B. Aarts, . Roelofs, and . Van-turennout, Anticipatory Activity in Anterior Cingulate Cortex Can Be Independent of Conflict and Error Likelihood, Journal of Neuroscience, vol.28, issue.18, pp.4671-4678, 2008.
DOI : 10.1523/JNEUROSCI.4400-07.2008

L. Abbott, Lapicque's introduction of the integrate-and-fire model neuron, 1907.

G. , A. , and M. Crutcher, Functional architecture of basal ganglia circuits : neural substrates of parallel processing, Trends Neurosci, vol.13, issue.7, pp.266-71, 1990.

G. Alexander, M. Delong, and P. Strick, Parallel Organization of Functionally Segregated Circuits Linking Basal Ganglia and Cortex, Annual Review of Neuroscience, vol.9, issue.1, pp.357-81, 1986.
DOI : 10.1146/annurev.ne.09.030186.002041

W. Asaad, E. Rainer, and . Miller, Neural Activity in the Primate Prefrontal Cortex during Associative Learning, Neuron, vol.21, issue.6, pp.1399-407, 1998.
DOI : 10.1016/S0896-6273(00)80658-3

G. Aston, -. , and J. D. Cohen, Adaptive gain and the role of the locus coeruleus-norepinephrine system in optimal performance, The Journal of Comparative Neurology, vol.30, issue.1, pp.99-110, 2005.
DOI : 10.1002/cne.20723

C. Azuar, . Reyes, . Volle, . Kinkingnehun, . Bravo et al., Architecture of cognitive control in the human prefrontal cortex : A lesion behavior mapping study in patients with prefrontal lesions, Society for Neuroscience, 2009.

A. Baddeley, Working memory, Science, vol.255, issue.5044, pp.556-565, 1992.
DOI : 10.1126/science.1736359

D. Badre, Cognitive control, hierarchy, and the rostro???caudal organization of the frontal lobes, Trends in Cognitive Sciences, vol.12, issue.5, pp.193-200, 2008.
DOI : 10.1016/j.tics.2008.02.004

D. Badre, D. Mark, and . Esposito, Functional Magnetic Resonance Imaging Evidence for a Hierarchical Organization of the Prefrontal Cortex, Journal of Cognitive Neuroscience, vol.84, issue.12, pp.2082-99, 2007.
DOI : 10.1046/j.1460-9568.1999.00718.x

D. Badre, D. Anthony, and . Wagner, Computational and neurobiological mechanisms underlying cognitive flexibility, Proceedings of the National Academy of Sciences, vol.103, issue.18, pp.7186-91, 2006.
DOI : 10.1073/pnas.0509550103

M. Hannah, . Bayer, W. Paul, and . Glimcher, Midbrain dopamine neurons encode a quantitative reward prediction error signal, Neuron, vol.47, issue.1, pp.129-170, 2005.

M. Paul, M. Bays, and . Husain, Dynamic shifts of limited working memory resources in human vision, Science, vol.321, issue.5890, pp.851-855, 2008.

M. Jeffrey, A. Beck, and . Pouget, Exact inferences in a neural implementation of a hidden markov model, Neural computation, vol.19, issue.5, pp.1344-61, 2007.

E. J. Timothy, . Behrens, T. Laurence, . Hunt, W. Mark et al., Associative learning of social value, Nature, vol.456, issue.7219, pp.245-249, 2008.

E. Timothy, . Behrens, W. Mark, . Woolrich, E. Mark et al., Learning the value of information in an uncertain world, Nat Neurosci, vol.10, issue.9, pp.1214-1221, 2007.

R. Bellman, The theory of dynamic programming, Proceedings of the National Academy of Sciences, 1952.

R. Ernest-bellman, Introduction to the mathematical theory of control processes?Äé -page 137, p.245, 1967.

M. Bertin, K. Schweighofer, and . Doya, Multiple model-based reinforcement learning explains dopamine neuronal activity, Neural Networks, vol.20, issue.6, pp.668-675, 2007.
DOI : 10.1016/j.neunet.2007.04.028

D. Bertsekas, Dynamic programming : deterministic and stochastic models, 1987.

G. Blasi, Effect of Catechol-O-Methyltransferase val158met Genotype on Attentional Control, Journal of Neuroscience, vol.25, issue.20, pp.5038-5045, 2005.
DOI : 10.1523/JNEUROSCI.0476-05.2005

D. Erie, T. E. Boorman, . Behrens, W. Mark, M. Woolrich et al., How green is the grass on the other side ? frontopolar cortex and the evidence in favor of alternative courses of action, Neuron, vol.62, issue.5, pp.733-776, 2009.

M. Botvinick, Hierarchical models of behavior and prefrontal function, Trends in Cognitive Sciences, vol.12, issue.5, pp.201-208, 2008.
DOI : 10.1016/j.tics.2008.02.009

M. Botvinick, C. Cohen, and . Carter, Conflict monitoring and anterior cingulate cortex: an update, Trends in Cognitive Sciences, vol.8, issue.12, pp.539-546, 2004.
DOI : 10.1016/j.tics.2004.10.003

M. Botvinick, Y. Niv, and A. Barto, Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective, Cognition, vol.113, issue.3, p.19, 2008.
DOI : 10.1016/j.cognition.2008.08.011

M. Botvinick, C. David, and . Plaut, Doing Without Schema Hierarchies: A Recurrent Connectionist Approach to Normal and Impaired Routine Sequential Action., Psychological Review, vol.111, issue.2, pp.395-429, 2004.
DOI : 10.1037/0033-295X.111.2.395

M. Matthew and . Botvinick, Multilevel structure in behaviour and in the brain : a model of fuster's hierarchy, Philosophical Transactions of the Royal Society B : Biological Sciences, vol.362, pp.1615-1626, 1485.

M. Matthew, . Botvinick, C. David, and . Plaut, Such stuff as habits are made on : A reply to cooper and shallice, Psychological Review, vol.113, issue.4, pp.917-928, 2006.

S. Bouret and S. Sara, Network reset: a simplified overarching theory of locus coeruleus noradrenaline function, Trends in Neurosciences, vol.28, issue.11, pp.574-582, 2005.
DOI : 10.1016/j.tins.2005.09.002
URL : https://hal.archives-ouvertes.fr/hal-00088131

K. Brodmann, Vergleichende lokalisationslehre der großhirnrinde in ihren prinzipien dargestellt auf grund des zellenbaues, 1909.

A. Brovelli, . Laksiri, . Nazarian, D. Meunier, and . Boussaoud, Understanding the Neural Computations of Arbitrary Visuomotor Learning through fMRI and Associative Learning Theory, Cerebral Cortex, vol.18, issue.7, pp.1485-1495, 2007.
DOI : 10.1093/cercor/bhm198

J. Brown and T. Braver, A computational model of risk, conflict, and individual difference effects in the anterior cingulate cortex, Brain Research, vol.1202, pp.99-108, 2008.
DOI : 10.1016/j.brainres.2007.06.080

J. Brown, T. Reynolds, and . Braver, A computational model of fractionated conflict-control mechanisms in task-switching, Cognitive Psychology, vol.55, issue.1, pp.37-85, 2007.
DOI : 10.1016/j.cogpsych.2006.09.005

W. Joshua, . Brown, S. Todd, and . Braver, Learned predictions of error likelihood in the anterior cingulate cortex, Science, vol.307, issue.5712, pp.1118-1139, 2005.

S. Budhani, A. Marsh, D. Pine, and R. J. Blair, Neural correlates of response reversal: Considering acquisition, NeuroImage, vol.34, issue.4, pp.1754-1765, 2007.
DOI : 10.1016/j.neuroimage.2006.08.060

P. Calabresi, . Gubellini, . Centonze, . Picconi, . Bernardi et al., Dopamine and camp-regulated phosphoprotein 32 kda controls both striatal long-term depression and long-term potentiation, opposing forms of synaptic plasticity, Journal of Neuroscience, issue.22, pp.208443-51, 2000.

M. Carandini and D. Heeger, Summation and division by neurons in primate visual cortex, Science, vol.264, issue.5163, pp.1333-1339, 1994.
DOI : 10.1126/science.8191289

S. Chamberlain, Neurochemical Modulation of Response Inhibition and Probabilistic Learning in Humans, Science, vol.311, issue.5762, pp.311861-863, 2006.
DOI : 10.1126/science.1121218

J. Chein and W. Schneider, Neuroimaging studies of practice-related change: fMRI and meta-analytic evidence of a domain-general control network for learning, Cognitive Brain Research, vol.25, issue.3, pp.607-623, 2005.
DOI : 10.1016/j.cogbrainres.2005.08.013

J. Cohen and D. Servan-schreiber, Context, cortex, and dopamine: A connectionist approach to behavior and biology in schizophrenia., Psychological Review, vol.99, issue.1, pp.45-77, 1992.
DOI : 10.1037/0033-295X.99.1.45

D. Jonathan, . Cohen, S. Todd, J. W. Braver, and W. Brown, Computational perspectives on dopamine function in prefrontal cortex, Current Opinion in Neurobiology, pp.1-7, 2002.

D. Jonathan, . Cohen, M. Samuel, A. J. Mcclure, and . Yu, Should i stay or should i go ? how the human brain manages the trade-off between exploitation and exploration, Philosophical Transactions of the Royal Society B : Biological Sciences, vol.362, pp.933-942, 1481.

X. Michael, . Cohen, J. Michael, and . Frank, Neurocomputational models of basal ganglia function in learning, memory and choice, Behavioural Brain Research, vol.199, issue.1, pp.141-156, 2009.

C. Constantinidis and E. Procyk, The primate working memory networks, Cognitive, Affective, & Behavioral Neuroscience, vol.4, issue.4, pp.444-65, 2004.
DOI : 10.3758/CABN.4.4.444
URL : https://hal.archives-ouvertes.fr/inserm-00906775

A. Conway, Working memory capacity and its relation to general intelligence, Trends in Cognitive Sciences, vol.7, issue.12, pp.547-552, 2003.
DOI : 10.1016/j.tics.2003.10.005

R. Cools, Differential Responses in Human Striatum and Prefrontal Cortex to Changes in Object and Rule Relevance, Journal of Neuroscience, vol.24, issue.5, pp.1129-1135, 2004.
DOI : 10.1523/JNEUROSCI.4312-03.2004

R. Cools, M. J. Frank, S. Gibbs, . Miyakawa, M. Jagust et al., Striatal Dopamine Predicts Outcome-Specific Reversal Learning and Its Sensitivity to Dopaminergic Drug Administration, Journal of Neuroscience, vol.29, issue.5, pp.1538-1543, 2009.
DOI : 10.1523/JNEUROSCI.4467-08.2009

R. Cools, L. Clark, M. Adrian, T. W. Owen, and . Robbins, Defining the neural mechanisms of probabilistic reversal learning using event-related functional magnetic resonance imaging, Journal of Neuroscience, vol.22, issue.11, pp.4563-4570, 2002.

R. Cools, W. Trevor, and . Robbins, Chemistry of the adaptive mind, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, vol.362, issue.1825, pp.2871-2888, 1825.
DOI : 10.1098/rsta.2004.1468

P. Richard, T. Cooper, and . Shallice, Contention scheduling and the control of routine activities, Cognitive Neuropsychology, vol.17, issue.4, pp.297-338, 2000.

P. Richard, T. Cooper, and . Shallice, Hierarchical schemas and goals in the control of sequential behavior, Psychological Review, vol.113, issue.4, pp.887-916, 2006.

P. Richard, T. Cooper, and . Shallice, Structured representations in the control of behavior cannot be so easily dismissed : A reply to botvinick and plaut, Psychological Review, vol.113, issue.4, pp.929-931, 2006.

G. Corrado and K. Doya, Understanding Neural Coding through the Model-Based Analysis of Decision Making, Journal of Neuroscience, vol.27, issue.31, pp.8178-8180, 2007.
DOI : 10.1523/JNEUROSCI.1590-07.2007

R. Crites and A. Barto, An actor/critic algorithm that is equivalent to q-learning, Advances in Neural Inf. Proc. Systems, 1995.

E. A. Crone, Neural Evidence for Dissociable Components of Task-switching, Cerebral Cortex, vol.16, issue.4, pp.475-486, 2005.
DOI : 10.1093/cercor/bhi127

J. Cummings, Anatomic and Behavioral Aspects of Frontal-Subcortical Circuits, Annals of the New York Academy of Sciences, vol.46, issue.1 Structure and, pp.1-13, 1995.
DOI : 10.1016/0166-2236(90)90104-I

D. Nathaniel and . Daw, Dopamine : at the intersection of reward and action, Nat Neurosci, vol.10, issue.12, pp.1505-1512, 2007.

D. Nathaniel, Y. Daw, P. Niv, and . Dayan, Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control, Nat Neurosci, vol.8, issue.12, pp.1704-1711, 2005.

D. Nathaniel, . Daw, P. John, P. O-'doherty, B. Dayan et al., Cortical substrates for exploratory decisions in humans, Nature, vol.441, issue.7095, pp.876-879, 2006.

P. Dayan and Y. Niv, Reinforcement learning: The Good, The Bad and The Ugly, Current Opinion in Neurobiology, vol.18, issue.2, pp.185-196, 2008.
DOI : 10.1016/j.conb.2008.08.003

P. Dayan, W. Bernard, and . Balleine, Reward, Motivation, and Reinforcement Learning, Neuron, vol.36, issue.2, pp.285-98, 2002.
DOI : 10.1016/S0896-6273(02)00963-7

P. Dayan and T. J. Sejnowski, Exploration bonuses and dual control, Machine Learning, vol.18, issue.1, 1996.
DOI : 10.1007/BF00115298

P. Dayan and A. Yu, Phasic norepinephrine: A neural interrupt signal for unexpected events, Network: Computation in Neural Systems, vol.22, issue.4, pp.335-350, 2006.
DOI : 10.1016/j.neuron.2005.04.026

R. Dearden, N. Friedman, and S. Russell, Bayesian q-learning, 1998.

S. Dehaene and J. Changeux, The Wisconsin Card Sorting Test: Theoretical Analysis and Modeling in a Neuronal Network, Cerebral Cortex, vol.1, issue.1, pp.62-79, 1991.
DOI : 10.1093/cercor/1.1.62

S. Dehaene and J. Changeux, A hierarchical neuronal network for planning behavior, Proceedings of the National Academy of Sciences, vol.94, issue.24
DOI : 10.1073/pnas.94.24.13293

S. Dehaene and J. Changeux, Reward-dependent learning in neuronal networks for planning and decision making, Prog Brain Res, vol.126, pp.217-246, 2000.
DOI : 10.1016/S0079-6123(00)26016-0

S. Deneve, Bayesian Spiking Neurons I: Inference, Neural Computation, vol.22, issue.7, pp.91-117, 2008.
DOI : 10.1038/370140a0

S. Deneve, Bayesian Spiking Neurons II: Learning, Neural Computation, vol.83, issue.39, pp.118-163, 2008.
DOI : 10.1177/1073858404272404

B. Doll, . Jacobs, M. Sanfey, and . Frank, Instructional control of reinforcement learning: A behavioral and neurocomputational investigation, Brain Research, vol.1299, 2009.
DOI : 10.1016/j.brainres.2009.07.007

U. Nico, . Dosenbach, M. Kristina, . Visscher, D. Erica et al., A core system for the implementation of task sets

K. Doya, Metalearning and neuromodulation, Neural networks : the official journal of the International Neural Network Society, pp.495-506, 2002.
DOI : 10.1016/S0893-6080(02)00044-8

K. Doya, Modulators of decision making, Nature Neuroscience, vol.55, issue.4, pp.410-416, 2008.
DOI : 10.1038/nn2077

K. Doya, K. Samejima, K. Ichi-katagiri, and M. Kawato, Multiple Model-Based Reinforcement Learning, Neural Computation, vol.3, issue.6, pp.1347-69, 2002.
DOI : 10.1016/S1364-6613(98)01221-2
URL : http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.107.6596

J. Dreher and K. F. Berman, Fractionating the neural substrate of cognitive control processes, Proceedings of the National Academy of Sciences, vol.99, issue.22, pp.14595-600, 2002.
DOI : 10.1073/pnas.222193299

J. Dreher, P. Kohn, B. Kolachana, R. Daniel, K. F. Weinberger et al., Variation in dopamine genes influences responsivity of the human reward system, Notes : N'apporte presque rien, pp.617-639, 2009.
DOI : 10.1073/pnas.0805517106

J. Dreher, J. Peter, P. Schmidt, D. Kohn, D. Furman et al., Menstrual cycle phase modulates reward-related neural function in women, Proceedings of the National Academy of Sciences, vol.104, issue.7, pp.2465-70, 2007.
DOI : 10.1073/pnas.0605569104

T. Egner, Multiple conflict-driven control mechanisms in the human brain, Trends in Cognitive Sciences, vol.12, issue.10, pp.374-380, 2008.
DOI : 10.1016/j.tics.2008.07.001

R. Engle, S. Tuholski, J. Laughlin, and A. Conway, Working memory, short-term memory, and general fluid intelligence: A latent-variable approach., Journal of Experimental Psychology: General, vol.128, issue.3, pp.309-340, 1999.
DOI : 10.1037/0096-3445.128.3.309

J. Fan, J. Fossella, T. Sommer, Y. Wu, I. Michael et al., Mapping the genetic variation of executive attention onto brain activity, Proceedings of the National Academy of Sciences, vol.100, issue.12, pp.7406-7417, 2003.
DOI : 10.1073/pnas.0732088100

M. Frank, R. Loughry, and . Reilly, Interactions between frontal cortex and basal ganglia in working memory: A computational model, Cognitive, Affective, & Behavioral Neuroscience, vol.1, issue.2, pp.137-60, 2001.
DOI : 10.3758/CABN.1.2.137

J. Michael and . Frank, Dynamic dopamine modulation in the basal ganglia : a neurocomputational account of cognitive deficits in medicated and nonmedicated parkinsonism, Journal of cognitive neuroscience, vol.17, issue.1, pp.51-72, 2005.

J. Michael and . Frank, Hold your horses : a dynamic computational role for the subthalamic nucleus in decision making, Neural networks : the official journal of the International Neural Network Society, vol.19, issue.8, pp.1120-1156, 2006.

J. Michael, . Frank, B. Bradley, J. Doll, F. Oas-terpstra et al., Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation, Nat Neurosci, vol.12, issue.8, pp.1062-1070, 2009.

J. Michael, . Frank, A. Ahmed, . Moustafa, M. Heather et al., Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning, Proc Natl Acad Sci, vol.104, issue.41, pp.16311-16317, 2007.

J. Michael, . Frank, C. Randall, T. O-'reilly, and . Curran, When memory fails, intuition reigns : midazolam enhances implicit inference in humans. Psychological science : a journal of the American Psychological Society, APS, vol.17, issue.8, pp.700-707, 2006.

J. Michael, . Frank, C. Lauren, . Seeberger, C. Randall et al., By carrot or by stick : cognitive reinforcement learning in parkinsonism, Science, vol.306, issue.5703, pp.1940-1943, 2004.

S. Fusi, . Asaad, X. Miller, and . Wang, A Neural Circuit Model of Flexible Sensorimotor Mapping: Learning and Forgetting on Multiple Timescales, Neuron, vol.54, issue.2, pp.319-333, 2007.
DOI : 10.1016/j.neuron.2007.03.017

J. Fuster and G. Alexander, Neuron Activity Related to Short-Term Memory, Science, vol.173, issue.3997, pp.652-656, 1971.
DOI : 10.1126/science.173.3997.652

M. Joaquín and . Fuster, Frontal lobe and cognitive development, J Neurocytol, vol.31, issue.3- 5, pp.373-85, 2002.

R. Charles, S. Gallistel, P. Fairhurst, and . Balsam, The learning curve : implications of a quantitative analysis, Proc Natl Acad Sci, vol.101, issue.36, pp.13124-13155, 2004.

J. Gittins and D. Jones, A dynamic allocation index for the sequential design of experiments, Colloq. Math. Soc. János Bolyai, vol.9, pp.241-266, 1974.

T. Goldberg and D. Weinberger, Genes and the parsing of cognitive processes, Trends in Cognitive Sciences, vol.8, issue.7, pp.325-335, 2004.
DOI : 10.1016/j.tics.2004.05.011

P. Goldman-rakic, The prefrontal landscape: implications of functional architecture for understanding human mentation and the central executive, Philos Trans R Soc Lond, B, Biol Sci, vol.351, pp.1445-53, 1346.
DOI : 10.1093/acprof:oso/9780198524410.003.0007

S. Jeffrey, S. Landau-eugene, and . Gollin, Successive reversal performance in young children as a function of the delay interval between reversls, Child development, p.11, 1966.

J. Grafman, The structured event complex and the human prefrontal cortex. Principles of frontal lobe function, 2002.

S. Graham, E. Phua, C. Siong-soon, T. Oh, C. Au et al., Role of medial cortical, hippocampal and striatal interactions during cognitive set-shifting, NeuroImage, vol.45, issue.4, pp.1359-1367, 2009.
DOI : 10.1016/j.neuroimage.2008.12.040

A. Green, M. Munafo, C. Deyoug, J. Fossella, J. Fan et al., Using genetic data in cognitive neuroscience: from growing pains to genuine insights, Nature Reviews Neuroscience, vol.26, issue.9, pp.710-720, 2008.
DOI : 10.1038/nrn2461

U. Halsband and R. Passingham, The role of premotor and parietal cortex in the direction of action, Brain Research, vol.240, issue.2, pp.368-72, 1982.
DOI : 10.1016/0006-8993(82)90239-6

A. Hampshire and A. Owen, Fractionating Attentional Control Using Event-Related fMRI, Cerebral Cortex, vol.16, issue.12, pp.1679-1689, 2005.
DOI : 10.1093/cercor/bhj116

A. Hampton, J. Bossaerts, and . Doherty, The Role of the Ventromedial Prefrontal Cortex in Abstract State-Based Inference during Decision Making in Humans, Journal of Neuroscience, vol.26, issue.32, pp.8360-8367, 2006.
DOI : 10.1523/JNEUROSCI.1010-06.2006

W. Carolyn and . Harley, Norepinephrine and dopamine as learning signals, Neural Plast, vol.11, issue.3-4, pp.191-204, 2004.

H. Harlow, The formation of learning sets., Psychological Review, vol.56, issue.1, pp.51-65, 1949.
DOI : 10.1037/h0062474

Y. Benjamin, . Hayden, M. John, . Pearson, L. Michael et al., Fictive reward signals in the anterior cingulate cortex, Science, vol.324, issue.5929, pp.948-50, 2009.

T. Hazy, R. Frank, and . Oreilly, Banishing the homunculus: Making working memory work, Neuroscience, vol.139, issue.1, pp.105-118, 2006.
DOI : 10.1016/j.neuroscience.2005.04.067

E. Thomas, . Hazy, J. Michael, . Frank, C. Randall et al., Towards an executive without a homunculus : computational models of the prefrontal cortex/basal ganglia system, Philosophical Transactions of the Royal Society B : Biological Sciences, vol.362, pp.1601-1613, 1485.

A. Seth, . Herd, T. Marie, . Banich, C. Randall et al., Neural mechanisms of cognitive control : an integrative model of stroop task performance and fmri data, Journal of cognitive neuroscience, vol.18, issue.1, pp.22-32, 2006.

A. Hodgkin and A. Huxley, Propagation of Electrical Signals Along Giant Nerve Fibres, Proceedings of the Royal Society B: Biological Sciences, vol.140, issue.899, pp.177-83, 1952.
DOI : 10.1098/rspb.1952.0054

A. Hyafil, C. Summerfield, and E. Koechlin, Two Mechanisms for Task Switching in the Prefrontal Cortex, Journal of Neuroscience, vol.29, issue.16, pp.5135-5142, 2009.
DOI : 10.1523/JNEUROSCI.2828-08.2009

H. Imamizu and M. Kawato, Neural Correlates of Predictive and Postdictive Switching Mechanisms for Internal Models, Journal of Neuroscience, vol.28, issue.42, pp.10751-10765, 2008.
DOI : 10.1523/JNEUROSCI.1106-08.2008

H. Imamizu, T. Kuroda, T. Yoshioka, and M. Kawato, Functional Magnetic Resonance Imaging Examination of Two Modular Architectures for Switching Multiple Internal Models, Journal of Neuroscience, vol.24, issue.5, pp.1173-81, 2004.
DOI : 10.1523/JNEUROSCI.4011-03.2004

H. Imamizu, N. Sugimoto, R. Osu, K. Tsutsui, K. Sugiyama et al., Explicit contextual information selectively contributes to predictive switching of internal models, Experimental Brain Research, vol.269, issue.3, pp.395-408, 2007.
DOI : 10.1007/s00221-007-0940-1

T. Jaakkola, S. Jordan, and . Singh, On the Convergence of Stochastic Iterative Dynamic Programming Algorithms, Neural Computation, vol.8, issue.6, 1994.
DOI : 10.1214/aoms/1177729586

R. Jacobs, A. Jordan, and . Barto, Task decomposition through competition in a modular connectionist architecture : The what, Machine learning : from theory to applications, 1993.

R. Jacobs, . Jordan, G. Nowlan, and . Hinton, Adaptive Mixtures of Local Experts, Neural Computation, vol.4, issue.1, 1991.
DOI : 10.1162/neco.1989.1.2.281

J. Harry, . Jerison, W. Dahlia, and . Zaidel, Evolution of the brain. neuropsychology. Neuropsychology . Handbook of perception and cognition, pp.53-82, 1994.

D. Joel, Y. Niv, and E. Ruppin, Actor???critic models of the basal ganglia: new anatomical and computational perspectives, Neural networks : the official journal of the International Neural Network Society, pp.4-6535, 2002.
DOI : 10.1016/S0893-6080(02)00047-3

K. Johnston, M. Helen, . Levin, J. Michael, S. Koval et al., Top-Down Control-Signal Dynamics in Anterior Cingulate and Prefrontal Cortex Neurons following Task Switching, Neuron, vol.53, issue.3, pp.453-462, 2007.
DOI : 10.1016/j.neuron.2006.12.023

M. Jordan and R. Jacobs, Hierarchical mixtures of experts and the em algorithm, Neural computation, 1994.

T. Jubault, C. Ody, and E. Koechlin, Serial Organization of Human Behavior in the Inferior Parietal Cortex, Journal of Neuroscience, vol.27, issue.41, pp.11028-11036, 2007.
DOI : 10.1523/JNEUROSCI.1986-07.2007

T. Kahnt, Q. Soyoung, . Park, X. Michael, A. Cohen et al., Dorsal Striatal???midbrain Connectivity in Humans Predicts How Reinforcements Are Used to Guide Decisions, Journal of Cognitive Neuroscience, vol.23, issue.7, pp.1332-1377, 2009.
DOI : 10.1016/j.neuroimage.2007.04.001

S. Kakade and P. Dayan, Dopamine: generalization and bonuses, Neural Networks, vol.15, issue.4-6, 2002.
DOI : 10.1016/S0893-6080(02)00048-5

J. Michael, . Kane, W. Randall, and . Engle, The role of prefrontal cortex in workingmemory capacity, executive attention, and general fluid intelligence : an individualdifferences perspective, Psychonomic bulletin & review, vol.9, issue.4, pp.637-71, 2002.

M. Kawato, Internal models for motor control and trajectory planning, Current Opinion in Neurobiology, vol.9, issue.6, pp.718-745, 1999.
DOI : 10.1016/S0959-4388(99)00028-8

A. Kepecs, N. Uchida, A. Hatim, Z. F. Zariwala, and . Mainen, Neural correlates, computation and behavioural impact of decision confidence, Nature, vol.10, issue.7210, pp.227-231, 2008.
DOI : 10.1038/nature07200

H. Kim, S. Shimojo, and J. Doherty, Is Avoiding an Aversive Outcome Rewarding? Neural Substrates of Avoidance Learning in the Human Brain, PLoS Biology, vol.304, issue.8
DOI : 10.1371/journal.pbio.0040233.sg005

C. Koch and I. Segev, The role of single neurons in information processing, Nature Neuroscience, vol.3, issue.Supp, pp.1171-1178, 2000.
DOI : 10.1038/81444

E. Koechlin, J. Anton, and Y. Burnod, Bayesian inference in populations of cortical neurons: a model of motion integration and segmentation in area MT, Biological Cybernetics, vol.80, issue.1, pp.25-44, 1999.
DOI : 10.1007/s004220050502

E. Koechlin, . Basso, . Pietrini, J. Panzer, and . Grafman, The role of the anterior prefrontal cortex in human cognition, Nature, vol.399, issue.6732, pp.148-51, 1999.

E. Koechlin and A. Hyafil, Anterior Prefrontal Function and the Limits of Human Decision-Making, Science, vol.318, issue.5850, pp.318594-598, 2007.
DOI : 10.1126/science.1142995

E. Koechlin and T. Jubault, Broca's Area and the Hierarchical Organization of Human Behavior, Neuron, vol.50, issue.6, pp.963-974, 2006.
DOI : 10.1016/j.neuron.2006.05.017

E. Koechlin and C. Summerfield, An information theoretical approach to prefrontal executive function, Trends in Cognitive Sciences, vol.11, issue.6, pp.229-235, 2007.
DOI : 10.1016/j.tics.2007.04.005

E. Koechlin, A. Danek, Y. Burnod, and J. Grafman, Medial Prefrontal and Subcortical Mechanisms Underlying the Acquisition of Motor and Cognitive Action Sequences in Humans, Neuron, vol.35, issue.2, pp.371-81, 2002.
DOI : 10.1016/S0896-6273(02)00742-0

E. Koechlin, C. Ody, and F. Kouneiher, The Architecture of Cognitive Control in the Human Prefrontal Cortex, Science, vol.302, issue.5648, pp.1181-1186, 2003.
DOI : 10.1126/science.1088545

F. Kouneiher, S. Charron, and E. Koechlin, Motivation and cognitive control in the human prefrontal cortex, Nature Neuroscience, vol.19, issue.7, pp.939-984, 2009.
DOI : 10.1016/j.neuroimage.2004.07.041

A. Kai, P. Krueger, and . Dayan, Flexible shaping : How learning in small steps helps, Cognition, vol.110, issue.3, pp.380-394, 2009.

L. Lapique, Recherches quantitatives sur l'excitation électrique de nerfs traitée comme une polarisation, J. Physiol. Pathol. Gen, 1907.

W. Ma, M. Jeffrey, . Beck, E. Peter, A. Latham et al., Bayesian inference with probabilistic population codes, Nature Neuroscience, vol.9, issue.11, pp.1432-1438, 2006.
DOI : 10.1038/nn1691

A. Farshad, K. Mansouri, . Tanaka, J. Mark, and . Buckley, Conflict-induced behavioural adjustment : a clue to the executive functions of the prefrontal cortex, Nat Rev Neurosci, vol.10, issue.2, pp.141-152, 2009.

A. Mcgovern, . Precup, S. Ravindran, and . Singh, Hierarchical optimal control of mdps, Proceedings of the Tenth Yale Workshop on Adaptive, 1998.

F. Mcnab and T. Klingberg, Prefrontal cortex and basal ganglia control access to working memory, Nature Neuroscience, vol.10, issue.1, pp.103-110, 2008.
DOI : 10.1037/0096-3445.132.1.47

V. Meininger, Neuro-anatomie, 1983.

E. Miller, The prefrontal cortex and cognitive control, Nature Reviews Neuroscience, vol.1, issue.1, pp.59-65, 2000.
DOI : 10.1038/35036228

D. Mitchell, . Rhodes, R. Pine, and . Blair, The contribution of ventrolateral and dorsolateral prefrontal cortex to response reversal, Behavioural Brain Research, vol.187, issue.1, pp.80-87, 2008.
DOI : 10.1016/j.bbr.2007.08.034

S. Monsell, Task switching, Trends in Cognitive Sciences, vol.7, issue.3, pp.134-140, 2003.
DOI : 10.1016/S1364-6613(03)00028-7

E. Murray, T. J. Bussey, and S. Wise, Role of prefrontal cortex in a network for arbitrary visuomotor mapping. Experimental brain research Experimentelle Hirnforschung Expérimentation cérébrale, pp.114-143, 2000.

L. Murray and C. Ranganath, The Dorsolateral Prefrontal Cortex Contributes to Successful Relational Memory Encoding, Journal of Neuroscience, vol.27, issue.20, pp.5515-5522, 2007.
DOI : 10.1523/JNEUROSCI.0406-07.2007

A. Nagano-saito, M. Leyton, Y. Monchi, Y. Goldberg, A. He et al., Dopamine Depletion Impairs Frontostriatal Functional Connectivity during a Set-Shifting Task, Journal of Neuroscience, vol.28, issue.14, pp.3697-3706, 2008.
DOI : 10.1523/JNEUROSCI.3921-07.2008

O. Randall, P. Nicolas, and . Rougier, Learning representations in a gated prefrontal cortex model of dynamic task switching, Cognitive Science, p.18, 2002.

G. Kimberly, . Noble, D. Bruce, M. J. Mccandliss, and . Farah, Socioeconomic gradients predict individual differences in neurocognitive abilities, Developmental Sci, vol.10, issue.4, pp.464-480, 2007.

J. O-'doherty, Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning, Science, vol.304, issue.5669, pp.452-454, 2004.
DOI : 10.1126/science.1094285

J. O-'doherty, M. Kringelbach, E. Rolls, C. Hornak, and . Andrews, Abstract reward and punishment representations in the human orbitofrontal cortex, Nat Neurosci, vol.4, issue.1, pp.95-102, 2001.

P. John, A. O-'doherty, H. Hampton, and . Kim, Model-based fmri and its application to reward learning and decision making, Ann N Y Acad Sci, vol.1104, pp.35-53, 2007.

R. C. O-'reilly, Biologically based computational models of high-level cognition, Science, issue.5796, pp.31491-94, 2006.

C. Randall, . Reilly, J. Michael, and . Frank, Making working memory work : a computational model of learning in the prefrontal cortex and basal ganglia, Neural computation, vol.18, issue.2, pp.283-328, 2006.

C. Randall, . O-'reilly, J. Michael, . Frank, E. Thomas et al., Pvlv : the primary value and learned value pavlovian learning algorithm, Behav Neurosci, vol.121, issue.1, pp.31-49, 2007.

A. Owen, Cognitive planning in humans: Neuropsychological, neuroanatomical and neuropharmacological perspectives, Progress in Neurobiology, vol.53, issue.4, pp.431-50, 1997.
DOI : 10.1016/S0301-0082(97)00042-7

A. Pasupathy, K. Earl, and . Miller, Different time courses of learning-related activity in the prefrontal cortex and striatum, Nature, vol.16, issue.7028, pp.873-879, 2005.
DOI : 10.1126/science.274.5286.427

M. Pessiglione, . Schmidt, . Draganski, . Kalisch, R. Lau et al., How the Brain Translates Money into Force: A Neuroimaging Study of Subliminal Motivation, Science, vol.316, issue.5826, pp.316904-906, 2007.
DOI : 10.1126/science.1140459

M. Pessiglione, P. Petrovic, J. Daunizeau, S. Palminteri, R. J. Dolan et al., Subliminal Instrumental Conditioning Demonstrated in the Human Brain, Neuron, vol.59, issue.4, pp.561-567, 2008.
DOI : 10.1016/j.neuron.2008.07.005

M. Pessiglione, B. Seymour, G. Flandin, R. J. Dolan, and C. D. Frith, Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans, Nature, vol.113, issue.7106, pp.4421042-1045, 2006.
DOI : 10.1002/1531-8257(200009)15:5<869::AID-MDS1016>3.0.CO;2-I

M. Petrides, Motor conditional associative-learning after selective prefrontal lesions in the monkey, Behavioural Brain Research, vol.5, issue.4, pp.407-420, 1982.
DOI : 10.1016/0166-4328(82)90044-4

M. Petrides, Visuo-motor conditional associative learning after frontal and temporal lesions in the human brain, Neuropsychologia, vol.35, issue.7, pp.989-97, 1997.
DOI : 10.1016/S0028-3932(97)00026-2

M. Petrides and D. Pandya, Association pathways of the prefrontal cortex and functional observations. Principles of frontal lobe function, 2002.

L. Michael, . Platt, A. Scott, and . Huettel, Risky business : the neuroeconomics of decision making under uncertainty, Nat Neurosci, vol.11, issue.4, pp.398-403, 2008.

R. Quilodran, E. Rothe, and . Procyk, Behavioral Shifts and Action Valuation in the Anterior Cingulate Cortex, Neuron, vol.57, issue.2, pp.314-325, 2008.
DOI : 10.1016/j.neuron.2007.11.031
URL : https://hal.archives-ouvertes.fr/inserm-00906686

A. Rangel, C. Camerer, and P. Montague, A framework for studying the neurobiology of value-based decision making, Nature Reviews Neuroscience, vol.10, issue.7, pp.545-556, 2008.
DOI : 10.1038/nrn2357

P. Redgrave, What is reinforced by phasic dopamine signals?, Brain Research Reviews, vol.58, issue.2, pp.322-339, 2008.
DOI : 10.1016/j.brainresrev.2007.10.007

P. Redgrave and K. Gurney, The short-latency dopamine signal: a role in discovering novel actions?, Nature Reviews Neuroscience, vol.9, issue.4, pp.967-75, 2006.
DOI : 10.1038/nrn2022

R. Rescorla, Informational variables in pavlovian conditioning. The psychology of learning and motivation : Advances, 1972.

J. Reynolds and R. Reilly, Developing PFC representations using reinforcement learning, Cognition, vol.113, issue.3, 2009.
DOI : 10.1016/j.cognition.2009.05.015

K. Richard-ridderinkhof, M. Ullsperger, A. Eveline, S. Crone, and . Nieuwenhuis, The Role of the Medial Frontal Cortex in Cognitive Control, Science, vol.306, issue.5695, pp.443-450, 2004.
DOI : 10.1126/science.1100301

P. Nicolas, . Rougier, C. David, . Noelle, S. Todd et al., Prefrontal cortex and flexible cognitive control : rules without symbols, Proc Natl Acad Sci, vol.102, issue.20, pp.7338-7381, 2005.

M. Rosario-rueda, K. Mary, . Rothbart, D. Bruce, L. Mccandliss et al., From The Cover: Training, maturation, and genetic influences on the development of executive attention, Proceedings of the National Academy of Sciences, vol.102, issue.41, pp.14931-14937, 2005.
DOI : 10.1073/pnas.0506897102

M. Rushworth, M. Walton, S. Kennerley, and D. Bannerman, Action sets and decisions in the medial frontal cortex, Trends in Cognitive Sciences, vol.8, issue.9, pp.410-417, 2004.
DOI : 10.1016/j.tics.2004.07.009

F. Matthew, T. E. Rushworth, and . Behrens, Choice, uncertainty and value in prefrontal and cingulate cortex, Nat Neurosci, vol.11, issue.4, pp.389-397, 2008.

F. Matthew, . Rushworth, J. Mark, T. E. Buckley, . Behrens et al., Functional organization of the medial frontal cortex, Curr Opin Neurobiol, vol.17, issue.2, pp.220-227, 2007.

M. Rushworth and M. Walton, The anterior cingulate cortex: reward-guided action selection and the value of actions, p.39, 2006.
DOI : 10.1093/acprof:oso/9780199231447.003.0006

K. Sakai, Task Set and Prefrontal Cortex, Annual Review of Neuroscience, vol.31, issue.1, p.29, 2008.
DOI : 10.1146/annurev.neuro.31.060407.125642

K. Samejima and K. Doya, Multiple Representations of Belief States and Action Values in Corticobasal Ganglia Loops, Annals of the New York Academy of Sciences, vol.20, issue.1, pp.213-241, 2007.
DOI : 10.1038/nrn1884

K. Samejima, Y. Ueda, K. Doya, and M. Kimura, Representation of Action-Specific Reward Values in the Striatum, Science, vol.310, issue.5752, pp.3101337-3101377, 2005.
DOI : 10.1126/science.1115270

A. Sanfey, Social Decision-Making: Insights from Game Theory and Neuroscience, Science, vol.318, issue.5850, pp.318598-602, 2007.
DOI : 10.1126/science.1142996

J. Susan and . Sara, The locus coeruleus and noradrenergic modulation of cognition, Nat Rev Neurosci, vol.10, issue.3, pp.211-223, 2009.

T. Schonberg, N. Daw, J. P. Joel, and . O-'doherty, Reinforcement Learning Signals in the Human Striatum Distinguish Learners from Nonlearners during Reward-Based Decision Making, Journal of Neuroscience, vol.27, issue.47, pp.12860-12867, 2007.
DOI : 10.1523/JNEUROSCI.2496-07.2007

F. Schubert, Quintette en ut majeur, pp.956-1889

W. Schultz, A Neural Substrate of Prediction and Reward, Science, vol.275, issue.5306, pp.2751593-1599, 1997.
DOI : 10.1126/science.275.5306.1593

W. Schultz, Getting Formal with Dopamine and Reward, Neuron, vol.36, issue.2, pp.241-63, 2002.
DOI : 10.1016/S0896-6273(02)00967-4

N. Schweighofer, C. Saori, K. Tanaka, and . Doya, Serotonin and the Evaluation of Future Rewards: Theory, Experiments, and Possible Neural Mechanisms, Annals of the New York Academy of Sciences, vol.23, issue.1
DOI : 10.1046/j.0953-816x.2001.01616.x

K. Semendeferi, . Lu, H. Schenker, and . Damasio, Humans and great apes share a large frontal cortex, Nature Neuroscience, vol.5, issue.3, pp.272-278, 2002.
DOI : 10.1038/nn814

B. Seymour, P. John, P. O-'doherty, M. Dayan, . Koltzenburg et al., Temporal difference models describe higher-order learning in humans, Nature, vol.19, issue.6992, pp.429664-429671, 2004.
DOI : 10.1016/S1053-8119(03)00073-9

M. Keith, P. Shafritz, A. Kartheiser, and . Belger, Dissociation of neural systems mediating shifts in behavioral response and cognitive set, NeuroImage, vol.25, issue.2, pp.600-606, 2005.

E. Sowell, P. Thompson, C. J. Holmes, T. Jernigan, and A. Toga, In vivo evidence for post-adolescent brain maturation in frontal and striatal regions, Nature Neuroscience, vol.2, issue.10, pp.859-61, 1999.
DOI : 10.1038/13154

A. Stemme, G. Deco, and A. Busch, The neurodynamics underlying attentional control in set shifting tasks, Cognitive Neurodynamics, vol.26, issue.4, pp.249-59, 2007.
DOI : 10.1007/s11571-007-9019-8

A. Stemme, G. Deco, and A. Busch, The neuronal dynamics underlying cognitive flexibility in set shifting tasks, Journal of Computational Neuroscience, vol.26, issue.4, pp.313-344, 2007.
DOI : 10.1007/s10827-007-0034-x

B. Strange, K. J. Henson, R. Friston, and . Dolan, Anterior Prefrontal Cortex Mediates Rule Learning in Humans, Cerebral Cortex, vol.11, issue.11, pp.1040-1046, 2001.
DOI : 10.1093/cercor/11.11.1040

R. Sutton, S. Precup, and . Singh, Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning, Artificial intelligence, 1999.
DOI : 10.1016/S0004-3702(99)00052-1

S. Tanaka, . Samejima, . Okada, Y. Ueda, . Okamoto et al., Brain mechanism of reward prediction under predictable and unpredictable environmental dynamics, Neural Networks, vol.19, issue.8, pp.1233-1241, 2006.
DOI : 10.1016/j.neunet.2006.05.039

C. Saori, K. Tanaka, G. Doya, K. Okada, Y. Ueda et al., Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops, Nat Neurosci, vol.7, issue.8, pp.887-93, 2004.

M. Turnock and S. Becker, A neural network model of hippocampal???striatal???prefrontal interactions in contextual conditioning, Brain Research, vol.1202, pp.87-98, 2008.
DOI : 10.1016/j.brainres.2007.06.078

M. Usher, J. Cohen, D. Servan-schreiber, J. Rajkowski, and G. , The Role of Locus Coeruleus in the Regulation of Cognitive Performance, Science, vol.283, issue.5401, pp.549-54, 1999.
DOI : 10.1126/science.283.5401.549

V. Valentin, J. P. Dickinson, and . O-'doherty, Determining the Neural Substrates of Goal-Directed Learning in the Human Brain, Journal of Neuroscience, vol.27, issue.15, pp.4019-4026, 2007.
DOI : 10.1523/JNEUROSCI.0564-07.2007

D. Tor, J. Wager, . Jonides, E. Edward, . Smith et al., Toward a taxonomy of attention shifting : individual differences in fmri during multiple shift types, Cognitive, affective & behavioral neuroscience, vol.5, issue.2, pp.127-170, 2005.

D. Jonathan and . Wallis, Orbitofrontal cortex and its contribution to decision-making

R. Wise and P. Rompre, Brain dopamine and reward. Annual review of psychology, pp.191-225, 1989.

C. Bianca, . Wittmann, D. Nathaniel, B. Daw, R. J. Seymour et al., Striatal activity underlies novelty-based choice in humans, Neuron, vol.58, issue.6, pp.967-73, 2008.

D. Wolpert and M. Kawato, Multiple paired forward and inverse models for motor control, Neural Networks, vol.11, issue.7-8, pp.1317-1329, 1998.
DOI : 10.1016/S0893-6080(98)00066-5

T. Wu, K. Kansaku, and M. Hallett, How Self-Initiated Memorized Movements Become Automatic: A Functional MRI Study, Journal of Neurophysiology, vol.91, issue.4, pp.1690-1698, 2004.
DOI : 10.1152/jn.01052.2003

W. Yoshida and S. Ishii, Resolution of Uncertainty in Prefrontal Cortex, Neuron, vol.50, issue.5, pp.781-789, 2006.
DOI : 10.1016/j.neuron.2006.05.006

A. Yu and P. Dayan, Uncertainty, Neuromodulation, and Attention, Neuron, vol.46, issue.4, pp.681-692, 2005.
DOI : 10.1016/j.neuron.2005.04.026