Volume 10, Issue 1 (January & February 2019)                   BCN 2019, 10(1): 1-12 | Back to browse issues page

DOI: 10.32598/bcn.9.10.125

XML Print

1- Department of Physics, Institute for Advanced Studies in Basic Sciences (IASBS), Zanjan, Iran.
2- School of Cognitive Sciences, Institute for Research in Fundamental Sciences (IPM), Tehran, Iran.
Neuromodulators modify intrinsic characteristics of the nervous system in order to reconfigure the functional properties of neural circuits. This reconfiguration is crucial for the flexibility of the nervous system to respond on an input-modulated basis. Such a functional rearrangement is realized by modification of intrinsic properties of the neural circuits including synaptic interactions. Dopamine is an important neuromodulator involved in motivation and stimulus-reward learning process, and adjusts synaptic dynamics in multiple time scales through different pathways. The modification of synaptic plasticity by dopamine underlies the change in synaptic transmission and integration mechanisms, which affects intrinsic properties of the neural system including membrane excitability, probability of neurotransmitters release, receptors’ response to neurotransmitters, protein trafficking, and gene transcription. Dopamine also plays a central role in behavioral control, whereas its malfunction can cause cognitive disorders. Impaired dopamine signaling is implicated in several neuropsychiatric disorders such as Parkinson’s disease, drug addiction, schizophrenia, attention-deficit/hyperactivity disorder, obsessive-compulsive disorder and Tourette’s syndrome. Therefore, dopamine plays a crucial role in the nervous system, where its proper modulation of neural circuits may enhance plasticity-related procedures, but disturbances in dopamine signaling might be involved in numerous neuropsychiatric disorders. In recent years, several computational models are proposed to formulate the involvement of dopamine in synaptic plasticity or neuropsychiatric disorders and address their connection based on the experimental findings.
Type of Study: Review | Subject: Computational Neuroscience
Received: 2017/10/31 | Accepted: 2018/02/5 | Published: 2019/01/1

1. Albin, R. L., Young, A. B., & Penney, J. B. (1989). The functional anatomy of basal ganglia disorders. Trends in Neurosciences, 12(10), 366-75. [DOI:10.1016/0166-2236(89)90074-X] [DOI:10.1016/0166-2236(89)90074-X]
2. Bayati, M., & Valizadeh, A. (2012). Effect of synaptic plasticity on the structure and dynamics of disordered networks of coupled neurons. Physical Review E, 86(1), 011925. [DOI:10.1103/PhysRevE.86.011925] [PMID] [DOI:10.1103/PhysRevE.86.011925]
3. Bayati, M., Valizadeh, A., Abbassian, A., & Cheng, S. (2015). Self-organization of synchronous activity propagation in neuronal networks driven by local excitation. Frontiers in Computational Neuroscience, 9(69), 1-15. [DOI:10.3389/fncom.2015.00069] [DOI:10.3389/fncom.2015.00069]
4. Berridge, K. C., & Robinson, T. E. (1998). What is the role of dopamine in reward: Hedonic impact, reward learning, or incentive salience?. Brain Research Reviews, 28(3), 309-69. [DOI:10.1016/S0165-0173(98)00019-8] [DOI:10.1016/S0165-0173(98)00019-8]
5. Bi, G. Q., & Poo, M. M. (1998). Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. Journal of Neuroscience, 18(24), 10464-72. [DOI:10.1523/JNEUROSCI.18-24-10464.1998] [PMID] [DOI:10.1523/JNEUROSCI.18-24-10464.1998]
6. Biederman, J., & Faraone, S. V. (2005). Attention-deficit hyperactivity disorder. Lancet, 366(9481), 237-48. [DOI:10.1016/S0140-6736(05)66915-2] [DOI:10.1016/S0140-6736(05)66915-2]
7. Blum, K., Chen, A. L. C., Braverman, E. R., Comings, D. E., Chen, T. J., & Arcuri, V., et al. (2008). Attention-deficit-hyperactivity disorder and reward deficiency syndrome. Neuropsychiatric Disease and Treatment, 4(5), 893-917. [PMID] [PMCID] [PMID] [PMCID]
8. Braver, T. S., Barch, D. M., & Cohen, J. D. (1999). Cognition and control in schizophrenia: a computational model of dopamine and prefrontal function. Biological Psychiatry, 46(3), 312-28. [DOI:10.1016/S0006-3223(99)00116-X] [DOI:10.1016/S0006-3223(99)00116-X]
9. Buse, J., Schoenefeld, K., Münchau, A., & Roessner, V. (2013). Neuromodulation in Tourette syndrome: Dopamine and beyond. Neuroscience & Biobehavioral Reviews, 37(6), 1069-84. [DOI:10.1016/j.neubiorev.2012.10.004] [PMID] [DOI:10.1016/j.neubiorev.2012.10.004]
10. Carlsson, A. (1959). The occurrence, distribution and physiological role of catecholamines in the nervous system. Pharmacological Reviews, 11(2), 490-3. [PMID] [PMID]
11. Carlsson, A., Lindqvist, M., Magnusson, T., & Waldeck, B. (1958). On the presence of 3-hydroxytyramine in brain. Science, 127(3296), 471-8. [DOI:10.1126/science.127.3296.471] [PMID] [DOI:10.1126/science.127.3296.471]
12. Cepeda, C., Radisavljevic, Z., Peacock, W., Levine, M. S., & Buchwald, N. A. (1992). Differential modulation by dopamine of responses evoked by excitatory amino acids in human cortex. Synapse, 11(4), 330-41. [DOI:10.1002/syn.890110408] [PMID] [DOI:10.1002/syn.890110408]
13. Chinta, S. J., & Andersen, J. K. (2005). Dopaminergic neurons. The International Journal of Biochemistry & Cell Biology, 37(5), 942-6. [DOI:10.1016/j.biocel.2004.09.009] [PMID] [DOI:10.1016/j.biocel.2004.09.009]
14. Cohen, M. X., & Frank, M. J. (2009). Neurocomputational models of basal ganglia function in learning, memory and choice. Behavioural Brain Research, 199(1), 141-56. [DOI:10.1016/j.bbr.2008.09.029] [PMID] [PMCID] [DOI:10.1016/j.bbr.2008.09.029]
15. Collins, A. G., & Frank, M. J. (2014). Opponent actor Learning (OpAL): modeling interactive effects of striatal dopamine on reinforcement learning and choice incentive. Psychological Review, 121(3), 337-66. [DOI:10.1037/a0037015] [PMID] [DOI:10.1037/a0037015]
16. Collins, A. G., & Frank, M. J. (2016). Surprise! Dopamine signals mix action, value and error. Nature Neuroscience, 19(1), 3-5. [DOI:10.1038/nn.4207] [PMID] [DOI:10.1038/nn.4207]
17. Cools, A. R., & Van Rossum, J. M. (1976). Excitation-mediating and inhibition-mediating dopamine-receptors: a new concept towards a better understanding of electrophysiological, biochemical, pharmacological, functional and clinical data. Psychopharmacologia, 45(3), 243-54. [DOI:10.1007/BF00421135] [PMID] [DOI:10.1007/BF00421135]
18. Dagher, A., & Robbins, T. W. (2009). Personality, addiction, dopamine: Insights from Parkinson's disease. Neuron, 61(4), 502-10. [DOI:10.1016/j.neuron.2009.01.031] [PMID] [DOI:10.1016/j.neuron.2009.01.031]
19. Dayan, P. (2012). Twenty-five lessons from computational neuromodulation. Neuron, 76(1), 240-56. [DOI:10.1016/j.neuron.2012.09.027] [PMID] [DOI:10.1016/j.neuron.2012.09.027]
20. Dayan, P., & Balleine, B. W. (2002). Reward, motivation, and reinforcement learning. Neuron, 36(2), 285-98. [DOI:10.1016/S0896-6273(02)00963-7] [DOI:10.1016/S0896-6273(02)00963-7]
21. DeLong, M. R. (1990). Primate models of movement disorders of basal ganglia origin. Trends in Neurosciences, 13(7), 281-5. [DOI:10.1016/0166-2236(90)90110-V] [DOI:10.1016/0166-2236(90)90110-V]
22. Di Chiara, G., & Imperato, A. (1988). Drugs abused by humans preferentially increase synaptic dopamine concentrations in the mesolimbic system of freely moving rats. Proceedings of the National Academy of Sciences, 85(14), 5274-8. [DOI:10.1073/pnas.85.14.5274] [DOI:10.1073/pnas.85.14.5274]
23. Durstewitz, D., Seamans, J. K., & Sejnowski, T. J. (2000). Dopamine-mediated stabilization of delay-period activity in a network model of prefrontal cortex. Journal of Neurophysiology, 83(3), 1733-50. [DOI:10.1152/jn.2000.83.3.1733] [PMID] [DOI:10.1152/jn.2000.83.3.1733]
24. Farries, M. A., & Fairhall, A. L. (2007). Reinforcement learning with modulated spike timing-dependent synaptic plasticity. Journal of Neurophysiology, 98(6), 3648-65. [DOI:10.1152/jn.00364.2007] [PMID] [DOI:10.1152/jn.00364.2007]
25. Flagel, S. B., Clark, J. J., Robinson, T. E., Mayo, L., Czuj, A., Willuhn, I., et al. (2011). A selective role for dopamine in stimulus-reward learning. Nature, 469(7328), 53-7. [DOI:10.1038/nature09588] [PMID] [PMCID] [DOI:10.1038/nature09588]
26. Florian, R. V. (2007). Reinforcement learning through modulation of spike-timing-dependent synaptic plasticity. Neural Computation, 19(6), 1468-1502. [DOI:10.1162/neco.2007.19.6.1468] [PMID] [DOI:10.1162/neco.2007.19.6.1468]
27. Frank, M. J. (2005). Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism. Journal of Cognitive Neuroscience, 17(1), 51-72. [DOI:10.1162/0898929052880093] [PMID] [DOI:10.1162/0898929052880093]
28. Frank, M. J., Santamaria, A., O'Reilly, R. C., & Willcutt, E. (2007). Testing computational models of dopamine and noradrenaline dysfunction in attention deficit/hyperactivity disorder. Neuropsychopharmacology, 32(7), 1583-99. [DOI:10.1038/sj.npp.1301278] [PMID] [DOI:10.1038/sj.npp.1301278]
29. Frank, M. J., Seeberger, L. C., & O'reilly, R. C. (2004). By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science, 306(5703), 1940-3. [DOI:10.1126/science.1102941] [PMID] [DOI:10.1126/science.1102941]
30. Frémaux, N., & Gerstner, W. (2016). Neuromodulated spike-timing-dependent plasticity, and theory of three-factor learning rules. Frontiers in Neural Circuits, 9(85), 1-19. [DOI:10.3389/fncir.2015.00085] [DOI:10.3389/fncir.2015.00085]
31. Frémaux, N., Sprekeler, H., & Gerstner, W. (2013). Reinforcement learning using a continuous time actor-critic framework with spiking neurons. PLoS Computational Biology, 9(4), e1003024. [DOI:10.1371/journal.pcbi.1003024] [PMID] [PMCID] [DOI:10.1371/journal.pcbi.1003024]
32. Friston, K. J. (2002). Dysfunctional connectivity in schizophrenia. World Psychiatry, 1(2), 66-71. [PMID] [PMCID] [PMID] [PMCID]
33. Gerstner, W., Kempter, R., van Hemmen, J. L., & Wagner, H. (1996). A neuronal learning rule for sub-millisecond temporal coding. Nature, 383(6595), 76-8. [DOI:10.1038/383076a0] [PMID] [DOI:10.1038/383076a0]
34. Gerstner, W., Kistler, W., Naud, R., & Paninski, L. (2014). Neuronal dynamics: From single neurons to networks and cognition. Cambridge: Cambridge University Press. [DOI:10.1017/CBO9781107447615] [DOI:10.1017/CBO9781107447615]
35. Greengard, P. (2001). The neurobiology of slow synaptic transmission. Science, 294(5544), 1024-30. [DOI:10.1126/science.294.5544.1024] [PMID] [DOI:10.1126/science.294.5544.1024]
36. Guillin, O., Abi‐Dargham, A., & Laruelle, M. (2007). Neurobiology of dopamine in schizophrenia. International Review of Neurobiology, 78(1), 1-39. [DOI:10.1016/S0074-7742(06)78001-1] [DOI:10.1016/S0074-7742(06)78001-1]
37. Gulledge, A. T., & Jaffe, D. B. (1998). Dopamine decreases the excitability of layer V pyramidal cells in the rat prefrontal cortex. Journal of Neuroscience, 18(21), 9139-51. [DOI:10.1523/JNEUROSCI.18-21-09139.1998] [PMID] [DOI:10.1523/JNEUROSCI.18-21-09139.1998]
38. Harris-Warrick, R. M., & Marder, E. (1991). Modulation of neural networks for behavior. Annual Review of Neuroscience, 14(1), 39-57. [DOI:10.1146/annurev.ne.14.030191.000351] [PMID] [DOI:10.1146/annurev.ne.14.030191.000351]
39. Hebb, D. O. (1949). The organization of behavior: A neuropsychological approach. New York: John Wiley & Sons. [PMCID] [PMCID]
40. Hepp, D. H., Foncke, E. M., Olde Dubbelink, K. T., van de Berg, W. D., Berendse, H. W., & Schoonheim, M. M. (2017). Loss of functional connectivity in patients with Parkinson disease and visual hallucinations. Radiology, 285(3), 170438. [DOI:10.1148/radiol.2017170438] [PMID] [DOI:10.1148/radiol.2017170438]
41. Hille, B. (2001). Ion channels of excitable membranes. Sunderland: Sinauer.
42. Hoffman, R. E., & McGlashan, T. H. (2001). Book Review: Neural network models of schizophrenia. The Neuroscientist, 7(5), 441-54. [DOI:10.1177/107385840100700513] [PMID] [DOI:10.1177/107385840100700513]
43. Howes, O. D., Kambeitz, J., Kim, E., Stahl, D., Slifstein, M., Abi-Dargham, A., et al. (2012). The nature of dopamine dysfunction in schizophrenia and what this means for treatment: meta-analysis of imaging studies. Archives of General Psychiatry, 69(8), 776-86. [DOI:10.1001/archgenpsychiatry.2012.169] [PMID] [PMCID] [DOI:10.1001/archgenpsychiatry.2012.169]
44. Izhikevich, E. M. (2007). Solving the distal reward problem through linkage of STDP and dopamine signaling. Cerebral Cortex, 17(10), 2443-52. [DOI:10.1093/cercor/bhl152] [PMID] [DOI:10.1093/cercor/bhl152]
45. Jay, T. M. (2003). Dopamine: A potential substrate for synaptic plasticity and memory mechanisms. Progress in Neurobiology, 69(6), 375-90. [DOI:10.1016/S0301-0082(03)00085-6] [DOI:10.1016/S0301-0082(03)00085-6]
46. Jay, T. M., Rocher, C., Hotte, M., Naudon, L., Gurden, H., & Spedding, M. (2004). Plasticity at hippocampal to prefrontal cortex synapses is impaired by loss of dopamine and stress: importance for psychiatric diseases. Neurotoxicity Research, 6(3), 233-44. [DOI:10.1007/BF03033225] [PMID] [DOI:10.1007/BF03033225]
47. Kaczmarek, L. K., & Levitan, I. B. (1987). Neuromodulation: The biochemical control of neuronal excitability. New York: Oxford University Press.
48. Kandel, E. R., Schwartz, J. H., & Jessell, T. M. (2000). Principles of neural science. New York: McGraw-Hill.
49. Kita, H., Oda, K., & Murase, K. (1999). Effects of dopamine agonists and antagonists on optical responses evoked in rat frontal cortex slices after stimulation of the subcortical white matter. Experimental Brain Research, 125(3), 383-8. [DOI:10.1007/s002210050694] [PMID] [DOI:10.1007/s002210050694]
50. Krames, E. S., Peckham, P. H., & Rezai, A. R. (2009). Neuromodulation. San Diego: Academic Press. [DOI:10.1016/B978-0-12-374248-3.00133-6] [DOI:10.1016/B978-0-12-374248-3.00133-6]
51. Law-Tho, D., Hirsch, J. C., & Crepel, F. (1994). Dopamine modulation of synaptic transmission in rat prefrontal cortex: an in vitro electrophysiological study. Neuroscience Research, 21(2), 151-60. [DOI:10.1016/0168-0102(94)90157-0] [DOI:10.1016/0168-0102(94)90157-0]
52. Lee, D. (2013). Decision making: From neuroscience to psychiatry. Neuron, 78(2), 233-48. [DOI:10.1016/j.neuron.2013.04.008] [PMID] [PMCID] [DOI:10.1016/j.neuron.2013.04.008]
53. Lisman, J., Grace, A. A., & Duzel, E. (2011). A neoHebbian framework for episodic memory; role of dopamine-dependent late LTP. Trends in Neurosciences, 34(10), 536-47. [DOI:10.1016/j.tins.2011.07.006] [PMID] [PMCID] [DOI:10.1016/j.tins.2011.07.006]
54. Loh, M., Rolls, E. T., & Deco, G. (2007). A dynamical systems hypothesis of schizophrenia. PLoS Computational Biology, 3(11), e228. [DOI:10.1371/journal.pcbi.0030228] [PMID] [PMCID] [DOI:10.1371/journal.pcbi.0030228]
55. Madadi Asl, M., Valizadeh, A., & Tass, P. A. (2017). Dendritic and axonal propagation delays determine emergent structures of neuronal networks with plastic synapses. Scientific Reports, 7(39682), 1-12. [DOI:10.1038/srep39682] [DOI:10.1038/srep39682]
56. Maia, T. V., & Frank, M. J. (2011). From reinforcement learning models to psychiatric and neurological disorders. Nature Neuroscience, 14(2), 154-62. [DOI:10.1038/nn.2723] [PMID] [PMCID] [DOI:10.1038/nn.2723]
57. Marder, E. (1998). From biophysics to models of network function. Annual Review of Neuroscience, 21(1), 25-45. [DOI:10.1146/annurev.neuro.21.1.25] [PMID] [DOI:10.1146/annurev.neuro.21.1.25]
58. Marder, E. (2012). Neuromodulation of neuronal circuits: back to the future. Neuron, 76(1), 1-11. [DOI:10.1016/j.neuron.2012.09.010] [PMID] [PMCID] [DOI:10.1016/j.neuron.2012.09.010]
59. Marder, E., & Thirumalai, V. (2002). Cellular, synaptic and network effects of neuromodulation. Neural Networks, 15(4), 479-93. [DOI:10.1016/S0893-6080(02)00043-6] [DOI:10.1016/S0893-6080(02)00043-6]
60. Markram, H., Lübke, J., Frotscher, M., & Sakmann, B. (1997). Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science, 275(5297), 213-5. [DOI:10.1126/science.275.5297.213] [PMID] [DOI:10.1126/science.275.5297.213]
61. McCormick, D. A. (1992). Neurotransmitter actions in the thalamus and cerebral cortex and their role in neuromodulation of thalamocortical activity. Progress in Neurobiology, 39(4), 337-88. [DOI:10.1016/0301-0082(92)90012-4] [DOI:10.1016/0301-0082(92)90012-4]
62. McCormick, D. A., & Pape, H. C. (1990). Properties of a hyperpolarization‐activated cation current and its role in rhythmic oscillation in thalamic relay neurones. The Journal of Physiology, 431(1), 291-318. [DOI:10.1113/jphysiol.1990.sp018331] [PMID] [PMCID] [DOI:10.1113/jphysiol.1990.sp018331]
63. Montague, P. R., Dayan, P., & Sejnowski, T. J. (1996). A framework for mesencephalic dopamine systems based on predictive Hebbian learning. Journal of Neuroscience, 16(5), 1936-47. [DOI:10.1523/JNEUROSCI.16-05-01936.1996] [PMID] [DOI:10.1523/JNEUROSCI.16-05-01936.1996]
64. Montague, P. R., Dolan, R. J., Friston, K. J., & Dayan, P. (2012). Computational psychiatry. Trends in Cognitive Sciences, 16(1), 72-80. [DOI:10.1016/j.tics.2011.11.018] [PMID] [PMCID] [DOI:10.1016/j.tics.2011.11.018]
65. Montague, P. R., Hyman, S. E., & Cohen, J. D. (2004). Computational roles for dopamine in behavioural control. Nature, 431(7010), 760-7. [DOI:10.1038/nature03015] [PMID] [DOI:10.1038/nature03015]
66. Nadim, F., & Bucher, D. (2014). Neuromodulation of neurons and synapses. Current Opinion in Neurobiology, 29(1), 48-56. [DOI:10.1016/j.conb.2014.05.003] [PMID] [PMCID] [DOI:10.1016/j.conb.2014.05.003]
67. Nicola, S. M., Surmeier, D. J., & Malenka, R. C. (2000). Dopaminergic modulation of neuronal excitability in the striatum and nucleus accumbens. Annual Review of Neuroscience, 23(1), 185-215. [DOI:10.1146/annurev.neuro.23.1.185] [PMID] [DOI:10.1146/annurev.neuro.23.1.185]
68. Noudoost, B., & Moore, T. (2011). Control of visual cortical signals by prefrontal dopamine. Nature, 474(7351), 372-5. [DOI:10.1038/nature09995] [PMID] [PMCID] [DOI:10.1038/nature09995]
69. Pedrosa, V., & Clopath, C. (2017). The role of neuromodulators in cortical plasticity. a computational perspective. Frontiers in Synaptic Neuroscience, 8(38), 1-9. [DOI:10.3389/fnsyn.2016.00038] [DOI:10.3389/fnsyn.2016.00038]
70. Popovych, O. V., & Tass, P. A. (2014). Control of abnormal synchronization in neurological disorders. Frontiers in Neurology, 5(268), 1-12. [DOI:10.3389/fneur.2014.00268] [DOI:10.3389/fneur.2014.00268]
71. Redish, A. D. (2004). Addiction as a computational process gone awry. Science, 306(5703), 1944-7. [DOI:10.1126/science.1102384] [PMID] [DOI:10.1126/science.1102384]
72. Rezende, D. J., & Gerstner, W. (2014). Stochastic variational learning in recurrent spiking networks. Frontiers in Computational Neuroscience, 8(38), 1-14.
73. Rolls, E. T., Loh, M., Deco, G., & Winterer, G. (2008). Computational models of schizophrenia and dopamine modulation in the prefrontal cortex. Nature Reviews Neuroscience, 9(9), 696-709. [DOI:10.1038/nrn2462] [PMID] [DOI:10.1038/nrn2462]
74. Sawaguchi, T., Matsumura, M., & Kubota, K. (1990). Catecholaminergic effects on neuronal activity related to a delayed response task in monkey prefrontal cortex. Journal of Neurophysiology, 63(6), 1385-1400. [DOI:10.1152/jn.1990.63.6.1401] [PMID] [DOI:10.1152/jn.1990.63.6.1401]
75. Sawaguchi, T., Matsumura, M., & Kubota, K. (1990). Effects of dopamine antagonists on neuronal activity related to a delayed response task in monkey prefrontal cortex. Journal of Neurophysiology, 63(6), 1401-12. [DOI:10.1152/jn.1990.63.6.1401] [DOI:10.1152/jn.1990.63.6.1401]
76. Schultz, W. (2002). Getting formal with dopamine and reward. Neuron, 36(2), 241-63. [DOI:10.1016/S0896-6273(02)00967-4] [DOI:10.1016/S0896-6273(02)00967-4]
77. Schultz, W. (2016).