Volume 10, Issue 1 (January & February 2019)                   BCN 2019, 10(1): 1-12 | Back to browse issues page


XML Print


Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:

Madadi Asl M, Vahabie A, Valizadeh A. Dopaminergic Modulation of Synaptic Plasticity, Its Role in Neuropsychiatric Disorders, and Its Computational Modeling. BCN 2019; 10 (1) :1-12
URL: http://bcn.iums.ac.ir/article-1-1062-en.html
1- Department of Physics, Institute for Advanced Studies in Basic Sciences (IASBS), Zanjan, Iran.
2- School of Cognitive Sciences, Institute for Research in Fundamental Sciences (IPM), Tehran, Iran.
Full-Text [PDF 723 kb]       |   Abstract (HTML) 
Full-Text:  
1. Introduction
The functional properties of neurons can be tuned based on the received input. The flexibility of the nervous system to adjust its function based on the input might even affect the structural connectivity patterns of the system, which is conceivable by the effect of synaptic plasticity on the global structures of neuronal networks (Bayati & Valizadeh, 2012; Bayati, Valizadeh, Abbassian, & Cheng, 2015; Madadi Asl, Valizadeh, & Tass, 2017). However, neuromodulators modify synaptic transmission and integration mechanisms, which in turn regulate intrinsic properties of the neural circuits including excitability of the pre- and postsynaptic neurons, probability of Neurotransmitters (NTs) release, and receptors’ response to neurotransmitters on multiple time scales (Marder & Thirumalai, 2002; Tritsch & Sabatini, 2012; Nadim & Bucher, 2014). 
Neuromodulation is referred to the modulation of the intrinsic properties of nervous system that might affect the performance of cells (Kaczmarek & Levitan, 1987; Krames, Peckham, & Rezai, 2009; Dayan, 2012; Marder, 2012). Therefore, neuromodulation controls the functional activity of the nervous system. The underlying mechanism involves targeted release of neuromodulators such as Acetylcholine (ACh), Dopamine (DA), Norepinephrine (NE), and serotonin (5-HT), which can attach to receptors of the postsynaptic neuron. The synaptic modulation of neuromodulators can be performed in multiple time scales, which affects both short-term and long-term dynamics of the nervous system. Among these neuromodulators associated with synaptic plasticity mechanisms, DA is the most important one involved in behavior and learning process (Montague, Hyman, & Cohen, 2004)
The role of DA in the Central Nervous System (CNS) is approved by numerous studies (Carlsson, Lindqvist, Magnusson, & Waldeck, 1958; Carlsson, 1959; Greengard, 2001), which is associated with attention, learning, and motivation. DA is involved in Reinforcement Learning (RL) that plays a significant role in the regulation of cognitive functions including working memory and decision making (Montague, Dayan, & Sejnowski, 1996; Berridge & Robinson, 1998; Dayan & Balleine, 2002; Tsai et al., 2009; Shohamy & Adcock, 2010; Flagel et al., 2011; Collins & Frank, 2016; Schultz, 2016)
In brain networks, the synaptic strengths can be modified based on the activity of neuronal populations that induce Long-Term Potentiation (LTP) and Long-Term Depression (LTD), which can also be affected by the action of several modulators such as DA. In dorsal striatum, DA signaling through specific pathways is required both for LTP and LTD (Pedrosa & Clopath, 2017).Experimental studies indicate that DA is involved in synaptic plasticity and memory mechanisms (Jay et al., 2004). It is shown that LTP of hippocampal-prefrontal synapses is driven by the level of mesocortical dopaminergic activity (Jay et al., 2004). 

 


While the appropriate function of DA signaling through nervous system leads to flawless synaptic plasticity and cognitive functions, malfunction of DA signaling can be potentially disadvantageous. DA’s dysfunction is engaged in several neuropsychiatric disorders such as Parkinson’s Disease (PD), drug addiction, schizophrenia, Attention-Deficit/Hyperactivity Disorder (ADHD), Obsessive-Compulsive Disorder (OCD), and Tourette’s Syndrome (TS) (Frank, 2005; Frank, Santamaria, O’Reilly, & Willcutt, 2007; Volkow, Fowler, Wang, Swanson, & Telang, 2007; Guillin, Abi‐Dargham, & Laruelle, 2007; Dagher & Robbins, 2009; Maia & Frank, 2011; Montague et al, 2012; Lee, 2013; Wang & Krystal, 2014). Patients with PD may show decreased levels of DA that could be due to the loss of dopaminergic neurons in the Substantia Nigra (SN).
Several studies on patients with PD reveal that DA is involved in motivational processes (Frank, 2005; Maia & Frank, 2011). On the contrary, patients with schizophrenia may show high DA levels that can be partially responsible for their condition. On the other hand, medical treatment of schizophrenia is often performed by the inhibition of DA’s activity (Halbach & Dermietzel, 2006). It is observed that addictive drugs enhance DA level in the forebrain structures (Di Chiara & Imperato, 1988; Volkow, Fowler, Wang, & Swanson, 2004). DA regulation mechanisms in people with ADHD are thought to be impaired (Biederman & Faraone, 2005). There are also several abnormalities in the DA signaling of some individuals with TS. Some people that are subjected to TS experience signs of OCD, while a significant percentage show symptoms of ADHD (Kandel, Schwartz, & Jessell, 2000)
Figure 1 classifies some features of neuronal dynamics that can be subjected to dopaminergic modulation of synaptic plasticity, and on the contrary, several neuropsychiatric disorders associated with disturbances in DA’s function. The current study briefly reviewed the effect of DA signaling on the cellular-level mechanisms of synaptic plasticity including the change in intrinsic properties of cells and introduced neuropsychiatric disorders which are thought to be triggered by malfunctioning of DA circulation in the nervous system. However, the current review study aimed at summarizing various computational models proposed in order to formulate the involvement of DA in the synaptic plasticity machinery, as well as its possible computational roles in several neuropsychiatric disorders. 

2. Cellular-Level Dopaminergic Modulation of Synaptic Plasticity
The substantial source of DA generation in the mammalian CNS is characterized by dopaminergic neurons reside in the midbrain (Chinta & Andersen, 2005).The Substantia Nigra pars compacta (SNc) and Ventral Tegmental Area (VTA) are two important centers that provide the significant amount of DA to the Basal Ganglia (BG) and forebrain embeds most of DA neurons in the CNS. In spite of their minority, midbrain dopaminergic neurons may have significant impact on the large-scale behavioral functions of the brain (Tritsch & Sabatini, 2012). The signaling of DA over nervous system is facilitated via a family of metabotropic G-Protein-Coupled Receptors (GPCRs) referred to as D1-D5 receptors (Cools & Van Rossum, 1976)
Typically, these DA receptors are grouped into two subsets based on their intrinsic characteristics including D1-like and D2-like types. D1-like type of DA receptors consists of D1 and D5 receptors, while D2-like family comprises D2-D4 receptors. One of the most important DA pathway signaling is by BG circuitry that is crucial for stimulus-response learning (Cohen & Frank, 2009). BG is involved in action selection (Maia & Frank, 2011) and its impairment is thought to be involved in several neuropsychiatric conditions that are linked to behavioral control (Stocco, Lebiere, & Anderson, 2010). The BG circuitry consists of direct (Go), indirect (NoGo), and hyperdirect (global NoGo) pathways from cortex to BG outputs (Albin, Young, & Penney, 1989; DeLong, 1990).
Neurons in direct pathway are influenced mostly by D1-like receptors, which boost desired actions in the ongoing state, while NoGo neurons are subjected to D2-like receptors and restrain actions that are inappropriate for the state (Maia & Frank, 2011). Therefore, the neuromodulatory signaling can regulate the intrinsic properties of cells at molecular level including the modulation of excitability of the pre- and postsynaptic neurons, probability of neurotransmitters release, and receptors’ response to neurotransmitters, which in turn determines the responsiveness of synaptic dynamics (Harris-Warrick & Marder, 1991; Marder, 1998; Jay, 2003).
Neurons can modify their intrinsic membrane characteristics based on the received input that can adjust the membrane excitability and its responses to synaptic input, which is under modulatory signals (Hille, 2001). Neuromodulators such as DA can modify the excitability of neurons by regulating the voltage- or ligand-gated channels (Kaczmarek & Levitan, 1987; Harris-Warrick & Marder). For instance, it is observed that DA exerts a significant influence on Striatal Projection Neurons (SPNs) intrinsic excitability (Nicola, Surmeier, & Malenka, 2000). The modulation of intrinsic activity properties of neurons may have significant impact on the behavioral state. In mammalian thalamus, the modulatory signaling leads to a transition between different patterns of firing frequency that might be related to the transition between awake and sleep phases (McCormick & Pape, 1990; McCormick, 1992; Steriade, McCormick, & Sejnowski, 1993)
DA signaling is also involved in the release process of neurotransmitters, which can change the release probability of both pre- and postsynaptic neurons by modifying their receptor signaling (Tritsch & Sabatini, 2012). However, the effects of DA on the transmitter release can be complicated since they highly depend on the context in which they are working. On the other hand, the major aim of DA signaling is to modify the postsynaptic neurotransmitters.
DA along with other neuromodulatory signals regulates function and trafficking of GABA, NMDA, and AMPA receptors through different pathways (Tritsch & Sabatini, 2012). Several physiological and computational studies revealed the ability of DA signaling to modify the ionic and synaptic currents (Sawaguchi, Matsumura, & Kubota, 1990a; Sawaguchi, Matsumura, & Kubota, 1990b; Yang & Seamans, 1996; Durstewitz, Seamans, & Sejnowski, 2000). DA’s activity may enhance the GABAergic transmitter release and related currents in the Prefrontal Cortex (PFC) (Zhou & Hablitz, 1999). On the other hand, DA signaling may reduce the Excitatory Postsynaptic Potential (EPSP) amplitude associated with AMPA- and NMDA-like synaptic currents (Cepeda, Radisavljevic, Peacock, Levine, & Buchwald, 1992; Kita, Oda, & Murase, 1999).
Although the detailed molecular-level mechanism governing the DA signaling in brain is unknown, several studies indicated that DA is able to regulate the physiological properties of different ionic and synaptic currents of PFC neuronal networks related to working memory tasks (Law-Tho, Hirsch, & Crepel, 1994; Yang & Seamans, 1996; Gulledge & Jaffe, 1998) or behavioral performance during RL in the PFC (Sawaguchi et al., 1990a; Sawaguchi et al., 1990b; Williams& Goldman-Rakic, 1995). It is demonstrated that certain DA levels in PFC are involved in the optimal control of visual cortical signals (Noudoost& Moore, 2011). There is also strong supporting evidence implicating that DA-related regulation of synaptic plasticity and neural signaling exist in several cortical layers (Soltani, Noudoost, & Moore, 2013). 

3. Computational models of DA-modulated synaptic plasticity
Synaptic plasticity is a process that adjusts synaptic strengths according to the correlated activity of pre- and postsynaptic firings. Based on Hebb’s rule (Hebb, 1949), simultaneous activation of pre- and postsynaptic neurons leads to an increase in the synaptic strength of the synapse connecting them. Spike-Timing-Dependent Plasticity (STDP) (Gerstner, Kempter, van Hemmen, & Wagner, 1996; Markram, Lübke, Frotscher, & Sakmann, 1997; Bi & Poo, 1998; Song, Miller, & Abbott, 2000) modifies synaptic strengths according to pre- and postsynaptic spike pairs related to Hebbian learning rules. 
STDP rule works based on the stimulus-stimulus learning protocol where naturally ignores the possible intervention of neuromodulatory signals that can be due to the events such as reward or punishment (Gerstner, Kistler, Naud, & Paninski, 2014; Frémaux & Gerstner, 2016). However, several studies revealed the crucial role of neuromodulators such as DA in novelty and reward-related processes (Berridge & Robinson, 1998; Dayan & Balleine, 2002; Schultz, 2002; Yu & Dayan, 2005; Tsai et al., 2009; Shohamy & Adcock, 2010; Flagel et al., 2011; Collins & Frank, 2016; Schultz, 2016). The involvement of DA signaling in synaptic plasticity and learning process can be taken into account by stimulus-reward learning protocol or the so-called neoHebbian formalism that depends on pre- and postsynaptic activities and the influence of neuromodulators (Montague et al., 1996; Lisman, Grace & Duzel, 2011; Frémaux & Gerstner, 2016)

3.1. DA-induced reinforcement learning
Several RL models are proposed in order to elucidate the role of DA modulation in behavioral and neural signaling during different learning tasks (Maia & Frank, 2011; Collins& Frank, 2014).These models such as Temporal-Difference RL (TDRL), or Temporal-Difference STDP (TD-STDP), assign a value to each specific action (Suttonand & Barto, 1998; Schultz, 2002; Frémaux & Gerstner, 2016). This assigned value can be altered by the induction of DA-related reward prediction errors in order to trigger learning. The choice between various actions is random that is completed by the comparison between different states. The mathematical formulation of RL rules are discussed shortly. Although, these models can justify different sets of data, each one has its own shortcomings in taking into account the modulatory influence of DA on the selected choice.

3.2. Mathematical formulation of DA-modulated Hebbian plasticity
The Hebbian learning rules work based on the joint activity (e.g. rate or timing) of pre- and postsynaptic neurons. However, introducing a new factor such as a neuromodulator defines a synaptic plasticity rule referred to as three-factor rule (Lisman et al., 2011;  Frémaux & Gerstner, 2016) that can be formulated as follows:

 
where  denotes the change in the strength of synapse connecting presynaptic neuron to the postsynaptic one, and parameter M is the modulatory agent, which can be a reward-related DA signal, for example. The function F denotes the nature of the learning rule. Assuming that the neuromodulatory signal is not involved, the conventional forms of synaptic plasticity rules can be recovered. One example is the classic STDP rule where the learning window F can be written as follows (Gerstner et al., 1996; Markram et al., 1997; Bi & Poo, 1998; Song, 2000):

 
where A± and τ± are amplitude and time constant of the synaptic modification. Δt=tpost-tpre denotes the difference between the timiΔng of pre- and postsynaptic firings.
The learning window F can be determined by several learning rules such as reward-modulated STDP (R-STDP) model (Farries & Fairhall, 2007; Florian, 2007; Izhikevich, 2007), which modifies synaptic strengths in the mean over several trials developed by the modulation of standard STDP with a reward-related DA term. The average of recent joint spike-timings in a conventional Hebbian STDP can be denoted by a function H (pre, post) (Frémaux & Gerstner, 2016). The modification of synaptic strengths is performed whenever the modulator M=R(t)-b, indicates a deviation of the reward R(t) from the baseline b (expected reward):

 
where the baseline can be set equal to the average reward b=(R(t)). The other types of learning protocols are spike-timing-dependent versions of TD learning, namely TD-STDP, where the change in synaptic strengths occurs after a single trial. Experimental studies show a reliable link between TD error arising in RL and activity patterns of dopaminergic neurons during reward-related experiments (Schultz, 2002; Schultz, 2016).It is shown that TD error, δ(t), also known as prediction error, can be written as follows (Montague et al., 1996):

 
where R(t) is reward, and V(t) is value expected at time t. Therefore, the explicit form of TD learning rule takes the form (Frémaux, Sprekeler, & Gerstner, 2013):

 
where δ(t) is TD error. Therefore, the role of dopaminergic neurons can be simulated by reward function R(t) in a TD model. However, a learning process could be triggered by a surprise signal that demonstrates the observed novelty compared to expected value, which apparently does not obey reward-based learning rules. In this case, the change in synaptic strengths depends on a variable that represents surprise S(t) (Rezende & Gerstner, 2014):

 
where this surprise term represents novelty with respect to the expected value and could be induced by the activity of dopaminergic neurons.

4. The Computational Role of DA in Neuropsychiatric Disorders
The crucial role of dopaminergic modulation in several brain processes and its participation in a variety of brain functions including cognitive functions and behavioral control indicates that impaired DA signaling can be potentially involved in several major brain disorders. Therefore, the study of clinical features of DA signaling can be a precious tool to reveal the complicated nature of dopaminergic modulation in the brain. Dysfunction of the dopaminergic circuitry can change the normal function of the nervous system, and as a result may cause several neurological and psychiatric disorders. However, along with the clinical applications of dopaminergic modulation of the nervous system, the role of DA in neuropsychiatric disorders can be simulated by computational models. Computational models may enhance the treatment techniques of the neuropsychiatric disorders since different agonists or antagonists of DA can be implemented and tested in the computer simulations, which do not require a human subject.

4.1. Parkinson’s disease 
Low levels of DA in the BG and in specific, striatum, are implicated in patients with PD that may have crucial impacts on the movement control or learning process (Frank, Seeberger, & O’reilly, 2004; Frank, 2005). It is believed that deterioration of dopaminergic neurons in PD may reduce the dopaminergic input to striatal areas, which can generate inappropriate tendency for NoGo pathway (Maia & Frank, 2011). Recently, it is shown that mean functional connectivity in patients with PD can be significantly lower than that of normal subjects (Hepp et al., 2017).PD symptoms are involved in cognitive and behavioral malfunctions due to the DA deficiency in motor system of the striatum. Although DA agonists can be used to treat patients with PD, standard therapy for PD is possible by desynchronization of pathologically strong synchronization of neurons performed using electrical High-Frequency (HF) Deep Brain Stimulation (DBS) techniques (Tass et al., 2012; Popovych & Tass, 2014; Tass, 2017).
Frankproposed a computational model in order to simulate PD with the hypothesis that the role of DA in PD lies in the reduction of DA dynamics that suppresses the learning process in BG through Go/NoGo pathways. The physiological constraints are implemented in a BG-based neural network model including Go/NoGo pathways that simulates the learning conditions. The simulation results show that the effect of low levels of DA on the PD model, and the simulated DA medication of PD are in accordance with the results observed in patients with PD. At the neural level, the model predicts that DA dynamics supports Hebbian learning by modulating synaptic dynamics in the indirect pathway: Suppressing NoGo neurons leads to LTD, while exciting them results in LTP, which is consistent with experimental observations.

4.2. Drug addiction
Drug addiction is entangled with difficulties in decision making which can be implicated by the potential effect of DA on the corticostriatal neurons. In fact, the crucial role of DA in reward-related process and RL generates a defective loop that the repeated use of addictive drugs can finally lead to compulsive and habitual behaviors (Montague et al., 2004). Most addictive drugs increase the level of DA in brain, therefore, the positive feedback interaction between DA neurons and such reinforcing drugs establishes a malfunctioning cycle, which results in an excessive increase in DA levels that may trigger persistent compulsive behaviors (Dagher & Robbins, 2009) implicated by a positive prediction error. Since these drugs are directly involved in DA-inspired RL, they provide a feedback loop that reinforces behavior leading to drug consumption and reveals the compulsive nature of drug addiction (Montague et al., 2004).
Redish proposed an RL-based computational model to simulate the behavioral states due to the increase of DA level using drug abuse. This model works on the basis of TDRL that its prediction error signal δ(t) is described by Equation (4) along with introducing a new variable yd that denotes raising the discounting factor y by the time delay d at time t-1:
 

 
In the TDRL model, learning ceases when the value function accurately estimates the reward, which produces no DA signal δ(t)=0. However, the enhancement of DA level due to the use of addictive drugs can be modeled by assuming that this mechanism induces a positive prediction error δ(t) that cannot be covered by changes in the corresponding value. In such a case, Equation (7) can be written as follows:

 
where D(t) is a DA flow occurring on entry into a new state at time t. Equation (8) reduces to typical TDRL if D(t)=0, but falls to a minimum δ(t) of D(t) if D(t)>0 that produces a positive prediction error δ(t)>0. Therefore, the values cause DA to flow tends to infinity. In conventional TDRL, the corresponding values leading to a normal reward asymptotically tend to a finite value. However, in the presented model, the values resulting to drug delivery increase with no limitation, establishing an unfavorable cycle of the reinforcing factor to choose an action that leads to the corresponding states (Redish, 2004).

4.3. Schizophrenia
Schizophrenia can be characterized by different cognitive symptoms including impaired attention and cognitive control, which may involve excessive levels of DA in the striatum, but reduced DA amounts in PFC (Maia & Frank, 2011). However, while the main cause of the disorder is not precisely known and remains controversial, it is indicated that schizophrenia might be related to some abnormalities in the dopaminergic synapses of the brain (Howes et al., 2012)
Some results from medical treatments indicate that schizophrenia can be due to excess activity of dopaminergic synapses (Squire et al., 2012); however, other studies postulated that some cognitive dysfunctions observed in schizophrenia are related to suppression of DA activity in PFC (Braver, Barch, & Cohen, 1999). It is shown that some of the clinical symptoms of schizophrenia could be simulated by stability considerations in computational models of neural networks in the sense that fluctuation of DA level can change the stability of corresponding attractor networks in the PFC, which is thought to be involved in some of the cognitive symptoms of schizophrenia (Loh, Rolls, & Deco, 2007; Rolls, Loh, Deco, & Winterer, 2008).
Hoffman and McGlashan introduced a computational model called spurious attractor states that can explain some of the symptoms of schizophrenia by the reduction of connectivity patterns in an attractor network that partly detaches the attractor network. This can be related to the disconnection hypothesis of schizophrenia in which the connectivity of some parts of the brain becomes relatively detached (Friston, 2002)
Another different approach called connectionist model simulates several cognitive impairments that occur in schizophrenia (Braver et al., 1999). It suggests that the cognitive symptoms of schizophrenia might be caused by a failure of cognitive control. However, more realistic and physiological neural models allow researchers to simulate and elaborate the effects of dopaminergic modulation on the ionic and synaptic currents (Rolls et al., 2008).

4.4. Attention-Deficit/ Hyperactivity Disorder (ADHD) 
ADHD is associated with abnormal hyperactivity and impulsivity behaviors that may be related to the dysfunction of PFC and its cortical units (Squire et al., 2012). The common viewpoint of ADHD is that it results from a defect in inhibitory system, which leads to executive malfunctions. However, other studies show that impaired function of DA reward pathways in patients with ADHD might be involved in motivation dysfunctions in this disorder (Volkow et al., 2011).ADHD is strongly heritable and is linked to the genetic transformations that interfere with NE or DA signaling. Reduced levels of DA in the brain can lead to difficulties in the control of impulsive behavior in patients with ADHD. It is shown that damage in DA neurons of the VTA can be the cause of hyperactivity and substandard response to stress (Blum et al., 2008).
Treatments of ADHD may include medication, therapy, or a combination of them, which can be a long-term approach. Symptoms of ADHD can often be treated with proper medications, which are typically based on the increased levels of DA. Conventional therapies are targeted at alleviating symptoms by regulating different neurotransmitters including DA (Blum et al., 2008).Although, there are several treatment procedures, an appropriate treatment is dopaminergic and serotonergic releaser combination therapy that consists of agonists. However, the most important pitfall of this approach is the possibility of medicine abuse due to the activation of DA neurons in CNS reward circuits.

4.5. Tourette’s syndrome
TS is indicated with abrupt and capricious movements referred to as tics. It is suggested that these tics may result from abnormal excitability or plasticity procedures in the direct (Go) and indirect (NoGo) pathways (Maia & Frank, 2011).This can be interpreted as an unfavorable positive feedback loop that the tic itself enhances the possibility of its occurrence through learning procedures, which in turn increases the tendency to tic by activation of the related motor systems. Evidence shows several abnormalities in the levels of DA signaling in some individuals with TS (Buse, Schoenefeld, Münchau, & Roessner, 2013). Several hypotheses concerning the dysfunction of the DA signaling in TS are developed including the dopaminergic modulation of pre- and postsynaptic receptors and firing patterns of DA neurons (Buse et al., 2013).On the other hand, some of the individuals subjected to TS experience signs of OCD, and a remarkable number comprise symptoms of ADHD (Buse et al., 2013).

5. Conclusion
The current review study considered the role of neuromodulator DA in one of the most important features of nervous system, i.e. the ability of neurons to tune their functional properties based on the required conditions of inputs. DA along with other modulatory signals regulates the performance of the synapses in order to function on a stimulus-reward learning basis, far from the classical view of stimulus-stimulus learning. Therefore, DA signaling in the learning process can regulate the efficiency of synaptic plasticity and learning procedures by modulating the cellular mechanisms that control the activity pattern of neuron. 
The impact of DA signaling on the synaptic plasticity at cellular level can be interpreted by understanding the molecular mechanisms that modify the intrinsic properties of cells. Such mechanisms are highly vulnerable to perturbations in their function, which can result in distorted circulation of DA signaling. In other words, the appropriate level of DA signaling across brain areas is crucial for proper function of the nervous system. However, the current review study summarized some important DA-induced computational models of synaptic plasticity associated with different learning rules. Finally, as it was mentioned earlier, disturbed DA signaling is associated with several important neuropsychiatric disorders.  Although diverse disorders such as PD, drug addiction, schizophrenia, ADHD, and TS might seem to have little in common, they are all entangled with disturbances in DA’s function through nervous system.
Experimental observations may provide a comprehensive, and in some cases controversial insights to the crucial role of neuromodulators such as DA in the nervous system, but mathematical formulation and computational modeling can be useful to understand their signaling machinery. The current review, however, summarized the significant findings regarding the understanding of computational role of DA in the brain through synaptic plasticity and its involvement in several neuropsychiatric disorders, which may shed light on the complicated task of DA in the nervous system. 

Ethical Considerations
Compliance with ethical guidelines

There is no ethical principle to be considered doing this research.

Funding
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Authors contributions
Conceptualization: All authors; Methodology: All authors; Investigation: Mojtaba Madadi Asl; Writing–original draft: All authors; Writing–review & editing: All authors; Resources: All authors; and Supervision: Abdol-Hossein Vahabie, Alireza Valizadeh.

Conflict of interest
The authors declared no conflict of interest.




References
  1. Albin, R. L., Young, A. B., & Penney, J. B. (1989). The functional anatomy of basal ganglia disorders. Trends in Neurosciences, 12(10), 366-75. [DOI:10.1016/0166-2236(89)90074-X]
  2. Bayati, M., & Valizadeh, A. (2012). Effect of synaptic plasticity on the structure and dynamics of disordered networks of coupled neurons. Physical Review E, 86(1), 011925. [DOI:10.1103/PhysRevE.86.011925] [PMID]
  3. Bayati, M., Valizadeh, A., Abbassian, A., & Cheng, S. (2015). Self-organization of synchronous activity propagation in neuronal networks driven by local excitation. Frontiers in Computational Neuroscience, 9(69), 1-15. [DOI:10.3389/fncom.2015.00069]
  4. Berridge, K. C., & Robinson, T. E. (1998). What is the role of dopamine in reward: Hedonic impact, reward learning, or incentive salience?. Brain Research Reviews, 28(3), 309-69. [DOI:10.1016/S0165-0173(98)00019-8]
  5. Bi, G. Q., & Poo, M. M. (1998). Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. Journal of Neuroscience, 18(24), 10464-72. [DOI:10.1523/JNEUROSCI.18-24-10464.1998] [PMID]
  6. Biederman, J., & Faraone, S. V. (2005). Attention-deficit hyperactivity disorder. Lancet, 366(9481), 237-48. [DOI:10.1016/S0140-6736(05)66915-2]
  7. Blum, K., Chen, A. L. C., Braverman, E. R., Comings, D. E., Chen, T. J., & Arcuri, V., et al. (2008). Attention-deficit-hyperactivity disorder and reward deficiency syndrome. Neuropsychiatric Disease and Treatment, 4(5), 893-917. [PMID] [PMCID]
  8. Braver, T. S., Barch, D. M., & Cohen, J. D. (1999). Cognition and control in schizophrenia: a computational model of dopamine and prefrontal function. Biological Psychiatry, 46(3), 312-28. [DOI:10.1016/S0006-3223(99)00116-X]
  9. Buse, J., Schoenefeld, K., Münchau, A., & Roessner, V. (2013). Neuromodulation in Tourette syndrome: Dopamine and beyond. Neuroscience & Biobehavioral Reviews, 37(6), 1069-84. [DOI:10.1016/j.neubiorev.2012.10.004] [PMID]
  10. Carlsson, A. (1959). The occurrence, distribution and physiological role of catecholamines in the nervous system. Pharmacological Reviews, 11(2), 490-3. [PMID]
  11. Carlsson, A., Lindqvist, M., Magnusson, T., & Waldeck, B. (1958). On the presence of 3-hydroxytyramine in brain. Science, 127(3296), 471-8. [DOI:10.1126/science.127.3296.471] [PMID]
  12. Cepeda, C., Radisavljevic, Z., Peacock, W., Levine, M. S., & Buchwald, N. A. (1992). Differential modulation by dopamine of responses evoked by excitatory amino acids in human cortex. Synapse, 11(4), 330-41. [DOI:10.1002/syn.890110408] [PMID]
  13. Chinta, S. J., & Andersen, J. K. (2005). Dopaminergic neurons. The International Journal of Biochemistry & Cell Biology, 37(5), 942-6. [DOI:10.1016/j.biocel.2004.09.009] [PMID]
  14. Cohen, M. X., & Frank, M. J. (2009). Neurocomputational models of basal ganglia function in learning, memory and choice. Behavioural Brain Research, 199(1), 141-56. [DOI:10.1016/j.bbr.2008.09.029] [PMID] [PMCID]
  15. Collins, A. G., & Frank, M. J. (2014). Opponent actor Learning (OpAL): modeling interactive effects of striatal dopamine on reinforcement learning and choice incentive. Psychological Review, 121(3), 337-66. [DOI:10.1037/a0037015] [PMID]
  16. Collins, A. G., & Frank, M. J. (2016). Surprise! Dopamine signals mix action, value and error. Nature Neuroscience, 19(1), 3-5. [DOI:10.1038/nn.4207] [PMID]
  17. Cools, A. R., & Van Rossum, J. M. (1976). Excitation-mediating and inhibition-mediating dopamine-receptors: a new concept towards a better understanding of electrophysiological, biochemical, pharmacological, functional and clinical data. Psychopharmacologia, 45(3), 243-54. [DOI:10.1007/BF00421135] [PMID]
  18. Dagher, A., & Robbins, T. W. (2009). Personality, addiction, dopamine: Insights from Parkinson’s disease. Neuron, 61(4), 502-10. [DOI:10.1016/j.neuron.2009.01.031] [PMID]
  19. Dayan, P. (2012). Twenty-five lessons from computational neuromodulation. Neuron, 76(1), 240-56. [DOI:10.1016/j.neuron.2012.09.027] [PMID]
  20. Dayan, P., & Balleine, B. W. (2002). Reward, motivation, and reinforcement learning. Neuron, 36(2), 285-98. [DOI:10.1016/S0896-6273(02)00963-7]
  21. DeLong, M. R. (1990). Primate models of movement disorders of basal ganglia origin. Trends in Neurosciences, 13(7), 281-5. [DOI:10.1016/0166-2236(90)90110-V]
  22. Di Chiara, G., & Imperato, A. (1988). Drugs abused by humans preferentially increase synaptic dopamine concentrations in the mesolimbic system of freely moving rats. Proceedings of the National Academy of Sciences, 85(14), 5274-8. [DOI:10.1073/pnas.85.14.5274]
  23. Durstewitz, D., Seamans, J. K., & Sejnowski, T. J. (2000). Dopamine-mediated stabilization of delay-period activity in a network model of prefrontal cortex. Journal of Neurophysiology, 83(3), 1733-50. [DOI:10.1152/jn.2000.83.3.1733] [PMID]
  24. Farries, M. A., & Fairhall, A. L. (2007). Reinforcement learning with modulated spike timing-dependent synaptic plasticity. Journal of Neurophysiology, 98(6), 3648-65. [DOI:10.1152/jn.00364.2007] [PMID]
  25. Flagel, S. B., Clark, J. J., Robinson, T. E., Mayo, L., Czuj, A., Willuhn, I., et al. (2011). A selective role for dopamine in stimulus-reward learning. Nature, 469(7328), 53-7. [DOI:10.1038/nature09588] [PMID] [PMCID]
  26. Florian, R. V. (2007). Reinforcement learning through modulation of spike-timing-dependent synaptic plasticity. Neural Computation, 19(6), 1468-1502. [DOI:10.1162/neco.2007.19.6.1468] [PMID]
  27. Frank, M. J. (2005). Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism. Journal of Cognitive Neuroscience, 17(1), 51-72. [DOI:10.1162/0898929052880093] [PMID]
  28. Frank, M. J., Santamaria, A., O’Reilly, R. C., & Willcutt, E. (2007). Testing computational models of dopamine and noradrenaline dysfunction in attention deficit/hyperactivity disorder. Neuropsychopharmacology, 32(7), 1583-99. [DOI:10.1038/sj.npp.1301278] [PMID]
  29. Frank, M. J., Seeberger, L. C., & O’reilly, R. C. (2004). By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science, 306(5703), 1940-3. [DOI:10.1126/science.1102941] [PMID]
  30. Frémaux, N., & Gerstner, W. (2016). Neuromodulated spike-timing-dependent plasticity, and theory of three-factor learning rules. Frontiers in Neural Circuits, 9(85), 1-19. [DOI:10.3389/fncir.2015.00085]
  31. Frémaux, N., Sprekeler, H., & Gerstner, W. (2013). Reinforcement learning using a continuous time actor-critic framework with spiking neurons. PLoS Computational Biology, 9(4), e1003024. [DOI:10.1371/journal.pcbi.1003024] [PMID] [PMCID]
  32. Friston, K. J. (2002). Dysfunctional connectivity in schizophrenia. World Psychiatry, 1(2), 66-71. [PMID] [PMCID]
  33. Gerstner, W., Kempter, R., van Hemmen, J. L., & Wagner, H. (1996). A neuronal learning rule for sub-millisecond temporal coding. Nature, 383(6595), 76-8. [DOI:10.1038/383076a0] [PMID]
  34. Gerstner, W., Kistler, W., Naud, R., & Paninski, L. (2014). Neuronal dynamics: From single neurons to networks and cognition. Cambridge: Cambridge University Press. [DOI:10.1017/CBO9781107447615]
  35. Greengard, P. (2001). The neurobiology of slow synaptic transmission. Science, 294(5544), 1024-30. [DOI:10.1126/science.294.5544.1024] [PMID]
  36. Guillin, O., Abi‐Dargham, A., & Laruelle, M. (2007). Neurobiology of dopamine in schizophrenia. International Review of Neurobiology, 78(1), 1-39. [DOI:10.1016/S0074-7742(06)78001-1]
  37. Gulledge, A. T., & Jaffe, D. B. (1998). Dopamine decreases the excitability of layer V pyramidal cells in the rat prefrontal cortex. Journal of Neuroscience, 18(21), 9139-51. [DOI:10.1523/JNEUROSCI.18-21-09139.1998] [PMID]
  38. Harris-Warrick, R. M., & Marder, E. (1991). Modulation of neural networks for behavior. Annual Review of Neuroscience, 14(1), 39-57. [DOI:10.1146/annurev.ne.14.030191.000351] [PMID]
  39. Hebb, D. O. (1949). The organization of behavior: A neuropsychological approach. New York: John Wiley & Sons. [PMCID]
  40. Hepp, D. H., Foncke, E. M., Olde Dubbelink, K. T., van de Berg, W. D., Berendse, H. W., & Schoonheim, M. M. (2017). Loss of functional connectivity in patients with Parkinson disease and visual hallucinations. Radiology, 285(3), 170438. [DOI:10.1148/radiol.2017170438] [PMID]
  41. Hille, B. (2001). Ion channels of excitable membranes. Sunderland: Sinauer.
  42. Hoffman, R. E., & McGlashan, T. H. (2001). Book Review: Neural network models of schizophrenia. The Neuroscientist, 7(5), 441-54. [DOI:10.1177/107385840100700513] [PMID]
  43. Howes, O. D., Kambeitz, J., Kim, E., Stahl, D., Slifstein, M., Abi-Dargham, A., et al. (2012). The nature of dopamine dysfunction in schizophrenia and what this means for treatment: meta-analysis of imaging studies. Archives of General Psychiatry, 69(8), 776-86. [DOI:10.1001/archgenpsychiatry.2012.169] [PMID] [PMCID]
  44. Izhikevich, E. M. (2007). Solving the distal reward problem through linkage of STDP and dopamine signaling. Cerebral Cortex, 17(10), 2443-52. [DOI:10.1093/cercor/bhl152] [PMID]
  45. Jay, T. M. (2003). Dopamine: A potential substrate for synaptic plasticity and memory mechanisms. Progress in Neurobiology, 69(6), 375-90. [DOI:10.1016/S0301-0082(03)00085-6]
  46. Jay, T. M., Rocher, C., Hotte, M., Naudon, L., Gurden, H., & Spedding, M. (2004). Plasticity at hippocampal to prefrontal cortex synapses is impaired by loss of dopamine and stress: importance for psychiatric diseases. Neurotoxicity Research, 6(3), 233-44. [DOI:10.1007/BF03033225] [PMID]
  47. Kaczmarek, L. K., & Levitan, I. B. (1987). Neuromodulation: The biochemical control of neuronal excitability. New York: Oxford University Press.
  48. Kandel, E. R., Schwartz, J. H., & Jessell, T. M. (2000). Principles of neural science. New York: McGraw-Hill.
  49. Kita, H., Oda, K., & Murase, K. (1999). Effects of dopamine agonists and antagonists on optical responses evoked in rat frontal cortex slices after stimulation of the subcortical white matter. Experimental Brain Research, 125(3), 383-8. [DOI:10.1007/s002210050694] [PMID]
  50. Krames, E. S., Peckham, P. H., & Rezai, A. R. (2009). Neuromodulation. San Diego: Academic Press. [DOI:10.1016/B978-0-12-374248-3.00133-6]
  51. Law-Tho, D., Hirsch, J. C., & Crepel, F. (1994). Dopamine modulation of synaptic transmission in rat prefrontal cortex: an in vitro electrophysiological study. Neuroscience Research, 21(2), 151-60. [DOI:10.1016/0168-0102(94)90157-0]
  52. Lee, D. (2013). Decision making: From neuroscience to psychiatry. Neuron, 78(2), 233-48. [DOI:10.1016/j.neuron.2013.04.008] [PMID] [PMCID]
  53. Lisman, J., Grace, A. A., & Duzel, E. (2011). A neoHebbian framework for episodic memory; role of dopamine-dependent late LTP. Trends in Neurosciences, 34(10), 536-47. [DOI:10.1016/j.tins.2011.07.006] [PMID] [PMCID]
  54. Loh, M., Rolls, E. T., & Deco, G. (2007). A dynamical systems hypothesis of schizophrenia. PLoS Computational Biology, 3(11), e228. [DOI:10.1371/journal.pcbi.0030228] [PMID] [PMCID]
  55. Madadi Asl, M., Valizadeh, A., & Tass, P. A. (2017). Dendritic and axonal propagation delays determine emergent structures of neuronal networks with plastic synapses. Scientific Reports, 7(39682), 1-12. [DOI:10.1038/srep39682]
  56. Maia, T. V., & Frank, M. J. (2011). From reinforcement learning models to psychiatric and neurological disorders. Nature Neuroscience, 14(2), 154-62. [DOI:10.1038/nn.2723] [PMID] [PMCID]
  57. Marder, E. (1998). From biophysics to models of network function. Annual Review of Neuroscience, 21(1), 25-45. [DOI:10.1146/annurev.neuro.21.1.25] [PMID]
  58. Marder, E. (2012). Neuromodulation of neuronal circuits: back to the future. Neuron, 76(1), 1-11. [DOI:10.1016/j.neuron.2012.09.010] [PMID] [PMCID]
  59. Marder, E., & Thirumalai, V. (2002). Cellular, synaptic and network effects of neuromodulation. Neural Networks, 15(4), 479-93. [DOI:10.1016/S0893-6080(02)00043-6]
  60. Markram, H., Lübke, J., Frotscher, M., & Sakmann, B. (1997). Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science, 275(5297), 213-5. [DOI:10.1126/science.275.5297.213] [PMID]
  61. McCormick, D. A. (1992). Neurotransmitter actions in the thalamus and cerebral cortex and their role in neuromodulation of thalamocortical activity. Progress in Neurobiology, 39(4), 337-88. [DOI:10.1016/0301-0082(92)90012-4]
  62. McCormick, D. A., & Pape, H. C. (1990). Properties of a hyperpolarization‐activated cation current and its role in rhythmic oscillation in thalamic relay neurones. The Journal of Physiology, 431(1), 291-318. [DOI:10.1113/jphysiol.1990.sp018331] [PMID] [PMCID]
  63. Montague, P. R., Dayan, P., & Sejnowski, T. J. (1996). A framework for mesencephalic dopamine systems based on predictive Hebbian learning. Journal of Neuroscience, 16(5), 1936-47. [DOI:10.1523/JNEUROSCI.16-05-01936.1996] [PMID]
  64. Montague, P. R., Dolan, R. J., Friston, K. J., & Dayan, P. (2012). Computational psychiatry. Trends in Cognitive Sciences, 16(1), 72-80. [DOI:10.1016/j.tics.2011.11.018] [PMID] [PMCID]
  65. Montague, P. R., Hyman, S. E., & Cohen, J. D. (2004). Computational roles for dopamine in behavioural control. Nature, 431(7010), 760-7. [DOI:10.1038/nature03015] [PMID]
  66. Nadim, F., & Bucher, D. (2014). Neuromodulation of neurons and synapses. Current Opinion in Neurobiology, 29(1), 48-56. [DOI:10.1016/j.conb.2014.05.003] [PMID] [PMCID]
  67. Nicola, S. M., Surmeier, D. J., & Malenka, R. C. (2000). Dopaminergic modulation of neuronal excitability in the striatum and nucleus accumbens. Annual Review of Neuroscience, 23(1), 185-215. [DOI:10.1146/annurev.neuro.23.1.185] [PMID]
  68. Noudoost, B., & Moore, T. (2011). Control of visual cortical signals by prefrontal dopamine. Nature, 474(7351), 372-5. [DOI:10.1038/nature09995] [PMID] [PMCID]
  69. Pedrosa, V., & Clopath, C. (2017). The role of neuromodulators in cortical plasticity. a computational perspective. Frontiers in Synaptic Neuroscience, 8(38), 1-9. [DOI:10.3389/fnsyn.2016.00038]
  70. Popovych, O. V., & Tass, P. A. (2014). Control of abnormal synchronization in neurological disorders. Frontiers in Neurology, 5(268), 1-12. [DOI:10.3389/fneur.2014.00268]
  71. Redish, A. D. (2004). Addiction as a computational process gone awry. Science, 306(5703), 1944-7. [DOI:10.1126/science.1102384] [PMID]
  72. Rezende, D. J., & Gerstner, W. (2014). Stochastic variational learning in recurrent spiking networks. Frontiers in Computational Neuroscience, 8(38), 1-14.
  73. Rolls, E. T., Loh, M., Deco, G., & Winterer, G. (2008). Computational models of schizophrenia and dopamine modulation in the prefrontal cortex. Nature Reviews Neuroscience, 9(9), 696-709. [DOI:10.1038/nrn2462] [PMID]
  74. Sawaguchi, T., Matsumura, M., & Kubota, K. (1990). Catecholaminergic effects on neuronal activity related to a delayed response task in monkey prefrontal cortex. Journal of Neurophysiology, 63(6), 1385-1400. [DOI:10.1152/jn.1990.63.6.1401] [PMID]
  75. Sawaguchi, T., Matsumura, M., & Kubota, K. (1990). Effects of dopamine antagonists on neuronal activity related to a delayed response task in monkey prefrontal cortex. Journal of Neurophysiology, 63(6), 1401-12. [DOI:10.1152/jn.1990.63.6.1401]
  76. Schultz, W. (2002). Getting formal with dopamine and reward. Neuron, 36(2), 241-63. [DOI:10.1016/S0896-6273(02)00967-4]
  77. Schultz, W. (2016). Dopamine reward prediction-error signalling: a two-component response. Nature Reviews Neuroscience, 17(3), 183-95. [DOI:10.1038/nrn.2015.26] [PMID] [PMCID]
  78. Shohamy, D., & Adcock, R. A. (2010). Dopamine and adaptive memory. Trends in Cognitive Sciences, 14(10), 464-72. [DOI:10.1016/j.tics.2010.08.002] [PMID]
  79. Soltani, A., Noudoost, B., & Moore, T. (2013). Dissociable dopaminergic control of saccadic target selection and its implications for reward modulation. Proceedings of the National Academy of Sciences, 110(9), 3579-84. [DOI:10.1073/pnas.1221236110] [PMID] [PMCID]
  80. Song, S., Miller, K. D., & Abbott, L. F. (2000). Competitive Hebbian learning through spike-timing-dependent synaptic plasticity. Nature Neuroscience, 3(9), 919-26. [DOI:10.1038/78829] [PMID]
  81. Squire, L., Berg, D., Bloom, F. E., Du Lac, S., Ghosh, A., & Spitzer, N. C. (2012). Fundamental of neuroscience. New York: Academic Press.
  82. Steriade, M., McCormick, D. A., & Sejnowski, T. J. (1993). Thalamocortical oscillations in the sleeping and aroused brain. Science, 262(5134), 679-85. [DOI:10.1126/science.8235588] [PMID]
  83. Stocco, A., Lebiere, C., & Anderson, J. R. (2010). Conditional routing of information to the cortex: A model of the basal ganglia’s role in cognitive coordination. Psychological Review, 117(2), 541-9. [DOI:10.1037/a0019077] [PMID] [PMCID]
  84. Sutton, R. S., & Barto, A. G. (1998). Reinforcement Learning: An Introduction. Cambridge: MIT Press.
  85. Tass, P. A., Qin, L., Hauptmann, C., Dovero, S., Bezard, E., & Boraud, T., et al. (2012). Coordinated reset has sustained aftereffects in Parkinsonian monkeys. Annals of Neurology, 72(5), 816-20. [DOI:10.1002/ana.23663] [PMID]
  86. Tass, P. A. (2017). Vibrotactile coordinated reset stimulation for the treatment of neurological diseases: Concepts and device specifications. Cureus, 9(8), 1-16. [DOI:10.7759/cureus.1535]
  87. Tritsch, N. X., & Sabatini, B. L. (2012). Dopaminergic modulation of synaptic transmission in cortex and striatum. Neuron, 76(1), 33-50. [DOI:10.1016/j.neuron.2012.09.023] [PMID] [PMCID]
  88. Tsai, H. C., Zhang, F., Adamantidis, A., Stuber, G. D., Bonci, A., & De Lecea, L., et al. (2009). Phasic firing in dopaminergic neurons is sufficient for behavioral conditioning. Science, 324(5930), 1080-4. [DOI:10.1126/science.1168878] [PMID] [PMCID]
  89. und Halbach, O. V. B., & Dermietzel, R. (2006). Neurotransmitters and neuromodulators: Handbook of Receptors and Biological Effects. New York: John Wiley & Sons. [DOI:10.1002/9783527609963]
  90. Volkow, N. D., Fowler, J. S., Wang, G. J., & Swanson, J. M. (2004). Dopamine in drug abuse and addiction: results from imaging studies and treatment implications. Molecular Psychiatry, 9(6), 557-69. [DOI:10.1038/sj.mp.4001507] [PMID]
  91. Volkow, N. D., Wang, G. J., Newcorn, J. H., Kollins, S. H., Wigal, T. L., Telang, F., et al. (2011). Motivation deficit in ADHD is associated with dysfunction of the dopamine reward pathway. Molecular Psychiatry, 16(11), 1147-54. [DOI:10.1038/mp.2010.97] [PMID] [PMCID]
  92. Wang, X. J., & Krystal, J. H. (2014). Computational psychiatry. Neuron, 84(3), 638-54. [DOI:10.1016/j.neuron.2014.10.018] [PMID] [PMCID]
  93. Williams, G. V., & Goldman-Rakic, P. S. (1995). Modulation of memory fields by dopamine Dl receptors in prefrontal cortex. Nature, 376(6541), 572-5. [DOI:10.1038/376572a0] [PMID]
  94. Yang, C. R., & Seamans, J. K. (1996). Dopamine D1 receptor actions in layers V-VI rat prefrontal cortex neurons in vitro: Modulation of dendritic-somatic signal integration. Journal of Neuroscience, 16(5), 1922-35. [PMID]
  95. Yu, A. J., & Dayan, P. (2005). Uncertainty, neuromodulation, and attention. Neuron, 46(4), 681-92. [DOI:10.1016/j.neuron.2005.04.026] [PMID]
  96. Zhou, F. M., & Hablitz, J. J. (1999). Dopamine modulation of membrane and synaptic properties of interneurons in rat cerebral cortex. Journal of Neurophysiology, 81(3), 967-976. [DOI:10.1152/jn.1999.81.3.967] [PMID]
Type of Study: Review | Subject: Computational Neuroscience
Received: 2017/10/31 | Accepted: 2018/02/5 | Published: 2019/01/1

References
1. Albin, R. L., Young, A. B., & Penney, J. B. (1989). The functional anatomy of basal ganglia disorders. Trends in Neurosciences, 12(10), 366-75. [DOI:10.1016/0166-2236(89)90074-X] [DOI:10.1016/0166-2236(89)90074-X]
2. Bayati, M., & Valizadeh, A. (2012). Effect of synaptic plasticity on the structure and dynamics of disordered networks of coupled neurons. Physical Review E, 86(1), 011925. [DOI:10.1103/PhysRevE.86.011925] [PMID] [DOI:10.1103/PhysRevE.86.011925]
3. Bayati, M., Valizadeh, A., Abbassian, A., & Cheng, S. (2015). Self-organization of synchronous activity propagation in neuronal networks driven by local excitation. Frontiers in Computational Neuroscience, 9(69), 1-15. [DOI:10.3389/fncom.2015.00069] [DOI:10.3389/fncom.2015.00069]
4. Berridge, K. C., & Robinson, T. E. (1998). What is the role of dopamine in reward: Hedonic impact, reward learning, or incentive salience?. Brain Research Reviews, 28(3), 309-69. [DOI:10.1016/S0165-0173(98)00019-8] [DOI:10.1016/S0165-0173(98)00019-8]
5. Bi, G. Q., & Poo, M. M. (1998). Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. Journal of Neuroscience, 18(24), 10464-72. [DOI:10.1523/JNEUROSCI.18-24-10464.1998] [PMID] [DOI:10.1523/JNEUROSCI.18-24-10464.1998]
6. Biederman, J., & Faraone, S. V. (2005). Attention-deficit hyperactivity disorder. Lancet, 366(9481), 237-48. [DOI:10.1016/S0140-6736(05)66915-2] [DOI:10.1016/S0140-6736(05)66915-2]
7. Blum, K., Chen, A. L. C., Braverman, E. R., Comings, D. E., Chen, T. J., & Arcuri, V., et al. (2008). Attention-deficit-hyperactivity disorder and reward deficiency syndrome. Neuropsychiatric Disease and Treatment, 4(5), 893-917. [PMID] [PMCID] [PMID] [PMCID]
8. Braver, T. S., Barch, D. M., & Cohen, J. D. (1999). Cognition and control in schizophrenia: a computational model of dopamine and prefrontal function. Biological Psychiatry, 46(3), 312-28. [DOI:10.1016/S0006-3223(99)00116-X] [DOI:10.1016/S0006-3223(99)00116-X]
9. Buse, J., Schoenefeld, K., Münchau, A., & Roessner, V. (2013). Neuromodulation in Tourette syndrome: Dopamine and beyond. Neuroscience & Biobehavioral Reviews, 37(6), 1069-84. [DOI:10.1016/j.neubiorev.2012.10.004] [PMID] [DOI:10.1016/j.neubiorev.2012.10.004]
10. Carlsson, A. (1959). The occurrence, distribution and physiological role of catecholamines in the nervous system. Pharmacological Reviews, 11(2), 490-3. [PMID] [PMID]
11. Carlsson, A., Lindqvist, M., Magnusson, T., & Waldeck, B. (1958). On the presence of 3-hydroxytyramine in brain. Science, 127(3296), 471-8. [DOI:10.1126/science.127.3296.471] [PMID] [DOI:10.1126/science.127.3296.471]
12. Cepeda, C., Radisavljevic, Z., Peacock, W., Levine, M. S., & Buchwald, N. A. (1992). Differential modulation by dopamine of responses evoked by excitatory amino acids in human cortex. Synapse, 11(4), 330-41. [DOI:10.1002/syn.890110408] [PMID] [DOI:10.1002/syn.890110408]
13. Chinta, S. J., & Andersen, J. K. (2005). Dopaminergic neurons. The International Journal of Biochemistry & Cell Biology, 37(5), 942-6. [DOI:10.1016/j.biocel.2004.09.009] [PMID] [DOI:10.1016/j.biocel.2004.09.009]
14. Cohen, M. X., & Frank, M. J. (2009). Neurocomputational models of basal ganglia function in learning, memory and choice. Behavioural Brain Research, 199(1), 141-56. [DOI:10.1016/j.bbr.2008.09.029] [PMID] [PMCID] [DOI:10.1016/j.bbr.2008.09.029]
15. Collins, A. G., & Frank, M. J. (2014). Opponent actor Learning (OpAL): modeling interactive effects of striatal dopamine on reinforcement learning and choice incentive. Psychological Review, 121(3), 337-66. [DOI:10.1037/a0037015] [PMID] [DOI:10.1037/a0037015]
16. Collins, A. G., & Frank, M. J. (2016). Surprise! Dopamine signals mix action, value and error. Nature Neuroscience, 19(1), 3-5. [DOI:10.1038/nn.4207] [PMID] [DOI:10.1038/nn.4207]
17. Cools, A. R., & Van Rossum, J. M. (1976). Excitation-mediating and inhibition-mediating dopamine-receptors: a new concept towards a better understanding of electrophysiological, biochemical, pharmacological, functional and clinical data. Psychopharmacologia, 45(3), 243-54. [DOI:10.1007/BF00421135] [PMID] [DOI:10.1007/BF00421135]
18. Dagher, A., & Robbins, T. W. (2009). Personality, addiction, dopamine: Insights from Parkinson's disease. Neuron, 61(4), 502-10. [DOI:10.1016/j.neuron.2009.01.031] [PMID] [DOI:10.1016/j.neuron.2009.01.031]
19. Dayan, P. (2012). Twenty-five lessons from computational neuromodulation. Neuron, 76(1), 240-56. [DOI:10.1016/j.neuron.2012.09.027] [PMID] [DOI:10.1016/j.neuron.2012.09.027]
20. Dayan, P., & Balleine, B. W. (2002). Reward, motivation, and reinforcement learning. Neuron, 36(2), 285-98. [DOI:10.1016/S0896-6273(02)00963-7] [DOI:10.1016/S0896-6273(02)00963-7]
21. DeLong, M. R. (1990). Primate models of movement disorders of basal ganglia origin. Trends in Neurosciences, 13(7), 281-5. [DOI:10.1016/0166-2236(90)90110-V] [DOI:10.1016/0166-2236(90)90110-V]
22. Di Chiara, G., & Imperato, A. (1988). Drugs abused by humans preferentially increase synaptic dopamine concentrations in the mesolimbic system of freely moving rats. Proceedings of the National Academy of Sciences, 85(14), 5274-8. [DOI:10.1073/pnas.85.14.5274] [DOI:10.1073/pnas.85.14.5274]
23. Durstewitz, D., Seamans, J. K., & Sejnowski, T. J. (2000). Dopamine-mediated stabilization of delay-period activity in a network model of prefrontal cortex. Journal of Neurophysiology, 83(3), 1733-50. [DOI:10.1152/jn.2000.83.3.1733] [PMID] [DOI:10.1152/jn.2000.83.3.1733]
24. Farries, M. A., & Fairhall, A. L. (2007). Reinforcement learning with modulated spike timing-dependent synaptic plasticity. Journal of Neurophysiology, 98(6), 3648-65. [DOI:10.1152/jn.00364.2007] [PMID] [DOI:10.1152/jn.00364.2007]
25. Flagel, S. B., Clark, J. J., Robinson, T. E., Mayo, L., Czuj, A., Willuhn, I., et al. (2011). A selective role for dopamine in stimulus-reward learning. Nature, 469(7328), 53-7. [DOI:10.1038/nature09588] [PMID] [PMCID] [DOI:10.1038/nature09588]
26. Florian, R. V. (2007). Reinforcement learning through modulation of spike-timing-dependent synaptic plasticity. Neural Computation, 19(6), 1468-1502. [DOI:10.1162/neco.2007.19.6.1468] [PMID] [DOI:10.1162/neco.2007.19.6.1468]
27. Frank, M. J. (2005). Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism. Journal of Cognitive Neuroscience, 17(1), 51-72. [DOI:10.1162/0898929052880093] [PMID] [DOI:10.1162/0898929052880093]
28. Frank, M. J., Santamaria, A., O'Reilly, R. C., & Willcutt, E. (2007). Testing computational models of dopamine and noradrenaline dysfunction in attention deficit/hyperactivity disorder. Neuropsychopharmacology, 32(7), 1583-99. [DOI:10.1038/sj.npp.1301278] [PMID] [DOI:10.1038/sj.npp.1301278]
29. Frank, M. J., Seeberger, L. C., & O'reilly, R. C. (2004). By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science, 306(5703), 1940-3. [DOI:10.1126/science.1102941] [PMID] [DOI:10.1126/science.1102941]
30. Frémaux, N., & Gerstner, W. (2016). Neuromodulated spike-timing-dependent plasticity, and theory of three-factor learning rules. Frontiers in Neural Circuits, 9(85), 1-19. [DOI:10.3389/fncir.2015.00085] [DOI:10.3389/fncir.2015.00085]
31. Frémaux, N., Sprekeler, H., & Gerstner, W. (2013). Reinforcement learning using a continuous time actor-critic framework with spiking neurons. PLoS Computational Biology, 9(4), e1003024. [DOI:10.1371/journal.pcbi.1003024] [PMID] [PMCID] [DOI:10.1371/journal.pcbi.1003024]
32. Friston, K. J. (2002). Dysfunctional connectivity in schizophrenia. World Psychiatry, 1(2), 66-71. [PMID] [PMCID] [PMID] [PMCID]
33. Gerstner, W., Kempter, R., van Hemmen, J. L., & Wagner, H. (1996). A neuronal learning rule for sub-millisecond temporal coding. Nature, 383(6595), 76-8. [DOI:10.1038/383076a0] [PMID] [DOI:10.1038/383076a0]
34. Gerstner, W., Kistler, W., Naud, R., & Paninski, L. (2014). Neuronal dynamics: From single neurons to networks and cognition. Cambridge: Cambridge University Press. [DOI:10.1017/CBO9781107447615] [DOI:10.1017/CBO9781107447615]
35. Greengard, P. (2001). The neurobiology of slow synaptic transmission. Science, 294(5544), 1024-30. [DOI:10.1126/science.294.5544.1024] [PMID] [DOI:10.1126/science.294.5544.1024]
36. Guillin, O., Abi‐Dargham, A., & Laruelle, M. (2007). Neurobiology of dopamine in schizophrenia. International Review of Neurobiology, 78(1), 1-39. [DOI:10.1016/S0074-7742(06)78001-1] [DOI:10.1016/S0074-7742(06)78001-1]
37. Gulledge, A. T., & Jaffe, D. B. (1998). Dopamine decreases the excitability of layer V pyramidal cells in the rat prefrontal cortex. Journal of Neuroscience, 18(21), 9139-51. [DOI:10.1523/JNEUROSCI.18-21-09139.1998] [PMID] [DOI:10.1523/JNEUROSCI.18-21-09139.1998]
38. Harris-Warrick, R. M., & Marder, E. (1991). Modulation of neural networks for behavior. Annual Review of Neuroscience, 14(1), 39-57. [DOI:10.1146/annurev.ne.14.030191.000351] [PMID] [DOI:10.1146/annurev.ne.14.030191.000351]
39. Hebb, D. O. (1949). The organization of behavior: A neuropsychological approach. New York: John Wiley & Sons. [PMCID] [PMCID]
40. Hepp, D. H., Foncke, E. M., Olde Dubbelink, K. T., van de Berg, W. D., Berendse, H. W., & Schoonheim, M. M. (2017). Loss of functional connectivity in patients with Parkinson disease and visual hallucinations. Radiology, 285(3), 170438. [DOI:10.1148/radiol.2017170438] [PMID] [DOI:10.1148/radiol.2017170438]
41. Hille, B. (2001). Ion channels of excitable membranes. Sunderland: Sinauer.
42. Hoffman, R. E., & McGlashan, T. H. (2001). Book Review: Neural network models of schizophrenia. The Neuroscientist, 7(5), 441-54. [DOI:10.1177/107385840100700513] [PMID] [DOI:10.1177/107385840100700513]
43. Howes, O. D., Kambeitz, J., Kim, E., Stahl, D., Slifstein, M., Abi-Dargham, A., et al. (2012). The nature of dopamine dysfunction in schizophrenia and what this means for treatment: meta-analysis of imaging studies. Archives of General Psychiatry, 69(8), 776-86. [DOI:10.1001/archgenpsychiatry.2012.169] [PMID] [PMCID] [DOI:10.1001/archgenpsychiatry.2012.169]
44. Izhikevich, E. M. (2007). Solving the distal reward problem through linkage of STDP and dopamine signaling. Cerebral Cortex, 17(10), 2443-52. [DOI:10.1093/cercor/bhl152] [PMID] [DOI:10.1093/cercor/bhl152]
45. Jay, T. M. (2003). Dopamine: A potential substrate for synaptic plasticity and memory mechanisms. Progress in Neurobiology, 69(6), 375-90. [DOI:10.1016/S0301-0082(03)00085-6] [DOI:10.1016/S0301-0082(03)00085-6]
46. Jay, T. M., Rocher, C., Hotte, M., Naudon, L., Gurden, H., & Spedding, M. (2004). Plasticity at hippocampal to prefrontal cortex synapses is impaired by loss of dopamine and stress: importance for psychiatric diseases. Neurotoxicity Research, 6(3), 233-44. [DOI:10.1007/BF03033225] [PMID] [DOI:10.1007/BF03033225]
47. Kaczmarek, L. K., & Levitan, I. B. (1987). Neuromodulation: The biochemical control of neuronal excitability. New York: Oxford University Press.
48. Kandel, E. R., Schwartz, J. H., & Jessell, T. M. (2000). Principles of neural science. New York: McGraw-Hill.
49. Kita, H., Oda, K., & Murase, K. (1999). Effects of dopamine agonists and antagonists on optical responses evoked in rat frontal cortex slices after stimulation of the subcortical white matter. Experimental Brain Research, 125(3), 383-8. [DOI:10.1007/s002210050694] [PMID] [DOI:10.1007/s002210050694]
50. Krames, E. S., Peckham, P. H., & Rezai, A. R. (2009). Neuromodulation. San Diego: Academic Press. [DOI:10.1016/B978-0-12-374248-3.00133-6] [DOI:10.1016/B978-0-12-374248-3.00133-6]
51. Law-Tho, D., Hirsch, J. C., & Crepel, F. (1994). Dopamine modulation of synaptic transmission in rat prefrontal cortex: an in vitro electrophysiological study. Neuroscience Research, 21(2), 151-60. [DOI:10.1016/0168-0102(94)90157-0] [DOI:10.1016/0168-0102(94)90157-0]
52. Lee, D. (2013). Decision making: From neuroscience to psychiatry. Neuron, 78(2), 233-48. [DOI:10.1016/j.neuron.2013.04.008] [PMID] [PMCID] [DOI:10.1016/j.neuron.2013.04.008]
53. Lisman, J., Grace, A. A., & Duzel, E. (2011). A neoHebbian framework for episodic memory; role of dopamine-dependent late LTP. Trends in Neurosciences, 34(10), 536-47. [DOI:10.1016/j.tins.2011.07.006] [PMID] [PMCID] [DOI:10.1016/j.tins.2011.07.006]
54. Loh, M., Rolls, E. T., & Deco, G. (2007). A dynamical systems hypothesis of schizophrenia. PLoS Computational Biology, 3(11), e228. [DOI:10.1371/journal.pcbi.0030228] [PMID] [PMCID] [DOI:10.1371/journal.pcbi.0030228]
55. Madadi Asl, M., Valizadeh, A., & Tass, P. A. (2017). Dendritic and axonal propagation delays determine emergent structures of neuronal networks with plastic synapses. Scientific Reports, 7(39682), 1-12. [DOI:10.1038/srep39682] [DOI:10.1038/srep39682]
56. Maia, T. V., & Frank, M. J. (2011). From reinforcement learning models to psychiatric and neurological disorders. Nature Neuroscience, 14(2), 154-62. [DOI:10.1038/nn.2723] [PMID] [PMCID] [DOI:10.1038/nn.2723]
57. Marder, E. (1998). From biophysics to models of network function. Annual Review of Neuroscience, 21(1), 25-45. [DOI:10.1146/annurev.neuro.21.1.25] [PMID] [DOI:10.1146/annurev.neuro.21.1.25]
58. Marder, E. (2012). Neuromodulation of neuronal circuits: back to the future. Neuron, 76(1), 1-11. [DOI:10.1016/j.neuron.2012.09.010] [PMID] [PMCID] [DOI:10.1016/j.neuron.2012.09.010]
59. Marder, E., & Thirumalai, V. (2002). Cellular, synaptic and network effects of neuromodulation. Neural Networks, 15(4), 479-93. [DOI:10.1016/S0893-6080(02)00043-6] [DOI:10.1016/S0893-6080(02)00043-6]
60. Markram, H., Lübke, J., Frotscher, M., & Sakmann, B. (1997). Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science, 275(5297), 213-5. [DOI:10.1126/science.275.5297.213] [PMID] [DOI:10.1126/science.275.5297.213]
61. McCormick, D. A. (1992). Neurotransmitter actions in the thalamus and cerebral cortex and their role in neuromodulation of thalamocortical activity. Progress in Neurobiology, 39(4), 337-88. [DOI:10.1016/0301-0082(92)90012-4] [DOI:10.1016/0301-0082(92)90012-4]
62. McCormick, D. A., & Pape, H. C. (1990). Properties of a hyperpolarization‐activated cation current and its role in rhythmic oscillation in thalamic relay neurones. The Journal of Physiology, 431(1), 291-318. [DOI:10.1113/jphysiol.1990.sp018331] [PMID] [PMCID] [DOI:10.1113/jphysiol.1990.sp018331]
63. Montague, P. R., Dayan, P., & Sejnowski, T. J. (1996). A framework for mesencephalic dopamine systems based on predictive Hebbian learning. Journal of Neuroscience, 16(5), 1936-47. [DOI:10.1523/JNEUROSCI.16-05-01936.1996] [PMID] [DOI:10.1523/JNEUROSCI.16-05-01936.1996]
64. Montague, P. R., Dolan, R. J., Friston, K. J., & Dayan, P. (2012). Computational psychiatry. Trends in Cognitive Sciences, 16(1), 72-80. [DOI:10.1016/j.tics.2011.11.018] [PMID] [PMCID] [DOI:10.1016/j.tics.2011.11.018]
65. Montague, P. R., Hyman, S. E., & Cohen, J. D. (2004). Computational roles for dopamine in behavioural control. Nature, 431(7010), 760-7. [DOI:10.1038/nature03015] [PMID] [DOI:10.1038/nature03015]
66. Nadim, F., & Bucher, D. (2014). Neuromodulation of neurons and synapses. Current Opinion in Neurobiology, 29(1), 48-56. [DOI:10.1016/j.conb.2014.05.003] [PMID] [PMCID] [DOI:10.1016/j.conb.2014.05.003]
67. Nicola, S. M., Surmeier, D. J., & Malenka, R. C. (2000). Dopaminergic modulation of neuronal excitability in the striatum and nucleus accumbens. Annual Review of Neuroscience, 23(1), 185-215. [DOI:10.1146/annurev.neuro.23.1.185] [PMID] [DOI:10.1146/annurev.neuro.23.1.185]
68. Noudoost, B., & Moore, T. (2011). Control of visual cortical signals by prefrontal dopamine. Nature, 474(7351), 372-5. [DOI:10.1038/nature09995] [PMID] [PMCID] [DOI:10.1038/nature09995]
69. Pedrosa, V., & Clopath, C. (2017). The role of neuromodulators in cortical plasticity. a computational perspective. Frontiers in Synaptic Neuroscience, 8(38), 1-9. [DOI:10.3389/fnsyn.2016.00038] [DOI:10.3389/fnsyn.2016.00038]
70. Popovych, O. V., & Tass, P. A. (2014). Control of abnormal synchronization in neurological disorders. Frontiers in Neurology, 5(268), 1-12. [DOI:10.3389/fneur.2014.00268] [DOI:10.3389/fneur.2014.00268]
71. Redish, A. D. (2004). Addiction as a computational process gone awry. Science, 306(5703), 1944-7. [DOI:10.1126/science.1102384] [PMID] [DOI:10.1126/science.1102384]
72. Rezende, D. J., & Gerstner, W. (2014). Stochastic variational learning in recurrent spiking networks. Frontiers in Computational Neuroscience, 8(38), 1-14.
73. Rolls, E. T., Loh, M., Deco, G., & Winterer, G. (2008). Computational models of schizophrenia and dopamine modulation in the prefrontal cortex. Nature Reviews Neuroscience, 9(9), 696-709. [DOI:10.1038/nrn2462] [PMID] [DOI:10.1038/nrn2462]
74. Sawaguchi, T., Matsumura, M., & Kubota, K. (1990). Catecholaminergic effects on neuronal activity related to a delayed response task in monkey prefrontal cortex. Journal of Neurophysiology, 63(6), 1385-1400. [DOI:10.1152/jn.1990.63.6.1401] [PMID] [DOI:10.1152/jn.1990.63.6.1401]
75. Sawaguchi, T., Matsumura, M., & Kubota, K. (1990). Effects of dopamine antagonists on neuronal activity related to a delayed response task in monkey prefrontal cortex. Journal of Neurophysiology, 63(6), 1401-12. [DOI:10.1152/jn.1990.63.6.1401] [DOI:10.1152/jn.1990.63.6.1401]
76. Schultz, W. (2002). Getting formal with dopamine and reward. Neuron, 36(2), 241-63. [DOI:10.1016/S0896-6273(02)00967-4] [DOI:10.1016/S0896-6273(02)00967-4]
77. Schultz, W. (2016).

Add your comments about this article : Your username or Email:
CAPTCHA

Send email to the article author


Rights and permissions
Creative Commons License This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

© 2024 CC BY-NC 4.0 | Basic and Clinical Neuroscience

Designed & Developed by : Yektaweb