V Srinivasa Chakravarthy

Although empirical and neural studies show that serotonin (5HT) plays many functional roles in the brain, prior computational models mostly focus on its role in behavioral inhibition. In this study, we present a model of risk based decision making in a modified Reinforcement Learning (RL)-framework. The model depicts the roles of dopamine (DA) and serotonin (5HT) in Basal Ganglia (BG). In this model, the DA signal is represented by the temporal difference error (δ), while the 5HT signal is represented by a parameter (α) that controls risk prediction error. This formulation that accommodates both 5HT and DA reconciles some of the diverse roles of 5HT particularly in connection with the BG system. We apply the model to different experimental paradigms used to study the role of 5HT: (1) Risk-sensitive decision making, where 5HT controls risk assessment, (2) Temporal reward prediction, where 5HT controls time-scale of reward prediction, and (3) Reward/Punishment sensitivity, in which the punishment prediction error depends on 5HT levels. Thus the proposed integrated RL model reconciles several existing theories of 5HT and DA in the BG. © 2014 Balasubramani, Chakravarthy, Ravindran and Moustafa.

We model the role played by the Basal Ganglia (BG) in the generation of voluntary saccadic eye movements. The BG model explicitly represents key nuclei like the striatum (caudate), Substantia Nigra pars reticulata (SNr) and compata (SNc), the Subthalamic Nucleus (STN), the two pallidal nuclei and Superior Colliculus. The model is cast within the Reinforcement Learning (RL) framework, with the dopamine representing the temporal difference error, the striatum serving as the critic, and the indirect pathway playing the role of the explorer. Performance of the model is evaluated on a set of tasks such as feature and conjunction searches, directional selectivity and a successive saccade task. Behavioral phenomena such as independence of search time on number of distractors in feature search and linear increase in search time with number of distractors in conjunction search are observed. It is also seen that saccadic reaction times are longer and search efficiency is impaired on diminished BG contribution, which corroborates with reported data obtained from Parkinson's Disease (PD) patients. © 2011 Elsevier Ltd.

Many computational models of the basal ganglia (BG) have been proposed over the past twenty-five years. While computational neuroscience models have focused on closely matching the neurobiology of the BG, computational cognitive neuroscience (CCN) models have focused on how the BG can be used to implement cognitive and motor functions. This review article focuses on CCN models of the BG and how they use the neuroanatomy of the BG to account for cognitive and motor functions such as categorization, instrumental conditioning, probabilistic learning, working memory, sequence learning, automaticity, reaching, handwriting, and eye saccades. A total of 19 BG models accounting for one or more of these functions are reviewed and compared. The review concludes with a discussion of the limitations of existing CCN models of the BG and prescriptions for future modeling, including the need for computational models of the BG that can simultaneously account for cognitive and motor functions, and the need for a more complete specification of the role of the BG in behavioral functions. © 2013 Helie, Chakravarthy and Moustafa.

Basal ganglia circuit is an important subcortical system of the brain thought to be responsible for reward-based learning. Striatum, the largest nucleus of the basal ganglia, serves as an input port that maps cortical information. Microanatomical studies show that the striatum is a mosaic of specialized input-output structures called striosomes and regions of the surrounding matrix called the matrisomes. We have developed a computational model of the striatum using layered self-organizing maps to capture the center-surround structure seen experimentally and explain its functional significance. We believe that these structural components could build representations of state and action spaces in different environments. The striatummodel is then integrated with other components of basal ganglia, making it capable of solving reinforcement learning tasks. We have proposed a biologically plausible mechanism of action-based learning where the striosome biases the matrisome activity toward a preferred action. Several studies indicate that the striatum is critical in solving context dependent problems. We build on this hypothesis and the proposed model exploits the modularity of the striatum to efficiently solve such tasks.

Basal ganglia (BG) constitute a network of seven deep brain nuclei involved in a variety of crucial brain functions including: action selection, action gating, reward based learning, motor preparation, timing, etc. In spite of the immense amount of data available today, researchers continue to wonder how a single deep brain circuit performs such a bewildering range of functions. Computational models of BG have focused on individual functions and fail to give an integrative picture of BG function. A major breakthrough in our understanding of BG function is perhaps the insight that activities of mesencephalic dopaminergic cells represent some form of 'reward' to the organism. This insight enabled application of tools from 'reinforcement learning,' a branch of machine learning, in the study of BG function. Nevertheless, in spite of these bright spots, we are far from the goal of arriving at a comprehensive understanding of these 'mysterious nuclei.' A comprehensive knowledge of BG function has the potential to radically alter treatment and management of a variety of BG-related neurological disorders (Parkinson's disease, Huntington's chorea, etc.) and neuropsychiatric disorders (schizophrenia, obsessive compulsive disorder, etc.) also. In this article, we review the existing modeling literature on BG and hypothesize an integrative picture of the function of these nuclei. © 2010 Springer-Verlag.

We present a model of basal ganglia (BG) that departs from the classical Go/NoGo picture of the function of its key pathways-the Direct and Indirect Pathways (DP and IP). Between the Go and NoGo regimes, we posit a third Explore regime, which denotes random exploration of action alternatives. Striatal dopamine (DA) is assumed to switch between DP and IP activation. The IP is modeled as a loop of the subthalamic nucleus (STN) and the Globus Pallidus externa (GPe). Simulations reveal that while the model displays Go and NoGo regimes for extreme values of DA, at intermediate values of DA, it exhibits exploratory behavior, which originates from the chaotic activity of the STN-GPe loop. We describe a series of BG models based on Go/Explore/NoGo approach, to explain the role of BG in three cases: (1) a simple action selection task, (2) reaching, and (3) willed action. © 2013 Elsevier B.V.

Parkinsonian handwriting is typically characterized by micrographia, jagged line contour, and unusual fluctuations in pen velocity. In this paper we present a computational model of handwriting generation that highlights the role of the basal ganglia, particularly the indirect pathway. Whereas reduced dopamine levels resulted in reduced letter size, transition of STN-GPe dynamics from desynchronized (normal) to synchronized (PD) condition resulted in increased fluctuations in velocity in the model. We also present handwriting data from PD patients (n = 34) who are at various stages of disease and had taken medication various lengths of time before the handwriting sessions. The patient data are compared with those of age-matched controls. PD handwriting statistically exhibited smaller size and larger velocity fluctuation compared to normal handwriting. © 2009 Elsevier B.V. All rights reserved.

We present a neural network model of basal ganglia that departs from the classical Go/NoGo picture of the function of its key pathways-the direct pathway (DP) and the indirect pathway (IP). In classical descriptions of basal ganglia function, the DP is known as the Go pathway since it facilitates movement and the IP is called the NoGo pathway since it inhibits movement. Between these two regimes, in the present model, we posit that there is a third Explore regime, which denotes random exploration of the space of actions. The proposed model is instantiated in a simple action selection task. Striatal dopamine is assumed to switch between DP and IP activation. The IP is modeled as a loop of the subthalamic nucleus (STN) and the globus pallidus externa (GPe), capable of producing chaotic activity. Simulations reveal that, while the system displays Go and NoGo regimes for extreme values of dopamine, at intermediate values of dopamine, it exhibits a new Explore regime denoting a random exploration of the space of action alternatives. The exploratory dynamics originates from the chaotic activity of the STN-GPe loop. When applied to the standard card choice experiment used in the imaging studies of . Daw, O'Doherty, Dayan, Seymour, and Dolan (2006), the model favorably describes the exploratory behavior of human subjects. © 2012 Elsevier Ltd.

Experimental data show that perceptual cues can either exacerbate or ameliorate freezing of gait (FOG) in Parkinson’s Disease (PD). For example, simple visual stimuli like stripes on the floor can alleviate freezing whereas complex stimuli like narrow doorways can trigger it. We present a computational model of the cognitive and motor cortico-basal ganglia loops that explains the effects of sensory and cognitive processes on FOG. The model simulates strong causative factors of FOG including decision conflict (a disagreement of various sensory stimuli in their association with a response) and cognitive load (complexity of coupling a stimulus with downstream mechanisms that control gait execution). Specifically, the model simulates gait of PD patients (freezers and non-freezers) as they navigate a series of doorways while simultaneously responding to several Stroop word cues in a virtual reality setup. The model is based on an actor-critic architecture of Reinforcement Learning involving Utility-based decision making, where Utility is a weighted sum of Value and Risk functions. The model accounts for the following experimental data: (a) the increased foot-step latency seen in relation to high conflict cues, (b) the high number of motor arrests seen in PD freezers when faced with a complex cue compared to the simple cue, and (c) the effect of dopamine medication on these motor arrests. The freezing behavior arises as a result of addition of task parameters (doorways and cues) and not due to inherent differences in the subject group. The model predicts a differential role of risk sensitivity in PD freezers and non-freezers in the cognitive and motor loops. Additionally this first-of-its-kind model provides a plausible framework for understanding the influence of cognition on automatic motor actions in controls and Parkinson’s Disease.

We present a computational model of basal ganglia as a key player in exploratory behavior. The model describes exploration of a virtual rat in a simulated water pool experiment. The virtual rat is trained using a reward-based or reinforcement learning paradigm which requires units with stochastic behavior for exploration of the system's state space. We model the Subthalamic Nucleus-Globus Pallidus externa (STN-GPe) segment of the basal ganglia as a pair of neuronal layers with oscillatory dynamics, exhibiting a variety of dynamic regimes such as chaos, traveling waves and clustering. Invoking the property of chaotic systems to explore state-space, we suggest that the complex exploratory dynamics of STN-GPe system in conjunction with dopamine-based reward signaling from the Substantia, Nigra pars compacta (SNc) present the two key ingredients of a reinforcement learning system. © World Scientific Publishing Company.