Experimental paradigms

Participants were instructed to perform reaching movements using a 2-dimensional joystick under three experimental paradigms: i) decision-making task (action selection), ii) Eriksen flanker task (decision conflict) and iii) stop-signal task (outright stopping) (Fig 1). In the decision-making task, participants had to respond to arrow stimuli presented on a computer screen by freely moving the joystick towards the left or right direction. Choice trials were interleaved with instructed trials in which all arrows pointed to the same direction. In the Eriksen flanker task, flanking arrows were presented on the screen, all pointing to the same direction. A target arrow was then presented to indicate the direction to move, either in the same (no conflict, congruent trials) or opposite (conflict, incongruent trials) direction as the flanking arrows. Finally, in the stop signal task, the participants were instructed to reach towards the direction of the arrows. In a minority of trials, the color of the arrows turned red after a short delay (stop signal delay, SSD), and the action had to be abandoned immediately. If the participants successfully inhibited their actions, the SSD increased by 50ms, making the next stopping trial more challenging. If the participants failed to inhibit their actions, the SSD decreased by 50ms, making the next stopping trial easier.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Experimental setup for action regulation tasks that require action inhibition.

A: Decision-making task, including instructed and choice trials. B: An arrow version of the Eriksen flanker task, including congruent (the flanker arrows point to the same direction as the central arrow) and incongruent (the flanker arrows point to the opposite direction from the central arrow) trials. C: A stop signal task with instructed trials. Individuals are prompted to stop the action when the arrows turn red.

https://doi.org/10.1371/journal.pcbi.1010111.g001

Motor behavior of neurotypical individuals and PD patients in action regulation tasks

We computed the reaction time (RT) for initiating an action as an index of motor preparation/programming of reaching actions. RT was computed as the time interval between the presentation of the target arrows on the screen and the initiation of the reaching movement. We computed the RT distribution across all participants in each group in the decision-making task (S1 Fig) and found that choice trials had longer RT on average than instructed trials in both populations (Fig 2A) (two-way ANOVA, p<0.001). However, although the neurotypical individuals responded faster than the PD patients in the instructed trials (two-way ANOVA, p<0.001), we found no significant difference in RT between the two groups in the choice trials (two-way ANOVA, p = 0.878), (Fig 2A). Additionally, we evaluated the performance of the participants by measuring the success rate of the instructed and choice trials. We found that both groups performed equally well on both conditions—i.e., the percentage of correct trials for neurotypical individuals was 98.80% ± 3.58% (mean ± SD) in the instructed trials and 94.65% ± 18.04% in the choice trials. The percentage of correct trials for PD patients was 98.33% ± 4.98% in the instructed trials and 97.69% ± 6% in the choice trials (two-tailed t-test comparing success rate between neurotypical individuals and PD patients; p = 0.7167 in instructed trials and p = 0.5297 in choice trials). Furthermore, we explored how often participants changed their initial choice and corrected their movement to the opposite direction. We found that both groups rarely changed their mind after initiating an action in the choice trials—i.e., the percentage of “change of mind” trials was 1.11% ± 1.88% for neurotypical individuals and 1.67% ± 3.07% for PD patients (two-tailed t-test comparing percentage of change of mind trials between neurotypical individuals and PD patients, p = 0.4733).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Behavioral findings from the decision-making task, the Eriksen flanker task and the stop signal task.

A: Bar plots of the mean RT for neurotypical individuals (Neurotypical) and PD patients (PD) across all participants in each group, in the instructed and choice trials of the decision-making task. B: Bar plots of the mean RT for neurotypical individuals (Neurotypical) and PD patients (PD) across all participants in each group, in the congruent and incongruent trials of the Eriksen flanker task. C: Bar plots of the mean RT for neurotypical individuals (Neurotypical) and PD patients (PD) across all participants in each group, in the go trials of the stop signal task. Error bars correspond to standard error (SE) across all participants in each group. White dots represent mean RT for each participant.

https://doi.org/10.1371/journal.pcbi.1010111.g002

In the Eriksen flanker task, we computed the RT distribution across all participants in each group (S1 Fig) and found that both groups exhibited shorter RT on average in the congruent trials than in the incongruent trials (Fig 2B) (two-way ANOVA, p<0.001 for both neurotypical individuals and PD patients). However, PD patients had slower responses than neurotypical individuals in both congruent and incongruent trials (two-way ANOVA, p<0.01 for congruent trials, p<0.05 for incongruent trials). We also found no significant difference on the success rate between the two populations in congruent trials and incongruent trials. The average percentage of correct trials in the neurotypical population was 98.98% ± 2.07% (mean ± SD) in congruent trials and 96.39% ± 4.85% in incongruent trials, whereas in the PD patient population was 99.49% ± 1.14% in congruent trials and 98.72% ± 2.59% in incongruent trials (two-tailed t-test comparing success rate between neurotypical individuals and PD patients; p = 0.3799 for congruent trials and p = 0.0895 for incongruent trials). Finally, we found no significant differences between the two populations regarding changing of mind in the incongruent trials. The neurotypical individuals switched actions to the opposite direction 9.01% ± 12.22% of the time (mean ± SD) whereas the PD patients changed their minds 9.23% ± 7.73% of the time (two-tailed t-test comparing percentage of change of mind trials between neurotypical individuals and PD patients, p = 0.9502).

We also computed the RT distribution in the go trials across all participants in the stop signal task (S1 Fig). Interestingly, the results showed that neurotypical individuals had slower responses than PD patients (Fig 2C) (two-tailed t-test, p<0.001). In particular, the neurotypical population seems to have strategically slowed down their responses in the go trials by 233 ms on average in order to be more successful in inhibiting their response once a stop signal is presented (two-tailed t-test on RT between instructed trials and go trials for the neurotypical population, p<0.001). On the other hand, PD patients exhibited much subtler modification of their response between instructed trials (decision-making task) and go trials (stop signal task)—the RT for go trials increased only by 47 ms on average compared to instructed trials (two-tailed t-test, p<0.001), suggesting that the anticipation of the stop signal had smaller effect on their motor planning behavior.

These findings predict that PD patients will perform worse in stop trials than neurotypical individuals, since a lower probability of stopping has often been associated with faster responses in go trials [49–51]. To test this hypothesis, we computed the probability to stop an action for different SSD values across all participants in each group. The results showed that the probability to successfully stop an action was inversely correlated with SSD, and consistent with the hypothesis, PD patients exhibited lower probability of stopping an action compared to neurotypical individuals (Fig 3).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Probability to successfully stop an action as a function of the SSD.

The probability to successfully stop an action as a function of the SSD for neurotypical individuals (Neurotypical, blue) and PD patients (PD, red).

https://doi.org/10.1371/journal.pcbi.1010111.g003

An integrated neurodynamical theory of action regulation predicts motor behavior

Our findings require a computational theory that could explain the mechanism of action regulation that involves motor inhibition and predicts how disruption of this mechanism can lead to motor impairments in PD patients. Building on our previous successful work in modeling visuomotor tasks [47, 48], we developed a neurodynamical theory to unify the action regulation mechanism that involves motor inhibition. The theory builds on the affordance competition hypothesis, according to which multiple actions are formed concurrently and compete over time until one has sufficient evidence to win the competition [6, 7, 12]. It combines dynamic neural field theory with stochastic optimal control theory and its architectural organization is illustrated in Fig 4. The basic architecture of this theory consists of a set of DNFs. Each DNF simulates the dynamic evolution of firing rate activity of a network of neurons over a continuous space with local excitation and surround inhibition. It consists of 181 neurons—with exception of the context signal field and the pause field—and each of them has a preferred direction between 0° and 180°. The “spatial sensory input” field encodes the angular representation of the competing actions (i.e., left vs. right movements in our study). The “expected outcome” field encodes the expected reward for reaching to a particular direction. The outputs of these two fields send excitatory projections (green arrows) to the “reach planning” field in a topological manner.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Model architecture.

The architectural organization of the neurodynamical theory to model tasks that involve action inhibition, such as decisions between competing options, decisions in the presence of conflicting information and outright stopping of actions.

https://doi.org/10.1371/journal.pcbi.1010111.g004

We also added to the model architecture a Basal Ganglia (BG)-type mechanism for implementing the inhibitory process. This mechanism consists of three DNF platforms: (a) two context signal fields (stop and conflict) that represent information related to the contextual requirement of the tasks; (b) a pause field that suppresses the activity of the reach planning field to inhibit planned or ongoing actions. Each of the context fields consist of 100 neurons which project to the corresponding sub-population of the pause field via one-to-all excitatory connections. The stop signal field and the conflict signal field are activated when they detect a stop cue and conflict cue, respectively. Regarding the action selection function, the model does not need a context field to signal the decision task, since it can collect this information from the spatial sensory input field. In particular, the spatial sensory input field projects to the corresponding sub-population on the pause field with one-to-all excitatory connections. If more than one targets is encoded in the spatial sensory input field, the corresponding population on the pause field is triggered. Notably, this architecture is consistent with experimental studies which suggest dissociable frontal-BG circuits for different action suppression functions [39]. The pause field consists of 3 sub-populations of 75 neurons, each of them associated with one of the action regulation functions (i.e., action decisions between multiple options, action selection in the presence of conflicting information and outright stopping of actions). Once the pause field is triggered, the activity of the reach planning field is suppressed to delay a decision when more time is needed (i.e. during action selection or decision with conflicting information), or to completely suppress an action when it is no longer wanted or rendered inappropriate (i.e., outright stopping).

Each neuron in the reach planning field is connected with a stochastic optimal controller. Once the activity of a reach neuron j exceeds the action initiation threshold (cyan discontinuous line in Fig 4) at the current time and state x_t, the corresponding controller initiates an optimal policy π_j(x_t) to move the joystick towards the preferred direction of that neuron (see Materials and methods section for more details). Reaching movements are generated as a mixture of active policies (i.e., policies in which the associated neuronal activity in the reach planning field is above the action initiation threshold) weighted by the normalized activity of the corresponding reaching neurons. The normalized activity is called relative desirability since it reflects the attractiveness of a policy with respect to alternatives (for more details see [19, 47]).

Modeling the computations of motor decision-making.

We modeled the decision-making task within the neurocomputational framework to simulate reaching behavior in instructed and choice trials. Fig 5A illustrates the activity of the reach planning field as a function of time for a representative instructed (top panel) and choice (bottom panel) trial. Initially, the field activity is in the resting state. After the target onset in the choice trial, two neuronal populations selective for the targets are formed and compete through mutual inhibitory interactions. The activity of the pause field also increased to further inhibit the reach planning field to delay the initiation of the action (Fig 6A blue trace shows the mean activity of the pause field across time in a choice trial). Once the activity of a neuronal population exceeds an action initiation threshold, the corresponding target is selected, the activity of the non-selected target is inhibited by the “winning” population, and a reaching movement is initiated. Under conditions when the competition between action plans is not resolved after movement onset, the activity of the unselected action is not completely suppressed, leading occasionally to “change-of-mind” that reverses the initial choice. When only one target is presented (Fig 5A top panel), the activity of the corresponding neuronal population exceeds the action initiation threshold faster due to the lack of inhibitory competition from an alternative option and the non-activation of the pause field (Fig 6A cyan trace shows that pause field activity remains on baseline). To get better insight on the model computations, consider two neurons from the reach planning field in the choice trial, one from each population, centered at the target locations (Fig 5D). The neuron that exceeds the action initiation threshold first (red continuous traces) dictates the RT and the selected target (i.e., the selected direction of movement). In the absence of action competition (instructed trial), the activity of the reach neuron (blue trace) exceeds the action initiation threshold faster than when two actions compete for selection (red traces). Hence, we predict that simulated instructed reaches have shorter RT than reaches in the choice trials. To test this prediction, we simulated 200 decision-making trials in which 50% of them involve choices between two competing options and the rest of them were instructed trials. Consistent with the prediction, we found that free choice movements have longer RT than instructed movements, as is shown in Fig 7A.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Simulated reach planning field neuronal activity changes in the decision making task, Eriksen flanker task and stop signal task.

A-C: Activity changes of the 181 neurons in the reach planning field during the decision making task (instructed trial and choice trial)(A), the Eriksen flanker task (incongruent trial and congruent trial)(B), and the stop signal task (go trial and stop trial)(C). D-F: Activity changes of single neurons in the reach planning field during the decision making task(D), the Eriksen flanker task(E), and the stop signal task(F).

https://doi.org/10.1371/journal.pcbi.1010111.g005

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Simulated pause field activity changes during the three tasks.

A: Activity changes of single neuron in the pause field during the decision making-task. Cyan trace, simulated pause field activity during an instructed trial for a neurotypical individual. Magenta trace, simulated pause field activity during an instructed trial for a PD patient. Blue trace, simulated pause field activity during a choice trial for a neurotypical individual. Red trace, simulated pause field activity during a choice trial for a PD patient. B: Activity changes of single neuron in the pause field during the Eriksen flanker task. Cyan trace, simulated pause field activity during a congruent trial for a neurotypical individual. Magenta trace, simulated pause field activity during a congruent trial for a PD patient. Blue trace, simulated pause field activity during an incongruent trial for a neurotypical individual. Red trace, simulated pause field activity during an incongruent trial for a PD patient. C: Activity changes of single neuron in the pause field during the stop signal task. Blue trace, simulated pause field activity during a stop trial for a neurotypical individual. Red trace, simulated pause field activity during a stop trial for a PD patient.

https://doi.org/10.1371/journal.pcbi.1010111.g006

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Simulated reaction times for the decision-making tasks, the Eriksen flanker task and the stop signal task.

A: Bar plots of the simulated RT for neurotypical individuals (Neurotypical) and PD patients (PD) in the instructed and choice trials of the decision-making task. B: Bar plots of the simulated RT for neurotypical individuals (Neurotypical) and PD patients (PD) in the congruent and incongruent trials of the Eriksen flanker task. C: Bar plots of the simulated RT for neurotypical individuals (Neurotypical) and PD patients (PD) in the go trials of the stop signal task. Error bars correspond to SE.

https://doi.org/10.1371/journal.pcbi.1010111.g007

Modeling the computations of outright stopping of actions.

Regarding the stop signal task, the model needs to generate actions while anticipating a stop signal. The experimental results showed that when people anticipate a stop signal, they have longer RT as compared to when they do not anticipate a stop signal (i.e., instructed trials). This suggests that the pause field is active even in the go trials to increase the chances of being able to abandon an action in case stopping is required. The reach planning field activity in the go task resembles that of an instructed trial in the decision-making task (Fig 5C top panel), the only difference being that in the go trials the pause field is continuously active (Fig 6C blue trace). Hence, the activity of the reach planning field increases slower compared to the instructed trial, resulting in longer RT. In a go trial, the reach neuronal population tuned to the target direction is formed preparing an action. Once the activity of the population exceeds an action initiation threshold, the action is performed. However, in some trials, a stop signal is cued and the pause field activity is further increased, which subsequently inhibits the activity of the reach planning field to completely stop the planned or the ongoing action (Fig 5C bottom panel). To better understand the mechanism for stopping actions, consider one neuron from the population centered at the location of the target. The activity of the neuron increases once the target is cued and an action is initiated when the activity exceeds the action initiation threshold (Fig 5F blue trace). However, if a stop signal is cued, the pause field inhibits the activity of the neuron to stop the ongoing action (Fig 5F black trace). The stop signal is given with some delay (stop signal delay, SSD) in each trial. The longer the SSD is, the harder it is for the pause field to suppress the activity of the reach neuron increasing the chance to fail to stop the action. To test the model prediction, we simulated 100 go trials, as well as 250 stop trials, in which a stop stimulus appeared at different SSDs, signaling the model to abandon the action. Consistent with the model predictions, we found that go trials have longer RTs than instructed trials (two-tailed t-test, p<0.001, comparison made between the mean RT on instructed trials in the decision-making task and mean RT on go trials in the stop signal task), and the probability to successfully stop a response reduces with increased SSD—i.e., the longer the stop signal delay, the harder it is for the model to stop an action (Fig 8 blue trace).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Simulated probability to successfully stop an action as a function of the SSD.

The (simulated) probability to successfully stop an action as a function of the SSD for neurotypical individuals (Neurotypical, blue) and PD patients (PD, red).

https://doi.org/10.1371/journal.pcbi.1010111.g008

Dysfunction of the pause mechanism predicts motor impairment in PD patients

So far the neurodynamical theory is capable of capturing the action regulation behavior of the neurotypical individuals in the 3 action regulation tasks. However, one of the main findings in our study is that PD patients exhibit overall slower responses in nearly all tasks compared to neurotypical individuals. This motor impairment can be explained within the neurodynamical theory as a deficit on the pause mechanism. That is, the pause field is active even in the absence of conflicting information (congruent trials in the Eriksen flanker task) or competition between multiple actions (instructed trials in the decision-making task). To get a better understanding on how dysfunction of the pause mechanism affects motor behavior, Fig 6 shows the activity of the pause field as a function of time for a single trial across all tasks when modeling the motor behavior of neurotypical individuals and PD patients. In the decision-making task, the pause field is activated even when no action competition is presented (i.e., instructed trials) to capture the RT of the PD patient (Fig 6A magenta trace). This explains the slower response on initiating an action on instructed trials from PD patients. Also, the lack of difference on RT between neurotypical individuals and PD patients in free choice trials suggests that the pause field exhibits similar activation levels when deciding between competing options after targets onset in both groups (Fig 6A). We need to point out here that although the pause field exhibits the same activation level in instructed and free choice trials in PD patients, the slower response in choice trials compared to instructed trials is due to the inhibitory action competition between the two alternative movement directions. Regarding the Eriksen flanker task, the pause field is active before the target onset in PD patients, explaining the slower response in both congruent and incongruent trials compared to neurotypical individuals (Fig 6B red and magenta traces).

Additionally, another important finding in our study is that PD patients have shorter RT in go trials than neurotypical individuals in the stop signal task. By comparing the RT of movements between go trials in the stop signal task and instructed trials in the decision-making task, we found that neurotypical individuals delayed their responses in the go trials because they anticipated a stop signal as compared to when they did not anticipate a stop signal (i.e., instructed trials). This response delay effect (RDE) has been reported in previous studies [52–55] and has been associated with an “active braking mechanism” that increases the chance of abandoning a response in case stopping is required [56]. Note that PD patients also exhibited this active braking mechanism, but the difference in RT between go trials with anticipation of stopping signal and instructed trials was much smaller compared to neurotypical individuals. Overall, these findings suggest that the pause field is active in the go trials for predicting the motor behavior in both groups of participants. In fact, the pause field activity is higher in neurotypical individuals than PD patients, before a stop signal is detected, to explain the slower response of the first group compared to the latter group (Fig 6C, blue and red traces). We also simulated 100 go trials and 250 stop trials for PD patients. Consistent with the behavioral findings, the go trials have longer RT in the simulated neurotypical individuals than PD patients (Fig 7C) (two-tailed t-test, p<0.001). Additionally, the model predicts that PD patients also have longer RTs in the go trials compared to instructed trials (two-tailed t-test, p<0.05), and that the probability to successfully stop a response is lower in PD patients than in neurotypical individuals (Fig 8) due to the faster response of PD patients.

General

Operating effectively in an ever-changing environment requires not only the ability to select the best action, but also the flexibility to re-program actions in response to environmental changes. This includes delaying actions in the presence of conflicting information and completely abandoning obsolete actions when they are rendered inappropriate [57–60]. Series of studies have explored how different brain regions contribute to programming, re-programming and stopping of actions using neural recordings and functional neuroimaging techniques [25, 26, 29, 30, 33, 34, 61, 62]. Furthermore, various computational theories including the drift-diffusion model (DDM), urgency-gating model (UGM), evidence accumulation model (EAM), race models and mutual inhibition models, have been constructed to explain how the brain selects between competing options, inhibits actions in the presence of conflicting information and abandons planned or ongoing actions when they are rendered inappropriate [63–66]. Although these theories provide significant insights into the action regulation mechanisms that involve inhibitory control, a major limitation is that they explored separately each of these three motor functions, making it challenging to develop a unified theory of action regulation. A computational theory that can simulate the mechanisms underlying selecting, inhibiting and outright stopping of actions is needed to unify and integrate these distinctly studied actions and mechanisms.

Our research focuses exactly on what has been missing from previous studies—to design a large scale computational theory that can predict: 1) how the brain selects between competing actions, delays actions in the presence of conflicting information and stops actions when they are rendered inappropriate, 2) how neuropsychiatric diseases, such as PD, affect the action regulation circuitry and lead to motor deficits. Building on our previous work [47, 48], we developed a neurodynamical framework to integrate the three action regulation functions into a unified computational theory. This computational theory is based on the affordance competition hypothesis, in which multiple actions are formed concurrently and compete over time until one has sufficient evidence to win the competition [6]. It is analogous to the DDM in perceptual decisions in which two accumulators integrate sensory evidence in favor of two alternative options [67, 68]. The accumulator that exceeds its upper bound faster dictates the RT and the choice between the two options (more details about the difference between our neurocomputational framework and the DDM framework in modeling action selection tasks can be found in [19]—Discussion section: 3.2 From signal-detection theory to evidence accumulation to desirability competition. We replace evidence accumulation with desirability—a continuously evolving quantity that integrates all sources of information about the relative value of an action with respect to alternatives. The winning action determines the RT and the direction of movement. The computational theory includes a BG-type mechanism of inhibiting actions in the presence of competing options, conflicting information and stopping signals. The framework includes circuitry for perception, expected outcome, context signal, pause, action planning and execution. Among which the reach planning field, the spatial sensory input field, the expected outcome field and the pause field are sufficient to explain most of the behavioral findings. However, the context signal fields are necessary for detecting the conflicting/stop signal, as they ensure that the pause field receives appropriate input. The stochastic optimal controllers are required for generating motor behavior (i.e., reaching movements) with biologically plausible properties, such as velocities, direct movements to the target locations and so on. They are also needed to determine whether a successful stop is made within a certain distance in the stop signal task. In our previous work, we included a “reach cost field” which encodes the effort required to move in a particular direction [19]. However, despite its significance in action execution when more than one targets are available, it does not contribute to the current study, therefore was removed from the computational framework.

We tested the computational model in a series of tasks that involve action selection, decision conflict and outright stopping using neurotypical individuals and PD patients. Our findings showed that the model captures many aspects on human behavior, such as the longer RT in the presence of competing actions and conflicting information, as well as the inverse relationship between the probability to successfully stop an action and stop signal delay (SSD). It also predicts the motor impairment on PD patients when performing these three motor tasks as a deficit in the pause mechanism. In particular, the model explains the longer responses in generating actions even without the presence of competing action and conflicting information in PD patients compared to neurotypical individuals as a consequence of hyperactivity on the pause field. This is consistent with experimental evidence showing that STN is overacting in PD patients [69] leading to longer responses in visuomotor tasks. Overall, to the best of our knowledge, our study presents the first neurocomputational theory that integrates the mechanisms of three action regulation functions and predicts how disruption of these mechanisms can lead to motor deficits reported in neurological diseases such as PD.

Mapping to neurophysiology

The computational theory presented in the current study is a systems-level framework aimed to qualitatively predict response patterns of neuronal activities in ensembles of neurons, as well as motor behavior, in action regulation tasks. It captures many key features of the functional properties of the cortical-subcortical network involved in action regulation. The spatial sensory input field mimics the organization level of the posterior parietal cortex (PPC) [70, 71]. The expected reward field can be associated with the ventromedial prefrontal cortex (vmPFC) and orbitofrontal cortex (OFC), two frontal areas with important role in computation of expected outcome [72, 73]. The reach planning field can be associated with the parietal reach region (PRR) [74, 75] and the premotor dorsal cortex (PMd) [76, 77], two cortical areas involved in planning of reaching movements. The stop signal field can be equated with rIFG, which is recruited when cues associated with response inhibition are detected [78, 79]. Regarding the conflict signal field, the popular view is that preSMA detects the co-activation of different but conflicted responses (e.g., naming the color of the word red written with green color) and activates STN to temporarily suppress a response [80, 81]. Finally, the pause field can be equated to STN,which is activated in tasks that require stopping or pausing behavioral outputs to suppress actions [28, 32, 35, 40].

Computational modeling of action inhibition deficits in PD patients

PD is a progressive neurodegenerative disease associated with progressive loss of dopaminergic neurons in the substantia nigra of the BG [69]. The disruption of frontal-BG circuitry is responsible for the development of major symptoms of PD, including rigidity, tremor, bradykinesia, and postural instability [82, 83]. In particular, impairment of response inhibition abilities, which greatly affects the life quality of PD patients, is considered to be a sensitive measure to the progression of PD [84]. As a key player in the frontal-BG circuit, STN is suggested to mediate a “pause” function by rapidly inhibiting the BG activity. Therefore, it is considered to have a prominent role in the pathology of PD [85]. An increase in the neuronal activity of STN has been demonstrated in electrophysiological and behavioral studies in non-human primate models of PD [86] as well as PD patients [87]. Our findings suggest that an increase in the STN-mediated “pause” signal is responsible for the impairment of action inhibition abilities in PD patients. In our model, we assigned higher baseline activation level of the pause field in the decision-making task and Eriksen flanker task in PD patients compared with neurotypical individuals. Consistent with the model predictions, PD patients exhibited longer RT than healthy individuals in the instructed trials of the decision-making task, as well as in both incongruent trials and congruent trials of the Eriksen flanker task. Notably, RT in the free choice trials of the decision-making task wasn’t significantly different between PD patients and neurotypical individuals. This suggests that the pause field, which is already highly active in instructed trials, is not further activated in the choice trials.

An interesting finding in our study was that neurotypical individuals had slower response than PD patients to initiate an action in the stop signal task. This is somewhat counter-intuitive since STN is overactive in PD patients and therefore we would expect that they would have slower response than neurotypical individuals. However, when we compared RT between instructed trials (decision-making task) and go trials (stop signal task) of the neurotypical individuals, we found that they responded slower when they anticipated a stop signal. On the other hand, we found much subtler difference in RT between instructed trials and go trials in PD patients. This suggests that the pause mechanism is activated in the stop signal task in neurotypical individuals even before a stop signal is presented. By activating the pause field to simulate the motor behavior of the neurotypical individuals, the model predicts that PD patients will have faster responses and lower probability to stop planned or ongoing actions compared to neurotypical individuals. In other words, the model explains the slower responses of the neurotypical individuals as a cognitive strategy adapted to minimize the probability to fail to stop an action if a stop signal is detected.

Activity suppression or increase of action initiation threshold?

In our theory, the pause field delays motor decisions by suppressing the activity of the reach planning field. However, an alternative hypothesis is that the pause field mediates the action inhibition function by increasing the action initiation threshold. Previous studies suggest that STN low-frequency oscillatory activity and medial prefrontal cortex (mPFC)-STN coupling are involved in determining the amount of evidence (i.e., action initiation threshold) needed before making a decision [88–91]. Additionally, clinical studies showed that deep brain stimulation (DBS) targeting STN in PD patients can modulate the amount of evidence, and therefore the action initiation threshold, required to initiate an action [89]. Hence, it is also likely that STN delays motor decisions in the presence of competing actions and/or conflicting information by increasing the action initiation threshold, instead of suppressing the activity of the motor areas that generate actions. Our computational theory is capable of modeling this hypothesis by adjusting the action initiation threshold in the reach planning field. However, it cannot dissociate between the two hypotheses on how STN pauses actions when needed. To do so, future neurophysiological or neuroimaging studies need to record activity from STN and motor areas during decision tasks with multiple options and/or conflicting information.

Hyperactive pause mechanism or altered cost/reward ratio in PD patients?

Although our study suggests that deficits in movement preparation in PD patients, such as slow RTs, are related to hyperactivity in STN that inhibits planned actions, other studies have associated motor impairments with motivational deficits [92]. In particular, motivational deficits seem to significantly contribute to bradykinesia in PD patients and lead to alternation in the amount of effort required to perform a movement at normal speed, as well as the perceived reward for successfully completing the action [93].

Motor decisions are frequently made based on expected reward and the associated effort cost required to obtain the reward. The cost has been considered to be detrimental, since we tend to choose the less costly actions especially when they are associated with similar expected rewards [94, 95]. The dopaminergic neurons seem to be critically involved in the process of cost versus reward (i.e., cost/reward ratio) evaluation. Dopamine depletion from rat results in decreased tolerance for effort cost, whereas enhanced dopamine levels have the opposite effect [94, 96]. Loss of dopaminergic neurons and their projections is a major pathological hallmark in PD patients. Clinical studies have shown that PD patients, regardless of medication status, tend to engage less effort for the lowest reward compared with neurotypical individuals in a hand-squeezing task [93]. However, dopamine medication motivates PD patients to engage more effort for a given reward, comparing to their off-medication state. In addition, DBS of STN establishes a reliable congruency between action and reward in PD patients and remarkably enhances it over the level observed in neurotypical individuals [97].

Overall, these studies provide evidence that impairment of movement preparation in PD patients can also be related to deficits in the mechanism that evaluates reward and effort cost associated with actions—i.e., alternation of the cost/reward ratio. Notably, this can be also modeled within our neurodynamical framework by increasing the amount of effort required to perform actions towards the cued directions. Additionally, the alternation of the cost/reward ratio in PD patients could be also related to the hyperactivity of STN—more effort is required to increase the activity of the motor population, which is continuously inhibited by STN, to initiate an action. Today, the mechanism for motor and information processing deficits in PD patients is still under extensive study. PD is considered not only a disease caused by degeneration of substantia nigra dopaminergic neurons, but also a system-level disease caused by dysfunction of the cortical-BG circuit [69]. Therefore, both the hyperactivity of STN and the altered cost/reward ratio could be considered parts of PD pathophysiology and contribute to the motor deficits in PD patients. In order to further dissociate between these two competing hypotheses, we have to design experiment paradigms that involve less “costly” motor actions, such as finger-tapping or saccadic movements. If altered cost/reward ratio, instead of hyperactive pause mechanism, is the major factor that contributes to the motor deficits in PD patients, it should have less impact on RT in actions that are associated with low effort cost. Such experimental paradigms combined with simultaneous recordings from multiple regions in the cortical-BG circuit, including STN, vmPFC and OFC during DBS surgeries, could further elucidate the mechanism of motor deficits in PD patients.

Limitations and theory-driven hypotheses

One limitation in our study is that full neuropsychological characterization of the PD patients (e.g., level of impulsivity) is not available. Furthermore, the age difference between the neurotypical and PD groups makes it challenging to disentangle the effects of age vs. PD. However, the overall behavioral performance of PD patients is consistent with previous studies, including slower movements [98] and prolonged RTs [2, 99]. Therefore, despite the fact that future work with age-matched controls and more detailed neuropsychological characterization of PD patients could provide more insights on how PD affects motor behavior, our study has significantly enhanced our understanding of the computational mechanisms of action regulation deficits in PD patients. In addition, although the neurodynamical theory introduced in the current study can predict many key features of motor behavior in neurotypical individuals and PD patients in action regulation tasks that involve action inhibition, it also has the same limitations as any computational theory on what it can predict and whether it can dissociate between alternative and competing hypotheses. For instance, as we discussed above, an alternative scenario on how STN contributes in motor inhibition is that it increases the action initiation threshold when the brain deals with competing options and conflicting information. Whether STN increases the action initiation threshold or suppresses the activity of motor areas cannot be dissociated in the current study, since both hypotheses predict comparable motor behavior—i.e., longer RTs. Similarly, we cannot exclude the hypothesis that inhibitory control is mediated by a global network, rather than dissociable brain networks that mediate the three action regulation functions. However, the current architecture can be easily modified to include a global pause function (i.e., one pause DNF) that mediates all the action regulation functions. Furthermore, we cannot exclude a hypothesis that the pause mechanism is involved only in outright stopping of actions. Selecting and switching actions in tasks with competing options (i.e., choice task) and conflicting information (i.e., Eriksen flanker task), respectively, could be made through a competition within the same circuit that guides the actions themselves (i.e., the reach planning field) without engaging the stopping circuitry. According to this hypothesis, action selection occurs only through an inhibitory competition between the alternative actions—i.e., the winning action determines the movement direction and RT. Similarly, switching actions in the incongruent trials may occur through an inhibitory competition between the action associated with the flanker stimuli, which appear earlier on the screen, and the action associated with the target—stopping circuitry is not involved in this scenario. These different hypotheses described above generate comparable motor outputs and therefore it is challenging, or even impossible, to dissociate them using the behavioral findings of the current study. However, we can easily refine the architecture of the computational theory as new evidence becomes available. In fact, this is one the roles of the proposed neurodynamical theory: to generate theory-driven hypotheses that can be tested in future experimental studies. For instance, neurophysiological studies that involve neural recordings and stimulation technologies in PD patients who undergo DBS implantation surgery can provide significant insights about the role of STN and other cortical regions in action inhibition.

Another challenge in our computational framework is the high degrees of freedom—i.e., the large number of parameters that need to be tuned to generate motor behavior and the underpinning neural activity. However, brain circuits are characterized by a high degree of structural and dynamic complexity in support of information processing. This biological reality is almost always reflected in the large number of free parameters in neurocomputational models [100]. Nevertheless, large-scale neurocomputational models, as the one proposed in the current study, are frequently over-parameterized resulting in more than one potential architectures that produce comparable behavioral outputs. Although this flexibility can make it difficult to falsify alternative architectures of the model, it could be argued that it is an advantage since the model builds multiple hypotheses that can be tested in future experimental studies.

We also need to point out that the model parameters have not been optimized so that the simulated motor behavior fits with the human behavioral data. This is common in systems-level models, since they aim to provide only qualitative predictions about the behavior and the neural mechanisms associated with visuomotor tasks. However, we can fit the model parameters to behavioral data using optimization methods, such as Genetic Algorithms (GAs) as presented in [101, 102]. In particular, the parameters that determine the DNF activity are the strength and the width of the interaction kernel, the resting level, the slope of the sigmoidal functions, and the strength of interactions with other DNFs. We can tune/optimize these parameters using GAs with appropriate cost functions to obtain the desired motor behavior that quantitatively predicts the human behavioral data.

Finally, although our model successfully predicts the RT of neurotypical participants and PD patients in the three action regulation tasks, it fails to predict the right-skewed distribution of the RTs. RTs, as many other quantities in neuroscience and psychology, are skewed to the right. Although there is no strong consensus yet, the right-skewed distribution of RT in tasks that people have to react to external stimuli has been associated with lapse of attention and/or tiredness and/or eye blink that cause some lengthy RTs [103]. Since our model does not involve mechanisms to simulate lapse of attention, fatigue, and eye blink, it cannot predict the right-skewed distribution of the RT.

Ethics statement

The study was approved by the University of California, Los Angeles Review Board and all individuals signed a written informed consent before participating.

Participants

A sample of 15 adults with PD (all males) and 32 neurologically healthy adults (13 females) participated in the study. The ages at the time of the experiment were 65.5 ± 8.6 (mean ± SD) years old for PD patients and 33.8 ± 6.4 years old for neurotypical individuals. The Hoehn and Yahr (H&Y) Scale for PD patients at the time of the experiment was 1.53 ± 0.64 (mean ± SD). All PD patients performed the experiments when they were on medications. The Levodopa equivalent dosing for PD patients was 610.5 ± 367.5 (mean ± SD) mg per day.

Stimuli and procedure

Statistical analysis

Cubic interpolating splines were used to smooth the reach trajectories and compute the velocity of the movements. RT was defined as the time between the target appearance and the time that reach velocity exceeded 10% of the maximum velocity. RTs faster than 100ms were removed because anticipation is considered to be involved prior to actions, as well as RTs longer than 1500 ms. RT outliers (RTs >3 standard deviations below or above the mean RT) were also excluded from the analysis. The trials in which the participant changed their mind (moving towards one direction past 5% of the maximum range, and then changed their mind to move towards the other direction) were also excluded from further analysis. RTs across all participants were pooled together, and for the decision making task and the Eriksen flanker task, two-way ANOVA analysis was performed to determine the group differences in RTs. For the stop signal task, two-tailed t-test was performed to determine the group differences in go trial RTs.

Computational model architecture

We developed a neurodynamical framework based on our previous studies [47, 48] to model the three action regulation functions. The computational framework combines dynamic neural field theory with stochastic optimal control theory, and includes circuitry for perception, expected outcome, context signal, pause, action planning and execution. Each DNF simulates the dynamic evolution of firing rate activity of a network of neurons over a continuous space with local excitation and surround inhibition. The functional properties of each DNF are determined by the lateral inhibition within the field and the connections with other fields in the architecture. The projections between the fields can be topologically organized—i.e., each neuron i in the field drives the activation at the corresponding neuron i in the other field (one-to-one connections), or unordered—i.e., each neuron in one field is connected with all neurons on the other field (one-to-all connections). The activity of a field j evolves over time under the influence of external inputs, local excitation and lateral inhibition interactions within the field, as well as interactions with other k fields, as described by Eq (1): (1) where u(x, t) is the local activity of the DNF at the position x and time t, and is the rate of change of the activity over time scale by a time constant τ. If there is no external input S(x, t), the field converges over time to the resting state h from the current level of activation. The first convolution term models interactions between the simulated neurons at different locations within the field j, and is shaped by the interaction kernel of Eq (2), which consists of both excitatory and inhibitory components: (2) where C_exc, C_inh, σ_exc and σ_inh describe the amplitude and the width of the excitatory and the inhibitory components, respectively. We convolved the kernel function with a sigmoidal transformation of the field so that the neurons with activity above a threshold participate in the intra-field interactions: (3) in which the steepness of the sigmoid function was controlled by β.

The function w_kj describes the connectivity kernel between fields u_j and u_k showing the contribution of field u_k to the dynamics of field u_j. The sigmoid f_k(u_k) and w_jk are convolved to determine the full contribution from field u_k to u_j.

The architectural organization of the framework is shown in Fig 4. The “reach planning” field encodes the potential movement directions, and is responsible for initiating the reaching movements. The “spatial sensory input” field encodes the angular representations of the competing targets. The “expected outcome” field encodes the expected reward for reaching to a particular direction centered on the hand position. The outputs of these two fields send excitatory projections (green arrows) to the reach planning field in a topological manner. The “pause” field suppresses the activity of the reach planning field to inhibit planned or ongoing actions via inhibitory projections to the reach planning field. The stop signal field and the conflict signal field encode information related to the contextual requirement of the task (i.e., stopping cue or flanker distractor), and send one-to-all excitatory projections to the corresponding population of the pause field. In particular, the stop signal field projects to the neuronal population of the pause field which is responsible for outright stopping of action, whereas the conflict signal field projects to the neuronal population of the pause field, which is responsible for delaying decisions when conflicting information is detected. Each of the context signal fields (stop signal field and conflict signal field) consists of 100 neurons, whereas the pause field consists of 3 neuronal sub-populations, each consists of 75 neurons. The rest of the fields consist of 181 neurons with a preferred direction between 0 to 180 degrees. The activity of the reach planning field S_action is given as the sum of the outputs of the fields encoding the position of the target v_pos, the expected reward v_reward, and the activity from the pause field v_pau, at any given time and state, corrupted by a Gaussian distributed additive noise ξ. (4) where η_pos, η_reward and η_pau are scalar values that weigh the influence of the spatial sensory input field, the expected outcome field and the pause field, respectively, to the activity of the reach planning field. The values of the model parameters are given in S1 Table. The normalized activity of the reach planning field describes the relative desirability d_i of each “reach neuron” with respect to the alternative options at time t—i.e., the higher the activity of a reach neuron j, the higher the desirability to move towards the preferred direction φ_j of this neuron with respect to the alternatives at a given time t. Each neuron j in the reach planning field is connected with a control scheme that generates reaching trajectories. Once the activity of that neuron exceeds the action initiation threshold γ, the controller is triggered and generates an optimal policy π_j, a sequence of motor actions towards the preferred direction of the neuron j. The optimal policy is given by minimization of the cost function: (5) where π_j(x_t) is the policy from the time t = 1 to t = T_J to reach towards the preferred direction φ_j; T_j is the time required to arrive at position p_j; p_j is the position planned to arrive (goal position) at the end of the reaching movement, given by p_j = [rcos(φ_j), rsin(φ_j)], in which r is the distance between the current location of the hand and the location of the stimulus encoded by the neuron j. is the state vector at the end of the reaching movement, and matrix S selects the actual position of the hand and the goal position at the end of the reaching movement from the state vector. Matrices and R define the cost dependent on precision and control, respectively. More details about the optimal control model are described in [47, 48]. Consequently, a action is initiated once a neuronal population exceeds the action initiation threshold and the executed action π_mix(x_t) is given as a mixture of the active policies (i.e., policies with active neurons) weighted by relative desirability values of the corresponding neurons at any given time and state. (6) where x_t is the state of the system at time t (i.e., position, velocity, orientation of the trajectory), d_j is the normalized activity of the neuron j (i.e., relative desirability value of the neuron j), and π_j is the optimal policy generated by the controller connected with neuron j. Because desirability is time- and state-dependent, the weighted mixture of the individual policies can change/correct the current trajectory in the presence of new incoming information—e.g., a stop signal cued while acting. In order to handle contingencies during the movement, the “receding horizon control” (RHC) [105, 106] technique, also known as model predictive control (MPC), which is widely used in stochastic optimal control models, was implemented in the framework. According to RHC, the framework would only execute the initial portion of the sequence of actions for a short period of time τ (τ = 9 in our framework), after which the framework would recompute the optimal policy π_mix(x_t + τ) from time t+τ to t+τ+T_i, and this approach would continue until the hand reaches one of the targets.