Fig 1.
(A) To perform a reaching movement the brain creates a motor program and sends it as an input to the cerebellum implemented as an artificial neural network. The cerebellum modifies the motor program and sends it as an input to the neuromechanical arm model. The perceived displacement between the movement endpoint and the target position (the vector error) is fed back to the cerebellum (visual feedback) and used to calculate the adjustment of weights in the cerebellar network using error back propagation. (B) We simulated reaching movements which started from a fixed initial position, aiming to reach a target located on a circle centered at the starting point with a radius of 20 cm. The direction of the movement relative to the body position was defined by the angle as shown.
Fig 2.
Simulation of cerebellar-based adaptation to mild perturbations.
The plot shows the movement error in degrees versus trial number during simulation of the 3-phase experiment when a visuomotor perturbation is introduced in the beginning of ADAPT phase and removed in the beginning of POST phase. The perturbations are the shift (red line) and 30-deg visuomotor rotation (blue line) (see text for details). Solid lines show the 32-run per trial average of the angle difference between the perceived target position and the perceived hand/cursor position versus the trial number. After the perturbation is introduced, the initial error is approximately equal to the shift/rotation angle. Then it converges to the asymptote below 10 degrees. The asymptote is nonzero due to “forgetting” effect presented in error-based learning, corresponding to the damping term in the cerebellum model. After the perturbation is removed, the error changes its sign and abruptly increases in magnitude again (perturbation aftereffect). Then it converges back to zero.
Fig 3.
Simulations of adaptation to the visuomotor rotation perturbations.
Each plot shows reaching angle dynamics which is defined here as a difference between the perceived target position and the movement direction. In this simulation (similar to Schlerf et al. [10] experiments) a visuomotor rotation first was introduced and removed in a stepwise manner (Multi Step phase), then it was introduced and removed abruptly (Single Step phase). The solid black line shows the actual target position relative to the perceived target position (rotation angle); the blue line shows simulation corresponding to the control group in Schlerf et al.; the red line shows the simulation with increased degradation rate and increased noise in the movement endpoint to mimic the cerebellar ataxia group data from Schlerf et al. (see text for details); the light grey areas show the standard error of the mean (SEM) over 16 runs. (A) one trial simulation of the control (blue) and ataxic (red) conditions; (B) 16-runs average multistep perturbation adaptation close-up (C) 16-runs average single step perturbation adaptation close-up.
Fig 4.
(Lack of) Error-based adaptation to strong visuomotor perturbation.
The plot shows movement error in degrees versus trial number during simulation of the 3-phase experiment when a visuomotor perturbation is introduced in the beginning of ADAPT phase and removed in the beginning of POST phase. Solid lines show the 32-run average angle difference between the perceived target position and the perceived hand/cursor position. (A) Simulation of adaptation to 90 degrees visuomotor rotation (blue line) and 90 degrees shift (red line). (B) Simulation of adaptation to the x-reflection perturbation. Note that the perturbation size is the same for the shift and rotation perturbation in (A). However, the trial-to-trial dynamics of the error are very different, as well as the magnitudes of the aftereffects.
Fig 5.
Adaptation to visuomotor rotation with increasing angle.
The panels show successive movement endpoints obtained from simulation during adaptation to rotation by 60, 75 and 85 degrees. The color of each point represents the trial number. The ‘+’ shows the target position. Only the adaptation phases of individual simulations are plotted. To emphasize the effect, the degradation rate in the cerebellum was set to zero. As the rotation angle increases, the trajectory first starts spiraling while converging to the target (the middle panel) and then exhibits oscillations around the target with non-decaying amplitude (right panel).
Fig 6.
General architecture of the model with cerebellum and basal ganglia.
In this version of the model basal ganglia select a motor program for execution among possible behaviors generated in the motor cortex (M1). The cerebellum (CB) receives a copy of the motor program and modifies it before the result is fed to the movement system.
Fig 7.
Adaptation to a mild perturbation in the model with cerebellum and basal ganglia.
The plot shows error magnitude versus trial number during simulation of the 3-phase experiment when a visuomotor perturbation is introduced in the beginning of ADAPT phase and removed in the beginning of POST phase. Solid line shows 32-runs average angle difference between perceived target position and perceived hand/cursor position. Grey area shows SEMs. Blue dashed lines show the rewarding range.
Fig 8.
Adaptation to strong perturbations in the model with cerebellum and basal ganglia.
The plot shows error magnitude versus trial number during simulation of the 3-phase experiment when a visuomotor perturbation is introduced in the beginning of the ADAPT phase and removed in the beginning of the POST phase. The solid black line shows the 32 runs average angle difference between the perceived target position and the perceived hand/cursor position during adaptation to the 90-degree rotation (A) and the x-reflection (B) perturbations. Dashed lines show the rewarding range. Grey area shows SEMs. BG addition neither improve adaptation to the strong rotation, nor does it make the adaptation to the reflection perturbation possible.
Fig 9.
Basal ganglia and cerebellum interaction through the critic and critic mechanism.
Panel A: In this version of the model the critic node controls the learning rate of the cerebellum (CB) and offsets the reinforcement signal in basal ganglia. It increases or decreases its output depending on whether the predicted error after correction corresponds to the observed one or not. The critic output is used to set the CB learning rate and add to the dopamine release in striatum. In case the error-based/cerebellar correction is successful, the CB learning rate is increased on the next trial, and the striatal dopamine concentration is increased to prevent negative reinforcement of the previously selected action. In case the error-based correction failed, the CB learning rate is reduced, and the striatal dopamine concentration is not modified thus allowing for negative reinforcement. Panel B: The diagram clarifies how the critic output is computed based on the different pieces of information from the previous (n) and current (n+1) trials. After the previous trial cerebellum modifies the synaptic weights and forms a prediction about the next error value based on that modification. At the current (n+1) trial the actually observed error can differ from the one predicted by the cerebellum due to influence of the factors that cerebellum cannot control and detect directly: possible change of the motor program (e.g. because of basal ganglia activity), perceptual perturbations and noise. If the observed error agrees with the predicted one, the critic signal is increased. If it strongly disagrees, the signal is decreased. Otherwise it is left unchanged. The critic output is used to set the future CB learning rate and add to the reward prediction error when calculating the dopamine level in striatum.
Fig 10.
Simulation of adaptation to strong perturbation with the critic-controlled learning rate.
The plot shows error size versus trial number during simulation of the 3-phase experiment when a visuomotor perturbation is introduced in the beginning of ADAPT phase and removed in the beginning of POST phase. Panels (A1), (B1) show the 32-runs average reaching error dynamics for the adaptation to 90-deg rotation (A1, blue line) and x-reflection (B1, red line). Panels (A2), (B2) show 32-runs average cerebellum (CB) learning rate dynamics for 90-deg rotation (A2, blue line) and x-reflection (B2, red line). Dashed lines show basal ganglia (BG) target size. Grey area shows SEMs. Critic control of the learning rate allows to adapt to both strong perturbations. Note absence of aftereffects.
Fig 11.
Basal ganglia take over from cerebellum during adaptation to a mild perturbation with the critic-controlled learning rate in cerebellum.
The plot shows error magnitude versus trial number during simulation of the 3-phase experiment when a visuomotor perturbation is introduced in the beginning of ADAPT phase and removed in the beginning of POST phase. (A) Solid green line shows 32-run average angle difference between the direction to the target and to the endpoint of the reaching movement; (B) 32-run average CB learning rate dynamics. Dashed lines show the rewarding range, dashed-dotted lines show endpoint noise amplitude. Grey area shows SEMs.
Fig 12.
Simulation of error-based and non-error-based adaptation with impaired basal ganglia function.
These simulations reproduce Gutierrez-Garralda et al [5] data. (A1, B1) 16-run averaged reaching error for the adaptation to a 30-degree shift perturbation. (A2, B2) 16-run averaged error for the adaptation to an x-reflection perturbation. Shape and darkness of the makers code different conditions: light squares show the simulation of Huntington’s Disease (HD) conditions, and light triangles are used for the simulation of Parkinson’s Disease (PD) conditions. Dark circles and diamonds are used to show control conditions for HD (CHD), and for PD (CPD), respectively. Green color is used for the shift perturbation which engages the error-based learning (EBL). X-reflection perturbation simulation (where non-error-based learning (NEBL) is involved) is shown in red. Error bars show SEMs. Controls reduce the error in both conditions but show almost no aftereffects after x-reflection perturbation. In the shift perturbation simulations (A1, B1), the error reduces equally for both control and PD/HD conditions. After the x-reflection (A2, B2), adaptation occurs in control conditions, but not in PD and HD conditions.
Fig 13.
Simulation of the reinforcement-dependent memory retention.
These simulations reproduce Shmuelof et al. [7] data. Blue lines show the 16-run average of the reaching angle dynamics for the model that constantly received both binary and vector error feedback (BE+VE). Orange lines show 16-run average of the reaching angle dynamics for the model that received only binary feedback between trials 80 and 160 (BE). Lighter blue and lighter orange lines correspond to the simulations where the error clamp was extended by 40 trials. Thick black horizontal lines show the target center, thin grey horizontal lines show target boundaries. Grey areas show SEMs. During error clamp the orange BE curves (unlike the blue BE+VE curves) approximately converge to the level where the binary feedback only was provided between trials 20 and 160.