2.1 Spino-cerebellar and cerebellar models perform the recorded kinematics

We extracted kinematics and EMG data from two healthy participants (P1 and P2) performing upper limb movements in the vertical plane involving the shoulder and elbow (see Methods). The motor tasks performed by P1 and P2 can be grouped in: i) flexion-extension movements, ii) hand-tracked circular trajectories. Both motor task groups were performed at different speeds, thus providing a set of natural upper limb trajectories which constituted our initial motor control benchmark. We used the joint kinematics (i.e., shoulder and elbow position, Q_d, and velocity, ) extracted from the recording sessions as the desired trajectory to be learnt by the spino-cerebellar (Fig 1A) and cerebellar (Fig 1B) models in the simulation framework. Both models performed 3 repetitions of the motor adaptation process for each desired trajectory, each repetition consisting of 2000 consecutive trials, a trial-and-error process that allowed motor adaptation to fully deploy from scratch. The performance metric was given by the position and velocity mean absolute error (MAE), i.e., difference between the desired and actual trajectory in joint space, allowing to assess motor behaviour (see Methods).

We first calculated the position and velocity MAE evolution for both the spino-cerebellar and cerebellar models performing the trajectories extracted from each participant (Fig 2A, P1’s 1.8s circle trajectory; Fig 2B, P2’s 1.2s flexion-extension); see Supporting Information for all P1 and P2 motor tasks MAE evolution (S1(A)–S9(A) Figs). As the trajectory was repeated over time, the cerebellar adaptation allowed position and velocity error reduction. At the end of the motor adaptation process, both the spino-cerebellar and cerebellar models followed the target kinematics (Fig 2A and 2B); see Supporting Information for all P1 and P2 motor tasks kinematics performance (S1(B)–S9(B) and S1(C)–S9(C) Figs).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Spino-cerebellar and cerebellar models kinematic performance for the lab recorded scenario.

A) Position and velocity mean absolute error (MAE) over the 3 repetitions of the 2000-trial motor adaptation process; and joint kinematics of the 3 repetitions last 200 trials (mean and standard deviation) for both the spino-cerebellar and cerebellar models performing P1’s slow circle trajectory (1.8s), and B) P2’s fast flexion-extension (1.2s). C) Mean and standard deviation of the position and velocity MAE (last 200 trials of the 3 trajectory repetitions) for all P1 recorded trajectories, and D) for all P2 recorded trajectories.

https://doi.org/10.1371/journal.pcbi.1011008.g002

We found that, attending to the MAE mean and standard deviation (std) of the last 200 trials of the motor adaptation process (Fig 2C and 2D), the spino-cerebellar model reached better performance in terms of position tracking for all trajectories except for P1’s slow (2.3s) and moderate (1.8s) flexion-extension (average position MAE for all motor tasks was 0.026 ± 0.011 rad for the spino-cerebellar model, and 0.037 ± 0.024 rad for the cerebellar model). Conversely, the cerebellar model reached better performance in terms of velocity tracking except for P1’s slow (1.8s) circle and P2’s moderate (1.6s) and fast (1.2s) circle (average velocity MAE for all motor tasks was 0.28 ± 0.07 rad/s for the spino-cerebellar model, and 0.26 ± 0.09 rad for the cerebellar model).

2.2 The spinal cord improves cerebellar learning convergence and speed

Once we revealed the adaptation capability of both the spino-cerebellar and cerebellar models, we studied the influence of the SC model on cerebellar learning over the adaptation process. Using the position and velocity MAE evolution of each P1 and P2 trajectory, we compared the spino-cerebellar and cerebellar models learning convergence and learning speed. To study learning convergence we applied control charts on the MAE data to determine the number of trials required to achieve a stable performance [43]. We computed the MAE mean (μ) and standard deviation (σ) using a temporal sliding window with a sample size of 200 trials and defined different MAE limits relating μ and σ (e.g., limit 1 = MAE ∈[μ − σ, μ + σ]). We then measured the percentage of trials with a MAE value within each limit (see Methods). To check learning speed we analysed the number of trials required for the mean MAE of 200 samples to reach a given target (i.e., 0.1 rad for position MAE, and 0.5 rad/s for velocity MAE). Learning convergence and speed were tested on both position and velocity tracking performance (see Fig 3A for an example of position and velocity MAE evolution and the metrics used, see Methods for a further description).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Spino-cerebellar and cerebellar models motor adaptation for all P1 and P2 recorded trajectories.

A) Position MAE for the spino-cerebellar and cerebellar models for P1 slow circle (left column), and velocity MAE for both models performing P2 fast flexion-extension (right column). Both MAE plots show the trials at which the learning convergence and learning speed metrics are fulfilled. Only one repetition of the motor adaptation process is displayed. B) Learning convergence for both models and all trajectories from P1 (left column) and P2 (right column). The bar plots display the number of trials required by each model to fulfill the learning convergence criteria (see Methods). C) Learning speed for both models and all trajectories from P1 (left column) and P2 (right column). The bar plots depict the inverse of the number of trials required to reach a position MAE of 0.1 rad and a velocity MAE of 0.5 rad/s.

https://doi.org/10.1371/journal.pcbi.1011008.g003

The SC was proven to facilitate cerebellar learning as it reduced learning convergence time (Fig 3B), and increased learning speed (Fig 3C) for both position and velocity for both P1 and P2 trajectories (Fig 3 left and right column, respectively). Thus, cerebellar motor adaptation was shown to be: i) stabilised by the SC: average convergence time for MAE_pos was 924 ± 423 trials for the spino-cerebellar model, and 1625 ± 526 trials for the cerebellar model; and for MAE_vel1015± 509 trials, and 1332 ± 571 trials, respectively; ii) accelerated by the SC: average learning speed for position was 3.2e−3 ± 1.0e−3 trials⁻¹ for the spino-cerebellar model, and 2.1e−3 ± 1.3e−3 trials⁻¹ for the cerebellar model; and for velocity 2.7e−3 ± 1.1e−3 trials⁻¹, and 2.0e−3 ± 1.3e−3 trials⁻¹, respectively.

2.3 The spinal cord simplifies cerebellar synaptic adaptation at GC-PC

Consistently with the Marr-Albus-Ito cerebellar theory, learning in the cerebellum was provided by means of an STDP mechanism adjusting the synaptic weights at GC to PC synapses (a connection established through PFs, i.e., GC axons). The effect of the SC on cerebellar learning, already checked in terms of motor performance in the previous section, must leave its trace at the level of cerebellar synaptic adaptation. To conduct a direct comparison between the synaptic adaptation of the spino-cerebellar and cerebellar models, it is necessary to establish a common synaptic foundation. Thus, in one of the three repetitions of the 2000-trial motor adaptation process for each motor task, the synaptic weights between GCs and PCs were homogeneously initialised; i.e., at trial 0 all GC-PC synapses in both the spino-cerebellar and cerebellar models started with the same synaptic weight (4.8 nS). This homogeneously-initialised, 2000-trial repetition for the different P1 and P2 motor tasks, served as our benchmark for synaptic adaptation. The common starting point concerning the synaptic weight distribution allowed for a fair comparison of the synaptic evolution between the two models.

During the motor adaptation process of both the spino-cerebellar and cerebellar models, we recorded the synaptic weight evolution at GC-PC connections every 200 trials for all P1 and P2 trajectories (Fig 4A and 4B). We then measured the entropy of the GC-PC synaptic weight distributions to quantify the synaptic complexity of both models: the higher the entropy, the more complex the synaptic weight distribution, i.e., higher heterogeneity of synaptic weights at the GC-PC population. The synaptic entropy metric can be grasped as measuring the complexity of the synaptic patterns displayed in Fig 4A and 4B, and the rest of the patterns recorded every 200 trials describing the full motor adaptation process. Contrasting the synaptic entropy of both models allowed evaluating the effect of the SC on cerebellar synaptic adaptation (Fig 4C and 4D). Noteworthy, results showed that for all motor tasks the SC reduced the entropy of the synaptic weight distribution: the mean entropy over all P1 and P2 trajectories was 3.65 ± 0.78 for the spino-cerebellar model, and 4.41 ± 1.14 for the cerebellar model. When the SC was lacking in the control loop, more complex synaptic patterns (i.e., higher specialisation) were required at cerebellar GC-PC connections. The spino-cerebellar model showed a simpler distribution of synaptic weights at GC-PC connections; in other words, the spinal cord was therefore shown to simplify learning in the cerebellum.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Spino-cerebellar and cerebellar synaptic adaptation.

A), B) Synaptic weights at granule cell—Purkinje cell (GC-PC) synapses after 200 and 2000 trials, respectively, for both models performing P1’s 1.8s circle trajectory. The heat map represents the normalised GC-PC synaptic weights, which could range from 0.0 to 15.0 nS. C), D) Evolution of the synaptic entropy at the GC-PC synapses over the 2000-trial motor adaptation process, for all P1 and P2 trajectories, respectively. The higher the entropy, the more complex the GC-PC synaptic distribution (i.e., higher heterogeneity in the synaptic weights of the GC-PC synapses). E), F) Percentage of GC-PC synapses that experienced a modification of their initial weight as motor adaptation progressed for all P1 and P2 trajectories; i.e., amount of GC-PC synapses required by both the spino-cerebellar and cerebellar models to succeed in the motor adaptation process.

https://doi.org/10.1371/journal.pcbi.1011008.g004

To deepen in the finding of the SC simplification of the cerebellar synaptic solutions, we analysed the amount of GC neurons required by the spino-cerebellar and cerebellar models. We measured the percentage of GC-PC synapses that experienced a modification of their initial weight as motor adaptation progressed, thus providing a measurement of how many GC-PC connections were effectively involved in motor learning (Fig 4E and 4F). Results showed that the spino-cerebellar model made use of fewer GC-PC connections for all P1 and P2 motor tasks. Thus, the SC allowed for a reduction of the GC neurons required for accurate execution of the motor tasks. The common synaptic starting point together with the synaptic entropy evolution and the amount of neurons involved in the motor adaptation process, support the divergence in the synaptic solutions acquired by the spino-cerebellar and cerebellar models, and the SC influence in facilitating cerebellar learning at the synaptic level.

2.4 Spino-cerebellar and cerebellar outcome in muscle space

We then evaluated the outcome in muscle space of both the spino-cerebellar and cerebellar models. We compared the recorded EMG envelopes to the main activated muscles from the spino-cerebellar and cerebellar models during performance of P1 and P2 trajectories. Fig 5 displays all the participants’ recorded EMG signals and corresponding joint cocontraction index (CCI). Fig 6A illustrates the deltoid posterior (DELTpost) and brachialis (BRA) muscles during P1 slow circle performed by both the spino-cerebellar and cerebellar models, whilst deltoid anterior (DELTant) and triceps lateral (TRIlat) muscles are depicted for P2 fast flexion-extension. Both models reproduced the main activation patterns of each muscle with a small shift for P2 DELTant and TRIlat. The correlation between the spino-cerebellar or cerebellar activation and the EMG signals was generally larger than 0.5 (see Supporting Information (S13 and S14 Figs)). The correlation was, however, larger for the spino-cerebellar model for most of the muscles and scenarios. Nevertheless, the correlation averaged over muscles was similar between the two models for all the movements and we could not conclude on a better muscle pattern reproduction by one or the other model.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. EMG recordings obtained from both P1 and P2 during flexion-extension and circular movements.

A) EMG recordings obtained from P1 during flexion-extension movements and B) circular movements. The EMG signals of all recorded movements performed at different speeds are interpolated and displayed together to highlight the main activation patterns. Muscles are grouped together by joint (left column for shoulder, right column for elbow) and direction of motion (top row for flexor muscles, bottom row for extensor muscles). For simplicity, biarticular muscles (i.e., biceps long and triceps long) are only displayed on the shoulder group (left column). C), D) Resulting joint cocontraction index (CCI) from P1 during flexion-extension and circular movements, respectively. E) EMG recordings obtained from P2 during flexion-extension movements and F) circular movements. G), H) Resulting joint CCI from P2 during flexion-extension and circular movements, respectively.

https://doi.org/10.1371/journal.pcbi.1011008.g005

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Spino-cerebellar and cerebellar model performance in muscle space for all P1 and P2 recorded trajectories.

A) Comparison of muscle activation signals with recorded EMGs: the comparison only shows the main activated muscles during recordings of P1’s slow circle (two left columns) and P2’s fast flexion-extension (two right columns). The plots show the muscle activity of the 200 trials reaching the learning convergence metric, as well as their mean and standard deviation (std), for the two models performance. EMG signals are scaled by the maximum of the activation signals for each muscle for the sake of representation. B) Joint cocontraction index (CCI) from EMG activity and both models performance, for the trajectories represented in A). EMG CCI are scaled by the maximum of the models CCI for the sake of representation. C) Joint CCI evolution over the 2000-trial motor adaptation process. D) Joint CCI values for both models and all P1 (two left columns) and P2 (two right columns) trajectories. E) Joint CCI correlation between the models and EMG for all P1 (two left columns) and P2 (two right columns) trajectories. F) CCI-MAE_vel relation: linear regression between joint CCI and joint MAE_vel over all the trajectories from P1 (two left columns) and P2 (two right columns).

https://doi.org/10.1371/journal.pcbi.1011008.g006

Results might not be conclusive when referred to a direct, muscle by muscle comparison between our models performance and the recorded EMG; note that our musculoskeletal upper limb model was actuated by eight muscles, a mere simplification of the complex muscle dynamics of the human upper limb. To overcome this, we further studied performance in muscle space using the joint cocontraction index (CCI), which unifies muscle activity per joint and provides a more comprehensive analysis (refer to the Methods section for details on the CCI computation procedure).

Both the spino-cerebellar and cerebellar model exhibited a gradual decrease in joint CCI as motor adaptation evolved (see S15 fig for CCI evolution). Due to computational constraints, each trajectory repetition was limited to 2000 trials, at the end of which CCI still showed a decreasing trend. The spino-cerebellar model displayed higher CCI values than the cerebellar model in some cases, whilst lower CCI values in others. Consequently, we could not draw definitive conclusions on the final CCI values. Instead, since we used convergence of the kinematic performance as our learning metric, we focused on the CCI values as the kinematic adaptation progressed.

During the early stages of learning, the spino-cerebellar model exhibited higher overall CCI values than the cerebellar model (see Table 1). Subsequently, we measured the joint CCI once an accurate internal model was developed, as indicated by a stable kinematic performance (see Methods for details on learning convergence metrics). We found that the spino-cerebellar model better reproduced the CCI patterns at the level of the elbow for P1 slow circle and at the level of the shoulder for P2 fast flexion-extension (Fig 6B). Significantly, the spino-cerebellar model provided higher CCI values at the kinematic convergence point (Fig 6C) for all P1 and P2 trajectories, both for the shoulder and elbow (Fig 6D). We then compared the CCI provided by both models with the CCI from the recorded EMG (Fig 6E). The correlation was mainly higher for the spino-cerebellar model. We observed a similar trend as that observed for MAE_vel, therefore, we performed a linear regression between the CCI and MAE_vel for each joint. The results (Fig 6F) highlighted a linear trend between these quantities for P1 elbow and P2 shoulder and elbow (with a coefficient of determination of 0.94, 0.63 and 0.53 respectively), whereas P1 shoulder presented a weaker relationship (with a coefficient of determination of 0.23).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Joint CCI mean and standard deviation values during the first 400 trials for each P1 and P2 trajectory, for both the spino-cerebellar and cerebellar model.

https://doi.org/10.1371/journal.pcbi.1011008.t001

Overall, we highlighted various findings that were consistent for various trajectories with various initial and final positions and speeds. The spino-cerebellar model provided more stable and faster learning with simpler cerebellar synaptic adaptation. Furthermore both models exhibited a gradual reduction in cocontraction as learning progressed; however, the spino-cerebellar model reached learning convergence with higher CCI values and better correlation to the recorded EMG.

2.5 The spinal cord increases the robustness against motor perturbations

The experimental setup used in the previous sections for the subjects recording sessions did not provide neural activity data, thus preventing any conclusions about the roles of the cerebellum and SC in response to external perturbations. It was not possible to determine whether the cerebellum or SC would be the main contributor to the subjects’ response to external perturbations. To circumvent this data limitation, we used our computational approach. By including or removing the SC from the control loop we were able to investigate the SC influence in responding to perturbations. To study the response against external perturbations of both the spino-cerebellar and cerebellar models, we used our lab designed benchmark: upper limb flexion-extension movements with bell-shaped velocity profiles, characteristic of reaching movements [44]. This kind of movement is usually used for addressing active-limb control malfunctioning, as cerebellar patients usually display upper limb oscillatory tremors that result in endpoint overshooting and undershooting when reaching a target [45].

Both models faced 2000 consecutive trials of the flexion-extension movement performed at different speeds (3s, 2.3s, 1.5s); after motor adaptation, both models succeeded in performing the target kinematics (see Supporting Information, S16 fig). Once both models adapted to perform the desired trajectories, we tested the contribution of the SC in handling motor perturbations. For that, we induced a set of external forces: i.e., 50 N for 30 ms applied to the hand in different directions and at different points along the flexion-extension movement, resulting in kinematic deviation (Fig 7A). We then measured the MAE deviation from the ideal, no-perturbation scenario (Fig 7B–7E). Each perturbation type was applied on 50 separate trajectory trials to get an average response (see Methods). Besides, to gain a deeper understanding of the SC involvement in handling perturbations, we individually assessed the influence of the stretch reflex and reciprocal inhibition mechanisms. To that end, we applied the aforementioned set of external perturbations to two additional cases: i) SC equipped with just stretch reflex (SR-cerebellar model); ii) SC equipped with just reciprocal inhibition (RI-cerebellar model). In order to focus on the contribution of the two SC mechanisms and conduct a direct comparison between them, for the SR-cerebellar and RI-cerebellar cases, we set the cerebellar synaptic weights to those developed by the spino-cerebellar model. More specifically, we equipped the cerebellar network with the synaptic solution acquired by the cerebellum after motor adaptation when the SC model was fully equipped, thus preventing the cerebellum from learning how to compensate for the deficiencies induced by the lack of spinal mechanisms. The cerebellar learning capability was disabled in all four cases to prevent adaptation to the perturbations. Note that, in previous research, we studied the cerebellar ability to effectively handle external perturbations during motor adaptation under changing task conditions [18, 20]. By disabling the cerebellar learning capability, we could specifically focus on the SC contribution.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Spino-cerebellar, SR-cerebellar, RI-cerebellar and cerebellar model responses to external motor perturbations during bell-shaped flexion-extension trajectories.

A) Kinematic performance of both the spino-cerebellar and cerebellar models under a forward perturbation at the flexed position whilst performing the 2.3s flexion-extension trajectory. 50 trials are displayed. B) Position deviation () caused by all the perturbations applied during the 2.3s flexion-extension trajectory for the four models. Mean and standard deviation (std) of 50 trials are displayed. C) Mean position and std for all the perturbations applied to the flexion-extension trajectories performed at different speeds (3s, 2.3s, 1.5s). 50 perturbed trials were used for each perturbation type. D) Velocity deviation () caused by all the perturbations applied during the 2.3s flexion-extension trajectory for the four models. Mean and std of 50 trials are displayed. E) Mean velocity and standard deviation for all the perturbations applied to the flexion-extension trajectories performed at different speeds (3s, 2.3s, 1.5s). 50 perturbed trials were used for each perturbation type. F) Shoulder and elbow CCI values for the four models. Mean and std of 50 trials without perturbation are displayed.

https://doi.org/10.1371/journal.pcbi.1011008.g007

The kinematic performance of both the full spino-cerebellar and cerebellar models under one perturbation type, whilst performing the moderate flexion-extension movement (2.3s), shows that the cerebellar model exhibits a larger kinematic deviation compared to the spino-cerebellar model, particularly at the elbow level Fig 7A). When analysing the responses of the four models (spino-cerebellar, cerebellar, SR-cerebellar, and RI-cerebellar) to the set of perturbations applied during the moderate flexion-extension trajectory (Fig 7B and 7D), we found that: i) the spino-cerebellar model exhibited significantly smaller Mean Absolute Error deviation (ΔMAE) compared to the cerebellar model, both in terms of position and velocity, for the majority of the applied perturbations; ii) the SR-cerebellar model exhibited smaller ΔMAE compared to the RI-cerebellar model. Additionally, the SR-cerebellar and spino-cerebellar models exhibited similar deviations for most of the perturbations. Similar results were obtained for the slow and fast bell-shaped flexion-extension trajectories (please refer to Supporting Information for the corresponding figure, S17 fig).

The kinematic performance analysis on the deviations of the four models in response to all the applied perturbations during the different trajectories (Fig 7C and 7E), confirmed the similar behaviour of the spino-cerebellar and SR-cerebellar models. Both models exhibited superior performance and greater robustness compared to the cerebellar and RI-cerebellar models. All these findings support the role of the SC in handling external motor perturbations, and suggest that the stretch reflex component plays a dominant and more effective role in dealing with perturbations, whilst the reciprocal inhibition mechanism is not as extensively involved.

Finally, we computed the joint CCI values of our four models (Fig 7F) during the three trajectories without perturbations. This analysis aimed to determine whether the CCI could serve as a biological marker to predict the performance against perturbations. The results showed that the SR-cerebellar model exhibited the highest CCI values, followed by the spino-cerebellar model, the RI-cerebellar model, and lastly the cerebellar model. This observation indicates a correlation between higher CCI values and greater robustness against perturbations, as models with higher CCI values exhibited smaller trajectory deviations. Based on these findings, we can conclude that the presence of the spino-cerebellar pathway contributes to better handling external motor perturbations, with the stretch reflex playing a prominent role, which leads to increased cocontraction levels.

3.1 Cerebellum and spinal cord modelling assumptions

The implemented cerebellar and spinal cord computational models are physiologically based and adhere to the principle of biological plausibility. Nonetheless, certain assumptions were made to facilitate interpretation of the results.

Regarding the cerebellum, it is well-known for its role in motor adaptation and learning, a role mainly supported by the cerebellar ability to acquire internal models of both body-plant dynamics and external objects [68]. These internal models are found to be either forward or inverse; a forward model maps the sensorimotor state to its predicted motor behaviour, whilst an inverse model maps the desired behaviour to the motor commands that will make it possible. The literature addresses the existence of both inverse and forward models in the cerebellum and, rather than confronting the two model alternatives, there is theoretical, computational, and behavioural experimentation that supports the coexistence and complementarity of both approaches [19, 68–71]. When modelled as a forward model, the cerebellum modulates the descending motor commands from the motor cortex to correct the mismatch between predicted and actual motor behaviour. In such cases, the body-plant dynamics are learnt at higher brain areas, and the cerebellum operates as a corrective mechanism in addition to the already learnt dynamics. Conversely, when functioning as an inverse model, the cerebellum does not rely on descending motor commands. Instead it can directly provide the entire motor output and bypass the motor cortex [72]. In our work, the cerebellum is implemented as an inverse model, therefore putting it in a worst-case scenario in which it needs to learn the entire body-plant dynamics. The purpose of this approach is to directly address the influence that the SC has on cerebellar motor learning.

If the implemented cerebellar model were a forward model, it might improve tracking performance; however, it would hinder the evaluation of the influence of the SC on cerebellar learning. The SC has a direct and fast action over muscles, thus leading to significant modifications of the arm-plant dynamics. The SC modification of the arm-plant dynamics would affect the cerebellar learning whilst also influencing the motor cortex output. If the SC control mechanisms were already being accounted for at higher brain areas, it would indeed pose challenges in evaluating and quantifying the SC effect on cerebellar learning. It would be difficult to compare the impact of the SC between our spino-cerebellar and cerebellar cases due to the potential overlapping factors incorporated by the already accounted SC control mechanisms at higher brain areas. Using the cerebellum as an inverse model provides a direct means to assess the effects of the SC on cerebellar motor learning. By isolating the role of the cerebellum in motor control, we were able to study how the SC influences the cerebellar motor learning, i.e., comparison between our spino-cerebellar and cerebellar cases is straightforward.

To further facilitate the comparison between our spino-cerebellar and cerebellar cases, we simplified the action of the cerebellar output commands, by omitting other inputs to the spinal cord, e.g. from the motor cortex and basal ganglia. In our cerebellar case, motor behaviour is solely driven by the cerebellum as the only nervous region present in the control loop. Therefore, in this case, the cerebellar output commands directly controlled muscle activation. In the spino-cerebellar case, we maintained the cerebellar descending commands to the SC as direct control signals, which acted upon motoneurons, thus ensuring equivalent cerebellar output commands in both the spino-cerebellar and cerebellar cases. This simplified descending pathway facilitated directly studying the influence of the SC on cerebellar motor learning. Notably, our findings suggest that the SC facilitates cerebellar learning, as evidenced by the differences in the developed cerebellar solutions with and without SC involvement (i.e., spino-cerebellar vs. cerebellar cases exhibited different GC-PC synaptic weight distributions).

However, the intricate reality of the nervous system presents greater complexity; a significant convergence of of cerebral and cerebellar efferent pathways occurs within the SC interneuronal circuitry before reaching motoneurons [73]. This raises a key question: do cerebellar internal models include this intermediary circuitry and account for the interaction amongst descending pathways, beyond the dynamics of the musculoskeletal system?

Our work revealed that cerebellar learning is impacted by the interaction with SC circuitry, and therefore that spinal cord properties should not be underestimated when studying learning. Other descending pathways, such as those from the pyramidal cortical system likely also impact cerebellar learning. Given that biological motor learning is determined by the interplay amongst various nervous regions, future research should expand this study to explore neural interactions that contribute to this intricate process.

Regarding the cerebellar sensory input, it has been previously stated that within the dorsal spino-cerebellar tract (DSCT) there is a subpopulation of neurons representing whole limb parameters and an equally-sized subpopulation representing joint-related signals. This halved structure allows for an early processing stage of sensory information that facilitates a reference frame for whole limb kinematics [74]. In our work, the focus was not on this early sensory processing in the DSCT. Instead, the proprioception signals provided by the SC in joint space were directly sent to the cerebellum. Nonetheless, despite its joint-based architecture, the cerebellum did account for whole limb kinematics as the driving force behind the motor adaptation process.

Our cerebellar model builds upon Ito’s inverse model assumption [72], which derived from oculomotor system studies. Here, we extend that cerebellar scheme to the upper limb musculoskeletal system, by including a subset of multi-articular muscles, with their associated dynamics, proprioceptive afferents and SC circuits. The structure of our cerebellar model involves assigning a cerebellar microcomplex to each joint, following Ito’s microcomplex theory [72]. Whilst the input sensory signals and output motor signals are joint-based, i.e., divided in a modular organisation of different cerebellar functional units [75], the cerebellar learning at GC − PC synapses, in contrast, considers the entire limb kinematics. GCs are believed to perform a recoding of sensory inputs [14, 76], ensuring an univocal and unambiguous representation of the kinematic state of the limb. GCs, in turn, transmit this sensory state to PCs which, driven by the CFs instructive action, adapt their activity to cope with the desired motor behaviour. Importantly, PCs are innervated by a massive number of GCs, with an estimated ∼175,000 GC-PC synapses in the rat cerebellar cortex for each PC [77]. Amongst that massive GC-PC innervation, PCs receive excitatory inputs not only from GCs within the very same microcomplex but also from GCs belonging to other microcomplexes. This connectivity allows the linkage of different cerebellar microcomplexes through GC-PC synapses [75, 78]. In our model, each PC receives excitatory input from all GCs, not just from those associated with the very same microcomplex. Therefore, each PC receives the entire sensory information from GCs, and a specific teaching/instructive signal through a particular CF corresponding to the same microcomplex, in accordance to CF-PC one-to-one synapses [79]. This connection arrangement makes synaptic plasticity at PCs to be affected by the sensory information from all joints and enables the cerebellum to acquire a comprehensive internal model representation of the upper-limb, rather than a joint-specific representation. Therefore, despite the microcomplex structure being based on individual joints, the ensemble activity of GCs across PCs allows for motor adaptation to account for the entire limb kinematics.

Regarding modelling of the SC circuits, some assumptions were also made. The synaptic weights of the spinal pathways were selected aiming to reproduce physiological connectivity by maintaining the ratio between the amplitudes of excitatory postsynaptic potentials (EPSPs) from Group la afferents reported in [80] and inhibitory postsynaptic potentials (IPSPs) from Group la inhibitory interneurons observed in [81].

The modelled SC did not include heteronymous excitatory projections for the sake of interpretability. Heteronymous projections in the human upper limb have been effectively reported in [82]. According to this study, inhibition occurs significantly more frequently than excitation between antagonist muscle pairs, with a higher frequency of inhibition received from antagonists muscles compared to muscles acting across a different joint. In [5, 83] it was noted that proximal to distal heteronymous Ia connections (from the arm to forearm) were absent, whilst wide connections from distal to proximal muscles were present, although weaker from proximal to shoulder muscles. The authors suggested that these stronger connections at wrist and elbow level may assist the hand muscles in grasping and lifting movements by providing stability to the corresponding joints. Consequently, heteronymous excitatory projections are reported to have less significance than heteronymous inhibitory projections in human upper limb movements. These findings are aligned with our SC model, in which reciprocal inhibition pathways allowed to capture this behaviour.

4.1 Ethics statement

Experiments were approved by the Commission cantonale d’éthique de la recherche sur l’être humain du canton de Vaud (CER-VD) under the license number 2017–02112 and performed in accordance with the Declaration of Helsinki. The two participants gave their written consent.

4.2 Spino-cerebellar control loop

The cerebellar and SC models operated in a closed loop with joint and muscle feedback (Fig 1A), in which cerebellar motor learning was assisted by fast reflex response and muscle activity regulation provided by the SC. The cerebellar model received sensory input signals describing the desired motor state (desired position, Q_d, and velocity, , per joint) and the actual motor state of the upper limb musculoskeletal model (actual position, Q_a, and velocity, , per joint). The comparison of the desired and actual motor states provided the instructive signal (ϵ per joint), also received by the cerebellum. The cerebellar output comprised a flexor-extensor (i.e., agonist-antagonist) pair of control signals (M_f and M_e per joint) that were sent to the SC model, which also received direct muscle feedback (length, l_m, and velocity, , per muscle). The SC generated the muscle excitation signals (u_m per muscle) resulting in muscle activation which finally actuated the upper limb musculoskeletal model, thus closing the loop. To contextualise the spino-cerebellar integration, we also implemented the control loop lacking the SC circuits (Fig 1B). In this scenario, the cerebellar output signals were directly used as muscle excitation signals. The control loop included sensory and motor delays, mimicking the biological pathways. In the cerebellar sensorimotor pathway, there exists a delay ranging from about 100 to 150ms (with inter and intraindividual variations), accounting for the time spent from the generation of a motor command until sensing back its effect [84]. Regarding the SC to muscles transmission, a delay of about 30ms has been reported for the upper limb [85, 86]. Our spino-cerebellar model included a 50ms sensorial delay affecting the reception of sensory inputs in the cerebellum; a transmission delay of 30ms from the cerebellum to the SC, and 30ms from the SC to the muscles, total motor delay of 60ms. The asymmetry between sensory and motor delay stands for the higher latency found in neuromuscular junction, electromechanical and force generation delays (involved in the motor pathway), compared to the sensing, nerve conduction and synaptic delays (involved in the sensory pathway) [87].

The following subsections describe the different components of our spino-cerebellar control loop. The various building blocks were integrated using Robot Operating System (ROS), allowing a modular implementation.

4.3 Cerebellar model

We implemented a spiking neural network (SNN) replicating some cerebellar neural layers and equipped with spike-timing-dependent plasticity (STDP) to allow motor learning and adaptation. The cerebellar SNN model was adapted from previous models [18–21], which have already been used to study cerebellar motor learning, adaptation capabilities, and robustness to dynamic changes. Importantly, the cerebellar model presented here and the ones presented in the aforementioned previous works, all share the same neural populations and learning mechanism. Thus, building upon previous research, the current work focused on the influence of the SC on cerebellar motor learning.

The cerebellar SNN structure was divided in the following neural layers: i) mossy fibres (MFs), constituted the sensory input layer conveying the desired and actual motor state signals (Q_d, , Q_a, ); ii) the spiking activity of MFs was transferred through excitatory afferents to the granule cell (GC) layer, where the sensory input information was univocally recoded; iii) the axons of the GCs, i.e., the parallel fibres (PFs), formed excitatory connections with the Purkinje cells (PCs); iv) PCs also received the excitatory action of the climbing fibres (CFs) conveying the instructive signal (ϵ); v) the deep cerebellar nuclei (DCN) layer received the inhibitory action from PCs and excitatory connections from both MFs and CFs. The DCN spiking activity was translated into output motor commands (flexor-extensor motor control signals, M_f and M_e) that constituted the cerebellar motor response to the sensory stimuli. Every neural layer was divided in two microcomplexes [72], being each microcomplex oriented to drive one of the two joints (shoulder or elbow). Each microcomplex at the PC-CF-DCN loop was partitioned into two regions: agonist and antagonist. The agonist region operated the joint flexor muscles, whereas the antagonist region operated the extensor muscles. This synergic agonist-antagonist (flexor-extensor) architecture allowed the cerebellar model to regulate the spatiotemporal muscle activity patterns [55], key for successful motor control [88]. See Fig 8 for a schematic representation of the cerebellar network, and Table 2 for network topology.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Cerebellum model.

A) Neural layers, connections, input and output sensorimotor signals. The input signals are conveyed by the mossy fibres (MFs), which project excitatory synapses to the granule cells (GCs). These perform a recoding of the input signals, and project excitatory connections through the parallel fibres (PFs) reaching Purkinje cells (PCs). PF-PC connections are endowed with plasticity, balanced between the long-term potentiation (LTP) caused by the input PF spikes, and long-term depression derived from the climbing fibres (CFs) activity reaching PCs. CFs convey the instructive signal. Finally, PCs project inhibitory synapses towards the deep cerebellar nuclei (DCN), the output layer of the cerebellar model, which also receives a baseline excitatory action from MFs and CFs. B) Detailed schematic of the cerebellar connections. Each GC receives the input excitatory action from a unique combination of four MFs. Each input signal (Q_d, , Q_a, ), is codified by ten MFs, being only one out of the ten MFs active at each time step. Hence, at each time step, four MFs will be active (one per input signal). That unique combination of four input MFs excites one single GC, allowing to perform a univocal representation of the sensory input at the granular layer. PCs then receive the excitatory action from all GCs in the cerebellar model and only one CF, allowing to relate the joint-specific instructive signal, to the global sensory state received from GCs. The PC-CF-DCN loop differentiates between agonist and antagonist regions, thus allowing simultaneous control of both flexor and extensor muscles.

https://doi.org/10.1371/journal.pcbi.1011008.g008

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Cerebellar neural topology.

Dashed entries stand for not applicable. Each GC-PC synapse was randomly initialised within the range [4.3, 5.2nS], except for one of the three repetitions of the 2000-trial motor adaptation process for each motor task. In that specific repetition, all GC-PC synapses were homogeneously initialised with 4.8nS¹.

https://doi.org/10.1371/journal.pcbi.1011008.t002

Consistently with the Marr-Albus-Ito theory on cerebellar motor adaptation [90–92], our cerebellar SNN model was equipped with synaptic plasticity at the GC-PC synapses. The synaptic weights were adjusted by means of an STDP mechanism that correlated the sensory information (univocally coded at GCs and transferred to PCs through PFs) and the instructive signal (conveyed to PCs by CFs). This STDP mechanism was a balanced process of long-term potentiation (LTP) and long-term depression (LTD). Each time a PC neuron received a GC spike through a PF, that synapse was potentiated (LTP) by a fixed amount as follows: (1) where stands for the synaptic weight change between GC i and PC j; α = 0.006nS is the synaptic weight increment; and δ_GCspike(t) is the Dirac delta function of a GC spike, received at PCs through PFs.

When the spiking activity of a CF conveyed an instructive signal to a PC neuron, the GC-PC connection that was involved in that error generation was depressed (LTD) as described by: (2) where β = −0.003nS stands for the synaptic weight decrement; and k(x) defines the integrative kernel with eligibility trace correlating past sensory inputs with the present instructive signal, i.e., the amount of LTD due to a CF spike depended on the previous GC activity received at PCs through PFs (see [20, 21] for a further description). A well-balanced LTP-LTD process changed the PF-PC synaptic weights, thus modifying the PCs output activity and the inhibitory action of PCs over DCN neurons, which ultimately varied the DCN output activity. Modulating the DCN activity allowed adaptation of the output motor response to the input stimuli. An iterative exposure to the sensory patterns defining the desired motor task, allowed adapting the motor response for error reduction.

We used leaky integrate and fire (LIF) neurons (see Supporting Information S1 Text) and EDLUT simulator [93] to build the cerebellar SNN model. Please see [20, 21] for a further review of the STDP mechanism and cerebellar layers.

4.4 Spinal cord model

Our SC model integrated the descending control signals from the cerebellum and the direct muscle feedback (Fig 9A). The SC model allowed fast reflex response and muscle activity regulation by means of monosynaptic Ia stretch reflex and disynaptic reciprocal inhibition pathways between antagonist muscles. The motoneuron (MN) of a given muscle received the following inputs: i) an excitatory connection conveying the cerebellar output signal (M_f or M_e, for flexor or extensor muscle); ii) an excitatory connection from the Ia afferent fibre of the muscle (i.e., stretch reflex); iii) an inhibitory connection from the Ia interneuron (Ia IN) innervated by the Ia afferent of the antagonist muscle (i.e., reciprocal inhibition). The antagonist relation between the muscles of the upper limb model is detailed below. The neuron leaky integrate dynamics of the MN firing rate, r, were modelled as follows: (3) where τ = 1ms stands for the spinal neuron activation time constant; with D = 8, emulating the on-off behaviour of neurons; i describes the MN input signals; w_i is the synaptic weight of the input connection, being 1.0 for excitatory synapses and 0.5 for the inhibitory to reproduce physiological connectivity [80, 81] (see Discussion for further details); r_i is the input activity; and τ_i = 30ms stands for the stretch reflex response delay. Depending on neuron size, τ can vary from 1 to 10ms [94], we only considered fast-response neurons as in [30]. For the upper limb, τ_i is about 30ms [85, 86]. The output rates of the MNs are finally provided as muscle excitation signals to the musculoskeletal model through a sigmoid (u(t) = σ(r(t))), thus inducing movement. The dynamics of Ia IN neurons followed the same description, with differing input activity including inhibitory connections between antagonist Ia IN (Fig 9B).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. Spinal cord model.

A) The spinal cord circuits were modelled as one motoneuron per muscle, receiving an excitatory input control signal (M) from the cerebellum, an excitatory connection from the Ia afferent fibre of the muscle (i.e., stretch reflex) and an inhibitory connection from the Ia interneuron (Ia IN) innervated by the Ia afferent of the antagonist muscle (i.e., reciprocal inhibition). We also included inhibitory connections between antagonist Ia interneurons. Each neuron is modelled with leaky integrate dynamics. B) Antagonist relation between the eight upper limb muscles: all the muscles shared the same synaptic weight for the stretch reflex and reciprocal inhibition pathways, i.e., 1.0 for excitatory synapses and 0.5 for the inhibitory.

https://doi.org/10.1371/journal.pcbi.1011008.g009

We used Prochazka’s model for the Ia afferent feedback dynamics [95], with a mean firing rate of 10Hz [30, 96, 97]: (4) where l_m and describes the muscle fibre length and velocity in mm and mm/s; and |x|₊ = max(|x|, 0.01). The output rate, r_Ia, was scaled by its maximum r_Ia,max to get a normalised value, i.e., r_Iaϵ[0,1].

To model the SC we used FARMS Python library, developed at the BioRobotics laboratory [98].

4.5 Musculoskeletal upper limb model

We used a two DOF musculoskeletal upper limb model as the front-end body to be controlled. The model, adapted from [41], included two flexion-extension joints: shoulder and elbow. The model was actuated by eight Hill-based muscles [99], with the following joint distribution: i) for the shoulder, flexion was carried by the deltoid anterior (DELTant) and the biceps long (BIClong), and extension was conducted by the deltoid posterior (DELTpost) and the triceps long (TRIlong); ii) for the elbow, flexion was provided by the biceps long and short (BICshort) and the brachialis (BRA), whilst extension was allowed by the triceps long, lateral and medial (TRIlat, TRImed). Note that BIClong and TRIlong were biarticular muscles, as they actuated both joints. The antagonist relation between muscles is depicted in Fig 9B. The Hill-based muscle dynamics were the following: (5) with f_m the muscle force, f_lv a combination of the force-length and force-velocity curves, f_p the passive force-length curve, θ the pennation angle, a the muscle activation (i.e., the concentration of calcium ions within the muscle), and u the muscle excitation (i.e., the firing of the MN) [99]. We used OpenSim physics engine to simulate the muscle and skeleton dynamics [42]. To allow using kinematics and EMG from lab recordings, an OpenSim upper limb model was scaled to match the morphology of each lab participant. This scaling process was achieved using OpenPifPaf Human Pose Estimation algorithm [100] during the static period and OpenSim scaling tool.

4.6 Benchmarking with various motor tasks

We used a set of different motor tasks to be performed by the spino-cerebellar and cerebellar models, differentiating between two scenarios: lab recorded and lab designed motor tasks.

For the lab recorded scenario, we used kinematics and EMG recordings from healthy participants performing different arm movements. Experiments were approved by the Commission cantonale d’éthique de la recherche sur l’être humain du canton de Vaud (CER-VD) under the license number 2017–02112 and performed in accordance with the Declaration of Helsinki in NeuroRestore laboratory at Lausanne CHUV. After obtaining their written consent, two participants, P1 and P2, were asked to perform planar reaching movements (flexion-extension) and continuous circular movements, both movements performed in the vertical plane and at various speeds (self-selected speeds). For flexion-extension movements both shoulder and elbow moved in the same direction, whilst during the continuous circular movements the joints moved in opposite directions. Thus, our benchmark includes interaction torques both assisting and resisting the movement. The recorded kinematics (i.e., joint position and velocity) constituted the desired motor state (Q_d, ) used as the control loop sensory input, whilst the EMG recordings supported model validation in muscle space. For each recorded motor task we ran the experimental setup with both the spino-cerebellar and cerebellar models, using an OpenSim upper limb model scaled to match the participant’s morphology. We then compared the models’ experimental performance to the lab recordings in both joint and muscle spaces.

P1 and P2 movements were recorded using an RGB-D camera, and we used OpenPifPaf human pose estimation algorithm [100] to extract the 2D positions of the participant’s anatomical joints at a frame rate of 25fps. Then 3D pose was deduced from the 2D pose, camera intrinsic, and depth information after accounting for distortion. The occlusions were removed using specially designed filters that ensure coherence in joint anatomy and time. We scaled an OpenSim upper limb musculoskeletal model to match the participant’s morphology, and ran inverse kinematics (IK) over the body segment kinematics, thus allowing the extraction of joint position and velocity from the participant’s motion. P1 generally performed fast movements, and the kinematics recordings of the fast circular movements were too noisy to extract joint position, thus we excluded this scenario from our analysis. For muscle activity, we recorded EMG using Delsys system and Trigno Avanti and Trigno Quattro sensors with an acquisition frequency of 1259.3Hz. We aligned the EMG with the kinematics signals thanks to a trigger inducing a pulse in an additional EMG channel and lightning a led in the camera range. We then computed the EMG envelopes to compare with our models muscle activation signals. For each recorded signal, we removed the mean and rectified the signal, which was then filtered using a low pass Butterworth filter with a 5Hz cutoff frequency. We applied the same processing steps to the maximal voluntary contraction (MVC) signal of each muscle (recorded at the beginning of the session), and used the maximal value of the processed MVC to normalise the corresponding muscle processed EMG signal.

For the lab designed scenario, we implemented a set of flexion-extension movements with different bell-shaped joint velocity profiles, characteristic of multi-joint arm reaching movements [44]. We then used the joint kinematics (Q_d, ) as the desired motor state to be performed by the spino-cerebellar and cerebellar models (please see Supporting Information for a depiction of the bell-shaped trajectories, S10–S12 Figs). We broadened the benchmark by adding a perturbation study using these bell-shaped trajectories. After cerebellar learning consolidation, we applied a set of motor perturbations whilst the trajectories were being performed: 50N for 30ms, applied to the hand in different directions and at different points along the flexion-extension movement. Each perturbation type was applied to 50 separate trials to get an average response, leaving 3 non-perturbed trials in between perturbed trials so that the model returned to its unperturbed state. Note that cerebellar learning was disabled during the perturbation study, to avoid cerebellar adaptation to the external forces and focus on SC response. To further extend the perturbation benchmark we tested two additional models in the aforementioned setup: i) SR-cerebellar model; i.e., spinal cord model equipped with just stretch reflex; ii) RI-cerebellar model; i.e., spinal cord model equipped with just reciprocal inhibition. This scenario allowed us to assess the influence of each implemented SC mechanism in handling perturbations.

Using this motor benchmark, and comparing the performance of the spino-cerebellar and cerebellar models, we could evaluate the cerebellum and spinal cord integration in terms of: muscle activity, motor adaptation and joint space performance, synaptic adaptation, and response to motor perturbations, for various trajectories with different initial and final positions and speeds. Please see Supporting Information for a representation of the motor tasks joint kinematics (S1–S12 Figs).

4.7 Cerebellar instructive signal

The cerebellar instructive signal ϵ(t) was obtained as the mismatch between the desired and actual joint state, combining in a single value per joint both position and velocity errors as follows: (6) where K_p = 3 and K_v = 1 are the position and velocity error gain, respectively. The trajectory error signal in joint space can be derived from the proprioceptive and sensory information conveyed by the spino-cerebellar tract from the muscle spindles (muscle length) and Golgi tendon organs (muscle force) to the cerebellum [101].

Both the spino-cerebellar and cerebellar models were trained exclusively using kinematics to highlight the influence of the SC in modulating muscle activity. If EMG data had been incorporated in training the models, the cerebellar network would adapt and learn to replicate the recorded muscle patterns, resulting in similar muscle recruitment strategies for both the spino-cerebellar and cerebellar models. In other words, the influence of the SC in muscle activity modulation would be diminished.

4.8 Performance metrics

4.8.2 Measuring learning performance.

To measure the learning convergence (i.e., number of trials required to reach a stable trajectory tracking), we used control chart metrics [43]. Throughout the MAE_pos and MAE_vel curve of each motor task (all performed 3 times, each repetition consisting of 2000 learning trials) we computed the mean (μ) and standard deviation (σ) using a sample size of 200 trials, which provided the following performance stability limits: (9)

We then checked the percentage of those 200 trials within each limit. As the limits were defined by the std, we also checked that the std value was below 0.012rad for position and 0.055rad/s for velocity. Thus, at trial x, the behaviour was stable if the percentage of the 200 previous trials within each limit fulfilled the metrics defined in Table 3, and the std was equal or below the aforementioned values. We then averaged each motor task metrics obtained from each of the three repetitions. By comparing the learning convergence of the spino-cerebellar and cerebellar models (i.e., number of trials required to reach a stable performance) we quantified the effect of the SC in the cerebellar motor adaptation process.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. MAE convergence criteria from control chart.

https://doi.org/10.1371/journal.pcbi.1011008.t003

Additionally, we assessed the learning speed of the two models by considering the number of trials required to reach a target MAE_pos of 0.1rad and a target MAE_vel of 0.5rad/s. We defined the learning speed metric as 1 over this number of trials (). The target values (0.1 rad and 0.5 rad/s) were such that they provided a common measure for all motor tasks whilst also taking into account the diversity of final MAE values amongst the different motor tasks. The final mean MAE of all motor tasks (including both spino-cerebellar and cerebellar models) was 0.03 rad for position, and 0.27 rad/s for velocity. To set the learning speed targets, we doubled these mean values and rounded them to the nearest tenth, resulting in 0.1 rad for position and 0.5 rad/s for velocity.

Thus, we evaluated how long it took for the performance to stabilise (learning convergence) and how fast the performance approached accurate tracking (learning velocity).