SHS is associated with BKIN-Technologies (which commercializes the robotic apparatus that was utilized) and was supported by a GlaxoSmithKlein chair in Neuroscience.
Conceived and designed the experiments: FC SHS. Performed the experiments: FC. Analyzed the data: FC SHS. Contributed reagents/materials/analysis tools: FC. Wrote the paper: FC SHS.
In every motor task, our brain must handle external forces acting on the body. For example, riding a bike on cobblestones or skating on irregular surface requires us to appropriately respond to external perturbations. In these situations, motor predictions cannot help anticipate the motion of the body induced by external factors, and direct use of delayed sensory feedback will tend to generate instability. Here, we show that to solve this problem the motor system uses a rapid sensory prediction to correct the estimated state of the limb. We used a postural task with mechanical perturbations to address whether sensory predictions were engaged in upper-limb corrective movements. Subjects altered their initial motor response in ∼60 ms, depending on the expected perturbation profile, suggesting the use of an internal model, or prior, in this corrective process. Further, we found trial-to-trial changes in corrective responses indicating a rapid update of these perturbation priors. We used a computational model based on Kalman filtering to show that the response modulation was compatible with a rapid correction of the estimated state engaged in the feedback response. Such a process may allow us to handle external disturbances encountered in virtually every physical activity, which is likely an important feature of skilled motor behaviour.
It is commonly assumed that the brain uses internal estimates of the state of the body to adjust motor commands and perform successful movements. A problem arises when external disturbances deviate the limb from the ongoing task. In such cases, the estimated state of the body must be corrected based on sensory feedback. Because neural transmission delays can destabilize feedback control, an important challenge for motor systems is to correct the estimated state as quickly as possible. In this paper, we tested whether such a rapid correction is performed following mechanical loads applied to the upper limb. Our results indicate that long latency responses (∼50–100 ms) exhibit knowledge of the relationship between the delayed sensed joint displacement and the current state of the limb at the onset of the motor response. Importantly, this knowledge can be adjusted from one perturbation response to the next, should a distinct perturbation profile be experienced. These results suggest that a correction of state estimation is performed within the limb rapid-feedback pathways, allowing fast and stable feedback control.
Neural transmission delays present a major challenge because the brain cannot directly use sensory feedback to guide motor actions. In order to compensate for feedback delays, the brain must build internal models of the dynamical interaction between the body and the environment, including sensory and motor prediction mechanisms. On the one hand, motor predictions use forward models to convert motor commands into estimates of the state of the body
An important question is whether the motor system uses similar processes to guide feedback responses to mechanical perturbations. Indeed, perturbation loads applied on the upper limb evoke very quick, task-related responses (long-latency, ∼50 ms)
The words ‘sensory prediction’ and ‘motor prediction’ have been often used in the literature to designate the same process, which is the prediction of the consequences of motor commands based on efference copy and internal forward models
In theory, sensory prediction is expected if optimal state estimation is performed while taking feedback delays into account (Kalman filter). In this framework, the present state of the limb (
A: Overhead sketch of a perturbation evoked displacement relative to the prescribed joint location (
In agreement with the model, we show that responses to step perturbations scaled with the step magnitude, regardless of whether changes in magnitude were expected or not. In contrast, initial responses to other unexpected perturbation profiles matched the response for the expected perturbation profile, suggesting that internal models are engaged in these rapid corrective responses. These priors started to influence the motor response within the long-latency time window (∼50–100 ms). Changes in long-latency responses correlated with the expected relationship between the initial joint displacement and the true state of the limb at the onset of the motor response as predicted by simulations using optimal state estimation. Altogether, our results suggest that state estimation guides long-latency motor responses to mechanical perturbations.
The effect of feedback delays on motor performances have been studied in the context of voluntary movement control, with feedback delays typically greater than those characterizing rapid motor responses to perturbations (for instance, visuomotor delays are >100 ms)
A: Computational model based on a state estimator that ignores feedback delays and directly integrates sensory feedback with prior beliefs about the body state. The controller outputs a motor command (
In order to produce stable and accurate feedback responses, we suggest that motor systems rely on optimal state estimation while taking feedback delays into account (
We tested the hypothesis that the brain uses sensory prediction to drive the motor response by exposing participants to a large number of step torque perturbations (1 Nm, 2 Nm and 3 Nm, see
A: Overhead representation of the initial joint configuration and of a typical perturbation related movement. The initial joint configuration is shown in black. Hand path from perturbation onset until the first hand-speed minimum (TH) is represented in solid trace. The remaining portion of the corrective movement is shown in dashed trace. B: Illustration of the different torque profiles: black traces illustrate the step perturbations (1 Nm and 3 Nm are displayed with thin traces), ramp-down perturbations in red and ramp-up perturbations in blue. C: Schematic representation of the effect of a ramp-down perturbation profile on the state estimation: the overestimation (gray region) result from the difference between the expected perturbation profile (step-function, dashed) and the actual perturbation (solid). The ramp-down profile was designed to produce an overestimation as illustrated in
A: Average elbow motion from one exemplar subject when step perturbations were expected (top) and when ramp-profiles were presented in blocked manner (bottom). Displays use the same color code as in
We found that the reversal time was a sensitive parameter that captured the effect of the profile on the kinematics of the corrective movements as well as the modulation of the feedback responses across contexts (catch or blocked, see
Importantly, the changes in reversal times observed in
The model based on Kalman filtering explains the effect of the perturbation profiles on the kinematics of the corrective movements (
A: Illustration of the actual (solid) and estimated (dashed) state variables. From top to bottom, displays are the external torque (TE), the joint velocity and the joint displacement. 2 Nm step perturbation is represented in black, ramp-down in red and ramp-up in blue. B: Estimation error for each perturbation profile, defined as the difference between the estimated state and the true state.
We collected the activity of elbow flexors and extensor muscles in order to determine the time when prior-related components of the response influenced the feedback correction. When participants expected a ramp-down perturbation (blocked condition), the evoked response diverged from the response evoked by 3 Nm step perturbations after 44 ms for Brachioradialis (
A: Top: Average shoulder (dashed) and elbow (solid) joint displacements following extension torques in three different cases, 3 Nm step torques (black), and ramp-down profiles in catch and blocked conditions (thin and thick red traces, respectively) averaged across subjects. Shoulder motion was identical across all conditions and the corresponding traces are superimposed. Middle: Perturbation-evoked response recorded from an elbow flexor (brachioradialis) with the same color code as in the top panel. Bottom: Difference between the 3 Nm step response and the ramp-down in the catch condition (black), and between ramp-down torques in the block and catch (red). Arrows indicate the onset of divergence from the 3 Nm evoked response. B: Response evoked by step-torque perturbations averaged across subjects. The shaded area represents the standard error. The vertical arrow depicts the latest divergence onset across all pair-wise comparisons. C: Changes in evoked response across the conditions where ramp-down (dark gray) or ramp-up (light gray) perturbations were presented in catch or blocked design. The grand average of Brachioradialis and Triceps Lateralis responses was considered for this analysis. Positive designates that the evoked activity decreased in the block condition.
Changes in activity resulting from mistakenly tracking the corresponding step function were significant in the long-latency time window. Following ramp-down profiles, the pre-perturbation activity (−50–0 ms, see
In all, the prior-related component influences the muscle response within about 60 ms of perturbation onset, in a way that correlated with changes in the expected relationship between the initial joint displacement and the state of the limb at the onset of the motor response.
A surprising result from Experiment 1 is that, on average, the difference between ramp-down responses across conditions persisted for a prolonged period of time (
The 2 Nm step and ramp-down were chosen based on the results of the first experiment. After the habituation blocks (see
A: Average hand path from one representative subject following step torques and ramp-down torques sorted by trial n-1. The blue traces are the average across all trials preceded by a step perturbation and the red traces are the average across all trials preceded by a ramp-down perturbation. B: Trial-by-trial modulation of the corrective response depending on the preceding trial for step (open symbols) and ramp-down (filled symbols) flexion and extension perturbations (disk and diamond, respectively). Significant differences in data from single perturbation profile (*) and from all perturbation pooled together (†) are shown at the level
We first addressed whether inverting the internal prior affected the response to the previously expected step perturbation profiles. As predicted, reversal times following step perturbations tended to be delayed when participants were expecting a ramp-down profile, although this trend was only close to significant (
A: Changes in reversal times relative to the average across all step responses from the block condition (similar as in
Second, this experiment was designed to investigate whether the response modulation persisted when the muscles were pre-activated. This experiment was motivated by the response differentiation found at ∼44 ms in the first experiment, which, in theory, indicates that the short-latency pathway may have contributed to the response modulation. We applied a background load on the elbow joint (−1 Nm) to evoked the same baseline activity across the two series of blocks in which ramp-down trials were presented as catch trials or in blocked fashion (Pre. across conditions, t(7) = 0.4,
In this experiment, we verified that the effect reported above was specifically related to the perturbation profiles independent of their magnitude. In theory, the controller only needs to know the perturbation profile to correct the state estimate, independently from the perturbation magnitude. A direct prediction of the model is that participants expecting a step torque should be able to respond to any perturbation magnitude provided that it follows a step function. Alternatively, if changes in control gains are involved, we expect to see a delayed corrective movement following the unexpected 3 Nm step torques since subjects were expecting a smaller perturbation (2 Nm). Feedback responses should also overcompensate for an unexpected 1 Nm perturbation. We tested these predictions by exposing participants to a large number of step torques of 2 Nm and presented step perturbations of 1 Nm or 3 Nm as catch trials following the same distribution as in the first experiment (see
A: Reversal times and hand speed minima across the tested values of step magnitudes as presented in
These results suggest that the variation in the kinematics of corrective movements emphasized above is specific to the shape of the perturbation. Muscle responses of an elbow flexor are shown in
This study shows that internal models of the perturbation loads influence long-latency responses to mechanical perturbations. Simulations based on optimal feedback control suggest that these priors reflect a rapid correction of the estimated state of the limb based on sensory prediction. In general, internal priors strongly influence decisional processes
Although previous studies have suggested that the brain uses sensory prediction following a perturbation
Our approach focuses on the rather simple case of a constant external torque, which is easy to model in the framework of linear systems. However, the limitations of linear systems are only theoretical and our data suggest that participants were able to learn more complex priors corresponding to non-linear ramp-up or ramp-down perturbations. Whether we are able to learn any perturbation profile, or equivalently any mapping between the sensed initial motion and the actual state of the limb, is an open question. Another important question is how multiple priors can be acquired. Our daily lives suggest that we can acquire motor skills in distinct tasks (such as biking and skating) without re-learning every time that we switch between tasks. A recent study in the context of force field learning has emphasized that multiple internal models can be acquired provided that the internal representation of the movements are distinct
Overall, the effects of prior expectations on the muscle response as well as on the kinematics of the corrective movements were quite small. This is not surprising as perturbations were manipulated over a very short time interval (∼50 ms), and the resulting unexpected change in limb motion can only be small. A clear difficulty is that it is not possible to investigate the case where no estimation at all is engaged in the response. Instead, we had to manipulate participants' expectations to extract the evidence for a sensory predictor. Although our approach evoked small effects in terms of magnitude, the results were consistently reproduced across experiments. Importantly, we also showed with simulations that ignoring the use of sensory predictions could lead to instability that should clearly be avoided at all cost.
We also demonstrate two key properties of the sensory predictor. First, we show that the influence of a prior during mechanical perturbations occurs from ∼45 ms to ∼60 ms, at which time the motor response started to diverge towards the appropriate profile. Assuming a contribution of the transcortical feedback with sensory and motor delays of about 30 ms
A second key property of sensory predictors is that it is modified on a trial-by-trial basis, which parallels the properties of the voluntary motor system observed in force-field learning studies
In principle, it is also possible that feedback gains were changed independently from any update in state estimation. Such changes in feedback gains may originate from internal set of the control strategy, or from changes in the peripheral motor apparatus through co-contraction and stiffness modulation
Second, we found that the modulation of long-latency responses according to prior expectation was present even after controlling for the pre-perturbation activity and short-latency reflex (Experiment 3). This experiment was partially motivated by the divergence onset between the expected ramp-down from the 3 Nm perturbations that we found at the end of the short-latency time window (Experiment 1). However, even with similar R1 responses, it is possible that rapid sensory predictions occurred at the periphery
Finally, unexpected changes in the step magnitude did not generate any over nor under compensation. Responses to 1 Nm and 3 Nm step perturbations were clearly similar regardless of whether changes in perturbation magnitudes were expected or not. Therefore, changes in reversal times evoked by ramp perturbations could not be explained by a possible modulation of control gains involved in response to unexpected changes in perturbation magnitude. These results were predicted by the model: the Kalman filter can correct the present estimate of the state of the limb by combining the sensed step magnitude of each individual trial with prior assumptions about the perturbation profile. As a consequence, time-varying feedback responses result from a constant feedback control policy applied to time varying estimates of the state of the limb, which does not require any prior knowledge about the perturbation magnitude. The controller only needs to know the perturbation profile.
Future studies should investigate the underlying neural pathway. The latency of the prior-related component already sets physiological constraints on the possible candidates. The cerebellum is clearly a candidate region given its known implication in prediction processes associated with descending commands
A sensory prediction is critical when abrupt perturbations induce large displacement as in the present study. However, disturbances can also be encountered at smaller scales including noise in neural circuits, and feedback responses are likely engaged at the level of small deviations corresponding to natural variability
The Queen's University Research Ethics Board approved the experimental protocol and participants gave written informed consent following standard procedures.
Subjects interacted with a virtual reality display showing visual targets and a right-hand aligned cursor in the horizontal plane. Participants' right arm was placed on an exoskeleton that can selectively apply torques at the shoulder and/or elbow joints (KINARM, BKIN Technologies, Kingston, ON
This experiment tested whether feedback responses to perturbation engaged internal priors about the perturbation profile. To do so, we used step torque perturbation of varying size and direction so that participants were expecting this perturbation profile. We addressed the effect of unexpected time-varying perturbation by using ramp-up or ramp-down perturbations randomly presented as catch trials (
We also performed an additional experiment to address the influence of the load magnitude on the response profiles to the ramp-up perturbations. Indeed, the response modulation following ramp-up perturbations was weaker, which is partially due to the large perturbation loads applied (3 Nm). We used the same paradigm on 8 participants while using load magnitudes reduced by 20% in order to see whether smaller load magnitudes leave more room for the response modulation following the ramp-up perturbations. Participants countered step torques of 0.8 Nm, 1.6 Nm or 2.4 Nm while ramp-up perturbations (from 0 to 0.8 Nm in 5 ms, and 0.8 to 2.4 in 50 ms) were presented as catch trials with the same distribution as in the main experiment. Participants were also exposed to a block of ramp-up perturbations. The sequence of blocks was randomized across participants. This control experiment reproduced the results of the main experiment and amplified the response modulation across conditions (see
In this experiment, we investigated the effect of a change in the perturbation profile on the feedback response strategy of the following trial by using a random adaptation paradigm. This experiment sought to examine whether the component of the rapid feedback responses that depends on prior expectations can be quickly adjusted from trial-to-trial as observed for voluntary control
This experiment tested two main effects. First, we used a similar paradigm as in Experiment 1 except that we had step perturbations as catch trials while participants were expecting a ramp-down perturbation. Second, this experiment was performed with a constant load applied on the elbow (−1 Nm) to in order to control for the pre-perturbation activity and short-latency reflex. In one series of blocks, participants (N = 8) were instructed to counter the perturbations including step torques of ±1 Nm, ±2 Nm and ±3 Nm. Perturbations were randomly interleaved and added to the constant background load. Ramp-down perturbations were presented as catch trials following the same distribution as in Experiment 1 (16 ramp-down perturbations for 60 step perturbations). In another series of blocks, ramp-down perturbations were blocked and 2 Nm step-torques were presented as catch trials. The sequence of each series of blocks was varied across subjects to eliminate possible order effects.
We finally examined the effect changes in control gains evoked by an unexpected step magnitude. We needed to verify that unexpected changes in the step magnitude did not produce variation in movement kinematics that could account for the effect emphasized in the first experiment. Participants (N = 8) had to counter step torques of ±2 Nm presented in blocks of 48 trials (24× flexion or extension). Step perturbation of ±3 Nm and ±1 Nm were presented as catch trials (4×3 Nm or 1 Nm×flexion or extension per block), summing to a total of 64 trials per block. Each subject performed three blocks.
Shoulder and elbow motion were collected at 1 kHz and digitally filtered at 50 Hz (4th order dual-pass Butterworth filter). We considered both the kinematics of elbow motion as well as hand paths in Cartesian coordinates to validate the use of the single joint model presented below. Muscle activity was collected by means of surface electrodes attached on the muscle belly after light abrasion of the skin with alcohol (DE-2.1, Delsys, Boston, MA). We concentrated on the mono-articular elbow muscles for Experiment 1, 3 and 4 (Brachioradialis, Br.; Triceps Lateralis, Tl.), and on the mono- and bi-articular elbow muscles for Experiment 2 (Biceps, Bc; and Triceps Long, Tg., in addition to Br. and Tl.). The raw EMG signal was amplified (gain = 104), digitally band-pass filtered (10–400 Hz), rectified, and averaged across trials. EMG signals were normalized to the average activity measured against a 2 Nm background load for all muscle samples (except in Experiment 3 where we used the activity evoked by the 1 Nm background load), while participants maintained postural control in the initial joint configuration (elbow = 90 deg and shoulder = 45 deg). The binned analysis of muscles activity was based on average EMG across the different epochs following classical definitions (Pre., −50 to 0 ms, R1, 20 to 45 ms; R2, 45 to 75 ms; R3, 75 to 105 ms and early voluntary from 120 to 180 ms
The importance of the model is to provide a rationale for the experimental design as well as predictions about the effect of the perturbation profile on the kinematics of the corrective movement. The hypothesis that the brain uses a process similar to a Kalman filter was found to be a very powerful approach to characterize the online combination of internal priors with multisensory information
We considered the angular motion of a rigid body as a model of the elbow joint. The choice of a single joint model was compatible with the perturbation-related motion immediately after the perturbation onset. Indeed, because we applied similar amounts of torque at the shoulder and elbow, the initial shoulder acceleration is zero as a result of the initial joint configuration and dynamics. Our data confirmed this property as the shoulder did not move until >100 ms following the perturbation. Therefore, the problem of state estimation following the perturbation reduces to the estimation of the elbow joint displacement in agreement with the single joint model. In addition, more complex models (e.g. nonlinear models including inter-segmental dynamics) are not necessary because the single-joint model captures the problem caused by feedback delays. Thus, we kept the model as simple as possible.
The differential equation of the joint motion was coupled with a first order, low-pass model of muscle dynamics linking the control variable to the muscular torque. The net torque was the sum of a viscous torque proportional to the angular velocity, a controlled torque (
This system was transformed into a discrete time control system by using classical Euler integration with 10 ms time step in order to take noise disturbances into account. Feedback delays were set to 60 ms. This value of feedback delay is compatible with the long-latency transmission delays, and also takes into account the fact that the controller, unlike EMG, can change the control value instantaneously. We therefore added on time step to the usual ∼50 ms considered for long latency delays in order to generate more realistic simulations. The state vector is composed of the joint angle, the joint velocity, the torques and the target location (noted
For this class of system, the Kalman filter gives an unbiased estimate of the state vector (
The task of the controller was to stabilize the joint at a given angle against the external torque and noise disturbances. The cost-function that penalized deviation from the prescribed joint angle was:
The simulations of reaching movements presented in
Finally, the blocked condition for the ramp-up/down perturbation profiles was simulated based on the assumption that the ideal control performance would be achieved if the controller could rely on perfect state information. To approximate this, we artificially set the control signal to 0 for a time interval corresponding to the feedback delay following the perturbation, and then applied the feedback gains to the true state of the system. In this case, the perfect state information corresponds to an estimation error that is zero, and the performance of the resulting control process corresponds to the best-case scenario. The artificial delaying of the response was used to generate a realistic displacement of the joint following the perturbation. We verified that the reversal times following step perturbations were identical with artificially delaying of the response, allowing us to compare changes in reversal times following ramp-perturbations. We should emphasize that the simulations based on perfect state information indicate what the system should do in the ideal case, without dealing explicitly with more complex priors. A theoretical limitation is that such complex profiles are difficult to reproduce within the framework of linear systems without additional dimensions and parameters. We performed additional simulations in which the external torque follows linear profiles (by setting the derivative of TE to a non-zero value), and found the same results as with perfect state information. We decided to concentrate on the simulations with veridical state information because it provided the same prediction with fewer assumptions.
In general, the variability in the reversal times from the simulations was lower than variability observed experimentally. The confidence interval was further reduced by considering the average reversal times across 50 simulation runs. In order to emphasize that effect of the estimation algorithm on corrective movements, we did not attempt to reproduce the experimental variability and chose to concentrate on the average reversal times across simulations (
A shortcoming of our approach is that we change the value of the external torque (
We want to thank K. Moore, H. Bretzke and J. Peterson for technical and logistic support. We also thank I. L. Kurtzer and A. J. Pruszynski for critical comments on an earlier version of the manuscript.