Figure 1.
Recalibration of internal sensory predictions.
Sensory afference can result both from external events (exafference) and, as this figure illustrates, from our own actions (reafference) [1]. According to the comparator model, the nervous system establishes the cause of the sensory afference by comparing the actual sensory input with the predicted sensory input [2], [3]. To this end, the sensorimotor system predicts the sensory input which will result from one’s actions on the basis of internal action-related information, such as corollary discharge [5] of the motor command. This prediction is computed by a forward model [6], [7] which additionally takes into account the current state of the motor system and the sensory system [34], [64]. The nervous system then makes a comparison between the actual sensory input and the predicted sensory input. In case of a match, the sensory afference should be interpreted as internally caused. Otherwise, in case of a mismatch, the difference between the actual and the predicted input should be interpreted as externally caused. This difference between the actual and the internally predicted sensory consequences of one’s actions constitutes a prediction error. However, such errors arise not only from external influences. Prediction errors can also result from internal changes, i.e. changes within the sensorimotor system such as growth, fatigue or disease. Thus, one’s internal sensory predictions need continuous recalibration. As previous research suggests [16], [43], [65], this recalibration should compensate only for those prediction errors which result from internal causes. However, internally and externally caused prediction errors do not differ per se. Addressing this issue, we here demonstrate that the recalibration of internal sensory predictions by prediction errors depends on the attribution of the prediction error to internal causes. Figure adapted from Wolpert and Miall, 1996 [66].
Figure 2.
(A) Setup. Subjects viewed the virtual image of their finger (white disc) on the feedback monitor via a mirror (solid orange arrow) while performing horizontal pointing movements. For geometric reasons, the virtual image appeared in the same plane as subjects’ finger movements (dotted orange arrow). Visual feedback could be either veridical, i.e. in spatiotemporal correspondence with subjects’ fingertip, or manipulated online by rotation around the starting point of the movements (solid red arrow: actual movement vector, solid white arrow: rotated visual feedback vector, dotted arrows correspond to projections of these vectors into the monitor or the movement plane, respectively). (B) and (C) Procedure. Feedback trials (B) and perceptual probe trials (C) followed on each other alternately. In both conditions, subjects were instructed to freely choose various motor pointing directions between the subjective directions of right (r = 0°) and anterior (a = 90°). In feedback trials (B), visual feedback (FB, dotted white line) about the pointing movement (MPD, dotted red line) was provided in real time. Feedback could be rotated around the starting point (green disc) of the movements by various angles, either in a clockwise (as in this example) or in a counterclockwise manner. When having completed a movement, subjects visually estimated the direction of their movement (referred to as the estimated pointing direction, EPD, solid grey arrow) by placing a trackball-guided cursor in the respective direction. The perceived pointing direction, defined as the difference PPD = EPD – MPD, allowed to us to estimate the component of the feedback manipulation – and thus of the visual prediction error – which subjects attributed to internal causes. In perceptual probe trials (C), subjects did not receive any visual feedback about their pointing movement (MPD). Consequently, they needed to rely entirely on internal action-related information when estimating their pointing direction (EPD). By analysing subjects’ perceived pointing direction (PPD = EPD – MPD) in perceptual probe trials as a function of the visual manipulation applied in the preceding feedback trial, we assessed how subjects’ internal sensory predictions recalibrated in response to visual prediction errors (see Figure 1 for background information).
Figure 3.
Error attribution and recalibration of internal sensory predictions.
(A) Perceived pointing direction in feedback trials. Subjects’ perceived pointing direction (PPD, blue curve), which is here plotted versus the manipulation applied to the visual feedback, allowed us to estimate the component of the feedback manipulation – and thus of the visual prediction error – which subjects attributed to internal causes. The PPD strongly reflected the size of the feedback manipulation (red line) if these manipulations were small. If manipulations were large, vice versa, subjects’ estimated pointing direction rather resembled the motor pointing direction (green line). By varying the size of the feedback manipulations, we thus gradually manipulated the error component which subjects attributed to internal causes. (B) Relative weight of visual information. We captured the internally attributed share of the prediction error by the relative weight which visual feedback obtained in subjects’ PPD. The relative weight of visual information was defined as the quotient of subjects’ offset-corrected PPD (compare A) and the manipulation applied to the visual feedback. Relative visual weights of 1 and 0 would indicate that a subject attributed the prediction error either entirely internally or, respectively, entirely externally. The relative visual weight was significant for all amounts of manipulation (one-sample one-tailed t-tests), but also quantitatively modulated by the absolute amount of manipulation: increasing error sizes resulted in decreasing shares of the internally attributed error component (paired one-tailed t-tests). (C) Perceived pointing direction in perceptual probe trials. The recalibration of subjects’ internal sensory predictions was reflected by the PPD in perceptual probe trials (blue curve), which is here plotted versus the feedback manipulation applied in the immediately preceding feedback trial. Recalibration was not proportional to the preceding prediction error, i.e. the preceding feedback manipulation, but instead resembled the error component which subjects had attributed internally, i.e. their PPD in the preceding feedback trial (compare A). By way of illustration, the green line shows the assumption that prediction errors would not induce any recalibration. Likewise, the red line corresponds to the assumption that subjects adjusted the perceived pointing direction to the entire amount of the preceding feedback manipulation. (D) Relative recalibration of internal sensory predictions. We defined the relative recalibration to compare the recalibration induced by prediction errors of variable size. The relative recalibration was the quotient of the offset-corrected PPD in a given perceptual probe trial (compare C) and the manipulation applied in the preceding feedback trial. The relative recalibration was significant for amounts of manipulation as large as 10°, 20° and 40° (one-sample one-tailed t-tests), which means that manipulations induced recalibration if exceeding a minimum threshold. Moreover, the amount of manipulation modulated the relative recalibration quantitatively: increasing amounts of manipulation resulted in decreasing values of relative recalibration (paired one-tailed t-tests). Diagrams show mean values ± standard errors calculated across subjects. All reported P-values are Bonferroni-corrected for multiple comparisons within each measure (*** P<.001, ** P<.01, n.s. P≥.10). Positive angles denote counterclockwise rotations.
Figure 4.
Distribution of the relative weight of visual information.
These histograms (mean ± standard error) display the density distribution of the relative weight of visual information across feedback trials. The relative weight of visual information was defined as the quotient of subjects’ offset-corrected perceived pointing direction and the feedback manipulation applied in the same trial (compare Figure 3B). For all amounts of feedback manipulation (5° in A, 10° in B, 20° in C, 40° in D), the histogram exhibits one single peak, indicating a unimodal density distribution. This peak corresponds to the mean relative weight of visual information (broken red line). This finding shows that subjects integrated the internally predicted and the actual sensory consequences of their actions on the level of individual trials when estimating their pointing direction. Alternatively, subjects could have based their estimates solely on internal signals in one trial (corresponding to a relative visual weight of 0) while relying entirely on visual information in another (corresponding to a relative visual weight of 1). This would have resulted in bimodal density distributions, which are not supported by our data. To further support this notion, we statistically examined the distribution of the relative visual weight in feedback trials separately for each subject (n = 11) and for each amount of feedback manipulation (5°, 10°, 20°, 40°). We applied the Shapiro-Wilk test to each of these 44 distributions, testing the null hypothesis that the sample came from a normally distributed population. We found that the null hypothesis was tenable in 41 of the 44 samples (P≥.05, uncorrected). This indicates that, indeed, the relative visual weight was normally and therefore unimodally distributed, both across subjects and feedback manipulations.
Figure 5.
Trial-by-trial recalibration of internal sensory predictions.
To investigate the recalibration of internal sensory predictions on a trial-by-trial basis, we performed a linear regression analysis using the perceived pointing direction in feedback trials (Figure 3A) to predict the perceived pointing direction in the consecutive perceptual probe trials (Figure 3C). This analysis was applied to single manipulation values as well as to all manipulation values. When analysing the correlation coefficients obtained for single manipulation values by a repeated-measures ANOVA, we found no significant main effect of orientation of manipulation (F(1, 10) = 0.66, P = .435), no significant main effect of amount of manipulation (F(3, 30) = 0.70, P = .557) and no significant interaction (F(3, 30) = 2.35, P = .092). We therefore pooled the correlation coefficients across counterclockwise and clockwise manipulations of the same amount. The figure displays the correlation coefficient r for all manipulation values, for veridical feedback (i.e. manipulations of 0°) and for single amounts of manipulation of 5°, 10°, 20° and 40° (mean ± standard error across subjects, compare Table S1 for details). If the amount of visual feedback manipulation was larger than 5°, the internally attributed component of the visual prediction error in a given feedback trial explained the recalibration of subjects’ internal sensory predictions. All reported P-values are Bonferroni-corrected for multiple comparisons (one-sample one-tailed t-tests, *** P<.001, ** P<.01, (*) P<.10, n.s. P≥.10).
Figure 6.
Motor pointing direction as a function of feedback manipulation.
(A) Feedback trials. The mean motor pointing direction in feedback trials (mean ± standard error) is plotted as a function of the visual manipulation, with 0° representing the rightward direction and 90° representing the anterior direction. Subjects showed no systematic online correction of their movement trajectories when presented with deviating visual feedback. (B) Perceptual probe trials. The mean motor pointing direction in perceptual probe trials (mean ± standard error) is plotted as a function of the visual manipulation applied in the preceding feedback trials. Subjects adjusted their motor performance in a manner which compensated for the recalibrated visual movement consequences (compare Figure 3C). Positive angles denote counterclockwise rotations. For the density distribution of the motor pointing direction in space, compare Figure S2.