^{1}

^{2}

^{¤}

^{3}

^{1}

^{2}

The authors have declared that no competing interests exist.

Current address: Lise-Meitner-Group for Environmental Neuroscience, Max Planck Institute for Human Development, Berlin, Germany

Plasticity in the oculomotor system ensures that saccadic eye movements reliably meet their visual goals—to bring regions of interest into foveal, high-acuity vision. Here, we present a comprehensive description of sensorimotor learning in saccades. We induced continuous adaptation of saccade amplitudes using a double-step paradigm, in which participants saccade to a peripheral target stimulus, which then undergoes a surreptitious, intra-saccadic shift (ISS) as the eyes are in flight. In our experiments, the ISS followed a systematic variation, increasing or decreasing from one saccade to the next as a sinusoidal function of the trial number. Over a large range of frequencies, we confirm that adaptation gain shows (1) a periodic response, reflecting the frequency of the ISS with a delay of a number of trials, and (2) a simultaneous drift towards lower saccade gains. We then show that state-space-based linear time-invariant systems (LTIS) represent suitable generative models for this evolution of saccade gain over time. This

Constant adjustments of saccade metrics maintain oculomotor accuracy under changing environments. This error-driven learning can be induced experimentally by manipulating the targeting error of eye movements. Here, we investigate oculomotor learning in healthy participants in response to a sinusoidally evolving error. We then fit a class of generative models to the observed dynamics of oculomotor adaptation under this new learning regime. Formal model comparison suggests a richer model parameterization for such a sinusoidal error variation than proposed so far in the context of classical, step-like disturbances. We identify and fit the parameters of a generative model as underlying those of a phenomenological description of adaptation dynamics and provide an explicit link of this generative model to more established state equations for motor learning. The joint use of the sinusoidal adaption regime and consecutive model fit may provide a powerful approach to assess interindividual differences in adaptation across healthy individuals and to evaluate changes in learning dynamics in altered brain states, such as sustained by injuries, diseases, or aging.

The accuracy of saccadic eye movement is maintained through mechanisms of saccade adaptation, which adjust the amplitude [

To induce saccade adaptation in the laboratory [

We recently presented a version of this paradigm in which the ISS (the disturbance responsible for inducing adaptation) follows a sinusoidal variation as a function of trial number ([

Here, we investigate whether a generative algorithm that models saccade gain modifications on a trial-by-trial basis by learning from errors made on previous trials can account for this response. To this end, we implemented and fit a series of state-space models in which a modified delta-rule algorithm updates a hidden or latent variable (for which the experimentally observed adaptation gain is a proxy) by weighting the last experienced visual error, in addition to other error-based and non-error based learning components [

We adopt the approach that these algorithms are linear time-invariant systems (LTIS), in that their coefficients are time and trial-independent. LTIS models, also known as linear dynamical systems (LDS) have been successfully used in a number of motor adaptation studies [

We first analyze the ability of a family of generative models to describe experimental recordings of saccade adaptation by fitting the relevant learning parameters. We then perform statistical model-selection analysis to determine those that best fitted the same data in the various experimental conditions. We fitted models to two data sets, a previously published one [

The Ethics Committee of the German Society for Psychology (DGPs) approved our protocols. We obtained written informed consent from all participants prior to the inclusion in the study. The present study conformed to the Declaration of Helsinki (2008).

We re-analyzed the data we recently collected using a fast-paced saccade adaptation paradigm with a sinusoidal disturbance. We had previously described these data by fitting a phenomenological model that we identified using statistical model selection. For details on the experimental procedures pertaining to this original data set (henceforth, ORIG) and to the selection of the functional form of this phenomenological model, please refer to our former communication [

We applied the same experimental procedure in collecting further data with an enhanced range of frequencies. In this case, thirteen participants ran two sessions with similar Two-way and Global adaptation protocols as used in previous reports [

In collecting this dataset (henceforth, FREQ), each session had 2370 trials divided in 11 blocks. Odd numbered blocks had 75 no-adaptation trials (zero ISS). The five even-numbered blocks consisted of 384 trials each with a sinusoidal disturbance similar to that used before but with frequencies of 1, 3, 6, 12 and 24 cycles per block (i.e., 384, 128, 64, 32, and 16 saccades per cycle, respectively). The order of adaptation blocks was randomly interleaved for each observer and type of adaptation. The program was paused after each adaptation block, giving participants some resting time, and we calibrated eye position routinely at the beginning of each non-adapting (odd-numbered) block. In each trial, the pre-saccadic target step was fixed at 8 degrees of visual angle (dva). The subsequent second step (ISS) then ranged between –25% and +25% of the first step, changing size according to a sine function of trial number.

In a double-step adaptation paradigm [

The second step of the McLaughlin paradigm (i.e., the target displacement inducing a feedback error) then shifts the target during the saccade to a position

In the general case, there would be a constant and a variable component in the second target step,

Saccade amplitude adaptation is usually described in terms of the changes in

The adaptation gain represents the residual of the saccade gain with respect to perfect landing. When a saccade lands exactly on the first target step (a perfectly accurate saccade), the saccade gain will be one while the adaptation gain will be zero. Therefore, the adaptation gain uses perfect landing as the origin of coordinates and quantifies departures from this ideal goal state. Clearly, in both descriptions the reference represents a state of no adaptation. The adaptation gain description may be viewed as following the evolution of the error rather than that of the full eye movement. As long as the true underlying learning model is strictly linear, both descriptions must be equivalent since they relate to each other by a shift. We used the adaptation gain,

In implementing the phenomenological parameter estimation, we adopted a Gaussian likelihood for the data given the model. This likelihood can be maximized with respect to the parameters at a fixed but unknown width. Instead we adopted the following procedure [

Throughout the manuscript we report results as mean ± SD for individual data and mean ± SEM when we discuss group data. In the phenomenological fittings, to determine average parameters from the parameter estimation other than the frequency, we computed the mean and variance for each parameter and participant as the first two moments of the corresponding posterior probability distribution and took the average of the means weighted by their standard deviations (square root of the estimated variance) to generate each point on the population plot. Alternative estimators (e.g., the modes of the posterior distributions, with and without weighting) gave qualitatively similar results.

To investigate generative models, we adopt the following rationale. In each trial, the oculomotor system must generate a motor command to produce the impending saccade. This needs to be calibrated against the actual physical size of the required movement [

If the saccade fails to land on target, the motor command needs to be recalibrated based on preexisting calibrations, and we will hypothesize that those changes take place in an obligatory manner (cf. [

We model the underlying sensorimotor learning using linear time-invariant systems (LTIS). The model parameters (or the learning coefficients) are time independent in each experimental block, although they can vary across experimental conditions or phases [

Because saccades are extremely rapid movements that do not admit reprogramming in mid-flight, it is assumed that all gain changes take place in between saccades. In our models, therefore, the error-based correction terms weight errors that were experienced in previous saccades. As a consequence, in the estimation of the forthcoming event, the post-saccadic stimulus gain is not compared against the adaptation gain measured for that trial but against the previous estimate of the gain. To justify these assumptions, it is usually assumed that the motor system sends an

We will assume that the values of saccade and adaptation gains observed and extracted from the recorded data (i.e.,

To be able to consistently compare results from this manuscript with the phenomenological analyses of the data presented in our earlier report, we will write the generative model in terms of a state variable associated to the

As suggested by Eqs

Because movement gains are computed from experimental observations, models of motor control often include a second equation that maps the estimates of the hypothesized internal variable to real-world observations (see, e.g., [

We conducted our analyses using the full form of

In view of these features of the generative model, a natural classification of the models tested arises as follows: given the parameters _{1},⋯, _{w},

All parameters of the generative models were estimated by fitting the model to the experimental data using MATLAB function nlinfit; 95% confidence intervals for the fitted parameters were computed using MATLAB function nlparci and predicted responses for the hidden variable

All 16 models were fitted to data from each individual participant parameters were extracted for each model, and models were compared using the Akaike information criterion (AIC; [

The adaptation gain of the oculomotor response to a sinusoidal disturbance is best described by a phenomenological function consisting of a decaying exponential added to a lagged but otherwise undistorted sinusoid [_{0}) where the baseline stabilizes at large trial number, a timescale (

We use here the same denominations used in our previous report [

The lag of the periodic response of the error gain derived from the (full version of the) generative model of

In models without next-to-last feedback term

The periodic component of the response to a sinusoidal disturbance in models where the next-to-last feedback is included can be written as:

Eqs

Following a sinusoidal disturbance, the baseline of the error gain will approach an asymptote at large trial number that can be written as a function of parameters of

The timescale

^{−λ} = (

To recap, _{0},

The amplitude of the decay of the baseline also bears dependence on the learning rates as well as on the initial condition. Because of the strong influence of the initial condition on this parameter, we refrain from a comparison of the behavioral fittings to the predictions from the generative model for this case.

Part of the material discussed in this contribution have been presented in the form of posters or slide presentations [

To obtain a general idea of patterns present in the data, we first collapsed the data for each stimulus frequency and adaptation type across participants (group data). We fit these data using a piecewise continuous function given by the addition of a monotonic (exponential) decay of the baseline–spanning both pre-adaptation and adapting trials- and a periodic entraining of the oculomotor response to the sinusoidal stimulus that begins at the onset of the adaptation block. This choice was supported by the fact that we had confirmed using statistical model selection criteria (i.e., AIC and BIC, [

The plots show adaptation gain (colored lines) averaged over individuals in the (

(

(

Some features are readily apparent from these plots. First, the frequency of the ISS is reliably estimated (cf.

The parameters that affect the observed drift in the baseline (i.e., asymptote and timescale,

To assess the generative model, we fit

The plots show adaptation gain (colored lines), averaged over individuals in the (

For all subsequent analyses, we fitted models to individual data. In particular, we compared 16 different models that differed from each other depending on which parameters were fitted (see

The label along the middle y-axis indicates the model for the weight displayed in the horizontal bars. Results from dataset ORIG (

Inspection of

Blue and red colors correspond to horizontal Two-way and Global adaptation, respectively. (

Blue and red colors correspond to horizontal Two-way and Global adaptation, respectively. (

Again, several features are readily apparent from these plots. The learning rates (

A feature observed in all cases is that in models that learn only from the last experienced error, the (single) learning rate (

The values of the parameters fitted with the best four models are shown in

The iteration of state-equations that learn from the last feedback already qualitatively predicts both components of the phenomenological response. In general, the complete response can be interpreted as a convolution of the stimulus with a

Above, we fitted the extended version of _{0}, timescale

When the learning algorithm includes several error-based terms,

We start with

A first significant observation about this expression is that in order to observe a drift in the baseline of the adaptation gain (i.e., in order to have an asymptote _{0} ≠ 0), a finite value of the drift parameter _{0}.

Experimentally, we observed drifts towards higher hypometria in all averages and in most of the individual data. Note that formatting the data in terms of

_{0} obtained from

y-axes show the values obtained with the phenomenological parameter estimation (_{0} in

A second parameter characteristic of the baseline drift is given by the timescale.

Asymptote and timescale are parameters traditionally investigated and reported in adaptation to fixed-step disturbances. Sinusoidal adaptation paradigms provide two additional parameters associated to the periodic component of the adaptation gain observed in these protocols.

The last comparison is provided by the lag of the periodic component.

We used a modified version of the traditional two-step saccade adaptation paradigm ([

The present study explored whether the phenomenology described by

In mathematical terms, the functional form in

We analyzed 16 models that differed in the specific parameters that were fitted and then used Akaike’s information criteria to attempt model selection. Since we were primarily modeling intrinsic error-based sensorimotor learning, the learning rate

To further discuss the effect of the generative parameters, we split the 16 models into four groups:

Models that neither included terms depending on the second learning rate

Models without terms depending on

Models including terms depending on

Models with both

We recall that in models where

The fits of the phenomenological model (

The persistent drift of the baseline towards higher hypometria is a distinctive feature in our data that cannot be accounted for on the basis of motor adaptation [

Saccadic eye movements slightly undershoot their target on average [

On the other hand, from the point of view of the internal model of the movement that the brain may implement [

On a neurophysiological level, the small systematic bias that gives rise to the drift of the baseline may originate from the dynamics of the responses in the neuronal substrates involved with saccade adaptation ([

The models that best explained the data featured a double error sampling, learning not only from the feedback experienced after the last saccade but also from the movement that occurred in a trial before that. Hence, the best models used a feedback reaching further back in time through the

To understand that, we return to

We can re-write _{+} and _{−}:
_{+} is the mean of the two samplings of the stimulus, i.e.,

Note that the representation in terms of these alternative internal variables would significantly alter the underlying structure of the noise-free learning model. But if we insist on keeping a close connection to the parameters extracted using the double-error-sampling algorithm, we would expect that the learning rate for learner _{+} would be the addition of the rates for the two errors, _{−} it would be _{+} will be much closer to unity than _{−} = (_{+} will learn and forget much slower than _{−}.

Using this double error sampling, the oculomotor system could track the rate of change of the stimulus from one saccade to the next, besides just its last change in size and it would approximate the learning efficiency of the double-error-sampling algorithm. The new internal learning variables (_{+} and _{−}) would learn from smoothed-out versions of the disturbance resulting from the average sum and difference of the two sampled inputs. Whether this constitutes an advantage over learning exclusively from the last feedback depends on the nature of stimulus. If the disturbance is constant or fully random there would be very little advantage in performing the double error sampling. In the former case, the inter-sampling variation is zero leaving nothing to learn. In the latter, the inter-sampling variation would be another random magnitude and there would be little advantage in learning from the variation in the feedback. However, if the mean of the disturbance varies in a systematic way—as it does during sinusoidal adaptation, and presumably in natural scenarios—learning from its rate of variation would be advantageous and could well justify a large learning rate. In the representation of the double-error-sampling model, unlearning actively the next-to-last sampled feedback error (i.e., with a large and negative _{+} and _{−} can be considered statistics in counterphase. To approximate the double-error-sampling learner, the system may hold in memory both samples, compute mean sum and differences between the samples and implement two learners based on those statistics rather than from bare values of errors or stimulus occurrences. To achieve that, the oculomotor system would need to keep memory and weight prediction errors from a former time scale besides the last feedback [

An important point to notice is that, even if there is double error sampling, it does not need to be strictly the next-to-last error. It would be enough that the brain keeps a correlation of errors over two different trials (cf. [

We further explored whether the values of the generative parameters exhibited dependence on the experimental condition, specifically with the type of adaptation and the frequency of the disturbance. The parameters of our models remained time-invariant across pre-adaptation and adaptation blocks. However, we did not rule out that these parameters may change with adaptation type and stimulus frequency. In fact, LTIS models with parameters not strictly time-invariant have been invoked to model (meta-learning in) savings in adaptation to visuomotor rotations [

We limit our discussion to the best four generative models selected in the

However, the dependence of the learning rate(s) on the frequency described above changed rather dramatically when double error sampling was included (cf.

In contradistinction, the retention rate

In summary, introducing a second error term increased the magnitude of both learning rates (

Multiple distinct learning processes contribute to sensorimotor adaptation [

We believe that our paradigm taps only the first, implicit component. Yet, we suggest that our analyses provide evidence for two separable subcomponents, although both would be intrinsic in nature [

We believe that the presence of a systematically varying disturbance enables a further decomposition of the implicit component of adaptation, perhaps into a primary one, that attempts to mitigate the end-point discrepancy regardless of self-correlations in the disturbance, and a second one that attempts to extract (and use) such correlations. It remains an open question how these putative subprocesses may map on distinct or overlapping anatomical structures, such as cerebellar cortices, deep cerebellar nuclei and extracerebellar structures [

A recent study suggested that learning in dynamic environments may be adequately modeled with an algorithm popular in industrial process control, the proportional-integral-derivative (PID) controller [

Having adequate generative models that describe eye movements have been stressed before [

Model name is shown at the top. The corresponding datasets can be identified by the stimulus frequencies tested: ORIG: 3, 4 and 6cpb. FREQ: 1, 3, 6, 12 and 24cpb.

(PDF)

Repeated-measures ANOVA (2 X 3 on data from ORIG; 2 X 5 on data from FREQ) with factors type of adaptation and stimulus frequency was run on each of the four best models. Model name is shown at the side of the table and parameter names are on the top. The dataset is indicated in the cell at the upper left corner next the the parameter names. Highlights indicate the cases where the corresponding factor shows significant effects.

(PDF)

(PDF)

We thank Thérèse Collins and members of the Rolfs lab for insightful discussions and help with data collection.