Ethics statement

The study was approved by the ethics committee of the Faculty of Social Sciences of Radboud University Nijmegen, the Netherlands (Approval number: ECSW-2022-082R1).

Participants

Twenty-two naive participants (6 men, 16 women), ranging in age from 18 to 33 years, without history of motion sickness, gave written informed consent to participate in the study. They were compensated with course credits or €15.00. The experiment lasted approximately 90 minutes per participant.

Setup

Participants were seated comfortably on a custom-built linear motion platform (dubbed the vestibular sled), where the motion axis is aligned with their interaural axis (see Fig 1A). Their head was restrained. They were further secured by a five-point seat belt in combination with vacuum cushions molded around the torso, as well as straps around their legs and feet. The track of the sled is 0.9 m long. The sled is powered by a linear motor (TB15N; Tecnotion, Almelo, The Netherlands) and controlled by a servo drive (Kollmorgen S700; Danaher, Washington, DC). Emergency stop buttons on either side of the sled could be pressed to stop the experiment at any time. A steering wheel (Logitech G27 Racing Wheel, Lausanne, Switzerland) was mounted at a comfortable handling distance. The participants placed both hands on on the steering wheel at the 9 and 3 o’clock positions. The steering wheel, which had a rotational range of to and a resolution of , controlled the speed of the sled at 60 Hz. Participants viewed a 55-inch OLED screen (55EA8809-ZC; LG, Seoul, South Korea) with a resolution of 1920x1080 pixels and a refresh rate of 60 Hz, positioned centrally in front of the sled at a distance of 1.6 m. The OLED screen depicted a cartoon traffic scenario, i.e., a car on a road (see Fig 1B). The road was generated as a trapezoid polygon with a central width of 0.15 m. The centrally presented car, visually representing the participant, had a width of 0.084 m. The experiment was controlled by custom code written in Python 3 [27] using the PsychoPy module (v.3.6.9 [28]).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Schematic of setup, task and perturbation signal.

A: Linear motion platform with the mounted steering wheel. B: Schematic representation of the steering task. Participants were asked to counteract a perturbation signal that either continuously displaced the platform and the visual car relative to the center of the road (visuoinertial condition) or solely displaced the car while the motion platform remained stationary (visual condition). C: Frequency spectrum of the sum-of-sines perturbation signal. The amplitude of all frequencies was set at 0.1 m/s and the phase of each sinusoid was randomly chosen between 0 and 2. D: Position and velocity time traces of a resulting perturbation signal.

https://doi.org/10.1371/journal.pcbi.1012659.g001

Behavioral task

Participants were seated on the motion platform in front of a screen, which depicted a cartoon traffic scenario, i.e., a car on a road (see Fig 1B). The car was perturbed laterally relative to the road and the task of the participant was to continuously adjust its position to keep it within the center of the road. Participants were tested in two conditions:

Analysis

Data were processed offline in Python 3 [27], using the NumPy [29] and SciPy [30] modules. We analyzed a total of 440 trials (22 participants x 2 conditions x 10 trials). Of each trial, the ramp-up and ramp-down phase were discarded such that only the central 120 s were analyzed.

As overall performance measures, we computed the root-mean-square of the car position relative to the center of the road of each trial, as well as for the first 60 s and last 60 s to examine within-trial changes.

To obtain insight in the frequency-dependent steering dynamics, we examined the compensatory velocity signal in terms of gain and relative phase as a function of frequency. First, we fast Fourier transformed the controlled vehicle velocity and perturbation signals separately to obtain the amplitude and its associated phase at each frequency. Gain is then defined as the ratio of the controlled over the perturbed amplitude. Relative phase is defined as the phase difference between participant generated velocity and the perturbation velocity at each frequency, which we obtained by finding the angle between the two signals. Ideal performance is defined by unity gain and a () phase difference, i.e., perfectly counteracting the perturbation signal resulting in no motion of the car on the screen.

Statistics

We performed a three-way repeated measures ANOVAs on the root-mean-squared-error (RMSE) of the car position to check for significant differences between the two conditions, changes in performance across trials and within-trial improvements. To investigate the frequency-dependent control strategies, we performed two two-way repeated measures ANOVAs on the gain and relative phase. Factors were the frequency (the 12 frequencies from the perturbation signal) and the two conditions. Paired t-tests, Bonferroni corrected, were used for post-hoc tests if significant main effects were observed.

Modelling

We represent the steering control task as a perturbation-rejection task. More specifically, the goal of the task is to steer the wheel to keep the vehicle centered on the road despite perturbations that are either perceived visuoinertially (visual, vestibular and somatosensory feedback) or visually (visual feedback only). While the task did not employ a first-person view of the road—making it perhaps less ecologically valid than actual steering—correction for perturbations relies on visual and inertial cues, which are inherently noisy and subject to time delays. The velocity of the vehicle can be regulated based on the position of the steering wheel, an interaction that depends on the participant’s neuromuscular system.

Following work from Cole and Nash [18], we formulated a optimal feedback control model to simulate this task (Fig 2). In brief (see S1 Appendix for details), the model consists of three components: the plant, the state estimator and the controller. The plant describes the true dynamics of the system, in our case, both the vehicle dynamics (Fig 2A) and the participant’s neuromuscular system interacting with the steering wheel (Fig 2D). The vehicle part of plant is perturbed by a band-limited white noise signal with discrete frequencies. The transfer function of the motion sled is near unity for frequencies up to 2.5 Hz. A noisy time-delayed output of the plant is observed by the sensory systems (Fig 2B)—the vestibular and visual systems. The state estimator optimally estimates the state of the plant based upon the received sensory measurements and predictions based on an internal model of the plant (Fig 2C). The optimal motor command for the neuromuscular control of the steering wheel is computed via a control policy based on inputs from the state estimator and keeping the visual error at zero with minimal effort being the goal (Fig 2C).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Schematic model visualization.

In our model, the plant consists of the motion platform, screen (A) and wheel control system (D). Information is fed back through delayed and noisy visual, vestibular and proprioceptive sensory systems (B). Through the feedback, states of the system are then estimated with a Kalman filter (C). Finally, an LQR controller (C) generates an optimal control policy (i.e., motor command) to steer the wheel and motion platform.

https://doi.org/10.1371/journal.pcbi.1012659.g002

More formally, we model the task using a linear quadratic Gaussian system (LQG [31]) operating at 60 Hz, as such, one time step () is approximately 17 ms. Accordingly, the true state of the system evolves over time according to a linear system subject to Gaussian noise,

(1)

where indicates the state of the system (both the vehicle and neuromuscular states) at time t, A is the state transition matrix (the passive system dynamics), B is the state transition matrix of active dynamics, is the motor command of the participant, is a multivariate standard normal representing various noise components (e.g motor noise), V is a matrix which transforms these noise components into the states and is the perturbation applied by the experimenter. Note that our motion platform is velocity controlled, and as a result, we modeled our system as a velocity-controlled first order system. As a result of this choice, we had to explicitly incorporate the acceleration perturbations into the model, using a separate matrix . This approach enables the model to detect changes not only in velocity and position, but also in acceleration.

For simplicity, we model the vehicle as a point mass with position and velocity . Similar to the behavioural task we make the velocity of the vehicle a linear function (scaled by λ) of the wheel position and perturb it via . Thus, in discrete time our vehicle dynamics are,

(2)

(3)

Because the wheel turns over a small range () we also modelled the wheel dynamics as linear. This allows us to write the dynamics of the wheel as,

(4)

(5)

where the force () represents the angular force applied to the wheel through the arms. Thus, the force exerted on the wheel () is equivalent to the acceleration component of the control signal. To emulate the properties of the neuromuscular system [32], a second-order low-pass filter is applied to the control prior to the conversion to force. This yields discrete time dynamics (note we split the second order filter to two first-order filters),

(6)

(7)

where the time constant of the neuromuscular low-pass filter was taken to be 40 ms, similar to previous biological control models [32,33]. Finally, the otolith system perceives the physical vehicle acceleration so we also keep track of this in the state description. For simplicity, we compute this via finite difference (see S1 Appendix for the exact form),

(8)

Thus, the full state consists of the position, velocity and acceleration of the vehicle and wheel, alongside the neuromuscular states that drive the wheel, . Rather than observing the true states, we assume the participant only has access to noisy and delayed observations of the states via the different sensory systems (e.g visual, vestibular and proprioceptive). The time delay of the visual and vestibular feedback are assumed to be approximately 120 and 10 ms, respectively, in the default model [34–36].

To this end, we assume the visual system provides measurements of the vehicle position and velocity delayed by 7 time steps ( 117 ms),

(9)

(10)

So, the participant effectively visually observes a noisy version of position and velocity from a previous time point (). Secondly, we assume that the vestibular and somatosensory system provides acceleration information during the visuoinertial condition (in the visual condition this observation is not available) but delayed by 1 step ( ms) that differs from ,

(11)

Finally, since the proprioceptive system of the arms do not provide direct information about the lateral translation of the car, we assumed that the position, velocity and force related to steering wheel were available with no delay ( = 0 ms),

(12)

(13)

(14)

This can be written more concisely in matrix notation as,

(15)

where is a vector of observations, is a multivariate standard normal vector representing observational noise, H is the observation matrix which maps from the true states to observations, and G is a matrix which maps the noise to the observation. G involves a scaling constant to scale noise levels without changing their relative magnitudes (see S1 Appendix for details).

From the standpoint of the participant, their goal is to determine the optimal motor command at each point in time to accomplish the task. To do this, it is necessary to specify a particular cost they wish to minimize which is indicative of their goals for the task. A crucial component of the task is keeping the car on the center of the road while minimizing the amount of effort to do so. In the model this is realized via a quadratic cost function which penalizes deviations of the vehicle’s position from zero (with a weight of 1) and penalizes the magnitude of the control command (with a weight r),

(16)

This can be rewritten in terms of Eq (1) as,

(17)

where Q is a matrix reflecting the state costs which, in our task, is the positional deviation of the car from the road. R is a matrix reflecting the cost of control (see S1 Appendix for their construction).

An ideal observer uses an internal model of the system (effectively, a model of Eqs 1 and 15), alongside the defined cost (Eq 17), to compute an optimal policy. However, the true state () in Eq (17) is not available to the participants, so they have to use an estimate of this state (). Because Eqs (1) and (15) constitute a linear system and Eq (17) is quadratic, the optimal policy [31] consist of using a Kalman filter to estimate the state. We utilized the filtering version [37] appose to the predictive version sometimes used in other control models [33]. First, the observer creates a prediction about the future state which is corrupted by noise, this prediction is then updated with the received measurement,

(18)

(19)

and the control command is found by applying a linear feedback control rule to this estimate,

(20)

where K is the optimal Kalman gain and L is the optimal feedback gain (see the S1 Appendix for details on how L and K are computed) and is noise added to the prediction of the state estimator [37].

The overall behaviour of the model is dictated by a second-order transfer function for the neuromuscular dynamics, alongside the scaling of the various noise levels in the system (all modelled as Gaussian white noise), the relative control costs, and the sensory delays (as detailed in Fig 3 and derived in the S1 Appendix).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Model predictions.

The upper row displays the model’s gain predictions, while the lower row shows phase predictions. Each column (from left to right) illustrates the model’s predictions as different parameters are varied: (sensory noise gain, color-coded), (control cost gain), and (visual delay, in milliseconds). Blue-green lines represent the visuoinertial condition, and red-yellow lines represent the visual condition. Left column: Control cost gain is fixed at the average of the best-fit values across conditions (see Table A in S1 appendix), and visual delay is set to 7 timesteps ( ≈ 117 ms). Sensory noise gain is varied, as indicated by the color bar. Middle column: Sensory noise gain is held constant at its best-fit value, and visual delay is at 7 timesteps. Control cost gain is varied. Right column: Sensory noise gain is fixed at its best-fit value, and control cost gain is set to the average of the best-fit values. Visual delay is varied.

https://doi.org/10.1371/journal.pcbi.1012659.g003

Model fitting

Naturally the predictions of the model are dependent on the specific choice of parameter values used. We fixed all the parameters (see S1 Table) except for the overall sensory magnitude , and the control cost r. We allowed the control cost to differ between the visual and visuoinertial conditions. We parameterized control cost as , where is the fitted control exponent. Thus we have three free parameter .

We optimized these parameters to minimize the RMSE between the observed steering wheel angle and the predicted steering wheel angle, similar to previous approaches in system identification [26].

Specifically, at each time point we compute the predicted steering wheel angle (given previous angles) from the model, and then RMSE between this and the observed wheel angle. In order to do so we follow previous inverse control approaches [38] and create a joint system of and and utilize this to compute the expected prediction error. For the minimization itself we used Bayesian Active Direct Search (BADS) [39,40]. We ran the fitting procedure with three separate initializations and selected the parameter set with the lowest error.

Model simulations

To understand the qualitative behaviour of the model, we performed simulations of the optimal feedback model and analyzed the central 120 s of the simulated time series in the same way as the experimental data. We simulated the expected response of the system to the perturbations for a single subject (11 trials per condition). For each trial we computed the model spectra (gain and phase) from the simulated time series and averaged them. To determine the sensitivity of these spectra to the specific parameter values, we simulated the model for a variety of parameter values for both the visuoinertial and visual condition. Specifically, we varied the magnitude of the sensory noise (), control cost exponent () and the delay of the visual system (), as explicated in the caption of Fig 3.

Fig 3 illustrates the results of the simulations as bode diagrams for both the visuoinertial (blue-green lines) and visual condition (red-yellow lines). The general observation across both conditions is that of a low-pass system, i.e., with increasing perturbation frequency the gain reduces and the phase is increasingly less compensatory to the perturbation signal. There are three primary aspects of the model that could contribute to this low-pass property. The first is the LQR controller which operates as a low-pass filter when control costs are not negligible [41], the second is the estimator, which can also operate as a low-pass filter, and the third is related to the inherent low-pass neuromuscular dynamics. The effect of the estimator and controller can be seen directly from our model simulations in Fig 3. For the controller, lowering control costs reduces the low-pass nature of the system, moving it closer towards a pass-through system where the gain is 1, and the phase is closer to . To examine the role of the estimator, we manipulated both the sensory noise and visual delays. Lowering sensory noise increases the compensatory gain and lowers the phase shift, but its effect is limited by the low-pass nature of the plant and the controller. By contrast, when visual delays are reduced, the impact on compensatory gain is minimal, primarily influencing the phase shift in the visual condition rather than in the visuoinertial condition. In the visuoinertial condition, the delays introduced by the visual system are offset by the vestibular system. Thus, vestibular input helps decrease the phase lag in the response, allowing the estimate to more closely match the actual state and resulting in a response that is less out of phase. Importantly, none of these manipulations can address how much the dynamics of the neuromuscular system contribute to the low-pass nature of our model. To address this, we simulated a simpler version of the model without the low-pass neuromuscular dynamics (see S1 Appendix). This simpler model showed very little difference in compensatory gain, but showed a relative phase closer to , indicating that some of the observed phase difference comes from this property. Combined, this indicates that the primary factor influencing the frequency response characteristics of the model is the controller, while the neuromuscular system and the estimator play a lesser role.

Task performance

Fig 4A shows the typical performance of a single participant in a selected part (30 s) of a visuoinertial and a visual trial, as well as the underlying perturbation signal. If the participant had not corrected for this perturbation, the car on the screen would move relative to the road along this profile, with displacements of more than 0.3 m relative to the center. In the visuoinertial condition (Fig 4A, middle panel), during which the participant sensed the motion of the car visually and vestibularly, there was a clear compensation for this perturbation. Compensation seems to be reduced in the visual condition (Fig 4A, bottom panel), where the participant does not receive vestibular cues, resulting in higher RMSE compared to the visuoinertial condition.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Task performance, expressed as RMSE, for the visuoinertial and visual condition.

A: Part (30 s) of an example trial. Top: Uncorrected perturbation signal. Middle: Vehicle position in the visuoinertial condition. Bottom: Vehicle position in the visual condition. B: Average RMSE () separately for each participant and averaged across participants, as well as the average value predicted by the model (see Methods). Visuoinertial condition in blue, visual condition in orange.

https://doi.org/10.1371/journal.pcbi.1012659.g004

To quantify the overall performance in these trials, we calculated the zero-mean RMSE of the car relative the center of the road across the full 120 s period. While the RMSE of the perturbation was 0.167 m, this reduced to 0.085 m and 0.133 m in the visuoinertial and visual condition, respectively, due to active steering control by the participant.

These RMSE findings are exemplary for all participants. Fig 4B shows the RMSE for the two conditions across trials, separately for each participant, and averaged across participants. We ran a three-way repeated measures ANOVA with condition, trial and epoch (first and last 60 s of each trial) to examine their effects on the RMSE. First, we found a main effect of condition (F(1,21) = 232.38, p< 0.001, = 0.92), where the RMSE was on average 1.5 times smaller in the visuoinertial condition (RMSE = 0.08 ± 0.01 m (mean ± SD)) compared to the visual condition (RMSE = 0.13 ± 0.02 m). Interactions of this factor with trial or epoch were not significant (condition x trial: F(9,189) = 1.63, p =  0.11, = 0.07; condition x epoch: F(1,21) = 3.47, p = 0.08, = 0.14). Also the three-way interaction effect was not significant (F(9,189) = 0.82, p = 0.59, = 0.04).

The ANOVA model revealed no significant effect of trial number either (F(9,189) = 0.93, p = 0.51, = 0.042). Fig 5A provides an illustration of this result, suggesting there is no across trial learning in our paradigm. There was neither an interaction effect of trial with epoch (trials x epoch: F(9,189) = 0.94, p = 0.49, = 0.04).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. RMSE changes within and across trials.

Error bars indicate the standard error of measurement. A: Across trial RMSE changes. No significant effect of trial was found, indicating no across trial performance changes. B: Within trial RMSE changes. The main 120 s portion of the trial is split into two 60 s epochs. Post-Hoc analysis revealed no significant differences between the first and second epoch in either condition, indicating no within trial performance changes.

https://doi.org/10.1371/journal.pcbi.1012659.g005

Finally, the ANOVA yielded a significant main effect of epoch (F(1,21) = 5.56, p = 0.03, = 0.21). This result is illustrated in Fig 5B. For the visuoinertial condition, both epochs had equal mean and standard deviation (, = 0.083 m, , = 0.011). The visual condition showed small differences (=0.126 m, =0.129 m,=0.018, =0.021). Given that the interaction effect between epoch and condition was close to statistical significance, we performed post-hoc paired t-tests for the visuoinertial and visual condition separately. This revealed neither a significant effect in the visuoinertial condition (t(21) = 0.45, p = 0.66, Cohen’s d = 0.03), nor significant effect of epoch in the visual condition (t(21) = -1.63, p = 0.12, Cohen’s d = -0.09).

These results suggest that participants have reached close to asymptotic performance levels after the practice trial, and that there were no significant performance improvements or deteriorations within a trial.

Dynamics of compensatory control

To analyze the dynamics of the compensation that resulted in the RMSE differences between conditions, Fig 6A presents the gain and phase of the compensatory steering signal at the perturbed frequencies of the data (dots), averaged across participant, i.e., a Bode plot. Ideal performance would be represented by a gain of one and phase of radians (). The gain of the data (Fig 6A, top panel) clearly demonstrates lowpass behavior, mimicking the simulation observations seen in Fig 3. In the visuoinertial condition, the gain is about one for the lowest perturbation frequency, and reduces with increasing perturbation frequency. In the visual condition, the gain is above one for the lowest perturbation frequency, but rapidly drops below the level of compensation of the visuoinertial condition with increasing frequency. A two-way repeated measures ANOVA revealed significant main effects of both condition (F(1,21) = 32.69, p < 0.01, = 0.609) and frequency (F(11,231) = 211.1, p < 0.01, = 0.910), as well as a significant interaction between them (F(11,231) = 18.39, p < 0.01, = 0.467). Post-hoc analysis revealed that the two conditions yielded different gains at all frequencies (t(21) > 2.41, p < 0.03, Cohen’s d > 0.53) except at 0.26 Hz (t(21) = 1.46, p = 0.16, Cohen’s d = -0.32). Note that, except for 0.1 Hz, all gains are lower in the visual condition.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Measured gain (top) and phase (bottom) alongside predictions.

A: Data alongside Model fits. Visuoinertial condition in blue; visual condition in orange. Data is plotted as as dots (mean ± SD), lines represent model predictions. B: Time series plots illustrating the steering response and model fit for a single trial from one participant.

https://doi.org/10.1371/journal.pcbi.1012659.g006

For both conditions, the phase of the lowest frequency compensatory signal is about radians, i.e., exactly in antiphase with, and thus compensatory for, the perturbation signal (Fig 6A, bottom panel). As the frequency increases, the phase increases as well, but much less for the visuoinertial than the visual condition. This is most likely due to the shorter sensory delay in the visuoinertial condition, i.e., the vestibular delay. A two-way repeated measures ANOVA showed a significant main effect of condition (F(1,21) = 1263.49, p< 0.01, = 0.984) and frequency (F(11,231) = 914.73, p< 0.01, = 0.978) as well as a significant interaction (F(11,231) = 428.4, p< 0.01, = 0.953). Post hoc paired t-tests revealed that the two conditions started to differ in phase for frequencies larger than 0.1 Hz (t(21) > 2.9, p< 0.01, Cohen’s d < -2.42). At 0.1 Hz, no significant difference was found between conditions (t(21) = 0.75, p = 0.23, Cohen’s d = -1.93).

To test how well our model can account for these data, we fitted its free parameters to the steering response in the time domain (see Methods). Fig 6B (top and bottom panels) illustrate that the fitted steering responses closely matched the measured responses for a representative participant in the visuo-inertial and visual-only condition, respectively. As shown in Fig 4B, the model fits closely match the observed RMSE in the visuoinertial condition but show a slight discrepancy in the visual-only condition. In term of fitted parameters, we found estimates of 0.5 cm visual position noise and 10 cm/s² of vestibular acceleration noise across participants, consistent with values reported in previous studies [42–44]. We also found that the fitted control cost exponent was negative and smaller (e.g control cost was higher) for the visual condition compared to visuoinertial condition, suggesting participants were willing to apply more control in the visuointertial condition.

Interestingly, when examining the frequency-resolved dynamics of the predicted compensation, the quality of the fitted pattern is slightly different. In the visual-only condition, the model provides a reasonably good fit for both the gain and phase data. However, when the same model is applied to the visuoinertial condition, it continues to fit the gain data well but tends to overestimate the phase compared to participants’ actual performance.