Assessing the contribution of active somatosensory stimulation to self-acceleration perception in dynamic driving simulators

Mattia Bruschetta; Ksander N. de Winkel; Enrico Mion; Paolo Pretto; Alessandro Beghi; Heinrich H. Bülthoff

doi:10.1371/journal.pone.0259015

Abstract

In dynamic driving simulators, the experience of operating a vehicle is reproduced by combining visual stimuli generated by graphical rendering with inertial stimuli generated by platform motion. Due to inherent limitations of the platform workspace, inertial stimulation is subject to shortcomings in the form of missing cues, false cues, and/or scaling errors, which negatively affect simulation fidelity. In the present study, we aim at quantifying the relative contribution of an active somatosensory stimulation to the perceived intensity of self-motion, relative to other sensory systems. Participants judged the intensity of longitudinal and lateral driving maneuvers in a dynamic driving simulator in passive driving conditions, with and without additional active somatosensory stimulation, as provided by an Active Seat (AS) and Active Belts (AB) integrated system (ASB). The results show that ASB enhances the perceived intensity of sustained decelerations, and increases the precision of acceleration perception overall. Our findings are consistent with models of perception, and indicate that active somatosensory stimulation can indeed be used to improve simulation fidelity.

Citation: Bruschetta M, de Winkel KN, Mion E, Pretto P, Beghi A, Bülthoff HH (2021) Assessing the contribution of active somatosensory stimulation to self-acceleration perception in dynamic driving simulators. PLoS ONE 16(11): e0259015. https://doi.org/10.1371/journal.pone.0259015

Editor: Jane Elizabeth Aspell, Anglia Ruskin University, UNITED KINGDOM

Received: October 1, 2020; Accepted: October 11, 2021; Published: November 18, 2021

Copyright: © 2021 Bruschetta et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the paper and its Supporting information files.

Funding: The author(s) received no specific funding for this work.

Competing interests: The authors have declared that no competing interests exist.

1 Introduction

Dynamic driving simulators are nowadays a widely used tool within automotive companies, both in R&D and in production departments. Originally designed for race applications, their role is becoming crucial also for autonomous driving and Advanced Driver Assistance Systems (ADAS), where human-machine interactions (HMI) can be analyzed in a way that ensures safety and repeatability.

In dynamic driving simulators, the driving experience is reproduced by combining visual information, generated by graphical rendering, with inertial stimuli, generated by translations and rotations of the motion platform. The strategies that have the aim of translating vehicle motion into feasible simulator motions are called Motion Cueing Algorithms (MCA). Since limitations of the simulator’s motion envelope preclude a 1:1 reproduction of the desired accelerations in most cases, MCA typically reproduce low-frequency dynamics by down-scaling, tilt-coordination, and a continuous calling back to the initial position (washout); thereby reducing the actual motion of the simulator, while trying not to compromise the subjective realism of the motion cues [1]. Although efforts are made to improve upon this washout strategy (see e.g., work on Model Predictive Control [2–6]), the inherent limitations of a motion simulator’s envelope cannot entirely prevent missing cues, false cues, and/or scaling errors. These limitations may in turn negatively affect the perceived realism and immersion in the simulation, and may even trigger motion sickness [1, 7–9].

In real-life, a vehicle’s inertial motion results in stimulation of different sensory systems of the driver throughout the body; notably the vestibular and somatosensory senses. The vestibular system comprises the semicircular canals and otoliths of the inner ear, which are directly sensitive to angular and linear accelerations, respectively. The somatosensory system consists of various sensory neurons, distributed throughout the body, that are indirectly responsive to physical motion. These neurons respond to force cues that are generated when, for instance, the body is pressed into the seat or safety belts during accelerations, due to its inertia. Consequently, it is possible to separately recreate somatosensory stimulation by actively providing specific pressure cues. This idea has been explored in flight simulators, in the form of the g-seat [10–13] and in driving simulators, in the form of the Active Seat (AS) and Active Belts (AB) [14], and has the potential to reproduce part of the low-frequency sustained accelerations that generally lack in compact motion simulators with a limited workspace [15].

There is a considerable neuroscientific literature that addresses how the human brain constructs a unified conscious experience out of multisensory cues. Several studies have shown that observers create a robust percept about the environment by weighing multiple sources of sensory information based on their reliability [16, 17]. Even though statistically optimal integration does not always apply [18], there is evidence that weighing is also involved in the perception of self-motion: for instance, visual and inertial cues to heading [19–23] and orientation [24–26] appear to be integrated optimally, provided that discrepancies between the signals remain within a certain range [27–29]. Similar findings have been reported for the perceived magnitude of angular [30] and linear motion [31–33]. Providing a detailed characterization of how multisensory stimuli interact in self-acceleration perception during the driving task is also a topic of current interest [34]. However, whereas visual-inertial interactions have thus received considerable attention in both real and simulated driving [35, 36], the relative contribution of somatosensory cues to the overall motion percept is, to the best of our knowledge, poorly understood. In particular, the skin receptor dynamics, that are of interest in the application at hand, are difficult to measure and quantify [34]. In the present study, we investigated how the perception of linear and angular motions is affected by active somatosensory stimulation. Our experimental setup consisted of a hexapod motion simulator with a head-mounted display (HMD) and an Active Seat (AS) and Active Belts (AB) integrated system (ASB). The ASB system consists of a seat equipped with a pneumatic system that can be used to recreate pressure cues acting on the body during accelerations by emulating the contact between seat, belts and body that result from inertial accelerations. We compared the perceived motion resulting from visual and platform movements (visual-vestibular motion condition) to perception resulting from visual and platform movements with additional somatosensory cues (somatosensory augmented motion condition).

We determined the effects of the active somatosensory stimulation in two different driving maneuvers: a straight line and a curve, that are characterized by longitudinal and lateral accelerations, respectively. Generally speaking, longitudinal accelerations are generated by forward and backward surges of the platform, whereas lateral accelerations are provided through left and right sways. Because longitudinal and lateral motions stimulate different groups of somatosensory neurons, we evaluated the ASB contribution in two experiments, respectively. In both experiments, participants were passively transported in an autonomous (virtual) vehicle. They were only asked to perform a Magnitude Estimation (ME) task, to provide a numerical estimate of the acceleration intensity, and a two-Interval Forced Choice (2IFC) task, to compare the acceleration intensities.

2 Materials and methods

2.1 Ethics statement

The experiments were carried out in accordance with the declaration of Helsinki. All participants provided written informed consent prior to participation. The experimental protocol was approved by the ethical committee of the medical faculty of the Eberhard-Karls University in Tübingen, Germany, reference number 800/2018BO1.

2.2 Participants

In total, ten different volunteers took part in the experiments (3 females, mean age 29.2 years, standard deviation 4.1). In accordance with motion platform safety guidelines, participation was limited to people measuring at most 1.95m long and weighing under 100 kg. All participants were familiar with the simulator and reported not to be susceptible to motion sickness and claustrophobia. Participants were assigned to one of two experimental groups: the first group experienced only cornering maneuvers (6 participants, 1 female, mean age 28.8 years, sd 4.2), whereas the second group experienced only car braking maneuvers (4 participants, 1 female, mean age 29.8 years, sd 4.5). Details of the experiments are given below, in the section Task and Stimuli. One participant participated in both experiments (i.e., both the longitudinal and lateral cases). One other participant (P05, longitudinal condition) was excluded from the formal analysis. During the debriefing, this participant reported to have been confused by the presence of active somatosensory stimulation and stated to have used different cognitive strategies to solve the tasks in the two conditions. This resulted in worse performance with active somatosensory stimulation.

2.3 Apparatus

Physical acceleration cues were generated using an eMotion-1500 hexapod-based motion system (Bosch Rexroth, 2015) at the Max Planck Institute for Biological Cybernetics. This platform is specifically designed for human-occupied vehicle simulations (aircraft, cars, trucks, buses, military vehicles, railway vehicles, etc.), entertainment, or research and development. The dynamic motion simulator was controlled using Simulink and MATLAB software (The MathWorks Inc., Natick, Massachusetts, United States). As the experiment is based on a passive driving experience, the platform was moved using motion references computed off-line. The proposed maneuvers are composed by a single event with a limited duration (below 10 seconds), hence the specific methodology used for the Motion Cueing does not play a critical role in the experiment design. However, to guarantee the possibility to recreate the experiment, the commercial software VI-DriveSim that includes VI-MotionCueing (VI-grade GmbH [37]) was used to generate the motion profiles. Tuning of the motion cueing was done following the guidelines recommended in the manual and refined during the study’s pilot phase to reach a subjectively optimal experience and was equal for all participants. A list of the parameters that have been modified w.r.t. the default tuning is reported in Tables 4 and 5 in S1 Appendix.

An Active Seat (AS) and 6-points Active Belts (AB) were used in combination (ASB) to provide active somatosensory stimulation (VI-grade GmbH). The seat was rigidly mounted on the motion platform and is shown in Fig 1. The AS part consists of eight air bladders placed in specific locations (see Fig 2, left panel). The bladders can be inflated in proportion to vehicle accelerations. Bladders 2 and 5 provide pressure to the lower sides of the trunk of the body (roughly at the location of the external oblique muscles); bladders 1 and 6 to the sides of the upper legs during lateral accelerations (vastus lateralis muscles); bladders 3 and 4 to the lower back during positive longitudinal acceleration (latissimus dorsi muscles) and, finally, bladders 7 and 8 provide pressure to the buttocks and rear of the upper legs (gluteus muscles) for accelerations in the vertical direction. The bladders are designed to have a distributed contact area and are placed such that they can be used to mimic pressure cues generated in a real car. Proportional valves are used to allow for a progressive and continuous variation of pressure, which can be varied in the range 0–2 bar.

Download:

Fig 1. Active seat.

The AS mounted on the motion platform with 6-point AB and Head-Mounted Display (HMD).

https://doi.org/10.1371/journal.pone.0259015.g001

Download:

Fig 2. ASB system: The left picture shows the position of the bladders, whereas the picture on the right shows the belts tensioned by a muscle.

https://doi.org/10.1371/journal.pone.0259015.g002

The AB are commanded by a pneumatic muscle (see Fig 2, right panel), and are intended to provide the driver with pressure cues to the torso (roughly the pectoralis major and trapezius muscles) during longitudinal acceleration of the vehicle.

The ASB system is controlled by VI-DriveSim 18.1 software. This software allows the tuning of the parameters directly from a user interface. As for the motion cueing, the editable parameters have been tuned in the study’s pilot phase to provide a subjectively optimal experience and were equal for all participants. The values used are given in tables in S1 Appendix.

Visual stimuli were presented using a Vive head-mounted display (HMD) (HTC, New Taipei City, Taiwan). The screens each have 1080 × 1200 pixels and a refresh rate of 90fps. The HMD was used to present participants with visual motion stimuli that depicted a view from the inside of a car driving over a straight road or a leftward curve, in the longitudinal and lateral case, respectively, as shown in Fig 3. To reduce the possibility that a participant focused on reference points to perform the task there were no elements in the scene such as trees or buildings. The scenes were generated using the Unity game engine (Unity Technologies, San Francisco, United States). Head motions were tracked, corrected for platform motion, and reproduced in the virtual environment using the Vive ‘lighthouse’ tracking boxes with OpenVR-InputEmulator [38].

Download:

Fig 3. View from HMD.

Screenshot of the view participants had from the inside of the virtual car.

https://doi.org/10.1371/journal.pone.0259015.g003

Sounds generated by the motion platform were masked using headphones (Plantronics, California, United States) that actively cancelled outside noise and played back simulated engine sounds. In addition, participants wore earplugs with a 33dB signal-to-noise ratio (Honeywell Safety Products, Roissy, France).

Verbal responses (ME task) were noted by the experimenter; binary responses (2IFC task) were collected using an Xbox controller (Microsoft, Redmond, WA, United States)

2.4 Task and stimuli

The study was designed to determine the contribution of somatosensory cues, as delivered by the ASB system, to the perception of accelerations in a dynamic driving simulator. Because the contribution of the AS and AB subsystems and the stimulated areas of the body differ for lateral and longitudinal accelerations, we created two experiments in which cornering (lateral motion) or braking maneuvers (longitudinal motion) were presented, respectively. Participants were randomly assigned to either experiment.

Similar to [39], who used two separate tasks to determine the relative contributions of accelerations and jerk to the perceived intensity of motions, we used (1) a Magnitude Estimation (ME) task to assess accuracy of perception, and (2) a 2-Interval Forced Choice (2IFC) task to assess precision of perception. Both tasks were used in each part of the experiment.

In the ME task, participants were presented with driving maneuvers with different acceleration intensities and were asked to provide verbal estimates of their intensity; in the 2IFC task, participants were presented with two successive motions on each experimental trial, and asked to judge which of the two stimuli was more intense.

We then evaluated the effects of active somatosensory stimulation by comparing perceived motion in visual-vestibular motion conditions, where combinations of visual stimulation and platform movements (visual-vestibular motion) were presented, to perceived motion in augmented conditions, with active somatosensory cues (somatosensory augmented motion) in addition to the visual and platform movements.

2.4.1 Tasks.

2.4.1.1 Magnitude estimation. In the ME tasks, participants were required to provide verbal estimates of different acceleration intensities of the proposed driving maneuvers. For both cornering and braking maneuvers, each of the seven stimulus intensities was presented three times. As a reference for their responses, participants received three consecutive training trials at the beginning of the experiments. These were chosen to have exactly the medium intensity within the two ranges: the peak acceleration intensity of the training stimuli for the braking maneuvers was 1.75 m/s², and the peak acceleration intensity of the training stimuli for the cornering maneuvers was 5 m/s². Participants were instructed to attribute the arbitrary number ‘100’ to the intensity of the training trials. In total, participants performed 24 ME trials for the visual-vestibular motion condition, and 24 for the somatosensory augmented motion condition. The order of stimuli was randomized.

2.4.1.2 2-interval forced choice. Although the ME method is thought to yield insights in the relationship between stimulus intensity and perceived intensity, it does not provide an unambiguous scale because idiosyncratic internal representations on the unit of acceleration may differ, and it does not provide accurate information on the precision of perception because numerical estimates are subject to cognitive biases such as a rounding off to the nearest decade [40]. Therefore, participants were asked to also perform a 2IFC task.

In a 2IFC task, participants experience two successive motions for each experimental trial, designated ‘reference’ and ‘comparison’. The participants’ task is to judge which of the two stimuli was more intense. The reference stimulus has a peak intensity that is kept constant among trials, whereas the peak intensity of the comparison stimulus is chosen from the range of values around the reference. By obtaining repeated judgments for several comparison intensities, it is possible to determine how the proportion of responses (or their probability) where the comparison is judged more intense than the reference depends on the intensity of the comparison stimulus, which in turn can be used to estimate perceptual precision. Each reference-comparison pair was presented 10 times, resulting in eighty trials for the M and S conditions, making for a total of 160 2IFC trials per participant. This number was the same for the cornering experiment and for the braking experiment. The order of trials and the order of reference and comparison motion within trials were randomized.

By combining the ME and 2IFC tasks, we aimed to obtain a fair characterization of perceived motion.

2.4.2 Stimuli.

The acceleration ranges are based on [41] and were tailored to achieve a realistic driving experience during pilot experiments. The specific values were selected to cover a range of accelerations that is commonly experienced during normal driving, and that provides a vestibular-somatosensory stimulation above the absolute perception threshold when reproduced on the simulator. In terms of subsystem capacity, the average ratio of AS÷AB at the peak intensity of the lateral maneuvers was 6%÷60%; this ratio was 48%÷1% during longitudinal maneuvers.

2.4.2.1 Cornering maneuvers. Trajectories were generated by imposing the same duration of the stimulus (5s) and the same curve angle (90 degrees), while varying the acceleration intensity, thereby manipulating the curvature radius and the travel velocity.

For the ME task, accelerations were varied between 4.1 and 5.9 m/s² in seven equidistant steps (Fig 4(a)). The corresponding platform accelerations, which include tilt coordination contribution, obtained from the VI-DriveSim software, are shown in Fig 4(c); exact quantification of the pressure cues generated by the ASB system is shown in Fig 4(e) for the 5.0m/s² maneuver.

Download:

Fig 4. Desired motion and stimulation profiles.

Top row: vehicle acceleration profiles for the lateral (a) and longitudinal (b) case; middle row: platform accelerations including tilt coordination for lateral (c) and longitudinal (d) case; bottom row: pressure cues generated by the ASB, in the 5.0m/s² lateral (e) and 1.75m/s² longitudinal (f) condition. Legends report the values of the acceleration peaks of the vehicle during the maneuver for the top and middle row. In the bottom row, the legend specifies the ASB actuator, as in Fig 2.

https://doi.org/10.1371/journal.pone.0259015.g004

For the 2IFC task, the reference intensity was 5.2 m/s², and the comparison intensities varied between 4.5 − 5.9 m/s² in nine equidistant steps, but omitting 5.2 m/s² as a comparison value, resulting in eight comparison stimuli. The profiles for vehicle accelerations, platform accelerations and displacements were identical to those of the ME-task except for their scaling, and are therefore not shown separately.

It may be noted that the cueing software introduced minor false cues (e.g., the negative accelerations around 6–8 s at the end of the profiles in Fig 4c). However, as these deflections are small and because participants did not report noticing them, nor any dizziness/sickness, we believe they did not have any notable effects on our findings.

2.4.2.2 Braking maneuvers. The trajectories featured a sustained deceleration (3s).

For the ME task, the range of peak accelerations was varied between 1 and 2.5 m/s², in seven equidistant steps (Fig 4(b)). The corresponding platform accelerations, which include tilt coordination contribution, are shown in Fig 4(d) and 4(f) shows an exact quantification of pressure exerted to the body by the ASB system for the maneuver with a 1.75m/s² acceleration peak.

For the 2IFC task reference intensity was 2 m/s², and the comparison magnitudes ranged between 1.5 − 2.5 m/s² in nine equidistant steps, but omitting 2 m/s² as a comparison value, resulting in eight comparison stimuli. As for the cornering experiment, the profiles for vehicle accelerations, platform accelerations and displacements were identical to those of the ME-task except for their scaling, and are therefore not shown separately.

The lateral and longitudinal maneuvers are designed to produce similar acceleration peaks on the platform while exploiting all the available working space.

2.5 Procedure

After participants had understood the instructions and safety requirements and had given their written informed consent, the experimenter guided them to the simulator platform and seated them in the AS. Then, the AB were put in place, as were the ear plugs, the headphones, and the HMD. The experimenter asserted that the belts were properly tightened and that the thighs of the participants rested on the seat. When participants were properly installed, they were given the controller, which they used to provide responses and to advance trials at their own pace.

Before the experiment started, participants were given three training trials to familiarize with the overall procedure and apparatus. At the end of the task, participants were debriefed and asked to fill out a questionnaire with questions regarding their subjective experience of the ASB contribution.

Altogether, the experiments took approximately three hours to complete per participant, including instructions, training sessions and pauses. Each participant completed the experiments in two sessions, both combining ME and 2IFC, one with the whole setup active and the other with ASB switched off. The sessions lasted one and a half hour each and were performed on two different days.

Assignment of participants to the cornering or braking maneuver; the order of the visual-vestibular motion and somatosensory augmented motion; and the order of ME and 2IFC tasks were randomized.

2.6 Data analysis

The data analyses were designed to determine how the addition of active somatosensory stimulation affected perception, in comparison to a visual-vestibular motion condition with visual and platform motion only. Because the ME and 2IFC tasks yield different types of response data (continuous and dichotomous, respectively), we performed separate analyses for the two measures. Data acquired are collected in SI 1 and 2.

For ME data, a linear mixed-effect model was fitted to the data as a psychometric function. In this model, we describe responses on the ME task y_ME in terms of fixed effects that were common to all participants and random effects that account for additional individual variations in these effects. The model has terms for the categorical dummy variables take on values of 0 or 1 to indicate the absence or presence of some categorical effect. variables ‘maneuver’ man and ‘condition’ con; a term for the continuous variable ‘peak intensity’ int, and terms for the interactions between these variables, which allow accounting for non-additive effects of different variables. In addition, we allow for variation between participants par by including random effects with the same structure as the fixed effects model.

In Wilkinson notation [42], the model fitted to the data of the ME task can then be defined as: (1)

In the 2IFC case, responses are binary, which means a linear regression model cannot be used. Instead, a probit mixed-effect model has been fitted to the data. Probit is a generalized regression model where the response variable can have two possible outcomes only. In Wilkinson notation [42], the model was defined as: (2) Here, Φ is the Cumulative Distribution Function (CDF) of the standard normal distribution. Note that this model includes effects for ‘peak intensity’ and all its interactions with ‘maneuver’ and ‘condition’, but lacks an intercept (‘-1’) and main effects for ‘maneuver’ and ‘condition’ (i.e, the possibility for the intercept to vary between conditions). This is so because, in a 2IFC task, the intercept is fixed to the value of the reference stimulus, which did not vary between conditions. Consequently, for the 2IFC task, we only account for variations in the slope of the psychometric function as a result of experimental manipulations.

Models were fitted to the data using Maximum-Likelihood Estimation. We assessed the (adjusted) coefficient of determination (, respectively) as an indicator of the quality of our fitted models. These statistics indicate the proportion of variance in the data explained by the model. P-values were computed for each regressor term in order to determine whether it provided a statistically significant effect: we accept a 5% chance of rejecting a true null hypothesis, that is, we accept there is an effect of a particular variable that is actually due to chance.

Because we are interested only in the general effect of the presence of active somatosensory stimulation, we do not explicitly consider the random effects in our analyses.

3 Results

3.1 Magnitude estimation

Verbal estimates collected in the ME tasks and the fitted psychometric functions are shown in Fig 5(a) and 5(b), for the lateral and longitudinal case, respectively. Overall, the model provided a fair approximation of the data, as evidenced by R² = 0.72 and .

Download:

Fig 5. Data acquired in the ME task for the cornering (left panel, (a)) and brake maneuver (right panel, (b)) cases.

Light blue and orange dots represent individual responses collected in the M and S conditions, respectively. A small amount of random noise was added to the x-coordinate of individual data points to improve their visibility. Lines are the four fitted psychometric functions obtained by model (1), which describe the four cases in which participants have been split, i.e. the braking and the cornering maneuver, both experienced with or without ASB active. The results suggest that the bladders of the ASB system do not increase perceived intensity in curves, but the belts do increase perceived intensity in braking maneuvers. No unit is provided for the y-axis as magnitude estimates are expressed on a subjective scale.

https://doi.org/10.1371/journal.pone.0259015.g005

The estimated coefficients for the psychometric functions and the associated p-values are presented in Table 1.

Download:

Table 1. Fixed-effects coefficients and associated p-values for the ME-task psychometric function.

https://doi.org/10.1371/journal.pone.0259015.t001

As the p-value for the int coefficient is smaller than 0.05, we can consider the effect significant. This means that responses on the ME task depend on the peak intensity of the stimulus, thereby confirming that participants were able to perform the task.

Significant p-values in man × int, con × int, man × con × int indicate that there are interaction effects: as can be seen in Fig 5(a) and 5(b), the slope of the psychometric function differs between experimental conditions. The significance of the effect of man combines with significant effect for man × int, which together reflect the fact that people were asked to associate the arbitrary value of 100 to the middle value of the range of intensities presented, whereas the range of intensities differed between maneuvers.

Notably, the slope of the psychometric function is steeper in the S condition than in the M condition, but only for longitudinal motion stimuli.

3.2 Two-interval forced choice

Binary responses and fitted psychometric functions task are shown in Fig 6(a) and 6(b). Overall, the model provided a fair explanation of the data, as evidenced by R² = 0.53 and .

Download:

Fig 6. Data acquired in the 2IFC task for the lateral (left panel, (a)) and longitudinal (right panel (b)) cases.

Light blue and orange dots represent the proportion of trials where the comparison stimulus was judged larger than the reference by an individual participant for each comparison stimulus intensity, for responses collected in the M and S conditions, respectively. A small amount of random noise was added to the x-coordinate of individual data points to improve their visibility. Lines are the four fitted psychometric functions obtained through model (2). The results suggest that active somatosensory stimulation increases discrimination sensitivity.

https://doi.org/10.1371/journal.pone.0259015.g006

Table 2 shows the estimated fixed-effects coefficients and the associated p-values.

Download:

Table 2. Fixed-effects coefficients and associated p-values for the 2IFC-task psychometric function.

https://doi.org/10.1371/journal.pone.0259015.t002

As the p-value related to int is lower than 0.05, we can conclude that the probability of response that the comparison stimulus was larger than the reference (C > R) increases with stimulus peak intensity, which means that participants were able to perform the task.

The interaction effects between maneuver and peak intensity (man × int) and between condition and peak intensity (con × int) were also significant. These findings indicate that the slope of the psychometric function differs depending on whether people performed the lateral or longitudinal task, and depending on whether there was active somatosensory stimulation, respectively.

Notably, the slope of the psychometric function was steeper when there was active somatosensory stimulation, and this effect was consistent between maneuvers (i.e. the effect for man × con × int was not significant).

3.3 Debriefing phase

After completing the main tasks, participants were asked to answer the following questions regarding their subjective experiences:

“Do you think the presence of the ASB system improves the realism of the simulation?” (0 no, 1 yes);
“Did you feel immersed; as if you were in a real car?” (0 no, 1 without ASB, 2 with ASB);
“Did the ASB system help you in distinguishing the different stimulus intensities?” (0 no, 1 yes);
“What do you think is the source of information that helped you most in performing the task?” (0 visual motion, 1 platform motion, 2 the ASB system, 3 other, namely …);
“Did you experience the ASB-system as a natural addition to the motion simulation?” (0 no, 1 yes).

The responses are shown in Table 3, and can be summarized as follows: (1) all participants considered the addition of the ASB to improve the realism of the simulation. (2) Nine participants (9/10) indicated that they felt more immersed in the simulation when ASB was present. (3) Eight (8/10) participants thought that ASB helped them to distinguish motion intensities. Participants considered platform motion and ASB as the main sources of information exploited to perform the tasks, whereas only three participants declared that they used also visual information. All participants except P05 (whose data were excluded from the quantitative analyses, see 2.2) considered the ASB-system a natural addition to the motion simulation. Given the high rating of realism provided by participants, it is unlikely that any interference or incompatibility has arisen between platform motion and somatosensory cues. During the debriefing, some participants suggested that there was room for improvement regarding the naturalness of the stimuli.

Download:

Table 3. Participants’ answers on the questionnaire.

The values for ‘case’ refer to whether participants were in the experiment with longitudinal (0) or lateral (1) motions.

https://doi.org/10.1371/journal.pone.0259015.t003

4 Discussion

The present work was designed to gather a first quantification of the contribution of active somatosensory stimulation to dynamic driving simulations. We assessed this by asking participants to rate the intensity of their motion percepts and to perform a discrimination task for visual-vestibular motion driving simulation scenarios (platform and visual motion) and for an augmented condition that featured additional active somatosensory stimulation. Because of the contribution of the AS and AB subsystems and the parts of the body that are stimulated differ for lateral and longitudinal motions, we performed separate experiments for both cases.

In our analysis, the data from the longitudinal (four participants) and lateral (six participants) maneuvers was pooled. Hence, assessments of ASB effects common to the two maneuvers were based on observations in ten participants [51–55].

Intensity ratings were obtained using a Magnitude Estimation (ME) paradigm. In this paradigm, participants attribute scores to stimuli of different intensities, with reference to a standard that was presented at the outset of the experiment. This paradigm allows expressing the perceived intensity as a function of stimulus intensity. Overall, a linear function provided a good approximation of the data. In early research on the relation between stimuli and perception, it was argued that the psychometric function that relates perceived to physical intensity should follow a logarithmic law [43] or a power law [44]. However, for a narrow range of stimuli, such as in the present experiments, the curvature of the function can be negligible, and a straight line can provide a reasonable approximation of the data [41]. In the longitudinal case, the slope coefficient of the fitted function was increased by a factor 1.3 when the ASB system was active (i.e., from to ). In this condition, somatosensory stimulation was primarily provided by the belts. This finding suggests that the belts provide somatosensory cues that can effectively augment the inertial stimulation for simulation of sustained decelerations in dynamic driving simulators.

In addition to the ME task, participants performed a 2-Interval Forced Choice task (2IFC). In this task, participants discriminate stimuli on the basis of their intensity. Although the binary data obtained from 2IFC tasks do not provide information about the size of the perceived intensity nor on the shape of the distribution of underlying percepts (as in the ME task), the binary responses on 2IFC tasks are not affected by cognitive strategies such as rounding off to the nearest decade and are not affected by the scaling of the internal representation of acceleration intensity. Consequently, the slope of the psychometric function fitted to the data in this task may be considered a more objective measurement of perceptual precision. The model fits for this data indicated that the slope was steeper in conditions with active somatosensory stimulation. For the lateral and longitudinal maneuvers, the slopes were approximately a factor 2 and a factor 1.5 steeper when there was active somatosensory stimulation (lateral: vs. ; longitudinal: vs. ). These findings thus indicate that the precision of the perceived intensity was higher when somatosensory information was present than when it was not. The direction of this effect, namely a reduction of perceptual uncertainty when more senses are stimulated, is consistent with predictions of psychophysical models of multisensory perception that state that information provided by multiple sensory systems is integrated in a statistically optimal fashion [16, 17, 23, 27, 28, 30, 45, 46]. When the precision of each contributing sensory system is known, these models can make exact predictions on the weights attributed to information provided by the different sensory systems and on the perceptual precision for combined multisensory conditions. To make such predictions for the present paradigm would require additional ME and 2IFC data for ASB stimulation in isolation from the other senses; without visual or inertial stimulation. Although it would be theoretically possible to collect this data, such an experimental condition (or isolated inertial stimulation, for that matter) would not bear any resemblance to a driving simulation, and is therefore beyond the scope of the present study. A consequence of this is that the present data do not allow us to evaluate the exact predictions of these perception models. This also means that on the basis of the present statistical evaluations of the data, the possibility cannot be excluded that the additional somatosensory cues were processed independently. There is however evidence to argue against this possibility: first, such processing would not be consistent with the subjective experiences reported by our participants during debriefing: if the cues had been processed independently, this would produce an unnatural situation where a somatosensory sensation co-exists with an inertial sensation, whereas our participants all indicated that adding the somatosensory cues actually improved simulation realism; and second, a number of studies show that cues of a somatosensory nature are merged with visual and vestibular cues for other aspects of self-motion and orientation perception. For instance, podokinetic cues (stepping) are combined with vestibular cues to generate vection during active turning [47, 48]. Moreover, it has been shown that the somatosensory cues are combined with visual and vestibular cues for spatial orientation [49] and postural sway [50].

Finally, it is important to note two potential limitations of the present study. First, the findings were obtained in the context of a dynamic driving simulator, and the ASB were tuned specifically to add to the physical cues provided by the motion base. Consequently, the findings may not generalize to fixed-base simulators. These simulators do not provide the physical motion cues that can be generated in dynamic driving simulators and can thus be expected to result in different sensory weightings. Second, the data were obtained from a limited sample of participants. However, such small numbers of participants are actually not unusual in studies using psychophysical procedures to quantify psychological effects. This may be justified by the notion that the low-level (neural) mechanisms responsible for perception do not typically differ between people, thus allowing generalizability of the results to the population [51–55]. To deal with the limited sample size in the best way, we chose to pool the data from the braking (four participants) and cornering (six participants) maneuvers for the statistical analysis, and to use multilevel models. These models account for individual differences by including random effects. The fixed effects shown as solid lines in Figs 5 and 6 may thus be interpreted as an average performance, corrected for individual differences. We believe that this approach provides a reasonable characterization of general effects.

5 Conclusion

We conclude that the addition of somatosensory information to dynamic driving simulations increases the precision of intensity percepts and that it can augment inertial longitudinal acceleration cues. The direction of the findings on perceptual precision is consistent with predictions made by psychophysical theories on multisensory perception, which state that this information is integrated in a statistically optimal way. Combined with subjective reports obtained via questionnaire, the findings suggest that ASB can add to simulation fidelity in dynamic driving simulators.

Supporting information

S1 Data. Data file containing participants’ answers obtained during ME tasks.

https://doi.org/10.1371/journal.pone.0259015.s001

(MAT)

S2 Data. Data file containing participants’ answers obtained during 2IFC tasks.

https://doi.org/10.1371/journal.pone.0259015.s002

(MAT)

S1 Questionnaire. Data file containing participants’ answers obtained during debriefing phase.

https://doi.org/10.1371/journal.pone.0259015.s003

(XLSX)

S1 Appendix.

https://doi.org/10.1371/journal.pone.0259015.s004

(PDF)

References

1. Berthoz A, Bles W, Bülthoff HH, Gracio BJC, Feenstra P, Filliard N, et al. Motion Scaling for High-Performance Driving Simulators. IEEE Transactions on Human-Machine Systems. 2013;43(3):265–276.
- View Article
- Google Scholar
2. Bruschetta M, Maran F, Beghi A. A Nonlinear, MPC-Based Motion Cueing Algorithm for a High-Performance, Nine-DOF Dynamic Simulator Platform. IEEE Transactions on Control Systems Technology. 2017;25(2):686–694.
- View Article
- Google Scholar
3. Bruschetta M, Maran F, Beghi A. A fast implementation of MPC-based motion cueing algorithms for mid-size road vehicle motion simulators. Vehicle System Dynamics. 2017;55(6):802–826.
- View Article
- Google Scholar
4. Venrooij J, Cleij D, Katliar M, Pretto P, Bülthoff HH, Steffen D, et al. Comparison between filter-and optimization-based motion cueing in the Daimler Driving Simulator. In: Driving Simulation Conference; 2016.
5. Katliar M, Fischer J, Frison G, Diehl M, Teufel H, Bülthoff HH. Nonlinear Model Predictive Control of a Cable-Robot-Based Motion Simulator. Ifac-papersonline. 2017;50(1):9833–9839.
- View Article
- Google Scholar
6. Katliar M, Drop FM, Teufell H, Diehl M, Bülthoff HH. Real-Time Nonlinear Model Predictive Control of a Motion Simulator Based on a 8-DOF Serial Robot. In: 2018 European Control Conference (ECC). IEEE; 2018. p. 1529–1535.
7. Cleij D, Venrooij J, Pretto P, Pool DM, Mulder M, Bülthoff HH. Continuous Subjective Rating of Perceived Motion Incongruence During Driving Simulation. IEEE Transactions on Human-Machine Systems. 2018;48(1):17–29.
- View Article
- Google Scholar
8. van Leeuwen TD, Cleij D, Pool DM, Mulder M, Bülthoff HH. Time-varying perceived motion mismatch due to motion scaling in curve driving simulation. Transportation Research Part F: Traffic Psychology and Behaviour. 2018.
- View Article
- Google Scholar
9. van der Steen FAM. Self-motion perception. PhD dissertation, University of Technology, Delft, The Netherlands, 1998. 2018.
10. Kron GJ. Advanced simulation in undergraduate pilot training: G-seat development. Singer Co Binghamton NY Simulation Products div; 1975.
11. Showalter TW, Parris BL. The Effects of Motion and g-Seat Cues on Pilot Simulator Performance of Three Piloting Tasks. NASA Technical Paper. 1980;1601.
12. Ashworth BR, McKissick BT, Parrish RV. Effects of motion base and g-seat cueing of simulator pilot performance. Hampton, VA, United States: NASA; 1984. NASA-TP-2247, L-15559, NAS 1.60:2247.
13. Ricard GL, Parrish RV. Pilot Differences and Motion Cuing Effects on Simulated Helicopter Hover. Human Factors. 1984;26(3):249–256.
- View Article
- Google Scholar
14. Vi-grade GmbH; https://www.vi-grade.com/.
15. Bruschetta M, Chen Y, Cunico D, Mion E, Beghi A. A nonlinear MPC based motion cueing strategy for a high performance driving simulator with active seat. In: IEEE 15th International Workshop on Advanced Motion Control (AMC); 2018. p. 23–28.
16. Ernst MO, Banks MS. Humans integrate visual and haptic information in a statistically optimal fashion. Nature. 2002; p. 415–429.
- View Article
- Google Scholar
17. Ernst MO, Bülthoff HH. Merging the senses into a robust percept. Trends in cognitive sciences. 2004;8(4):162–169. pmid:15050512
- View Article
- PubMed/NCBI
- Google Scholar
18. Kenney DM, and O’Malley S, and Song HM, and Townsend B, and von Mohrenschildt M, and Shedden JM. Velocity influences the relative contributions of visual and vestibular cues to self-acceleration. Experimental Brain Research 1423–1432, 2020 Springer-Verlag GmbH Germany pmid:32367145
- View Article
- PubMed/NCBI
- Google Scholar
19. Butler JS, Smith ST, Campos JL, Bülthoff HH. Bayesian integration of visual and vestibular signals for heading. Journal of Vision. 2010;10(11):23. pmid:20884518
- View Article
- PubMed/NCBI
- Google Scholar
20. Butler JS, Campos JL, Bülthoff HH, Smith ST. The Role of Stereo Vision in Visual–Vestibular Integration. Seeing and Perceiving. 2011;24(5):453–470. pmid:21888763
- View Article
- PubMed/NCBI
- Google Scholar
21. Angelaki DE, Gu Y, DeAngelis GC. Visual and vestibular cue integration for heading perception in extrastriate visual cortex. The Journal of Physiology. 2011;589(4):825–833. pmid:20679353
- View Article
- PubMed/NCBI
- Google Scholar
22. Fetsch CR, Turner AH, DeAngelis GC, Angelaki DE. Dynamic reweighting of visual and vestibular cues during self-motion perception. The Journal of Neuroscience. 2009;29(49):15601–15612. pmid:20007484
- View Article
- PubMed/NCBI
- Google Scholar
23. de Winkel KN, Katliar M, Bülthoff HH. Forced Fusion in Multisensory Heading Estimation. Plos one. 2015;10(5). pmid:25938235
- View Article
- PubMed/NCBI
- Google Scholar
24. Dyde RT, Jenkin MR, Harris LR. The subjective visual vertical and the perceptual upright. Experimental Brain Research. 2006;173(4):612–622. pmid:16550392
- View Article
- PubMed/NCBI
- Google Scholar
25. Vingerhoets RAA, De Vrijer M, Van Gisbergen JAM, Medendorp WP. Fusion of visual and vestibular tilt cues in the perception of visual vertical. Journal of neurophysiology. 2009;101(3):1321–1333. pmid:19118112
- View Article
- PubMed/NCBI
- Google Scholar
26. Nooij S, Nesti A, Bülthoff HH, Pretto P. Perception of rotation, path, and heading in circular trajectories. Experimental Brain Research. 2016;234(8):2323–2337. pmid:27056085
- View Article
- PubMed/NCBI
- Google Scholar
27. de Winkel KN, Katliar M, Bülthoff HH. Causal Inference in Multisensory Heading Estimation. Plos One. 2017;12(1). pmid:28060957
- View Article
- PubMed/NCBI
- Google Scholar
28. de Winkel KN, Katliar M, Diers D, Bülthoff HH. Causal Inference in the Perception of Verticality. Scientific reports. 2018;8(1):5483. pmid:29615728
- View Article
- PubMed/NCBI
- Google Scholar
29. Acerbi L, Dokka K, Angelaki DE, Ma WJ. Bayesian comparison of explicit and implicit causal inference strategies in multisensory heading perception. PLoS computational biology. 2018;14(7):e1006110. pmid:30052625
- View Article
- PubMed/NCBI
- Google Scholar
30. de Winkel KN, Soyka F, Barnett-Cowan M, Bülthoff HH, Groen EL, Werkhoven P. Integration of visual and inertial cues in the perception of angular self-motion. Experimental Brain Research. 2013;231. pmid:24013788
- View Article
- PubMed/NCBI
- Google Scholar
31. Harris LR, Jenkin M, Zikovitz DC. Visual and non-visual cues in the perception of linear self motion. Experimental Brain Research. 2000;135(1):12–21. pmid:11104123
- View Article
- PubMed/NCBI
- Google Scholar
32. Sun HJ, Campos JL, Chan GSW. Multisensory integration in the estimation of relative path length. Experimental Brain Research. 2004;154(2):246–254. pmid:14685814
- View Article
- PubMed/NCBI
- Google Scholar
33. Campos JL, Butler JS, Bülthoff HH. Multisensory integration in the estimation of walked distances. Experimental Brain Research. 2012;218(4):551–565. pmid:22411581
- View Article
- PubMed/NCBI
- Google Scholar
34. Nash CJ, Cole DJ, Bigler RS. A review of human sensory dynamics for application to models of driver steering and speed control. Biological Cybernetics. 2016;110(2):91–116. pmid:27086133
- View Article
- PubMed/NCBI
- Google Scholar
35. Houck JA, Telban RJ, Cardullo F. Motion Cueing Algorithm Development: Human-Centered Linear and Nonlinear Approaches. NASACR, vol 213747, [Online] Available: http://hdlhandlenet/2060/20050180246. 2005.
36. Markkula G, Romano R, Waldram R, Giles O, Mole C, Wilkie R. Modelling visual-vestibular integration and behavioural adaptation in the driving simulator. arXiv preprint arXiv:181012441. 2018.
37. VI-DriveSim Documentation; 2017.
38. OpenVR-InputEmulator; 2018. https://github.com/matzman666/OpenVR-InputEmulator.
39. de Winkel KN, and Florian S, and Bülthoff HH. The role of acceleration and jerk in perception of above-threshold surge motion. Experimental Brain Research 699–711, 2020 Springer-Verlag GmbH Germany pmid:32060563
- View Article
- PubMed/NCBI
- Google Scholar
40. Petzschner FH, Glasauer S, Stephan KE. A Bayesian perspective on magnitude estimation. Trends in cognitive sciences. 2015;19(5):285–293. pmid:25843543
- View Article
- PubMed/NCBI
- Google Scholar
41. Naseri AR, Grant PR. Human discrimination of translational accelerations. Experimental Brain Research. 2012;218(3):455–464. pmid:22354103
- View Article
- PubMed/NCBI
- Google Scholar
42. Wilkinson GN, Rogers CE. Symbolic description of factorial models for analysis of variance. Applied Statistics. 1973; p. 392–399.
- View Article
- Google Scholar
43. Fechner GT. Elemente der psychophysik. Breitkopf & Härtel.; 1860.
44. Stevens SS. The psychophysics of sensory function. American scientist. 1960;48(2):226–253.
- View Article
- Google Scholar
45. de Winkel KN, Weesie J, Werkhoven PJ, Groen EL. Integration of visual and inertial cues in perceived heading of self-motion. Journal of Vision. 2010;10(12):23–28. pmid:21047733
- View Article
- PubMed/NCBI
- Google Scholar
46. Greenlee MW, Frank SM, Kaliuzhna M, Blanke O, Bremmer F, Churan J, et al. Multisensory integration in self motion perception. Multisensory Research 2016, Brill 29, 6-7, 525–556
- View Article
- Google Scholar
47. Becker W, Nasios G, Raab S, and Jürgens R. Fusion of vestibular and podokinesthetic information during self-turning towards instructed targets. Experimental Brain Research 2002. Experimentelle Hirnforschung. Expérimentation Cérébrale, 144, 458–474. pmid:12037631
- View Article
- PubMed/NCBI
- Google Scholar
48. Becker W, Kliegl K, Kassubek J, and Jürgens R. Podokinetic circular vection: characteristics and interaction with optokinetic circular vection. Experimental Brain Research, 2016, 234 pmid:26965438
- View Article
- PubMed/NCBI
- Google Scholar
49. de Winkel KN, Edel E, Happee R, and Bülthoff HH. Multisensory Interactions in Head and Body Centered Perception of Verticality. Frontiers in Neuroscience, 2020, 14, 599226 pmid:33510611
- View Article
- PubMed/NCBI
- Google Scholar
50. Peterka RJ and Benolken MS Role of somatosensory and vestibular cues in attenuating visually induced human postural sway. Experimental Brain Research, 1995, 105, 1, pages 101–110, Springer. pmid:7589307
- View Article
- PubMed/NCBI
- Google Scholar
51. Campion G. The Synthesis of Three Dimensional Haptic Textures: Geometry, Control, and Psychophysics. Springer Science and Business Media. 2011;8.5.3:133–134.
- View Article
- Google Scholar
52. Bonnen K, Burge J, Yates J, Pillow J, Cormack L. Continuous psychophysics: Target-tracking to measure visual sensitivity. Journal of Vision. 2015;15:14–14. pmid:25795437
- View Article
- PubMed/NCBI
- Google Scholar
53. Kishishita Y, Tanaka Y, Kurita Y. Force perceptual bias caused by muscle activity in unimanual steering. Plos one. 2019;14. pmid:31639140
- View Article
- PubMed/NCBI
- Google Scholar
54. Smith PL, and Little DR. Small is beautiful: In defense of the small-N design. Psychonomic Bulletin and Review. 2018;25(6):2083–2101. pmid:29557067
- View Article
- PubMed/NCBI
- Google Scholar
55. Klinke R, Hartmann R. Hearing—Physiological Bases and Psychophysics. Springer Science and Business Media. 2012; p. 133–134.

[ref1] 1. Berthoz A, Bles W, Bülthoff HH, Gracio BJC, Feenstra P, Filliard N, et al. Motion Scaling for High-Performance Driving Simulators. IEEE Transactions on Human-Machine Systems. 2013;43(3):265–276.
View Article
Google Scholar

[2] View Article

[3] Google Scholar

[ref2] 2. Bruschetta M, Maran F, Beghi A. A Nonlinear, MPC-Based Motion Cueing Algorithm for a High-Performance, Nine-DOF Dynamic Simulator Platform. IEEE Transactions on Control Systems Technology. 2017;25(2):686–694.
View Article
Google Scholar

[5] View Article

[6] Google Scholar

[ref3] 3. Bruschetta M, Maran F, Beghi A. A fast implementation of MPC-based motion cueing algorithms for mid-size road vehicle motion simulators. Vehicle System Dynamics. 2017;55(6):802–826.
View Article
Google Scholar

[8] View Article

[9] Google Scholar

[ref4] 4. Venrooij J, Cleij D, Katliar M, Pretto P, Bülthoff HH, Steffen D, et al. Comparison between filter-and optimization-based motion cueing in the Daimler Driving Simulator. In: Driving Simulation Conference; 2016.

[ref5] 5. Katliar M, Fischer J, Frison G, Diehl M, Teufel H, Bülthoff HH. Nonlinear Model Predictive Control of a Cable-Robot-Based Motion Simulator. Ifac-papersonline. 2017;50(1):9833–9839.
View Article
Google Scholar

[12] View Article

[13] Google Scholar

[ref6] 6. Katliar M, Drop FM, Teufell H, Diehl M, Bülthoff HH. Real-Time Nonlinear Model Predictive Control of a Motion Simulator Based on a 8-DOF Serial Robot. In: 2018 European Control Conference (ECC). IEEE; 2018. p. 1529–1535.

[ref7] 7. Cleij D, Venrooij J, Pretto P, Pool DM, Mulder M, Bülthoff HH. Continuous Subjective Rating of Perceived Motion Incongruence During Driving Simulation. IEEE Transactions on Human-Machine Systems. 2018;48(1):17–29.
View Article
Google Scholar

[16] View Article

[17] Google Scholar

[ref8] 8. van Leeuwen TD, Cleij D, Pool DM, Mulder M, Bülthoff HH. Time-varying perceived motion mismatch due to motion scaling in curve driving simulation. Transportation Research Part F: Traffic Psychology and Behaviour. 2018.
View Article
Google Scholar

[19] View Article

[20] Google Scholar

[ref9] 9. van der Steen FAM. Self-motion perception. PhD dissertation, University of Technology, Delft, The Netherlands, 1998. 2018.

[ref10] 10. Kron GJ. Advanced simulation in undergraduate pilot training: G-seat development. Singer Co Binghamton NY Simulation Products div; 1975.

[ref11] 11. Showalter TW, Parris BL. The Effects of Motion and g-Seat Cues on Pilot Simulator Performance of Three Piloting Tasks. NASA Technical Paper. 1980;1601.

[ref12] 12. Ashworth BR, McKissick BT, Parrish RV. Effects of motion base and g-seat cueing of simulator pilot performance. Hampton, VA, United States: NASA; 1984. NASA-TP-2247, L-15559, NAS 1.60:2247.

[ref13] 13. Ricard GL, Parrish RV. Pilot Differences and Motion Cuing Effects on Simulated Helicopter Hover. Human Factors. 1984;26(3):249–256.
View Article
Google Scholar

[26] View Article

[27] Google Scholar

[ref14] 14. Vi-grade GmbH; https://www.vi-grade.com/.

[ref15] 15. Bruschetta M, Chen Y, Cunico D, Mion E, Beghi A. A nonlinear MPC based motion cueing strategy for a high performance driving simulator with active seat. In: IEEE 15th International Workshop on Advanced Motion Control (AMC); 2018. p. 23–28.

[ref16] 16. Ernst MO, Banks MS. Humans integrate visual and haptic information in a statistically optimal fashion. Nature. 2002; p. 415–429.
View Article
Google Scholar

[31] View Article

[32] Google Scholar

[ref17] 17. Ernst MO, Bülthoff HH. Merging the senses into a robust percept. Trends in cognitive sciences. 2004;8(4):162–169. pmid:15050512
View Article
PubMed/NCBI
Google Scholar

[34] View Article

[35] PubMed/NCBI

[36] Google Scholar

[ref18] 18. Kenney DM, and O’Malley S, and Song HM, and Townsend B, and von Mohrenschildt M, and Shedden JM. Velocity influences the relative contributions of visual and vestibular cues to self-acceleration. Experimental Brain Research 1423–1432, 2020 Springer-Verlag GmbH Germany pmid:32367145
View Article
PubMed/NCBI
Google Scholar

[38] View Article

[39] PubMed/NCBI

[40] Google Scholar

[ref19] 19. Butler JS, Smith ST, Campos JL, Bülthoff HH. Bayesian integration of visual and vestibular signals for heading. Journal of Vision. 2010;10(11):23. pmid:20884518
View Article
PubMed/NCBI
Google Scholar

[42] View Article

[43] PubMed/NCBI

[44] Google Scholar

[ref20] 20. Butler JS, Campos JL, Bülthoff HH, Smith ST. The Role of Stereo Vision in Visual–Vestibular Integration. Seeing and Perceiving. 2011;24(5):453–470. pmid:21888763
View Article
PubMed/NCBI
Google Scholar

[46] View Article

[47] PubMed/NCBI

[48] Google Scholar

[ref21] 21. Angelaki DE, Gu Y, DeAngelis GC. Visual and vestibular cue integration for heading perception in extrastriate visual cortex. The Journal of Physiology. 2011;589(4):825–833. pmid:20679353
View Article
PubMed/NCBI
Google Scholar

[50] View Article

[51] PubMed/NCBI

[52] Google Scholar

[ref22] 22. Fetsch CR, Turner AH, DeAngelis GC, Angelaki DE. Dynamic reweighting of visual and vestibular cues during self-motion perception. The Journal of Neuroscience. 2009;29(49):15601–15612. pmid:20007484
View Article
PubMed/NCBI
Google Scholar

[54] View Article

[55] PubMed/NCBI

[56] Google Scholar

[ref23] 23. de Winkel KN, Katliar M, Bülthoff HH. Forced Fusion in Multisensory Heading Estimation. Plos one. 2015;10(5). pmid:25938235
View Article
PubMed/NCBI
Google Scholar

[58] View Article

[59] PubMed/NCBI

[60] Google Scholar

[ref24] 24. Dyde RT, Jenkin MR, Harris LR. The subjective visual vertical and the perceptual upright. Experimental Brain Research. 2006;173(4):612–622. pmid:16550392
View Article
PubMed/NCBI
Google Scholar

[62] View Article

[63] PubMed/NCBI

[64] Google Scholar

[ref25] 25. Vingerhoets RAA, De Vrijer M, Van Gisbergen JAM, Medendorp WP. Fusion of visual and vestibular tilt cues in the perception of visual vertical. Journal of neurophysiology. 2009;101(3):1321–1333. pmid:19118112
View Article
PubMed/NCBI
Google Scholar

[66] View Article

[67] PubMed/NCBI

[68] Google Scholar

[ref26] 26. Nooij S, Nesti A, Bülthoff HH, Pretto P. Perception of rotation, path, and heading in circular trajectories. Experimental Brain Research. 2016;234(8):2323–2337. pmid:27056085
View Article
PubMed/NCBI
Google Scholar

[70] View Article

[71] PubMed/NCBI

[72] Google Scholar

[ref27] 27. de Winkel KN, Katliar M, Bülthoff HH. Causal Inference in Multisensory Heading Estimation. Plos One. 2017;12(1). pmid:28060957
View Article
PubMed/NCBI
Google Scholar

[74] View Article

[75] PubMed/NCBI

[76] Google Scholar

[ref28] 28. de Winkel KN, Katliar M, Diers D, Bülthoff HH. Causal Inference in the Perception of Verticality. Scientific reports. 2018;8(1):5483. pmid:29615728
View Article
PubMed/NCBI
Google Scholar

[78] View Article

[79] PubMed/NCBI

[80] Google Scholar

[ref29] 29. Acerbi L, Dokka K, Angelaki DE, Ma WJ. Bayesian comparison of explicit and implicit causal inference strategies in multisensory heading perception. PLoS computational biology. 2018;14(7):e1006110. pmid:30052625
View Article
PubMed/NCBI
Google Scholar

[82] View Article

[83] PubMed/NCBI

[84] Google Scholar

[ref30] 30. de Winkel KN, Soyka F, Barnett-Cowan M, Bülthoff HH, Groen EL, Werkhoven P. Integration of visual and inertial cues in the perception of angular self-motion. Experimental Brain Research. 2013;231. pmid:24013788
View Article
PubMed/NCBI
Google Scholar

[86] View Article

[87] PubMed/NCBI

[88] Google Scholar

[ref31] 31. Harris LR, Jenkin M, Zikovitz DC. Visual and non-visual cues in the perception of linear self motion. Experimental Brain Research. 2000;135(1):12–21. pmid:11104123
View Article
PubMed/NCBI
Google Scholar

[90] View Article

[91] PubMed/NCBI

[92] Google Scholar

[ref32] 32. Sun HJ, Campos JL, Chan GSW. Multisensory integration in the estimation of relative path length. Experimental Brain Research. 2004;154(2):246–254. pmid:14685814
View Article
PubMed/NCBI
Google Scholar

[94] View Article

[95] PubMed/NCBI

[96] Google Scholar

[ref33] 33. Campos JL, Butler JS, Bülthoff HH. Multisensory integration in the estimation of walked distances. Experimental Brain Research. 2012;218(4):551–565. pmid:22411581
View Article
PubMed/NCBI
Google Scholar

[98] View Article

[99] PubMed/NCBI

[100] Google Scholar

[ref34] 34. Nash CJ, Cole DJ, Bigler RS. A review of human sensory dynamics for application to models of driver steering and speed control. Biological Cybernetics. 2016;110(2):91–116. pmid:27086133
View Article
PubMed/NCBI
Google Scholar

[102] View Article

[103] PubMed/NCBI

[104] Google Scholar

[ref35] 35. Houck JA, Telban RJ, Cardullo F. Motion Cueing Algorithm Development: Human-Centered Linear and Nonlinear Approaches. NASACR, vol 213747, [Online] Available: http://hdlhandlenet/2060/20050180246. 2005.

[ref36] 36. Markkula G, Romano R, Waldram R, Giles O, Mole C, Wilkie R. Modelling visual-vestibular integration and behavioural adaptation in the driving simulator. arXiv preprint arXiv:181012441. 2018.

[ref37] 37. VI-DriveSim Documentation; 2017.

[ref38] 38. OpenVR-InputEmulator; 2018. https://github.com/matzman666/OpenVR-InputEmulator.

[ref39] 39. de Winkel KN, and Florian S, and Bülthoff HH. The role of acceleration and jerk in perception of above-threshold surge motion. Experimental Brain Research 699–711, 2020 Springer-Verlag GmbH Germany pmid:32060563
View Article
PubMed/NCBI
Google Scholar

[110] View Article

[111] PubMed/NCBI

[112] Google Scholar

[ref40] 40. Petzschner FH, Glasauer S, Stephan KE. A Bayesian perspective on magnitude estimation. Trends in cognitive sciences. 2015;19(5):285–293. pmid:25843543
View Article
PubMed/NCBI
Google Scholar

[114] View Article

[115] PubMed/NCBI

[116] Google Scholar

[ref41] 41. Naseri AR, Grant PR. Human discrimination of translational accelerations. Experimental Brain Research. 2012;218(3):455–464. pmid:22354103
View Article
PubMed/NCBI
Google Scholar

[118] View Article

[119] PubMed/NCBI

[120] Google Scholar

[ref42] 42. Wilkinson GN, Rogers CE. Symbolic description of factorial models for analysis of variance. Applied Statistics. 1973; p. 392–399.
View Article
Google Scholar

[122] View Article

[123] Google Scholar

[ref43] 43. Fechner GT. Elemente der psychophysik. Breitkopf & Härtel.; 1860.

[ref44] 44. Stevens SS. The psychophysics of sensory function. American scientist. 1960;48(2):226–253.
View Article
Google Scholar

[126] View Article

[127] Google Scholar

[ref45] 45. de Winkel KN, Weesie J, Werkhoven PJ, Groen EL. Integration of visual and inertial cues in perceived heading of self-motion. Journal of Vision. 2010;10(12):23–28. pmid:21047733
View Article
PubMed/NCBI
Google Scholar

[129] View Article

[130] PubMed/NCBI

[131] Google Scholar

[ref46] 46. Greenlee MW, Frank SM, Kaliuzhna M, Blanke O, Bremmer F, Churan J, et al. Multisensory integration in self motion perception. Multisensory Research 2016, Brill 29, 6-7, 525–556
View Article
Google Scholar

[133] View Article

[134] Google Scholar

[ref47] 47. Becker W, Nasios G, Raab S, and Jürgens R. Fusion of vestibular and podokinesthetic information during self-turning towards instructed targets. Experimental Brain Research 2002. Experimentelle Hirnforschung. Expérimentation Cérébrale, 144, 458–474. pmid:12037631
View Article
PubMed/NCBI
Google Scholar

[136] View Article

[137] PubMed/NCBI

[138] Google Scholar

[ref48] 48. Becker W, Kliegl K, Kassubek J, and Jürgens R. Podokinetic circular vection: characteristics and interaction with optokinetic circular vection. Experimental Brain Research, 2016, 234 pmid:26965438
View Article
PubMed/NCBI
Google Scholar

[140] View Article

[141] PubMed/NCBI

[142] Google Scholar

[ref49] 49. de Winkel KN, Edel E, Happee R, and Bülthoff HH. Multisensory Interactions in Head and Body Centered Perception of Verticality. Frontiers in Neuroscience, 2020, 14, 599226 pmid:33510611
View Article
PubMed/NCBI
Google Scholar

[144] View Article

[145] PubMed/NCBI

[146] Google Scholar

[ref50] 50. Peterka RJ and Benolken MS Role of somatosensory and vestibular cues in attenuating visually induced human postural sway. Experimental Brain Research, 1995, 105, 1, pages 101–110, Springer. pmid:7589307
View Article
PubMed/NCBI
Google Scholar

[148] View Article

[149] PubMed/NCBI

[150] Google Scholar

[ref51] 51. Campion G. The Synthesis of Three Dimensional Haptic Textures: Geometry, Control, and Psychophysics. Springer Science and Business Media. 2011;8.5.3:133–134.
View Article
Google Scholar

[152] View Article

[153] Google Scholar

[ref52] 52. Bonnen K, Burge J, Yates J, Pillow J, Cormack L. Continuous psychophysics: Target-tracking to measure visual sensitivity. Journal of Vision. 2015;15:14–14. pmid:25795437
View Article
PubMed/NCBI
Google Scholar

[155] View Article

[156] PubMed/NCBI

[157] Google Scholar

[ref53] 53. Kishishita Y, Tanaka Y, Kurita Y. Force perceptual bias caused by muscle activity in unimanual steering. Plos one. 2019;14. pmid:31639140
View Article
PubMed/NCBI
Google Scholar

[159] View Article

[160] PubMed/NCBI

[161] Google Scholar

[ref54] 54. Smith PL, and Little DR. Small is beautiful: In defense of the small-N design. Psychonomic Bulletin and Review. 2018;25(6):2083–2101. pmid:29557067
View Article
PubMed/NCBI
Google Scholar

[163] View Article

[164] PubMed/NCBI

[165] Google Scholar

[ref55] 55. Klinke R, Hartmann R. Hearing—Physiological Bases and Psychophysics. Springer Science and Business Media. 2012; p. 133–134.

Figures

Abstract

1 Introduction

2 Materials and methods

2.1 Ethics statement

2.2 Participants

2.3 Apparatus

2.4 Task and stimuli

2.4.1 Tasks.

2.4.2 Stimuli.

2.5 Procedure

2.6 Data analysis

3 Results

3.1 Magnitude estimation

3.2 Two-interval forced choice

3.3 Debriefing phase

4 Discussion

5 Conclusion

Supporting information

S1 Data. Data file containing participants’ answers obtained during ME tasks.

S2 Data. Data file containing participants’ answers obtained during 2IFC tasks.

S1 Questionnaire. Data file containing participants’ answers obtained during debriefing phase.

S1 Appendix.

References