Fig 1.
Visual stimuli and experimental procedure for the unimodal spatial-discrimination task.
(A) Example visual stimuli (contrast exaggerated). (B) Task timing (blue: auditory stimuli; pink: visual stimuli). Participants were successively presented with a standard stimulus (located straight ahead) and a test stimulus (located to the left or right) in random order. After stimulus presentation, they used a keypad to report which interval contained the stimulus located farther to the right. Feedback was provided.
Fig 2.
Results for the unimodal spatial-discrimination task.
(A) Psychometric functions for representative participant S4. The probability of judging the test stimulus as to the right of the standard stimulus is plotted as a function of the distance between the two stimuli, with negative numbers indicating that the test stimulus was located to the left of the standard stimulus (presented straight ahead). V: Visual. A: Auditory. Filled circles: binned response proportions (bin size = 1.8°). The area of each filled circle is proportional to the number of trials within the bin. Solid curves: psychometric functions fit to the data. (B) Estimated just-noticeable difference (JND) of each visual stimulus type for S4. Solid line: auditory JND. Error bars and blue area: 95% bootstrapped confidence intervals. (C) Group mean JNDs (± SEM).
Fig 3.
Experimental procedure and results for the bimodal spatial-discrimination task.
(A) In each trial, a visual stimulus (high reliability) and an auditory stimulus (four possible auditory locations, ±2.5 and ±7.5° relative to straight-ahead) were presented in random order. Participants reported whether the visual stimulus was to the left or right of the auditory stimulus. Feedback was not provided. (B) Psychometric functions for participant S4. Probability of judging the visual to the right of the auditory stimulus is plotted as a function of visual stimulus location. Filled circles: binned response proportions (bin size = 3°). Curves: psychometric functions fitted separately for the four auditory stimulus locations (shades of blue). The area of each filled circle is proportional to the number of trials in each bin.
Fig 4.
Modality-specific biases in spatial perception.
(A) Point of subjective equality (PSE) as a function of the location of the auditory stimulus. Dashed grey line: identity line; solid line: linear regression line; horizontal dashed lines: visual stimulus locations perceived as co-located with the four auditory standard locations based on the regression. (B) Estimated constant, location-independent (y-axis; regression intercept) and proportional, location-dependent (x-axis; regression slope) shift of visual relative to auditory stimulus location. The proportional and constant shifts equal the slope and intercept of the linear regression line through the PSEs (see panel A). Error bars: 95% bootstrapped confidence intervals. Vertical and horizontal dashed lines correspond to the absence of proportional and constant shifts, respectively.
Fig 5.
Experimental procedure and conditions during the recalibration experiment.
(A) Timeline for unimodal localization tasks (pre- and post-recalibration phase). In each trial, either a visual or an auditory stimulus was presented; participants indicated its location using a visual cursor displayed on the screen. Feedback was not provided. (B) Timeline for the bimodal localization task (recalibration phase). In each trial, participants were presented with a spatially discrepant audiovisual stimulus pair; they were asked to localize one of the modalities, cued after stimulus presentation (V: localize the visual component, A: localize the auditory component). Feedback was not provided. (C) Stimulus locations in physical and perceptual space. Top panel: the physical locations of the four perceptually aligned audiovisual stimulus pairs identified at the beginning of the study for participant S4, stimuli were always presented at one of these locations; bottom row: pairs with a constant perceptual spatial discrepancy were presented during the recalibration phase, solid lines: location pairs presented in the visual-left-of-auditory condition; dashed lines: location pairs presented in the visual-right-of-auditory condition.
Fig 6.
(A) Pre- (diamonds, jittered to the left) and post-recalibration (circles, jittered to the right) localization responses as a function of auditory and visual stimulus locations (relative to straight-ahead) measured for participant S4 in the visual-right-of-auditory and low-visual-reliability condition. Localization responses were summarized using linear regression (dashed lines: pre-recalibration; solid lines: post-recalibration). Grey shaded area: estimated probability distribution of spatial perception-unrelated localization noise centered at an example location. (B) Auditory and visual recalibration effects (the difference between the intercepts of the pre-and post-recalibration regression lines in panel A) as a function of visual stimulus reliability for three participants. Error bars: 95% bootstrapped confidence intervals.
Fig 7.
Simulated auditory recalibration effects based on three candidate models of cross-modal recalibration.
The effect of visual (horizontal axis) and auditory (panels) stimulus reliability in bimodal trials on the amount of auditory recalibration by vision, based on causal-inference (solid line), fixed-ratio (dashed line), and reliability-based (dot-dashed line) models for two different learning rates (top row: slow; bottom row: fast).
Fig 8.
Model predictions and model comparison.
(A) Observed (circles) and predicted (diamonds) auditory (blue) and visual (pink) final measurement shifts after the recalibration phase based on 1,000 runs using the best-fitting parameters given the causal-inference model (diamonds) as a function of visual reliability for all participants (panels). Error bars: 95% bootstrapped confidence intervals. (B) Model predictions by the fixed-ratio model (triangles in left panel) and the reliability-based model (inverted triangles in right panel) for participant S1. (C) Model comparison indices, smaller values indicate more evidence (RB = reliability-based, FR = fixed ratio, CI = causal inference, = causal inference with supramodal learning rate).
Fig 9.
Cue reliability and the amount of recalibration.
(A) Non-linear effect of visual stimulus reliability in bimodal trials (; different panels) on the auditory spatial estimate
(green vertical lines). Blue and red dashed vertical lines and curves: auditory and visual measurements (
and
in perceptual space) and likelihood functions, respectively. Blue: when separate causes are assumed, the auditory measurement and likelihood equal the auditory location estimate
and the posterior distribution over auditory locations (a flat prior over stimulus location is assumed). Grey dashed vertical lines and curves: audiovisual location estimates
and posterior distributions of audiovisual locations conditioned on a common cause. (B-D) The effect of visual reliability on the posterior probability of a common cause,
, the integrated location estimate, i.e., the estimate conditioned on a common audiovisual source
, and the distance between auditory measurement and location estimate
, which directly sets the amount of recalibration.
Fig 10.
Determinants of recalibration.
The joint effects of visual reliability in bimodal trials, , the distance between auditory and visual measurements in perceptual space,
, and the common-cause prior, p(C = 1) on the posterior probability of a common cause,
(panel A), and the distance between the final location estimate and the auditory measurement,
(panel B), which is proportional to the amount of recalibration.
Fig 11.
The influence of spatial discrepancy and modality-specific biases on the amount of auditory recalibration.
Top row: The auditory recalibration effects (color key) as a function of proportional and constant shift of visual relative to auditory location when spatial discrepancy (panels) is constant in physical space (i.e., sV = sA + spatial discrepancy). Bottom row: The auditory recalibration effects when spatial discrepancy is constant in perceptual space (i.e., visual stimulus locations are selected to adjust the perceptual biases in auditory relative to visual spatial perception, sV = proportional shift × (sA + spatialdiscrepancy) + constant shift).
Table 1.
Summary of model parameters in Θ3.