Fig 1.
The flash-lag effect (FLE) as a motion-induced predictive shift.
To follow the example given by [6], a football (soccer) player that would run along a continuous path (the green path, where the gradient of color denotes the flow of time) is perceived to be ahead (the red position) of its actual position at the unexpected moment a ball is shot (red star) even if these positions are physically aligned. A referee would then signal an “offside” position. Similarly, such a flash-lag effect (FLE) is observed systematically in psychophysical experiments by showing a moving and a flashed stimuli (here, a square). By varying their characteristics (speed, relative position), one can explore the fundamental principles of the FLE.
Fig 2.
In the current study, the estimated state vector z = {x, y, u, v} is composed of the 2D position (x and y) and velocity (u and v) of a (moving) stimulus. (A) First, we extend a classical Markov chain using Nijhawan’s diagonal model in order to take into account the known neural delay τ: At time t, information is integrated until time t − τ, using a Markov chain and a model of state transitions p(zt|zt−δt) such that one can infer the state until the last accessible information p(zt−τ|I0:t−τ). This information can then be “pushed” forward in time by predicting its trajectory from t − τ to t. In particular p(zt|I0:t−τ) can be predicted by the same internal model by using the state transition at the time scale of the delay, that is, p(zt|zt−τ). This is virtually equivalent to a motion extrapolation model but without sensory measurements during the time window between t − τ and t. Note that both predictions in this model are based on the same model of state transitions. (B) One can write a second, equivalent “pull” mode for the diagonal model. Now, the current state is directly estimated based on a Markov chain on the sequence of delayed estimations. While being equivalent to the push-mode described above, such a direct computation allows to more easily combine information from areas with different delays. Such a model implements Nijhawan’s “diagonal model”, but now motion information is probabilistic and therefore, inferred motion may be modulated by the respective precisions of the sensory and internal representations. (C) Such a diagonal delay compensation can be demonstrated in a two-layered neural network including a source (input) and a target (predictive) layer [44]. The source layer receives the delayed sensory information and encodes both position and velocity topographically within the different retinotopic maps of each layer. For the sake of simplicity, we illustrate only one 2D map of the motions (x, v). The integration of coherent information can either be done in the source layer (push mode) or in the target layer (pull mode). Crucially, to implement a delay compensation in this motion-based prediction model, one may simply connect each source neuron to a predictive neuron corresponding to the corrected position of stimulus (x + v ⋅ τ, v) in the target layer. The precision of this anisotropic connectivity map can be tuned by the width of convergence from the source to the target populations. Using such a simple mapping, we have previously shown that the neuronal population activity can infer the current position along the trajectory despite the existence of neural delays [44].
Table 1.
The model is implemented using the same paradigm as detailed in [16] while the extrapolation uses the same formalism as in [17]. We used a similar set of parameters and controlled by a set of iPython notebooks that are available on the corresponding author’s website. Note that the speed and diffusion parameters are given relative to one spatio-temporal period.
Table 2.
Stimuli are generated on a space defined in absolute values (ranging arbitrarily from −1 to 1) and time defined from t = 0 to t = T (in seconds). As such, stimulus parameters are defined in these units. To avoid border effects, the spatio-temporal domain is defined as a 3-dimensional torus (that is the cartesian product of the periodic real spaces ). By convention, a speed of 1 is defined as a motion of one spatial period in one temporal period.
Fig 3.
The diagonal motion-based prediction (dMBP) model accounts for the Flash-lag effect.
(A) We plot the histogram of estimated positions from the dMBP model with a neural delay τ = 100 ms for the moving and the flashed stimuli. These estimated positions are averaged across the five frames centered around the time at which the response to the flash reaches its maximal precision and across 20 trials. Comparing the distribution of estimated positions for the moving (green) and flashed (in red) stimuli shows that, at this particular instant, the (left) moving dot is perceived ahead of the estimated position of the flash. (B) We quantified this spatial lead by plotting the histograms of the inferred horizontal positions during these frames, both for the position-based predictive (PBP) and dMBP models. The red and green dashed vertical lines represent the average positions of the flashed and moving stimuli, respectively. One can observe a significant spatial lead in the dMBP model, but not in the PBP model. The motion component of the dMBP model is thus essential to explain the flash-lag effect. (C) We varied the speed of the dot motion to titrate its role in the amplitude of the spatial lead. The black dashed line illustrates the predicted linear relationship from an extrapolation model with a perfect knowledge about target speed (slope one). One can observe a nearly linear relationship at slow speeds, followed by a saturation for higher speeds. At the fastest extrema of the speed range, ones observes a decrease in the spatial lead of the moving spot, together with an higher variability across trials (error bars: ±1 SD), consistent with the experimental data from [36]. The nonlinear relationship in our model emerges from the decrease of precision in the representation of motion at higher speeds. It highlights the putative role of the dynamic, explicit representation of precision in explaining the flash-lag effect.
Fig 4.
Both flash-initiated and flash-terminated conditions can be explained by the diagonal motion-based prediction (dMBP) model.
With the same format as Fig 3-B, we plot the temporal evolution of the probability distributions of the inferred position for both the flashed (in red) and moving (in green) dots, in the (A) flash-initiated and (B) flash-terminated conditions. As in Fig 3-B, each curve corresponds to the five frames (respectively numbered from i − 2 to i + 2) centered on the time of the model’s maximal response to the flash. Dashed vertical lines indicate at each frame the estimated positions from the maximum a posteriori of the probability distributions for either the flash (red) or the moving (green) dot, together with the veridical position of the flashed dot (black). As expected, one can observe that the distribution of inferred positions is approximately correct for the flashed stimulus in all conditions. In the flash-initiated FLE condition, the distribution for the moving dot is biased towards its direction and develops very rapidly. Notice however that these biases are smaller than observed with the standard FLE. In the flash-terminated conditions, the bias is observed in the last frames before the maximum of the flash and then competes with another estimate with no bias which dominates near the moment of the flash’s maximum. Note that the a posteriori probability distributions around the flash’s maximum are very broad and indicate a high spatial uncertainty. Altogether, the absence of bias in the flash-terminated condition is similar to that reported psychophysically with human observers [28].
Fig 5.
Histogram of the estimated positions as a function of time for the dMBP model.
Histograms of the inferred horizontal positions (blueish bottom panel) and horizontal velocity (reddish top panel), as a function of time frame, from the dMBP model. Darker levels correspond to higher probabilities, while a light color corresponds to an unlikely estimation. We highlight three successive epochs along the trajectory, corresponding to the flash initiated, standard (mid-point) and flash terminated cycles. The timing of the flashes are respectively indicated by the dashed vertical lines. In dark, the physical time and in green the delayed input knowing τ = 100 ms. Histograms are plotted at two different levels of our model in the push mode. The left-hand column illustrates the source layer that corresponds to the integration of delayed sensory information, including the prior on motion. The right-hand illustrates the target layer corresponding to the same information but after the occurrence of some motion extrapolation compensating for the known neural delay τ.
Fig 6.
Estimating the dot position from the dMBP model during the motion reversal experiment.
In the motion reversal experiment, the moving dot reverses its direction at the middle of the trajectory (i.e., at t = 500 ms, as indicated by the mid-point vertical dashed line). In the left column (target layer) and as in Fig 5, we show the histogram of inferred positions during the dot motion and a trace of its position with the highest probability as a function of time. As expected, results are identical to Fig 5 in the first half period. At the moment of the motion reversal, the model output is consistent with previous psychophysical reports. First, the estimated position follows the extrapolated trajectory until the (delayed) sensory information about the motion reversal reaches the system (at t = 600 ms, green vertical dashed line). Then, the velocity is quickly reset and converges to the new (reversed) motion such that the estimated position “jumps” to a position corresponding to the updated velocity. In the right column (smoothed layer), we show the results of the same data after a smoothing operation of τs = 100 ms in subjective time. This different read-out from the inferred positions corresponds to the behavioral results obtained in some experiments, such as that from Whitney and Murakami [23].
Fig 7.
Dependence of the FLE with respect to contrast and duration of the stimuli.
(A) Using the same data format as in Fig 5, we show the spatial distribution of the estimated response (zoomed around its physical position at the perceived time of the flash at full contrast which is indicated by a cross) for different different relative contrast levels C indicated at each row. The different columns correspond from left to right to different conditions where the contrast of the dot is manipulated (first two columns)—respectively at the beginning of the cycle (i.e. flash-initiated) cycle, the mid-point (i.e. standard cycle)— or where the contrast of the flash is varied (right-end column). Note that in the standard FLE case (middle column), the model already responds to very low values of dot contrast in a nearly all-or-none fashion. By comparison, the responses to the dot or the flash during the initial phase of the trajectory gradually increased with contrast. In particular, the dot’s lag seems to increase more rapidly with respect to contrast. (B) These qualitative results are best illustrated by plotting in the first column the precision of the response as measured by the inverse standard deviation of the estimated position as a function of contrast of the different conditions. Coherent with the results illustrated in (A), the precision of the representation varies gradually against contrast of the flash or moving dot in the early phase whereas it changes more rapidly and abruptly as a function of the moving dot’s contrast in the standard FLE. Consequently, we estimated in the second column the spatial lag that is expected when changing the contrast of the stimuli (± one standard deviation). Coherently with psychophysical results, increasing the contrast of the moving dot gradually increases the FLE in the flash-initiated cycle but has only limited effects in the standard FLE when above a given precision as it rapidly reaches a saturating value of ≈0.2 corresponding to a full compensation of the fixed delay. Consistent with [38], these results show the role of spatial uncertainty in dynamically tuning the estimated position and, ultimately, in influencing the spatial lag in the FLE. (C) As shown by [47], flash duration modulates FLE. We show here the precision for the flash as a function of time with respect to duration. While the peak remained at t = .5 s (that is, at t = .6 s when including the delay), we tested for different durations, respectively .03, .05, .08, .13, .25 in s (as marked by colored horizontal bars). The respective measured time to reach the maximal precision are given by tmax (in s), showing that precision was high for T ≥.05 s (that is, 50 ms). Notice that this value was used for all the experiments described above.