Adaptive model of the peripheral visual system.

The peripheral visual system consisting of PRs and LMCs was modeled according to Li et al. [22] (Fig 1B). In each input line, the brightness signal is split into two signal branches. One branch is low-pass filtered with a small time constant of 9 ms, leading to a signal that follows even high-frequency intensity fluctuations; the other branch is low-pass filtered with a large time constant of 250 ms, leading to a signal that indicates the current light condition on a much slower timescale. By adding a constant to the latter branch and dividing the output of the fast branch by this signal, a saturation-like transformation function is obtained that shifts adaptively over time according to the current light condition. (1) In Eq (1), I is the input intensity, PR_resp is the photoreceptor response, LP1_PR and LP2_PR are first-order low-pass filters with small and large time constants, and C_PR is a constant. This adaptive photoreceptor model allows the visual system to operate over eight to ten decades of light intensities [22].

The photoreceptor output is then fed into the LMC model. The LMC model is a first-order high-pass filter that eliminates the information about the average brightness level. This high-pass filtering has been shown to be essential for extracting depth information by the motion detectors at the next processing stage [22]. As an elaboration of our earlier model [22], the LMC output is half-wave rectified and split into an ON and an OFF pathway according to its biological counterparts [33–35]. Furthermore, a saturation-like non-linearity was introduced to the LMC output by dividing the LMC output by the sum of the LMC output and a constant (Fig 1B).

Adaptive elementary motion detector model.

The LMC output of the ON and OFF pathway, respectively, is fed into the adaptive motion detector model (Fig 1B). This motion detector model is composed of a correlation-type motion detector and an adaptive processing of each half-detector output. The correlation-type motion detector is composed of two mirror-symmetric half-detectors sensitive to motion in opposite directions. Each half-detector detects motion by multiplying the delayed signal from one LMC output with the non-delayed signal from the corresponding neighboring LMC output. This leads to four EMD outputs for each column, two for preferred-direction (PD) motion of ON and OFF signals, respectively, and two for null-direction (ND) motion of ON and OFF signals. Each of the EMD outputs is processed by an adaptive mechanism similar to that of brightness adaptation of the photoreceptors (Eq (1), [22]), namely by dividing a fast signal branch following the fluctuations in the motion signal by a slow signal branch representing pattern velocities on a much slower timescale and embedding in a saturation-like Lipetz-transformation. Since motion adaption takes place on a much longer timescale than brightness adaptation in the peripheral visual system, the ‘fast’ and ‘slow’ time constants are much larger, 20 ms and 4 s, respectively, than the ‘fast’ and ‘slow’ time constants characteristic of brightness adaptation (see above). While the fast branches are the different half-detector outputs after being low-pass filtered with a small time constant, the slow branch is the average of all four half-detector outputs being low-pass filtered with a large time constant (Fig 1B). Adaptation of all four branches by the same slow signal was essential for achieving direction-independent motion adaptation. Moreover, to account for the increase of response transients to velocity changes an adaptive exponent is implemented in each component of the adaptation equation. (2) In Eq (2), EMD_adpt is the adapted EMD response of each branch, EMD_nadpt is the unadapted EMD response corresponding to the response after the multiplication, and EMD_ave is the average EMD_nadpt of all four branches. LP1_EMD and LP2_EMD are low-pass filters, C_EMD is a constant, and a is an adaptive exponent adjusted according to EMD_ave: (3) In Eq (3), a_max and a_min are the upper and lower boundaries of exponent a, p₁ and p₂ are constants determining the speed of recovery and the strength of adaptive modification. LP2_EMD(EMD_ave) is the unadapted EMD response of all four branches after they were averaged and low-pass filtered. Note that, the temporal frequency tuning of this correlation-type motion detector as well as its adaptive version are bell-shaped (Supplementary S2 Fig), i.e. with increasing stimulus temporal frequency the EMD response first increases, while with a further increase in temporal frequency the EMD response reaches an optimum and then decreases again.

LPTC model.

For simplicity, we assume that the outputs of the local motion detectors are linearly summated at the next stage of signal processing corresponding to the level of LPTCs (Fig 1B). Here, both half-detectors, i.e. ON and OFF, responding best to preferred-direction motion contribute to the sum with a positive sign, whereas both half-detectors responding best to null-direction motion contribute with a negative sign. The simplification of linearly summating the motion detector outputs instead of implementing a dynamic gain control at this processing stage [36, 37] is justified in the context of the current paper, since the pattern size in all model simulations was kept constant. (Note that, the LPTC model is only used for the model development and characterization (Figs 2–4), while the impact of motion adaptation on spatial vision is being analyzed at the level of adaptive EMD arrays (Figs 5–8)).

Stimulus set 1.

The first stimulus set was characterized by a sine-wave grating moving at a constant velocity (7420 ms) superimposed by eight short (50 ms) velocity transients at regular time intervals (780 ms) (Fig 2 upper panel of insets). Before and after this motion stimulus the grating was stationary for 500 ms, as in the corresponding experiments by Kurtz et al. [27] and Maddess and Laughlin [26]. For convenience, we used in many places the term ‘velocity’ instead of ‘temporal frequency’, i.e. the ratio between the velocity and spatial wavelength. This is justified, because the spatial wavelength was kept constant throughout our simulations and, thus, velocity and temporal frequency are proportional. The transients are either increments in temporal frequency (Fig 2A, from 2 Hz to 4 Hz) or decrements in temporal frequency (Fig 2B, from 4 Hz to 2 Hz); the temporal frequency of the constant background motion was selected to be either smaller (Fig 2A and 2B) or larger (Fig 2C, 8 Hz background to 12 Hz transients) than the optimum of the bell-shaped, steady-state velocity tuning curve of EMDs, or it matched the optimum (Fig 2D, 6 Hz background to 3 Hz transients); the brightness contrast was either high (Fig 2 red, c = 0.88) or low (Fig 2 green, c = 0.3). For all scenarios (Fig 2A–2D) the sine-wave grating was a 3 × 360 pixel² matrix with average intensity I_mean = 1000 a.u., and spatial wave length λ = 19 pixel. (See [27] for the corresponding parameters used in the electrophysiological experiments.)

In order to assess under which conditions the sensitivity to velocity discontinuities is enhanced by motion adaptation we used the same stimulation scheme as described above and systematically varied the temporal frequency (0.1, 0.2, 0.4, 0.8, 1.6, 3.2, 6.4, 12.8, and 25.6 Hz) and brightness contrast (0.05, 0.25, 0.45, 0.65, and 0.85) of the grating. For each combination of temporal frequency and contrast, the temporal frequency of the transients was either half (Fig 3A and 3B) or twice as large (Fig 3C and 3D) as the background temporal frequency. The model response was calculated for each of these conditions to assess whether the response contrast between the responses to the temporal frequency transients and the responses to constant background motion is enhanced by adaptation. The same stimulus scheme and response analysis was also done at a light level being brighter by eight decades (I_mean = 10¹² a.u. in Fig 3B and 3D smaller plots in contrast to I_mean = 10³ a.u. in Fig 3B and 3D main plots) to test model performance for a wide range of light intensities.

Stimulus set 2.

Stimulus set 2 was used to test the responses of model LPTCs to a moving grating before and after adaptation with constant-velocity motion of a grating, as used by Harris et al. [28]. Stimulus set 2 was characterized by the following sequence: a homogeneous screen of average brightness (500 ms), a reference stimulus consisting of motion of a sine-wave grating (1 s, c = 0.3, 3 Hz), a homogeneous screen of average brightness (50 ms), a long motion adaptation stimulus consisting of a grating of high contrast and constant velocity (4 s, c = 0.95, 5 Hz), immediately followed by a test stimulus of the same stimulus parameters as the reference stimulus, followed by a homogeneous screen (500 ms) (Fig 4A–4C). During the adaptation phase the grating moved either in the preferred direction (PD) (Fig 4A), in the null direction (ND) (Fig 4B) or in the orthogonal direction (Fig 4C). The sinewave grating was a 90 × 90 pixel² matrix with average brightness I_mean = 1000 a.u. and spatial wave length of 18 pixels.

Furthermore, under the same stimulus scheme as in Fig 4A and 4B, the contrast of the grating of the reference and test stimulus was varied systematically (20 logarithmically and equally distributed contrast levels between 0.005 and 1) in order to analyze how motion adaptation modifies the contrast gain (Fig 4D, see also Figure 2a in [28]).

Visual stimuli generated by translational motion in virtual 3D environments.

We used a virtual 3D environment consisting of a wall (1.1 m high, 16 m long, 0.55 m away from the flight trajectory) and a row of bars (5 cm wide, 1 m high, 1 m spacing, 0.5 m away from the flight trajectory) in front of the wall. An agent with one spherical eye (2° spatial resolution) moved parallel to the wall and the row of bars. It passed 1 bar/s during its 8 seconds of translational motion (Fig 5A). The wall and the bars were textured with a random cloud pattern with 1/f² statistics (f is the spatial frequency). The 3D environments were generated with Open Inventor 1.0 and the visual stimuli experienced by the agent were generated by Cyberfly toolbox developed by Lindemann et al. ([38]; Fig 5B).

The spatial discontinuities between the bars and the background cause discontinuities in retinal velocities during translational motion. In order to compare the impact of motion adaptation on different depth differences we increased the distance between the objects and the wall without changing the distance between the bars and the agent (Fig 6A). If we assume a flight speed of 1 m/s, the distance between the flight trajectory of the agent and the row of bars correspond to 0.5 m, and the walls in different scenarios to 0.55 m, 2 m, and 4 m. We adjusted the size of the wall accordingly to have the same-sized projection of the wall on the retina (Fig 6A). As a result, different spatial scenarios were characterized by the same retinal size of wall texture elements and the bars, but a lower background velocity with increasing wall distance. In order to distinguish the influence of depth transients on the responses from the influence of a specific pattern texture 50 different random cloud walls and bar patterns were included in our analysis.

Visual stimuli during translational motion in a natural 3D environment.

We also tested a stimulus mimicking what flies experience during translational motion in a cluttered natural environment. By taking a sequence of panoramic photographs along a linear track with the help of a hyperbolic mirror in natural environments (for example in a forest) and applying corresponding rendering methods, image sequences mimicking the retinal image flow during translational motion at 1 m/s in a forest were generated (for details see [21] and published data [39]). The available image sequences recorded in natural environments were too short for investigating the effect of motion adaptation (if we assume a flight speed of 1 m/s, the image sequences correspond to only 900 ms, whereas motion adaptation has a timescale of several seconds [27]). Therefore, we repeated the same image sequence eight times via concatenation. The concatenated image sequences were then fed into our adaptive model of the visual motion pathway. A potential influence of the discontinuity in the scenery due to concatenation was minimized by analyzing the influence of motion adaptation for the frames in the middle of the individual image sequences (Fig 7A).

Visual stimuli based on semi-natural flight dynamics.

Natural flight of flies consists of segments of translation interspersed with quick saccadic rotations. In order to investigate the impact of motion adaptation on spatial vision under conditions of natural flight dynamics we designed an artificial 3D environment and an artificial flight trajectory taking the dynamics of real flight of flies into account. According to [2] the saccade frequency of free blowfly flights in a cubic box is approximately 10/s; the yaw angle of saccadic turns varies by up to 90°, and the corresponding yaw velocity can reach up to several thousands of degrees per second. Considering these features of real flight dynamics, we generated a semi-realistic flight trajectory by bending each second of the linear translational flight trajectory of a total duration of 8 s, as described, for the pure translational scenario (Fig 8A) into a decagon (Fig 8D). This trajectory was centered in a decagonal flight arena, and the distance from the flight trajectory to the bar and the wall as well as the size of the bar and the height of the wall were kept the same as for the straight trajectory. The bar and the walls were also textured with a random cloud pattern, and the frequency at which a bar was passed by (1 bar/s) was also kept the same. One cycle of the decagonal trajectory was composed of a sequence of 80-ms-pure-translations along the walls of the decagon and 20-ms-pure-rotations that led to a 36° saccadic turn in the corners of the decagon. During the saccadic turns in the corners roll and pitch were kept constant, and the dynamics of yaw-velocity was based on recorded free flight data [3, 38] following the equation: (4) In this way, the saccade frequency, the yaw angle and velocity during saccades were within a realistic range and the total length and duration of the trajectory were identical to the pure translational trajectory.

Translational motion in virtual and natural 3D environments.

According to available electrophysiological data and our model simulations, motion adaptation can enhance the relative sensitivity to discontinuities in the motion stimulus, while reducing the overall response to sustained motion (Fig 2). This feature can potentially favor optic flow-based spatial vision, since during translational motion depth contours generate discontinuities in the optic flow profile, which might be enhanced as a consequence of motion adaptation. In order to test this hypothesis we first moved a virtual agent parallel to a row of bars in front of a wall (Fig 5A). The environment projected on the left eye is illustrated in Fig 5B.

We used the resulting motion sequence as the input to our model of the visual motion pathway and compared the response profile of the retinotopic EMD arrays before (Fig 5C left) and after (Fig 5C right) motion adaptation, i.e. when the first bar vs. when the last bar was passing the lateral part of the visual field. The responses to both the bars and the background wall were generally reduced after adaptation. However, as a consequence of motion adaptation, the response to the background wall pattern was much more reduced in comparison with the response to the bars making the bars more salient in the overall response profile of the EMDs (Fig 5C).

In order to quantify this impression we assessed the sensitivity to the depth discontinuities in the following way: First, we combined the temporal development of EMD responses to bars and background wall to one variable. To this end, we chose the lateral (azimuth = 90°) part of the visual field for our response analysis, because for geometric reasons bars passing the visual field at 90° azimuth led to the strongest responses and covered most of the vertical extent of the visual field. We then calculated the average of the motion energy (i.e. absolute value of EMD responses) at 90° azimuth over time (Fig 5D), which represents both bar and wall responses over time. Finally, based on these time-dependent bar and wall responses we calculated the response contrast according to Eq (5) for each of the eight consecutive bars and the corresponding wall sections (Fig 5E). Due to the strong reduction of background activity (Fig 5D inset), the response contrast increased almost monotonically with adaptation (Fig 5E).

To investigate the impact of the distance (and consequently retinal velocity) differences between the bars and the wall the wall was placed at a distance of 0.55 m, 2 m and 4 m from the flight trajectory in different scenarios, while the bars were kept unchanged at 0.5 m from the flight trajectory (Fig 6A). Moreover, to reduce potential effects of a specific cloud pattern we assessed the bar and wall responses (Fig 6B, 6D and 6F) and the response contrast between the bars and the wall (Fig 6C, 6E and 6G) as in Fig 5D and 5E for 50 random wall and bar patterns and averaged across textures (Fig 6B, 6D, 6F and 6H). The response contrast evoked by the bars increased for all wall distances tested (Fig 6H) as a consequence of a strong reduction of background wall responses with adaptation (Fig 6B, 6D and 6F insets). This effect was the more pronounced the closer the wall was to the bars and, thus, the smaller the response contrast was before motion adaptation (Fig 6H). Thus, motion adaptation enhances the sensitivity of the motion detection system to depth discontinuities.

This conclusion was further corroborated with more realistic stimuli mimicking the visual input during translational motion in a forest (Fig 7A). Because the original image sequence has a duration of only 900 ms (assuming 1 m/s flight speed) which is too short for investigating motion adaptation, the original image sequence was concatenated eight times. In order to see how motion adaptation affects the representation of the environment by arrays of motion detectors the response profiles of EMDs before adaptation (i.e. in the middle of the first repetition of a translational trajectory, Fig 7B) was compared with that after adaptation (i.e. in the middle of the eighth repetition, Fig 7C). Similar, but less prominent effects as in Fig 5C can be observed here: Motion adaptation led to a larger reduction of the responses to background structures than to the contours of nearby tree trunks, which makes the nearby tree trunks more salient. In order to better assess the influence of motion adaptation on signal representation by EMD arrays we calculated the local response contrast of the EMD response profile before and after motion adaptation and subtracted the local response contrast profile after adaptation from that before adaptation (Fig 7D). The red color indicates regions in the environment where the local response contrast was enhanced by adaptation, which corresponds mainly to the contours of nearby tree trunks.

During semi-natural flight.

According to our simulation results, motion adaptation enhances the segregation of foreground objects from their background if an agent performs pure translational motion (Figs 5–7). However, the translational periods of insect flight are never as long, but frequently interspersed with fast saccadic turns. To test whether the image flow induced by saccadic turns affects our conclusion that the representation of nearby contours is enhanced by motion adaptation, we designed a semi-natural flight trajectory that takes several features of natural flight dynamics into account [2]. Furthermore, we shaped the trajectory and the virtual 3D environments into decagons, so that the visual stimulus was as similar as possible to that used to obtain the results shown in Fig 6 (Fig 8A–8C). The same response analysis was performed as in Fig 6. Under such semi-natural flight conditions (Fig 8E and 8G), the background activity was dominated by the rotational response. This background activity was strong and rather independent of wall distance. Therefore, the response contrast curves increased with motion adaptation, but with a relatively shallow slope. Moreover, the response amplitudes did not differ as much for different wall distances as those obtained during linear motion without saccadic turns interspersed (compare Fig 8F and 8G).

Without saccadic turns, the response contrast between the bars and the background increased most when the distance between wall and bars was smallest and, accordingly, the background and bar responses most similar. Thus, the adaptation is most effective when an enhancement of the response contrast is particularly relevant to segregate objects from their background. However, this characteristic is hardly visible under semi-naturalistic conditions, in which large responses were induced by saccades. However, the general qualitative feature of enhanced response contrast between nearby objects and background with motion adaptation was maintained even with saccades, though to a much smaller extent. There is evidence that responses to saccadic turns are suppressed in visual neurons (measured in LPTCs) by efference copy signals [41, 42]. Although the exact location of the target of the efference copy is not yet clear, this mechanism can potentially counteract the above mentioned detrimental effects of saccades on the consequences of motion adaptation.