2.1 Quantitative analysis of pairwise swimming dynamics

Fig 1B illustrates the main kinematic and geometric quantities used throughout this study to characterize and quantify fish motion, as well as the influence of social interactions on it. A detailed definition and mathematical description of these variables are provided in the Materials and Methods section.

Fig 2 shows the experimental distributions obtained from two real fish, together with their counterparts from numerical simulations of the model described below, for a set of key descriptors of individual and collective motion. The probability density functions of the full 3D speed and inter-individual 3D distance are nearly indistinguishable from those of their horizontal projections and d, respectively (see S1 Fig), indicating that motion remains mostly planar. While this result reflects the strong tendency of fish to align vertically in pairwise interactions, it does not preclude the emergence of three-dimensional structure in larger groups, where interactions with multiple neighbors may lead to more complex spatial organization [35].

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Quantification of free swimming behavior in a fish pair within the hemispherical bowl.

Probability density functions (PDFs) of (A) swimming speed , (B) distance to the bowl wall (solid lines) and radial position r (dashed lines), (C) incidence angle relative to the wall , (D) swimming depth z, (E) elevation angle , (F) vertical inter-individual distance , (G) horizontal inter-individual distance d, (H) viewing angle of the neighbor when acting as the geometrical follower (solid lines) or geometrical leader (dashed lines), and (I) heading misalignment in the horizontal plane (solid lines) and vertical plane (dashed lines). Red curves represent experimental data from two real fish, whereas purple curves correspond to numerical simulations of the model. Horizontal gray lines in (C, E, H, I) indicate the uniform angle distribution value, 1/360 ().

https://doi.org/10.1371/journal.pcbi.1014544.g002

Fish swam at a preferred speed of approximately  cm/s (Fig 2A), rarely exceeding 10 cm/s. Note that this speed range is markedly lower than observed in quasi-2D experiments in shallow water [5], where the mean speed was close to 10 cm/s, which might be ascribed to lower light levels in the VR experimental setup. The fish remained close to the wall of the bowl, with a mean wall distance  cm (Fig 2B), corresponding to a radial position  cm from the central vertical axis. Their heading angle relative to the wall, , was concentrated slightly below (Fig 2C), indicating motion nearly tangential to the boundary. The fish also exhibited a preferred swimming depth around  cm (Fig 2D), with occasional excursions toward the bottom (uniform tail over  cm, with maxima up to 13 cm). Vertical motion was minimal overall: the elevation angle remained tightly centered around zero (Fig 2E), confirming the strong preference for horizontal navigation.

At the collective level, individuals stayed in close proximity, with a typical horizontal separation  cm (Fig 2G) and a very small vertical separation  cm (Figs 2F and S1). As shown in Fig 2H, a clear leader–follower configuration emerged: the follower typically viewed the leader directly ahead (solid curves peaked near zero), whereas the leader perceived the follower behind, with peaks near (dashed curves). Alignment was strong in both planes, as evidenced by the peaked distributions of heading differences and around zero (Fig 2I), the latter being particularly narrow due to minimal vertical excursions.

Another notable behavior of the fish is their spontaneous tendency to perform U-turns while swimming side by side. One fish suddenly turns around to swim in the opposite direction of its companion, which in turn performs a U-turn to rejoin the first. S3 Fig shows the probability density function (PDF) and the survival curve of the time intervals separating two successive U-turns, displaying an exponential decay.

2.2 Data-driven 3D model of pairwise coordination in fish

Building on our empirical results and extending previous 2D modeling frameworks [5–7,36], we develop a fully data-driven 3D dynamical model that accounts for the geometry of the hemispherical bowl and the social interactions underlying coordinated motion. Because of the limited precision of the three-dimensional tracking (especially along the vertical axis), segmentation of fish trajectories into discrete burst-and-coast phases was not feasible. As a consequence, we could not rely on the burst-and-coast–type description and model previously developed in two dimensions [5]. It was therefore necessary to modify the model structure and adopt a continuous-time formulation of fish motion.

The instantaneous behavioral state of a fish is represented by the 9 variables , , , , z, d, , , , respectively describing its horizontal and vertical speed, its distance and orientation with respect to the wall, its depth relative to the water surface, and its horizontal and vertical distance, viewing angle, and heading relative to another fish (see Fig 1B). The temporal evolution of these variables combines biomechanical constraints, interactions with the environment, and social cues from the neighbor. The associated equations of motion are based on principles of self-organization, symmetry, and collision avoidance, and their functional forms are quantitatively inferred from the experimental data using a reconstruction procedure generalized from flat-arena studies to a curved 3D geometry [5]. A detailed description of the model is provided in Materials and Methods.

We decompose fish motion into horizontal and vertical components:

(1)

(2)

In the horizontal plane, the acceleration results from six forces: (i) an effective friction emerging from hydrodynamic effects and the tendency of fish to swim at a preferred speed, , (ii) a term implementing the tendency of a fish to adapt its speed to that of its neighbor (or to the average speed of its neighbors for more than one neighbor), of magnitude , (iii) a radial repulsive interaction with the wall of the bowl, , (iv) a wall-induced rotational force guiding motion tangentially, , and (v-vi) the social attraction and alignment interactions with the neighbor, and . In the vertical direction, arises from (i) a friction acting on the vertical speed , encapsulating the tendency of a fish to swim horizontally, (ii) a tendency to swim at a preferred comfort depth, , which also effectively results from the interaction of the fish with the wall of the bowl and the surface of the water, and (iii) a vertical attraction toward the neighbor’s depth, . Additionally, a noise term is added to each component, and respectively, corresponding to the fish spontaneous decisions and random changes of heading.

A powerful assumption of our modeling process consists in considering that, once decomposed along the parallel and perpendicular components of the acceleration, the magnitudes of the forces are separable as products of single-variable functions, e.g., . Moreover, symmetry constraints apply to the angular functions to consistently reproduce the directional reactions of fish. For instance, the rotational force must be an odd function of at fixed (hence, including a factor ) ensuring that a fish approaching the wall with (wall to its right) would turn in the opposite direction to that taken if it were approaching the wall with the opposite angle . This constraint preserves the right-left symmetry, which was shown to consistently hold for the considered fish species [5]. Similarly, the alignment force must include a factor favoring a left turn when the neighbor is oriented counterclockwise with respect to the heading of the focal fish () by increasing the perpendicular acceleration, while if the neighbor is oriented relatively clockwise (), the negative contribution induces a right turn.

Thus, with these separability and symmetry considerations, the magnitudes of the forces induced by the wall are given by

(3)

(4)

where and are even functions of . As for the social interactions, they can be written as

(5)

(6)

(7)

where all g and h functions of an angle are even. These interactions involve interaction functions depending on the distance between the fish and its neighbor or the wall, which are modulated by angular functions effectively describing the fish anisotropic perception of its environment.

The model also includes simple rejection rules that prevent the fish from crossing the physical wall of the bowl or the water surface. When a predicted step places the fish beyond these boundaries, the movement is rejected and a new fish orientation or vertical movement is recalculated (see Material and Methods for more details).

The resulting stochastic differential system is solved numerically using the Euler–Maruyama scheme with experimentally informed initial conditions.

2.3 Reconstruction of empirical interaction functions and validation of the 3D behavioral model

In this section, we present the interaction functions empirically reconstructed from the experimental trajectories and analyze their biological and dynamical roles in shaping coordinated motion. This method was originally introduced in [5] and extended in [22] and was extensively exploited to analyze the interactions arising from effectively 2D burst-and-coast trajectories, relevant to the case of fish swimming in shallow water. Here, we simply extend the method to 3D trajectories and to a continuous-time formulation of the equations of motion.

The general idea of the method consists in describing the different single-variable interaction functions introduced in the previous section by polynomial expansions, Fourier series, or tabulations. Ultimately, the parameters involved in this description are optimized to minimize the discrepancy between the observed acceleration of a real fish and the acceleration predicted by the model.

Some of the functions introduced above have a general shape that can be guessed on physical principles. For instance, the magnitudes (denoted with lowercase f) of the friction terms and the speed adaptation term are expected to resemble the following functional forms:

(8)

The friction tends to maintain a speed over a relaxation time , while ensures that the mean vertical speed in a bounded bowl must necessarily be zero (i.e., that the fish is on average aligned with the horizontal plane). Finally, enforces the tendency of a fish to match its speed to the speed of its neighbor(s). In our reconstruction procedure, we are not imposing these simple but natural linear forms, and the functions are instead expanded as a finite order polynomial of their variable (in practice, we used third-order polynomials; see for instance, Eqs. (S1), (S2), and (S12) in Supplemental Material). Similarly, some interaction functions are expected to vanish linearly for some value of their corresponding variable and will also be described by finite-order polynomials. This is the case for (expected to vanish at a vertical separation between fish ) and (expected to vanish at some preferred depth z = z₀). Hence, in total, 5 functions are described as polynomials of their corresponding variable, whose coefficients are to be optimized by the reconstruction procedure.

Moreover, the model involves 8 angular functions (depending on the angles , , or ) modulating the magnitude of the interaction of a fish with the wall and its attraction/alignment to its neighbor(s), and encoding the fish’s anisotropic perception of its environment. These angular functions are expressed as even Fourier series (preserving the left-right symmetry closely maintained by the considered fish [5]) of the form , where the ’s are parameters to be optimized. We allowed up to K = 6 Fourier modes for each function, although for 2D trajectories, 2 or 3 modes were found to be sufficient to capture the different interaction functions [5].

Finally, for the 5 interaction functions , , , , and , no general insight on their shape can be ascertained from physical/biological considerations. Hence, following [5,22], we tabulate each of them over a finite grid (typically, 0.5 cm spacing for the grids for and d) of their corresponding variable and consider the value of the function at each point of the grid as a parameter to be optimized. After optimization, the resulting functions are fitted by simple functional forms, which are ultimately implemented in the model. These fitting functions and the different polynomial and Fourier expansions introduced above are detailed in the Supplemental Material.

The coefficients of the 5 polynomial expansions, those of the 8 Fourier series, and the parameters of the 5 tabulated functions are then optimized by minimizing the error between the experimentally measured accelerations and those predicted by the model. We use the total squared error , where each term measures the squared difference between the experimental and model components of the acceleration along the fish heading, perpendicular to it, and along the vertical axis, , , and , respectively (the detailed forms of , , and in the model are given in Materials and Methods). The matching error along the three directions are thus

for , , and z, and where each model acceleration is evaluated using the values of observed in the corresponding experimental data.

Experimental accelerations are computed by finite differences after a slight smoothing of the experimental trajectories (see Materials and Methods). The model accelerations depend on the unknown functions introduced above. The coefficients describing their polynomial or Fourier expansion, or their tabulation parameters, are therefore the unknowns of the minimization problem. Because each force contributes linearly to the acceleration due to the separability assumption, is quadratic with respect to the tabulated function values, thus allowing the efficient use of a gradient-descent optimization scheme to minimize the matching error .

Fig 3 depicts the reconstructed functions (colored curves) superimposed on the PDF of their corresponding variable (gray areas). This highlights the regions of highest data density, where the reconstruction is most reliable and which primarily determine the overall shape of the functions.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Experimentally reconstructed locomotion and interaction functions.

Colored solid curves: reconstructed force components inferred from the experimental data. Gray areas: corresponding probability density functions (PDFs) of the associated state variables. (A) Self-propulsion terms: hydrodynamic friction and relaxation toward a preferred speed, and tendency of a fish to adapt its speed to that of its neighbor, (arbitrarily centered at equal to the mean speed in this plot) as functions of the horizontal speed . (B) Interaction with the wall: radial repulsion and wall-induced rotation as functions of the distance to the wall . (C) Angular modulation of wall effects through and as functions of the incidence angle . (D) Social interactions as functions of the horizontal distance between individuals d: horizontal attraction , vertical attraction , and horizontal alignment . (E, F) Angular modulation of the social interactions as functions of the viewing angle and heading difference . (G) Vertical drag as a function of vertical speed . (H) Attraction toward a preferred depth  cm, , as a function of the depth z. (I) Vertical attraction between fish as a function of the vertical separation between fish . In these plots, the angular functions have been normalized such that .

https://doi.org/10.1371/journal.pcbi.1014544.g003

The horizontal drag reduces speeds greater than  cm/s, and increases speeds below it (Fig 3A). The speed adaptation function is essentially linear in the difference between the speed of a focal fish and that of its neighbor, as anticipated above (see Eq. (8)).

The wall-repulsion function is strong when the fish is within 6 cm of the wall (i.e., about 2 body lengths, consistent with the results of [5] for quasi-2D experimental setups) and essentially vanishes for  cm (see Fig 3B). The wall-induced rotational component has a similar spatial profile but a weaker magnitude. Both are modulated by even angular functions (see Fig 3C) encapsulating the fish’s anisotropic perception of the wall, with the modulation of the repulsion to the wall peaking at , when the fish is facing the wall, and being suppressed by a factor of 2 or more for .

The horizontal attraction and alignment interactions and initially grow with the distance between fish and then peak at –3 body lengths (7–9 cm), before slowly decaying for  cm, although the reliability of this long-range behavior is poor due to a lack of observed data, as fish are rarely found at such distances (Fig 3D). The vertical attraction extends over a slightly narrower range.

Angular modulations reveal that social attraction is stronger when the neighbor is located laterally (maxima of near , Fig 3E), and more pronounced as the fish become more misaligned, with increasing as goes from 0 to (Fig 3F). Alignment is stronger when the neighbor is in front (, Fig 3E) and becomes anti-alignment when the neighbor is behind ( when ). Furthermore, alignment weakens as the fish are more misaligned ( decreases as goes from 0 to ; see Fig 3F). This apparent anti-alignment should not be interpreted as an attraction-like response. Instead, it results from the angular dependence of the alignment interaction (Fig 3E), which promotes turning when a neighbor is located behind the focal fish, independently of their relative orientation.

As anticipated above (see Eq. (8)), the vertical friction is a nearly linear odd function that suppresses vertical excursions (see Fig 3G). The effective force driving the fish toward a preferred depth  cm (see Fig 3H) exhibits a stronger/steeper repulsive interaction with the surface of the water (z = 0) than with the bottom of the bowl. For the interaction between fish along the vertical direction, the observed modulation by and is weak, indicating that vertical attraction depends primarily on the depth difference (the weakly nonlinear and almost odd in Fig 3I) and on the distance between fish ( in Fig 3D, which is strongly suppressed for d > 10 cm).

Fig 2 shows that the agreement between experimental data and simulations of the model integrating the empirically reconstructed interaction functions is excellent. This agreement holds for both horizontal kinematic variables, such as speed, relative position, and heading, and vertical components, including swimming depth z and vertical orientation . Moreover, the model also accurately reproduces the time intervals separating two consecutive U-turns (S3 Fig).

2.4 Closed-loop experiments with a virtual conspecific controlled by the model

Having established the natural coordination patterns of two real fish and derived a data-driven model that accurately reproduces these dynamics, we next examined how a real fish behaves when it interacts with a virtual conspecific whose motion is controlled in real time by the model.

We tested three closed-loop conditions, each differing in the behavioral parameters assigned to the virtual fish, to probe the role of reciprocity and kinematics in social coordination. In preliminary observations, we found that real fish swim significantly faster in the VR setup than in real–real interactions, likely due to environmental differences such as illumination. To account for this discrepancy, we introduced conditions with different speed regimes for the virtual fish, allowing us to test whether interaction rules inferred at lower speeds generalize to higher-speed contexts:

• Condition C1: a perfect clone, reproducing the behavior of a real fish in a pair, swimming with a mean speed  cm/s;
• Condition C2: a faster clone moving at twice the speed used in condition C1 ( cm/s, similar to what is observed in quasi-2D experiments in shallow water and with a standard lighting [5]);
• Condition C3: a faster clone identical to the one used in condition C2 but which is independent of the real fish, creating a one-way interaction where only the real fish responds.

These closed-loop assays allow us to quantify how a real fish adjusts its behavior depending on the speed and responsiveness of the virtual partner. By comparing the resulting probability distributions of key behavioral measurements with those obtained for two real fish (Fig 4), we can evaluate both the realism of the model and the extent to which coordination mechanisms remain engaged when interaction becomes kinematically challenging or asymmetric. Remarkably, despite substantial speed differences introduced in C2 and C3, and the absence of reciprocal interaction in C3, the real fish still exhibits similar social responses (i.e., similar pairwise separation and alignment) in response to the virtual conspecific.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Model validation through real–virtual fish interactions.

Probability density functions (PDFs) of behavioral variables measured for the real fish (red) interacting with a virtual conspecific controlled in closed loop by the model, for the virtual fish (blue), and for numerical simulations involving two real fish (violet). (A) Swimming speed . (B) Distance to the wall . (C) Incidence angle relative to the wall . (D) Swimming depth z. (E) Elevation angle . (F) Vertical inter-individual distance . (G) Horizontal inter-individual distance d. (H) Viewing angle for geometrical leader (dashed lines) and follower (solid lines). (I) Heading alignment between the real and virtual fish in the horizontal plane (solid lines) and in the vertical direction (dashed lines).

https://doi.org/10.1371/journal.pcbi.1014544.g004

2.4.1 Behavioral responses of a real fish interacting with a realistic virtual partner.

We refer to the condition where both fish are real as the 2R (two real) baseline. Fig 4 reports the same observables presented in Fig 2 for condition 2R, but now corresponding to condition C1, where the real fish interacts with a model-controlled virtual clone.

The probability density functions for the real fish (red), the virtual fish (blue), and simulations of two real fish (violet) allow us to directly compare (i) the behavior of a real fish when paired with a real versus a virtual partner and (ii) how the model-defined behavior changes when the real fish deviates from its natural pairwise dynamics.

Overall, the qualitative features of bowl swimming remain present: real and virtual fish stay near the wall (Fig 4B), swim nearly tangentially to it (Fig 4C), maintain a preferred depth (Fig 4D), and exhibit small vertical angles (Fig 4E). Coordination is still visible, as shown in S1 Video and S2 Fig. The strong following behavior observed in these experiments relies on the perceptual realism of the projected stimulus. Because the virtual fish is rendered using a full 3D model and projected through an anamorphic transformation that accounts for the real fish’s viewpoint, the stimulus is perceived as a coherent object located inside the tank. As illustrated in S1 Video, fish respond to the virtual conspecific as if it were physically present, maintaining appropriate distances and relative positioning despite the projection occurring on the curved surface of the bowl. This result highlights that the visual system of the fish integrates both geometric and kinematic cues to reconstruct a socially meaningful representation of the stimulus.

However, substantial quantitative differences emerge. Most strikingly, the real fish swims markedly faster when interacting with the virtual clone than with a real conspecific: the PDF of its speed peaks near  cm/s in C1, reaching up to 20 cm/s, whereas in 2R the speed rarely exceeds 10 cm/s (Fig 4A). In fact, the swimming speed of the real fish now aligns with the characteristic speeds documented in quasi-2D experiments [5]. As noted earlier, the projection of the virtual fish may partially restore lighting conditions comparable to those of the 2D setup, which could explain the observed agreement in swimming speeds. The virtual fish also swims faster than implemented in the model (peak near  cm/s instead of 5 cm/s), thanks to the speed adaptation term (see Eq. (8) and the discussion below it). However, as it is constrained by the model’s speed parameters, it cannot accelerate sufficiently to keep up with the real fish (S1 Video). This speed mismatch produces an increase in the typical distance between fish (Fig 4G) and prevents the virtual fish from positioning itself appropriately to maintain alignment (Fig 4I).

A second major difference is a more frequent bottom exploration (Fig 4D). While the real fish in 2R mostly remains near its preferred depth  cm, the real fish in C1 frequently visits the lower region of the bowl, producing a secondary PDF peak around  cm. The virtual clone exhibits the same trend, though less strongly.

These changes weaken spatial cohesion: the PDF of the horizontal distance between fish d broadens substantially, with d > 20 cm observed frequently (Fig 4G). Alignment is also drastically reduced: the PDF of is almost flat (Fig 4I), indicating minimal heading correlation.

The PDF of the angle of perception shows that fish still display the classic spatial configurations of geometric leader and follower. As a follower, the real fish predominantly perceives the virtual one ahead (; red solid line, Fig 4H), consistent with 2R behavior (purple line), and as leader, it perceives its neighbor behind (; red dashed line, Fig 4H). However, the virtual fish tends to perceive the real one laterally (; blue solid line), revealing a breakdown in reciprocal leadership. As geometrical leaders, both fish maintain distributions similar to real-pair interactions. The virtual fish’s behavior differs subtly from that of the real one. As leader, its behavior remains similar to that observed in interactions between real fish (blue dashed line, Fig 4H). However, as a follower, it clearly keeps the real fish in front but at one side, as shown by the two peaks of the PDF of at (blue solid line, Fig 4H), thus breaking the reciprocity typically observed in pairwise interactions.

To investigate the origin of these deviations (faster motion, deeper swimming, larger separations, and reduced alignment) we quantified turning behavior and acceleration components.

The real fish exhibited much stronger turning dynamics when interacting with the virtual clone. Fig 5A shows the PDF of the signed angular speed , defined as the instantaneous heading change signed with the angle of incidence to the wall , so that left and right turns can be accumulated. The PDF peaks at  s^-1 in C1, compared with only  s^-1 in 2R, and shows a greatly increased probability of extreme turns (high-value tail for in purple line). We also note that the mean time separating a U-turn performed by one fish and that of its companion is higher in C1,  s, than in the 2R condition,  s (S4 Fig). Since the time intervals between U-turns executed by either the real or the virtual fish’s partner are similar, this suggests that the virtual fish’s inability to keep up with the real fish stems from its limited acceleration capability.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 5. Real–virtual interactions induce stronger turning accelerations.

Probability density functions (PDFs) of kinematic variables describing turning maneuvers when the focal fish interacts with a real (red) or a virtual (violet) partner: (A) Signed angular velocity (sign defined by the incidence angle ), (B) Parallel acceleration , (C) Perpendicular acceleration . Squares in (B, C) indicate the absolute mean accelerations and , showing that perpendicular accelerations are more than twice as large as parallel ones when swimming with a virtual partner.

https://doi.org/10.1371/journal.pcbi.1014544.g005

Kinematic forces are also amplified; the absolute mean parallel acceleration nearly doubles,  cm s^-2 in C1 vs. 2.5 cm s^-2 in 2R, and the perpendicular acceleration nearly triples:  cm s^-2 in C1 vs. 3.5 cm s^-2 (Figs 5B and 5C).

Together, these observations reveal that although the model-controlled virtual fish can elicit coordinated interactions, the lack of full reciprocity and the inability of the virtual fish to match the real fish’s acceleration lead to a fundamental disruption of pair cohesion and alignment.

2.4.2 Behavioral response of a real fish to a faster virtual conspecific, with or without reciprocal interactions.

In Condition C1, the real fish swam considerably faster than when paired with a real conspecific in the VR setup. This observation motivated us to examine how a real fish responds when the virtual partner swims at that same higher speed, which coincides with the typical real fish speed observed in quasi-2D setups [5]. We refer to this configuration as Condition C2. Furthermore, to test the role of reciprocal interaction, we implemented a third configuration (Condition C3) in which the virtual fish behaves as an isolated agent and does not respond to the real fish. Any coordination in C3 thus arises solely from the real fish adapting to the virtual one.

For each condition, we plotted the PDFs of the relevant behavioral variables: Condition C2 shown in the upper rows of Figs 6–8, and Condition C3 in the bottom rows. We also include numerical simulations in which both agents follow the behavior of the virtual fish. In C3, these simulations represent a “null model” without reciprocal interactions. The parameter values of the model used in these different conditions are provided in S1 Table. The kinematic differences between the virtual fish in conditions C1 and C2 are shown in S5 Fig.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 6. Individual behavioral responses to a faster virtual conspecific with and without social interactions.

Probability density functions (PDFs) of real fish (red), the virtual fish (blue), and simulations of the 2R-based model (violet) at a higher mean speed  cm/s. (A–C) Condition C2: with social interactions enabled (bidirectional coordination). (D–F) Condition C3: without social interactions (unilateral interaction). (A, D) Swimming speed . (B, E) Distance to the wall . (C, F) Incidence angle relative to the wall .

https://doi.org/10.1371/journal.pcbi.1014544.g006

Download:

• PNG
  larger image
• TIFF
  original image

Fig 7. Effects of reciprocal vs. unilateral interaction with a faster virtual partner on collective coordination.

Probability density functions (PDFs) of collective variables measured for pairs composed of a real fish (red) and a faster virtual fish (blue), controlled by the 2R-based model, compared with numerical simulations where both agents behave like the virtual one (violet). The mean speed of the virtual fish is increased to  cm/s. (A–C) Condition C2: bidirectional interaction, where the virtual fish responds to the real one. (D–F) Condition C3: unilateral interaction, where the virtual fish behaves independently of the real one. (A, D) Horizontal inter-individual distance d. (B, E) Viewing angle for geometrical leader and follower. (C, F) Heading alignment between the real and virtual fish in the horizontal plane and vertical direction .

https://doi.org/10.1371/journal.pcbi.1014544.g007

Download:

• PNG
  larger image
• TIFF
  original image

Fig 8. Effects of reciprocal vs. unilateral interaction with a faster virtual partner on vertical motion.

Probability density functions (PDFs) of vertical behavioral variables for a real fish (red), a faster virtual fish (blue), and simulations of the 2R-based model (violet), using a higher mean virtual speed  cm/s. (A–C) Condition C2: social interactions enabled (bidirectional coordination). (D–F) Condition C3: social interactions disabled (unilateral interaction). (A, D) Swimming depth z. (B, E) Elevation angle . (C, F) Vertical inter-individual distance .

https://doi.org/10.1371/journal.pcbi.1014544.g008

As expected, in both C2 and C3 the real fish swims faster than in 2R, but not substantially faster than in C1. The speed distributions peak at  cm/s (Fig 6A) and  cm/s (Fig 6D), close to  cm/s (Fig 4A). However, high-speeds ( cm/s) are more frequent in both C2 and C3 than in C1.

In C2, the virtual fish exhibits nearly identical speed and depth PDFs as the real fish (Figs 6A and 8A), showing that both fish successfully track each other. Their distance to the wall and their wall-incidence angle converge towards values observed for two real fish (Fig 6B and 6C). These results indicate that the higher speed of the virtual fish enhances mutual responsiveness and improves bidirectional coordination compared with C1.

From the collective perspective, the distance between fish is considerably smaller in C2 (Fig 7A) than in C1 (Fig 4G), and the distribution of heading differences ( and ) is much more peaked around zero (Fig 7C), demonstrating higher alignment (see also S2 Video and S2 Fig). This suggests that the speed of the virtual fish modulates the intensity of alignment interactions. When the virtual fish swims faster, alignment between the two fish is higher. Moreover, the mean time separating a U-turn performed by one fish and that of its companion is smaller in C2 than in C1 ( s) and close to the 2R condition (S4 Fig).

In sharp contrast, when the virtual fish ignores the real one (Condition C3), the virtual agent behaves exactly as in the null model, but the real fish still reacts socially (see also S3 Video and S1D Fig). As a result, the distance between fish is significantly larger than in C2, yet remains much smaller than expected from the null model in the absence of social interactions (Fig 7D). The alignment between the real and virtual fish, although reduced relative to C1, is still more pronounced than in C2 (Fig 7F), meaning that the alignment between fish is higher when interactions are unidirectional.

Moreover, the unilateral social response in C3 causes the real fish to behave predominantly as a follower, doing so for 57.2% of the time, compared to 38.8% in C1 and 46.6% in C2. In this configuration, the real fish is more frequently positioned behind the virtual partner and follows its trajectory, resulting in similar headings and thus increased alignment (; solid red line in Fig 7E). This effect is further reinforced by the low turning rate of the virtual fish in C3, which performs very few U-turns.

To quantify alignment in a single scalar, we computed the average heading difference for each condition, obtaining 0.59 for 2R, 0.03 for C1, 0.29 for C2, and 0.35 for C3. These values confirm that the alignment is highest for two real fish, approximately the same in C2 and C3 (although slightly higher in C3), and markedly reduced in C1. We also calculated the U-turn rate of each fish (per 1000 frames), finding 1.8 for 2R, 4.3 (real) and 1.0 (virtual) in C1, 7.7 (real) and 4.8 (virtual) in C2, and 4.5 (real) and 0.4 (virtual) in C3. Due to the high turning rate in C2, for both the real and virtual fish, the alignment is easily disrupted despite that social interactions are in both directions, while in C3, U-turns are practically absent in the trajectory of the virtual fish, making it highly predictable, thus reinforcing the alignment of the follower.

4.1 Ethics statement

Experiments were approved by the Animal Experimentation Ethics Committee C2EA-01 of the Toulouse Biology Research Federation and were performed in an approved fish facility (A3155501) under permit APAFIS#27303–2020090219529069 v8 in agreement with the French legislation. All procedures were designed to minimize stress and handling. Fish were transferred from rearing tanks to the experimental setup with minimal manipulation. Each individual was used in only one one-hour experimental session per week. Swimming ability was monitored throughout; fish exhibiting impaired or absent swimming activity were excluded and replaced. No animals were sacrificed during this study.

4.2 Study species

Rummy-nose tetras (Hemigrammus rhodostomus) were purchased from Amazonie Labège in Toulouse, France. Fish were kept in 16 L aquariums on a 12:12 hour, dark:light photoperiod, at 27.7° C () and were fed ad libitum with fish flakes. The average body length of the fish used in these experiments is 3.1 cm.

4.3 Experimental setup

We used a closed-loop virtual reality system specifically developed to study real-time interactions between freely swimming fish and computer-generated virtual conspecifics. The virtual fish consisted of a realistic three-dimensional animated model of the rummy-nose tetra (Hemigrammus rhodostomus), reproducing the species’ body shape, coloration, and swimming kinematics. The model includes characteristic visual features such as the red head and striped tail. The virtual fish is rendered in real time using a Unity-based engine and projected onto the inner surface of the hemispherical tank. Crucially, the projection is computed using an anamorphic transformation that continuously adapts the size, orientation, and position of the image to the tracked position of the real fish. This ensures that the virtual fish appears as a spatially coherent object located within the three-dimensional environment of the tank, rather than as a distorted image on its surface. The rendering pipeline reconstructs, for each pixel of the projection, the direction of the corresponding light ray relative to the fish’s viewpoint, effectively simulating the visual scene from its perspective. This approach creates a consistent perceptual illusion of a freely swimming conspecific [23].

The setup consisted of a hemispherical acrylic bowl (diameter 52.2 cm, depth 14.6 cm, filled with 15 L of water) placed above a DLP LED projector that displayed the anamorphically rendered virtual fish onto the bowl wall. Fish movements were tracked from above using an Intel RealSense D435 depth camera positioned 47.9 cm above the water surface. Infrared illumination (8 IR lamps, 850 nm) enabled robust tracking under controlled light conditions. A dedicated workstation (Dell Precision 3640 with NVIDIA RTX 3070) processed all tasks in real time, including 3D tracking, trajectory simulation of virtual fish, and rendering.

The system included three software modules: a Python-based 3D tracking pipeline, a C++ trajectory simulator, and a Unity rendering engine. In this study, the system was used in closed-loop mode: the position and heading of the virtual fish were continuously controlled by the behavioral model parameterized for rummy-nose tetras, exploiting the current position and speed of the real fish. Real and virtual fish positions were exchanged via UDP at 30–90 Hz, ensuring precise, biologically relevant interactions. The same setup was also employed to study the behavior and interactions of pairs of real fish, using the same structured background and identical lighting conditions as in experiments involving one real and one virtual fish.

4.4 Experimental procedure

We conducted four series of experiments combining behavioral recordings of real fish pairs and closed-loop VR assays. In the first series, we tracked pairs of H. rhodostomus swimming freely in a confined 3D space to characterize their spontaneous motion and social interactions. Sixteen one-hour replicates were performed. The collected trajectories provided reference values for swimming speed, spatial positioning, and alignment between individuals, and were used to extract analytical functions describing self-propulsion, wall avoidance, and social interaction terms. These functions formed the basis of the behavioral model controlling the virtual fish. In subsequent experiments, a single real fish was introduced into the hemispherical VR tank and exposed to a virtual conspecific whose motion was governed in real time by this model. The position and heading of the virtual fish were continuously updated in closed loop based on the current position and velocity of the real fish. Three experimental conditions were tested, each defined by a distinct set of model parameters (S1 Table). Condition C1 corresponded to the reference parameter set reproducing the dynamics of two real fish swimming together (see S1 Video). Condition C2 used higher self-propulsion parameters to match the increased swimming speed observed in C1 (see S2 Video). Condition C3 used the same parameters as C2 but with all social interaction terms disabled, such that the virtual fish moved independently of the real one (see S1 Video). Eight one-hour replicates were performed under C1 and C3, and twelve under C2. Each trial began after a 10 min acclimation period.

4.5 Preprocessing of fish trajectories

Before analyzing the trajectories extracted from the experiments, the data must be preprocessed to ensure accuracy and to retain only periods in which fish are actively swimming. Time intervals during which a fish remains immobile for at least 4 seconds are discarded, as well as segments in which its instantaneous speed exceeds 25 cm/s or when it swims in a straight line at approximately constant speed for more than one second. These empirical thresholds were determined from the statistical properties of spontaneous swimming behavior and correspond, respectively, to periods of inactivity and to tracking artifacts such as sudden position jumps.

The dataset is then filtered to remove any frames in which the reconstructed position lies outside the bowl. To recover continuous temporal trajectories, gaps created during this filtering procedure are linearly interpolated whenever the duration of missing data is shorter than one second. For longer gaps, reliable reconstruction is not possible, and the trajectory is left discontinuous at that point.

The resulting trajectories therefore consist of multiple consecutive temporal segments. Each segment is subsequently smoothed by a Gaussian kernel convolution in order to reduce tracking noise while preserving the essential kinematic features of the motion,

(9)

where , such that the exponential kernel weight is negligible at i = n. The kernel width is set to h = 0.5 s (which is the typical duration of a kick arising from the burst-and-coast swimming mode of H. rhodostomus), which provides minimal smoothing while reducing high-frequency noise.

4.6 Quantification of individual and collective behavior in pair of fish

Fish predominantly swim in the horizontal plane at a preferred comfort depth. Throughout the manuscript, three-dimensional quantities are denoted with a superscript , horizontal (2D) quantities without a superscript, and vertical components with a subscript z.

The experimental bowl is modeled as the lower cap of a sphere of radius R₀ = 26.19 cm, filled with 15 liters of water, resulting in a central depth of  cm. The coordinate origin is placed at the water surface above the center of the bowl, with the z-axis oriented downward so that depth values are positive and increase with distance from the surface. At depth z, the radius of the corresponding horizontal cross-section is

(10)

which vanishes at , as expected.

The fish position and velocity are given by the vectors and . We define the distance to the bowl’s vertical axis as and the distance to the horizontal boundary at depth z as . For consistency with previous work, we refer to this horizontal boundary as the wall.

The horizontal velocity is , and the full 3D speed and its horizontal component are and , respectively. The fish heading in the horizontal plane is defined as , ranging from to , while the elevation angle is , ranging in when . Heading is used to compute the angle of incidence to the wall, , where is the positional azimuth. By convention, positive headings are counterclockwise, and positive (negative) elevation indicates upward (downward) motion.

When swimming with a conspecific, the relative configuration is defined by their spatial separation and angular alignment. For a pair of fish i and j, the 3D distance between them is , the horizontal distance , and the vertical distance . The viewing angle , defined as the angle fish i has to turn to point toward fish j, is computed as . Note that, in general, , meaning interaction geometry is not necessarily reciprocal.

We define the geometrical leader as the individual with the larger perception angle magnitude , while the other fish is considered the geometrical follower. Alignment differences in the horizontal and vertical planes are given by and .

Since only dyads are studied here, notation is simplified by omitting indices when no ambiguity arises, writing instead of .

4.7 3D self-propelled particle model for fish swimming in a bowl

The horizontal velocity of a fish is decomposed into components parallel and perpendicular to its instantaneous heading. Defining the unit vectors

the velocity in the horizontal plane is . The horizontal acceleration is written as , where and are the longitudinal (speed-changing) and lateral (turning) acceleration components, respectively. Together with the vertical acceleration , these variables fully determine the temporal evolution of fish position.

Expressing each force as the product of a scalar with the unit vector of its direction, the components of fish’s acceleration in the horizontal plane are as follows:

(11)

where , with , is the position angle of fish i in the horizontal plane and is perpendicular to the wall closest to the fish in the horizontal plane, and , with , is oriented along the direction from i to j in the horizontal plane.

Using and , the vectors and can be expressed in the fish frame as and .

In the intrinsic frame of fish i, this gives the scalar components of the acceleration:

(12)

(13)

Using explicitly the decomposition of the interactions into products of single-variable functions, the final system reads

(14)

(15)

(16)

where is the average speed of the neighbor(s) of i, which for N = 2 fish becomes , i = 1,2.

The analytical expressions of the different functions arising in these equations of motion (5 polynomial expansions, 8 Fourier series, and 5 simple fits of tabulated functions), used in the reconstruction procedure and implemented in the model, are given in Supplemental Material.

4.7.1 Stochasticity.

Behavioral fluctuations are modeled as autocorrelated noise processes in the parallel, perpendicular, and vertical directions. Noise components are defined in the fish-centered reference frame and converted into Cartesian coordinates via

They evolve according to a discrete Ornstein-Uhlenbeck process with time-step index n,

where , , are independent random numbers normally distributed, generated with the Box-Muller transform: , . The parameters corresponding to the Ornstein-Uhlenbeck dynamics are given by

where , , and set noise amplitudes, and , are correlation times in the horizontal and vertical directions, respectively, that we took equal in practice. The occurrence of finite correlation times for the noises is consistent with the true burst-and-coast nature of the fish swimming mode, where a fish alternates a very brief acceleration period (of typical duration 0.1 s) with a passive gliding period lasting typically 0.5 s, where the fish barely changes its heading.

4.7.2 Boundary rejection rules.

In this agent-based model, (x,y,z) denotes the center of mass of the fish. Since fish have a finite body length, the predicted position at step n + 1 may place the head outside the water volume. Such moves are rejected.

The position of the head is obtained from , , and , where is the head-center distance.

Two rejection cases are implemented:

• Collision with the wall: If the head lies beyond the wall surface, , the previous position is restored, and the fish is reoriented to swim nearly tangentially to the wall (adding a small random angle of order radian):

(17)(18)

where . This ensures , i.e., near alignment with the wall.

• Exiting the water surface: In the very rarely observed cases where the head is predicted to lie above the surface or below the bowl bottom, the depth is restored and the vertical speed is reversed: and .