^{1}

^{1}

^{2}

^{3}

^{1}

^{2}

The authors have declared that no competing interests exist.

Conceived and designed the experiments: MG YRC. Performed the experiments: KG. Analyzed the data: KG MG YRC. Contributed reagents/materials/analysis tools: KG MG YRC. Wrote the paper: KG MG YRC.

How simple is the underlying control mechanism for the complex locomotion of vertebrates? We explore this question for the swimming behavior of zebrafish larvae. A parameter-independent method, similar to that used in studies of worms and flies, is applied to analyze swimming movies of fish. The motion itself yields a natural set of fish "eigenshapes" as coordinates, rather than the experimenter imposing a choice of coordinates. Three eigenshape coordinates are sufficient to construct a quantitative "postural space" that captures >96% of the observed zebrafish locomotion. Viewed in postural space, swim bouts are manifested as trajectories consisting of cycles of shapes repeated in succession. To classify behavioral patterns quantitatively and to understand behavioral variations among an ensemble of fish, we construct a "behavioral space" using multi-dimensional scaling (MDS). This method turns each cycle of a trajectory into a single point in behavioral space, and clusters points based on behavioral similarity. Clustering analysis reveals three known behavioral patterns—scoots, turns, rests—but shows that these do not represent discrete states, but rather extremes of a continuum. The behavioral space not only classifies fish by their behavior but also distinguishes fish by age. With the insight into fish behavior from postural space and behavioral space, we construct a two-channel neural network model for fish locomotion, which produces strikingly similar postural space and behavioral space dynamics compared to real zebrafish.

Behavior is a direct reflection of neural activity and its modulation by external stimuli. Many tools are available to study behavior—for example, electrophysiological techniques to probe neural circuitry [

The zebrafish (

Extensive studies have categorized the behavioral patterns of zebrafish [

Recent work [

In the present work, we apply similar approaches to zebrafish to understand the behavioral patterns in their free swimming. The key steps are summarized with actual data in

(A) Snapshots of a larval zebrafish free swimming movie in which the fish backbone is fitted to a 10-point spline. (B) A linear combination of three “eigenshapes” accurately reconstructs the backbone shapes of the zebrafish. (C) A swimming bout is represented as a trajectory in the “postural space” spanned by the three eigenshapes. (D) A “behavioral space” generated by multi-dimensional scaling reduces each cycle of a trajectory to a point, and clusters them by their similarity. (E) A 2-channel neuro-kinematic model is constructed based on the observed behavioral patterns and evaluated using the same work flow.

How can one quantify the free swimming behavior of zebrafish without _{i}.

(A) Still images of a swim bout from a representative movie of free swimming zebrafish larva, recorded at 500 fps. (B) Spline fit of fish backbone (cyan). Tangent vectors (black arrow) at 10 evenly spaced segments along the backbone from head (_{0} = 0) to tail (_{9} = 9) point along a direction _{j}). (C) Swim bout from A parameterized by Δ_{j},_{i}) = _{j},_{i})–_{i}). (D) Singular value decomposition of Δ_{k}(_{j}) (_{kk} are the singular values) of each eigenfunction _{k}(_{j}). The right axis in cyan shows the cumulative contribution of each eigenshape. The first three eigenshapes contribute 96% of the total variance in Δ

Next, we applied background subtraction algorithms and threshold procedures to the movies to extract the backbone shape of each fish as a sequence in time (see _{j},_{i}), measured along the normalized arc lengths of the fish from head (_{0} = 0) to tail (_{9} = 1). All spine angles are given relative to the head angle, Δ_{j},_{i}) = _{j},_{i})–_{0},_{i}).

Δ_{j},_{i})is an _{j},_{i}) was then decomposed into a set of _{k} by SVD [_{k}(_{i}) represents the amplitude of the _{k}(_{j}) at each time point _{i}. _{kk} is an _{kk} is conventionally normalized to 1, such that each singular value represents the fractional contribution a basis function makes to the overall swimming behavior. The basis functions are sorted from most important (largest singular value; _{kk} may be small, and thus many basis functions can be left out of the sum in _{j},_{i}) ^{r}(_{j},_{i}) using the first three eigenshapes (see

Plotting the three eigenshapes (light blue, red, and green in

Based on this analysis method, any zebrafish free swimming bout can be represented as a trajectory in the low-dimensional postural space spanned by the three collective eigenshapes described above. _{i} of the movie, the zebrafish backbone shape is represented by a set of three amplitudes {_{k}(_{i})} with _{k} vs. time, and in

(A) Still images of a representative turning bout during free swimming. As discussed in the text, it is convenient to divide the bout into a “turn” region (_{1}(_{2}(_{3}(_{1}(_{2}(_{3}(_{1}(_{1}(_{2}(_{3}(

Previous studies have categorized free swimming of zebrafish into two behavioral states, the “R-turn” and “scoot”, based on a limiting angle of the backbone [_{2} and _{3}, returning to a fixed point corresponding to a “resting” fish. In contrast, the turn bout in _{1}–_{3} plane lasting between one half and one whole oscillation of cycle 1, before shifting to cycles 2–4 in _{2} and _{3} similar to those in the scoot trajectory (see cycles 1–5 in _{1} corresponds to the direction of the turn, with _{1} < 0 corresponding to a right turn and _{1} > 0 to a left turn, and we consider these cases to be mirror images of the same behavior. To distinguish the different cycles of a trajectory in postural space, we found it useful to name the excursion in the _{1}–_{3} plane “turn region”, the oscillation cycles in the _{2}–_{3} plane “scoot region” and finally decaying to zero amplitude as “resting region.” Thus a swimming bout can progress through these regions in postural space in succession. Time is color coded from black to magenta on the trajectory in

How does this approach extend to an ensemble of swimming bouts? To determine if multiple trajectories can be similarly categorized, we compared trajectories from different fish, normalizing their time axes by matching the duration of the first oscillation period in _{1}

We next used multidimensional scaling (MDS) [_{αβ} between every pair of postural space trajectories

(A–B) Ensemble of swim bout trajectories (_{1}, _{2}, and _{3} vs. normalized time of the trajectories in A and B from younger larvae. (D) Same plot for older larvae. Shaded areas demarcate each cycle of the trajectory. (E–F) Postural space representation of the trajectories in C and D, respectively. Throughout, color represents the location in the behavioral space and cycle number: in the RGB colormap, red and green channels correspond to position along MDS dimension 1 and 2, respectively, and the blue channel to the cycle number.

In our analysis of an ensemble of swimming bouts, three dimensions (not to be confused with the three axes of postural space) captured ~85% of the behavioral variability contained in the set of all trajectories (

The behavioral space in

These observations confirm the qualitative picture formed from individual trajectories. Fig _{3} in the first cycle, as seen by inspection. Although the contribution of _{3} to the zebrafish shape is less than the other two modes (<5%, see _{1}–_{3} (a so-called “turn region”) and others (green) lying mostly along the _{2}–_{3} plane (a “scoot region”).

The behavioral space also reveals differences between age groups. In _{3} in cycle 1 in comparison to younger fish; their turn interval (red trajectories) is considerably more bent than that of the younger larvae, indicating a systematic difference in shape. Nevertheless, although fish of different age seem to adopt different postures, they undergo the same temporal sequence of patterns in the behavioral space (

Having established a robust, parameter-free method for analyzing zebrafish behavior, can we reproduce the observed patterns using a simple model of free swimming? We surmised that a simple neural network may suffice in describing the simple behavioral patterns observed. Constructing a network with a channel controlling the left and the right of the organism, we anticipated that signals of identical amplitude to both channels would produce a symmetric fish spine oscillation (i.e. a scoot) while unequal amplitudes would produce a turn in one direction. To test this idea, we adapted a coarse-grained neuro-kinematic model of

Our version of the neural network divides the fish backbone into a right and left half, each with 10 equal neural segments (_{j}) of neurons interconnected by appropriate synapses (see _{osc}(_{j}, _{i}) in the form of a train of sharp pulses whose firing times _{f}, amplitudes _{m}(_{j}, _{i}) at each segment on the left and right. From this force function, we determined the radius of curvature _{j},_{i}) = _{m}(_{j},_{i})/_{j}) of the zebrafish backbone at each segment position, where _{j}) is the fish stiffness along its spine. Finally we calculated the tangential angles Δ_{j},_{i}) by integrating ^{-1}(_{j},_{i}) = |_{j},_{i}) on which we could perform the same dimensionality reduction analysis detailed above.

(A) Schematic of neuro-kinematic model depicting the fish backbone divided into ten segments on the right and left sides. (B) Spike train generated by an optimized neural model. The right and left spike trains are shown in red and green, respectively. The height of each spike represents the amplitude _{f} is the firing time for each spike in the head segment. The segment-to-segment delay

Using a genetic algorithm, we optimized the model parameters _{f}, _{j}) against two experimental data traces (see _{j}) had to decrease monotonically from the head toward the tail, indicating greater flexibility towards the tail, then increase again over the last tail segment (

(A) Still images from a free swimming zebrafish movie. Each snapshot shows the fish backbone reduced to a thin skeleton fitted to a cubic spline fit (cyan) and obtained from the neural simulation (red), respectively. The neural model was optimized against the experimental data as described in _{k}(_{j}). The eigenshapes from the neural model of a turn (dashed light red, blue, and green lines) match those from experimental data of a turn (solid red, blue, and green). (C) Bar plot of % weights of each eigenshapes _{k}(_{j}) (_{j})) obtained after optimization of neuro-kinematic model against experimental data. (E) Simulated neuro-kinematic model trajectory of a turning bout in postural space.

We also tested the robustness of the model by adding varying levels of white Gaussian noise to the model parameters _{f}, and _{f}. Not surprisingly, noise that significantly altered the timing pattern between the left and right sides of the organism affected the cyclic motion required for free swimming. A behavioral space was constructed to compare the simulated trajectories with varying signal-to-noise ratio. _{f}, respectively, whereas the trajectories with the highest noise overlapped with those with the lowest noise in parameter

Prior studies of zebrafish locomotion applied boundary criteria to predefined parameters to quantify proposed behavioral states. In the current analysis, the shapes of the fish backbone themselves generate a basis set to describe their motion and reveal which basis functions contribute the most to that description. Drawing from studies of

It may be surprising at a first glance that fish larvae require only three basis functions to describe their backbone shapes, whereas worms analyzed in [

The behavioral space in

A low-dimensional postural space affords a direct visual comparison between real behavior and neural network models, as seen by comparing Figs _{2}–_{3} plane, while turns involve _{1}. It also shows the bent-ellipse-like distortion of the turn interval the _{3} direction, although it exaggerates it compared to real behavior. Such deviations are more obvious in behavioral space than by comparison to actual free swimming movies (

A key observation from the neuro-kinematic model is that increasing the amplitude

All experimental procedures in this study were approved by the University of Illinois Institutional Animal Care and Use Committee protocol # 13327.

All experiments were performed on wild-type AB genotype zebrafish (Danio rerio) larvae age 7–10 dpf (days post fertilization). The larvae were raised without food until 6 dpf and were fed food every day (Hatchfry 0, Argent Labs) before taking data. These larvae were obtained from breeding of wild-type zebrafish adults. Zebrafish were maintained in a Z-hab mini system (Aquatic habitats, Beverly, MA) fish facility at 28.5°C on a 14h:10h light:dark cycle. The embryos were obtained from adult fish breeding and were raised at 28.5°C in 10% Hanks solution [

We imaged free swimming larvae using a high speed camera (IDT vision N-3) mounted to a stereo microscope (Edmund Optics 6V head + 10X eyepiece). This allowed us to image of the entire 21-mm diameter Petri dish in which the larvae swam. A halogen light source (Edmund Optics MI-150 high-intensity illuminator) was used to illuminate the sample, and its light passed through a series of long-pass filters (780 nm and 830 nm) in order to obtain IR wavelengths (>810 nm). IR light is preferable to visible light since the larva cannot detect it [

For each experiment, a single 7–10 dpf fish larva was placed in a 21-mm Petri dish in 10% Hanks solution at room temperature. The larvae were illuminated from the bottom with IR light. We recorded videos of a free swimming larva at 500 fps using a high speed camera (Diagnostic Instruments) and the Motion Studio Software Suite. Each video typically had 4–5 swimming episodes ~250 ms with intermittent pauses. There were a total of 18 videos from 7–8 dpf and 21 from 8–10 dpf old fish. All videos were recorded when larvae were swimming in the center of the Petri dish, away from the dish edges that obscure the fish and making backbone shape extraction difficult. We used the frame difference method

All the zebrafish free swimming movies were analyzed using the image and video processing toolbox of MATLAB. A stepwise image segmentation algorithm was followed. The images were first preprocessed using a customized background subtraction algorithm. Next, the image was thresholded to 1 bit (black/white) to yield a fish outline. A skeletonization algorithm was applied to find the center of the outline, corresponding to the backbone. A cubic spline was then fitted to n points evenly sampling the fish backbone along the backbone arc length. Each step is explained in

Each fish swimming bout was of 180–200 ms duration which accounts for 90–100 frames in total. We took 140 frames in total, including the resting behavior preceding and following the swimming bout of the fish. We analyzed a total of 115 swimming bouts from 20 different fish. We performed singular value decomposition as described above on the matrix Δθ(s_{j},t_{i}) containing m = 16100 total frames from the catenation of all recordings of free swimming fish. To analyze multiple fish trajectories together, the time traces of the amplitudes {_{k}(_{i})} (with k = 1, 2 and 3) were scaled and shifted in time to maximize the overlap between trajectories (_{αβ} between pairs ^{α,β}(_{j},_{i}). Based on

We used a modified version of the neuro-kinematic model described in [_{f}, stimulus amplitudes _{c}, was based on our experimental values (6–8 half cycles observed in each swimming bout). The three model parameters were adjusted by a genetic algorithm to minimize the difference between experimental and simulated trajectories Δ_{j},_{i}). The genetic algorithm used 50 family members with constraints on each variable obtained from eigenshape data in each generation and typically converged after ~50 generations. The step-wise algorithm for optimization is explained in

(DOCX)

(A) A user-selected region of interest (ROI) around the fish in the first frame of a movie (blue polygon) is used to create a mask. (B) Background (

(TIF)

Each line on the plot is the cumulative contribution of each eigenshape (as in

(TIF)

The difference in residual error is insignificant for more than 3 eigenshapes.

(TIF)

(A) Snapshots of a movie of a free swimming zebrafish recorded at 500 fps. (B) Shown are the real-space shapes (gray lines) corresponding to the basis function _{k}(_{j}) where

(TIF)

(A) Still images of a representative scooting bout (from 50–250 ms) during free swimming. (B) Plot of the amplitudes _{1}(_{2}(_{3}(_{1}(_{2}(_{3}(_{1}(_{1}(_{2}(_{3}(

(TIF)

(A) The amplitudes _{1}, _{2}, _{3} of each eigenshape _{k} (_{1}, respectively (orange band). (B) Histogram of cycle 1 period for all trajectories. See

(TIF)

(A) Maximum relative error between the dissimilarity matrix

(TIF)

(A) Behavioral space in MDS dimensions 1 and 2 from

(TIF)

(A) _{1} = 275 and σ_{2} = 80 along MDS dimensions 1 and 2 were used, respectively).

(TIF)

(A) Behavioral space in MDS dimensions 1 and 2 from

(TIF)

(A) A schematic of a neural model depicting the fish backbone divided into ten segments on either side of the backbone. (B-C) Examples of Δ_{j},_{i}) for a turn and a scoot trajectory, respectively. The zero crossings of Δ_{j},_{i}) are labeled at

(TIF)

(A) Schematic of neuro-kinematic model depicting the fish backbone divided into ten segments on the right and left sides. (B) Spike train generated by an optimized neural model. The right and left spike trains are shown in red and green, respectively. The height of each spike represents the amplitude

(TIF)

(A-C) Response of the neural model to white Gaussian noise added to the model parameters

(TIF)

Contains additional information on data analysis methods, and details on the neural network model.

(DOCX)

Video of a freely swimming larva in a petri dish recorded at a rate of 500 frames per second (fps) using a high-speed camera. The video shows a total of 3 swimming episodes with 4 intermittent pauses (resting). The playback rate is 100 fps.

(MP4)

The left side of the movie shows free swimming larva at 10 fps and a spline curve in cyan fitted to its backbone. The right side shows the corresponding trajectory in postural space. The dot shows the contribution of amplitudes _{1}(_{2}(_{3}(_{1}, _{2}, and _{3}, respectively, for the corresponding frame on the left. In postural space, the bout involves a turn region (

(MP4)

The left side of the movie shows a free swimming larva (at 10fps) and a spline curve in cyan fitted to its backbone. The right side of the movie shows the corresponding trajectory in postural space. The dot shows the contribution of amplitudes _{1}(_{2}(_{3}(_{1}, _{2}, and _{3}, respectively, for the corresponding frame on the left. In postural space, the bout involves a scoot region (

(MP4)

The movie shows the trajectories from a population of younger fish in postural space. The animation sweeps over elevation angle to show how trajectories occupy a continuum between two extremes—turn-like trajectories and scoot-like trajectories—rather than clustering into distinct behavioral states. For details on the colormap see

(MP4)

The movie shows the trajectories from a population of older fish in postural space. The animation sweeps over elevation angle to show how trajectories occupy a continuum between two extremes—turn-like trajectories and scoot-like trajectories—rather than clustering into distinct behavioral states. For details on the colormap see

(MP4)

The left side of the movie shows a free swimming larva recorded at a rate of 500 fps and a spline curve in cyan fitted to its backbone. The right side shows the same free swimming larva and a spline curve obtained using the neural network model described in the text. The movie has two swimming episodes (a turning bout followed by a scooting bout) with one intermittent pause. The colored marker represents the tail position at each time point. The playback rate is 10 fps.

(MP4)

The authors acknowledge helpful discussions with Prof. Mark Nelson.