Fig 1.
Communication between different parts of a Stytra experiment. Each color represents a separate process in which the module(s) are running. Data flow between modules within one process is depicted by arrows, and between processes as double arrows. The classes belonging to the data flow elements are displayed in monospace. A more comprehensive diagram of the classes is provided in S1 Fig. The user interface, the stimulus update and related functions such as the screen calibration and data saving are performed in the main process, colored in green. The stimulation can be triggered by a triggering process (in orange) that listens for an external triggering signal. Frames can be acquired from a camera process (in blue), analyzed by a tracking function (in purple), and the result can be streamed to the main process for data saving and used in closed-loop experiments via the estimator.
Fig 2.
Screen capture of the software in use.
The various behavioral paradigms supported by Stytra provide the user with a consistent interface to control experiments. The toolbar on top controls aspects of running the experiment, a camera panel shows the tracking results superimposed on the camera image, a calibration panel enables quick positioning and calibration of the stimulus display and a monitoring panel plots a user-selected subset of experimental variables.
Fig 3.
Head-restrained tail tracking in Stytra.
A) The image is first pre-processed by inverting, down-scaling, blurring and clipping, resulting in the image on the right, where the fish is the only object brighter than the background. Then, tail tracing starts from a user-defined point, and in the direction determined by another user-defined point at the end of the tail at rest. For each segment, a square (outlined in white) in the direction of the previous segment (yellow) is sampled, and the direction for the next segment is chosen as the vector (red) connecting the previous segment end and the center of mass of the sampled square (blue). B) A heatmap showing the angles of the tail segments from the start to the end of the tail during a bout, and a trace representing the cumulative curvature sum from a behaving animal. The total curvature is just the difference in angle between the first and last tail segment (adding up angle differences between all segments, only these two terms remain).
Fig 4.
A) Eyes are detected by fitting an ellipse to the connected components of the image of the fish head after thresholding. B) Example trace of eye motion in response to a full-field rotating background.
Fig 5.
Example bouts tracked from freely-swimming fish.
From left to right: trajectories of bouts in different directions, the velocity magnitude and the total angle change during the course of the bouts. In the left-most panel, all trajectories were realigned such that the initial position and orientation of the fish were the same. The data was sampled at 300 Hz.
Fig 6.
Screenshot of DeepLabCut-based rat tracking in Stytra.
On the left, the 4 detected keypoints (snout, two ears and tail base) in red are superimposed on the video. On the right, traces tracking the coordinates of the animal are displayed, along with a parameter of of a closed-loop stimulus (a circle that would be tracking a rat). The video displayed was provided with the DeepLabCut repository [10].
Fig 7.
Dynamic update of the stimulus in a closed-loop assay for the optomotor response. From top: open-loop velocity of the gratings moving caudo-rostrally below the fish; cumulative tail angle (see the tail tracking section and Fig 3 for details); bout vigor, estimated by calculating the instantaneous standard deviation of the angle sum in a 50 ms window; final closed-loop velocity of the gratings, with backward movements induced by the fish swimming.
Fig 8.
Hardware for zebrafish behavior experiments.
A) Above: sample image of a behavioral setup that can be used to track head-restrained zebrafish tail end eyes (the opaque enclosure has been removed for visualization purposes). Below: sample traces for tail angle and grating velocity obtained from this setup with the closed-loop experiment described in Fig 7. B) A low-cost version of the setup presented in A) that can be used to investigate behavior in the head-restrained fish, and sample traces from this setup. A detailed description of the setup together with a complete list of parts can be found at www.portugueslab.com/stytra/hardware_list.
Fig 9.
Closed-loop protocol and simultaneous whole-brain calcium imaging.
A) A protocol consisting of either open- or closed-loop forward-moving gratings was presented to a seven day old Tg(elavl3:GCaMP6f) zebrafish larvae during two-photon imaging. The arrowhead points to the timepoint of receiving the trigger signal from the microscope. Colored stripes indicate periods when the gratings were moving: dark gray represents open loop trials (gain 0) and light gray represents closed-loop trials (gain 1). B) Left: Pixel-wise correlation coefficients with the grating velocity regressor. The square on the regressor map reports the position of the area that was used to compute the calcium trace displayed on the right. Right: z-scored fluorescence trace from the selected area, imposed over the regressor trace. C) Same as B, for the vigor regressor.
Fig 10.
Visual feedback changes inter-bout interval in a head-restrained optomotor assay.
Replication within Stytra of results published in [13]. A) Changing the gain that is used to convert the fish’s swimming vigor to relative velocity with respect to the grating affects the inter-bout interval. The line represents the average normalized inter-bout time, and bars represent standard error of the mean from n = 28 larvae (adapted from [13]). B) Replication in Stytra of the same experimental protocol (n = 24 larvae). Individual fish traces are shown in gray.
Fig 11.
Comparison of turning angle distribution in a closed-loop freely-swimming phototaxis experiment.
Left: a histogram of the angle turned per bout, redrawn from [20]. Right: the equivalent panel, with n = 10 fish and the protocol run with Stytra. The dark shading on the plot represents the dark side of the visual field.