Fig 1.
Schematic of analysis pipeline to segment and track neurons through time and extract their neural activity in a deforming brain.
Neurons are labeled with calcium insensitive red fluorescent proteins, RFP, and calcium sensitive green fluorescent proteins, GCaMP. Videos of the animal’s behavior and volumetric fluorescent images of the animal’s brain serve as input to the pipeline. The algorithm detects all neurons in the head and produces tracks of the neural activity across time as the animal moves.
Fig 2.
(A) Example images from all four video feeds from our imaging system. Both scale bars are 100μm. Fluorescent images are shown with false coloring. (B)A schematic illustrating the timings from all the devices that run in open loop in our imaging setup. The camera that collects high magnification images captures at 200Hz. The two low magnification images capture at 60Hz, and the focal plane moves up and down in a 3 Hz triangle wave. The cameras are synchronized post-hoc using light flashes and each image is assigned a timestamp on a common timeline.
Fig 3.
Straightening and segmentation.
(A) Centerlines are detected from the low magnification dark field images. The centerline is shown in green and the tip of the worm’s head is indicated by a blue dot. (B) The centerline found from the low magnification image is overlaid on the high magnification RFP images. The lines normal to the centerline, shown in blue, are used to straighten the image. All scale bars are 100 μm. (C) A maximum intensity projection of the straightened volume is shown. Individual neuronal nuclei are shown (D) before and (E) after segmentation.
Fig 4.
Schematic of Neuron Registration Vector Encoding.
(A) The registration between a sample volume and a single reference volume is done in several steps. I. The image is segmented into regions corresponding to each of the neurons. II. The image is represented as a Gaussian mixture, with a single Gaussian for each segmented region. The amplitude and the standard deviation of the Gaussians are derived from the brightness and the size of the segmented regions. III. Non-rigid point-set registration is then used to deform the sample points to best overlap the reference point-set. IV. Neurons from the sample and the reference point-sets are paired by minimizing distances between neurons. (B) Neuron registration vectors are constructed by assigning a feature vector vi,t to each neuron xi,t in a sample volume xt by performing the registration between the sample volume and a set of 300 reference volumes, each denoted by rk. Each registration of the neuron results in a neuron match, , and the set of matches becomes the feature vector vi,t. (C) Vectors from all neuron-times are clustered into similar groups in a two step process: Hierarchical clustering (illustrated in the figure) is performed on a subset of neurons to define clusters, each of which is given a label Sn. Then each feature vector vi,t is assigned to a cluster based on a distance metric (not illustrated). (D) The clustering of the feature vectors shown in (C) assigns an identity to each of the neurons in every volume. This allows us to track the neurons across different volumes of the recording.
Fig 5.
Example of consensus voting to correct a misidentified neuron.
In volume 735, neuron #111 is found successfully and is indicated in green. In volume 736, however, the neuron is misidentified, shown in red. During the correction phase, all other time points vote for what the position of neuron #111 should based on a thin-plate spline deformation. A sample of votes are shown (blue ‘x’). Since the initial estimate of the position is far from the majority of consensus votes, a corrected position is assigned to be the centroid of the votes weighted by image intensity. This process is repeated to correct any errors for every neuron at every time.
Fig 6.
Comparison of the automated Neuron Registration Vector Encoding algorithm with manual human annotation.
A previously published 4 minute recording of calcium activity (strain AML14) was annotated by hand, [10]. (A) Spheres show position of neurons that were detected by the automated algorithm. Grey indicates a neuron detected by both the algorithm and the human. All neurons detected by the human were also detected by the algorithm (70 neurons). Red indicates neurons that were missed by the human and detected only by the algorithm (49 neurons). (B) Histogram showing number of neurons that were mismatched for a given fraction of time-volumes when comparing automated and manual approaches. Only those neurons that were consistently found by both algorithm and human were considered. An automatically identified neuron was deemed correctly matched for a given time-volume if it was paired with the correct corresponding manual neuron.
Fig 7.
Neural activity traces from 156 neurons in the brain a C. elegans as it freely moves on an agarose plate for 8 minutes (strain AML32). The neural activity is expressed as a fold change over baseline of the ratio of GCaMP6s to RFP for each neuron. The behavior is indicated in the ethogram. On the right is the locations of all of the detected neurons (the head of the worm is towards the top of the page). The neurons that have significant correlation with reverse locomotion are indicated in red. White gaps indicate instances where neurons failed to segment. This is a newly acquired recording, different from that in Fig 6.
Table 1.
Breakdown of computation time and scalings for Neuron Registration Vector Encoding pipeline.
nframes is the total number of low magnification images used to detect centerlines, nvol is the total number of volumes in the recording, nref is the number of reference volumes used for creating feature vectors, nneurons is the total number of neurons detected, and nsubset is the number of neurons in the subset of volumes used for initial clustering.