Figure 1.
Varying response latencies and context dependent neural coding.
(A) Varying latencies. Sequence of 8 dimensional white noise stimuli (e.g. successive frames on a one dimensional screen with 8 pixels). An LNP model generates spikes (black bars) if a chunk of stimulus (dashed rectangles) is similar enough to its receptive field (dashed rectangles). Jitter-free or ideal spikes (vertical black bars, ‘ideal spiking’) are produced with some fixed latency (dashed diagonal lines). Jittered spikes (black bars, ‘observed spiking’) are produced by randomly jittering ideal spikes (gray bars) forward or backward in time (green arrows). The jitter of adjacent spikes can be independent or correlated. The jittered spikes are the basis for fitting neural response models. (B) Receptive field (RF) estimates using spike triggered stimulus averaging (STA) on unjittered spikes (true RF), jittered spikes (STA), and the MPH on jittered spikes (MPH). Noisy response latencies lead to blurring of STA RFs, but not of MPH RFs. (C) State-dependent coding. For the same white noise stimulus, spikes are generated from one of two LNP models depending on hidden states I and II (green lines) determining which model is used. (D) The true RFs are superimposed when estimated with STA. A two-states MPH can faithfully recover the two RFs.
Figure 2.
Two minimal MPHs for flexible timing and context dependent coding.
(A) Architecture of the minimal MPH that allows for neural codes with varying latencies, i.e. flexible timing. This MPH has 3 hidden states, one X-state that models only the stimulus, one R-state that models only the neural response, and one M-state that jointly models stimulus and response. The probability distributions over stimuli (bottom) are illustrated as low dimensional projections (stimulus dimension 2 coincides with the receptive field of the M-state). (B) Hidden state sequences in that model correspond to paths in the alignment matrix: a diagonal step leading into position implies that stimulus and response at times
and
are jointly modeled by an M-state, a horizontal step implies modeling of only the stimulus at time
, and a vertical step implies modeling of only the response at time
(deviations from the diagonal reflect jittered spikes detected by the model). Depicted stimulus and spiking responses are from figure 1A. (C) The minimal MPH for modeling state-dependent neural codes. The MPH can switch between several M-states, each of which represents a different RF. The (projected) stimulus distributions given a spike (spike triggered stimulus ensemble) are centered on the respective RFs (indicated by black arrows). (D) Adding states to the model turns the alignment matrix into an alignment tensor composed of several planes (strictly speaking, B depicts a tensor as well; we just projected all the states onto one plane). The switch from state 1 to State 2 is indicated (green arrow).
Figure 3.
The MXR-MPH applied to white noise stimuli and spike-time-jitter.
(A) A white noise stimulus (top) with spiking responses (black bars) generated by an LNP-type model neuron (LNP output, the LNP RF size is indicated by the black rectangle). The jittered versions (jittered) of the LNP spike trains with corresponding firing rate (thick gray line) are shown below. The MPH estimate of firing rate (black full line) is more accurate than the STA estimate (dashed line). (B) Applied spike jitter is i.i.d. among spikes and log-normally distributed with zero mean (3 different jitter distributions are shown; they differ in terms of variance and symmetric/asymmetric shape). Results for the jitter kernel with variance
are shown in panels A, C and D. (C) RFs estimated through STA on unjittered spikes (true RF), STA on jittered spikes (STA), and MPH on jittered spikes (MPH). The STA RF is blurred whereas the MPH RF closely resembles the true RF. Dotted black lines indicate the midpoints of the RFs. (D) Projections of all stimuli (gray lines) and the spike triggered stimulus ensembles (black lines) onto the underlying (true) RF for the unjittered spikes (left), the jittered spikes (middle), and the MPH reconstruction (right, obtained via dynamic alignment using the generalized Viterbi algorithm). (E) Response prediction. To evaluate the models we computed correlation coefficients (CCs) between predicted and actual firing rates on the validation set and for different jitter variances. For small spike jitter, performances of STA and MPH are comparable. As the jitter magnitude increases, STA performance drops much more severely than does MPH performance. Also shown is an upper bound for the CC computed by sampling and cross-correlating jittered responses. (F) MPH robustness to jitter is demonstrated also when assessed as similarity between the estimated RF and the true RF (similarity computed as normalized scalar product, i.e. cosine of angle between RFs). (G) We assessed the influence of different non-linearities (labeled A–E, ordered by steepness) on prediction quality for both the MPH as well as the cascaded MPH (cMPH). (H) Shallow non-linearities decrease the upper bound of prediction quality (black line) as well as the MPH (red lines) and STA (green line) performance for the unjittered (left) and the jittered case (right). The cascaded MPH (red line) shows slight improvements over the non-cascaded one (dotted red line).
Figure 4.
The MPH applied to natural stimuli and jittered spike responses.
(A) An example log-spectrogram of zebra finch song (top, high sound amplitudes in red and low amplitudes in blue), spiking responses generated by an LNP-type model (middle, LNP output), their jittered versions (below), and the corresponding jittered firing rate (bottom, gray line). The MPH-predicted response (MPH, full line) of the jittered firing rate is more accurate than the reverse correlation prediction (RC, dashed line). (B) Applied spike jitter is i.i.d. among spikes and log-normally distributed with zero mean. Two different jitter distributions are shown, they differ in terms of variance and symmetric/asymmetric shape (gray curves
left, and
right). The MPH-estimated jitter kernels are shown in black. The MPH misses some jittered spikes (right), as revealed by the excessive peak at zero time lag. Results for the jitter kernel with variance
are shown in panels A, C, and D. (C) RFs estimated through reverse correlation for unjittered data (true RF), jittered data (RC) as well as the MPH receptive field estimate (MPH). The STA RF is blurred whereas the MPH RF closely resembles the true RF. Dotted black lines indicate the midpoints of the RFs. (D) Projections of all stimuli (gray lines) and the spike triggered stimulus ensembles (black lines) onto the underlying (true) RF for the unjittered spikes (left), the jittered spikes (middle), as well the MPH reconstruction (right, obtained via dynamic alignment using the generalized Viterbi algorithm). (E) Correlation coefficients (CCs) between predicted and true firing rates on the validation set for different jitter variances. Also shown is an upper bound for the CC computed by sampling and cross-correlating jittered responses. For small overall jitter, performances of reverse correlation and MPH are comparable. As the overall jitter magnitude increases, reverse correlation performance drops much more severely than does MPH performance. (F) RC performance drops even stronger when assessed in terms of similarity between the estimated and the true RFs.
Figure 5.
The MPH applied to white noise stimuli and switched responses.
(A) A white noise stimulus (top), the randomly switched states of a switching LNP model (middle, black curve), and the observed spike train (middle, black rasters) and firing rate (bottom, gray line). The MPH-predicted firing rate (bottom, black line) to a test stimulus is closer to the observed firing rate than is the STA prediction (blue line) or the STC prediction (dotted green line). (B) The MPH RF estimates (MPH, 2nd column) capture well the underlying true RFs (True RFs, 1st column) for all relative angles, unlike the STA RF estimates (STA, 3rd column) or the STC RF estimates (STC, 4th column). (C) We evaluated the models by computing CCs between predicted and observed firing rates on a validation set and for different pairs of LNP filters that were generated by rotating one of the RFs. The cascaded MPH (black line) performs slightly better than the non-cascaded MPH (gray line). Both MPHs perform better than STC (green line) and STA (blue line). (D) Quality of RF reconstruction, shown is the cosine angle between true and model RFs (compare main text). The MPH reconstructed the true RFs more faithfully (black line) than did STA (blue line) and STC (green line). The occasional drops in MPH performance (larger error bars) are due to local optima that can be circumvented by starting the MPH-parameter optimization from different initial conditions (the orange line is from the best model – in terms of likelihood on the training set – out of 3 initial conditions). Both, panels (C) and (D) show average results from 10 simulations (with standard errors indicated).
Figure 6.
The MXR- and Mn-MPH applied to single-unit activity in NIF of a singing zebra finch.
(A) Raw extracellular voltage trace time-aligned to a log-power sound spectrogram of a zebra finch song (high sound amplitudes in red and low amplitudes in blue). (B) The MXR-MPH's RF estimate (left, high and low sound amplitudes in red and blue respectively). The red blob at about +30 ms is an indication that this cell is premotor. The width of the window is ∼0.25 s. The MXR-MPH's alignment kernel (right) is concentrated near −10 ms, yielding a total lead of NIf spikes on song of about 40 ms. (C) The RF estimated with reverse correlation (left) is similar to the MXR-MPH's RF. Middle: RF and jitter kernel of an MXR-MPH with much narrower RF window (about 10 ms wide). The total dimension of the RF is 605 (5 columns times 121 rows). Because the RF is so narrow, the spike latency is now clearly reflected in the alignment kernel (right), centered around a negative alignment shift of about 40 ms, implying that the model aligns spikes to portions of the song that occur about 40 ms after the spike. Hence, the alignment kernel strongly suggests a premotor function of this cell. (D) Predictions (5-fold cross validation) of the MXR-MPH (left, red bar) are similar to reverse correlation (blue bar). Using the non cascaded version (green bar) yields a slight drop in performance. An Mn-MPH yields a modest improvement in prediction performance (right, peaking at 8 states) in both the cascaded (cMPH) and non-cascaded forms (MPH, error bars depict 95% confidence intervals). (E) Results for a different data set (a different cell producing 1659 spikes during about 54 s of song data containing about 60 song motifs). The RF estimated using RC reveals diffuse spectrotemporal tuning, making it nearly impossible to decide whether this cell is sensory or motor in function. By contrast, the MPH alignment kernel (right) quite clearly reveals a motor function in this cell, evidenced by the predominance of negative alignment shifts. Also, the MPH RF shows a rather narrow frequency tuning near 2.6 kHz (middle). (F) The MXR-MPH firing-rate predictions for this cell were comparable to reverse correlation predictions; Mn-MPHs again yield a modest improvement in prediction performance.
Table 1.
The generalized Viterbi algorithm.
Table 2.
The generalized forward algorithm.
Table 3.
The generalized backward algorithm.
Table 4.
Extended Viterbi algorithm to compute most likely pair of hidden state and neural response sequences for a given stimulus.