Figure 1.
Tuning of retinal ganglion cell responses.
(A) Illustration of retinal ganglion cell recordings used in this work. Left, the eight grating patterns used as stimuli, along with the receptive field centers of two recorded fast-Off RGCs. One grating period measured 660 µm on the retina. Middle and right, spike rasters recorded from these two RGCs, plotted vs time since the onset of each grating stimulus. The gratings were presented multiple times randomly interleaved. (B) Cosine fit (solid line) to the first-spike latencies of the RGC from (A), middle column. Boxes depict mean latency ±1SD. Gray lines show the baseline (vertical) and phase offset (horizontal) of the cosine fit. (C) As in (B), but for the spike count. (D) Quality of the cosine fits, measured by coefficients of determination for single-trial first-spike latencies, , and of single-trial spike counts,
, in the highest contrast condition. Each point is one RGC, different symbols denote different experiments, the filled square represents the fits shown in (B) and (C). The cross marks the population means. (E) As in (D), but coefficients of determination for single-trial spike counts,
, plotted versus each fit’s coefficients of determination for mean spike counts,
. The
values are large even when the corresponding
is small, which indicates that
is affected by large noise in the spike counts rather than a systematic failure of the cosine model. (F) Scatter plot of phase offsets for the cosine fits of spike count (y-axis) and latency (x-axis). The solid black line depicts a relative phase shift of 180°. (G) As in (F), but for the baseline values. (H) As in (F), but for modulation amplitudes. (I) Contrast affects response latency similarly in different cells. For each RGC, cosine fits for the latency were obtained at all four studied contrast levels and the shift
of the latency baseline was measured relative to the highest contrast, i.e. from 47% to 39% (diamonds), 47% to 31% (squares), and 47% to 29% (circles). For all pairs of RGCs analyzed, this scatterplot shows the baseline shifts of the two members. Solid line is the identity.
Figure 2.
Spike-timing computations based on retinal spike trains.
(A) Schematic of the modeled readout neuron. The neuron receives input from multiple afferents (top left). Each afferent spike produces a PSP of stereotyped shape with an amplitude that depends on the synaptic strength (insets top right, x-scale = 20 ms, y-scale = spike threshold). For some RGC spike patterns (e.g. the one in blue), excitatory and inhibitory input spikes arrive segregated in time such that the resulting PSPs sum to a peak voltage, , above the spike threshold (
), and the model neuron fires an action potential (bottom left, blue trace). For other patterns (e.g. red), excitation and inhibition interfere such that the peak voltage (red trace) remains below threshold and the model neuron remains silent. (B) Two visual categorization tasks based on the grating stimuli. Top: The luminance task asks whether a particular location in the field (dotted line) is bright (left) or dark (right). The readout neuron should fire in the former case but not in the latter; the opposite rule is another version of this task. Bottom: The boundary task asks whether a particular location (dotted line) has a boundary of either polarity (left) or no boundary (right).
Figure 3.
Performance of the tempotron and other decoders on RGC population responses.
(A) Performance of the tempotron on the luminance (circles) and boundary (triangles) tasks in the highest contrast condition. Results are averaged over all realizations of each task and two separate populations of simultaneously recorded fast-Off RGCs, one with 7 neurons, the other with 8. The fraction of correct stimulus classifications is shown as a function of the maximal number of spikes admitted from each RGC. (B) Effect of synaptic depression on performance in the “all spikes” condition of the boundary task in (A). The fraction of correct responses is plotted as a function of the recovery time constant of synaptic depression and the synaptic utilization parameter
, which determines the degree of depression (cf. Equation 5);
(maximal depression, black), 0.8 (blue), 0.6 (red). The dotted line indicates the performance with static synapses from (A). (C) As in (A), but performance of the perceptron decoder based on spike counts. (D) Comparison of the tempotron (Temp) and perceptron (Perc) peak performances from (A) and (C) with other timing-based readout schemes for the luminance (left) and boundary (right) tasks: Rank-order decoder (Rank), temporal-winner-take-all decoder using the first spike (Twta 1) or the first three spikes (Twta 3). The boundary task includes performance of a perceptron with an optimized integration window of 80 ms duration (Perc Δ). (E) Contrast dependence of tempotron performance in the luminance (circles) and boundary (triangles) tasks when using at most the first spike of each afferent ganglion cell. The fraction of correct classifications was measured separately within each of four contrast conditions (x-axis) on the basis of a seven-cell input population of RGCs. Open symbols with dotted lines: after training on all four contrast levels. Filled symbols with solid lines: after training only on the lowest and highest contrast levels. Note that the tempotron performs well even when generalizing to intermediate stimulus contrasts that were not encountered during training (colored symbols).
Figure 4.
Mechanisms of spike-latency-based neuronal computing.
Illustration of sample tempotrons that solve the luminance (left) or the boundary (right) task. Each tempotron receives inputs from only two RGCs, and only their first spikes are processed. (A) The four grating stimuli that define each task and the relative locations of the two recorded RGC receptive fields. Solid line: Cell 1; dashed line: Cell 2. (B) Latency tuning of the two RGCs under the eight gratings. Lines show fitted cosine tuning curves. Colored horizontal bars highlight the spatial phases of the four stimuli used in each task, color-coded as in (A). For each of these stimuli, boxes depict the mean ±1 SD of the measured first-spike latency. (C) Histograms of the differential latencies of the two RGCs during the four task-relevant stimuli, color-coded as in (A). (D) Sample voltage traces of a tempotron that solves the task, color-coded according to the four stimuli. The input spike times of this example represent the median differential latencies observed experimentally. Horizontal dotted line depicts the spike threshold. Inset: The minimal readout circuit with two RGCs connected to one postsynaptic neuron. Traces show the PSPs of cell 1 (black) and cell 2 (gray) that underlie the solution shown in the main panel. Scale bars depict 20 ms in the x-direction and half of the spike threshold in the y-direction. In these implementations, both afferents are excitatory. D1:
, D2:
. (E) Like (D), but in these solutions, one afferent is excitatory and the other inhibitory. E1:
, E2:
. Note that the solution in (E2) encodes the identity of the stimulus within the target class by the latency of the output spike: While early responses signal the dark red stimulus, later responses signal the light red stimulus. The membrane time constants in (D) and (E) were chosen to minimize the generalization errors for the sample distributions of (C) in the investigation of threshold noise (Materials and Methods). (F) Histogram of the peak voltage of the tempotron in (D) for all experimental trials, color-coded by stimulus as in (A). Note that the target and null stimuli are well separated on either side of the threshold (dotted vertical line). (G) Like (F), but for the solutions in (E).
Figure 5.
Synaptic solution space of a ganglion cell pair decoder.
(A) In this simple version of the decoder, each of the two afferents fires exactly one spike per stimulus presentation at times and
. The two spikes produce PSPs with identical kinetics but different amplitudes
and
. The relative latency
of the two spikes determines whether the combined PSP crosses threshold for a spike. This classifier divides the range of
into three regions:
,
, and
. Depending on the two synaptic weights, the decoder fires only in the middle region (top right, e.g. Figure 4D2) or only outside that region (bottom right, e.g. Figure 4E2). (B) One can prove that for any desired location of the boundaries
and
there is a combination of synaptic weights
and
that provides the correct classification. Here this solution space is computed using PSP kinetics with
and
. The left hand plot shows for any combination of
and
the ratio of synaptic weights
that solves the task. In the region marked “++” both synaptic weights are positive and the readout neuron fires inside the range
. In the regions marked “+–” and “–+” the weights are of opposite sign, and the readout neuron fires outside the specified range. Dashed lines indicate the boundary between the three types of solutions. Their location depends on the PSP kinetics and asymptotically approaches the time-to-peak of the PSP (solid lines). See Materials and Methods for details. The right hand plots illustrate two specific solutions for the
combinations indicated by the arrows. In each case, the PSP is shown for several different latencies
; bold lines correspond to the limiting cases
and
, for which the PSP just reaches threshold.
Figure 6.
Effects of spike-time noise, threshold noise, and readout time on tempotron performance.
(A) Scatterplot of first-spike latencies of two RGCs on multiple trials for each of the eight stimulus phases in the highest contrast condition. For this cell pair, the latencies covary, with an average correlation coefficient of 0.46 over all eight grating phases. (B) Histogram of correlation coefficients for first-spike latencies observed for all simultaneously recorded cell pairs, stimuli, and contrasts (black). Note the excess of positive correlations. As a control, the gray line shows the analogous histogram obtained when correlating latencies of the two cells separated by one stimulus trial. (C) Top, histogram of relative latencies for the cell pair of (A) for the boundary task of Figure 4C2. Bottom, the same histogram of relative latencies, but obtained from shifted trials. Note the increased dispersion of the relative latencies and the increased overlap between the red and blue peaks. (D) Effect of latency correlations on readout performance. For all simultaneously recorded cell pairs, we obtained the minimal tempotron error rates with inputs from simultaneous trials and with inputs from shifted trials. The ratio of these two error rates is plotted against the error rate obtained for simultaneous trials. Squares: luminance task; circles: boundary task. Symbol colors represent the mean latency correlation across the four stimuli that constitute a given task (color bar). Note that most of the points lie below unity, showing that in most cases the readout performance degrades when trials are shifted and latency correlations destroyed. (E) Distributions of the peak voltage of the tempotron for the boundary task, based on the distributions of relative latencies shown in (C) (top; same color code). The two voltage distributions result from optimizing the tempotron weights for different PSP kinetics, namely (top) and
(bottom). In this example the shorter PSPs generate a much larger separation between the maximal voltages for target and null stimuli (red vs blue), such that the spike readout would be more robust to any noise in the neuron’s threshold. (F) Optimal classification performance on the boundary task with the input cell pair of (C) and assuming a Gaussian threshold noise whose standard deviation is 5% of the mean synaptic weight magnitude. The error was minimized for each PSP time constant (x-axis) over the synaptic efficacies for the purely excitatory solution (black, Figure 4D2) and the mixed solution with one excitatory and inhibitory input (gray, Figure 4E2). (G) Performance of optimal tempotrons operating with a pair of RGCs on the luminance task as a function of the maximal allowed latency,
, of input spikes. The fraction of correct classifications was averaged over all input cell pairs that allowed for error-free performance in the highest contrast condition at large
. Results are plotted for different values of the stimulus contrast (indicated in %). (H) As in (G) but for the boundary task and the fraction of correct classifications averaged over all input cell pairs with errors below 5% in the highest contrast condition at large
.
Figure 7.
Schematic model of orientation selectivity by latency processing.
(A) In this model, a population of 7 RGCs is stimulated by a sudden appearance of a bright/dark grating, and the resulting spike trains are processed by a tempotron. To emulate a cell that detects a single horizontally oriented pattern, reminiscent of a cortical simple cell, the tempotron should fire to the preferred grating (left), but remain silent to its inverse (middle), a rotated version (right), or any other pattern of illumination. To detect a horizontal grating independent of polarity, the tempotron should fire both to the preferred grating (left) and its inverse (middle), but reject all other patterns. (B) A set of synaptic weights assigned to the 7 RGCs (left) that solves this problem. Each RGC fires a spike either early or late (if its receptive field turns dark or bright, respectively) with a relative time difference of (Figure 1H). The resulting spike patterns produced by 4 different stimuli (top) are shown, with colors indicating each spike’s excitatory or inhibitory contribution. Bottom panels show the postsynaptic voltage traces elicited in the tempotron (
). All 126 binary stimulus patterns other than the preferred grating and its inverse produce a peak voltage of
in units of the unitary PSP amplitudes. The preferred grating and its inverse produce the two highest values with
; in the present model, this occurs if
. Because of the order of excitation and inhibition, the preferred grating always elicits a higher peak voltage than the inverse grating. Hence, if the spike threshold
is high (green line) the tempotron detects a single pattern, if
is lower (pink line) it detects horizontal gratings of both polarities.
Figure 8.
A tempotron model for orientation tuning with phase invariance.
(A) Orientation tuning curves of a tempotron model designed to show phase invariance. Extending the schematic model shown in Figure 7, this tempotron received inputs from 200 model RGCs whose Gaussian receptive fields were randomly placed in the circular region over which the grating was presented. The latency of each RGC was cosine-tuned to the fraction of its receptive field covered by dark grating bars as in the experimental recordings (Figure 1). The tempotron was trained to fire in response to orientations within ±15° (left) or ±30° (right) independently of the grating phase. The curves show average orientation tuning curves of the maximal voltage for four different grating phases (–90°: blue, 0: red, 90°: black, and 180°: green). Averages were obtained over 14 (left) and 15 (right) independent RGC populations and tempotrons. For each tuning width, insets show the four orientation tuning curves of one individual tempotron, overlaid on the mean ±1 standard deviation of the populations (shaded areas). (B) Spike timing of the two tempotron models shown in the insets of (A) with narrow (left) and wide (right) orientation tuning. The latency of the output spike (color code) was measured relative to the shortest-latency spike of the tempotron and plotted as a function of orientation and phase of the stimulus grating. Each tempotron spiked only for orientations within the dashed vertical lines.