Functional diversity among sensory neurons from efficient coding principles

In many sensory systems the neural signal is coded by the coordinated response of heterogeneous populations of neurons. What computational benefit does this diversity confer on information processing? We derive an efficient coding framework assuming that neurons have evolved to communicate signals optimally given natural stimulus statistics and metabolic constraints. Incorporating nonlinearities and realistic noise, we study optimal population coding of the same sensory variable using two measures: maximizing the mutual information between stimuli and responses, and minimizing the error incurred by the optimal linear decoder of responses. Our theory is applied to a commonly observed splitting of sensory neurons into ON and OFF that signal stimulus increases or decreases, and to populations of monotonically increasing responses of the same type, ON. Depending on the optimality measure, we make different predictions about how to optimally split a population into ON and OFF, and how to allocate the firing thresholds of individual neurons given realistic stimulus distributions and noise, which accord with certain biases observed experimentally.

We demonstrate that the equal coding theorem, where any number of ON and OFF cells codes the same information, does not follow trivially from changing ON cells into OFF cells by exchanging a spike for no spike. To illustrate this in the most intuitive way, we compare a system with one ON and one OFF cell with optimally adjusted thresholds θ 1 and θ 2 respectively, and two ON cells also with optimally adjusted thresholds, θ 1 and θ 2 . As stated in the main text, we assumed that ON (OFF) neurons fire Poisson spikes with an average mean count ν max whenever the stimulus intensity is above (below) their threshold θ i , and zero otherwise, i.e. ν i (s) = ν max Θ(s−θ i ) for ON neurons and ν i (s) = ν max Θ(θ i −s) for OFF neurons (Fig. 1B of the main text), where Θ is the Heaviside function. For such response functions with two firing rate levels, 0 and ν max , we can lump together all states with nonzero spike counts into a single state, which we denote as 1. Correspondingly, we denote the state with zero spikes as 0. We introduce the parameter q = e −νmaxT , which ranges from q = 0 in the noiseless limit of high firing rate, and q = 1 in the high noise limit of low firing rate. Then, Let (n 1 , n 2 ) denote the output vector, with (ν 1 , ν 2 ) being related to the input since the firing rate of each cell is determined by how the output distribution p(n 1 , n 2 ) is partitioned by the thresholds. For the ON-OFF system, this partitioning has the form: resulting in the conditional probability matrix p(n 1 , n 2 |s) given in S1 Table. Table S1. Conditional probability matrix p(n 1 , n 2 |s) for a mixed ON-OFF system. Now, we convert one OFF cell into an ON cell. Again, the output distribution p(n 1 , n 2 ) is partitioned by the thresholds, which for the ON-ON system has the form: resulting in the conditional probability matrix p(n 1 , n 2 |s) given in S2 Table. It is clear that changing one Table S2. Conditional probability matrix p(n 1 , n 2 |s) for a homogeneous ON-ON system.
OFF into an ON cell does not simply exchange the rows of the conditional distribution matrix, but alters them. In particular, while the first and second rows are exchanged between the ON-ON and ON-OFF systems, the third row is very different, regardless of threshold settings. Therefore, we conclude that there is no simple symmetry argument for why the information between the ON-OFF and ON-ON systems should be identical.

The identity of mutual information does not depend on the noise model
We consider two different noise models to demonstrate that the mutual information is very similar in a system of two cells, independent of whether the population is mixed (one ON and one OFF cell), or homogeneous (two ON cells). Figure S1. A framework with binary neurons that have two firing rate levels, r if the stimulus is smaller (bigger) than a threshold, and R if the stimulus is bigger (smaller) than a threshold for ON (OFF) cells. We compare two systems, left: one ON (red) and one OFF (blue) cells, and right, two ON cells, where the information is maximized by optimizing the cells thresholds. Information results in Table S3. Table S3. Mutual information for the systems comprised of binary cells, one ON and one OFF cells, vs. two ON cells (see S1 Figure) where the spike counts r and R have been varied, assuming Poisson noise on the spike count. In all cases, the information is nearly identical. Figure S2. Two sigmoidal nonlinearities for an ON cell (red) and an OFF cell (blue), describing the firing rate as a function of stimulus with the maximum expected spike count R, the gain β, and the threshold θ.
The shaded curve denotes the Laplace stimulus probability distribution. Information results in S4 Table. First, we continue as before with a Poisson noise model, but where we assume that the neural response of an ON (OFF) cell to a stimulus below (above) a certain threshold is nonzero (S1 Figure). The expected spike counts are determined by the firing rate levels whenever the stimulus is smaller (r) or bigger than a threshold (R). Because of this assumption, we can no longer make the simplification from Eq. S1, (also Eq. 12 in the main text) about the way noise enters the system and which allows us to do analytical calculations, but need to explicitly sum over all possible spike counts (up to some cut-off) given r and R, numerically. Therefore, we compared systems of two neurons, one ON and one OFF, vs. ON-ON in which the levels of firing in the high and low state were the same, but where we optimized the thresholds for a Laplace stimulus distribution. We obtained the results in S3 Table for different values of r and R, which confirm our initial conclusion about equality of ON-OFF and ON-ON coding.
Second, we considered sigmoidal nonlinearities fitted from recorded salamander retinal ganglion cells [1,2]. Since real ganglion cells do not conform exactly to Poisson statistics [3,4], we considered an output noise distribution that matches the observed sub-Poisson noise in spike counts of salamander retinal ganglion cells, as measured experimentally [1]. The form of the nonlinearity was and g OFF (s) = R 1 1 + e β(s−θOFF) (S4) for ON and OFF cells, respectively. These sigmoids have gain β and a maximal spike count of R (S2 Figure). In this model, we assumed that the output noise distribution was sub-Poisson, fitted directly form data [1]. In particular, for a given mean spike count c, the measured spike count distribution p(k|c) has width that stayed constant with c after an initial Poisson-like growth ([1] (see their Supp. Fig. 3). The distribution is well described by the heuristic formula where with a = 0.5 nd σ = 0.75 [1]. Using this noise model, we computed the mutual information numerically, and had to sum over the spike counts for each cell (up to some cutoff). The fitted sigmoids from the data from salamander ganglion cells had gains of β = 5.8 (expressed in inverse of the standard deviation of the stimulus), and β = 2.3 from the data from macaque ganglion cells. The maximal firing rates were also extracted from the data to be: median peak firing rate for salamander ganglion cells of 48 Hz, which with a coding window of 50 ms translates to R = 2.4, and median peak firing rate for macaque ganglion cells of 220 Hz, which with a coding window of 10 ms translates to R = 2.2. Based on these fits, we computed the mutual information as before for a system with two cells, one ON and one OFF, and two ON cells (S4 Table), demonstrating that the mutual information was identical.  Figure). The maximal firing rates and gains were extracted from the data. In all cases, the information is identical.