Fig 1.
Stimulus encoding with a population of neurons in the presence of input and output noise.
A. Framework: A static stimulus s (top) is encoded by a population of spike counts {k1, …kN} (bottom) in a coding time window ΔT. The stimulus is first corrupted by additive input noise z and then processed by a population of N binary nonlinearities {ν1, …νN}. Stochastic spike generation based on Poisson output noise corrupts the signal again. Thresholds {θ1, …, θN} of the nonlinearities are optimized such that the mutual information Im(k1, …, kN;s) between stimulus and spike counts is maximized. Inset: Introducing additive input noise and a binary nonlinearity can be interpreted as having a sigmoidal nonlinearity after the input noise is averaged, 〈…〉z. Shallower nonlinearities result from higher input noise levels. B. Two different scenarios of information transmission: In the independent-coding channel each neuron contributes with its spike count to the coding of the stimulus, while in the lumped-coding channel all spike counts are added into one scalar output variable that codes for the stimulus.
Fig 2.
Schematic illustrating the dominance in information of the independent- over the lumped-coding channel.
Here, we treat the case of N = 2 cells, vanishing input noise (σ = 0) and A. vanishing output noise (R → ∞), B. intermediate output noise when the total number of spikes k = k1 + k2 = 1, and C. high output noise (R → 0). Left: The relative positions of optimal thresholds of both the independent- (blue) and lumped-coding (red) channels. Right: The stimulus “estimation probabilities” for the two different channels. Yellow shading shows where the noise entropy is higher in the lumped-coding channel. α, α′, and β denote non-zero probability values (see text).
Fig 3.
Maximized mutual information for the lumped- and independent-coding channels for a population of three neurons.
A. Information of the independent-coding channel for different combinations of output noise R and input noise σ. Contours indicate constant information. B. Information of the lumped-coding channel. C. Absolute information difference between the two coding channels. D. Information ratio between the two coding channels. Both C and D show a region of intermediate output noise where the independent-coding channel substantially outperforms the lumped-coding channel. E. Information difference depending on input noise σ for various levels of output noise R, corresponding to vertical slices from C. We also include the special case of zero output noise, R → ∞. F. Information difference depending on output noise R for various levels of input noise σ, corresponding to horizontal slices from C.
Fig 4.
Optimal thresholds for the independent- and lumped-coding channels.
Optimal thresholds for the independent-coding channel (A-E) compared to the lumped-coding channel (F-J) for a population of three neurons. A. The optimal number of distinct thresholds depends on input noise σ and output noise R. B. The optimal thresholds as a function of output noise for a fixed value of input noise (σ = 0.4). C. The optimal thresholds as a function of input noise for a fixed value of output noise (R = 1). D. The optimal thresholds as a function of output noise in the limit of no input noise (σ = 0). E. The optimal thresholds as a function of input noise in the limit of vanishing output noise (R → ∞). F-J. As (A-E) but for the lumped-coding channel. Intermediate noise levels where bifurcations occur in (G,H) take smaller values of R and σ in the lumped- that in the independent-coding channel since lumping itself acts like a source of noise (G: σ = 0.1, H: R = 9).
Fig 5.
Threshold differences as phase transitions with respect to both noise sources.
A. Optimal thresholds for the independent-coding channel depending on output noise R. Insets: The first derivative of the mutual information as a function of noise is continuous, while the second derivative is discontinuous at the critical noise values where the thresholds separate, implying a second-order phase transition. B. As in A, but with respect to input noise σ. C. Optimal thresholds as in A but for the lumped-coding channel. The first derivative is discontinuous at the critical noise values where the thresholds separate, implying a first-order phase transition. D. As in C but with respect to input noise σ.
Fig 6.
Information landscape for the independent- and lumped-coding channels undergoes different phase transitions around critical noise levels.
A. Top: Optimal thresholds of the independent-coding channel for a population of two neurons as a function of output noise R. Bottom: Corresponding eigenvalues of the Hessian of the information landscape with respect to thresholds. At the critical noise value Rcrit ≈ 0.396 at which the threshold bifurcation occurs (vertical dashed line) one eigenvalue approaches zero. B. Information landscape Im(θ1, θ2) for the three output noise levels R indicated by arrows in A. Top: For R > Rcrit, there are two equal global maxima. Middle: At R = Rcrit, the eigenvectors of the Hessian are shown and scaled by the corresponding eigenvalue (the eigenvector with the smaller eigenvalue, , was artificially lengthened to show its direction). At the critical noise value the information landscape locally takes the form of a ridge. Bottom: For R < Rcrit, there is one global maximum, meaning that the optimal thresholds are equal (bottom). C. The mutual information as a function of the line x in (θ1, θ2) space connecting the two maxima in B. Top: For R > Rcrit (low noise), there are two inflection points (dashed vertical lines) with zero curvature along the line x. The point with equal thresholds corresponds to a local minimum. Middle: At R = Rcrit, the two maxima, the minimum, and the two inflection points merge into one point, thus the curvature is zero. Bottom: For R < Rcrit, there is a single global maximum with a negative curvature. D. As in A but for a population with N = 3 neurons. E. As in A but for the lumped-coding channel. Both the optimal thresholds and the eigenvalues show a discontinuity at the critical noise level. F. Information landscape as in B for the lumped-coding channel and noise values indicated by arrows in E. Local optima are shown in cyan, global ones in red. G. Similar to C for the lumped-coding channel. Here the abscissa denotes the (non-straight) path connecting the three optima in F. H. Illustration of discontinuous threshold bifurcations, where the global maximum at θ1 ≠ θ2 at low noise (red, solid) becomes a local maximum for high noise (cyan, solid), while θ1 = θ2 (dashed) becomes global. As their respective derivatives are different, there is a discontinuity in the first derivative when only taking the global maximum into account (red lines), corresponding to a first-order phase transition.
Table 1.
Critical exponents as a function of the two noise sources.
Critical exponents are obtained by fitting a monomial to the threshold differences or eigenvalues near the critical noise values (l denotes the left, and r the right side) as a function of each noise source (see S4 Fig).
Fig 7.
The tuning of auditory nerve fibers (ANFs) match predictions from optimal coding.
A. The sigmoidal function that we use to fit ANFs tuning curves. Spontaneous firing rate (r), maximal firing rate (rm), firing threshold (θ), and the dynamic range (σ) are labelled on the curve. B. An example showing original data from ref. [8] and fitted tuning curves. These two tuning curves come from the same mouse. C. Optimal configuration (cyan dots) in the contour plot of mutual information. Black dot denotes the fitted thresholds from the data. D. Optimal configuration (cyan dots) in the contour plot of average firing rate. Black dot denotes the fitted thresholds from the data.
Table 2.
Comparison of different studies with regards to the different optimization measures, constraints, information convergence strategies, sources of noise and neuronal population size.
MI stands for Mutual Information and MSE for Mean Square Error.