^{1}

^{*}

^{2}

Wrote the paper: WB TS. Other: Derived the general equation for optimal decision boundaries: WB TS. Found and analyzed specific solutions for several example probability distributions: WB TS.

The authors have declared that no competing interests exist.

We consider here how to separate multidimensional signals into two categories, such that the binary decision transmits the maximum possible information about those signals. Our motivation comes from the nervous system, where neurons process multidimensional signals into a binary sequence of responses (spikes). In a small noise limit, we derive a general equation for the decision boundary that locally relates its curvature to the probability distribution of inputs. We show that for Gaussian inputs the optimal boundaries are planar, but for non–Gaussian inputs the curvature is nonzero. As an example, we consider exponentially distributed inputs, which are known to approximate a variety of signals from natural environment.

What we know about the world around us is represented in the nervous system by sequences of discrete electrical pulses termed action potentials or “spikes”

Sensory inputs to the brain are intrinsically high dimensional objects. For example, visual neurons encode various patterns of light intensities that, upon moderate discretization, become vectors in 10^{2}–10^{3} dimensional space

We consider a much simplified version of the full problem. We look at a single neuron, and focus on a small window of time in which that cell either does or does not generate an action potential. We ignore, in this first attempt, coding strategies that involve patterns of spikes across multiple neurons or across time in single neurons, and ask simply how much information the binary spike/no spike decision conveys about the input signal. Let this input signal be a vector

We can always write the information as a difference between two entropies _{response}−_{noise}. This expression holds for any model of neural noise and response probability

In the absence of noise, the coding scheme which maps signals into spikes (or their absence) is a boundary in the

In this case, the decision boundary (red solid line) is shown as extending to infinity, but closed contours are also possible. Variations in contours' shape (as illustrated with a dashed curve) not only shift the position of the decision boundary relatively to the input probability distribution, but also change the overall length of the contour and its infinitesimal arc length element.

We will work in an approximation where the noise is small. We will allow for the noise magnitude to vary with the stimulus to account for the fact that, for example, noise level could be larger for inputs of large magnitude. Then if the boundary of the spiking domain is some (_{noise} should arise from a narrow strip surrounding the boundary

While our choice of threshold–like transitions between spiking and non–spiking regions considerably narrows the types of possible input-output transformations, it still leads, as we show below, to highly nontrivial, yet tractable, solutions. We will treat the local noise length scale σ(

Taking into account that the response entropy _{response} only depends on the average spike probability, the optimal contour providing maximal information may be found by minimizing

First, let us consider the simplest case where inputs are uniformly distributed and noise level is constant. In this case, the optimal contours obtained according to Eq. (5) are circles. The circle radius is determined by the average firing rate, which in this case equals its area. The fact that a circle turns out to be an optimal solution for uniform inputs is, perhaps, not surprising. After all, the optimization problem we consider is related to the theory of minimal surfaces, which have the smallest circumference for a given area. A circle is the most obvious example of a minimal surface. In the context of information transmission, fixing the average firing rate is equivalent to fixing the enclosed area, whereas minimizing noise entropy is equivalent to minimizing the circumference in the case of minimal surfaces. Despite its simplicity, the finding that optimal decision contours are circles for uniform probability distribution indicates possible functional advantages of the circular symmetry observed for receptive fields in the retina. After all, in the case of retinal processing, the probability to have certain intensity value is uniform across space. Below we solve Eq. (5) to find optimal decision boundaries for two example non-uniform probability distributions: a Gaussian and the exponential. The exponential distribution is important not only as an example of non–Gaussian inputs, but also because it captures some of the essential statistical properties found in real–world signals

Consider the case of uncorrelated Gaussian inputs

To choose between circles and straight lines, we calculate the noise entropy as a function of spike probability _{noise} is proportional to the noise level σ, so in what follows we compute the noise entropy in these units. For a circle in two dimensions, the noise entropy is given by

For straight lines a distance _{line}. From Eq. (6), we find that the Lagrange multiplier λ = _{line}. Therefore, similarly to the case of circular decision boundaries, within the family of planar threshold decisions there is also a one-to-one relationship between the average firing rate and the noise entropy.

Comparing the noise entropy as a function of the corresponding average spike probability both the family of circular and linear solutions, we find that threshold decisions with respect to straight lines lead to smaller values of noise entropy, and therefore larger values of information transmitted, for all values of average spike probability, cf.

This result can be generalized to inputs of arbitrary dimensionality. Expressions for entropy and probability for straight lines do not change with dimensionality

As an example of a non–Gaussian probability distribution, we consider an exponential distribution in two dimensions (2D):

For

Within a single quadrant, the optimal solution can be found explicitly in terms of angle _{0}_{0} = x_{0})_{0} = y_{0})

The possible types of global solutions are shown in

(A) closed “stretched circle” solutions are shown for

For all types of global solutions (A–C), boundary conditions specify a unique curve for each value of

For extended solutions B, the entropy and probability become

“stretched circles” with spiking outside A or inside A′, extended solutions B and C; straight lines parallel to coordinate axes (1) and at ±π/4 angle (2). Solutions (A-C) and (1) satisfy the optimality Eq. (9), but not (2), which is optimal only at a single point at

The most physiologically relevant regime corresponds to _{A}< β_{B}_{C}^{2})

We have presented a general approach to finding optimal binary separations of multidimensional inputs. In the small noise limit, the curvature of the optimal bounding surface is determined locally by the probability distribution. While Gaussian inputs are optimally separated by hyper-planes, this is not the case in general. For example, in the case of exponentially distributed inputs in two dimensions, the optimal decision contours are curved and could either be closed or extended. Closed contours are optimal at extreme probabilities, while extended ones are optimal for spike probabilities near 1/2. The ubiquity of non–Gaussian signals in nature, particularly of the exponential distributions considered here, suggests that these results might be relevant for neurons across different sensory modalities.