Figure 1.
Degradation of sensory signal.
Here we illustrate degradation of the image signal in the eye. The original signal is a portion of an unaltered standard test image. The blurred signal is computed with the blur function measured at 30° eccentricity of the human eye [50]. The observed signal (also called the raw sensory signal) simulates the noisy response of cone photoreceptors in a square lattice by adding white gaussian noise to the blurred signal.
Figure 2.
The number of output neurons is far more limited in the peripheral retina.
The graph shows the number of cone photoreceptors per midget RGC as a function of eccentricity in the macaque retina. The data at the fovea () and periphery (
) are from [93] and [70], respectively, and the smooth curve was a fit to the data using a cubic spline.
Figure 3.
(a) Network diagram. Nodes represent individual elements of the indicated variables (noise variables indicated by small gray nodes); lines represent dependencies between them. Bold lines highlight, respectively, a point spread function of the blur from a point in the original signal to the observed signal, an encoding filter (or receptive field) that transforms the observed signal into the neural representation in a single neuron (encoding unit), and a decoding filter (or projective field) which represents the patten of that neuron's contribution in the reconstructed signal (its amplitude is given by the neural representation). In this diagram, the number of coding units at the neural representation is smaller than that of sensory units at the observed signal, which is called an undercomplete representation. Note that the proposed model is general and could form an optimal code with an arbitrary number of neurons, including complete and overcomplete cases. (b) The block flow diagram of the same model using the model variables defined in Methods. Each stage of sensory representation is depicted by a circle; each transformation by a square; each noise by a gray circle.
Figure 4.
Image reconstruction examples.
We compare reconstructions from two different codes: whitening and the proposed, optimal model. The original signal (121×121 pixels) is degraded with blur and with different levels of sensory noise (−10 to 20 dB), resulting in the observed signals, where the percentage indicates the MSE relative to the original signal. These are encoded under two different cell ratios: 1∶1 (fovea) and 16∶1 (periphery) for each noise level. The reconstructed signals are obtained with the optimal decoding matrices, where the percentage indicates the MSE relative to the original signal, which can also be read out in Figure 5 (labeled by open and closed triangles for the respective eccentricities).
Figure 5.
The reconstruction error as a function of neural population size.
Two x-axes represent, respectively, the cone: RGC ratio (top) and the corresponding retinal eccentricity in the macaque retina (bottom; see Figure 2). The problem settings are the same as in Figure 4 with extended cell ratios; the common cell ratios (1∶1 and 16∶1) are indicated by the same labels (open and closed triangles, respectively). The signal dimension is 121×121 = 14,641 for all condition; the number of neurons with 16∶1 cell ratio is 915.
Figure 6.
Spectral analysis of the proposed model compared to whitening.
Every stage of sensory representations and their transformations are illustrated (cf. Figure 3). The signal is 100-dimensional, and the fovea and periphery conditions differ only in the neural population size (100 and 10, respectively). Each is analyzed under two sensory noise levels (20 and −10 dB). The horizontal axes represent the frequency (or spectrum) of the signal and are common across all plots. The vertical axes of the open plots (e.g., original signal) are common and represent the variance of the indicated sensory representations; those of the box plots (e.g., blur) are also common and represent gain (or modulation) with the indicated transformation, where the thin horizontal line indicates unit gain. The original signal (, yellow) is assumed to have a
power spectrum where f is the frequency of the signal. The blur (
, black) is assumed to be low-pass gaussian. The observed signal (
) is shown component-wise, i.e., the blurred signal (
, blue) and the sensory noise (
, red). The observed signal is transformed by the neural encoding (
, black). Solid and dashed lines indicate the gain as a function of frequency for the proposed and whitening model, respectively (and the same line scheme is used in the other plots). The neural representation (
) is also shown component-wise, i.e., the encoded signal (
, blue) and neural noise (
, red). The optimal decoding transform (
, black) is applied to the neural representation to obtain the reconstructed signal (
; blue), which is superimposed with the original signal (yellow); the percentage shows the MSE of reconstruction. Note all axes are in logarithmic scale. It is useful to recall that transforming a signal with a matrix is multiplicative, but it is simply summation in a logarithmic scale, and thus one can visually compute, for example, the blurred signal as the sum of the original signal and blur curves.
Figure 7.
A variety of equally optimal solutions obtained under different resource constraints.
Each panel shows a subset of five pairs of neural encoding (top, ) and decoding (bottom,
) filters in the foveal setting at four sensory SNRs (columns, −10 to 20 dB) in four conditions (rows): (a) No additional constraint (i.e., the base model). (b) Weight sparsity. (c) Response sparsity. (d) Spatial locality. Only the spatial locality constraint yields center-surround receptive fields. See Figure S6 for the resource costs in respective populations. Note that in (d) the center-surround structure is seen only in the filters, which transform the observed signal into the neural code (and hence correspond to receptive fields). The decoding filters have a different, gaussian-like structure. These features are used to optimally reconstruct the original signal from the neural code.
Figure 8.
Predicting different retinal light adaptations at different eccentricities.
Each panel consists of three plots. Top: The (smoothed) cross section of a typical receptive field through the peak. The horizontal line indicates the weight value of zero. Middle: The intensity map of the same receptive field. The bright and dark colors indicate positive and negative weight values, respectively, and the medium gray color indicates zero. Superimposed is the outline of the center subregion (the contour defined by the half-height from the peak) along with the average number of pixels (cones photoreceptors) inside the contour. Bottom: The half-height contours of the entire neural population which displays their tiling in the visual field. Two neurons are highlighted for clarity (one of which corresponds to the neuron shown above). The pixel lattice is depicted by the orange grid.