Figure 1.
Natural image correlations are highly regular in both space and spectrum.
(A) An RGB rendering of a hyperspectral image taken from the natural image database described by Chakrabarti and Zickler [12]. The code used to render this figure is included in our gitHub repository (https://github.com/DavidBrainard/ReceptorLearning/). (B) Image correlations for images from our combined database. Correlation is plotted as a function of distance (pixel). Each curve represents a different wavelength separation: the black curve represents no wavelength difference while the yellow curve represents a difference of 320 nm (i.e., the 400 nm channel correlated with the 720 nm channel).
Figure 2.
Inter-cone correlations carry information about cone spectral class and mosaic arrangement.
(A) A cone mosaic with an L∶M∶S ratio of 47∶47∶6; each cone is colored by class (red, green, and blue for L, M, and S cones, respectively). (B) The spectral sensitivities of L, M, and S cones in a typical fovea [52]. (C) The mosaic from Panel A with grayscale indicating the correlation of each cone's response to the response of the single M cone plotted in green. The correlations are calculated from the responses to 2,000,000 natural images. At this scale it is readily apparent that correlations decrease with inter-cone distance and that S cones are distinguished from M cones by their correlations. (D) Magnified view of the correlation of the immediately adjacent neighbors of the green M cone from Panel C, with grayscale indicating correlation. Red and green circles indicate the class of each neighboring cone. Of the four immediate neighbors of the central cone, the two neighboring M cones (green) have higher correlations than the neighboring L cones (red). (E) Magnified view of the correlation of the diagonal neighbors of the green M cone from Panel C, with grayscale indicating correlation. The neighboring M cones have higher correlations than the two neighboring L cones. (F) A smoothed histogram of the correlations of immediately neighboring longer-wavelength-sensitive cones in 54 simulated mosaics whose M cone
value was 530 nm and whose L cone
value was 558.9 nm. The 54 mosaics differed in their L∶M cone ratios and sizes. Correlations between neighboring L cones, neighboring M cones, and L-M neighbors are shown in red, green, and black respectively. The histograms represent correlations from 9,862 L-L cone pairs, 9,769 M-M cone pairs, and 8,369 L-M cone pairs.
Figure 3.
Multidimensional scaling allows classification of cones for a typical trichromatic retinal mosaic.
(A) 3D embeddings of the correlation matrix of the mosaic from Figure 2A. Each point represents a single cone and is colored red, green, or blue for L, M, or S respectively, according to its actual identity in the mosaic. The 3D embeddings shown here and in other figures in this paper are oriented so that the
-
plane (
horizontal,
vertical) of the representational space described in the text is shown. The absolute units on these axes are not meaningful, because MDS solutions are determined only up to a relative-distance preserving transformation. (B) The same 3D embedding shown in A zoomed in on the embedding of the L and M cones only. (C) The 3D embedding of the L and M cones from A after flattening. (D) A histogram of the
positions of the embedding from C (i.e., after flattening); best fit skew normals are shown in red and green. Rotating animations that show the three-dimensional structure of the embeddings are available online (http://color.psych.upenn.edu/supplements/receptorlearning).
Figure 4.
The algorithm detects the number of longer-wavelength-sensitive cone classes and correctly classifies individual cones for a range of trichromatic mosaic parameters.
(A) The fraction of cones correctly classified for various combinations of L∶M cone ratio and M cone value, when the number of longer-wavelength cone classes (L and M) was assumed to be 2. The S cone proportion was held at 6%, and S cones were given a
value of 420.7 nm in all simulations. The L cone
value was 558.9 nm. Each cell in the plot represents the aggregate results of three simulations, each with a different mosaic and each shown a different random sample of 2 million natural image patches. Mosaic responses for each simulation were obtained with different draws of natural image patches. Accuracies are reported as the average accuracy for L and M cones, each calculated separately. For example, if the algorithm correctly classified 354 of 354 L cones and 1 of 22 M cones in a mosaic with an L∶M ratio of 16∶1, the overall accuracy reported here would be 52% (the mean of 354/354 and 1/22) rather than 94% (the fraction of all cones correctly classified). (B) The number of simulations (of three) for each L∶M ratio and M cone
value where the algorithm correctly detected that there were two longer-wavelength-sensitive cone classes. The results for individual simulations are given in Figure S4.
Figure 5.
The algorithm detects dichromatic and tetrachromatic retinal mosaics.
On the left are embeddings of the dichromatic retinal mosaic: (A) the full embedding; (B) the embedding zoomed in on just the L cones; (C) the flattened L cone embedding; and (D) a histogram of the positions of the flattened L cone embedding with the best-fit of the single detected skew normal. On the right are embeddings of the tetrachromatic retinal mosaic: (E) the full embedding; (F) the embedding zoomed in on just the L, M, and anomalous (A) cones; (G) the flattened L, M, and A cone embedding; and (H) a histogram of the
coordinates of the flattened L, M, and A cone embedding with best fit of a mixture of the detected skew normals. Note that the units on Panels A, B, C, E, F, and G are arbitrary, as MDS does not produce meaningful units, but rather yields a relative-distance-preserving embedding. Spectral sensitivity curves for L, M, and S cones are shown in Figure 2A. Anomalous A cones were given
values of 545 nm, and the tetrachromatic retinal mosaic had an L∶M∶A ratio of 1∶1∶1. L, A, M, and S cones are colored red, yellow, green, and blue, respectively.