Receptive fields acquired through unsupervised learning of sparse representations of natural scenes have similar properties to primary visual cortex (V1) simple cell receptive fields. However, what drives in vivo development of receptive fields remains controversial. The strongest evidence for the importance of sensory experience in visual development comes from receptive field changes in animals reared with abnormal visual input. However, most sparse coding accounts have considered only normal visual input and the development of monocular receptive fields. Here, we applied three sparse coding models to binocular receptive field development across six abnormal rearing conditions. In every condition, the changes in receptive field properties previously observed experimentally were matched to a similar and highly faithful degree by all the models, suggesting that early sensory development can indeed be understood in terms of an impetus towards sparsity. As previously predicted in the literature, we found that asymmetries in inter-ocular correlation across orientations lead to orientation-specific binocular receptive fields. Finally we used our models to design a novel stimulus that, if present during rearing, is predicted by the sparsity principle to lead robustly to radically abnormal receptive fields.
The responses of neurons in the primary visual cortex (V1), a region of the brain involved in encoding visual input, are modified by the visual experience of the animal during development. For example, most neurons in animals reared viewing stripes of a particular orientation only respond to the orientation that the animal experienced. The responses of V1 cells in normal animals are similar to responses that simple optimisation algorithms can learn when trained on images. However, whether the similarity between these algorithms and V1 responses is merely coincidental has been unclear. Here, we used the results of a number of experiments where animals were reared with modified visual experience to test the explanatory power of three related optimisation algorithms. We did this by filtering the images for the algorithms in ways that mimicked the visual experience of the animals. This allowed us to show that the changes in V1 responses in experiment were consistent with the algorithms. This is evidence that the precepts of the algorithms, notably sparsity, can be used to understand the development of V1 responses. Further, we used our model to propose a novel rearing condition which we expect to have a dramatic effect on development.
Citation: Hunt JJ, Dayan P, Goodhill GJ (2013) Sparse Coding Can Predict Primary Visual Cortex Receptive Field Changes Induced by Abnormal Visual Input. PLoS Comput Biol 9(5): e1003005. doi:10.1371/journal.pcbi.1003005
Editor: Matthias Bethge, University of Tuebingen and Max Planck Institute for Biologial Cybernetics Tuebingen, Germany
Received: May 12, 2012; Accepted: February 10, 2013; Published: May 9, 2013
Copyright: © 2013 Hunt et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This work was supported by an Australian Postgraduate Award (to JJH), the Gatsby Charitable Foundation (to PD) and Australian National Health and Medical Research Council Project Grant 525459 (to GJG). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Simple cells in the mammalian primary visual cortex (V1) are among the cells in the brain that are best functionally characterised –. They have also been used as a key model system for studying the complex interplay of intrinsic and extrinsic factors, i.e., nature and nurture, in controlling development. For instance, there is ample evidence that receptive field structure exists prior to eye-opening [e.g. 4]–, being significantly present in dark-reared animals , . Yet numerous studies, many taking advantage of the fact that simple cells are the earliest in the visual pathway to encode input from both eyes , have demonstrated that receptive field properties are modified by visual experience during development [e.g. 10]–.
Developing a general theory of sensory coding has been an important goal of computational neuroscience. One famously powerful idea, Barlow's efficient coding hypothesis, is that early sensory coding attempts to remove redundancy by representing input in informationally optimal ways . Among other achievements, this hypothesis has provided compelling explanations for the characteristics of retinal receptive fields . However, redundancy reduction may be only a first step in sensory processing . For instance, V1 is many times overcomplete in its representation of input , a fact that, on the surface at least, increases rather than decreases the redundancy in the encoding of the input .
One possibility is that V1 is attempting to code visual input sparsely . Many variants of sparse coding have been mooted –, and, when tailored for natural scene input, almost ubiquitously lead to units with response properties similar to V1 simple cells. Other work has extended sparse coding models of V1 to complex cells , the dimension of time  and color ,  [reviewed in 33]. Sparse learning schemes often trade off the amount of sparsity and the error in the encoding. The justification for sparse coding has ranged from the energetic grounds of being metabolically efficient , to the statistical grounds of exposing underlying latent structure in the input , .
The boldest claim of the sparse coding hypothesis is that it offers more than just an interpretation of simple cell receptive fields, but rather that it can account for the outcome (if not necessarily the time-course) of cortical plasticity. Showing this would offer a more stringent response to criticisms about the utility of these forms of unsupervised learning models for understanding visual development , , and also license applications of the same principles at more advanced stages of sensory processing. However, bar some notable exceptions [e.g. 35], , models based on precepts such as sparse coding have typically been applied to the development under normal input, for which the role of nurture can be questioned, rather than under abnormal input, for which it cannot. Furthermore, apart from notable exceptions such as Hoyer and Hyvärinen , the models have typically focused on monocular rather than binocular receptive fields, thus not addressing many of the most important experimental conditions.
Here we tested whether receptive field changes in six abnormal rearing conditions applied to cats (table 1) can be captured by binocular versions of sparse coding models of receptive field development. The cat was chosen as the model organism to match since all the conditions have been examined experimentally by several different groups, leading to broad agreement in the results. The more limited range of experiments that have been conducted in other species, notably macaques, have led to similar results, as in .
To ensure the outcomes were due to the general principle of sparsity, rather than the specifics of a particular algorithm, we used three different generative models for learning sparsity: product of experts , k-means clustering and independent component analysis . We found that all three models qualitatively reproduced the receptive field changes observed in experiment in every rearing condition considered, and provided a good quantitative match in cases in which there was sufficiently ample sampling of receptive fields in the relevant experiments. This agreement provides evidence that receptive fields are indeed optimized during development in response to input statistics. Further, we used our models to design a novel rearing condition that we propose offers a strong test of the explanatory power of sparse coding. This involves presenting white noise with sparsity greater than that of natural scenes such that, even when augmented with natural input, it is still expected to lead to the development of highly localized receptive fields that are quite different from those of normal simple cells.
Overall, we suggest that examining abnormal rearing conditions will offer tests of functional accounts of development that are revealing and stringent, and look forward to the prospect of their application to higher visual areas and other sensory modalities.
We examined whether simple unsupervised learning models could capture the receptive field structures observed in abnormally reared animals. The models learn sparse responses which are conditionally independent given the input. We considered normal rearing, along with six abnormal rearing conditions (summarised in table 1).
All three unsupervised learning models gave qualitatively similar results across the rearing conditions. Figures shown in the main text, starting from the bottom rows in each column in Figure 1 which show sample receptive fields for each rearing condition, are for the results found using the product of experts model . In Text S1/S2 we provide the same figures with the results for independent components analysis/k-means clustering. Where there are notable differences between the models we mention this is the main text.
Representative examples of the V1 receptive fields over both eyes that result for the PoE model (lower 18 pairs). Subsequent figures quantify the changes in receptive field structure and distribution induced in each rearing condition. See table 1 for a summary of the receptive field changes seen experimentally for each condition. We model rearing with (A) normal (unfiltered) visual input, (B) stripe rearing, i.e. a single dominant orientation (in this case horizontal), (C) orthogonal stripe rearing, i.e. dominant orientations differing by 90 degrees between the two eyes (in this case horizontal and vertical), (D) monocular deprivation, i.e. one eye occluded, (E) one eye occluded but alternating the eye randomly during training, (F) one eye occluded most of the time, and (G) artificial strabismus (direction of gaze offset between the two eyes).
To facilitate a direct comparison between models and experiment, in each of the subsequent sections we first provide a brief literature review of the relevant experimental work for that condition, and then present the results of our models. Although some conditions required specialised additional comparisons, changes were observed in every condition in receptive field binocularities (figure 2), the fraction of oriented receptive fields devoted to each eye (figure 3) and the joint orientation-binocularity distribution (figure 4). These figures are referred to in each section. In sum, all three models are able to match the changes in receptive field properties observed across all the conditions considered.
Binocularity was measured on a 7 point scale as in Shouval et al. . Values 1 and 7 represent completely monocular responses while values in the middle correspond to at least somewhat binocular responses. (A) In the normal rearing condition, neurons had a range of binocular responses, although there were few completely monocular neurons. (B) In the stripe-reared condition, binocularity increased due to higher inter-ocular correlation caused by the reduction in off-axis spatial frequencies. Experiments have also reported varying amounts of increases in binocular responses. (C) In the orthogonal-reared condition, binocularity decreased. Experiments have also reported such a decrease. (D) In the monocular-reared condition, neurons developed responses primarily for the unoccluded eye, which led to strongly monocular responses for this eye. The primary experimental finding in this rearing condition has been the absence of responses to the occluded eye. (E) Alternating monocular rearing removes inter-ocular correlation as each eye is presented with stimuli only when the other eye is occluded. In the PoE model, this led to strongly monocular responses distributed equally between eyes. Experimentally, the primary finding has been a paucity of binocular responses, but equal responses to each eye. (F) Partial monocular rearing resulted in recovery of receptive fields for both eyes, albeit with fewer binocular neurons. Experimentally, a small amount of binocular experience has been found to result in a significant recovery of responses to the occluded eye, but also an increased degree of monocularity. (G) Strabismus decreases inter-ocular correlation, and thus led in the PoE model to increased monocularity. An increase in monocularity is the primary experimental finding of the effects of strabismus. Errorbars show the SEM. Each condition was repeated times. The binocularity distribution of all the modified rearing conditions were significantly different from the normal rearing condition (, Kolmogorov-Smirnov).
As in experiment, filtering the visual input resulted in a decrease in the fraction of orientation selective neurons (each eye shown separately). Neurons were considered selective when their circular variance was . Errorbars show the SEM. All modified rearing conditions had a significantly different fraction of orien- tation selective neurons compared with the normal condition (, Kolmogorov-Smirnov, each eye tested separately).
Only neurons that had a well defined orientation preference (circular variance ) in at least one eye were included. The color in the bars indicates the ocular dominance of the responses. (A) In the normal reared condition, there was a large over-representation of vertical orientations (90°), and, to a lesser degree, horizontal orientations (0°). Nonetheless, the full range of orientation preferences developed. This over-representation of the cardinal orientations has been reported in experiment, although not always to the same degree (see the Discussion). (B) In the stripe reared condition, there was a significant over-representation of neurons responding to horizontal lines (). These horizontal neurons were also strongly binocular (i.e. mostly green shading). The over-representation of vertical orientations also persisted. The over-representation of the reared orientation is the primary experimental finding in this rearing condition. Cardinal over-representation has not been examined closely in stripe-rearing. (C) In the orthogonally reared condition, there was over-representation of horizontal neurons in the left eye and vertical neurons in the right eye. As found experimentally, these neurons were strongly monocular for the eye that was over-exposed to their preferred orientation. (D) In the monocular reared condition, there was a broad representation of orientation preferences but only for the unoccluded eye. Experimentally, monocular reared animals have normal visual acuity with the non-deprived eye. (E) In the alternating monocular reared case, there is an even distribution of orientation selectivity and strong monocularity. Experimentally, alternate blind reared animals represent all orientations well. (F) In the partial monocular reared condition, there was a recovery of responsivity for both eyes across the full range of orientations. Experimentally, partial monocular reared animals have been demonstrated to have normal visual in each eye (but to suffer from defects in stereo vision). (G) In the strabismic case, there was an increase in monocularity, but with normal orientation coverage. This is in agreement with experiments which have not noted any orientation deficits in strabismic animals. Errorbars show the SEM. All modified rearing cases, except strabismus () had orientation preference distributions significantly different from the normal case (, Kolmogorov-Smirnov).
Normal rearing (figures 1A, 2A, 4A)
As expected from previous work in the monocular case , , , , the binocular receptive fields learned based on normal input were Gabor-like edge detectors (figure 1A). This property broadly survived the modified rearing conditions, up to some degradation and broadening. Given normal input, receptive fields were distributed over the full range of orientations (figure 4A) with primarily binocular responses (figure 2A). Note that the quantification of binocularity in some early experimental results is somewhat subjective, which complicates quantitative comparison. For example, most experimental groups classify monocular cells as ones which respond solely to one eye, which is difficult to define theoretically, since learned receptive fields are never entirely empty.
One salient feature of the orientation distribution is the over-representation of cardinal orientations. There is evidence that some degree of cardinal over-representation is present in normally reared animals  and in the visual environment , although it may be accentuated in our work due to the pixel representation of the training images (see later for further discussion and references).
An additional feature of note is the relationship observed in the normal rearing condition between orientation and binocularity. Li and Atick  examined 2nd-order correlations in visual input and predicted that vertically oriented receptive fields should be more monocular than horizontally oriented ones due to the asymmetry in inter-ocular correlations with horizontal disparity. This asymmetry in encoding can be seen in the normal case (figure 4A), with significantly more monocular receptive fields for vertical orientations. We are not aware of detailed experimental investigation of this phenomena (see Discussion).
One concern we examined for the case of normal input was the robustness of the models to training set size. This is particularly important since, for computational reasons, we trained our models with training examples, which is approximately a factor of four less than the number of degrees of freedom of the overcomplete models. By inspection, receptive fields appeared similar for different training set sizes. To test this more quantitatively, we examined the dependence on size of a key statistic of the receptive fields, namely orientation selectivity (figure 5). We found no dependence. Thus, the sparse constraints of the model result in a fit that is robust to training set size, even in the under-constrained regime.
The PoE model has approximately degrees of freedom. However, for computational reasons, we trained the model with only training examples (each training example is projected onto 150 principal components). We therefore examined the effect of increasing the training set size (for the case of normal input). The receptive field orientation selectivity was robust to changes in training set size. There was also no obvious visual change in receptive field structure (data not shown). This demonstrates that the sparsity constraints result in receptive field formation robust to training set size.
Stripe rearing (figures 1B, 2B, 4B, 6)
Stripe rearing refers to the condition in which animals are raised with visual experience consisting primarily of a single orientation. This can be achieved by the use of cylindrical lenses, lenses painted with stripes, or rearing chambers with striped walls. Early electrophysiological studies on the effects of stripe rearing conflict, with some reports of a complete absence of receptive fields responsive to the unexperienced orientations , , while others found no effect on receptive field distribution , and Freeman and Pettigrew  found a more limited over-representation of the experienced orientation and reduced selectivity to the unexperienced orientations. Later experiments found significant over-representation of the experienced orientation ,  and reduced orientation selectivity for unexperienced orientations. Unlike the other studies, Freeman and Pettigrew  and Blasdel et al.  reported a reduction in binocular responses; however Freeman and Pettigrew  attributed this to misalignment in the oriented lines between the two eyes. Blakemore  found that stripe reared animals have normal levels of binocularity.
More recently, optical imaging techniques have allowed simultaneous characterisation of large regions of V1. Using optical imaging, Sengpiel, Stawinski and Bonhoeffer  found a roughly 60% increase in the cortical area devoted to the over-represented orientation, with no change in the orientation selectivity between the experienced and unexperienced orientations. Tanaka et al. ,  used exclusive goggle rearing with cylindrical lenses and found a much more dramatic 3–6 fold over-representation and increased selectivity (and reduced variance) of the exposed orientation. Tanaka and colleagues also noted that older animals exhibited reduced over-representation despite continued goggle rearing, and that increased dark exposure limited the effect of the goggle rearing.
There are several possible explanations for some of the differences between studies. The method of over-representation varied: some studies used stripe-cylinders, while others used goggles containing lines or strong cylindrical lenses. The age and duration of exposure also varied, and some early studies may have suffered from sampling biases. However, there is broad agreement between studies that stripe rearing leads to increased binocularity and significant increases in over-representation of the exposed orientation. Further, several studies found increased selectivity of the exposed orientation.
We modeled stripe-rearing by filtering the input using oriented Gaussian blurring designed to attenuate the power of off-axis orientations sharply. To maintain stability of the algorithm, 10% normal images were included in the training mixture (as in , see Discussion). The output of the models was consistent with the experimental observations. Receptive fields trained on striped input showed increased binocularity (figure 2B, 8% monocular cells in the stripe-reared condition compared with 17% monocular cells in normal condition, two-sided t-test). As in the experiments, there was a slight reduction in the number of orientation selective responses (figure 3), and the over-exposed orientation (in our case, horizontal) had sharper tuning curves (figure 6) with a smaller variance in their tuning. The size of these changes was dependent on the strength of the input filtering (data not shown), which is another possible explanation for the differing effect sizes seen between groups using different rearing techniques. These changes collectively meant that many receptive fields were less Gabor-like, although orientation selectivity was largely preserved. Since we could not find empirical studies employing methods such as reverse correlation against which to compare our results, it is difficult to determine how faithful this result is. However, the loss of structure of some receptive fields is at least qualitatively consistent with the experimental finding of reduced orientation selectivity in stripe rearing.
In the PoE model, horizontal stripe rearing resulted in significantly sharper orientation tuning for the over-exposed orientation ( Kruskal-Wallis). Experimentally, there are conflicting results regarding the tuning of neurons representing the exposed orientation. Tuning width was measured as the full width at half maximum (FWHM) of the tuning curve.
Orthogonal rearing (figures 1C, 2C, 4C)
Orthogonal rearing is a binocular extension of stripe rearing in which the two eyes are exposed to orthogonal orientations. Hirsch and Spinelli ,  found that orthogonally reared animals had reduced binocularity and almost exclusively monocular responses for cells with well-defined orientations. They also found an almost perfect correlation between receptive field ocular and orientation preferences along with an overall reduction in the fraction of oriented responses. Subsequent groups found similar results although the quantitative changes reported varied, possibly due again to differences in the type of filtering and the strength of the manipulation between experiments. Freeman and Pettigrew  noticed reduced binocularity (25–35% binocular cells for orthogonally reared animals compared with 85% for normally reared animals) and a strong correlation between orientation and ocular preference along with broader orientation tuning away from the over-exposed orientations. Blakemore  also noted reduced binocularity (53% monocular responses).
Leventhal and Hirsch  observed a difference between horizontal and vertical orthogonal rearing and orthogonal rearing with oblique angles. In the cardinal case, they found a strong correlation between ocular and orientation preference. With oblique angles they found a continued dominance of horizontal and vertical orientations, but little response to the non-exposed non-cardinal orientation. In all cases they noted a reduction in binocularity. Stryker et al.  also found a reduction in the number of oriented responses (50% of cells were not responsive or not selective to orientation) and strong correlation between eye and orientation preference. They also observed over-representations of approximately two-fold for the exposed orientations and almost no binocular cells. More recently, albeit as yet only in abstract form, Tani and Tanaka  confirmed the over-representation of the exposed orientations using optical imaging.
As in the stripe reared case, we modelled orthogonal rearing by oriented Gaussian blurring. However, in this condition, the left eye viewed horizontally filtered images while the right eye viewed vertically filtered images. This led to similar results as in the experiments. When trained on this orthogonally filtered input, model receptive fields were significantly more monocular (figure 2C, 31% monocular compared with 17% in the normal case, , two-sided paired ttest), although this effect was not as pronounced as reported experimentally. Responses showed a strong correlation between ocular and orientation preference (figure 4C). Additionally, there was a reduced fraction of oriented responses compared with the normal case (figure 3, 37% compared with 63% in the normal case, ). The results were similar when oblique rearing orientations are considered, although in this case there was also a smaller, cardinal over-representation effect (data not shown). This cardinal over-representation is presumably driven by the same causes as in normal case (discussed further below): cardinal over-representation in the input and the square pixel representation.
Monocular rearing (figures 1D, 2D, 4D)
Monocular deprivation, in which one eye is deprived of visual input, is perhaps the best-studied manipulation. There is substantial variation in deprivation length and daily visual exposure between different studies. We only considered results based on experiments with no recovery period with normal visual input. We examine the recovery of binocular fields later in the section on partial monocular rearing.
Early work by Wiesel and Hubel ,  using electrophysiology found almost no response to the deprived eye (1/84 cells responded). Similarly, Hubel and Wiesel  observed only 7% of cells responsive to the deprived eye after 3 months of deprivation (all cells were classified as having ocular dominance values of 6 or 7 on a scale of 0–7). Blakemore and Van Sluyters  also demonstrated almost complete domination of V1 by the deprived eye, with normal levels of orientation selectivity in the non-deprived eye. Using autoradiography, Shatz, Lindstrom and Wiesel , Stryker  and Shatz and Stryker  found shrinkage of the deprived eye's territory with only 22–25% cortical area labelled by the deprived eye. Olson and Freeman  considered the effects of shorter periods of deprivation, finding pronounced decreases in binocularity after just 2.5 days of deprivation, and almost total loss of responsiveness to the deprived eye after longer periods, compared with 80% binocular responses in the normal animals. Similarly, Peck and Blakemore  found that, with just 20 hours of monocular deprivation, all oriented cells had ocular dominances in the range 4–7. Only a small number of unoriented responses remained exclusive to the deprived eye. Schechter and Murphy  noted 86% of cells responded exclusively to the open eye, 3% to the deprived eye, 3% had binocular responses and 7% were unresponsive. Kratz and Spear  observed a reduced number of orientation selective cells (65% compared with 85% in normal) and reduced direction selectivity (70% compared with 90% in normal). Blasdel and Pettigrew  found that 3 weeks of molecular deprivation led to most cells having ocular dominances of 7, with a small number reported as being 5 and 6. Olson and Freeman  also noted that only 3% of cells responded to the deprived eye (87% in normal) after 10 days of monocular deprivation. Singer et al.  found that the majority of cells were responsive only to the open eye.
Some variations have added insight about the changes occurring during monocular deprivation. Blakemore  and Wilson, Webb and Sherman  demonstrated that there was little difference between monocular deprivation with the nictitating membrane or the full eye-lids, showing that the loss of spatial patterns, rather than the change in luminance, is the critical component of deprivation. Blakemore and Hillman  showed that the open eye dominated whether optically stimulated, or driven electrically. Olson and Freeman  interspersed dark-reared intervals during the deprivation and continued to find almost no response to the deprived eye. Tumosa, Tieman and Hirsch  used behavioral assays to show that the animals had no functional visual acuity in the deprived eye. Mitchell  demonstrated that recovery was improved when the non-deprived eye was occluded during the recovery period. More recent experiments have used optical imaging, and found only 14–18% of cortex responded to the occluded eye  and confirmed that little functional visual acuity remains in the deprived eye .
We used the observation from Wilson, Webb and Sherman  that it is the spatial pattern of the input associated with the deprived eye that matters rather than the overall power to realize a stringent test of the model. We simulated this by using an extremely low-pass boxcar filter on the right eye's input, so that almost all contrast was destroyed. The model reproduced the experimental findings, producing primarily cells with ocular dominance values of 2, and none greater than 4 (figure 2D), indicating no cell responded more strongly to the deprived eye than the open eye. No oriented response was assigned to the deprived eye (figure 3).
Alternating monocular rearing (figures 1E, 2E, 4E)
In alternating monocular rearing, animals are monocularly deprived, but which eye is deprived is alternated regularly (every few hours of visual experience). This removes all inter-ocular correlations, while not favouring the development of either eye. Hubel and Wiesel  first performed this experiment and found that 91% of the resulting cells had monocular responses, evenly distributed between the two eyes. Behaviourally, the animals appeared to have normal spatial acuity in each eye. Blake and Hirsch  also measured normal acuity in each eye but observed defects in stereopsis. They also noted an almost complete absence of binocular responses even after animals were reared with normal input for a year after the critical period. Blakemore  found alternating monocular rearing resulted in 55% of neurons responding monocularly, similar to a strabismic animal. Blasdel and Pettigrew  also found reduced binocularity ( binocularity) except when they used a mechanised device to reduce the alternation interval to less than 1 second. They also observed a low correlation in orientation tuning between the two eyes ( versus in normal animals). Tumosa, Tieman and Hirsch  used behavioural assays and found normal visual acuity in each eye, and equal cortical coverage devoted to each eye . In all of these experiments, alternating monocular rearing had a similar effect to strabismic rearing: reduced binocularity while retaining an equal number of neurons devoted to each eye.
We simulated this condition in a similar way to monocular rearing, with the blind eye having its input low-pass filtered so that little contrast remained. The difference from monocular rearing was that the eye that was blind was alternated for each patch. This resulted in quantitative agreement with experiment. The PoE model predicted (figure 2E) 89% monocular responses (compared with 17% in the normal case, , two-sided paired t-test) and a symmetrical ocular dominance distribution. There was also a reduced fraction of oriented responses (figure 3).
Partial-monocular rearing (figures 1F, 7, 8)
Monocular rearing leads to almost complete loss of function in the deprived eye. There has thus been substantial interest in the question as to what features of the input are necessary for preventing this. Partial-monocular rearing, in which animals are exposed to a small fraction of binocular experience, allows the amount of normal input needed for the maintenance of responses to both eyes to be determined.
Olson and Freeman  monocularly deprived kittens for 4 hours while providing 14–20 hours of binocular experience per day. They found this limited deprivation had little effect. Similarly, Kind et al.  examined the effect of monocular deprivation for 10 days (at 5 weeks old) followed by binocular exposure for 14 days. Again, these kittens had nearly normal visual development, although non-aligned binocular input (i.e. strabismic) led to only 34% coverage for the deprived eye, demonstrating that correlated visual input may be important for recovery. Conversely, Malach and Van Sluyters  found that strabismic binocular input did lead to recovery of binocular responses, but this may be because the animals were dark-reared for 18 hours per day, a manipulation that is known have a protective effect .
Follow-up experiments interleaved binocular experience with monocular deprivation. Mitchell et al.  found that even 0.5 hours of binocular experience with 6.5 hours monocular deprivation preserved moderate spatial acuity in the deprived eye and 2 hours binocular experience with 5 hours monocular experience resulted in normal acuity. Again, they found the inter-ocular correlations were vital, as binocular experience in which artificial strabismus had been induced by prisms resulted in poor recovery of visual acuity. Mitchell et al.  found that splitting the binocular experience into discontiguous blocks impeded recovery.
Later experiments explored the neural basis of these changes. Schwarzkopf et al.  found normal cortical coverage of both eyes for animals with more than 30 minutes daily binocular experience, whether this was matched with 3.5 or 7 hours of monocular deprivation. Vorobyov et al.  examined interocular phase selectivity (a measure of disparity tuning) and found reduced phase selectivity in the partial monocularly reared animals. Mitchell et al.  demonstrated that, while partial monocularly reared animals recovered normal levels of spatial acuity in each eye, most had severe deficits in binocular depth perception (unlike normal animals, their depth estimates did not improve when they were allowed to use both eyes). Mitchell et al.  showed that animals developed normal spatial acuity provided they received at least 30% binocular experience (regardless of total exposure length) . These results broadly agree that a significant response to the deprived eye recovers with as little as binocular input, and that normal levels of spatial acuity occur with 30% binocular input. However, significant deficits in binocular integration remain even with 30% binocular experience.
We simulated this condition by including a fraction of normal visual input along with the same boxcar filtered input used for monocular rearing. This resulted in significant recovery of deprived eye responses with just 10% binocular input (figure 7B), and recovery of equal representation of each eye with 40% binocular input (figure 7E) with the PoE model. However, as in the experiments, recovered responses tended to be monocular, with fewer (67% compared with 83% in normal, two-sided paired ttest) strongly binocular responses. The deprived eye rapidly recovered responses to the full range of spatial frequencies (figure 8) which corresponds well with the behavioural experiments.
This figure shows the ocularity of the receptive fields learned from input with varying fractions of binocular experience (0% corresponds to complete monocular deprivation, 100% to normal binocular experience). (A) Complete monocular deprivation resulted in receptive fields unresponsive to the occluded eye. (B) Just 10% binocular experience led to a substantial recovery of response to the occluded eye. However, the recovered receptive fields were more monocular than in the normal case. (C–J) Further increases in the fraction of binocular experience caused a slow recovery of the number of binocular receptive fields. However, even with 90% normal visual experience, neurons were still significantly more monocular than in the normal case. (K) Full binocular integration requires normal visual input. Errorbars show the SEM. All the partial monocular rearing conditions had binocularity distributions significantly different from the monocular case (, Kolmogorov-Smirnov). Additionally, all cases with partial monocular experience had significantly different binocularity distributions from the normal case (, Kolmogorov-Smirnov).
Spatial acuity in the deprived eye recovered rapidly when a small amount of binocular experience was provided. Even 10% binocular experience (orange) was enough to lead to coverage of higher spatial frequencies. However, full binocular experience was required for complete recovery (blue).
This experimental condition was the only one which showed qualitative differences between the different unsupervised learning models. With the ICA model (Text S1) the recovery from monocular deprivation is much weaker than with the PoE (figure 6) or kmeans clustering (Text S2). Unlike the other models the ICA model is not overcomplete: it has half as many receptive fields to allocate, and may therefore be more susceptible to allocating receptive fields only to the majority input statistics. As we discuss later, there is much evidence that biological V1 is substantially overcomplete.
This recovery appears counterintuitive as it seems the normal input is exerting a disproportionate effect on receptive field development. One explanation may be that, since the sparse coding model strongly penalises representations which are insufficiently sparse, only a small amount of binocular experience is necessary before a significant number of receptive fields are allocated to the deprived eye. As we discuss later, this result suggests that a preferential mechanism for normal input may not be required to explain the recovery observed in partial monocular rearing: rather, it may be a natural consequence of development prior to patterned input.
Strabismic rearing (figures 1G, 2G, 4G)
Strabismus is of both clinical interest , as a condition which affects a significant fraction of the population, and theoretical interest, as it lowers inter-ocular correlations. The effects of convergent or divergent stabismus are similar , . Hubel and Wiesel  used electrophysiology to show that kittens raised with divergent strabismus develop a majority of neurons which are responsive to only a single eye. This finding was reinforced by Shatz, Lindstrom and Wiesel  who used histology to demonstrate that strabismic kittens have more bimodal distribution of ocular dominance columns. Chino et al. , using animals with strabismus greater than 10°, found no neuron in ocular dominance category 4 (strongly binocular). Unlike other investigators, they also found reduced spatial acuity in the deviating eye and a reduced contrast response. Yinon and Auerbach  found that approximately 70% of cells were monocular, and noted an increased number of unresponsive neurons. Similarly, Blakemore  measured 76% monocular responses. In a follow-up experiment they observed no strong binocular response when neurons were stimulated optically, and only a small fraction with direct electrophysiological stimulation . Van Sluyters and Levitt  used prisms rather than surgical manipulation, which allowed them to create symmetric strabismus. In both divergent and convergent conditions, they found a loss of binocularity with the majority of neurons being assigned ocular dominance categories 1, 2, 6 or 7.
Later results confirmed these findings. Levitt and Van Sluyters  found that kittens raised with strabismus during the critical period (2–4 weeks) had cells that were primarily monocular. Grunau  measured 80% binocular responses in normal animals and 26% in strabismic. Berman and Murphy  also noted a loss of binocularity ( binocular simple cells, compared with 65% for controls), and also observed increased receptive field sizes. Kalil, Spear and Langsetmo  found only 7% binocular cells in strabismus. They also reported that animals with divergent strabismus had equal representation of each eye while convergent strabismus resulted in a slight reduction in the representation of the periphery of the deviating eye. Eschweiler and Rauschecker  and Schmidt, Singer and Galuske  both confirmed that the majority of neurons were monocular. Schmidt Singer and Galuske  also found similar orientation preference map characteristics between normal and strabismus (and between maps measured in the deviant and normal eye). Vorobyov et al.  also noted a significant decrease in binocular responses compared with control in strabismic animals.
In sum, there is substantial agreement about the effects of strabismus. With 10–20° deviance, whether divergent or convergent, animals develop 80% monocular responses, no strongly binocular response at all, and a reduced number of responsive cells overall. There are mixed reports regarding preferences for the non-deviating eye.
We simulated the effect of strabismus in the models by choosing visual scene patches independently in each eye (focal points were still identical in each). This disrupts inter-ocular correlations, as each eye views different parts of the scene, and led to similar results to those found in the experiments. With the PoE model, only 2% of cells were in ocular dominance category 4, with the majority being in categories 2 and 6 (figure 2G). Additionally, there was a significant reduction in the total number of orientation selective cells responding to either eye (figure 3). The full range of orientation preferences continued to be expressed (figure 4G).
We have shown above that sparse coding provides an explanation for the RFs that result from several different abnormal rearing conditions. We therefore considered whether there was a novel experiment which could directly address the importance of sparsity in RF envelopment. To do this we exploited that fact that advances in experimental technology have made it possible to rear animals with visual experience which is almost entirely computer-driven (e.g. ). This provides nearly unconstrained scope for modification of visual experience after eye-opening.
We sought a stimulus for which sparsity was central, and that we would predict would lead to markedly different receptive fields from normal animals when used as input. It was important that the difference would persist even if some more naturalistic input was additionally provided, for instance from retinal waves present before eye-opening, or from small amounts of natural visual input that cannot be completely controlled in a practical experiment . We also required the stimulus to be statistically stationary in space, so that it would not be necessary to track eye position in presenting the stimulus.
We constructed stimuli by independently sampling each pixel of the patch from a sparse distribution (student-t with 2 degrees of freedom). This is intentionally close to the distribution of coefficients (rather than pixels) in natural scenes, so as to push the model towards learning the Cartesian basis. As a control, we also considered stimuli with a uniform or Gaussian distribution of coefficients. We examined the receptive fields predicted by the model when trained on mixtures of natural scenes and these noise stimuli. All input data were normalized to have the same mean and variance before combination.
We found that sparse noise provoked a disproportionately strong response in the receptive field development of the models (PoE results in figure 9). Even when trained on a mixture of 50% natural scenes, sparse noise resulted in strongly localized and distinctive receptive fields (figure 10). This effect was particular to sparse noise, as Gaussian or uniform noise had substantially less influence on receptive field development (figure 10).
We created very sparse noise patterns, with only few pixels with substantial input, and mixed those in various proportions with natural scene input (or other distributions in G, H) for input to unsupervised learning. (A) Training the PoE model with 100% sparse noise resulted in highly-localized receptive fields. (B–F) Sparse noise continued to have a marked effect on the learned receptive fields even in the presence of natural scene input. With 50% natural input, receptive fields remained strongly localized (D), and even with 90% natural scenes, some pixel localization is still discernable (F). (G–H) This result was specific to sparse noise with a coefficient distribution near that of natural scenes. Training the PoE model with a mixture of natural scenes and either uniform white noise (G) or Gaussian (H) mixtures produced weaker perturbations of the receptive fields (cf panel D).
(A) The fraction of orientation selective responses (defined as circular variance 0.6) as a function of the fraction of noise input during training (the remaining input always consisted of natural images). When sparse input constituted greater than 50% of the input, very little orientation selectivity remained. Uniform and Gaussian input also impacted the development of orientation selectivity; however, unlike sparse noise, this impact was more gradual. (B) To quantify the extreme localization that is visually apparent in the sparse noise receptive fields, we examined the fractions of weights in each receptive field that were more than 1 standard deviation from the mean of each filter. As with orientation selectivity, when sparse input constituted greater than 50% of the input to the model, most receptive fields were strongly localized to a small number of pixels. The opposite effect occurred with Gaussian and uniform noise, presumably because this input impaired the convergence of the model.
In this one instance, the k-means clustering model results (text S1) contained some deviation from the results of the other two models. In particular, the development of orientation selectivity was impeded more strongly than in the other models by Gaussian and flat noise, and the mixtures of sparse noise and natural scenes developed a high proportion of low-frequency spatial fields. The exact reasons for these deviations are unclear. However, even with k-means clustering the modification to receptive fields trained with sparse input is large, and robust to the inclusion into the training set of a substantial fraction of natural scene input.
We have shown that sparse coding models can simulate the structure of the V1 simple cell receptive fields that arise when animals are reared with normal and abnormal visual input. Receptive field structure exists prior to eye-opening, and there is ample evidence that important aspects of development are driven by intrinsic factors. However, that dramatic changes in receptive field properties occur that depend on the nature of the input suggests that substantial plasticity remains after eye opening. The account of these changes in our model provides evidence of a causal link between receptive field structure and optimal representations of visual input. This directly answers questions  that have been directed at models of purely normal development , , , , ,  about the necessity or sufficiency of an impetus towards sparsity. We suggest that sparse coding provides a unifying framework for modelling receptive field changes under a wide variety of rearing conditions.
Understanding the mechanisms by which visual responsivity can be harmed and indeed potentially cured by aberrant or benificent input has important implications. Take, for instance, the case of partial monocular rearing , –. Monocularly deprived animals do not develop substantial V1 responses to the occluded eye , , , . Recent experiments have demonstrated the recovery of near-normal visual acuity in animals allowed only a small fraction (1/7) of binocular experience. If this result depended on some intrinsic mechanism for detecting and reacting to this small amount of experience, then it might not generalize. Contrary to this, our findings suggest the enticing possibility that improvements may arise as a natural consequence of developmental optimisation of V1 coding.
Our results regarding the development of receptive fields in abnormal rearing conditions are in agreement with previous results which have examined other modalities and a more limited range of visual rearing conditions. Hsu and Dayan  showed that a monocular version of the products of experts model matched the over-representation observed in stripe rearing when trained on stripe filtered input, and a similar result has been found for other algorithms based on sparseness . Saxe et al.  also demonstrated that unsupervised learning algorithms could match receptive field changes in abnormal rearing conditions across a range of different sensory modalities.
We used three unsupervised learning methods – independent component analysis , product of experts  and k-means clustering  to acquire sparse codes, thus ensuring that our results did not depend on the specifics of one particular algorithm. As in Saxe et al. , we found that all these algorithms learned qualitatively similar sparse codes. One advantage of k-means clustering and the product of experts is that they can readily learn over-complete codes. This is important as V1 is indeed significantly over-complete , , . Due to the limitations of the experimental data, it is difficult to make any strong claims about which particular unsupervised learning approach provides a better fit. Although the mechanics of these algorithms are not biologically realistic, similar sparse learning algorithms have been implemented in neurobiologically realistic terms , .
We interpret our results, and those of others based on normal input, as placing the focus on the nature of the efficiency afforded by sparsity , , –. One notion is that sparse codes may represent a trade-off between the metabolic costs associated with neurons that are firing versus those that are quiescent (and maintaining their membrane potentials; ). Of more widespread note is the ability of sparse codes to capture the sort of latent statistical structure in input that can then underpin visual comprehension , , . The idea is that sensory input arises from the superposition of causes that themselves occur only sparsely. These causes are what it is important to determine. Then, finding a sparse representation of the input in a computationally suitable context (formally, in the recognition component of a pair of recognition and generative models; ) can unearth those causes.
The agreement between the experimental results and the output of sparse coding models trained on visual input is perhaps surprising. V1 consists of much more than the feature detectors we have modelled it as here. Real V1 neurons incorporate temporal integration , bottom-up and top-down attentional modulation , –, lateral connections between columns –, feature maps , , and significant feedback from other cortical areas . Additionally, our model does not include the influence of intrinsic activity which is known to be necessary for normal receptive field development , wiring constraints , and hemispheric asymmetries , . These mechanisms may explain the presence of structured receptive fields in dark-reared animals, particularly as spontaneous retinal waves have statistical similarities to natural visual input . However, the success of the model is evidence that, despite many mechanistic differences between the model and V1, sparse codes are good predictors of receptive field changes during the critical period.
Our models favor nurture at the expense of nature. One hint in our results that this favoritism may be too extreme comes from the critical importance of the 10% binocular input in the partial monocular rearing case (and thus the 10% normal input mixed in the stripe and orthogonal rearing conditions). From a technical viewpoint, the primary effect of the normal input is to avoid the collapse of the principal component step which results in the almost complete loss of oriented responses , figure 2b. The choice of 10% is somewhat flexible, and Hsu and Dayan  found that 75–95% stripe reared input resulted in strong over-representation without a collapse in oriented responses. It could be argued that the 10% is a stand-in for the ineluctable effects of the neural structures established prior to eye-opening, although further work would be required to establish this claim. In the models, the effect of normal input on stripe rearing appears to be more proportionate than in blind rearing; we are not aware of experimental tests of partial stripe rearing for comparison.
Trained with normal input, the models over-represented cardinal orientations. There is evidence that cardinal axes are indeed slightly over-represented in real animals , –, presumably due to a prevalence of cardinal edges in natural scenes , , . The degree of over-representation was in reasonably good agreement with a recent study of natural scenes . However, the over-representation in the model may be accentuated because biological retinas are not arranged on a square lattice, unlike most digital representations of images. Such differences in representation are known to exert a small effect on the results of sparse coding models .
Modeling binocularity allowed us to observe an effect on coding due to the asymmetry of inter-ocular correlations in visual input that was predicted by Li and Atick . They showed that, because inter-ocular correlation decays more rapidly with horizontal edges than vertical edges, a redundancy reducing code should have an increased number of monocular receptive fields for vertical edges and an increased number of binocular receptive fields for horizontal edges. In our models, despite higher order interactions not considered by Li and Atick, this effect is observed. Hoyer and Hyvärinen  have previously examined binocular encoding and reported that learned receptive fields had similar disparity preferences to experiment, but did not report on any relationships between orientation and ocularity. To our knowledge this asymmetry has not been observed in experiment. It is possible that supervised constraints on depth estimation  explain why this has not been observed in experiment.
We modelled the six conditions that have been the subject of the most intense investigation. Various other experimental manipulations of visual input have been performed in cats including rearing with random spots , exposure to constant speed and direction of motion –, astigmatism  and opposite rotations of visual input in each eye . We have not attempted to model these results here, either because the studies have not been replicated by other groups or because the effects involved temporal manipulation. In order to constrain the problem size, our models did not include the temporal dimension in receptive field responses, although previous work indicates that this is unlikely to change the results dramatically .
However, we did use the models to design a rearing regime that might provide a novel and strong test of the predictive power of sparse coding. According to our models, animals reared with exposure primarily to sparse white noise (similar to that used as a test of the sparse coding algorithm in Olshausen and Field ), should develop strongly localized receptive fields. This should be discernable with electrophysiological or optical imaging of receptive fields. Our models predict that the developed receptive fields will be small and mostly non-oriented. This effect is specific to noise with a sparsity near that of natural scenes, and Gaussian noise or distributions with a sparsity much greater than that of natural scenes (such as the spot stimuli examined in Ohshiro, Hussain and Weliky ) are predicted to have much less influence, particularly if the animal also receives some naturalistic input . An experimental test of this prediction would provide evidence that sparse coding is a key driver of early receptive field development, or alternatively provide insights into the limits of plasticity during early visual development.
Our models consider only a single cortical area. It would be interesting to look systematically at the effects of the abnormal input statistics on the responses of neurons in higher cortical areas and, concomitantly, on the receptive fields and responses of units in multi-layer, hierarchical ,  unsupervised learning models of those higher areas , –.
Unsupervised learning can only take us so far in understanding brain function. At some point, brains have goals, seek rewards and avoid punishments. However, transforming high-dimensional input into representations that are more useful is an essential part of artificial forms of machine learning , and has offered a critical and realisable metaphor for understanding representations in the brain and the way that they are malleable to changes in the input.
Stereo visual input
We acquired a training set of naturalistic binocular image patches from eighteen stereo images of high-quality, binocular images of natural scenes (from http://home.comcast.net/~toeppen/, this image library was the same as used in Hoyer and Hyvärinen ). Each image was photographed using a binocular camera with lenses spaced approximately a pair of human eye's distance apart at varying focal distances. As in Hoyer and Hyvärinen , 5 focal points were chosen randomly in each image, the two stereo images were aligned at the focal point and then stereo image patches were acquired in a 300×300 pixel square around the focal point. This has the effect of approximating the view of an observer focussing at 5 different points in the scene. Each patch was 2×25×25 pixels. These patches naturally contain varying degrees of disparity. All images were first converted from color to greyscale. A total of 100,000 training patches were created for each rearing condition.
To model modified rearing conditions, the training input was filtered to match the visual experience of the animals. For stripe and orthogonal rearing the off-axis spatial frequencies were attenuated by using an oriented Gaussian filter [as in 42]. For monocular rearing and alternating monocular rearing the occluded eye's images were convolved with a square kernel with a length of 150 pixels, which removed all but extremely low spatial frequencies. To simulate strabismus, the focal points were chosen independently for each eye. In keeping with previous work, 10% of the training input was unfiltered in the stripe and orthogonal rearing conditions. Hsu and Dayan  found that retaining 10% normal input gave a better match with experiment because it reduced the “collapse” of receptive field structure that occurred in the absence of any normal input.
We used three different models for learning sparse codes: product of experts, k-means (which is also known as k-nearest neighbour) and independent component analysis. In all cases, the training data was whitened and dimension reduced using principal component analysis. We retained the first 150 principal components.
We used FastICA  to learn independent components and the built-in MATLAB ‘kmeans’ function to learn k-means clustering. Since these algorithms are well-known we do not describe them further here.
The product of experts model ,  models the input distribution as a product of Student-t distributions. Representing each input patch as a column vector , and the ensemble average over all training examples as , the probability of input is modelled (with neurons) as:(1)(2)The parameters of this model (encapsulated in the term ) describe the receptive field and the sparseness of each neuron . The normalisation constant is dependent on . The model is trained by maximising the log-likelihood of the data with respect to the model parameters . When the number of neurons is equal to the dimensionality of the input there is a closed-form solution for and the model performs independent component analysis . However, when over-completeness is introduced there is no general closed-form solution for the normalisation constant . Contrastive divergence , which performs gradient descent on a cost function that is within a small constant of the log-likelihood function, was used to fit the model with a learning rate of and a batch size of . We used an over-completeness factor of , as in .
Characterisation of receptive field properties
Binocularity was quantified using the index described in Hoyer and Hyvärinen :(3)where and refer to the portions of the receptive field corresponding to each eye and is the norm. For comparison with experiment we binned the values of into 7 bins with boundaries at [as in 150]. Bins 1 or 7 correspond to highly monocular responses while values in the middle correspond to binocular responses.
The orientation and spatial frequency preferences for each receptive field was calculated as in Hyvärinen, Hurri and Hoyer [33, ch. 6]. Each eye was treated separately. The response of the receptive field to a quadrature sinusoidal grating over a range of spatial frequencies and orientations was recorded. We examined spatial frequencies between and 0.5 cycles/pixels with spacing of 0.02 cycles/pixels and all orientations with a spacing of 1°. This provided an orientation response curve from which the circular variance , which is a measure of the orientation selectivity, could be determined. Circular variance is defined as where:(4) is the response of the neuron to stimuli of orientation . When plotting distributions of orientation preference we only included receptive fields which had a circular variance , as receptive fields with low circular variance cannot be reliably assigned an orientation.
In order to test whether the changes observed in the receptive fields under different model conditions were significant, each condition was repeated 25 times with a different pseudo-random seed. The seed affected both the model fitting and the sampling of the training images. Error bars are shown as the standard error of the mean. Except where noted otherwise, tests were performed with a two-sided Kolmogorov-Smirnov test . This test makes only the assumption that the distributions are finite, and tests for differences in both distribution mean and shape.
A replication of all the figures in the manuscript using the independent components analysis model.
A replication of all the figures in the manuscript using the kmeans model.
The authors are grateful to Anne Hsu and Patrick Hoyer for helpful advice. We also thank the anonymous reviewers for their constructive comments that helped improve this work.
Conceived and designed the experiments: JJH PD GJG. Performed the experiments: JJH. Analyzed the data: JJH. Wrote the paper: JJH PD GJG.
- 1. Hubel D, Wiesel T (1959) Receptive fields of single neurones in the cat's striate cortex. J Physiol 148: 574–591.
- 2. Jones JP, Palmer LA (1987) An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. J Neurophysiol 58: 1233–1258.
- 3. Ringach DL (2002) Spatial structure and symmetry of simple-cell receptive fields in macaque primary visual cortex. J Neurophysiol 88: 455–463.
- 4. Fregnac Y, Imbert M (1978) Early development of visual cortical cells in normal and darkreared kittens: relationship between orientation selectivity and ocular dominance. J Physiol 278: 27–44.
- 5. Horton JC, Hocking DR (1996) An adult-like pattern of ocular dominance columns in striate cortex of newborn monkeys prior to visual experience. J Neurosci 16: 1791–807.
- 6. Crair MC, Gillespie DC, Stryker MP (1998) The role of visual experience in the development of columns in cat visual cortex. Science 279: 566–570.
- 7. White LE, Coppola DM, Fitzpatrick D (2001) The contribution of sensory experience to the maturation of orientation selectivity in ferret visual cortex. Nature 411: 1049–1052.
- 8. Elstrott J, Anishchenko A, Greschner M, Sher A, Litke A, et al. (2008) Direction selectivity in the retina is established independent of visual experience and cholinergic retinal waves. Neuron 58: 499–506.
- 9. Hubel DH, Wiesel TN (1977) Ferrier lecture. Functional architecture of macaque monkey visual cortex. Proc R Soc. Lond B 198: 1–59.
- 10. Hubel DH, Wiesel TN (1965) Binocular interaction in striate cortex of kittens reared with artificial squint. J Neurophysiol 28: 1041–1059.
- 11. Wiesel T, Hubel D (1965) Extent of recovery from the effects of visual deprivation in kittens. J Neurophysiol 28: 1060–1072.
- 12. Shatz C, Lindstrom S, Wiesel T (1977) The distribution of afferents representing the right and left eyes in the cat's visual cortex. Brain Res 131: 103–116.
- 13. Shatz CJ, Stryker MP (1978) Ocular dominance in layer IV of the cat's visual cortex and the effects of monocular deprivation. J Physiol 281: 267–283.
- 14. Stryker MP (1978) Postnatal development of ocular dominance columns in layer IV of the cat's visual cortex and the effects of monocular deprivation. Archives Italiennes de Biologie 116: 420–426.
- 15. Sengpiel F, Stawinski P, Bonhoeffer T (1999) Influence of experience on orientation maps in cat visual cortex. Nat. Neurosci 2: 727–732.
- 16. Kind PC, Mitchell DE, Ahmed B, Blakemore C, Bonhoeffer T, et al. (2002) Correlated binocular activity guides recovery from monocular deprivation. Nature 416: 430–433.
- 17. Schwarzkopf DS, Vorobyov V, Mitchell DE, Sengpiel F (2007) Brief daily binocular vision prevents monocular deprivation effects in visual cortex. Eur J Neurosci 25: 270–280.
- 18. Vorobyov V, Schwarzkopf DS, Mitchell DE, Sengpiel F (2007) Monocular deprivation reduces reliability of visual cortical responses to binocular disparity stimuli. Eur J Neurosci 26: 3553–3563.
- 19. Mitchell DE, Kennie J, Schwarzkopf DS, Sengpiel F (2009) Daily mixed visual experience that prevents amblyopia in cats does not always allow the development of good binocular depth perception. J Vis 9: 7–22.
- 20. Li Y, Fitzpatrick D, White LE (2006) The development of direction selectivity in ferret visual cortex requires early visual experience. Nat. Neurosci 9: 676–681.
- 21. Barlow HB (1961) Possible principles underlying the transformations of sensory messages. Sensory Communication 217–234.
- 22. Srinivasan M (1982) Predictive coding: A fresh view of inhibition in the retina. Proc R Soc. Lond B 216: 427–459.
- 23. Zhaoping L (2006) Theoretical understanding of the early visual processes by data compression and data selection. Network 17: 301–334.
- 24. Olshausen B (2003) Principles of image representation in visual cortex. In: Chalupa L and Werner J, editors. The Visual Neurosciences. 1603–1615.
- 25. Barlow H (2001) Redundancy reduction revisited. Network 12: 241–53.
- 26. Olshausen BA, Field DJ (2004) Sparse coding of sensory inputs. Curr. Opin. Neurobiol 14: 481–487.
- 27. Olshausen BA, Field DJ (1996) Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381: 607–609.
- 28. Olshausen BA, Field DJ (1997) Sparse coding with an overcomplete basis set: a strategy employed by V1? Vision Res 37: 3311–3325.
- 29. Smith EC, Lewicki MS (2006) Efficient auditory coding. Nature 439: 978–982.
- 30. Greene G, Barrett DGT, Sen K, Houghton C (2009) Sparse coding of birdsong and receptive field structure in songbirds. Network 20: 162–77.
- 31. Bell AJ, Sejnowski TJ (1997) The “independent components” of natural scenes are edge filters. Vision Res 37: 3327–3338.
- 32. Van Hateren JH, Van der Schaaf A (1998) Independent component filters of natural images compared with simple cells in primary visual cortex. Proc R Soc. Lond B 265: 359–366.
- 33. Hyvärinen A, Hurri J and Hoyer PO (2009) Natural Image Statistics: A Probabilistic Approach to Early Computational Vision. London: Springer-Verlag. 448 p.
- 34. Rehn M, Sommer FT (2007) A network that uses few active neurones to code visual input predicts the diverse shapes of cortical receptive fields. J Comp. Neurosci 22: 135–146.
- 35. Saxe A, Bhand M, Mudur R, Suresh B and Ng AY (2011) Unsupervised learning models of primary cortical receptive fields and receptive field plasticity. In: Shawe-Taylor J, Zemel R, Bartlett P, Pereira F and Weinberger K, Eds. Advances in Neural Information Processing Systems 24. Granada, Spain. 1971–1979.
- 36. Hyvärinen A, Hoyer P (2001) A two-layer sparse coding model learns simple and complex cell receptive fields and topography from natural images. Vision Res 41: 2413–2423.
- 37. Van Hateren JH, Ruderman DL (1998) Independent component analysis of natural image sequences yields spatio-temporal filters similar to simple cells in primary visual cortex. Proc R Soc. Lond B 265: 2315–2320.
- 38. Caywood MS, Willmore B, Tolhurst DJ (2004) Independent components of color natural scenes resemble V1 neurons in their spatial and color tuning. J Neurophysiol 91: 2859–2873.
- 39. Hoyer P, Hyvärinen A (2000) Independent component analysis applied to feature extraction from colour and stereo images. Network 11: 191–210.
- 40. Baddeley R (1996) An efficient code in V1? Nature 381: 560–561.
- 41. Eichhorn J, Sinz F, Bethge M (2009) Natural image coding in V1: How much use is orientation selectivity? PLoS Comp. Biol 5: e1000336.
- 42. Hsu AS, Dayan P (2007) An unsupervised learning model of neural plasticity: Orientation selectivity in goggle-reared kittens. Vision Res 47: 2868–77.
- 43. Rauschecker JP (1991) Mechanisms of visual plasticity: Hebb synapses, NMDA receptors, and beyond. Physiological Reviews 71: 587–615.
- 44. Teh Y, Welling M, Osindero S, Hinton G (2003) Energy-based models for sparse overcomplete representations. J Machine Learning Res 4: 1235–1260.
- 45. Bell AJ, Sejnowski TJ (1995) An information-maximization approach to blind separation and blind deconvolution. Neural Comput 7: 1129–59.
- 46. Coppola DM, White LE, Fitzpatrick D, Purves D (1998) Unequal representation of cardinal and oblique contours in ferret visual cortex. Proc. Natl. Acad. Sci 95: 2621–2623.
- 47. Girshick A, Landy M, Simoncelli E (2011) Cardinal rules: visual orientation perception reflects knowledge of environmental statistics. Nat. Neurosc 14: 926–932.
- 48. Li Z, Atick J (1994) Efficient stereo coding in the multiscale representation. Network 5: 157–174.
- 49. Blakemore C, Cooper GF (1970) Development of the brain depends on the visual environment. Nature 228: 477–478.
- 50. Blakemore C, Mitchell D (1973) Environmental modification of the visual cortex and the neural basis of learning and memory. Nature 241: 467–468.
- 51. Stryker M, Sherk H (1975) Modification of cortical orientation selectivity in the cat by restricted visual experience: A reexamination. Science 168: 904–906.
- 52. Freeman R, Pettigrew J (1973) Alteration of visual cortex from environmental asymmetries. Nature 246: 359–360.
- 53. Blasdel G, Mitchell D, Muir D, Pettigrew J (1977) A physiological and behavioural study in cats of the effect of early visual experience with contours of a single orientation. J Physiol 265: 615–636.
- 54. Stryker MP, Sherk H, Leventhal AG, Hirsch HV (1978) Physiological consequences for the cat's visual cortex of effectively restricting early visual experience with oriented contours. J Neurophysiol 41: 896–909.
- 55. Blakemore C (1976) The conditions required for the maintenance of binocularity in the kitten's visual cortex. J Physiol 261: 423.
- 56. Tanaka S, Ribot J, Imamura K, Tani T (2006) Orientation-restricted continuous visual exposure induces marked reorganization of orientation maps in early life. Neuroimage 30: 462–477.
- 57. Tanaka S, Tani T, Ribot J, O'Hashi K, Imamura K (2009) A postnatal critical period for orientation plasticity in the cat visual cortex. PloS One 4: e5380.
- 58. Hirsch HVB, Spinelli DN (1970) Visual experience modifies distribution of horizontally and vertically oriented receptive fields in cats. Science 168: 869–871.
- 59. Hirsch HV, Spinelli DN (1971) Modification of the distribution of receptive field orientation in cats by selective visual exposure during development. Exp. Brain Res 12: 509–527.
- 60. Leventhal A, Hirsch H (1975) Cortical effect of early selective exposure to diagonal lines. Science 190: 902–904.
- 61. Tani T and Tanaka S (2008) Interocular imbalance in visual experience affects orientation map formation in cat visual cortex, program no. 769.8/kk14. In: Society for Neuroscience Annual Meeting, Washington, DC, abstract only.
- 62. Wiesel T, Hubel D (1963) Single-cell responses in striate cortex of kittens deprived of vision in one eye. J Neurophysiol 26: 1003–1017.
- 63. Wiesel TN, Hubel DH (1965) Comparison of the effects of unilateral and bilateral eye closure on cortical unit responses in kittens. J Neurophysiol 28: 1029–40.
- 64. Hubel D, Wiesel T (1970) The period of susceptibility to the physiological effects of unilateral eye closure in kittens. J Physiol 206: 419–436.
- 65. Blakemore C, Van Sluyters R (1974) Reversal of the physiological effects of monocular deprivation in kittens: further evidence for a sensitive period. J Physiol 237: 195.
- 66. Olson C, Freeman R (1975) Progressive changes in kitten striate cortex during monocular vision. J Neurophysiol 38: 26–32.
- 67. Peck CK, Blakemore C (1975) Modification of single neurons in the kitten's visual cortex after brief periods of monocular visual experience. Exp. Brain Res 68: 57–68.
- 68. Schechter PB, Murphy EH (1976) Brief monocular visual experience and kitten cortical binocularity. Brain Res 109: 165–8.
- 69. Kratz KE, Spear PD (1976) Effects of visual deprivation and alterations in binocular competition on responses of striate cortex neurons in the cat. J Comp. Neurol 170: 141–51.
- 70. Blasdel G, Pettigrew J (1978) Effect of prior visual experience on cortical recovery from the effects of unilateral eyelid suture in kittens. J Physiol 274: 601–619.
- 71. Olson CR, Freeman RD (1978) Monocular deprivation and recovery during sensitive period in kittens. J Neurophysiol 41: 65–74.
- 72. Singer W, Yinon U, Tretter F, et al. (1979) Inverted monocular vision prevents ocular dominance shift in kittens and impairs the functional state of visual cortex in adult cats. Brain Res 164: 294.
- 73. Wilson J, Webb S, Sherman S (1977) Conditions for dominance of one eye during competitive development of central connections in visually deprived cats. Brain Res 136: 277–287.
- 74. Blakemore C, Hillman P (1977) An attempt to assess the effects of monocular deprivation and strabismus on synaptic efficiency in the kitten's visual cortex. Exp. Brain Res 30: 187–202.
- 75. Olson C, Freeman R (1980) Cumulative effect of brief daily periods of monocular vision on kitten striate cortex. Exp. Brain Res 38: 53–56.
- 76. Tumosa N, Tieman S, Hirsch H (1982) Visual field deficits in cats reared with unequal alternating monocular exposure. Exp. Brain Res 47: 119–129.
- 77. Mitchell D (1988) The extent of visual recovery from early monocular or binocular visual deprivation in kittens. J Physiol 395: 639–660.
- 78. Mitchell DE, Kind PC, Sengpiel F, Murphy K (2003) Brief daily periods of binocular vision prevent deprivation-induced acuity loss. Curr. Biol 13: 1704–1708.
- 79. Blake R, Hirsch HVB (1975) Deficits in binocular depth perception in cats after alternating monocular deprivation. Science 190: 1114–1116.
- 80. Blasdel G, Pettigrew J (1979) Degree of interocular synchrony required for maintenance of binocularity in kitten's visual cortex. J Neurophysiol 42: 1692–1710.
- 81. Tumosa N, Tieman SB, Tieman DG (1989) Binocular competition affects the pattern and intensity of ocular activation columns in the visual cortex of cats. Vis. Neurosci 2: 391–407.
- 82. Malach R, Van Sluyters R (1989) Strabismus does not prevent recovery from monocular deprivation: A challenge for simple Hebbian models of synaptic modification. Vis. Neurosci 3: 267–273.
- 83. Mitchell DE, Kind PC, Sengpiel F, Murphy K (2006) Short periods of concordant binocular vision prevent the development of deprivation amblyopia. Eur J Neurosci 23: 2458–66.
- 84. Mitchell D, Sengpiel F, Hamilton D, Schwarzkopf D, Kennie J (2011) Protection against deprivation amblyopia depends on relative not absolute daily binocular exposure. J Vis 11: 1–11.
- 85. Mitchell DE, Kennie J, Duffy KR (2011) Preference for binocular concordant visual input in early postnatal development remains despite prior monocular deprivation. Vision Res 51: 1351–9.
- 86. Beauchamp GR, Black BC, Coats DK, Enzenauer RW, Hutchinson AK, et al. (2005) The management of strabismus in adults–III. The effects on disability. Journal of AAPOS 9: 455–9.
- 87. Yinon U, Auerbach E (1975) The ocular dominance of cortical neurons in cats developed with divergent and convergent squint. Vision Res 15: 1251–1256.
- 88. Freeman RD, Tsumoto T (1983) An electrophysiological comparison of convergent and divergent strabismus in the cat: electrical and visual activation of single cortical cells. J Neurophysiol 49: 238–53.
- 89. Chino YM, Shansky MS, Jankowski WL, Banser Fa (1983) Effects of rearing kittens with convergent strabismus on development of receptive-field properties in striate cortex neurons. J Neurophysiol 50: 265–86.
- 90. Van Sluyters R, Levitt F (1980) Experimental strabismus in the kitten. J Neurophysiol 43: 686–699.
- 91. Levitt F, Van Sluyters R (1982) The sensitive period for strabismus in the kitten. Dev. Brain. Res 3: 323–327.
- 92. Grunau MW von (1982) Comparison of the effects of induced strabismus on binocularity in area 17 and the LS area in the cat. Brain Res 246: 325–9.
- 93. Berman N, Murphy EH (1982) The critical period for alteration in cortical binocularity resulting from divergent and convergent strabismus. Dev. Brain Res 2: 181–202.
- 94. Kalil RE, Spear PD, Langsetmo A (1984) Response properties of striate cortex neurons in cats raised with divergent or convergent strabismus. J Neurophysiol 52: 514–37.
- 95. Eschweiler GW, Rauschecker JP (1993) Temporal integration in visual cortex of cats with surgically induced strabismus. Eur J Neurosci 5: 1501–9.
- 96. Schmidt K, SingerWand Galuske R (2004) Processing deficits in primary visual cortex of amblyopic cats. J Neurophysiol 91: 1661–1671.
- 97. Ohshiro T, Hussain S, Weliky M (2011) Development of cortical orientation selectivity in the absence of visual experience with contour. J Neurophysiol 106: 1923–1932.
- 98. Hunt JJ, Ibbotson M, Goodhill GJ (2012) Sparse coding on the spot: Spontaneous retinal waves suffice for orientation selectivity. Neural Comput 24: 2422–33.
- 99. Einhäuser W, König P (2010) Getting real-sensory processing of natural stimuli. Curr. Opin. Neurobiol 20: 389–95.
- 100. Lewicki MS (2002) Efficient coding of natural sounds. Nat. Neurosci 5: 356–363.
- 101. Hinton G (2002) Training products of experts by minimizing contrastive divergence. Neural Comput 14: 1771–1800.
- 102. Lloyd S (1982) Least squares quantization in PCM. IEEE Trans. on Information Theory 28: 129–137.
- 103. Savin C, Joshi P, Triesch J (2010) Independent component analysis in spiking neurons. PLoS Comp. Biol 6: e1000757.
- 104. Zylberberg J, Murphy JT, Deweese MR (2011) A sparse coding model with synaptically local plasticity and spiking neurons can account for the diverse shapes of V1 simple cell receptive fields. PLoS Comp. Biol 7: e1002250.
- 105. Doya K, Ishii S, Pouget A and Rao RP, eds. (2011) Bayesian Brain: Probabilistic Approaches to Neural Coding. Cambridge, Massachusetts: MIT Press. 344 p.
- 106. Geisler WS (2008) Visual perception and the statistical properties of natural scenes. Annu. Rev. Psychol 59: 167–192.
- 107. Simoncelli EP, Olshausen BA (2001) Natural image statistics and neural representation. Annu. Rev. Neurosci 24: 1193–1216.
- 108. Attwell D, Laughlin SB (2001) An energy budget for signaling in the grey matter of the brain. Journal of Cerebral Blood Flow and Metabolism 21: 1133–45.
- 109. Mumford D (1994) Neuronal architectures for pattern-theoretic problems. In: Large-scale neuronal theories of the brain. Ed. by C Koch and J Davis. Cambridge MA: MIT Press. 125–152.
- 110. Hinton GE, Ghahramani Z (1997) Generative models for discovering sparse distributed representations. Proc R Soc. Lond B 352: 1177–90.
- 111. Motter BC (1993) Focal Attention Produces Spatially Selective Processing in Visual Cortical Areas Vl, V2, and V4 in the Presence of Competing Stimuli. J Neurophysiol 70: 909–919.
- 112. Ito M, Gilbert C (1999) Attention modulates contextual influences in the primary visual cortex of alert monkeys. Neuron 22: 593–604.
- 113. Poghosyan V, Ioannides AA (2008) Attention modulates earliest responses in the primary auditory and visual cortices. Neuron 58: 802–813.
- 114. Bosking WH, Zhang Y, Schofield B, Fitzpatrick D (1997) Orientation selectivity and the arrangement of horizontal connections in tree shrew striate cortex. J Neurosci 17: 2112–2127.
- 115. Das A, Gilbert CD (1999) Topography of contextual modulations mediated by short-range interactions in primary visual cortex. Nature 399: 655–61.
- 116. Stettler DD, Das A, Bennett J, Gilbert CD (2002) Lateral connectivity and contextual interactions in macaque primary visual cortex. Neuron 36: 739–750.
- 117. Goodhill GJ (2007) Contributions of theoretical modeling to the understanding of neural map development. Neuron 56: 301–311.
- 118. Swindale N (2008) Visual map. Scholarpedia 3: 4607.
- 119. Lund JS, Lund RD, Hendrickson AE, Bunt AH, Fuchs AF (1975) The origin of efferent pathways from the primary visual cortex, area 17, of the macaque monkey as shown by retrograde transport of horseradish peroxidase. J Comp. Neurol 164: 287–303.
- 120. Gjorgjieva J, Eglen SJ (2011) Modeling developmental patterns of spontaneous activity. Curr. Opin. Neurobiol 21: 1–6.
- 121. Chklovskii DB (2000) Binocular disparity can explain the orientation of ocular dominance stripes in primate primary visual area (V1). Vision Res 40: 1765–1773.
- 122. Albert MV, Schnabel A, Field DJ (2008) Innate visual learning through spontaneous activity patterns. PLoS Comp. Biol 4: e1000137.
- 123. Furmanski CS, Engel SA (2000) An oblique effect in human primary visual cortex. Nat. Neurosci 3: 535–6.
- 124. Li B, Peterson MR, Freeman RD (2003) Oblique effect: a neural basis in the visual cortex. J Neurophysiol 90: 204–17.
- 125. Hansen B, Essock E (2004) A horizontal bias in human visual processing of orientation and its correspondence to the structural components of natural scenes. J Vis 4: 1044–1060.
- 126. Mannion DJ, McDonald JS, Clifford CWG (2010) The influence of global form on local orientation anisotropies in human visual cortex. NeuroImage 52: 600–5.
- 127. Mannion DJ, Mcdonald JS, Clifford CWG (2010) Orientation Anisotropies in Human Visual Cortex. J Neurophysiol 103: 3465–3471.
- 128. Coppola DM, Purves HR, McCoy AN, Purves D (1998) The distribution of oriented contours in the real world. Proc. Natl. Acad. Sci 95: 4002.
- 129. Dragoi V, Turcu CM, Sur M (2001) Stability of cortical responses and the statistics of natural scenes. Neuron 32: 1181–1192.
- 130. Doi E, Inui T, Lee TW, Wachtler T, Sejnowski TJ (2003) Spatiochromatic receptive field properties derived from information-theoretic analyses of cone mosaic responses to natural scenes. Neural Comput 15: 397–417.
- 131. Franz A, Triesch J (2007) Emergence of disparity tuning during the development of vergence eye movements. IEEE 6th International Conference on Development and Learning 31–36.
- 132. Sluyters RC, Blakemore C (1975) Innate and environmental factors in the development of the kitten's visual cortex. J Physiol 248: 663–716.
- 133. Cynader M, Berman N, Hein A (1975) Cats raised in a one-directional world: effects on receptive fields in visual cortex and superior colliculus. Exp. Brain Res 22: 267–80.
- 134. Daw BYNW, Wyatt HJ (1976) Kittens reared in a unidirectional environmental: evidence for a critical period. J Neurophysiol 257: 155–170.
- 135. Berman BYN, Daw NW (1977) Comparison of the critical periods for monocular and directional deprivation in cats. J Physiol 265: 249–259.
- 136. Cynader M, Mitchell D (1977) Monocular astigmatism effects on kitten visual cortex development. Nature 270: 177–178.
- 137. Bruce CJ, Isley MR, Shinkman PG (1981) Visual experience and development of interocular orientation disparity in visual cortex. J Neurophysiol 46: 215–28.
- 138. Bengio Y (2009) Learning Deep Architectures for AI. Hanover, MA: Now Publishers Inc. 144 p.
- 139. Lee H, Grosse R, Ranganath R, Ng AY (2009) Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. Proceedings of the 26th Annual International Conference on Machine Learning 1–8.
- 140. Hoyer PO, Hyvärinen A (2002) A multi-layer sparse coding network learns contour coding from natural images. Vision Res 42: 1593–1605.
- 141. Perrinet L, Samuelides M, Thorpe S (2004) Sparse spike coding in an asynchronous feed-forward multi-layer neural network using matching pursuit. Neurocomputing 57: 125–134.
- 142. Schwartz O, Sejnowski TJ, Dayan P (2006) Soft mixer assignment in a hierarchical generative model of natural scene statistics. Neural Comput 18: 2680–718.
- 143. Lee H, Ekanadham C, Ng AY (2007) Sparse deep belief net model for visual area V2. NIPS 8.
- 144. Karklin Y, Lewicki M (2008) Emergence of complex cell properties by learning to generalize in natural scenes. Nature 457: 83–86.
- 145. Köster U, Hyvärinen A (2010) A two-layer model of natural stimuli estimated with score matching. Neural Comput 22: 2308–33.
- 146. Coen-Cagli R, Dayan P, Schwartz O (2012) Cortical Surround Interactions and Perceptual Salience via Natural Scene Statistics. PLoS Comp. Biol 8: e1002405.
- 147. Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313: 504–7.
- 148. Hyvarinen A (1999) Fast and robust fixed-point algorithms for independent component analysis. IEEE transactions on Neural Networks 10: 626–634.
- 149. Osindero S, Welling M, Hinton GE (2006) Topographic product models applied to natural scene statistics. Neural Comput 18: 381–414.
- 150. Shouval H, Intrator N, Law CC, Cooper LN (1996) Effect of binocular cortical misalignment on ocular dominance and orientation selectivity. Neural Comput 8: 1021–1040.
- 151. Ringach D, Shapley R, Hawken M (2002) Orientation Selectivity in Macaque V1: Diversity and Laminar Dependence. J Neurosci 22: 5639.
- 152. Marsaglia G, Tsang W, Wang J (2003) Evaluating Kolmogorov's distribution. Journal of Statistical Software 8: 1–4.
- 153. Hubel DH, Wiesel TN (1962) Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J Physiol 160: 215.
- 154. Payne BR and Peters A (2001) The concept of cat primary visual cortex. In: Payne BR and Peters A, eds. The cat primary visual cortex. San Diego: Academic Press. 1–108.