Sparse Coding Models Can Exhibit Decreasing Sparseness while Learning Sparse Codes for Natural Images
Figure 3
SAILnet multi-unit activity can become less sparse during receptive field formation.
A SAILnet simulation was performed in which the RFs, firing thresholds, and recurrent connection strengths were initialized with random numbers (see Methods section for details). (A) These initial RFs are shown for 196 randomly selected model neurons. Each box on the grid shows the RF of one neuron, with white corresponding to positive pixel values, and black corresponding to negative ones. (B) After training with natural images, these same SAILnet neurons have oriented, localized RFs. (C) All three of our “multi-unit” sparseness measures decrease during the training period, as has been observed in the visual cortex of maturing ferrets [14]. We made similar observations when we made measurements of single-neuron sparseness values (data not shown).