Sparse Coding Models Can Exhibit Decreasing Sparseness while Learning Sparse Codes for Natural Images
(A) For illustration purposes, we show 100 of the 250 macaque V1 receptive fields (courtesy of D. Ringach) against which we compared our SAILnet model neuronal RFs. To estimate the fraction of the variance in the pixel values of these RFs that could realistically be explained, we performed registration-based RF fitting for each macaque RF, using all other macaque RFs as comparators. The best-fit matches to the RFs in panel A are shown in panel B, and the distribution of values obtained with the macaque-vs-macaque fitting shows that, on average, approximately of the variance in the macaque RF pixel values can be explained by other macaque RFs (E). For the SAILnet model that experienced increasing sparseness during training, the best-fit RF matches are shown in panel C, and the distribution of corresponding values shows that, on average, approximately of the variance in the macaque RFs can be explained by these SAILnet model neuronal RFs (E). Similarly, for the SAILnet model that experienced decreasing sparseness during training (D), approximately of the variance in the macaque RFs can be explained by model neuronal RFs. For both of the SAILnet networks shown (C,D), there are a few macaque RFs for which the image registration fails completely. We include these in our goodness-of-fit statistics; they correspond to the cells with values near zero. There is no clear trend in the shapes of macaque RFs that cause this failure. For either increasing, or decreasing sparseness during training, the model neuronal RFs can, on average, account for approximately of the explainable variance in the measured macaque RFs. The error bars on the bars in panel (E) correspond to the standard deviation of the values over the sample of experimental RFs. There is a statistically significant, although small in magnitude, difference between the quality with which the macaque RFs fit each other (E), and the quality with which either model fits the data ( for either model, using a paired t test; n = 250). There is no statistically significant difference between the quality with which the two models fit the macaque RFs (, paired t test; n = 250).