Sparse Coding Models Can Exhibit Decreasing Sparseness while Learning Sparse Codes for Natural Images
To check that our conclusions apply to other models besides SAILnet, we performed simulations with the publicly available SparseNet code of Olshausen and Field , . (A) When the basis functions are initialized with large-amplitude white noise (see text for details), the sparseness increases over time contrary to the ferret data shown in Fig. 1. (B) However, when the bases are initialized with small-amplitude white noise, the sparseness decreases over time.