Optimal Learning Rules for Discrete Synapses
Figure 2
Information storage capacity per synapse versus the number of synaptic inputs, for dense (p = 0.5), sparse (p = 0.05), and very sparse (p = 0.005) coding. Lines show analytic results, while points show numerical results. For small number of synapses, each additional synapse contributes equally to the information. However, for many synapses, information per synapse decreases as .