The Equivalence of Information-Theoretic and Likelihood-Based Methods for Neural Dimensionality Reduction
Fig 6
Lower bound on the fraction of total information neglected by MID for a Bernoulli neuron, as a function of the marginal spike probability p(spike) = p(r = 1), for the special case of a binary stimulus.
Information loss is quantified as the ratio I0/(I0+Iss), the information due to no-spike events, I0, divided by the total information due to spikes and silences, I0+Iss. The dashed gray line shows the lower bound derived in the limit p(spike) → 0. The solid black line shows the actual minimum achieved for binary stimuli s ∈ {0,1} with p(s = 1) = q, computed via a numerical search over the parameter q ∈ [0, 1] for each value of p(spike). The lower bound is substantially loose for p(spike) > 0, since as p(spike) → 1, the fraction of information due to silences goes to 1.