The Equivalence of Information-Theoretic and Likelihood-Based Methods for Neural Dimensionality Reduction

doi:10.1371/journal.pcbi.1004141

The Equivalence of Information-Theoretic and Likelihood-Based Methods for Neural Dimensionality Reduction

Fig 5

A second example Bernoulli neuron for which ${\hat{k}}_{M I D}$ fails to identify the most-informative one-dimensional subspace.

The stimulus space has two dimensions, denoted s₁ and s₂, and stimuli were drawn iid from a standard Gaussian (0,1). (A) The nonlinearity f(s₁,s₂) = p(spike|s₁,s₂) is excitatory in s₁ and suppressive in s₂; brighter intensity indicates higher spike probability. (B) Contour plot of the stimulus-conditional densities given the two possible responses: “spike” (red) or “no-spike” (blue), along with the raw stimulus distribution (black). (C) Information carried by silences (I₀), single spikes (I_ss), and total Bernoulli information (I_Ber = I₀+I_ss) as a function of subspace orientation. The MID estimate ${\hat{k}}_{M I D} = 90^{\circ}$ is the maximum of I_ss, but the total Bernoulli information is in fact 13% higher at ${\hat{k}}_{B e r} = 0^{\circ}$ due to the incorporation of no-spike information. Although both stimulus axes are clearly relevant to the neuron, MID identifies the less informative one. As with the previous figure, equations (19) and (20) detail the equivalence between I_Ber and ℒ_lnb, so that this figure can be viewed from either an information-theoretic or likelihood-based perspective.

doi: https://doi.org/10.1371/journal.pcbi.1004141.g005