Optimal prediction with resource constraints using the information bottleneck
Fig 11
Encoding schemes with m > 2 representation variables.
The steady state is plotted as a dotted line and the representation for each realization of the value of are plotted as solid lines. The representations which carry maximum predictive information for (a) m = 2 at
bit, and (b) m = 3 at
bits. The optimal representations at large m tile space more finely and have higher predictive information. The optimal representations for m = 200 at fixed β = 1.01 (
,
) (c) and β = 20 (
,
). (d) At low
, many of the representations are redundant and do not confer more predictive information than the m = 2 scheme. A more explicit comparison is given in S3 Fig. At high
, the degeneracy is lifted. All computations done at N = 100, Nμ = 0.2, Ns = 0.001, Δt = 1.