Optimal prediction with resource constraints using the information bottleneck
Fig 5
Example of a sub-optimal compression.
An optimally predictive, compressed representation, in panel (a) compared to a suboptimal representation, in panel (b) for a prediction at Δt = 1 in the future, within the underdamped regime (ζ = 1/2). We fix the mutual information between the representations and Xt ( bits), but find that, as expected, the suboptimal representation contains significantly less information about the future.