Optimal prediction with resource constraints using the information bottleneck
Fig 1
A schematic representation our predictive information bottleneck.
On the left hand side, we have coordinates Xt evolving in time, subject to noise to give Xt+Δt. We construct a representation, , that compresses the Xt (minimizes
) while retaining as much information about Xt+Δt (maximizes
) up to the weighting of the prediction compared to the compression set by β.