Fig 1.
Lossless compression via predictive coding.
Feedforward (a) and feedback (b) predictive coding circuits (as in Eq (1)), showing the circuit structure necessary for reconstructing the input. (c) Lossless reconstruction (black dots) of an input (blue) from the transmitted output of a predictive coding circuit (red). The mean power up to each point in time (solid bars) for the input (blue) and transmission (red) demonstrates the reduction in the mean transmitted power via predictive coding.
Fig 2.
Parameters of the optimized linear predictive coding algorithm.
(a) β = β(τs) (b) Λ* = Λ*(β, σ): Eq (6) for two values of β.
Fig 3.
Schematic neural networks implementing predictive coding, through feedforward (a) or feedback (b) inhibition.
Networks’ parameters are: (a) Feedforward: ,
. (b) Feedback: α, Γ. In both circuits, the unit that outputs pt is termed the principal cell, and the unit that outputs nt the interneuron. There are two synapses onto each interneuron: (1) its external input and (2) a representation of its internal dynamics (right of the dashed line). We can describe its internal dynamics in this way because the interneuron is a leaky integrator, i.e. its internal dynamics (Eq (8)) are an infinite sum of past outputs, discounted at each time step by a fixed multiplier. Therefore, they can be represented by a delayed input across a synapse, with a multiplicative discounting factor, as in (a) and (b).
Fig 4.
Rectification nonlinearity and its effect on the feedback inhibitory circuit.
(a) Rectification nonlinearity (black), with threshold at v = 1. Linearized responses in color (cyan: min; magenta: max). (b) Nonlinear feedback inhibitory network. The nonlinearity (inset) is applied to the interneuron’s output. Nonlinearities with increasing thresholds are colored (green: min; red: max) (Methods). (c) Network gain at different input frequencies (note: the frequency space is in Z-space, i.e. defined with respect to the fixed time step of the network). The three colored curves are the nonlinear network response curves (computed using describing function analysis, colored as in (b)) (Methods). The dotted lines provide extreme parameter values for the linear network: Γ = 0,1.
Fig 5.
Comparing the performance (measured through network gain) of nonlinear and linear feedback inhibitory networks (measured through network gain: lower is better) (details in Methods).
(a) Two input mixtures, modeling rapid transition from predictable to unpredictable input components. (b) Description of simulations. Inputs constructed as in (a). At each time point, inputs are either pure predictable signal, or pure unpredictable noise, with an instantaneous transition from one type to the other, in the middle of the simulation period. (c-f) Simulation outputs (inputs shown in inset). The amplitude of the unpredictable component of the mixture varies along the x axis. Error bars are 1 std. dev. (c,e) Network gain of the linear network of type 1, optimized to the mixture (blue, non-adapted linear response), is significantly higher than that of the nonlinear network (red). In contrast, the nonlinear network gain is close to the response of the optimal linear network of type 2, which is allowed to adapt to each component of the mixture (dotted black, adapted linear response). (e) Green shading indicates region where the nonlinear response is more than one std. dev. lower than the non-adapted linear response. Diagonal hashing indicates region where the nonlinear response is within one std. dev. of the adapted linear response. (d,f) % improvement of the performance of the nonlinear network over the type 1 linear network at different amplitudes of the unpredictable component. Data taken from (c) and (e) respectively. (f) Green box indicates region where the improvement is more than one std. dev. different from 0.
Fig 6.
Linear filters from spike-triggered correlation analyses, estimated at different times around a sudden shift from low (L) to high (H) contrast.
(a,b) Adapted from [27]. (a) Timeline of experiment. (b) Experimentally measured filters (low contrast, blue; high contrast, red). High contrast filter is computed only from the response to the first 200 ms of high contrast stimulus (see timeline in (a)). (c) Simulated response filters of the principal cell of the nonlinear predictive coding circuit (Methods), showing oscillatory structure.
Fig 7.
Response filters of zebra finch auditory neurons ((a), adapted from [28]) compared to theoretically simulated response filters of a nonlinear feedback inhibitory circuit (d).
(a-c) Adapted from [28]. (a) Linear response filters for a single neuron stimulated with inputs of different mean sound amplitudes (colored to match (d)). (b) Ratio of the total positive to total negative component of neuronal filters is computed for an input with high mean amplitude, and an input with low mean amplitude. Each circle shows these values for a different recorded neuron (one example being (a)). The colored circle is derived from the simulated response filters of the principal cell of the nonlinear circuit (d). It uses the ratio of the total positive to total negative component of the simulated filters (plotted in (e)) at the highest input amplitude (red), and the lowest input amplitude (cyan). (c) The BMF (freq. of the peak of the Fourier transform of the linear response filters) for a high mean input, and a low mean input (for different neurons, one example being (a)). The colored circle is derived from the simulated responses of the principal cell of the nonlinear network (d). It uses the BMFs (shown in (f)) of the simulated filters at the highest input amplitude (red) and the lowest input amplitude (cyan). (d) Linear filters estimated to best approximate the response of the principal cell of the nonlinear circuit to white noise stimuli of different amplitudes (Methods). Curves colored to match (a). (e) The ratio of the total positive component of the curves in (d) to the total negative component. Points are colored as in (a,d). The ratios derived from inputs with highest (red) and lowest amplitude (cyan) define the colored circle in (b). (f) The BMF of the responses filters from (d). The BMF values derived from inputs with highest (red) and lowest amplitude (cyan) define the colored circle in (c).