The geometry of representational drift in natural and artificial neural networks
Fig 5
Artificial neural networks: Hyperparameter fits and drift geometry as a function of Δt and variance explained.
(a) Measure of fit to experimental data, Ztotal see Eq (19), as a function of noise hyperparameters, p (top labels) or σ (bottom labels). Dots are best fits, for which additional data is plotted here and in supplemental figures (S5 Fig, Methods). [b-c] Various metrics as a function of the time between earlier and later session, Δt. Colored curves are linear regression fits. All data is averaged over 10 initializations. (b) Average magnitude of drift relative to magnitude of mean response vector. (c) Average percent of drift vector that lies in the variational space of initial session. [d-i] Various drift metrics and their dependence on PC dimension of the earlier session’s variational space. Metrics are plotted as a function each PCi’s ratio of variance explained, vi, of the corresponding stimulus group Colored curves are linear regression fits. Grey curves are behavioral data fits from the novel sessions shown in Fig 4c, 4d and 4e. Middle row is for networks with additive Gaussian noise (σ = 0.1) and bottom row is with node dropout (p = 0.5). All data is averaged over 10 initializations. (d, g) Magnitude of drift along PCi direction, relative to full magnitude of drift. (e, h) Angle of drift with respect to PCi direction. (f, i) Post-drift variance explained along PCi direction (dotted line is equality). Linear regression fit to log(var. exp.).