Fig 1.
A. Left: model architecture. Right: sample activity traces from three randomly chosen neurons. B. Rank-one recurrent connectivity. C. Illustration of a sample activity trajectory in the high-dimensional space where each axis corresponds to the activity of a different neuron. Activity (black arrow) is given by the sum of two components (Eq 7, blue arrows); the direction of the component generated from recurrent interactions is fixed, and is aligned with the connectivity vector m.
Fig 2.
Rank-one RNN receiving one-dimensional stochastic inputs.
A. Model architecture. B. Activity covariance is low-dimensional, and is spanned by connectivity vector m together with the external input vector u. As a consequence, activity is contained within the plane collinear with these two vectors. C–D–E. Example of a simulated network with . In C: covariance spectrum. Components larger than 10 are not displayed (they are all close to zero). In D: overlap between the dominant principal components estimated from simulated activity and the theoretically-estimated PCs (left), or the vectors m and u (right). Overlaps are quantified via Eq 4, with input vectors u chosen to be normalized. Note that here, but not in G, only one principal component can be identified. In E: simulated activity projected on the two dominant PCs. F–G–H. Same as in C–D–E, example with
.
Fig 3.
Rank-one RNN receiving high-dimensional stochastic inputs.
A. Model architecture. B. The activity covariance is high-dimensional, with all eigenvalues taking identical values except for two – one larger and one smaller. The principal components associated with these two eigenvalues lie within the plane spanned by the connectivity vectors m and n. C. Covariance eigenvalues as a function of overlap between connectivity vectors. The dashed vertical line indicates the value of for which dynamics become unstable. Black arrows indicate the value of
that is used for simulations in Fig 4. D. Dimensionality. Horizontal black lines indicate the maximum (N) and the minimum (1) possible values. E. Components of
(or PC1 vector, left) and
(or PCN vector, right) along connectivity vectors m and n, as from Eq 24. F. Overlap (Eq 4) between the principal components
and
(after normalization) and the connectivity vectors m and n.
Fig 4.
Rank-one RNN receiving high-dimensional stochastic inputs.
A–B–C. Example of a simulated network with . In A: covariance spectrum. In B: overlap between two principal components (the strongest and the weakest) estimated from simulated activity and the theoretically-estimated vectors
and
(top), or vectors m and n (bottom). Overlaps are quantified via Eq 4. In C: simulated activity projected on two different pairs of PCs. D–E–F. Same as in A–B–C, example with
. Note that, although the qualitative behaviour of activity in the two examples is similar, activity in the example network in A–B–C is overall higher dimensional.
Table 1.
Summary of the main results obtained for recurrent neural networks with rank-one connectivity.
In the second column, we report the directions defining the part of the covariance matrix (Eq 11) that originates from recurrent activity. (Note that also input directions contribute, via the term .) In the fourth column, the structure of connectivity vectors determines the precise value taken by dimensionality. For intermediate input dimensionality, covariance spectra have not been formally analyzed. Simulations suggest that, among the C + 1 eigenvalues that do not vanish,
are approximately fixed to the reference value, while three depend on connectivity.
Fig 5.
Stochastic activity in rank-two recurrent neural networks.
A. Rank-two connectivity. B. Eigenvalues of the covariance matrix that are different than the reference value . As connectivity is rank-two, four eigenvalues are perturbed; we sort them in ascending order. Violin plots show the distribution of perturbed eigenvalues for different values of the parameters
and
. Note that, for all sets of parameters, two eigenvalues are increased and two decreased with respect to
. C. Dimensionality as a function of
and
. The black dashed lines indicate the parameter values for which dynamics become unstable. The tiny black square indicates the parameter values that are used for simulations in F–G. In both B and
, we keep the values of
and
fixed to zero (see Methods 7). D–E. Same as for B–C, but for a different parametrization, where we keep
and
fixed to zero. F–G. Example of a simulated network, parameters indicated in C. In F: covariance spectrum. In G: overlap between four selected principal components (the strongest and the weakest) estimated from simulated activity and the theoretically-estimated covariance eigenvectors (left) and the connectivity vectors (right). Overlaps are quantified via Eq 4. The theoretical expressions for this case are reported in Methods 7.
Fig 6.
Stochastic activity in a rank-one excitatory-inhibitory circuit.
A. E-I circuit with high-dimensional inputs. B. Variance explained by PC1 (top) and PC2 (bottom) as a function of the overall recurrent connectivity strength w and the relative dominance of inhibition g. In B–C–D, the black solid line separates the regions for which the non-zero eigenvalue λ is larger or smaller than one. The black dotted line separates the regions for which the non-zero eigenvalue λ is larger or smaller than zero. Note the different color scales in the top and bottom plots. C. Overlap between PC1 and the sum (top) and diff (bottom) directions. D. Non-zero eigenvalue of the synaptic connectivity matrix −
. E. E-I circuit with one-dimensional inputs. F. Variance explained by PC1 (top) and PC2 (bottom) as a function of the overall recurrent connectivity strength w and the direction of the input vector u. The input direction is parametrized by an angle θ (see Methods 8), so that
(resp.
) correspond to inputs entering only E (resp. I), while
(resp.
) corresponds to inputs aligned with the sum (resp. diff) direction. G. Variance explained by PC1 (top) and PC2 (bottom) as a function of the relative dominance of inhibition g and the direction of the input vector u. H. Overlap between PC1 and the sum (top) and diff (bottom) direction.