Table 1.
List of notations.
Fig 1.
Covariance spectrum under random Gaussian connectivity.
A. Compare theory (Eq (5)) with finite-size network covariance using Eq (2) at N = 100, g = 0.5. The histogram of eigenvalues is a single realization of the random connectivity. B. Same as A. at N = 400. C. Covariance eigenvalue distribution at various value of g. As g increases the distribution develops a long tail of large eigenvalues. D. Dimension (normalized by network size) vs g. The dots and error bars are mean and sd over repeated trials from finite-size networks (Eq (2) and use Eq (4)). Note some error bars are smaller than the dots E. Covariance eigenvalues vs. their rank (in descending order). The circles are covariance eigenvalues from a single realization of the random connectivity with N = 100 (Eq (2)). The crosses are predictions based on the theoretical pdf (Eq (5)). F. Same as E. but for g = 0.9 and on the log-log scale. The red dashed line is the power law with exponent −3/2 derived from Eq (5), see Section 3.2.
Fig 2.
A. The exact pdf (solid line) of the covariance spectrum compared with the power-law approximation (dashed line, Eq (7)) at g = 0.7. Inset shows the log-log scale. B. Same as A. for g = 0.8. The approximation improves as g approaches the critical value 1. C. The log error between the exact pdf and approximation as a function of g and “distance” from the support edges. We quantify this “distance” as the minimum ratio of x/x− and
(more details and motivations in Section A.2 in S1 Text. The plot shows the log error is small when this ratio is large, which means x being far away from the edges. The dashed line shows the attainable region of the ratio which increases with g.
Fig 3.
Covariance spectrum for the symmetric and anti-symmetric random connectivity.
A. The pdf of a covariance spectrum with random symmetric J with different g (note for stability). B. Same as A., but for random anti-symmetric Jij = −Jji. The pdf diverges at x = 1 as
.
Fig 4.
A. Compare theoretical covariance spectrum for random connectivity with reciprocal motifs and a finite-size network covariance using Eq (2)(g = 0.4, κ = 0.4, N = 400). B. The impact of reciprocal motifs on dimension for various gr = g/gc (Eq (18)). For small gr, the dimension increases sharply with κ. C. The spectra at various κ while fixing g = 0.4. The black dashed line is the i.i.d. random connectivity (κ = 0). D. Same as C. but fixing relative gr = 0.4 to control the main effect (see text). The changes in shape are now smaller and the support narrows with increasing κ.
Fig 5.
Robustness of the covariance spectrum to low rank perturbations of the connectivity.
A. Eigenvalues of a Gaussian random connectivity J (Eq (3)), g = 0.4, N = 400. As N → ∞, the limiting distribution of eigenvalues is uniform in the circle with radius g ([34] red solid line). The black dashed line is the 0.995 quantile of the eigenvalue radius calculated from 1000 realizations. B. Same as A. but for the rank-1 perturbed J + xuvT. ,
and x = 4.03. This example also corresponds to adding diverging motifs (Section 3.3 and Section C.1 in S1 Text). C. The histogram of covariance eigenvalues (Eq (2)) under the J in A. D. The bulk histogram of eigenvalues with J + xuvT has little change and remains well described by the Gaussian connectivity theory (red line, Eq (5)). Besides the bulk, there are two outlier eigenvalues to the left and right (inset, arrows) E,F, Analytical predictions (solid and dashed lines) of the outlier locations given g and |x| when u, v are (asymptotically) orthogonal unit vectors that are independent of J (see other cases in Section C.3 in S1 Text). The y-axis is the outlier location subtracting the corresponding edge x±, Eq (6), so it is zero for small |x| before the outlier emerges (dashed line). The dots are the mean of the smallest (for the left outlier) or largest (right outlier) eigenvalues averaged across 100 realizations of the random J, N = 4000. The errorbars are the standard error of the mean (SEM, many are smaller than the dots).
Fig 6.
A. One realization of the covariance eigenvalues by Eq (2) with an EI network satisfying the stability condition (see text). The bulk spectrum is well described by the Gaussian random connectivity theory (solid line, Eq (5)). There is one small outlier to the left of the bulk (arrow). The parameters are g = w0 = 0.4, ,
,
,
, K = 60, N = 1000. To improve the accuracy of the theory to finite K, N, here we use a slightly modified connection weight,
, for all excitatory non-zero connections, and similarly
for inhibitory connections, such that
holds exactly for finite N. B. Similar as A but for balanced EI network (see text) with kαβ = k = 1, g = 0.4, K = 40, N = 400. Note there are two outliers on both sides of the bulk.
Fig 7.
Effects of sampling in time and space on the covariance spectrum.
A. For the i.i.d. Gaussian random connectivity, how different levels of time samples α change the spectrum (Eq (22)). The non-sampled case corresponds to α = 0. g is fixed at 0.4. Inset: The relative dimension vs. α (Eq 23). The dots correspond to the pdfs with matched colors. B. Same as A. but for the spatial subsampling (Section I.2 in S1 Text), at g = 0.5. The non-sampled case corresponds to f = 1. The relative dimension in the inset is based on Section I.1 in S1 Text.
Fig 8.
Fitting the theoretical spectrum to data.
A. The anatomical map of neurons (dots) in the example functional clusters (different colors) across a larval zebrafish brain (scale bar is 50 μm, see text and [52]). B. Comparing the fitting error of the time-sampled random connectivity theory (Section 3.7) and the Marchenko–Pastur law. The errors are measured by Eq (38). The red dashed line is the diagonal. labeled on each plot is the relative dimensionality (Eq 4). The calcium activity is recorded at a frame rate of 2 Hz and a total of 600 frames of spontaneous activity [52] are used in here. See more details in Methods. Fitting results for all other clusters are in Fig O in S1 Text.
Fig 9.
Covariance spectrum in nonlinear dynamics.
A. Top: A histogram of covariance eigenvalues calculated from firing rate activities ri(t) of simulating a N = 400 network model according to Eq 33. The eigenvalues are normalized to have a mean equal to 1 for easy comparison of the shape (or equivalently the eigenvalues of the correlation matrix Section 3.8). Here g = 0.4 and σ = 0.5 (see Methods for additional numerical details). The green dashed line is the time-sampled theoretical spectrum (Section 3.7) using actual g and α, only shown in A for clarity of the plots. The length of the simulated data corresponds to α = N/T = 0.1. The orange curve is also the time-sampled theoretical spectrum except for using an effective that is fitted numerically to best match the simulated eigenvalues. Bottom: The blue curve is the histogram of hi(t) (aggregated across i and t) and the orange dashed curve is the histogram of ri(t). The overlaying red curve shows the nonlinearity ϕ(x) as a reference. The 〈⋅〉 in the title of each plot represents averaging ϕ′(hi(t)) over all i and t. B-D. Same as A except for σ = 1 and g = 0.6, 0.8, 1.2, respectively. Only the fitted theoretical spectrum (orange curve) is shown for clarity.
Fig 10.
Covariance spectra under some deterministic connectivity models.
A. Histogram of the covariance eigenvalues of a ring network with a long-range connection profile (inset, N = 100). Most eigenvalues are close to 1 and the rest of eigenvalues converge to discrete locations predicted by top Fourier coefficients (crosses) of the connection profile (Eq (36)). B. Same as A. but for a ring network with Nearest-Neighbor connections: Ji−1 = 0.4, Ji+1,i = 0.2. The solid line is theoretical spectrum in large N limit which has two diverging singularities at both support edges. The effect of such singularities is also evident in the finite-size network at N = 400 (a single realization). C-F. Higher dimensional Nearest-Neighbor ring network (ad = 0.6, see Methods). As the dimension increases, the singularities in the pdf become milder and less evident, and the overall shape becomes qualitatively similar to the random connectivity case (Fig 1).