Fig 1.
Comparison of low- and high-variance networks.
A) Cartoon of network dynamics in time. Time is measured in units of τx. Light (dark) gray corresponds to low (high) firing rates. With low synaptic variance, fluctuations in firing rates are small, and a relatively fixed and dense subset of units contribute to firing. Right: firing rate traces of five example units (each with a distinct color). Gray arrow indicates the extent of fluctuations in the network. B) Same as A except for the high-variance model. The network exhibits a small and shifting ensemble of cells that respond robustly at any given time. The magnitude of fluctuations is increased substantially (right). C) Mean response in both networks follows a scaling (fits to the data yield
for high variance and ∼ 1/K0.513 for low variance; J0 is adjusted so that
values in the two networks overlaps). D) Fraction of active units (inverse sparsity). High-variance model exhibits a rapid sparsening in K while, in the low-variance network, this fraction remains roughly constant. E) Mean response of the active subset. The trend in D is flipped: the low-variance network exhibits a rapidly vanishing μ, which is not the case in the high-variance model. Input current I0 is set to one. F) Network’s response to the external input current I0 with K = 1000. G) Despite similar
values, the high-variance network is more sparsely active by more than twofold. H) Active neurons respond more robustly in the high-variance network than in its low-variance counterpart. (Model parameters: J0 = 2 for high variance and 1.05 for low variance, g = 2, Jij ∼ gamma, N = K, ϕ = [tanh]+).
Fig 2.
Sparse balance yields non-Gaussian dynamics and a subthreshold mean.
Distribution of currents x (over time and units) for gamma-distributed synapses. Dashed lines denote the mean of each distribution, i.e., . Area above threshold (set to zero; solid line) corresponds to the fraction of active units f. A) With low synaptic variance (ν = 1), the distribution of x is a Gaussian centered around a mean that tends to zero for larger K. B) Same as in A except for high synaptic variance (ν = 1/2). Note the larger range of the horizontal axis compared to B. The distribution is no longer Gaussian.
is relatively insensitive to K and lies below threshold. (Model parameters are g = J0 = 2, I0 = 1, Jij ∼ gamma, N = K, ϕ = [tanh]+).
Fig 3.
Asynchronous irregular activity in the sparse balance model.
A) Responses of network neurons in time for four different nonlinear response functions: Heaviside step function, rectified tanh, rectified linear, and rectified quadratic. B) Rates ϕ(x) (dark) superimposed on the currents x (light) for four example units. Cells respond robustly and infrequently across choices of the response functions. The synchrony index, as defined in [19], is approximately 10−4 for each of the networks shown. C) Fractions of active neurons, or the inverse sparsity. D) Normalized distributions for the fraction of ON-time, defined as the fraction of (simulation) time a unit spends above threshold. For better visualization, histograms are smoothened using kernel density estimation. E) Normalized distributions of x, showing non-Gaussian dynamics. F) Population-averaged autocorrelation functions of x. At this fixed value of in-degree (K = 1000), all response functions produce qualitatively similar results. (Model parameters: g = J0 = I0 = 2, Jij ∼ gamma, N = K).
Fig 4.
Time-scale of fluctuations adjusts to maintain sparse activity.
A) Population-averaged autocorrelation function of the synaptic input normalized by its zero-lag value. Note the faster decay of the autocorrelation for increasing K. B) The decorrelation rate β is constant in the low-variance network but increases logarithmically with K in the sparse balance model, resembling the (inverted) trends of sparsity (Fig 1D). C) β (solid) and the ratio var(x)/var(η) (dashed) in the high-variance model plotted on the same panel, aligned to different y-axes. var(⋅) refers to the total variance; similar result is obtained for the temporal variance. Error bars indicate SEM, averaged over 10 random realizations of the connectivity. (Model parameters: J0 = 2 for high variance and 1.05 for low variance, g = 2, I0 = 1, Jij ∼ gamma, N = K, ϕ = [tanh]+).
Fig 5.
Comparison between network simulations and dynamic mean-field theory (DMFT).
Mean response (A) and fraction active (B) as functions of in-degree K. A crucial feature of the sparse balance model is the rapid decay of mean response and sparsity, but not the mean active response (Fig 1E), with increasing K. C) Autocorrelation of the synaptic input Rη(τ), normalized by its zero-lag value, exhibits more rapid decorrelation for larger K. D) β grows logarithmically as a function of K. Blue data points illustrate averages over 10 random realizations of the connectivity; SEM error bars fall roughly within the size of data points. Crosses indicate self-consistent solutions to Eqs (23)–(25) and show good agreement with the simulation results. (Model parameters: J0 = 3, g = 2, I0 = 1, Jij ∼ Gaussian, ϕ = [tanh]+).
Fig 6.
Asynchronous irregular activity in an E-I network with small input current.
A) Responses of 500 excitatory (red) and 500 inhibitory (blue) units in two networks, one with lognormal (left) and the other with gamma (right) weight distributions. Responses are sparse and distributed across the population. B) Rates (dark) superimposed on the currents (light) for four example cells from each population. Response is infrequent as fluctuations occasionally push the current above threshold. C) Fraction of active units for individual populations (red and blue) and across the entire network (gray). The inhibitory population is more active than its excitatory counterpart. D) Fraction of (simulation) time units spend above threshold for each population and connectivity distribution. This distribution is wide and skewed. Both choices of the connectivity distribution produce qualitatively similar results. (Model parameters: g = 1, JEE = JIE = 1, JEI = 2, JII = 1.2, IE = 2, II = 1, NE = NI = 3000, K = 600, ϕ = [tanh]+).