## Figures

## Abstract

The brain exhibits temporally complex patterns of activity with features similar to those of chaotic systems. Theoretical studies over the last twenty years have described various computational advantages for such regimes in neuronal systems. Nevertheless, it still remains unclear whether chaos requires specific cellular properties or network architectures, or whether it is a generic property of neuronal circuits. We investigate the dynamics of networks of excitatory-inhibitory (EI) spiking neurons with random sparse connectivity operating in the regime of balance of excitation and inhibition. Combining Dynamical Mean-Field Theory with numerical simulations, we show that chaotic, asynchronous firing rate fluctuations emerge generically for sufficiently strong synapses. Two different mechanisms can lead to these chaotic fluctuations. One mechanism relies on slow I-I inhibition which gives rise to slow subthreshold voltage and rate fluctuations. The decorrelation time of these fluctuations is proportional to the time constant of the inhibition. The second mechanism relies on the recurrent E-I-E feedback loop. It requires slow excitation but the inhibition can be fast. In the corresponding dynamical regime all neurons exhibit rate fluctuations on the time scale of the excitation. Another feature of this regime is that the population-averaged firing rate is substantially smaller in the excitatory population than in the inhibitory population. This is not necessarily the case in the I-I mechanism. Finally, we discuss the neurophysiological and computational significance of our results.

## Author Summary

Cortical circuits exhibit complex temporal patterns of spiking and are exquisitely sensitive to small perturbations in their ongoing activity. These features are all suggestive of an underlying chaotic dynamics. Theoretical works have indicated that a rich dynamical reservoir can endow neuronal circuits with remarkable computational capabilities. Nevertheless, the mechanisms underlying chaos in circuits of spiking neurons remain unknown. We combine analytical calculations and numerical simulations to investigate this fundamental issue. Our key result is that chaotic firing rate fluctuations on the time scales of the synaptic dynamics emerge generically from the network collective dynamics. Our results pave the way in the study of the physiological mechanisms and computational significance of chaotic states in neuronal networks.

**Citation: **Harish O, Hansel D (2015) Asynchronous Rate Chaos in Spiking Neuronal Circuits. PLoS Comput Biol 11(7):
e1004266.
https://doi.org/10.1371/journal.pcbi.1004266

**Academic Editor: **Olaf Sporns,
Indiana University, UNITED STATES

**Received: **December 22, 2014; **Accepted: **April 3, 2015; **Published: ** July 31, 2015

**Copyright: ** © 2015 Harish, Hansel. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited

**Data Availability: **All relevant data are within the paper and its Supporting Information files.

**Funding: **This work was funded by Agence Nationale de la Recherche, DH, France-Israel Laboratory of Neuroscience, DH, and France-Israel High Council for Science and Technology, DH. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

**Competing interests: ** The authors have declared that no competing interests exist.

## Introduction

Single cell recordings [1] and electro-encephalography [2, 3] suggest the existence of chaotic dynamics in the brain. Consistent with chaotic dynamics, *in-vivo* experiments have demonstrated that cortical circuits are sensitive to weak perturbations [4, 5]. Remarkably, the misplacement of even a single spike in a cortical network has a marked effect on the timing of subsequent spikes in the network [6].

Chaotic states in extended dynamical systems can be classified as synchronous or asynchronous, depending on the spatial patterns of the dynamics. In synchronous chaos the temporal fluctuations exhibit spatial correlations. If the temporal fluctuations are spatially incoherent, the chaotic state is classified as asynchronous.

EEG measures the activity of a large population of neurons. Therefore, it is probable that chaoticity observed in EEGs reflects synchronous chaos in brain regions of rather large size. Models of local cortical circuits exhibiting synchronous chaos have been studied in [7–12]. A computational advantage of synchronous chaos in the brain is that it enables neuronal populations to respond quickly to changes in their external inputs [7] and facilitates the access of the network to states (e.g. limit cycles or fixed points) that encode different stimuli [3]. A large body of experimental data, however, has reported that cortical neurons exhibit very weak correlations [13, 14] and thus are more compatible with asynchronous than with synchronous chaos. Moreover, recent studies have demonstrated that the richness, the complexity and the high dimension of the dynamics in systems operating in asynchronous chaos endows them with remarkable computational capabilities [15–18]. The present paper focuses on the mechanisms underlying the emergence of asynchronous chaos in local neuronal circuits.

Asynchronous chaos was studied in a seminal work by Sompolinsky, Crisanti and Sommers (SCS) [19], who investigated a large network of *N* neuronal-like units fully connected with random weights drawn from a zero mean Gaussian distribution (called hereafter as the SCS model). The dynamics of the network are those of a “rate” model [20], in which the activity of a unit, *S*(*t*), is characterized by a continuous variable which is a non-linear function, *S* = *ϕ*(*h*), of the total input to the unit. In the SCS model the activity variables take values between [-1, 1] and the function *ϕ*(*h*) is sigmoidal and odd. Using Dynamical Mean-Field Theory (DMFT) SCS showed that if the standard deviation of the weight distribution is sufficiently large, the dynamics bifurcate from fixed point to asynchronous chaos. The SCS model in its original form or in its discrete time version has been used in numerous studies in theoretical and computational neuroscience [15–17, 21–25].

However, the connectivity of the SCS model violates Dale’s Law, whereby in biological networks a given neuron is either excitatory or inhibitory [26]. Also, the equation of the SCS model dynamics are invariant under the transformation *h* → −*h*, a symmetry not fulfilled in more realistic neuronal network models. More importantly, as this is the case frequently for rate models, the physiological meanings of the dynamical “neuronal” variables and of the parameters are not clear in the SCS network. Should these variables and the time constant of their dynamics—which sets the time scale of the chaotic fluctuations—be interpreted as characterizing neurons, or synapses?

In this paper we address the following general and fundamental issues: To what extent are asynchronous chaotic states generic in networks of spiking neurons? How does this depend on single neuron properties? How do excitation and inhibition contribute to the emergence of these states? To what extent these chaotic dynamics share similarities with those exhibited by the SCS model? We first study these questions in one population of inhibitory neurons receiving feedforward excitation. We then address them in networks of two populations, one inhibitory and the other excitatory, connected by a recurrent feedback loop. A major portion of the results presented here constitutes the core of the Ph.D thesis of one of the authors (O.H) [27].

## Results

### One population of inhibitory neurons: General theory

We consider *N* randomly connected inhibitory spiking neurons receiving an homogeneous and constant input, *I*. The voltage of each neuron has nonlinear dynamics, as e.g. in the leaky integrate-and fire (LIF model, see Materials and Methods) or in conductance-based models [20].

The connection between two neurons is *J*_{ij} = *JC*_{ij} (*i*, *j* = 1, 2…*N*), with *J* ≤ 0, and *C*_{ij} = 1 with probability *K*/*N* and 0 otherwise. The outgoing synapses of neuron *j* obey
(1)
where *S*_{j}(*t*) is the synaptic current at time *t* and *τ*_{syn} the synaptic time constant. When neuron *j* fires a spike (time ), *S*_{j} increments by *J*. Thus, the total input to neuron *i*, *h*_{i}(*t*) = *I*+∑_{j} *J*_{ij} *S*_{j}(*t*), satisfies:
(2)
We assume *K* ≫ 1, hence the number of recurrent inputs per neuron is . Scaling *J* and *I* as: , , the time-averaged synaptic inputs are and their spatial (quenched) and temporal fluctuations are *O*(1) [28, 29]. Finite neuronal activity requires that excitation and inhibition cancel to the leading order in *K*. In this *balanced* state, the mean and the fluctuations of the *net* inputs are *O*(1) [28, 29]. The properties of the balanced state are well understood if the synapses are much faster than all the typical time constants of the intrinsic neuronal dynamics [30]. Temporally irregular *asynchronous* firing of spikes is a hallmark of this regime [13, 28, 29, 31, 32]. However, this stochasticity does not always correspond to a true chaotic state [28, 29, 33–36]. In fact, this depends on the spike initiation dynamics of the neurons [37]. The opposite situation, in which some of the synapses are slower than the single neuron dynamics, remains poorly understood. This paper mostly focuses on that situation.

When the synaptic dynamics is sufficiently slow compared to the single neuron dynamics, the network dynamics can be reduced to the set of non-linear first order differential equations:
(3) (4)
where *r*_{i}(*t*) is the instantaneous firing rate of neuron *i* and *g*(*h*) is the neuronal input-output transfer function [20]. These are the equations of a *rate model* [20, 38] in which the activity variables correspond to the net synaptic inputs in the neurons. Eqs (3)–(4) differ from those of the SCS model in that they have a well defined interpretation in terms of spiking dynamics, the time constant has a well defined physiological meaning, namely, the *synaptic* time constant, the transfer function quantifies the *spiking* response of the neurons and is thus positive, the interactions satisfy Dale’s law and the neuronal connectivity is partial.

#### Dynamical mean-field theory (DMFT).

We build on a DMFT [19] to investigate the dynamics, Eqs (3)–(4), in the limit 1 ≪ *K* ≪ *N*. Applying this approach, we rewrite the last two terms in the right hand side of Eq (3) as a Gaussian noise whose statistics need to be self-consistent with the dynamics. This yields a set of self-consistency conditions which determine the statistics of the fluctuations, from which the synaptic net inputs and the firing rates of the neurons can be calculated. This approach is described in detail in the Materials and Methods section.

The DMFT shows that, for a given transfer function, depending on the parameters *J*_{0} and *I*_{0}, the dynamics either converge to a fixed point state or remain in an asynchronous, time-dependent state. In the fixed point state, the net inputs to the neurons, , (*i* = 1…*N*) are constant. Their distribution across the population is Gaussian with mean *μ* and variance . The DMFT yields equations for *μ*, *q*, as well as for the distribution of firing rates (*i* = 1…*N*) (Eqs (24)–(25) and (36)). In the time-dependent state, *h*_{i}(*t*) exhibit Gaussian temporal fluctuations, which are characterized by a mean, *μ* = [⟨*h*(*t*)⟩], and a population-averaged autocovariance (PAC) function, *σ*(*τ*) = [⟨*h*(*t*)*h*(*t*+*τ*)⟩] − *μ*^{2} ([⋅] and ⟨ ⋅ ⟩ denote means over the population and over time, respectively). Solving the set of self-consistent equations which determine *σ*(*τ*) and *μ* (Eqs (25), (27) and (37)–(38)) indicates that *σ*(*τ*) decreases monotonically along the flow of the deterministic dynamics, thus suggesting that the latter are chaotic. To confirm that this is indeed the case one has to calculate the maximum Lyapunov exponent of the dynamics (which characterizes the sensitivity of the dynamics to initial conditions [39]) and verify that it is positive. This can be performed analytically in the framework of DMFT [19]. However, this is beyond the scope of the present paper. Therefore, in the specific examples analyzed below we rely on numerical simulations to verify the chaoticity of the dynamics.

For sufficiently small *J*_{0}, the fixed point state is the only solution of the dynamics. When *J*_{0} increases beyond some critical value, *J*_{c}, the chaotic solution appears. We show in the Materials and Methods section that *J*_{c} is given by:
(5)
where *q* and *μ* are computed at the fixed point state and .

#### On the stability of the fixed point state.

The NxN matrix characterizing the stability of the fixed point is with **I** the NxN identity matrix and:
(6)
where is the total input in neuron *j* at the fixed point. This is a sparse random matrix with, on average, K non zero elements per line or column. In the limit *N* → ∞, these elements are uncorrelated, have a mean and variance (for large *N*, the second moment of the matrix elements is equal to their variance). Interestingly, Eq (5) means that the SD of the elements of **M** crosses 1 (from below) at *J*_{c}. As *J*_{0} increases, the fixed point becomes unstable when the real part of one of the eigenvalues crosses 1. Note that that for large *K*, **D** always has a negative eigenvalue, which is .

In the specific examples we investigate below, simulations show that when the chaotic state appears the fixed point becomes unstable. This implies that for *J* < *J*_{c} given by Eq (5) the real parts of all the eigenvalues of are smaller than 1 and that for *J* = *J*_{c}, the real part of one of the eigenvalues, the eigenvalue with maximum real part, crosses 1. This suggests the more general conjecture that in the limit 1 ≪ *K* ≪ *N* the eigenvalue with the largest real part of is:
(7)
Below we compare this formula to results from numerical diagonalization of .

### One population of inhibitory neurons: Examples

The above considerations show that when synapses are slow, the dynamics of inhibitory networks is completely determined by the transfer function of the neurons. Therefore, to gain insights into the way dynamics become chaotic in such systems we proceed by investigating various spiking models that differ in the shape of their transfer functions.

#### Sigmoidal transfer functions.

Neurons in a strong noise environment can be active even if their net inputs are on average far below their noiseless threshold, whereas when these inputs are large the activity saturates. The transfer functions of the neurons can therefore be well approximated by a sigmoid. Specifically here we consider the dynamics described in Eqs (3)–(4) with a sigmoidal transfer function:
(8)
This form of the sigmoid function makes analytical calculations more tractable. Fig 1A shows that for *J*_{0} = 4, *I*_{0} = 1, the simulated network dynamics converge to a fixed point. This is not the case for *J*_{0} = 6 and *J*_{0} = 15 (Fig 1B, 1C). In these cases the activities of the neurons keep fluctuating at large time. Note also that the mean level of activity is different for the three neurons. This is a consequence of the heterogeneities in the number of inputs the neurons receive.

Activity of 3 neurons in simulations (*N* = 32,000, *K* = 800, *τ*_{syn} = 10 ms). A: *J*_{0} = 4. B: *J*_{0} = 6. C: *J*_{0} = 15.

These differences in the network dynamics for these three values of *J*_{0} are consistent with the full phase diagram of the DMFT in the parameter space *I*_{0} − *J*_{0}. Fig 2A depicts the results obtained by solving numerically the self-consistent equations that define chaos onset with *g*(*x*) = *ϕ*(*x*) (Eqs (17)–(18) in S2 Text). In the region above the line a chaotic solution exists whereas it does not exist below it. Simulations indicate that in the region above the line, very small perturbations from the fixed point state drive the network toward the time dependent state. In other words, the fixed point solution is unstable above the line: the bifurcation to the time dependent state is thus supercritical.

A: Phase diagram. Solid line: DMFT; Dots indicate where the largest Lyapunov exponent, Λ, changes sign in simulations (*N* = 32,000, *K* = 800, *τ*_{syn} = 10 ms). Inset: Λ vs. *J*_{0}. *I*_{0} = 2 (black), 4 (red), 6 (blue). Parameters used in Fig 1A, 1B abd 1C are marked by ×, + and ▫, respectively. B: *σ*(*τ*) for *I*_{0} = 1, *J*_{0} = 15. Black: DMFT. Red and blue dots: Simulations for *N* = 32,000, *K* = 800, and *N* = 256,000, *K* = 2000, respectively (results averaged over 8 network realizations).

The instability of the fixed point on this line is also confirmed by direct diagonalization of the matrix (see Eq (6)). To this end, we solved numerically the mean field equations for different values of *J*_{0} to obtain *μ* and *q*, randomly sampled values from the distribution defined by *μ* and *q* to generate the random matrix matrix , and then computed numerically the spectrum of the matrix (for *N* = 10000). Examples of the results are plotted in Fig 3A for two values of *J*_{0}, one below and one above the critical value *J*_{c}. In both cases, the bulk of the spectrum is homogeneously distributed in the disk of radius *λ*_{max} centered at the origin. Fig 3B plots *λ*_{max} computed numerically (dots) and compare the results to our conjecture, Eq (7) (solid line). The agreement is excellent. The instability of the fixed point corresponds to *λ*_{max} crossing 1.

The matrix was diagonalized numerically for *N* = 10000, *K* = 400, *I*_{0} = 1 and different values of *J*_{0}. A: The bulk of the spectrum for *J*_{0} = 6 (blue) and for *J*_{0} = 1.12 (red). Left: The imaginary parts of the eigenvalues are plotted vs. their real parts for one realization of **M**. This indicates that the support of the spectrum is a disk of radius *λ*_{max}. Right: Histograms of *N*_{eig}/*R* (one realization of **M**) where *N*_{eig} is the number of eigenvalues with a modulus between *R* and *R*+Δ*R* (Δ*R* = 0.0428 (top), 0.0093 (bottom)) for *J*_{0} = 6 (top) and *J*_{0} = 1.12 (bottom). The distribution of eigenvalues is uniform throughout the spectrum support. B: The largest real part of the eigenvalues (black dots), *λ*_{max}, is compared with the conjecture, Eq (7) (solid line). The fixed point loses stability when *λ*_{max} crosses 1.

To verify the chaoticity of the time dependent state predicted by the DMFT in the region above the bifurcation line we simulated the dynamics and computed numerically the largest Lyapunov exponent, Λ, for different values of *I*_{0} and *J*_{0} (see Materials and Methods for details). The results plotted in Fig 2A (red dots and inset) show that Λ crosses zero near the DMFT bifurcation line and is positive above it. Therefore the dynamics observed in simulations are chaotic in the parameter region above this line as predicted by the DMFT.

We solved numerically the parametric self-consistent differential equation which determined the PAC, *σ*(*τ*), (Eqs (25), (29) and (37)–(38)) for different values of *J*_{0} and *I*_{0}. An example of the results is plotted in Fig 2B. It shows that numerical simulations and DMFT predictions are in very good agreement. Moreover, simulations with increasing values of *N* and *K* indicate that the small deviations from the DMFT predictions are due to finite *N* and *K* effects; a detailed study of these effects is reported in S1 Text.

Fig 4A shows the bifurcation diagram of the PAC amplitude, *σ*_{0} − *σ*_{∞}. For *J*_{0} below the bifurcation point (BP) the PAC amplitude is zero, which corresponds to the fixed point state (solid blue line). At the bifurcation the fixed point loses stability (dashed blue line) and a chaotic state with a strictly positive PAC amplitude emerges (black line).

A: The PAC amplitude, *σ*_{0} − *σ*_{∞}, is plotted against *J*_{0}. At fixed point *σ*_{0} − *σ*_{∞} = 0 (blue). When *J*_{0} = *J*_{c} ≈ 4.995 (black dot, BP) the chaotic state appears. For *J*_{0} > *J*_{c}, the fixed point is unstable (dashed blue) and the network settles in the chaotic state (*σ*_{0} − *σ*_{∞} > 0, black). Red: Perturbative solution in the limit *J*_{0} → *J*_{c} (see S2 Text). Inset: *σ*_{0} − *σ*_{∞} vanishes linearly when *δ* = *J*_{0} − *J*_{c} → 0^{+}. Black: Numerical solution of the DMFT equations. Red: Perturbative solution at the leading order, *O*(*δ*). B: (*σ* − *σ*_{∞})/*δ* is plotted for different values of *δ* > 0 showing the convergence to the asymptotic form (Eq (11) in S2 Text) in the limit *δ* → 0. C: Blue dots: Decorrelation time, *τ*_{dec} vs. PAC amplitude. The PAC, *σ*(*τ*) − *σ*_{∞}, was obtained by solving numerically the DMFT equations and *τ*_{dec} was estimated by fitting the result to the function *A*/*cosh*^{2}(*τ*/*τ*_{dec}). Red: In the whole range, *J*_{0} ∈ [5, 7] considered, *τ*_{dec} can be well approximated by . This relation becomes exact in the limit *σ*_{0} − *σ*_{∞} → 0. Inset: Numerical solution of the DMFT equations for *J*_{0} = 6.65 (blue dots) and the fit to *A*/*cosh*^{2}(*τ*/*τ*_{dec}) (red). The fit is very good although this is far from bifurcation.

We studied analytically the critical behavior of the dynamics at the onset of chaos. We solved perturbatively the DMFT equations for 0 < *δ* = *J*_{0} − *J*_{c} ≪ 1, as outlined in the Materials and Methods section and in S2 Text. This yields (*σ*(*τ*) − *σ*_{∞}) ∝ *δ*^{α}/cosh^{2}(*τ*/*τ*_{dec}), with *α* = 1 and a decorrelation time scaling like *τ*_{dec} ∝ *δ*^{β} with *β* = −1/2. Therefore at the onset of chaos, the PAC amplitude vanishes and the decorrelation time diverges. We show in the Materials and Methods section that this critical behavior with exponents *α* = 1, *β* = −1/2, is in fact a general property of the model, Eqs (3)–(4), whenever *g*(*h*) is twice differentiable. It should be noted that in the SCS model the PAC also vanishes linearly at chaos onset. However, the critical exponent of the decorrelation time is different (*β* = −1) [19].

The inset in Fig 4A compares the PAC amplitude obtained by numerically solving Eq (27) (black line) with the corresponding perturbative result (red line) for small *δ*. The agreement is excellent. In fact, the perturbative calculation provides a good estimate of the PAC even if *δ* is as large as 0.2*J*_{c} (Fig 4A, main panel and Fig 4B). More generally, the PAC can be well fitted with the function (*σ*_{0} − *σ*_{∞}) ⋅ cosh^{ − 2}(*τ*/*τ*_{dec}) (Fig 4C, inset) providing an estimate of the decorrelation time, *τ*_{dec}, for all values of *J*_{0}. Fig 4C plots *τ*_{dec} vs. *σ*_{0} − *σ*_{∞} for *I*_{0} = 1. It shows that the formula we derived perturbatively for small *δ* provides a good approximation of the relationship between the PAC amplitude and the decorrelation time even far above the bifurcation.

#### Threshold power-law transfer function.

We next consider the dynamics of the network (Eqs (3)–(4)) with a transfer function
(9)
where *γ* > 0 and *H*(*x*) = 1 for *x* > 0 and 0 otherwise. Non-leaky integrate-and-fire neurons [40] (see also S3 Text) and *θ*-neurons [41–44] correspond to *γ* = 1 and *γ* = 1/2, respectively. The transfer functions of cortical neurons *in-vivo* can be well fitted by a power-law transfer function with an exponent *γ* ≈ 2 [45, 46].

Fig 5A plots the phase diagrams in the *J*_{0} − *I*_{0} parameter space by solving the DMFT equations (see S4 Text) for different values of *γ* > 1/2. For fixed *I*_{0}, *J*_{c} varies non-monotonically as *γ* decreases. This non-monotonicity is also clear in Fig 5B. When *γ* → (1/2)^{+}, *J*_{c} → 0 as as we show analytically in S4 Text. For *γ* < 1/2, the integral in the right hand side of Eq (5) diverges. Equivalently, the elements of the stability matrix have infinite variance. Therefore, the DMFT predicts a chaotic dynamics as soon as *J*_{0} > 0.

A: *γ* = 3 (gold), 1 (black), 0.7 (red), 0.51 (purple). B: *J*_{c} vs. *γ* for *I*_{0} = 1. Black: DMFT. Blue and red: Simulations with *N* = 32000, *K* = 400. Blue: Zero-crossing of Λ. Red: The fraction of networks with stable fixed point is 50%, 5% and 95% on the solid, bottom-dashed and top-dashed lines respectively.

To compare these predictions with numerical simulations, we simulated different realizations of the network (N = 32000, K = 400, *I*_{0} = 1) for various values of *J*_{0}. For each value of *J*_{0} and *γ* we determined whether the dynamics converge to a fixed point or to a time dependent state as explained in the Materials and Methods section. This allowed us to compute the fraction of networks for which the dynamics converge to a fixed point. The solid red line plotted in Fig 5B corresponds to a fraction of 50% whereas the dotted red lines correspond to fractions of 5% (upper line) and 95% (lower line). We also estimated the Lyapunov exponent, Λ, for each values of *J*_{0} and *γ*. The blue line in Fig 5B corresponds to the location where Λ changes sign according to our estimates (see Materials and Methods for details).

For *γ* ⪆ 0.6, the fraction of networks with an unstable fixed point varies sharply from 0 to 100% in the vicinity of the bifurcation line predicted by the DMFT. Moreover, for these values of *γ*, the spectrum of the matrix is homogeneously distributed in the disk of radius *λ*_{max} centered at the origin and the values of *λ*_{max} agrees with Eq (7). This is shown in Fig 6A for *γ* = 1. Finally, simulations indicate that the values of *J*_{0} where the largest Lyapunov Λ becomes positive in numerical simulations (blue line in Fig 5B) are very close to the DMFT bifurcation values.

A-B: *γ* = 1. The matrix was diagonalized numerically for *N* = 10000, *K* = 400, *I*_{0} = 1 and different values of *J*_{0}. A: The bulk of the spectrum (one realization). Left panel: Imaginary vs. real parts of the eigenvalues for one realization of **M**. Blue: *J*_{0} = 2.045. Red: *J*_{0} = 0.307. Right panel: Histograms (100 realizations) of *N*_{eig}/*R* where *N*_{eig} is the number of eigenvalues with modulus between *R* and *R*+Δ*R* (Δ*R* = 0.0479 (top), 0.0122 (bottom)) for *J*_{0} = 2.045 (top) and *J*_{0} = 0.307 (bottom). The eigenvalues are almost uniformly distributed throughout the disk of radius *λ*_{max} (except very close to the boundary). B: The largest real part of the eigenvalues, *λ*_{max} (one realization, black dots) is compared with the conjecture Eq (7) (solid line). C,D: Same as in A, B, for *γ* = 0.55. Blue: *J*_{0} = 3.01, Δ*R* = 0.0491; red: *J*_{0} = 0.75, Δ*R* = 0.0246 (red). The agreement with Eq (7) is good for *J*_{0} not too large but the eigenvalues distribution is non-uniform. Quantitatively similar results are found for *N* = 20000, *K* = 400 as well as *N* = 40000, *K* = 600 (not shown).

However, as *γ* → (1/2)^{+}, the discrepancies between DMFT and simulations become more pronounced. Very close to *γ* = (1/2)^{+} there is a whole range of values of *J*_{0} for which the DMFT predicts chaos whereas in numerical simulations the dynamics always converge to a fixed point. This discrepancy can be understood by observing that the integral over the Gaussian measure in Eq (5) corresponds to a population average over neurons. When *γ* → (1/2)^{+}, the region where *z* is just above dominates the integral; in other words, the neurons with positive close-to-threshold net inputs are those that make the largest contribution to the destabilization of the fixed point. On the other hand, the DMFT shows that these neurons become extremely rare as *γ* → (1/2)^{+}: in that limit *μ*_{c} increases sharply, thus shifting the center of the Gaussian distribution to very large positive values. Therefore, we would need to simulate outrageously large networks to obtain a quantitative agreement with the DMFT predictions for the locations of the bifurcation to chaos. Similar arguments explain why when *γ* < 1/2 we find a transition from fixed point to chaos in numerical simulations for *J*_{0} ⪅ 0.9 although according to the DMFT the fixed point is always unstable since the integral in Eq (5) diverges.

Numerical diagonalization of shows that when *γ* ⪅ 0.6 (i) the eigevalues in the bulk of the spectrum are distributed in a disk centered at the origin and that this distribution is less and less homogeneous as *γ* → (1/2)^{+} (ii) the eigenvalue *λ*_{max} governing the instability exhibits substantial deviations from Eq (7) especially for large *J*_{0} (Fig 6C) (iii) *λ*_{max} exhibits large sample to sample fluctuations (results not shown). We conjecture that these features are due to large finite *N* and *K* effects and stem from the fact that the SD of the elements of diverges when *γ* → (1/2)^{+}.

We studied the dynamics in detail for *γ* = 1. The DMFT predicts that for all *I*_{0} and *K* (*K* large). As already mentioned, the simulations agree well with this result (Fig 5B). We studied analytically the dynamics for *J*_{0} close to this transition (Fig 7A-7C). To this end, we solved the self-consistent DMFT equations in the limit *δ* = *J*_{0} − *J*_{c} → 0^{+}. The perturbative calculation, explained in S4 Text, is less straightforward than in the case of a sigmoid transfer function. This stems from the fact that at the threshold, the threshold-linear transfer function is only differentiable once. It yields that *σ* − *σ*_{∞} ∼ *δ*^{α} *σ*_{s}(*τ*/*δ*^{β}) with *α* = 2, *β* = −1/2 and the function *σ*_{s}(*x*)) has to be determined numerically. The function *σ*_{s} is plotted in Fig 7B. It can be well fitted to the function *A*[cosh(*x*/*x*_{dec})]^{−1} with *A* = 12.11 and *x*_{dec} = 2.84 (see Fig 7B, inset). In particular, for small *δ*, the amplitude and the decorrelation time of the PAC are related by *τ*_{dec} ∝ 1/(*σ*_{0} − *σ*_{∞})^{1/4}. Note that the amplitude of the PAC vanishes more rapidly (*α* = 2) than for sigmoidal transfer functions (*α* = 1) whereas the decorrelation time diverges with the same critical exponent (*β* = −1/2) in the two cases.

A: The PAC amplitude, *σ*_{0} − *σ*_{∞}, is plotted against *J*_{0}. At fixed point *σ*_{0} − *σ*_{∞} = 0 (blue). When (black dot, BP) a bifurcation occurs and the chaotic state appears. For *J*_{0} > *J*_{c}, the fixed point is unstable (dashed blue) and the network settles in the chaotic state (*σ*_{0} − *σ*_{∞} > 0, black). Red: Perturbative solution in the limit *J*_{0} → *J*_{c} (see S4 Text). Inset: plotted against *δ* = *J*_{0} − *J*_{c} showing that *σ*_{0} − *σ*_{∞} vanishes quadratically when *δ* → 0^{+}. Black: Full numerical solution of the DMFT equations. Red: Perturbative solution at the leading order, *O*(*δ*). B: (*σ* − *σ*_{∞})/*δ*^{2} is plotted for different values of *δ* > 0 to show the convergence to the asymptotic function derived perturbatively in S4 Text. Inset: The function (*σ*(*τ*) − *σ*_{∞})/*δ*^{2} (black) can be well fitted to *A*/cosh(*x*/*x*_{dec}) (red dots, *A* = 12.11, *x*_{dec} = 2.84). C: Decorrelation time, *τ*_{dec} vs. PAC amplitude (blue). The function *σ*(*τ*) − *σ*_{∞} was obtained by integrating numerically Eq (29) and *τ*_{dec} was estimated by fitting this function to *A*/cosh(*τ*/*τ*_{dec}). Red: In the whole range of *J*_{0} considered (*J*_{0} ∈ [1.4, 1.9] the relation between *τ*_{dec} and *σ*_{0} − *σ*_{∞} can be well approximated by . Inset: The PAC computed by solving the DMFT equations for *J*_{0} = 1.81 (blue dots) and the fit to 0.93/cosh(*τ*/4.6). D: The PAC for *J*_{0} = 2 and *K* = 1200. Blue: Numerical integration of Eq (29). Red: Numerical simulations for *N* = 256,000.

Fig 7A-7C compares the results of the perturbative analysis to those of the numerical integration of the differential equation, Eq (27). Unlike what we found for the sigmoid transfer function, *δ* must be very small (*δ* ⪅ 0.03*J*_{c}) to achieve a good quantitative agreement. It should be noted, however, that the quality of the fit of *σ* − *σ*_{∞} to *A*[cosh(*τ*/*τ*_{dec})]^{−1} does not deteriorate by much even far from the bifurcation (Fig 7C, inset; *δ* = 0.4), and that the relation holds with good approximation even if *δ* is not small (Fig 7C, main panel).

Finally, Fig 7D compares DMFT and numerical simulations results for *σ*(*τ*) when *J*_{0} = 2. The agreement is reasonably good but not perfect. We show in S1 Text that the discrepancy between the two improves as the network size increases but that finite size effects are stronger here than in the rate model with sigmoid transfer function.

#### Leaky integrate-and-fire (LIF) inhibitory networks.

Our objective here is to obtain further insights into the relevance of the chaotic behavior exhibited by rate dynamics, Eqs (3)–(4), to understand *spiking* network dynamics. The dynamics of one population of LIF spiking neurons reduces to Eqs (3)–(4) with the transfer function
(10)
in the limit where the synapses are much slower than the cell membrane time constant, *τ*_{m}. Our goal is twofold: 1) to study the emergence of chaos in this rate LIF rate model and 2) to compare it to full spiking dynamics and characterize the range of values of the synaptic time constant for which the two dynamics are quantitatively or at least qualitatively similar.

Figs 8, 9 depict typical patterns of neuronal activity in simulations of the inhibitory spiking LIF model. For strong and fast synapses (*τ*_{syn} = 3 ms, Fig 8A), neurons fire spikes irregularly and asynchronously (Fig 8A). Fig 8B shows that when *τ*_{syn} = 100 ms the population average firing rate remains essentially the same (∼ 14.1 Hz) and the network state stays asynchronous. The spiking patterns, however, change dramatically: with large *τ*_{syn} neurons fire irregular bursts driven by slowly decorrelating input fluctuations (Fig 9A, blue). Fig 9B shows that reducing *J*_{0} increases the firing rate, reduces the amplitude of the fluctuations (Fig 9B, inset) and slows down their temporal decorrelation. Eventually, for small enough *J*_{0}, *σ*(*τ*) becomes flat and the fluctuations are negligible.

*N* = 10000, *K* = 800, *J*_{0} = 2, *I*_{0} = 0.3. Voltage traces of single neurons (top), spike trains of 12 neurons (middle) and population averaged firing rates (in 50 ms time bins, bottom) are plotted. A: *τ*_{syn} = 3 ms. Neurons fire irregular spikes asynchronously. B: *τ*_{syn} = 100 ms. Neurons fire bursts of action potentials in an irregular and asynchronous manner.

Simulation results for *N* = 40,000, *K* = 800, *I*_{0} = 0.3, *τ*_{syn} = 100 ms. From left to right: *J*_{0} = 2 (blue), 1.5 (red) and 1 (black). A: Examples of single neuron membrane voltages (top) and net inputs, *h*, (bottom). For the three values of *J*_{0}, the mean firing rate of the example neuron is 11 Hz. As *J*_{0} decreases, the temporal fluctuations in the net input become smaller whereas the temporal average increases such that the firing rate remains nearly unchanged. B. Top: Population average firing rate increases like 100*I*_{0}/*J*_{0} as implied by the balance condition. Bottom: PAC (*σ* − *σ*_{∞}, bottom). The dots correspond to the fit of the PAC to (*σ*_{0}−*σ*_{∞})⋅[cosh(*τ*/*τ*_{dec})]^{−1} which yields *τ*_{dec}/*τ*_{syn} = 2.5 (blue), 3.0 (red), 3.8 (black) for the three values of *J*_{0}. Inset in the right panel: *σ*_{0} − *σ*_{∞} vs. *J*_{0}.

Fig 10 compares the dynamics of the rate to those of the spiking LIF networks. Panels A,B show that for *J*_{0} = 2, *I*_{0} = 0.3 and *τ*_{syn} = 100 ms, *σ*(*τ*), the distributions of the time averages of neuronal firing rates and net inputs, ⟨*r*_{i}⟩ and ⟨*h*_{i}⟩, are essentially the same in the simulations of the two networks. When reducing *τ*_{syn} down to *τ*_{syn} ⪆ 15 ms, the function *σ*(*τ*/*τ*_{syn}) measured in the spiking network simulations, changes only slightly. In fact, this function is remarkably similar to what is found for the corresponding function in the DMFT and in simulations of the LIF rate model (Fig 11A). Fitting *σ*(*τ*) with the function *B*+*A*[cosh(*τ*/*τ*_{dec})]^{−1} yields *τ*_{dec} ≈ 2.45⋅*τ*_{syn}.

*N* = 40,000, *K* = 800. *J*_{0} = 2, *I*_{0} = 0.3, *τ*_{syn} = 100 msec. A: *σ*(*τ*/*τ*_{syn}). B: Distributions of neuronal mean firing rates, ⟨*r*_{i}⟩, and net inputs, ⟨*h*_{i}⟩, (inset) in the spiking network (black) and rate model (red; dots: simulations, solid line: DMFT).

All the results are from numerical simulations with *N* = 40,000, *K* = 800. A: *J*_{0} = 2, *I*_{0} = 0.3. B: *J*_{0} = 3, *I*_{0} = 0.3. C: *J*_{0} = 4, *I*_{0} = 0.6. D: *J*_{0} = 1, *I*_{0} = 0.3. In all four panels the PACs are plotted for the spiking network with *τ*_{syn} = 10 (gray), 20 (red) and 40 (green) ms. The results for the rate model are also plotted (black). The firing rates are ∼ 15 Hz in A and C, ∼ 10 Hz in B and ∼ 30 Hz in D, in good agreement with the prediction from the balance condition ([⟨*r*⟩] = 100*I*_{0}/*J*_{0} Hz). As the population firing rate increases, a larger *τ*_{syn} is needed for good agreement between the spiking and the rate model.

How small can *τ*_{syn} be for the two models to still behave in a quantitatively similar manner? Simulations show that this value increases with the mean activity of the network (see examples in Fig 11) but that for reasonable firing rates, fewer than several several tens of Hz, the fluctuations have similar properties in the two models even for *τ*_{syn} ≈ 20 ms.

We conducted extensive numerical simulations of the inhibitory LIF rate and spiking models (*N* = 40000, *K* = 800) to compute their phase diagrams in the *I*_{0} − *J*_{0} parameter space. The results for the rate model are plotted in Fig 12. For sufficiently small *J*_{0} the dynamics always converge to a fixed point whereas for sufficiently large *J*_{0} the network always settles in a state in which the activity of the neurons keeps fluctuating at large time. We show in S5 Text that in this regime the maximum Lyapunov exponent is strictly positive, therefore the dynamics are chaotic. Between these two regimes, whether the dynamics converge to a fixed point or to a chaotic state depends on the specific realization of the connectivity matrix. The fraction of networks for which the convergence is to a fixed point depends on *J*_{0}. The range of *J*_{0} where this fraction varies from 95% to 5% is notably large as shown in Fig 12. Additional simulation results on this issue are depicted in S5 Text. The counterpart of this behavior in the spiking network is that when *J*_{0} is small, neurons fire regular spikes tonically whereas for sufficiently large *J*_{0} they fire highly irregular bursts. The transition between the two regimes occurs for similar values of *J*_{0} in the rate and in the spiking networks. In both networks this transition is driven by the neurons with low firing rates; i.e., with larger numbers of recurrent inputs. These neurons are the first to become bursty as *J*_{0} increases (see S6 Text).

All the results are from numerical simulations with *N* = 40,000, *K* = 800. Black: zero-crossing of the maximum Lyapunov exponent, Λ. The fraction of networks for which the dynamics converge to a fixed point is 50%. 5% and 95% on the solid, top-dashed and bottom-dashed red lines respectively. Insets: *I*_{0} = 0.3. Voltage traces of a neuron in the inhibitory LIF spiking model for *J*_{0} = 2 (top inset), 0.3 (bottom inset) and *τ*_{syn} = 100 ms.

In Fig 13A we plot the bifurcation diagram of the model as obtained in the numerical solution of the DMFT equations (black line) and as calculated in simulations of the rate model (blue dots) and of the spiking network with *τ*_{syn} = 25 ms (red ×’s) and *τ*_{syn} = 7.5 ms (green ×’s). The rate model simulations are in good agreement with DMFT for 0.8 ⪅ *J*_{0} ⪅ 2. For larger *J*_{0} the discrepancy becomes significant and increases with *J*_{0}. This is because of finite *K* effects that grow stronger as *J*_{0} increases as shown in the right inset in Fig 13A, for *J*_{0} = 3 (blue) and *J*_{0} = 4 (red). Fig 13A also shows that, as discussed above, the amplitude of the PACs obtained in simulations of the LIF rate and spiking networks are barely different provided the synaptic time constant is sufficiently large.

All simulation results depicted here were obtained in networks with *N* = 40,000, *K* = 800, *I*_{0} = 0.3. A: The PAC amplitude, *σ*_{0} − *σ*_{∞}, vs. inhibitory coupling, *J*_{0}. Black: DMFT. Blue dots: Simulations of the rate model. Red ×’s: Simulations of the spiking network with *τ*_{syn} = 25 ms. Green ×: Spiking network with *τ*_{syn} = 7.5 ms. Right inset: The difference between PAC amplitudes obtained in simulations (Δ*σ*_{sim}) and DMFT (Δ*σ*_{th}) plotted against *K* (in log scale) for *J*_{0} = 3 (blue) and *J*_{0} = 4(red). Left inset: Closeup (*J*_{0} ∈ [0.2 0.5]) of the numerical solution of the DMFT equations. B: PACs were fitted as explained in the text to estimate *τ*_{dec}. The estimated decorrelation time, *τ*_{dec}, is plotted vs. the amplitude of the PAC for the rate (blue), spiking (black) networks and DMFT (red). Inset: The PAC in the rate model for *J*_{0} = 2 (black dots: simulation; red line: fit).

Fig 13B shows the relation between the decorrelation time, *τ*_{dec} and the PAC amplitude. To get these results, simulations of the rate and the spiking networks were performed for *J*_{0} ∈ [0.8, 3.5] and *τ*_{dec} was estimated by fitting the PACs with the function *A*⋅[cosh(*τ*/*τ*_{dec})]^{−1}. We also solved the DMFT equations for the same values of *J*_{0} and computed the PAC that we fitted to the same function. The results from the simulations (rate model: blue; spiking network: black) and DMFT (red) agree fairly well. Note that *τ*_{dec} decreases more slowly as *σ*_{0} − *σ*_{∞} increases than in the models with a sigmoid or threshold-linear transfer function (compare to Figs 4C and 7C).

Finally, according to the DMFT the fixed point should be always unstable since for the LIF transfer function the elements of the stability matrix always have an infinite variance or, equivalently, the integral in Eq (5) always diverges. This can be seen in the close-up in the left inset of Fig 13A, indicating that the PAC amplitude is non-zero for small *J*_{0} and that it approaches 0 very slowly as *J*_{0} decreases. By contrast, in numerical simulations in the same range of *J*_{0}, the dynamics are not chaotic for most of the realizations of the network: they converge to a fixed point, as shown in Fig 12. The explanation for this difference is as for the rate model with threshold power-law transfer function with *γ* < 1/2 (see above).

### Two asynchronous chaos mechanisms in excitatory-inhibitory recurrent networks

We now consider EI spiking networks with recurrent feedback interactions between the two populations. The synaptic strengths and time constants are and *τ*_{αβ} (*α*, *β* ∈ {*E*, *I*}). Assuming slow synapses, the dynamics can be reduced to four sets of equations for the four types of synaptic inputs, (Materials and Methods, Eq (17)). The DMFT yields self-consistent equations for the statistics of these inputs. These equations can be analyzed straightforwardly for the fixed point state. In contrast to purely inhibitory networks where the fixed point loses stability only via a bifurcation to chaos, it can now also lose stability via a Hopf bifurcation. This depends on the synaptic time constants. When this happens the network develops synchronous oscillations which break the balance of excitation and inhibition (the oscillation amplitude diverges for large *K*).

We focus here on instabilities which lead to chaos. Their locations in the 6 dimensional parameter space (4 synaptic strengths, 2 external inputs) of the model can be derived for a general transfer function (Eqs (54)–(55)). Differential equations for the PAC functions, *σ*_{αβ}(*τ*), can also be derived in the chaotic regime. However, general analytical characterization of their solutions is particularly difficult. Leaving such study for future work, we mostly focus below on numerical simulations. Our key result is that in EI networks asynchronous chaos emerges in two ways, one driven by I-I interactions (II mechanism) and the other by the EIE loop (EIE mechanism).

#### EI network with threshold-linear transfer function.

We first study a EI network in which all the neuronal transfer functions are threshold-linear. Fig 14 plots for different *K* the phase diagram of the DMFT of this model in the parameter space, when and *I*_{E} = *I*_{I} = 1, .(The phase-diagram for a non-zero of value , , is plotted and briefly discussed in S7 Text). On the lines, the dynamics bifurcate from fixed point (below the lines) to chaos (above). As decreases the lines go to infinity. Numerical simulations indicate the maximum Lyapunov exponent changes sign very close to these lines (compare red line and red dots) in good agreement with DMFT. For any finite *K*, the instability line exhibits a re-entrance, crossing the -axis at , where the instability occurs in a purely inhibitory network; in this respect, the limit *K* → ∞ is singular. Solving the self-consistent equations for the average firing rates, *r*^{E} and *r*^{I}, one finds that the two populations can have a comparable firing rate for large when is not too large. As becomes small, the activity in the E population becomes much lower than in the I population. In fact, for *K* → ∞, *r*^{E} vanishes on the line and is zero for (white region in Fig 14). In other words, in the latter case, inhibition is not balanced by excitation in the E population.

, . The bifurcation lines predicted by the DMFT are plotted in the parameter space for *K* = 400 (red), 10^{3} (blue), 10^{4} (black), and *K* → ∞ (green). Red dots: Zero-crossing of the largest Lyapunov exponent (average over 5 network realizations) in numerical simulations for *K* = 400. Color code: Ratio of the population average firing rate of the two populations (I/E) in *log* scale (right). White region: The activity of the E population is very small for finite *K* and goes to zero in the limit *K* → ∞. The boundary to the right of that region is given by: .

As shown above, in the single inhibitory population case with threshold-linear transfer functions the transition to chaos occurs at . Fig 14 shows that in the two population network the chaotic regime extends below . This suggests that the EIE loop can also play the key role in the emergence of chaos. To assess further the role of the II and of the EIE interactions in generating chaotic activity, we simulated the network for different values of and *τ*_{αβ}. Traces of the synaptic inputs are displayed in Fig 15 for large (panel A) and small (panel B) . The gray traces correspond to the case where all time constants are equal (10 ms, reference case). Multiplying *τ*_{IE} by 10 (black) slows down the fluctuations in all inputs when is small, but when is large this happens only for *h*_{IE}. By contrast, dividing *τ*_{II} by 10 (purple) has very little effect when is small but the fluctuations of all the inputs are substantially faster when is large.

Simulations were performed for *N*_{E} = *N*_{I} = 8000, *K* = 400, *I*_{E} = *I*_{I} = 1, , . A: II mechanism for , . Left panels: Examples of traces of excitatory (*h*_{IE}) and inhibitory inputs (*h*_{EI}, *h*_{II}) into one neuron. Right: PAC of the net inputs to the E neurons. Gray: *τ*_{IE} = *τ*_{EI} = *τ*_{II} = 10 ms; Black: *τ*_{IE} = 100 ms, *τ*_{EI} = *τ*_{II} = 10 ms; Blue: *τ*_{II} = 100 ms, *τ*_{IE} = *τ*_{EI} = 10 ms; Purple: *τ*_{II} = 1 ms, *τ*_{EI} = *τ*_{IE} = 10 ms. Inset: All PACs plotted vs. *τ*/*τ*_{II}. B: EIE mechanism for , . Other parameters are as in A. Inset: All PACs plotted vs. *τ*/*τ*_{IE}.

Fig 15 also demonstrates the effect of changing *τ*_{αβ} on the PAC of the net inputs to the E neurons, (corresponding results for the I population are shown in S8 Text). The PAC in the reference case is plotted in gray. For *large* , a ten-fold *increase* in *τ*_{II} causes the PAC width to become ten times larger and the PAC amplitude increases (Fig 15A, blue; see also inset). For a ten-fold *decrease* in *τ*_{II} (purple) compared to reference, the width of the PAC is smaller but by a smaller factor whereas its amplitude is greatly reduced. By contrast, a ten-fold increase in *τ*_{IE} has no noticeable effect, either on the width or on the amplitude of the PAC (black). Fig 15B plots the PAC of the total input to the E population for *small* . Here, decreasing *τ*_{II} by a factor of 10 (purple line) weakly affects the width as well as the amplitude of the PAC. In contrast, a ten-fold increase of *τ*_{IE} (black) widens the PAC by a comparable factor (see also inset). A similar widening occurs if *τ*_{EI} is increased ten-fold (see S8 Text).

This phenomenology can be understood as follows. In the large regime, the II interactions play the key role in the generation of chaos. Therefore, the time scale of the fluctuations in the activity of the I neurons is essentially determined by *τ*_{II}. Thus if the latter is 10 times larger than reference, the I inputs to the E neurons are slowed down by the same factor. At the same time, the filtering effect of the EI synapses becomes weaker and thus the amplitude of the PAC of the net input in the E neurons increases. The effect of decreasing *τ*_{II} stems from the filtering effect of the EI synapses which is now stronger than in the reference case. Finally, changing *τ*_{IE} has no noticeable effect since the fluctuations are generated by the II interactions. By contrast, when is small, II interactions are not sufficient to generate chaotic fluctuations. In this regime, the EIE loop drives these fluctuations if is sufficiently large. That is why the time scale of the activity fluctuations depends primarily on *τ*_{IE} and to a much smaller extent on *τ*_{II}.

These results point to the existence of two mechanisms for chaos emergence in two population networks; they differ by the type of the dominant interactions (EIE or II) and therefore on the synaptic time constants which settle the time scale of the activity fluctuations. Another difference is that in the EIE mechanism, the E population is always significantly less active than the I population. This is not the case in the II mechanism.

#### Two-population spiking LIF network.

We ran a similar analysis for LIF networks. Fig 16A, 16C plot the PACs of for the LIF spiking and rate models (PACs of are shown in S9 Text). In all panels , , and *τ*_{EI} = 3 ms. For (Fig 16A), increasing *τ*_{II} slows down the fluctuations. By contrast, changing *τ*_{IE} only has a very mild effect (S10 Text). This is because the fluctuations are essentially driven by the II interactions. For *τ*_{II} ⪆ 15 ms, the fluctuation statistics are quantitatively similar in the spiking and the rate models: in both, the decorrelation time, *τ*_{dec} ≈ 2*τ*_{II} (Fig 16A, inset). Moreover, simulations indicate that the dynamics of the rate model are chaotic (Λ ≈ 1.7/*τ*_{II}). The trace in Fig 16B shows that with large *τ*_{II} (=100 ms) the spiking pattern is bursty. The membrane potential between bursts exhibit slow fluctuations because they are generated by the slow II connections.

Simulations were performed with *N*_{E} = *N*_{I} = 16000, *K* = 400, *I*_{E} = 0.2, *I*_{I} = 0.1, , , . A: II mechanism. PACs of the net inputs in E neurons are plotted for , *τ*_{IE} = 100 ms, *τ*_{EI} = 3 ms and *τ*_{II} = 3, (red), 10 (black), 40 (blue) and 100 ms (purple). Solid line: Spiking model. Dots: Rate model. Inset: All PACs (spiking network) are plotted vs. *τ*/*τ*_{II}. B: Voltage of one E neuron for parameters as in A, purple. C: EIE mechanism. PACs of the net inputs in E neurons are plotted for , *τ*_{EI} = *τ*_{II} = 3 ms and *τ*_{IE} = 100, (green), 200 (red) and 400 ms (black). Solid line: Spiking model. Dots: Rate model. Inset: All PACs (spiking network) are plotted vs. *τ*/*τ*_{IE}. D: Voltage of one E neuron in the spiking network with parameters as in C, green.

Fig 16C plots the PACs of for . Here also, the LIF rate model operates in a chaotic regime (Λ ≈ 120*s*^{−1}). In the spiking model the PACs exhibit a slow time scale but also a fast one (the sharp peak around *τ* = 0). These correspond to the slow and fast fluctuations observable in the voltage traces in Fig 16D. Increasing *τ*_{IE} while keeping *τ*_{EI} = *τ*_{II} = 3 msec has a substantial effect on the slow component but hardly affects the fast component. When plotted vs. *τ*/*τ*_{IE}, the slow components of the PACs all collapse onto the same curve (Fig 16C, inset). This indicates that the EIE loop is essential in generating the slow, but not the fast, fluctuations. Fitting this slow component with the function *A*⋅[cosh(*τ*/*τ*_{dec})]^{−1} yields *τ*_{dec} ≈ 2.4*τ*_{IE}. Furthermore, increasing *τ*_{II} suppresses the fast fluctuations and amplifies the slow ones. These two effects saturate simultaneously when *τ*_{II} ≈ 10 ms (S11 Text). Thus, it can be inferred that fast fluctuations are mostly generated by II interactions. Their amplitude is suppressed as *τ*_{II} is increased because they become more filtered. Concomitantly, the slow fluctuations become amplified. This is because fast fluctuations smooth the effective transfer function of the E neurons in the low firing rate regime. Thus, their suppression increases the gain of this transfer function. This explains the quantitative differences between the PACs in the spiking and the rate LIF network when II synapses are fast and why these differences are lessened as *τ*_{II} increases (S11 Text).

In the simulations reported in Fig 16 there is no recurrent excitation in the E population (). Moreover, all the excitatory synapses to the I population are slow. Both assumptions were made to reduce the number of parameters in order to simplify the analysis. However, in cortical networks in general, fast (AMPA) and slow (NMDA) excitation coexist (in fact AMPA synapses are required to open the NMDA receptors). Moreover, recurrent excitation is thought to be in general substantial (see however [47]). Results depicted in S12 Text show that the EIE loop can induce slow rate fluctuations in our network when it combines slow and fast excitatory synapses and when substantial recurrent excitation is present in the E population.

## Discussion

Networks of neurons operating in the so-called balanced regime exhibit spiking activity with strong temporal variability and spatial heterogeneity. Previous theoretical studies have investigated this regime assuming that excitatory and inhibitory synapses are sufficiently fast compared to the neuronal dynamics. The nature of the balanced state is now fairly well understood in this case. By contrast, here we focused on networks in which some of the synapses are slow. To study the dynamics in these networks, we reduced them to a rate dynamics that we investigated by combining Dynamical Mean-Field Theory and simulations. Our key result is that when synaptic interactions are sufficiently strong and slow, chaotic fluctuations on the time scales of the synaptic dynamics emerge *naturally* from the network collective behavior. Moreover, the nature of the transition to chaos and the behavior in the chaotic regime are determined only by the neuronal *f* − *I* curve and not by the details of the spike-generation mechanism.

We identified two mechanisms for the emergence of asynchronous chaos in EI neuronal networks. One mechanism relies on II interactions whereas in the other the EIE feedback loop plays the key role. These mechanisms hold in rate models (Eq (3)) as well as in LIF spiking networks. By computing the maximum Lyapunov exponent, we provided direct evidence that in rate models these states are indeed chaotic. For LIF spiking networks, we argued that when the synapses are sufficiently slow, the observed activity fluctuations are chaotic since their statistics are quantitatively similar to those observed in the corresponding rate model. This similarity persists for synaptic time constants as small as the membrane time constant. This is in agreement with [33–35] which relied on numerical integration of the LIF model to compute the Lyapunov spectra of networks of various sizes and increasing synaptic time constants. They found that the LIF dynamics are chaotic *only* if the synapses are sufficiently slow.

In these two mechanisms, the dynamics of the synaptic currents play the key role whereas dependence on the intrinsic properties of the neurons only occurs via their nonlinear instantaneous input-output transfer function. Since the synaptic currents are filtered versions of the neuronal spike trains, and that the temporal fluctuations of the activity occur on the time scales of the synaptic currents, it is natural to qualify the dynamical regime as *rate* chaos. Although the features of the bifurcation to chaos may depend on the shape of the transfer function, as we have shown, the qualitative features of the chaotic state are very general, provided that the synaptic currents are sufficiently slow. Rate chaos is therefore a generic property of networks of spiking neurons operating in the balanced regime. We show in S3 Text that rate chaos occurs also in networks of *non-leaky* integrate-and-fire spiking neurons. In that case, the statistics of the fluctuations are similar to those of the model in Eq (3) with a threshold-linear transfer function. We also found rate chaos in biophysically more realistic network models in which the dynamics of the neurons and of the synapses are conductance-based (results not shown). In these cases, the dynamics of the synaptic conductances give rise to the chaotic fluctuations.

Quantitative mappings from spiking to rate models have been derived for networks in stationary asynchronous non chaotic states [38] or responding to external fluctuating inputs [48]. Spiking dynamics also share *qualitative* similarities with rate models for networks operating in synchronous states [9–11, 38, 43]. To our knowledge, the current study is the first to report a *quantitative* correspondance between spiking and rate model operating in chaotic states.

The SCS model [19] has been widely used to explore the physiological [22, 49] and computational significance of chaos in neuronal networks. Recent works have shown that because of the richness of its chaotic dynamics, the SCS model has remarkable learning capabilities [15–18]. Our work paves the way for an extension of these results to networks of spiking neurons with a connectivity satisfying Dale’s law, which are biologically more realistic than the SCS model.

Another interesting implication of our work is in the field of random matrices. Given a *dense* NxN random matrix, **A**, with i.i.d elements with zero mean and finite standard deviation (SD), in the large *N* limit, the eigenvalue of with the largest real part is real, and it is equal to SD [50, 51] (more generally, the eigenvalues of are uniformly distributed within a disk of radius SD centered at the origin [50, 51]). Several results regarding the spectra (bulk and outliers) of dense random matrices with structures reflecting Dale’s law have been derived recently [52–54]. Less is known when the matrices are sparse. A byproduct of our approach are two conjectures for the maximal eigenvalue of such *sparse* random matrices, namely Eqs (7) and (62) that we verified numerically.

Neuronal spiking statistics (e.g., firing rate, spike counts, inter-spike intervals) exhibit a very broad range of time scales during spontaneous or sensory evoked activity in-vivo (see e.g [55, 56]). Fluctuations on time scales larger than several 100s of millisecond can be accounted for by neuromodulation which changes the global excitability of the cortical network or changes in behavioral state. Very fast fluctuations are naturally explained in the framework of the standard model of balance of excitation and inhibition [28–30]. By contrast, it is unclear how to explain modulations in the intermediate temporal range of a few 10s to several 100s of milliseconds. In fact, the *standard* framework of balanced networks predicts that fluctuations on this time scale are actively suppressed because the network state is very stable. Our work extends this framework and shows two mechanisms by which modulations in this range can occur. In the II mechanism, inhibitory synapses must be strong and slower than 10 − 20 ms. GABA_{A} inhibition may be too fast for this [57] (see however [58]), but GABA_{B}[59] are sufficiently slow. In contrast, the EIE mechanism is achieved when inhibition in fast. It requires slow recurrent excitation to inhibitory neurons, with a time constant of a few to several tens of ms, as is typically the case for NMDA receptors (see e.g [60–62]). Hence, the combination of GABA_{A} and NMDA synapses can generate chaotic dynamics in the cortex and fluctuations in activity on a time scale of several tens to a few hundreds of ms.

Note added in production: Following a request from the editors after formal acceptance of our article, we note that a recent paper [63] claims that spiking networks with instantaneous delayed synapses exhibit an asynchronous state similar to the chaotic state of the SCS model. However, this claim is incorrect and has been shown to rely on flawed analysis [64].

## Models

### Two population leaky integrate-and-fire spiking network

The two population network of leaky integrate-and-fire (LIF) neurons considered in this work consists of *N*_{E} excitatory (E) and *N*_{I} inhibitory neurons. The subthreshold dynamics of the membrane potential, , of neuron *i* in population *α* (i = 1, …, *N*_{α}; *α*, *β* ∈ {*E*, *I*}) obeys:
(11)
where *τ*_{m} is the membrane time constant (we take *τ*_{m} = 10 msec for both populations), and *J*^{αβ} are respectively the connectivity matrix and the strength of the connections between the (presynaptic) population *β* and (postsynaptic) population *α* and *I*^{α} the external feedforward input to population *α*. For simplicity we take *N*_{E} = *N*_{I} = *N*. However, all the results described in the paper are also valid when the number of neurons is different in the populations (provided both numbers are large)., The variables , which describe the synapses connecting neuron *j* in population *β* to population *α*, follow the dynamics:
(12)
where *τ*_{αβ} is the synaptic time constant and the sum is over all the spikes emitted at times .

Eqs (11), (12) are supplemented by a reset condition. If at time *t*_{sp}, , the neuron emits a spike and . For simplicity we do not include the neuronal refractory period.

We assume that the connectivity is random with all the uncorrelated and such that with probability *K*/*N* and 0 otherwise. Hence each neuron is connected, on average, to *K* neurons from its population as well as to *K* neurons from the other population. When varying the connectivity *K* we scale the interaction strength and the feedforward inputs according to: and [29].

### Network of inhibitory leaky integrate-and-fire neurons

The dynamics of the network of the one-population spiking LIF neurons considered in the first part of the paper are:
(13)
supplemented with the reset condition at threshold. The elements of the connectivity matrix, *C*_{ij}, are uncorrelated and such that *C*_{ij} = 1 with probability *K*/*N* and 0 otherwise. All neurons are inhibitory, thus *J* < 0.

The synaptic dynamics are:
(14)
where *τ*_{syn} is the synaptic time constant of the inhibition and the sum is over all the spikes emitted at times *t*_{j} < *t*. The interaction strength and the feedforward inputs scale with K as: and with *J*_{0} > 0.

### Network of non-leaky integrate-and-fire neurons

We consider briefly this model in S3 Text. The network architecture as well as the synaptic dynamics are as above. The single neuron dynamics of non-leaky integrate-and-fire (NLIF) neurons are similar to those of LIF neurons except for the first terms on the right-hand side of Eqs (11), (13) which are now omitted.

### Rate dynamics for spiking networks with slow synapses

If the synapses are much slower than the membrane time constant, the full dynamics of a spiking network can be approximated by the dynamics of the synapses driven by the instantaneous firing rates of the neurons, namely:
(15)
where *g*(*x*) is the transfer function of the neuron (the *f* − *I* curve) [20]. In particular, for the LIF networks,
(16)
with *H*(*x*) = 1 for *x* > 0 and *H*(*x*) = 0 otherwise. For the NLIF networks, the transfer function is threshold-linear: *g*(*x*) = *xH*(*x*).

Defining , the dynamics of are given by
(17)
We will denote by the total input into neuron *i* in population *β*: . For networks comprising only one population of inhibitory spiking neurons we will drop the superscript *β* = *I* and denote this input by *h*_{i}. The dynamics then yield:
(18)
where *τ*_{syn} is the inhibitory synaptic time constant.

## Dynamical Mean-Field Theory of the Single Inhibitory Population

A Dynamical Mean-Field Theory (DMFT) can be developed to investigate the rate model, Eq (17), for a general transfer function under the assumption, 1 ≪ *K* ≪ *N*.

Here we provide a full analysis of a one-population network of inhibitory neurons whose dynamics are given in Eq (18). We take as the external input and as the coupling strength. In this case, a functional integral derivation shows that these dynamics can be written as:
(19)
where *η*_{i}(*t*) is a Gaussian noise:
(20)
with *z*_{i}, i.i.d Gaussian quenched variables with zero mean and unit standard deviation (SD), *ξ*_{i}(*t*) are Gaussian noises with ⟨*ξ*_{i}(*t*)⟩_{t} = 0, and ⟨*ξ*_{i}(*t*)*ξ*_{j}(*t*+*τ*)⟩_{t} = *C*_{ξ}(*τ*)*δ*_{i, j} where ⟨ ⋅ ⟩_{t} stands for averaging over time. Therefore, in general, the inputs to the neurons display temporal as well as quenched fluctuations.

The self-consistent equations that determine the mean, temporal correlations and quenched fluctuations yield:
(21) (22) (23)
where ⟨ ⋅ ⟩ and [⋅] stand for averaging over noise and quenched disorder, respectively. Thus the quantities *q* and *μ* obey:
(24)
and:
(25)
where *σ*(*τ*) = [⟨*h*(*t*)*h*(*t*+*τ*)⟩] − *μ*^{2} is the population-averaged autocovariance (PAC) of the input to the neurons and we define: *σ*_{0} = *σ*(0) and . In the limit *K* → ∞, *μ* must remain finite. This implies that the population averaged firing rate, [⟨*g*(*h*)⟩] = *I*_{0}/*J*_{0} does not depend on the specifics of the transfer function of the neurons and varies linearly with *I*_{0}. This is a key outcome of the balance between the feedforward excitatory and the recurrent inhibitory inputs to the neurons.

To express *C*_{ξ}(*τ*) in terms of *σ*, we note that the vector (*h*(*t*), *h*(*t*+*τ*))^{T} is a bivariate Gaussian, so in fact we need to calculate *E*[*g*(*μ*+*x*)*g*(*μ*+*y*)] where (*x*, *y*)^{T} has zero mean and a covariance matrix
and *E*[⋅] stands for averaging over temporal noise and quenched disorder. Defining
where *ξ*, *θ* and *z* are independent Gaussian variables with zero mean and unit variance yields
(26)
A straightforward derivation shows that *σ*(*τ*) obeys:
(27)
with initial conditions:
(28)
where the last condition results from *σ*(*τ*) = *σ*(−*τ*).

Eq (27) can be rewritten as:
(29)
where the “potential” *V*(*σ*;*σ*_{0}) which depends parametrically on *σ*_{0} is:
(30)
with *G*(*x*) = ∫*g*(*x*)*dx*. Note that for positive *σ* this equation yields
(31)
Therefore the quantity
(32)
is conserved under the dynamics, Eq (29). Hence:
(33)

To simplify notations, we drop the parameter *σ*_{0} and denote the potential by *V*(*σ*). The first, second and third order derivatives of the potential with respect to *σ* are denoted *V*′(*σ*), *V*′′(*σ*) and *V*′′′(*σ*).

For illustrative purpose, we consider a sigmoid transfer function, . In this case we have
Using the identities:
and
the potential *V*(*σ*) can be written as:
Fig 17A_{1–3} plots *V* for *σ* ∈ (−*σ*_{0}, *σ*_{0}) for *J*_{0} = 4, fixed *I*_{0} = 1 and different values of *σ*_{0}. When *V*′(*σ*_{0}) > 0 (Fig 17*A*_{1}), the solution to Eq (29), *σ*(*τ*), decreases monotonically from *σ*_{0} to −*σ*_{0} that it reaches in finite time with a strictly negative velocity; this solution does not correspond to an autocovariance function. For *σ*_{0} such that *V*′(*σ*_{0}) = 0 (Fig 17A_{2}) the solution is *σ*(*τ*) = *σ*_{0}. It corresponds to a fixed point of the dynamics, Eq (18) in which all the inputs to the neurons are constant in time, , and has a Gaussian distribution. Finally, for *σ*_{0} such that *V*′(*σ*_{0}) < 0 (Fig 17A_{3}), there is no solution to Eq (33) with *σ*(0) = *σ*_{0}.

The potential, *V*(*σ*, *σ*_{0}) is plotted for different values of *σ*_{0} as a function of *σ*. A_{1–3}: *J*_{0} = 4 < *J*_{c} (=4.995). B_{1–5}: *J*_{0} = 15 > *J*_{c}.

Fig 17B_{1–3} plots *V* for *J*_{0} = 15. For small *σ*_{0}, the solution Eq (33) does not correspond to an autocovariance function. As *σ*_{0} increases, *V*(*σ*) becomes non-monotonic in the vicinity of *σ* = *σ*_{0} with local maxima and minima at *σ* = *σ*_{max} and *σ* = *σ*_{min}, respectively (Fig 17B_{2}). However, here also the solution for *σ*(*τ*) does not correspond to an autocovariance because *σ*_{0} is the global maximum in the range *σ* ∈ [−*σ*_{0}, *σ*_{0}]. For , such that (Fig 17B_{3}) an acceptable solution appears, in which *σ* decays monotonically from and converges to *σ*_{max} as *τ* → ∞, i.e. *σ*_{max} = *σ*_{∞}. This solution corresponds to a chaotic state of the network. If *σ*_{0} is further increased beyond , *V*(*σ*_{max}, *σ*_{0}) > *V*(*σ*_{0}) (Fig 17B_{4}), and the solution exhibits oscillations around *σ*_{min}. For *σ*_{0} ≈ 11.77, *V*′(*σ*_{0}) = 0, and the solution corresponds to a fixed point (Fig 17B_{5}). Finally, for *σ*_{0} larger, *V*′(*σ*_{0}) is negative (Fig 17B_{6}) and there is no solution to Eq (18) with *σ*(0) = *σ*_{0}.

A bifurcation between these behaviors occurs at some critical value, *J*_{c}, such that for *J*_{0} < *J*_{c} the self-consistent solutions of Eq (29) are either oscillatory or constant as a function of *τ*, whereas for *J*_{0} > *J*_{c} they are either oscillatory or decay monotonically. A stability analysis of these different solutions is beyond the scope of this paper; instead, we rely on numerical simulations of the full dynamics. They indicate that the network dynamics always reach a fixed point for sufficiently small *J*_{0}. For sufficiently large *J*_{0} the fixed point is unstable and the network settles in a state in which *σ*(*τ*) decays monotonically with *τ*. Simulations also show that the maximum Lyapunov exponent in these cases is positive (see below); i.e. the network is in a chaotic state. For values of *J*_{0} in between these two regimes, the network displays oscillatory patterns of activity. However, for increasing network sizes, *N*, the range of *J*_{0} in which oscillations are observed vanishes (not shown). Therefore for large *N* the bifurcation between a fixed point and chaos occurs abruptly at some critical value *J*_{c}. A similar phenomenology occurs for other non-linear positive monotonically increasing transfer functions.

In summary, for a fixed feedforward input, *I*_{0}, there are two regimes in the large *N* limit:

- for
*J*_{0}<*J*_{c}: the stable state is a fixed point. The distribution of the inputs to the neurons is a Gaussian whose mean,*μ*, and variance,*σ*are determined by the self-consistent mean-field equations: (34) (35) For a transfer function,*g*(*x*), which is zero when*x*is smaller than some threshold*T*(functions without threshold correspond to*T*= −∞), the distribution of the neuronal firing rates,*r*_{i}, in this state is given by: (36) - for
*J*_{0}>*J*_{c}: the stable state is chaotic. The distribution of time average inputs is Gaussian with mean*μ*and variance and the autocovariance of the inputs is determined by Eq (29) which depends on*σ*_{0}. The quantities*μ*,*σ*_{0}and*σ*_{∞}are determined by the self-consistent equations: (37) and (38) together with Eq (25).

## Two-population networks

### Self-consistent DMFT equations

A DMFT approach can also be developed to investigate the dynamics of the two population network model, Eq (17). To that end, the last term in Eq (17) is written as a Gaussian random process with mean *μ*^{αβ} and autocorrelation function *C*^{αβ}(*τ*) and derives the self-consistent equations that these quantities satisfy. The quantity *μ*^{αβ} is therefore
where:
(39)
is the net input to neuron *i* in population *β*.

The synaptic inputs is also a Gaussian random process. We denote its mean over time and over all the neurons in population *α* by *μ*^{αβ} = *E*[*h*^{αβ}(*t*)] and its PAC by *σ*^{αβ}(*τ*) = *E*[*h*^{αβ}(*t*)*h*^{αβ}(*t*+*τ*)]−(*μ*^{αβ})^{2}. Taking we can write the mean of as
(40)
The PAC of then reads:
We can now write the balance condition in the large *K* limit:
(41)
where
(42)
is the neuronal firing rate averaged over cells in population *α*. Here, .

We can also express *C*^{αβ}(*τ*) in terms of *σ*^{α}(*τ*) as:
(43)
where:
(44)

Let us denote by Δ^{αβ}(*τ*) the autocorrelation of *h*^{αβ}(*t*). We can express the relation between *C*^{αβ}(*τ*) and Δ^{αβ}(*τ*) by their Fourier transforms as Δ^{αβ}(*ω*) = *H*(*ω*)*H**(*ω*)*C*^{αβ}(*ω*), where *H*(*ω*) = 1/(1+*iτ*_{αβ} *ω*). Transforming back to the time domain yields:
(45)
Since Δ^{αβ} = *σ*^{αβ}+(*μ*^{αβ})^{2} we get:
(46)

Thus we get a set of self-consistent equations for the four PACs *σ*^{αβ}. The relevant soutions have to satisfy the four boundary conditions:
(47)

In general, these dynamical equations cannot be written like those of a particle in some potential. This makes the study of their solutions substantially more difficult than in the one population case.

### Separation of time scales

A potential function can be written for the DMFT if the time scale of one type of synapses is substantially larger than the others, which makes it possible to consider the latter as instantaneous. We carry out this analysis below assuming *τ*_{IE} ≫ *τ*_{EI}, *τ*_{EE}, *τ*_{II}.

Setting all the synapses except those from E neurons to I neurons to be instantaneous implies that except for *σ*_{IE} one has:
(48)
where is defined in Eq (44). Since *τ*_{IE} is now the only time scale we can take *τ*_{IE} = 1. Also, *σ*^{EE}, *σ*^{EI}, *σ*^{II} and the potential *V* are now functions of a single variable, *σ*^{IE}. Therefore, the differential equation for *σ*^{IE} can be written as
where
(49)

The instability of the fixed point occurs when, *V*′(*σ*_{IE}) and *V*′′(*σ*_{IE}), the first and the second derivatives of *V* with respect to *σ*^{IE}, vanishes. Using Eq (49) one has:
(50)
Since *σ*^{α} = *σ*^{αE}+*σ*^{αI}:
(51)
and
(52)
where
From Eqs (51)–(52) one gets:
and:
(53)
Thus at chaos onset, together with Eq (41), , *σ*^{α} and *μ*^{α} obey:
(54) (55)
where:

For instance for the threshold-linear transfer function we have
and
where *F*_{i}(*a*, *b*) are defined in Eq (28).

It should be noted that if the transition to chaos occurs for the same parameters for which the fixed point loses stability and that this is controlled by a real eigenvalue crossing zero, the location of the transition will not depend on the synaptic time constant. If this is the case, Eq (54) will characterize the location of the transition to chaos in the parameter space of the network in general and not only under the assumption of the separation of time scales under which we have established this condition.

### On the stability of the fixed point

Let us denote the fixed point solution of the dynamics, Eq (17), by: . Writing with , linearizing the dynamics and looking for solution of the form **δ h** ∝ *e*^{λt}) one gets:
(56)
where the (*α* = *E*, *I*, *β* = *E*, *I*) are *N*×*N* sparse matrices with elements
(57)
(**C**^{αβ} is the matrix of connectivity between populations *β* (presynaptic) and *α*). We are interested in instability onsets at which a real eigenvalue crosses 0.

Using Eq (56), it is straightforward to show that such an instability happens if the synaptic strength are such that: (58)

If , one can rewrite Eq (58) as: (59) with: (60)

Let us assume that is fixed and such that for small enough the fixed point is stable. When increasing, the fixed point loses stability when the value of is the smallest for which Eq (59) is satisfied, that is for which the largest real eigenvalue, *λ*_{max} of the matrix *M* crosses 1. If this instability also corresponds to chaos onset, Eq (54), this would imply that the condition *λ*_{max} = 1 is equivalent to:
(61)
Interestingly, this condition means that the variance of the elements of the matrix is equal to one leading us to conjecture that more generally the eigenvalue of the latter which has the largest real part and is given by:
(62)

## Numerical simulations

### Integration of network dynamics and mean-field equation solutions

The integration of differential equations, Eq (15) and Eq (18) (Eq (3) in main text), was performed with a C code using the Euler method with fixed Δ*t* = *τ*_{syn}/20 (the validity of the results was verified using smaller values of Δ*t*).

Simulations of the LIF spiking networks were done using a second-order Runge-Kutta integration scheme supplemented by interpolation of spike times as detailed in [65]. In all the spiking network simulations the time step was Δ*t* = 0.1 ms.

Self-consistent mean-field equations were solved with MATLAB function *fsolve*, which implements a ‘trust-region-dogleg’ algorithm or the Levenberg-Marquardt algorithm for non-square systems. Numerical calculations of integrals was done with MATLAB function *trapz*.

### Population-averaged autocovariance

The population average autocovariance (PAC) functions of neuronal quantities *f*_{i}(*t*) (*i* = 1…*N*) were computed as
where *N*_{t} is the number of time samples for the calculation of the PAC. In all figures *f*_{i}(*t*) = *h*_{i}(*t*) except in Fig 16 where . All PACs of spiking networks were calculated over 163.84 sec, and averaged over 10 realizations of the connectivity. For models Eq (15) and Eq (18), PACs were calculated over 2048*τ*_{syn} after discarding 200*τ*_{syn} of transient dynamics and averaged over 8 realizations.

### Largest Lyapunov exponents

To calculate the maximal Lyapunov exponent, Λ, of the inhibitory network, Eq (3), we simulated the system for a sufficiently long duration (200*τ*_{syn}) so that it settled on the attractor of the dynamics. Denoting by the network state at that time, we then ran two copies of the dynamics, one with initial conditions and the other with slightly perturbed initial conditions, (, where ∣∣⋅∣∣ is the *l*^{2} norm). Monitoring the difference, we computed and . We then reinitialized the dynamics of the second network copy to . We iterated the process *n* times and estimate the Lyapunov exponent according to:
A similar method was used for two population networks, Eq (15), the only difference being that the vector now had dimension 4*N*. Throughout the article we take *n* = 100, *T*_{max} = 5*τ*_{syn}, *D*_{max} = 10^{−3} and *ϵ* = 10^{−6}. The Lyapunov exponent values reported in this article are averages over 5 realizations of the networks.

### Fraction of networks with a stable fixed point in rate dynamics

Fig 10D in the main text plots the lines in the *J*_{0} − *I*_{0} phase diagrams of the threshold-power law rate model, for which 5%,50%,95% of randomly chosen networks have dynamics which converge to a fixed point. To compute these lines we simulated, for each value of *γ* and *J*_{0}, 100 realizations of the network. For each realization, we computed the population average of the temporal variance the synaptic inputs, *ρ*:
where *N*_{tot} is the total number of time steps of the simulations after discarding a transient with a duration of 256*τ*_{syn}. The fixed point was considered to be unstable if *ρ* > 10^{−9}. The fraction of unstable networks, *F*_{u}, was fitted with a logistic function: *F*_{u}(*J*_{0}) = 100[1+exp(−(*J*_{0} − *J*_{m})/Δ*J*)]^{−1}. The thick red line and red dots plot the values of *J*_{m} vs. *γ*, and the dashed lines are the values of *J*_{0} for which *F*_{u} = 95 and *F*_{u} = 5.

## Supporting Information

### S1 Text. Finite size effects in inhibitory rate networks with sigmoid or threshold-linear transfer functions.

https://doi.org/10.1371/journal.pcbi.1004266.s001

(PDF)

### S2 Text. Chaos onset in inhibitory rate models with twice differentiable transfer functions.

https://doi.org/10.1371/journal.pcbi.1004266.s002

(PDF)

### S3 Text. Asynchronous rate chaos in non-leaky integrate-and-fire networks.

https://doi.org/10.1371/journal.pcbi.1004266.s003

(PDF)

### S4 Text. Onset of chaos in inhibitory rate models with threshold-power-law transfer functions.

https://doi.org/10.1371/journal.pcbi.1004266.s004

(PDF)

### S5 Text. Maximum Lyapunov exponents in the inhibitory LIF rate model.

https://doi.org/10.1371/journal.pcbi.1004266.s005

(PDF)

### S6 Text. Firing statistics in the inhibitory LIF spiking network.

https://doi.org/10.1371/journal.pcbi.1004266.s006

(PDF)

### S7 Text. Two-population rate model with a threshold-linear transfer function with .

https://doi.org/10.1371/journal.pcbi.1004266.s007

(PDF)

### S8 Text. The two mechanisms underlying asynchronous chaos in the two-population rate model with a threshold-linear transfer function.

https://doi.org/10.1371/journal.pcbi.1004266.s008

(PDF)

### S9 Text. The two mechanisms underlying asynchronous chaos in two-population LIF networks: Results for inhibitory neurons.

https://doi.org/10.1371/journal.pcbi.1004266.s009

(PDF)

### S10 Text. Two-population LIF rate and spiking models: In the II mechanism the PAC depends very mildly on *τ*_{IE}.

https://doi.org/10.1371/journal.pcbi.1004266.s010

(PDF)

### S11 Text. Two-population LIF rate and spiking models: In the EIE mechanism the slow component of the PAC depends very mildly on *τ*_{II}.

https://doi.org/10.1371/journal.pcbi.1004266.s011

(PDF)

### S12 Text. Two-population integrate-and-fire network with recurrent EE excitation, AMPA and NMDA synapses and fast inhibition.

https://doi.org/10.1371/journal.pcbi.1004266.s012

(PDF)

### S1 Fig. Finite size effects in inhibitory rate models with sigmoid or threshold-linear transfer functions.

https://doi.org/10.1371/journal.pcbi.1004266.s013

(EPS)

### S2 Fig. The potential function in inhibitory rate models with *g*(*x*) = *ϕ*(*x*).

https://doi.org/10.1371/journal.pcbi.1004266.s014

(EPS)

### S3 Fig. PACs in non-leaky integrate-and-fire networks.

https://doi.org/10.1371/journal.pcbi.1004266.s015

(EPS)

### S4 Fig. The potential function in inhibitory rate models with a threshold-linear transfer function.

https://doi.org/10.1371/journal.pcbi.1004266.s016

(EPS)

### S5 Fig. Lyapunov exponent in simulations of the inhibitory LIF rate model.

https://doi.org/10.1371/journal.pcbi.1004266.s017

(EPS)

### S6 Fig. Firing statistics in the LIF inhibitory spiking model.

https://doi.org/10.1371/journal.pcbi.1004266.s018

(EPS)

### S7 Fig. Two-population rate model with a threshold-linear transfer function and EE connections.

https://doi.org/10.1371/journal.pcbi.1004266.s019

(EPS)

### S8 Fig. The two mechanisms underlying chaos in the two-population rate model with a threshold-linear transfer functions.

https://doi.org/10.1371/journal.pcbi.1004266.s020

(EPS)

### S9 Fig. The PAC for the inhibitory neurons in the mechanisms underlying chaos in two-population LIF spiking and rate networks.

https://doi.org/10.1371/journal.pcbi.1004266.s021

(EPS)

### S10 Fig. The PAC depends very mildly on *τ*_{IE} in the II mechanism.

https://doi.org/10.1371/journal.pcbi.1004266.s022

(EPS)

### S11 Fig. In the EIE mechanism the slow component of the PAC depends very mildly on *τ*_{II}.

https://doi.org/10.1371/journal.pcbi.1004266.s023

(EPS)

### S12 Fig. Two-population integrate-and-fire network with AMPA and NMDA synapses.

https://doi.org/10.1371/journal.pcbi.1004266.s024

(EPS)

## Acknowledgments

We thank Gerard Benarous, Ran Darshan, Gianluigi Mongillo, Carl van Vreeswijk and Fred Wolf for insightful discussions. We thank Andrea Crisanti for sharing with us unpublished notes related to [19].

## Author Contributions

Conceived and designed the experiments: OH DH. Performed the experiments: OH DH. Analyzed the data: OH DH. Contributed reagents/materials/analysis tools: OH DH. Wrote the paper: OH DH.

## References

- 1. Faure P, Korn H (1997) A nonrandom dynamic component in the synaptic noise of a central neuron. Proc Natl Acad Sci U.S.A. 94:6506–6511. pmid:9177248
- 2. Babloyantz A, Destexhe A (1986) Low-dimensional chaos in an instance of epilepsy. Proc Natl Acad Sci U.S.A. 83:3513–3517. pmid:3085091
- 3. Skarda CA, Freeman WJ (1987) Rate models for Conductance-Based Cortical Neuronal Networks. Behav Brain Sci 10: 161–195.
- 4. Brecht M, Schneider M, Sakmann B, Margrie TW (2004) Whiskers evoked by stimulation of single pyramidal cells in rat motor cortex. Nature 427:704–710. pmid:14973477
- 5. Yang Y, DeWeese MR, Otazu GHH, Zador AM (2008) Millisecond-scale differences in neural activity in auditory cortex can drive decisions. Nat Neurosci 11:1262–1263. pmid:18849984
- 6. London M, Roth A, Beeren L, Hausser M, Latham PE (2010) Sensitivity to perturbations in vivo implies high noise and suggests rate coding in cortex. Nature 466:123–127. pmid:20596024
- 7. Hansel D, Sompolinsky H (1992) Synchronization and computation in a chaotic neural network. Phys Rev Lett 68:718–721. pmid:10045972
- 8. Hansel D, Sompolinsky H (1996) Chaos and synchrony in a model of a hypercolumn in visual cortex. J Comput Neurosci 3:7–34. pmid:8717487
- 9. Roxin A, Brunel N, Hansel D (2005) Role of delays in shaping spatiotemporal dynamics of neuronal activity in large networks. Phys Rev Lett 94:238103. pmid:16090506
- 10. Battaglia D, Brunel N, Hansel D (2007) Temporal decorrelation of collective oscillations in neural networks with local inhibition and long-range excitation. Phys Rev Lett 99:238106. pmid:18233419
- 11. Battaglia D, Hansel D (2011) Synchronous chaos and broad band gamma rhythm in a minimal multi-layer model of primary visual cortex. PLoS Comput Biol 7:e1002176. pmid:21998568
- 12. Rangan AV, Young LS (2013) Emergent dynamics in a model of visual cortex. J Comput Neurosci 35:155–67. pmid:23519442
- 13. Renart A, de la Rocha J, Bartho P, Hollender L, Parga N, Reyes A, Harris KD (2010) The asynchronous state in cortical circuits. Science 327:587–590. pmid:20110507
- 14. Ecker AS, Berens P, Keliris GA, Bethge M, Logothetis NK, Tolias AS (2010) Decorrelated Neuronal Firing in Cortical Microcircuits. Science 327:584–587. pmid:20110506
- 15. Sussillo D, Abbott LF (2009) Generating coherent patterns of activity from chaotic neural networks. Neuron 63: 544–557. pmid:19709635
- 16. Barak O, Sussillo D, Romo R, Tsodyks M, Abbott LF (2013) From fixed points to chaos: Three models of delayed discrimination. Prog Neurobiol 103: 214–222. pmid:23438479
- 17. Toyoizumi T, Abbott LF (2011) Beyond the edge of chaos: amplification and temporal integration by recurrent networks in the chaotic regime. Phys Rev E 84:051908.
- 18.
Nishikawa I, Aihara K, Toyoizumi T (2013) Signal processing in neural networks that generate or receive noise. Cosyne Abstracts 2013, Salt Lake City USA.
- 19. Sompolinsky H, Crisanti A, Sommers HJ (1988) Chaos in random neural networks. Phys Rev Lett 61:259–262. pmid:10039285
- 20.
Dayan P, Abbott LF (2001) Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems. MIT Press.
- 21. Hansel D, Sompolinsky H (1993). A solvable model of spatiotemporal chaos. Phys. Rev. Lettersi 71:2710–2713.
- 22. Rajan K, Abbott LF, Sompolinsky H (2011) Stimulus-dependent suppression of chaos in recurrent neural networks. Phys Rev E 82:011903.
- 23. Garcia del Molino LC, Pakdaman K, Touboul J, Wainrib G. (2013) Synchronization in random balanced networks. Phys. Rev. E. Stat. Nonlin. Soft Matter Phys. 88(4):042824. pmid:24229242
- 24. Wainrib G, Touboul J. (2013) Topological and dynamical complexity of random neural networks. Phys. Rev. Lett. 110(11):118101. pmid:25166580
- 25. Stern M, Sompolinsky H, Abbott LF. (2013) Dynamics of random neural networks with bistable units. Phys. Rev. E. Stat. Nonlin. Soft Matter Phys. 90(6–1):062710.
- 26. Dale HH (1934) Pharmacology and nerve endings: first Dixon memorial lecture by sir Henry Dale. Br Med J 2:1161–1163.
- 27.
Harish O (2013) Network mechanisms of working memory: from persistent dynamics to chaos. Ph.D. thesis: Université Paris Descartes, France
- 28. van Vreeswijk C, Sompolinsky H (1996) Chaos in neuronal networks with balanced excitatory and inhibitory. Science 274:1724–1726. pmid:8939866
- 29. van Vreeswijk C, Sompolinsky H (1998) Chaotic Balanced State in a Model of Cortical Circuits. Neural Comput 10:1321–1372. pmid:9698348
- 30.
van Vreeswijk C, Sompolinsky H (2005) Les Houches Lectures LXXX on Methods and models in neurophysics. Elsevier 341–402.
- 31. Hansel D, van Vreeswijk C (2012) The mechanism of orientation selectivity in primary visual cortex without a functional map. J Neurosci 32:4049–4064. pmid:22442071
- 32. Hansel D, Mato G (2013) Short-term plasticity explains irregular persistent activity in working memory tasks. J Neurosci 33:133–149. pmid:23283328
- 33. Jahnke S, Memmesheimer RM, Timme M (2008) Stable irregular dynamics in complex neural networks. Phys Rev Lett 100:048102. pmid:18352336
- 34. Jahnke S, Memmesheimer RM, Timme M (2009) How Chaotic is the Balanced State?. Front Comput Neurosci 3:13. pmid:19936316
- 35. Zilmer R, Brunel N, Hansel D (2009) Very long transients irregular firing and chaotic dynamics in networks of randomly connected inhibitory integrate-and-fire neurons. Phys Rev E 79:031909.
- 36. Monteforte M, Wolf F (2010) Dynamical entropy production in spiking neuron networks in the balanced state. Phys Rev Lett 105:268104. pmid:21231716
- 37. Monteforte M, Wolf F (2011) Single Cell Dynamics Determine Strength of Chaos in Collective Network Dynamics. Front Neurosci Conference Abstract: Computational and Systems Neuroscience 2010.
- 38. Shriki O, Hansel D, Sompolinsky H (2003) Rate models for Conductance-Based Cortical Neuronal Networks. Neural Comput 15:1809–1841. pmid:14511514
- 39.
Strogatz SH (1994), Nonlinear dynamics and chaos. Westview Press.
- 40. Mongillo G, Amit DJ (2001) Oscillations and irregular emission in networks of linear spiking neurons. J Comput Neurosci 11:249–261. pmid:11796941
- 41. Brumberg JC, Gutkin BS (2007) Cortical pyramidal cells as non-linear oscillators: experiment and spike-generation theory. Brain Res 1171:122–137. pmid:17716635
- 42. Gutkin BS, Ermentrout GB (1998) Dynamics of membrane excitability determine interspike interval variability: a link between spike generation mechanisms and cortical spike train statistics. Neural Comput 10:1047–1065. pmid:9654767
- 43. Ermentrout GB, Kopell N (1986) Parabolic Bursting in an Excitable System Coupled with a Slow Oscillation. SIAM J Appl Math 46:233–253.
- 44. Hansel D, Mato G (2003) Asynchronous states and the emergence of synchrony in large networks of interacting excitatory and inhibitory neurons. Neural Comput 15:1–56. pmid:12590818
- 45. Hansel D, van Vreeswijk V (2002) How noise contributes to contrast invariance of orientation tuning. J Neurosci 22:5118–5128. pmid:12077207
- 46. Priebe NJ, Ferster D (2008) Inhibition spike threshold and stimulus selectivity in primary visual cortex. Neuron 57:482–497. pmid:18304479
- 47. Atallah BV, Bruns W, Carandini M, Scanziani M (2012) Parvalbumin-Expressing Interneurons Linearly Transform Cortical Responses to Visual Stimuli. Neuron 73, 159170.
- 48.
Ostojic S, and Brunel N (2011) From spiking neuron models to linear-nonlinear models.
*PLoS Comput. Biol*7:e1001056. pmid:21283777 - 49. White B, Abbott LF, Fiser J (2012) Suppression of cortical neural variability is stimulus- and state-dependent. J Neurophysiol 108:2383–2392. pmid:22896720
- 50. Girko VL (1984) Circular Law Theory Probab Appl 29:694–706.
- 51. Tao T, Vu V (2010) Random matrices: Universality of ESD and the Circular Law (with appendix by M. Krishnapur). Annals of Probability 38:20232065.
- 52. Rajan K, Abbott LF (2006) Eigenvalue spectra of random matrices for neural networks. Phys Rev Lett 97:188104. pmid:17155583
- 53. Tao T (2013) Outliers in the spectrum of iid matrices with bounded rank perturbations. Probab Theory Relat Fields 115:231–263.
- 54. Ahmadian Y, Fumarola F, Miller KD (2014) Properties of networks with partially structured and partially random connectivity, Phys Rev E 91:012820.
- 55. Goris RLT, Movshon AJ, Simoncelli EP (2014) Partitioning neuronal variability. Nat Neurosci 17:858–65. pmid:24777419
- 56. Murray JD, Bernacchia A, Freedman DJ, Romo R, Wallis JD, Padoa-Schioppa C, Pasternak T, Seo H, Lee D, Wang XJ (2014) A hierarchy of intrinsic timescales across primate cortex. Nat Neurosci 12:1661–1663.
- 57. Bartos M, Vida I, Frotscher M, Geiger JR, Jonas P (2001) Rapid signaling at inhibitory synapses in a dentate gyrus interneuron network. J Neurosci 21:2687–2698. pmid:11306622
- 58. Sceniak MP, Maciver MB (2008) Slow GABA(A) mediated synaptic transmission in rat visual cortex. BMC Neurosci 9:8. pmid:18199338
- 59. Gassmann M, Bettler B (2012) Regulation of neuronal GABA B receptor functions by subunit composition. Nat Rev Neurosci 13:380–394. pmid:22595784
- 60. Carmignoto G, Vicini S (1992) Activity-dependent decrease in NMDA receptor responses during development of the visual cortex. Science. 6:1007–1011.
- 61. Gotz T, Kraushaar U, Geiger J, Lubke J, Berger T, Jonas P (1997) Functional properties of AMPA and NMDA receptors expressed in identified types of basal ganglia neurons. J Neurosci 17:204–215. pmid:8987749
- 62. Wang H, Stradtman GG, Wang XJ, Gao WJ (2008) A specialized NMDA receptor function in layer 5 recurrent microcircuitry of the adult rat prefrontal cortex. Proc Natl Acad Sci U.S.A. 105:16791–16796. pmid:18922773
- 63. Ostojic S (2014) Two types of asynchronous activity in networks of excitatory and inhibitory spiking neurons. Nature Neuroscience 17:514–601.
- 64. Engelken R, Farkhooi F, Hansel D, van Vreeswijk C, Wolf F (2015) Comment on Two types of asynchronous activity in networks of excitatory and inhibitory spiking neurons. bioRxiv
- 65. Hansel D, Mato G, Meunier C, Neltner L (1998) On numerical simulations of integrate-and-fire neural networks. Neural Comput 10:467–483. pmid:9472491