Decorrelation of Neural-Network Activity by Inhibitory Feedback

Correlations in spike-train ensembles can seriously impair the encoding of information by their spatio-temporal structure. An inevitable source of correlation in finite neural networks is common presynaptic input to pairs of neurons. Recent studies demonstrate that spike correlations in recurrent neural networks are considerably smaller than expected based on the amount of shared presynaptic input. Here, we explain this observation by means of a linear network model and simulations of networks of leaky integrate-and-fire neurons. We show that inhibitory feedback efficiently suppresses pairwise correlations and, hence, population-rate fluctuations, thereby assigning inhibitory neurons the new role of active decorrelation. We quantify this decorrelation by comparing the responses of the intact recurrent network (feedback system) and systems where the statistics of the feedback channel is perturbed (feedforward system). Manipulations of the feedback statistics can lead to a significant increase in the power and coherence of the population response. In particular, neglecting correlations within the ensemble of feedback channels or between the external stimulus and the feedback amplifies population-rate fluctuations by orders of magnitude. The fluctuation suppression in homogeneous inhibitory networks is explained by a negative feedback loop in the one-dimensional dynamics of the compound activity. Similarly, a change of coordinates exposes an effective negative feedback loop in the compound dynamics of stable excitatory-inhibitory networks. The suppression of input correlations in finite networks is explained by the population averaged correlations in the linear network model: In purely inhibitory networks, shared-input correlations are canceled by negative spike-train correlations. In excitatory-inhibitory networks, spike-train correlations are typically positive. Here, the suppression of input correlations is not a result of the mere existence of correlations between excitatory (E) and inhibitory (I) neurons, but a consequence of a particular structure of correlations among the three possible pairings (EE, EI, II).


Introduction
Neurons generate signals by weighting and combining input spike trains from presynaptic neuron populations. The number of possible signals which can be read out this way from a given spiketrain ensemble is maximal if these spike trains span an orthogonal basis, i.e. if they are uncorrelated [1]. If they are correlated, the amount of information which can be encoded in the spatiotemporal structure of these spike trains is limited. In addition, correlations impair the ability of readout neurons to decode information reliably in the presence of noise. This is often discussed in the context of rate coding: for N uncorrelated spike trains, the signal-to-noise ratio of the compound spike-count signal can be enhanced by increasing the population size N. In the presence of correlations, however, the signal-to-noise ratio is bounded [2,3]. The same reasoning holds for any other linear combination of spike trains, also for those where exact spike timing matters (for example for the coding scheme presented in [4]). Thus, the robustness of neuronal responses against noise critically depends on the level of correlated activity within the presynaptic neuron population.
Several studies suggested that correlated neural activity could be beneficial for information processing: Spike-train correlations can modulate the gain of postsynaptic neurons and thereby constitute a gating mechanism (for a review, see [4]). Coherent spiking activity might serve as a means to bind elementary representations into more complex objects [5,6]. Information represented by correlated firing can be reliably sustained and propagated through feedforward subnetworks ('synfire chains'; [7,8]). Whether correlated firing has to be considered favorable or not largely depends on the underlying hypothesis, the type of correlation (e.g. the time scale or the affected frequency band) or which subpopulations of neurons are involved. Most ideas suggesting a functional benefit of correlated activity rely on the existence of an asynchronous 'ground state'. Spontaneously emerging correlations, i.e. correlations which are not triggered by internal or external events, would impose a serious challenge to many of these hypotheses. Functionally relevant synfire activity, for example, cannot be guaranteed in the presence of correlated background input from the embedding network [9]. It is therefore-from several perspectives-important to understand the origin of uncorrelated activity in neural networks.
It has recently been shown that spike trains of neighboring cortical neurons can indeed be uncorrelated [10]. Similar results have been obtained in several theoretical studies [11][12][13][14][15][16][17]. From an anatomical point of view, this observation is puzzling: in general, neurons in finite networks share a certain fraction of their presynaptic sources. In particular for neighboring neurons, the overlap between presynaptic neuron populations is expected to be substantial. This feedforward picture suggests that such presynaptic overlap gives rise to correlated synaptic input and, in turn, to correlated response spike trains.
A number of theoretical studies showed that shared-input correlations are only weakly transferred to the output side as a consequence of the nonlinearity of the spike-generation dynamics [15,[18][19][20][21]. Unreliable spike transmission due to synaptic failure can further suppress the correlation gain [22]. In [9], we demonstrated that spike-train correlations in finite-size recurrent networks are even smaller than predicted by the low correlation gain of pairs of neurons with nonlinear spike-generation dynamics. We concluded that this suppression of correlations must be a result of the recurrent network dynamics. In this article, we compare correlations observed in feedforward networks to correlations measured in systems with an intact feedback loop. We refer to the reduction of correlations in the presence of feedback as ''decorrelation''. Different mechanisms underlying such a dynamical decorrelation have been suggested in the recent past. Asynchronous states in recurrent neural networks are often attributed to chaotic dynamics [23,24]. In fact, networks of nonlinear units with random connectivity and balanced excitation and inhibition typically exhibit chaos [11,25]. The high sensitivity to noise may however question the functional relevance of such systems ( [26,27]; cf., however, [28]). [29] and [27] demonstrated that asynchronous irregular firing can also emerge in networks with stable dynamics. Employing an analytical framework of correlations in recurrent networks of binary neurons [30], the balance of excitation and inhibition has recently been proposed as another decorrelation mechanism [17]: In large networks, fluctuations of excitation and inhibition are in phase. Positive correlations between excitatory and inhibitory input spike trains lead to a negative component in the net input correlation which can compensate positive correlations caused by shared input.
In the present study, we demonstrate that dynamical decorrelation is a fundamental phenomenon in recurrent systems with negative feedback. We show that negative feedback alone is sufficient to efficiently suppress correlations. Even in purely inhibitory networks, shared-input correlations are compensated by feedback. A balance of excitation and inhibition is thus not required. The underlying mechanism can be understood by means of a simple linear model. This simplifies the theory and helps to gain intuition, but it also confirms that low correlations can emerge in recurrent networks with stable, non-chaotic dynamics.
The suppression of pairwise spike-train correlations by inhibitory feedback is reflected in a reduction of population-rate fluctuations. The main effect described in this article can therefore be understood by studying the dynamics of the macroscopic population activity. This approach leads to a simple mathematical description and emphasizes that the described decorrelation mechanism is a general phenomenon which may occur not only in neural networks but also in other (biological) systems with inhibitory feedback. In ''Results: Suppression of population-rate fluctuations in LIF networks'', we first illustrate the decorrelation effect for random networks of N leaky integrateand-fire (LIF) neurons with inhibitory or excitatory-inhibitory coupling. By means of simulations, we show that low-frequency spike-train correlations, and, hence, population-rate fluctuations are substantially smaller than expected given the amount of shared input. As shown in the subsequent section, the ''Suppression of population-activity fluctuations by negative feedback'' can readily be understood in the framework of a simple onedimensional linear model with negative feedback. In ''Results: Population-activity fluctuations in excitatory-inhibitory networks'', we extend this to a two-population system with excitatory-inhibitory coupling. Here, a simple coordinate transform exposes the inherent negative feedback loop as the underlying cause of the fluctuation suppression in inhibition-dominated networks. The population-rate models of the inhibitory and the excitatory-inhibitory network are sufficient to understand the basic mechanism underlying the decorrelation. They do, however, not describe how feedback in cortical networks affects the detailed structure of pairwise correlations. In ''Results: Population averaged correlations in cortical networks'', we therefore compute self-consistent population averaged correlations for a random network of N linear excitatory and inhibitory neurons. By determining the parameters of the linear network analytically from the LIF model, we show that the predictions of the linear model are-for a wide and realistic range of parameters-in excellent agreement with the results of the LIF network model. In ''Results: Effect of feedback manipulations'', we demonstrate that the active decorrelation in random LIF networks relies on the feedback of the (sub)population averaged activity but not on the precise microscopic structure of the feedback signal. In the ''Discussion'', we put the consequences of this work into a broader context and point out limitations and possible extensions of the presented theory. The ''Methods'' contain details on the LIF network model, the derivation of the linear model from the LIF dynamics and the derivation of population-rate spectra and population averaged correlations in the framework of the linear model. This section is meant as a supplement; the basic ideas and the main results can be extracted from the ''Results''.

Author Summary
The spatio-temporal activity pattern generated by a recurrent neuronal network can provide a rich dynamical basis which allows readout neurons to generate a variety of responses by tuning the synaptic weights of their inputs. The repertoire of possible responses and the response reliability become maximal if the spike trains of individual neurons are uncorrelated. Spike-train correlations in cortical networks can indeed be very small, even for neighboring neurons. This seems to be at odds with the finding that neighboring neurons receive a considerable fraction of inputs from identical presynaptic sources constituting an inevitable source of correlation. In this article, we show that inhibitory feedback, abundant in biological neuronal networks, actively suppresses correlations. The mechanism is generic: It does not depend on the details of the network nodes and decorrelates networks composed of excitatory and inhibitory neurons as well as purely inhibitory networks. For the case of the leaky integrate-and-fire model, we derive the correlation structure analytically. The new toolbox of formal linearization and a basis transformation exposing the feedback component is applicable to a range of biological systems. We confirm our analytical results by direct simulations.

Results
In a recurrent neural network of size N, each neuron i[½1,N receives in general inputs from two different types of sources: External inputs j i (t) representing the sum of afferents from other brain areas, and local inputs resulting from the recurrent connectivity within the network. Depending on their origin, external inputs j i and j j to different neurons i and j can be correlated or not. Throughout this manuscript, we ignore correlations between these external sources, thereby ensuring that correlations within the network activity arise from the local connectivity alone and are not imposed by external inputs [17]. The local inputs feed the network's spiking activity s(t)~(s 1 (t), . . . ,s N (t)) T back to the network (we refer to spike train s i (t), the i th component of the column vector s(t) [the superscript ''T'' denotes the transpose], as a sum over deltafunctions centered at the spike times t k i : s i (t)~P k d(t{t k i ); the abstract quantity 'spike train' can be considered as being derived from the observable quantity 'spike count' n Dt i (t), the number of spikes occurring in the time interval ½t,tzDt), by taking the limit Dt?0: s i (t)~lim Dt?0 1 Dt n Dt i (t)). The structure and weighting of this feedback can be described by the network's connectivity matrix J (see Fig. 1 A). In a finite network, the local connectivity typically gives rise to overlapping presynaptic populations: in a random (Erdös-Rényi) network with connection probability E, for example, each pair of postsynaptic neurons shares, on average, E 2 N presynaptic sources. For a network size of, say, N~10 4 and a connection probability E~0:1, this corresponds to a fairly large number of 100 identical inputs. For other network structures, the amount of shared input may be smaller or larger. Due to this presynaptic overlap, each pair of neurons receives, to some extent, correlated input (even if the external inputs are uncorrelated). One might therefore expect that the network responses s 1 (t), . . . ,s N (t) are correlated as well. In this article, we show that, in the presence of negative feedback, the effect of shared input caused by the structure of the network is compensated by its recurrent dynamics.

Suppression of population-rate fluctuations in LIF networks
To illustrate the effect of shared input and its suppression by the recurrent dynamics, we compare the spike response s(t)( s 1 (t), . . . ,s N (t)) T of a recurrent random network (feedback scenario; Fig. 1 A,C,E) of N LIF neurons to the case where the feedback is cut and replaced by a spike-train ensemble q(t)~(q 1 (t), . . . ,q N (t)) T , modeled by N independent realizations of a stationary Poisson point process (feedforward scenario; Fig. 1 B,D,F). The rate of this Poisson process is identical to the time and population averaged firing rate in the intact recurrent system. In both the feedback and the feedforward case, the (local) presynaptic spike trains are fed to the postsynaptic population according to the same connectivity matrix J. Therefore, not only the in-degrees and the synaptic weights but also the shared-input statistics are exactly identical.
For realistic size N and connectivity E, asynchronous states of random neural networks [12,31] exhibit spike-train correlations which are small but not zero (compare raster displays in Fig. 1 C and D; see also [15]). Although the presynaptic spike trains are, by construction, independent in the feedforward case ( Fig. 1 D), the resulting response correlations, and, hence, the population-rate fluctuations, are substantially stronger than those observed in the feedback scenario (compare Fig. 1 F and E). In other words: A theory which is exclusively based on the amount of shared input but neglects the details of the presynaptic spike-train statistics can significantly overestimate correlations and population-rate fluctuations in recurrent neural networks.
The same effect can be observed in LIF networks with both purely inhibitory and mixed excitatory-inhibitory coupling (Fig. 2). To demonstrate this quantitatively, we focus on the fluctuations of the population averaged activity s(t)~N {1 P N i~1 s i (t). Its powerspectrum (or auto-correlation, in the time domain) is determined both by the power-spectra (auto-correlations) of the individual spike trains and the cross-spectra represents the Fourier transform of the spike train s k (t)). We observe that the spike-train power-spectra A i (v) (and auto-correlations) are barely distinguishable in the feedback and in the feedforward case (not shown here; the main features of the spike-train auto-correlation are determined by the average singleneuron firing rate and the refractory mechanism; both are identical in the feedback and the feedforward scenario). The differences in the population-rate spectra C SS (v) are therefore essentially due to differences in the spike-train cross-spectra C ij (v). In other words, the fluctuations in the population activity serve as a measure of pairwise spike-train correlations [32]: small (large) population averaged spike-train correlations are accompanied by small (large) fluctuations in the population rate (see lower panels in Fig. 1 C-F). The power-spectra C SS (v) of the population averaged activity reveal a feedback-induced suppression of the population-rate variance at low frequencies up to several tens of Hertz. For the examples shown in Fig. 2, this suppression spans more than three orders of magnitude for the inhibitory and more than one order of magnitude for the excitatory-inhibitory network.
The suppression of low-frequency fluctuations does not critically depend on the details of the network model. As shown in Fig. 2, it can, for example, be observed for both networks with zero risetime synapses (d-shaped synaptic currents) and short delays and for networks with delayed low-pass filtering synapses (a-shaped synaptic currents). In the latter case, the suppression of fluctuations is slightly more restricted to lower frequencies (v10 Hz). Here, the fluctuation suppression is however similarly pronounced as in networks with instantaneous synapses.
In Fig. 2 C,D, the power-spectra of the population activity converge to the mean firing rate at high frequencies. This indicates that the spike trains are uncorrelated on short time scales. For instantaneous d-synapses, neurons exhibit an immediate response to excitatory input spikes [33,34]. This fast response causes spiketrain correlations on short time scales. Hence, the compound power at high frequencies is increased. In a recurrent system, this effect is amplified by reverberating simultaneous excitatory spikes. Therefore, the high-frequency power of the compound activity is larger in the feedback case (Fig. 2 B). Note that this high-frequency effect is absent in networks with more realistic low-pass filtering synapses (Fig. 2 C,D) and in purely inhibitory networks (Fig. 2 A).
Synaptic delays and slow synapses can promote oscillatory modes in certain frequency bands [12,31], thereby leading to peaks in the population-rate spectra in the feedback scenario which exceed the power in the feedforward case (see peaks at *25 Hz in Fig. 2 C,D). Note that, in the feedforward case, the local input was replaced by a stationary Poisson process, whereas in the recurrent network (feedback case) the presynaptic spike trains exhibit oscillatory modes. By replacing the feedback by an inhomogeneous Poisson process with a time dependent intensity which is identical to the population rate in the recurrent network, we found that these oscillatory modes are neither suppressed nor amplified by the recurrent dynamics, i.e. the peaks in the resulting power-spectra have the same amplitude in the feedback and in the feedforward case (data not shown here). At low frequencies, however, the results are identical to those obtained by replacing the feedback by a homogeneous Poisson process (i.e. to those shown in Fig. 2; see ''Results: Effect of feedback manipulations''). In the present study, we mainly focus on these lowfrequency effects.
The observation that the suppression of low-frequency fluctuations is particularly pronounced in networks with purely inhibitory coupling indicates that inhibitory feedback may play a key role for the underlying mechanism. In the following subsection, we demonstrate by means of a one-dimensional linear population model that, indeed, negative feedback alone leads to an efficient fluctuation suppression.

Suppression of population-activity fluctuations by negative feedback
Average pairwise correlations can be extracted from the spectrum (1) of the compound activity, provided the single spiketrain statistics (auto-correlations) is known (see previous section). As the single spike-train statistics is identical in the feedback and in the feedforward scenario, the mechanism underlying the decorrelation in recurrent networks can be understood by studying the dynamics of the population averaged activity. In this and in the next subsection, we consider the linearized dynamics of random Figure 1. Spiking activity in excitatory-inhibitory LIF networks with intact (left column; feedback scenario) and opened feedback loop (right column; feedforward scenario). A,B: Network sketches for the feedback (A) and feedforward scenario (B). C,D: Spiking activity (top panels) and population averaged firing rate (bottom panels) of the local presynaptic populations. E,F: Response spiking activity (top panels) and population averaged response rate (bottom panels). In the top panels of C-F, each pixel depicts the number of spikes (gray coded) of a subpopulation of 250 neurons in a 10 ms time interval. In both the feedback and the feedforward scenario, the neuron population f1, . . . ,Ng is driven by the same realization x(t)~(j 1 (t), . . . ,j N (t)) T of an uncorrelated white-noise ensemble; local input is fed to the population through the same connectivity matrix J. The in-degrees, the synaptic weights and the shared-input statistics are thus exactly identical in the two scenarios. In the feedback case (A), local presynaptic spike-trains are provided by the network's response s(t)~(s 1 (t), . . . ,s N (t)) T , i.e. the pre-(C) and postsynaptic spike-train ensembles (E) are identical. In the feedforward scenario (B), the local presynaptic spike-train population is replaced by an ensemble of N independent realizations q(t)~(q 1 (t), . . . ,q N (t)) of a Poisson point process (D). Its rate is identical to the time-and population-averaged firing rate in the feedback case. See Table 1 and Table 2 for details on network models and parameters. doi:10.1371/journal.pcbi.1002596.g001 networks composed of homogeneous subpopulations of LIF neurons. The high-dimensional dynamics of such systems can be reduced to low-dimensional models describing the dynamics of the compound activity (for details, see ''Methods: Linearized network model''). Note that this reduction is exact for networks with homogeneous out-degree (number of outgoing connections). For the networks studied here (random networks with homogeneous in-degree), it serves as a sufficient approximation (in a network of size N where each connection is randomly and independently realized with probability E [Erdös-Rényi graph], the [binomial] in-and out-degree distributions become very narrow for large N [relative to the mean in/out-degree]; both in-and outdegree are therefore approximately constant across the population of neurons). In this subsection, we first study networks with purely inhibitory coupling. In ''Results: Population-activity fluctuations in excitatory-inhibitory networks'', we investigate the effect of mixed excitatory-inhibitory connectivity.
Consider a random network of N identical neurons with connection probability E. Each neuron i~1, . . . ,N receives K~EN randomly chosen inputs from the local network with synaptic weights {J. In addition, the neurons are driven by external uncorrelated Gaussian white noise j i (t) with amplitude g, i.e. E t j i (t) ½ ~0 and E t j i (t) ½ j j (tzt)~d ij g 2 d(t). For small input fluctuations, the network dynamics can be linearized. This linearization is based on the averaged response of a single neuron to an incoming spike and describes the activity of an individual neuron i by an abstract fluctuating quantity r i (t) which is defined such that within the linear approximation its auto-and crosscorrelations fulfill the same linearized equation as the spiking model in the low-frequency limit. Consequently, also the lowfrequency fluctuations of the population spike rate are captured correctly by the reduced model up to linear order. This approach is equivalent to the treatment of finite-size fluctuations in spiking networks (see, e.g., [31]). For details, see ''Methods: Linearized network model''. For large N, the population averaged activity r(t)~E i r i (t) ½ ~N {1 P N i~1 r i (t) can hence be described by a onedimensional linear system with linear kernel h(t), effective coupling strength w w~Kw and the population averaged noise x(t)~E i x i ½ (t) (see ''Methods: Linearized network model'' and Fig. 3 B). The coupling strength w w represents the integrated linear response of the neuron population to a small perturbation in the input rate of a single presynaptic neuron. For a population of LIF neurons, its relation to the synaptic weight J (PSP amplitude) is derived in ''Methods: Linearized network model'' and ''Methods: Response  Table 1 and Table 2 for details on network models and parameters. In C and D, local synaptic inputs are modeled as currents I i (t)~P j J ij P l psc(t{t j l {d) with a-function shaped kernel psc(t)~ett  Table 1 for details). Simulation time T~100 s. Single-trial spectra smoothed by moving average (frame size 1 Hz). kernel of the LIF model''. The normalized kernel h(t) (with Ð ? 0 dt h(t)~1) captures the time course of the linear response. It is determined by the single-neuron properties (e.g. the spikeinitiation dynamics [35,36]), the properties of the synapses (e.g. synaptic weights and time constants [37,38]) and the properties of the input (e.g. excitatory vs. inhibitory input [39]). For many real and model neurons, the linear population-rate response exhibits low-pass characteristics [13,[34][35][36][37][38][39][40][41][42][43][44][45][46]. For illustration (Fig. 3), we consider a 1st-order low-pass filter, i.e. an exponential impulse response h(t)~t {1 exp({t=t)H(t) with time constant t (cutoff frequency f c~( 2pt) {1 ; see Fig. 3 A, light gray curve in E). The results of our analysis are however independent of the choice of the kernel h(t). The auto-correlation E t x(t)x(tzt) ½ r r 2 d(t) of the external noise is parametrized by the effective noise amplitude r r~r= ffiffiffiffi ffi N p .
Given the simplified description (2), the suppression of response fluctuations by negative feedback can be understood intuitively: Consider first the case where the neurons in the local network are unconnected (Fig. 3 A; no feedback, w w~0). Here, the response r(t) (Fig. 3 A 3 ) is simply a low-pass filtered version of the external input x(t) (Fig. 3 A 1 ), resulting in an exponentially decaying response auto-correlation (Fig. 3 D; light gray curve) and a drop in the response power-spectrum at the cutoff frequency f c (Fig. 3 E). At low frequencies, r(t) and x(t) are in phase; they are correlated. In the presence of negative feedback (Fig. 3 B), the local input { w wr(t) (Fig. 3 B 2 ) and the low-frequency components of the external input x(t) (Fig. 3 B 1 ) are anticorrelated. They partly cancel out, thereby reducing the response fluctuations r(t) (Fig. 3 B 3 ). The autocorrelation function and the power-spectrum are suppressed (Fig. 3 D,E; black curves). Due to the low-pass characteristics of the system, mainly the low-frequency components of the external drive x(t) are transferred to the output side and, in turn, become available for the feedback signal. Therefore, the canceling of input fluctuations and the resulting suppression of response fluctuations are most efficient at low frequencies. Consequently, the autocorrelation function is sharpened (see inset in Fig. 3 D). The cutoff frequency of the system is increased (Fig. 3 E; black curve). This effect of negative feedback is very general and well known in the engineering literature. It is employed in the design of technical devices, like, e.g., amplifiers [47]. As the zero-frequency power is identical to the integrated auto-correlation function, the suppression of low-frequency fluctuations is accompanied by a reduction in the auto-correlation area (Fig. 3 D; black curve). Note that the suppression of fluctuations in the feedback case is not merely a result of the additional inhibitory noise source provided by the local input, but follows from the precise temporal alignment of the local and the external input. To illustrate this, let's consider the case where the feedback channel is replaced by a feedforward input q(t) (Fig. 3 C) which has the same auto-statistics as the response r(t) in the feedback case ( Fig. 3 B 3 ) but is uncorrelated to the external drive x(t). In this case, external input fluctuations The feedback induced suppression of response fluctuations can be quantified by comparing the response power-spectra and in the feedback (Fig. 3 B) and the feedforward case ( Fig. 3 C), respectively (see ''Methods: Population-activity spectrum of the linear inhibitory network''). Here, R(v) andR R(v) denote the Fourier transforms of the response fluctuations in the feedback and the feedforward scenario, respectively, H(v) the transfer function (Fourier transform of the filter kernel h(t)) of the neuron population, and E x ½ the average across noise realizations. We use the power ratio as a measure of the relative fluctuation suppression caused by feedback. For low frequencies (v?0) and strong effective coupling D w wD~DKwD&1, the power ratio (5) decays as w w {2 (see Fig. 4 A): the suppression of population-rate fluctuations is promoted by strong negative feedback. In line with the observations in ''Results: Suppression of population-rate fluctuations in LIF networks'', this suppression is restricted to low frequencies; for high frequencies (v??, i.e. H(v)?0), the power ratio a(v) approaches 1. Note that the power ratio (5) is independent of the amplitude r r of the population averaged external input x(t). Therefore, even if we dropped the assumption of the external inputs x i (t) being uncorrelated, i.e. if E t x i (t)x j (tzt) Â Ã =0 for i=j, the power ratio (5) remained the same. For correlated external input, the power r r of the population average x(t) is different from r= ffiffiffiffi ffi N p . The suppression factor a(v), however, is not affected by this. Moreover, it is straightforward to show that the power ratio (5) is, in fact, independent of the shape of the external-noise spectrum C XX (v)~E x DX (v)D 2 Â Ã . The same result (5) is obtained for any type of external input (e.g. colored noise or oscillating inputs).
For low frequencies, the transfer function H(v) approaches unity (lim v?0 H(v)~1); the exact shape of the kernel h(t) becomes irrelevant. In particular, the cutoff frequency (or time constant) of a low-pass kernel has no effect on the zero-frequency power (integral correlation) and the zero-frequency power ratio a(0) (Fig. 4). Therefore, the suppression of low-frequency fluctuations does not critically depend on the exact choice of the neuron, synapse or input model. The same reasoning applies to synaptic delays: Replacing the kernel h(t) by a delayed kernel h(t{d) leads to an additional phase factor e {ivd in the transfer function H(v). For sufficiently small frequencies (long time scales), this factor can be neglected (lim v?0 e {ivd~1 ).
For networks with purely inhibitory feedback, the absolute power (3) of the population rate decreases monotonously with increasing coupling strength w w. As we will demonstrate in ''Results: Population-activity fluctuations in excitatoryinhibitory networks'' and ''Results: Population averaged correlations in cortical networks'', this is qualitatively different in networks with mixed excitatory and inhibitory coupling w w E~ w ww0 and w w I~{ g g w wv0, respectively: here, the fluctuations of the compound activity increase with w w. The power ratio a(v), however, still decreases with w w.

Population-activity fluctuations in excitatory-inhibitory networks
In the foregoing subsection, we have shown that negative feedback alone can efficiently suppress population-rate fluctuations and, hence, spike-train correlations. So far, it is unclear whether the same reasoning applies to networks with mixed excitatory and inhibitory coupling. To clarify this, we now consider a random network composed of a homogeneous excitatory and inhibitory subpopulation E and I of size N E~D ED and N I~D I D~cN E , respectively. Each neuron receives K~EN E excitatory and cK~EN I inhibitory inputs from E and I with synaptic weights ww0 and {gwv0, respectively. In addition, the neurons are driven by external Gaussian white noise. As demonstrated in ''Methods: Linearized network model'', linearization and averaging across subpopulations leads to a two-dimensional system describing the linearized dynamics of the subpopulation averaged activity r(t)~(r E (t),r I (t)) T . Here, x(t)~(x E (t),x I (t)) T denotes the subpopulation averaged external uncorrelated white-noise input with correlation functions E x,t x p (t),x q (tzt) Â Ã r r 2 p d pq d(t) ( r r p~r = ffiffiffiffiffiffi N p p , p,q[fE,Ig), and h(t) a normalized linear kernel with Ð ? 0 dt h(t)~1. The excitatory and inhibitory subpopulations are coupled through an effective connectivity matrix with effective weight w w~Kww0 and balance parameter g g~cgw0.
The two-dimensional system (6)/(7) represents a recurrent system with both positive and negative feedback connections (Fig. 5  A). By introducing new coordinates , we obtain an equivalent representation of (6)/(7), describing the dynamics of the sum and difference activity r z (t) and r { (t), respectively, i.e. the in-and anti-phase components of the excitatory and inhibitory subpopulations (see [48][49][50]). The new coupling matrix reveals that the sum mode r z (t) is subject to self-feedback (S 11~{ w z~ w w(1{ g g)) and receives feedforward input from the difference mode r { (t) (S 12~wFF~ w w(1z g g)). All remaining connections are absent (S 21~S22~0 ) in the new representation (8) (see Fig. 5 B). The correlation functions of the external noise in the new coordinates are given by . The feedforward coupling is positive (w FF w0): an excitation surplus (r { (t)w0) will excite all neurons in the network, an excitation deficit (r { (t)v0) will lead to global inhibition. In inhibition dominated regimes with g g~cgw1, the self-feedback of the sum activity r z (t) is effectively negative ({w z v0). The dynamics of the sum rate in inhibition-dominated excitatoryinhibitory networks is therefore qualitatively similar to the dynamics in purely inhibitory networks (''Results: Suppression of population-activity fluctuations by negative feedback''). As shown below, the negative feedback loop exposed by the transform (8) leads to an efficient relative suppression of population-rate fluctuations (if compared to the feedforward case).
Mathematically, the coordinate transform (8) corresponds to a Schur decomposition of the dynamics: Any recurrent system of type (6) (with arbitrary coupling matrix W) can be transformed to a system with a triangular coupling matrix (see, e.g., [50]). The resulting coupling between the different Schur modes can be ordered so that there are only connections from modes with lower index to modes with the same or larger index. In this sense, the resulting system has been termed 'feedforward' [50]. The original coupling matrix W is typically not normal, i.e. W T W=WW T . Its eigenvectors do not form an orthogonal basis. By performing a Gram-Schmidt orthonormalization of the eigenvectors, however, one can obtain a (normalized) orthogonal basis, a Schur basis. Our new coordinates (8) correspond to the amplitudes (the time evolution) of two orthogonal Schur modes.
The spectra C RERE (v), C RIRI (v), C RERI (v) and C RzRz (v) of the subpopulation averaged rates r E , r I and the sum mode r z , respectively, are derived in ''Methods: Population-activity spectra of the linear excitatory-inhibitory network''. In contrast to the purely inhibitory network (see ''Results: Suppression of population-activity fluctuations by negative feedback''), the population-rate fluctuations of the excitatory-inhibitory network increase monotonously with increasing coupling strength w w. For strong coupling, C RzRz (v) approaches lim w w??
from below with w FF =w z~( g gz1)=( g g{1). Close to the critical point ( g g^1), the rate fluctuations become very large; (11) diverges. Increasing the amount of inhibition by increasing g g, however, leads to a suppression of these fluctuations. In the limit g g??, C R z R z (v) and (11) approach the spectrum lim w w?0 C R z R zD HD 2 r 2 (1zc {1 )=(2N E ) of the unconnected network. For strong coupling ( w w&1), the ratio C R E R E (v)=C R I R I (v) approaches g g 2 : the fluctuations of the population averaged excitatory firing rate exceed those of the inhibitory population by a factor g g 2 (independently of H(v) and v).
Similarly to the strategy we followed in the previous subsections, we will now compare the population-rate fluctuations of the feedback system (6), or equivalently (9), to the case where the feedback channels are replaced by feedforward input with identical auto-statistics. A straight-forward implementation of this is illustrated in Fig. 5 C: Here, the excitatory and inhibitory feedback channels R E and R I are replaced by uncorrelated feedforward inputs Q E and Q I , respectively. The Schur representation of this scenario is depicted in Fig. 5 D. According to (6), the Fourier transforms of the response fluctuations of this system read , and using we can express the spectrum CR R zR R z (v) of the sum activity in the feedforward case in terms of the spectra C R E R E (v) and C R I R I (v) of the feedback system (see eq. (55)). For strong coupling ( w w&1), the zero-frequency component (H(0)~1) becomes Thus, for strong coupling, the zero-frequency power ratio reveals a relative suppression of the population-rate fluctuations in the feedback system which is proportional to 1= w w 2 (see Fig. 4 B; black dashed line). The power ratio a z (0) for arbitrary weights w w is depicted in Fig. 4 B (black dotted curve). For a network at the transition point g g~1, (14) equals 1=(2 w w 2 ). Increasing the level of inhibition by increasing g g leads to a decrease in the power ratio: in the limit g g??, (14) approaches 1=(8 w w 2 ) monotonously. Above, we suggested that the negative self-feedback of the sum mode R z , weighted by {w z (Fig. 5 B), is responsible for the fluctuation suppression in the recurrent excitatory-inhibitory system. Here, we test this by considering the case where this feedback loop is opened and replaced by uncorrelated feedforward input Q z , weighted by {w z , while the feedforward input from the difference mode R { , weighted by w FF , is left intact (see Fig. 5 D9). As before, we assume that the auto-statistics of Q z is identical to the auto-statistics of R z as obtained in the feedback case, i.e. C QzQz (v)~C RzRz (v). According to the Schur representation of the population dynamics (9)/(10), the Fourier transform of the sum mode of this modified system is given bỹ With CR R zR R z (v) given in (54) and C R z R z (v), we obtain the power ratio Its zero-frequency component a 0 z (0) is shown in Fig. 4 B (gray dotted curve). For strong coupling, the power ratio decays as 1=(2w 2 z ) (gray dashed line in Fig. 4 B). Thus, the (relative) power in the recurrent system is reduced by strengthening the negative self-feedback loop, i.e. by increasing w z . So far, we have presented results for the subpopulation averaged firing rates r E (t) and r I (t) and the sum mode r z (t). The spectrum of the compound rate i.e. the activity averaged across the entire population, reads In the feedforward scenario depicted in Fig. 5 C, the spectrum of the compound rateR R~H( w wQ E z w w g gQ I zX ) For strong coupling, the corresponding low-frequency power ratio a(0)~C RR (0)=CR RR R (0) (black solid curve in Fig. 4 B) exhibits qualitatively the same decrease ! w w {2 as the sum mode.
To summarize the results of this subsection: the population dynamics of a recurrent network with mixed excitatory and inhibitory coupling can be mapped to a two-dimensional system describing the dynamics of the sum and the difference of the excitatory and inhibitory subpopulation activities. This equivalent representation uncovers that, in inhibition dominated networks ( g gw1), the sum activity is subject to negative self-feedback. Thus, the dynamics of the sum activity in excitatory-inhibitory networks is qualitatively similar to the population dynamics of purely inhibitory networks (see ''Results: Suppression of population-activity fluctuations by negative feedback''). Indeed, the comparison of the compound power-spectra of the intact recurrent network and networks where the feedback channels are replaced by feedforward input reveals that the (effective) negative feedback in excitatory-inhibitory networks leads to an efficient suppression of population-rate fluctuations.

Population averaged correlations in cortical networks
The results presented in the previous subsections describe the fluctuations of the compound activity. Pairwise correlations ½ are outside the scope of such a description. In this subsection, we consider the same excitatory-inhibitory network as in ''Results: Population-activity fluctuations in excitatory-inhibitory networks'' and present a theory for the population averaged spike-train cross-correlations. In general, this is a hard problem. To understand the structure of cross-correlations, it is however sufficient to derive a relationship between the cross-and auto-covariances in the network, because the latter can, to good approximation, be understood in mean-field theory. The integral of the auto-covariance function of spiking LIF neurons can be calculated by Fokker-Planck formalism [12,31,51]. To determine the relation between the cross-covariance and the auto-covariance, we replace the spiking dynamics by a reduced linear model with covariances obeying, to linear order, the same relation. We present the full derivation in ''Methods: Linearized network model''. There, we first derive an approximate linear relation between the auto-and cross-covariance functions a(t) and c(t), respectively, of the LIF network. A direct solution of this equation is difficult. In the second step, we therefore show that there exists a linear stochastic system with activity u and correlations a u (t) and c u (t) fulfilling the same equation as the original LIF model. This reduced model can be solved in the frequency domain by standard Fourier methods. Its solution allows us, by construction, to determine the relation between the integral cross-covariances C(0) and the integral auto-covariances A(0) up to linear order.
As we are interested in the covariances averaged over many pairs of neurons, we average the resulting set of N linear selfconsistency equations (56) for the covariance matrix in the frequency domain C(v) over statistically identical pairs of neurons and many realizations of the random connectivity (see ''Methods: Population averaged correlations in the linear EI network''). This yields a four-dimensional linear system (76) describing the population averaged variances A E and A I of the excitatory and inhibitory subpopulations, and the covariances C E E and C I I for unconnected excitatory-excitatory and inhibitory-inhibitory neuron pairs, respectively (note that we use the terms ''variance'' and ''covariance'' to describe the integral of the auto-and cross-correlation function, respectively; in many other studies, they refer to the zero-lag correlation functions instead). The dependence of the variances and covariances on the coupling strength w w, obtained by numerically solving (76), is shown in Fig. 6. We observe that the variances A E and A I of excitatory and inhibitory neurons are barely distinguishable (Fig. 6 A). With the approximation A :~A E~AI , explicit expressions can be obtained for the covariances (thick dashed curves Fig. 6 E): The deviations from the full solutions (thin solid curves in Fig. 6 E), i.e. for A E =A I , are small. In the reduced model, both the external input and the spiking of individual neurons contribute to an effective noise. As the fluctuations in the reduced model depend linearly on the amplitude r of this noise, the variances A and covariances C pq (p,q[fE,Ig) can be expressed in units of the noise variance r 2 . Consequently, the correlation coefficients C pq =A are independent of r 2 (see Fig. 6). The analytical form (19) of the result shows that the correlations are smaller than expected given the amount of shared input a pair of neurons receives: The quantity C in shared in the first line is the contribution of shared input to the covariance. For strong coupling w w&1, the prefactor I causes a suppression of this contribution. Its structure is typical for a feedback system, similar to the solution (3) of the one-population or the solution (52) of the two-population model. The term w w(1{ g g) in the denominator represents the negative feedback of the compound rate. The prefactor II in the second line of (19) is again due to the feedback and suppresses the contribution of the factor III, which represents the effect of direct connections between neurons.
Our results are consistent with a previous study of the decorrelation mechanism: In [17], the authors considered how correlations scale with the size N of the network where the synaptic weights are chosen as J!1= ffiffiffiffi ffi N p . As a result, the covariance C in shared in (19) caused by shared input is independent of the network size, while the feedback w w(1{ g g)!EN(1{ g g) JzO(J 2 ) À Á scales-to leading order-as ffiffiffiffi ffi N p (see (45)). Consequently, the first line in (19) scales as 1=N. The same scaling holds for the second line in (19), explaining the decay of correlations as 1=N found in [17].
The first line in (19) is identical for any pair of neurons. The second line is positive for a pair of excitatory neurons and negative for a pair of inhibitory neurons. In other words, excitatory neurons are more correlated than inhibitory ones. Together with the third line in (19), this reveals a characteristic correlation structure: Figure 6. Dependence of population averaged correlations and population-rate fluctuations on the effective coupling w w~Kw in a linearized homogeneous network with excitatory-inhibitory coupling. A: Spike-train variances A E (black) and A I (gray) of excitatory and inhibitory neurons. B: Spike-train covariances C EE (black solid), C EI (dark gray solid) and C II (light gray solid) for excitatory-excitatory, excitatoryinhibitory and inhibitory-inhibitory neuron pairs in the recurrent network, respectively, and shared-input contribution C in shared (black dotted curve; 'feedforward case'). C: Decomposition of the total input covariance C in (light gray) into shared-input covariance C in shared (black) and weighted spike-train covariance C in corr (dark gray). Covariances in A, B and C are given in units of the noise variance r 2 . D: Input-correlation coefficient C in =A in in the recurrent network (black solid curve). In the feedforward case, the input-correlation coefficient is identical to the network connectivity E (horizontal dotted line). E: (dark gray) and C II =A I (solid light gray curve) for excitatory-excitatory, excitatoryinhibitory and inhibitory-inhibitory neuron pairs, respectively. Thick dashed curves represent approximate solutions assuming A E~AI . F: Lowfrequency (LF) power ratios a (black), a E (dark gray), a I (solid light gray) for the population rate r(t) and the excitatory and inhibitory subpopulation rates r E (t) and r I (t), respectively. The LF power ratio represents the ratio between the LF spectra in the recurrent network and for the case where the feedback channels are replaced by feedforward input with C EI~0 (cf. Fig. 5 C). Thick dashed curves in F show power ratios obtained by assuming that the autocorrelations are identical in the feedback and the feedforward scenario (see main text). Vertical dotted lines mark the stability limit of the linear model (see ''Methods: Linearized network model''). A-F: K~1000, E~0:1, c~1=4, g~6, g g~cg~3=2, N~K(1zc)=E~12500. doi:10.1371/journal.pcbi.1002596.g006 C EE wC EI wC II (Fig. 6 B,E). For strong coupling w w&1, the difference between the excitatory and inhibitory covariance is . The difference decreases as the level g g of inhibition is increased, i.e. the further the network is in the inhibition dominated regime, away from the critical point g g~1. To understand the suppression of shared-input correlations in recurrent excitatory-inhibitory networks, consider the correlation between the local inputs I k=l~½ Wr k=l of a pair of neurons k, l. The input-correlation coefficient can be expressed in terms of the averaged spike-train covariances: (see ''Methods: Population averaged correlations in the linear EI network'': The input covariance C in equals the average quantity C xy,B given in (67), the input variance A in is given by (63) as A x,B ). The term C in shared represents the contribution due to the spiketrain variances of the shared presynaptic neurons (see (19)). This contribution is always positive (provided the network architecture is consistent with Dale's law; see [15]). In a purely feedforward scenario with uncorrelated presynaptic sources, C in shared is the only contribution to the input covariance of postsynaptic neurons. The resulting response correlation for this feedforward case is much larger than in the feedback system ( Fig. 6 B, black dotted curve). The correlation coefficient between inputs to a pair of neurons in the feedforward case is identical to the network connectivity E (horizontal dotted curve in Fig. 6 D; see [15]). In an inhibition dominated recurrent network, spike-train correlations between pairs of different source neurons contribute the additional term C in corr , which is negative and of similar absolute value as the shared-input contribution C in shared . Thus, the two terms C in shared and C in corr partly cancel each other (see Fig. 6 C). In consequence, the resulting input correlation coefficient C in A in is smaller than E (see Fig. 6 D; here: E~0:1). The correlations in a purely inhibitory network can be obtained from (19) by replacing N E ?N, taking into account the negative sign of w in w w~{Kw and setting g~0 and c~0: For finite coupling strength w ww0, this expression is negative. The contributions of shared input and spike-train correlations to the input correlation are given by C in shared~ w w 2 A N w0 and C in corr~ w w 2 C, respectively (see (19) and (20)). Using (21), we can directly verify that C in corr v0, because pairwise correlations C are negative, leading to a partial cancellation C in shared zC in corr~ w w 2 the right hand side is smaller in magnitude by a factor of^1 w w 2 compared to each individual contribution. Hence, as in the network with excitation and inhibition, shared-input correlations are partly canceled by the contribution due to presynaptic pairwise spike-train correlations. In the feedforward scenario with zero presynaptic spike-train correlations, in contrast, the response correlations are determined by shared input alone and are therefore increased. The suppression of shared-input correlations in the feedback case is what we call 'decorrelation' in the current work. In purely inhibitory networks, this decorrelation is caused by weakly negative pairwise correlations (21). For sufficiently strong negative feedback, correlations are smaller in absolute value as compared to the feedforward case. The absolute value of these anti-correlations is bounded by A=N.
The similarity in the results obtained for purely inhibitory networks and excitatory-inhibitory networks demonstrates that the suppression of pairwise correlations and population-activity fluctuations is a generic phenomenon in systems with negative feedback. It does not rely on an internal balance between excitation and inhibition.
As discussed in ''Results: Suppression of population-rate fluctuations in LIF networks'', the suppression of correlations in the recurrent network is accompanied by a reduction of population-activity fluctuations. With the population averaged correlations (19), the power (1) of the population activity r(t) reads In ''Results: Population-activity fluctuations in excitatory-inhibitory networks'', we showed that the populationactivity fluctuations are amplified if the local input in the recurrent system is replaced by feedforward input from independent excitatory and inhibitory populations (see Fig. 5 C). This manipulation corresponds to a neglect of correlations C EI between excitatory and inhibitory neurons. All remaining correlations (A E , A I , C EE , C II ) are preserved. With the resulting response auto-and cross-correlationsÃ A andC C given by (84), the power (1) of the population activity becomes For large effective coupling w w, the power ratio a~C RR =CR RR R decays as 1= w w 2 (black curve in Fig. 6 F). Note that the power ratio a derived here is indistinguishable from the one we obtained in the framework of the population model in ''Results: Populationactivity fluctuations in excitatory-inhibitory networks'' (black solid curve in Fig. 4 B). Although the derivation of the macroscopic model in ''Results: Population-activity fluctuations in excitatory-inhibitory networks'' is different from the one leading to the population averaged correlations described here, the two models are consistent: They describe one and the same system and lead to identical power ratios.
The fluctuation suppression is not only observed at the level of the entire network, i.e. for the population activity r(t), but also for each individual subpopulation E and I , i.e. for the subpopulation averaged activities r E (t) and r I (t). The derivation of the corresponding power ratios a E and a I is analog to the one described above. As a result of the correlation structure C EE wC II in the feedback system (see Fig. 6 B), the power of the inhibitory population activity is smaller than the power of the excitatory population activity. In consequence, a E wa I (gray curves in Fig. 6 F).
In (22) and (23), the auto-correlations are scaled by 1=N, while the cross-correlations enter with a prefactor of order unity. For large N, one may therefore expect that the suppression of population-activity fluctuations is essentially mediated by pairwise correlations. In the recurrent system, however, the cross-correlations C xy (x,y[fE,Ig) are of order A=N (see Fig. 6 and (19)). It is therefore a priori not clear whether the fluctuation suppression is indeed dominated by pairwise correlations. In our framework, one can explicitly show that the auto-correlation is irrelevant: Replacing the auto-correlationÃ A in (23) by the average autocorrelation (N E A E zN I A I )=N of the intact feedback system has no visible effect on the resulting power ratio (dashed curves in Fig. 6 F). The difference in the spectra of the population activities C RR and CR RR R is therefore essentially caused by the crosscorrelations.
The absolute population-activity fluctuations in purely inhibitory and in excitatory-inhibitory networks show a qualitatively different dependence on the synaptic coupling w w, in agreement with the previous sections. In networks with excitation and inhibition, the correlation coefficient increases with increasing synaptic coupling (see Fig. 6 E). Hence, the population-activity fluctuations grow with increasing coupling strength. In purely inhibitory networks, in contrast, the pairwise spike-train correlation decreases monotonously with increasing magnitude of the coupling strength w w, see (21). In consequence, the population-activity fluctuations decrease. The underlying reason is that, in the inhibitory network, the power of the population activity is directly proportional to the covariance of the input currents, which is actively suppressed, as shown above. For excitatory-inhibitory networks, these two quantities are not proportional (compare (20) and (1)) due to the different synaptic weights appearing in the input covariance.
To compare our theory to simulations of spiking LIF networks, we need to determine the effect of a synaptic input on the response activity of the neuron model. To this end, we employ the Fokker-Planck theory of the LIF model (see ''Methods: Response kernel of the LIF model''). In this context, the steady state of the recurrent network is characterized by the mean m and the standard deviation s of the total synaptic input. Both m and s depend on the steady-state firing rate in the network. The steadystate firing rate can be determined in a self-consistent manner [12] as the fixed point of the firing rate approximation (42). The approximation predicts the firing rate to sufficient accuracy of about +1 s {1 (see Fig. 7 A). We then obtain an analytical expression of the low-frequency transfer which relates the fluctuation n j (t)~ n nzEd(t) of a synaptic input to neuron i to the fluctuation of neuron i's response firing rate to linear order, so that Ð ? 0 dn i (t) dt~Ew(J ij ). This relates the postsynaptic potential J ij in the LIF model to the effective linear coupling w ij~w (J ij ) in our linear theory. The functional relation w(J) can be derived in analytical form by linearization of (42) about the steady-state working point. Note that w(J) depends on m and s and, hence, on the steady-state firing rate in the network. The derivation outlined in ''Methods: Response kernel of the LIF model'' constitutes an extension of earlier work [21,33] to quadratic order in J. The results agree well with those obtained by direct simulation for a large range of synaptic amplitudes (see Fig. 8). Fig. 7 B compares the population averaged correlation coefficients C=A obtained from the linear reduced model, see (19), and simulations of LIF networks. Note that the absolute value of the noise amplitude r in the reduced model does not influence the correlation coefficient C=A, as both quantities C and A depend linearly on r 2 . Theory and simulation agree well for synaptic weights up to J&1 mV. For larger synaptic amplitudes, the approximation of the effective linear transfer for a single neuron obtained from the Fokker-Planck theory deviates from its actual value (see Fig. 8 B). Fig. 7 C shows that the cancellation of the input covariance in the LIF network is well explained by the theory.
Previous work [17] suggested that positive correlations between excitatory and inhibitory inputs lead to a negative component in the input correlation which, in turn, suppresses shared-input correlations. The mere existence of positive correlations between excitatory and inhibitory inputs is however not sufficient. To explain the effect, it is necessary to take the particular correlation structure C EE wC EI wC II into account. To illustrate this, consider the case where the correlation structure is destroyed by replacing all pairwise correlations in the input spike-train ensemble by the overall population average C~(N E C EE zN I C II )=(N E zN I )w0 (homogenization of correlations). The resulting response correlations (upper gray curve in Fig. 7 B) are derived in ''Methods: Population averaged correlations in the linear EI network'', eq. (86). In simulations of LIF networks, we study the effect of homogenized spike-train correlations by first recording the activity of the intact recurrent network, randomly reassigning the neuron type (E or I) to each recorded spike train, and feeding this activity into a second population of neurons. Compared to the intact recurrent network, the response correlations are significantly larger (Fig. 7 B). The contribution of homogenized spike-train correlations to the input covariance C in (see (20)) is given by C in corr,hom~ w w 2 (1{ g g) 2 C §0. For positive spike-train correlations Cw0, this contribution is greater or equal zero (zero for g g~1). Hence, it cannot compensate the (positive) shared-input contribution C in shared (see Fig. 7 C). In consequence, input correlations, output correlations and, in turn, populationrate fluctuations (Fig. 7 D) cannot be suppressed by homogeneous positive correlations in the input spike-train ensemble. Canceling of shared-input correlations requires either negative spike-train correlations (as in purely inhibitory networks) or a heterogeneity in correlations across different pairs of neurons (e.g. C EE wC EI wC II ).

Effect of feedback manipulations
In the previous subsections, we quantified the suppression of population-rate fluctuations in recurrent networks by comparing the activity in the intact recurrent system (feedback scenario) to the case where the feedback is replaced by feedforward input with some predefined statistics (feedforward scenario). We particularly studied the effect of neglecting the auto-statistics of the compound feedback, (the structure of) correlations within the feedback ensemble and/or correlations between the feedback and the external input. In all cases, we observed a significant amplification of population-activity fluctuations in the feedforward scenario. In this subsection, we further investigate the role of different types of feedback manipulations by means of simulations of LIF networks with excitatory-inhibitory coupling. To this end, we record the spiking activity of the recurrent network (feedback case), apply different types of manipulations to this activity (described in detail below) and feed this modified activity into a second population of identical (unconnected) neurons (feedforward case). As before, the connectivity structure (in-degrees, shared-input structure, synaptic weights) is exactly identical in the feedback and the feedforward case.
In ''Methods: Linearized network model'', we show that the low-frequency fluctuations of the population rate s(t) of the spiking model are captured by the reduced model r(t) presented in the previous subsections. To verify that the theory based on excitatory and inhibitory population rates is indeed sufficient to explain the decorrelation mechanism, we first consider the case where the sender identities of the presynaptic spike train are randomly shuffled. Fig. 9 A shows the power-spectrum of the population activity recorded in the original network (FB) as well as the spectra obtained after shuffling spike-train identities within the excitatory and inhibitory subpopulations separately (Shuff2D), or across the entire network (Shuff1D). As shuffling of neuron identities does not change the population rates, all three compound spectra are identical. Fig. 9 B shows the response power-spectra of the neuron population receiving the shuffled spike trains. Shuffling within the subpopulations (Shuff2D) preserves the population-specific fluctuations and average correlations. The effect on the response fluctuations is negligible (compare black and light gray curves in Fig. 9 B). In particular, the power of low-frequency fluctuations remains unchanged (Fig. 9 C). This result confirms that population models which take excitatory and inhibitory activity separately into account are sufficient to explain the observations. Shuffling of spike-train identities across subpopulations (Shuff1D), in contrast, causes an increase in the population fluctuations by about one order of magnitude ( Fig. 9 B,C; dark gray). This outcome is in agreement with the result obtained by homogenizing pairwise correlations (see Fig. 7) and demonstrates that the excitatory and inhibitory subpopulation rates have to be conserved to explain the observed fluctuation suppression.
The shuffling experiments and the results of the linear model in the previous subsections suggest that the precise temporal structure of the population averaged activities within homogeneous subpopu-lations is essential for the suppression of population-rate fluctuations. Preserving the exact structure of individual spike trains is not required. This is confirmed by simulation experiments where new sender identities were randomly reassigned for each individual presynaptic spike (rather than for each spike train; data not shown). This operation destroys the structure of individual spike trains but preserves the compound activities. The results are similar to those reported here.
So far, it is unclear how sensitive the fluctuation-suppression mechanism is to perturbations of the temporal structure of the population rates. To address this question, we replaced the excitatory and inhibitory spike trains in the feedback ensemble by independent realizations of inhomogeneous Poisson processes (PoissI) with intensities given by the measured excitatory and inhibitory population rates s E (t) and s I (t) of the recurrent network, respectively. Note that the compound rates of a single realization of this new spike-train ensemble are similar but not identical to the original population rates s E (t), s I (t) (in each time window ½tzDt), the resulting spike count is a random number drawn from a Poisson distribution with mean and variance proportional to s E (t) and s I (t), respectively). Although the compound spectrum of the resulting local input is barely distinguishable from the compound spectrum of the intact recurrent system (Fig. 9 D; black and dark gray curves), the response spectra are very different: replacing the feedback ensemble by inhomogeneous Poisson processes leads to a substantial amplification of lowfrequency fluctuations (Fig. 9 E; compare black and dark gray curves). The effect is as strong as if the temporal structure of the population rates was completely ignored, i.e. if the feedback channels were replaced by realizations of homogeneous Poisson processes with constant rates (PoissH; light gray curves in Fig. 9 D,E). This result indicates that the precise temporal structure of the population rates is essential and that even small deviations can significantly weaken the fluctuation-suppression mechanism. The  Table 1 and Table 2. doi:10.1371/journal.pcbi.1002596.g008 ; logarithmic scaling). Note that in A, the compound-input spectra (FB, Shuff1D, Shuff2D) are identical. In D, the input spectra for the intact recurrent network (FB) and the inhomogeneous-Poisson case (PoissI) are barely distinguishable. See Table 1 and  Table 2 for details on the network model and parameters. Simulation time T~100 s. Single-trial spectra smoothed by moving average (frame size 1 Hz). doi:10.1371/journal.pcbi.1002596.g009 results of the Poisson experiments can be understood by considering the effect of the additional noise caused by the stochastic realization of individual spikes. Considering the auto-correlation, a Poisson spike-train ensemble with rate profile n(t) is equivalent to a sum of the rate profile and a noise term resulting from the stochastic (Poissonian) realization of spikes, q(t)~n(t)1z ffiffiffiffi ffi n 0 p z(t). Here, z(t) denotes a Gaussian white noise with auto-correlation E t z(t)z(tzs) ½ 1d(s) and n 0~Et n(t) ½ the mean firing rate. The response fluctuations of the population driven by the rate modulated Poisson activity are, to linear approximation, given bỹ R R~H(WQzX). Inserting Q, we obtain an additional noise term n 0 DHD 2 WW T in the spectrum CR RR R~R RR R Ã which explains the increase in power compared to the spectrum C RR of the recurrent network. As a generalization of the Poisson model, one may replace the noise amplitude ffiffiffiffi ffi n 0 p by some arbitrary prefactor g. In simulation experiments, we observed a gradual amplification of the population-rate fluctuations with increasing noise amplitude g (data not shown).

Discussion
We have shown that negative feedback in recurrent neural networks actively suppresses low-frequency fluctuations of the population activity and pairwise correlations. This mechanism allows neurons to fire more independently than expected given the amount of shared presynaptic input. We demonstrated that manipulations of the feedback statistics, e.g. replacing feedback by uncorrelated feedforward input, can lead to a significant amplification of response correlations and population-rate fluctuations.
The suppression of correlations and population-rate fluctuations by feedback can be observed in networks with both purely inhibitory and mixed excitatory-inhibitory coupling. In purely inhibitory networks, the effect can be understood by studying the role of the effective negative feedback experienced by the compound activity. In networks of excitatory and inhibitory neurons, a change of coordinates, technically a Schur decomposition, exposes the underlying feedback structure: the sum of the excitatory and inhibitory activity couples negatively to itself if the network is in an inhibition dominated regime (which is required for its stability; see, e.g., [12). This negative feedback suppresses fluctuations in a similar way as in purely inhibitory networks. The fluctuation suppression becomes more efficient the further the network is brought into the inhibition dominated regime, away from the critical point of equal recurrent excitation and inhibition ( g g~1). Having identified negative feedback as the underlying cause of small fluctuations and correlations, we can rule out previous explanations based on a balance between (correlated) excitation and inhibition [17]. We presented a self-consistent theory for the average pairwise spike-train correlations which illuminates that the suppression of population-rate fluctuations and the suppression of pairwise correlations are two expressions of the same effect: as the single spike-train auto-covariance is the same in the feedforward and the feedback case, the suppression of population-rate fluctuations implies smaller correlations. Our theory enables us to identify the cancellation of input correlations as a hallmark of small spike-train correlations.
In previous studies, shared presynaptic input has often been considered a main source of correlation in recurrent networks (e.g. [15,52]). Recently [17], suspected that correlations between excitatory and inhibitory neurons and the fast tracking of external input by the excitatory and the inhibitory population are responsible for an active decorrelation. We have demonstrated here that the mere fact that excitatory and inhibitory neurons are correlated is not sufficient to suppress shared-input correlations. Rather, we find that the spike-train correlation structure in networks of excitatory and inhibitory networks arranges such that their overall contribution to the covariance between the summed inputs to a pair of neurons becomes negative, canceling partly the effect of shared inputs. This cancellation becomes more precise the stronger the negative compound feedback Kw(1{cg) is. In homogeneous networks where excitatory and inhibitory neurons receive statistically identical input, the particular structure of correlations is C EE wC EI wC II . It can further be shown that this structure of correlations is preserved in the limit of large networks N?? (K=N~const:). For non-homogeneous synaptic connectivity, if the synaptic amplitudes depend on the type of the target neuron (i.e. J EE =J IE or J EI =J II ), the structure of correlations may be different. Still, the correlation structure arranges such that shared input correlation is effectively suppressed. Formally, this can be seen from a self-consistency equation similar to our equation (80).
The study by [17] has shown that correlations are suppressed in the limit of infinitely large networks of binary neurons receiving randomly drawn inputs from a common external population. Its argument rests on the insight that the population-activity fluctuations in a recurrent balanced network follow the fluctuations of the external common population. An elegant scaling consideration for infinitely large networks N?? with vanishing synaptic efficacy !1= ffiffiffiffi ffi N p shows that this fast tracking becomes perfect in the limit. This allows to determine the zero-lag pairwise correlations caused by the external input. The analysis methods and the recurrent networks presented here differ in several respects from these previous results: We study networks of a finite number of spiking model neurons. The neurons receive uncorrelated external input, so that correlations are due to the local recurrent connectivity among neurons, not due to tracking of the common external input [17]. Moreover, we consider homogeneous connectivity where synaptic weights depend only on the type of the presynaptic neuron (as, e.g., in [12]), resulting in a correlation structure C EE wC EI wC II . For such connectivity, networks of binary neurons with uncorrelated external input exhibit qualitatively the same correlation structure as reported here (results not shown).
In purely inhibitory networks, the decorrelation occurs in an analog manner as in excitatory-inhibitory networks. As only a single population of neurons is available here, population averaged spike-train correlations C II are negative. This negative contribution compensates the positive contribution of shared input.
The structure of integrated spike-train covariances in networks constitutes an experimentally testable prediction. Note, however, that the prediction (19) obtained in the current work rests on two simplifying assumptions: identical internal dynamics of excitatory and inhibitory neurons and homogeneous connectivity (i.e. J EE~JIE , J EI~JII ; see ''Results: Population-activity fluctuations in excitatory-inhibitory networks''). For such networks, the structure of correlations is given by C EE wC EI wC II . Further, the relation between subthreshold membrane-potential fluctuations and spike responses is the same for both neuron types. Consequently, the above correlation structure can be observed not only at the level of spike trains but also for membrane potentials, provided the assumptions hold true. A recent experimental study [53] reports neuron-type specific cross-correlation functions in the barrel cortex of behaving mice, both for spike trains and membrane potentials. It is however difficult to assess the integral correlations from the published data. A direct test of our predictions requires either a reanalysis of the data or a theory predicting the entire correlation functions. The raw (unnorma-lized) II and EI spike-train correlations in [53] are much more pronounced than the EE correlations ( Fig. 6 in [53]). This seems to be in contradiction to our results. Note, however, that the firing rates of excitatory and inhibitory neurons are very different in [53]. In our study, in contrast, the average firing rates of excitatory and inhibitory neurons are identical as a consequence of the assumed network homogeneity. Future theoretical work is needed to generalize our model to networks with heterogeneous firing rates and non-homogeneous connectivity. Recent results on the dependence of the correlation structure on the connectivity may prove useful in this endeavor [54][55][56].
Correlations in spike-train ensembles play a crucial role for the en-and decoding of information. A set of uncorrelated spike trains provides a rich dynamical basis which allows readout neurons to generate a variety of responses by tuning the strength and filter properties of their synapses [1] In the presence of correlations, the number of possible readout signals is limited. Moreover, spike-train correlations impair the precision of such readout signals in the presence of noise. Consider, for example, a linear combination y(t)~P N i~1 (s i Ã h i )(t) of N presynaptic spike trains with arbitrary (linear) filter kernels h i (t) (e.g. synaptic filters). In a realistic scenario, the individual spike trains s i (t) typically vary across trials [3,57]. To understand how robust the resulting readout signal y(t) is against this spike-train variability, let's consider the variability of its Fourier transform Y . Assuming homogeneous spiketrain statistics, the (squared) signal-to-noise ratio of the readout signal Y (v) is given by Here, E . . . ½ denotes the average across the ensemble of spike-train realizations, k(v)~C(v)=V (v) the spike-train coherence, and the coefficients H H 1 :~E i H i ½ and H H 2 :~E i DH i D 2 Â Ã the 1st-and 2ndorder filter statistics. For uncorrelated spike trains, i.e. k(v)~0, and S(v)=0, the signal-to-noise ratio SN 2 grows unbounded with the population size N. Thus, even for noisy spike trains (V w0), the compound signal y(t) can be highly reliable if the population size N is sufficiently large. In the presence of correlations, k(v)=0, however, SN 2 converges towards a constant value k {1 DSD 2 V {1 as N grows. Even for large populations, the readout signal remains prone to noise. These findings constitute a generalization of the results reported for population-rate coding, i.e. sums of unweighted spike counts (see, e.g., [2,3]). The above arguments illustrate that the same reasoning applies to coding schemes which are based on the spatio-temporal structure of spike patterns.
In a previous study [9], we demonstrated that active decorrelation in recurrent networks is a necessary prerequisite for a controlled propagation of synchronous volleys of spikes in embedded feedforward subnetworks ('synfire chains'; Fig. 10): A synfire chain receiving background input from a finite population of independent Poisson sources amplifies the resulting sharedinput correlations, thereby leading to spontaneous synchronization within the chain (Fig. 10 B). A distinction between these spurious synchronous events and those triggered by an external stimulus is impossible. The synfire chain loses its asynchronous ground state [58]. A synfire chain receiving background inputs from a recurrent network, in contrast, is much more robust. Here, shared-input correlations are actively suppressed by the recurrent-network dynamics. Synchronous events can be triggered by external stimuli in a controlled manner (Fig. 10 A). Apart from the spontaneous synchronization illustrated in Fig. 10, decorrelation by inhibition might solve another problem arising in embedded synfire structures: In the presence of feedback connections between the synfire chain and the embedding background network, synchronous spike volleys can excite (high-frequency) oscillatory modes in the background network which, in turn, interfere with the synfire dynamics and prevent a robust propagation of synchronous activity within the chain ('synfire explosion'; see [59,60]). The decorrelation mechanism we refer to in our work is efficient only at low frequencies. It cannot prevent the build-up of these oscillations. [61] demonstrated that the 'synfire explosion' can be suppressed by adding inhibitory neurons to each synfire layer ('shadow inhibition') which diffusely project to neurons in the embedding network, thereby weakening the impact of synfire activity on the embedding network.
In the present work we focus on the integral of the correlation function, nurtured by our interest in the low-frequency fluctuations. An analog treatment can however easily be performed for the zero-lag correlations. In contrast to infinite networks with sparse connectivity (N??, K~const), in the case of finite networks, pairs of neurons must be distinguished according to whether they are synaptically connected or not in order to arrive at a self-consistent theory for the averaged correlations. Providing explicit expressions for correlations between connected and unconnected neurons, the current work provides the tools to relate experimentally observed spiking correlations to the underlying synaptic connectivity.
The quantification of pairwise correlations is a necessary prerequisite to understand how correlation sensitive synaptic plasticity rules, like spike-timing dependent plasticity [62], interact with the recurrent network dynamics [63]. Existing theories quantifying correlations employ stochastic neuron models and are limited to purely excitatory networks [63][64][65]. Here, we provide an analytical equivalence relation between a reduced linear model and spiking integrate-and-fire neurons describing fluctuations correctly up to linear order. A formally similar approach has been employed earlier to study delayed cumulative inhibition in spiking networks [66]. We show that the correlations observed in recurrent networks in the asynchronous irregular regime are quantitatively captured for realistic synaptic coupling with postsynaptic potentials of up to about 1 mV. The success of this approach can be explained by the linearization of the neural threshold units by the afferent noise experienced in the asynchronous regime. For linear neural dynamics, the second-order description of fluctuations is closed [67]. We exploit this finding by applying perturbation theory to the Fokker-Planck description of the integrate-and-fire neuron to obtain the linear input-output transfer at low frequencies [33], thereby determining the effective coupling in our linear model.
The scope of the theory presented in the current work is limited mainly by three assumptions. The first is the use of a linear theory which exhibits an instability as soon as a single eigenvalue of the effective connectivity matrix assumes a positive real part. This ultimately happens when increasing the synaptic coupling strength, because the eigenvalues of the random connectivity matrix are located in a circle centered in the left half of the complex plain with a radius given by the square root of the variance of the matrix elements [68,69]. Nonlinearities, like those imposed by strictly positive firing rates, prevent such unbounded growth (or decay) by saturation. For nonlinear rate models with sigmoidal transfer functions it has been shown that the activity of recurrent random networks of such units makes a transition to chaos at the point where the linearized dynamics would loose stability [70]. However, this point of transition is sharp only in the limit of infinitely large networks. From the population averaged firing rate and the pairwise correlations averaged over pairs of neurons considered in Fig. 7 we cannot conclude whether or not a transition to chaos occurs in the spiking network. In simulations and in the linearized reduced model, we could however observe that the distribution of pairwise correlations broadens when approaching the point of instability. Future work needs to examine this question in detail, e.g. by considering measures related to the Lyapunov exponent. Recently developed semi-analytical theories accounting for nonlinear neural features [71] may be helpful to answer this question. The second limiting factor of the current theory is the use of a perturbative approach to quantify the response of the integrate-and-fire model. Although the steady-state firing rate of the network is found as the fixed point of the nonlinear self-consistency equation, the response to a synaptic fluctuation is determined up to linear order in the amplitude of the afferent rate fluctuation, which is only valid for sufficiently small fluctuations. For larger input fluctuations, nonlinear contributions to the neural response can become more important [33]. Also for strong synaptic coupling, deviations from our theory are to be expected. Thirdly, the employment of Fokker-Planck theory to determine the steady-state firing rate and the response to incoming fluctuations assumes uncorrelated presynaptic firing with Poisson statistics and synaptic amplitudes which are vanishingly small compared to the distance between reset and threshold. For larger synaptic amplitudes, the Fokker-Planck theory becomes approximate and deviations are expected [33,34,72,73]. This can be observed in Fig. 7 A, showing a deviation between the self-consistent firing rate and the analytical prediction at about J^1 mV. In this work, we obtained a sufficiently precise selfconsistent approximation of the correlation coefficient C=A by relating the random recurrent network of spiking neurons in the asynchronous irregular state to a reduced linear model which obeys the same relation between C and A up to linear order. This reduced linear model, however, does not predict the absolute values of the variance A and covariance C. The variance A of the LIF model, for example, is dominated by nonlinear effects, such as the reset mechanism after each action potential. Previous work [12,31] has shown that the single spike-train statistics can be approximated in the diffusion approximation if the recurrent firing rate in the network is determined by mean-field theory. One may therefore extend our approach and determine the integral autocorrelation function as A~nF with the Fano factor F (see [51]). For a renewal process and long observation times, the Fano factor is given by F~CV 2 [74,75]. The coefficient of variation CV can be obtained from the diffusion approximation of the membranepotential dynamics (App. A.1 in [12). The covariance C can then be determined by (19). Another possibility is the use of a refractory-density approach [76,77].
The spike-train correlation as a function of the time lag is an experimentally accessible measure. Future theoretical work should therefore also focus on the temporal structure of correlations in recurrent networks, going beyond zero-lag correlations [15,17] and the integral measures studied in the current work. This would allow to compare the theoretical predictions to direct experimental observations in a more detailed manner. Moreover, the relative spike timing between pairs of neurons is a decisive property for Hebbian learning [78] in recurrent networks, as implemented by spike timing-dependent plasticity [62], and suspected to play a role for synapse formation and elimination [79].
The simulation experiments performed in this work revealed that the suppression of correlations is vulnerable to certain types of manipulations of the feedback loop. One particular biological source of additional variability in the feedback loop is probabilistic vesicle release at synapses [80]. In feedforward networks, such unreliable synaptic transmission has been shown to decrease the Figure 10. Recurrent network dynamics stabilizes dynamics of embedded synfire chains. Spiking activity in a synfire chain (L~10 layers, layer width b~50) receiving background input from an excitatory-inhibitory network (A, cf. Fig. 1 C) or from a finite pool of excitatory and inhibitory Poisson processes (B, cf. Fig. 1 D). Average input firing rates, in-degrees and amount of shared input are identical in both cases. Neurons of the first synfire layer (neuron ids 1,:::,b) are stimulated by current pulses at times t~500 and 1000 ms. Each neuron in layer k[½2,L receives inputs from all b neurons in the preceding layer k{1 (synaptic weights J sfc~0 :8 mV, spike transmission delays d sfc~2 ms), and K{b and cK excitatory and inhibitory background inputs, respectively, randomly drawn from the presynaptic populations. Neurons in the first layer k~1 receive K and cK excitatory and inhibitory background inputs, respectively. Note that there is no feedback from the synfire chain to the embedding network. See Table 2  transmission of correlations by pairs of neurons [22]. Stochastic synaptic release is very similar to the replacement of the population activity in the feedback branch by a rate modulated Poisson processes that conserves the population rate. In these simulations we observed an increase of correlations due to the additional noise caused by the stochastic Poisson realization. Future work should investigate more carefully which of the two opposing effects of probabilistic release on correlations dominates in recurrent networks.
The results of our study do not only shed light on the decorrelation of spiking activity in recurrent neural networks. They also demonstrate that a standard modeling approach in theoretical neuroscience is problematic: When studying the dynamics of a local neural network (e.g. a ''cortical column''), it is a common strategy to replace external inputs to this neural population P by spike-train ensembles with some predefined statistics, e.g. by stationary Poisson processes. Most neural systems, however, exhibit a high degree of recurrence. Nonlocal input to the population P, i.e. input from other brain areas, therefore has to be expected to be shaped by the activity within P. The omission of these feedback loops can lead to qualitatively wrong predictions of the population statistics. The analytical results for the correlation structure of recurrent networks presented in this study provide the means to a more realistic specification of such external activity.

LIF network model
In the present study, we consider two types of sparsely connected random networks: networks with purely inhibitory coupling (''I networks'') and networks with both excitatory and inhibitory interactions (''EI networks''). To illustrate the main findings of this study and to test the predictions of the linear model described in ''Methods: Linearized network model'', both architectures were implemented as networks of leaky integrate-and-fire (LIF) neurons. The model details and parameters are reported in Table 1 and Table 2, respectively. All network simulations were carried out with NEST (www.nest-initiative.org, [81]).

Linearized network model
In this section we show how the dynamics of the spiking network can be reduced to an effective linear model with fluctuations fulfilling, by construction, the same relationship as the original system up to linear order. We first outline the conceptual steps of this reduction, and then provide the formal derivation.
We make use of the observation that the effect of a single synaptic impulse on the output activity of a neuron is typically small. Writing the response spike train of a neuron as a functional of the history of all incoming impulses therefore allows us to perform a linearization with respect to each of the afferent spike trains. Formally, this corresponds to a Volterra expansion up to linear order, the generalization of a Taylor series to functionals. In ''Methods: Response kernel of the LIF model'', we perform this linearization explicitly for the example of the LIF model. This determines how the linear response kernel depends on the parameters of the LIF model. The linear dependence on the input leads to an approximate convolution equation (31) linearly connecting the auto-and the cross-correlation functions in the network. As this equation is complicated to solve directly, we introduce a reduced linear model (35) obeying the same convolution equation. The reduced linear model can be solved by standard Fourier methods and yields an explicit form for the covariance matrix in the frequency domain (37). The diagonal and off-diagonal elements of the N~N E zN I dimensional covariance matrix C(v) in (56) correspond to the power-spectra of individual neurons and the cross-spectra of individual neuron pairs, respectively. As, in this linear approximation, both the auto-and the cross-covariances are proportional to the variance of the driving noise, the resulting correlation coefficients are independent of the noise amplitude (see ''Methods: Population averaged correlations in the linear EI network''). As shown in ''Results: Suppression of population-activity fluctuations by negative feedback'' and ''Results: Populationactivity fluctuations in excitatory-inhibitory networks'', the suppression of fluctuations in recurrent networks is most pronounced at low frequencies. It is therefore sufficient to restrict the discussion to the zero-frequency limit v?0. Note that the zero-frequency variances and covariances correspond to the integrals of the auto-and cross-correlation functions in the time domain. In this limit, we may combine the two different sources of fluctuations caused by the spiking of the neurons and by external input to the network into a single source of white noise with variance r 2 (see (39)).
In general, the spiking activity s i (t) of neuron i at time t is determined by the entire history fs(t 0 )Dt 0 vtg of the activity of all neurons s~(s 1 , . . . ,s N ) in the network up to time t. Formally, this dependence can be expressed by a functional The subscript t in G i t indicates that t 0 vt (causality). In the following, we use the abbreviation G i t ½s:G i t ½s(t 0 ). The effect of a single synaptic input on the state of a neuron is typically small. We therefore approximate the influence of an incoming spike train on the activity of the target neuron up to linear order. The sensitivity of neuron i's activity to the input from neuron k can be expressed by the functional derivative of G i t with respect to input spike train s k : It represents the response of the functional to a single d-shaped perturbation in input channel k at time t 00 , normalized by the perturbation amplitude E. In (27), e k~( 0, . . . ,0,1,0, . . . ,0) denotes the unity vector with elements e k k~1 and e k i~0 for all i=k. By introducing the vector sk k (t)~(s 1 (t), . . . ,s k{1 (t),0,s kz1 (t), . . . ,s N (t)) of spike trains with the k-th component set to zero, G i t ½s can be approximated by Eq. (28) is a Volterra expansion up to linear order, the formal extension of a Taylor expansion of a function of N variables to a functional, truncated after the linear term. With the linearized dynamics (28), the pairwise spike-train cross-correlation function between two neurons i and j=i is given by Ss k (t 00 )s s j (t)T s k T s\s k dt 00 (Vtw0): Note that (29)   -Spike emission at times t i k with V i (t i k ) §h -Subthreshold dynamics: [87] with temporal resolution dt Initial membrane-potential distribution at t~0: uniform between 0 and h E Synapse independent of the actual realization of s k . A consistent approximation up to linear order is equivalent to the assumption that for all j the linear dependence of the functional on s j is completely contained in the respective derivative with respect to s j (28). Dependencies beyond linear order include higher-order derivatives and are neglected in this approximation. This is equivalent to neglecting the dependence of where we used the fact that Sf ½sk k T s\s k~S f ½sk k T s for any functional f that does not depend on s k . The average of the functional derivative has the intuitive meaning of a response kernel with respect to a d-shaped perturbation of input s k at time t 00 . Averaged over the realizations of the stationary network activity this response can only depend on the relative time tzt{t 00 . In a homogeneous random network, the input statistics (number of synaptic inputs and synaptic weights) and the parameters of the internal dynamics are identical for each cell, so that the temporal shape h(t) of the response kernel can be assumed to be the same for all neurons. The synaptic coupling strength from neuron k to neuron i determines the prefactor w ik : In this notation, the linear equation connecting the auto-correlations a k and the cross-correlations c ij takes the form Eq. (31) can be solved numerically or by means of Wiener-Hopf theory taking the symmetry c ij (t)~c ji ({t) into account [82]. Our aim is to find a simpler model which is equivalent to the LIF dynamics in the sense that it fulfills the same equation (31). Let's u(t) denote the vector of dynamic variables of this reduced model. Analog to the original model, we define the crosscorrelation for i=j and tw0 as The simplest functional L i t ½u consistent with equation (31) is linear in u. Since we require equivalence only with respect to the ensemble averaged quantities, i.e. c u ij (t)~c ij (t), the reduced activity and therefore L i t ½u can contain a stochastic element which would disappear after averaging. The linear functional with a pairwise uncorrelated, centralized white noise z i (t) (Sz i (tzt)z k (t)T z~r 2 z d ij d(t)) fulfills the requirement, since for tw0 and i=j This equation has the same form as (31), so both models, within the linear approximation, exhibit an identical relationship between the auto-and cross-covariances. The physical meaning of the noise z(t) is the variance caused by the spiking of the neurons. The autocorrelation function of a spike train of rate n has a d-peak of weight n. The reduced model (33) exhibits such a d-peak if we set r 2 z~n . A related approach has been pursued before (see Sec. 3.5 in [31]) to determine the auto-correlation of the population averaged firing rate. This similarity will be discussed in detail below.
So far, we considered a network without external drive, i.e. all spike trains s(t) originated from within the network. If the network is driven by external input, each neuron receives, in addition, synaptic input y i from neurons outside the network. We assume uncorrelated external drive Sy i (tzt)Ty j (t)T~r 2 y d ij d(t). In the reduced model, this input constitutes a separate source of noise: Here, (f Ã g)(t)~Ð t {? dt'f (t')g(t{t') denotes the convolution and h iy (t) the response kernel with respect to an external input. For simplicity, let's assume that the shape of these kernels is identical for all pairs of pre-and postsynaptic sources, i.e. h ix (t)~h(t). If we further absorb the synaptic amplitude of the external drive in the strength of the noise r y , the linearized dynamics (34) can be written in matrix notation with W~fw ij g. The reduced model (35) can be solved directly by means of Fourier transform: The full covariance matrix follows by averaging over the sources of noise Z and Y as The diagonal elements of C u represent the auto-covariances, the off-diagonal elements the cross-covariances. Both are proportional to the driving noise r 2 z zDH(v)D 2 r 2 y . This is consistent with (31) which is a linear relationship between the cross-and autocovariances.
For networks which can be decomposed into homogeneous subpopulations, the N dimensional system (35) can be further simplified by population averaging. Consider, for example, a homogeneous random network with purely inhibitory coupling. Assume that the neurons are randomly connected with probability E and coupling strength {wv0. The average number of in/ outputs per neuron (in/out-degree) is thus given by K~EN. By introducing the population averaged external input y(t)~E i y i (t) ½ , the averaged spiking noise z(t)~E i z i (t) ½ , and the effective coupling strength w w~Kw, the dynamics of the population averaged activity becomes Here we assumed that E i w ij Â Ã is independent of the presynaptic neuron j and can be replaced by {Ew~{ w w=N. Note that this replacement is exact for networks with homogeneous out-degree, i.e. if the number of outgoing connections is identical for each neuron j. For large random networks with binomially distributed out-degrees (e.g. Erdös-Rényi networks or random networks with constant in-degree), (38) serves as an approximation.
To relate our approach to the treatment of finite-size fluctuations in [31], consider the population-averaged dynamics (38) of a single population with mean firing rate n. We set r 2 z~n for all single neuron noises z i in order for the reduced model's auto-covariances to reproduce the d-peak of the spiking dynamics. In the population averaged dynamics, this leads to the variance of the noise z(t) given by This agrees with the variance of the population rate in [31]. Therefore, the dynamics of the population averaged quantity u in (38) agrees with the earlier definition of a population averaged firing rate for the spiking network [31]. In equation (38), two distinct sources of noise appear: The noise due to external uncorrelated activity y and the noise z which is required to obtain the d-peak of the auto-correlation functions of the reduced model. The qualitative results of ''Results: Suppression of population-activity fluctuations by negative feedback'' and ''Results: Population-activity fluctuations in excitatory-inhibitory networks'', however can be understood with an even simpler model. As we are mainly concerned with the low-frequency fluctuations, we only need a model with the same limit v?0. As we normalized the kernel so that H(0)~1 we can combine both sources of noise and require X (0):Y (0)zZ(0) in (36) in the zero frequency limit. Hence, in ''Results: Suppression of population-activity fluctuations by negative feedback'' and ''Results: Populationactivity fluctuations in excitatory-inhibitory networks'', we consider the model with a pairwise uncorrelated centralized white noise to explain the suppression of fluctuations at low frequencies.
As a second example, consider a random network composed of an excitatory and an inhibitory subpopulation E and I with population sizes N E~D ED and N I~D I D~cN E , respectively. Assume that each neuron receives excitatory and inhibitory inputs from E and I with coupling strengths ww0 and {gwv0, respectively, and probability E, such that the average excitatory and inhibitory in/outdegrees are given by K~EN and cK, respectively. The dynamics of the subpopulation averaged activities u(t)~(u E (t),u I (t)) T is given by (35) with subpopulation averaged noise y(t)~(y E (t),y I (t)) T and z(t)~(z E (t),z I (t)) T and effective coupling Here, w w~Kw denotes the effective coupling strength, g g~cg the effective balance parameter and y ½ the (sub)population averaged external and spiking sources of noise, respectively. Again, the reduction of the Ndimensional linear dynamics to the two-dimensional dynamics (40) is exact if the out-degrees are constant within each subpopulation. As before, both sources of noise can be combined into a single source of noise, if we are only interested in the low-frequency behavior of the model, leading to the dynamics (39) with the effective coupling (40).
The linear theory is only valid in the domain of its stability, which is determined by the eigenvalue spectrum of the effective coupling matrix W. For random coupling matrices, the eigenvalues are located within a circle with a radius equal to the square root of the variance of the matrix entries [69] ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffi Var w ij Â Ã q~w ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffi NE(1{E)(1zcg 2 ) p . Writing the effective dynamics for the exponential kernel as a differential equation t Lr Lt~( W{1)rzx(t), the eigenvalues of the right hand side matrix W{1 are confined to a circle centered at {1 in the complex plain with radius ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffi Var w ij Â Ã q . Given Var w ij Â Ã w1, eigenvalues might exist which have a positive real part, leading to unstable dynamics. This condition is indicated by the vertical dotted lines in Fig. 6 A-F and Fig. 7 B-D near J~2:8 mV. Beyond this line, the linear model predicts an explosive growth of fluctuations. In the LIF-network model, an unbounded growth is avoided by the nonlinearities of the single-neuron dynamics.

Response kernel of the LIF model
We now perform the formal linearization (30) for a network of N LIF neurons i~1, . . . ,N. A similar approach has been employed in previous studies to understand the population dynamics in these networks [12,31]. We consider the input P j J ij s j (t) received by neuron i from the local network, where s j denotes the spike train of the neuron j projecting to neuron i with synaptic weight J ij . Given the time dependent firing rate n j (t) of each afferent, and assuming small correlations and small synaptic weights, the total input to neuron i can be replaced by a Gaussian white noise with mean m i (t) and variance s 2 i (t), where j sums over all synaptic inputs. J ij [fJ,{gJg denotes the amplitude of the postsynaptic potential evoked by synapse j?i. t m is the membrane time constant of the model. In the stationary state, the firing rate of each afferent is well described by the constant time average n n j~Et n j Â Ã . The working point at which we perform the linearization of the neural response (30) is then given by analog equations as (41), resulting in a constant mean m m i~tm P j J ij n n j and variance s s 2 i~t m P j J 2 ij n n j . If the amplitude of each postsynaptic potential is small compared to the distance of the membrane potential to threshold, the dynamics of the LIF model can be approximated by a diffusion process, employing Fokker-Planck theory [83]. The stationary firing rate of the neuron is then given by [12,31,84] with the reset voltage V reset , the threshold voltage h and the refractory time t ref .
In homogeneous random networks, the stationary rate ( Fig. 7 A) is the same for all neurons. It is determined in a selfconsistent manner [12] as the fixed point of (42). The stationary mean m m i and variance s s 2 i are determined by the stationary rate. To determine the kernel (30) we need to consider how a d-shaped deflection in the input to this neuron at time point t 0 affects its output up to linear order in the amplitude of the fluctuation. In the stationary state, we may set t

0~0
. It is therefore sufficient to focus on the effect of a single fluctuation We therefore ask how the density of spikes per time n i (t)~SG i t ½sT s\s k of neuron i, averaged over different realizations of the remaining inputs to neuron i, changes in response to the fluctuation (43) of the presynaptic neuron k in the limit of vanishing amplitude a. This kernel w ik h (30) is identical to the impulse response of the neuron and can directly be measured in simulation by trial averaging over many responses to the given d-deflection (43) in the input (see Fig. 8 A). For the theory of low-frequency fluctuations, we only need the integral of the kernel, also known as the DC susceptibility, The second equality follows from the equivalence of the integral of the impulse response and the step response in linear approximation [21,33]. Following from [41], both mean and variance are perturbed as dm i~a t m J ik and ds 2 i~a t m J 2 ik in response to a step a in the afferent rate n j . Moreover, we used the chain rule ds i~1 2 s s i ds 2 i . The variation of the afferent firing rate hence co-modulates the mean and the variance and both modulations need to be taken into account to derive the neural response [31]. Although the finite amplitude of postsynaptic potentials has an effect on the response properties [33,34], the integral response is rather insensitive to the granularity of the noise [33]. We therefore employ the diffusion approximation to linearize the dynamics of the LIF neuron around its working point characterized by the mean m m i and the variance s s 2 i of the total synaptic input. In (44), we evaluate the partial derivatives of n n i with respect to m m i and s s 2 i using (42). First, observe that by chain rule Note that the modulation of m i results in a contribution to w ik that is linear in J ik , whereas the modulation of s i causes a quadratic dependence on J ik . This expression therefore presents an extension to the integral response presented in [21,85]. Fig. 8 B shows the comparison of the analytical expression (45) and direct simulation. The agreement is good over a large range of synaptic amplitudes J ik [½{4,4 mV in the case of constant background noise caused by small synaptic amplitudes (here 0:1 mV for excitation and {0:4 mV for inhibition). For background noise caused by stronger impulses, the deviations are expected to grow [33].
Population-activity spectra in the linear model: feedback vs. feedforward scenario The recurrent linear neural dynamics defined in the previous section is conveniently solved in the Fourier domain. The driving external Gaussian white noise X is mapped to the response R(v)~T(v)X(v) by means of the transfer matrix T(v). According to (39), it is given by . The covariance matrix in the frequency domain, the spectral matrix, thus reads where we used E x XX Ã ½ ~r 2 1 and the expectation operator E x ½ represents an average over noise realizations. To identify the effect of recurrence on the network dynamics, we replace the local feedback input by a feedforward input Q with spectral matrix C QQ . The resulting response firing rate is given bỹ R R~H(WQzX). Assuming that the feedforward input Q is uncorrelated to the external noise source X (C QX~0 ) yields a response spectrum Population-activity spectrum of the linear inhibitory network In the Fourier domain, the solution of the mean-field dynamics (38) using the spectrum of the noise E x X (v)X (v) Ã ½ r 2 . We compare this power-spectrum to the case where the feedback loop is opened, i.e. where the recurrent input is replaced by feedforward input with unchanged auto-statistics C QQ (v)C RR (v), but which is uncorrelated to the external input C QX (v)~0. The resulting power-spectrum is given by (47) as CR RR R~D HD 2 w w 2 C RR zr 2 À Á .

Population-activity spectra of the linear excitatoryinhibitory network
In a homogeneous random network of excitatory and inhibitory neurons, the population averaged activity (40) can be solved in the Schur basis (9) introduced in ''Results: Population-activity fluctuations in excitatory-inhibitory networks'' with w z~{ w w(1{ g g) and w FF~ w w(1z g g). The power of the population rate therefore is The fluctuations of the excitatory and the inhibitory population follow as So the power-spectra are Replacing the recurrent input of the sum activity R z by activity Q z with the same auto-statistics, but which is uncorrelated to the remaining input into R z (Fig. 5 D9) results in the fluctuations The power-spectrum of the sum activity therefore becomes If, alternatively, the excitatory and the inhibitory feedback terms R E and R I are replaced by uncorrelated feedforward input Q E and Q I with power-spectra C RERE and C RIRI (Fig. 5 C,D), the spectrum of the sum activity reads The limit (14) for inhibition dominated networks with g gw1 can be obtained from this and the former expressions by taking H(0)~1 and assuming strong coupling w w&1.

Population averaged correlations in the linear EI network
In this subsection, we derive a self-consistency equation for the covariances in a recurrent network. We start from (37) (we drop the superscript u of C u for brevity) multiply by 1{WH(v) from left and its transpose from right to obtain We assume a recurrent network of N E excitatory and N I inhibitory neurons, in which each neuron receives K excitatory inputs of weight w and cK inhibitory inputs of weight {gw drawn randomly from the presynaptic pool of neurons. To obtain a theory for the variances and covariances at zero frequency (with H(0)~1) we may abbreviate r 2 z (0)zDH(0)D 2 r 2 y (0) by r 2 (0). For a population averaged theory, we need to replace in (56) the variances A i of an individual neuron by the population average and replace the covariance C ij for a given pair of neurons (i,j) by the average over pairs that are statistically equivalent to (i,j). For a pair (i,j) of neurons we will show that the set of equivalent pairs depends on the current realization of the connectivity since unconnected pairs are not equivalent to connected ones. Therefore it is necessary to first average the covariance matrix over statistically equivalent neuron pairs given a fixed connectivity and to subsequently average over all possible realizations of the connectivity. The latter will be denoted as E W ½. For compactness of the notation, first we perform the averaging for the general case, where neuron i belongs to population x and neuron j to population y. We denote by X , Y the sets of neuron indices belonging to populations x and y, respectively. Subsequently replacing x and y by all possible combinations x,y[fE,Ig, we obtain the averaged self-consistency equations for the network. We denote the number of incoming connections to a neuron of type x from the population of neurons of type y as K xy and the strength of a synaptic coupling as w xy . Rewriting the self-consistency equation (56) explicitly with indices yields C ij~X k w ik C kj z X k w jk C ik { X k,l w ik C kl w jl zr 2 d ij : ð57Þ The last equation shows that for a connected pair (i,j) of neurons (w ij =0 or w ji =0) either of the first two sums contains a contribution w ij C jj or w ji C ii proportional to the variance of the projecting neuron. We therefore need to perform the averaging separately for connected and for unconnected pairs of neurons. We use the notation for the average covariance over pairs of neurons of types x,y[fE,Ig with a connection from neuron j[Y to neuron i[X , where N pairs,x/y is the number neuron pairs connected in this way. An arrow to the right, i?j, denotes a connection from neuron i to neuron j. Note that we use the same letter C for the population averaged covariances and for the covariances of individual pairs. The distinction can be made by the indices: i,j,k,l throughout indexes a single neuron, u,v,x,y,z identifies one of the populations fE,Ig. We denote the covariance averaged over unconnected pairs as We further use for the integrated variance averaged over all neurons of type x. Connected and the unconnected averaged covariances differ by the term proportional to the variance of the projecting neuron, as mentioned above C x/y~Cx y zw xy A y C x?y~Cx y zw yx A x : As a consequence, we can express all quantities in terms of the averaged variance (60) and the covariance averaged over unconnected pairs (59). We now proceed to average the integrated variance over population x. Since there are no self-connections in the network, we do not need to distinguish two cases here. Replacing C ii on the right hand side of (60), the first term of (57) contributes A From the second to the third step we used that the sum over k (l) yields non-zero contributions only if neuron k (l) connects to neuron i. This happens in K xE (K xI ) cases with the coupling weight w xE (w xI ). Therefore the covariance averaged over connected pairs appears on the right hand side. In the last line we used the relation (61) to express the connected covariance in terms of the variance and the covariance over unconnected pairs. The second term in (60) is identical because of the symmetry C ik~Cki . Up to here, the structure of the network only entered in terms of the in-degree of the neurons. The contribution of the third term follows from a similar calculation From the second to the third step we assumed that among the K xz K xw pairs of neurons k[Z,l[W projecting to neuron i, the fraction K wz N z has a connection k?l. These pairs contribute with the connected covariance. The connections in opposite direction contribute the other term of similar structure. We ignore multiple and reciprocal connections here, assuming the connection probability is low. We introduce the shorthand C xy for the covariance averaged over all neuron pairs including connected and unconnected pairs C xy~Cx y zw xy K xy N y A y zw yx This is the covariance which is observed on average when picking a pair of neurons of type x and y randomly. In this step, beyond the indegree, the structure of the network entered through the expected number of connections between two populations. Taken all three terms together, we arrive at where due to the absence of a direct connection between i and j, the term linear in the coupling and proportional to the variance is absent. From the symmetry C kl~Clk it follows that the second term corresponds to an exchange of x and y in the last expression. The third sum in (57) In summary, the contributions from (66) and (67) We now simplify the expressions by assuming that the in-degree of a neuron and the incoming synaptic amplitudes do not depend on the type of the neuron, i.e. that excitatory and inhibitory neurons receive statistically the same input. Formally this means that we need to replace K xy by K y , the number of incoming connections from population y and w xy by w y , the coupling strength of a projection from a neuron of type y. The covariance C x y then has two distinct contributions, C xy,sep that depends on the type of neurons x,y, and C com that does not. In particular C xy,B and C xy,corr do not depend on x,y and we omit their subscripts in the following. The variances fulfill The disjoint part C xy,sep determines the difference between the covariances for pairs of neurons of different type. Using the parameters K E~K , K I~c K, w E~w , w I~{ gw, the explicit form is Therefore, also the covariances in the network obey the relation i.e. the mixed covariance can be eliminated and is given by the arithmetic mean of the covariances between neurons of same type. In matrix representation with the vector Q~(A E ,A I ,C E E ,C I I ), the self-consistency equation is The self consistent covariance can then be obtained by solving the system of linear equations This self-consistency equation yields the argument, why the sharedinput correlation C in shared (19) cancels the contribution C in corr (20) due to spike-train correlations in the covariance to the input currents (see Fig. 6 C,D). Rewriting (80) in terms of these quantities results in If a self-consistent solution with small correlation DC x y D,DC xy DvE exists, the right hand side of (81) must be of the same order of magnitude. The right hand side of this equation has a prefactor Kw which typically is &1 (for the parameters in Fig. 6, Kw becomes larger than 1 for ww10 {3 ). The first term in the bracket is proportional to the contribution of shared input, the second term is due to correlations among pairs of different neurons. Each of these terms is of order E. Due to the prefactor Kw, however, the sum of the two terms needs to be of order E=(Kw) to fulfill the equation. Hence, the terms must have different signs to cause the mutual cancellation.
To illustrate how the correlation structure is affected by feedback, let us now consider the case where the feedback activity is perturbed (''feedforward scenario''). We start from (47) and, again, only consider the fluctuations at zero frequency, First, we consider a manipulation that preserves the singleneuron statistics A E , A I and the pairwise correlations C EE , C II within each subpopulation, but neglects correlations C EI between excitatory and inhibitory neurons. Formally, this corresponds to the block diagonal correlation matrix Here, we have replaced the individual entries of the correlation matrix by the corresponding subpopulation averaged correlations. The calculation of the response auto-and cross-correlationÃ A and C C is similar as for the expressions (63) and (67), with the difference that terms containing C EI are absent: As an alternative type of feedback manipulation, we assume that all correlations are equal, irrespective of the neuron type. To this end, we replace all spike correlations by the population average C~(N 2 E C EE zN 2 I C II z2N E N I C EI )=(N E zN I ) 2~( N E C EE z N I C II )=(N E zN I ). Thus, the covariance matrix reads C QQ ij~d ij Az(1{d ij )C: The calculation follows the one leading to the expressions (63) and (67) and results iñ A A~w 2 K(1zcg 2 )Az(wK) 2 (1{cg) 2 Czr 2