A review of the mean field theory of balanced networks.

To capture the notion that the net external synaptic input to neurons is strong, we define the small number, where K_EX = p_EX N_X is the average number of external synaptic projections received by each neuron in E from all neurons in X, p_EX is connection probability, and J_EX is the synaptic strength of each connection. Specifically, J_EX is the total postsynaptic current induced in a postsynaptic neuron in E by a single spike in a presynaptic neuron in X. Hence, 1/ϵ quantifies the synaptic current that would be induced in each neuron in E (on average) if every neuron in X spiked once simultaneously. For the simulations in Fig 1a–1e and in Fig 2, ϵ = 0.0027 mV. Using this convention, the mean synaptic input to each neuron in populations E and I from all sources can be written in vector form as (1) where I = [I_E I_I]^T (superscript T denotes the transpose) is the vector of mean synaptic input to neurons in each population and similarly for their mean rates, r = [r_E r_I]^T. The rescaled external synaptic input, X = [X_E X_I]^T, is given by where r_X is the average rate of neurons in population X and s/ϵ is the strength of the inward current induced by optogenetic stimulation (s = 0 when stimulation is off). The recurrent and feedforward mean-field connectivity matrices are given by (2) respectively where w_ab = K_ab J_ab/(K_EX J_EX) quantifies the relative number, K_ab = p_ab N_b, and strength, J_ab, of synaptic connections from population b to a. To achieve moderate firing rates when ϵ is small, local input, Wr, must be net-inhibitory and partially cancel the strong external excitation, X, in Eq (1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Imbalanced amplification and suppression under partial optogenetic stimulation.

Same as Fig. 2a-d except the inward current was only provided to 20% of the excitatory neurons, modeling ChR2-expressing pyramidal cells. a) Schematic. A population of recurrently connected excitatory (red) and inhibitory (blue) spiking neuron models receive synaptic input from an external population (X; green) of Poisson-spiking neurons. Optogenetic stimulation of excitatory neurons was modeled by an extra inward current to 20% of the excitatory population at 5s. b) Spike rasters from 10 randomly selected ChR2-expressing and 40 non-expressing excitatory (red) neurons and 50 inhibitory (blue) neurons from the recurrent network. c) Average firing rate of ChR2-expressing excitatory neurons from simulations (light solid) and from the corrected approximation (Eq (4); dashed). d) Mean synaptic currents to 200 randomly selected ChR2-expressing excitatory neurons from external inputs (X; green), from the local population (E + I; purple) and the total synaptic current (black). e) Same as c, but for non-expressing excitatory neurons (red) and inhibitory neurons (blue). f) Same as d, but for non-expressing excitatory postsynaptic neurons. g) Same as c and d, but averaged over all excitatory neurons (expressing and non-expressing). Currents are measured in units of the neurons’ rheobase (rheobase/C_m = 10.5 V/s). Firing rates predicted by Eq (3) are not shown in c and e because Eq (3) is not applicable to those cases.

https://doi.org/10.1371/journal.pcbi.1006048.g002

Balanced network theory [6, 7] takes this cancellation to its extreme by considering the limit of large number of neurons, N = N_E + N_I, while scaling connection strengths and probabilities in such a way that . Under this scaling, Eq (1) would seem to imply that mean synaptic currents diverge in the limit, but this divergence is avoided in balanced networks by a precise cancellation between external and recurrent synaptic input. To achieve this cancellation, firing rates must satisfy the mean-field balance equation, in the large N limit, so that [6–10] (3) Hence, balanced network theory provides a closed form, linear expression for firing rates in the large network limit. Generally speaking, the firing rate of a neuron depends nonlinearly on the mean and variance of its input current [22, 31, 32]. Notably, however, the derivation of the fixed point in Eq (3) did not require us to specify the exact form of this dependence. Instead, Eq (3) represents the unique fixed point firing rates for which synaptic currents remain bounded as N → ∞. More specifically, if Eq (3) is not satisfied as N → ∞ then ∥I∥ → ∞ (where ‖ ⋅ ‖ is the Euclidean norm). The existence of this fixed point does not guarantee that it is stable. Precise, general conditions on the accuracy of Eq (3) for spiking network models are not known and the investigation of such conditions is outside the scope of this study. However, the approximation tends to be accurate in the N → ∞ limit whenever all eigenvalues of W have negative real part, the solution in Eq (3) is strictly positive, and inhibitory synaptic kinetics are sufficiently fast [6–10, 32, 33]. Indeed, Eq (3) provides a reasonable, but imperfect approximation to firing rates in our spiking network simulation (Fig 1c, compare light and dark solid).

Balanced network theory has some critical limitations. Local cortical circuits are, of course, finite in size so the N → ∞ (equivalently ϵ → 0) limit may not be justified. Moreover, excitation and inhibition in cortex may not be as perfectly balanced and spike trains not as asynchronous as predicted by balanced network theory [13–19, 34]. More importantly, under many biologically relevant connectivity structures, precise cancellation cannot be realized so Eq (3) cannot even be applied [8–10]. We next review a simple, linear correction to Eq (4) that partially resolves these issues.

A linear correction to precise balance.

A correction to Eq (3) can be obtained by considering ϵ non-zero, but this requires making assumptions on the relationship between neurons’ input statistics and firing rates. A simple approximation is obtained by assuming that population-averaged firing rates, r, depend only on population-averaged mean inputs, I, yielding the fixed points problem r = f(I) = f([Wr + X]/ϵ) where f is the population-level f-I curve. When f is an increasing function over relevant ranges of I, this fixed point equation can be re-written as Hence, in strongly coupled networks (ϵ small), the shape of f-I curves has a small effect on steady-state firing rates under such an approximation. Indeed, in the ϵ → 0 limit, the f-I curve has no effect and firing rates are determined by Eq (3). This conclusion easily generalizes to the case where f also depends on the average temporal variance of neurons’ inputs.

A simple case of this approximation is obtained by using a rectified linear approximation, r = g[I]₊ where [⋅]₊ denotes the positive part. We fit such a function to the relationship between neurons’ mean synaptic inputs and firing rates from our spiking network simulation (Fig 1e). Assuming that the average firing rates of all populations are positive, this rectified linear approximation produces a linear rate model [35] with mean firing rates given by solving Wr + X = ϵ/g r to obtain (4) where The AdEx neuron model used in our simulations has a nonlinear f-I curve (Fig 1e; gray dots) and its firing rate depends on all statistics of its input, not just the mean [26, 36]. Nevertheless, the linear approximation in Eq (4) was accurate in predicting firing rates in our simulations (Fig 1c, dashed), outperforming the balanced network approximation from Eq (3). This can be explained by the fact that the balanced approximation in Eq (3) is already somewhat accurate and the linear approximation in Eq (4) corrects for some of the error introduced by imperfect balance, even though the true dependence of r on I is nonlinear.

To further investigate the relative accuracy of Eqs (3) and (4), we repeated the spiking network simulations from Fig 1a–1d while proportionally scaling the number of neurons (N_E, N_I, and N_X) in each population and scaling connection weights and probabilities in such a way that (see Methods). As predicted by balanced network theory, excitatory and inhibitory firing rates increased toward the limit in Eq (3) (Fig 1f, compare light and dark solid lines). The linear correction in Eq (4) tracks this increase in firing rates and is more accurate than the approximation in Eq (3), particularly for smaller N (Fig 1f, dashed). It is worth noting that, in applying Eq (4) to obtain the dashed curve in Fig 1f, we fixed the value of g to the one obtained from the simulation in Fig 1a–1e. Hence, a single estimate of the gain yields an accurate approximation even under different parameter values.

The predictive power of Eq (4) in these examples is, of course, limited by the fact that it was only applied after estimating the gain of the neurons using firing rates obtained in simulations. Moreover, highly nonlinear f-I curves could introduce additional error. However, the purpose of Eq (4) in this work is to provide a first-order approximation to and general understanding of firing rates in networks under which Eq (3) cannot be applied. For these purposes, Eq (4) is sufficient.

Firing rates are increased by stimulating fewer neurons.

The model of optogenetic stimulation considered in Fig 1 is somewhat inaccurate since optogenetic stimulation of excitatory neurons is often incomplete. For example, only a fraction of cortical pyramidal neurons express the channelrhodopsin 2 (ChR2) protein targeted in many optogenetic experiments [3, 24, 37, 38]. To more accurately model optogenetic stimulation, we modified the example above so the extra inward current was provided to a randomly chosen 20% of the excitatory neurons (Fig 2a), modeling ChR2-expressing pyramidal cells. This change produced surprising results. The ChR2-expressing neurons increased their firing rates by a larger amount than they did when all excitatory neurons received the current (Fig 2b and 2c; compare to Fig 1b and 1c). Hence, counterintuitively, stimulating fewer neurons actually amplifies the effects of stimulation on the targeted cells. In contrast, non-expressing excitatory neurons were suppressed during stimulation and inhibitory neurons increased their rates, but by a smaller amount than they did under complete stimulation (Fig 2e; compare to Fig 1c).

Similar effects were observed in experiments by Adesnik and Scanziani [3]. In that study, pyramidal neurons in layers (L) 2/3 of mouse somatosensory cortex (S1) were stimulated optogenetically, but only about 23% of the pyramidal neurons expressed ChR2. During stimulation, non-expressing L2/3 pyramidal neurons were suppressed and inhibitory synaptic currents increased, implying an increase in inhibitory neuron firing rates.

To understand these effects, we first extended the mean-field theory above to account for multiple subpopulations by defining to be the vector of average firing rates for the ChR2-expressing (exp), non-expressing (nexp) excitatory neurons and inhibitory (I) neurons. The vector of average input to the network is again given by Eq (1) where W_X = [w_EX w_EX w_IX]^T, and q = 0.2 represents the proportion of neurons that express ChR2.

Importantly, W is singular (i.e., not invertible), so classical balanced network theory fails for this example since Eq (3) cannot be evaluated. More specifically, it is impossible for I in Eq (1) to remain finite as ϵ → 0 since there is no vector, r, such that Wr = −X. Intuitively, this can be understood by noting that expressing and non-expressing excitatory neurons receive the same local input on average (Fig 2d and 2f, purple), since local connectivity is not specific to ChR2 expression, but they receive different external input during stimulation (Fig 2d and 2f, green). Therefore, local synaptic input cannot simultaneously cancel the external input to both sub-populations, so the precise cancellation required by classical balanced network theory cannot be achieved (Fig 2d and 2f, black). A similar mechanism has been used to explain a lack of cancellation between positive and negative correlations in balanced networks [12, 39].

Amplification in the nullspace: A general analysis.

We now give a general analysis of network responses when W is singular. The example of partial optogenetic stimulation is then considered as a special case. If W is a singular matrix then only vectors, X, that are in the column space of W admit solutions to Wr + X = 0. The column space of W is defined as the linear space of all vectors, u, such that u = Wr for some r. The column space of a matrix, W, is the orthogonal complement of the nullspace of W^T. We can therefore decompose where is the projection of X onto the nullspace of W^T and X₁ = proj_C(W) X is the projection onto the column space of W. Moreover, note that since Wr is in the column space of W. Therefore, the projection of the total input onto the nullspace of W^T is (5) Hence, the projection of the total synaptic input onto the nullspace of W^T is whenever X has an component in the nullspace of W^T. Note that, despite the 1/ϵ term in Eq (1), the total synaptic input, I, is when balance is realized due to cancellation (as in Fig 1d). Hence, the singularity of W introduces large, synaptic currents where they would not occur if W was non-singular. In other words, external input in the nullspace of W^T produces strong synaptic currents in the network. Importantly, this conclusion does not rely on any assumptions about neurons’ f-I curves or other properties. This result is a fundamental property of balanced networks or, more generally, networks receiving strong feedforward input.

To understand the implications of this result on firing rates in the network, however, we must specify an f-I curve. We again consider the linear rate approximation quantified by Eq (4). Importantly, unlike Eq (3) from classical balanced network theory, the approximation in Eq (4) is applicable to this example because it accounts for imperfect cancellation between local and external inputs. Specifically, the regularized matrix, ϵD − W, is invertible so Eq (4) can be evaluated even though Eq (3) cannot. The resulting firing rate solution from Eq (4) agrees well with spiking network simulations (Fig 2c and 2e). Hence, Eq (4) provides an accurate approximation to firing rates in networks to which classical balanced network theory is not applicable at all.

Eq (4) also provides a concise mathematical quantification of firing rates when W is singular. Namely, if X₀, then firing rates can be expanded as (6) where r₀ is in the nullspace of W and r₀, . To derive this result, first note that Eq (4) can be rewritten as (7) If X has components in the nullspace of W^T then we can project both sides of this equation onto this nullspace to obtain where we again used the fact that since Wr is in the column space of W. Since and D are assumed , this equation is only consistent when . We can therefore decompose r = (1/ϵ)r₀ + r₁ where r₀, . We next show that r₀ is in the nullspace of W. From Eq (7), we have Isolating the terms gives Wr₀ = 0 and therefore r₀ is in the nullspace of W. In summary, components of external input in the nullspace of W^T partially break balance to evoke amplified firing rates in the nullspace of W.

In the special case that W has a one-dimensional nullspace, a more precise characterization of r₀ is possible. Let v₀ be in the nullspace of W with ∥v₀∥ = 1. Note that W^T also has a one-dimensional nullspace (since W is a square matrix). Let v₂ be in the nullspace of W^T with ∥v₂∥ = 1. Since r₀ is in the nullspace of W, we can write r₀ = av₀ for some scalar, a. Now, dot product both sides of Eq (7) by v₂ to obtain where we have used that v₂ · Wr = 0 since v₂ is in the nullspace of W^T, which is orthogonal to Wr in the column space of W. Keeping only terms and making the substitution r₀ = av₀, we get so that (8) yielding a concise expression for the amplified component of firing rates when W has a one-dimensional nullspace. Note that v₀ is in the nullspace of W, so this result is consistent with the more general conclusions above.

Amplification in the nullspace under partial optogenetic stimulation.

For the specific example of partial optogenetic stimulation considered in Fig 2, the nullspace of W^T is spanned by and the projection of X onto the nullspace of W^T is X₀ = [s/2 − s/2 0]^T. The nullspace of W is spanned by . We therefore have r = (1/ϵ)r₀ + r₁ where Eq (8) gives Hence, ChR2-expressing neurons are amplified and non-expressing neurons are suppressed by optogenetic stimulation, as observed in simulations. A more precise result is given by expanding the full approximation from Eq (4) to obtain (9) Here, is a constant proportional to s, c = |w_IE/w_II| and r_off is the vector of firing rates in the balanced, ϵ → 0, limit when stimulation is off (s = 0). Specifically, r_off is the unique vector that satisfies Wr_off + W_XrX = 0, which is solvable even though W is singular because W_X r_X is in the column space of W, so balance can be maintained when s = 0.

The gs/ϵ term in Eq (9) quantifies the amplification and suppression observed in simulations: Non-expressing neurons are suppressed by stimulation since −q < 0 and the response of ChR2-expressing neurons is amplified since 1 − q > 0 and s/ϵ is large. Stimulating fewer excitatory neurons (q smaller), increases the rate of the stimulated neurons, as indicated by the 1 − q factor. The term shows why inhibitory neurons increase their rates by a smaller amount. In summary, the optogenetically induced suppression observed experimentally by Adesnik and Scanziani [3] is a generic feature of balanced or strongly coupled networks under partial stimulation.

Local imbalance with global balance explains intralaminar suppression and interlaminar facilitation.

Interestingly, despite the break of balance at the level of ChR2-expressing and non-expressing subpopulations, global balance is maintained in this example. This can be understood by repeating the mean-field analysis above without partitioning neurons into ChR2-expressing and non-expressing sub-populations, thereby quantifying the global average of firing rate of all excitatory neurons. In particular, the average synaptic input, I = [I_E I_I]^T, to excitatory and inhibitory neurons is given by Eq (1) where W and W_X are as in Eq (2), and to account for the fact that only a proportion q of the excitatory neurons receive the inward current from optogenetic stimulation. In this case, W is non-singular so the balanced solution in Eq (3) is applicable. Indeed, the average firing rates of all excitatory neurons in our spiking network simulation is close to the prediction from Eq (3) and even closer to the prediction from Eq (4) (Fig 2g; compare to Fig 1c). The average feedforward input to all excitatory neurons is canceled by net-inhibitory local input (Fig 2h; compare to Fig 1d). Hence, balance is maintained globally even though the network is imbalanced at the level of ChR2-expressing and non-expressing populations.

In the same study by Adesnik and Scanziani considered above [3], recordings were made in L5, which was not directly stimulated optogenetically, but receives synaptic input from L2/3. Interestingly, despite the fact that most excitatory neurons in L2/3 were suppressed during stimulation, firing rates in L5 increased.

To model these experiments, we interpreted the recurrent network from Fig 2 as a local neural population in L2/3, which sends synaptic projections to L5 (Fig 3a). We modeled a neural population in L5 identically to the L2/3 population, except its feedforward input came from excitatory neurons in the L2/3 network, instead of from Poisson-spiking neurons. As in experiments [3], L5 neurons increased their firing rates during stimulation (Fig 3b) and approximate balance was maintained (Fig 3c). This can be understood by noting that, in our model, L5 receives synaptic input sampled from all excitatory neurons in L2/3. Hence, the feedforward excitatory current to L5 neurons increases proportionally to the average excitatory firing rates in L2/3 during stimulation. As we showed above, this average rate increases (Fig 2e), despite the fact that most excitatory neurons in L2/3 are suppressed by stimulation. Hence, the combination of intralaminar suppression and interlaminar facilitation observed during optogenetic stimulation in experiments [3] results from the fact that the stimulated layer is locally imbalanced, but globally balanced during partial stimulation. These conclusions rely on an assumption that synaptic projections to L5 come from both ChR2-expressing and non-expressing excitatory neurons in L2/3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Interlaminar facilitation despite intralaminar suppression under optogenetic stimulation.

a) Multi-layer network schematic. L2/3 was identical to the recurrent network in Fig 2 and provided external excitatory input to L5, which had the same internal structure as the L2/3 model. b) Average firing rates of excitatory (red) and inhibitory (blue) neurons in L5 from simulations (light solid) and from the corrected approximation (Eq (4); dashed). c) Mean synaptic currents to 200 randomly selected excitatory neurons in L5 from external inputs (X; green), from the local population (E + I; purple) and the total synaptic current (black). Currents are measured in units of the neurons’ rheobase (rheobase/C_m = 10.5 V/s).

https://doi.org/10.1371/journal.pcbi.1006048.g003

Imbalanced amplification of weak stimuli.

Sufficiently small ϵ or large s would introduce negative rates in Eq (9), representing a regime in which non-expressing neurons cease spiking and the firing rate of ChR2-expressing neurons saturate at a high value. In this sense, firing rates do not truly have a component for ϵ very small. However, smaller values of ϵ allow weak stimuli (small s) to be strongly amplified. Strictly speaking, if one takes , then under the linear approximation in Eq (4), partial optogenetic stimulation would have an effect on the average firing rate of stimulated and unstimulated subpopulations, but an effect on globally averaged firing rates. In practical terms, this means that, in strongly coupled networks (ϵ small), partial optogenetic stimuli can have a moderate effect on the firing rates of stimulated neurons while having a negligible effect on the average firing rates of all excitatory neurons.

To demonstrate this idea, we repeated the simulations from Fig 2 in a network with four times as many neurons (N = 2 × 10⁴) where synaptic weights and probabilities were scaled so that (as in Fig 1f) and we reduced stimulus strength, s, as well. In this simulation, ChR2-expressing neurons’ firing rates nearly doubled (Fig 4a) and non-expressing neurons were noticeably suppressed (Fig 4b). However, the change in the average firing rate of all excitatory neurons was nearly imperceptible (Fig 4c) and similarly for the firing rates of inhibitory neurons (Fig 4b and 4c). As a result, the firing rates in a downstream layer were unnoticeably modulated during stimulation (Fig 4d; compare to Fig 3). This effect could mask the effects of optogenetic stimulation in recordings.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Imbalanced amplification of weak stimuli.

Same simulations as Figs 2 and 3 except with N increased by a factor of four, ϵ decreased by a factor of two, and a weaker stimulus. a) Schematic. A recurrent network (stimulated layer), which receives external input from Poisson-spiking neurons (green X) and from partial optogenetic stimulation, sends excitatory synaptic input to an identical network (downstream layer). b) Average firing rate of ChR2-expressing excitatory neurons in the stimulated layer from simulations (light solid) and from the corrected approximation (Eq (4); dashed). c) Same as b, but for non-expressing excitatory (red) and inhibitory (blue) neurons in the stimulated layer. d) Same as b, but averaged over all excitatory neurons in the stimulated layer. e) Same as d, but for the downstream layer. Mean firing rates from simulations of the stimulated layer changed from 5.8 Hz before stimulation to 10.0 Hz during stimulation for ChR2-expressing neurons, from 5.9 to 5.1 Hz for non-expressing excitatory neurons, from 5.9 to 6.1 Hz averaged over all excitatory neurons, and from 7.8 to 8.0 Hz for inhibitory neurons. Mean firing rates from simulations of the downstream layer changed from 7.2 Hz to 8.1 Hz for excitatory neurons and from 8.5 Hz to 9.5 Hz for inhibitory neurons.

https://doi.org/10.1371/journal.pcbi.1006048.g004

Imbalanced amplification with nearly singular connectivity matrices.

An apparent limitation of the results above is that they rely on the singularity of the connectivity matrix, W. Singularity is a fragile property of matrices that arises from structural symmetries. In the example above, singularity arises from our implicit assumption that local synaptic connectivity is independent of whether neurons express ChR2. Even a slight difference in connectivity to or from ChR2-expressing neurons would make W non-singular so that its nullspace would be empty, rendering Eq (6) vacuous. We now show that Eq (6) and the surrounding analysis naturally extends to connectivity matrices that are approximately singular, with similar overall conclusions.

A matrix, W, is singular if it has λ = 0 as an eigenvalue. A matrix can therefore be considered approximately singular if it has an eigenvalue with small magnitude. Specifically, let λ be an eigenvalue of W with |λ| ≪ 1. Note that λ is also an eigenvalue of W^T. Now let v be the associated eigenvector so that W^T v = λv and assume that ∥v∥ = 1 without loss of generality. Take the projection of each term in Eq (7) onto the subspace spanned by v to get Now note that proj_v[Wr] = λ proj_v r. Hence, If and proj_v[Dr] ∼ proj_v r then this implies Hence, where δ = |λ| + ϵ. This generalizes Eq (6) to the case where W is only approximately singular. In summary, the mechanism of imbalanced amplification is a general property of strongly coupled networks with singular or nearly singular connection matrices.

We next show that networks with connection probabilities that depend on continuous quantities like distance or tuning preference necessarily have singular or nearly singular connectivity kernels and are therefore naturally susceptible to the amplification and suppression mechanisms described above.

Amplification and suppression in simulations with spatially narrow stimuli.

An oriented stimulus localized in the animal’s visual field (Fig 5c) was modeled by imposing firing rate profiles in L4 that were peaked at the associated location in physical and tuning space, again with a Gaussian profile (Fig 5d and 5e). This produced external input to L2/3 that was similarly peaked, but was nearly perfectly canceled by net-inhibitory lateral input (Fig 5f and 5g). Excitatory and inhibitory firing rate profiles in L2/3 were also peaked at the associated location in physical and tuning space (Fig 5h and 5i), demonstrating that neurons in L2/3 were appropriately tuned to the stimulus.

A smaller visual stimulus was modeled by shrinking the spatial profile of firing rates in L4 while leaving the orientation-dependence of L4 rates unchanged (Fig 5j and 5k). As above, synaptic inputs and firing rate profiles were appropriately peaked in physical and orientation tuning space (Fig 5l–5o). However, the smaller stimulus produced a surprising change to firing rates in L2/3. Despite the fact that L2/3 neurons at all locations received less excitation from L4 (Fig 5l), peak firing rates in L2/3 increased and a surround suppression dynamic emerged (Fig 5n). Hence, a more localized external input produced an amplification and suppression dynamic similar to the one observed in our model of optogenetic stimulation (compare to Fig 2). On the other hand, responses in orientation tuning space were mostly unchanged by the smaller stimulus size (Fig 5m and 5o).

Mean-field theory of balance in two-dimensional spatial networks with orientation-tuning-specific connectivity.

The mean-field theory of balanced networks was previously extended to continuously indexed networks in one and two dimensions [8, 12, 43]. We now review a straightforward extension to two spatial dimensions and one orientation dimension. Eq (1) generalizes naturally to (10) where r(x, θ) = [r_E(x, θ) r_I(x, θ)]^T is the vector of mean firing rates of excitatory and inhibitory L2/3 neurons near spatial coordinates x = (x, y) with preferred orientation near θ, and similarly for the neurons’ external input, X(x, θ), and total input, I(x, θ). The external input is given by where r_X(x, θ) is the profile of firing rates in L4 The connectivity kernels, and , are convolution integral operators defined by and Here, w_ab(x, θ) is the mean-field connection strength between neurons separated by x in physical space and θ in orientation space (see Methods), and [w_ab * r_b](x, θ) denotes circular convolution with respect to x and θ, i.e., convolution with periodic boundaries. These convolution operators implement low-pass filters in orientation and physical space, capturing the effects of synaptic divergence and tuning-specific connection probabilities. Similar filters describe feedforward connectivity in artificial convolutional neural networks used for image recognition [44].

Taking ϵ → 0 in Eq (10) shows that that firing rates must satisfy (11) This is an analogue to Eq (7) for spatial networks. From here, one may be tempted to invert the integral operator to obtain a spatial analogue of Eq (3). However, integral operators are never invertible [45]. Specifically, since Eq (11) is an integral equation of the first kind, there necessarily exist external input profiles, X(x, θ), for which Eq (11) does not admit a solution so that the classical balanced state cannot be realized [8]. This implies that there always exist inputs that prevent a continuously indexed network from maintaining excitatory-inhibitory balance. To better understand why this is the case, we follow previous work [8, 12, 43, 46, 47] in transitioning to the spatial Fourier domain to rewrite Eq (11) as (12) Here, is a Fourier coefficient of r(x, θ) and similarly for where n = (n₁, n₂) is the two-dimensional spatial Fourier mode and k is the Fourier mode in tuning space. Importantly, the convolution operators above become ordinary matrices in the Fourier domain. Specifically, and where is a Fourier coefficient of w_a(x, θ). Note that going from Eq (11) to Eq (12) requires that is a convolution operator and that the boundaries of the network are treated periodically, i.e., the convolutions are circular.

Solving Eq (12) gives an analogue to Eq (3) for spatial networks in the Fourier domain, (13) This equation gives all Fourier coefficients, . However, this solution is only viable when the inverse transform exists, i.e., when the Fourier series of converges, which in turn requires that converges to zero faster than as n → 0 and k → 0. More specifically, in Eq (13) must be square-summable. Hence, balance can only be realized when recurrent connectivity, quantified by , has more power at high spatial frequencies than external input, . In other words, for balance to be realized, external input, X(x, θ), cannot have “sharper” spatial features than the recurrent connectivity kernels, w_ab(x, θ) for a, b = E, I.

The use of Fourier series to solve Eq (11) relies on the translation invariance of the integral operator, , but the existence of external input profiles for which Eq (11) does not admit a solution is a more general feature that is true for any integral operators with Hilbert-Schmidt kernels. Specifically, the convolutions, w_ab * r_b, that define and can be replaced by integrals of the form ∫ w_ab(u, v)r_b(v)dv and the result that there exist X(u) for which Eq (11) does not admit a solution is still true whenever the kernels, w_ab(u, v) are finite (no delta functions) and square-integrable [45].

Balance and imbalance in networks with Gaussian-shaped connectivity kernels.

A more intuitive understanding of when and why balance is broken is provided by considering the Gaussian-shaped connectivity profiles used in our spiking network simulations. This explanation applies equally to the spatial profile of firing rates and connectivity in physical and orientation space, so we do not distinguish between the two in this discussion. Similar calculations were performed previously for spatial networks [8], so we only review the results here and discuss some of their implications here. Details of the calculations are provided in S1 Text.

When L4 firing rates and all connectivity kernels are Gaussian-shaped, all firing rates and synaptic input profiles are also Gaussian-shaped in the balanced state (see [8] and S1 Text), so the analysis can be performed solely in terms of the widths of each of these profiles.

Let σ_a be the width of the firing rate profile in population a, α_a the width of outgoing synaptic connections from the presynaptic neurons in population a, and β_a the width of the spatial profile of synaptic input from population a = X, E, I (Fig 5a, 5d, 5f and 5h). Since L4 is the feedforward population in this context, we use X subscripts for L4 and E, I for L2/3 neurons.

Synaptic divergence broadens the profile of synaptic currents so that (14) for a = E, I, X. This is due to the Gaussian shape of firing rates and connectivity, as well as the convolution that describes the resulting mean-field synaptic inputs (the convolution of a Gaussian with a Gaussian is Gaussian). For balance to be maintained, feedforward synaptic input from L4 must be precisely canceled by lateral synaptic input in L2/3. This, in turn, requires that Combined with Eq (14), this implies that balance requires the widths of firing rate profiles in L2/3 to satisfy [8] (15) This approximation accurately predicted firing rate profiles in our first spiking network simulation (Fig 5h and 5i, solid, dark curves have widths given by Eq (15)). Hence, by Eq (15), the requirement of cancellation in balanced networks implies that recurrent connectivity sharpens neurons’ tuning, both in physical and orientation space.

Interestingly, Eq (15) implies that the amount by which excitatory and inhibitory firing rate profiles are sharpened in balanced networks is determined by the width of their outgoing synaptic projections. Pyramidal neurons in L2/3 of mouse V1 preferentially target similarly tuned neurons in L2/3, but the tuning of these lateral connection probabilities is much broader than the tuning of pyramidal neurons’ firing rates [41] (α_E > σ_E in orientation space). This observation is consistent with Eq (15): Excitatory neuron tuning curves are sharpened precisely because their outgoing connections are broadly tuned. Hence, sharpening of excitatory neuron tuning curves in L2/3 is naturally achieved in balanced networks with lateral excitation, without requiring lateral inhibition. Following the same line of reasoning, the broader orientation tuning of inhibitory neurons [40] (σ_I larger) suggests that they project more locally in orientation tuning space than pyramidal neurons (α_I < α_E in orientation space).

Eq (15) also clarify when and why balanced network theory fails for continuously indexed networks. If external inputs are sharper than lateral connectivity (β_X < α_E or β_X < α_I) in physical or orientation space, then Eq (15) do not yield solutions for σ_E or σ_I. In other words, balance requires that because Eq (11) does not admit a solution when these inequalities are broken (see [8] and S1 Text). Intuitively, when the inequalities above are broken, recurrent connectivity paints with too broad a brush to cancel the sharper feedforward input, so balance cannot be realized. As a result, balanced network theory cannot be applied to the example in Fig 5j–5o with a smaller visual stimulus.

A linear correction to balance quantifies amplification and suppression in continuously indexed networks.

We next derive a linear correction to Eq (13) that accounts for imperfect cancellation and, in doing so, gives firing rate approximations where classical balanced network theory fails. Specifically, we generalize the derivation of Eq (4) to continuously indexed networks. Under this linear approximation, firing rate profiles are given by solving (16) This is an integral equation of the second kind, which generically admits firing rate solutions, r, even when Eq (11) does not [45]. We again transition to the Fourier domain so Eq (16) becomes (17) From Eq (17), firing rates, r(x, θ), can be computed numerically through an inverse transform (the Fourier series over n and k), yielding an accurate approximation to firing rates from spiking network simulations even where classical balanced network theory fails (Fig 5n and 5o). Interestingly, the linear correction is slightly less accurate at predicting firing rates than the classical theory in Fig 5i. However, the main purpose of the corrected theory is not increased accuracy, but that it is applicable in situations where the classical theory is not.

The amplification and suppression caused by the smaller visual stimulus can be roughly explained by the balanced amplification mechanism discussed previously. Since is a low-pass filter, it approximately cancels high frequency components of firing rate profiles. Hence, high frequency components are in the approximate nullspace of the local connectivity operator, , and are therefore amplified by the network through the same mechanism discussed for discrete networks previously.

A more precise explanation is given by first averaging firing rates over orientation preference by setting k = 0 in Eq (17) to give that depends only on spatial frequency, and similarly for and . The convolution operator, , implements a low-pass filter, so is in magnitude at low spatial frequencies and converges to zero at higher frequencies (large ∥n∥). The regularized inverse, , is therefore in magnitude at low frequencies and at higher frequencies (Fig 6a, purple). When external input, X(x), has sharp features, has power at higher spatial frequencies (Fig 6a, green), which are amplified by the component of while low frequencies remain . The result is that the magnitude of has a peak at a non-zero spatial frequency (Fig 6a, black), introducing a high-amplitude, non-monotonic rate profile (as in Fig 5n; see [47] for a similar analysis). When X(x) has spatially broad features, has little power at high spatial frequencies so that this amplification dynamic is weak or absent (as in Fig 5h). An identical argument applies in orientation space. In summary, high-frequency components of external input profiles are transmitted more strongly than low-frequency components in strongly coupled networks, and the cutoff frequency is determined by the width (α_E or α_I) of lateral synaptic projections.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Spatial filtering of external input and the dependence of suppression on outgoing synaptic projection width.

a) The magnitude of the spatial filter, , imposed by recurrent connections (purple), the external input (, green) and the resulting firing rate profile (, black) as a function of the spatial frequency, , from the simulation in Fig 5j–5o. Magnitude is measured by the Frobenius norm for . Curves normalized by their peaks. b) Firing rates of excitatory (red) and inhibitory (blue) neurons with receptive fields at the center of a grating stimulus plotted as the width of the stimulus increases (represented by increasing σ_X) using parameters from Fig 5j–5o. c) Same as b, but the excitatory rate is plotted for different widths of the excitatory synaptic projection width, α_E. d) Same as c, but firing rates in L4 are shaped like a disc with radius σ_X instead of a Guassian with width parameter σ_X.

https://doi.org/10.1371/journal.pcbi.1006048.g006

It is worth noting that the average firing rates (over all orientations and spatial positions) are given by the zero Fourier coefficient . When balance is broken by sharp external input features, the zero Fourier mode is not affected as long as mean firing rates, r(x, θ), remain non-zero at all locations and orientations. Hence, sharp input features can break balance locally without breaking global, network-averaged, balance. This is analogous to the global balance obtained in the optogenetic example when local balance was broken at the level of subpopulations (Fig 2).

Implications of imbalanced amplification on receptive field tuning.

We next considered a study by Adesnik et al. [48] in which drifting grating stimuli of varying sizes were presented to mice while recording from neurons in L2/3 of V1. In that study, pyramidal neurons’ firing rates first increased then decreased as the stimulus size was increased. On the other hand, somatostatin-expressing (SOM) neuron’s firing rates increased monotonically with stimulus size. Intracellular recordings combined with optogenetic stimulation in that study showed that SOM neurons project locally and pyramidal neurons form longer range projections.

To test our model against these findings, we applied Eq (17) to a network with local inhibition and longer-range excitation (α_E > α_I) with increasing size of a visual stimulus (increasing σ_X). Our results are consistent with recordings in Adesnik et al., 2012 [48]: Excitatory neuron firing rates changed non-monotonically with stimulus size, while inhibitory neuron firing rates monotonically increased (Fig 6b). The non-monotonic dependence of excitatory firing rates on stimulus size in Fig 6b is explained by the mechanism of imbalanced amplification. When σ_X is sufficiently small, balance is broken so imbalanced amplification introduces a large peak firing rate surrounded by suppression (as in Fig 5n). However, the total amount of external excitation introduced by the stimulus is proportional to the size of the stimulus, so a very small σ_X introduces very little excitation and peak firing rates are small. As σ_X increases, more excitation is recruited and the network is still imbalanced, which leads to increasingly large peak firing rates (as in Fig 5n). Once σ_X becomes large, balance begins to be restored and the peak excitatory firing rate decreases to moderate values (as in Fig 5h).

The degree to which excitatory neurons suppress depends on the spatial width, α_E, of lateral excitatory projections (Fig 6c) and suppression of inhibitory neurons similarly depends on the spatial width, α_I, of lateral inhibition (not pictured). Specifically, suppression occurs when lateral connectivity is broader than feedforward input (α_E > β_X or α_I > β_X) because this is when the balanced solution in Eq (15) disappears. When a sub-population’s lateral connectivity is more localized than feedforward connectivity from L4 (α_E < α_X as in the lightest gray curve in Fig 6c; or α_I < α_X), that sub-population cannot exhibit suppression since feedforward input width () is always larger than lateral connectivity, regardless of the stimulus size (σ_X).

A similar line of reasoning explains why peak inhibitory neuron firing rates increase monotonically with stimulus size in Fig 6b. Inhibitory neurons in that example project locally (α_I = α_X), so the inequality α_I < β_X is always satisfied because whenever α_I ≤ α_X. Whenever α_I < β_X, inhibitory firing rates reflect their balanced state values which increase monotonically with the increase in total excitation induced by a larger stimulus.

Unlike SOM neurons, parvalbumin-expressing (PV) neurons were found to exhibit suppression by Adesnik et al., 2012 [48]. Hence, our theory predicts that PV neurons project more broadly in space than SOM neurons. Indeed, PV interneurons in L2/3 are primarily basket cells whose axons project to larger lateral distances than other inhibitory neuron subtypes such as Martinotti cells that comprise most SOM neurons [49].

We observed a unimodal dependence of firing rate on stimulus size (Fig 6c, all curves have a single peak). However, Rubin, et al. [47] observed a multi-modal, oscillatory dependence of firing rate on stimulus size in recordings and in a computational model. In that study, the drifting grating stimuli were disc-shaped with a sharp cutoff of contrast at the edges of the disc. Above, we considered a Gaussian-shaped contrast profile with soft edges (Fig 6c, inset). Repeating our calculations with a sharp-edged, disc-shaped stimulus (Fig 6d, inset) produced an oscillatory dependence of firing rate on stimulus size (Fig 6d), as observed by Rubin et al.. This oscillation only arose when lateral synaptic projections were narrower than the stimulus size (α_E small). The oscillation results from a Gibbs phenomenon: The sharp edge in the stimulus produces high-frequency power in , which passes through the high-pass filter when α_E is small.

We next explored the functional consequences of these results on receptive field tuning. We first considered a disc-shaped grating stimulus (Fig 7a), producing a disc-shaped firing rate profile in L4 (Fig 7b). Synaptic divergence causes the profile of synaptic input from L4 to L2/3 to be “blurred” at the edges (Fig 7c), as quantified by the low-pass filter, . This illustrates a fundamental problem in receptive field tuning: Synaptic divergence from one layer to another implements a low-pass filter that blurs sharp features. This problem is resolved by our observation above that lateral, recurrent connectivity implements a high-pass filter. If the width of lateral, excitatory connections in L2/3 is similar to that of feedforward connections from L4, the high-pass filter implemented by the recurrent network cancels the low-pass filter implemented by feedforward connectivity, effectively implementing a deconvolution that can recover the sharpness of firing rate profiles in L4 (Fig 7d–7f). Hence, counterintuitively, broader lateral excitation actually sharpens receptive field tuning. Broadening lateral connections further increases the sharpness of the firing rate profiles, but introduce oscillatory, Gibbs phenomena near sharp features (Fig 7f). These points are illustrated more clearly in an example with an asymmetrically shaped stimulus (Fig 7g–7l). Hence, the high-pass filter described above corrects the blurring caused by synaptic divergence between layers in V1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Imbalanced amplification and suppression reverse the blurring introduced by interlaminar synaptic divergence.

a) A disc-shaped grating stimulus gives rise to b) a disc-shaped firing rate profile, r_X(x), in L4 with slightly blurred edges (achieved by convolving contrast from a with a Gaussian kernel). c) Input, X_E(x), from L4 to excitatory neurons in L2/3 is blurred by synaptic divergence, which effectively applies a low-pass filter, , to the L4 rates. d) Excitatory firing rates in L2/3 are sharper than external input when lateral excitation is similar, but smaller, in width than interlaminar excitation (α_E = 0.85α_X). e) Same as c, but lateral excitation is exactly as broad as interlaminar excitation (α_E = α_X), which sharpens the edges further, making firing rates in L2/3 similar to those in L4. f) Same as c, but lateral excitation is broader than interlaminar excitation (α_E = 1.1α_X), which sharpens the edges even further, but also introduced suppressed regions due to Gibbs phenomena. g-l) Same as a-f, but contrast was determined by the brightness of a photograph. Horizontal and vertical axes are neurons’ receptive fields.

https://doi.org/10.1371/journal.pcbi.1006048.g007

In summary, imbalanced amplification and linear rate models provide a concise and parsimonious theoretical basis for understanding how suppression, amplification and tuning depends on the profile of neuron’s incoming and outgoing synaptic projections in physical and orientation tuning space.