The high input/high rate regime.

When the mean inputs are far above threshold (u_max ≪ − 1), neural firing is dominated by deterministic drift in the neuron membrane potential, and noise has only a weak effect on firing. In this regime, we use the first order term in the expansion (11) valid for large and negative u, to obtain a simplified expression of Eq (3), (12) Eq (12), is the transfer function of a single neuron receiving deterministic input, as expected from the fact that the mean input is far above threshold in units of the input noise. Higher order terms in expansion (11) provide corrections due to membrane fluctuations. Eq (12) shows that the relation between ν and ν_X in this regime is nonlinear. With the additional assumption |(u_max − u_min)/u_max| ≪ 1 (which is equivalent to θ − V_r ≪ μ − θ, a condition satisfied for μ sufficiently above threshold θ), we can expand Eq (12) and obtain a direct relation between ν and ν_X (13) where is the fraction of recurrent excitatory inputs that are needed to fire simultaneously to drive the membrane from reset to threshold, and ν_th = θ/KJτ is the external input needed to generate spikes in the absence of noise, respectively. Eq (13) shows explicitly that the nonlinearity in this regime is generated by the presence of the refractory period τ_rp, while the parameter ϵ controls the deviation from the linear prediction.

In Fig 2 first row, we compare the prediction of Eq (13) with the numerically computed solutions of Eqs (3)–(5) for various values of K. Numerical solutions of the mean-field equations show that our approximation gives a good description of the transfer function, capturing the nonlinearity observed close to saturation. As expected from Eq (13), the width of the non-linear region close to saturation expands as ϵ increases. For instance, for K = 1, 000, we have ϵ = 0.05, and the deviations from linear behavior are already significant when the firing rate is less than half its maximal value. On the other extreme, for the unrealistically high value K = 100, 000, ϵ = 0.0005 and the non-linear region becomes extremely small.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Types of nonlinearities in network response.

(A) Network transfer function obtained solving numerically Eqs (3)–(5) for different values of K (indicated on top of each column). (B) CV of interspike interval distribution obtained solving numerically Eq (6) (red lines). (C,D) Plots as in A, B but zoomed in the region of response onset. In all panels, dotted green lines correspond to the values at which u_max = 1 and u_max = − 1, i.e. they indicate the separation between the different operating regimes mentioned in the main text. Firing nonlinearities at response onset and saturation are captured by approximated forms (black dashed lines) obtained for u_max ≪ − 1 (Eq (13)) and u_max ≫ 1 (Eq (15)), respectively. For u_max ∼ 0, the transfer function approaches the balanced-state solution (gray dashed lines, Eq (17)); in the region − 1 < u_max < 1, the first order corrections are of order and become negligible as K increases. In the suprathreshold regime (u_max ≪ − 1), the CV approaches zero, i.e. firing becomes regular, with a decay with input strength captured by Eq 14 (dashed line). Parameters: J = 0.2 mV, g = 5.0, g_X = 1.

https://doi.org/10.1371/journal.pcbi.1008165.g002

Consistent with the idea that, for u_max ≪ − 1, firing is driven by deterministic drift, the CV of the ISI is found to be smaller than one in the high input regime (see Fig 2 second row); with a value that becomes smaller and smaller as ν_X increases. Using Eq (11), we derive a simplified expression for the CV starting from Eq (6) given by (14) This expression captures the decay of the CV observed in Fig 2, and shows that, as the neural firing rate ν approaches its maximum value, the CV goes to zero as 1 − τ_rpν.

Low input/low rate regime.

For membrane potential far below threshold (u_max ≫ 1), firing is driven by large stochastic fluctuations, so that noise can no longer be neglected. In this regime, in the simplified framework of model A, Eq (3) is well approximated by [4] (15) The transfer function obtained solving Eq (15) is nonlinear and provides a good description of the response for small rates. Specifically, the response shows threshold like nonlinearities, with threshold close to ν_th; we will characterize this in more detail in the following sections. Consistent with the fact that firing is produced by large fluctuations, the response is highly irregular (CV of order one or above) (see Fig 2 second and fourth row). Note that the CV depends non-monotonically on input rate, as shown in [36].

Intermediate linear region.

Up to now, we have shown that network response is expected to be nonlinear in the regions of rates corresponding to u_max ≪ − 1 and u_max ≫ 1; we will now show that in the region between these two regimes the response is expected to be approximately linear. For fixed values of ν_X and ν, we can write u_max = ω with ω constant of order one. In this regime, neural firing is driven by input fluctuations if ω > 0, or a combination of deterministic drift and fluctuations if ω < 0. The transfer function can be found iteratively for every value of ν and ν_X as (16) Eq (16) shows that, for ω = 0, the transfer function always matches the balanced-state solution [1, 4] (17) For ω of order one, deviation from this solution are expected to be of order . To test this, we plot in Fig 2 first row the balanced-state solution (gray dashed-lines): network responses are found to be close to this solution, with a distance that decreases as K increases. The balanced-state model [1] is characterized by a linear transfer function (given by Eq (17) with our notation). Consistent with the fact that this model has been derived in the strong coupling limit, we find that as K increases the network response converges to the balanced-state solution.

The above argument can be generalized for the more general case of model B. Using the approximations derived above, we can deduce that nonlinearities in the network response will appear any time that one or both populations have mean inputs far below threshold or above threshold. Moreover, an approximately linear solution appears when both mean inputs are close to threshold, with a scaling close to the one predicted by the balanced-state model up to corrections of order . In the following sections we systematically classify these nonlinearities as a function of the network parameters.

Model A.

We start our analysis of saturation nonlinearities by computing numerically responses in the mean field theory, i.e. solving Eqs (3)–(5), for different parameter values. The network response, for mean input μ above threshold, depends on g, K, and J; it depends only weakly on g_X since, in this regime, responses depend weakly on noise amplitude. Results of the numerical analysis are shown in Fig 3. Note that in this figure input firing rates are rescaled so that the balanced limit is the same regardless of the parameters. The transfer function shows a sublinear scaling as the network rate approaches saturation (Fig 3A–3C), but nonlinear effects are seen at low rates, especially for small K; the CV decreases monotonically for sufficiently large inputs (Fig 3D–3E). To understand the generality of these results, we turn to the approximated expressions of Eqs (13) and (14).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Saturation nonlinearities in Model A.

Transfer function (first row) and CV (second row) computed numerically from Eqs (3)–(5) and (6) (continuous lines) for different g (first column), C (second column) and J (third column). As in Fig 2, colored dotted lines in the first row represent values of the rates at which u_max = − 1. Black dashed lines solutions of the approximated Eqs (13) and (14). This validation of the approximated rate equation motivates the perturbative approach used in the main text and allows to classify nonlinearities in a general way. Simulation parameters: J = 0.2 mV, K = 10³ in (A,D); g = 5.0, J = 0.1mV in (B,E); g = 5.0, K = 10³ in (C,F); in all plots g_X = 1, τ = 20ms.

https://doi.org/10.1371/journal.pcbi.1008165.g003

As shown in Fig 3A–3C, solutions of Eq (13) capture the observed nonlinearities across input strength and parameter values. Surprisingly, they provide a good approximation of the network transfer function also for |u_max| ≈ 1, i.e. beyond the range of validity of Eq (13). These results suggest that Eq (13) can be used to understand nonlinearities in the whole range of activity levels above response onset.

To classify possible nonlinear responses, we solve Eq (13) using a perturbative expansion in ϵ, (18) The zeroth order solutions are found taking ϵ = 0 (strong coupling limit) in Eq (13); there are two such solutions: (19) The first solution corresponds to the balanced-state solution of Eq (17) while the second solution corresponds to saturated activity, with firing at maximum rate.

Using the balanced-state solution in the ϵ expansion, we get (20) In the inhibition dominated regime (gγ > 1), which is thought to underlie cortical dynamics [4], the first order correction is always negative and its absolute value becomes larger as the rate increases; this shows that the output rate increases sublinearly with input rate, regardless of the choice of parameters. Eq (20) also shows that the deviations from linear response increase with ϵ. At finite coupling (ϵ > 0), the first order correction is linear if τ_rpν₀ ≪ 1; nonlinear corrections become large when ϵ ≈ (γg − 1)(1 − τ_rp ν₀).

Using the saturation solution in the ϵ expansion, we get (21) This equation shows that the rate approaches saturation as 1/ν_X for all connectivity parameters. Note that, in the inhibition dominated regime, corrections with τ_rp(ν_X − ν_th) < (γg − 1) produce rates above saturation, which are not realizable. It follows that the saturation solution appears only when the balanced-solution is larger than 1/τ_rp, i.e. the two solutions found at ϵ = 0 are mutually exclusive at finite coupling; this will not be true in model B.

As in the previous section, the approximate CV expression of Eq (14) captures the decay as activity approaches saturation, ensuring that the suppression of irregular firing is expected for all parameter values. However, unlike what happens for the rates, the approximated expression departs from the mean field value as soon as u_max ≈ − 1 (see Fig 3 second row).

Model B.

In this section, we characterize saturation nonlinearities in model B. Using the perturbative method introduced for model A, we show that the network transfer function, at a fixed external drive, has in some cases multiple solutions. In cases in which there is a unique rate value, depending on the connectivity matrix and coupling strength, the response can be a sub-linear, linear or supra-linear function of the input.

As discussed in the methods section, model B features one excitatory and one inhibitory population with rates ν_E,I and external drive ν_EX, ν_IX = α_E,I ν_X. The transfer function of the network is obtained solving Eqs (3)–(5) which, in the limit in which the inputs to inhibitory neurons are much larger than threshold, is approximated by (22) where is the fraction of recurrent excitatory inputs that are needed to fire simultaneously to drive the membrane of an excitatory neuron from reset to threshold, is a parameter measuring the ratio of total recurrent excitatory synaptic strength onto excitatory and inhibitory neurons, (A = E,I) is the external firing rate needed to bring population A = E,I at firing threshold, in the absence of recurrent inputs and input fluctuations.

Admissible solutions of Eq (22), i.e. with rates in the range [0, 1/τ_rp], provide a good approximation of the network transfer function. The advantage of using Eq (22) is that it is polynomial in the rates and can be easily solved with an ϵ expansion of the form (23) Moreover, as discussed above, solutions describe, up to corrections of order , the network response for |u_max| ∼ 1.

We now investigate the structure of the network transfer function using the ϵ expansion. To simplify expressions, in what follow we omit contributions coming from ν_th,A. For ϵ = 0, there are two solutions of Eq (22) in which neither population is saturated. The first of these solutions, which will be called s1 or regular, has the first two terms of the expansion given by (24) This solution corresponds to the classic ‘balanced’ solution [1, 2]. In the strong coupling limit, regular solutions feature excitatory and inhibitory rate increasing linearly with input strength. For finite coupling, the network transfer function deviates from this linear scaling. In particular, the response of one population is supralinear (sublinear) if the first order correction is positive (negative). For instance, the excitatory rate is a supralinear function of inputs if (25) sublinear scaling appears if the above inequality is not satisfied. Analogous conditions holds for the inhibitory population. In agreement with this result, we find numerically that solutions of Eqs (3)–(5) which approach s1 solutions for ϵ = 0 transition from sublinear to supralinear as β increases (Fig 4A).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Saturation nonlinearities in model B.

(A-F) Numerical solutions of Eqs (3)–(5) for different parameters (colored lines) and linear approximations predicted by Eqs (24), (26) and (27) in the strong coupling limit (dotted lines). In each panel, the first (second) row shows the excitatory (inhibitory) firing rate as a function of ν_X. (A,B) Nonlinear solutions obtained at finite coupling starting from solution s1 and s2 for different β values. (C-F) All admissible cases of coexistence of multiple solutions at low ν_X; note that, as expected from our analysis, the number of solutions changes as ν_X increases. Simulation parameters are: (A) g_I = 3.9, g_E = 8, α_I = 1 α_E = 7; (B) g_I = 3.9, g_E = 8, α_I = 10, α_E = 7; (C) g_I = 4, g_E = 3, α_I = 5, α_E = 2; (D) g_I = 8, g_E = 6, α_I = 5, α_E = 2; (E) g_I = 1, g_E = 2, α_I = 0.3, α_E = 2; (F) g_I = 1, g_E = 2, α_I = 3, α_E = 2. In panels (A-F), g_EX = g_IX = 1; J_EE = J_IE = 0.2mV and K_EE = K_IE = 10³; except in A and B where , .

https://doi.org/10.1371/journal.pcbi.1008165.g004

The second unsaturated solution of Eq (22) obtained for ϵ = 0 will be called s2 or supersaturated and is given by (26) In the strong coupling limit, the inhibitory rate increases linearly with the external drive while the excitatory population remains silent due to overwhelming inhibition. Applying the same approach as the one used for s1 solutions, we find that s2 solutions at finite coupling admit only sublinear scaling (Fig 4B).

Up to now, we have assumed that only one of the two solution types appears for any given value of ν_X. In what follows we discuss under what conditions the assumption is valid and what happens if these conditions are not met. In this analysis, a key role is played by saturated solutions of Eq (22) which are of three types (27) In the strong coupling limit, in s3 solutions, both populations are saturated; in s4 and s5 solutions, only one population is saturated while the other changes linearly with inputs.

To analyze solution admissibility, for any given value of ν_X, we compute the rates predicted by the ϵ expansion up to first order and investigate for what parameters each solution is within the [0, 1/τ_rp] range. Conditions obtained for general values of ν_X are given in the Supporting Information (S1 Text). For simplicity, here we analyze results obtained for ν_X = 0; because of response continuity, these results are valid also in a range of sufficiently small external inputs. In this range of inputs, we find that there are seven possible scenarios, 3 with a single admissible solution (s1, s2 and s3), and 4 with three admissible solutions ({s1, s2, s3}, {s1, s2, s5}, {s1, s3, s4}, {s2-s3-s4}). When there are three solutions, we expect at least one of these solutions to be unstable; we will come back to this point during the analysis of network simulations and in the discussion.

We find that s1 solutions are the only one admissible if (28) analogously, s2 solutions are the only one that appear if (29) Note that the conditions in Eq (29) are analogous to those for supersaturating solutions in the SSN [24]. The first two inequalities ensure suppression of excitatory rate with increasing input strength; reabsorbing J_EA into α_A, they correspond to the conditions Ω_E < 0 and det J > 0 found in [24]. The last two inequalities prevent the existence of saturated solutions. Because of the presence of the refractory period in our model, the mathematical expressions are different in the SSN—the analogous condition in [24] being . Any time Eqs (28) or (29) are violated, multiple solutions appear. All the above-mentioned combinations of coexisting multiple solutions for ν_X ∼ 0 are shown explicitly in Fig 4C–4F. Note that the admissibility conditions derived in SI depends on the value of ν_X. Therefore, as ν_X increases, the number and the identity of solutions is expected to change; examples of this phenomena are given in Fig 4, where s1-s5 (Fig 4D) or s1-s4 (Fig 4E) solutions merge for ν_Xτ_rp ∼ 0.05 leaving only one admissible solution for larger inputs. Using the relations derived in SI, we also derive relations between solutions that must be satisfied for arbitrary values of ν_X. First, we find that solutions s3 and s5 are mutually exclusive and can never appear at the same time. Second, any time that solution s1 and s2 are admissible at the same time, s4 solutions are not admissible. These two results imply that, for any value of ν_X at most three saturation-generated solutions can coexist. Finally, for large ν_X, only s3 solutions are admissible, i.e. as the input intensity increases, both populations eventually saturate.

To summarize, we have shown that in model B at finite coupling, the network response can have one or multiple solutions. We have provided a general framework which, given the network parameters, predicts the expected number of solutions and their nonlinearities at finite coupling.

Model A.

To understand the possible nonlinearities appearing at response onset in model A, we rely on Eq (15), whose solution provide a good approximation of the network response for u_max ≫ 1. In this regime, we were not able to find a useful perturbative expansion and hence we will use a different approach, analyzing how the response evolves as a function of input strength ν_X.

For small enough ν_X, effects of recurrent interactions in the network are negligible, feedforward inputs dominate over recurrent inputs, and the transfer function is given by (30) As shown in Fig 5A, Eq (30) captures the response for small rates: response rises exponentially as ν_X approaches ν_th.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Response-onset nonlinearities in model A.

(A) Network response computed from Eqs (3)–(5) (red). (B) As in panel A but in log-log scale to highlight the behavior at low rate. Response at low rates is determined only by feedforward inputs (black, Eq (30)); effects of recurrent noise (blue) and mean (green) become relevant at higher rates, with opposite effects on the number of solutions. (C-F) Effects of different parameters on the number of solutions in network response; responses are shown in linear (top) and log-log (bottom) scale. Simulation parameters, unless otherwise specified in legend, are K = 10³, J = 0.5 mV, g = 5.0, g_X = 1, τ = 20ms.

https://doi.org/10.1371/journal.pcbi.1008165.g005

As ν_X increases, the network rate increases and recurrent inputs become relevant. In particular, recurrent inputs impacts network response as soon as their contributions to current mean or noise are of the same order as feedforward input. Specifically, using Eq (15) we find that this happens as soon as ν approaches the smaller of (ν_X − ν_th)/(1 − gγ) and ν_Xg_X/(1 + g² γ). At this point, the transfer function is found solving the implicit Eqs (3)–(5) (or its simplified form Eq (15)). We find numerically (see Fig 5B–5D) that, up to the point at which the membrane potential is close to threshold (u_max ∼ 1), there is either a single solution, or three solutions to the equations for the population firing rate. When a unique solution is present, firing increases supralinearly with inputs. For larger input values, the transfer function is determined by the linear response of the balanced-state regime.

As pointed out in [37, 38], multiple solutions emerge because of the positive feedback induced by the variance of the fluctuations in recurrent synaptic inputs and can be understood as follows. Let us consider the fictitious dynamics given by [38] (31) The fixed points of the above equation gives the network response at fixed input, i.e. the network transfer function; when multiple solutions are present, one solution must be unstable. This means there must be a solution for which the linearized dynamics (32) has λ > 0, with (33) Since the above equation is valid for u_max ≫ 1, multiple solutions can be generated only if the derivative of u_max with respect to ν is sufficiently large and negative. This derivative is given by (34) The above equation shows that, in an inhibition dominated network (gγ > 1), recurrent input mean and fluctuation have opposite effects; while the mean inputs provide negative feedback and hence cannot generate more than one solution, fluctuations provide positive feedback and therefore can potentially lead to multiple solutions.

This is verified in Fig 5A, where we compare network response computed from Eqs (3)–(5) with and without either recurrent input mean (μ_rc = ν(1 − gγ)) or noise (); it show that noise is the key factor to produce multiple solutions.

We characterize the number of solutions numerically, since the nonlinearities make an analytic approach unfeasible. First we note that, out of the 5 model parameters in Eq (15) (τ, K, g_X, J, and g) only 4 are relevant in determining the response structure, since τ fixes the overall scale of the rates. Results of the numerical investigation are shown in Fig 5B–5E. For all parameters, the low rate response rises exponentially up to the point at which recurrent inputs are no longer negligible. After this point, the network shows either supralinear increases or multiple solutions. Increasing J and K (Fig 5B–5C) decreases the response onset point, as expected by the definition of ν_th, and lead to multiple solutions. Changing g (Fig 5D) has little effect on response onset but it affects the slope of the balanced-solution. Finally, increasing g_X (Fig 5E) reduces the likelihood of having multiple solutions; this is due to the fact that recurrent input noise becomes negligible compared to the feedforward input noise.

Model B.

In this section we analyze the behavior of model B at response onset. As discussed above, at low input rates, responses are determined by feedforward inputs, and the contribution of recurrent interactions are negligible. At larger input rates, when at least one of the two population has u_max of order one, recurrent interactions are relevant, the network response is approximately linear and approaches one of the solutions derived in the previous section (e.g., when only one solution is present, regular or supersaturated). The response region connecting these two regimes is expected to be nonlinear; characterizing possible nonlinearities is the goal of this section. Unlike the case of model A, however, the large number of parameters makes an extensive exploration of possible behaviors unfeasible. Therefore, we focus our investigation on the role of coupling strength, which in model A was found to have a major role in determining the type of response nonlinearity.

Examples of excitatory and inhibitory activity for regular and supersaturated solutions are shown in Fig 6. We find that, in the region of inputs connecting response onset to the balanced solution, rates can either have a unique or multiple solutions. When only a unique solution is present, response increases supralinearly in the regular case (Fig 6A) and has a supralinear increase which eventually becomes sublinear in the supersaturated case (Fig 6B). In model B, there are two independent ways in which coupling strength can be modified to affect these responses: uniform change (e.g. increase K_EE with K_EE = K_IE) and relative change (e.g. fix K_EE and change K_IE).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Onset-nonlinearities in model B.

Responses obtained solving numerically Eqs (3)–(5) for regular (A) and supersaturated (B) solutions at different coupling strengths and for different β = K_EE J_EE/K_IE J_IE. The size of the nonlinear region decreases as the coupling strength increases and as the inhibitory activation threshold decreases. In B, the lack of purple curve in the first row means that the firing rates are very close to zero in the whole range of inputs. In the plots, response onset emerges around 0.1ν_th because of the value g_EX = g_IX = 10 used in simulations. As showed in Fig 5F, this choice enhances the amplitude of the nonlinear region by decreasing the likelihood of having multiple solutions. Other simulation parameters are:, J_EE = J_IE = 0.1mV, g_I = 3.9, g_E = 8. In (A) α_I = 1 α_E = 7; in (B) α_I = 10, α_E = 7.

https://doi.org/10.1371/journal.pcbi.1008165.g006

In regular solutions, a uniform increase in coupling strength reduces the region of nonlinear response, promoting a more sudden transition to linear scaling (Fig 6A, first vs second columns). The same modification in supersaturated solutions reduces the peak response of excitatory cells and increases the likelihood of having multiple solutions (Fig 6A, first vs second columns).

The main effect of changing the relative coupling is to move the relative onset point of the two populations, since it controls the ratio of ν_th,E and . For instance, increasing K_EE at fixed K_IE makes the excitatory population respond at lower input rates than the inhibitory population. For small modifications, this leads to a larger region of supralinear response (e.g. first column of Fig 6B light blue vs yellow lines). For larger modifications, it produces multiple solutions, since the excitatory population is unstable on its own (e.g. first column of Fig 6B red line). Finally, we note that similar effects could be produced modifying the relative onset point of the two populations through changes in other parameters, such as J_EE of J_IE.

In model B, there are two independent sources of positive feedback which can generate multiple solutions: excitatory-to-excitatory connectivity [39] and noise. Which of these sources generates the observed multiple responses? We already know that, when network structure reduces model B to model A, i.e. when excitatory and inhibitory populations have the same properties, multiple solutions are generated by noise feedback. Are noise-generated multiple solutions limited to this specific case or a more general feature of model B? To answer this question, inspired by model A, we computed numerically the network response with and without recurrent noise, for different values of network parameters; results are shown in Fig 7. At all coupling values, results along the line g_E = g_I are analogous to model A, as expected. At strong coupling, for a significant fraction of parameter space, the presence of multiple solutions is generated by noise, as they disappear when noise is removed. Interestingly, at weak coupling, noise seems to stabilize activity in a finite fraction of the parameter space; this phenomenon could be due to the larger increase in inhibitory response produce by its positive feedback coming from recurrent noise of inhibitory population.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Role of recurrent noise in generating multiple solutions at onset in model B.

Number of solutions for different combinations of g_E and g_I (red and blue correspond to single solution and multiple solutions, respectively) computed without (first column) and with (second column) recurrent noise; the third column shows transfer functions for specific combinations (indicated by triangles in the first two columns). Both for small (first row) and large (second row) K, noise influences the number of solutions. (A-C) For some parameters, noise reduces the size of the region with multiple solutions. (D-F) For others, noise increases the size of the region with multiple solutions. Parameters: J_IE = J_EE, g_IX = g_EX = 1, α_I = α_E = 1.

https://doi.org/10.1371/journal.pcbi.1008165.g007

These results show that, in model B, response onset is nonlinear and that a full characterization of response nonlinearities requires recurrent noise to be included in the model. This represents a qualitative difference with respect to rate models, where recurrent noise is not taken into account. In the next section we show how the results obtained so far compare with rate models.

Network simulations.

We perform numerical simulation in networks with either uniform or Erdös–Rényi connectivity. In the uniform case, all neurons receive exactly the same numbers of external, recurrent excitatory and recurrent inhibitory connections, i.e. there is a fixed in-degree for all types of connections. In the Erdös-Rényi (ER) case, the adjacency matrix specifying the existence of a synapse from any presynaptic to any (distinct) post-synaptic neuron is composed of i.i.d. Bernoulli variables, with connection probability given by ratio between mean number of connections and number of neurons in the presynaptic population. This leads to fluctuations in numbers of in-degrees between neurons which generates, in the balanced limit, a wide distribution of firing rates [1–3, 5]. The mean-field theory we have used so far assumes a fixed in-degree. Simulations of networks with uniform connectivity are useful to check that MF results are accurate, in spite of all the involved approximations (see Materials and Methods); simulations of ER networks allow us to assess the robustness of our results to heterogeneities.

Network simulations were performed with the simulator BRIAN2 [40] using networks of N_E = 11K_EE excitatory and N_I = 11K_EI inhibitory neurons, receiving excitatory inputs from ensembles of N_EX,IX = 11K_EX,IX independent Poisson units firing at rates ν_EX,EX, respectively. We used uniformly distributed delays of excitatory and inhibitory synapses. Delays were drawn randomly and independently at each existing synapse from uniform distributions in the range [0, 100]ms (E synapses) and [0, 1]ms (I synapses) [38]. These extreme values ensure no synchronized oscillations are present (see Discussion). For fixed network parameters, rates were computed starting from a given initial condition and gradually increasing the external drive. We explored different initial conditions to check for multiple solutions. Stationary responses obtained for different representative parameter sets are shown in Fig 8. For connectivity parameters leading to saturation nonlinearity, results of the network simulations closely match predictions of the mean field theory. In the case in which multiple solutions are expected from the mean field theory, either generated at saturation or at response-onset, simulations capture the upper and lower branches of the response; the absence of the middle branch suggests that it corresponds to unstable fixed points of the dynamics. Simulations obtained for parameters generating supersaturation in the mean field model are found to depend on the connectivity structure. For uniform connectivity, simulation follows the same trend of the mean field prediction, with inhibitory rate increasing monotonically while excitatory rate show an increase with inputs at low intensity and suppression at larger inputs. We note that, although this mean response follows the trend of the mean field prediction, the network showed oscillatory activity in the region of maximum excitatory response despite the broad distribution of synaptic delays. In the case of random connectivity, on the other hand, we found that the suppression of excitatory response is not present. This lack of supersaturation at the population level is generated by an heterogeneity in responses of excitatory cells, with 66% of cells which are silent while the remaining 33% show a weak increase in rate with the external drive. Despite this difference, a common features of supersaturating solutions in networks of spiking neurons, observed both in the mean filed theory and in simulations, is that the peak response of excitatory cells is small. The intuitive reason why firing rates of excitatory neurons are generically small in this scenario is that rates go to zero in the strong coupling limit.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Spiking-networks nonlinearities in network simulations and rate models.

Comparison of responses of excitatory (red) and inhibitory (blue) neurons computed with: mean field theory of spiking networks (colored lines in first column and black dashed lines in second to fourth columns); network simulation with uniform (second column) and random (third column) connectivity; rate models (Ricciardi model, fourth column, and the SSN, fifth column). Different rows correspond to different values of α_E,I and g_E,I. Prediction of the mean field theory match numerical simulations, with the only exception of the lack of supersaturation in networks with random connectivity. Spiking networks present nonlinearities at saturation and at response-onset. Saturation nonlinearities are generated by the refractory period and hence are captured only by the Ricciardi model and not by the SSN, which lack this ingredient. These nonlinearities have effects also at low rate and constrain the parameter space over which the response is unique. Response-onset nonlinearities have similar structure in the three models but in spiking networks are smaller in size and feature multiple solutions generated by noise feedback. Network structure (from top): g_E = 8, g_I = 7, α_E = 4, α_I = 2; g_E = 2.08, g_I = 1.67, α_E = α_I = 1; g_E = 4.5, g_I = 2.9, α_E = α_I = 1; g_E = 4.1, g_I = 2.46, α_E = 1, α_I = 0.2; g_E = 7, g_I = 6, α_E = 1, α_I = 0.7. In all simulations, for spiking-networks mean field, simulations and Ricciardi model: J_EE = J_IE = J; K_EX = K_EE = K_IE = K; g_EX = g_IX = 1. (except second row, where J_EE = J_IE 2.5/2.4 = J, and fourth row, where K_EX = K_EE = 2K_IE = 2K); values are 400 for K and 0.2mV for J in all simulations except for noise driven bistability where J = 0.5mV. In Ricciardi model, σ_E = σ_I = σ; σ matches noise at threshold in spiking neurons and is (from the top): 25mV, 10mV, 3mV, 5mV, 7mV. In the SSN, k = 0.04, n = 2, W_EE = W_IE = 1, W_AI = γg_A except in second row, where W_IE = 2.4/2.5 and W_EI = γg_I 2.4/2.5, and fourth row where W_EE = 2 and W_EI = 2γg_E.

https://doi.org/10.1371/journal.pcbi.1008165.g008

These numerical results validate the analysis derived above as a good description of the response in spiking networks, with the only exception given by the lack of supersaturation in networks with random connectivity. In what follows we compare how rate models compare to results obtained in spiking networks.

Rate models.

Rate models are characterized by a fixed relation between input current μ and firing rate response (here referred to as f − μ curve); this is not the case in spiking networks, where the firing rate also depends on the noise level which, in turns, depends on the input rate. Here we focus on two specific models: a ‘Ricciardi’ model, in which the single unit transfer function is given by the Ricciardi nonlinearity at fixed noise value, and the SSN [24], where the single unit transfer function is a power law with an exponent that is larger than one; a mathematical definition of these models is given in the Methods section. The comparison between different models is performed computing responses in networks of given input structure α_E,I and of recurrent inhibition g_E,I; this choice ensures the same balanced-solutions in all models and limits differences to the nonlinear response regions. In the Ricciardi-model, parameters are as in spiking network, the only exception is the noise level, which must be specified. We fix this to be the value observed in the spiking-network model at threshold. In the SSN, we fix parameters as in [24], i.e. we assume a powerlaw nonlinearity with exponent n = 2 and proportionality constant k = 0.04.

For networks producing saturation nonlinearities (Fig 8 first and second rows), the Ricciardi-model recapitulates quantitatively responses observed in spiking networks. As mentioned during our analysis of saturation nonlinearities, this is due to the fact that in spiking network the response in the balanced-regime and for larger rates depends weakly on the noise level. It follows that the agreement between spiking networks and the Ricciardi model is valid for all saturation nonlinearities discussed in model A and B. The SSN, on the the hand, shows substantial discrepancies. In particular, for network structure generating sublinear response in spiking network (Fig 8 first row), the SSN responds linearly; in the parameter conditions in which spiking network shows multiple solutions at zero inputs and maximum firing at large inputs (Fig 8 second row), the SSN shows supersaturation. These discrepancies are expected, since the SSN does not include a refractory period, i.e. the ingredient needed to generate saturation nonlinearity. While these nonlinearities can be captured by adding a refractory period to the model, our results show that consequences of the refractory period appear at rates that are much lower than 1/τ_rp. First, for moderate coupling, response is nonlinear for rates that are far from the maximum firing rate. Second, saturation nonlinearities limit the parameter space in which the network has a unique solution. In particular, the network structure used in Fig 8 second row is the one used for supersaturating solutions in [24, 25]; our results show that these parameters produce multiple solutions and saturation when the refractory period is taken into account (see Supporting Information for more details).

We next focus on response-onset nonlinearities. In spiking networks, rates at responses onset can either increase supralinearly or go through a region with multiple solutions; for larger inputs, rates approach the balanced solutions. Example of these transitions are shown in the last three rows of Fig 8. For network structure leading to supersaturating solution in spiking networks, the same qualitative behavior is observed in rate models but quantitative differences are observed. First, because of feedback coming from recurrent noise, the supralinear region at response onset is stiffer in spiking network. Indeed, the response in the Ricciardi-model, where the noise amplitude is fixed and there is no positive feedback coming from noise, is much less steep. A second major quantitative difference is that the peak excitatory rate is smaller in spiking network and Ricciardi model with respect to the SSN. More generally, our numerical analysis shows that response-onset nonlinearities occur in a smaller range of firing rates in spiking networks and in the Ricciardi model than in the SSN. In the Supporting Information (S3 Text), we show that this difference comes from the different shape of the f − μ curve in these models. For network structures leading to multiple solutions at response-onset, we should distinguish two cases: mean and noise generated multiple solutions. For some parameters, positive feedback due to mean recurrent input generates multiple solutions; this mechanism is also present in the Ricciardi model and in the SSN (Fig 8, fourth row). On the other hand, when multiple solutions are generated by noise feedback in spiking networks, there are no multiple solutions in rate models (Fig 8, fifth row); this is expected because in rate models there is no positive feedback generated by recurrent noise.