Mean-field approximations.

In order to derive a mesoscopic mean-field theory for populations of GIF neurons, we first approximate the conditional intensity of an individual neuron by an effective rate that only depends on its last spike time and on the history of the population activity , t′ < t (as expressed by the subscript “A”). This is called the quasi-renewal approximation [63]. Taking into account the dependence on the last spike is particularly important because of neuronal refractoriness.

To obtain such a quasi-renewal description we make two approximations. Firstly, we approximate the random connectivity by an effective full connectivity with proportionally scaled down synaptic weights (“mean-field approximation”). As a result, all neurons of the same population are driven by identical synaptic input (see Methods). This implies that for all neurons that had the same last spike time, the time course of the membrane potential is identical, . Secondly, we make the quasi-renewal approximation for GIF neurons [63], which replaces the threshold ϑ_i(t) by an effective threshold . Again, the effective threshold only depends on the last spike time and the history of the population activity. As a final result we obtain the conditional intensity for all neurons with a given last spike time as (4) (Fig 1C, inset). A comparison of Eq (4) with Eq (3) shows that the explicit dependence on all past spike times of a given neuron i has disappeared. Instead, the conditional intensity now only depends on the last firing time and the past population activity , t′ < t. To keep the notation simple, we omit in the following the population label α at all quantities.

Finite-size mean field theory.

In this section, we present the main theoretical results with a focus on the finite-size effects arising from neuronal refractoriness. So far, we have effectively reduced the firing probability of a neuron to a function that only depends on its last spike time (Fig 2A). This allows us to describe the evolution of the system by the density of the last spike time [23, 68, 88, 89]. Because the last spike time characterizes the refractory state of the neuron, this density will also be referred to as the refractory density. Before we describe the novel finite-N theory, it is instructive to first recall the population equations for the case of infinite N (Fig 2B and 2C). Let us look at the population of neurons at time t and ask the following question: What fraction of these neurons has fired their last spike between and ? This fraction is given by the number of neurons that have fired in this interval multiplied by the survival probability that such a neuron has not fired again until time t. In other words, the product evaluated at time t represents the density of last spike times . Because a neuron with last spike time emits a spike with rate , the total population activity at time t is given by the integral [23] (5) This situation is depicted in Fig 2B. At the same time, the survival probability of neurons that fired their last spike at decays according to (6) with initial condition (Fig 2E and 2F red line). Eqs (5) and (6) define the population dynamics for N → ∞ [16, 23, 89].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. How fluctuations of the refractory density effect the population activity.

(A) The conditional intensity shown as a function of (left) and t (right). Typically, the conditional intensity increases as the time since the last spike grows and the neuron recovers from refractoriness. (B) For N → ∞, the population activity A(t) (hatched bin) results from averaged over the last spike times with a weighting factor (blue) corresponding to the density of last spike times. Here, is the survival probability. (C) The last spike times are discretized into time bins. In the bin immediately before t, a large fluctuation (blue peak) was induced by forcing some of the neurons with last spike time around t* to fire. At time t, the density of last spike times (blue) has a hole and a peak of equal probability mass. The red line shows the pseudo-density that would be obtained if we had used the survival probability of the unperturbed system. The peak at does not contribute to the activity A(t) because of refractoriness, but the hole at contributes with a non-vanishing rate (A), implying a reduction of A(t) (hatched bin). (D) For a finite population size (here N = 400), the refractory density (blue), determines the expectation (hatched bin) of the fluctuating activity A_N(t). Analogous to the forced fluctuation in (C), the finite-size fluctuations are associated with negative and positive deviations in the refractory density (holes and overshoots) compared to the non-normalized density (red line) that would be obtained if only the mean and not the exact survival fraction had been taken into account. The variance of the deviations is proportional to given by Eq (12). As a function of , shows the range of where deviations are most prominent (bottom). (E, F) Given the number of neurons firing in the bin , , the fraction of neurons that survive until time t is shown for ten realizations (gray lines, one highlighted in black for clarity). The mean fraction equals the survival probability (red line, top panel). The variance of the number of survived neurons at time t, , is shown at the bottom (red line: semi-analytic theory, Eq (12); circles: simulation). (E) , (F) .

https://doi.org/10.1371/journal.pcbi.1005507.g002

In the limit N → ∞, the dynamics of A_N(t) = A(t) is deterministic because microscopic noise averages out. Nevertheless, the infinite-N case is useful to understand the main effect of fluctuations in the finite-N case. To this end, let us perform the following thought experiment: in a small time interval of length Δt immediately before time t, we induce a large, positive fluctuation in the activity by forcing many of the neurons with last spike close to a given time to fire a spike (Fig 2C). As a result, the density of last spike times at time t exhibits a large peak just prior to time t corresponding to the large number of neurons that have been forced to fire in the time interval [t − Δt, t). At the same time, these neurons leave behind a “hole” in the density around . Because the number of neurons is conserved, this hole exactly balances the area under the peak, and hence, the density of last spike times remains normalized. However, the two fluctuations (the hole and the peak) have two different effects on the population activity after time t. Specifically, the hole implies that some of the neurons which normally would have fired at time t with a nonzero rate λ_A(t|t*) > 0 are no longer available. Moreover, neural refractoriness implies that neurons which fired in the peak have a small or even vanishing rate λ_A(t|t − Δt) ≈ 0 at time t. As a result, the population activity is reduced shortly after the perturbation (Fig 2C). This example highlights the importance of the normalization of the refractory density as well as the state-dependence of the firing probability for understanding the effect of fluctuations. In particular, the normalization condition and neuronal refractoriness imply that a positive fluctuation of the population activity is succeeded by a negative fluctuation, and vice versa. This behavior is characteristic for spiking neurons, which are known to exhibit negative auto-correlations of their mean-centered spike trains at short time lags (see e.g. [14, 53, 90]).

We now turn to the finite-size case. In this case, it is advantageous to discretize the last spike times into small bins of size Δt that begin at the grid points , . Furthermore, we adopt the definition of the coarse-grained population activity, Eq (2), i.e. we consider the number of spikes in the time bin . Instead of the survival probability, we introduce the fraction of survived neurons , , such that is the number of neurons from bin k that have not fired again until time t (Fig 2D and 2E). Dividing this number by NΔt and taking the continuum limit Δt → 0, yields the density of last spike times in the case of finite N. Since all neurons are uniquely identified by their last spike time, this density is normalized [23] (7) We note that differentiating this equation with respect to time t yields the population activity as the formal finite-size analog of Eq (5). The change of the survival fraction , however, is not deterministic anymore as in Eq (6) but follows a stochastic jump process (Fig 2E and 2F): In the time step [t, t + Δt), the number of survived neurons for a given bin in the past, , makes a downstep that corresponds to the number of neurons that fire in the group with last spike time . For sufficiently small Δt, this number is Poisson-distributed with mean . Hence, the fraction of survived neurons evolves in time like a random stair case according to the update rule . The activity A_N(t) = Δn(t)/(NΔt) in the time bin starting at t is given by the sum of all the downsteps, , where the sum runs over all possible last spike times. This updating procedure represents the evolution of the density of last spike times, , for finite N under the quasi-renewal approximation (cf. Methods, Eqs (41) and (42)). Although it is possible to simulate such a finite-N stochastic process using the downsteps , this process will not yield the reduced mesoscopic dynamics that we are looking for. The reason is that the variable refers to the subpopulation of neurons that fired in the small time bin at . For small Δt, the size of the subpopulation, , is a small number, much smaller than N. In particular, in the limit Δt → 0, the simulation of for all in the past would be as complex as the original microscopic dynamics of N neurons. Therefore we must consider such a simulation as a microscopic simulation. To see the difference to a mescoscopic simulation, we note that the population activity A_N(t) involves the summation of many random processes (the downsteps ) over many small bins. If we succeed to simulate directly A_N(t) from an underlying rate dynamics that depends deterministically on the past activities A_N(t′), t′ < t, we will have a truely mescoscopic simulation. How to arrive at a formulation directly at the level of mescocopic quantities will be the topic of the rest of this section.

The crucial question is whether the stochasticity of the many different random processes can be reduced to a single effective noise process that drives the dynamics on the mesoscopic level. To this end, we note that for small Δt and given history , , each bin contributes with rate a Poisson random number of spikes to the total activity at time t (Fig 2D). Therefore, the total number of spikes Δn(t) is Poisson distributed with mean , where is the expected population rate (8) Here, the integral extends over all last spike times up to but not including time t. Eq (8) still depends on the stochastic variables . The main strategy to remove this microscopic stochasticity is to use the evolution of the survival probability , given by Eq (6), and the normalization condition Eq (7). For finite N, the quantity is formally defined as the solution of Eq (6) and can be interpreted as the mean of the survival fraction (Fig 2E and 2F, see Methods). Importantly, is a valid mesoscopic quantity since it only depends on the mesoscopic population activity A_N (through , cf. Eq (6)), and not on a specific microscopic realization. Combining the survival probability with the actual history of the mesoscopic activity for yields the pseudo-density . In contrast to the macroscopic density in Eq (5) or the microscopic density , the pseudo-density is not normalized. However, the pseudo-density has the advantage that it is based on mesoscopic quantities only.

Let us split the survival fraction into the mesoscopic quantity and a microscopic deviation, . In analogy to the artificial fluctuation in our thought experiment, endogenously generated fluctuations in the finite-size system are accompanied by deviations of the microscopic density from the pseudo-density (Fig 2C and 2D, red line). A negative deviation () can be interpreted as a hole and a positive deviation () as an overshoot (compare red curve and blue histogram in Fig 2D). Similar to the thought experiment, the effect of these deviations needs to be weighted by the conditional intensity in order to arrive at the population activity. Eq (8) yields (9) Analogously, the normalization of the refractory density, Eq (7), can be written as (10) We refer to the second integral in Eq (9) as a correction term because it corrects for the error that one would make if one sets S_N = S in Eq (8). This correction term represents the overall contribution of the holes (δS < 0) and overshoots (δS > 0) to the expected activity.

To eliminate the microscopic deviations in Eq (9) we use the normalization condition, Eq (10). This is possible because the correction term is tightly constrained by the sum of all holes and overshoots, , which by Eq (10), is completely determined by the past mesoscopic activities. Eqs (9) and (10) suggest to make the deterministic ansatz for the correction term. As shown in Methods (“Mesoscopic population equations”), the optimal rate Λ(t) that minimizes the mean squared error of this approximation is given by (11) Here, the quantity , called variance function, obeys the differential equation (12) with initial condition (see Methods, Eq (51). Importantly, the dynamics of v involves mesoscopic quantities only, and hence v is mesoscopic. As shown in Methods and illustrated in Fig 2D (bottom), we can interpret as the variance of the number of survived neurons, . To provide an interpretation of the effective rate Λ(t) we note that, for fixed t, the normalized variance is a probability density over . Thus, the effective rate Λ(t) can be regarded as a weighted average of the conditional intensity that accounts for the expected amplitude of the holes and overshoots.

Using the effective rate Λ(t) in Eq (9) results in the expected activity (13) Looking at the structure of Eq (13), we find that the first term is the familiar population integral known from the infinite-N case, Eq (5). The second term is a correction that is only present in the finite-N case. In fact, in the limit N → ∞, the pseudo-density converges to the macroscopic density , which is normalized to unity. Hence the correction term vanishes and we recover the population Eq (5) for the infinite system.

To obtain the population activity we consider an infinitesimally small time scale dt such that the probability of a neuron to fire during an interval [t, t + dt) is much smaller than one, i.e. . On this time scale, the total number of spikes dn(t) is an independent, Poisson distributed random number with mean , where is given by Eq (13). From Eq (2) thus follows the population activity (14) Alternatively, the population activity can be represented as a δ-spike train, or “shot noise”, , where is a random point process with a conditional intensity function . Here, the condition denotes the history of the point process {t_pop,k} up to (but not including) time t, or equivalently the history of the population activity for . The conditional intensity means that the conditional expectation of the population activity is given by , which according to Eq (13) is indeed a deterministic functional of the past activities. Finally, we note that the case of large but finite populations permits a Gaussian approximation, which yields the more explicit form (15) Here, ξ(t) is a Gaussian white noise with correlation function 〈ξ(t)ξ(t′)〉 = δ(t − t′). The correlations of ξ(t) are white because spikes at t′ and t > t′ are independent given the expected population activity at time t. However, we emphasize that the expected population activity does include information on the past fluctuations ξ(t′) at time t′. Therefore the fluctuations of the total activity A_N(t) are not white but a sum of a colored process and a white-noise process ξ(t) [55]. The white noise gives rise to the delta peak of the auto-correlation function at zero time lag which is a standard feature of any spike train, and hence also of A_N(t). The colored process , on the other hand, arises from Eq (13) via a filtering of the actual population activity which includes the past fluctuations . For neurons with refractoriness, is negatively correlated with recent fluctuations (cf. the thought experiment of Fig 2B) leading to a trough at small time lags in the spike auto-correlation function [14, 53, 90].

The set of coupled Eqs (6), (12), (11)–(14) constitute the desired mesoscopic population dynamics and is the main result of the paper. The dynamics is fully determined by the history of the mesoscopic population activity A_N. The Gaussian white noise in Eq (15) or the independent random number involved in the generation of the population activity via Eq (14) is the only source of stochasticity and summarizes the effect of microscopic noise on the mesoscopic level. Microscopic detail such as the knowledge of how many neurons occupy a certain microstate has been removed.

One may wonder where the neuronal interactions enter in the population equation. Synaptic interactions are contained in the conditional intensity which depends on the membrane potential , which in turn is driven by the synaptic current that depends on the population activity via Eq (29) in Methods. An illustration of the derived mesoscopic model is shown in Fig 1C (inset). In this section, we considered a single population to keep the notation simple. However, it is straightforward to formulate the corresponding equations for the case of several equations as shown in Methods, Sec. “Several populations”.

Stochastic population dynamics can be efficiently simulated forward in time.

The stochastic population equations provide a rapid means to integrate the population dynamics on a mesoscopic level. To this end, we devised an efficient integration algorithm based on approximating the infinite integrals in the population equation Eq (13) by discrete sums over a finite number of refractory states (Methods, Sec. “Numerical implementation”). The algorithm involves the generation of only one random number per time step and population, because the activity is sampled from the mesoscopic rate . In contrast, the microscopic simulation requires in each time step to draw a random number for each neuron. Furthermore, because the population equations do not depend on the number of neurons, we expect a significant speed-up factor for large neural networks compared to a corresponding microscopic simulation. For example, the microscopic simulation of the cortical column in Fig 1B took 13.5 minutes to simulate 10 seconds of biological time, whereas the corresponding forward integration of the stochastic population dynamics (Fig 1D) took only 6.6 seconds on the same machine (see Sec. “Comparison of microscopic and mesoscopic simulations”).

A pseudocode of the algorithm to simulate neural population dynamics is provided in Methods (Sec. “Numerical implementation”). In addition to that, a reference implementation of this algorithm is publicly available under https://github.com/schwalger/mesopopdyn_gif, and will be integrated in the Neural Simulation Tool (NEST) [91], https://github.com/nest/nest-simulator, as a module presumably named gif_pop_psc_exp.

Mesoscopic equations capture refractoriness.

Our theory describes the interplay between finite-size fluctuations and spike-history effects. The most prominent spike-history effect is refractoriness, i.e. the strong effect of the last spike on the current probability to spike. To study this effect, we first focus on a population of uncoupled neurons with a constant threshold corresponding to leaky integrate-and-fire (LIF) models without adaptation (Fig 3). The reset of the membrane potential after each spike introduces a period of relative refractoriness, where spiking is less likely due to a hyper-polarized membrane potential (Fig 3A). Because of the reset to a fixed value, the spike trains of the LIF neurons are renewal processes. In the stationary state, the fluctuation statistics as characterized by the power spectrum is known analytically for the case of a population of independent renewal spike trains (Eq (134) in Methods).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Population activity of uncoupled leaky integrate-and-fire neurons without adaptation.

(A) Neurons were stimulated by a step current I_ext(t) such that μ = u_rest + RI_ext(t) increased from μ = 15 mV to μ = 30 mV (top). Voltage trace of one of 500 neurons (bottom). Stationary firing statistics (rate and coefficient of variation (CV) of the interspike intervals) corresponding to the two stimuli are indicated above the step current. (B) Realizations of the population activity A_N(t) for the microscopic (top) and mesoscopic (bottom, blue line) simulation. The magenta line shows the expected population rate given the past actual realization A_N(t′) for t′ < t. (C) The power spectrum of the stationary activity A_N(t) obtained from renewal theory, Eq (134), (black solid line) and from the mesoscopic simulation (blue circles). The top and bottom panel corresponds to weak (μ = 15 mV) and strong (μ = 30 mV) constant stimulation (transient removed).

https://doi.org/10.1371/journal.pcbi.1005507.g003

A single realization of the population activity A_N(t) fluctuates around the expected activity that exhibits a typical ringing in response to a step current stimulation [23, 93]. The time course of the expected activity as well as the size of fluctuations are roughly similar for microscopic simulation (Fig 3B, top) and the numerical integration of the population equations (Fig 3B, bottom). We also note that the expected activity is not constant in the stationary regime but shows weak fluctuations. This is because of the feedback of the actual realization of A_N(t′) for t′ < t onto the dynamics of , Eq (13).

A closer inspection confirms that the fluctuations generated by the mesoscopic population dynamics in the stationary state exhibit the same power spectrum (Fig 3C) as the theoretically predicted one, which is given by Eq (134). In particular, the mesoscopic equations capture the fluctuation statistics even at high firing rates, where the power spectrum strongly deviates from the white (flat) power spectrum of a Poisson process (Fig 3C bottom). The pronounced dip at low-frequencies is a well-known signature of neuronal refractoriness [94].

Mesoscopic equations capture adaptation and burstiness.

Further important spike-history effects can be realized by a dynamic threshold. For instance, spike-frequency adaptation, where a neuron adapts its firing rate in response to a step current after an initial strong response (Fig 4A and 4B), can be modeled by an accumulating threshold that slowly decays between spikes [47, 49]. In single realizations, the mean population rate as well as the size of fluctuations appear to be similar for microscopic and mesoscopic case (Fig 4B, top and bottom, respectively). For a more quantitative comparison we compared the ensemble statistics as quantified by the power spectrum. This comparison reveals that the fluctuation statistics is well captured by the mesoscopic model (Fig 4C). The main effect of adaptation is a marked reduction in the power spectrum at low frequencies compared to the non-adaptive neurons of Fig 3. The small discrepancies compared to the microscopic simulation originate from the quasi-renewal approximation, which does not account for the individual spike history of a neuron before the last spike but only uses the population averaged history. This approximation is expected to work well if the threshold kernel changes slowly, effectively averaging the spike history locally in time [63].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Population dynamics captures adaptation and burstiness.

(A) 500 adapting leaky integrate-and-fire neurons were stimulated by a step current I_ext that increased μ = u_rest + RI_ext(t) from μ = 12 mV to μ = 27 mV (top). Voltage trace of one neuron (bottom). Stationary firing statistics (rate and coefficient of variation (CV)) corresponding to the two stimuli are indicated above the step current. (B) Realizations of the population activity obtained from microscopic simulation (black) and mesoscopic population equation (blue) as well as (magenta). (C) Power spectra corresponding to the stationary activity (averaged over 1024 trials each of 20 s length) at low and high firing rates as in (A), circles and lines depict microscopic and mesoscopic case, respectively. Parameters in (A)–(C): u_th = 10 mV, u_r = 25 mV, threshold kernel for t ≥ t_ref with J_θ,1 = 1.5 mV⋅s, τ_θ,1 = 0.01 s, J_θ,2 = 1.5 mV⋅s, τ_θ,2 = 1 s. (D) Bursty neuron model. (i) Biphasic threshold kernel θ(t), where a combination of a negative part (facilitation) and a positive part (adaptation) yields a bursty spike pattern, (ii) sample firing pattern of one neuron. (iii) The interspike interval distribution with values of rate and CV. (E) Power spectrum of the population activity A_N(t) shown in (D)-(iv). Parameters in (D) and (E): μ = 20, u_th = 10 mV, u_r = 0 mV, τ_m = 0.01 s, facilitation: J_θ,1 = −0.45 mV⋅s, τ_θ,1 = 0.05 s; adaptation: J_θ,2 = 2.5 mV⋅s, τ_θ,2 = 1 s.

https://doi.org/10.1371/journal.pcbi.1005507.g004

In the case of fast changes of the threshold kernel, we do not expect that the quasi-renewal approximation holds. For example, a biphasic kernel [95] with a facilitating part at short interspike intervals (ISI) and an adaptation part for large ISIs (Fig 4D-(i)) can realize bursty spike patterns (Fig 4D-(ii)). The burstiness is reflected in the ISI density by a peak at small ISIs, corresponding to ISIs within a burst, and a tail that extends to large ISIs representing interburst intervals (Fig 4D-(iii)). Remarkably, the mesoscopic equations with the quasi-renewal approximation qualitatively capture the burstiness, as can be seen by the strong low-frequency power at about 1 Hz (Fig 4E). At the same time, the effect of adaptation manifests itself in a reduced power at even lower frequencies. The systematic overestimation of the power across frequencies implies a larger variance of the empirical population activity obtained from the mesoscopic simulation, which is indeed visible by looking at the single realizations (Fig 4D-(iv)). As an aside, we note that facilitation which is strong compared to adaptation can lead to unstable neuron dynamics even for isolated neurons [96].

Recurrent network of randomly connected neurons.

So far, we have studied populations of uncoupled neurons. This allowed us to demonstrate that the mesoscopic dynamics captures effects of single neuron dynamics on the fluctuations of the population activity. Let us now suppose that each neuron in a population α is randomly connected to presynaptic neurons in population β with probability p^αβ such that the in-degree is fixed to p^αβ N^β connections. In the presence of synaptic coupling, the fluctuations at time t are propagated through the recurrent connectivity and may significantly influence the population activity at time t′ > t. For instance, in a fully-connected network (p^αβ = p = 1 for all α and β) of excitatory and inhibitory neurons (E-I network, Fig 5B and 5C), all neurons within a population receive identical inputs given by the population activities (cf. Eq (29)). Finite-size fluctuations of generate common input to all neurons and tend to synchronize neurons. This effect manifests itself in large fluctuations of the population activity (Fig 5B). Since the mean-field approximation of the synaptic input becomes exact in the limit p → 1, we expect a good match between the microscopic and mesoscopic simulation in this case. Interestingly, the power spectra of the population activities obtained from these simulations coincide well even for an extremely small E-I network consisting of only one inhibitory and four excitatory neurons (Fig 5C). This extreme case of N = 5 neurons with strong synapses (here, w^EE = w^IE = 12 mV, w^II = w^EI = −60 mV) highlights the non-perturbative character of our theory for fully-connected networks, which does not require the inverse system size or the synaptic strength to be small. In general, the power spectra reveal pronounced oscillations that are induced by finite size fluctuations [43]. The amplitude of these stochastic oscillations decreases as the network size increases and vanishes in the large-N limit.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Mesoscopic dynamics of E-I network for varying network size N, connection probability p and number of synapses per neuron C.

(A) Left: Schematic of the network of N^E excitatory and N^I = N^E/4 inhibitory leaky integrate-and-fire (LIF) neurons, each receiving C^E = pN^E (C^I = pN^I) connections from a random subset of excitatory (inhibitory) neurons. Total numbers are N = N^E + N^I and C = C^E + C^I. At C = 200, the synaptic strength is w = 0.3 mV and −gw = −1.5 mV for excitatory and inhibitory connections, respectively. To preserve a constant mean synaptic input, the synaptic strength is scaled such that Cw = const‥ Right: Schematic of a corresponding mesoscopic model of two interacting populations. (B) Trajectories of u(t) for five example neurons (top) and of the excitatory population activity obtained from the network simulation (middle) and the mesoscopic simulation (bottom, dark green) for C = N = 50; time resolution Δt = 0.2 ms. The light green trajectory (bottom panel) depicts the expected population activity given the past activity. (C) Power spectra of for different network sizes while keeping p = 1 fixed (microscopic: symbols, mesoscopic: solid lines with corresponding dark colors). (D) Sample trajectories corresponding to the green curve in (E) (N = 1000, p = 0.2). (E) Analogously to (C) but varying the connection probabilities while keeping C = pN = 200 fixed. (F) Sample trajectories corresponding to the green curve in (G) (N = 200, C = 40). (G) Analogously to (C) but varying the number of synapses C while keeping N = 200 fixed. Note that the mesoscopic theory (black solid line) is independent of C because the product Cw, which determines the interaction strength in the mesoscopic model (see, left panel of (A)), is kept constant. Parameters: μ^E/I = 24 mV, mV and θ^E/I(t) ≡ 0 (no adaptation).

https://doi.org/10.1371/journal.pcbi.1005507.g005

If the network is not fully but randomly-connected (0 < p < 1), neurons still share a part of the finite-size fluctuations of the population activity. Earlier theoretical studies [58, 68, 97] have pointed out that these common fluctuations inevitably yield correlated and partially synchronized neural activity, as observed in simulations (Fig 5D and 5F). This genuine finite-size effect decreases for larger networks approaching an asynchronous state [98] (Fig 5C and 5E). As argued in previous studies [58–60], the fluctuations of the synaptic input can be decomposed into two components, a coherent and an incoherent one. The coherent fluctuations are given by the fluctuations of the population activity and are thus common to all neurons of a population. This component is exactly described by our mean-field approximation, used in Eq (4) (cf. Methods, Eq (31)). The incoherent fluctuations are caused by the quenched randomness of the network (i.e. each neuron receives input from a different subpopulation of the network) and have been described as independent Poisson input in earlier studies [58–60]. If we compare the membrane potential of a single neuron with the one expected from the mean-field approximation (Fig 6A, 6C and 6E, top), we indeed observe a significant difference in fluctuations. This difference originates from the incoherent component. Differences in membrane potential will lead to differences in the instantaneous spike emission probability for each individual neuron; cf. Eq (3). However, in order to calculate the population activity we need to average the conditional firing rate of Eq (3) over all neurons in the population (see Methods, Eq (28) for details). Despite the fact that each neuron is characterized by a different last firing time , the differences in firing rate caused by voltage fluctuations will, for sufficiently large N and not too small p, average out whereas common fluctuations caused by past fluctuations in the population activity will survive (Fig 6A, 6C and 6E, bottom). In other words, the coherent component is the one that dominates the finite-N activity whereas the incoherent one is washed out. Therefore, mesoscopic population activities can be well described by our mean-field approximation even when the network is not fully connected (Figs 5E, 5G and 6B, 6D, 6F). Remarkably, even for C = 200 synapses per neuron and p = 0.05, the mesoscopic model agrees well with the microscopic model. However, if both N and p are small, the mesoscopic theory breaks down as expected (Fig 5G, blue circles). Furthermore, strong synaptic weights imply strong incoherent noise, which is then passed through the exponential nonlinearity of the hazard function. This may lead to deviations of the population-averaged hazard rate from the corresponding mean-field approximation (Fig 6E, bottom), and, consequently, to deviations between microscopic and mesoscopic population activities in networks with strong random connections (Fig 6F).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Mean-field approximation of synaptic input for randomly connected networks.

The same E-I network as in Fig 5 with N = 500 neurons and connection probability p = 0.2 was simulated for increasing synaptic strength w^EE = w^IE = w (w^EI = w^II = −5w) of excitatory (inhibitory) connections: (A, B) w = 0.25 mV, (C, D) w = 0.5 mV (E, F) w = 1 mV. (A, C, E) Top: Membrane potential of one example neuron shows fluctuations due to spike input from C = 100 presynaptic neurons (black line), which represent a random subset of all 500 neurons. The mean-field approximation of the membrane potential (dashed red line) assumes that the neuron had the same firing times but was driven by all neurons, i.e. by the population activities and , with rescaled synaptic strength . Although individual membrane potentials differ significantly from the mean-field approximation (top), the relevant population-averaged hazard rates (bottom) are well predicted by the mean-field approximation. (B, D, F) Corresponding power spectra of the (excitatory) population activity for microscopic (circles) and mesoscopic (blue solid line) simulation. Parameters as in Fig 5 except μ^E/I = 18 mV.

https://doi.org/10.1371/journal.pcbi.1005507.g006

Finite-size induced switching in bistable networks.

In large but finite E-I networks, the main effect of weak finite-size fluctuations is to distort the deterministic population dynamics of the infinitely large network (stable asynchronous state or limit cycle motion) leading to stochastic oscillations and phase diffusion that can be understood analytically by linear response theory [43, 55, 60, 69] and weakly nonlinear analysis [58]. This is qualitatively different in networks with multiple stable states. In such networks, finite-size fluctuations may have a drastic effect because they enable large switch-like transitions between metastable states that cannot be described by a linear or weakly nonlinear theory. We will show now that our mesoscopic population equation accurately captures strongly nonlinear effects, such as large fluctuations in multistable networks.

Multistability in spiking neural networks can emerge as a collective effect in balanced E-I networks with clustered connectivity [99], and, generically, in networks with a winner-take-all architecture, where excitatory populations compete through inhibitory interactions mediated by a common inhibitory population (see, e.g. [14, 100–104] and Fig 7A). Jumps between metastable states have been used to model switchings in bistable perception [32–34]. To understand such finite-size induced switching in spiking neural networks on a qualitative level, phenomenological rate models have been usually employed [32, 104]. In these models, stochastic switchings are enabled by noise added to the deterministic rate equations in an ad hoc manner. Our mesoscopic mean-field equations keep the spirit of such rate equations, however with the important difference that the noisy dynamics is systematically derived from the underlying spiking neural network without any free parameter. Here, we show that the mesoscopic mean-field equations quantitatively reproduce finite-size induced transitions between metastable states of spiking neural networks. We emphasize that the switching statistics depends sensitively on the properties of the noise that drive the transitions [105]. Therefore, an accurate account of finite-size fluctuations is expected to be particularly important in this case.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Finite-size induced switching in a bistable network.

(A) Schematic of a winner-take-all network architecture: Two competing excitatory populations (N^E1 = N^E2 = 400) interact with a common inhibitory population (N^I = 200). (B)-(D) In the absence of adaptation (θ^E/I(t) ≡ 0), the excitatory populations switch between low and high activities in an irregular fashion (B). Activities in (B, E) are low-pass-filtered by a moving average of 100 ms. Top: full network simulation. Inset: Magnified view of the activities for 1 s (without moving average) showing fast large-amplitude oscillations. Bottom: mesoscopic simulation. (C) The power spectrum of the activity of the excitatory populations exhibits large low-frequency power and a high-frequency peak corresponding to the slow stochastic switching between high- and low-activity states and the fast oscillations, respectively. (D) The density of the dominance times (i.e. the residence time in the high-activity states) has an exponential form. (E-G) Like (B-D) but excitatory neurons exhibit weak and slow adaptation ( with J_θ = 0.1 mV⋅s, τ_θ = 1 s for t ≥ t_ref). Switching between high- and low-activity states is more regular than in the non-adapting case as revealed by a low-frequency peak in the power spectrum (F) and a narrow, unimodal density of dominance times (G). In (C, D) and (F, G) microscopic and mesoscopic simulation correspond to cyan symbols/bars and dark blue solid lines, respectively. Parameters: μ^E/I = 36 mV except for μ^E = 36.5 in (E-G) to compensate adaptation. Time step Δt = 0.01 ms (microscopic), Δt = 0.2 ms (mesoscopic). Efficacies of excitatory and inhibitory connections: w^E = 0.0624 mV and w^I = −0.2496 mV (B-D), and w^E = 0.096 mV and w^I = −0.384 mV (E-G), p = 1, mV.

https://doi.org/10.1371/journal.pcbi.1005507.g007

We consider a simple bistable network of two excitatory populations with activities A^E1 and A^E2, respectively, that are reciprocally connected to a common inhibitory population with activity A^I (Fig 7A). We choose the mean input and the connection strength such that in the large-N limit the population activities exhibited two stable equilibrium states, one corresponding to a situation, where A^E1 is high and A^E2 is low, the other state corresponding to the inverse situation, where A^E1 is low and A^E2 is high. We found that in smaller networks, finite-size fluctuations are indeed sufficient to induce transitions between the two states leading to repeated switches between high- and low-activity states (Fig 7B and 7E). The regularity of the switching appears to depend crucially on the presence of adaptation, as has been suggested previously [32, 34]. Remarkably, both in the presence and absence of adaptation, the switching dynamics of the spiking neural network appears to be well reproduced by the mesoscopic mean-field model.

For a more quantitative comparison, we use several statistical measures that characterize the bistable activity. Let us first consider the case without adaptation. As before, we compare the power spectra of the population activity for both microscopic and mesoscopic simulation and find a good agreement (Fig 7C). The peak in the power spectrum at relatively high-frequency reveals strong, rapid oscillations that are visible in the population activity after a switch to the high-activity state (inset of Fig 7B with magnified view). In contrast, the large power at low frequencies corresponds to the slow fluctuations caused by the switching of activity between the two excitatory populations, as revealed by the low-pass filtered population activity (Fig 7B). The Lorentzian shape of the power spectrum caused by the slow switching dynamics is consistent with stochastically independent, exponentially distributed residence times in each of the two activity states (i.e., a homogeneous Poisson process). The residence time distribution shows indeed a monotonic, exponential decay (Fig 7D) both in the microscopic and mesoscopic model. Furthermore, we found that residence times do not exhibit significant serial correlations. Together, this confirms the Poissonian nature of bistable switching in our three-population model of neurons without adaptation.

In models for perceptual bistability, residence times in the high-activity state are often called dominance times. The distribution of dominance times is usually not exponential but has been described by a more narrow, gamma-like distribution (see, e.g. [106]). Such a more narrow distribution emerges in a three-population network where excitatory neurons are weakly adaptive. When the population enters a high-activity state, the initial strong increase of the population activity is now followed by a slow adaptation to a somewhat smaller, stationary activity (Fig 7E). Eventually, the population jumps back to the low-activity state. The switching dynamics is much more regular with than without adaptation leading to slow stochastic oscillations as highlighted by a second peak in the power spectrum at low frequencies (Fig 7F) and a narrow distribution of dominance times (Fig 7G), in line with previous theoretical studies [32–34]. We emphasize, however, that in contrast to these studies the underlying deterministic dynamics for N → ∞ is in our case not oscillatory but bistable, because the adaptation level is below the critical value necessary in the deterministic model to switch back to the low-activity state.

The emergence of regular switching due to finite-size noise can be understood by interpreting the residence time of a given population in the high-activity state as arising from two stages: (i) the initial transient of the activity to a decreased (but still large) stationary value due to adaptation and (ii) the subsequent noise-induced escape from the stationary adapted state. The first stage is deterministic and hence does not contribute to the variability of the residence times. The variability results mainly from the second stage. The duration of the first stage is determined by the adaptation time scale. If this time covers a considerable part of the total residence time, we expect that the coefficient of variation (CV), defined as the ratio of standard deviation and mean residence time, is small. In the case without adaptation, a deterministic relaxation stage can be neglected against the mean noise-induced escape time so that the CV is larger.

Mesoscopic dynamics of cortical microcolumn.

As a final example, we applied the mesoscopic population equations to a biologically more detailed model of a local cortical microcircuit. Specifically, we used the multi-laminar column model of V1 proposed by Potjans and Diesmann [5] (see also [72, 107] for an analysis of this model). It consists of about 80′000 non-adapting integrate-and-fire neurons organized into four layers (L2/3, L4, L5 and L6), each accommodating an excitatory and an inhibitory population (see schematic in Fig 1A). The neurons are randomly connected within and across the eight populations. We slightly changed this model to include spike-frequency adaptation of excitatory neurons, as observed in experiments (see e.g. [26]). Furthermore, we replaced the Poissonian background noise in the original model by an increase of mean current drive and escape noise (both in the microscopic and mesoscopic model). The mean current drive was chosen such that the firing rates of the spontaneous activity were matched to the firing rates in the original model. We note that the fitting of the mean current was made possible by the use of our population equations, which allow for an efficient evaluation of the firing rates. The complete set of parameters is listed in Methods, Sec. “Modified Potjans-Diesmann model”.

Sample trajectories of the population activities have already served as an illustration of our approach in Fig 1, where neurons in layer 4 and 6 are stimulated by a step current of 30 ms duration, mimicking input from the thalamus as in the original study [5]. Individual realizations obtained from the microscopic and mesoscopic simulation differ due to the marked stochasticity of the population activities (Fig 1B and 1D). However, trial-averaging reveals that the mean time-dependent activities that can be estimated from a peri-stimulus-time histogram (PSTH) obtained from microscopic and mesoscopic simulations indeed agree well, except for a slight underestimation of the oscillatory peak during stimulus offset compared to the microscopic simulation (Fig 8A). However, during the short moments where the mean time-dependent activity (PSTH) of the mescoscopic and microscopic simulation do not match, the time-dependent standard deviation across hundreds of trials (Fig 8B) is extremely high in both mesoscopic and microscopic simulation, indicating that fluctuations of the activity between one trial and the next are high after stimulus offset at 0.09s. The standard deviation as a function of time (Fig 8B) agrees overall nicely between microscopic and mesoscopic simulation, suggesting a good match of second-order statistics. A closer look at the second-order statistics, as provided by the power spectra of spontaneous activities (“ground state” of cortical activity), also reveals a good agreement at all frequencies (Fig 9). This agreement is remarkable in view of the low connection probabilities (p < 0.14, see table 5 in [5]) that violate the assumption of dense random connectivity used in the derivation of the mesoscopic mean-field equations. More generally, this example demonstrates that the range of validity of our mesoscopic theory covers relevant cortical circuit models.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Time-dependent statistics of the population activities in a cortical column model.

(A) Trial-averaged population activities (peri-stimulus-time histogram, PSTH) in the modified Potjans-Diesmann model as illustrated for a single trial in Fig 1. Circles and blue solid line show microscopic simulation (250 trials, simulation time step Δt = 0.01 ms) and mesoscopic simulation (1000 trials, Δt = 0.5 ms), respectively. A step current mimicking thalamic input is provided to neurons in layer 4 and 6 during a time window of 30 ms as indicated by the gray bar. Rows correspond to the layers L2/3, L4, L5 and L6, respectively, as indicated. Columns correspond to excitatory and inhibitory populations, respectively. (B) Corresponding, time-dependent standard deviation of A_N(t) measured with temporal resolution Δt = 0.5 ms.

https://doi.org/10.1371/journal.pcbi.1005507.g008

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. Stationary statistics of population activities in a cortical column model.

Power spectra of the spontaneous population activities A_N(t) in the modified Potjans-Diesmann model in the absence of time-dependent thalamic input (corresponding to the activities shown in Fig 1B (microscopic) and Fig 1D (mesoscopic) outside of the stimulation window. Circles and blue solid lines represent microscopic and mesoscopic simulation, respectively. Rows correspond to the layers L2/3, L4, L5 and L6, respectively, as indicated. Columns correspond to excitatory and inhibitory populations, respectively.

https://doi.org/10.1371/journal.pcbi.1005507.g009

Finally, we mention that the numerical integration of the mesoscopic population equations yields a significant speed-up compared to the microscopic simulation. While a systematic and fair comparison of the efficiencies depends on many details and is thus beyond the scope of this paper, we note that a simulation on a single core of 10s of biological time took 811.2s using the microscopic model, whereas that of the mesoscopic model only took 6.6s. This corresponds to a speed-up factor of around 120 achieved by using the mesoscopic population model. In the simulation, we employed the same integration time step of Δt = 0.5 ms for both models for a first naive assessment of the performance. However, a more detailed comparison of the performance should be based on simulation parameters that achieve a given accuracy. In this case, we expect an even larger speed-up of the mesoscopic simulation because for the same accuracy the temporally coarse-grained population equations allow for a significantly larger time step than the microscopic simulation of spiking neurons.

Network setup.

We consider a network of M populations each consisting of N^α interconnected neurons of the same type (the superscript α = 1, …, M labels the populations). Neuron i in population α receives p^αβN^β connections (synapses) from a random subset of presynaptic neurons in population β. Here, p^αβ denotes the probability for a connection from a neuron in population β to a neuron in population α. That is, the connections between any two populations are random with fixed in-degree.

Let the spike train of neuron i in population α be denoted by (21) where is its k-th spike time and δ denotes the Dirac δ-function. The neuron receives spike train input from its presynaptic partners in population β with a transmission delay Δ^αβ and synaptic weight w^αβ. More precisely, the synaptic input current is modeled as a sum of post-synaptic currents caused by each spike of presynaptic neurons: (22) where R^α and are the membrane resistance and membrane time constant of a neuron in population α, respectively, and w^αβ sets the synaptic weights in units of mV. The synaptic kernel ϵ^αβ(t) is defined as the postsynaptic current (PSC) normalized by its charge that is induced by one input spike from a neuron of population β. More precisely, ϵ^αβ is the PSC divided by its integral, and therefore it has units of 1/sec. In Eq (22), the first sum runs over all populations β, whereas the second sum runs over the set of all neurons in population β that project onto neuron i in population α.

In general, the filtered total synaptic input from population β, , may be modeled by a set of differential equations for a finite number of synaptic variables , ℓ = 1, …, L. In simulations, we model the synaptic kernel by a single exponential with constant delay Δ^αβ = Δ, , where Θ(t) denotes the Heaviside step function. The synaptic time constants are ms and ms for excitatory and inhibitory synapses, respectively. This kernel can be realized by a single synaptic variable , which obeys the first-order kinetics with β ∈ {E, I}.

Generalized integrate-and-fire model.

Neurons are modeled by a leaky integrate-and-fire model with a dynamic threshold [14, 47, 49, 80] and an escape noise mechanism [14, 26–28]. Following [27], we refer to this model as the generalized integrate-and-fire (GIF) model. The crucial variables of this model are the membrane potential and the dynamic threshold . The membrane potential obeys the subthreshold dynamics (23) where is the membrane time constant and is the drive in the absence of synaptic input consisting of a constant resting potential u_rest and an external stimulus . The synaptic current has been defined in Eq (22).

After each spike the voltage is reset to the potential u_r, where it is clamped for an absolute refractory period t_ref = 4 ms. Furthermore, each spike adds a contribution to the dynamic threshold: (24) where is a baseline threshold and θ^α(t) is called the spike-triggered kernel [26, 27]. Since the increases in spike threshold accumulate over several spikes, the spike-triggered kernel causes spike-frequency adaptation. We set θ^α(t) = ∞ for t ∈ (0, t_ref) so as to ensure absolute refractoriness. For the sake of simplicity, we assumed here that all spike-triggered accumulation effects can be lumped into the threshold variable (cf. Sec. “Mapping onto a generalized linear model” below). However, if realistic membrane potentials are needed (e.g., for fitting membrane potential data [27] or in a conductance-based extension of the model (see Discussion) and [88]), adaptation mechanisms affecting the voltage should be kept in the voltage dynamics.

Spikes are elicited stochastically by a conditional intensity (also called hazard rate, escape rate or conditional rate) (25) which depends on the momentary distance between the membrane potential and the threshold via the exponential link function . The parameter c^α is the escape rate at threshold and the parameter characterizes the softness of the threshold (Fig 1A, inset). Intuitively, a neuron fires immediately if its membrane potential is millivolts above the threshold and is unlikely to fire if its membrane potential is millivolts below the threshold [14]. In the limit , the model turns into a deterministic (but adaptive) leaky integrate-and-fire model with a hard threshold. We emphasize that our standard choice of mV is consistent with the intrinsic stochasticity of neurons in cortical slices [26, 85]. Alternatively, the softness of the threshold may also be regarded as a phenomenological parameter that accounts for all incoherent noise sources that are individual to each neuron. This includes, e.g., any intrinsic noise but also fluctuations of external background input from other neural populations that are not modeled explicitly. For instance, to account for the external Poisson input used in the original cortical column model by Potjans an Diesmann [5], we increase in Figs 1, 8 and 9 the softness to mV. We note that for more detailed comparisons with the original model, more elaborate approximations of the escape rate for the case of colored noise exist [89], which in principle could be used to approximate external Poisson noise without a free parameter. However, because such a mapping is not the focus of the current study, we stick here for the sake of simplicity to the phenomenological escape rate, Eq (25), of the exponential form.

The parameters of the model used in simulations (unless specified differently) are summarized in Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Default values of parameters used in simulations unless stated otherwise.

https://doi.org/10.1371/journal.pcbi.1005507.t001

Mapping onto a generalized linear model.

We also considered a slightly different variant of the model, called spike-response model [14] or generalized linear model (GLM) [55, 66, 84, 108, 109]. This model does not reqire the reset rule of the integrate-and-fire model but instead relies on spike-triggered kernels to implement refractoriness and other spike-history effects. Specifically, the membrane potential is given by (26) where is the free membrane potential given by (27) For a membrane filter kernel , where Θ(t) denotes the Heaviside step function, the dynamics of is equivalent to the dynamics of (Eq (23)), except that is not reset upon spiking. Spike-history effects on the level of the membrane potential are captured by the second term in Eq (26). This term represents the convolution of the output spike train with a spike-triggered kernel η(t) and generates a spike-after-potential that accumulates over spikes. As before, the threshold obeys Eq (24). Given the membrane potential and the dynamic threshold , spikes are generated by the same hazard rate given by Eq (25).

At low firing rates, the spike-triggered kernel η can be used to approximate the integrate-and-fire dynamics by choosing . However, this is not an exact mapping because the value of the membrane potential is not reset to a fixed value u_r after spiking, in contrast to the GIF model. This is due to the accumulation of the threshold and the variability in the voltage at the moment of firing.

We also mention that the kernel η can be transformed into the kernel θ of the threshold dynamics [14]. This is possible because we are only interested in the spike emissions of the neurons and not the membrane potentials. In fact, the conditional firing rate, Eq (25), is invariant under the transformation θ → θ − η, η → 0.

Mean-field approximation of synaptic input.

In the special case of a fully connected network (p^αβ = 1), the membrane potential can be completely inferred from the last spike time and the mean field . In this case, the synaptic input Eq (22) can be rewritten as (29) Thus, in a fully connected network all neurons in population α “see” the same synaptic input given by the “mean field” A_N(t). From Eq (23) follows that GIF neurons with the same last spike time all have the same membrane potential that obeys the differential equation (30) with J^αβ = p^αβ N^β w^αβ. The initial condition is corresponding to the reset of the membrane potential after the last spike. If we use this insight for the conditional intensity we see that the explicit dependence upon can be dropped as long as we keep track of the last spike time , cf. Eq (28); hence .

In a randomly connected network (p^αβ < 1), the synaptic input is different for each neuron. On the population level, however, this heterogeneity is averaged allowing us to still use the mean-field approximation, Eq (30). To see this, we note that in our network with fixed in-degree, each neuron i in population α has p^αβN^β presynaptic neurons in population β (β = α is possible). This means that in Eq (22) we can approximate the sum over the p^αβ N^β presynapic neurons by (31) (cf. definition of in Eq (1)). The mean-field approximation, Eq (31), only depends on the population activity and is thus identical for all neurons. Therefore, fluctuations of the population activity can be regarded as common input fluctuations that are coherent across neurons. On the other hand, the deviation from the mean-field approximation (i.e. the difference between the left- and right-hand side of Eq (31)) is different for each neuron and can be regarded as incoherent noise. For low connection probabilities, this incoherent part of the fluctuations may lead to a significant deviation of the membrane potential from the mean-field approximation (Fig 6A, 6C and 6E, top). On the mesoscopic scale, however, the total number of spikes in a small time step Δt is determined by the population-averaged conditional firing rate, Eq (28), (cf. Eq (46) below). Hence, for sufficiently large N^α, incoherent noise average out, whereas common finite-size fluctuations survive (Fig 6A, 6C and 6E, bottom). Note, however, that incoherent noise may cause a small bias because we average a nonlinear function of the noisy membrane potential on the l.h.s. of Eq (28). Effectively, the incoherent noise softens the threshold of the escape noise mechanism.

Quasi-renewal approximation.

So far we have reduced the conditional intensity to . This expression still involves the individual threshold of neuron i in population α, which depends on the full spike history of that neuron. This means that the spike-train is generally not a time-dependent renewal process. Here, we employ the quasi-renewal approximation [63] and average over the spikes before the last spike time assuming that they occurred according to an inhomogeneous Poisson process with rate , . Averaging the conditional intensity, Eq (25), in this way, conditioned on a given last spike time and a given history , , yields [63, 134, 135] (32) where is an effective dynamic threshold given by (33) Here, is the so-called quasi-renewal kernel [63], while describes the increase of the threshold induced by the last spike. Note that as a result of the two approximations, the conditional firing rate no longer depends on the precise spiking history of a given neuron and its presynaptic neurons, but only on its last firing time, cf. Eqs (25) and (32). This ends our explanation of Eq (28).

Microscopic dynamics of the refractory density.

Because the firing probability of a neuron only depends on its last spike time and the mesoscopic population activity in the past, we can use a population density description of all last spike times in the population. To derive such representation it is useful to discretize time by introducing the discrete time points t_k = t₀ + kΔt, , and the corresponding intervals . Time is measured relative to a reference time t₀, which, however, is irrelevant for the following arguments. We require that the size of the intervals Δt is sufficiently small so that each neuron fires at most once during any interval. Specifically, we require that Δt ≤ t_ref. We also require that the sum of axonal and synaptic delays is not smaller than Δt. Furthermore, we identify the discrete time point t_l as the current time, whereas indices k with k < l correspond to the past. In the population density approach, we do not keep track of the last spike time of each individual neuron but for each past time interval we only track the number of those neurons that have their last spike time in this interval. This number is denoted by m(t_l, t_k). The collection of these numbers for all intervals , k < l, represents the current distribution of last spike times in the population at time t_l (Fig 10A). Because each neuron has exactly one last spike time, the distribution m(t_l, t_k) is normalized to the total number of neurons: (34) Since the last spike time determines the refractory state of a neuron, the distribution m(t_l, t_k) will be also called refractory distribution and the function Q_N(t_l, t_k) ≡ m(t_l, t_k)/(NΔt) can be regarded as the corresponding refractory density. The refractory distribution fully characterizes the microscopic state of the population.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 10. Different interpretations of the function (red line).

(A) As a function of (or as a function of k in discrete time), m(t_l, t_k) represents the distribution of last spike times across the population at time t = t_l. (B) As a function of time t (or as a function of the index l in discrete time), m(t_l, t_k) represents the survival number, i.e. the number of neurons which fired in the interval [t_k, t_k + Δt) which survived (did not fire) until time t = t_l. The activity Δn(t_k), i.e. the number of neurons that fired in the k-th time bin, is depicted by a blue line. The population size is N = 1000.

https://doi.org/10.1371/journal.pcbi.1005507.g010

We now introduce the number of neurons that fired a spike in the interval (not necessarily the last spike). This number is denoted by Δn(t_k) (Fig 10A) and is related to the population activity by A_N(t_k) = Δn(t_k)/(NΔt). Therefore, Δn(t_k) will be often referred to as simply the “activity” at time t_k. Knowing the past activities Δn(t_k′) for k′ < l and the last spike time t_k fully determines the membrane potentials u(t_l, t_k) and thresholds ϑ(t_l, t_k), and hence the escape rate λ(t_l|t_k) = f(u(t_l, t_k) − ϑ(t_l, t_k)) associated with the interval . Thus, the knowledge of the past activities and the distribution of last spike times at time t_l is sufficient to statistically determine these quantities at time t_l+1. In other words, the evolution of the system can be described by a Markov process if we define the microscopic state of the population at time t_l by the sequence of pairs (35) In the following, the main task will be to derive the statistics of the number of spikes Δn(t_l) in the next time interval [t_l, t_l + Δt) and the distribution m(t_l + Δt, t_k) of last spike times at time t_l+1 given the state at time t_l. We mention that what we have lost in this population density description is only the information about the identity of the neurons, which, however, is irrelevant for the mesoscopic description of homogeneous populations.

There is a second interpretation of m(t_l, t_k): let us consider the group of neurons that have fired in the interval , k < l. The number of neurons from this group that have “survived” (i.e. that have not fired again) until time t_l is exactly given by m(t_l, t_k). We will therefore also call it number of survived neurons or survival number for the refractory state k. Correspondingly, the ratio S_N(t_l|t_k) = m(t_l, t_k)/Δn(t_k) is the fraction of survived neurons. As time t_l evolves, the number of survived neurons diminishes whenever there is a spike in that group (Fig 10B). Thus, if the group fires X_lk spikes in the time step [t_l, t_l + Δt), then m(t_l, t_k) decreases by X_lk. For l > k, this gives rise to the evolution equation (36) The initial condition is given by m(t_k + Δt, t_k) = Δn(t_k), which follows from the absolute refractoriness during the first time step after a spike. Absolute refractoriness also entails that each neuron can fire only one spike per time step (Δt ≤ t_ref) with a firing probability (37) In the last step, we introduced the average hazard rate . Because the past activities Δn(t_k), k < l, completely determine the probability to fire P_λ(t_l|t_k), each neuron decides independently from the others whether it fires in the next time step. Furthermore, there is a total number of m(t_l, t_k) neurons from the considered group that could potentially fire in the interval [t_l, t_l + Δt). Therefore, the number of spikes X_lk is the result of m(t_l, t_k) independent Bernoulli trials with success probability P_λ(t_l|t_k). This implies that X_lk follows a binomial distribution: (38) Moreover, the random numbers X_lk associated with different past time intervals are conditionally independent given the current state of the system (cf. Eq (35)).

The total number of spikes emitted in the current interval [t_l, t_l+Δt) is equal to the total reduction of survivals in that interval, hence (39) Eqs (36)–(39) define the microscopic kinetics in discrete time. In a simulation, for each past time interval one independent Poisson random number X_lk needs to be drawn per time step and population. These random numbers determine the current spike count via Eq (39) and the update of the distribution of last spike times m(t_l, t_k), via Eq (36). We call this description microscopic because for small time steps, there will be many (order of N) intervals that contain survived neurons, i.e. for which m(t_l, t_k) > 0 and for each of which one needs to draw a random number X_lk in a simulation. In the limit Δt → 0, such a simulation would be as complex as the original microscopic simulation of N neurons.

The microscopic population density description can be summarized in a particularly compact form by performing the continuum limit Δt → 0 and by assuming large N. For large N, the statistics of X(t_l, t_k) becomes Gaussian with mean and variance P_λ(t_l|t_k)m(t_l, t_k). Thus, the dynamics of , Eq (36), can be rewritten as (40) where are independent, standard normal random numbers and the ramp function [x]₊ = xΘ(x) ensures non-negativity of m. Using the density of last spike times Q_N(t_l, t_k) ≡ S_N(t_l|t_k)A_N(t_k) ≡ m(t_l, t_k)/(NΔt), setting t_l = t and , and expanding for small Δt we arrive at the following dynamics in the limit Δt → 0: (41) Here, is a Gaussian random field with zero mean and correlation function . For a given last spike time , Eq (41) has the form of a Langevin equation. Its initial condition is . The population activity results from Eq (39) as the integral of changes of the refractory density over all refractory states, i.e. , or using Eq (41): (42) Eqs (41) and (42) represent the microscopic population density equations in continuous time under the Gaussian and quasi-renewal approximations.

Mesoscopic description.

At the mesoscopic level, we want to describe the state of the population at time t_l only by the mesoscopic variables Δn(t_k), k < l, that have been observed so far. Therefore, we define the history of n at time t_l by (43) which completely determines the mesoscopic state. In contrast to the microscopic state defined in Eq (35), the mesoscopic state does not require the knowledge of the detailed distribution of last spike times m(t_l, t_k). We call a variable mesoscopic if it only depends on the history . Likewise, an equation is called mesoscopic if it only involves mesoscopic variables. In the following sections, all averages at a given time t_l have to be understood as conditional averages given the history . We will therefore often omit an explicit notation of this condition.

To derive a mesoscopic equation, we want to find an approximate dynamics with only one effective, mesoscopic noise term that summarizes the effect of all microscopic random variables X_lk. In a simulation, this would imply to draw only one random number per time step and per population. Towards that end, we assume that Δt can be chosen sufficiently small such that P_λ(t_l|t_k) ≪ 1, which is always possible if neurons are stochastic und hence do not perfectly synchronize. Under this assumption, the binomially-distributed random numbers X_lk are approximately Poisson-distributed, i.e. (44) where (45) is the conditional mean of X_lk given the current survival number m(t_l|t_k). Given the conditional independence of X_lk for different k, the Poisson property implies that the global activity Δn(t_l) in Eq (39) is also Poisson-distributed given the current refractory distribution {m(t_l, t_k)}_k<l, i.e. (46) with mean (47) Because of the definition of refractory densities and P_λ ≤ 1, we find that is automatically satisfied at any moment in time. However, for the numerical implementation with finite Δt later on we need to keep in mind that the Poisson number Δn(t) could become larger than N, if is close to N. In this case, using a binomial statistics will be more appropriate, as explained in Sec. “Numerical implementation”.

Eqs (46) and (47) suggest the possibility to generate Δn(t_l) by a single Poisson-distributed random number. However, Eq (47) is not a mesoscopic equation yet because it still depends on the dynamics of m(t_l, t_k), Eq (36), which contains the microscopic random variables X_lk. There is another, more subtle problem if we want to use Eqs (46) and (47) as a mesoscopic dynamics that generates the activities Δn(t_l). If we regard Δn(t_l) as an independent random variable, the conservation of neurons, Eq (39), imposes a constraint on the microscopic random numbers {X_lk}_k<l, which will therefore not be independent anymore. Conversely, if we consider both {X_lk}_k<l and Δn(t_l) as independent variables, we almost certainly violate the conservation of neurons, Eq (39), or equivalently, the normalization condition Eq (34). This problem does not occur in the microscopic dynamics, where Δn(t_l) is a dependent variable generated from the independent random variables {X_lk}_k<l via Eq (39), and hence the correct normalization is guaranteed at any time. Nevertheless, the “non-normalized” or “unconstrained” process, in which {X_lk}_k<l and Δn(t_l) are drawn independently, will be useful for deriving mesoscopic equations because it allows us to calculate the moments of the survival numbers m(t_l, t_k). Our main strategy is to use these moments in conjunction with the normalization condition to express the expected spike count , Eq (47), as a deterministic functional of the past activities. In this way, will not depend anymore on the actual microscopic realizations of the constrained noise (constrained by a given history {Δn(t_k)}_k<l via Eq (39)) and can thus be used to generate Δn(t_l) as a Poisson random number from the knowledge of the past activities.

Moment equations.

To achieve such deterministic relationship, we first derive mesoscopic equations for the mean and variance of m(t_l, t_k) given the history in the so-called non-normalized ensemble or unconstrained ensemble. This means that the history determines the initial conditions of the dynamics of m(t_l, t_k), Eq (36), as well as the conditional intensities λ(t_l|t_k), but it does not impose the constraint Eq (39) on the random numbers {X_{l′, k}}_l′≤l. Although this unconstrained noise leads to a non-normalized distribution , it still yields a very good approximation of its mean and variance in the actual constrained ensemble. Taking the average of Eq (36), and using Eq (44) yields the evolution of the mean: (48) with initial condition . Here and in the following, is short-hand for to simplify the notation, and 〈⋅〉 denotes the ensemble average of the unconstrained process for a given history . Actually, the condition for the average 〈⋅〉 can be extended to the history (and to any future activities) because in the unconstrained ensemble neither nor depend on the most recent activity Δn(t_l) (clearly, this also holds for any future activity). Importantly, Eq (48) is a mesoscopic equation because it is fully determined by the past activities.

As a next step we derive an equation for the variance of . To this end, let (49) denote the deviation from the mean. Using the law of total variance, we find for the variance in the next time step (50) The conditional mean of m_l+1,k given the current value m_l,k, denoted by E[m_l+1,k|m_l,k], follows from the evolution Eqs (36) and (45) as [1 − P_λ(t_l|t_k)]m_l,k. Therefore, its variance is . For the second term in Eq (50), we note that the conditional variance Var[m_l+1,k|m_{l, k}] is equal to the variance Var[X_lk|m_{l, k}] of the decrement X_lk. Because X_lk is a Poisson variable, this variance is equal to the mean given by Eq (45). Taken together, we find the following update rule for the total variance (51) with initial condition . As a function of t_l (Fig 2C bottom), the variance is initially zero because all neurons have still survived immediately after firing at time t_k. On the other hand, at long times t_l ≫ t_k, the variance also vanishes because according to Eq (48), the mean number of survived neurons 〈m(t_l, t_k)〉 appearing in Eq (51) goes to zero. As a consequence, the variance obtains a maximum at an intermediate time. Similarly, the dependence of the variance at time t_l for different last spike times shows the same limiting behavior which implies a maximum at an intermediate last spike time (Fig 2B bottom). However, the rugged shape of this function with many local maxima reflects the discontinuity of the driving force 〈m(t_l, t_k)〉 as a function of t_k that arises from the stochastic initial condition 〈m(t_k+1, t_k)〉 = Δn(t_k).

Mesoscopic population equations.

Let us return to Eq (47) for the expected activity , which is used to draw the activity Δn(t_l) as a Poisson random number (cf. Eq (46)). Because we condition on the history {Δn(t_l′)}_l′<l, the processes m_l,k in this equation belong to the “constrained” ensemble, in which the normalization condition, Eq (34), is obeyed. We note that these constrained processes could in principle be generated microscopically by Eq (36) if at each time t_l′ in the past, the microscopic noise X_l′k was sampled from a joint distribution that ensures the conservation of neurons, Eq (39), i.e. ∑_k X_l′k = Δn(t_l′). However, as we will show in the following, such a construction is not needed because the dependence of the expected activity on a specific realization of m_l,k can be eliminated by exploiting the normalization condition, Eq (34). To this end, we take advantage of the fact that the mean of the unconstrained process is a mesoscopic variable. This suggests to split the constrained processes m_l,k into the mean of the unconstrained process and a fluctuation part: (52) The first contribution is deterministic given the past activities Δn(t_k) while the second contribution represents the microscopic fluctuations. We note that the fluctuation δm_l,k is not equivalent to the deviation of the unconstrained process because does not obey the normalization condition, Eq (34), whereas does.

To remove the microscopic fluctuations δm_l,k, we require that both Eqs (34) and (47) are simultaneously satisfied. Substituting Eq (52) into these equations leads to (53) (54) The microscopic fluctuations δm_l,k enter the dynamics only in the form of two sums. First, the normalization condition Eq (53) imposes a strict relation between the summed deviation ∑_k δm_l,k and the means of the unconstrained processes, , irrespective of the specific, underlying microscopic dynamics of m_l,k. In particular, we can solve for with terms on the r.h.s. that are completely determined given the past activities. Second, the total effect of the deviations on the expected activity is given by the weighted sum ∑_k P_λ(t_l|t_k)δm_l,k in Eq (54) with P_λ(t_l|t_k) ≤ 1 for all k < l. The weighted sum ∑_k P_λ(t_l|t_k)δm_l,k is therefore tightly constrained by the value of the summed fluctuation ∑_k δm_l,k. These considerations suggest to make the following decoupling approximation: (55) with a still unknown factor P_Λ(t_l), that we call effective firing probability. To determine the effective firing probability, we require that in the unconstrained ensemble the corresponding approximation (56) minimizes the mean squared error , where P_Λ is short-hand for P_Λ(t_l). We use the unconstrained deviations here because we are only interested in the typical error. Note that if ε_l is Gaussian distributed, minimizing the mean squared error yields the optimal estimation of P_Λ in the sense that it maximizes the log-likelihood of given under the linear approximation Eq (56). The error can be rewritten as ε_l = ∑_k[P_λ(t_l|t_k) − P_Λ(t_l)]Δm_{l, k}, which for N ≫ 1 is a sum of many independent variables that can indeed be considered to be Gaussian. The derivative of with respect to P_Λ is (57) where we have exploited that P_λ(t_l|t_k) is deterministic given the past activities Δn(t_k), k < l. Furthermore, under this condition, the fluctuations and with k ≠ k′ are conditionally independent. Using this property and setting we find that the optimal effective firing probability is (58) The variance in this formula obeys the mesoscopic dynamics derived above in Eq (51). Hence, the effective firing probability is itself mesoscopic.

Using Eqs (53) and (55), ∑_k δm_{l, k} can be eliminated in Eq (54) resulting in (59) Thus, we obtain an equation that yields the mean spike count at the present time as a function of the past spike counts {Δn(t_k)}_k<l. Eq (59) is the desired mesoscopic equation in discrete time. For sufficiently small time steps Δt, the present spike count Δn(t_l) can be generated by drawing a Poisson random number with mean according to Eq (46).

Dynamic threshold of refractory and free neurons.

For free neurons, i.e. for , we use the average threshold under the assumption that spikes occurred in the range according to an inhomogeneous Poisson process with rate . This average is given by [63, 134] (87) where in the last step we used Eq (86). We assume that for t > T the spike-triggered kernel can be sufficiently well approximated by a sum of exponentials This allows us to express the threshold for free neurons as (88) where the variables g_ℓ(t) satisfy the differential equations (89) For refractory neurons, i.e. if , we need to evaluate in the effective threshold, Eq (33), an integral over the exact quasi-renewal kernel . Splitting this integral into the free and refractory part yields the threshold of refractory neurons: (90) We can use the threshold for free and refractory neurons, Eqs (88), (89) and (90), respectively, to obtain the respective conditional intensities: (91) where h(t) is the free membrane potential given by Eq (27). Let us remind the reader that h(t) obeys the dynamics Eq (23) but without resetting of the membrane potential after a spike.

Population equations.

We now apply the split of the history to the integrals that appear in the population equations, specifically Eqs (65) and (66). By definition, the conditional intensity of free neurons does not depend explicitly on the last spike time . That is, for , where the free hazard rate λ_free(t) is given by Eq (91). In the free part of the integrals in Eqs (65) and (66), the free hazard rate can be pulled out of the integral, which yields (92) (93) Here we have introduced the expected number of free neurons and the partial integral over the variance function . Differentiating these new variables and employing Eqs (63) and (69), we find that they obey the differential equations (94) (95) Thus, the integrals no longer run from −∞ to t but are now limited to the range [t − T, t). The long tails over the past have been reduced to differential equations.

Discretization of time.

We discretize the time axis into a grid with step size Δt and grid points (96) Because we keep track of a finite history with the oldest last spike time , the history consists of a finite number K of bins such that T = KΔt. If the index k = l corresponds to the current time, the oldest last spike time of the explicit history corresponds to an index k = l − K and the most recent one corresponds to the index k = l − 1. Note that the numerical implementation requires the absolute refractory period t_ref to be at least as large as the integration time step Δt (see below).

To facilitate the notation of the update rules, it is convenient to introduce the following notations: (97) (98) (99) (100) In particular, and v_k correspond to the mean and variance of the unconstrained survival numbers, respectively. We also recall that the population index α is dropped wherever confusion is not possible, while the index β as well as double indices αβ will be kept.

Choice of Δt.

A crucial assumption of the derivation of the population equations in discrete time was that the time step Δt is small enough such that each neuron fires at most one spike per time step. This can be achieved by the condition (101) (cf. Sec. “Discretized population density equations”). Clearly, this condition implies that the total number of spikes per time step must be bounded by the number of neurons, i.e. the population activity must obey Δn(t_l) ≤ N. The equality sign corresponds to the case that all neurons fire in the same time step. In addition to condition Eq (101), we also had to require that Δt is not larger than the transmission delay Δ, i.e. (102) In order to justify the use of the Poisson statistics in the derivation of the population equations, we further assumed that Δt is sufficiently small such that the expected number of spikes per time step, , is much smaller than N, or equivalently . While this does not pose a problem for the theory, which ultimately concerns with the continuum limit Δt → 0, an efficient numerical integration of the population equations benefits from a time step that is as large as possible and should thus not be limited by such a condition. In particular, we should allow a large fraction of neurons to fire during one time step, either as a result of an external synchronization of many neurons (e.g. by a strong, sudden stimulus) or because of synchronous oscillations emerging from the network dynamics. In this case, a Poisson-distributed spike count Δn(t) may exceed the number of neurons N. This problem can be remedied by drawing Δn(t) from a binomial distribution with mean and maximal value N. For , this binomial distribution agrees with the Poisson distribution used in our theory, whereas at large it ensures that the spike count is bounded by the total number of neurons N. Although the binomial distribution does not follow strictly from our theory, it is expected to yield a very good approximation even at large . The reason is as follows: A statement analogous to our argument that the sum of Poisson numbers [Eq (39)] yields again a Poisson number [Eq (46)] is, in general, not valid for binomial random numbers if the random numbers (i.e. the firing probabilities P_λ(t_l|t_k) in our model) are very different. However, if neurons are strongly synchronized, and hence , they fire with a similar probability, which implies indeed a binomial distribution of the spike count Δn(t).

Besides Eqs (101) and (102), a third condition concerns the approximation of the integral in Eq (37) by (trapezoidal rule). This approximation is valid if the membrane potential u and threshold ϑ do not vary too strongly during a time step. More precisely, the absolute error of the trapezoidal rule is known to be Δt³|λ′′|/12, which we require to be much smaller than λΔt. Thus, the relative error is of order Δt². Using the definition of λ, Eq (25), this leads to the condition (103) In summary, Δt should be chosen such that all three conditions, Eqs (101)–(103) are satisfied for all populations.

Update of the membrane potential.

To compute the firing probabilities, we need to update both the membrane potential and the threshold. In the presence of an exponential synaptic filter , the membrane potential of free neurons u(t) = h(t) in population α obeys the differential equation (104) (105) Assuming that the external stimulus I_ext(t) and the population activity A_N(t) are constant during one time step, the solution over one time step is given by (106) (107) where h_tot is the total input of population α given by (108) For refractory neurons, we obtain the membran potential in the GLM model by the simple formula u_k(t_l+1) = h(t_l+1) + η(t_l+1 − t_k). For the GIF model, the same update rule as for h(t), Eq (106), can be applied for k = l − K, …, l − k_ref: (109) For the absolute refractory period, l − k_ref < k < l, the membrane potential remains at u_k(t_l+1) = u_r. Note that the total integrated input h_tot needs to be computed only once per time step.

Update of the threshold.

Let us first discuss, how to compute the threshold at time t_l given the values of g_ℓ(t_l) and Δn(t_k) for k = l − K, …, l − 1. For free neurons, the threshold ϑ_free(t_l) is given by Eq (88) evaluated at time t = t_l. For refractory neurons, we find from Eq (90) that the threshold can be written in the discretized form (110) k = l − K, …, l − k_ref. Eq (110) can be rewritten as (111) where the variables can be calculated iteratively starting at k = l − K: (112) with initial condition . Thus, at each time step t_l, the threshold can be rapidly evaluated in one sweep via Eqs (88), (111) and (112).

For the computation of the firing probabilities below, it is necessary to compute the threshold one time step ahead, i.e. at time t_l+1. To this end, we first update the variables g_ℓ according to Eq (89): (113) This yields the threshold ϑ_free(t_l+1) of free neurons via the formula Eq (88). For refractory neurons we find from Eqs (111) and (112) (114) where can be iterated by (115) with initial condition (116)

Firing probabilities.

The firing probabilities for free and refractory neurons are given by (117) respectively. Here, (118) (119) are the arithmetic mean of the respective intensities at the beginning and end of the time interval (cf. Eq (37)). The free intensity λ_free(t) is given by Eq (91). For refractory neurons, the conditional intensities are given by (120) where the last case corresponds to the absolute refractory period.

Population dynamics.

We can directly use the discretized form of the population equations given by Eqs (58) and (59). As in Eq (92) the infinite sums in Eq (59) can be split into an explicit, finite history of length K and a remaining part corresponding to k < l − K. This results in (121) where (122) is the firing probability of “neurons” belonging to the “holes and overshoots” δm_l,k (cf. Results, Sec. “Mesoscopic population equations”). The variables x and z have the discrete time definition (123) corresponding to a discretization of their integral definition above. Having calculated the expected spike count , the actual spike count Δn(t_l) is obtained by drawing a binomially distributed random number (124) as discussed above. In Eq (124), B(N, p_B) denotes the binomial distribution corresponding to N Bernoulli trials with success probability p_B.

The discrete evolution equations for and v_k(t_l) are given by Eqs (48) and (51), respectively, which we repeat here for convenience: (125) (126) To find the update rule for x, we use the definition Eq (123) and the update rule for , Eq (125): (127) Here, we have exploited that P_λ(t_l|t_k) = P_free(t_l) for k < l − K. Using again Eq (125) we find (128) This equation is the discrete analog of the continuous-time Eq (94). Note that is given by Eq (125).

An update rule for z can be found from the definition Eq (123) and the update rule for , Eq (126). A similar calculation that led to Eq (128) results in (129) This equation is the discrete analog of the continuous-time Eq (95). Note that v_l−K(t_l+1) is given by Eq (126).

Finally, the initial conditions can be accounted for by setting (130) The last update step corresponding to the reset of the membrane potential only needs to be performed for the GIF model.

Initialization and storage of history.

One simple way to initialize the system is to fully synchronize the network at time −Δt such that at time t₀ = 0 all neurons are refractory. This gives rise to the sharp initial condition and v_k(0) = u_k(0) = 0 for the refractory epoch (k = −K, …, −1). Here, δ_k,l denotes the Kronecker delta, which is unity for k = l and zero otherwise. After synchronization there are no free neurons, hence x(t₀) = z(t₀) = 0 and, if there were no further spikes in the past, g_ℓ(t₀) = 0 for ℓ = 1, …, N_θ. The initialization of g_ℓ corresponds to a zero adaptation level at the beginning of the simulation.

For the representation of the variables , v_k(t_l), u_k(t_l) and λ_k(t_l), k = l − K, …, l − 1, in memory, it is convenient to employ circular buffers. That is, the “running” range of the explicit history k = l − K, …, l − 1 is mapped to a static range in memory by applying the modulo operation (131) to all temporal indices.

Summary of the update step and pseudocode.

Let us summarize the steps needed to evolve the population equation from time t_l to time t_l+1:

1. Calculate the total integrated input h_tot using Eq (108) and then update the synaptic variables y^αβ(t_l+1) according to Eq (107).
2. Update the free membrane potential h(t_l+1) and threshold variable g_ℓ(t_l+1) for free neurons using Eqs (106) and (113) and use these values to compute the threshold ϑ_free(t_l+1) and conditional intensity λ_free(t_l+1) of free neurons by means of Eqs (88) and (91). This yields the firing probability of free neurons P_free(t_l) via Eqs (117) and (118).
3. For all refractory states k = l − K, …, l − k_ref, compute the membrane potential u_k(t_l+1) from Eq (109), the threshold ϑ_k(t_l+1) from Eq (115) and the conditional intensity λ_k(t_l+1) from Eq (120). The firing probabilities P_λ(t_l|t_k) are then given by Eqs (117) and (119).
4. Calculate the effective firing probability P_Λ(t_l) from Eq (122).
5. Calculate the expected activity by Eq (121). The empirical population activity Δn(t_l) can be obtained by drawing a binomially distributed random number according to Eq (124).
6. Update the mean and variance of the survival numbers , v_k(t_l+1), x(t_l+1) and z(t_l+1) using Eqs (125), (126), (128) and (129).
7. Realize the boundary conditions at according to Eq (130).

These steps have to be performed for all populations α = 1, …, M. A detailed implementation of the algorithm is provided by the pseudocode shown in Figs 11 and 12.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 11. Pseudocode for the integration of the mesoscopic population equation.

Note that procedure UpdatePopulation in line 12 is shown in Fig 12.

https://doi.org/10.1371/journal.pcbi.1005507.g011

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 12. Pseudocode for the update of the variables of a given population.

Note that the adaptation kernel θ_k′ ≡ θ((K − k′)Δt), the quasi-renewal kernel , Eq (33), as well as the exponentials and can be precomputed.

https://doi.org/10.1371/journal.pcbi.1005507.g012

Whereas the complexity of the microscopic simulation is of order O(NΔt⁻¹), the integration of the population equation is of order O(Δt⁻²) because in each time step one has to update a history of length K = T/Δt (step 3, 4 and 6). Hence, at low neuron numbers (e.g. N < 100), the direct simulation of the microscopic system may become more efficient. We emphasize, however, that to achieve a comparable accuracy, the integration of the mesoscopic population equation can be performed on a coarser, millisecond time scale (e.g., Δt = 1 ms), whereas the microscopic simulation requires precise spike times and hence a sub-millisecond simulation (e.g., Δt = 0.1 ms). If we take advantage of this fact, the mesoscopic population model performs well even at low neuron numbers.