Mesoscopic description of microscopic network dynamics

We study the dynamics of a network of N spiking neurons that, on the microscopic level, are modeled as linear-nonlinear-Poisson (LNP) neurons [47–49] with dynamic synapses [44, 50]. The network of LNP neurons without dynamic synapses is also referred to as a multivariate nonlinear Hawkes process [51, 52] in the mathematical neuroscience literature. The LNP model consists of a cascade of three steps: a temporal linear filter acting on the synaptic input, a static nonlinearity, which accounts for nonlinearities of the output firing rate, and a stochastic spike generation mechanism. While the simplicity of the LNP model enables us to derive mesoscopic dynamics with spiking noise and dynamic synapses, we note that the model does not capture spike correlations due to refractoriness and other spike-history effects [47]. The synaptic dynamics is given by the Tsodyks-Markram model of short-term plasticity (STP) [23]. For simplicity, we focus here, in the Results part, on the special case where the synaptic dynamics corresponds to pure depression [53] and the linear filter of the LNP model corresponds to a leaky integration of the input [54]. The general theory for the full Tsodyks-Markram model with depression and facilitation as well as the straightforward extension to general linear filters, enabling biologically more realistic neuronal dynamics [54], is provided in the Methods part.

Specifically, each LNP neuron i = 1, …, N is characterized by an input potential h_i(t), which can be loosely interpreted as its membrane potential at time t. Given the current value of h_i(t), neuron i emits a spike independently with state-dependent hazard rate f(h_i(t)), also known as conditional intensity [55]. Here, f is a non-negative, increasing function that represents the static nonlinearity of the LNP model (see Eq (6) below for a concrete choice of f). Thus, the conditional probability that neuron i fires a spike in an infinitesimally small time interval [t, t + dt) given the input potential h_i(t) is (1a) where t⁻ denotes the time just before t. The corresponding spike train is the sum of Dirac delta pulses (1b) where are the spike times of neuron i. Mathematically, these spike times are a realization of a stochastic point process with conditional intensity f(h_i(t⁻)). In numerical simulations of networks of N LNP neurons, in each small time step [t, t + Δt), we draw N independent Bernoulli random variables z_t,1, …, z_t,N ∈ {0, 1} with success probabilities f(h₁(t⁻))Δt, …, f(h_N(t⁻))Δt, respectively. If z_t,i = 1, then neuron i emits a spike at time t.

We further consider fully-connected networks with homogeneous connectivity and depressive synapses: each time neuron i emits a spike, all the input potentials in the network make a jump proportional to , where x_i(t⁻) is the amount of synaptic resources available at the outgoing synapses of neuron i just before its spike. In contrast to standard leaky integrate-and-fire models [49], the potential h_i(t) is not reset to a fixed value after each spike of neuron i. Together with Eqs (1a) and (1b), the microscopic network model with LNP neurons and synaptic short-term depression (STD), in short LNP-STD model, reads (1c) (1d)

Here, μ(t) represents a common external current mimicking, e.g., feedforward input from other areas, and τ can be interpreted as the membrane time constant. The synaptic parameters are given by the overall synaptic weight factor J, the relative depletion of neurotransmitter by a single transmitted spike U₀ and the time scale of synaptic depression τ_D. Note that h_i(t) = h_j(t) for all t ≥ 0 and for all i and j if at time 0, all the h_i share the same initial condition.

From a mathematical point of view, the microscopic model (1) can be rigorously interpreted as a system of Poisson-noise-driven stochastic differential equations with solutions taken in the sense of Itô.

Diffusion model of the mesoscopic dynamics (Gaussian noise).

Our goal is to derive a mean-field model for the microscopic dynamics, Eq (1), that accounts both for the finite number of neurons as well as for the dynamic synapses undergoing Tsodyks-Markram STP, see Fig 1. The mean-field description will be based on the dynamics of the following mesoscopic variables defined as the empirical averages (2)

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. From microscopic to mesoscopic population dynamics.

(A) Network with microscopic short-term plasticity. Dashed region shows a zoom into a pair of interconnected neurons: presynaptic neuron 1 sends out an unmodulated spike train to postsynaptic neuron 2 that receives the spike-train modulated by short-term depression. (B) Mesoscopic mean-field model with one effective synapse undergoing short-term depression.

https://doi.org/10.1371/journal.pcbi.1010809.g001

The desired dynamics of h(t), x(t) and Q(t) are supposed to no longer depend on (the index i of) individual neurons, so we will approximate terms such as, e.g., the sum , by a diffusion term which only involves the mesoscopic variables. To this end, we follow the temporal coarse-graining approach by Gillespie [56] for the derivation of a “chemical Langevin equation”, see the Methods section for a detailed derivation. In brief, we first use a macroscopically infinitesimal time step Δt [56] and approximate the coarse-grained sum by a Gaussian random variable with variance proportional to Q(t⁻). In a second step, we derive the dynamics of Q(t), discarding the fluctuations whose effect on h(t) and x(t) is of order N^−3/2. The resulting mesoscopic mean-field dynamics is given by the diffusion model (3a) (3b) (3c) where ξ(t) is a Gaussian white noise with auto-correlation function 〈ξ(t)ξ(s)〉 = δ(t − s). Although it is possible to deduce Eq (3) from the detailed doubly-stochastic mesoscopic dynamics derived in [44] (see Methods), the derivation summarized above and presented in Methods Section “Diffusion approximation for the mesoscopic dynamics with short-term depression” is much simpler as it relies on a direct application of the diffusion approximation (avoiding the detour via the model presented in [44]).

The stochastic differential equation (SDE) (3), as all the SDEs in this work, is integrated in the sense of Itô. Numerically, this means that for each integration time step, the factor multiplying the noise is taken at the time just before the random increment.

Microscopic vs. mesoscopic dynamics of a single population exhibiting metastability

The mesoscopic descriptions Eqs (3) and (4) of the full network of N interacting spiking neurons with short-term depression (STD) effectively reduce the high-dimensional microscopic dynamics Eq (1) to a system of three stochastic differential equations in h, x, and Q. In the limit N → ∞, finite-size fluctuations in the mesoscopic dynamics vanish and the variable Q becomes superfluous. The two-dimensional macroscopic dynamics Eq (5) readily allows for a comprehensive phase-plane analysis, see, e.g., [16, 46], which reveals the deterministic backbone of, and can therefore yield theoretical insights about, the full network dynamics.

To demonstrate the high accuracy of our mesoscopic description and also its usefulness for studying the effect of finite-size fluctuations on metastable dynamics, we will focus in this Section on two traditional examples of metastability in a single excitatory population (J > 0) of LNP-STD neurons: populations spikes and spontaneous transitions between Up and Down states. For reasons of comparability with previous models [16, 21], we assume in this paper that the static nonlinearity of the microscopic LNP model (1) has the form (6) with slope parameter r, smoothness a, and threshold h₀. The function f(h) has an exponential sub-threshold tail and a linear supra-threshold part [21, 57]. In the limit a → 0, f(h) = r[h − h₀]⁺ becomes a threshold linear function with slope r and threshold h₀ [16]. The larger a > 0, the smoother the transition at the threshold.

Population spikes in an excitatory population.

As a first example, we study the emergence of spontaneous bursts of synchronized activity due to finite-size fluctuations and short-term depression (Fig 2). To this end, we tune the parameters of our model such that the macroscopic dynamics for N → ∞ exhibits a unique stable fixed point (red dot in Fig 2A) together with a pair of unstable fixed points (orange, green). In the absence of external inputs or internal finite-size fluctuations, the system will remain in the stable, low-activity state forever. This state, however, is excitable: Fluctuations can lead to rapid, transient excursions of the neural trajectory, when the system is kicked across a separatrix (red-dashed curve = stable manifold emanating from the unstable saddle point (orange diamond)), see the blue traces in Fig 2A. During an excursion along the unstable manifold of the saddle point (orange-dotted), the input potential h(t), and with it the population firing rate f(h), rapidly increases, which corresponds to a short synchronized burst of activity. The increased firing of spikes leads to a strong suppression of the depression variable x, which in turn pulls the firing rate down. Once the depression variable x(t) has recovered sufficiently, finite-size fluctuations can again trigger a synchronized burst of activity (Fig 2B and 2C). Such bursts of activity, called population spikes, have been studied theoretically in the context of STP [44, 58–60] and have also been observed experimentally [60, 61].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Population spikes in excitatory populations of finite size.

(A) Phase-plane analysis of the macroscopic model (Eq (3) for N → ∞) reveals the backbone of the metastable dynamics due to the proximity of a separatrix (red-dashed) near the unique stable fixed point (red dot = cross-section of the black-dashed nullclines). Trajectories (blue) of the mesoscopic model reproduce population spikes by following the unstable manifold (orange dotted line) of the saddle fixed point (orange diamond). Population spikes have variable amplitude and inter-population spike intervals (ISI), see also (B,C). (D) The mesoscopic models with hybrid noise (jump-diffusion model; blue) and Gaussian noise (diffusion model; orange) accurately capture finite-size fluctuations in the input potential h—note the logarithmic y-scale—and population spikes of the microscopic network dynamics (black) of N = 30 neurons. (E) Power spectra of the input potential h and (F) ISI distributions coincide for all three models. (G-H) same as (D-F) for N = 200. Statistics are for simulations of length T_sim = 100′000s. Model parameters can be found in Table 1.

https://doi.org/10.1371/journal.pcbi.1010809.g002

As predicted by the mesoscopic model, the corresponding microscopic network model also exhibits population spikes with a similar rate, and this rate increases when the network size decreases (Fig 2D and 2G). Quantitatively, the mesoscopic model accurately reproduces the statistics of population spikes in the microscopic model: There is an excellent match between the power spectra (Fig 2E and 2H) and between the distributions of the inter-population spike intervals (Fig 2F and 2I). The slight deviations of the diffusion model Eq (3) with Gaussian noise (orange, dashed traces) for a network of N = 30 neurons disappear for a network of N = 200 neurons: as expected, the diffusion approximation becomes better with increased network size. Remarkably, the jump-diffusion model Eq (4) with hybrid noise (both Gaussian and Poisson; blue traces) perfectly matches the microscopic network dynamics even when N = 30. Given the close correspondence between microscopic and mesoscopic models, the mechanism of excitable dynamics driven by finite-size noise found above for the mesoscopic model thus explains the emergence of population spikes in the microscopic network model. Note that the population spikes are endogenously generated without the need for external (noisy) inputs as in previous microscopic [44, 58] and mesoscopic [44, 60] models.

Up-Down dynamics in an excitatory population.

In a second example, we change the model parameters slightly so that our system now exhibits two co-existing stable fixed points: a high-activity “Up” state and a low-activity “Down” state. In the macroscopic model Eq (5), which corresponds to the one studied in [16] in the absence of noise, only one of the two states can be realized depending on the initial conditions. In the mesoscopic models, Eqs (3) and (4), however, finite-size fluctuations lead to irregular transitions between Up and Down states. An exemplary stochastic trajectory in Fig 3A starts close to the Down state, but soon gets kicked across the separatrix (red-dashed stable manifold of the (orange) saddle fixed point), from where it follows the (orange-dotted) unstable manifold and undergoes a sharp excursion in phase-space, resembling a population spike as described in the foregoing section. On its way back to the stable Down state, the trajectory approaches the unstable limit cycle (green dashed) that acts as the boundary of the basin of attraction of the Up state. Finite-size fluctuations can induce attractor hopping: from the low-activity node (Down state), the trajectory can cross the basin boundary and starts spiraling into the high-activity focus (Up state), until it crosses the basin boundary again and converges towards the low-activity node (Down state), see also Fig 3B and 3C. The seemingly ongoing oscillations in the Up state are a pure finite-size effect, which will be damped out in the macroscopic model. As an aside, the frequency of the oscillations in the Up state coincides with the imaginary part of the eigenvalue of the high-activity focus, cf. [46].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Up-down dynamics due to finite-size fluctuations.

Mesoscopic model reproduces noisy bistable population dynamics. (A) Phase-plane analysis of macroscopic dynamics (Eq (3) for N → ∞) reveals two stable fixed points (red): a high-activity focus representing the Up, and the low-activity node the Down state of the system. From the saddle fixed point (orange diamond), an unstable (orange dotted line) and a stable manifold (red dashed line) emerge. The latter acts as a separatrix—trajectories (blue curve) starting from above make an excursion around the unstable limit cycle (green dashed) and converge towards the down state. Finite-size fluctuations can make the trajectory cross the limit cycle into the basin of attraction of the Up state. (B,C) Stochastic trajectory of the mesoscopic dynamics (3) with N = 100 transitioning between Down and Up states. (D) The mesoscopic models with hybrid noise (jump-diffusion model; blue) and Gaussian noise (diffusion model; orange) qualitatively capture Up-Down-dynamics of the microscopic network (black). (E) Power spectrum and (F) histogram of input potential h over simulation of length T_sim = 100′000s. Model parameters can be found in Table 1.

https://doi.org/10.1371/journal.pcbi.1010809.g003

To assess the accuracy of our mesoscopic description of this finite-size induced metastable regime, we performed extensive simulations and compared them to the microscopic network Eq (1). In Fig 3D, we show exemplary time series of the network dynamics for N = 100 neurons of the jump-diffusion model Eq (4) with hybrid noise (blue), of the diffusion model Eq (3) with Gaussian noise (orange) and of the microscopic model (black). Qualitatively, there is an excellent agreement between micro- and mesoscopic simulations. However, closer inspection of the time series reveal that the Up states in the diffusion model are, on average, of shorter duration than in the microscopic and the jump-diffusion model. This slight shortcoming of the diffusion model also becomes evident when looking at the power spectrum and the bimodal distribution of the input potential h(t) computed over a long simulation of T_sim = 100′000s (Fig 3E and 3F, respectively). The jump-diffusion model perfectly captures the full statistics of the microscopic network, but the diffusion model slightly underestimates the time spent in the Up states (see the zoom in Fig 3F), which also manifests in small deviations of the power spectrum.

Mesoscopic model for hippocampal replays

We now turn to a more complex biological example for metastability in neural circuits: the spontaneous replay of activity sequences across hippocampal place cells [62, 63]. Sequential activation patterns of place cells have been widely observed in experiments when an animal explores its environment [64–67] and have been related to neural representations of animal trajectories, which may subserve navigation and spatial learning [68–71]. Once an internal representation, or map, unique to one environment [72] is formed, it can later be replayed spontaneously in the absence of sensory input—a feature that is believed to contribute to memory consolidation and retrieval [73–75] as well as to route planning [76, 77]. Spontaneous replay occurs within so-called sharp waves (SW)—i.e. population bursts of co-active pyramidal cells that give rise to large amplitude extracellular waves— during quiet wakefulness [78–80] and sleep [81, 82], typically has a much faster, compressed time scale [78, 82, 83] and the replayed trajectories can either be in the originally experienced order or backwards [84]. Spontaneous replay events appear and disappear abruptly and repeatedly, and can therefore be regarded as metastable states separated by states of low activity. In the following, we extend our mesoscopic theory to a ring-attractor model consisting of multiple neuronal populations in order to provide a mechanistic description of hippocampal replay with a direct link to microscopic networks of spiking neurons.

A circular environment.

We first consider a single circular environment and assume that, after exploration, the animal has formed an internal representation (map) of the corresponding environment, which is encoded in the synaptic connectivity of the place cells in hippocampal area CA3. Accordingly, we assign to all neurons in population α ∈ {1, …, M} a place field at angle θ_α = 2πα/M, so that the place field locations are equally spaced on a ring, see Fig 4A. The synaptic strength J_αβ depends on the distance between locations according to (9) where J₁ scales the strength of map-specific interactions and J₀ corresponds to uniform feedback inhibition. For J₁ > J₀ ≥ 0, populations with adjacent place fields excite each other, whereas populations with place fields far apart from each other are inhibitory. This form of symmetric interaction is known to generate spatially coherent activity, leading to so-called bump attractors [86–89]. STD has been shown to destabilize stationary bumps that move around the environment [90], creating burst-like nonlocal traveling wave events (NLEs) that resemble hippocampal replay patterns on a population level.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Hippocampal replay in micro- and mesoscopic ring-attractor network model.

(A) Ring-attractor model of M population units of N LNP spiking neurons with STD. Synaptic weights J_α,β are excitatory for units with nearby place field positions θ_α and inhibitory at longer distances, see the coupling function on the right. (B) Mesoscopic and (C) microscopic network simulations reveal (i) spontaneous bursts of the averaged activity, resembling SWs, during which replay patterns evolve as (ii) metastable traveling waves, or nonlocal replay events (NLE), along the circular environment—the expected activity r_j = f(h_j) at location θ_j = 2πj/M, j = 1, …, M is color-coded. Statistics of the (D) mesoscopic and (E) microscopic simulations perfectly match each other with respect to: (i) the distribution of event duration, (ii) the correlation between the number of peaks per burst and its duration, (iii) the correlation between the length of the traveled path during an event and its duration (the red curves in panels (ii,iii) are linear regression curves), as well as (iv) the distribution of average bump speed during an event, computed from events with more than one peak; see Methods for more details, Table 1 for model parameters and Table 2 for simulation results.

https://doi.org/10.1371/journal.pcbi.1010809.g004

In our network simulations, we consider M = 100 populations around the ring with N = 50 neurons each. The external input is homogeneous, i.e. μ^α = μ for all α = 1, …, M. As shown in Fig 4Bi, the network spontaneously generates a burst of elevated activity with one or more peaks, that lasts up to a few hundred milliseconds and is then terminated due to STD. Another burst is generated only after a recovery period—the so-called interburst interval (IBI). The intermittently elevated activity on the ring is strongly localized because of the synaptic connectivity. STD makes the localized bump move to neighboring place field locations and, thus, gives rise to an NLE, Fig 4Bii; we define an NLE (= nonlocal replay event) as a burst of the averaged activity with more than one peak, see also Methods. As the elevated activity locally alters the spatial profile of the slow synaptic depression variable, highly irregular activity patterns emerge with bursts that start at seemingly random locations, travel in either backward or forward direction, and vary both in duration and distance (note, however, that a single wave of activity never travels further than once around the circle). This type of metastable dynamics strongly resembles the behavior observed by Romani and Tsodyks in their deterministic firing rate model [21], hereafter referred to as the RT model, but with the important difference that here the emergence of metastability is a finite-size effect—increasing the population size from N = 50 to N = 5000 renders any initiation of burst-like activity impossible (see the orange curve in Fig 4Bi). Put differently, our model reveals a novel dynamical regime, in which metastable burst states are fluctuation driven, in contrast to the RT model, where bursts are induced deterministically when depression slowly abates (for a similar mechanism of noise-induced traveling waves, see [91]).

Comparing the simulations of our mesoscopic model with the microscopic network, we find an excellent agreement both from a qualitative (Fig 4B and 4C) and from a quantitative perspective (Fig 4D and 4E), where we follow the statistical analysis of [21], see also Methods. For comparison, we considered two deterministic models with fatigue-induced bursts: (i) the original RT model [21], and (ii) our model with N → ∞ and slightly increased external drive (μ = −0.9 instead of μ = −1.4). This second model, which will be referred to as the macroscopic model in the following, essentially reproduces the dynamics of the original RT model [21]. We included this second model in the comparison (see also Fig A in S1 Text) because the macroscopic limit N → ∞ of our mesoscopic model yields slightly different model equations compared to the heuristic RT model. A closer inspection of the statistical properties of NLEs and IBIs reveals that mesoscopic and microscopic simulations not only match almost perfectly, but the fluctuation-induced bursts may also have more biologically realistic properties than the fatigue-induced regime. For example, experiments in rodents exposed to long linear tracks [83, 92] had an experimentally estimated number of ∼10 SWs/s. Assuming that each peak in the average activity corresponds to one SW, the micro- and mesoscopic models closely match the experimental observations with 9.3 SWs/s in contrast to less than 8 SWs/s in the macroscopic and original RT model, see Methods for more details.

Furthermore, our fluctuation-driven model reveals larger temporal variability with a unimodal IBI distribution (Fig 5A) and low serial correlations of the event speeds (Fig 5B). In marked contrast, in the macroscopic and RT models of fatigue-induced bursts, the IBI distributions are bimodal, and clearly not exponentially distributed as observed experimentally [62, 93, 94] and expected for a Poisson process. By contrast, our fluctuation-driven model shows the tendency towards exponential IBI distributions with longer tails, showing in particular a larger mean (0.65 s vs. 0.29 s) and a higher coefficient of variation (CV = 0.85 vs. 0.79). Moreover, the macroscopic and RT models exhibit strong correlations between forward and backward replay events as seen in the serial correlations of the event speed (Fig 5Bii). The alternating structure of the serial correlation coefficient with strong anti-correlations at lag 1 means that forward and backward motion alternate almost perfectly. In contrast, the motion directions in the sequence of NLEs in our fluctuation-driven model are almost uncorrelated (Fig 5Bi). Another difference to the deterministic models is that the onset location of the fluctuation-induced NLEs is independent of the offset location of the preceding event. By contrast, in the deterministic fatigue-induced (macroscopic and RT) models, the activity bursts start at the location where the slow depression variable has had most time to recover, leading to more regular event patterns (see also [21] and Fig A in S1 Text).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Comparison of fluctuation-induced and depression-induced hippocampal replay dynamics.

(A) Interburst-interval distributions and (B) serial correlations of the event speeds of consecutive NLEs for (i) the meso- and microscopic models and (ii) the deterministic models (green: original Romani-Tsodyks model [21], purple: macroscopic model in the fatigue-driven regime obtained by setting N^α → ∞ and μ = −0.9 (instead of μ = −1.4)). (C) On shorter time scales, all models capture the discontinuous nature of the replayed trajectory: (i) Single NLE of the mesoscopic model. (ii) Place field positions decoded using PVA in bins of 50ms (black dots). This “replayed trajectory” deviates from a straight line (red-dashed) corresponding to a hypothetical uniform motion. (iii) Increments of the movement trajectory exhibit strongly irregular movement features (black dots) in contrast to the constant increments expected for a straight line (red-dashed). (D) (i): The distributions of reconstructed step-sizes of the meso- (blue) and microscopic models (gray histogram) coincide and strongly deviate from the narrow distributions of average step sizes (computed for each NLE by fitting straight lines to individual movement trajectories) for the meso- (red) and microscopic models (gray thin line). The average step sizes vary for different NLE’s, causing the non-zero width of their distribution. (ii) Similar behavior is observed for the macroscopic model (purple; with increased external input μ = −1.4 ↦ −0.9) and the Romani-Tsodyks (RT) model (green histogram); red/gray thin lines correspond to average step sizes in the macro/RT-models, respectively. See main text and Methods for more details.

https://doi.org/10.1371/journal.pcbi.1010809.g005

On shorter time scales, all models (micro-, meso-, macroscopic and RT) exhibit the experimentally observed discontinuous nature of replay events [95]. In Fig 5Ci, we zoom into one exemplary NLE of around 350ms, during which a metastable wave travels around the ring in backward direction. The place field activity (color coded) varies critically in location and activity. Binning activity per location in moving 50ms windows, we estimate the animal’s (hypothetical) position along the ring by applying the population vector average (PVA; [96]), see Fig 5Cii. The decoded trajectory (black dots) does not follow a smooth path, but the step sizes vary irregularly in length (Fig 5Ciii). In Fig 5D, we show the step-size distributions for the four different models, featuring a large number of very short steps and a long tail of larger steps. The broad distributions significantly differ from the narrow step-size distribution that would be expected if the movement trajectory was uniform (thin lines in Fig 5D). More precisely, the latter distribution describes the variation of the average step sizes (computed for each NLE) across different NLEs, which corresponds to approximating each replay trajectory by a straight line (red-dashed curves in Fig 5C), see also [95] for more details on computing the step-size distributions.

In conclusion, the mesoscopic model Eq (8) perfectly recovers the metastable replay dynamics of the microscopic network model Eq (7). While our fluctuation-driven model is similar to the deterministic models with respect to the structure of single replay trajectories (Fig 5C and 5D), the fluctuation-driven model features unimodal IBI distributions and low serial correlations of motion directions, in marked contrast to the bimodal IBI distribution and the strong serial correlations of the deterministic fatigue-induced replay model. Thus, the IBI distribution and the sequence of motion directions provide experimentally testable predictions that may be useful to disentangle the contributions of deterministic and stochastic sources of metastable hippocampal dynamics.

Multiple circular environments.

In a next step, we assume that an animal has internalized multiple environments. The ability to code for spatial locations in multiple environments is considered one of the hallmarks of place cell activity in the hippocampus. Experimental results have shown that rodents, when exposed to two distinct environments of similar shape, most place cells are active in only one environment. However, a few place cells are active in both environments but they typically exhibit place fields at different spatial locations, which is referred to as global remapping [97, 98]. Replay events can then be observed in both neural maps corresponding to each of the two environments [79], see also [99, 100].

Following [21], we consider K circular environments and store their respective maps within the synaptic connectivity J_α,β of the network model Eq (7). To this end, we endow each population α with a binary vector of selectivities for these K environments, , where indicates that the neurons in population α are selective for environment k ∈ {1, …, K} (i.e., the neurons contribute to the encoding of this environment) [21, 101]. Otherwise, if , population α is not selective for environment k. Selectivity to particular environments is assigned randomly, but with the constraint that with f ∈ [0, 1], i.e. exactly fM populations are selective for environment k. Furthermore, we introduce place field locations for each environment k and randomly assign a unique place field angle to each of the fM populations selective for environment k. We then define the synaptic weights as (10) where map-specific interactions of strength J₁ only occur within environments, and J₀ represents uniform feedback inhibition as before [21]. In our simulations, we use K = 3, f = 0.3 and M = 300, and find parameters of synaptic strength J₀ and J₁ as well as the homogeneous external input μ such that replay events are again a pure finite-size effect: When increasing the population size from N = 50 to N = 500, the network remains in a quiescent state and bursts of elevated activity no longer occur in our simulation.

As can be seen in Fig 6, bursts of elevated activity occur spontaneously and with high temporal variability. During these nonlocal replay events (NLE) a metastable traveling wave state is generated randomly in one of the three stored environments; activity in the other environments is suppressed due to global inhibition. As expected, our meso- and microscopic network simulations show qualitatively very similar behavior (cf. Fig 6A and 6B). The metastable dynamics exhibit high variability with respect to the duration of NLEs and the interburst intervals (Fig 6E), the length of the traveled path during a NLE within the active environment, as well as the order of environment activation.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Spontaneous replay switches between multiple environments.

In the (A) mesoscopic and (B) microscopic ring-attractor network storing multiple environments, metastable replay dynamics spontaneously emerge due to finite-size fluctuations when decreasing the population size from N = 500 (orange/red in panels i) to N = 50 (blue/black) per unit. Nonlocal replay events (NLEs) occur randomly in exclusively one of three environments, while activity in the respective other two is suppressed. The resulting activation sequences of replayed environments—in (A) the activation sequence reads 313123122312321—are analyzed with respect to (C) the transition probabilities between subsequently active environments and (D) sequential activation patterns. In the meso- and microscopic models, transitions from environment k to j are equally likely for all pairs (k, j) ∈ {1, 2, 3}². But the deterministic (macro and RT) models show a clear preference for transitions 1 → 3 → 2 → 1, which is also apparent in the high probability of the corresponding subsequences of three distinct, subsequently active environments. (E) Larger heterogeneity with respect to NLE duration and interburst intervals (IBI), as assessed by the respective means μ and coefficients of variation CV, further distinguishes the more variable metastable regime of the micro-/mesoscopic vis-à-vis macroscopic/deterministic models. Model parameters can be found in Table 1.

https://doi.org/10.1371/journal.pcbi.1010809.g006

In more detail, we statistically analyzed the patterns of sequential activations. For instance, in Fig 6A the order of environment activation reads 313123122312321. To quantify whether replay of distinct environments, i.e. their order of activation, is random or correlated, we first computed the transition probabilities between environments. While the stochastic (meso- and microscopic) models did not show strong preference for any environment transition, in the deterministic (macroscopic and RT) models, preferred transitions were clearly visible (Fig 6C). Moreover, we checked whether particular sequences of recalled environments are more probable than others, which may point at some (hidden) deterministic origin of sequence generation and recall. In order to avoid spurious deterministic effects that may be inherited from asymmetries in the synaptic weights, we constructed the selectivity vectors ζ_α symmetrically and guaranteed that bursts within the environments were equally distributed, see Table 3 in the Methods section. The deterministic (macroscopic and RT) models, nonetheless, exhibited a strong preference for specific order of environment activation (3 → 2 → 1 → 3 in the example of Fig 6). By contrast, the stochastic models did not show any preference for a particular order, Fig 6D, which underlines once more the strong variability in the finite-size induced metastable dynamical regime of hippocampal replay.

Modeling population spikes and Up-Down dynamics

In the modeling literature, theoretical models of metastability are typically based on an interplay between the network’s tendency to posit itself in a self-excitable dynamical regime and some fatigue mechanism that generates activity-dependent self-inhibition in response to elevated network activity [102, 103]. Such a fatigue mechanism can be implemented in neural networks via neural spike-frequency adaptation (SFA) or via synaptic short-term depression (STD). Here, we focused on STD, but acknowledge that similar behavior can, in principle, also be achieved with SFA. One possibility of self-excitability is that a stable low-activity state of asynchronous activity is close to a Hopf bifurcation, at which it becomes destabilized in favor of stable global oscillations. In the subcritical regime, noise can promote transient departures from the fixed point, resembling populations spikes [60]. Another possibility is that the system exhibits two stable fixed points, a high-activity (Up) and a low-activity (Down) state. Switching between the states can be induced by internal or external noise. In addition, the Up state can get destabilized in a saddle-node bifurcation by an adaptive fatigue mechanism described above [18, 20, 104, 105]. These scenarios are effectively low-dimensional and, in consequence, firing rate models describing the mean population activity can successfully explain metastable dynamics. As noted in the Introduction, however, the firing rate models largely miss a clear link to microscopic circuit models. While neuronal and synaptic properties can be partly accounted for by mean-field models, incorporating biologically realistic fluctuations in such models is more difficult as it often lacks a rigorous footing. Neglecting fluctuations may explain purely deterministic, fatigue-induced metastable activity patterns with low variability. Experimentally observed metastability in the brain, however, shows larger variability suggesting fluctuation-induced metastable dynamics. Fluctuations can have manifold origins that range from external noisy (cortical or thalamic) inputs or background noise [106], via specific network connectivity topologies (random, sparse, or clustered) [107–111], heterogeneity [112], (loose or strong) balance between excitation and inhibition, up to finite-size effects [60, 113, 114], or even a combination of them [115, 116]. On the mesoscopic scale, such fluctuations are often modeled heuristically by adding noise terms to the mean-field equations ad hoc.

With the mesoscopic mean-field model that we have proposed here, we restricted ourselves to explaining the fluctuations observed on a mesoscopic population level that are due to a finite number of stochastic neurons. To minimize confounding factors, we considered just one excitatory population of Poisson spiking neurons, all-to-all coupling, and no external noise. Indeed, it has been shown that inhibition is not necessarily needed to generate population spikes and Up-Down transitions, neither is short-term synaptic facilitation needed; see [16, 45, 46] who used a mean-field model with STD and Gaussian white noise in the voltage dynamics. Building on their previous work and complementing that mean-field model by an additional facilitation dynamics, Holcman and co-workers more recently provided an improved, and analytically tractable, description of network bursts, Up-Down dynamics and slow oscillations that is compatible with experimental recordings on various scales [117, 118] and allows for detailed stochastic analysis [119–121]. In the Methods, we extend our mesoscopic description considering also short-term facilitation. Our mesoscopic mean-field model Eq (3) yields exactly the same deterministic dynamics of their depression(-facilitation) mean-field model in the limit N → ∞, see also [122]. The important difference, however, is how to deal with noise. While Holcman, Tsodyks and co-workers rather vaguely motivated an additive Gaussian white noise term that is meant to represent the fluctuations from independent vesicular release events and/or closings and openings of voltage gated channels, we here provide an explicit and rigorous derivation of the multiplicative noise terms in our mesoscopic mean-field model. Our model can thus accurately account for the finite-size fluctuations of the microscopic network including the thereby induced heterogeneity of synaptic depression across the neurons.

Previous approaches to model finite-size effects in networks with STP included a multiplicative noise term ad hoc in the firing rate equation [123, 124]. By contrast, we here derived Langevin equations directly from the underlying microscopic network of finitely many LNP spiking neurons. In our bottom-up approach we explicitly take into account fluctuations due to the variability of the individual depression variables across synapses, that are typically neglected in the other approaches. Our resulting mesoscopic model remains nonetheless simple enough to allow for an efficient analysis of finite-size induced metastable dynamics in the presence of a slow fatigue mechanism in form of STD.

Diffusion- vs. jump-diffusion model: It depends on network size and dynamical regime

For a single population of LNP-STD neurons, we showed that microscopic network simulations were accurately captured by our mesoscopic description. In general, the agreement with the jump-diffusion model Eq (4) was excellent both for the population spike and the Up-Down states dynamics. The fit between the diffusion model Eq (3) and the microscopic network became better, the larger the network size N in line with the diffusion approximation requiring a rather large N. At least this is true concerning the population spikes dynamics (Fig 2). In the Up-Down dynamics, however, even for intermediate network sizes of N = 100 neurons, deviations between microscopic simulations and the diffusion model still prevailed, see inset in Fig 3F. An increase in network size hardly led to a better fit. On top of that, a larger N resulted in finite-size fluctuations with smaller amplitude, which made attractor hopping between Up and Down states more difficult and less frequent, and thus required longer simulation times. The origin of the deviations can be found in the phase plane shown in Fig 3A: the unstable manifold of the saddle fixed point (orange dotted line) leads the neural trajectory close to the basin boundary (green dashed) of the Up state only in a small region of the phase space. In this region, fluctuations need to perturb the neural trajectory in a particular direction so that it can enter (and then also remain within) the basin of attraction of the Up state. The jump-diffusion model actually recapitulates the correct fluctuations, whereas noise in the diffusion model is too diffusive. This discrepancy can already be anticipated from the Q- and y-dynamics in Eqs (3) and (4), respectively, which directly influence the finite-size fluctuations. In fact, the steady state profiles of Q and y predict that the fluctuations are strongest for intermediate firing rates f(h) and depression levels 0 < x < 1, that is, right in the aforementioned critical region of the phase space, where a delicate balance between Poisson and Gaussian noise is important to recover the microscopic network dynamics. Consequently, the accuracy of the simpler diffusion model Eq (3), and hence its choice over the more complex jump-diffusion model Eq (4), to describe the microscopic network dynamics depends not only on the network size N, but also on the dynamical regime under investigation.

In the case of the hippocampal replay dynamics in the ring-attractor model with multiple populations of LNP-STD neurons, we saw a very similar effect. For fatigue-induced hippocampal replay, the actual characteristics of the finite-size fluctuations were less important than the deterministic backbone provided by the spatio-temporal profile of the slow fatigue (depression) variable. Hence, the diffusion model was able to accurately capture the metastable replay dynamics of the microscopic ring network. By contrast, for the chosen parameter set in Fig 4, hippocampal replay patterns were finite-size induced and vanished completely for large network sizes. As such, capturing the finite-size noise properties of the microscopic network was necessary for observing the NLE dynamics. The finite-size induced hippocampal replay dynamics could be recovered by both mesoscopic model versions, albeit the jump-diffusion model yielded a better quantitative agreement.

Network size variation resembles long-term plasticity

Our study of the Up-Down-dynamics has been guided by the work of Holcman, Tsodyks and co-workers [16, 45], who considered a firing rate model with external stochastic input as a necessary ingredient to realize the metastable dynamics and irregular transitions between Up and Down states in a network with STD, which was supported by experimental observations in [46]. The Up and Down transitions described by our mesoscopic model in Fig 3A–3C solely stem from internally generated finite-size fluctuations and no external noise is needed. In addition, the mesoscopic model can explain certain dynamical features that Holcman and Tsodyks ascribed to long-term synaptic plasticity by changing the network size. Specifically, in [16] Holcman and Tsodyks attested deviations from typical Up-Down dynamics to external stimulation (by changing the input parameter μ) or to long-term synaptic plasticity (by changing the recurrent coupling strength J). A depolarization injection current (larger μ) had a similar effect as long-term potentiation (LTP; stronger recurrent coupling J), which led to longer Up states. On the other hand, a hyperpolarization injection current (smaller μ), or similarly long-term depression (LTD; smaller J), led to shorter Up states and more frequent population spikes. While our mesoscopic description predicts analogous behavior of the microscopic network when changing μ and/or J, it also predicts that similar effects can be realized by varying the network size N. Performing simulations with various network sizes between N = 50 and N = 150 while keeping the other parameters unchanged, we found that for N = 100, metastable activity featuring populations spikes is interspersed with Up states that last on average 10.6 s (mean taken over all Up states that last at least 1 s), see Fig 3D. For smaller networks with N = 50 (simulations not shown), population spikes become more frequent and Up states become shorter (mean duration 3.1s): Over a 10′000 s simulation, Up states followed population spikes in 16.9% of all 2249 cases for N = 50, whereas for N = 100 this only occurred in 15.6% of all 1218 cases. Larger networks with N = 150, by contrast, exhibit significantly longer Up states (mean duration 41.9 s) and even less frequent population spikes (see Fig B in S1 Text). Computing the corresponding histograms of the input potential revealed only for sufficiently large network sizes a second peak at around 5.5 mV (as in Fig 3F), which coincides with the Up state in Fig 3A. Furthermore, the fluctuation-driven oscillations in the long-lasting Up states (Fig 3B and 3C for N = 100) become visible in the power spectrum for N = 150 as another peak at around 1.5 Hz emerges. This frequency corresponds to the imaginary part of the eigenvalues of the stable focus (λ ≈ (−1.54 + 9.24i) Hz hence Im(λ)/(2π) ≈ 1.5 Hz).

We can conclude that in our theoretical model the network size critically affects the Up-Down dynamics. Recent findings in mouse primary sensory cortex show that the network size changes dynamically as the proportion of stimulus-responsive cells increases with learning [125]. Likewise, learning or, more general, context-dependence may dynamically control Up and Down states by recruiting more (or less) neurons in the respective brain area. One needs to keep in mind, however, that when varying the number N of neurons in our model, we have simultaneously rescaled the synaptic weights in proportion to 1/N. Although this specific change of parameters is useful for the theoretical analysis, a corresponding biological implementation is difficult to perceive. In any case, for a given network size and synaptic weights, our mesoscopic mean-field models Eqs (3) and (4) provide accurate descriptions of the microscopic network with short-term synaptic depression and therefore allow for an efficient analysis of how finite-size fluctuations contribute to and shape metastable dynamics.

Modeling hippocampal replay

Given the recent success in explaining a wide range of experimental observations on hippocampal dynamics by means of firing-rate models with STD [21, 71], we extended our mesoscopic description for a single population to a ring-attractor model of spiking neurons. Aiming at a minimal spiking neuron network model that can offer unique insights in the generative mechanisms of hippocampal dynamics in area CA3—in particular those underlying sharp waves and bidirectional activity replay—, we ignored various degrees of biological plausibility on purpose, but see below for possible extensions. In contrast to previous models that incorporated more biological details [10, 126–131], our derived stochastic ring model provides a relatively simple framework for a concise and efficient analysis of hippocampal sharp waves. Moreover, we were able to overcome the notorious difficulties of accommodating both the capability to endogenously generate sequential activity [132–134] and trial-to-trial temporal variability [6, 9].

In previous approaches to model metastable sequential activation patterns, as evident in hippocampal replay, successful candidate mechanisms to account for both of the above characteristics have already been implemented with the help of firing rate models. Recanatesi et al. [135] proposed a two-area mesoscale attractor network very much in the spirit of the winnerless competition model by Seliger, Tsimring and Rabinovich [22], in which the combination of asymmetric synaptic connectivity, arising from reciprocal coupling between fast and slow systems, with stochastic synaptic efficacy is crucial for the generation of sequences. Alternatively, Romani and Tsodyks [21] proposed a fully deterministic mechanism for hippocampal replay based on short-term depression (STD) and without the need for asymmetric connectivity, see also [71]: A ring-attractor network model with symmetric synaptic connectivity with local excitation and long-range inhibition exhibits transitions between a global quiescent state and various spatially localized bump states [48, 86, 87]. STD, as a slow fatigue mechanism, destabilizes these bumps and gives rise to traveling wave states [90]. In combination with the high-dimensional character of the network, the resulting dynamics appears effectively stochastic and spontaneously switches between quiescence and metastable traveling waves. As the approaches by Recanatesi et al. [135] and by Romani and Tsodyks [21] rely on heuristic firing-rate models, both of them suffer from the limitations already discussed in the Introduction and neither can account for a systematic investigation of finite-size effects on metastable activity. It is also unclear to what extent the deterministic Romani-Tsodyks (RT) model describes neural variability. For these reasons, we saw the need for a mesoscopic theory and derived a stochastic ring-attractor model bottom-up, thereby creating a direct link to microscopic networks of spiking neurons.

We first corroborated the findings of [21, 71] as we recovered, in the limit of infinitely many neurons per population, an analogous deterministic regime of spontaneously emerging hippocampal replay patterns as in the RT model [21], see Fig A in S1 Text. Given the similarity between our macroscopic model and the RT model, we subsequently probed our mesoscopic ring-attractor models by changing first the network size and then also by tuning the strength of the common input. In the regime of fatigue-induced hippocampal replay, already the diffusion model with Gaussian noise proved an accurate description of the microscopic network dynamics with finite population sizes, see Table A in S1 Text. When decreasing the common input μ = −0.9 ↦ −1.4, the microscopic network simulations featured finite-size induced hippocampal replay dynamics for small population sizes N that were accurately recovered by the mesoscopic jump-diffusion ring network model Eq (8) with hybrid noise, see Fig 4. The simpler diffusion model also captured these metastable dynamics but the quantitative match was not as good as in the jump-diffusion model, cf. Table B in S1 Text. Importantly, the macroscopic model could not retrieve the NLE regimes at all and remained in the globally quiescent state. The good agreement between the ring network simulations for single and multiple environments using either the microscopic or the mesoscopic ring network models, makes us confident that our mesoscopic description readily allows for capturing realistic hippocampal dynamics not only on periodic tracks—which we considered here for simplicity and as a proof of concept—but also on linear tracks, in T-maze environments, and planar (and higher dimensional) fields; we leave these extensions for future work as well as the formation of theta sequences and phase precession [21, 71, 136], see also [137, 138].

Recent experiments highlight the importance of stochastic models of hippocampal replay that can explain (i) replay patterns that resemble Brownian diffusion [139] or even super-diffusion [140], (ii) interburst intervals that exhibit significant variability [62, 93, 94], and (iii) replay episodes that do not always draw a smooth continuous path, but often follow a jumpy, discontinuous trajectory [95, 141]. Our mesoscopic description can be a useful step towards such a stochastic modeling framework. It may allow for a mechanistic understanding of the neuronal, synaptic and network constituents for generating spontaneous hippocampal activity, which are critical for memory consolidation, recall and spatial working memory, navigational planning, as well as reward-based learning [63, 75, 77, 80].

Predictions and possible functional roles of variable hippocampal dynamics

Our mesoscopic description opens a new perspective on the variability of hippocampal dynamics. Using the mesoscopic model, we uncovered a novel, finite-sized induced metastable regime of hippocampal replay (see also [91] for a similar mechanism of noise-induced traveling waves). At first glance, these fluctuation-induced quasi-traveling waves are similar in nature to those in [21]. On time-scales of a few hundred milliseconds, both the deterministic RT model and our mesoscopic model exhibit spontaneously emerging bursts of activity that resemble a nonlocal replay event. As a single replay event unfolds, we showed that both models feature discontinuous replay trajectories—consistent with experimental observations [95, 140, 141], which also underlines the biological plausibility of the ring-attractor assumption. On longer time-scales, however, our mesoscopic model allows for a much richer repertoire of replay dynamics. We found that the mesoscopic ring-attractor network can exhibit significantly larger variability for finite-sized induced replay dynamics than for deterministic, fatigue-induced dynamical regimes. This variability manifests in the spatio-temporally irregular succession of replay events (Figs 4 and 5 and Fig A in S1 Text) and could be tested experimentally, following, e.g., [62, 93, 94]. Consequently, our results insinuate that different replay dynamics can have distinct dynamical origins. In particular, replay dynamics in rodents are reportedly different during awake rest versus sleep [140, 142], possibly relating to fluctuation- versus fatigue-induced metastability.

Replay in brains and machines

Our findings may also be relevant for human neuroscience, where a paradigm shift to decode cognition rather from off-task, than from task-based, neural activity seems to be imminent [143]. Hippocampal replay in rodents is a prime example for a “representation-rich” approach to spontaneous neural activity by uncovering the temporal structure of task-related representations. In understanding how the temporal dynamics of a particular neural activity pattern (e.g., a nonlocal traveling wave) unfolds, researchers could shed light on various cognitive functions that are subserved by spontaneous neural activity, including memory, learning, and decision-making [77, 144]. Recent technical advances in human neuroimaging have inspired “human replay” studies that investigate spontaneous task-related neural reactivations [145–147], which bears strong resemblance to rodent replay. Instead of the spontaneous recall of an environment map in rodent hippocampal replay, the focus lies now on reactivation of a more abstract “cognitive map” of task space. As the associated cognitive processes include memory retrieval, planning and inference, and thus lie at the heart of sophisticated model-based reasoning, our results can also be regarded as a proof-of-concept for the model-based representation-rich approach advocated in [143] when re-interpreting the environmental maps stored in synaptic connectivities as cognitive maps. Intriguingly, also in human replay studies, replay can occur in forward and backward direction, with putative functional roles for spatial and non-spatial learning [148, 149]. In particular, Liu and co-workers suggest that nonlocal backward replay may serve as a neural mechanism for model-based reinforcement learning [149]. A comprehensive view about the different roles the wide variety of replay dynamics may subserve, however, remains elusive.

Insights from machine learning, where replay is commonly implemented in artificial agents, may help to find answers about the putative computational functions [150, 151]. “Experience replay” was already introduced as a reinforcement learning technique in the early 90s [152] and is nowadays a crucial ingredient in building human-level intelligence in deep neural networks [153, 154]. Note also that in reinforcement learning, a ‘model’ has a similar meaning to the notion of a ‘cognitive map’, which thus naturally bridges the gap from (human) cognition to artificial intelligence [155]. Nonetheless, research on replay in neuroscience and machine learning has progressed largely in parallel, so that insights from the latter can also inform future neuroscientific studies. An outstanding problem in the field of deep learning is the catastrophic forgetting problem in online learning [151, 156, 157]. This problem is due to the fact that during online learning, data is not guaranteed to be independent and identically distributed (i.i.d.), which is a challenge for standard optimization methods. Replay-like methods are used to overcome this problem, but current replay implementations are computationally expensive to deploy. By contrast, uniform sampling of past experiences has proven to be remarkably efficient both in supervised learning [158–160] and in reinforcement learning tasks [153, 161]. These insights provide some indirect, yet important evidence for the computational benefit of the stochastic replay that we have uncovered in this work and hint at an important role of the novel fluctuation-induced replay regime.

Biological limitations and extensions

In this paper, we aimed at a minimal bottom-up population model that accounts for spiking noise, short-term synaptic plasticity and basic neuronal properties. The result can be regarded as a proof of concept that a simple nonlinear mesoscopic model, which enables the analysis of metastable dynamics in terms of network size and the aforementioned properties, can be derived from an underlying microscopic model. However, our model has several biological limitations and lacks some important features. First, neurons exhibit post-spike refractoriness, which is not captured by the LNP model. While the response of the instantaneous firing rate can be well reproduced by choosing the linear filter function κ(t) and the nonlinearity f(h) of the LNP model corresponding to realistic dynamical transfer functions of neurons with refractoriness [54], the temporal spike-train correlations caused by refractoriness violate the Poisson assumption of our derivation. Although the strict Poisson assumption can be relaxed to some degree [44], strong spike-train auto-correlations influence the noise properties of the mesoscopic model and hence the fluctuations of the population activities. This effect has already been described for the mesoscopic model of [44] in the case of leaky integrate-and-fire neurons with pronounced refractoriness. Because our theory is based on the previous mesoscopic model, we expect that these effects carry over to the present model. How to account for non-Poisson statistics due to refractoriness in a mesoscopic theory with STP is a challenging theoretical problem that is left for future research. We also mention that a related neuronal property is spike-triggered adaptation, which—similar to synaptic depression—is a slow negative feedback mechanism. Incorporating adaptation into a mesoscopic theory, either instead of or in addition to depression, is interesting for two reasons: first, it represents an alternative slow fatigue mechanisms driving metastable dynamics, and second, adaptation is an important biological feature found in many cell types [162]. A promising generalization of the present theory to adaptation could be based on the quasi-renewal approximation [163] and its extension to mesoscopic theories [26, 164].

Apart from a more realistic description of neuronal properties, also the synaptic dynamics can be improved towards important biological features. First, the synaptic conductances exhibit temporal filtering and, in a conductance-based model, they enter the voltage dynamics in a multiplicative way. Both effects are neglected in our model. At least, the synaptic filtering of the conductance dynamics is straightforward to include in our theory as shown in [44]. For an extension of the mesoscopic model to a conductance-based description of synaptic input, the interested reader is referred to the discussion of [26], see also [116, 165]. Second, the Tsodyks-Markram model considered here is a phenomenological and deterministic model of the synaptic dynamics. However, synaptic transmission is stochastic, and thus a stochastic STP model [166] would be biologically more realistic (see also discussion in [44]), but how to treat such stochasticity within a mesoscopic theory remains unclear.

A biologically difficult problem is how to account for realistic network topology. Our theory applies to networks of multiple interacting homogeneous populations. In turn, each population is a fully-connected network of neurons. However, the connectivity of biological neural networks is not fully connected and often exhibits a large degree of heterogeneity. Regarding the first issue, we have shown previously that the full connectivity model represents an effective model that faithfully reproduces the dynamics of a non-fully-connected, random network with fixed in-degree if the synaptic weights are rescaled correspondingly [44] (see also [26] for the case of static synapses). The second issue is a principle challenge for mean-field theories as they rely on the possibility to use averages over many neurons, and hence cannot describe heterogeneous networks that are strongly affected by single neurons. However, in many cases it might be valid to subdivide the network into many small subpopulations that can be regarded as roughly homogeneous. Following this strategy, it is crucial to have a mesoscopic description of the subpopulations because the grouping of neurons with similar properties may result in small population sizes. For example, in our hippocampal network model, we have grouped place cells with highly overlapping place fields into one homogeneous subpopulation. If the number of these similarly tuned place cells is small, the mesoscopic framework can show its full strength because in this case, the jump-diffusion model still provides an accurate description (see Fig 3 for N = 30 neurons). Finally, we note that a basic type of network heterogeneity in biology, the separation of excitatory and inhibitory neurons (Dale’s law), is not realized in our hippocampal network with “Mexican-hat”-type connectivity. However, it has been shown that such connectivity can be re-implemented in accordance with Dale’s law by two layers of neurons, one excitatory and one inhibitory layer [167].

Theoretical challenges

The diffusion model Eq (3) and its derivation based on temporal coarse-graining [56] greatly simplify our previous theoretical work [44] but both results are consistent as shown below in Methods. The simplicity of our new derivation and the remarkable agreement of the diffusion model with microscopic simulations beg the question of whether the diffusion model is the exact diffusion approximation [168]. The only way to give a positive answer to this question would be to propose a rigorous diffusion approximation proof (as in [169] for the case of LNP neurons without STP), which we leave as an open mathematical problem. A second open theoretical question is the convergence of the multi-population model Eq (8) to a stochastic neural field equation. The circular environment we study can be regarded as a space-discretized stochastic neural field but the exact expression of the continuous equation and its physical interpretation is unclear and will be subject of future work. Ideally, one would want to be able to prove the convergence to such an equation, as in [170] in the case without STP.

The low-dimensional mesoscopic model could be also interesting from a data-analytical perspective. Recently, Bayesian state-space models have been developed to infer replay events from spiking data [141]. Such inference does not exploit any knowledge about the dynamical mechanisms underlying the data and the likelihood function is assumed ad hoc. Our low-dimensional mesoscopic could provide an analytical likelihood function so as to enable improved data assimilation methods to infer replay events.

In conclusion, we have put forward a multiscale framework for systematically investigating metastable network dynamics in finite-sized networks of LNP-STD neurons using a bottom-up mesoscopic model. This model is efficient to analyze and simulate and is also versatile for incorporating more biological realism. Thanks to a unique link between the underlying microscopic network and its mesoscopic description, it becomes possible to disentangle the differential roles of neuronal, synaptic and network properties—in particular the network size—for emerging metastable brain dynamics. The mesoscopic model may also be instrumental for distinguishing between fatigue-driven and fluctuation-driven metastability because of their distinct statistical predictions—as in the case of hippocampal replay. Such predictions could be tested experimentally and reveal the dynamical origin of spontaneous neural activity.

Diffusion approximation for the mesoscopic dynamics with short-term depression

The microscopic dynamics of a network of N LNP spiking neurons with short-term synaptic depression and an exponential linear filter (leading to leaky input integration) are given by Eq (1). To derive the diffusion model of the mesoscopic dynamics Eq (3), we focus on the mesoscopic variables h(t), x(t) and Q(t) defined as in Eq (2):

From Eq (1) and using the definition of x(t), we get (11a) (11b)

Note that, as mentioned in the presentation of the microscopic LNP-STD model (1), if all the h_i have the same initial condition at time 0, then h_i(t) = h_j(t) for all times t ≥ 0 and, consequently, h_i(t) = h(t) for all t ≥ 0. This means that for this type of initial conditions (also used in [50]), the microscopic h_i are equivalent to the mesoscopic h. Here, we consider only this case.

To approximate the sum by a diffusion term which only involves mesoscopic variables, we follow the coarse-graining approach by Gillespie [56] for the derivation of a “chemical Langevin equation”. To this end, we study the stochastic increments , where Δt > 0 is assumed to be a macroscopically infinitesimal time step [56]: Δt is small enough such that (i) the x_i’s can be assumed to jump at most once in the time interval [t, t + Δt], (ii) if neuron i has a spike at time τ_i ∈ [t, t + Δt], and (iii) for all ; and Δt is large enough such that many neurons spike in the time interval [t, t + Δt]. These assumptions are expected to hold if Δt ≪ τ, τ_D and 1 ≪ Nf(h(t))Δt ≪ N for all t. By the smallness assumption, we have (12) where are i.i.d. Bernoulli random variables with mean f(h(t⁻))Δt. Conditioned on the past (where (h_i(t′))_{t′ < t} ≡ (h(t′))_{t′ < t} by our assumption on the initial condition), all the variables x₁(t⁻), …, x_N(t⁻), z₁(t), …, z_N(t) are approximately independent, and the are approximately i.i.d. (all these variables become truly independent as N → ∞ [50]). Hence, we use the Gaussian approximation,

We now use the empirical averages Eq (2) to approximate the conditional expectation and variance:

We can now approximate the increment Eq (12) by a Gaussian:

Taking the limit Δt → 0, we obtain the diffusion approximation (13a) (13b) where ξ(t) is a Gaussian white noise with auto-correlation function 〈ξ(t)ξ(t′)〉 = δ(t − t′).

To close the system (13), we have to derive the dynamics of Q(t). Going back to Eq (1d), by Itô’s formula for jump processes, we have

Taking the empirical average, we get (14)

We could follow the same steps as before and try to obtain a diffusion approximation for Eq (14). However, the fluctuations of such a diffusion approximation would be of order and since Q(t) affects the dynamics of Eq (13) only through the term , the effect of the fluctuation of Q(t) on h(t) and x(t) are of order N^−3/2 and can therefore be neglected when N is large. Hence, we approximate the increments by their (approximate) expectation: for the time step Δt > 0, whence, (15)

Finally, gathering Eqs (13) and (15), we obtain the mesoscopic dynamics in the form of the diffusion model Eq (3).

The fact that we are here considering the diffusion approximation (or Langevin dynamics) allows us to significantly shorten the original derivation of the mesoscopic model presented in [44]. In particular, in the present derivation, we do not need to approximate the distribution of the x_i(t⁻) in Eq (12) by a Gaussian, since we only need to approximate the sum in Eq (12) by a Gaussian. Note that the arguments enabling the present derivation were already hinted at in the Section “Remarks on the approximation” and Appendix B of [44] but not put together. Both derivations lead to the same mesoscopic model, except that here, for simplicity, we neglect fluctuations of order N^−3/2 and we consider the diffusion limit; see also the Methods Section “Reduction to a pure diffusion process” below for an explicit derivation of the diffusion model from the original mesoscopic model presented in [44].

Jump-diffusion model with synaptic depression and facilitation

Starting from our previous work [44], we can derive an improved mesoscopic model that also accounts for synaptic facilitation and the shot-noise character of the finite-size spiking noise. When allowing only for short-term depression while keeping the facilitation variable constant, the resulting mesoscopic dynamics boils down to the jump-diffusion model Eq (4) with hybrid noise. For large N ≫ 1, the Poisson shot noise can be simplified under a diffusion approximation, which recovers the diffusion model Eq (3) derived in the previous section.

Mesoscopic model with pure synaptic depression

In order to obtain the jump-diffusion model Eq (4) only with short-term synaptic depression but without facilitation as considered in the Results section, we set and R ≡ U₀x. Then, μ^Q(u, x, P, Q, R) = −U₀(2−U₀)Q and ξ_u(t) ≡ 0, and the dynamics Eq (28) reduce to (30a) (30b) with the Gaussian white noise process ξ_x(t) defined as in Eq (28i). In Eq (30b), we have neglected the noise terms and replaced the population activity A(t) by its mean f(h(t)) because they enter the dynamics of x only to order N^−3/2. Finally, introducing the new mesoscopic variable y = Q − x², we can combine Eqs (30) and (28i) to arrive at the jump-diffusion model Eq (4).

Again, in the case of a general linear filter κ, the stochastic differential Eq (4a) for h(t) needs to be replaced by the corresponding integral expression (31)

Recurrent network parameters and numerical simulations

In the Results section, we presented and analyzed the network dynamics of a single excitatory population consisting of N LNP-STD spiking neurons following the microscopic dynamics Eq (1) or the mesoscopic dynamics Eqs (3)/(4). For the ring-attractor network model to study hippocampal replay dynamics, we used the microscopic dynamics Eq (7) and the mesoscopic dynamics Eq (8). The parameters to obtain Figs 2–6 are detailed in Table 1. The number N of neurons per population α = 1, …, M is indicated inside the Figures. The model specification and parameters of the Romani-Tsodyks (RT) model for fatigue-induced hippocampal replay, with which we compared our results for the mesoscopic ring-attractor network model in the single and in the multiple environment case in Figs 4–6, respectively, can be found in [21].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Parameters used in Figs 2–6.

https://doi.org/10.1371/journal.pcbi.1010809.t001

We performed numerical simulations of the microscopic, mesoscopic and macroscopic dynamics using an Euler–Maruyama scheme with time step dt = 10⁻⁴s. In the single-population scenario considered in Figs 2 and 3, we ran the simulations for T_sim = 100′000s to obtain significant statistics.

A circular environment.

In the ring-attractor network with a single environment stored in the synaptic connectivity, we ran the simulations long enough to have at least 5′000 burst events, which allows for a meaningful comparison of the different models. In Table 2, we list the simulation length T_sim together with an overview over the simulation results.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Simulation results complementing Figs 4 and 5.

https://doi.org/10.1371/journal.pcbi.1010809.t002

In more detail, we defined burst events as contiguous epochs of the averaged population activity above a certain threshold (taken as the activity averaged across neurons and the whole simulation period). There is an almost perfect agreement between the microscopic network and our mesoscopic description: First, the number of bursts (5′040 vs. 5′030) coincides up to an error of less than 0.2%. Second, the slopes of the linear regression between the number of peaks and the duration per burst are almost the same, see also Fig 4Dii and 4Eii, and our model prediction (∼9.3) is closer to the value observed experimentally (∼10.0, [83, 92]) than the macroscopic model predictions. Third, linear regression between the distance traveled during an event and its duration, yield almost identical slopes for the micro- and the mesoscopic models (Fig 4Diii and 4Eiii). Fourth, also the means and the coefficients of variation (CV) of the interburst intervals, i.e. the time from the end of the kth burst until the start of the k + 1st burst, coincide. Next, we defined nonlocal replay events (NLEs) as bursts of the average activity with more than one peak so that a transient traveling wave is visible in the density plots in Fig 4Bii and 4Cii, resembling a hippocampal replay pattern. Among all bursts events, around 20% are NLEs, which is consistent across all four models. Around half of all the NLEs travel in anti-clockwise/negative direction (“forward replay”) and the other half in clockwise/positive direction (“backward replay” or “preplay”); also this feature is consistent across the different models and matches experimental observations, where in individual experimental sessions both forward and backward replay events are detected with similar proportion [83, 92]. To determine the absolute event speed per NLE, see Fig 4Div and 4Eiv, we divided the distance of the traveled path (irrespective of forward or backward direction) by the duration of the NLE. The means of the absolute NLE speeds in all four models were again close to each other, which stresses the robustness of our findings across parameters and models. Finally, we also investigated serial correlations of the NLEs. Positive serial correlations of the event speed (now taking into account also the travel direction by considering negative speed for backward replays) at lag n indicate that the kth and the k + nth NLE are more likely to travel in the same direction, whereas negative correlations indicate these NLEs to travel in opposite directions. While the macroscopic and the RT model showed strong negative correlations at lag 1 and positive correlations at lag 2, correlations in the mesoscopic and microscopic models were negligible, see also Fig 5B. Besides, computing the serial correlations not for the (directional) event speed, but for a binary vector of forward/backward replay directions (by taking the sign of the directional event speed) yielded comparable results (see Table 2).

Multiple circular environments.

In the ring-attractor network with three circular environments stored in the synaptic connectivity, we ran micro- and mesoscopic simulations long enough to match the number of bursts in the deterministic (macroscopic and RT) model simulations of length T_sim = 1′000s, see Table 3. To reduce confounding asymmetries in the underlying synaptic connectivity structure, we constructed the selectivity vectors ζ_α in Eq (10) pseudo-randomly while guaranteeing that the number of bursts was equally distributed across the three environments, see Table 3. More precisely, we created the binary selectivity vectors ζ_α under the constraint that exactly fM = 90 of in total M = 300 units α = 1, …, M are selective for each environment k ∈ {1, 2, 3}. Out of these 90 selective units for environment k, units 1, …, 7 were selective for all three environments, units 8, …, 17 were selective for environments k and j and units 18, …, 27 for environments k and l with j, l ∈ {1, 2, 3}, k ≠ j ≠ l ≠ k. The remaining 63 units were exclusively selective for environment k. After this selection process, we randomly shuffled these 90 units and drew unique, evenly distributed place field angles for each unit α = 1, …, 90 selective for environment k.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Simulation results complementing Fig 6.

https://doi.org/10.1371/journal.pcbi.1010809.t003

To quantify which subsequences occurred more frequently than others, we computed the probabilities for subsequences with three distinct environments (k, j, l) ∈ {1, 2, 3}³ with k ≠ j ≠ l ≠ k by dividing the number of occurrences of a particular subsequence by the number of all possible 3-sequences (= number of all bursts−2). The results are shown in Fig 6.