^{*}

The authors have declared that no competing interests exist.

Conceived and designed the experiments: SM. Performed the experiments: RI VM. Analyzed the data: RI VM MB CK SM. Wrote the paper: RI VM MB CK SM.

The manner in which different distributions of synaptic weights onto cortical neurons shape their spiking activity remains open. To characterize a homogeneous neuronal population, we use the master equation for generalized leaky integrate-and-fire neurons with shot-noise synapses. We develop fast semi-analytic numerical methods to solve this equation for either current or conductance synapses, with and without synaptic depression. We show that its solutions match simulations of equivalent neuronal networks better than those of the Fokker-Planck equation and we compute bounds on the network response to non-instantaneous synapses. We apply these methods to study different synaptic weight distributions in feed-forward networks. We characterize the synaptic amplitude distributions using a set of measures, called tail weight numbers, designed to quantify the preponderance of very strong synapses. Even if synaptic amplitude distributions are equated for both the total current and average synaptic weight, distributions with sparse but strong synapses produce higher responses for small inputs, leading to a larger operating range. Furthermore, despite their small number, such synapses enable the network to respond faster and with more stability in the face of external fluctuations.

Neurons communicate via action potentials. Typically, depolarizations caused by presynaptic firing are small, such that many synaptic inputs are necessary to exceed the firing threshold. This is the assumption made by standard mathematical approaches such as the Fokker-Planck formalism. However, in some cases the synaptic weight can be large. On occasion, a single input is capable of exceeding threshold. Although this phenomenon can be studied with computational simulations, these can be impractical for large scale brain simulations or suffer from the problem of insufficient knowledge of the relevant parameters. Improving upon the standard Fokker-Planck approach, we develop a hybrid approach combining semi-analytical with computational methods into an efficient technique for analyzing the effect that rare and large synaptic weights can have on neural network activity. Our method has both neurobiological as well as methodological implications. Sparse but powerful synapses provide networks with response celerity, enhanced bandwidth and stability, even when the networks are matched for average input. We introduce a measure characterizing this response. Furthermore, our method can characterize the sub-threshold membrane potential distribution and spiking statistics of very large networks of distinct but homogeneous populations of 10s to 100s of distinct neuronal cell types throughout the brain.

Experiments analyzing the distribution of synaptic weights impinging onto neurons typically observe low-amplitude peaks with only few, large-amplitude excitatory (EPSPs) or inhibitory post-synaptic potentials (IPSPs)

The relation between probabilistic synaptic weight distribution and population dynamics can be studied using simulations

The equation for the membrane potential

The first term on the RHS in

The stationary solution for

Although the model can include both excitatory and inhibitory connections, all the results presented below, except for those in

Intuitively, this model quickly diverges from the Fokker-Planck formalism when the synaptic strength is large. Consider the hypothetical case of a neuron starting at rest which has all the synapses equal and large. After a short time step, the Fokker-Planck formalism produces a narrow Gaussian distribution in membrane potential near rest, while the DiPDE formalism yields a membrane potential distribution which is a sum of two scaled delta functions: a large one at rest, and a small one at the synaptic weight. Over time, the Fokker-Planck equation converges to a single broader Gaussian, while the DiPDE formalism leads to a larger coefficient for the delta function at the synaptic weight value. A generalization to conductance-based synapses is presented in (

We solve

Panels (a)–(e) show results for excitatory, low-frequency, large amplitude current-based synapses of constant weight. Topmost panels show time evolution of the probability distribution of membrane potentials in the neuronal population obtained with Poisson input for

The steady state firing rate obtained from DiPDE is

By comparison, the Fokker-Planck formalism results in a lower steady state firing rate of

We used the DiPDE formalism to investigate the effect of generic synaptic weight distribution on the steady-state, subthreshold membrane potential distribution and output firing rates in feed-forward networks. Gaussian synaptic weight distributions with different mean weights whose input firing rates are adjusted to produce the same average synaptic current, result in different transient and steady-state firing rates as well as different equilibrium voltage distributions (top row in

Top panels: a) Two self-similar Gaussian synaptic weight distributions. b) The distribution of the sub-threshold, steady-state membrane potential when the two Gaussian synaptic inputs are activated with either a low (solid curves) or a high input firing rate (dashed curves) adjusted such that the mean input currents are equal. The low-amplitude distribution always has twice the input rate of the high amplitude one. In the absence of a threshold, these synaptic input would depolarize

We then examined six different distributions tuned to have both the same mean (1 mV) synaptic weight and input rate (1,000 Hz), so that the average synaptic current was the same (see

Mean input current matched (1 mV,1000 Hz) | Drift (1 |
|||||

Distribution | 95% CI Output rates | 95% CI Output rates | ||||

delta | 19.6 | 16.2 | (18.5,20.6) | 22.0 | 11.0 | (20.5,23.5) |

Gaussian | 20.5 | 13.4 | (19.5,21.6) | 21.8 | 10.6 | (20.5,23.1) |

exponential | 21.3 | 10.6 | (20.3,22.4) | 21.5 | 10.2 | (20.4,22.7) |

lognormal | 21.5 | 8.2 | (20.4,22.6) | 21.1 | 9.6 | (20.1,22.1) |

bimodal | 28.7 | 2.6 | (27.5,29.9) | 15.9 | 4.2 | (15.3,16.6) |

power law | 28.3 | 0.9 | (27.0,29.6) | 13.6 | 2.7 | (13.3,13.9) |

Since the network we study is a feed-forward network, the mean synaptic delay results in a simple time translation of the responses. However, the overshoot of steady state seen in

To analyze what characteristic of the synaptic distribution is most important for the fast response observed for our heavy-tailed distributions, we generated more than 1000 random distributions matched to have the same mean synaptic weight and input rate (

Moment | ||||

mom2 | 0.1739 | 0.1030 | 0.0391 | |

mom3 | 0.0479 | 0.0582 | ||

mom4 | 0.0622 | 0.1580 | 0.2550 | |

mom5 | 0.1179 | 0.2226 | 0.3874 | 0.5136 |

Tail wt. no. | ||||

Q2 | 0.0397 | 0.1082 | 0.1862 | |

Q3 | 0.0427 | 0.0222 | 0.0594 | |

Q4 | 0.0967 | 0.0418 | 0.0157 | |

Q5 | 0.1631 | 0.0910 | 0.0290 |

Having established a measure of population activity when all neurons start at rest, we examined the dynamics resulting from an equilibrium different from rest (see

Top row: Protocol used to investigate the response of the neuronal population with a given excitatory synaptic weight distribution to a sudden perturbation in its synaptic input. a) Input rate as a function of time. For the first 500 ms, the cumulative synaptic input rate is varied between 500 and 1,000 Hz, expressed as a fraction of the 1,000 Hz base input rate (see

The DiPDE formalism also enables insights into the influence of short-term synaptic depression (STSD), which is known to play a key role in neural network homeostasis and in the generation of multiple network states

In the preceding analysis, we saw how perturbations of the input event rate in a population with different synaptic weight distributions affect the entire time-course of the population's evolution from an initial to a final equlibrium. In contrast, we also analyzed how fluctuations in inputs (see

The graphs show the relative excitability

To quantify the instantaneous change in the equilibrium firing rate, we make use of a closely-related variant of the tail weight numbers defined in

Using

A few strong synapses can exert an undue large influence on the mean response of the population. Consider the unimodal

Our study provides an advance on two fronts, computationally and neurobiologically. First, we developed and validated a semi-analytic method to model the sub-threshold membrane potential probability distribution and the firing rate of homogeneous neuronal populations with finite synaptic inputs. Second, we apply this method to explore the effect of varying synaptic weight distributions on equilibrium and transient population characteristics. From a methodological standpoint, the DiPDE formalism reproduces population behavior from aggregate simulations of identical point neurons, without the need to run the large-scale simulations themselves with the attendant computational costs (see

This

Teramae and colleagues

Such investigations highlight the need to focus the attention of electrophysiologists studying synaptic transmission

Given the large biological and instrumental noise present in synaptic measurements, in particular under

A simplified case can be obtained if all EPSPs are assumed to have the same value

If

In

If

Numerical simulations, against which the Fokker-Planck (for current-based synapses) and DiPDE formalisms were compared, were performed by invoking the NEST simulator after writing the code in PyNN.

For current-based synapses, simulations were performed for a population of

For conductance-based synapses, simulations were performed for a population of

Simulations were also performed for a population of

To solve the evolution

For the standard LIF neuron with membrane time-constant

For more general forms of the leak term (for e.g, the exponential integrate-and-fire (EIF) neuron; see

At each time-step

In order to compare results with simulations, we need to match the total synaptic input to the neuronal population with that from simulations. The DiPDE formulation is exact when implementing instantaneous synapses

The synaptic weight

For our choice of simulation parameters outlined above

The underlying stochastic process in

Expected 95% intervals for output spike counts can then be obtained as follows. At each time step, the solution obtained from

For non-instantaneous synapses, upper and lower bounds on the output firing rate can be obtained. The exponentially decaying current-based synapses used here in simulations (see (Materials and Methods: Current-based synapses)) result in an exact EPSP given by

Setting

A lower bound on the output firing rate can obtained by equating the maximum depolarization,

For

For the lower bound, consider the difference between

For current-based synapses, the change in membrane potential resulting from synaptic input is independent of the initial membrane potential. For a conductance-based synapse however, this change is proportional to the difference between the initial membrane potential and the reversal potential

Because of the additional dependence on the membrane voltage and synaptic reversal potential

For our choice of simulation parameters, the maximum depolarization achieved by a neuron starting from rest due to synaptic input with

For multiple neuronal populations connected to each other, one can generalize

The DiPDE formalism (

For the exponential integrate-and-fire (EIF) neuron, the numerical solution of the equation with exponentially decaying conductance-based synapses gives

The Fokker-Planck equation for current-based synapses was solved using an explicit Forward-Time Centered-Space (FTCS) scheme. As in the DiPDE, the leak term was solved analytically, although the voltage discretization was kept uniform in order to be compatible with the standard Centered-Space discretization used for the diffusion term. To ensure that the discretization was sufficiently fine (especially since FTCS is only first-order accurate), we compared the equilibrium firing rate generated by this scheme against the analytical solution

The effect of various matched synaptic weight distributions on the population dynamics in feed-forward networks was investigated using the DiPDE formalism. The top panel of

If the firing is driven exclusively by variations in input (Gaussian distributions with balanced excitation and inhibition), the differences in output firing rates are large as shown in the bottom panel of

These results imply that population response is determined not only by the total current, but also by the mean synaptic weights.

For the bottom row of

Delta :

Gaussian :

Exponential :

Lognormal :

Bimodal :

Power-law :

The ^{2}), Gaussian (1.3 mV^{2}), Exponential (1.8 mV^{2}), Lognormal (2.2 mV^{2}), Bimodal (8.0 mV^{2}) and Power-law (8.7 mV^{2}). The firing rates and transient times corresponding to distributions for which the mean input current was matched (

An alternate way to match synaptic weight distributions is to match the drift and diffusion, corresponding to what the mean and variance of the membrane potential would be if the neuron did not have a threshold, while allowing the input rates to vary. In the Fokker-Planck formalism, such distributions matched for drift and diffusion would have led to the same results.

Delta :

Gaussian :

Exponential :

Lognormal :

Bimodal :

Power-law :

The

The heavier-tailed distributions still lead to faster transients, however the steady-state firing rates now decrease with increasing heaviness of the tail in the distributions. This is in contrast to the case when the input currents were matched (

In order of increasing heaviness of the tail distributions, the equilibrium output firing rates and the transient firing times for the different distributions are provided in

We also implement synaptic delays within the DiPDE formalism. This is done by a using a queue to store the output firing rate, which is then accessed and updated depending on the distribution of synaptic delays.

The tail-heaviness of the synaptic weight distribution can be characterized by either constructing the moments of the distribution or a set of

To test which key characteristic of synaptic weight distributions better describes the transient times, we generated 1222 random distributions between 0 and

Input-output curves in

We quantified the spread in the input-output relationship due to different values of the fraction

The most comprehensive treatment of synaptic utility would involve a single 3-D integro-differential equation where the probability distribution

Given the exponential form of

When a non-zero proportion of the probability for a given pre-synaptic neuronal population is at threshold (i.e. there is spiking in the population), the output synaptic weight distribution from that population to the target population is convolved with the synaptic utility distribution,

To analyze the effect of fluctuations generated by different synaptic distributions, the DiPDE numerical solution was evolved with a given synaptic distribution for 300 ms, long enough for the voltage distribution to reach equilibrium. At this point, to simulate fluctuations which occur on timescales

This same analysis was then conducted with synaptic depression during the fluctuation stage, starting with the stationary distribution obtained without synaptic depression. Unlike in the no-depression case, the synaptic distribution was scaled by the synaptic utility (using the method outlined in (

All the results for DiPDE simulations were obtained using MATLAB. The entire code base is available for download at

Comparisons for excitatory, high-frequency and low-amplitude current-based synapses. Top panels show time evolution of the probability distribution of membrane potentials in the neuronal population obtained from a) simulations of 10,000 leaky integrate-and-fire neurons, b) the numerical solution to

(EPS)

Solutions to

(EPS)

Comparisons for excitatory conductance-based synapses. Top panels show time evolution of the probability distribution of membrane potentials in the neuronal population obtained from a) simulations, and b) numerical solution to

(EPS)

For non-instantaneous synapses, estimates of the output firing rate can be obtained by using instantaneous synapses equated for the total charge (with normalized capacitance) or the maximum depolarization respectively. These are controlled by an additional parameter

(EPS)

EPSPs from instantaneous current-based, excitatory synapses used for obtaining estimates of the output firing rate for non-instantaneous synapses (see

(EPS)

Gaussian distributions of instantaneous synaptic weights with the same mean input current. Top Panels: a) Four Gaussian synaptic weight distributions. b) The output firing rates as a function of time when the four Gaussian synaptic inputs are activated with an input firing rate adjusted such that the mean input currents are equal. Both low-amplitude distributions (red and light-blue curves) have twice the input rate of the high-amplitude ones (dark-blue and green curves). In the absence of a threshold, the synaptic input would depolarize

(EPS)

Distributions of synaptic weights with same mean input current and variance of membrane potential. a) Semi-log plot of different synaptic distributions, matched for drift^{2}/ms. b) Steady state sub-threshold membrane potential distributions. c) Output firing rates. Heavier-tailed distributions still produce quicker transients, but result in lower steady state output firing rates in contrast to (

(EPS)

Effect of

(EPS)

Top Panels: Tail weight numbers

(EPS)

Time taken to reach 20% of steady-state firing rate (

(EPS)

Even

(EPS)

Quantifying response to sudden changes in input rate for different synaptic weight distributions matched for mean input current. Peak output firing rates for different fractions of base input rate as a function of base input rate for a)

(EPS)

Effect of external fluctuations on sub-threshold membrane potential distribution and excitability for different synaptic weight distributions, without and with synaptic depression. Simulations start from the equilibrium of

(EPS)

Effects of fluctuations in synaptic input, starting from the equilibrium obtained for different synaptic weight distributions with mean = 1 mV and input rate

(EPS)

Computational times. Table shows the simulation times in (s) with NEST and DiPDE for different choices of time step dt. The middle four columns correspond to simulation times with different numbers of neurons

(PDF)

Differentiating synaptic weight distributions matched for mean input current. Table shows the number of independent recordings of sub-threshold steady state membrane potential required to differentiate between synaptic distributions matched for mean input current. The values above the diagonal are the sample sizes needed for p = 0.01, and below the diagonal are the sizes for p = 0.05.

(PDF)

Differentiating synaptic weight distributions matched for drift and diffusion. Table shows the number of independent recordings of sub-threshold steady state membrane potential required to differentiate between synaptic distributions matched for drift and diffusion. The values above the diagonal are the sample sizes needed for p = 0.01, and below the diagonal are the sizes for p = 0.05.

(PDF)

Dynamical ranges. Table shows the upper (90%) and lower (10%) limits of steady state output firing rates and dynamical ranges (ratio of upper and lower steady state output firing rates) for synaptic weight distributions (as in

(PDF)

Relative excitabilities. Table shows the maximum relative excitability for synaptic weight distributions matched for mean input current, without and with synaptic depression. Heavier-tailed distributions lead to smaller changes in relative excitability.

(PDF)

A brief review of stochastic processes focusing on elements needed for this study.

(PDF)

Comparison of DiPDE and Fokker-Planck formalisms.

(PDF)

Computational complexity with DiPDE.

(PDF)

p-values for differences between sub-threshold steady-state membrane potential distributions obtained from different synaptic weight distributions.

(PDF)

We wish to thank the Allen Institute founders, Paul G. Allen and Jody Allen, for their vision, encouragement and support. We also thank N. Cain for useful discussions and M. Hawrylycz for feedback on the manuscript.