Fig 1.
Dynamics approaching the firing-rate instability in threshold-quadratic networks.
A) Average firing rate of the excitatory neurons as synaptic weights are scaled. While the ordinate axis shows the excitatory-excitatory synaptic weight, all other weights are scaled with it. Solid lines: prediction of mean field theory. Dots: result of simulation. Inset: threshold-quadratic transfer function. B) Spectral radius of the stability matrix of mean field theory as synaptic weights are scaled. Stars indicate the weight values for the simulations below. C) Example realizations of activity for three different interaction strengths. As synapses become stronger, correlated activity becomes apparent. When synapses are strong enough the activity becomes unstable, even though the mean field theory is stable. All plotted firing rates in A) are averaged over the time period before the rates diverged (if they did). Left: (WEE, WEI, WIE, WII) = (.025, -.1, .01, -.1) mV. C). Middle: (WEE, WEI, WIE, WII) = (1, −4, .4, −4) mV. Right: (WEE, WEI, WIE, WII) = (1.5, −6, .6, −6) mV.
Fig 2.
Feynman diagrams for the first three cumulants of the inhomogeneous Poisson process.
Each dotted edge corresponds to a delta-function connecting the time indices of its two nodes. White nodes denote the measurement times, while gray nodes denote the times at which spikes are generated. The cumulants are constructed by convolving the term corresponding to the gray node with the product of all outgoing edges’ terms (Eq 5).
Fig 3.
Feynman diagrams for the first three cumulants of the filtered inhomogeneous Poisson process.
A) Cumulant corresponding to the graph. B) Diagrammatic expressions using the filter and the underlying Poisson process. Each dotted edge corresponds to a delta-function connecting the time indices of its two nodes. Each wavy edge corresponds to the filter g connecting the time indices of its two nodes. C) Diagrammatic expressions using the propagator. In all graphs, external white nodes (leaves of the graph) denote measurement times. Gray nodes denote the times at which spikes are generated in the input spike train. Internal white nodes (with time indices t′) denote the times at which input spikes arrive at the postsynaptic neuron. The cumulants are constructed by convolving the term corresponding to the gray node with the product of all outgoing edges’ terms (Eq 8).
Fig 4.
Diagrammatic expansion for the mean firing rate and linear response of the self-exciting process.
A) First-order approximation of the firing rate. B) Diagrams corresponding to the re-summing of the expansion of the mean field rate (Eq 25), which is represented by the black dot. C) Diagrams corresponding to the re-summing calculation of the propagator (Eq 28), which is represented by the solid edge. In all diagrams, time indices associated with internal vertices have been suppressed.
Fig 5.
Diagrammatic expansion for the second cumulant for the self-exciting process.
A) First-order approximation of the second cumulant. B) Re-summing to obtain the full second cumulant. Compare the expansions within the square brackets adjacent to external vertices to the expansion of the propagator, Fig 4c, and compare the expansion of the source term to that of the mean field rate, Fig 4b.
Fig 6.
Diagrams corresponding to third order cumulants.
A) Diagrams corresponding to the third cumulant of the first-order self-exciting process. B) Diagrams corresponding to the third cumulant of the self-exciting process, after resumming the perturbative expansion. Nodes and edges correspond to the same terms as in Fig 5.
Fig 7.
Feynman rules for the self exciting process.
These rules provide an algorithm for computing the expansion of the cumulants around the mean field solution . The dots between the legs of the first two vertices indicate that there are such vertices with any number of outgoing legs greater than or equal to two.
Fig 8.
Expansion of the mean firing rate and propagator for the nonlinearly self-exciting process.
A) One of the first nonlinear terms of the expansion of the mean-field firing rate, to first order in the quadratic term of the nonlinearity and zeroth order in the linear term. The two diagrams shown correspond to the two terms in Eq 42. B) First nonlinear terms of the expansion of the propagator around the mean-field firing rate.
Fig 9.
Corrections to the mean-field firing rate and propagator of the nonlinearly self-exciting process, to quadratic order in the nonlinearity ϕ.
A) The correction to mean field theory for the firing rate to first order in the quadratic term of ϕ. B) The full one-loop correction to the mean-field rate. C) The full one-loop correction to the propagator.
Fig 10.
Feynman rules for the nonlinearly self exciting process.
These rules provide an algorithm for computing the expansion of the cumulants around the mean field solution . The dots between the outgoing legs of the first vertex indicate that there any number of outgoing legs greater than or equal to two. The number b of incoming edges of the second vertex correspond to its factor containing the bth derivative of ϕ, evaluated at the mean field input. The a dots between the outgoing edges of the second vertex indicate that it can have any number of outgoing edges such that a + b ≥ 3.
Fig 11.
Feynman rules for networks of stochastically spiking neurons with nonlinear input-rate transfer ϕ.
These rules provide an algorithm for computing the expansion of the cumulants around the mean field solution . The dots between the outgoing legs of the first vertex indicate that there are any number of outgoing legs greater than or equal to two. The number b of incoming edges of the second vertex correspond to its factor containing the bth derivative of ϕ, evaluated at the mean field input. The a dots between the outgoing edges of the second vertex indicate that it can have any number of outgoing edges such that a + b ≥ 3.
Fig 12.
Fluctuation expansion links single-neuron nonlinearities and network structure to determine network activity.
The linear response for linear neurons depends on the network structure both explicitly and implicitly, through the mean-field rates. The first nonlinear correction brings in additional explicit and implicit dependencies on the connectivity.
Fig 13.
Construction of Feynman diagrams for the first nonlinear correction to the two-point cumulant (graphs containing one loop).
In each panel, we add a new layer of vertices to the diagrams, until we arrive at a source vertex. When there are multiple potential ways to add vertices, we add diagrams to account for each of those constructions. A) External vertices corresponding to the two measurement times, with incoming propagator (Δ) edges. B) Diagrams with one internal vertex added. t1 ↔ t2 corresponds to switching the two external vertices in the bottom diagram; the top diagram is symmetric with respect to that switch. C) Diagrams with two layers of vertices. The top diagram finishes that of B, top. The second two arise from the second diagram of B, and each also have copies with t1 ↔ t2. D) Last diagrams containing one loop. The final diagrams corresponding to the one-loop correction to the second cumulant are the top two of C) and that of D).
Fig 14.
Dynamics approaching the firing-rate instability in threshold-linear networks.
A) Threshold-linear input-rate transfer function. B,C) Raster plots of 1 second realizations of activity for weak and strong synaptic weights. Neurons 0–199 are excitatory and 200–240 are inhibitory. B) (WEE, WEI, WIE, WII) = (.025, −.1, .01, −.1) mV. C) (WEE, WEI, WIE, WII) = (.2, −.8, .08, −0.8) mV. D-F) Average firing rate of the excitatory neurons (D), integral of the auto-covariance function of the summed population spike train (E), and spectral radius of the stability matrix of mean field theory (F) vs. excitatory-excitatory synaptic weight. While excitatory-excitatory weight is plotted on the horizontal axis, all other synaptic weights increase proportionally with it. Black lines: tree-level theory: mean-field firing rates and covariance computed by linearizing dynamics around it, for each value of synaptic weights. Dots: simulation.
Fig 15.
Correlation-driven instability in nonlinear networks.
A) Threshold-quadratic input-rate transfer function. B,C) Raster plots of 6 second realizations of activity for weak and strong synaptic weights. Neurons 0–199 are excitatory and 200–240 are inhibitory. B) (WEE, WEI, WIE, WII) = (.025, −.1, .01, −.1) mV. C) (WEE, WEI, WIE, WII) = (1.5, −6, .6, −6) mV. D-F) Average firing rate of the excitatory neurons (D), integral of the auto-covariance function of the summed population spike train (E), and spectral radius of the stability matrix of mean field theory (F) vs. excitatory-excitatory synaptic weight. While excitatory-excitatory weight is plotted on the horizontal axis, all other synaptic weights increase proportionally with it. Black line: tree-level theory. Red line: one-loop correction accounting for impact of the next order (pairwise correlations’ influence on mean and triplet correlations’ influence on pairwise). Dots: simulation. All dots after the one-loop spectral radius crosses 1 represent results averaged over the time period before the activity diverges.
Fig 16.
Calculation of the one-loop stability correction.
A) Loop expansion of the full propagator. B) Factorization of the loop and resumming of the full propagator after that factorization.
Fig 17.
Correlation-driven instability in a non-Erdős-Rényi network with broadly distributed excitatory-excitatory weights.
A) Threshold-quadratic input-rate transfer function. B) Histogram of excitatory-excitatory synaptic weights with location parameter of 1.42 (mean of.29 mV), corresponding to the simulation in panel C. C) Raster plots of 6 second realizations of activity. Neurons 0–199 are excitatory and 200–240 are inhibitory. (WEE, WEI, WIE, WII) = (1.125, −4.5, .45, −4.5) mV. D-F) Average firing rate of the excitatory neurons (D), integral of the auto-covariance function of the summed population spike train (E), and spectral radius of the stability matrix of mean field theory (F) vs. excitatory-excitatory synaptic weight. While the mean excitatory-excitatory weight is plotted on the horizontal axis, all other synaptic weights increase proportionally with it. Black line: tree-level theory. Red line: one-loop correction. Dots: simulation. If a simulation exhibits divergent activity, the spike train statistics are averaged over the transient time before that divergence for visualization.
Fig 18.
Stability of a network with exponential transfer functions.
A) Mean firing rate of the excitatory neurons. B) Integral of the auto-covariance function of the summed population spike train. C) Spectral radius of the stability matrix of mean field theory, all (A-C) vs. excitatory-excitatory synaptic weight. While the mean excitatory-excitatory weight is plotted on the horizontal axis, all other synaptic weights increase proportionally with it. Black line: tree-level theory. Red line: one-loop correction. Dots: simulation. If a simulation exhibits divergent activity, the spike train statistics are averaged over the transient time before that divergence for visualization.
Fig 19.
Failure of one-loop corrections with exponential transfer functions.
A) Mean firing rate of the excitatory neurons. B) Integral of the auto-covariance function of the summed population spike train. C) Spectral radius of the stability matrix of mean field theory, all (A-C) vs. excitatory-excitatory synaptic weight. While the mean excitatory-excitatory weight is plotted on the horizontal axis, all other synaptic weights increase proportionally with it. Black line: tree-level theory. Red line: one-loop correction. Dots: simulation. If a simulation exhibits divergent activity, the spike train statistics are averaged over the transient time before that divergence for visualization.
Fig 20.
Potential impacts of correlations on coding in a presence of a nonlinearity.
A) The independent Poisson assumption for neurons gives rise to uncorrelated distributions of population activity. B) Correlations could increase or decrease the overlap of those distributions by stretching them, decreasing or increasing the decoding performance (top and bottom, respectively). C) The impact of correlations on the mean responses can shift those distributions, potentially counteracting the impact of stretching the distributions (as shown), or exaggerating it.
Table 1.
Model parameters (unless otherwise specified in text).