Pairwise maximum-entropy model.

Neuronal activity is modelled as a set of sequences of spikes of N neurons during a finite time interval [0, T]. These spike sequences are discretized: we divide the time interval into n bins of identical length Δ equal to T/n, indexed by t in 1, …, n. For each neuron i, the existence of one or more spikes in bin t is represented by s_i(t) = 1, and lack of spikes by s_i(t) = 0. With this binary representation, the activity of our population at time bin t is described by a vector: s(t) ≔ (s_i(t)). We will switch freely between vector and component notation for this and other quantities.

Time averages are denoted by a circumflex: . The time-averaged activity of neuron i is denoted by m_i: (1) and the time average of the product of the activities of the neuron pair ij, called raw correlation or coupled activity, is denoted by g_ij: (2)

These time averages will be used as constraints for the maximum-entropy model.

The pairwise maximum-entropy statistical model [34, 35, 39, 40] assigns a time-independent probability distribution for the population activity s(t) of the form (time is therefore omitted in the notation): (3) the Lagrange multipliers h(m, g) and J(m, g) are determined by enforcing the equality of the time averages Eqs (1) and (2) with the single- and coupled-activity expectations, with their definitions (4) that is, enforcing (5)

By introducing the covariances c and Pearson correlation coefficients ρ, (6) the constraints above are jointly equivalent to (7) or (8)

The maximum-entropy distribution is unique if the constraints are convex. The covariance constraints c_ij = g_ij − m_im_j alone are not convex. In this case, uniqueness has to be checked separately [54, 55, 84]. On the other hand, the constraints E_p(s_i) = m_i and E_p(s_is_j) = g_ij are separately convex, thus their conjunction [E_p(s_i) = m_i] ∧ [E_p(s_is_j) = g_ij] is convex too. The bijective correspondence of the latter with [E_p(s_i) = m_i] ∧ [c_ij = g_ij − m_im_j] guarantees that the latter set of constraints is convex as well. What we have said about the covariances c also holds for the correlations ρ.

Reduced maximum-entropy model.

If the time-averaged activities m are homogeneous, i.e. equal to one another and to their population average , and the N (N − 1)/2 time-averaged coupled activities g are also homogeneous with population average , , then the pairwise maximum-entropy distribution has homogeneous Lagrange multipliers by symmetry: h_i = h_r and J_ij = J_r. It reduces to the simpler and analytically tractable form (9) which assigns equal probabilities to all those activities s that have the same population-averaged activity , defined as

We denote population averages that are normalized to the number of neurons by an overbar: . Then the quantity represents the sum of the activities of all neurons in the population at time bin t; we call it total activity, and its plot is called “population time-histogram” in some works [cf 85, 86].

In this homogeneous case, the values of the multipliers appearing in Eq (9) are equal to their population averages: and . This distribution hence contains only information about how many neurons are active at any given point in time, but not the particular composition s of active neurons. This simpler distribution can therefore also be interpreted as an approximation of the pairwise maximum-entropy one, achieved by disregarding population inhomogeneities of the constraints m_i and g_ij. But it is also an exact maximum-entropy distribution in its own right, obtained by only constraining the expectations for the population averages of the single and coupled activities, to be equal to their measured time averages (10)

For this reason we call the model Eq (9) a reduced pairwise maximum-entropy model. But in the inhomogeneous case the multipliers of the reduced model are not equal to the averages of the pairwise one: , .

It is straightforward to derive the probability distribution for the population average in this model, owing to its symmetry: the total number of active neurons in the population is , and there are equally probable ways in which this is possible, each with probability by Eq (9). Therefore, (11)

This probability distribution can, in turn, also be obtained applying a maximum-relative-entropy principle [31, 83], i.e. minimizing the relative entropy (or discrimination information) (12) of with respect to the reference distribution while constraining its first two moments, or equivalently its first two factorial moments [87] .

It is easy to see that in this model, by symmetry, we also have (13) (14) and , , are equivalent sets of constraints ( and by themselves are not convex).

This reduced maximum-entropy model is mathematically very convenient because the Lagrange multipliers h_r, J_r can be easily found numerically (with standard convex-optimization methods like downhill simplex, direction set, conjugate gradient, etc. [56, ch. 10]) with high precision even for large (e.g. thousands) population sizes N.

The pairwise and reduced models are very similar to the Gaussian ensemble of statistical mechanics [88–90, and refs therein], in which the mean and variance of a system’s energy are constrained; it is intermediate in properties between the canonical and microcanonical ensembles.

The maximum-entropy models reviewed above use time-averaged data. Their probabilities are therefore time-invariant; they are stationary statistical models.

Glauber dynamics and Boltzmann learning.

The normalization Z_p(h, J) appearing in the probability distribution Eq (3) requires the summation over 2^N states, typically in the range of N ≈ 100. This calculation may require prohibitive amounts of time; we need a way to calculate the distribution that avoids the computation of the normalization. The probability P_p(s|h, J) Eq (3) is identical in form to the stationary distribution of a Markov chain s(t) ↦ s(t + 1), the so-called asynchronous Glauber dynamics [63]. If this dynamics is ergodic, after an initial transient it generates states with a relative frequency distribution approximately equal to the stationary one. In this context the symmetric Lagrange multipliers J are sometimes also referred to as “couplings”, in analogy to synaptic interactions between the units. The Lagrange multipliers h are referred to as “biases” and may be interpreted as either a threshold or external input controlling the base activity of individual neurons. The temporal sequence of states produced by the Glauber dynamics is predominantly controlled by the time constant of the update rule and in general does not reflect the temporal evolution of the neuronal population activity.

If we assume that our uncertainty about the evolution of the population activity can be modelled by the Glauber dynamics of a binary network, we can employ the Glauber dynamics, with h, J parameters determined by the constraints Eq (5), to generate surrogate data that would allow us to implement a null-hypothesis for a statistical test, that includes the average activities and pairwise correlations as observed in the data. This procedure requires that the dynamics should be ergodic, sampling the entire state space. Otherwise, time averages obtained from the surrogate would not coincide with those of our experimentally observed data.

The Glauber dynamics is used as the sampling step in Boltzmann learning [64], as mentioned in the “Introduction”, to find the parameters h(m, g) and J(m, g) of the maximum entropy distribution Eq (3) having mean activities m and raw pairwise correlations g. Starting with some values of the multipliers and , the distribution is sampled by the Glauber dynamics and the its averages of the single activity and the coupled activity are found for the current values of the multipliers. The latter are then adjusted in relation to the mismatch , between the sampled averages and the required values from the experimental data. A new sampling is then performed, and so on until the mismatch lies below a prescribed accuracy.

Experimental data: Preliminary approximations.

The data, provided by A. Riehle and T. Brochier (INT, CNRS-AMU Marseille, France), consist of the activity of a population of N = 159 single neurons recorded from motor cortex of macaque monkey for 15 minutes, using a 100-electrode “Utah” array as described in [60], but with a different behavioural design: here the monkey was awake and alert, but did not perform any task during the recording. This behavioural protocol is chosen for retrieving “resting” (or “ongoing”) state [91] data to characterize the “ground” state, in contrast to a task or functional state.

Fig 1A shows a raster plot (2s out of 15min for better visibility) of the activities s(t) of all recorded neurons. The population-averaged activity for this period is shown underneath. The distributions of the time-averaged single and coupled activities m_i, g_ij, and the corresponding empirical covariances c_ij measured in the full data set of 15 min are shown in panels B, C, D. The population averages of these time-averaged quantities are (15)

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Experimental data and their empirical first- and second-order statistics.

(A) Example raster display (snippet of 2s from the total data of 15 min) of N = 159 parallel spike recordings of macaque monkey during a state of “ongoing activity”. The experimental data are recorded with a 100-electrode “Utah” array (Blackrock Microsystems, Salt Lake City, UT, USA) with 400 μm interelectrode distance, covering an area of 4 × 4 mm² (session: s131214-002). The total activity shows the number of active neurons within each time bin t of width Δ = 3 ms. (B) Population distribution of the time-averaged activities m_i (in spikes/Δ) of each of the neurons i, Eq (1). The vertical line marks the population average . (C) Population distribution of the time-averaged raw correlations (coupled activities) g_ij, Eq (2). The vertical line marks the population average, . (D) Population distribution of the covariances c_ij = g_ij − m_im_j. The vertical line marks the slightly positive population average, . Histograms bins in B, C, D are computed with Knuth’s rule [92] and calculated over the full 15-minute long recording. Data courtesy by A. Riehle and T. Brochier.

https://doi.org/10.1371/journal.pcbi.1005762.g001

As discussed in the previous section, the pairwise maximum-entropy model is a stationary statistical model. If we intended to analyse the dataset above with this model for a specific purpose—for example, characterizing a “ground state” of behavioural activity—then we would have to assess whether a stationary model would really suit these particular data and purpose. It would not be suitable, for example, to model transient aspects of neural activity. Our goal, however, is rather to analyse the general presence of bimodality in the model for data with ranges and orders of magnitude typical of recorded brain activity. In section “Bimodality of pairwise models for massively parallel data” we will see that our conclusions regarding bimodality are valid even if the population average is doubled or halved and if the Pearson correlation becomes ten times smaller or larger; thus amply valid within any non-stationarity corrections [93]. Moreover, stationary maximum-entropy models have also been used with highly fluctuating data, e.g. from retinal cells [e.g. 35, 40, 43, 94], with the purpose of analysing some of their information-theoretic properties rather than of modelling the data. For these reasons, and also for brevity, we do not address stationarity analyses and corrections in the present work.

We now need to find the parameters h(m, g) and J(m, g) of the maximum entropy distribution Eq (3) so that the mean activities m and the raw pairwise correlations g correspond to those measured in the data shown in Fig 1B and 1C. We try to find these parameters via Boltzmann learning with a Glauber dynamics, as explained in “Glauber dynamics and Boltzmann learning”.

We choose the sampling phase of the Boltzmann learning to have 10⁶ timesteps; an example is shown in Fig 2A. This number of timesteps is large compared to Roudi et al. [45] (200 timesteps) or Broderick et al. [95] (400 timesteps). The preliminary approximations of the Lagrange multipliers obtained in this way are shown in Fig 2D. The final single and coupled activities are shown in Fig 2C, compared to the experimental data. The first and second moments are highly correlated with the experimental ones and seem to describe the data well. The preliminary approximation of the population-average probability distribution is shown in red in Fig 2B. Its tail disagrees with that of the empirical frequency distribution (dashed); but, before discussing about curve-fitting properties, we want to make sure that our initial approximations are correct. In fact, we shall now see that these preliminary approximations are not correct in this case.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Preliminary results from Boltzmann learning.

(A) Evolution of the total-population activity of N = 159 neurons produced by the Glauber dynamics (implemented in NEST [96]; see section “Simulation of Glauber dynamics with NEST”) with 10⁶ steps, during the sampling phase of the last Boltzmann-learning iteration. (B) Red, solid: Preliminary approximation of the probability distribution of the population-averaged activity, obtained via Boltzmann learning. Blue, dashed: empirical distribution of the population-averaged activity from the dataset shown in A. (C) Preliminary values of the time averages m_i and g_ij obtained from Boltzmann learning described in (A), versus the experimental ones shown in Fig 1. (D) Preliminary approximations of the population distributions of the Lagrange multipliers and (associated with the averages m_i and g_ij) obtained via Boltzmann learning. Histogram bins in D are computed with Knuth’s rule [92].

https://doi.org/10.1371/journal.pcbi.1005762.g002

Appearance of bimodality.

The preliminary results from Boltzmann learning do not show any inconsistency at this point. But now we sample the distribution for a much longer time: 5 × 10⁷ steps, to verify whether these approximations have truly converged. The result is shown in Fig 3A. We find that after roughly 2 × 10⁶ steps the whole population jumps to a high-activity regime and remains there until the end of the sampling. We have thus discovered that:

• our preliminary approximations of the Lagrange multipliers are wrong; their mismatch is shown in Fig 3C–3D;
• our preliminary approximation of the pairwise distribution, Fig 2B, is therefore wrong in two ways: it does not correspond to the (wrong) approximations of the Lagrange multipliers, and is not a pairwise maximum-entropy distribution;
• the Glauber dynamics has an additional metastable high-activity regime.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Longer sampling: Bistability.

(A) Evolution of the total activity produced by Glauber dynamics, as in Fig 2A, but with 5 × 10⁷ steps. The dashed grey line marks the end of the previous shorter sampling of Fig 2A. (B) Evolutions of the total-population activity obtained from several instances of Glauber dynamics. Each instance starts with a different value of the initial total activity , i.e. with initially active neurons (chosen at random), and is represented by a different red shade, from (light red) to (dark red). Note the two convergence basins, one at and one at . (C, D) New values of the time averages m_i and g_ij versus the experimental ones. These new values are obtained from the longer Glauber dynamics described in (A) using the values of the Lagrange multipliers shown in Fig 2D, obtained from the previous Boltzmann learning. The plots clearly show that the values of found with the Boltzmann learning are not the ones yielding the constraints m_i, g_ij.

https://doi.org/10.1371/journal.pcbi.1005762.g003

It is legitimate to wonder whether there are other metastable regimes. To test this possibility we start the dynamics with different numbers of initially active neurons. Two metastable regimes are observed (see Fig 3B): one at high activity and one at low activity. This means that the distribution associated with the initial, wrong approximations of the Lagrange multipliers of Fig 2D is bimodal, not unimodal as Fig 2B seemed to show.

Note that choosing as initial condition for the sampling procedure a state in the low-activity regime will not prevent the Glauber dynamics from entering the high activity state. As shown in Fig 3B, the system may still spontaneously transition into the high activity regime with a small but not negligible probability.

What do these facts imply? Let us recall that our primary goal is to model the data with an inhomogeneous pairwise maximum-entropy distribution. Boltzmann learning is just a procedure to find this distribution. This procedure explores the space of distributions in a particular way to find the correct one. What we just found says that this procedure entered a region of bimodal distributions in such space and got stuck there, without finding the correct distribution yet.

We must reflect on three main issues:

1. Boltzmann learning becomes impractically slow when it enters the bimodal region, because the Glauber dynamics that is part of this procedure becomes almost non-ergodic. Therefore it is an inefficient method to find the pairwise maximum-entropy distribution. Non-ergodicity is a known phenomenon in Monte Carlo methods; its solution requires longer sampling times or algorithms different from Glauber sampling [61, 62, 97, 98]. This problem also appears, for example, in the calculation of extensive parameters in finite-size statistical mechanics in phase-transition regimes [99]. Note that bistability is not an effect of longer sampling; rather, longer sampling becomes a necessity because of bistability. This bistability is an inherent mathematical phenomenon caused by the positivity of the average correlations together with the large number N of units, as shown in sections “Bimodality ranges and population size” and “Bimodality of the inhomogeneous model for large N”.
   We could then try to use alternative procedures to find our desired distribution. The Thouless-Anderson-Palmer approximation [66, 67] and the Sessak-Monasson approximation [68, 69], for example, have been successfully used in the literature for this purpose. But unfortunately we find that these two approximations do not give the correct distribution either: properly sampled, the distributions they yield do not match the correlations and means of our data, just as in Fig 3C–3D. Evidently our data lie outside the domains of validity of these two approximations. Notably, the incorrect distributions given by these two approximations are also bimodal.
2. There may be one more problem ahead, though. Never mind that the procedures we know do not work; suppose we find a procedure that gives us the inhomogeneous pairwise distribution we are seeking. What should we do if this distribution turns out to be bimodal with a second mode at unrealistically high activities? Its Glauber dynamics would be bistable, yielding sustained periods of high activity, which would not be useful to generate meaningful, realistic surrogate data. Would we still be willing to use this maximum-entropy statistical model, or should we reject it altogether? And is the appearance of a bimodal distribution only peculiar to our data, or a more widespread feature of brain-activity data?
3. Besides, it is a pity that our initial approximation of the maximum-entropy distribution, Fig 2B, was incorrect. We had found some probability distribution, but it was not a pairwise maximum-entropy distribution; yet that distribution was modelling our data in an interesting way—and such modelling is our priority. This situation can be confusing, so let us explain it with an analogy. Imagine that we have some datapoints and we say “we want to fit the points with a parabola”. An incorrect fitting algorithm, however, gives us a curve that is not parabola. If this curve covers the datapoints in an interesting way, we may want to investigate what kind of curve it is. It could turn out to be a hyperbole, for example. We may then want to broaden our point of view and say “we want to fit the points with a quadric”, and use that curve. (We should still fix the fitting algorithm, though.) In our case, can we find out more about the probability distribution of Fig 2B? Could it also be a member of an extended maximum-entropy family for example?

We will shortly show that there is one solution that addresses these three issues all at once. We think, in fact, that it also addresses a fourth issue of maximum-entropy models, to be discussed later. We first analyse issues 2. and 3. in more detail; issue 1. above is subordinate to them.

Is the correct pairwise model bimodal?

We would like to know whether the correct maximum-entropy distribution, which we have not found yet, is also bimodal like its incorrect approximation.

We make an educated guess by examining the analytically tractable reduced maximum-entropy model P_r, Eq (9). Using the population-averaged single and coupled activities as constraints, and from Eq (15), we numerically find the Lagrange multipliers of the reduced model: (16)

Note that in this case there is no sampling involved: the distribution can be calculated analytically, and the values Eq (16) are correct within the numerical precision of the maximization procedure (interior-point method [56, chap. 10]). The values of the expected single and couple activities, re-obtained by explicit summation (not sampling) from the corresponding reduced maximum-entropy distribution, agree with the values Eq (15) to seven significant figures.

The resulting reduced maximum-entropy distribution for the population-averaged activity, , is shown in Fig 4A, together with the experimental frequency distribution of our data. Its corresponding Glauber dynamics with two metastable regimes is shown in Fig 4B. It shows a second maximum at roughly 90% activity.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Reduced maximum-entropy model.

(A) Red, solid: Probability distribution for the population-averaged activity, given by the reduced model for our dataset Eq (15); note the two probability maxima. Blue, dashed: empirical frequency distribution of the population-averaged activity from our dataset. (B) Population-averaged activities obtained from several instances of the Glauber dynamics associated with the reduced model, with homogeneous couplings, J_ij = J_r, and biases, h_i = h_r, of Eq (16). As in Fig 3, each instance starts with an initial population activity s(0) having different values of population average , and is represented by a different red shaded curve. The initial values range from (light red) to (dark red).

https://doi.org/10.1371/journal.pcbi.1005762.g004

A mathematically exact analysis of smaller subsets of our population with the inhomogeneous maximum-entropy model, and an analysis of the full population and large subsets of it with a reduced maximum-entropy model having higher-order constraints , , (the latter corresponding to the variance of the second moments), show that if a reduced maximum-entropy model is bimodal, the full inhomogeneous model is also bimodal, with a heightened second mode shifted towards lower activities with respect to the reduced model.

The bimodality encountered in the Boltzman learning, the bimodality of the reduced maximum-entropy model, the bimodality of the full maximum-entropy model for small populations, and finally the bimodality for the reduced model with higher-order constraints, together constitute strong evidence that the correct pairwise maximum-entropy distribution for our data is bimodal. In section “Bimodality of the inhomogeneous model for large N” we prove that this must be true for large N, even if it were not true for our specific population size N = 159.

Bimodality ranges and population size.

Next we want to address whether it is common or rare that the pairwise maximum-entropy method yields bimodal distributions for neuronal brain-activity data. For this purpose we first estimate the ranges of firing rates and correlations for which maximum-entropy yields a bimodal distribution; then we check whether typical experimental values fall within these ranges. We are particularly interested in how bimodality depends on the recorded population size N.

We again make an educated guess using the reduced maximum-entropy model, with distribution for the population average , , Eq (11). The distribution has two maxima if it has one minimum for some value , . An elementary study of the convexity properties (second derivative) of this distribution shows that it has one minimum at if (17)

These conditions can be solved analytically and give the ranges of the multipliers (h_r, J_r) for which bimodality occurs, parametrically in (, J_r): (18) where Ψ(x) ≔ d ln Γ(x)/dx, Γ being the Gamma function [100, chs 43–44]. We express the population-averaged activity and the Pearson correlation , typically used in the literature, in terms of (h_r, J_r) using the definitions Eq (14) and the probability Eq (11). In this way we finally obtain the bimodality range for , parametrically in (, J_r): the results are shown in Fig 5A for various values of the number of neurons N.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Bimodality ranges for the reduced model and effects of inhomogeneity.

(A) The reduced maximum-entropy model Eq (9) yields a distribution that is either unimodal or bimodal, depending on the number of neurons N and the values of the experimental constraints . Each curve in the plot corresponds to a particular N (see legend) and separates the values yielding a unimodal distribution, below the curve, from those yielding a bimodal one, above the curve. The curves are symmetric with respect to (ranges not shown). Note how the range of constraints yielding bimodality increases with N. Coloured dots show the experimental constraints from our dataset for different time-binnings with widths Δ = 1 ms, Δ = 3 ms, Δ = 5 ms, Δ = 10 ms: all these binnings yield a bimodal distribution. (B) Probability distributions of the reduced model for the population-averaged activity, , obtained using the constraints Eq (15) from our data set (3 ms purple dot in panel A) and different N (same colour legend as panel A). (C) Population-averaged activities from several instances of Glauber dynamics, all with the same normally-distributed couplings J_ij and biases h_i, with means as in Eq (16) and Fig 4B, and standard deviations σ(J_ij) = 0.009, σ(h_i) = 0.8. Each instance starts with an initial population activity s(0) having different values of the population average , and is represented by a different red shade, from (light red) to (dark red). Note how the basins of attraction of the two metastable regimes are wider than in the homogeneous case of Fig 4B. (D) The same as panel C, but with larger standard deviations σ(J_ij) = 0.012, σ(h_i) = 1.08; the jumps between the two metastable regimes become more frequent than in Fig 4B, indicating that the minimum between the modes becomes shallower as the inhomogeneity increases.

https://doi.org/10.1371/journal.pcbi.1005762.g005

For each N we have a curve in the plane ,. Values of above such curves yield a bimodal pairwise maximum-entropy distribution in the homogeneous case, for the corresponding population size N. The plot notably shows that the range of constraints yielding bimodality increases with N. Fig 5B displays the probability distribution of the population-averaged activity for the constraints from our dataset Eq (15) but different values of N. When N ≲ 150 the distribution has only one maximum at low activity, , and when N ≳ 150 a second probability maximum at high activity, , appears. The probability at this second maximum increases sharply until N ≈ 200 and thereafter maintains an approximately stable value, roughly 6000 times smaller than the low-activity maximum. The minimum between the two modes becomes deeper and deeper as we increase N above 200.

As mentioned in the previous section, exact studies with small samples and studies with large samples and a reduced model with higher-order constraints indicate that the high-activity maximum in the inhomogeneous case is even larger (roughly 2000 times smaller than the low-activity one when N = 1000) and shifted towards lower activities ( when N = 1000).

This can also be seen by adding a Gaussian jitter to the multipliers of the reduced case h_i = h_r, J_ij = J_r, thereby making the model inhomogeneous. The results for small and large jitter are shown in Fig 5C–5D, respectively. The basin of attraction of the second metastable regime is shifted to lower activities, and transitions between the two metastable regimes become more likely for larger jitters. This means that inhomogeneity makes the minimum in between the two modes shallower.

The population-averaged activity and Pearson correlation of our data (violet “3 ms” point in Fig 5A) fall within the bimodality range.

Bimodality of pairwise models for massively parallel data.

Having found the bimodality ranges of firing rates and correlations in section “Bimodality ranges and population size”, we now ascertain whether our dataset is a typical representative leading to bimodal pairwise distributions, or an outlier. We take as reference the data summarized in Table 1 of Cohen & Kohn [37], which reports firing rates and spike-count correlations r_SC for several experimental recordings of brain activity. The reported firing rates correspond to population-averaged activities ranging between 0.02 and 0.25 if we use 3 ms time bins; thus our data are well within this range.

The values reported for the correlations r_SC in [37] are given for the spike counts measured in large time intervals: several hundred of milliseconds. We therefore need a coarse estimate of the Pearson correlation coefficient ρ that would be measured on a fine temporal scale of 3 ms in the same experiment. Both r_SC and ρ are particular cases of the Pearson correlation coefficient r_CCG of spike counts in a window τ, as introduced by Bair et al. [101, App. A]: (19) i.e. ν_i(τ) is the number of spikes of neuron i in the time window τΔ. Here s_i(t) are the binary representations of the spike trains binned with bin width Δ, in line with section “Pairwise maximum-entropy model”. This metric also equals the area between times −τΔ and τΔ under the cross-correlogram of neurons i and j (stationarity is assumed), calculated on the fine temporal time scale Δ. The spike count correlation r_SC corresponds to τ = n ≡ T/Δ, and our Pearson correlation ρ to τ = 1. Several studies [we analysed: 101–105] report either measured values of r_CCG(τ) for different windows τ, or measured cross-correlograms. We studied, one by one, all the measures reported in the cited studies and numerically found that each of them satisfies ρ ≳ r_SC/20. We decide to take the lower bound as a coarse estimate of ρ given r_SC, because it leads to points as far away from bimodality as possible, i.e. because it is biased against our conjecture.

Under these approximations—and notwithstanding the choice of estimates that keeps the data as far away from bimodality as possible—the largest part of the data summarized by Cohen & Kohn does fall in the bimodality region of Fig 5A for N = 250, and almost all data lies in the bimodality region for N = 500; see Fig 6. These data points have only an indicative value, but suggest that our dataset is not an outlier for the bimodality problem. If those data had been recorded from a population of 500 neurons, they would have yielded a bimodal pairwise maximum-entropy model because, as shown in section “Bimodality ranges and population size”, the more neurons we are able to record, the more likely the bimodality occurs. Thus the bimodality problem and its consequences need to be taken seriously. Our next question is then: Is there any way to eliminate the bimodality problem?

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Bimodality for experimental data from neuroscientific literature.

Mean activities and correlations inferred from experimental data reported in Cohen & Kohn [37, Table 1], plotted upon the curves separating bimodal from unimodal maximum-entropy distributions of Fig 5A. The plot suggests that typical experimental neural recordings of 250 neurons and above are likely to lead to bimodal maximum-entropy pairwise distributions.

https://doi.org/10.1371/journal.pcbi.1005762.g006

Analysis of the erroneous preliminary approximation of the distribution.

In the last three sections we have partly addressed the second issue raised in section “Appearance of bimodality”: the distribution given by the pairwise maximum-entropy method is bound to be bimodal, not only for our data, but also for typical neuronal recordings of a couple of hundreds neurons or more.

We now address the third issue: to find out more about the first erroneous approximation of the probability distribution, shown in Fig 2B. It is important to remember that it is not the correct pairwise maximum-entropy distribution, and the initial erroneous approximations , of the multipliers do not match the data. The approximation was erroneous because the sampling phase of the Boltzmann-learning algorithm was too brief. The time required to explore the full distribution is so long that the dynamics is non-ergodic for computational purposes. This non-ergodicity effectively truncates the sampling at states s for which , where θ is the population-averaged activity at the trough between the two metastable regimes. This means that if the wrong multipliers , are used in a “truncated” distribution (20) then the expectations of this distribution are close to the experimental time averages for the single and coupled activities: E_t(s_i) = m_i, E_t(s_i s_j) = g_ij. This truncated distribution, though, is obviously not a pairwise maximum-entropy distribution. It is an interesting distribution nevertheless, as Fig 2B shows. Could it be obtained or approximated with the maximum-entropy method in some other way?

Intuitive understanding of the bimodality: Mean-field picture.

From the point of view of a network evolving with a Glauber dynamics with couplings J and biases h, the bimodality and bistability appear because the couplings J are on average positive and make the network dominantly excitatory. The positivity of the couplings appears because the average correlation between the neurons, empirically measured, is positive (Figs 1D and 7A). This phenomenon is not unknown: bimodalities in the distribution of extensive quantities have a similar explanation in the statistical mechanics of finite-size systems [99, 106–108].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Mean-field picture.

(A) Illustration of a self-coupled symmetric network that is self-excitatory on average. Arrow-headed blue lines (→) represent excitatory couplings; circle-headed red lines (⊸) represent inhibitory couplings. (B) Self-consistency solution of the naive mean-field equation, illustrated for different J_r. Larger J_r lead to two additional intersections, corresponding to an unstable and a stable solution. The red curve corresponds to the J_r calculated from our experimental data Eq (16).

https://doi.org/10.1371/journal.pcbi.1005762.g007

A naive mean-field analysis also confirms this. In such an approximation we imagine that each neuron is coupled to a field representing the mean activities of all other neurons [65; 109, ch. 6]. From the point of view of entropy maximization, we are replacing the maximum-entropy distribution with one representing independent activities, having minimal Kullback-Leibler divergence from the original one [66, chs 2, 16, 17]. Given the couplings J and biases h, the mean activities m must satisfy N self-consistency equations (21)

In the homogeneous case they reduce to the equation , corresponding to the intersection of two functions of : the diagonal line , and the curve that depends parametrically on (h_r, J_r); see Fig 7B. For the Lagrange multipliers of our data, these curves intersect at two different values of , meaning that there are two solutions to the self-consistency equation, corresponding to two different mean activities. These approximately correspond to the maxima of the probability distribution for the population average in Fig 4A.

Importance of inhibition: Modified Glauber dynamics.

In the neuronal network dynamics just analysed, with positive correlations on average, the second peak of high activity can be suppressed introducing an effective negative feedback loop. Such inhibitory mechanism can be represented by an additional neuronal unit I, having positive incoming couplings J_Ik > 0 and negative outgoing couplings J_kI < 0 with all other units k. This unit is therefore activated if the total activity of the other units is high, and once activated it provides negative input back to all other units. This situation is illustrated in Fig 8A. Such a stabilizing mechanism acts much in the same way as inhibition stabilizes the low-activity state in neuronal networks [110, 111]; it has also been used in the simulations by Bohte et al. [34]. The asymmetry of this mechanism is in contradiction with the symmetry of the couplings J_ij of the Glauber dynamics, however. We want to break this symmetry and add an asymmetric inhibitory feedback to the Glauber dynamics to avoid the bimodality of the probability distribution.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Asymmetric inhibition and elimination of bimodality and non-ergodicity.

(A) Illustration of self-coupled network with additional asymmetric inhibitory feedback. Each neuron receives inhibitory input J_I < 0 from the additional neuron whenever the population-average becomes greater than the inhibition threshold θ. (B) Population-averaged activities from several instances of the inhibited Glauber dynamics, with J_I = −24.7, θ = 0.3, and homogeneous J_ij = J_r, h_i = h_r of Eq (16), as used for Fig 4B. Each instance starts with an initial population activity s(0) having different values of the population average , and is represented by a different grey shade, from (light grey) to (black). Note the disappearance, thanks to inhibition, of the bistability that was evident in the “uninhibited” case of Fig 4B. (C) Analogous to panel B, with J_I = −24.7, θ = 0.3, but inhomogeneous normally distributed couplings and biases as in the uninhibited case of Fig 5C. The bistability again disappears thanks to inhibition. (D) Comparison of a longer (5 × 10⁶ timesteps) Glauber sampling in the inhibited (black, J_I = −24.7, θ = 0.3) and uninhibited (red) case, using the couplings and biases of Fig 2D obtained from our first Boltzmann learning. (E) Time averages m_i and g_ij obtained from Boltzmann learning for the inhibited model P_i, versus experimental ones. (F) Probability distribution of the population-averaged activity given by the inhibited model Eq (22) for our dataset Eq (15), compared with the one previously given by the reduced model , Fig 4A.

https://doi.org/10.1371/journal.pcbi.1005762.g008

We preliminarily implement this idea in our Glauber dynamics, to observe its consequences, using first the incorrect approximations of the multipliers that yielded a bimodal inhomogeneous pairwise distribution, and then the (correct) multipliers h_i = h_r, J_ij = J_r, Eq (16), of the reduced pairwise model. We connect all N neurons to a single inhibitory neuron that instantaneously activates whenever their average activity exceeds a threshold θ ∈ {0, 1/N, 2/N, …, (N − 1)/N, 1}. Upon activation the inhibitory neuron inhibits the other N neurons (see Fig 8A) via N identical negative couplings J_I < 0. The results from the simulation of the inhibited Glauber dynamics are shown in Fig 8; in all cases the inhibitory coupling was J_I = −24.7 and the inhibition threshold θ = 0.3.

The algorithm for sampling from this “inhibited” Glauber dynamics is explained in section “Inhibited Glauber dynamics”. It can be seen that the additional inhibitory neuron eliminates the bistability, leaving only the stable low-activity regime. The resulting homogeneous and inhomogeneous stationary distributions (in the inhomogeneous case J_ij and h_i are normally distributed as in Fig 5C) are either unimodal or have a second mode that is completely negligible, being tens or hundreds of orders of magnitude smaller than the first mode.

The inhibited Glauber dynamics can suppress the bistability for any network size N, with an appropriate choice of the inhibitory coupling J_I < 0 and threshold θ.

Continuing our exploration, we check what happens if we use the inhibited Glauber dynamics in the sampling phase of Boltzmann learning for our initial problem, when we tried to find a stationary distribution having means and correlations shown in Fig 1B–1D. As shown in Fig 8E, the addition of the inhibitory neuron (again with J_I = −24.7, θ = 0.3) eliminates the second metastable state that appeared after 2 × 10⁶ steps, cf. Fig 3. The resulting couplings and biases of the final stationary distribution are distributed as in Fig 2D—but note that this time the Boltzmann learning has converged, hence these are its correct final values.

However, it must be stressed that the stationary distribution thus found, using the inhibited Glauber dynamics, is not a pairwise maximum-entropy distribution, because the latter is the stationary distribution of the original Glauber dynamics, not of the modified one. If we use the inhibited dynamics in Boltzmann learning, we are abandoning the standard pairwise maximum-entropy model.

There is nevertheless a positive result: in the next section we show that the stationary distribution of the inhibited Glauber dynamics belongs to a generalized maximum-entropy family.

Inhibited maximum-entropy model.

The pairwise maximum-entropy distribution Eq (3) is the stationary distribution of the Glauber dynamics with symmetric couplings. It is not the stationary distribution of the inhibited Glauber dynamics. But the following fact holds: The stationary distribution of the inhibited Glauber dynamics (Fig 8) belongs to the maximum-entropy family. Its analytic expression is (22) where J_I is the (negative, in our case) coupling strength from the inhibitory neuron to the other neurons, θ is the activation threshold of the inhibitory neuron, and H is the Heaviside step function. We call Eq (22) the inhibited pairwise maximum-entropy model. The proof that it is the stationary distribution of the inhibited Glauber dynamics is given in section “Inhibited Glauber dynamics”.

This maximum-entropy model is characterized by the new term in the exponential, which we call “inhibition term”. The function is plotted in Fig 9 together with its exponential. We show in section “Expansion of the inhibition term in terms of higher-order coupled activities” that it can also be written as a linear combination of population-averaged K-tuple activities, , for K equal to Nθ + 1 and larger: (23) the coefficients being generalized binomial coefficients [100, ch. 6; see also 112], which have alternating signs. For example, if N = 5 and θ = 3/5, (24)

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. Reference measure.

The function and its exponential for J_I < 0.

https://doi.org/10.1371/journal.pcbi.1005762.g009

This function differs from the additional function appearing in the maximum-entropy model by Tkačik et al. [50, 94, 113], which consists in N + 1 constraints enforcing the observed frequency distribution of the population average . For large data samples the constraints used in those works typically equal 0 for larger values of ; for reasons discussed in section “Range of applicability of maximum-entropy models”, the use of such extreme constraints may not be justified or meaningful.

The inhibited distribution P_i(s) belongs to the maximum-entropy family in two different ways, the first preferable to the second:

1. It can be obtained by application of the maximum-relative-entropy (minimum-discrimination-information) principle [31, 83], with the pairwise constraints Eq (5), with respect to the reference distribution (25) also called “reference measure”, which assigns decreasing probabilities to states with average activities above θ; see Fig 9B. This relative-maximum-entropy model can be interpreted as arising from a more detailed model in which we know that external inhibitory units make activities above the threshold θ increasingly improbable, like Bohte et al.’s model [34] for example. We discuss this in the “Discussion”. In this interpretation the parameters J_I and θ are chosen a priori.
2. Alternatively, it results from the application of the “bare” maximum-entropy principle given the pairwise constraints Eq (5) and an additional constraint for the expectation of : (26)
   This is a constraint on what could be called the “tail first moment” of the distribution for the population-averaged activity : it determines whether the right tail of has a small (J_I < 0) or heavy (J_I > 0) probability. It can also be seen as a constraint on the Nθ-th and higher moments, owing to Eq (23). In this interpretation the parameter J_I is the Lagrange multiplier associated with this constraint, hence it is determined by the data; the parameter θ is chosen a priori. Note, however, that experimental data are likely to give a vanishing time average of , so that J_I = −∞. This interpretation has therefore to be used with care, for the reasons discussed in section “Range of applicability of maximum-entropy models”.

Two features of the inhibited maximum-entropy model Eq (22) are worth remarking upon:

1. The family of inhibited distributions P_i includes the pairwise family P_p, Eq (3), as the particular case J_I = 0. Note that if J_I ≠ 0 the inhibited and uninhibited models with identical Lagrange multipliers (h, J) have different expectations for single and coupled activities: (27) and therefore different covariances and correlations.
2. If the inhibitory parameter J_I is very large and negative, all activities s such that are assigned a negligible probability by the inhibited model. We therefore have the approximation (technically, pointwise convergence as J_I → −∞) (28) where C is an appropriate normalization constant.

But the last expression is identical to Eq (20). Thus, the inhibited maximum-entropy distribution P_i is approximately equal to the truncated distribution P_t—the incorrect one Eq (20) obtained with our initial Boltzmann learning—having the same multipliers (h, J) and threshold θ: (29) and their expectations are also approximately equal. We have therefore addressed the third issue discussed in section “Appearance of bimodality”: the initial, incorrect but interesting approximation can actually be rescued, as we now explain.

Summary: Application of the inhibited maximum-relative-entropy model.

Let us find a probability distribution for our data using the maximum-relative-entropy method, with reference measure Eq (25), J_I = −24.7, θ = 0.3, given the single and pairwise constraints Eq (5) summarized in Fig 1C and 1D. To find the Lagrange multipliers (h, J) and the distribution Eq (22) we use the Boltzmann learning procedure with 5 × 10⁶ timesteps. For the sampling phase we must use the inhibited Glauber dynamics, because its stationary distribution is Eq (22). As proven earlier (Fig 8) no bistability arises, so we do not need to worry about wrong approximations from undersampling; in fact, a much smaller number of timesteps would suffice.

The resulting multipliers and distribution are not shown because their plots are indistinguishable to the naked eye from the homologous ones in Fig 2B–2D. We have thus found that the interesting distribution shown in red in Fig 2B, although not a pairwise maximum-entropy distribution, still belongs to an enlarged maximum-entropy family: it is an inhibited, inhomogeneous, pairwise maximum-relative-entropy distribution.

The inhibited maximum-relative-entropy model therefore solves all there issues presented in section “Appearance of bimodality”:

1. The multipliers and distribution of this model can efficiently be found via Boltzmann learning with the typical number of timesteps used in the literature (200–400 [45, 95]), because its Glauber dynamics is not affected by bistability (Fig 8).
2. This model produces a distribution that has no unrealistic modes at high activities.
3. This distribution has interesting features and fitting properties (Fig 2B).

We can ask whether the solution of these three issues by the inhibited model is enough to warrant its use. In particular, does the use of the reference measure (Eq (25)) make sense from a neurobiological standpoint? In the following “Discussion” we argue that it does, and that it solves in fact a fourth issue of standard (i.e. uniform-measure) maximum-entropy models for neuronal networks.

Inhibited Glauber dynamics.

In the “inhibited” Glauber dynamics, the network of N neurons with states s_i(t) has an additional neuron with state s_I(t). The dynamics is determined by the following algorithm starting at time step t with states s = s(t), s_I = s_I(t):

1. One of the N units is chosen, each unit having probability 1/N of being the chosen one. Suppose i is the selected unit.
2. The chosen unit i is updated to the state with probability
   Note the additional coupling from the neuron s_I, with strength J_I. This strength can have any sign, but we are interested in the J_I ⩽ 0 case; we therefore call s_I the “inhibitory neuron”.
3. The inhibitory neuron is deterministically updated to the state given by (33) corresponding to a Kronecker-delta conditional probability (34)
   In other words, the inhibitory neuron becomes active if the population-averaged activity of the other neurons is equal to or exceeds the threshold θ.
4. The time is stepped forward, t + 1 → t, and the process repeats from step 1.

The original Glauber dynamics, described in the previous section, is recovered when J_I = 0, which corresponds to decoupling the inhibitory neuron s_I.

The total transition probability can be written as (35) the product of Kronecker deltas in the last term ensures that at most one of the N neurons changes state at each timestep.

The transition probabilities for the chosen neuron s_i and the inhibitory neuron s_I are independent, conditional on the state of the network at the previous timestep: so the transition probability for the N neurons only can be written as (36) (37)

This formula also shows that the transition probability for the network can alternatively be derived without explicitly introducing an inhibitory unit: starting from the modified activation function with F_i defined by Eq (39), the transition probability Eq (36) for the network follows from the additional requirement that only a single unit changes state within a single update.

Proof that the inhibited maximum-entropy model is the stationary distribution of the inhibited Glauber dynamics.

The modified maximum-entropy distribution P_i, Eq (22), is the stationary distribution of a slightly modified version of the above dynamics, with the update rule (38) and the use of N inhibitory neurons, one for each of the original N units. This dynamics has a slightly different transition probability, with activation function (39) instead of Eq (37). Note that the two dynamics are very similar for large enough N. To prove the stationarity of inhibited maximum-entropy distribution P_i, we show that P_i satisfies the detailed-balance equality (40) which is a sufficient condition for stationarity [146–148].

First note that if s′ and s differ in the state of more than one neuron, the transition probability p(s′|s) vanishes and the detailed-balance above is trivially satisfied. Also the case s′ = s is trivially satisfied. Only the case in which s′ and s differ in the state of one unit, say s_i, remains to be proven. Assume then that (41) by symmetry, if the detailed balance is satisfied in the case above it will also be satisfied with the values 0 and 1 interchanged.

Substituting the transition probability Eqs (36) and (39) in the left-hand side of the fraction form of the detailed balance Eq (40), and noting that F_i(s′) = F_i(s), we have (42)

Using the expression for the inhibited model P_i, Eq (22), in the right-hand side of the fraction form of the detailed balance Eq (40), we have (43) where we have used the equality NG(x + 1/N) − NG(x) = H(x), valid if and Nθ ∈ Z. Comparison of formulae Eqs (42) and (43) finally proves that the detailed balance is satisfied also in the case Eq (41).