Recapitulation of MCMC sampling

In machine learning, sampling is often considered the “gold standard” of inference methods, since, assuming that we can sample from the distribution in question, and assuming enough computational resources, any inference task can be carried out with arbitrary precision (in contrast to some deterministic approximate inference methods such as variational inference). However sampling from an arbitrary distribution can be a difficult problem in itself, as, e.g., many distributions can only be evaluated modulo a global constant (the partition function). In order to circumvent these problems, elaborate MCMC sampling techniques have been developed in machine learning and statistics [38]. MCMC algorithms are based on the following idea: instead of producing an ad-hoc sample, a process that is heuristically comparable to a global search over the whole state space of the random variables, MCMC methods produce a new sample via a “local search” around a point in the state space that is already (approximately) a sample from the distribution.

More formally, a Markov chain (in discrete time) is defined by a set of states (we consider for discrete time only the case where has a finite size, denoted by ) together with a transition operator . The operator is a conditional probability distribution over the next state given a preceding state . The Markov chain is started in some initial state , and moves through a trajectory of states via iterated application of the stochastic transition operator . More precisely, if is the state at time , then the next state is drawn from the conditional probability distribution . An important theorem from probability theory (see, e.g., p. 232 in [39]) states that if is irreducible (i.e., any state in can be reached from any other state in in finitely many steps with probability ) and aperiodic (i.e., its state transitions cannot be trapped in deterministic cycles), then the probability converges for to a probability that does not depend on the initial state . This state distribution is called the invariant distribution of . The irreducibility of implies that it is the only distribution over the states that is invariant under its transition operator , i.e.(1)

Thus, in order to carry out probabilistic inference for a given distribution , it suffices to construct an irreducible and aperiodic Markov chain that leaves invariant, i.e., satisfies equation (1). Then one can answer numerous probabilistic inference questions regarding without any numerical computations of probabilities. Rather, one plugs in the observed values for some of the random variables (RVs) and simply collects samples from the conditional distribution over the other RVs of interest when the Markov chain approaches its invariant distribution.

A convenient and popular method for the construction of an operator for a given distribution is looking for operators that satisfy the following detailed balance condition,(2)for all . A Markov chain that satisfies (2) is said to be reversible. In particular, the Gibbs and Metropolis-Hastings algorithms employ reversible Markov chains. A very useful property of (2) is that it implies the invariance property (1), and this is in fact the standard method for proving (1). However, as our approach makes use of irreversible Markov chains as explained below, we will have to prove (1) directly.

Neural sampling

Let be some arbitrary joint distribution over binary variables that only takes on values . We will show that under a certain computability assumption on a network consisting of spiking neurons can sample from using its inherent stochastic dynamics. More precisely, we show that the stochastic firing activity of can be viewed as a non-reversible Markov chain that samples from the given probability distribution . If a subset of the variables are observed, modelled as the corresponding neurons being “clamped” to the observed values, the remaining network samples from the conditional distribution of the remaining variables given the observables. Hence, this approach offers a quite natural implementation of probabilistic inference. It is similar to sampling approaches which have already been applied extensively, e.g., in Boltzmann machines, however our model is more biologically realistic as it incorporates aspects of the inherent temporal dynamics and spike-based communication of a network of spiking neurons. We call this approach neural sampling in the remainder of the paper.

In order to enable a network of spiking neurons to sample from a distribution of binary variables , one needs to specify how an assignment of values to these binary variables can be represented by the spiking activity of the network and vice versa. A spike, or action potential, of a biological neuron has a short duration of roughly . But the effect of such spike, both on the neuron itself (in the form of refractory processes) and on the membrane potential of other neurons (in the form of postsynaptic potentials) lasts substantially longer, on the order of to . In order to capture this temporally extended effect of each spike, we fix some parameter that models the average duration of these temporally extended processes caused by a spike. We say that a binary vector is represented by the firing activity of the network at time for iff:(3)

In other words, any spike of neuron sets the value of the associated binary variable to 1 for a duration of length .

An obvious consequence of this definition is that the binary vector that is defined by the activity of at time does not fully capture the internal state of this stochastic system. Rather, one needs to take into account additional non-binary variables , where the value of at time specifies when within the time interval the neuron has fired (if it has fired within this time interval, thereby causing at time ). The neural sampling process has the Markov property only with regard to these more informative auxiliary variables . Therefore our analysis of neural sampling will focus on the temporal evolution of these auxiliary variables. We adopt the convention that each spike of neuron sets the value of to its maximal value , from which it linearly decays back to during the subsequent time interval of length .

For the construction of the sampling network , we assume that the membrane potential of neuron at time equals the log-odds of the corresponding variable to be active, and refer to this property as neural computability condition:(4)where we write for and for the current values of all other variables with . Under the assumption we make in equation (4), i.e., that the neural membrane potential reflects the log-odds of the corresponding variable , it is required that each single neuron in the network can actually compute the right-hand side of equation (4), i.e., that it fulfills the neural computability condition.

A concrete class of probability distributions, that we will use as an example in the remainder, are Boltzmann distributions:(5)with arbitrary real valued parameters which satisfy and (the constant ensures the normalization of ). For the Boltzmann distribution, condition (4) is satisfied by neurons with the standard membrane potential(6)where is the bias of neuron (which regulates its excitability), is the strength of the synaptic connection from neuron to , and approximates the time course of the postsynaptic potential in neuron caused by a firing of neuron with a constant signal of duration (i.e., a square pulse). As we will describe below, spikes of neuron are evoked stochastically depending on the current membrane potential and the auxiliary variable .

The neural computability condition (4) links classes of probability distributions to neuron and synapse models in a network of spiking neurons. As shown above, Boltzmann distributions satisfy the condition if one considers point neuron models which compute a linear weighted sum of the presynaptic inputs. The class of distributions can be extended to include more complex distributions using a method proposed in [40] which is based on the following idea. Neuron representing the variable is not directly influenced by the activities of the presynaptic neurons, but via intermediate nonlinear preprocessing elements. This preprocessing might be implemented by dendrites or other (inter-) neurons and is assumed to compute nonlinear combinations of the presynaptic activities (similar to a kernel). This allows the membrane potential , and therefore the log-odds ratio on the right-hand side of (4), to represent a more complex function of the activities , giving rise to more complex joint distributions . The concrete implementation of non-trivial directed and undirected graphical models with the help of preprocessing elements in the neural sampling framework is subject of current research. For the examples given in this study, we focus on the standard form of the membrane potential (6) of point neurons. As shown below, these spiking network models can emulate any Boltzmann machine (BM) [36].

A substantial amount of preceding studies has demonstrated that BMs are very powerful, and that the application of suitable learning algorithms for setting the weights makes it possible to learn and represent complex sensory processing tasks by such distributions [37], [41]. In applications in statistics and machine learning using such Boltzmann distributions, sampling is typically implemented by Gibbs sampling or more general reversible MCMC methods. However, it is difficult to model some neural processes, such as an absolute refractory period or a postsynaptic potential (PSP) of fixed duration, using a reversible Markov chain, but they are more conveniently modelled using an irreversible one. As we wish to keep the computational power of BMs and at the same time to augment the sampling procedure with aspects of neural dynamics (such as PSPs with fixed durations, refractory mechanisms) to increase biological realism, we focus in the following on irreversible MCMC methods (keeping in mind that this might not be the only possible way to achieve these goals).

Neural sampling in discrete time

Here we describe neural dynamics in discrete time with an absolute refractory period . We interpret one step of the Markov chain as a time step in biological real time. The dynamics of the variable , that describes the time course of the effect of a spike of neuron , are defined in the following way. is set to the value when neuron fires, and decays by at each subsequent discrete time step. The parameter is chosen to be some integer, so that decays back to in exactly time steps. The neuron can only spike (with a probability that is a function of its current membrane potential ) if its variable . If however, , the neuron is considered refractory and it cannot spike, but its is reduced by 1 per time step. To show that these simple dynamics do indeed sample from the given distribution , we proceed in the following way. We define a joint distribution which has the desired marginal distribution . Further we formalize the dynamics informally described above as a transition operator operating on the state vector . Finally, in the Methods section, we show that is the unique invariant distribution of this operator , i.e., that the dynamics described by produce samples from the desired distribution . We refer to sampling through networks with this stochastic spiking mechanism as neural sampling with absolute refractory period due to the persistent refractory process.

Given the distribution that we want to sample from, we define the following joint distribution over the neural variables:(7)

This definition of simply expresses that if , then the auxiliary variable can assume any value in with equal probability. On the other hand necessarily assumes the value if (i.e., when the neuron is in its resting state).

The state transition operator can be defined in a transparent manner as a composition of transition operators, , where only updates the variables and of neuron , i.e., the neurons are updated sequentially in the same order (this severe restriction will become obsolete in the case of continuous time discussed below). We define the composition as , i.e., is applied prior to . The new values of and only depend on the previous value and on the current membrane potential . The interesting dynamics take place in the variable . They are illustrated in Figure 1 where the arrows represent transition probabilities greater than 0.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 1. Neuron model with absolute refractory mechanism.

The figure shows a schematic of the transition operator for the internal state variable of a spiking neuron with an absolute refractory period. The neuron can fire in the resting state and in the last refractory state .

https://doi.org/10.1371/journal.pcbi.1002211.g001

If the neuron is not refractory, i.e., , it can spike (i.e., a transition from to ) with probability(8)where is the standard sigmoidal activation function and the denotes the natural logarithm. The term is the current membrane potential, which depends on the current values of the variables for . The term in (8) reflects the granularity of a chosen discrete time scale. If it is very fine (say one step equals one microsecond), then is large, and the firing probability at each specific discrete time step is therefore reduced. If the neuron in a state with does not spike, relaxes into the resting state corresponding to a non-refractory neuron.

If the neuron is in a refractory state, i.e., , its new variable assumes deterministically the next lower value , reflecting the inherent temporal process:(9)

After the transition of the auxiliary variable , the binary variable is deterministically set to a consistent state, i.e., if and if .

It can be shown that each of these stochastic state transition operators leaves the given distribution invariant, i.e., satisfies equation (1). This implies that any composition or mixture of these operators also leaves invariant, see, e.g., [38]. In particular, the composition of these operators leaves invariant, which has a quite natural interpretation as firing dynamics of the spiking neural network : At each discrete time step the variables are updated for all neurons , where the update of takes preceding updates for with into account. Alternatively, one could also choose at each discrete time step a different order for updates according to [38]. The assumption of a well-regulated updating policy will be overcome in the continuous-time limit, i.e., in case where the neural dynamics are described as a Markov jump process. In the methods section we prove the following central theorem:

Variation of the discrete time model with a relative refractory mechanism.

For the previously described simple neuron model, the refractory process was assumed to last for time steps, exactly as long as the postsynaptic potentials caused by each spike. In this section we relax this assumption by introducing a more complex and biologically more realistic neuron model, where the duration of the refractory process is decoupled from the duration of a postsynaptic potential. Thus, this model can for example also fire bursts of spikes with an interspike interval . The introduction of this more complex neuron model comes at the price that one can no longer prove that a network of such neurons samples from the desired distribution . Nevertheless, if the sigmoidal activation function is replaced by a different activation function , one can still prove that the sampling is “locally correct”, as specified in equation (12) below. Furthermore, our computer simulations suggest that also globally the error introduced by the more complex neuron model is not functionally significant, i.e. that statistical dependencies between the RVs are still faithfully captured.

The neuron model with a relative refractory period is defined in the following way. Consider some arbitrary refractory function with , and for . The idea is that models the readiness of the neuron to fire in its state . This readiness has value when the neuron has fired at the preceding time step (i.e., ), and assumes the resting state when has dropped to . In between, the readiness may take on any non-negative value according to the function . The function does not need to be monotonic, allowing for example that it increases to high values in between, yielding a preferred interspike interval of a oscillatory neuron. The firing probability of neuron in state is given by , where is an appropriate function of the membrane potential as described below. Thus this function is closely related to the function (called afterpotential) in the spike response model [5] as well as to the self-excitation kernel in Generalized Linear Models [42]. In general, different neurons in the network may have different refractory profiles, which can be modeled by a different refractory function for each neuron . However for the sake of notational simplicity we assume a single refractory function in the following.

In the presence of this refractory function one needs to replace the sigmoidal activation function by a suitable function that satisfies the condition(11)for all real numbers . This equation can be derived (see Methods section Lemma 0) if one requires each neuron to represent the correct distribution over conditioned the variables . One can show that, for any as above, there always exists a continuous, monotonic function which satisfies this equation (see Lemma 0 in Methods). Unfortunately (11) cannot be solved analytically for in general. Hence, for simulations we approximate the function for a given by numerically solving (11) on a grid and interpolating between the grid points with a constant function. Examples for several functions and the associated are shown in Figure 2B and Figure 2C respectively. Furthermore, spike trains emitted by single neurons with these refractory functions and the corresponding functions are shown in Figure 2D for the case of piecewise constant membrane potentials. This figure indicates, that functions that define a shorter refractory effect lead to higher firing rates and more irregular firing. It is worth noticing that the standard activation function is the solution of equation (11) for the absolute refractory function, i.e., for and for .

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 2. Neuron model with relative refractory mechanism.

The figure shows the transition operator , refractory functions and activation functions for the neuron model with relative refractory mechanism. (A) Transition probabilities of the internal variable given by . (B) Three examples of possible refractory functions . They assume value when the neuron cannot spike, and return to value (full readiness to fire again) with different time courses. The value of at intermediate time points regulates the current probability of firing of neuron (see A). The x-axis is equivalent to the number of time steps since last spike (running from 0 to from left to right). (C) Associated activation functions according to (11). (D) Spike trains produced by the resulting three different neuron models with (hypothetical) membrane potentials that jump at time from a constant low value to a constant high value. Black horizontal bars indicate spikes, and the active states are indicated by gray shaded areas of duration after each spike. It can be seen from this example that different refractory mechanisms give rise to different spiking dynamics.

https://doi.org/10.1371/journal.pcbi.1002211.g002

The transition operator is defined for this model in a very similar way as before. However, for , when the variable was deterministically reduced by in the simpler model (yielding ), this reduction occurs now only with probability . With probability the operator sets , modeling the firing of another spike of neuron at this time point. The neural computability condition (4) remains unchanged, e.g., for a Boltzmann distribution. A schema of the stochastic dynamics of this local state transition operator is shown in Figure 2A.

This transition operator has the following properties. In Lemma 0 in Methods it is proven that the unique invariant distribution of , denoted as , gives rise to the correct marginal distribution over , i.e.

This means that a neuron whose dynamics is described by samples from the correct distribution if it receives a static input from the other neurons in the network, i.e., as long as its membrane potential is constant. Hence the “local” computation performed by such neuron can be considered as correct. If however, several neurons in the network change their states in a short interval of time, the joint distribution over is in general not the desired one, i.e., , where denotes the invariant distribution of . In the Methods section, we present simulation results that indicate that the error of the approximation to the desired Boltzmann distributions introduced by neural sampling with relative refractory mechanism is rather minute. It is shown that the neural sampling approximation error is orders of magnitudes below the one introduced by a fully factorized distribution (which amounts to assuming correct marginal distributions and independent neurons).

To illustrate the sampling process with the relative refractory mechanism, we examine a network of neurons. We aim to sample from a Boltzmann distribution (5) with parameters , being randomly drawn from normal distributions. For the neuron model, we use the relative refractory mechanism shown in the mid row of Figure 2B. A detailed description of the simulation and the parameters used is given in the Methods section. A spike pattern of the resulting sampling network is shown in Figure 3A. The network features a sparse, irregular spike response with average firing rate of . For one neuron , indicated with orange spikes, the internal dynamics are shown in Figure 3B. After each action potential the neuron’s refractory function drops to zero and reduces the probability of spiking again in a short time interval. The influence of the remaining network is transmitted to neuron via PSPs of duration and sums up to the fluctuating membrane potential . As reflected in the highly variable membrane potential even this small network exhibits rich interactions. To represent the correct distribution over conditioned on , the neuron continuously adapts its instantaneous firing rate. To quantify the precision with which the spiking network draws samples from the target distribution (5), Figure 3C shows the joint distribution of neurons. For comparison we accompany the distribution of sampled network states with the result obtained from the standard Gibbs sampling algorithm (considered as the ground truth). Since the number of possible states grows exponentially in the number of neurons, we restrict ourselves for visualization purposes to the distribution of the gray shaded units and marginalize over the remaining network. The probabilities are estimated from samples, i.e., from successive states of the Markov chain. Stochastic deviations of the estimated probabilities due to the finite number of samples are quite small (typical errors ) and are comparable to systematic deviations due to the only locally correct computation of neurons with relative refractory mechanism. In the Methods section, we present further simulation results showing that the proposed networks consisting of neurons with relative refractory mechanism approximate the desired target distributions faithfully over a large range of distribution parameters.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 3. Sampling from a Boltzmann distribution by spiking neurons with relative refractory mechanism.

(A) Spike raster of the network. (B) Traces of internal state variables of a neuron (# 26, indicated by orange spikes in A). The rich interaction of the network gives rise to rapidly changing membrane potentials and instantaneous firing rates. (C) Joint distribution of 5 neurons (gray shaded area in A) obtained by the spiking neural network and Gibbs sampling from the same distribution. Active states are indicated by a black dot, using one row for each neuron , the columns list all possible states of these neurons. The tight match between both distributions suggests that the spiking network represents the target probability distribution with high accuracy.

https://doi.org/10.1371/journal.pcbi.1002211.g003

In order to illustrate that the proposed sampling networks feature biologically quite realistic spiking dynamics, we present in the Methods section several neural firing statistics (e.g., the inter-spike interval histogram) of the network model. In general, the statistics computed from the model match experimentally observed statistics well. The proposed network models are based on the assumption of rectangular-shaped, renewal PSPs. More precisely, we define renewal (or non-additive) PSPs in the following way. Renewal PSPs evoked by a single synapse do not add up but are merely prolonged in their duration (according to equation (6)); renewal PSPs elicited at different synapses nevertheless add up in the normal way. In Methods we investigate the impact of replacing the theoretically ideal rectangular-shaped, renewal PSPs with biologically more realistic alpha-shaped, additive PSPs. Simulation results suggest that the network model with alpha-shaped PSPs does not capture the target distribution as accurately as with the theoretically ideal PSP shapes, statistical dependencies between the RVs are however still approximated reasonably well.

Neural sampling in continuous time

The neural sampling model proposed above was formulated in discrete time of step size , inspired by the discrete time nature of MCMC techniques in statistics and machine learning as well as to make simulations possible on digital computers. However, models in continuous time (e.g., ordinary differential equations) are arguably more natural and “realistic” descriptions of temporally varying biological processes. This gives rise to the question whether one can find a sensible limit of the discrete time model in the limit , yielding a sampling network model in continuous time. Another motivation for considering continuous time models for neural sampling is the fact that many mathematical models for recurrent networks are formulated in continuous time [5], and a comparison to these existing models would be facilitated. Here we propose a stochastically spiking neural network model in continuous time, whose states still represent correct samples from the desired probability distribution at any time . These types of models are usually referred to as Markov jump processes. It can be shown that discretizing this continuous time model yields the discrete time model defined earlier, which thus can be regarded as a version suitable for simulations on a digital computer.

We define the continuous time model in the following way. Let , for , denote the firing times of neuron . The refractory process of this neuron, in analogy to Figure 1 and equation (8)-(9) for the case of discrete time, is described by the following differential equation for the auxiliary variable , which may now assume any nonnegative real number :(12)

Here denotes Dirac’s Delta centered at the spike time . This differential equation describes the following simple dynamics. The auxiliary variable decays linearly with time constant when the neuron is refractory, i.e., . Once arrives at its resting state it remains there, corresponding to the neuron being ready to spike again (more precisely, in order to avoid point measures we set it to a random value in , see Methods). In the resting state, the neuron has the probability density to fire at every time . If it fires at , this results in setting , which is formalized in equation (12) by the sum of Dirac Delta’s . Here the current membrane potential at time is defined as in the discrete time case, e.g., by for the case of a Boltzmann distribution (5). The binary variable is defined to be 1 if and 0 if the neuron is in the resting state . Biologically, the term can again be interpreted as the value at time of a rectangular-shaped PSP (with a duration of ) that neuron evokes in neuron . As the spikes are discrete events in continuous time, the probability of two or more neurons spiking at the same time is zero. This allows for updating all neurons in parallel using a differential equation.

In analogy to the discrete time case, the neural network in continuous time can be shown to sample from the desired distribution , i.e., is an invariant distribution of the network dynamics defined above. However, to establish this fact, one has to rely on a different mathematical framework. The probability distribution of the auxiliary variables as a function of time , which describes the evolution of the network, obeys a partial differential equation, the so-called Differential-Chapman-Kolmogorov equation (see [43]):(13)where the operator , which captures the dynamics of the network, is implicitly defined by the differential equations (12) and the spiking probabilities. This operator is the continuous time equivalent to the transition operator in the discrete time case. The operator consists here of two components. The drift term captures the deterministic decay process of , stemming from the term in equation (12). The jump term describes the non-continuous aspects of the path associated with “jumping” from to at the time when the neuron fires.

In the Methods section we prove that the resulting time invariant distribution, i.e., the distribution that solves , now denoted as it is not a function of time, gives rise to the desired marginal distribution over :(14)where and if and otherwise. denotes Kronecker’s Delta with if and otherwise. Thus, the function simply reflects the definition that if and 0 otherwise. For an explicit definition of , a proof of the above statement, and some additional comments see the Methods section.

The neural samplers in discrete and continuous time are closely related. The model in discrete time provides an increasingly more precise description of the inherent spike dynamics when the duration of the discrete time step is reduced, causing an increase of (such that is constant) and therefore a reduced firing probability of each neuron at any discrete time step (see the term in equation (8)). In the limit of approaching , the probability that two or more neurons will fire at the same time approaches , and the discrete time sampler becomes equal to the continuous time system defined above, which updates all units in parallel.

It is also possible to formulate a continuous time version of the neural sampler based on neuron models with relative refractory mechanisms. In the Methods section the resulting continuous time neuron model with a relative refractory mechanism is defined. Theoretical results similar to the discrete time case can be derived for this sampler (see Lemmata 9 and 10 in Methods): It is shown that each neuron “locally” performs the correct computation under the assumption of static input from the remaining neurons. However one can no longer prove in general that the global network samples from the target distribution .

Demonstration of probabilistic inference with recurrent networks of spiking neurons in an application to perceptual multistability

In the following we present a network model for perceptual multistability based on the neural sampling framework introduced above. This simulation study is aimed at showing that the proposed network can indeed sample from a desired distribution and also perform inference, i.e., sample from the correct corresponding posterior distribution. It is not meant to be a highly realistic or exhaustive model of perceptual multistability nor of biologically plausible learning mechanisms. Such models would naturally require considerably more modelling work.

Perceptual multistability evoked by ambiguous sensory input, such as a 2D drawing (e.g., Necker cube) that allows for different consistent 3D interpretations, has become a frequently studied perceptual phenomenon. The most important finding is that the perceptual system of humans and nonhuman primates does not produce a superposition of different possible percepts of an ambiguous stimulus, but rather switches between different self-consistent global percepts in a spontaneous manner. Binocular rivalry, where different images are presented to the left and right eye, has become a standard experimental paradigm for studying this effect [44]–[47]. A typical pair of stimuli are the two images shown in Figure 4A. Here the percepts of humans and nonhuman primates switch (seemingly stochastically) between the two presented orientations. [16]–[18] propose that several aspects of experimental data on perceptual multistability can be explained if one assumes that percepts correspond to samples from the conditional distribution over interpretations (e.g., different 3D shapes) given the visual input (e.g., the 2D drawing). Furthermore, the experimentally observed fact that percepts tend to be stable on the time scale of seconds suggests that perception can be interpreted as probabilistic inference that is carried out by MCMC sampling which produces successively correlated samples. In [18] it is shown that this MCMC interpretation is also able to qualitatively reproduce the experimentally observed distribution of dominance durations, i.e., the distribution of time intervals between perceptual switches. However, in lack of an adequate model for sampling by a recurrent network of spiking neurons, theses studies could describe this approach only on a rather abstract level, and pointed out the open problem to relate this algorithmic approach to neural processes. We have demonstrated in a computer simulation that the previously described model for neural sampling could in principle fill this gap, providing a modelling framework that is on the one hand consistent with the dynamics of networks of spiking neurons, and which can on the other hand also be clearly understood from the perspective of probabilistic inference through MCMC sampling.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 4. Modeling perceptual multistability as probabilistic inference with neural sampling.

(A) Typical visual stimuli for the left and right eye in binocular rivalry experiments. (B) Tuning curve of a neuron with preferred orientation . (C) Distribution of dominance durations in the trained network under ambiguous input. The red curve shows the Gamma distribution with maximum likelihood on the data. (D) 2-dimensional projection (via population vector) of the distribution encoded in the spiking network showing that it strongly favors coherent global states of arbitrary orientation to incoherent ones (corresponding to population vectors of small magnitude). (E) 2-dimensional projection of the bimodal posterior distribution under an ambiguous input consisting of two different orientations reminiscent of the stimuli shown in A. The black trace shows the temporal evolution of the network state for 500 ms around a perceptual switch. (F) Network states at 3 time points marked in E. Neurons that fired in the preceding 20 ms (see gray bar in G) are plotted in the color of their preferred orientation. Inactive neurons are shown in white. While states and represent rather coherent orientations, shows an incoherent state corresponding to a perceptual switch. Clamped neurons (which the posterior is condition on) are marked by a black dot. (G) Spike raster of the unclamped neurons during a 500 ms epoch marked by the black trace in E. Gray bars indicate the 20 ms time intervals that define the network states shown in F. Altogether this figure shows that a theoretically rigorous probabilistic inference process can be carried out by a network of spiking neurons with a spike raster that is similar to generic recorded data.

https://doi.org/10.1371/journal.pcbi.1002211.g004

In the following we model some essential aspects of an experimental setup for binocular rivalry with grating stimuli (see Figure 4A) in a recurrent network of spiking neurons with the previously described relative refractory mechanism. We assigned to each of the 217 neurons in the network a tuning curve , centered around its preferred orientation as shown in Figure 4B. The preferred orientations of the neurons were chosen to cover the entire interval of possible orientations and were randomly assigned to the neurons. The neurons were arranged on a hexagonal grid as depicted in Figure 4F. Any two neurons with distance were synaptically connected (neighboring units had distance ). We assume that these neurons represent neurons in the visual system that have roughly the same or neighboring receptive field, and that each neuron receives visual input from either the left or the right eye. The network connections were chosen such that neurons that have similar (very different) preferred orientations are connected with positive (negative) weights (for details see Methods section).

We examined the resulting distribution over the dimensional network states. To provide an intuitive visualization of these high dimensional network states , we resort to a 2-dimensional projection, the population vector of a state (see Methods for details of the applied population vector decoding scheme). Only the endpoints of the population vectors are drawn (as colored points) in Figure 4D,E. The orientation of the population vector is assumed to correspond to the dominant orientation of the percept, and its distance from the origin encodes the strength of this percept. We also, somewhat informally, call the strength of a percept its coherence and a network state which represents a coherent percept a coherent network state. A coherent network state hence results in a population vector of large magnitude. Each direction of a population vector is color coded in Figure 4D,E, using the color code for directions shown on the right hand side of Figure 4F. In Figure 4D the distribution of the network is illustrated by sampling of the network for , with samples taken every millisecond. Each dot equals a sampled network state . In a biological interpretation the spike response of the freely evolving network reflects spontaneous activity, since no observations, i.e., no external input, was added to the system. Figure 4D shows that the spontaneous activity of this simple network of spiking neurons moves preferably through coherent network states for all possible orientations due to the chosen recurrent network connections (being positive for neurons with similar preferred orientation and negative otherwise). This can directly be seen from the rare occurrence of population vectors with small magnitude (vectors close to the “center”) in Figure 4D.

To study percepts elicited by ambiguous stimuli, where inputs like in Figure 4A are shown simultaneously to the left and right eye during a binocular rivalry experiment, we provided ambiguous input to the network. Two cells with preferred orientation and two cells with were clamped to . Additionally four neurons with resp. were muted by clamping to . This ambiguous input is incompatible with a coherent percept, as it corresponds to two orthogonal orientations presented at the same time. The resulting distribution over the state of the 209 remaining neurons is shown for a time span of of simulated biological time (with samples taken every millisecond) in Figure 4E. One clearly sees that the network spends most of the time in network states that correspond to one of the two simultaneously presented input orientations ( and ), and virtually no time on orientations in between. This implements a sampling process from a bimodal conditional distribution. The black line marks a trace of network states around a perceptual switch: The network remained in one mode of high probability – corresponding to one percept – for some period of time, and then quickly traversed the state space to another mode – corresponding to a different percept.

Three of the states around this perceptual switch (, and in Figure 4E) are explicitly shown in Figure 4F. Neurons that fired during the preceding interval of ms (marked in gray in Figure 4G) are drawn in the respective color of their preferred orientation. Inactive neurons are drawn in white, and clamped neurons are marked by a black dot ().

Figure 4G shows the action potentials of the non-clamped neurons during the same trace around the perceptual switch. One sees that the sampling process is expressed in this neural network model by a sparse, asynchronous and irregular spike response. It is worth mentioning that the average firing rate when sampling from the posterior distribution is only slightly higher than the average firing rate of spontaneous activity ( and respectively), which is reminiscent of related experimental data [7]. Thus on the basis of the overall network activity it is indistinguishable whether the network carries out an inference task or freely samples from its prior distribution. It is furthermore notable, that a focus of the network activity on the two orientations that are given by the external input can be achieved in this model, in spite of the fact that only two of the neurons were clamped for each of them. This numerical relationship is reminiscent of standard data on the weak input from LGN to V1 that is provided in the brain [48], [49], and raises the question whether the proposed neural sampling model could provide a possible mechanism (under the modelling assumptions made above) for cortical processing of such numerically weak external inputs.

The distribution of the resulting dominance durations, i.e., the time between perceptual switches, for the previously described setup with ambiguous input is shown for a continuous run of in Figure 4C (a similar method as in [18] was used to measure dominance durations, see Methods). This distribution can be approximated quite well by a Gamma distribution, which also provides a good fit to experimental data (see the discussion in [18]). We expect that also other features of the more abstract MCMC model for biological vision of [17], [18], such as contextual biases and traveling waves, will emerge in larger and more detailed implementations of the MCMC approach through the proposed neural sampling method in networks of spiking neurons.

Mathematical details

Details to the computer simulations

The simulation results shown in Figure 2, Figure 3 and Figure 4 used the biologically more realistic neuron model with the relative refractory mechanism. During all experiments the first second of simulated time was discarded as burn-in time. The full list of parameters defining the experimental setup is given in Table 1. All occurring joint probability distributions are Boltzmann distributions of the form given in equation (5). Example Python [64] scripts for neural sampling from Boltzmann distributions are available on request and will be provided on our webpage. The example code comprises networks with both absolute and relative refractory mechanism. It requires standard Python packages only and is readily executable.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. List of parameters of the computer simulations.

https://doi.org/10.1371/journal.pcbi.1002211.t001

Firing statistics of neural sampling networks

In previous sections it was shown that a spiking neural network can draw samples from a given joint distribution which is in a well-defined class of probability distributions (see the neural computability condition (4)). Here, we examine some statistics of individual neurons in a sampling network which are commonly used to analyze experimental data from recordings. The spike trains and membrane potential data are taken from the simulation presented in Figure 3.

Figure 5A,B exemplarily show the distribution of the membrane potential and the interspike interval (ISI) histogram of a single neuron, namely neuron which was already considered in Figure 3B. The responses of other neurons yield qualitatively similar statistics. The bell-shaped distribution of the membrane potential is commonly observed in neurons embedded in an active network [66]. The ISI histogram reflects the reduced spiking probability immediately after an action potential due the refractory mechanism. Interspike intervals larger than the refractory time constant roughly follow an exponential distribution. Similar ISI distributions were observed during in-vivo recordings in awake, behaving monkeys [67].

Figure 5C shows a scatterplot of the coefficient of variation (CV) of the ISIs versus the average ISI for each neuron in the network. The neurons exhibited a variety of average firing rates between and . Most of the neurons responded in a highly irregular manner with a CV . Neurons with high firing rates had a slightly lower CV due to the increased influence of the refractory mechanism The dashed line marks the CV of a Poisson process, i.e., a memoryless spiking behavior. The CV of neuron is marked by a cross. The structure of this plot resembles, e.g., data from recordings in behaving macaque monkeys [68] (but note the lower average firing rate).

Approximation quality of neural sampling with different neuron and synapse models

The theory of the neuron model with absolute refractory mechanism guarantees sampling form the correct distribution. In contrast, the theory for the neuron model with a relative refractory mechanism only shows that the sampling process is “locally correct”, i.e., that it would yield correct conditional distributions for each individual neuron if the state of the remaining network stayed constant. Therefore, the stationary distribution of the sampling process with relative refractory mechanism only provides an approximation to the target distribution. In the following we examine the approximation quality and robustness of sampling networks with different refractory mechanisms for target Boltzmann distributions with parameters randomly drawn from different distributions. Furthermore, we investigate the effect of additive PSP shapes with more realistic time courses.

We generated target Boltzmann distributions with randomly drawn weights and biases (excitabilities) and computed the similarity between these reference distributions and the corresponding neural sampling approximations. The setup of these simulations is the same as for the simulation presented in Figure 3. As we aimed to compare the distribution sampled by the network with the exact Boltzmann distribution , we reduced the number of neurons per network to . This resulted in a state space of possible network states for which the normalization constant for the target Boltzmann distribution could be computed exactly. The weight matrix was constraint to be symmetric with vanishing diagonal. Off-diagonal elements were drawn from zero-mean normal distributions with three different standard deviations , and , whereas the were sampled from the same distribution as in Figure 3. For every value of the hyperparameter we generated 100 random distributions. For Boltzmann distributions with small weights (), the RVs are nearly independent, whereas distributions with intermediate weights () show substantial statistical dependencies between RVs. For very large weights (), the probability mass of the distributions is concentrated on very few states (usually 90% on less than 10 out of the states). Hence, the range of the hyperparameter considered here covers a range a very different distributions.

The approximation quality of the sampled distribution was measured in terms of the Kullback-Leibler divergence between the target distribution and the neural approximation (21)

We estimated from samples for each simulation trial using a Laplace estimator, i.e., we added a priori to the number of occurrences of each state .

Table 2 shows the means and the standard deviations of the Kullback-Leibler divergences between the target Boltzmann distributions and the estimated approximations stemming from neural sampling networks with three different neuron and synapse models: the exact model with absolute refractory mechanism and two models with different relative refractory mechanisms shown in the bottom and middle row in Figure 2B. Additionally, as a reference, we provide the (analytically calculated) Kullback-Leibler divergences for fully factorized distributions, i.e., with correct marginals but independent variables for .

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Approximation quality of networks with different refractory mechanisms.

https://doi.org/10.1371/journal.pcbi.1002211.t002

The absolute refractory model provides the best results as we expected due to the theoretical guarantee to sample from the correct distribution (the non-zero Kullback-Leibler divergence is caused by the estimation from a finite number of samples). The models with relative refractory mechanism provide faithful approximations for all values of the hyperparameter considered here. These relative refractory models are characterized by the theory to be “locally correct” and turn out to be much more accurate approximations than fully factorized distributions if substantial statistical dependencies between the RVs are present (i.e., , ). As expected, a late recovery of the refractory function is beneficial for the approximation quality of the model as it is closer to an absolute refractory mechanism. Figure 6 shows the full histograms of the Kullback-Leibler divergences for the intermediate weights group (). Systematic deviations due to the relative refractory mechanism are on the same order as the effect of estimating from finite samples (as can be seen, e.g., from a comparison with the absolute refractory model which has 0 systematic error). For completeness, we mention that the divergences of the fully factorized distributions of out of the networks with are not shown in the plot.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 6. Comparison of neural sampling with different neuron and synapse models.

The figure shows a histogram of the Kullback-Leibler divergence between different Boltzmann distributions over K = 10 variables (with parameters randomly drawn, see setup of Figure 3) and approximations stemming from different neural sampling networks. Networks with absolute refractory mechanism provide the best approximation (as expected from theoretical guarantees). Networks consisting of neurons with relative refractory mechanisms, with only “locally” correct sampling, also provide a close fit to the true distribution (see inset) compared to a fully factorized approximation (assuming correct marginals and independent variables). Furthermore, it can be seen that sampling networks with more realistic, alpha-shaped, additive PSPs still fit the true distribution reasonably well.

https://doi.org/10.1371/journal.pcbi.1002211.g006

The theorems presented in this article assumed renewed (i.e., non-additive), rectangular PSPs. In the following we examine the effect of additive PSPs with more realistic time courses. We define additive, alpha-shaped PSPs in the following way. The influence of each presynaptic neuron on the postsynaptic membrane potential is modeled by convolving the input spikes with a kernel :(22)where for and for , and for are the spike times of the presynaptic neuron . The time constant governing the rising edge of the PSPs was set to . The time constant controlling the falling edge was chosen equal to the duration of rectangular PSPs, . The scaling parameter was set such that the time integral over a single PSP matches the time integral over the theoretically optimal rectangular PSP, i.e., . These parameters display a simple and reasonable choice for the purpose of this study (an optimization of , and is likely to yield an improved approximation quality). Figure 7A shows the resulting shape of the non-rectangular PSP. Furthermore the time course of the function caused by a single spike of neuron is shown in order to illustrate that the time constants of and of a PSP are closely related due to the assumption made above. Preliminary and non-exhaustive simulations seem to suggest that the choice yields better approximation quality than setting or ; however it is very well possible that a mismatch between and can be compensated for by adapting other parameters, e.g., the PSP magnitude or a specific choice of the refractory function . Figure 7B shows the results of an experiment, similar to the one presented in Figure 3C , with additive, alpha-shaped PSPs and relative refractory mechanism. While differences to Gibbs sampling results are visible, the spiking network still captures dependencies between the binary random variables quite well.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 7. Sampling from a Boltzmann distribution with more realistic PSP shapes.

(A) The upper panel shows the shape of a single PSP elicited at time . The lower panel shows the time course of the refractory function caused by a single spike of neuron at . The grey-shaded area of length indicates the interval of neuron being active (i.e., ) due to a single spike of neuron at time . (B) Shown is the probability distribution of 5 out of 40 neurons. The plot is similar to Figure 3C, however it is generated with a sampling network that features alpha-shaped, additive PSPs. It can be seen that the network still produces a reasonable approximation to the true Boltzmann distribution (determined by Gibbs sampling).

https://doi.org/10.1371/journal.pcbi.1002211.g007

For a quantitative analysis of the approximation quality, we repeated the experiment of Figure 6 with additive, alpha-shaped PSPs (shown as green bars). The Kullback-Leibler divergence to the true distribution is clearly higher compared to the case of renewed, rectangular PSPs. Still networks with this more realistic synapse model account for dependencies between the random variables and yield a better approximation of than fully factorized distributions.