Spiking neural network model.

The network architecture in Fig 1D comprises network neurons and input neurons. The spiking activity of input neurons is fed externally into the network. For the network neurons, we employ the stochastic spike response neuron model from Buesing et al. [17] that describes the state of each network neuron by a binary variable z_k(t) ∈ {0,1} according to Eq (1): After a spike of the k-th network neuron, z_k(t) turns active for duration τ. After that period, the variable switches back to the inactive state z_k(t) = 0. Similarly, the state of the i-th input neuron is described by a binary variable y_i(t) ∈ {0,1} with y_i(t) = 1 for duration τ after a spike of the i-th input neuron. The membrane potential of the k-th network neuron is given by (3) Here, b_k denotes a neuron’s intrinsic excitability and captures, for instance, the influence of the voltage gap between resting and threshold potential in more detailed neuron models [19, 33]. and are recurrent synaptic weights between network neurons to instantiate sparse recurrent excitation [30] and local lateral inhibition [34]. For simplicity, we model the effect of inhibition as direct negative connections between network neurons, i.e., we do not model interneurons explicitly. The afferent weights V_ki denote the strength of synapses from the i-th input to the k-th network neuron. For notational convenience, weight 0 is assigned to non-existing connections. As in [17], neurons communicate via rectangular post-synaptic potentials (PSPs) of duration τ and with amplitude , and V_ki respectively. Throughout this work, we chose τ = 10 ms as an estimate for PSP durations. Thus, the membrane potential u_k(t) integrates the current value of all presynaptic variables z_j(t) and y_i(t) at any time.

Network neurons emit spikes stochastically with instantaneous firing probability (4) Here, (1 − z_k(t)) describes a refractory period that is inversely related to the state z_k(t), i.e., when z_k(t) is active, the neuron is refractory and cannot emit another spike. The exponential dependence of the firing probability ρ_k on u_k was confirmed to be a reasonable modeling assumption for neurons in a noisy environment [19, 33].

The network response as the result of a meaningful Bayesian computation.

We aim to understand the activity of the network neurons in response to spiking input as the result of a Bayesian computation. To establish a link between the stochastic spike response properties of individual neurons and the joint activity distribution of the entire network, we build upon the neural sampling theory [17] where states z_k(t) and y_i(t) of the neurons are formally treated as random variables z_k and y_i. This probabilistic description of stochastic network activity in response to given input amounts to a conditional probability distribution p (z ∣ y, θ) for each possible input configuration y. In order to assign a computational meaning to the conditional distributions p (z ∣ y, θ), we will identify a generative model, consisting of prior p(z ∣ θ) and likelihood p (y ∣ z, θ), for which the conditional distributions p (z ∣ y, θ) arise from Bayes rule (2) as the correct posterior. The network’s spike response can then be understood as probabilistic inference through sampling from the posterior p (z ∣ y, θ).

To decide whether the posterior distribution of a generative model is compatible with the neural dynamics of the spiking network, we make use of a sufficient condition, called neural computability condition [17], which connects the membrane potentials u_k of individual spike response neurons (4) with the activity distribution of the recurrent network. For our case of a posterior distribution p (z ∣ y, θ) with arbitrary (but fixed) input y, the neural computability condition reads: (5) with z_\k = (z₁, .., z_k−1, z_{k + 1}, .., z_K) denoting the state of the remaining network. If the condition holds for all possible input states y and all possible network states z, the trajectory of network states z(t) is guaranteed to be distributed according to p (z ∣ y, θ) in the limit t → ∞ for any fixed input y, i.e., the spiking network samples from p (z ∣ y, θ).

To establish the equivalence between the spiking network in Fig 1D and the Bayesian model in Fig 1E, we introduce prior p(z ∣ θ) and likelihood distributions p (y ∣ z, θ) which are shaped by parameters θ and match the graphical model in Fig 1E. From these distributions, we then calculate the posterior p (z ∣ y, θ) according to Bayes rule (2). Finally, we apply the sufficient condition (5) to decide under what conditions the spiking network will sample from the correct posterior. In particular, this will allow us to determine the connectivity structure of the network in detail, and furthermore, to precisely map the parameters of the generative model to the synaptic efficacies and intrinsic excitabilities of the neural sheet model.

We set out with a class of prior distributions p(z ∣ θ), namely Boltzmann distributions, that have been shown [17] to be compatible with the neural sampling dynamics of the recurrent spiking network model (without input): (6) where Z is a normalizing factor. The distribution assigns probabilities to binary random vectors z = (z₁, .., z_K)^⊺ with z_k {0,1}. The parameters shape the distribution and consist of a real-valued bias vector and symmetric, zero-diagonal K × K coupling matrices and . We endow the prior (6) with the spatial structure sketched in the upper row of Fig 1E. Orange circles depict the random variables z_k that correspond to network neurons. The random variables z_k maintain sparse excitatory recurrent connections (red links) on an intermediate range. Excitatory links between variables make their coactivation more probable in the prior (6) and thus encode network-wide state combinations z that are more likely to occur than others. This associative memory aspect will turn out to be particularly powerful in the context of recurrent learning where structural knowledge is integrated by the prior and will allow the network to maintain coherent network states even in face of ambiguous input. The sparse excitatory links on intermediate distances are complemented with strong negative connections (theoretically , blue links) on a local scale among nearby network neurons. These negative connections will correspond to local lateral inhibition in the network and, as we will see in the context of learning, facilitate self-organized statistical model optimization through synaptic plasticity.

The likelihood distribution p (y ∣ z, θ) establishes the connection between the network variables z_k and the input variables y_i. We adopt a generative perspective in order to identify the likelihood distribution p (y ∣ z, θ) and view the network variables z_k as “hidden causes” of their inputs y_i. Thus, we assume that each input y_i is (hypothetically) generated by the corresponding subset of connected network variables z_k. This amounts to the downward arrows in Fig 1E where each input y_i (green circles) receives converging arrows only from nearby network variables z_k, the so-called parents of y_i. For our idealized architecture, we further assume that no two parents z_k and z_j of an individual input y_i can be active simultaneously. This is ensured by z_k and z_j sharing strong negative connections , i.e., the parents of y_i are assumed to lie within the range of lateral inhibition (blue links in Fig 1E). As a consequence, each input y_i is generated by at most one of its parents at a time. We denote the probability of y_i to be active by π_ki (with 0 < π_ki < 1) given that it is generated by the active hidden cause z_k. If none of its parents is active, the chance of activity is assigned a constant default value π_0i. This is summarized in the following Bernoulli distribution: (7) To obtain the full likelihood distribution p (y ∣ z, θ) over all N input variables, we note that the y_i’s are conditionally independent and, hence, . This completes the definition of prior and likelihood distributions that match the graphical model sketched in Fig 1E. A formal definition of the probabilistic model is provided in Generative model in Methods. There we also describe extensions of the likelihood distribution to support natural and real-valued inputs, i.e., to y_i ∈ ℕ and y_i ∈ ℝ.

The neural network can sample from the posterior of the generative model.

From the prior (6) and the likelihoods (7), the posterior distribution p (z ∣ y, θ) can be calculated in closed-form using Bayes rule. By applying the neural computability condition (5), a straightforward derivation reveals that the membrane potential (3) is compatible with the posterior p (z ∣ y, θ) of the generative model. The closed-form posterior and the derivation are provided in Inference in the generative model in Methods. Furthermore, the calculation yields a translation between the network parameters and the abstract parameters (and the constants π_0i), and results in the following corollary:

Corollary 1 (Inference).

Let network parameters and abstract parameters be identified via (8) (9) (10) with , and let the recurrent weights and be symmetric matrices with zero diagonal. Then, for any fixed input instantiation y(t) = y, the response z(t) of a recurrent spiking network consisting of stochastic neurons (4) with linear membrane potential (3) is distributed according to in the limit t → ∞, i.e. the network samples from the posterior distribution of the probabilistic model defined by the prior (6) and the likelihoods (7).

Corollary 1 explains the structural similarity of the graphical model in Fig 1E and the network architecture in Fig 1D: Each red (blue) recurrent link between latent variables z_k and z_j in the graph corresponds to a symmetric reciprocal excitatory (inhibitory) synaptic connection between the k-th and j-th network neuron; and each downward arrow from z_k to y_i in the graph gives rise to a synapse V_ki from the i-th input neuron to the k-th network neuron. In particular, the assumption in the abstract model, that at most one parent z_k explains a dependent input variable y_i at a time, translates to a local lateral inhibition motif in the spiking network: Any two network neurons, that share common input, inhibit each other through lateral inhibition. This theoretically derived network motif is reminiscent of cortical lateral inhibition frequently reported across areas and species [29]. In the neural sheet model, local lateral inhibition introduces Winner-take-all (WTA) competition among nearby network neurons which thus play the role of local feature detectors. However, the local WTA circuits are not separated in the sheet, but rather interwoven such that each network neuron can participate in multiple overlapping WTA sub-circuits. In addition, lateral excitatory connections encode associations between spatially distant feature combinations. This generalized concept of WTA circuits in the continuous sheet contrasts with existing models of interacting WTAs [35] which investigated disjoint non-overlapping WTA sub-circuits that operate in parallel. The full probabilistic model p(y, z ∣ θ) of the neural sheet can thus be understood as a spatially extended reservoir of contiguous competing local feature detectors (corresponding to the likelihood p (y ∣ z, θ)) and an associative memory over the feature set (corresponding to the prior p(z ∣ θ)).

The implementation of Bayes rule (2) by the spiking network is illustrated in Fig 2A and 2B for the minimal example of two neighboring neurons with an overlapping, but not identical, subset of inputs y_i. Both neurons z₁ and z₂ maintain an implicit likelihood model for their respective local inputs (top-down arrows in A, bottom-up synapses in B). Additionally, the prior installs competition between the neurons via negative reciprocal connections W^inh (blue edge in A, blue synapses in B). Consequently, the neurons z₁ and z₂ preferentially fire to different input patterns y. Corollary 1 guarantees that the frequency of occurrence of each network state (z₁, z₂) will be proportional to p (z₁, z₂ ∣ y, θ) for any input instantiation y. This property is particularly important for inputs y that are roughly equally compatible with the preferred activity patterns of both network neurons. In such ambiguous cases, the network trajectory will sample the posterior states (z₁, z₂) = (0,0), (0,1) and (1,0) in accordance with Bayes rule. As a consequence, the network response carries information on not only the most likely solution but also on the (un-)certainty of all possible outcomes—a pivotal aspect of Bayesian information processing.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Sheets of spiking neurons can perform Bayesian inference on distributed spiking input.

(A) Local generative model with two competing hidden causes and five inputs. Each hidden cause stores a specific input pattern in the top-down parameters π_ki. (B) Corresponding local neural network. Top-down parameters π_ki translate to bottom-up synaptic weights V_ki, turning each network neuron into a probabilistic expert for a specific local input pattern. (C) Example network with six network neurons. Neighboring neurons with overlapping input inhibit each other (dashed line: range of lateral inhibition for red neuron). The spiking network is linked to a generative model p (y, z ∣ θ) according to Corollary 1. (D) The network in C performs sample-based inference of hidden causes z under time-varying input y(t) in the associated generative model. Comparison of posterior marginals p (z_k = 1 ∣ y(t), θ) between the analytically calculated exact posterior (top) and the average network response (bottom, estimated from 1000 simulation runs) under the time-varying input y(t) in panel E. All traces were smoothed with a 20ms box kernel for visual clarity. (E) Top: Network spike response in a single simulation run. Bottom: Input spike trains (colored: structured input, gray: background activity). The network response is an ongoing sampling process from the posterior p (z ∣ y(t), θ). Time points marked by a ‘*’ exemplify characteristic properties of sample-based Bayesian information processing. (F) Top: Bottom-up weights V_ki of the red neuron. Bottom: Average input activity while the matching pattern is presented (during the 300ms period marked in E).

https://doi.org/10.1371/journal.pone.0134356.g002

Demonstration of sample-based inference on transient spiking input.

Corollary 1 ensures that the network will sample from the correct posterior distribution p (z ∣ y, θ) in arbitrarily large network architectures for any fixed input configuration y(t) = y. However, Corollary 1 offers no strict guarantees in case of time-varying input since it only ensures that the network activity converges to the equilibrium distribution p (z ∣ y, θ) in the limit t → ∞. For transient spiking input, the underlying Markov chain cannot fully converge to its equilibrium distribution and the sampling network will “lag behind” the transient posterior p (z ∣ y(t), θ).

We therefore assessed the network’s ability to sample from the analytically correct posterior distribution in case of time-varying input y(t) in a computer simulation. The computer simulation was performed in the setup of the small, analytically tractable network architecture shown in Fig 2C. This 6-neuron network will furthermore serve to illustrate salient response properties of the neural sheet during neural sampling-based inference. The prior features both sparse excitatory connections and local lateral inhibition such that neighboring neurons compete with each other through inhibitory connections. More distant neurons maintain excitatory connections (red links: ), representing knowledge that these two hidden causes are likely to co-occur. Furthermore, each neuron has a (randomly generated) preferred local input pattern stored in its synaptic weights V_ki. For instance, the top panel of Fig 2F shows the afferent weights of the red neuron. A demonstration of neural sampling by this circuit in response to time-varying input y(t) is shown in Fig 2E: High-dimensional (N = 108) input spike trains are presented to the network (bottom). Input spikes consist of uninformative background spikes (gray), interleaved with periods of more structured inputs (colored spikes). Uninformative background spikes are Poisson spike trains with a uniform rate chosen such that the average activation of input neurons is 〈 y_i 〉 = 0.2 (ca. [20]Hz). Structured input spikes are Poisson spike trains with rates that were chosen to match the activity patterns stored in the afferent weights V_ki of the network neurons. For example, the red neuron is particularly good at detecting the red spike pattern. Fig 2E (top) shows the network response to the input spike pattern (bottom) during a single simulation run. Whenever a structured local input pattern occurs, the corresponding network neuron starts firing. Yet, the network response is not deterministic: sometimes network neurons elicit spikes even when presented with unstructured input (see e.g. at the 1^st time point marked by a ‘*’); in other cases, a competing neighboring neuron emits a spike (see e.g. at the 2^nd time point marked by a ‘*’). Due to the stochastic nature of the network response, the exact spike pattern of the network will be slightly different in each simulation run. This apparent trial-to-trial variability is an inherent feature of sample-based representations.

We next turn to the question how well the network trajectory z(t) approximates the correct Bayesian posterior p (z ∣ y(t), θ) at any time t. A quantitative comparison addressing this question is provided in Fig 2D. From many repetitions of the experiment—all with the same input spike pattern y(t)—we can estimate the distribution the network actually samples from. The top row of Fig 2D shows the exact marginal probabilities p_theo (z_k = 1 ∣ y(t), θ) at any time t, analytically calculated from Bayes rule. The bottom row shows the average network response p_net (z_k = 1 ∣ y(t), θ), estimated from the samples from 1000 repetitions of the experiment. The comparison indicates that the sampling network approximates the correct posterior probabilities with high accuracy at almost any time, capturing not only qualitative aspects of the transient posterior distribution but also the quantitative composition of the distribution in face of input fluctuations. Only for particularly rapid input fluctuations, which lead to sharp peaks in the posterior, the network shows a slightly delayed and sometimes inaccurate response. A more detailed statistical evaluation of the sampling quality is provided in Details to the computer simulations in Methods, along with a brief discussion on the origins of stochastic and systematic deviations in the sampled distribution. In conclusion, the quantitative comparison in Fig 2D shows that the stochastic network response z(t) can be understood as an ongoing Bayesian inference process, even in case of time-varying input y(t).

Fig 2D and 2E also exemplify characteristic properties of sample-based Bayesian information processing. At times, input instantiations y(t) may be noisy or ambiguous such that the hidden causes of the presented input cannot be inferred with certainty. Two typical examples are marked with a ‘*’ symbol: At the first time point marked, the unstructured background input accidentally bears some resemblance to the red and the yellow input patterns, such that the posterior probability for inferring the red/yellow patterns temporarily jumps to p (z_k = 1 ∣ y(t), θ) ≈ 1/2 for k = 3 and k = 5. This brief moment of uncertainty due to stochastic fluctuations in the input is represented by the network via a small number of spikes of the red (k = 3) and the yellow (k = 5) network neurons. While the exact timing of the spikes is a stochastic process, the instantaneous spiking probability in the sampling network is well in line with the analytically calculated posterior. At the second time point marked, two competing neurons (dark blue and green) could explain their local input well, as can be seen from the marginal posterior p (z_k = 1 ∣ y, θ). In such ambiguous situations, both neurons are eager to fire, yet they compete due to lateral inhibition. As a result, network activity switches between two local interpretations where either one of the two hidden causes is active. Due to the stochastic nature of the sampling process, the particular switching times between the competing network states change with each repetition of the experiment, leading to trial-to-trial variability from the perspective of an external observer.

Corollary 2 (Learning of afferent synapses).

The implicit probabilistic model defined by (6) and (7) can be optimized with respect to the afferent synaptic parameters V_ki by a Generalized Expectation Maximization algorithm which continuously increases a lower bound of the log-likelihood ℒ(θ) until a local optimum of ℱ is reached. The concurrent application of the learning rules (12) and (13) implements an online approximation of this optimization algorithm.

The approximate character of the network implementation only arises from incomplete convergence of the network’s Markov chain during inference and from the non-infinitesimally small learning rates η_V and η_b. Ignoring these effects, i.e., in the limit of small learning rates (η_b → 0 and η_V/η_b → 0) and assuming instantaneous convergence of the Markov chain, we find that the plasticity rules become exact. Then, the direction of the expected learning update is given by, (14) where the expectation 〈⋅〉 is taken with respect to the presented input y(t) and the network response z(t). In other words, plastic changes in the synaptic weights V_ki point on average in the direction of increasing ℱ. Maximizing a lower bound ℱ on ℒ, instead of direct optimization of ℒ, is a common trick [37] in machine learning to obtain a tractable learning problem. In our model this approach serves to obtain a spiking network implementation of maximum likelihood learning in which all required information is available locally at the neurons and synapses. The local availability of information during learning comes at a cost. The network does not sample from the exact posterior anymore, but from a well-defined variational posterior [36]: The variational posterior, the network samples from, is the closest distribution to the analytically exact posterior (measured in terms of the Kullback-Leibler divergence) that satisfies the homeostatic long-term target activations m_k. A full derivation of the plasticity rules (12) and (13), including a precise definition of the lower bound ℱ and the variational posterior distribution, is provided in Model optimization via Generalized Expectation Maximization in Methods.

Demonstration of self-organized learning in the neural sheet.

We tested the derived learning rules for afferent weights V_ki and intrinsic excitabilities b_k in a computer simulation of a sheet of 7×3 network neurons (Fig 3A). The sheet model receives synaptic input from a total 21×6 afferent cells. Each network neuron receives input from a subset of 6×6 inputs. The network neurons are arranged such that in each column there are three neurons sharing the same 6×6 input (such as the green/red/blue neuron in Fig 3A). Neighboring columns of network neurons receive inputs from an overlapping subset of afferent cells (6×6 input subset shifted by three columns of input neurons to the left and right respectively). Neurons in the same column as well as neurons in neighboring columns inhibit each other. The input data was generated in a similar manner as in Fig 2, by interleaving a background Poisson spike train with periods of structured Poisson spikes generated from a small number of stereotypical rate patterns. At each 6×6 input field there are three such recurring activity patterns. At any moment, at most one such pattern is presented at any spatial position in the input. The resulting spike trains have complex spatio-temporal structure, as shown in Fig 3G (bottom).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Emergence of probabilistic local experts through synaptic plasticity.

(A) Network architecture with 21×6 inputs and 7×3 network neurons. The green, red and blue neuron receive input from the same 6×6 subset of input neurons. The input subsets of neighboring groups are shifted by three. The dashed line indicates the range of lateral inhibition. (B) At each of the overlapping 6×6 locations, three randomly drawn activity patterns can occur. Shown are the activity patterns for the location highlighted in A. (C) Synaptic and intrinsic plasticity shape the probabilistic model that is encoded by the network. The log-likelihood function ℒ(θ) measures how well the network is adapted to the presented input. (D) One-to-one comparison of the synaptic weights V_ki to analytically calculated optimal weight values, at the end of learning (T = [10, 000] s). (E) 2-dimensional projection of the local input distribution p*(y). Each dot is one input instantiation y(t) at the 6×6 input field in A. Dots are colored according to which of the three neurons (green/red/blue) fired in response (gray: none fired). The neural plasticity rules achieve a clustering of local inputs into local categories. (F) Evolution of synaptic weights V_ki of the green/red/blue neuron over the course of learning. Each neuron becomes a probabilistic local expert for a certain input pattern (cp. panel B). (G) The plastic network develops a sparse, structured spike code that conveys compressed information about the presented input. Bottom: input spike trains. Top: Network response at different stages of learning.

https://doi.org/10.1371/journal.pone.0134356.g003

Initially all afferent weights are set to V_ki = 0. Afferent weights are plastic and follow Eq (12). For the purpose of neuroscientific modeling, synaptic weights V_ki were restricted to positive values. The theory for inference and learning would support positive and negative weights, including sign changes. Excitabilities are uniformly initialized at b_k = −2 and follow Eq (13) with the average target response 〈z_k〉 of each network neuron set to m_k = 6.5%. Whenever a network neuron spikes, synaptic plasticity is triggered and the current activity of the local 6×6 input field, that is connected to the active network neuron, leaves a small trace in the afferent synaptic weights of that neuron. As a result, the same neuron is more likely to fire again when a similar input pattern occurs in the future, which leads to further strengthening of the synaptic weights. Due to the combined effect of synaptic plasticity and local competition among neurons, network neurons start specializing on different salient input patterns of their respective 6 × 6 input fields (Fig 3F). Homeostatic intrinsic plasticity ensures during this learning process that neurons which specialize on weak patterns (weak synaptic weights) are not disadvantaged compared to neurons which focus on strong patterns (strong synaptic weights), by regulating intrinsic excitabilities such that all network neurons maintain their long-term average activity m_k. An example that illustrates how homeostatic intrinsic plasticity contributes to synaptic learning of input patterns with different intensities is provided in Illustration of learning with homeostatic intrinsic plasticity in Methods.

A direct consequence of the gradual specialization of network neurons on local salient input patterns is that the network response becomes increasingly more structured and reliable during learning (Fig 3G, top): each network neuron becomes a probabilistic local expert for one of the salient activity patterns in its local 6×6 input field. This assigns a particular meaning to each of the random variables z_k of the network: The activity of neuron z_k represents the presence or absence of the salient local input feature which is encoded in its afferent weights V_ki. Nearby network neurons (such as the blue/red/green neurons in Fig 3A) are in competition due to lateral inhibition. Therefore, whenever an input is presented to the network, the activity in the local 6×6 input field is effectively categorized by similarity to the preferred patterns of the three neurons. This is visualized in Fig 3E: Each dot in the 2D-projection represents one instantiation of the local 6×6 input field highlighted in Fig 3A. Grey dots indicate that none of the three neurons (green/red/blue) fired in response to the local input (≈ 80% of the time in accordance with the targets m_k). Colored dots indicate that the respective network neuron fired (colors as in Fig 3A). After learning, the input space is segmented into three regions (Fig 3E, right). Ambiguous input instances at borders between regions evoke probabilistic responses (e.g. between blue and red region). In this case, the network stochastically responds with one of the two (or three) possible interpretations in order to approximate the posterior probabilities of each hidden cause. The resulting representation on the network level is a sparse structured spike response that conveys highly compressed information about the input.

The qualitative changes in network behavior described above are paralleled by a quantitative improvement of network performance measured by the log-likelihood ℒ(θ) (Fig 3C), as predicted by Corollary 2: From the perspective of statistical model optimization, the learning dynamics due to synaptic and homeostatic plasticity guide a local stochastic search in parameter space which on average increases the lower bound ℱ of the log-likelihood ℒ(θ). At the end of learning, after T = [10, 000] s, the network has identified a faithful representation of the actually presented local activity patterns (see Fig 3D). For the shown comparison, theoretically optimal weights were calculated from the presented activity patterns according to Eq (9). The reason for the small but systematic differences (learned weights are a bit stronger than predicted) can be found in the facts that the network samples from a variational posterior distribution and that the plasticity rules (12) and (13) optimize a low bound ℱ instead of the log-likelihood ℒ.

In summary, we have demonstrated in this subsection how a neural-sampling network can adapt its internal parameters to perform probabilistic inference on distributed spiking input streams. The theoretically derived plasticity rules (12) and (13) enable the sampling network to develop a sparse and reliable spike code that carries the most salient information of the input stream. This statistical optimization process evolves in a fully self-organized manner by turning network neurons into probabilistic local experts that compete in explaining the presented spike input according to the rules of probability theory.

The distinct role of lateral inhibition for synaptic learning.

Before we address plasticity of recurrent synapses, an important contribution of inhibition to synaptic learning deserves a brief discussion. The simplicity of the synaptic plasticity rule (12) arises from the salient lateral inhibition network motif. To identify the role of lateral inhibition for synaptic learning, it is instructive to review the derivation of (12) in the absence of inhibition. By repeating the derivation with W^inh = 0, we obtain: (15) The sum on the right-hand side in (15) depends on the activity of neighboring cells z_j, j ≠ k, as well as on their afferent weights V_ji, thereby rendering statistically optimal learning non-local. In contrast, local inhibition introduces competition among nearby neurons such that each input variable y_i is explained by at most one hidden cause z_k at a time, and, as a consequence, the complex non-local term in Eq (15) vanishes. Notably, this outcome is not an artifact of the specific probabilistic model we use, but rather is a general consequence of explaining away effects in any graphical model with converging arrows. This finding suggests that lateral inhibition among nearby neurons assists synaptic learning in a Bayesian framework of model optimization. On the other hand, when lateral inhibition extends beyond neurons with shared afferent input, the expressive power of the probabilistic model is reduced since less hidden causes z_k are allowed to be active simultaneously. These theoretical considerations suggest that the emergence of efficient representations benefits from network architectures in which the range of local inhibition matches the spatial extent of excitatory cells that share common afferent input.

Demonstration of self-organized integration of structural knowledge.

We tested the learning capabilities of the simple plasticity rule (17) and compared it with the theoretically optimal wake-sleep rule (16) in a computer simulation with seven spatially separate network populations (see Fig 4A). Each population consisted of three neurons and received spiking input from a group of 6×6 inputs. Within each population neurons are subject to lateral inhibition. Neurons from different populations are linked via all-to-all recurrent excitatory connections. At each spatially separate 6×6 input location one of three local activity patterns may occur. The local activity patterns are simple stripe patterns (see Fig 4B top) that can easily be learned by the afferent weights V_ki. For brevity, we refer to the three activity patterns as the ‘red’, ‘green’ and ‘blue’ pattern in the following.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Plastic recurrent synapses integrate structural knowledge.

(A) Network architecture with seven recurrently connected local populations (all-to-all beyond inhibition). Input weights V_ki and recurrent excitatory weights are plastic. One of three stripe activity patterns (visualized as red/green/blue) is presented as spiking input to each local population. (B) Examples of the input structure. Top: input activity of the strip patterns (250 ms average). Bottom: The two outer locations serve as left and right cue (LQ,RQ). Cue input patterns are chosen independently. The cues determine the type (“color”) of the inner patterns in that the inner inputs are always different from both cues. In addition, inner patterns are always consistent. Hence, inner network neurons must consider both cues and the state of the other inner network neurons to infer their own state. (C) Recurrent weights of the highlighted blue-tuned neuron after 25, 000 s of learning with the simple local plasticity rule (17). Network neurons are colored according to the local pattern they have become experts for. Line width encodes the synaptic weight (min. line width 0.2 for ). In accordance with the input structure, the inner blue neuron has developed strong excitatory connections to the other inner blue neurons and moderately strong connections to the red and green cue neurons. (D) Comparison of the log-likelihood over the first 10, 000 s of learning for three recurrent plasticity conditions: with the simple recurrent plasticity rule (red), with the wake-sleep algorithm (dashed black), and without recurrent excitation (dashed gray). Recurrent plasticity significantly enhances the network’s learning capabilities. (E) The knowledge on the input structure, that was learned by the recurrent synapses , enables inference of correct global network states in face of incomplete input. Only the outer cues are presented while all inner inputs show an uninformative (“gray”) pattern. With red and green cues, the network correctly infers that the inner hidden causes should be blue, most of the time. Horizontal bars: mean activity 〈z_k〉 of network neurons (average over 100 s). (F) When both outer cues are blue, the already incomplete input is furthermore ambiguous. During inference, the network switches stochastically between the two consistent global interpretations.

https://doi.org/10.1371/journal.pone.0134356.g004

In order to assess the network’s ability to detect and integrate highly interdependent correlation structures among features at different locations, we introduced complex dependencies between the input presented to the seven populations: While the two outer input locations served as a left cue (LQ) and right cue (RQ), and were chosen independently, the five inner locations were chosen to be (a) different from the cues, and (b) consistent with each other. For instance, if the outer cue patterns were red and green, all inner input patterns were blue. If, however, both cues showed the same pattern (e.g., both blue), an ambiguous situation arose: In this case, the inner patterns were still chosen to be different from the cue, but additionally all inner patterns were consistently of the same type (either all red or all green). Several examples illustrating this correlation structure are sketched in Fig 4B. As a consequence, an inner network neuron, that is tuned to one of the three local patterns, has to consider both cues and the state of all other inner network neurons in order to infer its own state correctly.

We tested the network’s ability to recover the statistical structure of the input in a simulation with concurrent learning of afferent weights and recurrent weights (initial values: and restricted to positive values). In a first simulation run, recurrent synaptic plasticity followed the theoretically optimal wake-sleep plasticity rule (16). As expected from the theory, all network neurons developed a tuning to one of the local input patterns, and recurrent weights correctly reflected the correlation structure of the task after 25, 000 s of learning: Similarly tuned inner network neurons formed strong excitatory recurrent links ; in addition, inner neurons developed excitatory connections with compatible cue neurons (e.g. with the red and green cue for a blue-tuned inner neuron). The ongoing adaptation of the network is reflected in the log-likelihood function ℒ(θ) shown in Fig 4D. The black dashed line shows the learning progress with the theoretically optimal learning rule. For comparison, we performed an independent simulation with recurrent plasticity switched off (dashed gray, all fixed at zero). Without recurrent excitation, the log-likelihood settled at a significantly lower value. The gap in ℒ between the two simulations corresponds to the structural information stored in the prior p(z ∣ θ).

Building on the theory-based learning result, we constructed a suitable LTD function ϕ_ϑ to obtain a simple plasticity rule. The set of target weights , that emerged during the first simulation run, and the set of coactivations 〈z_kz_j〉, that had led to these weights, are provided in Details to the computer simulations in Methods. Based on the observed functional dependence between and 〈z_kz_j〉, we chose the following LTD function: (18) with parameters ϑ = (W^max = 1.41, γ = 31.6) fitted to the data. The LTD function (18) has two components: The term m_k ⋅ m_j describes the expected coactivation 〈z_kz_j〉 if the neurons were statistically independent. This case is associated with weight when synaptic plasticity follows Eq (17). The second term accounts for positive correlations between z_k and z_j through a stabilizing weight dependence. While the specific tangent-shape is a heuristic, the functional form has some properties that are generally expected for Hebbian-type plasticity: The LTD function is strictly monotonically increasing such that higher coactivations settle at stronger efficacies; and, the maximum efficacy of a synapse is bounded since the LTD contribution goes to infinity as approaches W^max. The scaling parameter γ sets the strength of LTD.

In a second simulation run, all recurrent and afferent weights were reset to zero, and recurrent synaptic plasticity followed the simple local plasticity rule given by Eqs (17) and (18). As seen in Fig 4D, the simple rule is virtually indistinguishable from the theoretically optimal rule in terms of the log-likelihood. The resulting recurrent connectivity structure after 25, 000 s of learning is shown in Fig 4C for the example of an inner network neuron that has specialized on the blue input pattern: The neuron developed strong recurrent weights with all other inner neurons of similar tuning, and moderately strong weights with red- and green-tuned cue neurons. This connectivity matrix mirrors the connectivity structure obtained with theoretically optimal wake-sleep learning (see Details to the computer simulations in Methods for numerical values).

The learned structural knowledge can be exploited by the sampling network during inference. Most distinctly, the benefits of well-adapted recurrent excitatory connections become apparent in response to incomplete, or even ambiguous, stimuli (Fig 4E and 4F). To emulate a scenario of incomplete observations, only outer cues were presented to the network while all inner inputs showed uninformative, uniform activity (indicated as gray). For example, when the cues are set to red and green, as shown in Fig 4E, the network correctly infers that all inner features should be blue most of the time. In the given experimental protocol, this knowledge can only be communicated via recurrent excitatory synapses. In a second example shown in Fig 4F, the already incomplete input is furthermore chosen to be ambiguous by presenting a both-blue cue. Hence, the inner features should be either all red or all green. In this case, the network activity stochastically switches between the two inferred consistent interpretations (note that the switching of network neurons at inner locations is synchronized), as expected for sampling from a bi-modal posterior distribution p (z ∣ y, θ).

In conclusion, the computer experiment in Fig 4 demonstrates that the sampling network model can concurrently identify salient features of its input stream and recover complex correlations among spatially distributed features through synaptic plasticity. Self-organized learning of recurrent excitatory connections can be understood as an ongoing refinement of the prior distribution in the network’s internal model of presented input. The obtained structural knowledge on the input distribution significantly enhances the network’s ability to maintain globally coherent network states in face of incomplete and ambiguous observations. For the examined inference task, which required integration of two independent cues as well as the state of multiple other network neurons, we observed that even the simple local plasticity rule (17) + (18) endowed the network with close-to-perfect learning capabilities. This observation indicates that the architecture of the neural sheet model eases the complexity of recurrent learning in that the local input integration through probabilistic experts can guide learning of recurrent connections. However, as a word of caution, owing to theoretical considerations it is unlikely that such simple plasticity rules will be sufficient to solve arbitrarily sophisticated learning problems. For instance, probabilistic causal relations of the type “A often implies B” but “B does not cause A” lie beyond the expressive power of the employed single-layer associative prior (6). Yet, as shown in [47], it is possible to implement such probabilistic relations into spiking neural networks by using complex connectivity structures and asymmetric recurrent weights. Regarding the ability of the local plasticity rule (17) to approximate the theoretically optimal rule (16), it can be expected that reliable learning is limited to scenarios where only few recurrently connected neurons are typically active simultaneously. For instance, it is unlikely that deep learning architectures with additional hidden units (without synaptic input connections) could be trained with the simple plasticity rule.

The relative spike timing carries essential information.

In the proposed neural sheet model, multivariate posterior distributions p (z ∣ y, θ) are represented by interpreting network spikes as realizations of random variables z_k according to Eq (1): After a spike of the k-th network neuron, the associated RV z_k turns active for a short time τ (10 ms in this study). This gives rise to vector-valued network states z(t) in which multiple RVs z_k will typically be active, simultaneously. As a direct consequence, the relative spike timing of neuronal subgroups carries important information on a millisecond time scale: overlapping on-times encode the probability of coactivation of the associated RVs in the multivariate distribution p (z ∣ y, θ). The computational importance of incorporating coactivations during inference becomes particularly evident in ambiguous situations that support multiple coherent—but mutually exclusive—explanations. The sampling network’s spike response in such an ambiguous scenario was demonstrated in Fig 4 where the network switched stochastically between two coherent perceptual modes in a synchronized manner. The information density conveyed by this structured, bimodal response goes far beyond the quality of conclusions that could be drawn from just observing the average firing of individual neurons. Figuratively, considering only marginal responses would disregard the insight that an obstacle could be circumvented either on the left hand or the right hand side, but would instead suggest to steer a happy medium.

Lateral inhibition facilitates local synaptic learning.

This information-rich neural representation of globally coherent network states was shown to emerge fully autonomously through the interplay of local synaptic and intrinsic plasticity rules. The derivation of the weight-dependent plasticity rule (12) for afferent synapses V_ki furthermore revealed an essential role of the local lateral inhibition network motif during learning. The contribution of lateral inhibition becomes apparent when the derivation of Eq (12) is repeated in the absence of the inhibition motif, resulting in the update rule (15). The latter rule (15) requires information not only on the pre- and post-synaptic spiking activity of a plastic synapse, but also on the specific weight values of other nearby synapses, thereby rendering learning non-local. This increased complexity of parameter learning is a general problem in graphical models with converging arrows, known as explaining away, and thus, the locality of synaptic learning displays a conceptual challenge for a wide range of Bayesian network architectures. In the neural sheet model, lateral inhibition establishes a particularly strong form of explaining away by ensuring that each input y_i is explained by at most one network variable z_k, at a time. The resulting competition among network neurons restores the locality of information required for synaptic learning. We therefore suspect that the competition introduced by local lateral inhibition could assist synaptic learning in a wide range of network models in that it facilitates global statistical model optimization through local synaptic plasticity rules. In particular, in order to maximize the expressive power of the system while preserving local synaptic learning, our theory suggests that self-organized learning of efficient representations benefits from network architectures in which the range of lateral inhibition matches the spatial extent of network neurons that share common afferent input.

Recurrent plasticity integrates structural knowledge.

Regarding plasticity of recurrent connections in the neural sheet, we found optimal learning rules to be fundamentally non-local; a hardly surprising finding for recurrent systems. We therefore explored in computer simulations to what extent even simple local plasticity rules with approximately matching convergence properties could be sufficient to recover complex structural correlations in the presented input. In the small computer experiments of Figs 4 and 5, we observed no significant impairment in the emergent weight configuration when simple local plasticity rules were used (compared with the theoretically optimal non-local rule). This observation indicates that, for tasks of “not too high” complexity, the prior p(z ∣ θ) could indeed be adjusted by only local synaptic plasticity. The specific shape of the employed plasticity rule, however, needs to be tailored to statistical properties of the given task. For biological systems, it is thus conceivable that evolution forged tailored simple plasticity dynamics for certain neuronal populations and brain areas that interact with more elaborate plasticity types.

Robust evasion of suboptimal solutions.

Finally, the learning dynamics of the underlying stochastic online Expectation Maximization algorithm, that performs local gradient ascent in the synaptic weights, deserves a brief discussion. Like every local optimization algorithm, learning in the neural sheet is only guaranteed to converge to a local optimum of the objective function ℱ. This is a general issue in unsupervised learning since the likelihood functions of complex probabilistic models may possess many local optima. However, two properties of the neural sheet model are expected to mitigate this issue: First, the network employs stochasticity in two ways, via the random presentation of input samples y ∼ p*(y) and via the stochastic nature of the network response z ∼ p (z ∣ y, θ). This induces stochastic fluctuations in the synaptic weights and facilitates the evasion of local optima (compared with a fully deterministic batch algorithm as often used in machine learning). Second, homeostatic intrinsic plasticity forces all neurons to participate in explaining the input. Thereby, many “particularly bad” local optima, which recruit only a small fraction of hidden causes z_k in the average posterior, are automatically evaded. In accordance with these properties, we observed in the computer simulations that learning was generally very robust and that the network reliably identified near-optimal parameter settings most of the time.

Disynaptic inhibition establishes local competition among pyramidal cells.

In vitro experiments indicate that central aspects of the ubiquitous lateral inhibition network motif in the neural sheet model are established in layer 2/3 by soma-targeting, fast-spiking (FS) interneurons. FS interneurons preferably form synapses locally [31] and show particularly high connection probability with nearby pyramidal cells (PCs) [50]. These GABAergic connections typically involve many (ca. 15) contacts [34] per cell-pair and inhibit the target close to its cell body [34]. In addition, FS-PC connections were reported to feature very short transmission delays of only approx. 1ms [50]. The dense and fast inhibition of PCs by FS interneurons is complemented with an increased probability for reciprocal connections [51], i.e., bidirectional links between nearby PC-FS pairs. Notably, reciprocal PC-FS connections were reported to be especially strong in either direction (see [51, 52] for PC → FS, and [51] for PC ← FS). Furthermore, also the PC → FS connection was found to have below-average transmission delays [50]. This led the authors of [50] to the conclusion that, “[t]aken together, our in vitro data (this study) and our related in vivo data [52] suggest that disynaptic inhibition driven by FS GABAergic neurons in the neocortex mediates competition among excitatory neurons such that perhaps only a small fraction of excitatory layer 2/3 neurons can be active at any given time.” The finding of lateral competition among PCs in layer 2/3 was also confirmed in vivo by [53].

Pyramidal cells are clearly tuned while local lateral inhibition is unspecific.

In vivo studies with awake animals characterized typical response properties of PCs and interneurons. During quiet wakefulness, PCs show a much lower firing activity than FS cells [54]. This was explained by the stronger synaptic drive required for exciting PCs compared with FS cells, which maintain an average membrane potential close below threshold [54]. Furthermore, detailed patch-clamp recordings, that separated individual conductance contributions, pointed to significantly sharper spatial tuning of excitation than inhibition [55]. This finding is supported by studies with anesthetized mice that reported clear orientation selectivity of excitatory neurons, while GABAergic cells showed only little tuning [56, 57], but see [58]. Indeed, experimental data suggests that the stimulus dependence of interneuron responses could be explained by the integrated activity of surrounding neurons [59].

Sparse coding and structured connectivity of excitatory neurons.

Experimental data on the spike response of PCs in layer 2/3 appears to be compatible with a sparse coding scheme. For instance, it was observed in vivo by [60] that even neurons with overlapping receptive fields showed only low correlation. Also [61] reported generally weak noise correlations between neighboring neurons of similar tuning, suggesting a mechanism of active decorrelation among nearby neurons. A recent review [62], that assessed experimental evidence for sparse coding, emphasized the particularly sparse response of layer 2/3 and pointed to the generally low firing activity in superficial layers compared with neurons from layer 4, which are considered to be a major input to layer 2/3. The functional characterization of PC responses is complemented by studies on physical connections between excitatory neurons. While inhibition was shown to act mostly locally, recurrent excitation spans larger distances [31]. Furthermore, neurons with positively correlated responses were found to also have an increased probability to maintain reciprocal connections [63]. More specifically, in [30] an increased connection probability was reported between neurons with similar orientation tuning and cell body distances larger than 500 μm.

Cortical models propose similar network architectures and function.

Based on the rich experimental data on cortical layer 2/3 both in vivo and in vitro, experimental neurobiologists and computational neuroscientists have proposed functional models of information processing in layer 2/3 that appear compatible with the architecture of the neural sheet model in this article. Douglas and Martin [29] as well as Lansner [35] sketched functional models of layer 2/3 that rely on local competitive circuits which communicate via associative excitatory links. The local competitive aspect of information processing in layer 2/3 was examined in detail by [50] and [52]. Finally, the architecture of the neural sheet model is highly reminiscent of the connectivity structures within a cortical column model of barrel cortex, as proposed in a recent review by Petersen and Crochet [64].

Especially cortical network models, which extend the idea of Bayesian confidence propagation neural networks (BCPNNs) [65], have been linked to probabilistic inference and statistical learning. In these networks, the mean activity of a small neuronal population (“cortical minicolumn”) is interpreted as the probability of a certain realization of a random variable. Different (disjoint) minicolumns form a “cortical hypercolumn” and compete in a winner-take-all manner to represent the possible values of a random variable. Excitatory synapses between hypercolumns realize associative memory function. BCPNN-type networks have been proposed as a model of human working memory [66]. Similar to the neural sheet model examined in this article, BCPNN-type networks are capable of developing abstract representations of input patterns [67] and forming associative memory [68], and support implementations with spiking neurons [69]. Furthermore, BCPNN-type networks have successfully been implemented with highly detailed neuron and synapse models [70]. A key difference between the neural sheet model and BCPNN-type networks is found in the local network architecture: In BCPNN-type networks, minicolumns are disjoint sets, i.e., each neuron participates in exactly one WTA circuit. In contrast, a neural sheet hosts a plethora of interwoven WTA subcircuits by virtue of the continuous nature of lateral inhibition. As a consequence, each network neuron can take part in multiple overlapping WTA subcircuits.

Neural representations of Bayesian computation.

The algorithms that underlie Bayesian computations in neural network models are diverse. However, regarding the representation of the arising posterior distributions p (z ∣ y, θ) two general lines of research can be identified, namely sample-based codes and distributional codes. In sample-based codes (as employed by the neural sheet model) the observed network state is interpreted as an instantiation of one or more random variables z. By observing the sequence z(t) of network states over time, the distribution p (z ∣ y, θ) is represented with increasing precision through the relative frequency of state occurrences [23, 24]. Examples of a direct (one-to-one) mapping between neurons and random variables are found in [38, 17, 25] and [20]. A conceptual separation between network neurons and the represented RVs was explored by [47] where only a subset of so-called principle neurons carry meaningful information for downstream populations. In [16], a spiking network model was proposed that enables a sample-based Bayesian interpretation of perceptual bistability based on the average activity of neuronal populations. The idea to rigorously separate neuronal spike patterns from the represented RVs was recently explored by [21], thereby allowing the simultaneous operation of multiple entangled sampling chains within a single network. In a different direction, recent theoretical work [18] has shown that the notion of “network states” can be extended to also cover entire population trajectories. On a general account, it has been emphasized [24] that sample-based codes offer several conceptual advantages, such as high representational flexibility, easy marginalization, natural emergence of response variability, and general suitability for learning.

Distributional codes provide a complementary (and at first glance, irreconcilable) neural representation of probability distributions. A characteristic property of distributional codes is the (almost) instantaneous representation of either the entire posterior distribution or, at least, pivotal statistical properties thereof (e.g. marginals or mean values). Just as the case for sampling networks, the exact inference algorithms implemented by distributional code network models are manifold. One line of research [11, 12, 14, 15], builds upon the belief propagation algorithm that aims to calculate the marginal posteriors 〈z_k〉_{p (z ∣ y, θ)} for all variables. This approach enables inference in complex graphical models, including temporal integration. However, since correlations among RVs in the posterior are not accommodated, the precise relative spike timing between different neurons carries no relevant information: what matters is the spike count in a given time interval, not the spike timing. A second line of research is established by (probabilistic) population codes (see e.g., [10, 13, 71]). These models aim to infer the posterior of a hidden (’true’) stimulus parameter (e.g., bar orientation) from the observed input activity in a known generative model. The inference process relies on integrating the spike count of many input neurons with known tuning properties, simultaneously. Neuronal trial-to-trial variability arises in this deterministic inference scheme solely from stochasticity in the spiking input.

Despite their different origins, sample-based codes and probabilistic population codes can provide mutually compatible interpretations, at least in some scenarios. Consider, for instance, a local column in the neural sheet model with multiple network neurons within the range of lateral inhibition. This local network is very similar to the competitive networks examined in [25]. Over the course of learning, each neuron will develop a tuning, e.g., a preferred bar orientation (cp. Fig 5 in [25]), such that the local network neurons will jointly cover the entire local input space. Due to the width of the likelihood distribution associated with each hidden cause, the response curves of roughly similarly tuned neurons will partially overlap. Formally, the resulting spike response of the local population to a given stimulus displays a sample-based code of a multinomial (mixture) posterior distribution. However, by knowing the tuning curves of each network neuron, the response of the sampling network could equally well be interpreted as a distributional code of the stimulus parameter (cp. Fig 2 in [71]). Conversely, the continuous hidden stimulus parameter in PPC models could always be mapped to a set of locally competing network neurons in a sampling sheet. Thus, it appears that, at least in some cases, sample-based codes and probabilistic population codes are mutually compatible, and further experimental research will be required to investigate spike response characteristics in more complex scenarios.

Integration into larger Bayesian spiking networks.

In this work, we have employed a rather basic prior distribution p(z ∣ θ) for associative memory formation, namely a single-layer Boltzmann distribution. The theory of the neural sheet model, however, also supports more complex network structures such as deep learning architectures [39] that constitute one of the most powerful tools for unsupervised learning in machine learning theory. Such deep network architectures would add hierarchical information exchange to the system (e.g., top-down or contextual information). Furthermore, arbitrary Bayesian networks, i.e., directed graphical models, could likely be used for the prior distribution as long as the graphical model features the local lateral inhibition network motif (mutually exclusive activity of hidden variables with shared input). Several spiking network implementations for sample-based inference in general graphical models can be found in [47]. Notably, these implementations overcome the constraint of symmetric recurrent connections in the network. An asymmetric recurrent connectivity structure is also a salient property of temporal models. To endow the spatial model with the ability of temporal integration, it would therefore be intriguing to combine the proposed sheet model with the Hidden Markov Model network implementation by [20] or the neural particle filtering approach by [22]. In a different research direction, we observed in computer simulations that relaxing the assumption of strong lateral inhibition still leads to reasonable learning results in many scenarios. This indicates that even soft inhibition could be sufficient to govern the emergence of probabilistic local experts—an important property for network models that feature a high neuron density. However, our current theory does not provide a proper interpretation of the resulting network response since the arising coactivation of neighboring neurons likely demands a generative model that supports soft explaining away. Finally, it would be interesting to combine the fully self-organized network model with reinforcement learning signals from the environment (e.g., via top-down feedback or third-factor plasticity rules). This could endow the spiking network with the ability of Bayesian decision making and action selection.

Definition of variables.

We introduce a generative model (20) over N observed variables y₁, .., y_N, subsumed in the vector y, and K latent variables z₁, .., z_K, subsumed in the vector z. The latent variables are binary, z_k ∈ {0,1}, and said to be “active” iff z_k = 1. The possible values of the observed variables y_i depend on the employed likelihood model. The full model is governed by parameters with real-valued K × N afferent weight matrix V, N-dimensional default vector V₀ (which will be a constant during learning), K × K recurrent weight matrix , and K-dimensional bias vector . Furthermore, each latent variable z_k is connected to a subset of the input variables (its afferent field) which is described by an index set ℐ_k ⊆ {1, 2, .., N}. Likewise, we define the projection field for each input variable y_i as the index set . We refer to all-but-one variables of a vector by the shorthand notation z_\k ≔ (z₁, .., z_k−1, z_{k + 1}, .., z_K).

Generative Model: Likelihood.

Provided the state of the hidden units z, the inputs are defined to be independent (local “Naive Bayes”): (21) For each y_i, the prior will ensure that at most one hidden unit z_k in is active. We assume that, if present, this single active unit z_k governs the distribution of y_i. In case all hidden variables in are 0, a “default hypothesis” is used. The resulting input distribution is assumed to be in the natural exponential family. For instance, Bernoulli, Poisson, or Gaussian distributions are in this class. In natural exponential family form, p (y_i ∣ z, θ) reads: (22) (23) where h_i(y_i) denotes the base measure, and the normalization constants A_ki and A_0i depend on the parameters V_ki and V_0i. In anticipation of (25), the parameters in (23) were written as (V_ki + V_0i), i.e. relative to the default hypothesis. We adopt the convention that V_ki = 0 for to obtain a closed form expression for (22) and (23): (24) (25) By combining (21) and (25) we obtain the full likelihood (26) with A = (A₁, .., A_K)^⊺ and . Here we use the short hands z^⊺ V y = ∑_k∑_i z_k V_kiy_i and z^⊺ A = ∑_k z_kA_k. The function h(y)≔ Π_i h_i(y_i) e^{V_0iy_i−A_0i} comprises only terms that do not depend on z, and hence, it will play no role in the inference. In Results, we employed a Bernoulli likelihood model: (27) or in closed form: (28) By rewriting (28) in the exponential family form (25), we identify (29) In particular, the cluster centers can be recovered via π_ki = σ(V_ki + V_0i).

Likewise, Poisson and Gaussian distributions can be written in the exponential family form (25). This extends the input domain to y_i ∈ ℕ and y_i ∈ ℝ respectively. For a Poisson model with expected values λ_0i and λ_ki (for the default hypothesis and the hidden causes resp.), (30) we obtain (31) For Gaussians with centers μ_0i and μ_ki and fixed variance σ², (32) we obtain (33)

Generative model: Prior.

The prior p(z ∣ θ) guarantees that no two (or more) hidden units z_k, z_j with overlapping input fields, ℐ_k ∩ℐ_j ≠ ∅, are active simultaneously, i.e., it ensures that at most one unit z_k generates each input variable y_i at the same time. For this work, we choose a Boltzmann machine prior which can introduce dependencies between units with non-overlapping receptive fields through symmetric parameters . Furthermore each variable z_k has a bias value which directly affects its prior probability of activity: (34) with δ denoting the Kronecker delta and 0⁰ ≔ 1. The double-product factor ensures, that the assumptions on z made in the likelihood (26) are satisfied, and can be approximated with arbitrary precision by using strong inhibitory weights (35) and setting . Note that the range of strong inhibition could also extend beyond the minimal required range (35). For notational brevity, we subsume the excitatory and inhibitory recurrent weight matrices in a single matrix for the derivation. is symmetric ( and has zero diagonal (). Using this notation, the prior simply reads (36)

E-step: Homeostatic intrinsic plasticity.

During the E-step (43), we seek the distribution that minimizes the Kullback-Leibler divergence to the model posterior p (z ∣ y, θ), and thus, maximizes the lower bound ℱ on the likelihood ℒ. This constrained optimization problem can be solved with Lagrange multipliers. We examine the Lagrange function (46) where the apostrophes indicate that y′ and z′ are summation variables, and β_k are the Lagrange multipliers for the K constraints. The additional Lagrange multiplier λ ensures correct normalization of q. The root of the derivative with respect to q(z ∣ y), with any particular choice of z and y, fulfills and, thus, the optimal solution q* has the form (47) Note that the variational distribution in (47) is already correctly normalized through the free constant exp(λ−1).

However, the optimal multipliers β = (β₁, .., β_K) are still to be determined. Analogous to [36, 37], gradient ascent on the dual function (48) (49) yields an iterative update rule to determine the optimal Lagrange multipliers β_k in q* for the E-step (43). During the E-step, the synaptic weights V_ki remain constant (synaptic weight updates are the M-step). Thus, optimizing β_k is equivalent to optimizing since b_k and β_k differ only by an additive constant. In particular, the update rule (50) remains unchanged and describes a form of homeostatic plasticity. It compares the average activation 〈z_k〉_{p*(y)q(z∣y)} with the target activation m_k and adapts the intrinsic excitability accordingly: When the average activity 〈z_k〉 is too low, the excitability b_k will be increased; if the activity is too high, the excitability will be reduced. Importantly, the homeostatic rule (50) requires only local information and “overwrites” the non-local terms A_k in (47) and in b_k.

M-step in V_ki: Weight-dependent plasticity of afferent weights.

During the M-step (44), we perform gradient ascent on 〈log p (y, z ∣ θ)〉_{p*(y)q(z∣y)} with respect to the parameters θ. This increases the lower bound ℱ on the likelihood ℒ(θ) since the entropy 〈H(q(z∣y))〉_p*(y) does not depend on θ. For the afferent weights V, the derivative of the log-joint models (26) × (36) reads (51) For a Bernoulli likelihood distribution, we obtain from Eq (29) that ∂_Vki A_ki = σ(V_ki + V_0i) = π_ki and hence: (52) The update rule Eq (52) only depends on local information, namely pre-(y_i) and post-(z_k) synaptic activity and the current synaptic weight V_ki. The same holds true for Poisson distributions with ∂_Vki A_ki = exp(V_ki + V_0i) and Gaussian distributions with ∂_Vki A_ki = σ² ⋅ (V_ki + V_0i). Intuitively, since ∂_Vki A_ki = 〈y_i〉_{p (yi ∣ zk = 1, θ)} in any natural exponential family, the plasticity rule (51) compares the true input value y_i ∼ p*(y) with the current expectation of the probabilistic model whenever z_k is active.

M-step in : Wake-sleep plasticity of recurrent weights.

Similarly, we can examine the derivative of ℱ with respect to the recurrent weights and biases in the prior: (53) (54) These update rules compare expected values from the variational posterior with expected values from the model prior, i.e., they are a variant of the wake-sleep algorithm. In the neural sheet model, we will apply recurrent learning only to lateral excitatory connections and keep lateral inhibitory connections fixed since lateral inhibition is the foundation for local synaptic plasticity rules of the afferent connections V_ki.

Plasticity rules for sample-based online learning (Corollary 2).

The above learning scheme revealed local update rules for unsupervised model optimization via Generalized Expectation Maximization. The derived algorithm relies on three ingredients:

1. the variational posterior distribution q(z ∣ y) in Eq (47),
2. the homeostatic update rule (50) to solve the E-step, and
3. the synaptic update rule (52) to solve the M-step.

Ingredient 1: By applying the sufficient condition (5) on the homeostatic posterior q(z ∣ y), we find that the network can sample from q(z ∣ y) when we use the same form of the membrane potential u_k, but with (instead of for sampling from the model posterior p (z ∣ y, θ)). This establishes an equivalent to Corollary 1 for the variational posterior.

Ingredient 2: A sample-based online approximation of Eq (50) is established by (55) with η_b denoting a small learning rate. This homeostatic intrinsic plasticity rule approximates the expected values 〈z_k〉_{p*(y)q(z ∣ y)} in Eq (50) through samples z ∼ q(z ∣ y) in response to the input y ∼ p*(y). The required samples z ∼ q(z ∣ y) ⋅ p*(y) are naturally provided by the network in response to presented input. The homeostatic plasticity rule Eq (55) uses only locally available information. When homeostatic plasticity has converged, i.e., when the average update vanishes for all neurons, the network implements the variational posterior distribution Eq (47) with optimal multipliers β_k, i.e., the network solves the E-step by sampling from from q*(z ∣ y). Due to the non-infinitesimal learning rate η_b, this equilibrium is subject to small stochastic fluctuations.

Ingredient 3: A sample-based online approximation of Eq (52) is established by (56) with a small learning rate η_V. This synaptic plasticity rule approximates the gradient in Eq (52) from samples y ∼ p*(y) and the evoked response z ∼ q(z ∣ y). The required samples z ∼ q(z ∣ y) ⋅ p*(y) are provided by the network if the homeostatic posterior q(z ∣ y) is correctly implemented in the E-step. Hence, it must be ensured that homeostatic intrinsic plasticity acts on significantly faster time scales than synaptic plasticity. This can be achieved by separating the time scales of intrinsic and synaptic plasticity via the learning rates η_b and η_V, such that homeostatic intrinsic plasticity can react quickly to changes in the synaptic weights (see also Interaction of time scales below).

Since Eq (55) approximates the E-step, and Eq (56) approximates the M-step, the joint application of these rules approximates the previously described Generalized Expectation Maximization algorithm. This proves Corollary 2.

In addition, theoretically optimal learning of recurrent connections can be realized by sample-based implementations of Eqs (53) and (54). This gives rise to the plasticity rule Eq (16). The LTD term can be determined by using an independent prior sampler, thereby giving rise to a sample-based sleep phase. In computer simulations with wake-sleep learning, we used such a prior sampler. This sampler maintained independent parameters which were updated according to a sample-based approximation of Eq (54), i.e., according to the difference of samples from the variational posterior and the prior. Since it is not known if (or how) the LTD term can be calculated by a single spiking network, we did not include recurrent plasticity in Corollary 2. Nevertheless, algorithmically the theory supports concurrent learning of input synapses and recurrent synapses.

Stochastic neuron model (common to all figures).

We employed the simplest neuron model from [17] with an absolute refractory period. Neurons are characterized by their membrane potential u_k and a refractory time τ that matches the time constant the associated RV z_k is active after a spike. In discrete time, the active period lasts for time steps. The spiking probability in each time step reads if the neuron is non-refractory. A neuron is non-refractory if z_k = 0 or if it is in its last active time step (to allow an uninterrupted active state, see [17]). Neurons in a network were updated sequentially such that state transitions of one cell are visible in the same time step to subsequently updated neurons. In the computer simulations, the update order was chosen randomly, in every time step. Just as for the continuous time neuron model of the Results section, networks that employ this discrete time neuron model are proven [17] to sample from the correct target distribution in the sense of Corollary 1. For a discrete time implementation of homeostatic intrinsic plasticity, we updated the excitability b in every time step according to δb_k = δt ⋅ η_b ⋅ (m_k − z_k). This is a simple Euler integration of the continuous time plasticity rule.

In the limit δt → 0 and while keeping τ = const., we obtain the continuous time neuron model from the discrete time model. In continuous time, the sequential update policy in the network disappears and all network neurons evolve in parallel. The Markov Chain that underlies continuous time implementations is expected to show even better mixing properties (in terms of biological real-time, not the number of time steps) than the discrete time model used in the simulations: In continuous time, if multiple local WTA neurons compete for explaining the input, an active neuron will (almost surely) switch back to the inactive state at the end of its active phase; the next spike of the local WTA population will then immediately be a correct sample from the local posterior distribution. In contrast, a discrete time neuron has a non-vanishing probability to re-spike in the last time step of its active phase, thereby generating strongly correlated samples. As a consequence, we expect that all results obtained from discrete time computer simulations remain valid without any restrictions in the continuous time limit.

Synaptic transmission and plasticity (common to all figures).

A spike of the i-th neuron elicits a rectangular post-synaptic potential (PSP) at the k-th neuron with duration τ and amplitude V_ki (W_ki) for afferent (recurrent) connections. Synaptic transmission has zero-delay and is non-additive, and thus, PSPs encode the value of the pre-synaptic random variable times the synaptic weight at any time. In discrete time, PSPs last for time steps, accordingly. For a discrete time implementation of plasticity, we updated the weights V, W^exc in every time step according to δV_ki = δt ⋅ η_V ⋅ z_k ⋅ (y_i − σ(V_ki + V_0i)) and . Here ϕ denotes the LTD term of wake-sleep learning and the approximate plasticity rule, respectively. Again, this is a simple Euler integration of the continuous-time plasticity rules.

Spiking input generation (common to all figures).

In all simulation, spiking input was presented to the network in form of Poisson spike trains with time varying firing rate. In accordance with the synaptic transmission in the network via non-additive, rectangular PSPs, an input RV y_i(t) has value 1 if a spike had occurred in the i-th input channel within (t − τ, t], and value 0 otherwise. Firing rates are chosen such that the expected value of y_i matches a target activity x_i, i.e., 〈y_i〉 = x_i. The specific target values x_i used for the simulations are provided below for each figure. In order to achieve 〈y_i〉 = x_i in a discrete time simulation, the spiking probability of an input is set to for each time step. This assignment originates from the following thought. An input is inactive when there was no spike in the last time steps, i.e., . The expected value 〈y_i〉 is thus given by . Solving for p_i yields the above assignment. In the limit δt → 0 such that , these spiking dynamics yield a Poisson process with firing rate log[(1 − x_i)^−1/τ].

Figure 2: Sheets of spiking neurons can perform Bayesian inference on distributed spiking input.

The network consists of N = 18 × 6 input neurons and K = 6 network neurons. Each network neuron receives local input from 6 × 6 inputs, with local connections being shifted by 3 between neighboring network neurons. Neurons with overlapping input inhibit each other, resulting in nearest-neighbor inhibition. In addition, three neuron pairs ((k, j) = (1, 3), (3, 5) and (4,6)) maintain excitatory recurrent connections of weight . Preferred local activity patterns x_kj ∈ (0.2,0.55), 1 ≤ k ≤ 6 and 1 ≤ j ≤ 36, were drawn for each network neuron from a uniform distribution. Background activity was set to π_0i = 0.2, and afferent synaptic weights were set to match the patterns according to Eq (29). Neuronal excitabilities were set to with and A_k = ∑_i A_ki being calculated for each neuron according to Eq (29).

Spiking input was generated with the aim that the input distribution p*(y) closely resembles the model distribution p(y ∣ θ) of the network: At each location and time point, at most one local input pattern x_kj is active. If pattern x_kj is active, it governs the firing rate of all those inputs that are connected to neuron k, i.e., 〈y_i〉 = x_kj for i = [(18 ⋅ (k − 1) + (j−1) − 6) mod N] + 1. The presence of an input pattern is indicated by colored spikes in Fig 2E. If no dedicated pattern is active, inputs fire with the background activity, i.e., 〈y_i〉 = π_0i (gray spikes). The data shown in Fig 2 and evaluated Fig 7A covers 1.5s simulation time. The total simulation time was 2.5s with a short period before and after the shown data being discarded. This serves to provide a burn-in phase for the sampling network, and to prevent boundary artifacts when smoothing the posterior marginals. In panel D, the posterior marginals p_net were estimated from the network response z(t) during 1000 simulation runs, all with exactly the same input (spike-level identity). The correct posterior p_theo is given by Eq (37). For visual clarity, the traces of the posterior marginals have been smoothed with a 20ms box kernel.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Sampling quality and heuristic learning rule.

(A) Sampling quality of the spiking network. Red: Histogram over the difference between the traces in Fig 2D for every neuron and time point. For visual clarity, the data in Fig 2D had been smoothed with a 20ms box kernel. Gray: Histogram over the non-smoothed, raw data. (B) Recurrent plasticity function in the setup of Fig 4. Weights obtained with wake-sleep learning vs. the covariance of network variables (blue dots), alongside the fitted plasticity function W(c_kj)(red line). (C) Same for Fig 5.

https://doi.org/10.1371/journal.pone.0134356.g007

A comparison of non-smoothed data is provided in Fig 7A. The figure shows a more systematic analysis of the sampling quality of the spiking network by means of a histogram over the mismatch (p_net(z_k = 1 ∣ y(t)) − p_theo(z_k = 1 ∣ y(t)). The red dotted counts were evaluated on the smoothed data of Fig 2D; the gray bars depict the non-smoothed (raw) data, evaluated on a millisecond basis. This quantitative analysis confirms the excellent approximation quality of the sampling network. In conclusion, the data shown in Figs 2D and 7A indicate that the sampling network can calculate and represent the general structure as well as quantitative specificities of the time-varying posterior distribution with high accuracy. However, small differences between the traces in Fig 2D are visible. Most notably, rapid and sharp peaks in the posterior, which arise from pronounced but transient jumps in the input, are not fully integrated by the network. The origin of these deviations is of stochastic and systematic nature. Stochastic fluctuations arise from the general sample-based representation. For time-varying input signals y(t), only few independent samples can be drawn from the target posterior distribution p (z ∣ y(t), θ) under (almost) stable conditions, i.e., before the target distribution changes. Any representation based on a limited number of samples can approximate the posterior only with limited precision. Systematic deviations result from incomplete convergence of the Markov chain. The Markov chain, that underlies the network dynamics, is guaranteed to converge to the correct equilibrium distribution p (z ∣ y, θ) only for any constant input y in the limit t → ∞. For time-varying input y(t), convergence of the network will typically “lag behind” the “moving target” p (z ∣ y(t), θ). Theoretical work [18] has shown that the network distribution converges exponentially fast to its equilibrium in almost arbitrary network architectures. Notably, the local WTA architecture is expected to facilitate mixing of the Markov chain since typically only a few competing neurons will attempt to fire in response to the presented input at the same time.

Figure 3: Emergence of probabilistic local experts through synaptic plasticity.

The network consists of N = 21 × 6 input neurons and K = 7 × 3 network neurons. Network neurons are organized in 7 local populations. The 3 neurons within a population share the same 6 × 6 field of afferent connections. Afferent fields of neighboring populations are shifted by 3 (measured in the domain of input neurons). Due to the torus-like topology, every network neuron has overlapping inputs with 8 other network neurons (2 in its population and 2 × 3 in the neighboring populations). This is the range of lateral inhibition.

Spatio-temporal spiking input (that determines the samples y and thus p*(y)) was generated as follows. For each of the 7 afferent field locations, 3 random activity patterns , (1 ≤ p ≤ 3, 1 ≤ l ≤ 7, 1 ≤ j ≤ 36), were drawn. To facilitate the generation of locally different activity patterns, each input location was drawn from a Dirichlet distribution and scaled to the activity range [0.1,0.6]: . Whenever a local activity pattern was presented, Poisson spike trains were generated such that with i = (18 ⋅ l + j mod N) + 1. The presence of local activity patterns was determined as follows. Three chains c = 1,2,3 were started at time t = 0. Each chain can either be active or inactive. If it is active, it appears at a location l and presents one of the local activity patterns . Initially, all chains were inactive and the initial duration of inactivity (in ms) was drawn for each chain from a Gamma distribution Γ(k = 10, θ = 10), leading to an average initial inactivity of 100 ms. Whenever inactivity of a chain ends, it turns active for a duration (in ms) drawn from Γ(k = 10, θ = 20), leading on average to 200 ms duration of activity. When a chain turns active, a random pattern p is drawn uniformly, and a random location l is drawn such that the invoked activity pattern does not overlap with the local activity pattern of a different currently active chain. Such a valid location always exists in the given architecture. After a chain’s active phase, it turns inactive again for a duration (in ms) drawn from Γ(k = 10, θ = 10). Inputs, that are not covered by a currently active chain, maintain a background activity with 〈y_i〉 = π_0i = 0.1. This process leads to spatially non-overlapping, but temporally interleaved input spike patterns as shown in Fig 3F.

The network was exposed to this spiking input for 10000 s. Afferent weights V_ki and intrinsic excitabilities b_k were plastic; recurrent connections and were non-plastic, and .

Details to the plotting: Panel B shows the activity patterns , p = 1,2,3, for the input region highlighted in panel A. For estimating the log-likelihood in panel C, S = 10000 input samples y^s were randomly drawn from the training data as a proxy for p*(y). Furthermore, the network has 337 possible z-states that respect the inhibition structure. Thus, the log-likelihood can be calculated via , with given by Eq (25), for any afferent weight configuration V that emerges over the course of learning. The joint distribution depends on the parameters W^inh, V and . While the synaptic parameters W^inh, V are directly accessible in the spiking network, the biases in the prior must be determined differently. We calculated the biases offline such that the prior exhibited the homeostatic target activity, i.e. 〈z_k〉_{p(z ∣ θ)} = m_k for all k. This is a canonical choice since these are the biases a Bayesian observer would determine from observing the network response. In panel D, optimal weights were calculated from the generating input patterns according to Eq (29). This is possible since the data distribution p*(y) is structurally similar to the model distribution p(y ∣ θ). For each of the seven input locations, each of the three local network neurons was assigned to the best matching pattern of optimal weights. The assignment was unambiguous, since each network neuron had clearly specialized on one of the local input patterns, and determines the one-to-one mapping between learned weights V_ki and optimal weights plotted in panel D. For panels E and G, four additional simulations were run with network parameters (V(t), b(t)) taken from different training time points t = 0 s, 1000 s, 3000 s, 10000 s. Identical 100s-spike patterns were presented to the network in these four simulations. Panel G shows [2.5]s of the input spike pattern alongside the network response for these simulations. Panel F shows the corresponding afferent weights V(t) for the three highlighted network neurons. For the 2-dimensional linear projection in panel E, the input states y(t) of each 100s-simulation were sampled every 10 ms (i.e. 10000 data points per scatter plot) and projected onto the 2d plane. The color of each data point is determined by the network response: red, green, blue if one the neurons marked in panel A responded; and gray otherwise. The projection plane is spanned by the two leading principle components (PCA) of those input samples the three highlighted network neurons responded to at the end of learning, i.e., the PCA is based on the colored samples in the rightmost panel. This biased selection only concerns the choice of the projection plane with the aim to visually discern the clusters of interest; the plotted data points are unbiased.

Figure 4: Plastic recurrent synapses integrate structural knowledge.

The network consists of N = 7 × 6 × 6 input neurons and K = 7 × 3 network neurons. Network neurons are organized in 7 local populations. The 3 neurons within a population share the same 6 × 6 field of afferent connections. The afferent fields of different populations are disjoint. Network neurons within the same population share lateral inhibition. Network neurons, which belong to different populations, maintain excitatory recurrent connections .

For the spiking input, a simpler temporal input structure than in Fig 3 was used since the focus of this simulation was set at the correlation structure of the input beyond the range of individual input fields. There are three local activity patterns , p = 1,2,3, referred to as the red, green and blue pattern, each consisting of “vertical stripes” at shifted locations: if ⌊(j−1)/⌋ mod 3 = (p−1), and 0.1 otherwise, with ⌊⋅⌋ denoting the floor function. To generate a global activity pattern x, two (potentially identical) cue patterns were picked for the outermost locations l = 1 and l = 7. Then the inner locations l = 2—6 were filled with a consistent valid pattern, with “validity” referring to the condition to show a pattern that differs from the cues. Thus, in case of two different cue patterns, the choice of the inner pattern was fully determined (e.g. a red and green cue leads to blue inner patterns); in case of two identical cues, the inner pattern could show either of two valid patterns with all inner locations showing the same pattern (e.g. a double-green cue leads to either all-red or all-blue inner patterns). Hence, both cues (l = 1 and l = 7) must be taken into consideration to determine the validity of inner patterns (2 ≤ l ≤ 6) during inference, and all inner patterns are supposed to be of equal type. As in previous simulations, input Poisson spike trains were generated such that the average value 〈y〉 matched the activity pattern x. Input activity patterns x were presented for a fixed duration before the global activity pattern switched: The patterns iterated over the nine possible cue combinations. In case of differently colored cues the global pattern was presented for 500 ms; in case of identical cues the two valid global patterns were presented for 250 ms each. Thus all cue combinations were presented for an equal amount of time. After all cues were presented, the presentation was repeated.

For the learning experiments, the network was exposed to this spiking input for 25000 s, i.e., each cue combination was presented 25, 000 s/0.5 s/9 ≈ 5, 500 times. In total, three learning experiments were conducted with recurrent plasticity being (a) governed by the theoretically optimal wake-sleep rule, (b) governed by the simple heuristic rule, (c) switched off. In all simulations, afferent weights V_ki and intrinsic excitabilities b_k were plastic. In (a) and (b) recurrent connections were plastic and restricted to positive (excitatory) weight values. For (a) “wake-sleep learning”, the theoretically derived learning rule (53) was used with η_W = 0.05, and prior samples being drawn from an independent sampling network which shared its recurrent weights with the learning network but maintained independent (homeostatically regulated) biases and was not exposed to any input. After learning, the covariances c_kj were calculated from the posterior samples z(t) of the last 1000s of the simulation. To obtain data points () for fitting the function W(c_kj), only excitatory synapses were considered that had a weight at the end of learning. This is to prevent distortions in the fit due to synapses between negatively correlated neurons that would have developed negative weights during wake-sleep learning (but were bounded to in the simulation). Fitting the function (60) to this data yielded W_max = 1.4113 and γ = 31.606 for the free parameters. Data points and fitted function are shown in Fig 7. For (b) “heuristic learning”, the fitted learning rule was used with η_W = 0.005. All recurrent weights converged to stable values. Weights connecting inner neurons (2 ≤ l ≤ 6), that had specialized on equal activity patterns, settled at . Weights between cue neurons and compatible inner neurons settled at . Weights between cue neurons responsive to the same pattern settled at . All other weights settled close to zero ().

For the demonstration of inference in face of incomplete observations (panels E and F), activity patterns x were generated as follows. Two cues were chosen at the outer locations. These cues had increased contrast to ensure that the spike pattern at the cue locations was unambiguous: if ⌊ (j−1)/3⌋ mod 3 = (p−1), and 0.1 otherwise (for l = 1 and l = 7). All inner locations l = 2—6 showed uninformative uniform activity of moderate intensity:x_lj = (0.6 + 0.1)/2 = 0.35 for all j. We refer to the uninformative patterns as “gray” patterns. For each cue combination, Poisson spike trains with 〈y〉 = x were presented to the network for 100 s. The vertical bars in panel E and F show the mean activity 〈z_k〉 of each network neuron during the simulation given an unambiguous cue (panel E) and an ambiguous cue (panel F).

The log-likelihood ℒ(θ(t)) in panel D was estimated as follows.S = 10000 input samples y^s were randomly drawn from the training data as a proxy for p*(y). Furthermore, the network has 4⁷ = 16384 possible z-states (zero or one active neuron in each local population). Thus, the log-likelihood can be calculated via for each time point t and each network type (a)-(c). The joint distribution depends on the parameters W^exc, W^inh, V and . While the synaptic parameters W^exc, W^inh, V are directly accessible in the spiking network, the biases in the prior must be determined differently. We calculated the biases offline for each weight configuration (W^exc(t), W^inh) such that the prior matched the homeostatic target activity, i.e. 〈z_k〉_{p(z ∣ θ)} = m_k. The log-likelihood is shown only for the first 10,000s of the simulation in order to highlight the early stage of learning.

Figure 5: Emergence of excitatory subnetworks in neural sheets.

The network consists of N = 24 × 24 input neurons and K = 12 × 12 network neurons. Network neurons are organized in a sparse grid with twice the distance of inputs. Each network neuron maintains afferent connections with 6 × 6 inputs, such that the input field of neighboring neurons is shifted by two. Lateral inhibition has a range of 2 (in the network grid; maximum norm). Additionally, any pair of network neurons (beyond the range of inhibition) maintains a plastic reciprocal excitatory connection with 25% probability. All existing excitatory connections are plastic with initial values V_kj(t = 0) = 0 and . “Non-existing” recurrent excitatory connections have a weight fixed at . Note that non-existing connections are fully supported by the theory for inference and learning: reducing the number of plastic synapses in the network means to reduce the number of free parameters θ of the generative model p(y, z ∣ θ). The derivative (53) in the direction of existing weights remains unaffected. Thus, the reduction only decreases the expressive power of the generative model.

Spiking input is composed of three prototypic rate patterns , 1 ≤ p ≤ 3 (grid, diagonal stripes, checkerboard), occurring locally at random locations (cp. panel B). Activity patterns are binary with a high rate of and a low rate of . The diameter of local patterns is variable, but exceeds the 6 × 6 input field of individual network neurons. Two randomly selected (and possibly equal) patterns are presented simultaneously at non-overlapping locations. New patterns and locations are drawn every 100ms during training. All inputs not covered by a pattern fire with the background activity 〈y_i〉 = 0.1. Homeostatic target activations m_k = 0.025 are chosen such that the network explains on average approx. 130 inputs; this is roughly half of the average area covered by rate patterns , i.e., ca. 50% of the input is on average being explained by the network.

The network was first trained with wake-sleep learning and variable learning rates until all parameters had converged to stable values. The simple learning rule was fitted to the resulting ()-pairs just as for Fig 4, yielding parameters W_max = 2.7049 and γ = 733.69. See Fig 7 for data and fitted function W(c_kj). The high value of the sensitivity γ, compared to Fig 4, likely originates from the generally much lower covariance c_kj, which in turn arises from the lower average network activity m_k.

For the subsequent spiking network simulation with the simple plasticity rule, a new network with different connected pairs of network neurons was generated. Thus, the extracted plasticity rule is only tailored to the general learning setup, but not to a specific network instance. The total simulation time was T = 10,000s, i.e, the network was presented with 100,000 input examples. Learning rates were set to quite high values (η_b = 10, η_V = 2, η_W = 1) since the focus of this simulation was on the general structure of emerging weight configurations, rather than on numerical precision. After training, recurrent weights covered the entire range of positive weight values with . To verify that the emergent weight configuration was stable even in face of high learning rates, the simulation was continued for another 10,000s, showing no signs of instability.

Details to the plotting: For panels C and D, the trained network was exposed to spiking input with patterns switching every 500ms. This was to obtain more stable estimates for the “expected input” 〈y^gen〉 in panel D. For panel E, all neurons could unambiguously be labeled to be responsive to one of the three local patterns, since their afferent weights V_ki showed an evident preference for either one of them. Panel F uses the euclidean distance in the lattice coordinates of the network neurons, i.e., directly neighboring network neurons have distance one.