Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Oscillation, Conduction Delays, and Learning Cooperate to Establish Neural Competition in Recurrent Networks

Abstract

Specific memory might be stored in a subnetwork consisting of a small population of neurons. To select neurons involved in memory formation, neural competition might be essential. In this paper, we show that excitable neurons are competitive and organize into two assemblies in a recurrent network with spike timing-dependent synaptic plasticity (STDP) and axonal conduction delays. Neural competition is established by the cooperation of spontaneously induced neural oscillation, axonal conduction delays, and STDP. We also suggest that the competition mechanism in this paper is one of the basic functions required to organize memory-storing subnetworks into fine-scale cortical networks.

Introduction

In mice experiments [1, 2], a memory is recalled when neurons that are active during a learning process are activated with optogenetic stimulation. A specific memory is considered to be stored in a subnetwork consisting of a small population of neurons. For such memory formation, competition among neurons might be necessary to embed memory-storing subnetworks into neural circuits [36]. Furthermore, synaptic plasticity is thought to play a critical role in subnetwork organization [7, 8].

In the last decade, studies in the field of neuroscience have revealed that synaptic modification depends on presynaptic and postsynaptic neuronal activities; this is called spike timing-dependent synaptic plasticity (STDP) [912]. Experimental observations suggest that the precise timing of presynaptic and postsynaptic neuronal action potentials is a crucial factor in information processing and/or memory formation in the brain. Therefore, STDP is thought to be one of the mechanisms to encode information into patterns of their synaptic weights [13, 14].

Based on experimental observations, several studies have proposed models of STDP window functions [1518]. For example, Song et al. [15] modeled synaptic learning that is independent of the synaptic weight; this is called an additive model or a hard-bound model. In contrast, Rossum et al. Rossum et al. [16] and Rubin et al. [17] proposed a model that linearly depends on the synaptic weight; this is called a multiplicative model or a soft-bound model. The dependence on synaptic weights in STDP models is generalized [17, 18]. Temporal development and equilibrium states of synaptic distribution have been well investigated through these STDP models and are supported by the Fokker-Plank theorem [1618]. Extended models were also recently proposed [19, 20].

Several studies on memory formation have reported that STDP has the ability to establish specific structures like neural clusters in recurrent neural networks [2128]. This clustering can be regarded as a form of neural competition. In a spontaneously organized network, neurons in a cluster simultaneously emit spikes that force neurons in the next cluster to fire. Therefore, the synchrony of each cluster is successively conducted to the next cluster and is evoked cyclically in the network. The reason for the synchrony distributed in time is that individual neuronal activities are under the theoretical limit of integration or the limit cycle. These conditions are, however, unusual in the cortical and hippocampal neurons in the homeostatic process [29, 30]. In addition, it has been presented that STDP enables recurrent neural networks to organize a feed-forward topology in presence of background noise, that does not lead neurons to the theoretical limit of integration or the limit cycle [23].

In this paper, we aimed to present that STDP leads excitable neurons in a sparse recurrent network to competition that might be essential in memory formation. Detailed analyses of neural mechanisms showed that competition is accomplished by the cooperation of spontaneously induced neural oscillations, axonal conduction delays, and STDP.

Materials and Methods

Neural network model

We employ Izhikevich’s simple neuron model as the basis of our neural network [31, 32]. This model is not only computationally effective as the leaky integrate-and-fire model, but can also realize firing patterns as rich as those in the Hodgkin-Huxley model [33]. Dynamics of the jth (j = 1, 2, …, N) neuron is described by the following two-dimensional ordinary differential equations: (1) (2) where vj is the membrane potential and uj is the recovery variable of the jth neuron. The membrane potential and the recovery variable of the neuron model is reset to cj mV and uj + dj when vj reached 30mV. The variable Ij(t) represents inputs to the jth neuron at time t. The inputs are the summation of external inputs () and synaptic inputs (). For the sake of simplicity, we model the synaptic inputs with the delta function δ(⋅), (3) where wij is the synaptic weight from the ith neuron to the jth neuron, and ti, k is the arrival time of the kth (k = 1, 2, …, nj, i where nj, i represents the number of spikes of the ith presynaptic terminal of the jth neuron) spike at the ith presynaptic terminal. Our neural network consists of N (= 1,000) neurons including both excitatory and inhibitory neurons. The ratio of the excitatory neurons to the inhibitory neurons is 4: 1 [34]. In this paper, we use regular-spiking excitatory neurons and fast-spiking inhibitory neurons. The parameters for the excitatory neurons are set as aj = 0.02, bj = 0.2, cj = − 65, and dj = 8 and those for the inhibitory neurons are set as aj = 0.1, bj = 0.2, cj = − 65, and dj = 2 [31, 32]. These neurons are randomly connected. Because, in many experimental studies, the connection probability has been estimated between 0.1 and 0.3 [3539], we choose 0.1 connection probability. In the neural network, no connections exist between any pairs of the inhibitory neurons. Furthermore, no neurons are self-connected. Excitatory connections have conduction delays of 1 to 10 ms, with a uniform distribution [40]. A time of 1 ms is required to transmit spikes on all inhibitory connections [40]. It has been shown that dendritic delays tend to strengthen self-feedback, whereas axonal delays weaken it [41, 42]. We assume that the conduction delays are only axonal.

Each neuron in the network receives an independent and uncorrelated Poisson spike train with the fixed firing rate of f spk/s during our simulation through a non-plastic excitatory feed-forward connection. The spike train is statistically identical for both excitatory and inhibitory neurons. In the simulations, we test f = 1, 10, and 40 spk/s. The reason for the usage of the Poisson spike train is based on the observation that in vivo neuronal behaviors in cortical areas are highly irregular [29]. The amplitude of each spike in the spike sequence is set to 20 mV. In other words, in Eq (1), which corresponds to a suprathreshold input when a neuron is in the resting state. A spike train for a neuron is statistically equivalent to spike trains for the other neurons.

All excitatory synaptic weights are initially set to 6 mV, except for the simulation in the section of Independence of initial distribution of plastic synapses on neural competition. whereas all inhibitory synaptic weights are set to − 5 mV. Research has shown the significance of synaptic types [12], and therefore STDP is applied only to the excitatory synapses between excitatory neurons in the network.

STDP is a type of Hebbian synaptic plasticity that has attracted considerable attention [9, 11, 12]. In this synaptic plasticity, if a postsynaptic action potential follows a presynaptic action potential within tens of milliseconds, the synaptic weight between them is strengthened; this is long-term potentiation (LTP). On the other hand, a synapse is depressed if a presynaptic action potential follows a postsynaptic action potential, which is long-term depression (LTD). In this paper, we adopt the additive STDP rule proposed by Song et al. [15]. Its window function is expressed in terms of the exponential functions as follows: (4) where Δt = tjti is the relative spike timing between a presynaptic terminal and a postsynaptic neuron. Hard bounds is assumed for plastic synapses. Therefore, the plastic synapses are constrained in the range of [0, wmax], where wmax is set to 10 (except for the simulation in the section of Influence of neural network parameters on neural competition). The variable λ(= 0.1 mV) is the learning rate [40]. The variable α is the degree of asymmetry between LTD and LTP. This parameter is typically set to 1.2, but is varied in the simulation in the section of Influence of neural network parameters on neural competition. We use the same time constant τ (= 20 ms) for both the LTP and the LTD [12]. Synaptic derivatives are changed at individual firing events, and actual synaptic weights are updated once a second. In all numerical simulations, spike interactions in the STDP rule are limited to nearest-neighbor pairs, except for the simulation in the section of Influence of spike interactions in STDP on neural competition. [42]. Potentiation and depression, which are independent of firing events, are also included in the synaptic modifications as in Izhikevich [40].

Strength correlations

To quantify network structures, the degree of individual nodes is usually measured. If connections in networks are directional, we can take into account two types of degrees: indegree and outdegree. The indegree and the outdegree of the jth node can be defined by the total numbers of incoming (afferent) and outgoing (efferent) connections, respectively, and are expressed as (5) (6) where H0(x) is the Heaviside step function in which H0(x) = 1 if x > 0; H0(x) = 0 if x ≤ 0.

A high indegree implies that a neuron is affected by many other neurons, whereas a high outdegree implies that a neuron affects many other neurons through synaptic connections. If indegrees and outdegrees of neurons are biased, the bias is visualized in a joint degree distribution matrix (JDDM). The imbalance of indegrees and/or outdegrees of neurons in a network appears in distances from the main diagonal of the matrix. Degree distributions in real networks often have the scale-invariant or scale-free property [43].

In the case of neural networks, synaptic connections do not have only directions, but also weights. The quantification of such weighted directional networks needs a natural extension of the degrees defined in Eqs (5) and (6) [44]. These are called instrength and outstrength. The instrength and the outstrength of the jth neuron are defined by the sum of the normalized synaptic weights of afferent and efferent connections, respectively: (7) (8)

An instrength indicates how much a neuron is affected by other neurons, whereas an outstrength indicates how much a neuron influences the other neurons. The imbalance between instrengths and outstrengths of neurons in a network is visualized in a joint strength distribution matrix (JSDM).

In the following of this study, we quantify self-organized neural network structures through STDP using the instrength and the outstrength defined by Eqs (7) and (8). Only excitatory connections between excitatory neurons in our network are plastic; therefore, we focus on a subnetwork consisting of the excitatory neurons for the network structure quantification.

To evaluate connectivity among neurons in our neural network, we introduce measures that we call instrength- and outstrength-correlation coefficient. The correlations are an extension of the degree correlation [45]. The degree correlation coefficient is usually computed from the total degree of the jth neuron: . To calculate the degree correlation coefficient, remaining degrees are used. The remaining degree is the number that is one less than the total degree. However, in our calculation, instrengths and outstrengths are directly used. We define the instrength- and the outstrength-correlation coefficient as follows: (9) where M = H0(wij), , , , and (See also the section of Connectivity of winner and loser neurons). The subscripts t and h denote the tail and the head of a connection and the superscript a indicates in or out.

Identification of winner or loser neurons

To identify the composition of a winner and a loser group, winner and loser neurons are defined based on their instrength. This is because, as seen in Fig 1, the instrengths tended to spread more widely than the outstrengths. As such, the threshold of the winner and the loser neurons is able to be easily determined. It should be noted that even if we have used the outstrength as a standard, our results will have been consistent in principle. The method to identify the winner groups and the loser groups is as follows. First, neurons are organized in descending order of their instrengths, and a neuron that satisfies the condition of sin < sout is identified. This neuron is set as the threshold and neurons before it are treated as winner neurons, and the remaining neurons are treated as loser neurons.

thumbnail
Fig 1. Neuronal competition in the neural network induced by STDP.

(A)–(C) Histograms of plastic synaptic weights. (D)–(F) Joint strength distribution matrices (JSDMs) of excitatory neurons. The mean firing rates of the external inputs are (A), (D) 1 spk/s, (B), (E) 10 spk/s, and (C), (F) 40 spk/s. In the JSDMs, the colors represent the frequency of the excitatory neurons. In (E), the schematics of the winner and the loser neurons are illustrated. The winner neurons obtain many strong incoming connections but their many outgoing connections are weak. The loser neurons have properties opposite to those in winner neurons. All the results in this figure are generated from the networks at t = 3,600 s.

https://doi.org/10.1371/journal.pone.0146044.g001

Definition of phase

To characterize activities of neural networks or populations of presynaptic terminals, we also obtain phases from oscillatory firing rates. These firing rates are low-pass filtered to define the phases, and the cut-off frequency of the filter is determined to be 35 Hz based on the result of our spectral analysis in the section of Neural competition is organized in neural oscillation. After filtering, we define the phases [46] as (10) where tk and tk + 1 correspond to any pairs of neighboring negative peaks of the oscillatory firing rate. The subtraction of π is to arrange the positive peak between tk and tk + 1 at ϕ(t) = 0. We obtain the phases from the firing rates of the entire network or the presynaptic terminals.

Estimation of firing rates

The mean firing rate of N neurons is estimated by the following equation: (11) where fj(t) is the firing rate of the jth neuron at time t. The firing rate of each neuron is given by (12) where T ( = 10 ms) is the width of a temporal window, and ρj(t) is the spike train of the jth neuron: (13) where nj is the number of spikes of the jth neuron and tj, k denotes the time of the kth spike of the jth neuron. If the neuron emits a spike ρj(t) = 1 and otherwise ρj(t) = 0. In the case of presynaptic terminals, we only consider the conduction delays. That is, we use the timings of the presynaptic-terminal firings instead of the timings of the somatic firings and then conduct the same estimation.

Kendall’s correlation coefficient

Kendall’s correlation coefficient τK is a non-parametric value to quantify correlation of a paired data set. Let us define a pair of data as xi and yi (i = 1, 2, …, m). We describe their ranks as Xi and Yi and consider pairs of rank data (Xi, Yi). The data pairs are sorted in ascending order of Xi. For each Yi (i = 1, 2, …, m − 1), the number Pi satisfying Yi > Yj (j = i + 1, i + 2, …, m) is counted. Analogously, the number of Yi < Yj is described as Qi. Note that Pi + Qi = mi is always satisfied. Using the values Pi and Qi (i = 1, 2, …, m − 1), Kendall’s correlation coefficient is computed as (14) where − 1 ≤ τK ≤ 1.

Watson’s U2-test

Watson’s U2-test is a non-parametric test for phase data. This test identifies significant differences of the mean value and/or the variance of phase distributions. Here we describe ϕx(i) (i = 1, 2, …m1) and ϕy(j) (j = 1, 2, …, m2) as two samples where both are sorted in ascending order. Therefore, the indices of i and j represent the ranks of phase data in ϕx(i) and ϕy(j), respectively. The total amount of data is M = m1 + m2. Next two variables Xk and Yk (k = 1, 2, …, M) are prepared. For all the data of ϕx(i) and ϕy(j), the following process is repeated:

  1. Set i = j = k = 1.
  2. Compare ϕx(i) with ϕy(j).
  3. If ϕx(i) < ϕy(j), Xk = i/m1 and Yk = 0, otherwise Xk = 0 and Yk = j/m2.
  4. When ϕx(i) < ϕy(j), i is incremented, otherwise j is incremented.
  5. k is incremented.
  6. Repeat 2–5 until k = M.

Afterwards, dk = XkYk is calculated for all k. Using dk, Watson’s U2-value is computed as (15)

In the table of significant values, the significance of the mean value and/or the variance in two-phase distributions is evaluated.

Results

Equilibrium states of distributions of plastic synapses

First, we show synaptic distributions to check the temporal behavior of the synapses (Fig 1A–1C). In agreement with the previous numerical and theoretical studies with the Fokker-Plank theory [15, 17, 18, 47], it is observed in our simulations that plastic synapses in the network bimodally distribute after the long-time simulations (t = 3,600 s). Due to the ability of the STDP to prevent a firing rate in networks from drastically increasing, many plastic synapses go to the lower bound if the firing rate in the networks is high [15, 17, 18]. Also after t = 3,600 s, individual synapses continually change due to firing events. Nevertheless, the influence of these changes are small and trivial, and the form of the synaptic distributions is almost invariant (S1 Fig). Then, we regard the networks after 3,600 s as being enough stable to quantify the network organization using synaptic weights. The stability of the organized network is further discussed in the following section.

STDP induces neural competition: emergence of winner and loser neurons

The joint strength distribution matrix (JSDM) represents the imbalance between the instrengths (sin) and the outstrengths (sout) observed in our neural network (Fig 1D–1F). The JSDM exhibits a two-dimensional Gaussian distribution before the STDP learning (results are not shown) because excitatory synapses between excitatory neurons are randomly connected and the weights of the synapses are homogeneous. After the STDP learning, the outstrengths are widely distributed, while the instrength distribution is narrow when the firing rates of the external inputs are 1 spk/s (Fig 1D).

For external inputs of 10 spk/s, the bias of the instrengths is magnified (Fig 1E). Moreover, the neurons in the neural network compete and two peaks emerge, indicating the existence of two assemblies. The neurons in one assembly have high instrength but low outstrength, while the neurons in the other assembly exhibit an opposite trend. The instrengths of the neurons in these two assemblies display clear differences. The outstrengths are also widely distributed but are narrower than those with 1 spk/s external input (Fig 1D and 1E).

In the case of external inputs with a higher firing rate of 40 spk/s, almost all excitatory neurons have similar instrengths and outstrengths, however, a few neurons achieve high instrength (Fig 1F). In comparison to the outstrengths, the instrengths form a relatively wide distribution in the space of sinsout.

In all the cases, synaptic competition is observed, but the ratio of synapses reaching the upper-bound and the lower-bound depends on the mean firing rate of the external inputs (Fig 1A–1C). Additionally, the degree of neuronal competition changes depending on the mean firing rates (Fig 1D–1F). According to these results, the neuronal competition is related to the ratio of depressed synapses to potentiated synapses.

Stability of neural competition through STDP

To evaluate the stability of the neural competition, we count the number of neurons that move between the winner and the loser assembly from t to t + 1 s (Fig 2). In both cases, i.e. movement from the winner to the loser, or from the loser to the winner assemblies, a maximum of three neurons moved, corresponding to 0.38% of the total excitatory neurons. This change is negligible on a whole network. We, therefore, regard the neural competition as stable.

thumbnail
Fig 2. Stability of the neural competition in the STDP network.

The red line indicates the number of neurons that move from the winner assembly to the loser assembly. The winner and the loser group are identified at each second. The blue line is the same as the red one, but progresses from loser to winner status.

https://doi.org/10.1371/journal.pone.0146044.g002

Connectivity of winner and loser neurons

To analyze how the competitive neurons are connected in the network, we characterize the neural network with instrength- and outstrength-correlation coefficient, rin and rout. These coefficients quantify the similarity of neurons at the ends of connections in networks (See also Fig 3A). When two neurons at the ends of connections in a network tend to have similar instrengths or outstrengths, the coefficients are positive. When instrengths or outstrengths are dissimilar, these coefficients are negative.

thumbnail
Fig 3. Connectivity of winner and loser neurons.

(A) Schematic diagram of assortativity and disassortativity that are evaluated using the instrength- and the outstrength-correlation coefficient. Positive (negative) values of the coefficients represent assortativity (disassortativity) of networks, and zero corresponds to random networks. (B), (C) The time traces of the instrength- and the outstrength-correlation coefficient during STDP with external input firing rates of 1 spk/s (gray line), 10 spk/s (red line), and 40 spk/s (blue line). The coefficients are computed at every second.

https://doi.org/10.1371/journal.pone.0146044.g003

The time courses of rin and rout are shown in Fig 3B and 3C. At t = 0 s, both rin and rout are zero in all cases because the neurons in the neural network are randomly connected through synapses under the initial condition. Evidently, rin and rout only significantly decrease from the initial condition at 10 spk/s. They then converge at approximately − 0.1 and − 0.12 (P < 0.001, t-test), respectively. In the other cases, rin and rout do not reach this level of dissimilarity. Such significant differences of the coefficient values for 10 spk/s case come from the clear competition between the winner and the loser assembly (Fig 1E). We should note that it is impossible for the coefficients to be negative at the significant level if the neurons do not compete and if the neurons in the individual assemblies have the reverse trend of the instrengths and the outstrengths as Fig 1E. Taking into account the results in Fig 1, the neurons are competitive and the dissimilar neurons tend to locate at the ends of individual connections. In other words, connections between the winner and the loser assembly are more strengthened and internal connections in the individual assemblies are easily pruned off.

In addition to the dismilarity, we also analyze the small-world property of the neural network [48]. At the beginning of the simulation (t = 0 s of Fig 2 in [48]), the network has as a large clustering coefficient and a characteristic path length as regular networks. The effect of the STDP leads to a much smaller characteristic path length relatively to the regular networks, although the large clustering coefficient is maintained (t = 600 s of Fig 2 in [48]). This indicates that the small-world network emerges in the connectivity among the winner and the loser neurons. In this study, neural competition is primary interest and the analyses therefore focus on cases of 10 spk/s-external input.

Independence of the numbers of excitatory and inhibitory presynaptic neurons on neural competition

We have shown that neurons in the network are competitive when 10 spk/s-external inputs are given to each neuron. Since presynaptic and postsynaptic activities determine synaptic modifications, the factors determining postsynaptic activity might be the key to neural competition. When considering how to construct our network (See Materials and Methods), one of the differences of individual neurons is the number of presynaptic neurons because we have adopted statistically equivalent external inputs. Therefore, for individual neurons receiving 10 spk/s external inputs, we plot the number of excitatory presynaptic neurons against their instrengths after learning (t = 3,600 s) as shown in Fig 4A. They clearly do not have a linear correlation so their correlation is quantified by the Kendall correlation coefficient τK—a non-parametric index (See details in Materials and Methods). However, even this coefficient cannot identify a correlation (τK = 0.18).

thumbnail
Fig 4. Influence of the initial conditions on the neuronal competition.

Scattergram (A) shows the number of excitatory presynaptic neurons versus the instrength after learning at t = 3,600 s (τK = 0.18), and (B) shows the number of inhibitory presynaptic neurons versus the instrength after learning at t = 3,600 s (τK = − 0.3).

https://doi.org/10.1371/journal.pone.0146044.g004

In the same way, the number of inhibitory presynaptic connections is plotted against the instrengths after STDP (t = 3,600 s) in Fig 4B. Since inhibitory synaptic weights are negative, they have a negative correlation, but the coefficient is small and negative (τK = − 0.3). Therefore, we conclude that the number of excitatory and inhibitory synapses on individual neurons has a minimal effect on whether neurons obtain many or few strong synapses.

Independence of initial distribution of plastic synapses on neural competition

For further analysis of the independence of the initial network architecture, we also change a given distribution of excitatory synaptic weights in the initial condition and observed the JSDMs after STDP (t = 3,600 s) for the 10 spk/s external input samples in S1 File. The variety of the initial synaptic distribution does not affect the JSDM after learning, and STDP induces neuronal competition. The three JSDMs are in perfect agreement with Fig 1B, which indicates that the structures are robust for certain synaptic weights under the initial condition. Therefore, synaptic weights before learning has little influence on neuronal competition.

Relation between axonal conduction delay and synaptic modifications

Another conceivable difference in the statistics of presynaptic connections for individual neurons is the distribution of axonal conduction delays. As such, we analyze the ratio of each conduction delay to the total number of presynaptic connections for each neuron. Fig 5A shows the mean values of this ratio in the winner and the loser assembly. The smaller the conduction delays are, the higher their ratio is in the winner neurons. In contrast, the ratio of larger conduction delays is higher in the loser neurons. From this result, the high ratio of shorter-delay connections seems to be advantageous for neurons to become winners.

thumbnail
Fig 5. Statistics of conduction delays of synaptic connections to the winner and the loser neurons and their relation with the plastic synaptic weights.

(A) The conduction delay distribution in excitatory presynaptic connections of a neuron. The value in each bin is averaged over all the winner neurons or all the loser neurons. (B) The average synaptic weight of each conduction delay. Red and blue bars represent winner and loser neurons, respectively. The data are generated from the network at t = 3,600 s.

https://doi.org/10.1371/journal.pone.0146044.g005

We also analyze the relation between the conduction delays and the mean synaptic weights after learning (t = 3,600 s) of both winner and loser neurons (Fig 5B). For both winner and loser neurons, the smallest delay has the largest mean weight, and the mean weight decreases as the conduction delay increases. This result indicates that smaller conduction delays lead to stronger synapses. However, during the first 5 ms of the conduction delays, the mean synaptic weights of the winner neurons are twice as strong as those of the loser neurons. This explains why winner neurons can get a high instrength but loser neurons cannot.

Neural competition is organized in neural oscillation

We have shown that winner neurons have a higher ratio of presynaptic connections with smaller conduction delays that are capable of more potentiation. However, it is still unclear why the small delay connections of the winner neurons are more strengthened than those of the loser neurons. To unveil origins of strong potentiation of the small delay synaptic connections of the winner neurons, we observe the network activity in the STDP network because it has a strong effect on synaptic modifications.

To show changes of the network activity during the learning process, we pick up the activity for 0–0.5 s, 3–3.5 s, and 5–5.5 s (Fig 6A–6C). The neural network exhibits oscillatory behaviors with frequency variations over time. At the beginning of the learning process, the network exhibits slow oscillation (Fig 6A), which speed up over time (Fig 6B and 6C). Additionally, the mean excitatory synaptic weight gradually decreases due to STDP and normalizes at around 2 (Fig 6D).

thumbnail
Fig 6. Neuronal activity and the mean synaptic weight in the network for 10 spk/s-external input.

Rastergrams and the average firing rates of all excitatory (light blue) and all inhibitory (dark red) neurons at (A) t = 0 to 0.5 s, (B) 3 to 3.5 s, and (C) 5 to 5.5 s. The lower panel of (C) is the enlargement of the selected area 5–5.2 s of the upper panel. Two black arrows represent the peaks of the firing rates of excitatory and inhibitory neurons. The average firing rates are estimated by Eq (11). (D) Time course of the mean synaptic weight of the plastic synapses. (E) Power spectra of the firing rates of excitatory (upper) and inhibitory (lower) neurons. The colors indicate the normalized power intensity.

https://doi.org/10.1371/journal.pone.0146044.g006

To gain further understanding, we conduct a spectrum analysis on these oscillations (Fig 6E). We notice that the time course of the power spectrum of the excitatory population looks very similar to that of the inhibitory population. This is explained as follows: By receiving the external inputs, excitatory and inhibitory neurons begin to fire and increase their firing rates. Triggered by the increase of the firing rate of the excitatory neurons, the inhibitory neurons are strongly activated. Indeed, the local maxima between the oscillations of excitatory and inhibitory neurons exhibit a small gap (Black arrows in Fig 6C bottom panel). Excessive firing of inhibitory neurons inactivates excitatory neurons by large amounts of negative feedback. This inhibition also leads to the silencing of the inhibitory neurons because of diminished excitatory inputs. The excitatory and the inhibitory neurons, however, are excited by the external inputs and begin to fire again. This cycle is repeated in the neural network, and therefore, this cycle results in the stable oscillation. This looks like the phenomenon known as the pyramidal-interneuronal gamma [4952].

For comparison, we also observe the network behavior for the 1 spk/s-external input (Fig 7), where neural competition did not emerge (See also Fig 1D). Analogously to the 10 spk/s case, the network exhibits the slow oscillation at the early stages of learning (Fig 7A and 7B). As learning progress, the oscillation vanishes and the neuronal firing rate diminishes (Fig 7C). As seen in Fig 7E, no apparent peaks exist in the power spectra. Even though the settings are different than in the simulation using 10 spk/s-external input, the mean synaptic weight declines and normalizes at same point as in the case of 10 spk/s-external input (Fig 7D). A case comparison suggest that the neural oscillation supports enhancing synaptic potentiation more on the winner neurons than on the loser neurons.

Mechanisms of neural competition

To understand the competition mechanisms, we observed presynaptic and postsynaptic activities of a winner and a loser neuron randomly selected from individual populations (Fig 8A).

thumbnail
Fig 8. Mechanisms of emergence of winner and loser neurons in the neural network.

(A) The lower panel is the time courses of the membrane potential of (red) a winner and (blue) a loser neuron. The upper panel is the average firing rates of (black) all excitatory neurons, (red) presynaptic terminals of the winner neuron, and (blue) presynaptic terminals of the loser neuron. These average firing rages are estimated by Eq (11). For the black line, timings of somatic firings are used. The red and the blue line are estimated with firing timings of presynaptic terminals, at which delay lengths are added to somatic firing timings of presynaptic neurons of the winner and the loser neuron. The winner and the loser neuron are randomly picked up from the identified groups at t = 3,600 s. (B) Schematic of a phase difference of the presynaptic oscillations from the global oscillation. The black, red and blue lines represent the mean firing rate of all excitatory neurons, the presynaptic terminals of the winner neuron, and the presynaptic terminals of the loser neuron, respectively. (C) Phase distributions of the local maxima in presynaptic terminal firing rate for the winner (red) and the loser (blue) neurons. The local maxima for the presynaptic firing rates are characterized by estimated phases relative to phases of a global firing rate of the excitatory neurons with Eq (10). The data for 45 s (from 5 s to 50 s) is used for the estimation. Results are however not different when using data after t = 50 s. Watson’s U2-test is used to test for significant differences in the distribution pattern between the winner and the loser neurons. This is the non-parametric test for phase data used to indicate any significant differences in the mean value or the variance (P < 0.001). (D) Schematic of the phase difference between the mean firing rate of presynaptic terminals and a postsynaptic firing in a winner neuron (left) and a loser neuron (right). The dashed and the solid lines represent the mean firing rate of presynaptic terminals and the postsynaptic potential, respectively. (E) Same as (C), but the local maxima of the presynaptic firing rates are replaced by the spikes of the winner (red) and the loser (blue) neuron firing rates (P < 0.001).

https://doi.org/10.1371/journal.pone.0146044.g008

The mean firing rates of the presynaptic terminals of the sampled winner and loser neurons oscillate in a similar manner as the firing rates of the excitatory neurons. However, the oscillation of the presynaptic terminals is slightly delayed from the oscillation of the excitatory neurons because of the axonal conduction delays (Fig 8A asterisks).

To quantify the delays, we evaluated gaps in the phases of the presynaptic oscillations from the excitatory oscillation as illustrated in Fig 8B. The mean probabilities of the phase gaps are plotted in Fig 8C. For both types of neurons, the positive gaps are collected, indicating that the presynaptic oscillations are delayed by the excitatory oscillation. The probability of phases gaps [0, π/4] in the winner neurons (red bars) is always higher than in the losers (blue bars). In contrast, the probabilities of the loser neurons exceeded those of the winner neurons in the larger phase range. This result is in good agreement with the previous result (Fig 5A) because the winner neurons have a higher ratio of smaller conduction delays. The distributions of the phase lags of the winners and the losers are significantly different (P < 0.001, Watson’s U2-test). For the details of Watson’s U2-test, see Materials and Methods.

To understand the potentiation and the depression processes in the neural oscillation, it is also necessary to observe the behaviors of the winner and the loser neurons. An example of the time trace of the membrane potentials of the sampled winner and the sampled loser neuron is depicted in Fig 8A (Lower panel). These neurons are the postsynaptic neurons of the synapses shown in Fig 8A (Upper panel). The winner neuron tends to fire just after the local maxima of its presynaptic firing rate (Black arrows for the red line in Fig 8A), while the loser neuron seems to emit spikes before the peaks of its presynaptic firing rate (Black arrows for the blue line in Fig 8A) as depicted in Fig 8D. Here, we characterized spikes of the winner and the loser neurons by using the phases of the presynaptic oscillations. The probability distributions of the phases are shown in Fig 8E. In both distributions, the largest peaks locate around π/2. In the range of [− π, 0], the probabilities of the loser neurons always exceed those of the winner neurons. The positions of the winners and the losers are reversed in [0, π/2]. This fact indicates that synapses on winner neurons are more potentiated and less depressed than those on loser neurons. The mean synaptic potentiation (or depression) between on winner neurons and on loser neurons is, on average, significantly different because there is significant difference in their probability distributions of phases (P < 0.001, Watson’s U2-test). Strong synapses are then, developed more on winner neurons than loser neurons.

Influence of neural network parameters on neural competition

In the previous sections, we have showed that the cooperation of spontaneously induced oscillatory behaviors, axonal conduction delays, and learning interacted with each other and resulted in the neuronal competition. The behaviors of neurons and synapses must however be influenced by certain parameters in the network, which in turn affect neural competition. As such, we investigated the influence of various parameters on the neural competition.

The first two parameters are α determining the balance between LTD and LTP, and the inhibitory synaptic weight because they have strong impacts on the synaptic distributions [15, 16, 47, 4952]. To evaluate organized neural network structures in these two parameter spaces, we used the instrength- and the outstrength-correlation coefficient. As shown in the results in Fig 3, when neural competition occurs, these coefficients become negative.

As expected, these parameters drastically changed the network connectivity (Fig 9).

thumbnail
Fig 9. Influence of the balance between LTD and LTP and inhibition level on the organized networks.

The firing rates of the external inputs are (A) 1 spk/s, (B) 10 spk/s, and (C) 40 spk/s. Colors correspond to the values of the instrength- (upper) and the outstrength- (lower) correlation coefficient. We plotted the instrength- and the outstrength-correlation coefficients of the neural network at t = 3,600 s for all the parameters.

https://doi.org/10.1371/journal.pone.0146044.g009

The network only established neural competition at 10 and 40 spk/s. In addition, the competition is observed for α ≤ 1.2. The parameter is in the suitable range of the experimentally observed STDP window function [12, 53]. The influence of the inhibition level on the competition is not observed in our result.

The next parameter is the upper bound of plastic synapses. Any other parameters are the same as Fig 1B. In spite of the greater or the smaller maximum synaptic weight, neural competition does not emerge (rin = 0.002 and rout = 0.006 in Fig 10A, rin = 0.13 and rout = 0.05 in Fig 10B, rin = 0.3 and rout = 0.2 in Fig 10C). It can be considered that spike effects dramatically affects neural competition [54, 55].

thumbnail
Fig 10. Influence of the upper bound of synaptic weights on the organized networks.

The upper bound of synaptic weights is (A) wmax = 7, (B) wmax = 15, and (C) wmax = 20. The other parameters are the same as those in Fig 1B. The instrength- and outstrength-correlation coefficients are (A) 0.002 and −0.006, (B) 0.13 and 0.05, and (C) 0.3 and 0.2, respectively. All the results are obtained from the networks at t = 3,600 s.

https://doi.org/10.1371/journal.pone.0146044.g010

Influence of spike interactions in STDP on neural competition

In addition to certain parameters, STDP learning similarly influences neural competition in the network. In this section, we analyzed the influence of the spike interactions of STDP on neural competition. S2 File shows the JSDMs, when we simulated the neural network with the all-to-all interactions of STDP. All parameters of the network are the same as Fig 1. The neural competition is quantitatively and qualitatively identical to those in the case of the nearest-neighbor interactions. These results are expected because the all-to-all spike interactions only increase the frequency of the synaptic potentiation and depression and do not affect the competition mechanisms shown in the previous sections. Hence, the spike interactions in STDP are not a key factor for neural competition.

Discussion

In this paper, we showed that excitable neurons in a recurrent network spontaneously competed and organized into two assemblies: winners and losers. Our analyses revealed that spontaneously induced neural oscillation, axonal conduction delays, and learning cooperated to establish neural competition.

In our numerical simulations, on average, STDP decreased synaptic weights in the network. At the same time, a certain level of neural activation spontaneously induced neural oscillation due to the existence of inhibitory neurons in the network. Around the local maxima of the oscillation, instantaneous firing rates of neurons in the network were high. If a postsynaptic neuron received inputs through many small delay connections, the inputs from the other neurons immediately arrived at the neuron. The spikes arrived before the next local oscillation maximum. Then, the presynaptic inputs could effectively induce firings of the postsynaptic neuron. This leads to potentiation of many synapses. Neurons that had high ratio of small axonal conduction delays were able to be winners. In contrast, the spike arrival at presynaptic terminals is delayed in neurons with many large delay connections. Therefore, many synapses failed to be potentiated. In addition, the LTD window is dominant in STDP, which already depressed the synapses in the network. For these reasons, the neurons with many large delay connections became the losers.

Because a specific memory is thought to be stored in a subnetwork consisting of a small population of neural circuits [1, 2], the neural competition shown in this paper is an important property for memory formation [36]. The competition mechanism in this paper is applicable to real neural circuits because all key factors of competition are observable in neural circuits. Because the length of a axonal conduction delay is proportional to the distance between a pair of neurons, our result indicates that synaptic connections of closer neurons were strengthened by neural oscillations. In other words, the neurons organized locally dense but globally sparse neural circuits. This might be related to the distance-dependent high-order correlations of neuronal activities [7, 8]. It is implied that highly nonrandom local connectivity is organized by the distance-dependent high-order correlations [36, 37].

In our analyses, the neural competition is accomplished when certain parameters of numerical simulations were set within physiologically reasonable limits. For example, the firing rate of external inputs, the slight dominance of the depression in the STDP window, and the upper bound of plastic synapses. The parameters tested in this paper might be unique in different areas of the brain [9, 56]. As shown in this paper, it is suggested that local circuits organize very differently in different brain areas.

Iglesias et al. conducted similar simulations and analyses using the leaky-integrate and fire units and the additive STDP, in which spike interaction is the nearest neighbor, without axonal conduction delays [23]. They extended their simulations and analyses by introducing neuron death with larger size of neural networks [57, 58]. In their simulations of Ref. [23], they assumed the application of inputs to a fraction of the population, that shaped a bar column, that dynamically moved, in 2D lattice. In Fig 4 of Ref. [23], Eqs (5) and (6) were used in their analyses. In the result, the neurons exhibited clear competition in the excitatory neural population. The indgrees in two neural groups were much different but their outdegrees were the same level. However, our analyses with the degrees show a reverse trend. The main difference of our simulation settings from Ref. [23] is the way of the application of inputs and the existence of conduction delays. In our result, we do not observe large difference of indegrees between the two groups (S2 Fig). Rather, there is the gap of outdegrees between them. This difference might come from the effect of the way of the external stimulation and the axonal conduction delays, especially the latter one is important for the neural competition shown in the current study because of the competition mechanisms (See also the section of Mechanisms of neural competition). Then, it is considered that the competition in the STDP neural networks in the current study originates in the different mechanisms from ones of Ref. [23]. So, it is suggested that there are some possible mechanisms of the neural competition. Accordingly, the neural competition in STDP recurrent networks should be further analyzed.

Supporting Information

S1 File. Influence of initial synaptic distributions on the organization of networks.

Initial excitatory synaptic weights have a uniform distribution in the range of (Figure A) [5, 7], (Figure B) [4, 8], and (Figure C) [3, 9]. The panels show JSDMs. The instrength- and outstrength-correlation coefficients for each case are (Figure A) − 0.09 and − 0.11, (Figure B) − 0.09 and − 0.11, and (Figure C) − 0.09 and − 0.11, respectively. The external input rate is fixed to 10 spk/s. All the results are obtained from the neural networks at t = 3,600 s.

https://doi.org/10.1371/journal.pone.0146044.s001

(EPS)

S2 File. Influence of spike interactions of STDP on the organized networks.

All spike pairs contribute to synaptic modifications (all-to-all interaction) [1518]. The panels show the JSDMs after the STDP learning (t = 3,600 s) with (Figure A) 1 spk/s, (Figure B) 10 spk/s, and (Figure C) 40 spk/s external inputs. The instrength and outstrength correlation coefficients are (Figure A) − 0.01 and 0.008, (Figure B) − 0.09 and − 0.1, and (Figure C) − 0.03 and − 0.03, respectively.

https://doi.org/10.1371/journal.pone.0146044.s002

(EPS)

S1 Fig. The stability of the synaptic weight distributions for 1 spk/s (gray), 10 spk/s (red) and 40 spk/s (blue) external input rate.

Left panels are enlargements of right panels for 1–50 s. All the parameters of the network and STDP implementation are the same as in Fig 1. The stability is quantified with the Jensen-Shannon divergence between the synaptic distributions at t and t + 1 s.

https://doi.org/10.1371/journal.pone.0146044.s003

(EPS)

S2 Fig. Neurons are not apparently competitive from a viewpoint of their indegrees.

Joint degree distribution matrices (JDDMs) of 10 spk/s. For JDDMs, Eqs (5) and (6) were used instead of Eqs (7) and (8).

https://doi.org/10.1371/journal.pone.0146044.s004

(EPS)

Acknowledgments

This research is partially supported by Research Fellowships of JSPS (No. 22⋅8144) for HK, and Grant-in-Aids for Challenging Exploratory Research (No. 24650116) and Grant-in-Aids for Scientific Research (C) (Generative Research Fields) (No. 15KT0112) for TI.

Author Contributions

Conceived and designed the experiments: HK TI. Performed the experiments: HK. Analyzed the data: HK TI. Contributed reagents/materials/analysis tools: HK. Wrote the paper: HK TI.

References

  1. 1. Silva AJ, Zhou Y, Rogerson T, Shobe J, Balaji J. Molecular And Cellular Approaches to Memory Allocation in Neural Circuits. Science. 2009;326(5951):391–395. pmid:19833959
  2. 2. Liu X, Ramirez S, Pang PT, Puryear CB, Govindarajan A, Deisseroth K, et al. Optogenetic Stimulation of A Hippocampal Engram Activates Fear Memory Recall. Nature. 2012;484:381–385. pmid:22441246
  3. 3. Dawkins R. Selective Neurone Death as A Possible Memory Mechanism. Nature. 1971;229:118–119. pmid:4923095
  4. 4. Edelman GM, Mountcastle VB, editors. The Mindful Brain: Cortical Organization and the Group-Selective Theory of Higher Brain Function. The MIT Press; 1982.
  5. 5. Han JH, Kushner SA, Yiu AP, Cole CJ, Matynia A, Brown RA, et al. Neuronal Competition And Selection during Memory Formation. Science. 2007;316(5823):457–460. pmid:17446403
  6. 6. Yasuda M, Johnson-Venkatesh EM, Zhang H, Parent JM, Sutton MA, Umemori H. Multiple Forms of Activity-Dependent Competition Refine Hippocampal Circuits in Vivo. Neuron. 2011;70(6):1128–1142. pmid:21689599
  7. 7. Schneidman E, II MJB, Segev R, Bialek W. Weak Pairwise Correlations Imply Strongly Correlated Network States in A Neural Population. Nature. 2006;440:1007–1012. pmid:16625187
  8. 8. Ohiorhenuan IE, Mechler F, Purpura KP, Schmid AM, Hu Q, Victor JD. Sparse Coding And High-Order Correlations in Fine-Scale Cortical Networks. Nature. 2010;466:617–621. pmid:20601940
  9. 9. Abbott LF, Nelson SB. Synaptic Plasticity: Taming The Beast. Nature Neuroscience Supplement. 2000;3:1178–1183.
  10. 10. Bell CC, Han VZ, Sugawara Y, Grant K. Synaptic Plasticity in A Cerebellum-Like Structure Depends on Temporal Order. Nature. 1997;387:278–281. pmid:9153391
  11. 11. Markram H, Lübke J, Frotscher M, Sakmann B. Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs. Science. 1997;275:213–215. pmid:8985014
  12. 12. Bi GQ, Poo MM. Synaptic Modifications in Cultured Hippocampal Neurons: Dependece on Spike Timing, Synaptic Strength and Postsynaptic Cell Type. The Journal of Neuroscience. 1998;18(24):10464–10472. pmid:9852584
  13. 13. Gerstner W, Kempter R, Van Hemmen JL, Wagner H. A Neuronal Learning Rule for Sub-Millisecond Temporal Coding. Nature. 1996;383:76–78. pmid:8779718
  14. 14. Masuda N, Aihara K. Self-Organizing Dual Coding Based on Spike-Time-Dependent Plasticity. Neural Computation. 2004;16:627–663. pmid:15006094
  15. 15. Song S, Miller KD, Abbott LF. Competitive Hebbian Learning through Spike-Timing-Dependent Synaptic Plasticity. Nature Neuroscience. 2000;3(9):919–926. pmid:10966623
  16. 16. Van Rossum MCW, Bi GQ, Turrigiano GG. Stable Hebbian Learning from Spike Timing-Dependent Plasticity. The Journal of Neuroscience. 2000;20(23):8812–8821. pmid:11102489
  17. 17. Rubin J, Lee DD, Sompolinsky H. Equilibrium Properties of Temporally Asymmetric Hebbian Plasticity. Physical Review Letters. 2001;86(2):364–367. pmid:11177832
  18. 18. Gütig R, Aharonov R, Rotter S, Sompolinsky H. Learning Input Correlations through Nonlinear Temporally Asymmetric Hebbian Plasticity. The Journal of Neuroscience. 2003;23(9):3697–3714. pmid:12736341
  19. 19. Pfister JP, Gerstner W. Triplets of Spikes on A Model of Spike Timing-Dependent Plasticity. The Journal of Neuroscience. 2006;26(38):9673–9682. pmid:16988038
  20. 20. Clopath C, Büsing L, Vasilaki E, Gerstner W. Connectivity Reflects Coding: A Model of Voltage-Based STDP with Homeostasis. Nature Neuroscience. 2010;13:344–352. pmid:20098420
  21. 21. Levy N, Meilijson DHI, Ruppin E. Distributed Synchrony in A Cell Assembly of Spiking Neurons. Neural Networks. 2001;14:815–824. pmid:11665773
  22. 22. Câteau H, Kitano K, Fukai T. Interplay between A Phase Response Curve And Spike-Timing-Dependent Plasticity Leading to Wireless Clustering. Physical Review E. 2008;77:051909.
  23. 23. Iglesias J, Eriksson J, Pardo B, Tomassini M, Villa AEP. Stimulus-Driven Unsupervised Synaptic Pruning in Large Neural Networks. Lecture Notes in Computer Science. 2005;3704:59–68.
  24. 24. Shin CW, Kim S. Self-Organized Criticality And Scale-Free Properties in Emergent Functional Neural Networks. Physical Review E. 2006;74:045101.
  25. 25. Takahashi YK, Kori H, Masuda N. Self-Organization of Feed-Forward Structure And Entrainment in Excitatory Neural Networks with Spike-Timing-Dependent Plasticity. Physical Review E. 2009;79:051904.
  26. 26. Gilson M, Burkitt A, Van Hemmen JL. STDP in Recurrent Neuronal Networks. Frontiers in Computational Neuroscience. 2010;4(23):1–15.
  27. 27. Kozloski J, Cecchi GA. A Theory of Loop Formation And Elimination by Spike Timing-Dependent Plasticity. Frontiers in Neural Circuits. 2010;4(7):1–11.
  28. 28. Stone DB, Tesche CD. Topological Dynamics in Spike-Timing Dependent Plastic Model Neural Networks. Frontiers in Neural Circuits. 2013;7(70):1–18.
  29. 29. Softky WR, Koch C. The Highly Irregular Firing of Cortical Cells Is Inconsistent with Temporal Integration of Random EPSPs. The Journal of Neuroscience. 1993;13(1):334–350. pmid:8423479
  30. 30. Shinomoto S, Shima K, Tanji J. Differences in Spiking Patterns Among Cortical Neurons. Neural Computation. 2003;15:2823–2842. pmid:14629869
  31. 31. Izhikevich EM. Simple Model of Spiking Neurons. IEEE Transactions on Neural Networks. 2003;14(6):1569–1572. pmid:18244602
  32. 32. Izhikevich EM. Which Model to Use for Cortical Spiking Neurons? IEEE Transactions on Neural Networks. 2004;15(5):1063–1070. pmid:15484883
  33. 33. Hodgkin AL, Huxley AF. A Quantitative Description of Membrane Current And Its Application to Conduction And Excitation in Nerve. Journal of Physiology. 1952;117:500–544. pmid:12991237
  34. 34. Kandel ER, Schwartz JH, Jessell TM, editors. Principles of Neural Science. 4th ed. Elsevier; 2000.
  35. 35. Liley DTJ, Wright JJ. Intracortical connectivity of pyramidal and stellate cells: estimates of synaptic densities and coupling symmetry. Computation in Neural Systems. 1994;5:175–189.
  36. 36. Song S, Sjöström PJ, Reigl M, Nelson S, Chklovskii DB. Highly Nonrandom Features of Synaptic Connectivity in Local Cortical Circuits. Public Library of Science Biology. 2005;3(3):0507–0519.
  37. 37. Lefort S, Tomm C, Sarria JCF, Petersen CCH. The Excitatory Neuronal Network of The C2 Barrel Column in Mouse Primary Somatosensory Cortex. Neuron. 2009;61:301–316. pmid:19186171
  38. 38. Perin R, Berger TK, Markram H. A Synaptic Organizing Principle for Cortical Neuronal Groups. Proceedings of the National Academy of Sciences. 2011;108(13):5419–5424.
  39. 39. McAssey MP, Bijma F, Tarigan B, Van Pelt J, Van Ooyen A, de Gunst M. A Morpho-Density Approach to Estimating Neural Connectivity. Public Library of Science One. 2014;9(1):e86526. pmid:24489738
  40. 40. Izhikevich EM. Polychronization: Computation with Spikes. Neural Computation. 2006;18:245–282. pmid:16378515
  41. 41. Lubenov EV, Siapas AG. Decoupling through Synchrony in Neuronal Circuits with Propagation Delays. Neuron. 2008;58:118–131. pmid:18400168
  42. 42. Morrison A, Diesmann M, Gerstner W. Phenomenological Models of Synaptic Plasticity Based on Spike Timing. Biological Cybernetics. 2008;98:459–478. pmid:18491160
  43. 43. Barabási AL, Albert R. Emergence of Scaling in Random Networks. Science. 1999;286:509–512. pmid:10521342
  44. 44. Costa LF, Rodrigues FA, Travieso G, Boas PRV. Characterization of Complex Networks: A Survey of Measurements. Advances in Physics. 2007;56(1):167–242.
  45. 45. Newman MEJ. Assortative Mixing in Networks. Physical Review Letters. 2002;89:208701. pmid:12443515
  46. 46. Bhattacharya J, Petsche H. Drawing on Mind’s Canvas: Differences in Cortical Integration Patterns Between Artists And Non-Artists. Human Brain Mapping. 2005;26:1–14. pmid:15852480
  47. 47. Câteau H, Fukai T. A Stochastic Method to Predict The Consequence of Arbitrary Forms of Spike-Timing-Dependent Plasticity. Neural Computation. 2003;15:597–620. pmid:12620159
  48. 48. Kato H, Ikeguchi T. Self-Organized Complex Neural Networks through Nonlinear Temporally Asymmetric Hebbian Plasticity. Lecture Notes in Computer Science. 2008;5163:623–631.
  49. 49. Börgers C, Kopell N. Synchronization in Networks of Excitatory and Inhibitory Neurons with Sparse, Random Connectivity. Neural Computation. 2003;15:509–538. pmid:12620157
  50. 50. Börgers C, Epstein S, Kopell NJ. Background Gamma Rhythmicity And Attention in Cortical Local Circuits: A Computational Study. Proceedings of The National Academy of Science of The United States of America. 2005;102(19):7002–7007.
  51. 51. Börgers C, Epstein S, Kopell NJ. Gamma Oscillations Mediate Stimulus Competition And Attentional Selection in A Cortical Network Model. Proceedings of The National Academy of Science of The United States of America. 2008;105(46):18023–18028.
  52. 52. Lee S, Sen K, Kopell N. Cortical Gamma Rhythms Modulate NMDAR-Mediated Spike Timing Dependent Plasticity in A Biophysical Model. Public Library of Science Computational Biology. 2009;5(12):e1000602. pmid:20011119
  53. 53. Bi GQ, Poo MM. Synaptic Modification by Correlated Activity: Hebb’s Postulate Revisited. Annual Review of Neuroscience. 2001;24:139–166. pmid:11283308
  54. 54. Kempter R, Gerstner W, Van Hemmen JL. Hebbian Learning And Spiking Neurons. Physical Review E. 1999;59(4):4498–4514.
  55. 55. Song S, Abbott LF. Cortical Development And Remapping through Spike Timing-Dependent Plasticity. Neuron. 2001;32:339–350. pmid:11684002
  56. 56. Caporale N, Dan Y. Spike Timing-Dependent Plasticity: A Hebbian Learning Rule. Annual Review of Neuroscience. 2008;31:25–46. pmid:18275283
  57. 57. Iglesias J, Chibirova OK, Villa AEP. Nonlinear Dynamics Emerging in Large Scale Neural Networks with Ontogenetic and Epigenetic Processes. Lecture Notes in Computer Science. 2007;4668:579–588.
  58. 58. Iglesias J, Villa AEP. Recurrent Spatiotemporal Firing Patterns in Large Spiking Neural Networks with Ontogenetic and Epigenetic Processes. Journal of physiology, Paris. 2010;104:137–46. pmid:19944154