Advertisement

  • Loading metrics

Open Access

Peer-reviewed

Research Article

Bursting dynamics and network structural changes towards and away from a Pavlovian-conditioned neural network

  • In Hoi Jeong,

    Roles Formal analysis, Investigation, Software, Visualization, Writing – review & editing

    Affiliation Department of Physics, Korea University, Seoul, Korea

    ⨯
  • Kyoung J. Lee

    Roles Conceptualization, Funding acquisition, Investigation, Project administration, Resources, Supervision, Validation, Writing – original draft

    kyoung@korea.ac.kr

    Affiliation Department of Physics, Korea University, Seoul, Korea

    ⨯

Abstract

Understanding the intricate relationship between neural network morphology, spiking dynamics, and function is a fundamental challenge in neuroscience. This study delves into this relationship in the renowned Pavlovian conditioning model proposed by Izhikevich to address the “distal reward problem.” The model consists of spiking neurons connected by synapses whose weights evolve based on dopamine-modulated spike-timing dependent plasticity and undergoes a specific Pavlovian conditioning protocol. Throughout conditioning, the network architecture undergoes a dramatic transformation, shifting from random recurrent connections to a predominant feed-forward structure, positioning a specific subpopulation of neurons targeted during conditioning at the network’s apex. Remarkably, this new structure, termed Pavlovian memory, can persist despite spontaneous population bursting activity driven by stochastic noise and residual recurrent connections. However, significantly elevated basal dopamine levels gradually induce morphological multi-stability, including a state of complete memory loss. These findings vividly illustrate the interplay of spiking dynamics, network morphology, and memory state within a dopamine-modulated Pavlovian conditioning system. We anticipate that employing a similar systematic approach could advance our understanding of more complex functions of spiking neural networks in both machine learning and the brain.

Author summary

Deciphering the intricate connections between neural network morphology, spiking dynamics, and function is a key challenge in neuroscience. This study investigates this relationship in the Pavlovian learning model proposed by Izhikevich to tackle the “distal reward problem”—how a neural network can recognize a stimulus when its corresponding reward is delivered after a significant delay amidst numerous deterrent stimuli. The model consists of spiking neurons connected by synapses, with connectivity strengths evolving based on dopamine levels and spike-timings of pre- and post-synaptic spikes, undergoing a specific Pavlovian conditioning protocol. Our results reveal a dramatic transformation in the network architecture throughout the learning process, transitioning from random connections to a predominant feed-forward structure. Notably, this new structure, representing Pavlovian memory, persists despite spontaneous population bursting activity driven by stochastic noise and residual recurrent connections. However, significantly elevated basal dopamine levels gradually induce morphological multi-stability, including a state of complete memory loss. These findings vividly illustrate the interplay of spiking dynamics, network morphology, and memory state within a dopamine-modulated neural network.

Citation: Jeong IH, Lee KJ (2025) Bursting dynamics and network structural changes towards and away from a Pavlovian-conditioned neural network. PLOS Complex Syst 2(2): e0000035. https://doi.org/10.1371/journal.pcsy.0000035

Editor: Mamiko Koshiba, Yamaguchi University, JAPAN

Received: May 21, 2024; Accepted: December 23, 2024; Published: February 18, 2025

Copyright: © 2025 Jeong, Lee. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: The source code and data are available at http://github.com/JeongInHoi/plos_data.git.

Funding: This work was supported by the National Research Foundation of Korea (NRF) Grant funded to KL by the Korea Government (MEST) (No. 2019R1A2C2005989 and RS-2024-00335928) and Electronics and Telecommunications Research Institute(ETRI) grant funded to KL by the Korean government [ETRI 4011-2022-00020, Next generation SNN model development (ML application purpose)]. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Introduction

The learning process of biological neural networks involves intricate dynamics, where the network’s connection morphology adapts to achieve specific objectives, such as identifying relevant sensory or contextual information among various stimuli, as demonstrated in studies such as those by Chong et al. (2020) [1] and Singer et al. (2021) [2]. This morphological evolution is driven by the network’s ability to generate action potentials (spikes), triggered by external sensory stimuli, system noise, or intrinsic self-sustained dynamics. Neurons communicate through spikes and chemical synapses, with plastic synaptic connection strengths that are subject to change by the spiking times and rate of pre- and post-synaptic neuron. Consequently, the morphological network configuration and spiking dynamics of neural networks are tightly intertwined, playing crucial roles in various functions.

However, dissecting the morphology of large biological neural networks at the level of single cells and synapses and understanding the implications of their spiking dynamics, especially during learning processes, is often a daunting task. The challenges associated with studying real biological neural networks or conducting relevant large-scale brain simulations arise from several factors. These include the vast degree of freedom inherent in neural networks, significant heterogeneity among neurons and synapses, limited knowledge of relevant system parameters, and a lack of suitable tools for analyzing and interpreting complex nonlinear population spiking dynamics. These dynamics encompass self-organized autonomous activity as well as induced activity resulting from sensory inputs.

In this study, we delve into a well-defined model network to fully dissect and quantify the process of a memory formation and its subsequent dissociation from the viewpoint of network morphology and population dynamics. Specifically, we investigate a famous mathematical model neural network proposed by Izhikevich years ago [3]. Briefly, this illuminating model is based on dopamine-modulated spike-timing-dependent plasticity (STDP), often referred to as the “three-factor STDP learning rule.” With this model, Izhikevich demonstrated how “distal rewards” can lead to successful Pavlovian conditioning, wherein the network could be trained with rewarded current stimuli, even when the rewards (“dopamine doses”) occur seconds later following the matching current stimuli. Upon the completion of the Pavlovian conditioning, during the inference stage, a target (a specific subpopulation of neurons) receiving a current stimulus generates a far-enhanced population burst (PB) compared to much weaker PBs of all other non-targeted subpopulations. However, the same study did not analyze the accompanying structural transformation of the network undergoing the conditioning process, prompting our investigation.

A closely related inquiry concerning the structural evolution of the network pertains to the potential influence of internally generated spiking activity within the network, and this important issue was not investigated in the original work by Izhikevich [3]. Various studies have proposed that spontaneous spiking in the brain plays a significant role in information processing, as discussed in references [4, 5], among others. Notably, it has been suggested that uncorrelated random spiking rapidly dissipates the effects of prior inputs, facilitating swift adaptation to new sensory input [4]. Expanding on this notion, we systematically investigate how autonomous spiking activity influences the established Pavlovian memory encoded within the Izhikevich network. To address this question, we introduce a new parameter, D0, representing a low basal level of dopamine, into the Izhikevich model. The presence of D0 implies a constant low-level activation of STDP, allowing spontaneously generated single spikes and bursts to alter the landscape of synaptic weights.

Results

Pavlovian conditioning resulting in a feedforward network

Our model comprises 2,000 neurons, including 1,600 excitatory and 400 inhibitory neurons, interconnected randomly with a sparsity of 10% (see Fig 1 for visualization). Neuronal connections operate via chemical synapses, where excitatory synaptic weights (red arrows) dynamically vary within a fixed range from 0 to 4, while inhibitory synaptic weights (blue arrows) remain fixed at -8. (Further details are provided in the Methods section).

thumbnail
Download:
Fig 1. Schematic illustration of an Izhikevich model network undergoing Pavlovian learning.

The network consists of 1,600 excitatory neurons (filled red dots) and 400 inhibitory neurons (blue circles), interconnected with a sparsity of 0.1. The thickness of each connecting arrow represents the synaptic weight strength. To facilitate the Pavlovian conditioning protocol, 100 subpopulations are formed, each comprising 100 randomly selected neurons from the overall population (only 4 subpopulations, each of which contains only 3 neurons, are shown in the scaled-down schematic diagram). During the conditioning process, a non-rewarded electrical stimulation is applied to a randomly chosen subpopulation at a time interval between 100 to 300 ms. In contrast, a rewarded stimulation is administered to a specific subpopulation (S1) at approximately 5-second intervals. The synaptic weights evolve dynamically in accordance with the three-factor dopamine-modulated STDP model described in the Model Section.

https://doi.org/10.1371/journal.pcsy.0000035.g001

Fig 2 illustrates the temporal progression of Pavlovian conditioning in the model network. For Pavlovian conditioning, 100 subpopulations are formed as illustrated in Fig 1 by randomly selecting 100 neurons from the total pool of 2,000. Sequential delivery of brief current (Iinject = 40 pA) pulses to these subpopulations occurs in random order as marked by arrows in Fig 2A, with time intervals varying randomly around a mean of 200 ms. Each current stimulus elicits a population burst, as depicted in the raster plot of Fig 2A. The raster plots also display many randomly scattered single spikes, representing system noise, which result from random current injections.

thumbnail
Download:
Fig 2. Amplification of rewarded stimulus-evoked bursts during Pavlovian conditioning.

(A) Raster plots showcasing neural spikes and stimulus-triggered bursts at different conditioning stages. The arrows atop each plot denote stimulation timings: the green (black) arrows mark the times of rewarded (non-rewarded) stimulations given to the target (non-target) subpopulation(s) (refer to Fig 1). Beneath each raster plot, the corresponding spike density function (SDF) is depicted. Population SDF was computed by convolving spikes with a Gaussian kernel of σ = 10 ms width. Magenta triangle markers pinpoint reward times. (B) Burst profiles (green lines) evoked by a rewarded stimulus. Overlaying each graph is the average non-rewarded stimulus-evoked burst profile (black solid line), accompanied by gray shades illustrating the corresponding standard deviation across a 4-second time-frame. Vertical dashed lines and green (black) arrows denote the timings of rewarded (non-rewarded) stimuli. It is noteworthy that the population burst SDF corresponding to the green arrows undergoes significant enhancement as the conditioning progresses, while those corresponding to the black arrows remain more or less unchanged. A small level of basal dopamine D0 = 0.0015 μM was assumed.

https://doi.org/10.1371/journal.pcsy.0000035.g002

In the three-factor STDP learning rule, heightened dopamine levels augment the influence of STDP. Dopamine release is exclusively triggered only when a specific subpopulation (e.g., S1 in Fig 1) is stimulated (as denoted by green arrows in Fig 2A), whereas no release occurs for other subpopulations (Si where i ≠ 1, marked by black arrows in Fig 2A). Dopamine rewards are administered with a variable time delay (depicted as magenta triangles), ranging from 1 to 3 seconds, following the stimulation of the target S1 subpopulation (each green arrow). This variability reflects the imprecision inherent in the timing of distal rewards.

In the initial phase of conditioning (as depicted in Fig 2A, top row), the induced bursts exhibit a wide range of shapes, as evidenced by the spike density function (SDF) trace below the raster plot. Notably, the burst shape corresponding to the rewarded stimulus (highlighted by the solid green line in the enlarged image in Fig 2B, top frame) does not significantly differ from the others (represented by the solid black line, the average profile of them, in the top frame of Fig 2B). However, with the progression of conditioning, the burst associated with the rewarded stimulus gains increasing prominence. Eventually, its amplitude becomes several times larger, on average, than that of the unrewarded stimuli (as seen in Fig 2B, third row). Additionally, the population burst profiles linked with the rewarded stimuli become more symmetrical about their peaks. In essence, the designed Pavlovian conditioning paradigm proves successful, as it elicits a superior and distinctive burst profile that aligns with the rewarded stimulus.

We now utilize graph-theoretical methods to examine the structural changes within the network, an aspect previously unexplored, as it undergoes the remarkable dynamic transformation during the Pavlovian learning. Fig 3 unveils a striking evolution in the network’s structural characteristics. The initial synaptic weight distribution, originally a delta function with a peak at 2 (magenta color in Fig 3A), undergoes a rapid transformation over time: it gradually shifts towards a U-shaped distribution and eventually matures into a quite extreme bimodal distribution (solid blue line) with peaks at 0 (indicating no functional connectivity) and 4 (representing saturated, maximal connectivity).

thumbnail
Download:
Fig 3. Evolution to feed-forward network architecture during Pavlovian conditioning.

(A) Synaptic weight distribution (top) and recurrent loop-length statistics (bottom) at various conditioning stages [t = 0 (magenta), t = 24 s (red), 240 s (green), 3,600 s (blue)]. Loop-length ratios are relative to the networks that are randomly shuffled (black dashed line). (B) Scatter plots illustrating the sum of afferent weights (∑Win) versus the sum of efferent weights (∑Wout) of all excitatory neurons. Orange points identify neurons within the subpopulation (S1)receiving rewarded stimuli, while purple points represent the top 100 neurons with the highest ∑Win values at t = 3,600 s. (C) Two-dimensional “spring-force layout” visualization of the network corresponding to the scatter plot in B (t = 3,600 s). Notably, the central cluster of orange points becomes a “source,’’ and the peripheral group of purple points on the outskirts becomes a “sink.’’ The orange and purple insets highlight intra-connectivity among excitatory neurons in the source (occupying the core area) and in the sink (occupying the peripheral area), respectively: The intra-connections are emphasized with thicker lines (black: weight = 4, white: weight = 0)compared to the dense, fuzzy background inter-connections.

https://doi.org/10.1371/journal.pcsy.0000035.g003

To visualize this dynamic network transformation process, we generate scatter plots portraying the relationship between ∑Win and ∑Wout. Here, each value of ∑Win (∑Wout) corresponds to the cumulative sum of synaptic weights from afferent (to efferent) connections attributed to a given excitatory neuron. Each scatter plot encompasses 1,600 points, each denoting an excitatory neuron within the network. Initially, at the outset (Fig 3B, t = 0 s), the points congregate in a compact cluster around the coordinates (320, 320), indicative of each excitatory neuron’s approximately 160 excitatory-excitatory connections. The small dispersiveness of them is caused by the inherent probabilistic nature of physical connections, adhering to a sparsity of 0.1. In short order, this initial grouping expands and metamorphoses into an elongated “rugby ball” configuration (Fig 3B, t = 24 s). Subsequently, this structure transitions into a linear “band” formation, characterized by a notable negative cross-correlation (Pearson correlation coefficient of -0.95) between ∑Win and ∑Wout, evident particularly at t = 3,600 s (Fig 3B).

Intriguingly, during this progression, neurons belonging to the particular subpopulation (S1), recipients of the rewarded stimuli (depicted as orange points in Fig 3B), systematically gravitate toward the top-left corner of the scatter plot. In the same plot, we define a set of purple points to represent neurons with the 100 highest values of ∑Win at t = 3,600 s. Notably, retracing the positions of these purple points backward in time reveals their widely dispersed nature at t = 24 s and t = 240 s, and even within the point cloud of the initial stage (t = 0) of Pavlovian conditioning. It’s interesting to note that at the very beginning (t = 0) these purple points exhibit, on average, slightly elevated ∑Win values compared to those of the whole population. As we will soon confirm, the scatter plot derived from the Pavlovian-conditioned data in Fig 3B (t = 3,600 s) distinctly portrays a robust feed-forward network structure, starting from the densely clustered orange points and ending at the purple points.

Since the established Pavlovian conditioned network has a quite extreme bimodal weight distribution (solid blue in Fig 3A), naturally we can set a threshold weight value of 2, beyond (below) which edge connectivity is assumed to be fully active (inactive) and with this criterion we can assign the in-degree (number of active input edges) and out-degree (number of active output edges) of each neuron. By evaluating in-degrees and out-degrees of all excitatory neurons in the Pavlovian conditioned network, a strong similarity is observed between the scatter plot of (∑Win, ∑Wout) and that of (in-degree, out-degree) (see S1 Fig). In fact, for a given excitatory neuron of the conditioned network, ∑Win (∑Wout) is almost linearly proportional to in-degree (out-degree). Furthermore, the statistical analysis of loop lengths (see Methods for details), as presented in the lower frame of Fig 3A, reveals a swift reduction in the prevalence of recurrent closed loops across all lengths, particularly in the case of larger (5 ∼ 10) loops. This observation signifies a substantial decrease in recurrent connections as the Pavlovian conditioning progresses.

The migration of the orange points towards the top-left corner of Fig 3B (t = 3,600 s) can be readily attributed to their dual role as recipients of rewarded stimuli and initiators of widespread population bursts. This migration is a direct consequence of the increasing emphasis placed on their presynaptic roles via the STDP mechanism during each rewarded stimulus event. The migration pattern of the orange points bears a resemblance to the coordinated movement of crawling amoebae in traveling-wave chemotaxis [6] (refer to S2 Fig for visualizing trajectories) as they gravitate towards the source of “chemo-attractant’’ (which, in our context, is the dopamine-rewarded electrical stimulation). We should note that while the dopamine reward is global, the specificity of the subpopulation receiving the accompanying electrical stimulation imposes the locality. Simultaneously, other points dynamically find their final positions in the scatter plot of Fig 3B (t = 3,600 s), although their positions exhibit considerable fluctuations during the transient period (see S2 Fig).

The initial spikes generated by the orange points (i.e., S1 subpopulation) initiate a cascading effect, leading to a series of spikes that flow downwards along the ∼ -1 slope within the scatter plot, a phenomenon visualized in Fig 4A. This sequence of events gives rise to what we term a superlative burst. While the precise spike timings of individual neurons within the superlative burst display large variability from one burst occurrence to another, our analysis reveals a robust positive correlation between the average spiking time of each neuron with respect to the stimulation onset and its ∑Win (or ∑Wout), as depicted in Fig 4B. Considering that each excitatory neuron can be characterized by its unique ∑Win value, this correlation clearly suggests that every neuron possesses a preferred (average) spiking time within a population burst triggered by a dopamine-rewarded stimulus. This correspondence underscores the intricate relationship between network connectivity and spiking dynamics.

thumbnail
Download:
Fig 4. Spreading spiking activity within a superlative burst rendered shown in the space of ∑Wout vs. ∑Win (A) and average spiking times of 1,600 excitatory neurons vs. their ∑Win in (B).

The figure in (A) showcases the propagation of spiking activity within an evoked burst, which is linked to a rewarded stimulation. The colored points represent neurons firing at specific labeled times. Notably, the spiking activity exhibits a clear propagation pattern towards the lower right-hand corner of the plot. The superlative burst corresponding to this propagation is indicated by a green arrow in Fig 2A (3rd row). The figure shown in (B) presents the average spiking times of all 1,600 excitatory neurons across 92 induced population burst events facilitated by a fully Pavlovian conditioned network. These times are then plotted as a function of corresponding neurons’ ∑Win. The larger points (located in the lower left-hand corner) correspond to neurons that receive direct electrical stimulation. Notably, there exists a small (∼1 ms) time delay between the source (large points) and the remaining neurons (small points).

https://doi.org/10.1371/journal.pcsy.0000035.g004

By deciphering the relative spike timings of all neurons during the recurrent occurrence of the superlative burst, the intricate web of network connectivity can be readily unveiled, largely due to its foundation on the principles of the STDP mechanism. To elucidate this concept, we employ a simplified arrangement of six neurons in a toy system, initially interconnected in an all-to-all manner. The assumed firing sequence of these neurons manifests as a compact population burst, as illustrated in a schematic raster plot of Fig 5A. Through the application of STDP, solely the synaptic connections linked with causal (non-causal) pairing of firing events exhibit functional activation (inactivation) with long-term potentiation (LTP) and long-term depression (LTD) mechanisms, respectively. This selective strengthening or weakening of synapses leads to an emergent feed-forward network structure, showcased in the schematic of Fig 5B. Here, node #1 serves as a (global) source, while node #6 serves as a (global) sink. Consequently, the (in-degree, out-degree) points of the six neurons converge to align along a linear trajectory with a slope of -1 (Fig 5C). Conversely, the negative correlation apparent in the scatter plot, characterized by the slope of -1, vividly reflects the network’s inherent feed-forward structure (Fig 5B).

thumbnail
Download:
Fig 5. Heuristic explanation of the formation and deformation of the feed-forward network.

(A) A toy model raster plot illustrating a neural burst in a 6-neuron system, where each neuron fires once and has its own fixed spiking time [#1 (black) first and #6 (violet) last]. The neurons are assumed to be physically connected all-to-all. (B) Schematic illustration of a functionally feed-forward network of the toy model, created by assuming the classical STDP mechanism based on causality assumption. Each colored line with an arrow-head represents a functionally active efferent connection. The two numbers in each bracket represent in-degree and out-degree, respectively. (C) Plot depicting the (in-degree, out-degree) pairs of the feed-forward network shown in (B). (D) Schematic diagram of the feed-forward network from (B), with the addition of two extra recurrent connections (thick red and orange arrows). (E) Scatter plot displaying the (in-degree, out-degree) pairs of the modified feed-forward network depicted in (D).

https://doi.org/10.1371/journal.pcsy.0000035.g005

Therefore, the striking negative cross-correlation evident in the scatter plot of Fig 3B (at t = 3,600 s) (and those given in S1 Fig), coupled with the network’s prowess in sustaining a system-wide population burst, strongly indicates the establishment of a feed-forward network through the process of Pavlovian conditioning. This observation harmonizes with the substantial decay observed in recurrent connections across various loop lengths as the Pavlovian conditioning evolves (Fig 3A, lower frame).

Lastly, Fig 3C employs a two-dimensional “spring-force layout” to visually portray the Pavlovian-conditioned network depicted in Fig 3B at t = 3,600 s. In this layout, points, akin to identical charges, exert repulsive Coulomb-like forces on one another while being connected by linear springs (with spring constants represented by Wij). As a result, the two-dimensional network diagram can attain a state of force equilibrium. Examining this conditioned network, we note that the connectivity percentages for orange-to-orange, purple-to-purple, orange-to-purple, and purple-to-orange points stand at approximately 50, 82, 90, and 0% (functionally connected only if weight >2), respectively. That is, inter-connectivity within the target S1 subpopulation (orange) and that within the purple subpopulation occupying the sink region are quite significant (refer to the two accompanying insets of Fig 3C). Given the nearly identical number (in-degree + out-degree ≈160) of springs attached to all nodes, it is conceivable that these distinct connectivity levels dictate the arrangement of orange (and purple) points within the central (and peripheral) region of Fig 3C, respectively.

Memory retention and erosion

The observed structural transformation of the network during Pavlovian conditioning is indeed remarkable and prompts another important question regarding the sustainability of the established feed-forward network morphology. Specifically, it raises the question of how long the conditioned state will persist in the absence of further conditioning. In the brain, self-sustained spiking activities arise from intrinsic dynamics as well as with parasitic system noise at various temporal and spatial scales [7]. Moreover, the presence of basal dopamine (D0) can continue to activate the STDP action, potentially leading to the disintegration of the established network into different configurations. To address this issue, we investigate the impact of varying levels of D0 concentration on the network dynamics and structure.

We begin by examining a case of D0 = 0.001 μM, which corresponds to only 0.2% of the instantaneous dopamine reward injection level, Dr = 0.5 μM. In Fig 6, we illustrate the temporal evolution of the feed-forward network established through Pavlovian conditioning (the shaded first row). The subsequent rows of Fig 6 illustrate the self-evolution of the network after the cessation of the Pavlovian conditioning. As anticipated, the system autonomously generates numerous random single spikes and small-scale population bursts (see second row, Fig 6), which are gradually increasing in strength and intermittently emerging in varying packet sizes. Notably, there are several rounds of bursts exhibiting a fairly regular inter-burst interval of approximately 40 ms (see the burst SDF profiles in the 3rd row of Fig 6). While the autonomous bursting dynamics matures, the underlying network morphology undergoes some changes, as depicted in the sequence of scatter plots in Fig 6B. The previously almost perfect negative cross-correlation between ∑Win and ∑Wout is altered, albeit only to a modest extent (Pearson cross-correlation: from -0.95 to -0.87). Furthermore, upon examining the scatter plots, we notice that the distinct territories occupied by three different subpopulations of excitatory neurons (represented by differently colored points) as indicated in Fig 6B remain predominantly unchanged: These subpopulations were categorized based on their inter-spike-interval histograms (see S3 Fig and Ref. [8] for additional details on the relevant hierarchical clustering method). In essence, it appears that the Pavlovian memory embedded within the network morphology is largely unaffected by the autonomous dynamics of the system.

thumbnail
Download:
Fig 6. Persistence of memory following Pavlovian conditioning: D0 = 0.001 μM case.

(A) Raster plots and corresponding SDF profiles depicting the final stage of conditioning (highlighted in the shaded first row) and the subsequent periods after conditioning completion (second to fourth rows). After the conditioning stage, population bursts are self-generated. Note that the SDF profile (blue line) graph shown in the first row has a scale different from those (black lines) of other rows. (B) Scatter plots showcasing the relationship between ∑Win and ∑Wout for all 1,600 excitatory neurons. They are evaluated at the ending time of the corresponding raster plots on their left. The three distinct point colors represent different clusters of excitatory neurons, classified based on their dynamic characteristics (refer to S4 Fig for detailed classification information). Notably, even long after the conclusion of Pavlovian conditioning, the territories occupied by the three neuronal clusters remain quite well-maintained. The self-organized spontaneous burst used for the analysis in Fig 6 is marked with a star symbol in the fourth row. [See S1 Video for the whole process].

https://doi.org/10.1371/journal.pcsy.0000035.g006

The gradual ‘bulging’ (as indicated by a red circle) observed in the band structure of Fig 6B, deviating from the -1 slope in the middle region, can be attributed to the formation of local feed-backward loops. An intuitive explanation supporting this argument is provided in Fig 5D and 5E. We assume that many pairs of noise-driven, (temporarily) adjacent single spikes, occurring at irregular intervals, counteract the natural downward flow of spiking activity in the established feed-forward network. This counter-action promotes the emergence of recurrent, feed-backward connections, as illustrated by the two backward (thicker) arrows in Fig 5D, consequently affecting the values of (in-degree, out-degree) of the relevant neurons and leading to the bulged line of Fig 5E.

Importantly, the spiking activity associated with a spontaneously generated population burst tends to propagate in the forward direction of the already established feed-forward network, thereby reinforcing its existing structure. Fig 7 demonstrates the spread of spiking activity associated with a representative self-generated burst, indicated by a star symbol in the third row of Fig 6A (third row): While the spikes at each time instance of Fig 7 are quite distributed across the population, it is evident that the center of the spread, marked by a diamond symbol, tends to gradually move towards the lower right-hand corner of the scatter plot. (See S1 Video for the whole process).

thumbnail
Download:
Fig 7. Spreading spiking activity associated with a spontaneously generated burst in a previously Pavlovian-conditioned network.

This figure highlights the propagation of spiking activity within a burst that occurs spontaneously in a Pavlovian-conditioned network. The colored points represent neurons firing at specific labeled times. While the firing neurons at each time instance are dispersed across the scatter plot, the centroids of their distributions exhibit a clear tendency to move towards the lower right-hand corner. This observation suggests a directional propagation of spiking activity within the burst. The burst corresponding to this propagation, which is selected randomly, is indicated by a star symbol in Fig 6A (third row). The time labels are given with respect to the temporal position (time = 0) of the maximum SDF of the star-marked burst.

https://doi.org/10.1371/journal.pcsy.0000035.g007

Even with a much high basal dopamine level (e.g., D0 = 0.0026 μM), the Pavlovian conditioning process remains effective, as evidenced by the generation of highly similar enhanced superior bursts following the rewarded stimuli (see Fig 8A, 1st row). Furthermore, the resulting network morphology still maintains a strong feed-forward nature (refer to Fig 8B, 1st frame). This outcome is expected since the dopamine reward is set at a much higher value of 0.5. However, the long-term stability of the Pavlovian conditioned feed-forward network proves to be quite sensitive to the level of D0. Fig 8 (2nd to 4th rows) vividly illustrates how a conditioned feed-forward network disintegrates into an almost random, recurrent network (also, refer to S4 Fig). In other words, when D0 = 0.0026 μM, the Pavlovian memory (i.e., feed-forwardness) gets completely erased, while it persists when D0 = 0.001 μM.

thumbnail
Download:
Fig 8. Loss of memory following Pavlovian conditioning: D0 = 0.0026 μM case.

(A) Raster plots and corresponding SDF profiles: during the final stage of conditioning (highlighted in the shaded first row) and the subsequent periods after conditioning completion (second to fourth rows). The green shaded area in the fourth row of (A) is magnified in S5 Fig to reveal the multi-hump nature (i.e., superburst) of the self-organized bursts. Note that the SDF profile (blue line) graph shown in the first row has a scale different from those (black lines) of other rows. (B) Scatter plots depicting the relationship between ∑Win and ∑Wout for all 1,600 excitatory neurons. They are evaluated at the ending time of the corresponding raster plots on their left. The three distinct point colors indicate different clusters of excitatory neurons, categorized based on their dynamic characteristics (see S3 Fig). Note that the Pavlovian conditioned, feed-forward network quickly disintegrates into a random network. [See S2 Video for the whole process being discussed].

https://doi.org/10.1371/journal.pcsy.0000035.g008

The transition to a new network morphology unfolds swiftly within a brief temporal window of approximately 1,000 seconds, contrasting with the gradual development of recurrent connections as demonstrated in the case of D0 = 0.001 μM. A visual inspection of the initial and subsequent frames in Fig 8B reveals that this transformation is primarily instigated by the orange and cyan nodes, originally serving as a ‘sink’ within the conditioned feed-forward network. This observation aligns with the expectation, as nodes within the sink region possess substantially higher in-degree metrics compared to those in the source region. Consequently, their connectivity proves more susceptible to change. Subsequently, nodes within the sink region migrate towards a region characterized by reduced ∑Win values (≈ 500) and increased ∑Wout values (≈ 500), prompting them to extend invitations to all other nodes within the source and body regions of the original feed-forward network. A comprehensive visual representation of this process is available in S2 Video.

It is intriguing to note that this morphological change coincides with a transformation in the self-organized neural bursts. In contrast to the stimulus-driven sharp, single-hump bursts observed in the SDF profiles shown in the first row of Fig 8, the self-generated bursts in the fourth row often exhibit multiple (2∼3) humps in the SDF and a much wider width, forming what is known as “superburst” [Some of these superbursts (green shade area of the fourth row of Fig 8) are blown-up in S5 Fig]. The existence of these superbursts may imply the presence of some self-organized, non-trivial features within the recurrent network, as explored in our earlier analysis on a related matter [8]. Additionally, it is worth noting that the SDF sizes of spontaneously generated bursts at the early stage of disintegration, as shown in the second row of Fig 8, are much smaller than those of fully blown system-wide bursts depicted in the fourth row of Fig 8.

The self-organized recurrent networks, which have evolved freely from a Pavlovian-conditioned, feed-forward network, can be multistable. Fig 9 showcases three distinct self-organized bursting states (Fig 9A) and their associated statistics (Fig 9B), based on 50 independent simulations (using different random initial conditions) for each different value of D0. Notably, within certain ranges of D0, the self-organized networks can manifest in two or even three different forms, which we label T1, T2, and T3. It is obvious that without the prior initialization of Pavlovian conditioning, the network state T1 depicted in Fig 9A [with S1 target subpopulation (orange points) forming a cohesive source] would not have been achievable. Naturally, the higher the value of D0 is the more significant the role of STDP (alongside random spikes) becomes and the percentage of T1 state decreases mainly at the expense of growing percentage of T3, which represents a completely memory-erased bursting state. The T2 also represents a completely memory-erased, newly organized state: see the orange points are widely scattered in the second frame in Fig 9A. Yet, the T2 state exhibits a strong, negative value (-1.76) of Pearson cross-correlation.

thumbnail
Download:
Fig 9. Multistability of the network structure long after the cessation of Pavlovian conditioning.

(A) Scatter plots of (∑Win, ∑Wout) for three different network types. These representative cases are selected from the dopamine level of D0 = 0.0011 μM (T1, blue), D0 = 0.0040 μM (T2, green), and D0 = 0.0048 μM (T3, red). (B) Graph illustrating the fractions of three different morphological states, which are illustrated in (A), for various values of D0. Different network types are realized stochastically without any controls from different initializations.

https://doi.org/10.1371/journal.pcsy.0000035.g009

Discussion

Within the context of the Pavlovian conditioning paradigm under examination, the notable augmentation in the SDF amplitude of the population burst, particularly in response to the rewarded stimulus directed at a specific subpopulation, serves as a prominent indicator of successful learning. Concurrently, the establishment of a feed-forward network architecture, marked by the relocation of the targeted subpopulation to the apex (source) region of the feed-forward network, represents another pivotal facet of effective conditioning. Although the conditioning task at hand may seem elementary, the Pavlovian conditioning process emerges as an excellent model for illustrating the bidirectional relationship between spiking dynamics and network morphology.

Expanding upon this notion, given the inherently complex nature of sensory input signals for real bio-organisms, an intriguing direction for future research may involve integrating Pavlovian sensory cues that encompass both temporal and spatial information [9]. For that matter, we would like to highlight our earlier research, wherein we utilized Δt-paired electrical pulse stimulation to effectively manipulate the network morphology of neuron populations cultured on multi-electrode array plates [10]. This stimulation method entails delivering bipolar electric pulse stimulation to two distinct groups of neurons (electrodes) with a slight time offset of Δt (0 ∼ 100 ms). The unique aspect of Δt-paired stimulation lies in its alignment with the time scales associated with the STDP mechanism, rendering it a compelling form of stimulation that strongly interacts with STDP [11].

Building on this foundation, a novel dopamine-rewarded stimulus cue could be devised as a stimulus pair delivered to two distinct subpopulations (e.g., S1 and S2), characterized by a controlled temporal delay Δt between them. Unrewarded stimuli (deterrents) could then involve identical (or distinct) pairs of subpopulations receiving electrical stimuli, introduced with equivalent or differing time intervals relative to that of the rewarded stimulus. The question arises: can the conditioned network accurately recall a specific Δt value associated with the rewarded stimulus pair administered to two different subpopulations? Furthermore, this concept can be broadened to encompass more intricate spatiotemporal sensory cues, involving multiple subpopulations and numerous temporal offsets. The memory formation within such a network may manifest as a complex set of assemblies characterized by strongly enhanced recurrent intra-connections (see, for example, Ref. [12]). This extension could also naturally intertwine with the overarching question of “how the brain encodes temporal sensory information and forms memory?” [13–16].

During the inference stage of the current Pavlovian conditioning protocol, distinguishing the stimulation given to the target subpopulation from those given to non-target subpopulations relied primarily on the SDF amplitudes of accompanying population bursts [as illustrated in Fig 2A (third row, SDF plot)]. However, in tasks involving more complex (spatiotemporal) sensory information, there could be benefits from more intricate characterizations of SDFs. This may entail a deeper exploration of the specifics of SDF shapes of accompanying superbursts.

In the original Izhikevich model for Pavlovian learning, a dopamine reward is a step-increase of Dr = 0.5 μM decaying exponentially in time, mimicking a burst of activity of dopaminergic neurons. Accordingly, when all the dopamine rewards have decayed, no STDP activation will be allowed to change the existing synaptic weights. In this study, we have introduced a low basal level of dopamine D0 into the model and systematically varied its value to see subsequent effects on the acquired memory. Influenced by noise-driven spikes and spontaneously generated bursts, which engage the same STDP mechanism in the presence of basal dopamine, the Pavlovian conditioned feed-forward network can change or disintegrate. Interestingly, this study finds that within the realm of low basal dopamine levels (approximately up to D0 = 0.0015 μM, the acquired memory persists in more than 90% of cases, yet in a form of modified network featuring both recurrent and feed-forward connections. It’s noteworthy that a recent study by Singer et al. [2] discusses how such a morphological coexistence can facilitate the concurrent accommodation of rate and temporal neural codings.

As the Pavlovian memory dissipates in the presence of high basal dopamine levels (beyond D0 = 0.0015 μM, the free-running network often manifests a captivating sequence of superbursts, characterized by distinct profiles that randomly alternate over time (as depicted in Fig 8, fourth row). In a previous study involving a comparable Izhikevich model system (albeit lacking the dopamine-reward framework), we unveiled that the emergence of superbursts stems from a unique interplay of activations across (several) self-organized neuronal subpopulations, each distinguished by distinct local graph-theoretic measures and different spiking activity patterns [8]. Consequently, the network’s asymptotic state captured by the scatter plot in Fig 8B (fourth frame) is likely to harbor intricate graph-theoretical properties. At this juncture, however, we remain unable to definitively identify the underlying features that drive the occurrence of superbursts. Considering its loop length statistics, which exhibit only a slight deviation from that of a wholly random network [refer to S4A(bottom) Fig], we conjecture that the essential components could potentially reside in localized, small-scale feed-forward connections, which remain concealed amidst the broader expanse of randomly recurrent connections.

It is widely acknowledged that dopamine plays a pivotal role in regulating memory functions and modulating sensations of pleasure, motivation, and movement. Consequently, deviations in dopamine levels, whether low or high, have been associated with a spectrum of neurological disorders [17–19]. However, the mechanisms through which abnormal dopamine levels contribute to these medical conditions remain poorly understood due to the intricate nature of the underlying brain circuitry and signaling pathways. For instance, evidence suggests that the encoding and consolidation of memories necessitate dopamine receptor stimulation within a hippocampal–striatal–prefrontal loop, orchestrating the formation of new memories. Nonetheless, our current network model does not aim to replicate biologically realistic brain circuits. Therefore, at this stage, we refrain from offering physiologically plausible interpretations of our findings regarding the impact of varying basal dopamine levels, aside from its close association with memory retention and erosion. Consequently, as an extension of our current investigation, exploring the dynamical evolution of a modularized, heterogeneous complex network that emulates biologically plausible network structures holds promise for further elucidating these phenomena.

Finally, it is important to acknowledge that there are other types of STDP mechanisms that are different from the additive STDP rule that we used in this work. While the classical additive STDP mechanism [20–24] remains the most commonly employed in spiking neural networks for both machine learning applications and neurodynamics in general, other models such as multiplicative STDP [25] and log-STDP [26, 27] are deemed more biologically plausible. Indeed, extensive literature exists on various synaptic plasticity mechanisms [12, 28–32]. Consequently, a pertinent question arises as to whether the findings presented in this study would hold under these different STDP models. Addressing this question will be a focus of our future investigations.

Conclusion

In this study, we conducted a comprehensive analysis of the evolution of network morphology and population burst dynamics across various phases of a renowned neural network model for Pavlovian conditioning. A key finding is the profound metamorphosis triggered by the conditioning process, transitioning from an initial random network to a structured feed-forward architecture. This transformation aligns with the establishment of a robust and enhanced population burst characterized by a sequence of temporally ordered spiking neurons, maintained upon receiving a “targeted stimulus.” This linkage between the emergence of specific network structure and enhanced burst pattern stems directly from the influence of the dopamine-modulated STDP mechanism.

Furthermore, we explored the impact of varying basal dopamine levels on Pavlovian memory. As expected, the presence of inherent system noise, leading to isolated random single spikes, contributes to memory degradation. However, under low basal dopamine levels, Pavlovian memory endures. Regarding memory retention, our findings indicate that spontaneously generated noise-induced population bursts, resembling spike avalanches of varying magnitudes, facilitate the consolidation of pre-established feed-forward network architecture. Consequently, an intriguing equilibrium emerges, regulated by basal dopamine levels: stochastic single spikes and autonomous population bursts vie for predominance, influencing either memory erosion or preservation.

In conclusion, the Izhikevich model for Pavlovian conditioning with a distal reward has proven to be a valuable framework for meticulously examining the transformation of neural spiking dynamics and network structure concurrently during the learning process. We are confident that similar methodologies can be systematically employed to unravel the complexities inherent in spiking neural networks when tackling more intricate tasks.

Methods

Mathematical model

The neurotransmitter dopamine is known to play a crucial role in the learning process of associative cues and rewards [33–36], as observed in Pavlovian conditioning [37]. Additionally, dopamine is widely recognized as a significant modulator of the STDP mechanism [38, 39]. In this context, years ago, Izhikevich proposed an illuminating mathematical model neural network that incorporates a dopamine-modulated STDP mechanism, effectively addressing the “distal reward problem” commonly encountered in classical conditioning paradigms. This simple, yet elegant model system for Pavlovian conditioning provides an excellent opportunity to thoroughly investigate the interaction and evolution of neural spiking dynamics, particularly population bursting, and network structure during memory formation and erosion, delving into intricate details.

Our model network, which consists of 2,000 neurons, comprising 1,600 excitatory and 400 inhibitory Izhikevich neurons, is a slightly modified version of the original model introduced by Izhikevich [3]. The neurons evolve dynamically according to the following set of equations: (1) (2) where V and r represent the membrane potential and recovery variable (subsequent to a spike firing), respectively. When V crosses a prescribed threshold value, the state variables reset:

Throughout this study, we employ a fixed set of parameter values: c = -65 mV, b = 0.2 nS, C = 1 nF, k = 0.04 pA/mV2, Vr = -82.7 mV, Vt = -42.3 mV, and a = 0.02 (0.1) ms−1 and d = 8 (2) pA for excitatory (inhibitory) neurons. The input current I comprises synaptic current inputs Isyn, noisy current Iη, and conditioning current Iinject. The noisy current Iη is randomly chosen from a uniform distribution of [−5.5, 5.5) and added at all iteration time steps. This addition introduces irregular spontaneous activity (noise) that is unrelated to external stimulation and accounts for trial-to-trial fluctuations in neuronal responses. During the Pavlovian conditioning period, Iinject is administered to a subpopulation approximately once every 200 ms, selected uniformly from the range of (100 ms, 300 ms).

Each neuron has efferent connections to randomly chosen 100 other neurons. Excitatory neurons connect to inhibitory as well as excitatory neurons with dynamically varying synaptic weights Wij ranging from 0 to 4, whereas inhibitory neurons only connect to excitatory neurons with a fixed synaptic weight of -1.0. No conduction time delays are assumed. So, the input synaptic current of the ith neuron can be written as: (3) where tj stands for the arrival times of incoming spikes from presynaptic neurons.

The model incorporates a “three-factor STDP learning rule’’ [3], where each weight Wij coevolves with its matching eligibility trace cij according to the following equations: (4) (5) (6)

Here, STDP(Δt) represents the effect of the temporal difference Δt = tpost − tpre (i.e., the relative timing between pre- and post-synaptic spikes) on the eligibility trace cij, which denotes the current level of synaptic activation (reflecting all the history of past synaptic activations). The Dirac delta function δ(t − tpre/post) confers step <increases in cij whenever there is a pre-post pair event. In the absence of significant pre-post pair events, cij gradually decays with a time constant τc = 1000 ms. The employed STDP function is defined as follows: K+ exp(−Δt/τ+) for Δt > 0 and K− exp(Δt/τ−) for Δt < 0, where K+ = 0.12, K− = −0.10, and τ+ = τ− = 20 ms. A+ and A− account for the strength of potentiation and depression, respectively, and τ+ and τ− represent the relevant STDP time range.

The rate of change of synaptic weight Wij is the product of cij and Dr + D0, where Dr and D0 are global varables representing the levels of rewarded dopamine, which is time-dependent, and constant basal dopamine, respectively. In our Pavlovian conditioning protocol, whenever a dopamine reward is given, an increment of 0.5 is added to the pre-existing level of Dr: that is, DA(t) = 0.5δ(t − trew), where trew stands for the time of reward. Dr decays with a time constant ms.

At this stage, it’s pertinent to highlight several modifications made to the original model proposed by Izhikevich [3]. Firstly, we have doubled the size of our model network to encompass 2,000 neurons, compared to the original model, which included 1,000 neurons. This expansion aims to scrutinize any finer-scale features of network morphology that may exist and to evaluate the scalability of the observed properties. Secondly, certain parameter values have been adjusted from those of the original study. These adjustments include increasing the stimuli set size to 100, raising A+ to 0.12 and A− to -0.10, and adjusting the inhibitory weight to -8, in contrast to the original study’s stimuli set size of 50, A+ of 0.10, A− of -0.15, and inhibitory weight of -1. Particularly noteworthy is the augmentation of A+, intended to promote more frequent spontaneous population bursts, a crucial element for addressing the consolidating memory effect of such bursts beyond the conditioning phase. Additionally, we have systematically varied the value of D0 from 1 nM to 5 nM to investigate the impact of tonic dopamine levels on memory retention or erosion. Despite these substantial modifications, the Pavlovian conditioning remains essentially unchanged.

Loop length statistics

The number of closed paths having a length l is obtained by: where A is the adjacency matrix of a given network [40]. We have set Aij = 1 (for Wij ≥ 2) or 0 (for Wij < 2). Then, a new variable ‘loop length ratio’ (or Ratio) is defined as: where Nshuffle(l) denotes the number of loops calculated from the randomly shuffled version of Aij.

Simulation

For all simulations, we utilized a customized version of the Python code (PyGeNN implementation of three-factor STDP) provided by Knight and Nowotny [41]. The simulations and subsequent analyses were performed using a GIGABYTE RTX 3090 GPU installed on an Ubuntu system.

Supporting information

S1 Fig. Similarity of two scatter plots.

(A) Scatter plot displaying the relationship between ∑Win and ∑Wout of a Pavlovian conditioned network at a specific time point (t = 3,600 s). This scatter plot is extracted from Fig 3B. (B) Scatter plot illustrating the relationship between in-degree and outdegree of the same Pavlovian conditioned network. This figure highlights the striking resemblance between the two scatter plots, providing evidence for the consistency of network connectivity patterns as revealed by two distinct measures.

https://doi.org/10.1371/journal.pcsy.0000035.s001

(TIFF)

S2 Fig. Exemplary trajectories of neurons across the ∑Wout vs. ∑Win plane throughout the entire Pavlovian conditioning process.

Some sample trajectories of neurons are depicted, illustrating their movements across the ∑Wout vs. ∑Win plane throughout the entire Pavlovian conditioning process. Orange traces denote the paths of 8 randomly selected neurons from the target subpopulation that receive direct electrical stimulations. In contrast, purple traces represent the trajectories of 8 neurons randomly chosen from the top 100 neurons with the highest ∑Win values at t = 3,600 s. Grey traces depict the trajectories of 8 neurons not belonging to these two specified groups.

https://doi.org/10.1371/journal.pcsy.0000035.s002

(TIFF)

S3 Fig. Classification of excitatory neurons based on their dynamic features.

(A) Three representative ISI histograms are shown. Inter-spike intervals (ISIs) are calculated for all 1,600 excitatory neurons within a time window of last 100 seconds of the conditioning protocol. Three representative ISI histograms, each selected from a different subpopulation, are displayed. The subpopulations are identified through a hierarchical clustering process as depicted in B: the distance measure used for the clustering is the vectorial distance between two “ISI histogram vectors.” The scatter plot in (C) showcases the distinct territories occupied by the three subpopulations of neurons. (D) Three return maps of ISI time series that are used for the histograms in (A) are presented.

https://doi.org/10.1371/journal.pcsy.0000035.s003

(TIFF)

S4 Fig. Free evolution of a Pavlovian conditioned feed-forward network (with D0 = 0.0026).

(A) Synaptic weight distribution (top) and recurrent loop-length statistics (bottom) at various free-running stages [t = 3,576 (red), t = 3,948 s (green), 4,188 s (blue), 4,668 s (black)]. Loop-length ratios are relative to the initial random network. (B) Scatter plots illustrating the passage towards a complete memory loss. The initial, feed-forward network (t = 3,576 s) transforms quickly into an almost random, recurrent network (t = 4,668 s).

https://doi.org/10.1371/journal.pcsy.0000035.s004

(TIFF)

S5 Fig. Spontaneously generated superbursts supported by a completely memory-lost Pavlovian conditioned network (D0 = 0.0026 case).

This is a blown-up image of the highlighted area in the fourth row of Fig 8.

https://doi.org/10.1371/journal.pcsy.0000035.s005

(TIFF)

S1 Video. Persistence of memory following Pavlovian conditioning.

This video matches the result discussed in Fig 6.

https://doi.org/10.1371/journal.pcsy.0000035.s006

(MP4)

S2 Video. Loss of memory following Pavlovian conditioning.

This video matches the result discussed in Fig 8.

https://doi.org/10.1371/journal.pcsy.0000035.s007

(MP4)

Acknowledgments

This work was supported by the National Research Foundation of Korea (NRF) Grant funded to KL by the Korea Government (MEST) (No. 2019R1A2C2005989 and RS-2024-00335928) and Electronics and Telecommunications Research Institute(ETRI) grant funded to KL by the Korean government [ETRI 4011-2022-00020, Next generation SNN model development (ML application purpose)]. We thank Dr. J. Knight for helping us in using GENN library and Jeongmu Kim and Woojun Park for critical reading of the manuscript.

References

  1. 1. Chong E, Moroni M, Wilson C, Shoham S, Panzeri S, Rinberg D. Manipulating synthetic optogenetic odors reveals the coding logic of olfactory perception. Science. 2020;368(6497):eaba2357. pmid:32554567
  2. 2. Singer W. Recurrent dynamics in the cerebral cortex: Integration of sensory evidence with stored knowledge. Proceedings of the National Academy of Sciences. 2021;118(33):e2101043118. pmid:34362837
  3. 3. Izhikevich EM. Solving the Distal Reward Problem through Linkage of STDP and Dopamine Signaling. Cerebral Cortex. 2007;17(10):2443–2452. pmid:17220510
  4. 4. van Vreeswijk C, Sompolinsky H. Chaos in Neuronal Networks with Balanced Excitatory and Inhibitory Activity. Science. 1996;274(5293):1724–1726. pmid:8939866
  5. 5. Beggs JM, Timme N. Being Critical of Criticality in the Brain. Frontiers in Physiology. 2012;3:163. pmid:22701101
  6. 6. Goldstein RE. Traveling-Wave Chemotaxis. Phys Rev Lett. 1996;77:775–778. pmid:10062899
  7. 7. Borges FS, Protachevicz PR, Pena RFO, Lameu EL, Higa GSV, Kihara AH, et al. Self-sustained activity of low firing rate in balanced networks. Physica A: Statistical Mechanics and its Applications. 2020;537:122671.
  8. 8. Kim B, Lee KJ. Self-organized neuronal subpopulations and network morphology underlying superbursts. New Journal of Physics. 2022;24(4):043047.
  9. 9. Bakkum DJ, Chao ZC, Potter SM. Spatio-temporal electrical stimuli shape behavior of an embodied cortical network in a goal-directed learning task. J Neural Eng. 2008;5:310–323. pmid:18714127
  10. 10. Choi JH, Kim JH, Heo R, Lee KJ. Modulating the precision of recurrent bursts in cultured neural networks. Phys Rev Lett. 2012;108:138103. pmid:22540729
  11. 11. Kim JH, Lee HJ, Choi W, Lee KJ. Encoding information into autonomously bursting neural network with pairs of time-delayed pulses. Scientific Reports. 2019;9(1):1394. pmid:30718675
  12. 12. Zenke F, Agnes EJ, Gerstner W. Diverse synaptic plasticity mechanisms orchestrated to form and retrieve memories in spiking neural networks. Nature Communications. 2015;6(1):6922. pmid:25897632
  13. 13. Raphan T, Dorokhin E, Delamater AR. Modeling Interval Timing by Recurrent Neural Nets. Frontiers in Integrative Neuroscience. 2019;13. pmid:31555104
  14. 14. Zhou S, Masmanidis SC, Buonomano DV. Neural Sequences as an Optimal Dynamical Regime for the Readout of Time. Neuron. 2020;108(4):651–658.e5. pmid:32946745
  15. 15. Barri A, Wiechert MT, Jazayeri M, DiGregorio DA. Synaptic basis of a sub-second representation of time in a neural circuit model. Nature Communications. 2022;13(1):7902. pmid:36550115
  16. 16. Buonomano DV. Decoding Temporal Information: A Model Based on Short-Term Synaptic Plasticity. Journal of Neuroscience. 2000;20(3):1129–1141. pmid:10648718
  17. 17. Fallon SJ, Gowell M, Maio MR, Husain M. Dopamine affects short-term memory corruption over time in Parkinson’s disease. npj Parkinson’s Disease. 2019;5(1):16. pmid:31396548
  18. 18. Meder D, Herz DM, Rowe JB, Lehéricy S, Siebner HR. The role of dopamine in the brain—lessons learned from Parkinson’s disease. NeuroImage. 2019;190:79–93. pmid:30465864
  19. 19. Tsetsenis T, Broussard JI, Dani JA. Dopaminergic regulation of hippocampal plasticity, learning, and memory. Frontiers in behavioral neuroscience. 2023;16:1092420. pmid:36778837
  20. 20. Levy WB, Steward O. Temporal contiguity requirements for long-term associative potentiation/depression in the hippocampus. Neuroscience. 1983;8(4):791–797. pmid:6306504
  21. 21. Bi Gq, Poo Mm. Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type. Journal of Neuroscience. 1998;18(24):10464–10472. pmid:9852584
  22. 22. Caporale N, Dan Y. Spike Timing–Dependent Plasticity: A Hebbian Learning Rule. Annual Review of Neuroscience. 2008;31(1):25–46. pmid:18275283
  23. 23. Tsodyks M. Spike-timing-dependent synaptic plasticity—the long road towards understanding neuronal mechanisms of learning and memory. Trends in Neurosciences. 2002;25(12):599–600. pmid:12446119
  24. 24. Feldman DE. The spike-timing dependence of plasticity. Neuron. 2012;75:556–571. pmid:22920249
  25. 25. Rubin J, Lee DD, Sompolinsky H. Equilibrium Properties of Temporally Asymmetric Hebbian Plasticity. Phys Rev Lett. 2001;86:364–367. pmid:11177832
  26. 26. van Rossum MCW, Bi GQ, Turrigiano GG. Stable Hebbian Learning from Spike Timing-Dependent Plasticity. Journal of Neuroscience. 2000;20(23):8812–8821. pmid:11102489
  27. 27. Gilson M, Fukai T. Stability versus Neuronal Specialization for STDP: Long-Tail Weight Distributions Solve the Dilemma. PLoS ONE. 2011;6(10):e25339. pmid:22003389
  28. 28. Bliss TVP, Gardner-Medwin AR. Long-lasting potentiation of synaptic transmission in the dentate area of the unanaesthetized rabbit following stimulation of the perforant path. The Journal of Physiology. 1973;232(2):357–374. pmid:4727085
  29. 29. Artola A, Bröcher S, Singer W. Different voltage-dependent thresholds for inducing long-term depression and long-term potentiation in slices of rat visual cortex. Nature. 1990;347(6288):69–72. pmid:1975639
  30. 30. Markram H, Lübke J, Frotscher M, Sakmann B. Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs. Science. 1997;275(5297):213–215. pmid:8985014
  31. 31. Sjs̎trm̎ PJ, Turrigiano GG, Nelson SB. Rate, Timing, and Cooperativity Jointly Determine Cortical Synaptic Plasticity. Neuron. 2001;32(6):1149–1164.
  32. 32. Chistiakova M, Bannon NM, Bazhenov M, Volgushev M. Heterosynaptic Plasticity: Multiple Mechanisms and Multiple Roles. The Neuroscientist. 2014;20(5):483–498. pmid:24727248
  33. 33. Schultz W, Dayan P, Montague PR. A Neural Substrate of Prediction and Reward. Science. 1997;275(5306):1593–1599. pmid:9054347
  34. 34. Calabresi P, Gubellini P, Centonze D, Picconi B, Bernardi G, Chergui K, et al. Dopamine and cAMP-Regulated Phosphoprotein 32 kDa Controls Both Striatal Long-Term Depression and Long-Term Potentiation, Opposing Forms of Synaptic Plasticity. Journal of Neuroscience. 2000;20(22):8443–8451. pmid:11069952
  35. 35. Schultz W. Getting Formal with Dopamine and Reward. Neuron. 2002;36(2):241–263. pmid:12383780
  36. 36. Seamans JK, Yang CR. The principal features and mechanisms of dopamine modulation in the prefrontal cortex. Progress in Neurobiology. 2004;74(1):1–58. pmid:15381316
  37. 37. Saunders BT, Richard JM, Margolis EB, Janak PH. Dopamine neurons create Pavlovian conditioned stimuli with circuit-defined motivational properties. Nature Neuroscience. 2018;21(8):1072–1083. pmid:30038277
  38. 38. Louth EL, Jørgensen RL, Korshoej AR, Sørensen JCH, Capogna M. Dopaminergic Neuromodulation of Spike Timing Dependent Plasticity in Mature Adult Rodent and Human Cortical Neurons. Frontiers in Cellular Neuroscience. 2021;15. pmid:33967700
  39. 39. Pawlak V, Wickens J, Kirkwood A, Kerr J. Timing is not Everything: Neuromodulation Opens the STDP Gate. Frontiers in Synaptic Neuroscience. 2010;2:556–571. pmid:20876200
  40. 40. Newman M. Networks. London: Oxford University Press; 2018.
    • 41. Knight JC, Komissarov A, Nowotny T. PyGeNN: A Python Library for GPU-Enhanced Neural Networks. Frontiers in Neuroinformatics. 2021;15:659005. pmid:33967731