Fig 1.
Schematic illustration of an Izhikevich model network undergoing Pavlovian learning.
The network consists of 1,600 excitatory neurons (filled red dots) and 400 inhibitory neurons (blue circles), interconnected with a sparsity of 0.1. The thickness of each connecting arrow represents the synaptic weight strength. To facilitate the Pavlovian conditioning protocol, 100 subpopulations are formed, each comprising 100 randomly selected neurons from the overall population (only 4 subpopulations, each of which contains only 3 neurons, are shown in the scaled-down schematic diagram). During the conditioning process, a non-rewarded electrical stimulation is applied to a randomly chosen subpopulation at a time interval between 100 to 300 ms. In contrast, a rewarded stimulation is administered to a specific subpopulation (S1) at approximately 5-second intervals. The synaptic weights evolve dynamically in accordance with the three-factor dopamine-modulated STDP model described in the Model Section.
Fig 2.
Amplification of rewarded stimulus-evoked bursts during Pavlovian conditioning.
(A) Raster plots showcasing neural spikes and stimulus-triggered bursts at different conditioning stages. The arrows atop each plot denote stimulation timings: the green (black) arrows mark the times of rewarded (non-rewarded) stimulations given to the target (non-target) subpopulation(s) (refer to Fig 1). Beneath each raster plot, the corresponding spike density function (SDF) is depicted. Population SDF was computed by convolving spikes with a Gaussian kernel of σ = 10 ms width. Magenta triangle markers pinpoint reward times. (B) Burst profiles (green lines) evoked by a rewarded stimulus. Overlaying each graph is the average non-rewarded stimulus-evoked burst profile (black solid line), accompanied by gray shades illustrating the corresponding standard deviation across a 4-second time-frame. Vertical dashed lines and green (black) arrows denote the timings of rewarded (non-rewarded) stimuli. It is noteworthy that the population burst SDF corresponding to the green arrows undergoes significant enhancement as the conditioning progresses, while those corresponding to the black arrows remain more or less unchanged. A small level of basal dopamine D0 = 0.0015 μM was assumed.
Fig 3.
Evolution to feed-forward network architecture during Pavlovian conditioning.
(A) Synaptic weight distribution (top) and recurrent loop-length statistics (bottom) at various conditioning stages [t = 0 (magenta), t = 24 s (red), 240 s (green), 3,600 s (blue)]. Loop-length ratios are relative to the networks that are randomly shuffled (black dashed line). (B) Scatter plots illustrating the sum of afferent weights (∑Win) versus the sum of efferent weights (∑Wout) of all excitatory neurons. Orange points identify neurons within the subpopulation (S1)receiving rewarded stimuli, while purple points represent the top 100 neurons with the highest ∑Win values at t = 3,600 s. (C) Two-dimensional “spring-force layout” visualization of the network corresponding to the scatter plot in B (t = 3,600 s). Notably, the central cluster of orange points becomes a “source,’’ and the peripheral group of purple points on the outskirts becomes a “sink.’’ The orange and purple insets highlight intra-connectivity among excitatory neurons in the source (occupying the core area) and in the sink (occupying the peripheral area), respectively: The intra-connections are emphasized with thicker lines (black: weight = 4, white: weight = 0)compared to the dense, fuzzy background inter-connections.
Fig 4.
Spreading spiking activity within a superlative burst rendered shown in the space of ∑Wout vs. ∑Win (A) and average spiking times of 1,600 excitatory neurons vs. their ∑Win in (B).
The figure in (A) showcases the propagation of spiking activity within an evoked burst, which is linked to a rewarded stimulation. The colored points represent neurons firing at specific labeled times. Notably, the spiking activity exhibits a clear propagation pattern towards the lower right-hand corner of the plot. The superlative burst corresponding to this propagation is indicated by a green arrow in Fig 2A (3rd row). The figure shown in (B) presents the average spiking times of all 1,600 excitatory neurons across 92 induced population burst events facilitated by a fully Pavlovian conditioned network. These times are then plotted as a function of corresponding neurons’ ∑Win. The larger points (located in the lower left-hand corner) correspond to neurons that receive direct electrical stimulation. Notably, there exists a small (∼1 ms) time delay between the source (large points) and the remaining neurons (small points).
Fig 5.
Heuristic explanation of the formation and deformation of the feed-forward network.
(A) A toy model raster plot illustrating a neural burst in a 6-neuron system, where each neuron fires once and has its own fixed spiking time [#1 (black) first and #6 (violet) last]. The neurons are assumed to be physically connected all-to-all. (B) Schematic illustration of a functionally feed-forward network of the toy model, created by assuming the classical STDP mechanism based on causality assumption. Each colored line with an arrow-head represents a functionally active efferent connection. The two numbers in each bracket represent in-degree and out-degree, respectively. (C) Plot depicting the (in-degree, out-degree) pairs of the feed-forward network shown in (B). (D) Schematic diagram of the feed-forward network from (B), with the addition of two extra recurrent connections (thick red and orange arrows). (E) Scatter plot displaying the (in-degree, out-degree) pairs of the modified feed-forward network depicted in (D).
Fig 6.
Persistence of memory following Pavlovian conditioning: D0 = 0.001 μM case.
(A) Raster plots and corresponding SDF profiles depicting the final stage of conditioning (highlighted in the shaded first row) and the subsequent periods after conditioning completion (second to fourth rows). After the conditioning stage, population bursts are self-generated. Note that the SDF profile (blue line) graph shown in the first row has a scale different from those (black lines) of other rows. (B) Scatter plots showcasing the relationship between ∑Win and ∑Wout for all 1,600 excitatory neurons. They are evaluated at the ending time of the corresponding raster plots on their left. The three distinct point colors represent different clusters of excitatory neurons, classified based on their dynamic characteristics (refer to S4 Fig for detailed classification information). Notably, even long after the conclusion of Pavlovian conditioning, the territories occupied by the three neuronal clusters remain quite well-maintained. The self-organized spontaneous burst used for the analysis in Fig 6 is marked with a star symbol in the fourth row. [See S1 Video for the whole process].
Fig 7.
Spreading spiking activity associated with a spontaneously generated burst in a previously Pavlovian-conditioned network.
This figure highlights the propagation of spiking activity within a burst that occurs spontaneously in a Pavlovian-conditioned network. The colored points represent neurons firing at specific labeled times. While the firing neurons at each time instance are dispersed across the scatter plot, the centroids of their distributions exhibit a clear tendency to move towards the lower right-hand corner. This observation suggests a directional propagation of spiking activity within the burst. The burst corresponding to this propagation, which is selected randomly, is indicated by a star symbol in Fig 6A (third row). The time labels are given with respect to the temporal position (time = 0) of the maximum SDF of the star-marked burst.
Fig 8.
Loss of memory following Pavlovian conditioning: D0 = 0.0026 μM case.
(A) Raster plots and corresponding SDF profiles: during the final stage of conditioning (highlighted in the shaded first row) and the subsequent periods after conditioning completion (second to fourth rows). The green shaded area in the fourth row of (A) is magnified in S5 Fig to reveal the multi-hump nature (i.e., superburst) of the self-organized bursts. Note that the SDF profile (blue line) graph shown in the first row has a scale different from those (black lines) of other rows. (B) Scatter plots depicting the relationship between ∑Win and ∑Wout for all 1,600 excitatory neurons. They are evaluated at the ending time of the corresponding raster plots on their left. The three distinct point colors indicate different clusters of excitatory neurons, categorized based on their dynamic characteristics (see S3 Fig). Note that the Pavlovian conditioned, feed-forward network quickly disintegrates into a random network. [See S2 Video for the whole process being discussed].
Fig 9.
Multistability of the network structure long after the cessation of Pavlovian conditioning.
(A) Scatter plots of (∑Win, ∑Wout) for three different network types. These representative cases are selected from the dopamine level of D0 = 0.0011 μM (T1, blue), D0 = 0.0040 μM (T2, green), and D0 = 0.0048 μM (T3, red). (B) Graph illustrating the fractions of three different morphological states, which are illustrated in (A), for various values of D0. Different network types are realized stochastically without any controls from different initializations.