Figure 1.
Definitions of network connectivity.
Illustration of different connectivity measures for a synaptic network connecting neuron populations
to
(which may be identical for recurrent networks). A, Anatomical connectivity
and potential connectivity
are fractions of neuron pairs
connected by an actual (black circles) and potential synapse (blue rectangles), respectively. B, The consolidation signal
specifies the ensemble of neuron pairs that request a synapse (
, red circles) to support storage of a given memory set. The corresponding effectual connectivity
is then the fraction of neuron pairs requesting a synapse that are already connected by an actual synapse. The consolidation load
is the fraction of neuron pairs that request a synapse.
Figure 2.
Model of structural plasticity and consolidation.
A, State/transition model of a single potential synapse (see text for details). B, In the following we consider potential synapses in a network , for example, connecting two cortical neuron populations
and
. Memories correspond to associations between activity patterns
and
. We will specifically analyze how well noisy activity patterns
can reactivate the corresponding memories
in order to estimate storage capacity. C, D: LTM storage (solid) by structural plasticity requires repetitive reactivation of activity patterns in cortical populations
and
to provide an appropriate consolidation signal
to the synapses. This may happen by repeated bottom-up stimulation (D) or, for episodic memories, by top-down replay (C) from a HC-type STM buffer (dashed). LTM = long-term memory; STM = short-term memory; HC = hippocampus.
Figure 3.
Learning in Willshaw-type associative networks.
A, Memory storage by Hebbian weight plasticity (Eq. 5) in a fully connected network (). Address patterns
are associated to content patterns
where
(here
). Each memory is represented by a binary activity vector of length
having
active units (which define the corresponding cell assembly). B, One-step retrieval of the first memory from a noisy query pattern
having two of the four active units in
(
). Here
can perfectly reactivate the corresponding memory pattern in population
(
) applying a firing threshold
on dendritic potentials
. C, As a simple form of structural plasticity, silent synapses can be pruned after learning. The resulting network has only 28 (instead of 49) synapses corresponding to a lower anatomical connectivity
, whereas the effectual connectivity is still
. Thus, pruning does not change network function, but increases stored information per synapse. D, Ongoing structural plasticity can similarly increase storage capacity during more realistic learning in networks with low anatomical connectivity (here
). During each time step
, Hebbian weight plasticity potentiates and consolidates synapses
with non-zero consolidation signal
(which equals
of panel A), whereas the remaining silent synapses are eliminated and replaced by new synapses at random locations. Note that the resulting network at
is the same as in panel C.
Figure 4.
Increase of effectual connectivity during memory consolidation with ongoing structural plasticity.
Each curve shows the evolution of effectual connectivity as a function of time
for different parameters
(anatomical connectivity),
(potential connectivity),
(consolidation load), and
(fraction of initially consolidated synapses). Data are from single microscopic network simulations (solid black; cf. Eq. 4; network size
) and macroscopic theory (dashed gray; Eq. 11). See Table 1 for further simulation parameters. A:
for different consolidation loads
and constant
,
,
. B:
for different fractions of initially consolidated synapses
and constant
,
,
. C:
for different anatomical connectivities
and constant
,
,
.
Figure 5.
Storage capacities for a finite Willshaw network having the size of a cortical macrocolumn ().
A, Contour plot of pattern capacity (number of stored memories) as a function of assembly size
(number of active units in a memory vector) and effectual network connectivity
assuming output noise level
and noise-free input patterns (
,
). B, Weight capacity
for the same setting as in panel A. C, Total storage capacity
including structural plasticity for the same setting as in A. Note that even modest increases of
can strongly increase storage capacity, in particular for sparse neural activity (small
) [82]. All data computed from Gaussian approximation of dendritic potential distributions (see appendix II. 2).
Figure 6.
Simulation of catastrophic forgetting, Ribot gradients, and the spacing effect.
A, Networks without structural plasticity suffer from catastrophic forgetting (top), but networks with structural plasticity do not (bottom). Plots show output noise over time
simulating networks of size
and activity
storing 25 memory blocks one after the other (only the interesting part between storage of blocks 6 and 21 are visible). Each curve (with a distinct color) corresponds to
for noisy test patterns of a particular memory block with
correct and
false active units. The steep descent of each curve corresponds to the time when the Hippocampus started to replay the corresponding memory block for 5 time steps. B, Networks employing structural plasticity show Ribot gradients after a cortical lesion (top) due to gradients in effectual connectivity (bottom). The lesion was simulated by deactivating half of the neurons in population
at time
. C, Networks employing structural plasticity reproduce the spacing effect of learning. In the first simulation (blue) novel memories were rehearsed once for 20 time steps (blue arrow at
). In a second simulation (red) the same total rehearsal time was “spaced” or distributed to four brief intervals of five steps each (red arrows at
,
,
, and
). Here the network achieves a higher effectual connectivity
(bottom) and less retrieval noise
(top). See Sections 2, 3 and Table 1 for further details and simulation parameters.
Table 1.
Simulation parameters.
Figure 7.
Sketch of network connectivity reflecting lifelong structural plasticity.
During development anatomical connectivity (thick solid) quickly increases reaching a peak level (around 2–3y in humans), where the initial increase is followed by a short period of stable connectivity (until age 5y in humans), a phase of significant decrease of connectivity until puberty, and finally a phase of stable connectivity during adulthood [14], [51], [77]. Recent experiments suggest a temporary novelty-driven (thick arrows) increase of connectivity during adulthood [23], [68], [116]. Our model of structural plasticity predicts that learning is fastest for high levels of anatomical connectivity and structural plasticity. Thus, memories acquired during early phases can reach higher levels of effectual connectivity (
,
; thin solid lines) compared to memories acquired during later phases (
,
). The resulting gradients in effectual connectivity can explain various memory phenomena (see Section 7 for details).