Implications and Predictions of storage degradation vs. Retrieval failure

The distinction between these two families of theories is not merely academic; it has important implications for both memory research and clinical practice. Consider, for example, Alzheimer’s Disease (AD), a dramatic form of abnormal forgetting. If the storage degradation theory is correct, the promising new clinical treatments that remove amyloid beta deposits (e.g., [19,20]) would stop the progression of the disease but would not restore memory function and would not bring back lost memories, as the underlying neural substrate has already been compromised. However, according to the retrieval failure theory, at least some lost memories might be retrievable, and function could be partially restored. Knowing which theory is correct, therefore, is important to guide treatment and future clinical research.

The current understanding of the mechanisms by which memories are encoded, maintained, and retrieved makes it possible to distinguish between these two theories. Most theories assume that the initial representation of a memory occurs in the hippocampus, by creating synapses between hippocampal cells that are also connected to the sensory areas that were initially involved at the moment of encoding [21,22]. The re-activation of this cell assembly, sometimes referred to as engram [4,8,23,24], results in the re-experience of the associated memory [25,26]. Replays of this activity, either through active retrieval or spontaneous brain activity, result in consolidation. This internal re-activation of the memories is supported by the coordinated activity of the so-called Default Mode Network (DMN), a large-scale brain network that includes the hippocampus, the medial frontal cortex, and the precuneus [27]. The DMN is believed to play a key role in propagating the re-activation of the original engrams, mediating the interactions between the hippocampus and the neocortical regions during consolidation [28]. In fact, the DMN’s activity at rest is explained as a way of rehearsing and maintaining memory as a form of “priors” for later processing [10,29,30].

The controlled retrieval of declarative memories, on the other hand, is a multifaceted process supported by the activity of the DMN and multiple dorsal and lateral prefrontal circuits. The DMN might be involved in mediating access to declarative memories, and episodic traces in particular [31–33]. Dorsal prefrontal regions are involved in voluntary and controlled memory retrieval as well as working memory, and provide a natural connection between the two forms of memory [34]. Ventral lateral prefrontal regions, on the other hand, are specifically involved in the selection between competing memories [35], including the resolving conflict between potential candidates [36] and maintaining retrieval cues [37–39]. Ventral and medial prefrontal regions, finally, seem to play a critical role in monitoring the retrieval process, as evidenced by their involvement in meta-memory judgments and feeling-of-knowing [40–43] and in confabulation in frontal patients [44,45]. Damage to these frontal regions, as well as the degradation of connectivity between these regions and the DMN, might result in the appearance of forgetting even when the underlying neural representations are intact—and might be directly awakened through electrical stimulation [14].

Since consolidation and retrieval processes have different neural substrates, the storage degradation and retrieval failure theories also make different predictions in terms of the neural correlates of forgetting. According to the storage degradation framework, individual differences in forgetting should be accompanied by differences in the functional connectivity between the DMN [28,46] and the sensory and association areas where the individual features of a memory are initially encoded—and where long-term memories might be stored as the end-product of systems consolidation [47–49].

According to the retrieval failure framework, however, forgetting occurs because of an inability to voluntarily access previously stored memories, even if the same memories can be involuntarily re-activated (for example, through local brain stimulation). Because the voluntary retrieval of memories is controlled by lateral prefrontal cortices [34,35,39], the theory predicts that forgetting would be associated with the functional connectivity between the prefrontal and parietal regions that oversee controlled retrieval and the encoding and consolidation regions of the DMN [50].

Obstacles to the testing the theories

Since the two theories make different predictions, it should be possible to empirically test them by examining which features of individual anatomy and functional connectivity co-vary with individual differences in forgetting. One important obstacle, however, remains: it is extremely difficult to quantify forgetting in the first place. As noted above, forgetting is typically operationalized as failing to correctly recall or recognize an item in a memory test: performance failure is the only ground truth available. Unfortunately, this carries significant limitations. First, common behavioral measures such as the proportion of correctly remembered items are prone to statistical misinterpretations [51,52]. Second, performance in memory tests is also affected by the specifics of the paradigms used. For example, memory performance differs greatly when the same materials are tested using recognition or recall; recognition is generally easier, but recall, being more effortful, leads to better subsequent consolidation and retention. Third, this retrieval accuracy-based operationalization does not easily account for possible temporal dynamics of forgetting. If forgetting is conceptualized as a process, a more precise assessment of memory accessibility is needed even if a memory test results in a successful retrieval.

The goal of this study is to overcome these limitations through computational phenotyping [53,54], that is, by fitting a formal model of cognitive processes to individual task data, and using the model’s parameters to quantify individual differences. Model parameters obtained through computational phenotyping are easier to interpret and more reliable than raw behavioral measures [54–56], such as accuracies or response times. In this study, individual differences in forgetting were quantified by fitting a computational model to individual learning data in a sample of healthy individuals for whom functional connectivity data were available. By analyzing which connectivity features best predict individual differences in forgetting, we can empirically compare the storage degradation and retrieval failure theories.

A computational theory of forgetting

As referred to above in the context of proportion of correct responses, purely behavioral measures of psychological constructs often have poor reliability [52], which leads, in turn, to poor brain-behavioral associations [57]. One way to obviate this limitation is to fit realistic computational models to individual behaviors, and use the fitted model parameters as dependent variables [58,59]. When the model captures the dynamics of the underlying neural process, its parameters can be more reliable and interpretable than pure behavioral measures. This has been shown, for example, with the threshold and drift parameters of accumulator models in perceptual decision-making [60,61] and in learning rate parameters in reinforcement learning models of learning [56]. Finally, using realistic models of long-term memory avoids the pitfalls that were originally pointed out by Loftus [51,62] in applying statistical linear models to non-linear effects in memory performance.

In this study, the estimation of the speed of forgetting was based on a well-established model of episodic memory [55,63–66] that is also the cornerstone of the ACT-R theory [67–69]. This model, coupled with an adaptive learning task that dynamically adjusts the presentation schedules of the stimuli [70], yields individual speeds of forgetting for individuals that can be measured within a short time (< 15 mins) but can be reliably extrapolated over much longer intervals spanning days or weeks [55,66,71,72].

In the model, the probability of retrieving a memory m at time t is proportional to its activation A(m, t), a scalar quantity that reflects a memory’s availability. As in the multiple trace theory [73,74], each memory m is associated with multiple episodic traces, each of which corresponds to specific (but potentially overlapping) neural representations that were created when m was retrieved or (re)encoded, and each of which decays over time.

The activation of each trace decays as a power function of time [75], and the activation of m is the log sum of the decaying activations of all its traces:

(1)

where t_i and d_i are, respectively, the time of the creation of the i-the trace and its characteristic “decay rate”. The decay rate can be thought of as the degree of damage (caused by either biological decay, system consolidation, or interference) that a trace accumulates per unit time, and which reduces that trace’s quality and, therefore, retrievability [69]. A trace’s decay rate, in turn, is related to the m’s activation at the moment each trace is created, thus capturing the fact that traces with higher initial activation decay faster than traces with lower activation [64]. Specifically:

(2)

with A(m, t_i) being the activation of m at the moment the i-th trace was encoded, and c is a scaling factor fixed to 0.25, as previously proposed by [70]. Thus, if the times at which a memory m has been encoded, re-encoded, and retrieved are known and controlled for, its activation depends only on φ, which is the only free parameter in the above equations. After determining the value of φ of each memory, their average was taken to represent an individual-specific speed of forgetting (SoF).

The activation of a memory, thus defined, determines its retrieval probability. Specifically, the retrieval probability P(m) of a memory m can be approximated as:

(3)

Previous studies have shown that individual values of φ are stable across time and materials and predictive of real-world outcomes such as learning and test scores [55,66,76–78], thus meeting the psychometric criteria of reliability and validity [79]. Finally, at least one previous study has established that speed of forgetting, thus measured, reliably correlates with individual differences in EEG activity, thus establishing a connection between SoF and brain function [80].

Experimental predictions

To distinguish between the retrieval failure and storage degradation hypotheses, this study applied a statistical learning approach [81] to resting-state fMRI data from healthy participants. Specifically, a regularized linear model’s hyperparameter was tuned to minimize the cross-validated prediction error of an individual’s speed of forgetting from their resting-state connectivity. The specific patterns of connectivity that were identified to successfully predict individual differences in forgetting would shed light onto the biological nature of the phenomenon by indicating whether the involved networks align best with the retrieval or the storage degradation hypothesis.

Resting-state activity fMRI was chosen because of its spatial precision and its reliability within participants [82]. It has been speculated that brain activity at rest reflects the invariant functional architecture of the brain [30,83]; small variations within this architecture are characteristic to every individual, to the point that it can be used to “fingerprint” individual participants [84,85]. Consistent with this finding, individual differences in the resting-state functional connectivity between different regions have been shown to be predictive of performance differences in a variety of cognitive abilities, including working memory [86], executive functions [87,88], motor learning [89], perceptual discrimination [90], and, most importantly, recognition memory [91]. Thus, resting-state functional connectivity provides an ideal target to identify the biological underpinnings of individual differences in long-term memory retention and forgetting.

Specifically, we hypothesized that the speed of forgetting would reliably co-vary with the functional connectivity between different sub-networks of the DMN. This prediction is common to both hypotheses and follows from the central role played by the DMN in systems consolidation and episodic memory retrieval.

In addition, we also hypothesized that individual differences in the speed of forgetting will be reflected in the connectivity between the DMN and other networks. These patterns of connectivity would allow us to distinguish between the storage degradation and the retrieval failure hypotheses. If the storage degradation framework is correct, then we expect to see that individual differences in forgetting are best predicted by patterns of connectivity between the DMN, sensory, and associative cortical areas [49,74,92], including higher-level visual areas such as those associated with the ventral and dorsal visual attention networks.

If, on the other hand, the retrieval failure framework is correct, then we would expect to find that individual differences in forgetting are best predicted by the connectivity between the DMN and brain networks associated with frontal and parietal control regions.

Ethics statement

All of the recruitment and testing procedures were approved by the University of Washington’s Institutional Review Board and all participants provided written informed consent.

Participants

A total of 33 monolingual English-speaking participants (19 females) aged between 18–33 years old were recruited from a pool of University of Washington undergraduates for whom resting-state fMRI data had already been acquired in a previous, unrelated experiment between 6 and 12 months before the paired-associate learning task was administered. All participants provided written informed consent and were compensated with a $25 gift card for their participation in the paired-associate study.

Adaptive paired associate learning task

Individual speeds of forgetting were estimated using the adaptive paired-associate learning task described in [55] and illustrated in Fig 2. The stimuli consisted of 25 Swahili-English word pairs selected from a previous study [93,94]. The task dynamically interleaved study trials and test trials following a retrieval practice design [95]. On study trials, a Swahili word (e.g., “samaki”) and its corresponding English equivalent (“fish”) were presented simultaneously on the screen, and participants typed in the English word to proceed. On test trials, only the Swahili word was presented on the screen as a cue (e.g., “samaki”), and participants were again asked to respond by typing the corresponding English word (“fish”) in an empty textbox. Trials were self-paced, with no cap on the time allowed for a response. Consecutive trials are separated by a 600 ms ISI after a correct response and a participant paced ISI following an incorrect response during which corrective feedback was shown on screen. The order of repetitions and moment of introduction for each item were determined by the adaptive scheduling algorithm outlined below and described in more detail elsewhere [71,76]. The task lasted 12 minutes, during which participants learned as many of the paired associates as possible. In the allotted time, participants learned, on average, 23.3 different pairs (range: 10–25), across 142.7 trials (range: 91–221), with a mean accuracy of 86.5% on test trials (range: 62.4% - 98.9%).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Activation over time of a memory that has been encoded in three different episodes at times t = 1, 20, and 75 seconds.

Each episode leaves an independent memory trace (colored lines) with a characteristic decay rate. The total activation of the memory is the sum of the decaying traces (gray line).

https://doi.org/10.1371/journal.pcbi.1013485.g001

The scheduling algorithm was designed to optimally interleave the stimuli so that items would be repeated before the internal cognitive model predicted that they would be forgotten as determined by the continuously updated speed of forgetting (φ) for each item. In the model, each memory m is a paired associate (see [96]) that links a new Swahili word (“samaki”) to a known English word (“fish”). In response to a study probe, participants are given the Swahili word (“samaki”) as a cue to retrieve m (“samaki” - “fish”). Only when m is retrieved can participants type in the correct answer (“fish”). In turn, φ is estimated from the time T it takes participants to respond (first key press), which is related to each pair’s activation by the equation:

(4)

where A(m) is the current memory activation (Eq. 1), F is a scaling constant fixed to 1, and t₀ represents a fixed offset (300 ms) for non-memory related processes, such as the time spent for encoding the visual stimulus and making a motor response [68,97]. Thus, responses that are either incorrect or slower than expected result in upwards adjustments of φ, while responses that are faster than expected result in downwards adjustments of the parameters.

Although it might seem surprising that long-term memory forgetting can be assessed within such a relatively short period, this is a consequence of the fact that it follows a power law [75]: forgetting occurs most quickly in the first few minutes and then gradually slows down. For example, in a forgetting curve with an SoF value of 0.3 (which is a reasonable estimate for the average φ for undergraduate students; in this study the average is 0.305, see below), a memory’s retrieval probability falls < 25% within the first 10 mins, unless it is rehearsed. This means that the first minutes are an ideal interval to precisely estimate the speed of forgetting, as a significant part of the decline happens in this short timeframe.

rs-fMRI data collection and preprocessing

Each resting-state run lasted 7 minutes and consisted of 210 echo-planar images (EPI) collected with TR = 2,000 ms, TE = 25 ms, FA = 72 degrees. Each image in the series consisted of 36 oblique 3-mm bi-dimensional slices with no gap in-between, with an in-plane resolution of 80x80 3x3 mm voxels. Each functional series was preprocessed using the AFNI [98] software package. Specifically, each resting-state time series was de-spiked, corrected for differences in slice acquisition time, realigned to the mean image in the series, normalized to the MNI ICBM152 template, and spatially smoothed with a 3D 8-mm FWHM Gaussian filter. Artifacts induced by subject movement were removed by regressing out the six motion parameters and their temporal derivatives from each voxel’s time series. In addition to the functional data, a high-resolution anatomical MP-RAGE was also collected to aid in the normalization process.

Brain parcellation

To calculate functional connectivity, each participant’s brain was divided into discrete regions using the network parcellation originally proposed by Yeo et al [99]. Fig 3 depicts the 17 brain networks identified by Yeo, using the color scheme proposed in the original paper.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Visual illustration of one complete study trial and the onset of a test trial from the adaptive vocabulary learning task.

Note that study items are repeated based on an estimate of when their memory activation would drop below a predefined threshold.

https://doi.org/10.1371/journal.pcbi.1013485.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. The 17 networks in the Yeo et al (2011) parcellation.

Subcortical regions, not included in the parcellation, are marked in black.

https://doi.org/10.1371/journal.pcbi.1013485.g003

To test the storage degradation and retrieval failure hypotheses, each network was assigned to one of three possible groups:

• The Default Mode Network, consisting of Default A, B, and C as identified by Yeo et al. [99];
• The Retrieval group, consisting of networks including key prefrontal and parietal regions implicated in controlled and voluntary memory retrieval [34,35,37,39,50] as well as the ventral and medial prefrontal regions monitoring of the retrieval process [41–43]. These regions include the subnetworks Control A and B, Dorsal Attention A, Salience B, Limbic B, and Temporal Parietal in the Yeo parcellation.
• The Storage group, consisting of regions associated with the encoding of the memory’s sensory and affective features and its later consolidation of memory; thus, they encompass both perceptual and sensorimotor areas [100–106] and the insular and medial regions involved in assessing stimulus salience [107–109]. These regions include Yeo’s subnetworks Visual A and B, Somatomotor A and B, Dorsal Attention B, Salience A, Limbic A, and Control C.

As would be expected of functionally related regions, the retrieval and storage networks, thus defined, cluster into spatially contiguous portions of the brain, as illustrated in Fig 4.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. The grouping of the networks from the Yeo et al (2011) parcellation according to their involvement in consolidation (DMN, black) storage (blue) or retrieval (brown) processes.

https://doi.org/10.1371/journal.pcbi.1013485.g004

Connectome extraction

Each participant’s global pattern of functional connectivity, known as the connectome [110], was extracted using a standard procedure [111,112]. For each of the 17x17 pairs of regions in the parcellation, the Pearson correlation coefficient between the respective time series was computed.

One problem with this canonical connectome is that the raw correlation coefficients are almost all positive and partially driven by common, correlated factors. To remove these confounds, we followed the procedure of Cole and colleagues [111], and recalculated the correlation matrices using partial correlation coefficients, so that, when computing the correlation between two networks, the correlations between each network in the pair and the remaining 15 networks were partialled out. The partial correlation coefficients were the functional connectivity data that was fed into the statistical analysis (see below).

Statistical analysis

Functional connectivity data were analyzed with the Least Absolute Shrinkage and Selection Operator [LASSO: 113], a statistical learning method [114,115] that combines linear regression and feature selection in a single framework. Like linear regression, LASSO estimates a vector of regression weights β* that minimizes the squared sum of the differences between the observations y and their predicted values, that is argmin_β||y – βX||₂ (where the notation ||v||_n represents the Ln norm of the vector v). Unlike traditional regression, the difference is further penalized in proportion to the sum of the absolute values of the regressor weights (their L1 norm), so that:

(5)

The use of the L1 norm allows the least informative regressors to shrink to zero and, effectively, be dropped from the model—thus combining shrinkage and feature selection [113]. In turn, this reduces the complexity of the model, favoring generalizability and reducing overfitting. The hyperparameter λ represents the tradeoff between model simplicity (encouraged by the ||β||₁ penalty) and accuracy (encouraged by the least-squares minimization term ||y – βX||₂). As with all machine learning approaches, careful validation is needed to estimate the values of λ and to identify the proper predictors; in this project, Leave-One-Out cross-validation was used to select the proper value of λ by minimizing the prediction error over the out-of-sample observation; the result of this process, therefore, is a set of sparse regressors that have the greatest predictive weight. Once the best value of λ was identified, a final model was then constructed by refitting the entire dataset.

Speed of forgetting

One SoF parameter was calculated for each individual by averaging across the φ values of each word pair they studied and which was presented at least three times. The resulting values ranged from 0.253 to 0.385, with a mean of 0.305 and a standard deviation of 0.029 (Fig 5). The mean and the range of values are in line with previously published studies [55,80]; no difference was found between male and female participants [Male: 0.309 ± 0.029; Female: 0.304 ± 0.031; t(24.47) = –0.48, p > 0.63].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Distribution of the observed speeds of forgetting (N = 33).

Rug plot lines on the x-axis indicate values from each individual participant.

https://doi.org/10.1371/journal.pcbi.1013485.g005

It is worth examining how well the model, and especially its predicted SoF parameters, actually fit the participants’ data (unfortunately, the trial-level data from two of our participants have been lost, and could not be entered into this analysis). Because they responded correctly in > 86% of the trials (close to ceiling due to the adaptive nature of the task), we focused on the model’s ability to predict individual trial-level response times; additional analyses, including trial-level accuracies (S1 Fig), predicted mean individual accuracies (S2 Fig), trial-level response times (S3 Fig), and mean individual response times (S4 Fig), are provided in the Supporting Information (S1-S4 Figs). Specifically, we recalculated the predicted activation for each test trial of every participant, using Eq. 1 and each participant’s final SoF. Then, for each of the 3,354 trials, we computed the predicted response times using Eq. 4 and keeping its remaining parameters to the same default values (t₀ = 300 ms and F = 1) used during the task. Response times taking longer than 8 seconds were excluded as outliers; resulting in the removal of 155 trials (4.4% of the total). As it is common for computational models of behavior, we estimated the model’s fit by comparing the predicted and observed distributions of response times; in this case, we used the Kullback-Lieber (KL) divergence metric and the χ² test [116]. KL values can be interpreted, in information theory, as the mean number of bits lost when the observed response times are replaced by the model’s predictions, while the χ² values quantify how likely it is that the observed and predicted values are generated by the distribution; in both cases, values closer to zero are better. To compute both metrics, predicted and observed response times were divided into ten 800ms bins, and the number of predicted vs. observed values falling into each bin was compared. As shown in Fig 6, the model is successful at capturing individual differences in response times, producing remarkably matching distributions for each participant. The mean KL value was 1.61 across all participants and 0.35 across all trials from all participants. Similarly, the mean χ² value was 25.4 across participants and 42.0 across all trials, implying that the predicted and observed distributions of response times were not significantly different from each other (that is, p > 0.05). Taken together, the results of this analysis confirm not only the validity of the model from which the SoF values were derived, but also the choice of the default parameters used in the adaptive paired associate learning paradigm.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Predicted vs. Observed distributions of response times in the adaptive paired associate learning task.

The predicted values are derived from the memory model (Equations 1–4), using participant-specific values for the SoF parameter.

https://doi.org/10.1371/journal.pcbi.1013485.g006

Resulting connectome and LASSO model

Fig 7 shows the group-averages of the resulting individual connectome matrices. The left side shows a canonical distribution of r values, including the zones of maximum r values along the diagonal (corresponding to connectivity within the same networks). This pattern is entirely consistent with previous findings using the same parcellation scheme.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Mean raw Pearson correlations (left) and partial correlations (right) between each of the 17 networks across all participants.

The mean correlation coefficient across participants was calculated by first turning each individual coefficient into a z-value using Fisher’s r-to-z transformation, then averaging the individual z-values, and finally transforming the average z value back into an equivalent r coefficient. Network names are color-coded based on whether the network belongs to the Storage, Retrieval, or DMN group (Fig 4).

https://doi.org/10.1371/journal.pcbi.1013485.g007

The right side of Fig 7 shows the average of the partial correlation connectomes. As expected, this second matrix is sparser and includes both negative and positive correlations [111,117], giving a clearer picture of the underlying functional organization. The raw partial connectomes were the features entered into the LASSO model.

Although LASSO reduces the number of predictors, it has been shown that its performance is hindered when the number of regressors is much larger than the number of participants [118], which would be the case if we considered the full 17 x 16/2 = 136 possible connections between networks. Thus, to reduce the number of regressors, we limited the analysis to the connections between the three default mode networks (Default A, B, and C in Fig 1), including their connectivity with each other, and all of the other 14 networks. Thus, the total number of regressors included in this analysis was 3 x 14 + 3 = 45. Notice that this does not reduce the generality of our findings, since our theoretical hypotheses concern precisely the relationship between the default mode networks and other networks (associative vs. control networks).

Network connectivity predictive of individual speed of forgetting

The cross-validation procedure yielded a hyperparameter value of λ = 0.0044, which resulted in only 8 regressors representing network connections being selected: it was the individual variations in these regressors that were found to be most predictive of individual speeds of forgetting. Table 1 lists these eight network connections, while Fig 7 provides a visual breakdown of their connectivity, organized according to the different DMN subnetworks involved. All of the six networks whose connectivity to the DMN was identified as a reliable predictor of forgetting (Somatomotor A, Visual A, Salience A, Control C, Limbic A, and Dorsal Attention B: Table 1 and Fig 8) belong to the storage-related group (see Fig 4), suggesting that forgetting can be better understood within the storage degradation framework. Further analysis of these connectivity patterns, described below, confirmed this hypothesis.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. List of network connections predictive of individual SoF values, together with their β weights, in decreasing order.

https://doi.org/10.1371/journal.pcbi.1013485.t001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Functional network connectivity predictive of individual speeds of forgetting.

The connectivity values are organized according to the DMN subnetwork they refer to (in dark gray); The color of each network reflects the value of the corresponding β weight of its functional connectivity.

https://doi.org/10.1371/journal.pcbi.1013485.g008

The predictions from this model were then compared to the observed data. Fig 9A plots the predicted speeds of forgetting against the observed values estimated from the adaptive paired-associate learning task. The final model’s predictions have a mean correlation of r(33) = 0.77 (p < 0.001) with the observed data.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. (A) Observed vs. predicted SoF values across all 33 participants, using the eight regressors identified by the LASSO Model. (B) Leave One Out (LOO) validation of the model, where each participant’s SoF value is predicted from beta weights fit to the remaining 32 individuals. In both plots, each dot represents a single participant; the black dashed line is the identity line, while the blue line and ribbon represent the linear regression estimate and its standard error.

https://doi.org/10.1371/journal.pcbi.1013485.g009

Validation of the LASSO results

Given the relatively small sample size of this study, it is possible that the results of Fig 9A are partially due to overfitting and would not generalize. As a safeguard against this possibility, a Leave One Out cross-validation procedure was run on the final model. In this procedure, a prediction for each participant was generated by first fitting the beta weights of the eight regressors on the remaining 32 participants, and then by applying these weights to the specific connectivity values of the withheld participant. The results of this procedure are depicted in Fig 9B. As the panel shows, the model generalizes well to the held-out data, with the cross-validated predictions still having a significant correlation with the observed results [r(33) = 0.59, p < 0.001].

Because of its penalty term, LASSO might sometimes force the algorithm to select between collinear predictors at random, providing an inadequate and partial picture of which regressors are indeed essential [119]. If this were the case, other equally valid predictors could have been overlooked. A simple test to exclude this possibility is to remove the 8 connections from the original matrix of 45 regressors X, and re-run the LASSO procedure. If the procedure converges on a different set of features, then other important regressors were accidentally excluded. Alternatively, if no other regressors are left that can account for variability in the speed of forgetting, then it would be impossible for the cross-validation procedure to find a satisfying value of λ that allows for generalizable results. When applied to our data, this test shows that the cross-validation procedure does not converge anymore. Specifically, it selects a larger hyperparameter value (λ = 0.0103) under whose constraints all regressors are set to zero; under these conditions, the resulting linear model reduces to the intercept and fails to capture any individual differences in forgetting.

To ascertain whether the eight regressors are consistently linked to the SoF values, we also performed eight follow-up LASSO analyses, each time removing only one of the eight regressors from the list of features. The removal of the connection between the Default C and Visual A sub-networks (which has the largest weight in Table 1) resulted in the model failing to converge. In each of the other cases, the model still converged to a solution and each of these solutions included all of the remaining seven regressors, suggesting that they are specifically and consistently associated with SoF values. In three of these seven cases, LASSO compensated by including 1–3 additional connections to the list of regressors. Only one such connection was common to these augmented solutions: the connection Default B and Visual B. Like one of our regressors, this connection links a DMN region with a visual one (thus suggesting a similar role) and also belongs to the Storage network group.

Taken together, these tests confirm that the eight regressor values identified by the LASSO model are reliably and consistently predictive of individual differences in SoF. However, since the number of predictors in the model (p = 8) remains proportionally large when compared to the number of observations (n = 33), it is possible that the surviving regressors are highly collinear, which would suggest that all connectivity values are similarly related to the speed of forgetting. To exclude this hypothesis, the eight regressors in the model were examined using the Variance Inflation Factor (VIF) metric as implemented in R’s car package [120]. The VIF of a regressor is the ratio of the variance of its estimate (that is, its β weight) in the full model over its variance in a reduced model that includes only that regressor. As a rule of thumb, VIF factors > 10 indicate collinearity [121]. This analysis revealed that all of the predictors had a VIF value < 2.5, which suggests that they contributed independently to an individual speed of forgetting.

Finally, a series of permutation tests was also conducted to examine whether any distribution of SoF values could be generated by an appropriate mixing of the full set of 45 regressors. To exclude this possibility, the LASSO procedure was repeated after randomly shuffling the vector of SoF values 1,000 times. In most cases (56.3%), the LASSO procedure failed to converge altogether. While possible linear solutions were identified in the remaining 43.6% of the simulations, models that were comparable to our results in terms of fit to the data were found in only 2.9% of the cases. Thus, it is statistically unlikely (p < 0.029) that the relationship between the distribution of SoF values and the distributions of connectivity values is is coincidental.

Relative importance of DMN networks

Having confirmed the internal validity of the model, we can now further investigate the nature of its predictors. The connections in Table 1 can be thought of as weighted edges connecting nodes in a network, with the weights corresponding to their β values; therefore, they can be analyzed using network-level measures. In particular, it is possible to compute the relative contribution of each of the Yeo networks as the importance of the corresponding node, which is the L1 norm of all of its connections to other nodes. In this case, each node is one of the Yeo networks, and its importance is the sum of the absolute value of the regressors that connect it to other Yeo networks in Table 1. As a first step, we computed the importance of each of the three DMN subnetworks (Default A, B, and C in Fig 3). The results are shown in Fig 10A.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 10. (A) Relative importance of each DMN subnetwork, calculated as the L1 norm of its connections. (B) Relative importance of each non-DMN network, calculated as the L1 norm of its connections.

https://doi.org/10.1371/journal.pcbi.1013485.g010

Relative importance of storage vs. retrieval networks

Fig 10B illustrates the importance of each of the six non-DMN subnetworks networks identified by the LASSO model (Somatomotor A, Visual A, Salience A, Control C, Limbic A, and Dorsal Attention B). As noted above, all of them belong to the storage-related group. Even accounting for the fact that our classification contains more storage networks (eight, accounting for 24 connections to the DMN subnetworks) than retrieval networks (six, for a total of 18 connections), the probability that all eight regressors would involve only storage networks is p < 0.003.

If the storage degradation hypothesis is correct, we would expect that the storage-related networks of Fig 4 would not only be selected more frequently, but would also have significantly greater importance than the retrieval-related networks. This is indeed confirmed by Fig 11, which illustrates the distribution of the importance scores of all of the non-DMN networks. A Wilcoxon test further confirmed that, even when all of the non-DMN networks were included (using a regressor value of zero for those eliminated by LASSO), the mean importance of the storage networks (M = 0.021, SD = 0.019) was significantly greater than zero [t(7) = 3.179, p = 0.016] and, as a consequence, greater than the mean across the retrieval networks (which is zero).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 11. Connectivity importance of storage and retrieval networks to predict memory performance.

https://doi.org/10.1371/journal.pcbi.1013485.g011

Limitations

While these findings are encouraging, it is important to recognize some of their inherent limitations. Although the use of a computational model allows a finer-grained understanding of the underlying memory process, our estimates of individual speeds of forgetting are dependent on the model’s default parameters for perceptual and motor processing. Although the values of these parameters are approximately correct for the current population (as shown by the model’s ability to fit the data, as illustrated in the Fig 6 and in the Supplementary Materials, S1-S4 Figs), different estimates might be necessary to correctly measure memory in different populations.

Most importantly, the data comes from a small number of fairly homogeneous participants who studied a relatively small set of foreign vocabulary items, which potentially limits external validity. Thus, future studies should expand the sample size and explore other sets of study materials. The limited sample size also prevents us from entirely ruling out a role for retrieval failure. It is possible, for example, that the contribution of retrieval networks is significant and consistent but not visible in our limited sample. It is also possible that the relative contributions of storage degradation and retrieval failure are mediated by external factors, such as age and clinical conditions. Thus, storage degradation may be the main contributor to forgetting in healthy young adults (our sample’s population), but retrieval failure might play a greater role in elderly adults or in patients affected by dementia. Once more, future studies will be needed to further examine these alternatives.

A related limitation concerns the interpretability of the specific connectivity results in Fig 8. It is impossible to judge how much the connectivity between two networks affects forgetting, because its final effects depend on both the size of its beta value and the value of the correlation. Even more strikingly, it is impossible to judge whether a higher or lower beta value for a functional connection translates to higher or lower forgetting, because the effect depends on the direction of the original functional connectivity. For example, if two regions were anti-correlated but their functional connectivity had a positive weight, an increase in functional connectivity would decrease, instead of increase, the predicted speed of forgetting. In a previous study by Yang et al. [122], the authors worked around this problem by multiplying the beta weight of a connection by the sign of its mean, group-level connectivity value, so that the sign of the product would correctly reflect the direction of the effect and the absolute value would predict the true predictive strength. If applied to our data, this approach yields the results illustrated in Fig 12.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 12. Interpretable functional connectivity predictive of individual speeds of forgetting, organized according to the DMN subnetwork they refer to (in dark gray).

The color of each network reflects the value of the corresponding β weight multiplied by its mean functional connectivity; positive values are associated with increases (and negative values with decrease) in the Speed of Forgetting.

https://doi.org/10.1371/journal.pcbi.1013485.g012

The results in Fig 12 suggest that greater connectivity at rest between storage-related networks and the DMN are associated with increased forgetting—with the only exception of the connectivity between the salience network and the DMN. This observation runs against the notion that spontaneous brain activity at rest supports consolidation; in turn, this suggests two possible interpretations. The first is that, while a tonic amount of functional activity does support memory consolidation and maintenance, too much of it might result in greater interference during memory reactivation and replay.

The second interpretation is that greater resting-state activity between these networks might be associated with shallower encoding of the initial engram, thus setting the stage for faster forgetting. For example, stronger spontaneous activity in the background might interfere with the recruitment of neurons to create the initial representation ensemble. Indeed, it is possible that individual differences in the speed of forgetting might be ascribed to individual differences in the depth of encoding. Although the computational model used in this research can successfully track differences in retention curves, it is agnostic about their underlying causes, leaving open the possibility that differences in forgetting are due to encoding. This is also consistent with a recent study [123] in which the authors experimentally manipulated the levels of processings [124,125] associated with different items and tracked their recall at different intervals and found that shallower processing at encoding indeed resulted in faster-declining retention curves.

Although both hypotheses are worth pursuing in future research, we should note that any interpretation of our results relies on having reliable estimates of the sign of group-level mean functional connectivity. Thus, given the limited size of our sample and the intrinsic limits of interpreting functional connectivity [126,127], the results in Fig 12 should be taken with caution.

Considerations

These limitations notwithstanding, we believe it is important to stress four characteristics of our findings. First, they are based on a measure that directly quantifies forgetting, a latent and unobservable process, through computational phenotyping. This approach provides an advantage over simpler behavioral tests of memory performance. As noted in the introduction, forgetting is a process [3,7] and there is no consensus on how long after encoding a memory should be tested to precisely quantify retention. Since long-term memory retention spans from seconds to decades, it is difficult to test it in a laboratory setting. Consequently, previous neuropsychological studies have relied on individuals exhibiting dramatic differences in memory function, such as amnesic patients (e.g., patients suffering from medial temporal lobe injury [128–130]), while most laboratory experiments focus on relatively short retention intervals. Two participants, however, might have comparable accuracy-based performance a few minutes after studying but vastly different performances weeks later. In contrast, our findings rely on a novel metric, the speed of forgetting, that quantifies the rate at which memories are forgotten. This metric is better suited for capturing forgetting and retention at different intervals and, ultimately, for making inferences as to where forgetting might happen in the brain.

Second, it is worth noting that our findings could be applied in the opposite direction: to decode an individual’s speed of forgetting from distributed patterns of connectivity using a standard brain parcellation. Such an approach could be used, for example, to detect the early onset of abnormal forgetting in individuals with known memory pathologies and cognitive degenerative diseases. Most importantly, this finding suggests the possibility of decoding model parameters directly from resting state data, rather than from a combination of specific tasks. This procedure could play an important role in the development of high-fidelity idiographic models. This procedure also has the benefit of showing reflections of parameters in brain connectivity network regions, shedding light on a more holistic and comprehensive way of doing functional anatomy research on human cognition.

Third, our results are broadly consistent with the existing literature. As predicted, we did find that individual differences in forgetting covary with connectivity with and within the DMN. Not only have these sets of networks been theorized to play a role in memory replay and consolidation [10], but their connectivity with the salience network and the dorsal attention network (both of which were identified in our analysis) are altered in pathologies that negatively affect long-term memory function, such as Alzheimer’s Disease [12], mild traumatic brain injury [131] and depression [132]. Our findings are also consistent with previous studies that have experimentally tested the root causes of drug-induced amnesia [133,134], which similarly strongly favor the storage degradation hypothesis.

Finally, our results are also consistent with the findings of a previous study that previously attempted to identify the neural underpinnings of forgetting using the same model-based approach but with a different neuroimaging modality—EEG instead of fMRI [80]. In that study, a significant positive correlation between spectral power in the beta band (12–20 Hz) and the speed of forgetting was observed distributing at middle frontal (AF3 and AF4), bilateral posterior (channel O1, O2), and parietal (channel P8) regions; these spatial locations are consistent with the cortical regions of the DMN subnetworks A and B (see Figs 10 and 11).

Conclusions

In summary, although further research will be needed to clarify the specific mechanisms of forgetting, our findings do provide a preliminary answer to the question of where memories are lost—that is, in the connections between the DMN and the sensory networks that are involved in the encoding, re-experience, and consolidation of a memory. Thus, our results imply that forgetting can be interpreted as a storage degradation process, rather than a retrieval access problem, and that future clinical and neuroscientific research should focus on the storage networks as a target to better understand, improve, or support memory function.