https://orcid.org/0000-0001-8380-2866
Figures
Abstract
Even though there is a clear link between Alzheimer’s Disease (AD) related neuropathology and cognitive decline, numerous studies have observed that healthy cognition can exist in the presence of extensive AD pathology, a phenomenon sometimes called Cognitive Resilience (CR). To better understand and study CR, we develop the Alzheimer’s Disease Cognitive Resilience Score (AD-CR Score), which we define as the difference between the observed and expected cognition given the observed level of AD pathology. Unlike other definitions of CR, our AD-CR Score is a fully non-parametric, stand-alone, individual-level quantification of CR that is derived independently of other factors or proxy variables. Using data from two ongoing, longitudinal cohort studies of aging, the Religious Orders Study (ROS) and the Rush Memory and Aging Project (MAP), we validate our AD-CR Score by showing strong associations with known factors related to CR such as baseline and longitudinal cognition, non AD-related pathology, education, personality, APOE, parkinsonism, depression, and life activities. Even though the proposed AD-CR Score cannot be directly calculated during an individual’s lifetime because it uses postmortem pathology, we also develop a machine learning framework that achieves promising results in terms of predicting whether an individual will have an extremely high or low AD-CR Score using only measures available during the lifetime. Given this, our AD-CR Score can be used for further investigations into mechanisms of CR, and potentially for subject stratification prior to clinical trials of personalized therapies.
Citation: Yao T, Sweeney E, Nagorski J, Shulman JM, Allen GI (2020) Quantifying cognitive resilience in Alzheimer’s Disease: The Alzheimer’s Disease Cognitive Resilience Score. PLoS ONE 15(11): e0241707. https://doi.org/10.1371/journal.pone.0241707
Editor: Stephen D. Ginsberg, Nathan S Kline Institute, UNITED STATES
Received: March 13, 2020; Accepted: October 19, 2020; Published: November 5, 2020
Copyright: © 2020 Yao et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: In order to protect research subject confidentiality, and in compliance with Rush University Medical Center Institutional Review Board, HIPAA, and NIH guidelines, a signed Data Use Agreement is required for distribution and use of clinicopathologic data from the Religious Orders Study (ROS) and Rush Memory and Aging Project (MAP). All data are available for request. The data underlying the results presented in the study are available from the Rush Alzheimer’s Disease Center (RADC) Research Resource Sharing Hub (https://www.radc.rush.edu/). Additional details of the RADC data sharing policies and the data request form can also be found at this site (https://www.radc.rush.edu/requests/data.htm).
Funding: T.Y. and G.I.A. acknowledge support from NSF DMS-1264058, NSF DMS-1554821, and NSF NeuroNex-1707400. E.S. acknowledges support from Rice Academy Postdoctoral Fellowship. J.N. was supported in this work by the NIH Training Award in Biostatistics for Cancer Research Award 5T32CA09652010. J.M.S. was additionally supported by NIH/NIA grants (R01AG053960, R01AG050631, R01AG057339, U01AG046161, U01AG061357), Huffington Foundation, Jan and Dan Duncan Neurological Research Institute at Texas Children’s Hospital, and a Career Award for Medical Scientists from the Burroughs Wellcome Fund. Study data were provided by the Rush Alzheimer’s Disease Center, Rush University Medical Center, Chicago. Data collection was supported through funding by NIA grants P30AG10161, R01AG15819, R01AG17917, R01AG30146, R01AG36836, U01AG32984, U01AG46152, the Illinois Department of Public Health, and the Translational Genomics Research Institute. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Alzheimer’s Disease (AD) is a debilitating, irreversible, and progressive brain disorder that destroys memory and cognitive skills. AD is a major public health concern, as it is the sixth leading cause of death in the United States and is the only cause of death in the top ten that cannot be prevented, cured, or slowed. A definitive diagnosis of AD requires the presence of AD pathology at autopsy (amyloid plaques and neurofibrillary tangles in the brain) in conjunction with cognitive symptoms observed during the lifetime [1]. Numerous studies show a link between AD-related pathology and cognitive decline, yet it has also been observed that healthy cognition can exist in the presence of extensive AD pathology [2]. In fact, only about 40% of the variation in late life cognition can be explain by pathology [3]. Consistently in community based cohorts with autopsy, around a third of people without dementia have the pathology meeting the criteria for intermediate or even high likelihood of AD [4–7]. A model for explaining this anomaly is the concept of cognitive resilience (CR). In [8], CR in AD is defined as the mechanism which enables some individuals to be more resilient to the pathological brain changes associated with AD than others; individuals with high CR show few or no clinical symptoms of AD during their lifetime, but have a pathological diagnosis of AD at autopsy. Understanding and characterizing CR could transform the way that AD is viewed from both a public health and scientific perspective, leading to population-level and targeted interventions as well as disease-modifying treatments and therapies to prevent, or slow progression of disease.
CR is an abstract concept with no consensus definition and therefore has been quantified using numerous approaches in the literature. Associations between levels of CR and a number of characteristics have been found in the literature, such as years of education, gender, social connectedness, and personality traits. One of the simplest and most frequently used methods to define CR is using one or more of these proxy variables [2, 9, 10]. In [11], a composite global enrichment score based on education, occupation and leisure activities is used as a surrogate for CR. Defining CR through the use of proxy variables has been cautioned against in the literature [12], as many of these proxy variables are highly correlated with other variables (such as socioeconomic status and health behaviors) already known to be associated with AD risk. In addition, many of the proxy variables do not reflect the same life experience among different individuals [9]. For example, one year of education could be very different among two different individuals.
CR has also been characterized using in-vivo structural magnetic resonance imaging (MRI) of the brain as a proxy for AD-related pathology. In [13], baseline episodic memory is decomposed into three components with a latent variable model: brain pathology (as characterized from measures derived from in-vivo structural MRI), demographic variables, and a latent person-specific factor that captures difference from the baseline test performance predicted for an average person with similar brain pathology and demographics. This latent factor defines CR and is extended to longitudinal settings in [14]. These methods rely heavily on parametric models, which may be prone to model misspecification, and do not produce stand-alone metrics of CR. And while measures derived from in-vivo structural MRI are available during the lifetime (unlike AD-related pathology measures), these measures have not been validated with ‘gold-standard’ postmortem pathology of AD and may not actually be related to brain pathology.
A number of studies have characterized AD-related CR by explaining the variation in cognition that remains after adjusting for AD-related postmortem pathology. In [15] and [16], CR is quantified by regressing a global cognition score on a global postmortem pathology score and an interaction between the pathology score and another characteristic (for example years of education). The interaction term tests the hypothesis that the characteristic modifies the effect of pathology on cognition, and is therefore associated with CR. In [17], a longitudinal global cognition score is modeled using a random effects mixture model, adjusting for common AD associated pathologies. After adjusting for these pathologies, four distinct groups of cognitive decline are found. Among the four groups, statistically significant differences are observed in depressive symptoms, measures of social isolation, and measures of cognitive and physical engagement activities. These residual-cognition methods have allowed for important discoveries of a number of important associations with CR, including experiential, psychological, and genetic findings [18, 19]. Yet, characterizing CR as the variation in cognition that remains after adjusting for AD-related pathology is not ideal, as these characterizations also use parametric models, which may be incorrectly specified, and do not produce stand-alone metrics of CR. Further, incorporating adjustment for AD pathologic burden in estimates of CR may obscure specific mechanisms conferring resilience to AD pathology, which may not necessarily be generalizable to other brain lesions. In addition, the characterizations using postmortem pathology are not available during an individual’s lifetime, and therefore cannot be used in clinical settings.
Here we propose an alternative framework for defining CR, the Alzheimer’s Disease Cognitive Resilience Score (AD-CR Score). Given AD-related pathology, the AD-CR Score is the difference between the observed and expected cognition. The AD-CR Score is closest to residual-cognition methods of characterizing CR, as it is computed using a global measure of cognition obtained proximal to death and ‘gold standard’ postmortem AD pathology. Yet unlike the residual-cognition methods, the AD-CR Score is fully non-parametric, stand-alone, and produces individual-level quantifications of CR that are derived independently of other measures. To show that the AD-CR Score is measuring CR, we validate the score by exploring its association with measures already known to be associated with CR, as well as longitudinal cognition. And while the AD-CR Score is computed using ‘gold-standard’ postmortem pathology of AD, we also propose a framework for predicting whether an individual will have an extremely high or low AD-CR Score using measures collected during premortem assessments, providing potential clinical utility to the score.
Methods
Participants
Data from two ongoing, longitudinal cohort studies of aging, the Religious Orders Study (ROS) and the Rush Memory and Aging Project (MAP) [18–20], is used for this analysis. The two studies were designed and are managed by the same team of investigators that collect the same measures, making the studies ideal to be combined for analysis. All participants are free of dementia when enrolled into the studies and agree to annual clinical evaluations and brain donation upon death. Both studies are approved by the Institutional Review Board of Rush University Medical Center.
At the time of this analysis, 3190 participants had completed the baseline evaluation and 1378 participants were deceased with completed brain autopsy. Clinical diagnoses were made following National Institute of Neurological and Communicative Disorders and Stroke-Alzheimer’s Disease and Related Disorders Association recommendations [21], and including AD, mild cognitive impairment (MCI), or no cognitive impairment (NCI). Such clinical diagnoses were made blind of all postmortem data. During autopsy, a modified NIA-Reagan score of AD pathology presenting as neurofibrillary tangles and neuritic plaques in the brain was also assigned [22]. This assessment is performed without knowledge of clinical information (e.g. a clinical diagnosis of dementia) and is a postmortem diagnosis of Alzheimer’s Disease based entirely on the neuropathology. Specifically, the modified NIA-Reagan score was dichotomized into low probability for having a pathologic diagnosis of AD and high probability for having a pathologic diagnosis of AD based entirely on the postmortem neuropathologic evaluation. Low probability means no or low likelihood of AD based on the postmortem neuropathologic evaluation. High probability means high or intermediate likelihood of AD based on the postmortem neuropathologic evaluation.
Of the 1378 participants who had died and gone to autopsy, 24 were excluded from this analysis because they were diagnosed with non-AD dementia at death, and 46 were missing either a clinical diagnosis at death or a modified NIA-Reagan score. Furthermore, the computation of our proposed AD-CR Score requires a global measure of cognition proximal to death and a global measure of AD pathology burden (described in the Neuropsychological performance testing section and the Neuropathologic evaluation section, respectively). Therefore, seven participants were excluded for not having the global AD pathology measure at autopsy. Similarly, we also excluded 319 participants who did not have the full battery of cognitive tests used to calculate the global cognition measure within two years of death. After applying this exclusion criteria, 980 subjects were used in the analysis.
We show a demographic summary for the 980 participants used in our analysis in Table 1 in terms of the premortem clinical diagnosis (i.e. AD, MCI, and NCI) as well as the postmortem pathologic diagnosis (i.e. the modified NIA-Reagan score). Nearly 15% of the participants have NCI from premortem clinical diagnosis with a high probability for having a pathologic diagnosis of AD based on postmortem neuropathologic evaluation. In other words, even though these participants actually have NCI before death, their postmortem neuropathologic evaluations show extensive AD-related pathology that would fulfill criteria for having a pathologic diagnosis of AD. These participants would be among those considered to have ‘high cognitive resilience (CR)’. In addition, over 18% of the participants have either MCI or AD from premortem clinical diagnosis despite a low probability for having a pathologic diagnosis of AD based on postmortem neuropathologic evaluation. These participants would be among those considered to be ‘cognitively vulnerable (CV)’. We want to note that Table 1 is only meant to provide a demographic summary for the participants and the actual definition/computation of our proposed AD-CR Score does not depend on the categorizations in Table 1 at all.
Neuropsychological performance testing
A battery of 21 cognitive tests are administered annually and nineteen of them are used to assess a variety of cognitive abilities across five cognitive domains (episodic memory, semantic memory, working memory, perceptual speed, and visuospatial ability) [20]. The specific tests used for each domain are summarized as follows: i) Logical Memory Ia, Logical Memory IIa, immediate story recall, delayed story recall, Word List Memory, Word List Recall, Word List Recognition for assessing episodic memory; ii) Boston Naming Test, Category Fluency (fruits, animals), National Adult Reading Test for assessing semantic memory; iii) Digit Span Forward, Digit Span Backward, Digit Ordering for assessing working memory; iv) Symbol Digit Modalities Test, Number Comparison, Stroop word reading, Stroop color naming for assessing perceptual speed; and v) Judgment of Line Orientation, Standard Progressive Matrices for assessing visuospatial ability [23]. A global measure of cognition is computed by converting each test to a z-score and then averaging the z-scores, as previously described in [24, 25]. Therefore, the global cognition measure is a summary measure of a subject’s overall cognitive abilities across five cognitive domains on a continuous scale—negative values for the global cognition measure indicate lower overall cognitive abilities than the average of the entire cohort whereas positive values signify higher cognitive abilities than the average of the cohort. This global cognition measure is used in the computation of our proposed AD-CR Score in our analysis.
Neuropathologic evaluation
Following the procedures recommended by the National Alzheimer’s Disease Coordinating Center [26], the postmortem neuropathologic evaluation includes assessment of AD pathology, cerebral infarcts, lewy body disease, and other pathologies common in aging and dementia. In addition to the modified NIA-Reagan score, a global measure of AD pathology burden is constructed using three AD pathologies (neuritic plaques, diffuse plaques, and neurofibrillary tangles) from 5 regions of the brain (midfrontal cortex, midtemporal cortex, inferior parietal cortex, entorhinal cortex, and hippocampus), as previously described in [27]. This global measure of AD pathology burden is used in the computation of our proposed AD-CR Score in our analysis. As we are also interested in other, non-AD causes of dementia and cognitive decline, we accounted for the presence of Lewy bodies [28], the presence of hippocampal sclerosis [29], as well as measures of vascular infarcts: gross infarcts, microinfarcts, gross chronic infarcts, and chronic microinfarcts [30–33].
Other measures
For this analysis, we investigate measures that may be correlated with CR and the AD-CR Score. We group these measures into the following categories to test association: demographic, cognition, non-AD pathology, education, personality, vision, APOE, pain, alcohol, smoking, BMI, comorbidity, parkinsonism, depression, life activities, and physical activities. The demographic group includes sex and age. The cognition group includes the summary measures of the five cognitive domains described in the Neuropsychological performance testing section (episodic memory, semantic memory, working memory, perceptual speed, and visuospatial ability). The non-AD pathology group consists of the non-AD pathology measures introduced in the Neuropathologic evaluation section, namely the presence of Lewy bodies, hippocampal sclerosis, gross infarcts, microinfarcts, gross chronic infarcts, and chronic microinfarcts. The education group is the number of years of education. The personality group includes of a measure of neuroticism [34] and a measure of anxiety [35, 36]. The vision group includes a test of visual acuity. The APOE genotype group includes an indicator of whether the individual has allele E2 as well as allele E4. The former has been shown to be protective for AD while the latter has been shown to increase an individual’s risk of AD [37, 38]. The pain group includes the measures of self-reported pain in the upper and lower extremities. The alcohol group includes self reported measures of the average grams of alcohol consumed per day in the last 12 months and the average number of alcoholic drinks consumed per day over the individual’s lifetime. The smoking group includes an indicator of being a former smoker or a current smoker. The BMI group is a measure of the individual’s BMI. The comorbidity group includes any history of hypertension, cancer, diabetes, head injury with loss of consciousness, thyroid disease, heart disease, and stroke. The parkinsonism group includes measures of bradykinesia, gait quality, rigidity and tremor all modified from the United Parkinson’s Disease Rating Scale (mUPDRS) [39] as well as an overall diagnosis of parkinsonism from a trained nurse based upon the mUPDRS scale [39]. The depression group is an assessment of depression using a modified Center for Epidemiologic Studies Depression (CESD) scale [40]. The life activities groups consists of a measure of the ability to perform instrumental activities of daily living [41], a measure of the ability to perform basic activities of living [42], and a measure of mobility disability [43]. The physical activities groups includes measures of general physical activity, swimming physical activity, walking physical activity, and physical activity spent doing yard work. Table 2 summarizes the measures in each of the groups.
Statistical methods
We validate the AD-CR Score as a measure of CR by (1) demonstrating the score’s association with measures already known to be associated with CR and (2) demonstrating the association of the score with longitudinal global cognition trajectories. In addition, we also build a machine learning framework to predict whether an individual will have a high or low AD-CR Score using only clinical measures available during the individual’s lifetime.
AD-CR Score associations.
We validate the proposed AD-CR Score as a measure of CR by establishing the associations of the score with measures already known to be associated with CR in the literature. Individuals with higher cognition at baseline [8], higher educational attainment [2, 8, 15], less neuroticism [44], less depressive symptoms [44, 45], less disability in daily life activities [44, 46] and more engagement in physical activities [46] have been previously found to have higher CR. Having the APOE E4 allele has been found to increase the risk of AD, while the APOE E2 allele has been shown to be protective for AD [37, 38]. Non-AD pathology [2] and other comorbidities, especially Parkinson’s disease [47], may also explain cognition that is less than expected in relation to AD related pathology. Alcohol consumption and smoking have not been found to be associated with CR [48], so we also investigate these associations to confirm the previous findings. In addition, we explore the relationship with CR and other available measures for the participants (anxiety, vision, lower and upper extremity pain, and BMI) in an effort to find new associations with the AD-CR Score and CR. To reduce the number of tests performed, we grouped these measures into 16 groups as described in the section Other measures: demographic, cognition, non-AD pathology, education, personality, vision, APOE, pain, alcohol, smoking, BMI, comorbidity, parkinsonism, depression, life activities, and physical activities (Table 2).
For this analysis, we first use the baseline value (collected at the participant’s first evaluation) for each of the measures as well as the non-AD pathology measures collected at autopsy. We perform a univariate nested ANOVA (or F-test) to test whether the model with the demographic group has a significantly better fit compared to the null model (in terms of the variance explained in the AD-CR Score). We then use the univariate nested ANOVA to determine if adding each of the groups from Table 2 individually fits significantly better than the demographic group model alone. Next we performed a multivariate nested ANOVA (or F-test), testing whether a model fit with all groups fits significantly better than each of the models fit after removing one of the groups. Note that the demographic group is excluded from such removal and is thus always included in all multivariate models. Additionally, as baseline cognition and pathology are highly associated with CR and many of the other measures of interest, we fit another set of multivariate nested ANOVA excluding the cognition and pathology groups from removal, which always include these two groups in addition to the demographic group in all multivariate models. Participants with missing measurements were excluded from models for which there was missingness. To adjust our inference for multiple testing, we performed multiple-comparison correction within the nested univariate ANOVA and the two nested multivariate ANOVA settings by controlling the false discovery rate at 10% [49].
Acknowledging that participants in the ROS and MAP studies were recruited into these studies at different ages, we also perform the univariate and multivariate ANOVAs using measures from participants in cross-sectional models at age 75, 80, 85, and 90, as well as the pathology measures collected at autopsy. This analysis is done in an attempt to temporally register the subjects using age. The number of participants in these groups was significantly smaller than the full study population, n = 198, 353, 413, and 284 respectively, with missingness of 1%, 2%, 3%, and 4% respectively. We therefore imputed the data with nonparametric missing value imputation using a random forest implemented in the missForest R package [50, 51]. The sensitivity analysis for this imputation can be found in the S2 Appendix. For all of the nested ANOVA models we use only sex from the demographic group; adjusting for age is not necessary as all participants in the models are the same age. Just as with the nested ANOVA models with the baseline data, we also adjust for multiple testing using the false discovery rate.
Association with cognitive decline.
In addition to demonstrating the cross sectional associations with the AD-CR Score, we also examine the association of the AD-CR Score with the longitudinal trajectories of global cognition to provide validation for the score. Previous studies have identified a relationship between cognitive decline and CR [17]; those individuals with higher CR experience a slower rate of decline in cognition than those with lower CR, who experience a steeper drop in cognition over time. To explore the association between the AD-CR Score and cognitive decline, we consider a non-linear longitudinal mixed effects model, using splines to fit non-linear longitudinal effects [52]. In the model, we include the fixed effects of age at baseline, AD-CR Score, an interaction of age at baseline and AD-CR Score, a quadratic B-spline basis of age (henceforth referred to as the non-linear age term), and an interaction between the non-linear age term and the AD-CR Score, as well as a random effect for each subject. The baseline age and AD-CR Score fixed-effects in the model allow us to test for an overall shift in the cognition trajectory based upon age of entry into the study and CR. We include a fixed effect interaction between the AD-CR Score and the baseline age to test if the AD-CR Score modifies the relationship between the age at baseline and the longitudinal trajectories. The non-linear age term is used to capture the expected overall decline in cognition as an individual ages. In addition, we include an interaction with the AD-CR Score and this non-linear age term to test if the AD-CR Score modifies this decline. The random subject effect allows for individual differences in cognition among the subjects.
Prediction of the AD-CR Score.
To assess the clinical utility of the AD-CR Score, we seek to predict the AD-CR Score and specifically whether subjects will have high or low AD-CR Score using only measures that are available at the baseline visit (we exclude pathology as this is only available after death). First, we randomly assign 60% of the 791 participants with complete data to a training set (490) and assign the remaining participants to a test set (301). We focus on predicting subjects that will have extreme AD-CR Scores as these are the most clinically relevant groups. Thus, we employ a two stage predictive modeling process: First, we use a predictive model to predict subjects as having extreme or average AD-CR Scores; and second, we build separate predictive models for each group to classify subjects as having high or low AD-CR Scores (defined as positive or negative AD-CR Scores respectively). For the first stage, we fit weighted regression models to predict the continuous AD-CR Score, where subjects with extreme AD-CR Scores are up-weighted. We implement regression models such as regularized regression (R package glmnet), random forests (R package randomForest), and gradient boosting (R package mboost and XGBoost). All models are trained using the training set. In particular, we use 10-fold cross-validation to choose the optimal tuning parameters, the optimal weighting scheme, as well as the best overall model using the training set. After the first stage, subjects with an absolute predicted AD-CR Score greater than 0.1 are placed into the extreme AD-CR group and the rest are placed into the average AD-CR group. For the second stage, we fit separate models to the extreme and average AD-CR groups to further classify subjects as having high or low AD-CR Scores. For this, we implement weighted classification models such as regularized logistic regression, random forests, support vector machines (Python toolbox scikit-learn), and gradient boosting; again, we use 10-fold cross-validation to choose tuning parameters and decide on the best model using the training set. The predictive performance of our machine learning framework is reported on the test set in terms of Sensitivity, Specificity, F-Score as well as accuracy for predicting whether subjects will have high or low AD-CR Scores.
Results
The Alzheimer’s Disease Cognitive Resilience Score
Numerous studies have shown a link between AD-related pathology and cognitive decline [2]. Our objective is to develop a quantitative definition of CR in AD that is stand-alone, non-parametric, and produces individual quantifications independent of other measures. To this end, we first characterize the expected level of cognition given an observed level of AD pathology. The AD-CR Score is then defined as the difference between the observed and expected cognition for a given AD pathology burden. Fig 1A shows a diagram of this definition. Our AD-CR Score is computed using the global cognition measure and the global AD pathology measure (described in the Neuropsychological performance testing and Neuropathologic evaluation sections respectively).
(A) Diagram of the AD-CR Score definition. (B) Plot of the observed global cognition at last visit versus AD pathology burden of the participants where (B1) are the original values and (B2) are the quantile transformed values. A participant with a AD-CR Score of 0.55 is shown as the circled blue point. The geometric interpretation of the AD-CR Score (as the normalized shortest distance from an individual’s observed cognition and pathology to the expected values of cognition and pathology) is also shown in the plot with the red lines. Colors for the plots are determined as follows: AD and MCI (AD and MCI with high AD pathology), CR (NCI with high AD pathology), CV (AD and MCI with low AD pathology) and normal (NCI with low AD pathology).
To mathematically define our AD-CR Score, let c be the observed global cognition at last visit and let p be the observed global AD pathology. We first quantile transform the global cognition and global AD pathology: Let and
be the empirical distribution functions for c and p respectively. The quantile transformed values of global cognition and pathology are
and
. Fig 1B shows plots of the observed and quantile transformed global cognition at last visit versus the AD pathology for all participants in the analysis. We then define the expected level of cognition given the level of AD pathology to be the line at which the two quantiles are equal,
, denoted as the red forty-five degree line in Fig 1B2. Quantile matching provides a non-parametric way to characterize the expected cognition given the level of AD pathology. Finally, we define the AD-CR Score for participant i with global cognition score of ci and global AD pathology burden of pi as:
In other words, the AD-CR Score for each individual is the difference between their quantile-transformed cognition and their quantile-transformed AD pathology level. Hence, the AD-CR Score is a continuous measure, taking values between −1 and 1. Positive values indicate that there is higher cognition than expected, negative values indicate that there is lower cognition than expected, and values near 0 indicate that the cognition is as expected for the observed pathology. Our AD-CR Score can also be interpreted as the normalized shortest distance from an individual’s observed cognition and pathology to the expected values of cognition and pathology; a proof of the mathematical equivalence of the two definitions is provided in the S1 Appendix. Fig 1B illustrates how our AD-CR Score is calculated by highlighting a participant with a AD-CR Score of 0.55 shown on both the observed and quantile transformed plots.
Cross sectional associations
Next, we seek to validate our AD-CR Score by testing its association to previously implicated indicators of cognitive resilience in AD. Table 3 shows cross-sectional association results from the three baseline nested ANOVA models: the univariate ANOVA, the multivariate ANOVA, and the multivariate ANOVA excluding cognition and pathology. Note that we do not test the demographics group in the multivariate models as we adjust for age in all of the multivariate models. After correcting for multiple testing, we found associations with the AD-CR Score and demographic information, baseline cognition, non-AD related pathology, education, personality, APOE genotype, parkinsonism, depression, and life activities in the univariate ANOVA. These findings replicate previous findings in the literature [2, 8, 37, 38, 47]. In the univariate ANOVA, the only finding that we were not able to replicate from the literature was physical activity; this is perhaps due to how these measures were obtained in the ROS and MAP cohorts. In the multivariate ANOVA, cognition, non-AD pathology, and APOE were found to be associated with the AD-CR Score. As baseline cognition and pathology are highly associated with CR and many of the other measures of interest, we also fit a multivariate ANOVA model excluding cognition and pathology. In this model, APOE genotype, parkinsonism, and life activities were found to be associated with the AD-CR Score.
Table 4 shows the coefficient estimates from the baseline nested ANOVA models. Almost all of the coefficient estimates from the ANOVA regressions are in the same direction of already established associations of CR in the literature. Notably however, the APOE allele associations with the AD-CR Score are opposite to the well established associations between APOE genotype and AD risk. The APOE 2 allele is known to be protective for AD, while the APOE 4 allele increases the risk of AD [37, 38], and these alleles show consistent associations with AD pathologic burden [53, 54]. We observe that the APOE 2 allele is negatively associated with the AD-CR Score and the APOE 4 allele is positively associated with the AD-CR Score. In further investigation, the univariate regressions of the AD-CR Score on the APOE 2 allele and the APOE 4 allele maintain the same direction of the association (coefficient estimates of -0.061 and 0.121 respectively with p-values of 0.024 and < 0.001). The reason that the association for the AD-CR Score and APOE are in opposite directions likely relates to its strong association with AD pathologic burden, and how we have defined the AD-CR Score. A substantial body of literature supports that the association of APOE genotype with cognition is mediated by an increase or decrease in the level of AD pathology in the brain [27, 54]. Because APOE genotype impacts cognition through AD pathology and the AD-CR Score is intended to capture resilience in cognitive manifestations conditioning on a given level of AD pathology, it is not surprising that APOE does not show a strong positive correlation with AD-CR Score. In addition, the AD-CR Score is defined as the difference in the observed and expected cognition given the observed global AD pathology burden. Therefore, a positive AD-CR Score requires the presence of some AD pathology burden. And this is reasonable for modeling cognitive resilience because Alzheimer’s Disease cognitive resilience describes the phenomenon that healthy cognition can still exist in the presence of AD pathology. Because the APOE 2 likely protects against development of AD pathology in the first place, individuals with the APOE 2 allele are less likely to have AD pathology and therefore will not have a high positive AD-CR Score. And this agrees with the literature that APOE likely affects the development of pathology rather than cognitive resilience in the presence of pathology [27]. In contrast those with the APOE 4 allele will have more AD pathology burden and will therefore be more likely to have a positive AD-CR Score.
In addition to fitting the nested ANOVA models using the baseline visit for each participant, we also fit cross sectional nested ANOVA models at age 75, 80, 85, and 90. The results from these models are shown in Fig 2 and indicate that different factors are associated with CR at different ages. Note that we do not test the sex group in the multivariate models as we adjust for sex in all of the models. An overall pattern in all of the models is that there are more significant results in the younger age groups (75 and 80) than the older groups (85 and 90). This is suggestive of the fact that there may be irreversible AD-related disease burden that cannot be modified by behaviors and other factors at these older ages. In addition, we see in the univariate model that non-AD pathology is more significant for the older groups. This indicates that non-AD pathology may account for CV, especially as participants age. The results from the cross sectional ANOVA models are similar to those in the baseline model and also validated those findings already established in the literature. In both the univariate and multivariate ANOVA models, cognition is found to be significant at all time points. The APOE genotype was also found to be significant at all time points in the multivariate ANOVA, and at ages 75 and 80 in the univariate and multivariate ANOVA that excluded cognition and pathology. Life activities were found to be significant at all time points in the univariate ANOVA. Non-AD pathology, education, personality, and parkinsonism were also found to be significant at different time point in the three models.
The FDR corrected p-values are shown up to values of 0.20. We see that different measures are associated with an individual’s AD-CR Score at different time points in their lifetime and that there are more associations found in the younger age groups (75 to 80). Sex in the multivariate ANOVA model is shown in gray as all models are adjusted for sex and we do not test for an association of sex and AD-CR Score in these models. Cognition and non-AD pathology are also shown in gray as these are excluded from the multivariate ANOVA that excludes cognition and pathology.
Longitudinal cognition
Fig 3A shows the longitudinal global cognition for the participants versus age. The participants are grouped by AD-CR Score quintiles (Lowest, Low, Medium, High and Highest) and are smoothed using a non-linear age term within each of the quintile groups (also shown on the plot with 95% confidence bands). While all groups follow an overall decline trajectory in cognition over time, we see that participants with high AD-CR Score (those in the Highest and High quintile groups) have a much slower decline trajectory in global cognition compared to those in the lower quintile groups (those in the Low and Lowest quintile groups). This relationship has been observed in previous studies in the literature [17]; individuals with higher CR experience a slower rate of decline in cognition than those with lower CR. In addition we see that the global cognition score increases with age for those in the highest quintile of AD-CR Scores. This may be due to the fact that there are very few participants in the study who live to be over age 100; the estimates for the global cognition in these ages is less accurate than at other ages, as evident from the wider confidence bands.
(A) Longitudinal global cognition trajectories grouped by AD-CR Score quintile group. For each of the quintile groups a smoothed curve with 95% confidence band is shown. (B) The predicted longitudinal cognition trajectories for the AD-CR Score quintile groups from the fitted mixed effects model. The plot is divided into three panels, one panel each for baseline age of 70, 80, and 90 with 10 years of predictions for each of the panels.
In the longitudinal smoothing spline ANOVA model, all fixed effects were found to be statistically significant: baseline age (p < .001), the AD-CR Score (p < .001), the interaction between baseline age and AD-CR Score (p < .001), the non-linear age term (p < .001) and the interaction with AD-CR Score and the non-linear age term (p < .001). To illustrate the results from the model, we plot the predicted non-linear global cognitive decline trajectories from the model in Fig 3B. Each of the predicted curves in the figure represents the prediction of the cognitive trajectory for 10 years for each of the quintile group cutoffs of cognition (4 cutoffs at the quintile values). The plot is split into three panels, one panel each for baseline age of 70, 80, and 90. We note that for all quintile cutoffs, the global cognition decrease over time, and the rate of cognitive decline becomes larger with age. Participants in the higher (lower) AD-CR Score quintiles have higher (lower) overall cognition levels. Additionally we note that the rate of cognitive decline is less steep (steeper) for subjects with high (low) AD-CR Scores. Finally we note there is a discontinuity of trajectories between the three panels, demonstrating the baseline age effect. There is an increase in each of the trajectories at the start of the new panel for participants in the lower AD-CR quintile groups. This phenomenon is attributable to selection bias; subjects who enter the study at older ages have lived dementia free until the time of enrollment and are therefore healthier than the general population.
Prediction
We applied our two stage machine learning framework to classify subjects as having extremely high or low AD-CR Scores, as these are the most clinically relevant groups. For these analyses, we restricted to baseline factors that are measured through clinical assessments during lifetime. Out of the machine learning models examined, cross-validation on the training set determined that gradient boosting (cross-validation-selected tuning parameters: mstop = 2000, nu = 0.1) with component-wise linear models (R package mboost) performed best in the first stage for separating subjects into an extreme AD-CR group and an average AD-CR group; additionally, cross-validation on the training set determined that linear support vector machines (SVM) were best for classifying subjects as having high or low AD-CR in stage two for both groups. Specifically, the linear SVM for the extreme AD-CR group has cross-validation-selected tuning parameter C = 0.003 and the linear SVM for the average AD-CR group has cross-validation-selected tuning parameter C = 0.0046. The predictive performance reported on the test set is visually depicted in Fig 4 and shows that our machine learning framework is able to predict whether new subjects will have extremely high (positive) AD-CR Scores vs. extremely low (negative) AD-CR Scores reasonably well (Sensitivity = 0.803, Specificity = 0.720, Accuracy = 0.767, and F-Score = 0.797 on the test set). We are not able to as accurately predict the AD-CR Scores of subjects in the average AD-CR group whose AD-CR Scores are close to zero (Sensitivity = 0.602, Specificity = 0.524, Accuracy = 0.568, and F-Score = 0.608 on the test set). Overall, our machine learning framework shows promising results in terms of predicting whether an individual will have an extremely high or low AD-CR Score using only baseline measures that are available during the lifetime.
The first stage model separates subjects into an Extreme AD-CR group and an Average AD-CR group based on whether their predicted AD-CR Scores are above a threshold, . Within each group separately, the second stage model classifies subjects as having either high or low AD-CR,
. We report the Sensitivity, Specificity, Accuracy, and F-Score of our machine learning framework when applied to the test set and also report the number of test subjects with high (orange) or low (blue) AD-CR Scores as well as the average AD-CR Score (
) in each of the predicted categories. Overall, our machine learning framework shows promising results in terms of predicting whether an individual will have an extremely high or low AD-CR Score using only baseline measures that are available during the lifetime.
Discussion
CR is the mechanism that allows some individuals to be more resilient to the pathology associated with AD than others. We provide a quantitative definition of CR with the AD-CR Score; given the AD related pathology, the AD-CR Score is the difference between the observed and expected cognition. The AD-CR Score is not the first definition of CR to be proposed. Previous work has defined CR using proxy variables [2, 9–11], as a latent factors [13, 14] and using residual cognition [15–17]. Proxy variables are a poor surrogate for CR as the variables used are not standardized among individuals and are correlated with other variables known to be associated with AD. Both the latent factor and residual cognition definitions rely on parametric models, which may be misspecified. The latent factor model uses in-vivo MRI to characterize AD-related pathology in place of ‘gold standard’ postmortem pathology. Residual cognition, the definition that is closest to the AD-CR Score, uses postmortem pathology, but can only be calculated at death. In contrast to the previously proposed definitions, the AD-CR Score is a fully non-parametric, stand-alone, individual-level quantification of CR that is derived independently of other factors or proxy variables. The AD-CR Score is computed with a global measure of cognition collected proximal to death and a ‘gold standard’ AD pathology measures collected at autopsy. Unlike residual cognition methods, we also provide a framework for prediction of the AD-CR Score using information available during the lifetime.
To validate the AD-CR Score as a measure of CR, we demonstrated the association of the AD-CR Score with baseline cognition, non-AD pathology, education, personality, APOE, parkinsonism, depression and life activities in the univariate ANOVA models. These measures have previously been shown to be associated with CR in the literature [2, 8, 37, 38, 47]. Consistent with prior studies [48], we also replicated the findings that smoking and alcohol consumption are not related to CR. We were unable to replicate the association between CR and physical activity. This could be due to the fact that the available activity measures are self reported. Self reported measures of physical activity are known to be biased, especially in elderly populations such as the population in this study [55]. Recent advances in monitoring physical activity in the elderly have included the use of activity monitors to obtain a less biased measure of activity [56], which could be useful for an analysis like this. We also found that the associations with the AD-CR Score and the measures changed over time. Many of the results from the baseline ANOVA models were confirmed in the cross-sectional ANOVA models for age. We also found associations with variables that were previously not found to be associated with CR: pain in the upper and lower extremities, vision and BMI. Due to the small sample sizes for these analysis further investigation is warranted. In addition we also found associations with the AD-CR Score and longitudinal cognitive decline that were consistent with those already established in the literature. Subjects in the upper quintiles of AD-CR Score showed slower cognitive decline than those in the lower quintiles of AD-CR Score.
In addition to validating the AD-CR Score by establishing previously observed associations with CR, we also built a framework for predicting the AD-CR Score. The AD-CR Score is calculated using postmortem pathology that is only available after death. This is ideal as ‘gold standard’ postmortem pathology is the only way to definitely diagnose AD, yet it is also a limitation as the AD-CR Score cannot be calculated during an individual’s lifetime. To increase the utility of the score, we built a machine learning framework to predict the AD-CR Score using measures collected at the baseline visit. Our study shows promising results in terms of classifying extremely high AD-CR Score from extremely low AD-CR Score. Future work includes expanding the prediction model to include environmental factors, activity measures, genomic information, in-vivo brain imaging measures to further increase the predictive accuracy. Accurate prediction of the AD-CR Score would allow the score to be used in clinical practice and in clinical trial settings. If individuals are identified as having low CR using the AD-CR Score, potential interventions could increase CR and reduce their risk of cognitive decline. In clinical trials for AD disease modifying therapies, it may be useful to enrich for participants with lower CR that have the potential to show the most improvement with the therapy.
Additionally, even though we used the global cognition measure and global AD pathology measure from the ROSMAP study to compute and validate our AD-CR Score in this study, the mathematical definition for the proposed AD-CR Score is very flexible and can accommodate a variety of cognition measures as well as other AD-related pathology measures. We look to further validate our proposed AD-CR Score using other measures of cognition such as MMSE and/or other AD-related pathology in future works and evaluate whether the AD-CR Score approach works well on other populations in other studies. Furthermore, we would love to further explore the use of various smoothing techniques in future works to mitigate the potential high variability from cognitive measures in subjects with AD.
In the current form, the AD-CR Score can be used as a research tool to further understand CR and factors that are associated with CR in a quantitative manner. As demonstrated here, the AD-CR Score may be applied for discovery of epidemiological factors that are associated with and/or predictive of CR. The AD-CR Score may also be used to discover genomic and in-vivo brain imaging markers of CR. Such advances would not only enhance our understanding of the mechansims of CR in AD pathogenesis, but also has the potential to powerfully inform stratification for clinical trials of potential disease-modifying therapies for AD.
Supporting information
S1 Appendix. Geometric definition of the AD-CR Score.
https://doi.org/10.1371/journal.pone.0241707.s001
(PDF)
S2 Appendix. Sensitivity analysis for cross sectional age ANOVA.
https://doi.org/10.1371/journal.pone.0241707.s002
(PDF)
Acknowledgments
Study data were provided by the Rush Alzheimer’s Disease Center, Rush University Medical Center, Chicago. We especially thank Greg Klein and Dr. Sue Leurgans for assistance with obtaining these data.
References
- 1. Dubois B, Feldman HH, Jacova C, DeKosky ST, Barberger-Gateau P, Cummings J, et al. Research criteria for the diagnosis of Alzheimer’s disease: revising the NINCDS–ADRDA criteria. The Lancet Neurology. 2007;6(8):734–746. pmid:17616482
- 2. Negash S, Bennett DA, Wilson RS, Schneider JA, Arnold SE. Cognition and neuropathology in aging: multidimensional perspectives from the Rush Religious Orders Study and Rush Memory and Aging Project. Current Alzheimer Research. 2011;8(4):336–340. pmid:21222592
- 3. Boyle PA, Wilson RS, Yu L, Barr AM, Honer WG, Schneider JA, et al. Much of late life cognitive decline is not due to common neurodegenerative pathologies. Annals of neurology. 2013;74(3):478–489. pmid:23798485
- 4. Bennett DA, Schneider JA, Arvanitakis Z, Kelly JF, Aggarwal NT, Shah RC, et al. Neuropathology of older persons without cognitive impairment from two community-based studies. Neurology. 2006;66(12):1837–1844. pmid:16801647
- 5. O’Brien RJ, Resnick SM, Zonderman AB, Ferrucci L, Crain BJ, Pletnikova O, et al. Neuropathologic studies of the Baltimore longitudinal study of aging (BLSA). Journal of Alzheimer’s Disease. 2009;18(3):665–675. pmid:19661626
- 6. White L. Brain lesions at autopsy in older Japanese-American men as related to cognitive impairment and dementia in the final years of life: a summary report from the Honolulu-Asia aging study. Journal of Alzheimer’s Disease. 2009;18(3):713–725. pmid:19661625
- 7. Jellinger KA, Attems J. Neuropathological evaluation of mixed dementia. Journal of the neurological sciences. 2007;257(1):80–87.
- 8. Stern Y. Cognitive reserve in ageing and Alzheimer’s disease. The Lancet Neurology. 2012;11(11):1006–1012.
- 9. Jones RN, Manly J, Glymour MM, Rentz DM, Jefferson AL, Stern Y. Conceptual and measurement challenges in research on cognitive reserve. Journal of the International Neuropsychological Society. 2011;17(04):593–601.
- 10. Stern Y. What is cognitive reserve? Theory and research application of the reserve concept. Journal of the International Neuropsychological Society. 2002;8(03):448–460.
- 11. Serra L, Musicco M, Cercignani M, Torso M, Spanò B, Mastropasqua C, et al. Cognitive reserve and the risk for Alzheimer’s disease: a longitudinal study. Neurobiology of aging. 2015;36(2):592–600. pmid:25433459
- 12. Stern Y, Arenaza-Urquijo EM, Bartres-Faz D, Belleville S, Cantilon M, et al. Whitepaper: Defining and investigating cognitive reserve, brain reserve, and brain maintenance. Alzheimer’s & Dementia. 2018;(18):S1552–5260.
- 13. Reed BR, Mungas D, Farias ST, Harvey D, Beckett L, Widaman K, et al. Measuring cognitive reserve based on the decomposition of episodic memory variance. Brain. 2010; p. awq154. pmid:20591858
- 14. Zahodne LB, Manly JJ, Brickman AM, Narkhede A, Griffith EY, Guzman VA, et al. Is residual memory variance a valid method for quantifying cognitive reserve? A longitudinal application. Neuropsychologia. 2015;77:260–266. pmid:26348002
- 15. Bennett DA, Wilson RS, Schneider JA, Evans DA, De Leon CM, Arnold SE, et al. Education modifies the relation of AD pathology to level of cognitive function in older persons. Neurology. 2003;60(12):1909–1915. pmid:12821732
- 16. Bennett DA, Schneider JA, Tang Y, Arnold SE, Wilson RS. The effect of social networks on the relation between Alzheimer’s disease pathology and level of cognitive function in old people: a longitudinal cohort study. The Lancet Neurology. 2006;5(5):406–412.
- 17. Yu L, Boyle PA, Segawa E, Leurgans S, Schneider JA, Wilson RS, et al. Residual decline in cognition after adjustment for common neuropathologic conditions. Neuropsychology. 2015;29(3):335. pmid:25495832
- 18. Bennett DA, Schneider JA, Arvanitakis Z, Wilson RS. Overview and findings from the religious orders study. Current Alzheimer Research. 2012;9(6):628–645.
- 19. Bennett DA, Schneider JA, Buchman AS, Barnes LL, Boyle PA, Wilson RS. Overview and findings from the rush Memory and Aging Project. Current Alzheimer Research. 2012;9(6):646–663.
- 20. Bennett DA, Buchman AS, Boyle PA, Barnes LL, Wilson RS, Schneider JA. Religious Orders Study and Rush Memory and Aging Project. Journal of Alzheimer’s Disease. 2018;64(s1):S161–S189.
- 21. McKhann G, Drachman D, Folstein M, Katzman R, Price D, Stadlan EM. Clinical diagnosis of Alzheimer’s disease: report of the NINCDS-ADRDA Work Group under the auspices of Department of Health and Human Services Task Force on Alzheimer’s Disease. Neurology. 1984;34(7):939–944.
- 22. National Institute on Aging and Reagan Institute Working Group on Diagnostic Criteria for the Neuropathological Assessment of Alzheimer’s Disease. Consensus recommendations for the postmortem diagnosis Of Alzheimers Disease. Neurobiol Aging. 1997;18(4 Suppl):S1–S2.
- 23. Bennett DA, Schneider JA, Buchman AS, Barnes LL, Boyle PA, Wilson RS. Overview and findings from the Rush Memory and Aging Project. Current Alzheimer Research. 2012;9(6):646–663.
- 24. Wilson RS, Beckett LA, Barnes LL, Schneider JA, Bach J, Evans DA, et al. Individual differences in rates of change in cognitive abilities of older persons. Psychology and aging. 2002;17(2):179. pmid:12061405
- 25. Wilson RS, Barnes LL, Bennett DA. Assessment of lifetime participation in cognitively stimulating activities. Journal of Clinical and Experimental Neuropsychology. 2003;25(5):634–642.
- 26. Honig LS, Kukull W, Mayeux R. Atherosclerosis and AD Analysis of data from the US National Alzheimer’s Coordinating Center. Neurology. 2005;64(3):494–500.
- 27. Bennett DA, Wilson RS, Schneider JA, Evans DA, Aggarwal NT, Arnold SE, et al. Apolipoprotein E ε4 allele, AD pathology, and the clinical expression of Alzheimer’s disease. Neurology. 2003;60(2):246–252. pmid:12552039
- 28. McKeith IG, Galasko D, Kosaka K, Perry EK, Dickson DW, Hansen LA, et al. Consensus guidelines for the clinical and pathologic diagnosis of dementia with Lewy bodies (DLB) Report of the consortium on DLB international workshop. Neurology. 1996;47(5):1113–1124. pmid:8909416
- 29. Nag S, Yu L, Capuano AW, Wilson RS, Leurgans SE, Bennett DA, et al. Hippocampal sclerosis and TDP-43 pathology in aging and Alzheimer disease. Annals of neurology. 2015;77(6):942–952. pmid:25707479
- 30. Schneider JA, Bienias JL, Wilson RS, Berry-Kravis E, Evans DA, Bennett DA. The apolipoprotein E ε4 allele increases the odds of chronic cerebral infarction detected at autopsy in older persons. Stroke. 2005;36(5):954–959.
- 31. Arvanitakis Z, Leurgans SE, Barnes LL, Bennett DA, Schneider JA. Microinfarct pathology, dementia, and cognitive systems. Stroke. 2011;42(3):722–727.
- 32. Schneider JA, Wilson RS, Cochran EJ, Bienias JL, Arnold SE, Evans DA, et al. Relation of cerebral infarctions to dementia and cognitive function in older persons. Neurology. 2003;60(7):1082–1088. pmid:12682310
- 33. Schneider JA, Boyle PA, Arvanitakis Z, Bienias JL, Bennett DA. Subcortical infarcts, Alzheimer’s disease pathology, and memory function in older persons. Annals of neurology. 2007;62(1):59–66.
- 34. Costa PT, McCrae RR. The revised neo personality inventory (neo-pi-r). The SAGE handbook of personality theory and assessment. 2008;2:179–198.
- 35. Wilson RS, Bienias JL, de Leon CFM, Evans DA, Bennett DA. Negative affect and mortality in older persons. American Journal of Epidemiology. 2003;158(9):827–835.
- 36. Wilson RS, Arnold SE, Schneider JA, Li Y, Bennett DA. Chronic distress, age-related neuropathology, and late-life dementia. Psychosomatic Medicine. 2007;69(1):47–53.
- 37. Boyle PA, Buchman AS, Wilson RS, Kelly JF, Bennett DA. The APOE ε4 allele is associated with incident mild cognitive impairment among community-dwelling older persons. Neuroepidemiology. 2009;34(1):43–49.
- 38. Hensley K, Barnes LL, Christov A, Tangney C, Honer WG, Schneider JA, et al. Analysis of postmortem ventricular cerebrospinal fluid from patients with and without dementia indicates association of vitamin E with neuritic plaques and specific measures of cognitive performance. Journal of Alzheimer’s Disease. 2011;24(4):767–774. pmid:21321395
- 39. Bennett DA, Shannon KM, Beckett LA, Goetz CG, Wilson RS. Metric properties of nurses’ ratings of parkinsonian signs with a modified Unified Parkinson’s Disease Rating Scale. Neurology. 1997;49(6):1580–1587.
- 40. Kohout FJ, Berkman LF, Evans DA, Cornoni-Huntley J. Two shorter forms of the CES-D depression symptoms index. Journal of aging and health. 1993;5(2):179–193. pmid:10125443
- 41. Lawton MP, Brody EM. Assessment of older people: self-maintaining and instrumental activities of daily living. Nursing Research. 1970;19(3):278.
- 42. Katz S, Akpom CA. A measure of primary sociobiological functions. International journal of health services. 1976;6(3):493–508. pmid:133997
- 43. Rosow I, Breslau N. A Guttman health scale for the aged. Journal of gerontology. 1966;21(4):556–559.
- 44. Vance DE, Crowe M. A proposed model of neuroplasticity and cognitive reserve in older adults. Activities, Adaptation & Aging. 2006;30(3):61–79.
- 45. Geerlings MI, Deeg DJ, Penninx BW, Schmand B, Jonker C, Bouter LM, et al. Cognitive reserve and mortality in dementia: the role of cognition, functional ability and depression. Psychological medicine. 1999;29(05):1219–1226. pmid:10576313
- 46. Scarmeas N, Stern Y. Cognitive reserve and lifestyle. Journal of clinical and experimental neuropsychology. 2003;25(5):625–633. pmid:12815500
- 47. O’Brien JPJT. Parkinson’s Disease with Dementia. Psychiatry of Parkinson’s Disease. 2012;27:103.
- 48. Broe GA, Creasey H, Jorm AF, Bennett HP, Casey B, Waite LM, et al. Health habits and risk of cognitive impairment and dementia in old age: a prospective study on the effects of exercise, smoking and alcohol consumption. Australian and New Zealand journal of public health. 1998;22(5):621–623. pmid:9744220
- 49. Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the royal statistical society Series B (Methodological). 1995; p. 289–300.
- 50. Stekhoven DJ. missForest: nonparametric missing value imputation using Random Forest; 2013.
- 51. Stekhoven DJ, Buhlmann P. MissForest—non-parametric missing value imputation for mixed-type data. Bioinformatics. 2012;28(1):112–118. pmid:22039212
- 52.
Gu C. Smoothing spline ANOVA models. vol. 297. Springer Science & Business Media; 2013.
- 53. Raber J, Huang Y, Ashford JW. ApoE genotype accounts for the vast majority of AD risk and AD pathology. Neurobiology of aging. 2004;25(5):641–650.
- 54. Bennett DA, De Jager PL, Leurgans SE, Schneider JA. Neuropathologic intermediate phenotypes enhance association to Alzheimer susceptibility alleles. Neurology. 2009;72(17):1495–503. pmid:19398704
- 55. Sallis JF, Saelens BE. Assessment of physical activity by self-report: status, limitations, and future directions. Research quarterly for exercise and sport. 2000;71(sup2):1–14.
- 56. Washburn RA, Smith KW, Jette AM, Janney CA. The Physical Activity Scale for the Elderly (PASE): development and evaluation. Journal of clinical epidemiology. 1993;46(2):153–162.