Study population

The English Longitudinal Study of Ageing (ELSA) is an observational multidisciplinary panel study of older adults living in the England community [26]. In ELSA, data of participants are measured every 2 years by computer-assisted personal interviews (CAPI) and self-completion questionnaires. At the time of this study, ELSA includes eight waves over a follow-up period of up to 14 years from wave 1 (2002–2003) to wave 8 (2016–2017) [26].

In the present study, data of 2733 participants were selected spanning 12 years across wave 2 (2004–2005) to wave 8 (2016–2017). The exposure variables were derived from a broad range of domains including sociodemographic, social engagement, health, physical functioning, health behaviors, and cognitive tests at baseline (wave 2). Participants aged 50–85 years without dementia at the baseline assessment were included. The ethical approval of ELSA was obtained from the National Research Ethics Committee and all participants signed the informed consent (http://www.hra.nhs.uk/about-the-hra/our-committees/res/).

Outcome variable

In the present study, verbal episodic memory was used to measure cognitive functioning in seven waves (wave 2- wave 8). According to the literature, verbal episodic memory is an early and common sign of age-related cognitive decline that is associated with the daily activities of aging populations [27, 28]. Immediate and delayed word recall tests (range 0–10) were used to assess episodic memory, where participants listened to 10 common words. Then, they were asked to recall as many words as possible, in any order immediately and after a short delay (about 5 minutes). The total episodic memory score was obtained using the correct number of recalled words in both tests (range 0–20). As such, a better episodic memory is indicated by a higher score. The word lists used in the present study were originally provided in the Health Retirement Study (HRS) [29], which presented good construct validity and consistency [30].

The participant-specific slopes of episodic memory from wave 2 (baseline) to wave 8 over 12 years of follow-up were estimated using the linear model with random intercepts and slopes [1]. This resulted in defining two groups of major and minor cognitive decliners. Participants with minor cognitive decline were those with predicted slopes more than 1 SD below the mean, while those with predicted slopes less than 1 SD below the mean were categorized as major cognitive decliners.

Independent variables

A wide range of factors was derived both directly and indirectly from the collected data in ELSA. The factors were selected and categorized carefully based on their features and the literature [2, 14, 21, 31]. In total, the factors were categorized into 7 groups. For more information S1 Table.

Machine learning framework

Development of the prediction model for discrimination of future minor cognitive decliners from those with major cognitive decline involved multiple steps. First, the original ELSA dataset was randomly split into the train and test sets, where 85% and 15% of all data were allocated to the training and test set, respectively. In total, there were 2733 samples. As such, 2323 and 410 samples were used as the training and test data. Second, we applied several data pre-processing techniques including the normalization of data to have an average of 0 and a standard deviation of 1, encoding categorical data using the One-hot encoding method. Third, feature selection methods were applied to select the optimal subset of important features. Lastly, four different prediction models were individually implemented. In the following sections, a more in-depth description of the process is provided.

Imbalanced data resampling

A problem with the classification of imbalanced data is that the number of samples in the minority class is too few, which results in decreasing the effectiveness of model learning. In the present study, the included dataset used to predict longitudinal cognitive changes was imbalanced where the prevalence of individuals with major cognitive changes was about 13.6%. Accordingly, we combined Synthetic Minority Oversampling Technique (SMOTE) with random under-sampling to resolve the imbalanced data problem in the training dataset before fitting a model.

Synthetic Minority Oversampling Technique (SMOTE) is an oversampling method that generates new synthetic samples from the minority class between the minority samples and their selected nearest neighbors [42]. This process is a type of data augmentation for tabular data that results in having larger and less specific decision regions. However, a disadvantage of the over-sampling method is that the majority class is not considered in generating the synthetic samples. This may cause creating an ambiguous sample if there is a strong overlap between the majority and minority classes.

In order to address the issue, we combined the SMOTE with under-sampling that has shown better performance compared to plain under-sampling [42]. Therefore, first, we oversampled the minority class, following that under-sampled the majority class to transform and balance the dataset. Furthermore, at the level of algorithm, the built-in parameters of each ML algorithm were tuned to deal with imbalanced data.

Feature selection

Feature selection has a key role in machine learning-based models that removes irrelevant and redundant information from the original features. Appropriate feature selection would result in better interpretability, simplified modeling, shorter learning time, and enhanced generalization [43, 44]. In this regard, we used Ridge regression (RR) as a feature selection method to select the important features and rank each individual feature by reducing the size of the coefficients instead of setting them equal to zero. In the Ridge regression model, the L2 regularization technique is used that penalizes the L2-norm of the coefficients in linear regression. More specifically, the magnitude of feature coefficients is penalized while the error between actual and predicted values is minimized.

ML-based models

In this study, multiple types of machine learning methods are provided. The * indicates the a including extreme gradient boosting (XGBoost), adaptive Boosting (AdaBoost), support vector machine (SVM), and multi-layer perceptron (MLP) neural network. These algorithms are selected due to their high performance in previous studies to implement prediction models for binary classification [45–48]. Further details of the optimal parameters of the selected machine learning algorithms are provided in S2 Table. In each individual algorithm of AdaBoost, XGBoost, SVM, and MLP the grid search technique was used separately to determine the optimal parameters based on the specific goals of this study. The grid search technique is a hyperparameter optimization method based on a defined subset of hyper-parameter space. The lower bound, upper bound, and the number of steps are required to specify the hyper-parameters in this technique [49]. All models were implemented with Python 3.7.6.

D. MLP.

Multi-layer perceptron or MLP is a feed-forward neural network that consists of several layers of neurons to flow the information from the input layer to the output layer [57]. The connections between interacting neurons are weighted. The degree of correlation between the activity levels of the connected neuron is measured by weights. Back-propagation algorithms are commonly used to correct the weights from output layers backward and to train the neural network [45]. The performance MLP model is related to multiple parameters such as the numbers of hidden layers, neurons, learning rate, momentum, and the number of iterations. The speed and effectiveness of the MLP model are balanced by the learning rate and the momentum. In Fig 1, the architecture of an MLP neural network is represented. In this study, a four fully connected (FC) layers of the perceptron neural network was used. The sigmoid activation function was used for the node output (output layer of the model) where it takes a real-valued input and squashes it to range between 0 and 1. In contrast, the ReLU (Rectified Linear Unit) was used for the hidden layers, where it outputs the input directly if the output is positive, otherwise, it outputs the input zero. Besides, the “AdaBound” and “binary cross-entropy” were used as the optimizer and loss (cost) function to train the model. The grid search which is based on a defined subset of hyper-parameter space was used as the model hyperparameter optimization technique. In Fig 2, the framework of the proposed machine learning prediction model is presented.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Structure of an MLP model with two hidden layers, and one output.

https://doi.org/10.1371/journal.pone.0280029.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Framework of the proposed machine learning prediction model.

https://doi.org/10.1371/journal.pone.0280029.g002

Evaluation metrics

After random distribution of the original ELSA dataset into the training and test sets the validation approach was applied to the training set. We used the repeated stratified 5-fold cross-validation method to train and validate the prediction models and to achieve a reliable measure. In repeated stratified 5-fold cross-validation, the original sample is randomly partitioned into 5 equal-sized subsets in a way that maintains the same class distribution in each subset. This could preserve the imbalanced class distribution in each fold and enforce the class distribution in each split of the data to match the distribution in the complete training dataset. Then, the validation process is repeated 3 times, with each of the 5 subsamples used exactly once as the validation data [21]. In this study, the goal was to ensure that each fold had the same proportion of major cognitive decline observations.

Three indexes of sensitivity (SEN), specificity (SPE), and area under the curve (AUC) are used to calculate the results of models for binary classification. The “sensitivity” indicates the ability of the model to determine major cognitive decliners, correctly. This measure is defined as TP/(TP+FN), where TP and FN stand for the proportion of true positive (i.e., minor cognitive decliners who are classified correctly) and false negative cases (i.e., major cognitive decliners who are classified as minor cognitive decliners), respectively. In contrast, “specificity” indicates the ability of the model to determine minor cognitive decliners, correctly. This measure is defined as TN/(TN+FP), where TN and FP stand for the proportion of true negative (i.e., minor cognitive decliners who are correctly classified) and false positive case (i.e., minor cognitive decliners who are classified as major cognitive decliners), respectively. The derived measure of AUC determines the inherent ability of the test to discriminate between individuals with minor and major cognitive decline. Another interpretation of AUC is “the average value of sensitivity for all the possible values of specificity” [58]. A higher score on these indexes indicates a better model performance. We did not include the index of accuracy as a performance metric. This measure does not provide meaningful information regarding the performance of classification models due to the unequal number of participants in two groups of minor and major cognitive decliners [59].

Cognitive groups (major vs minor cognitive decliners)

In total, 2733 participants are included in the present study where 2361 individuals (86.4%) were categorized as minor cognitive decliners and 372 individuals (13.6%) as major cognitive decliners. Fig 3 illustrated the estimated two cognitive groups with major cognitive decline (slope > 1 SD below mean) and minor cognitive decline (slope ≤ 1 SD below mean) based on the changes in the level of memory function. Memory function was measured using the combination of immediate and delayed word recall tests from baseline (year 1) to year 12. The slope (and intercept) of the estimated linear model for 12-year memory changes among minor and major cognitive decliner groups were -0.03 (intercept, 11.35) and -1.02 (intercept, 13.49) respectively.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3.

Two cognitive groups of major cognitive decliners (slope > 1 SD below mean) and minor cognitive decliners (slope ≤ 1 SD below mean) were determined based on the changes in the level of memory function (combination of immediate and delayed word recall tests) from baseline (year 1) to year 12. The * indicates the average memory function for participants in each group during the assessment year. The lines are fitted using the linear mixed model.

https://doi.org/10.1371/journal.pone.0280029.g003

Participant characteristics

Descriptive results for characteristics of participants at baseline year are presented in Table 1. As shown, the characteristics of both minor and major cognitive decliner groups in seven domains including sociodemographic, social engagement, health, psychology, physical functioning, health-related behaviors, and baseline cognitive tests are presented. In the sociodemographic domain, compared to minor decliners, major decliners were more likely to be older, female, unmarried/divorced, relatively lower levels of education and employment, as well as lower levels of occupation status. However, participants in both groups were likely to have similar socioeconomic statuses. In the social engagement domain, those with participation in three or more community activities (i.e., engaging in a hobby or pastime, going on a holiday) were more likely to be major cognitive decliners. In contrast, the percentage of individuals who had the membership in four or more social organizations (i.e., resident groups, religious groups, charitable associations) were relatively similar in both cognitive groups. In the health domain, participants in the minor-decline group self-reported their health, eyesight, and hearing as good, very good, or excellent (vs fair or poor) compared to the major decliners. Furthermore, those with more than one CVD type disease (i.e., high cholesterol, hypertension, Angina) and eyesight were more likely to be major cognitive decliners. By contrast, the percentage of participants with one or more than one eye non-CVD type disease (i.e., Asthma, Cancer, or a malignant tumor) was relatively similar in both groups. In the physical functioning domain, major decliners were more likely to have at least one limitation with ADL (i.e., kneeling, climbing) and one limitation with IADL (i.e., showering, taking medications,). Furthermore, those in the group of major decline were more likely to need at least one type of mobility aid (i.e., cane or walking stick, manual wheelchair). In the psychology domain, those with relatively lower levels of depression symptoms (CES-D based depression), and lower level of loneliness feeling (UCLA based loneliness) was more likely to be in the group of minor cognitive decliners. However, in both groups, the level of life satisfaction (SWLS based life satisfaction) was relatively similar. In the health-related behaviors domain, participants in the minor cognitive decline group self-reported taking part in mild and moderate physical activities once or more per week. However, participants in the major were more likely to take part in vigorous physical activities once or more per week. Furthermore, the number of participants who drank more than one alcoholic drink per week during the last 12 months and current smokers were relatively higher in the group with minor cognitive decline. In the baseline cognitive tests domain, participants in the minor cognitive decline group self-reported their health literacy and eyesight as good, very good, or excellent (vs fair or poor) compared to the major decliners, their score of delayed word recall was likely higher as well. However, the scores on other cognitive tests including immediate word recall, time orientation, verbal fluency, self-rated memory change, processing speed, and health literacy were relatively similar among the participants of both cognitive groups.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Characteristics of the participants in two groups of individuals with major and minor cognitive decline at baseline (wave 2 of ELSA dataset).

https://doi.org/10.1371/journal.pone.0280029.t001

Performance of ML-based models

A total of 2733 samples was split to have 410 samples (15%) in the test set and 2323 samples (85%) in the training set. Following that, in the training stage, the pre-processing procedure including normalization and imbalanced data techniques are performed. Then, feature selection was applied using the Ridge regression method that resulted in the selection of optimal features. Following that, the performance of four ML-based predictive models in terms of three evaluation metrics including AUC, specificity, and sensitivity on the test data were obtained. The results are illustrated in Table 2. The best result was achieved using XGBoost with 72.84% AUC, 78.23% sensitivity, and 67.41% specificity overall. We further investigated the effects of the feature selection technique on the highest performance model (XGBoost). In this regard, the results of the XGBoost model in two conditions of using the original features (without any feature selection) and using the optimal subset of important features (with feature selection) were compared. The findings indicated that the performance of the model remained relatively stable. This shows the robustness of the XGBoost model and its less sensitivity to irrelevant and redundant features.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Classification performance of four machine learning algorithms on the test set.

https://doi.org/10.1371/journal.pone.0280029.t002

Top-ranking predictors of longitudinal cognitive changes

XGBoost is a tree-based ensemble learning algorithm that could be used for both classification and feature ranking [60]. Among all models, XGBoost showed the highest performance for discrimination of future major and minor cognitive decliners using the test set (see Table 2). As the next step, we used this algorithm as the feature ranking method to sort features according to the feature importance degree using Gini importance (mean decrease in impurity). In XGBoost, three measures of feature importance scores including ‘weight’, ‘gain’, and ‘cover’ could be used that represent the relative importance.

The ‘weight’ is a valid and commonly used measure to estimate the importance score of the features. This measure indicates how many times a feature is used to split the data across all trees. As such, in the present study, we used this measure for feature ranking [61]. Fig 4 illustrated the results of this method that determined the most influencing predictors on the longitudinal changes of memory function. Based on the results among all 43 baseline features the top 7 ranked features that had an important role in predicting major cognitive decliners vs minor decliners included age, employment status, socioeconomic status, self-rated memory changes, immediate word recall, feeling of loneliness, and vigorous physical activity. In contrast, the five least important baseline features consisted of smoking, instrumental activities of daily living, eye disease, life satisfaction, and cardiovascular disease.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Top 25 ranked features with their score when predicting major cognitive decliners vs minor decliners using fitted XGBoost.

https://doi.org/10.1371/journal.pone.0280029.g004