Framingham Risk Score and Alternatives for Prediction of Coronary Heart Disease in Older Adults

Background Guidelines for the prevention of coronary heart disease (CHD) recommend use of Framingham-based risk scores that were developed in white middle-aged populations. It remains unclear whether and how CHD risk prediction might be improved among older adults. We aimed to compare the prognostic performance of the Framingham risk score (FRS), directly and after recalibration, with refit functions derived from the present cohort, as well as to assess the utility of adding other routinely available risk parameters to FRS. Methods Among 2193 black and white older adults (mean age, 73.5 years) without pre-existing cardiovascular disease from the Health ABC cohort, we examined adjudicated CHD events, defined as incident myocardial infarction, CHD death, and hospitalization for angina or coronary revascularization. Results During 8-year follow-up, 351 participants experienced CHD events. The FRS poorly discriminated between persons who experienced CHD events vs. not (C-index: 0.577 in women; 0.583 in men) and underestimated absolute risk prediction by 51% in women and 8% in men. Recalibration of the FRS improved absolute risk prediction, particulary for women. For both genders, refitting these functions substantially improved absolute risk prediction, with similar discrimination to the FRS. Results did not differ between whites and blacks. The addition of lifestyle variables, waist circumference and creatinine did not improve risk prediction beyond risk factors of the FRS. Conclusions The FRS underestimates CHD risk in older adults, particularly in women, although traditional risk factors remain the best predictors of CHD. Re-estimated risk functions using these factors improve accurate estimation of absolute risk.


Introduction
Guidelines for the prevention of coronary heart disease (CHD) recommend the use of risk scores to identify adults at higher risk of CHD for whom preventive therapy-e.g., by lipid lowering drugshas higher absolute benefits [1]. Several scoring systems exist to help clinicians assess the 10-year CHD risk [2,3,4], with the Framingham risk score (FRS) [2] the most widely used. US Guidelines for the prescription of lipid-lowering drug therapy [5] and aspirin in primary prevention [6] are based on the risk estimations provided by the FRS.
Most risk scores were developed in white middle-aged populations [2,3,4]. Thus, it is uncertain whether risk estimates based on these scores can be generalized to the elderly. The FRS, for example, was developed in a white middle-aged population with a mean age of 49 years and included persons as young as 30 and none older than 74 [2]. Actual risk prediction with FRS might perform less well in older adults compared to middle-aged adults, and some traditional risk factors have weaker associations with CHD risk in the elderly; for example, total and LDL-cholesterol are strong cardiovascular risk factors in middle-aged but not in older adults [7].
As it remains unclear whether and how CHD risk prediction might be improved in the growing population of elderly [8] to facilitate primary prevention strategies, we aimed to compare the prognostic performance of 1) the FRS, directly and 2) after recalibration [9], and 3) with functions derived from the Health ABC Study, a cohort of elderly white and black men and women [10]. We also aimed to assess 4) the utility of adding routinely available lifestyle and simple laboratory variables not part of the FRS but which have been shown to predict CHD in older adults, such as creatinine [11], glucose [12] and lifestyle factors (alcohol consumption [13], physical activity [14]).

Study population
Participants were part of the Health, Aging, and Body Composition Study (Health ABC Study), a population-based cohort of 3075 community-dwelling men and women, aged 70-79 during the study enrollment period in 1997-1998. Participants were identified from a random sample of white and all black Medicare-eligible adults living in designated zip codes areas surrounding Pittsburgh, PA, and Memphis, TN. Eligibility criteria at baseline included the ability to walk J mile, up 10 stairs without rest and perform basic activities of daily living independently [10]. All participants gave written informed consent and the Pittsburgh and Memphis Institutional Review Boards approved the protocol.
Among the 3075 participants, we excluded 841 who had overt cardiovascular disease (CVD) at baseline, defined as diagnosis of CHD (angina, prior myocardial infarction, angioplasty of coronary arteries or coronary artery surgery), stroke or transient ischemic attack, peripheral arterial revascularization, carotid artery disease, heart failure or having a pacemaker. We also excluded 41 participants with missing data for any of the traditional cardiovascular risk factors. The final sample for our analyses was 2193 participants.

Measurements
Cardiovascular risk factors. Participants reported smoking history and were classified as never, current, or former smoker. Fasting total cholesterol, HDL-cholesterol, and blood pressure were measured as previously described [15]. Hypertension was defined as self-report and use of anti-hypertensive medications, or measured blood pressure $140 and/or $90 mm Hg. Diabetes was defined as self-reported medical diagnosis and/or using any hypoglycemic medication [16]. Physical activity was assessed by questionnaire about all types of walking and exercise performed in the prior week [14].
Cardiovascular events. During 8-year follow-up, we assessed incident CHD events and mortality among participants without overt CVD at baseline [16]. Using algorithms mirroring those of the Cardiovascular Health Study [16], diagnoses and cause of death were adjudicated until 2006-2007 based on interview, review of all hospital records, death certificates, and other documents by a panel of clinicians. CHD events included nonfatal myocardial infarction or coronary death (corresponding to ''hard'' events, as defined in the current FRS [5]), and hospitalization for angina or revascularization (coronary angioplasty or surgery) [17].

Statistical Analyses
The FRS predicts 10-year CHD risk based on a Cox model estimated using data from the Framingham Heart Study [2]. The Framingham cohort included 5345 subjects aged 30-74 years at the time of their examination in [1971][1972][1973][1974]. For this analysis, we used the sex-specific Framingham equations of Wilson [2], because they include diabetes, a strong independent CHD risk factor [18,19]. This FRS Cox model includes age, total and HDL cholesterol, blood pressure, diabetes, and smoking status.
In this study, we compared the prognostic performance of the FRS, directly and after recalibration (taking into account different prevalence of risk factors and underlying rates of developing CHD), with functions entirely derived from the Health ABC cohort, similar to previous studies [9]. Analyses were stratified by gender. We first estimated the FRS using regression coefficient estimates and values of the risk factor means reported by Wilson [2]. To account for the shorter follow-up in the Health ABC study and to avoid extrapolation beyond the range of the data [17], we examined 7.5-year risk and adapted accordingly the estimated baseline survival function used in computing the FRS. Participants who died from non-CHD death were censored at the time of death.
We then examined whether the predictive performance of the FRS could be improved with recalibration or with refitting model coefficients. For the recalibrated version of the FRS [9], we reestimated predicted risks for Health ABC by retaining the original coefficient estimates reported by Wilson [2] but adapted the risk factor means to the present cohort and the Kaplan Meier estimate of the baseline survival function of Health ABC data. For the refit version of the FRS (the ''Health ABC function''), we estimated the regression coefficients with a Cox model fitted to the Health ABC data, obtaining an estimated predicted risk entirely based on Health ABC data. In this model, some adjacent risk factor categories were combined to avoid cells with limited numbers of events and/or unpredictive trends.
To compare prediction of these three risk models, we examined different statistical measures. To assess discrimination, we used Harrell's C-index [20], an adaptation of the C-statistic an adaptation of the C-statistic or area under the ROC curve for use with survival data. As the model validation for Health ABC functions was performed on the same dataset used for estimating the Cox model and the sample included too few events for splitsample validation, we calculated an optimism-corrected C-index using bootstrap resampling [21] with 1000 replications [20]. To assess model calibration, we used Parzen's adaptation [22] of the Hosmer-Lemeshow test to the Cox model.
In exploratory analysis, we sought to determine whether alternative sets of predictors would improve risk prediction. To evaluate the utility of adding to the FRS different lifestyle and simple laboratory variables, we initially considered predictor variables with p,0.20 in unadjusted Cox models for CHD events in Health ABC data. We then used three model selection procedures: a backward selection with a retention criterion of p,0.10, and two forward stepwise selection procedures minimizing the Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC), respectively [23]. In these models, total cholesterol, HDL-cholesterol, and blood pressure were modeled as continuous predictors. Statistical analyses were performed using the software R, version 2.9.1 (R Foundation for Statistical Computing, Vienna, Austria).

Results
At baseline, the mean age of the study participants was 73.5 years; 55% were women, and 41% were black ( Table 1). The mean 10-year risk based on the FRS was 14.9%. Most participants had a 10-year CHD risk ranging from 5 to 19.9%.
During a median follow-up of 8.3 years (maximum, 10.2 years), 351 participants developed a CHD event (197 of which had a ''hard'' CHD event). In unadjusted analyses, all traditional cardiovascular risk factors were associated with CHD events Table 1. Baseline characteristics and unadjusted associations with incident CHD events (n = 2193; number of CHD events = 351).

Variable
Mean ± SD / n (%) HR (95% CI) p  (Table 1). Abdominal circumference, glucose, and creatinine were also associated with CHD events, but not glomerular filtration rate, alcohol use or physical activity levels. Results were similar for hard CHD events with larger confidence intervals because of lower number of events (data not shown), except that the association with abdominal circumference disappeared (HR = 1.00). Number of participants in different risk factor categories and CHD events are shown in Table 2 for women and Table 3 for men. The original FRS had poor discrimination in these older adults (C-index: 0.577 in women; 0.583 in men). Using risk factors as continuous variables yielded similar C-indexes. Calibration of the original FRS was also poor in older adults (Figure 1), particularly among women, for whom the absolute risk was underestimated by 51% (vs. 8% in men, Table S1). Recalibration of the FRS improved calibration, particularly for women, and produced a better match between observed and expected CHD risk ( Figure 1, Table S1). Statistically significant differences between observed and expected risks across deciles remained, at least in in women (p value remaining ,0.05, larger p-values indicating better calibration), with an overestimation of the predicted risk for those above the median risk by a factor of 1.4 for women and 1.3 for men. For both genders, the Health ABC function significantly improved calibration ( Figure 1). For discrimination, the C-index for the Health ABC function, after correction for optimism, was comparable to the C-index of the FRS (p = 0.54 for women and 0.90 for men, Tables 2 and 3). Total cholesterol and age 2 were not predictive in women and were therefore omitted in the Health ABC function. Overall, results did not differ between whites and blacks. C-indexes for the unmodified FRS, the recalibrated FRS and the Health ABC function stratified by gender did not significantly differ between whites and blacks (all p for interaction .0.20; C-indexes ranging from 0.550 to 0.603). Calibration became reasonable (with p.0.20 for comparison of observed with expected) in white men for the unmodified FRS and in white men and women for the recalibrated FRS, but the best calibration remained for the Health ABC function (p.0.20 for comparison of observed with expected in the four subgroups stratified by race and gender).
We used a variety of model selection procedures when considering the addition of routinely available measures not included in the Framingham risk factor set to the Health ABC function. The procedures based on p-values and the AIC lead to very similar final models (Table S2); in contrast, the BIC, which strongly penalizes the complexity of the model, lead to the omission of a larger number of risk factors. All final models mainly retained traditional risk factors included in the FRS. The additions of lifestyle variables (alcohol, physical activity), waist circumference, and creatinine did not improve risk prediction in terms of discrimination or model fit beyond using the traditional risk factors from the FRS. Selection procedures stratified by gender yielded similar results.

Discussion
In this population-based study of older adults, the FRS poorly discriminated between persons who experienced a CHD event and those who did not (C-index: 0.577 in women; 0.583 in men) and underestimated the absolute CHD risk by 51% in women and 8% in men. Nevertheless, traditional risk factors remained the best predictors of CHD events. Physical activity, alcohol consumption, waist circumference and creatinine did not improve risk prediction beyond traditional risk factors of the FRS. Recalibration of the FRS improved the accuracy of absolute risk estimation, particularly for women. For both genders, the Health ABC function significantly improved estimation of absolute risk, with a discrimation similar to the FRS. Neither refitting equations nor including other routinely available measurements in risk equations provided substantial benefits in terms of discriminating between high-and low-risk older adults over FRS.
Our study adds new data on the performance of recalibration of the FRS, refit functions and the utility of adding other routinely  available risk parameters to FRS among older adults. Previous studies also found lower performance of risk prediction based on the FRS associated with increasing age, but did not examine how CHD risk prediction might be improved among older adults. For example, the C-index for the FRS was 0.63/0.66 in men/women aged 65-74 enrolled in the Cardiovascular Health Study [24] and 0.63 in a patient cohort with a mean age of 66 years [25], compared to 0.79/0.83 in men/women enrolled in the Framingham Heart Study (mean age of 49 years) [24]. Performance of the FRS may be worse in the very old, with a C-index of 0.53 in adults aged 85 years or older [26]. In different ethnic populations in the US and other countries, FRS often overestimates CHD risk Abbreviations: FRS: Framingham risk score; CHD: coronary heart disease; coef: coefficient; CI: confidence interval; SD: standard deviation; TC: total cholesterol; HDL-C: high-density lipoprotein cholesterol. *Some of the Framingham risk factors categories were collapsed to avoid cells with limited numbers of events and /or unpredictive trends. Total cholesterol and age 2 were omitted because they were unpredictive in these older women. The proportionality assumption was tested using the Therneau and Grambsch statistics, which is based on the Schoenfeld residuals. The assumption was accepted (p = 0.14).
{Based on Wilson et al. [2]. {Cholesterol categories proposed by the National Cholesterol Education Program [24]. I Blood pressure categories: Optimal (Systolic,120, Diastolic.80); Normal (Systolic,130, Diastolic.85); High normal (Systolic,140, Diastolic.90); Stage I (Systolic,160, Diastolic,100); Stage II-IV (Systolic $160, Diastolic $100) [24]. "Estimated from the Framingham adjusted survival rate (survival rate at the mean value of the risk factors) at 10 years: S 0 (10) = 0.96246 [2], as: Ŝ 0 (7.5) = S 0 (10) 0.75 = 0.9717 (exponential model). #Kaplan-Meier survival function at t = 7.5 years on HABC data, similar to reference [24]. **Adjusted survival rate at t = 7.5 years obtained on the HABC cohort as the baseline survival functions of the multivariate Cox model, similar to reference [9]. {{After bootstrap correction for the optimism (1000 bootstrap samples from the original dataset [20]), c-index = 0.564 (p = 0.54 for comparison with Framingham function). I I Adaptation to the Cox model of the Hosmer-Lemeshow test of goodness of fit [33], comparing observed and expected failures within deciles of predicted risk. Larger p values indicate better calibration [21]. doi:10.1371/journal.pone.0034287.t002 [9,24,27]. Recalibration of the FRS was shown to improve the estimation of absolute risk in these different ethnic populations [9,24]. In the present analysis among older adults, the FRS underestimated absolute CHD risk, particularly in women. Although recalibration of the FRS yielded a better estimation of absolute risk, the function specific to the Health ABC cohort yielded the best estimation of absolute risk, becoming statistically acceptable. Compared to recalibration among other ethnic groups [9,24], the recalibrated FRS showed worse risk prediction in our study of older adults. Our results indicate that the FRS not only underestimates CHD risk in older adults but that some traditional risk factors, such as total and LDL-cholesterol, have weaker associations with CHD risk in older adults, as previoulsy found [7]. In particular, total cholesterol did not predict CHD events in older women in our present study.
Our study has several strengths and limitations. These data are drawn from a well-characterized population-based cohort of older adults, with a high number of CHD events over a 8-year follow-up Abbreviations: FRS: Framingham risk score; CHD: coronary heart disease; coef: coefficient; CI: confidence interval; SD: standard deviation; TC: total cholesterol; HDL-C: high-density lipoprotein cholesterol. *Some of the Framingham risk factors categories were collapsed to avoid cells with limited numbers of events and /or unpredictive trends. The proportionality assumption was tested using the Therneau and Grambsch statistics, which is based on the Schoenfeld residuals. The assumption was accepted (p = 0.33).
{Based on Wilson et al. [2]. {Cholesterol categories proposed by the National Cholesterol Education Program [24]. I Blood pressure categories: Optimal (Systolic,120, Diastolic.80); Normal (Systolic,130, Diastolic.85); High normal (Systolic,140, Diastolic.90); Stage I (Systolic,160, Diastolic,100); Stage II-IV (Systolic $160, Diastolic $100) [24]. "Estimated from the Framingham adjusted survival rate (survival rate at the mean value of the risk factors) at 10 years: S 0 (10) = 90015 [2], as: Ŝ 0 (7.5) = S 0 (10) 0.75 = 0.9241 (exponential model). #Kaplan-Meier survival function at t = 7.5 years on HABC data, similar to reference [24]. **Adjusted survival rate at t = 7.5 years obtained on the HABC cohort as the baseline survival functions of the multivariate Cox model, similar to reference [9]. {{After bootstrap correction for the optimism (1000 bootstrap samples from the original dataset [20]), c-index = 0.580 (p = 0.90 for comparison with Framingham function). I I Adaptation to the Cox model of the Hosmer-Lemeshow test of goodness of fit [33], comparing observed and expected failures within deciles of predicted risk. Larger p values indicate better calibration [21]. doi:10.1371/journal.pone.0034287.t003 period, and included a larger sample of black older adults compared to previous studies [24]. CHD events were formally adjudicated. The cohort included both white and black older adults, but did not include other ethnic groups. After stratification by gender, our power for subgroup analyses was limited for comparisons between whites and blacks. Lower performance of the FRS might partly be related to ascertainment of CHD events limited to those requiring hospitalization in the Health ABC, but not in the Framingham cohort [2]. However, all our comparisons in the present data examined CHD outcomes limited to those requiring hospitalization; we also found similar associations for hard CHD events (nonfatal myocardial infarction or coronary death).
What are the potential clinical and research implications of these findings? Clinicians should use the FRS with caution in older adults, as it underestimates the absolute CHD risk by 51% in women and 8% in men and does not discriminate effectively between those who will have CHD events and those who will not. We could not identify additional, routinely available variables that might improve risk prediction beyond traditional risk factors comprising the FRS, similar to several previous studies that did not clearly identify factors improving risk prediction of the FRS [28]. Re-estimated risk functions using these factors improve accurate estimation of absolute risk, but did not meaningfully improve discrimination, or the ability to distinguish between low, intermediate, and high-risk adults. Substantial improvements in discrimination may require novel CHD risk markers or other strategies for risk prediction in the elderly. We have previously found that ankle-arm index and interleukin-6, but not highsensitive C-reactive protein, improved risk prediction beyond traditional risk factors, but only modestly [17]. Other potential markers that might improve CHD risk prediction in the elderly include homocysteine [26] or coronary calcification [29]. Future investigations should examine whether markers of atherosclerosis [29] or novel CHD risk markers [30] might improve risk prediction beyond FRS in older adults, which still requires additional studies [31]. For current clinical use, recalibrated Framingham functions seem an attractive option to better assess absolute CHD risk for older adults (Methods S1), given that no currently available new risk factors have been clearly and consistently shown to improve CHD risk prediction [28] and that the Health ABC function needs to be externally validated in another cohort.
In summary, our study suggests that the FRS underestimates CHD risk in the growing population of elderly [8], particularly in older women. However, traditional risk factors remain the best predictors of future CHD events. Recalibrating risk functions in older adults is important to improve the accuracy of absolute CHD risk estimates, especially for women, and might be useful to better identify older individuals at increased risk who will benefit from preventive therapies, such as statins or aspirin. However, substantial improvements in discrimination may require novel CHD risk markers or other strategies for better CHD risk prediction and risk stratification in the elderly. Methods S1