Framingham Ten-Year General Cardiovascular Disease Risk: Agreement between BMI-Based and Cholesterol-Based Estimates in a South Asian Convenience Sample

The goal of this analysis was to determine the agreement between body mass index-based and cholesterol-based ten-year Framingham general cardiovascular disease risk scores among a convenience sample of 773 South Asian Canadian adults attending community-based screening clinics. Scores were calculated using age, systolic blood pressure, antihypertensive use, current smoking, diabetes, and total cholesterol and high density lipoprotein (for cholesterol-based risk) or height and weight (for body mass index-based risk). Mean risk score differences (body mass index-based risk minus cholesterol-based risk) were estimated using paired t-tests. Bland-Altman plots were used to assess agreement between scores. Finally, agreement across risk categories (low [<10%], moderate [10% to <20%], high [> = 20%]) was examined using the kappa statistic. Average agreement between the two risk scores was quite good overall (mean differences of 0.6% for men and 0.5% for women), but increased to about 3% among participants 60–74 years of age. However, Bland-Altman plots revealed that the differences between the two scores and the variability of the differences increased with increasing average 10-year risk. In terms of clinical importance, the limits of agreement were reasonable for women < 60 years (95% confidence interval: -3.2% to 3.1%), but of concern for women 60-74 years (95% confidence interval: -6.0% to 12.3%), men < 60 years (95% confidence interval: -7.1% to 7.3%) and men 6-074 years (95% confidence interval: -13.8% to 18.8%). Agreement across categories was moderate for most sex and age groups examined (kappa values: 0.51 for women < 60 years, 0.50 for women 60-74 years, 0.65 for men < 60 years), except for men 60-74 years, where agreement was only fair (kappa = 0.26). In light of these disagreements, evaluation of a participant’s change in cardiovascular disease risk over time will necessitate use of the same risk score (i.e., either body mass index-based or cholesterol-based) at all screening sessions.


Introduction
Canada Research Ethics Board (Protocol Number 2011-0034). All participants provided written informed consent.

Study Methods
The University of Calgary research team was approached by SA community leaders to implement a national program similar to three local projects that addressed CVD risk reduction [8,12,13]. The program methodology was developed collaboratively, incorporating the principles of community based participatory research [14]. At a two-day national training workshop, the research team met with the regional core teams (i.e., project managers and clinic leads) from six cities across Canada. Using a standardized, culturally-adapted training manual, the research team provided information on CVD and its major risk factors, reviewed the research methodology and screening clinic procedures, and provided hands-on training for: collection of participant health histories, measurement of height and weight, calculation of body mass index (BMI), assessment of blood pressure (BP) and capillary total and HDL cholesterol, Framingham risk score calculation, and participant counseling and referral. Using a "train the trainer" approach, the core teams returned to their respective cities to provide the same training to their local volunteer teams. All volunteers, a mix of health professionals and lay community members, were also members of the participating SA communities.

Study Population
From February through August 2012, 792 predominantly SA participants (mostly originating from East Africa, India, Pakistan, and Central Asia), 30-74 years of age with no prior history of CVD, underwent cholesterol-based screening at community-based clinics. Ethnicity was not directly assessed in the study but we estimate that 98% of the participants were of SA origin (personal communication, N. Surani, May 9, 2014).

Lifestyle and Health Data
Participants completed a brief health history questionnaire which assessed basic demographic data and self-report of: a prior history of heart disease or stroke; a physician diagnosis of, or prescription for, hypertension, high cholesterol, and/or diabetes; current use of any medication (s) for blood pressure, cholesterol and/or blood sugar; and current smoking status. They then underwent measurement of height, weight, blood pressure (BP) and random capillary total cholesterol (TC) and high density lipoprotein (HDL). Height and weight were measured using rigid tape measures and simple bathroom scales, purchased locally by each team. BP was measured using the BPTru (model BPM 100; VSM MedTech Ltd.), an automated device that yields a more reliable and accurate assessment of resting BP (when compared with 24-hour ambulatory measurement) than a manual measurement taken with a stethoscope and mercury sphygmomanometer [15,16]. To reduce clinic waiting times, resting BP was taken using a modification of the Canadian guidelines [17]. The BPTru monitor was set to take six readings at one minute intervals. If the first reading was normal [systolic and diastolic pressures < 140 mmHg and < 90 mmHg, respectively (if not diabetic), or < 130 mmHg and < 80 mmHg, respectively (if diabetic)], the first reading was recorded as the screening BP. Otherwise, the first reading was discarded and the average of the last five readings was recorded (as per the usual Canadian guidelines). TC and HDL were measured using the Cholestech point-of-care desktop reflometer (Hayward, CA, USA), which delivers results that correlate well with laboratory results from venous blood samples [18,19]. Ten-year CVD risk scores were then calculated and explained to participants by volunteer health professionals (licensed and practicing registered nurses, pharmacists or physicians).

Statistical Analyses
Questionnaire and measured health data were summarized using counts and percentages (categorical data), means and standard deviations (normally distributed continuous variables) and medians, ranges and interquartile ranges (non-normally distributed continuous variables). While the clinic volunteers calculated CVD risk using the paper-based charts published by D'Agostino et al [11], for the purposes of this analysis, we calculated exact cholesterol-based and BMI-based risk scores using the risk functions derived from gender-specific Cox proportional-hazards regression models published in the same paper. The models, based on risk factors and outcomes ascertained from the original Framingham Heart Study (1968-71) and the Framingham Offspring Study (1971-75 and 1984-87), include age, systolic BP, antihypertensive use, current smoking, diabetes, and TC and HDL (for cholesterol-based risk) or BMI (for BMI-based risk). The risk functions yield an estimate of 10-year absolute CVD risk, including coronary death, myocardial infarction, coronary insufficiency, angina, ischemic stroke, hemorrhagic stroke, transient ischemic attack, peripheral artery disease and heart failure.
For each participant, the difference between the BMI-based risk estimate and the cholesterol-based risk estimate was determined (BMI-based risk minus cholesterol-based risk). Mean risk score differences and 95% confidence intervals (CIs) were estimated separately for men and women, and by the Framingham age categories used in the risk calculations, using paired t-tests.
The differences were then examined using Bland-Altman plots [20]. In a Bland-Altman plot, for each participant, the difference between the two scores is plotted against their mean. As a participant's true CVD risk is unknown, the mean of her/his BMI-based and cholesterolbased scores is the best estimate available. The 95% limits of agreements (i.e., the mean difference between the scores +/-two standard deviations of the differences) were then estimated to provide an interval within which 95% of the differences between the two scores would be expected to lie [20].
Given that the Framingham risk functions that we used do not account for use of cholesterol medications, we repeated the analysis after excluding the 91 participants who reported hypercholesterolemia treatment. All analyses were conducted using Stata/IC 12.0 (College Stn, TX).

Participant Characteristics
Of the 792 participants who underwent cholesterol-based risk assessment, 773 (98%) had their height and weight measured, permitting comparison of cholesterol-based CVD risk with BMIbased CVD risk. Four hundred one women (52%) and 372 men (48%) 30-74 years of age (mean: 48.9 years) were included in this analysis. The majority of participants were not born in Canada, but of those, about half had lived in Canada for longer than 20 years (Table 1). Ninety two percent selected English as their language of choice during screening. Twenty nine percent of all participants reported a first degree family history of CVD in one or more relatives before the age of 60 years. The prevalence of self-reported hypertension (physician-diagnosed) was the same among men and women (about 15%), and of those reporting the condition, 71% of women and 67% of men were being treated with medication(s). The prevalence of self-reported diabetes was similar among women and men (7.2% overall), but of those affected, women were less likely than men to report current treatment (71% vs. 92%). Women were less likely than men to report hypercholesterolemia (18% vs. 30%) and slightly less likely to report treatment if affected (45% vs. 52%). Smoking was rare overall, but more common among men than women (4% vs. 2%). On average, measured BP was slightly higher among men than women, while TC, HDL and BMI were slightly lower. Median 10-year BMI-based and cholesterol-based CVD risk scores were similar for women (5.1% and 5.2%, respectively), though the variability in BMIbased scores was higher, as evidenced by the overall and interquartile ranges. A similar trend was noted for men, who had median risk scores approximately double those of the women (12.4% for BMI-based risk and 11.7% for cholesterol-based risk).

Mean Differences in Risk Scores
The average differences between BMI-based and cholesterol-based scores for women and men were 0.6% (95% CI: 0.3% to 0.8%) and 0.5% (95% CI: 0.0% to 1.0%), respectively. When examined by the age categories used in the Framingham calculations, the average differences were larger for both women and men in the three youngest age groups (30-39, 40-49 and 50-59 years) compared with those in the two older age groups (60-69 years and 70-74 years). Accordingly, the average difference was calculated for two age groups: < 60 years and 60-74 years. For women, the mean difference between the scores for those < 60 years was-0.1% (95% CI: -0.3% to 0.1%) compared with 3.1% (2.1% to 4.2%) for those 60-74 years. For men, the same pattern was observed. Mean differences were 0.1% (95% CI: -0.3% to 0.5%) among those < 60 years and 2.5% (95% CI: 0.4% to 4.5%) among those 60-74 years.

Bland-Altman Plots / Limits of Agreement
While these differences suggest that BMI-based and cholesterol-based scores agree well on average, they offer no insight into our primary goal of determining whether or not the two forms of the general CVD risk score yield similar estimates of overall risk for a given individual. Bland-Altman plots showing the agreement between the two risk scores for men and women by age group (<60 years, 60-74 years) are shown in Fig. 1. The limits of agreement shown on the plots (i.e., the mean difference +/-two standard deviations of the difference) provide estimates of the 95% range of agreement for individuals. For women, both the magnitude of the differences between the scores and the variability of the differences increase as the average 10year CVD risk increases. The limits of agreement are reasonable in terms of clinical importance for women < 60 years (-3.2% to 3.1%), but are a concern for those > = 60 years (-6.0% to 12.3%). Similar trends were noted for men, though the limits of agreement were wide from a clinical perspective for both age groups: -7.1% to 7.3% for those <60 years; -13.8% to 18.8% for those 60-74 years.

Categorical Agreement
Categorical agreement between the two risk scores is shown in Table 2 (women) and Table 3 (men). Overall agreement in risk category classification was 87% for women (kappa = 0.63, standard error (SE) = 0.04). While disagreements were observed in both directions, there was a tendency for BMI-based risk to be higher than cholesterol-based risk classifications. BMIbased risk categories were higher than cholesterol-based risk categories for 36 women (9%), but lower for only 18 (5%). Agreement across risk categories was 91% for women < 60 years and 69% for women 60-74 years, yielding similar kappa coefficients: 0.51 (SE = 0.05) and 0.50 (SE = 0.08), respectively. Among the younger women, disagreements were equally likely to occur in both directions. Relative to their cholesterol-based risk classifications, 15 (5%) had higher and 17 (5%) had lower BMI-based risk classifications. Among the older women, disagreements almost always reflected higher BMI-based risk; 24 (30%) had higher but only 1 (1%) had lower BMI-based risk classifications compared with their cholesterolbased classifications.
Overall agreement in risk category classification was 80% for men (kappa = 0.69, SE = 0.04). As with women, there was a tendency for BMI-based risk to be higher than cholesterol-based risk classifications. BMI-based risk categories were higher than cholesterol-based risk categories for 50 men (13%) and lower for only 26 (7%). Agreement across risk categories was 79% for men < 60 years (kappa = 0.65, SE = 0.04). Among the younger men, 39 (13%) had higher and 25 (8%) had lower BMI-based risk classifications relative to their cholesterol-based classifications. None of the men 60-74 years were classified as low risk by either scoring system. Overall agreement among the moderate and high risk categories was 81% (kappa = 0.26, SE = 0.10). Eleven (17%) classified as moderate risk according to their cholesterol-based measures were considered high risk based on their BMI-based scores, while 1 (2%) classified as high risk according to his cholesterol-based score was considered moderate risk based on his BMIbased score.

Discussion
In our convenience sample of largely SA participants in a national screening program, average agreement between BMI-based and cholesterol-based 10-year general Framingham CVD risk scores at the group level was quite good overall, with average differences of 0.6% for women and 0.5% for men. These increased to about 3% among participants 60-74 years of age. Bland- Altman plots illustrating agreement between scores for individuals, however, revealed that both the differences between the two scores and the variability of the differences increased as the average of the two scores increased. In terms of clinical importance, the limits of agreement were reasonable for women < 60 years (-3.2% to 3.1%), but of concern for women 60-74 years (-6.0% to 12.3%), men < 60 years (-7.1% to 7.3%) and men 60-74 years (-13.8% to 18.8%). Considered from the perspective of low (<10%), moderate (10% to <20%) and high (> = 20%) risk, which is what our participants were counseled on, agreement as assessed by the kappa statistic was moderate to good among all sex and age groups examined, except for men 60-74 years, where agreement was only fair [22]. Limiting the analysis to those not on medication(s) for hypercholesterolemia did not appreciably change the observed agreement. In our study, practical and budget considerations prevented us from standardizing the measurement of height and weight. Volunteers purchased bathroom scales and tape measures  locally, and it was not possible to calibrate the scales or use the same models across the country. Accordingly, both BMI measures and BMI-based risk scores for some individuals may have been under-or overestimated. This, in turn, may have increased or decreased the observed differences between their BMI-based and cholesterol-based risk scores. However, BMI measurement errors were likely randomly distributed across the country and were not likely related to the measurement of any other variables used in the risk score calculations. Therefore, while the specific impact of BMI measurement error on the agreement between the two scores is unknown, it is unlikely that it contributed to any systematic differences between the BMI-based and the cholesterol-based CVD risk scores. Practical considerations also led us to measure BP only once among participants for whom the first reading was considered normal, whereas we took the average of the final five BPTru readings for all others. While we may have overestimated BP among participants with a single reading (since BP readings tend to decrease with serial BPTru measures [23]), this overestimation would have had a similar impact on both CVD risk scores (i.e., the same BP was used in the calculation of an individual's BMI-based and cholesterol-based risk). So, while both risk scores may have been overestimated for these individuals, any systematic impact on the agreement between the two scores was unlikely.
Few publications have compared laboratory and non-laboratory based CVD risk scores, and to the best of our knowledge, none report on agreement among South Asians. D'Agostino et al [11] did not publish the agreement between the two scores that we used in our screening clinics. Using data from the First National Health and Nutrition Examination Survey (NHANES I) and the NHANES I Epidemiologic Follow-up Study (NHEFS) cohort in the United States, Gaziano et al [24] developed a laboratory-based model incorporating age, systolic BP, smoking status, TC, reported diabetes status, and current treatment for hypertension to predict first fatal and non-fatal cardiovascular events. They also developed a non-laboratory-based model, substituting BMI for TC. Both models demonstrated good predictive discrimination, but within-person agreement by risk category was not reported. A subsequent analysis [25] conducted in the NHANES III population compared the non-laboratory-based model developed in the NHANES I population with four laboratory-based risk scores [11,26,27]. With risk categories defined as low risk (< = 10%) versus high risk (>10%), agreement between the non-laboratory-based model and the Framingham score that we also used [11] was 93% for women and 92% for men, somewhat higher than the agreement we observed. This could in part be explained by the different risk categorizations used. Green et al [28] calculated the same two risk CVD risk prediction scores that we did using data from the electronic health records of a group health cooperative in the United States and found that risk categories (low, moderate and high) were concordant for 78% of patients, a finding comparable to ours. Similarly, BMI-based risk classifications also tended to be higher than cholesterol-based risk classifications. This was more pronounced than it was among our participants, with 20% and 1% of their patients having higher and lower BMI-based classifications, respectively, compared with cholesterolbased classifications. We selected the Framingham 10-year general CVD risk score [11] because it offered information on the global risk of developing any major atherosclerotic CVD event, included a number of risk factors that we were able to collect information on, and offered simple, paper-based calculation tables for both cholesterol-based and BMI-based risk assessment. We recognize that risk tools developed and validated in one population may not accurately predict CVD risk in another population due to secular, geographical and ethnic differences. While Aarabi et al [29] developed a modified Framingham-based coronary heart disease (CHD) risk score specifically for use among SAs, the adjusted calculations were deemed too complicated for volunteers in the community-based screening setting. Further, the tool is limited to the prediction of CHD. Other risk calculators that include ethnicity (such as QRISK2 [30]), adjust for ethnicity post-hoc (such as JBS2 [31]), or have been calibrated for use in SA populations (such as ETHRISK [32]) either include variables that we were unable to assess or were again deemed difficult for our volunteers to use. Perhaps most importantly, no published CVD risk prediction tool has been developed, validated or calibrated for any Canadian SA population. To do so would require population-based, longitudinal data on both risk factors and CVD events specific to SAs, data that are currently unavailable in Canada. We selected the two Framingham-based scores recognizing that they would provide general estimates of CVD risk, and we were careful to explain to participants that the extent to which the scores over-or underestimate risk for SAs in Canada is not known.
An important component of our screening program will be the evaluation of change in CVD risk scores from baseline to follow-up. Accordingly, we needed to ascertain whether or not the two scoring tools we used similarly classified participants. While differences between scores were acceptable at the aggregate level, our Bland-Altman plots and comparison of categorical agreement show that the two risk scores cannot be used interchangeably in our study population.
Further, in this analysis, we computed the two Framingham risk scores using the formulae as published [11]. In practice at our screening clinics, however, volunteers calculated less exact scores using the paper-based risk charts included in the same paper. We also employed simple recommendations in an effort to adjust risk scores for SA ethnicity by adding 10 years to each participant's age [29] and doubling the final risk score for those with a family history of premature CVD in a first degree relative before the age of 60 [33]. Adding 10 years to age would have a variable impact on the differences between scores, depending upon the participant's age. We noted that the average difference between the two scores was higher for older participants, so the calculated difference between scores for a given participant would depend upon her/his adjusted-age categorization in the paper-based risk calculation sheets. Doubling the final risk scores based on family history would double the observed difference between the two scores. For example, for a participant with a BMI-based risk score of 8% and a cholesterol-based risk score of 14%, the difference between the risk estimates is 6%. If both risks were doubled based on family history, the difference between the estimates would increase to 12%. This would be of little practical significance when original differences between the scores are small (i.e., plus/ minus 1 or 2 percent), but of substantial importance for participants with relatively large differences between BMI-based and cholesterol-based risks.
In light of the observed differences between the two risk scores as calculated, combined with the implications of adapting our scores, evaluation of a participant's change in CVD risk over time will necessitate use of the same risk score (i.e., either BMI-based or cholesterol-based) at all screening sessions. It will be important for both participants and volunteer counselors to understand that the CVD risk score is an estimate, and only and estimate, useful as part of a comprehensive risk factor assessment and risk reduction plan. As our SA community-based research partners consider the continuance of a simple and sustainable program, they will need to adopt a protocol that ensures consistent risk appraisal within individuals.

Conclusion
Despite acceptable average differences in BMI-based vs. cholesterol-based Framingham tenyear general CVD risk scores in our SA convenience sample, only moderate agreement between the scores was found when risk was categorized as low, moderate or high. While disagreements occurred in both directions, BMI-based risk was assessed as higher than cholesterol-based risk more often than it was found to be lower. Accordingly, the two risk scores cannot be used interchangeably in our national, community-based screening program. Valid assessment of change in participants' CVD risk over time will require that one or the other be used consistently within participants who attend repeated screening sessions. Evaluation of the accuracy of either risk score in our population will require population-based longitudinal cohort studies among SAs in Canada.