Validation of the MAGGIC (Meta-Analysis Global Group in Chronic Heart Failure) heart failure risk score and the effect of adding natriuretic peptide for predicting mortality after discharge in hospitalized patients with heart failure

Background In clinical practice, a risk prediction model is an effective solitary program to predict prognosis in particular patient groups. B-type natriuretic peptide (BNP)and N-terminal pro-b-type natriuretic peptide (NT-proBNP) are widely recognized outcome-predicting factors for patients with heart failure (HF).This study derived external validation of a risk score to predict 1-year mortality after discharge in hospitalized patients with HF using the Meta-analysis Global Group in Chronic Heart Failure (MAGGIC)program data. We also assessed the effect of adding BNP or NT-proBNP to this risk score model in a Korean HF registry population. Method and results We included 5625 patients from the Korean acute heart failure registry (KorAHF) and excluded those who died in hospital. The MAGGIC constructed a risk score to predict mortality in patients with HF by using 13 routinely available patient characteristics (age, gender, diabetes, chronic obstructive pulmonary disorder (COPD), HF diagnosed within the last 18 months, current smoker, NYHA class, use of beta blocker, ACEI or ARB, body mass index, systolic blood pressure, creatinine, and EF). We added BNP or NT-proBNP, which are the most important biomarkers, to the MAGGIC risk scoring system in patients with HF. The outcome measure was 1-year mortality. In multivariable analysis, BNP or NT-proBNP independently predicted death. The risk score was significantly varied between alive and dead groups (30.61 ± 6.32 vs. 24.80 ± 6.81, p < 0.001). After the conjoint use of BNP or NT-proBNP and MAGGIC risk score in patients with HF, a significant difference in risk score was noted (31.23 ± 6.46 vs. 25.25 ± 6.96, p < 0.001).The discrimination abilities of the risk score model with and without biomarker showed minimal improvement (C index of 0.734 for MAGGIC risk score and 0.736 for MAGGIC risk score plus BNP or NT-proBNP, p = 0.0502) and the calibration was found good. However, we achieved a significant improvement in net reclassification and integrated discrimination for mortality (NRI of 33.4%,p < 0.0001 and IDI of 0.002, p < 0.0001). Conclusion In the KorAHF, the MAGGIC project HF risk score performed well in a large nationwide contemporary external validation cohort. Furthermore, the addition of BNP or NT-proBNPto the MAGGIC risk score was beneficial in predicting more death in hospitalized patients with HF.


Introduction
Heart failure (HF) imposes a great health problem worldwide. The associated risks of HF are particularly vulnerable in countries with aging populations where diagnosis, treatment, and prevention of re-hospitalization are difficult [1]. Hospitalized patients with HF have poor prognosis, but re-hospitalization rates are increasingly, consequently elevating the in-hospital and post-discharge mortality rates. The prognosis of HF remains poor, and repeated hospitalization exerts a huge cost on national health care budgets and threatens quality of life [2][3][4][5]. Therefore, taking urgent measures to predict a patient's clinical course as early as possible is necessary by selecting evidence-based management strategies to improve care of patients with HF. To date, risk stratification in these patients is a challenge [6].
Risk prediction models are frequently used to triage patients and ease treatment decisions, helping physicians estimate prognosis in a more neutral manner and interpret the consequence of prognosis studies [7,8].Among all the datasets, the most eminently prognostic information could facilitate appropriate implication of monitoring and treatment that improves the standard of nursing and outcomes of hospitalized patients with HF [7]. Several predictors are applied in a prediction model to assess the risk of occurrence of a specified event (death, rehospitalization) in the future [9].
Recently, the Meta-analysis Global Group in Chronic Heart Failure (MAGGIC) has performed a literature-based meta-analysis by collecting each patient data from 30studies including six randomized clinical trials. These data consist of important prognostic variables, such as demographics, medical history, medical treatment, symptom status, clinical variables, laboratory variables, and outcome [10,11]. Hence, a risk score was established to predict mortality in patients with HF from 13 regularly available patient characteristics at a baseline level, and an easily accessible online calculator was made available (www.heartfailurerisk.org).Tracing risk stratification depicts a prospective outcome and is beneficial for patients with advanced HF (AHF) of multiple etiologies. Meanwhile, risk modeling serves as a comprehensive and conducive assessment for clinicians and plays a facilitating role to patients and health service provider to achieve better recognition of likely outcomes [12].MAGGIC also provides an advantage for determining numerous individual risk factors, such as serum sodium, gender, and survival of patients with HF with preserved or reduced LVEF [13][14][15].However, B-type natriuretic peptide(BNP), a widely recognized outcome-predicting factor for patients with HF, has not been included in a risk prediction model, as it was only available in less than 25% of the enrolled patients [16,17].BNP and N-terminal pro-b-type natriuretic peptide (NT-proBNP) are valuable prognostic predictors for grading severities and predicting mortality risk in patients with HF [18,19]. Measurement of serum BNP is essential in guiding decision-making process and predicting patient status, as well as in establishing therapeutic strategies [20][21][22]. Thus, considering a major risk factor in a risk scoring system is necessary to predict important outcome. Although BNP and NT-pro BNP are of accurate prognosis-predictive ability for hospitalized patients with HF, other clinical factors may also play pivotal roles in affecting outcomes [23].
An established, simple, cost-effective model with the highest accuracy, which consists of selective number of predictors, is significant for use in other populations [8]. MAGGIC risk score in HF was previously conducted and validated in a Swedish population, and an excellent discrimination was obtained [9]. Nevertheless, the real-world performance of prediction models for new patient populations and original advancing target populations including Asians is not consistent [24]. Therefore, in this study, we aimed to derive and externally validate a risk score to predict 1-year mortality after discharge in hospitalized patients with HF using the MAGGIC program data. We also assessed the effect of adding BNP or NT-proBNP to this risk score model in a Korean heart failure registry population.

Study population
The Korean Acute Heart Failure (KorAHF) registry is a prospective multicenter cohort study designed to measure the outcomes of Korean patients admitted for HF. The registry data consist of patient demographics, clinical characteristics, and evidence-based treatments [3,25]. Patients who had signs or symptoms of HF and met one of the following criteria were eligible for this study: (1) lung congestion or (2) objective findings of left ventricular systolic dysfunction or structural heart disease. Patients hospitalized for AHF from 10 tertiary university hospitals throughout the country were consecutively enrolled from March 2011 to February 2014. The patients were planned to be followed up until 2018. Data were collected by each site and entered into a webbased case-report form in the Clinical Data Management System from the Korea National Institute of Health. Information about patient demographics, medical history, signs, symptoms, results of laboratory tests, electrocardiogram, echocardiography, medications, hospital course, and outcomes was collected at admission, discharge, and follow-up (30days, 90days, 180days, 1to 5years annually).Written informed consent was obtained from each patient. If patients were unable to give consent due to disease severity, informed consent was obtained from a relative or legal representative. In-hospital mortality and the mode of death have been adjudicated by an independent event committee. The mortality data for patients lost to follow-up were collected from the national insurance data or national death records. The study protocol was approved by the ethics committee/institutional review board (IRB) at each hospital.

Measurement of BNP or NT-proBNP
Plasma BNP or NT-proBNP levels were measured at admission for acute HF. Blood sampling and tests were conducted as routine practice by laboratories at each center certified by the Korean Association of Quality Assurance for Clinical Laboratories [26]. Measurement of NT-proBNP was performed with the electro-chemiluminescence immunoassay method using an Elecsys 2010 analyzer (Roche Diagnostics) or NT-proBNP assay for Dimension platform, Siemens Medical Solutions Diagnostics. Plasma was tested for BNP using the Biosite Triage assay, a point-of-care device that uses a fluorescence immunoassay technique (Biosite, San Diego, CA, USA) [27].

Statistical analysis
Descriptive statistics are used to summarize demographic and clinical characteristics, clinical care during hospitalization, and outcomes of the patients. Continuous variables are presented as mean ± standard deviation (SD) and were analyzed with Student's t-test to demonstrate the statistical significance of the difference between two groups. Categorical variables are presented as frequency (percentage) and were analyzed with Chi-square test. The logistic regression model was applied to verify predictors of all cause mortality. Score discrimination and calibration were evaluated by the c statistic and Hosmer-Lemeshow statistic [9,11]. To test the incremental prognostic utility of B-type natriuretic peptide level (BNP) or N-terminal pro-Btype natriuretic peptide (NT-proBNP), we used the AUC comparison as DeLoog, net reclassification index (NRI) and integrated discrimination improvement (IDI). P-value less than 0.05 were considered statistically significant. All analyses were performed using SAS statistical software (version 9.4; SAS Institute Inc., Cary, NC).

Demographic characteristics and clinical profiles
We enrolled 5,625 AHF subjects from 10 tertiary university hospitals in Korea. We excluded 269 patients who died in hospital, 784 patients with missing patient characteristics, and patients without death information. Finally, we analyzed 4572 patients including 1623 (35.5%) dead and 2949 (64.5%) alive (Fig 1).
Descriptive statistics for patients who are still alive and patients who died during follow-up are summarized in Table 1.The mean age was68.42±14.52 years, and 2392(52.32%)were men. The overall 1-year mortality in the study population was 17.58%. Among the 13 variables, age, creatinine (Cr), smoker and diabetes (41.53%),COPD(14.17%), HF duration > 18 month (52.37%) and NYHA 4(53.73%) as well as not taking medications (ACEI or ARB and beta blocker) were significantly higher in patients who died during follow-up compared with those who are alive. Additionally, body mass index (BMI)was almost similar between the 2 groups. However, male gender, systolic blood pressure (BP), and ejection fraction did not differ significantly between the two groups. High plasma BNP(>887.9) or NT-proBNP (>4751.05) according to the median value was significantly higher in the mortality group (61.31% vs 44.63%, p<0.0001), whereas low BNP or NT-proBNP was higher in the alive group (38.69% vs 55.37%, p<0.0001).

Predictors of mortality
In multivariable analysis, 11 variable included in the MAGGIC score were associated with mortality. However, when BNP or NT-pro-BNP were associated with MAGGIC. ejection fraction could not predict mortality ( Table 2).Here 11variables(age, male sex, BMI,SBP, diabetes, HF duration > 18 months, NYHA(2-4),Cr, ACEI or ARB, and beta blocker, along with high BNP or NT-proBNP) were independent predictors in multivariable analysis (Table 2). However, smoker and COPD were non-significantly associated with mortality in both groups. The risk score was significantly different between alive and dead groups (30.61 ± 6.32 vs. 24.80 ± 6.81, p<0.001). After the conjoint use of BNP or NT-proBNP and MAGGIC risk score in patients with HF, the risk score was also significantly different (31.23 ± 6.46 vs. 25.25 ± 6.96, p<0.001) ( Table 3).There were strongly significantly difference between them [0.61 (0.59~0.64) for death group and 0.45 (0.43~0.47) for alive group].

Validation of the risk score models
In this study, we selected 13 predictor variables from the multivariable model to formulate a risk score. Good discrimination abilities were found,with a C-indexof 0.734 from the risk score model to predict 1-year mortality. Hosmer-Lemeshow goodness of fit test for risk score mortality indicated a good calibration (p = 0.5559) by plotting predicted versus observed mortality using 10 groups (Fig 2). Additionally, the predicted mortality rates corresponded with the observed mortality rates in each decile.

Effect of adding natriuretic peptide
The addition of BNP or NT-proBNP to the MAGGIC risk score using the 13 variables resulted in minimal improvement(C index of 0.734 for MAGGIC risk score and 0.736 for MAGGIC risk score plus BNP or NT-proBNP, p = 0.0502).However, we achieved a significant improvement in net reclassification and integrated discrimination for mortality (NRI of 33.4%,p <0.0001 and IDI of 0.002, p <0.0001) ( Table 4).

Discussion
This study demonstrates the performance of the MAGGIC project HF risk score using 13 variables by external validation to predict 1-year mortality after discharge in Korean hospitalized patients with HF. We found that the discrimination and calibration were good for the MAG-GIC risk score. This risk score is useful to predict mortality after discharge for hospitalized patients with HF. Furthermore, the addition of BNP or NT-proBNP to the MAGGIC risk score showed significant improvement in risk reclassification of patients and minimal improvement in discrimination ability.
HF is a complex and fatal medical condition that progresses rapidly with escalating aging population and causes considerable morbidity, mortality, and re-hospitalization, resulting in a tremendous burden on the global healthcare system [28][29][30]. Physicians always need to determine the best therapy for high-risk patients and make estimation of risk in patients by integrating patient characteristics, clinical signs, and laboratory tests [8].Prediction is naturally inconsistent. Although a physician assigns a relative weight to each variable, which relies on his previous experiences, personal beliefs, clinical judgment, as well as current mood, it could be inaccurate and ambiguous. To determine an individual's cardiac mortality risk score in daily clinical practice, adding points assigned for predictors that exist in a patient is only required. For this reason, the MAGGIC risk score is beneficial for busy clinicians and nurses. Using the 13 available baseline variables, including age, EF, NYHA class, serum Cr, diabetes, systolic BP, BMI, HF duration, current smoker, COPD, male gender, evidence-based medication beta blocker, and ACE inhibitor or ARBs, can construct the risk score and identify the risks of hospitalized patients easily at low cost. In this MAGGIC score, each variable was independently associated with mortality, except COPD and smoker. Several risk stratification models that are used to predict mortality in hospitalized patients with HF have limitations. For example, the Seattle Heart Failure Model and the CHARMmodel use not readily available variables. The former requires lymphocyte count, uric acid, and total cholesterol, and the latter requires atrial fibrillation, cardiomegaly, and pulmonary edema. Meanwhile, in the HF-ACTION predictive risk score model, inclusion of exercise duration on the baseline cardiopulmonary exercise test and serum urea nitrogen, which require difficult estimations, is necessary [31][32][33].Moreover, in assessing long-term mortality, other predictive models are used, but included participants from clinical trials. The models also rely on not readily available variables in clinical care, such as peak exercise oxygen consumption for the Heart Failure Survival Score model; and hemoglobin, GGT, eGFR, 12-lead ECG, and 24-h Holter monitoring variables including non-sustained VT and frequent VPBs, for the MUSIC risk score model [34,35]. Although these predictive models are informative and important for research purposes, their drawbacks limit their use in daily clinical practice. By contrast, the MAGGIC project HF risk score has several advantages and is superior to other HF mortality risk models. Itis targeted for a great number of population-based patients with manifold demographic characteristics regardless of LV systolic function and can be used for nationwide contemporary external validation cohort. Thus, this model is extensively pertinent and appealing as it contains nearly a modest number of variables regularly collected during admission [9]. MAGGIC risk score can be calculated by entering admission data into an online risk calculator (www.heartfailurerisk.org).This calculator can estimate1-year all-cause mortality for patients with HF. It is not only easily understandable and beneficial to physicians, but also helpful for patients and relatives [11]. The MAGGIC projectincluded39,372 patients from 30 studies. Of these studies, 6 were randomized controlled trials (24,041 patients) and the remaining 24 were registries (15,331 patients).Approximately 40% of patients died during a median follow-up of 2.5 years.The six largest studiescontributed76% of patients and 76% of deaths [11]. Recently, another study performed the MAGGIC risk score in a Swedish population and found excellent discrimination. However, good calibration was not found as moderate increase in low-risk patients and depreciation of high-risk patients accounted for the 3-year mortality risk [9]. In this present study, we observed good discrimination and calibration for MAGGIC risk score in hospitalized patients with HF derived from the KorAHF after discharge. Discharge is often the first step of  treatment progress and initial assessment for prediction of prognosis. We adopted the risk score to patients with HF after discharge from hospital. This study is the first to use the MAGGIC risk score of patients with HFin a population in Korea, which is part of East Asia. Some differences exist in the baseline clinical characteristics and co-morbidity between Western and East Asian countries [36].Several studies reported that the analytical histories of IHD and hypertension in Koreawere45.4% and 43.6%, respectively, which were considerably lower than in the high-income Western nations [37]. On the contrary, the prevalence of hypertension as a co-morbidity of patients with HF is widely varied (39.2% to 77.6%) in Japan than in other studies, such as in the Acute Decompensated Heart Failure Syndromes registry (71%),the Organized Program to Initiate Lifesaving Treatment in Hospitalized Patients with Heart Failure (71%),and the Euro-Heart Failure Survey II (63%) [38][39][40][41]. Although the prevalence of HF is lower in Korea (1.53%) compared with that in the USA (2.2%) or other Western and European nations, the increase in aging population and adoption of a Western life style promote the growth of HF prevalence in East Asian countries, including Korea, China, and Japan [37,42].With gender consideration, male is the most classic demographic risk factor for the development of HF, along with older age, ethnicity, and low socioeconomic state, in East Asia [43].The prevalence of overweight/obesity is much lower(21.6-26.2%) in Southeast Asia than in the UK (66.7%) and USA (69.6%). However, the recent prevalence of blood glucose/diabetes as a notable risk factor is higher in Asian countries than in the UK and USA [44].During the follow-up period, the clinical effect of evidenced-based medication between Western and Asian countries was also different [45]. Therefore, establishment of MAGGIC HF risk score model is not only important for Korean population but also for other Asian countries.
BNP and NT-proBNP are the most powerful prognostic factors; they are expected to serve as an important basis for diagnosis and play a crucial role in therapeutic decision-making process [22]. High plasma BNP or NT-proBNP at admission predicts greater risk of outcomes in hospitalized patients with HF [22,46]. On the contrary, we added BNP or NT-proBNP to the MAGGIC risk score and found minimal improvement in discrimination (C index of 0.734 for MAGGIC risk score and 0.736 for MAGGIC risk score plus BNP or NT-proBNP, p = 0.0502). In multivariable analysis, we observed that high natriuretic peptide level was significantly associated with mortality. Furthermore, when we used new statistical analysis methods (NRI and IDI indices), BNP or NT-proBNP (NRI of 33.4%, p <0.0001 and IDI of 0.002, p <0.0001) showed the greatest increase in discrimination and net reclassification for mortality. Based on this study, we suggest that the validated risk score is readily generalizable, and the addition of BNP or NT-proBNPto the MAGGIC risk score was beneficial in predicting more death. Therefore, a quick and cost-effective important indicator is established to anticipate adverse events in patients with HF. This indicator supports the performing cardiologist and referring clinician to take direct care and improve communication with patients with HF about their prognosis [46].

Study limitations
This cohort study has several limitations. First, its design had several inherent limitations including selection bias and missing values. Second, we did not use MAGGIC risk score inhospital patients with unstable health condition during admission. Thus, we excluded in-hospital mortality, and analysis was considered after discharge of patients. Third, although in-hospital mortality and post-discharge mortality were confirmed by an independent event committee, causes of post-discharge mortality are unknown. We could not differentiate cardiovascular mortality and causes of re-hospitalization, which were not validated. Fourth, we did not perform the specific categories of BNP or NTproBNP levels to be widely used as a prognostic score. We could not also validate and compare MAGGIC risk score with other risk score such as 3C-HF score which is widely used to predict 1-year mortality using routinely available predictors. Finally, we evaluated short-term mortality (1-year mortality), and the biomarker natriuretic peptide(BNP or NT-proBNP)was only used in this risk score. The measurement time and median value of BNP and NT-proBNP were different.

Conclusions
In the KorAHF, the MAGGIC project HF risk score performed well in a large nationwide contemporary external validation cohort. Furthermore, addition of BNP or NT-proBNPto the MAGGIC risk score was beneficial in predicting more death in hospitalized patients with HF.