https://orcid.org/0000-0001-9475-0344
Figures
Abstract
The ability to accurately predict the one-year survival of older adults is challenging for clinicians as they endeavour to provide the most appropriate care. Standardised clinical needs assessments are routine in many countries and some enable application of mortality prediction models. The added value of blood biomarkers to these models is largely unknown. We undertook a proof of concept study to assess if adding biomarkers to needs assessments is of value. Assessment of the incremental value of a blood biomarker, Brain Naturetic Peptide (BNP), to a one year mortality risk prediction model, RiskOP, previously developed from data from the international interRAI-HomeCare (interRAI-HC) needs assessment. Participants were aged ≥65 years and had completed an interRAI-HC assessment between 1 January 2013 and 21 August 2021 in Canterbury, New Zealand. Inclusion criteria was a BNP test within 90 days of the date of interRAI-HC assessment. The primary outcome was one-year mortality. Incremental value was assessed by change in Area Under the Receiver Operating Characteristic Curve (AUC) and Brier Skill, and the calibration of the final model. Of 14,713 individuals with an interRAI-HC assessment 1,537 had a BNP within 90 days preceding the assessment and all data necessary for RiskOP. 553 (36.0%) died within 1-year. The mean age was 82.6 years. Adding BNP improved the overall AUC by 0.015 (95% CI:0.004 to 0.028) and improved predictability by 1.9% (0.26% to 3.4%). In those with no Congestive Heart Failure the improvements were 0.029 (0.004 to 0.057) and 4.0% (0.68% to 7.6%). Adding a biomarker to a risk model based on standardised needs assessment of older people improved prediction of 1-year mortality. BNP added value to a risk prediction model based on the interRAI-HC assessment in those patients without a diagnosis of congestive heart failure.
Citation: Pickering JW, Scrase R, Troughton R, Jamieson HA (2022) Evaluation of the added value of Brain Natriuretic Peptide to a validated mortality risk-prediction model in older people using a standardised international clinical assessment tool. PLoS ONE 17(11): e0277850. https://doi.org/10.1371/journal.pone.0277850
Editor: Alonso Soto, Universidad Ricardo Palma Facultad de Medicina Humana, PERU
Received: April 6, 2022; Accepted: November 3, 2022; Published: November 18, 2022
Copyright: © 2022 Pickering et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Because of restrictions of use of clinical data we cannot make the data set freely available. However, we can make it available if contacted subject to ethics approval from a New Zealand ethics board and a formal use agreement between ourselves and those who wish to use the data. For New Zealand data this is particularly important because some of the data is from Māori, the indigenous people of New Zealand, who are acknowledged to have sovereignty of their data and its use. The restrictions are imposed by the New Zealand Ministry of Health and Disability Ethics Committee (HDEC). Data access queries may be sent to the Department of Medicine, University of Otago Christchurch, Christchurch, New Zealand (contact via christchurch@otago.ac.nz).
Funding: Health Research Council of New Zealand grant 17/363 The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
At a global level the population is ageing and western countries in particular are doing so with increasing levels of comorbidities and complex health conditions [1]. To be able to provide timely and appropriate care for older individuals with complex conditions, clinicians need reliable mortality prediction tools to aid both clinical decision making and inform conversations with patients and family. In this way, wherever possible, common understanding can be reached about appropriate interventions and treatment options including making comprehensive future health care plans [2].
Reliable and accurate mortality prediction can be of significant value to clinicians when considering the risks and benefits of specific medications or when formulating individualised care plans for older adults with considerable variation in health status and disability [3–5]. The development of mortality tools which are more easily applied in the real-world environment and which can provide clinically meaningful information to guide mortality assessment have been facilitated by the evolution of standardised assessment tools and the accompanying big data [6]. The international Residential Assessment Instrument (interRAI) is one such suite of standardised assessment tools containing questions around demographics, function, comorbidities, and living conditions, and which is now used in over 35 countries and in a multitude of clinical settings [7,8]. Each tool is specific to a setting, e.g. long-term care facilities, or discipline e.g mental health, although a large proportion of the questions are identical. This enables monitoring of changes of status between assessments as the patient moves through their health journey and life course [9]. Although the primary purpose of any interRAI assessment is to provide a standardised assessment to create individualised care plans, they also provide a comprehensive dataset at a population level which can be valuable to aid research aimed at improving outcomes for our most vulnerable older population [10].
The interRAI-Home Care (interRAI-HC) is mandated for use throughout all of New Zealand with older adults requiring assessment for Government funded support or who are being considered for possible entry into aged residential care [8]. This means the individuals being assessed tend to have complex co-morbidities and at a population level are frailer and more vulnerable than their peers. This clinically vulnerable group is, therefore, the exact older cohort clinicians would most want to have a greater understanding of their morbidity trajectory, so that they can more appropriately support with care better tailored to the individual.
The authors recently developed and validated a mortality risk model (RiskOP) for those people aged 65 plus using data from the interRAI-HC only [11]. The RiskOp model predicts one-year mortality with good discrimination and excellent calibration.
The broad context for this research is the question of whether individual biomarkers, not currently part of most needs assessments, can improve risk prediction models that are based on standardised assessments in older adults. To answer this question, we undertook a proof of concept using a validated prediction model with good discrimination and calibration (therefore with less room for improvement than a poor one), RiskOp and with a biomarker associated with mortality across a variety of clinical settings, Brain Natriuretic Peptide (BNP) [12,13]. In western countries in particular heart failure is a leading cause of death among the older population [14]. BNP is a good predictor of mortality in adults both with and without a prior diagnosis of heart failure [13,15,16]. We also chose BNP because it is in frequent clinical use, it is inexpensive and it is clinically accessible. We hypothesised that BNP may add useful prognostic information to the RiskOP one-year mortality risk prediction model for older adults.
Methods
Design
We assessed the added value of BNP to the RiskOP one-year mortality prediction model which uses the interRAI-HC instrument [11]. The interRAI-HC is administered to older people to assess their need for home support or for entry to a long-term care facility. The inteRAI -HC assessment tool is administered by a trained health professional who is usually either a Registered Nurse or a Social Worker. These health professionals are required to attend a training course and are then need to complete yearly updates to ensure that the quality of their assessments is maintained. These assessments consist of over 200 questions across 20 domains asked of the patient and sometimes their care givers. Data quality and completeness has been previously demonstrated [8]. RiskOP uses the answer to 16 questions concerning Age, Sex, Body Mass Index, Cancer, Congestive Heart Failure (CHF), Previous Stroke, Parkinsons, Dyspnea, Weight loss, Fatigue, Oxygen therapy, Skin ulcer, frequency of going out, exercise, time since last hospital stay and change in activities of daily living.
Participants
Participants included all people aged ≥65 years who completed an interRAI-HC assessment between 1 January 2013 and 21 August 2020 in Canterbury, New Zealand, and who had given written consent for their data to be used for research. Previous research has demonstrated 93% of all those who are assessed give this consent [8]. The RiskOP model had been developed in a subset of these patients (up to March 2018) [11]. Ethical approval was provided by the New Zealand Ministry of Health and Disability Ethics Committee (HDEC) (14/STH/140/AM08).
Inclusion criteria was a single BNP test within 90 days of the date of interRAI-HC assessment. Exclusion criteria was those identified as being on a palliative pathway. This was identified by a question which asked if they had been told that in the best clinical judgment of the physician the individual has end stage disease with approximately 6 or fewer months to live.
The interRAI-HC assessment data and BNP test results was provided by the Canterbury District Health Board. The National Mortality Collection Register administered by the New Zealand Ministry of Health provided the dates of deaths. Linkage between the data sets was made by encrypted national health index numbers as every person who has ever interacted with the public health system in New Zealand has a unique number. All participants were followed for a minimum of 12 months or until death.
Statistical analysis
Data are presented as n (%) for categorical variables, mean and standard deviation for normally distributed quantitative variables and median (lower quartile and upper quartile) for non-normally distributed variables. All confidence intervals are 95% calculated using bootstrapping.
The RiskOP model was applied to each patient to obtain a prediction of one year mortality and recalibrated for the local data. This is referred to as the baseline model. A logistic regression model containing the predictions from the Baseline model, the log-base-2 transformed BNP concentrations (as a continuous variable), and the time from BNP measurement to interRAI-HC assessment was constructed and one-year mortality predictions calculated. This was called the new model.
The added value of BNP to RiskOP was assessed by the relative change in Brier score (Brier skill), difference in area under the receiver operator characteristic curve (δAUC), Integrated Discrimination Improvement (IDI) for those who died (IDIevent) and those who did not die separately (IDInon-event), risk assessment plots (RAP), calibration plots and decision curves. The Brier score is a measure of the variation in risk prediction from the actual outcomes and the Brier skill measures the improvement in this measurement with the addition of a biomarker [17]. The AUC is a measure of discrimination and is the probability that if we were to draw at random a person who died and a person who survived that the risk prediction for the person who died would be greater than for the person who survived. An AUC of 0.5 suggests no discrimination. An AUC of 1 means perfect discrimination. The IDIevent represents the mean increase in predicted risk for those who had the event (those who died) whereas the IDInon-event represents the mean decrease in predicted risk for those who did not have the event [18]. RAPs plot Sensitivity verse risk prediction and 1-Specificity verse risk prediction [19]. Improved performance with the addition of a biomarker would be observed by increased separation of the curves. Calibration plots divide the data into quantiles and plot the mean predicted risk verses the actual risk (nevent/n) within each quantile. Decision curves enable the assessment of the additional net benefit of BNP at specific risk thresholds of relevance to the clinician and patient taking into account both benefits and harms of a proposed treatment [20]. In this case the proposed treatment is simply telling the person that they are at risk of dying in one year, and the benefit of being correct is weighted the same as the harm of being incorrect. The prediction threshold on the x-axis is the threshold of predicted risk above which each person is classified as high risk (in this case high risk of dying in one year) and below which they are classified as low risk. The net benefit is true positives. It is maximal and equal to the prevalence at a prediction threshold of zero. A net benefit of say, 0.1, in this case is equivalent to a strategy that correctly identifies as dying in one year 10 out of 100 at risk people. The higher a decision curve is, the better it performs at identifying true positives.
Results
There were 18,720 interRAI-HC assessments from 14,713 individuals in the data set. Of these individuals 6,899 (46.9%) had a BNP test within the period of which 1,690 (24.5%) were within 90 days preceding the assessment and had at least one year follow-up, (Fig 1). 1537 individuals had all the required data to calculate one-year mortality predictions with the RiskOP model. The mean age was 82.6 years and 54.2% were female (Table 1). Of these 785 had CHF of whom 324 (41.3%) died within a year. Within those with CHF the median (IQR) BNP was 139 (58–297) pg/mL. Of the 752 without CHF 229 (30.5%) died within one year. Within those without CHF the median (IQR) BNP was 57 (23–130) pg/mL.
Overall there was a modest improvement in prediction with the addition of BNP, (Table 2).
The AUC increased marginally (δAUC = 0.015) from 0.716 to 0.731, and the Brier skill of 1.9% suggests a reduction in overall prediction error. The mean prediction for those who died (IDIevent) increased marginally (0.9%) and the mean prediction for those who did not die (IDInonevent) decreased marginally (0.6%). In the cohort with CHF, BNP had minimal impact on mortality prediction (Figs 2 and 3. There was no apparent improvement in risk for those who did or who did not die in one year, (Fig 3B). The decision curve suggests a net benefit of the addition of BNP in the mid-prediction threshold range of 0.4 to 0.7, (Fig 4B). In contrast, in those without CHF BNP improved discrimination (δAUC = 0.029 (0.004 to 0.057)) with a final AUC of 0.75. There was no apparent improvement in risk for those who did or did not die in one year, (Fig 3C). The improvement was greatest for those who died with probabilities above about 0.3. The decision curve shows that the addition of BNP improves the application of the risk model amongst those with baseline risk up to 0.75, (Fig 4C). The new model remained well calibrated overall in each of the CHF and no CHF cohorts, (Fig 5).
Sensitivity verse 1-Specificity. Dotted lines are baseline model RiskOP model, solid lines are the new model with addition of BNP.
Plots illustrating the change in predicted probabilities for participants who died (teal Sensitivity Metric curves) and did not die (red 1-Specificity Metric curves). The dotted line is the predicted probability curve of the baseline model, RiskOP. The solid line is the predicted probability curve with the addition of BNP to RiskOP (New model). Improved prediction is where the teal Sensitivity curve moves towards the upper right corner (higher average risk) with the addition of BNP and/or where the red 1-Specificity curve moves towards the bottom left corner (lower average risk) with the addition of BNP. All patients included in panel A, with separate models for Congestive Heart Failure participants in panel B, and non-Congestive Heart Failure participants in panel C.
At specific prediction of mortality thresholds the new model (teal curve) with the addition of BNP has higher net benefit to the baseline model (red curve). All patients included in panel A, with separate models for Congestive Heart Failure participants in panel B, and non-Congestive Heart Failure participants in panel C.
Actual percentage of people with outcomes (and 95% CI) verse predicted percentage (and 95%CI) for all patients split into 10 groups. Perfect calibration is the dashed line. The new model (teal curve) has similar or better calibration than the baseline model (red curve). All patients are included in panel A. Panel B is for participants with Congestive Heart Failure, and Panel C for participants without Congestive Heart Failure.
Discussion
The addition of BNP measurements to a one-year mortality risk prediction model designed for use with a standardised needs assessment for older people marginally added value overall. The additional value was in people without a diagnosis of CHF. More specifically, the study indicated it had additional prognostic information for clinicians when treating those patients without a diagnosis of CHF and with a risk prediction of below 0.75 (<75% probability of dying within 1-year).
Several previous studies have developed mortality risk prediction models utilising existing health data. These include a large UK study which included a patient questionnaire and biochemical tests [21]. In 2020 Canadian research which utilises an earlier version of the interRAI-HC, developed the RESPECT tool [22]. Similar to RiskOP, this model was developed in a large group of older adults (mean age 79.6) and had similar discriminatory performance. Given that the interRAI suite of tools are standardised and well established internationally, RESPECT or RiskOP could be expected to be transferrable to other countries using the interRAI-HC.
The large UK based QRISK3 study [6] focussed on future risk of cardiovascular disease. This study utilised the QResearch database at over 1300 General Practices throughout the UK. It included a blood test for high density lipoprotein cholesterol. Adjusted hazard ratios for cardiovascular disease in men was 1.19 (1.18 to 1.19; 95% CI) per unit increase of total cholesterol: HDL cholesterol ratio. It is a rare example of the value of standardised and connected data bases that support not just the individual but population health.
The utilisation of specific blood tests, such as BNP have long been identified as predicting mortality in older adults, including those without any known cardiovascular issues [13]. Indeed, the value of BNP as a predictor of mortality continues to be recognized [13,15,23], including most recently in relation to BNP mortality and its association with Covid-19 [24]. Furthermore, frailty, which is in itself a risk factor for mortality [25], was associated with an increased risk of elevated BNP in a recent study of over 1300 community dwelling elders with no previous cardiovascular history [26].
Given the value of BNP in terms of identifying both frailty and risk of mortality, the addition of this relatively low cost and easy to acquire biomarker to the RiskOP mortality model adds a modest effect to the accuracy of a comprehensive risk prediction, although its value is greater for those without a prior diagnosis of heart failure. That there is little additional prognostic value of BNP in those with heart failure may seem surprising. This could simply be that as the people with CHF are more likely to die the covariates in the RiskOP model better correlate with BNP meaning it has less affect on overall mortality prediction. While intriguing, further investigation is beyond the scope of this manuscript.
Like all assessments, and clinical outcome measures, the prediction models here are designed to be used as a valuable aid to clinical decision making but not as a replacement to skilled clinical judgement and of course individual patient choice which is an important part of any decision-making discussion. Added to this is the important cultural context when any health professional is preparing to discuss a specific patients clinical trajectory and the options available [27]. This group had low numbers of Māori (indigenous New Zealanders) and specific ethnic analysis was not viable. Further research with appropriate numbers of Māori is vital in creating a culturally appropriate model. Additionally, incorporating Haoura Māori values would be evaluated and included before considering the roll-out of any prediction model.
A limitation of this study is that the model has been developed in a national New Zealand based cohort of older adults and the results need to be validated in an international context. However, the advantage of using the interRAI-HC to develop the model is that interRAI-HC is used internationally with similarly trained assessors require ongoing review. This increases the likelihood that this model would validate externally. We also acknowledge that for practical reasons, the interRAI-HC assessments and the BNP blood tests were not usually completed at the same time in the patient journey. Ideally blood draws for biomarker testing would be cotemporaneous with the interRAI assessment. Given many of the interRAI assessors are trained nurses, this is practical. How this cotemporaneous measurement affects the accuracy of the new model would need further investigation. Additionally, while the RiskOP model incorporates Age, Sex, and Body Mass Index, other factors which may be associated with BNP such as inflammation, kidney function, and atrial fibrillation were not measured or included as covariates. If this was able to be done these may well improve the accuracy of the risk prediction. Further investigation would also need to assess performance in sex, age, and ethnicity sub-groups. Finally, the net benefit and decision curve analysis is merely a tool for understanding if BNP could be of added value in decision making. For it to be seen as suitable for clinical practice, a more comprehensive net-benefit analysis would be needed weighing both benefits and harms at each decision point, e.g. for risk stratifying patients to low, intermediate, or high risk.
The addition of a BNP blood test to the RiskOP mortality model improves its accuracy, particularly for those that do not have a diagnosis of congestive heart failure. This is a proof of concept that the addition of a biomarker to risk prediction from a standardised assessment can improve mortality prediction. Further research would need to be undertaken in order to ascertain the value of including additional commonly used blood tests for the older adult population such as sodium, creatinine, albumin, and C-reactive protein. This research would not only need to ascertain the statistical improvement in risk prediction, but also look at a cost-benefit analysis, and ideally would include calibration of models for specific groups including in New Zealand Māori and Pacific Peoples. Improved mortality prediction gives clinicians, patients and family more confidence that they have received the necessary information to make informed decisions about appropriate interventions and treatment options for an individual.
References
- 1. Rechel B, Doyle Y, Grundy E, McKee M. How can health systems respond to population ageing? World Health Organization, on behalf of the European Observatory on Health Systems and Policies; 2009 p. 43. Report No.: 10.
- 2. Kim P, Daly JM, Berry-Stoelzle MA, Schmidt ME, Michaels LC, Dorr DA, et al. Prognostic Indices for Advance Care Planning in Primary Care: A Scoping Review. J Am Board Fam Med. 2020;33: 322–338. pmid:32179616
- 3. Kim NH, Cho HJ, Kim S, Seo JH, Lee HJ, Yu JH, et al. Predictive Mortality Index for Community-Dwelling Elderly Koreans. Medicine (Baltimore). 2016;95: e2696. pmid:26844511
- 4. Campbell AJ, Diep C, Reinken J, McCosh L. Factors predicting mortality in a total population sample of the elderly. J Epidemiol Community Health. 1985;39: 337–342. pmid:4086965
- 5. Gagne JJ, Glynn RJ, Avorn J, Levin R, Schneeweiss S. A combined comorbidity score predicted mortality in elderly patients better than existing scores. J Clin Epidemiol. 2011;64: 749–759. pmid:21208778
- 6. Hippisley-Cox J, Coupland C, Brindle P. Development and validation of QRISK3 risk prediction algorithms to estimate future risk of cardiovascular disease: prospective cohort study. BMJ. 2017; j2099. pmid:28536104
- 7. Hirdes JP, Ljunggren G, Morris JN, Frijters DH, Finne Soveri H, Gray L, et al. Reliability of the interRAI suite of assessment instruments: a 12-country study of an integrated health information system. BMC Health Serv Res. 2008;8: 434. pmid:19115991
- 8. Schluter PJ, Ahuriri-Driscoll A, Anderson TJ, Beere P, Brown J, Dalrymple-Alford J, et al. Comprehensive clinical assessment of home-based older persons within New Zealand: an epidemiological profile of a national cross-section. Aust N Z J Public Health. 2016;40: 349–355. pmid:27197797
- 9. de Stampa M, Cerase V, Bagaragaza E, Lys E, Alitta Q, Gammelin C, et al. Implementation of a Standardized Comprehensive Assessment Tool in France: A Case Using the InterRAI Instruments. Int J Integr Care. 2018;18: 5. pmid:30127689
- 10. Jamieson HA, Schluter PJ, Pyun J, Arnold T, Scrase R, Nisbet-Abey R, et al. Fecal Incontinence Is Associated With Mortality Among Older Adults With Complex Needs: An Observational Cohort Study. 2017;112: 1431–1437. pmid:28762377
- 11. Pickering JW, Abey-Nesbit R, Allore H, Jamieson H. Development and validation of multivariable mortality risk-prediction models in older people undergoing an interRAI home-care assessment (RiskOP). EClinicalMedicine. 2020; 100614. pmid:33437945
- 12. Wallen T, Landahl S, Hedner T, Nakao K, Saito Y. Brain natriuretic peptide predicts mortality in the elderly. Heart. 1997;77: 264–267. pmid:9093047
- 13. York MK, Gupta DK, Reynolds CF, Farber-Eger E, Wells QS, Bachmann KN, et al. B-Type Natriuretic Peptide Levels and Mortality in Patients With and Without Heart Failure. J Am Coll Cardiol. 2018;71: 2079–2088. pmid:29747827
- 14. Reilev M, Lundby C, Jensen J, Larsen SP, Hoffmann H, Pottegård A. Morbidity and mortality among older people admitted to nursing home. Age Ageing. 2020;49: 67–73. pmid:31763682
- 15. Doust J, Lehman R, Glasziou P. The Role of BNP Testing in Heart Failure. 2006;74: 6.
- 16. Khanam SS, Son J-W, Lee J-W, Youn YJ, Yoon J, Lee S-H, et al. Prognostic value of short-term follow-up BNP in hospitalized patients with heart failure. BMC Cardiovasc Disord. 2017;17: 215. pmid:28774268
- 17. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiol Camb Mass. 2010;21: 128–138. pmid:20010215
- 18. Pencina MJ, D’Agostino RB, Vasan RS. Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond. Stat Med. 2008;27: 157–172. pmid:17569110
- 19. Pickering JW, Endre ZH. New Metrics for Assessing Diagnostic Potential of Candidate Biomarkers. Clin J Am Soc Nephrol. 2012;7: 1355–1364. pmid:22679181
- 20. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Mak Int J Soc Med Decis Mak. 2006;26: 565–574. pmid:17099194
- 21. Ganna A, Ingelsson E. 5 year mortality predictors in 498 103 UK Biobank participants: a prospective population-based study. The Lancet. 2015;386: 533–540. pmid:26049253
- 22. Hsu AT, Manuel DG, Spruin S, Bennett C, Taljaard M, Beach S, et al. Predicting death in home care users: derivation and validation of the Risk Evaluation for Support: Predictions for Elder-Life in the Community Tool (RESPECT). Can Med Assoc J. 2021;193: E997–E1005. pmid:34226263
- 23. Voors AA, Ouwerkerk W, Zannad F, van Veldhuisen DJ, Samani NJ, Ponikowski P, et al. Development and validation of multivariable models to predict mortality and hospitalization in patients with heart failure: Mortality and hospitalization models in heart failure. Eur J Heart Fail. 2017;19: 627–634. pmid:28247565
- 24. Qin J-J, Cheng X, Zhou F, Lei F, Akolkar G, Cai J, et al. Redefining Cardiac Biomarkers in Predicting Mortality of Inpatients With COVID-19. Hypertension. 2020;76: 1104–1112. pmid:32673499
- 25. Cornman JC, Glei DA, Goldman N, Weinstein M. Physiological Dysregulation, Frailty, and Risk of Mortality Among Older Adults. Res Aging. 2017;39: 911–933. pmid:26865398
- 26. Wang Z-D, Yao S, Shi G-P, Wang Y, Shi J-M, Guo J-H, et al. Frailty index is associated with increased risk of elevated BNP in an elderly population: the Rugao Longevity and Ageing Study. Aging Clin Exp Res. 2020;32: 305–311. pmid:31004283
- 27. DeSouza R. Wellness for all: the possibilities of cultural safety and cultural competence in New Zealand. J Res Nurs. 2008;13: 125–135.