Beyond the revised cardiac risk index: Validation of the hospital frailty risk score in non-cardiac surgery

Frailty is an established risk factor for adverse outcomes following non-cardiac surgery. The Hospital Frailty Risk Score (HFRS) is a recently described frailty assessment tool that harnesses administrative data and is composed of 109 International Classification of Disease variables. We aimed to examine the incremental prognostic utility of the HFRS in a generalizable surgical population. Using linked administrative databases, a retrospective cohort of patients admitted for non-cardiac surgery between October 1st, 2008 and September 30th, 2019 in Alberta, Canada was created. Our primary outcome was a composite of death, myocardial infarction or cardiac arrest at 30-days. Multivariable logistic regression was undertaken to assess the impact of HFRS on outcomes after adjusting for age, sex, components of the Charlson Comorbidity Index (CCI), Revised Cardiac Risk Index (RCRI) and peri-operative biomarkers. The final cohort consisted of 712,808 non-cardiac surgeries, of which 55·1% were female and the average age was 53·4 +/- 22·4 years. Using the HFRS, 86.3% were considered low risk, 10·7% were considered intermediate risk and 3·1% were considered high risk for frailty. Intermediate and high HFRS scores were associated with increased risk of the primary outcome with an adjusted odds ratio of 1·61 (95% CI 1·50–1.74) and 1·55 (95% CI 1·38–1·73). Intermediate and high HFRS were also associated with increased adjusted odds of prolonged hospital stay, in-hospital mortality, and 1-year mortality. The HFRS is a minimally onerous frailty assessment tool that can complement perioperative risk stratification in identifying patients at high risk of short- and long-term adverse events.


Introduction
Pre-operative frailty is associated with short-and long-term adverse outcomes [1][2][3]. However, there is no gold standard for defining frailty, which most commonly is described as a clinically recognizable state of increased vulnerability due to aging-associated decline in reserve and function across multiple physiological domains [4]. While there are numerous frailty scores that accurately identify and risk stratify frail individuals, these tools can be time consuming and not easily applied at the bedside [5]. Furthermore, such frailty scores cannot be applied retrospectively in the development of prognosis tools, use for risk stratification and benchmarking of services. Despite these challenges, many key stakeholders advocate for wide-scale implementation of frailty assessment. The benefits of early identification of frailty include preventing/slowing further decline, prognostication, avoidance of inappropriately aggressive therapies and encouraging goals of care discussions [6]. Currently the gold standard of risk stratification in pre-operative settings include the use of the Revised Cardiac Risk Index (RCRI) and pre-operative B-type natriuretic peptides (BNP) [7,8]. However, frailty is not captured or taken into account using these measures. Recently the use of administrative databases has been targeted as a convenient, cost-effective, automated method of identifying frailty. Gilbert et al. developed and validated the Hospital Frailty Risk Score (HFRS) in medical inpatients, which is composed of 109 International Classification of Disease (ICD-10) diagnostic codes to create a numerical score [9]. This score was demonstrated to have fair to moderate overlap with previously validated clinical frailty scores such as the Fried Phenotype and Rockwood Frailty Index and identified high frailty risk patients that had a significantly higher 30-day mortality and readmission rates.
We assessed the validity of the HFRS in a large perioperative cohort in the peri-operative period to predict adverse outcomes.

Study design
A retrospective cohort of all patients selected elective non-cardiac surgery (S1 Table) between October 1st, 2008 and September 30th, 2019 in Alberta, Canada, was created using linked administrative databases to identify 798,918 admissions for non-cardiac surgery. Surgeries were excluded if they occurred within one-year of the index surgery (n = 86,110; i.e. only the first surgery for each patient within the time period was included), with a final cohort of 712,808 non-cardiac surgeries. The following databases were linked using individual patient provincial health numbers: 1) Alberta Inpatient Discharge Abstract Database (DAD) that includes information on all admissions to acute care facilities including most responsible admission diagnosis (coded using International Classification of Diseases, Canadian Enhancement; ICD-10) and in-patient surgical procedures (coded using Canadian Classification of Health Intervention codes); 2) The Pharmaceutical Information Network (PIN) Database that captures outpatient medication dispensations in Alberta, regardless of medication insurance coverage. Over the counter medications are not captured in this database; 3) The Alberta Health Care Practioner Claims Database, which identifies physician billing claims; 4) Alberta Provincial Registry data which ascertains start and stop dates of Alberta Health Care Insurance Plan coverage, including dates of death; 5) Alberta Health Services (AHS) Laboratory Database, which is the repository for all in-patient and out-patient laboratory investigations.

Hospital frailty risk score and RCRI score calculation
Using the methodology previously described by Gilbert et al. [9], the Hospital Frailty Risk Score was calculated using 109 ICD codes identified in any position. (S2 Table) with variable weighting to calculate the Hospital Frailty Risk Score [9]. Eligible ICD codes included those within 2 years of surgery date. A score of < 5 indicated low risk, 5 Our method for calculating the RCRI score using administrative data has previously been reported [10]. Briefly, the RCRI was calculated by 1-point assignments for the presence of each of the following variables: 1) history of ischemic heart disease 2) heart failure 3) stroke or transient ischemic attack 4) insulin treated diabetes 5) creatinine � 177umol/L and 6) high-risk surgery (intra-thoracic, vascular and intra-peritoneal), for a maximum score of 6. These data were extrapolated from the Discharge Abstract Database using the International Classification of Disease (ICD) codes. ICD-10 codes used included I20-I25 for ischemic heart disease; I50 for heart failure, I60-I69 for cerebrovascular disease; E10-14 for diabetes. Insulin use was determined from the PIN Database within 100 days prior to surgery. The most recent preoperative creatinine, up to 91 days prior to surgery, was obtained from the AHS Laboratory database.

Outcomes
Our primary outcome was a composite of death, myocardial infarction or cardiac arrest at or before 30-days. Other short-term outcomes included: prolonged hospital stay, in-hospital mortality, 30-day mortality, hospital readmission for any reason or emergency department presentation for any reason at 30-days. A prolonged hospital stay was defined as greater than expected length of stay for that particular surgery plus one day. Expected length of stay for surgery type was based on institutional standards (S3 Table). Long-term outcomes included: oneyear mortality, hospital readmission for any reason or emergency department presentation for any reason at one-year and a composite of death, myocardial infarction or cardiac arrest at one-year.

Statistical analysis
Descriptive analysis was undertaken to demonstrate differences between HFRS risk categories. Categorical variables were summarized using frequency and column percentage and continuous variables were described using mean and standard deviation. Non-normal distributed data is presented as median and inter-quartile ranges. Unadjusted odds ratio (OR) and 95% confidence intervals (CI) were derived from univariate logistic regression. Adjusted odds ratio (OR) and 95% confidence intervals (CI) were derived from multivariable logistic regression adjusted for age, sex, categorized number of preadmissions, categorized RCRI score, troponin, b-natriuretic peptide and the 17 components of Charlson Comorbidity Index (CCI). Stepwise variable selection (with p = 0.05 as enter and stay criterion) was adopted to get rid of redundant variables.
The incremental predictive power of including HFRS for outcomes was assessed using area under the Receiver Operating Characteristic (AUROC) curve and a net reclassification improvement (NRI) analysis. For both analyses the study cohort was randomly and evenly split into a training and a validation dataset. The following predictive models were built on the training and tested on the validation datasets for assessing predictive power of including HFRS. One is the gold standard model (reference) derived from a multivariable logistic regression which included the following candidate variables: age, sex, components of the CCI, RCRI as a categorical variable, categorized pre-operative troponin and pre-operative BNP values. The second is the proposed model which included the addition of HFRS as a categorical variable. Stepwise variable selection was used to include informative predictors into the final predictive models with p = 0.1 as enter and stay criterion.
NRI generates three values of relevance to be considered. The first is the NRI for events, which examines the proportion of patients with a correctly predicted observed outcome. An improvement in the events NRI represents a reduction in type I error (false positives), whereas a reduction in the events NRI represents an increase in type I error. NRI for non-events examines the proportion of patients who were correctly predicted to not experience an outcome. An improvement in the non-events NRI represents a reduction in type II error (false negatives), whereas a reduction in the non-events NRI represents an increase in type II error. The overall NRI is the sum of the events NRI and the non-events NRI.
All Statistical analysis was performed using Statistical Analysis System (SAS) Enterprise Guide 7.1 (Cary, NC, USA) and R (R Core Team (2021)).

Ethics
This study complies with the Declaration of Helsinki and was approved by the University of Alberta Health Research Ethics Board (Pro00081737) with a waiver of informed patient consent due to the minimal risk and being infeasible to contact all patients.

Primary outcome
The primary outcome composite of death, myocardial infarction or cardiac arrest at 30-days occurred in 4636 patients, ranging from 0�4% in the HFRS low risk group to 2�8% in the HFRS high risk group (Table 2 and Fig 1). In the adjusted model, an intermediate and high HFRS score was associated with an increased risk of the primary outcomes (OR 1�61; 1�5-1�74 and 1�55, 1�38-1�73). The breakdown of major adverse cardiovascular events (MACE) categorised by RCRI and HFRS can be seen in Fig 2 and S5 Table.

Short-term outcomes
A higher HFRS score was associated with a prolonged length of stay, ranging from 27�7% in the low HFRS group to 67�6% in the high HFRS group (Fig 1). HFRS was strongly associated with increased odds of in-hospital mortality (Fig 1), with the intermediate HFRS group demonstrating an adjusted OR of 6�99 (6�63-7�37) and the high HFRS group demonstrating an aOR of 9�67 (8�99-10�39). Odds of 30-day mortality (Fig 1) and 30-day readmission/ED visit (Fig 1) were similarly higher in the intermediate and high risk HFRS groups (Table 2).

Incremental benefit of HFRS
The NRI analysis demonstrated that for all outcomes, the addition of HFRS improved overall predictions (Fig 3 and S6 Table). For the primary composite outcomes of death, hospitalisation for MI or cardiac arrest at 30-days the overall NRI, NRI for events and NRI for non-events was 0.133, -0.175 and 0.288 respectively. When examining NRI events and NRI non-events separately, we observe that the inclusion of HFRS consistently improved the ability to identify nonevents, but was generally associated with an increase in false positives. The only exception was in-hospital mortality, where the HFRS improved the predictive abilities for both events (0.320) and non-events (0.634). The AUROC analysis demonstrated similar results (S7 Table), with

PLOS ONE
statistically significant improvements for all outcomes with the greatest AUROC improvement observed for in-hospital mortality (3.0%).

Discussion
In a large, generalizable cohort of patients undergoing non-cardiac surgery, frailty identified by the Hospital Frailty Risk Score was associated with adverse short-and long-term outcomes. The hospital frailty risk score provides incremental prognostic information to traditional RCRI risk estimation. For example, for patients with an RCRI of one, the risk of 30-day death, MI and cardiac arrest ranges from 0.35% in the low frailty risk group to 2.12% in the high-risk frailty group.
Since the development and validation of the hospital frailty risk score in the acute care setting [9], the association between HFRS score and adverse events has been replicated in numerous in-patient [11][12][13][14] and procedural settings [15][16][17]. In the peri-operative setting, higher HFRS scores have been associated with adverse events in patients undergoing spinal surgery [18], joint arthroplasty [19], vascular surgery [20] and cirrhotic patients undergoing surgery [21]. In a generalizable cohort of 487,197 patients over the age of 50 undergoing surgery Harvey et al. demonstrated that a high HFRS was associated with prolonged length of stay, 30-day mortality and 28-day readmission [22]. However, the addition of HRFS to CCI did not significantly improve model performance. In contrast, in our analysis, after adjusting for RCRI and CCI, HFRS remained a significant predictor of both short and long-term outcomes. The difference is likely the result of variations in coding practices-in our analysis, we are able to capture ICD codes from all in-patient, emergency department and outpatient encounters. In comparison, in the analysis by Harvey et al. the CCI and HFRS were not calculated in 17% and 32% of individuals due to a lack of hospitalisation in the previous two and five years respectively [22]. This highlights the potential importance of a broad, standardized approach to collecting administrative data for HFRS calculation and the implications of coding depth for accurate HFRS calculation [23].
Overall, frailty defined using a variety of assessment tools has been demonstrated to be associated with adverse peri-operative outcomes [24,25]. It is important to consider that frailty takes many shapes and form, and as a result, a multidimensional approach to assessment that assesses physical, mental, nutritional, and socioeconomic factors is the ideal. This approach has been demonstrated to be valid and correlated with adverse events in community-dwelling older adults and in patients with heart failure [26,27]. However, HFRS provides a unique opportunity for implementation in electronic medical record systems, due to its ability to be tabulated by readily available administrative data without the need for the manual application of frailty assessment. The availability of this point-of-care estimate of frailty provides valuable information to physicians and surgeons, allowing for a more nuanced discussion of peri-operative risk. The HFRS may also be used to identify a high-risk patient population that would warrant dedicated frailty assessment and consideration for enrollment in pre-operative prehabilitation interventions that have been shown to be beneficial in surgical setting [28,29]. Based on our NRI analysis, the greatest strength of the HFRS is its ability to identify patients that would traditionally be deemed high risk based on their RCRI score and reclassify them to a lower risk based on the absence of frailty metrics included in the HFRS. Further studies are warranted to examine whether the HFRS can provide clinically actionable information to surgeons and pre-operative assessors, identify appropriate candidates for prehabilitation and be integrated into routine clinical practice and be utilised to judiciously utilise peri operative testing. In addition, further analysis is required to determine whether the impact of frailty as measured by the HFRS varies based on the spectrum of surgical risk for any given procedure.
Limitations of an ICD-based frailty assessment revolve around the accuracy and depth of coding to accurately define frailty. In addition, ICD coding does not allow for severity of conditions to be considered. As previously mentioned, administrative data is not primarily intended for research purposes, further increasing the risk of incorrect coding or missing data bias. In our analysis, we retrospectively utilised this administrative data, however it is unclear it can be prospectively used in the future to be utilised in direct patient care. Further studies are required to assess the feasibility of implementation of HFRS into existing electronic medical records and its impact on peri-operative decision making. In large sample size populations, such as in our study, even small effect sizes may demonstrate statistical significance. Clinical relevance of such small sizes should be taken into consideration when interpreting these statistically significant results. In the present study, risk of MI, cardiac arrest and death was significantly lower compared to pooled event rates from previous observational studies [30], however these prior studies were frequently small, included a small proportion of surgical procedures and were conducted more than a decade ago.
In conclusion, in a large, inclusive population of all adults undergoing non-cardiac surgery, the HFRS provides important short-and long-term prognostic information above and beyond traditional peri-operative risk stratification.
Supporting information S1