Introduction

In 2022, there were 236,749 estimated number of new lung cancer cases and 130,180 estimated deaths from lung cancer [1]. As the projected number of people 65 years and older grows from 59 million in 2022 to 73 million in 2030 [2], thoracic surgeons are increasingly presented with the task of assessing operative candidacy and performing risk stratification for older patients with lung cancer. Chronological age does not always reflect biological age, and elderly people have a range of functional status that varies from robust to frail. Frailty has been identified as the progressive loss of physical and mental function that leads to an abnormal response to physiologic stressors [3]. In surgical fields, frailty correlates well with both mortality and morbidity across a broad spectrum of operations [4–6]. We recently characterized the 5-factor modified frailty index as predictive of most outcomes after major lung resection, but this metric was only marginally better than more commonly used risk assessment tools such as the American Society of Anesthesiology (ASA) Physical Status Classification System and Charlson Comorbidity Index (CCI) [7].

The original Risk Analysis Index was developed as a measure of frailty and used to predict 6-month mortality in nursing home residents [8]. This was later revised for surgical patients (Revised Risk Analysis Index, hereafter referred to as the Risk Analysis Index and abbreviated as RAI), and was externally validated using various national databases [9, 10]. Despite growing literature on RAI as a predictor of postoperative outcomes in high-risk operations including lung cancer resections [10, 11], there has not been any dedicated, in-depth analysis of RAI as a predictor of postoperative outcomes following major lung resection for cancer using the Society of Thoracic Surgery (STS) database. In the present study, we mapped the RAI variables to patient data from the STS General Thoracic Surgery Database and analyzed whether the RAI correlates with postoperative outcomes in patients following lung cancer resection.

Materials and methods

Metrics

RAI was assessed by mapping ten variables in the STS database to eleven factors: age, sex, weight loss, poor appetite, congestive heart failure, dyspnea, renal failure, presence of cancer, functional status, cognitive decline, and living status (Table 1). An RAI score from 0 to 81 was calculated for each patient, with an increasing RAI score related to an increasing incidence of frailty. RAI was categorized into four subgroups (≤34, 35–39, 40–44, ≥45) as previously described [13]. Patients who did not have all the variables necessary for RAI calculation were included in a “missing” RAI category. The modified CCI was calculated as previously described [14]. ASA status was abstracted from STS data, which used a standard definition [15].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Society of thoracic surgery database variables mapped to the Risk Analysis Index (RAI).

https://doi.org/10.1371/journal.pone.0303281.t001

Results

Patients

Of 29,420 patients identified in the STS General Thoracic Surgery Database according to the inclusion criteria and exclusion criteria, 22,848 (77.6%) had all the factors necessary for calculation of RAI (S1 Fig). Patients who lacked the necessary data to calculate RAI were most similar demographically and clinically to patients with an RAI score of 35–39 or 40–44, particularly with regards to age, gender, living status, shortness of breath, congestive heart failure, dialysis status, functional status, and neurocognitive dysfunction. These preoperative variables were included in the calculation of RAI and correlated positively with RAI (S4 Table). Among preoperative variables not included in the calculation of RAI, many comorbidities also had clinically significant correlations with RAI. There was no clinically significant change in distribution of BMI across RAI categories. Similarly, no consistent trends were seen with induction therapy with respect to RAI scores. However, patients with higher RAI scores were more likely to have higher pathological and clinical T stages for lung cancer. We compared the cohort that had the data needed for calculation of RAI to the cohort that did not and found no evidence of important clinical differences (S5 Table).

The median score for ASA and CCI was 3, whereas it was 36 for RAI (Table 2). The RAI and CCI scores were reasonably well distributed among three and four score categories, respectively, with sparse population of the one remaining category for RAI and among the three remaining relevant categories for CCI. Of note, CCI scores were always 2 or greater and RAI scores were always 28 or greater because all patients had a diagnosis of lung cancer. ASA was well distributed among three categories, with the remaining three categories being sparsely populated.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Distribution of scores for the different risk assessment metrics among 29,420 patients.

https://doi.org/10.1371/journal.pone.0303281.t002

Multivariate analyses

On multivariable analyses, RAI was significantly associated with an incremental pattern with almost all outcomes (Table 3). Multivariate analyses for ASA demonstrated a significant association with almost all postoperative and administrative outcomes with the exception of infectious complications, unexpected ICU admission, and unanticipated surgical approach conversion (S6 Table). CCI was significantly associated with almost all postoperative and administrative outcomes with the exception of infectious complications (S7 Table).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Multivariate analysis for postoperative outcomes.

https://doi.org/10.1371/journal.pone.0303281.t003

Assessment of performance as risk metrics

Monotonicity of marginal probability was strict or partial among all outcome categories for RAI and ASA. Monotonicity of marginal probability was strict or partial for CCI among all outcome categories with the exception of gastrointestinal outcomes. The incidence of strict monotonicity was similar among CCI (11/15), RAI (9/15), and ASA (10/15). Of note, ASA was at less risk for partial monotonicity based on having only 3 categories compared to 4 for RAI and CCI (S8 Table).

ROC analyses demonstrated “good” AUC values (.70 to .80) for all three metrics related to mortality and discharge to other than home, whereas AUC values for other outcomes were in the fair category for all metrics (0.60 to 0.70; Table 4). Comparison of AUC values between each pair of the instruments using the DeLong test showed that RAI outperformed CCI and ASA in 6 of the 15 outcome categories, although the differences were not clinically important. There was no significant difference between CCI and ASA on ROC analyses (Table 4).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Results of ROC analyses.

https://doi.org/10.1371/journal.pone.0303281.t004

Discussion

In many surgical fields, frailty has been shown to correlate with postoperative complications, discharge to institutional care, and mortality [17, 18]. The original Risk Analysis Index is a tool based on the accumulation of deficits model of frailty derived from the Minimum Set Mortality Risk Index-Revised (MMRI-R) instrument used to predict 6-month mortality in nursing home residents [8]. Arya et al. revised the original Risk Analysis Index scoring system for nonveteran surgical patients, externally validated the revised RAI using the National Surgical Quality Improvement Program Database (NSQIP) in patients undergoing elective noncardiac operations, and reported that RAI had improved discrimination and calibration as a frailty-screening tool in surgical patients compared to the original RAI [9]. In contrast, Wan et al. further investigated the accuracy of RAI in predicting postoperative morbidity and mortality in patients undergoing high-risk operations using NSQIP data and reported it to be an ineffective predictor of 30-day morbidity and mortality for patients undergoing high-risk operations including lung resection [11]. To date, no study has investigated the predictive value of the revised RAI (hereafter abbreviated as RAI) focusing on patients undergoing lung resection using the using the Society of Thoracic Surgery (STS) database. The current study mapped RAI variables to patient data from the STS database and assessed whether frailty as assessed by RAI is independently associated with postoperative outcomes in patients following anatomic lung resection for lung cancer.

We found RAI to be an independent predictor of all postoperative outcomes following anatomic lung section for lung cancer. On multivariable analysis, increasing RAI scores were associated with incremental changes in the odds of adverse outcomes that were clinically meaningful. We also demonstrated that RAI was associated with a wide variety of administrative outcomes. RAI performed similarly to ASA and CCI based on overall accuracy. Contrary to ROC analyses using the NSQIP database which identified RAI as a poor predictor of operative mortality [11], our analyses using the STS database suggested RAI to be a good predictor of postoperative mortality. However, RAI was similarly weak in predicting other postoperative outcomes compared to ASA and CCI, suggesting that its routine use in assessing surgical risk is only of moderate benefit.

An additional strength of RAI may be its potential ability to assess frailty. Tools assessing the “frailty phenotype” are sometimes impractical for routine screening in busy surgical clinics because they require time and effort from clinical staff to measure walking speed and grip strength [19]. RAI requires no special equipment and can be calculated in less than 60 seconds, allowing easy implementation and resulting in high participation compliance among surgical clinics [20]. However, although associations of RAI with surgical outcomes have been demonstrated, and its performance is similar to that of other frailty metrics generated using data from the electronic medical record, correlations between RAI and components of physical frailty such as weight loss, low gait speed, weakness, sarcopenia, and low energy have not been adequately assessed. In our study RAI did not correlate strongly with underweight status, suggesting a weak relationship with weight loss and weakness. Thus, whether RAI merely represents a potentially useful surgical risk tool or in addition has the added benefit of assessing frailty in a lung resection population has yet to be determined.

Although RAI may have the advantage of ease of calculation using the electronic medical record, its design as an accumulation of deficits model precludes extensive use of physical performance metrics for such calculation. Regardless of whether RAI is an accurate measure of frailty, the information from RAI cannot be used to identify individual elements in patients that might be targeted for mitigation to reduce operative risk, and RAI values for an individual patient will be unlikely to decrease in response to such mitigation. Therefore, RAI may serve primarily as a means for improved patient selection for surgery.

There are potential limitations to this study. Selecting covariates for multivariate analyses is sometimes challenging in studies such as this. There are a number of known factors that are associated with adverse clinical outcomes after resection for lung cancer that have been derived through assessment of the STS General Thoracic Surgery database [16]. Unfortunately, Version 2.41 of the STS data selected for the current study did not capture steroid use and Zubrod performance status, which have been shown to be positively correlated with postoperative mortality. Furthermore, gender and renal dysfunction could not be included in the multivariable model because they were included in the calculation of RAI. Review of the distribution of gender and renal dysfunction across the different RAI categories revealed strong collinearity between RAI and these variables (S4 Table). To directly compare the performance of ASA, CCI and RAI as surgical risk metrics, we removed ASA from the multivariable model when analyzing RAI and CCI.

There were a large number of missing values related to some outcome and covariate variables within the STS database. The general thoracic surgery STS database currently only receives data from 271 participant sites across North America, with center-level penetration between 40% and 50% [21], thus our findings may not be generalizable to all centers in North America. Data on post-discharge status are limited, largely restricting our analyses to in-hospital outcomes with no survival data. It is likely that a minimally invasive approach may lead to lower complication rates than an open approach, as is being studied in the ongoing VIOLET trial [22]. However, we did not assess surgical approach (open, VATS, robotic) and its possible effect on the utility of RAI in predicting complications, as this is the topic of a separate ongoing study using STS database data. Finally, we limited our study to Version 2.41 of the STS database due to changes in collected variables that permitted calculation of the RAI score using only this version.

We conclude that the RAI is associated with clinical and administrative outcomes following lung cancer resection. However, its overall accuracy as a surgical risk predictor is only moderate and similar to ASA and CCI. Our findings invite the question of whether RAI is a good frailty metric or is more accurately a surgical risk predictor in disguise. Future studies will be needed to investigate the relationship between RAI, physical frailty metrics, and sarcopenia.

Supporting information

Acknowledgments

The data for this research were provided by The Society of Thoracic Surgeons’ National Database Participant User File Research Program. The views and opinions presented in this article are solely those of the author(s) and do not represent those of The Society of Thoracic Surgeons. Data analysis was performed at the investigators’ institution.