https://orcid.org/0000-0002-1207-2189
Figures
Abstract
Background
High-flow nasal cannula (HFNC) therapy is commonly used to treat acute respiratory failure in patients with COVID-19 pneumonia. However, predictors of successful weaning from HFNC in these patients has not been investigated.
Objective
To assess predictors of successful separation from HFNC in patients with COVID-19 pneumonia.
Methods
We conducted a retrospective cohort study at a university hospital in Thailand. Patients with COVID-19 pneumonia requiring HFNC therapy between April 2020 and June 2022 were included. ROX index was defined as the ratio of oxygen saturation (SpO2) / fraction of inspired oxygen (FiO2) to respiratory rate. Heart-ROX (HROX) index was defined as ROX multiplied by heart rate (HR) improvement. HR improvement (delta-HR) was defined as a percentage of the difference between the baseline HR and the morning HR at HFNC weaning day 1 divided by the baseline HR. Weaning success was defined as ability to sustain spontaneous breathing after separation from HFNC without any invasive or non-invasive ventilatory support for ≥48 hours or death.
Results
A total of 164 patients (54% male) were included. Mean age was 61.1±16.1 years. Baseline SpO2/FiO2 was 265.3±110.8. HFNC weaning success was 77.4%. The best cut-off value of ROX index to predict HFNC weaning success was 7.88 with 100% sensitivity, 97.3% specificity, and area under the ROC curve (AUC) of 0.98 (95% CI: 0.937–1.000, p<0.001). The best cut-off value of delta-HR 3.7 with 88.2% sensitivity, 75.7% specificity, and AUC of 0.83 (95% CI: 0.748–0.919, p<0.001). The best cut-off value of HROX index was 59.2 with 88.2% sensitivity, 81.1% specificity, and AUC of 0.89, (95% CI: 0.835–0.953, p<0.001).
Conclusions
The ROX index has the highest accuracy for predicting successful weaning off HFNC treatment in patients with COVID-19 pneumonia. While HROX and delta-HR indices can serve as alternative tools, it is recommended to verify these indices and determine the optimal cut-off value for determining separation from HFNC therapy through a large prospective cohort study.
Citation: Ruchiwit P, Madua J, Saiphoklang N (2024) Comparison of ROX, HROX, and delta-HR indices to predict successful weaning from high-flow oxygen therapy in hospitalized patients with COVID-19 pneumonia. PLoS ONE 19(2): e0297624. https://doi.org/10.1371/journal.pone.0297624
Editor: Andrea Cortegiani, University of Palermo, ITALY
Received: September 9, 2023; Accepted: January 10, 2024; Published: February 15, 2024
Copyright: © 2024 Ruchiwit et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files.
Funding: I have received financial support from the Faculty of Medicine, Thammasat University, Thailand. The funders had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Coronavirus Disease 2019 (COVID-19) is an infectious disease that is still spreading. Most patients have respiratory infection symptoms such as a cough, runny nose, sore throat, and fever. Some people have COVID-19 pneumonia, resulting in decreased oxygen levels in the blood. These patients need oxygen supply via an oxygen cannula, a high-flow nasal cannula (HFNC), or non-invasive ventilation, depending on oxygen concentration and the severity of pneumonia [1, 2]. Although the use of non-invasive respiratory support in acute respiratory failure due to viral infection and pandemics is still debated, without evidence in favor or against their use at the early stage of the pandemic, non-invasive respiratory support strategies were widely and variably used during the pandemic [3].
If the inflammatory process continues, respiratory failure can occur, requiring invasive mechanical ventilation and increasing the risk of death [4]. HFNC has been widely used during the COVID-19 pandemic with different reported outcomes [5]. A recent systematic review and meta-analysis [4] indicated that HFNC may reduce the intubation rate compared to conventional oxygen therapy. Nevertheless, there is still conflicting evidence regarding the benefits of its use [6, 7]. However, delayed intubation can have adverse effects on patients [8]. Therefore, proper intubation and use of HFNC can make treatment more effective. The ROX index, defined as the ratio of SpO2/FiO2 to respiratory rate, is used to predict successful HFNC treatment [9]. Treatment success is determined by a ROX index value of 12.7 with 85% sensitivity and 41% specificity [10]. Humidified high-flow nasal cannula (HFNC) has been identified as a safe and effective therapy for hypoxemic acute respiratory failure in COVID-19 patients [11]. It reduces respiratory rate and increases oxygen saturation, potentially decreasing the need for mechanical ventilation. However, delaying intubation through non-invasive approaches may lead to worse outcomes [8].
Despite the clinical benefits of HFNC, previous studies have not fully explored predictors of weaning success in COVID-19 pneumonia patients. Previous research has found the ROX index to be a useful indicator of disease severity and predictor for intubation. A value of less than 4.88 is associated with intubation, with an HR of 0.273, CI95% 0.121–0.618, and p value of .002 [9].
This study aims to identify new predictors of successful weaning from HFNC among COVID-19 patients hospitalized with pneumonia. We will focus on two novel predictors: 1) the HROX index, which is calculated by using the ROX index in conjunction with heart rate; 2) the HR improvement (delta-HR) index, which is defined as the percentage difference between the baseline heart rate and the morning heart rate on the first day of HFNC weaning, divided by the baseline heart rate.
Methods
Study design and participants
This retrospective cohort study was conducted at Thammasat University Hospital in Thailand. Patient data between 1st April 2020 and 30th June 2022 were extracted through electronic medical records, nursing record, and ICD-10. The data were accessed for research purposes in July 2022. The inclusion criteria were patients aged 18 years or older, hospitalized in respiratory wards, including both ICU and non-ICU wards with COVID-19 pneumonia, requiring HFNC treatment within 24 hours of admission, and with confirmed diagnosis by a positive SARS-CoV-2 reverse transcription-polymerase chain reaction test and infiltrates observed on chest radiographs. Patients receiving HFNC treatment after extubation, those with cardiac arrhythmias, those taking medications that affect heart rate (such as beta-blockers, non-dihydropyridine calcium channel blockers, amiodarone, or digoxin), and those with advanced-stage disease or a “do not intubate order (DNI)” were excluded.
Clinical data including demographics, co-morbidities, clinical characteristics, laboratory results, baseline oxygenation level and SpO2/FiO2 ratio, respiratory rate, baseline heart rate before HFNC weaning, morning heart rate on the first day of HFNC weaning, and the success or failure of weaning from HFNC, were analyzed.
Ethic approval was obtained from the Human Research Ethics Committee of Thammasat University (Medicine), Thailand (IRB No. MTU-EC-IM-0-065/65, COA No. 150/2022), in full compliance with international guidelines such as Declaration of Helsinki, The Belmont Report, CIOMS Guidelines and the International Conference on Harmonisation-Good Clinical Practice (ICH-GCP) (see S1 File). All methods were performed in accordance with these guidelines and regulations. This study was registered in the Thai Clinical Trials Registry (TCTR); thaiclinicaltrials.org (number: TCTR20221108004). Written informed consent was waived because this study was a retrospective study.
Procedures and outcomes
The primary outcomes of this study were to determine the best cut-off values of the ROX, HROX, and delta-HR indexes for predicting weaning success in patients with COVID-19 pneumonia who received HFNC treatment. The ROX index was calculated as the ratio of SpO2/FiO2 to respiratory rate at 1 hour before HFNC weaning, while the HROX index was defined as ROX multiplied by heart rate (HR) improvement. HR improvement (delta-HR) was calculated as the percentage difference between the baseline HR and the morning HR on the first day of HFNC weaning, divided by the baseline HR. Weaning success was defined as the ability to sustain spontaneous breathing after separation from HFNC without any invasive or non-invasive ventilatory support for at least 48 hours, or until death.
Statistical analysis
Data were expressed as number (%) and mean ± standard deviation. The chi-squared test was used to compare categorical data between the successful weaning group and the failed weaning group. The student’s t-test was used to compare continuous data between the two groups. A two-sided p-value < 0.05 was considered statistically significant. The ROC curve with sensitivity/specificity for each cut-off value was drawn, and the area under that curve was calculated. Statistical analyses were performed using SPSS version 26.0 software (IBM Corp., Armonk, NY, USA).
Results
Out of the 510 hospitalized patients with COVID-19 who were screened, 191 had pneumonia that required HFNC treatment. Of these, 27 patients included a “do not intubate order (DNI)” were excluded (see Fig 1). The final analysis included a total of 164 patients. Fifty-four percent of the patients (n = 89) were male, and the mean age was 61.1±16.1 years. The baseline SpO2/FiO2 was 265.3±110.1. The most common comorbidities were hypertension (present in 47.6% of patients), diabetes (36%), and dyslipidemia (33.5%) (see Table 1). Of these, 37 patients (22.6%) failed to wean off HFNC, and 10 patients in this group died.
Data shown as n (%) or mean±SD.
ROX, HR improvement (Delta-HR), and HROX indices
The proportion of COVID-19 pneumonia patients who successfully weaned off HFNC was 77.4%. The best cut-off value for the ROX index at 1 hour before HFNC weaning in predicting HFNC weaning success was 7.88, with a sensitivity of 100%, specificity of 97.3%, positive predictive values (PPV) of 99.2%, negative predictive values (NPV) of 100%, and an area under the receiver operating characteristic curve (AUC) of 0.98 (95% CI: 0.937–1.00, p<0.001). For the delta-HR index, the best cut-off value to predict HFNC weaning success was 3.7, with a sensitivity of 88.2%, specificity of 75.7%, PPV of 92.6%, NPV of 65.2%, and an AUC of 0.83 (95% CI: 0.748–0.919, p<0.001). Regarding the HROX index, the best cut-off value to predict HFNC weaning success was 59.2, with a sensitivity of 88.2%, specificity of 81.1%, PPV of 94.1%, NPV of 66.7%, and an AUC of 0.89 (95% CI: 0.835–0.953, p<0.001).(see Fig 2 and Table 2).
The best cutoff values of ROX, delta-HR, and HROX indices are 7.88, 3.7 and 59.2, with the area under the ROC curve of 0.98, 0.83 and 0.89, respectively.
Discussion
Our study is the first cohort investigation of predictors for successful weaning off HFNC treatment in hospitalized patients with COVID-19 pneumonia. The findings show that the ROX index had both the highest sensitivity, specificity and AUC, followed by the HROX and delta-HR indices.
The ROX index is a valuable tool in predicting intubation after HFNC treatment in patients with acute hypoxic respiratory failure. Numerous studies have demonstrated a significant correlation between ROX index and deteriorating outcomes. A multicenter, prospective observational cohort study by Roca O. et al. [12], conducted over two years, found that the ROX index was effective in identifying patients at high risk of intubation. Of the 191 patients treated with HFNC, 68 (35.6%) required intubation. The study also showed that the prediction accuracy of the ROX index increased over time (AUC: 0.679 at 2 h, 0.703 at 6 h, and 0.759 at 12 h). Additionally, an ROX greater than 4.88 measured at 2 h after HFNC initiation had a significant association with a lower risk for intubation [12]. In line with the findings of the retrospective cohort study involving 129 patients with COVID-19 pneumonia conducted by Patel M et al. [13], the study indicated that an ROX index value of less than 5 at the initiation of HFNC suggested a progression to invasive mechanical ventilation (odds ratio 2.137, P = .052). Furthermore, any subsequent decrease in the ROX index value after HFNC initiation was predictive of intubation (OR 14.67, P < .001). Another retrospective review examined the medical records of 105 patients with COVID-19 who were treated with HFNC and successfully withdrawn from HFNC, conducted by Hu Ming et al. [14]. The study found that the ROX index after 6 hours of HFNC initiation (AUROC, 0.798) demonstrated good predictive capacity for HFNC outcomes, with a ROX index greater than 5.55 at 6 hours after initiation significantly associated with HFNC success (OR, 17.821; 95% CI, 3.741–84.903, p<0.001). A multicenter, prospective study by Mellado-Artigas R et al. [15] demonstrated factors associated with endotracheal intubation in patients with COVID-19 pneumonia. The study revealed that the ROX index, SOFA score, and pH were predictors of endotracheal intubation. The SOFA score and ROX index showed excellent performance with an AUC of 0.88 (95% CI 0.80–0.96). Another retrospective observational study by Maeva Rodriguez et al. [10] conducted over two years found that FiO2 ≤ 40% and ROX index ≥ 9.2 were predictors of successful separation from HFNC in patients with pneumonia and acute respiratory failure.
In a prospective observational cohort study by Goh KJ et al. [16], 145 patients with acute hypoxemic respiratory failure treated with HFNC were followed a planned extubation. The study showed that lower ROX and ROX-HR index values recorded at different time points within 48 hours were associated with HFNC failure. Between the first 12 hours, the highest AUC for both indices occurred at 10 hours, with 0.723 and 0.739 for the ROX and ROX-HR indices, respectively. At 10 hours, a ROX-HR index above 6.80 was associated with a lower risk of HFNC failure (hazard ratio 0.301).
Our study is the first to evaluate the effectiveness of combining heart rate with ROX (HROX) or subtracting the baseline heart rate from the heart rate during HFNC treatment (delta-HR) in predicting HFNC weaning outcomes in patients who have recovered from COVID-19 pneumonia. We found that the HROX index had significant predictive value in HFNC weaning success with the best cut-off value of 59.2. Similarly, the delta-HR index showed predictive value with the best cut-off value of 3.7. However, the ROX index had the highest sensitivity and specificity in predicting HFNC weaning outcomes with the best cut-off value of 7.88.
We hypothesized that incorporating heart rate into the ROX formula would improve accuracy and predict successful weaning from HFNC. Given the challenges of assessing patients during the COVID-19 outbreak, particularly through physical examinations, the decision to wean HFNC is largely dependent on vital signs and symptoms. A notable strength of these indexes is that they are rapid bedside tools that are non-invasive, fast, easily reproducible, inexpensive, and highly accessible. Therefore, they can be used in low-resource settings and might be calculated by nurses, which is crucial during the pandemic. Our findings suggest that the HROX and delta-HR indices could serve as effective tools for the early prediction of successful weaning from HFNC therapy among hospitalized patients with COVID-19 pneumonia. However, the ROX index demonstrated the highest accuracy, sensitivity, and specificity compared to the HROX and delta-HR indices. Adding heart rate assessment does not add value to the ROX index, as heart rate is a variable that can change based on various conditions such as activity, emotions, illness, etc.
The study had a few limitations. Firstly, the timing of the heart rate record to calculate HROX or delta-HR indices may have affected their accuracy. As heart rate is a sensitive factor, patients with arrhythmias or those taking heart rate control medications (e.g., beta blockers) on the day of HFNC weaning may not have had accurate indices to predict the outcome of HFNC weaning. The timing of the heart rate record is debated, and we chose 2 a.m. as patients have less activity during that time, leading to less interference for heart rate analysis. Secondly, the parameters involved in the ROX index can vary throughout the day or in different clinical situations, which may lead to possible biases. Recording parameters at the same time every day when patients had no activity for treatment decreased interference with result analysis. Thirdly, for the calculation of HR improvement and HROX, patients with arrhythmia or those taking medications such as beta-blockers were excluded. This exclusion may limit the generalizability of the results, as hospitalized patients often have chronic cardiovascular diseases. These results should be confirmed in future validation cohorts before proposing the use of these indices for predicting successful weaning from HFNC in patients with COVID-19 pneumonia. However, our results suggest that monitoring COVID-19 pneumonia patients weaning from HFNC therapy with the ROX, HROX, and delta-HR indices could predict clinical outcomes.
Conclusions
The ROX index has the highest accuracy for predicting successful weaning off HFNC treatment in patients with COVID-19 pneumonia. While HROX and delta-HR indices can serve as alternative tools, it is recommended to verify these indices and determine the optimal cut-off value for determining separation from HFNC therapy through a large prospective cohort study.
Supporting information
S1 File. Dataset of patients with COVID-19 pneumonia.
https://doi.org/10.1371/journal.pone.0297624.s001
(XLSX)
Acknowledgments
The authors would like to thank Sam Ormond, Faculty of Medicine, Thammasat University, for proofreading this manuscript.
References
- 1. Yd Gao, Ding M, Dong X, Zhang Jj, Kursat Azkur A, Azkur D, et al. Risk factors for severe and critically ill COVID‐19 patients: a review. Allergy. 2021;76:428–55. pmid:33185910
- 2. Frat J-P, Thille AW, Mercat A, Girault C, Ragot S, Perbet S, et al. High-flow oxygen through nasal cannula in acute hypoxemic respiratory failure. New England Journal of Medicine. 2015;372:2185–96. pmid:25981908
- 3. Crimi C, Noto A, Cortegiani A, Impellizzeri P, Elliott M, Ambrosino N, et al. Noninvasive respiratory support in acute hypoxemic respiratory failure associated with COVID-19 and other viral infections. 2020:2020.05. 24. pmid:32756535.
- 4. Chandel A, Patolia S, Brown AW, Collins AC, Sahjwani D, Khangoora V, et al. High-flow nasal cannula therapy in COVID-19: using the ROX index to predict success. Respiratory care. 2021;66:909–19. pmid:33328179
- 5. Crimi C, Pierucci P, Renda T, Pisani L, Carlucci AJRC. High-flow nasal cannula and COVID-19: a clinical review. 2022;67:227–40. pmid:34521762
- 6. Perkins GD, Ji C, Connolly BA, Couper K, Lall R, Baillie JK, et al. Effect of noninvasive respiratory strategies on intubation or mortality among patients with acute hypoxemic respiratory failure and COVID-19: the RECOVERY-RS randomized clinical trial. 2022;327:546–58. pmid:35072713
- 7. Crimi C, Noto A, Madotto F, Ippolito M, Nolasco S, Campisi R, et al. High-flow nasal oxygen versus conventional oxygen therapy in patients with COVID-19 pneumonia and mild hypoxaemia: a randomised controlled trial. 2023;78:354–61. pmid:35580898
- 8. Kang BJ, Koh Y, Lim C-M, Huh JW, Baek S, Han M, et al. Failure of high-flow nasal cannula therapy may delay intubation and increase mortality. Intensive care medicine. 2015;41:623–32. pmid:25691263
- 9. Roca O, Messika J, Caralt B, García-de-Acilu M, Sztrymf B, Ricard J-D, et al. Predicting success of high-flow nasal cannula in pneumonia patients with hypoxemic respiratory failure: The utility of the ROX index. Journal of critical care. 2016;35:200–5. pmid:27481760
- 10. Rodriguez M, Thille AW, Boissier F, Veinstein A, Chatellier D, Robert R, et al. Predictors of successful separation from high-flow nasal oxygen therapy in patients with acute respiratory failure: a retrospective monocenter study. Annals of intensive care. 2019;9:1–7. pmid:31511996
- 11. Sztrymf B, Messika J, Mayot T, Lenglet H, Dreyfuss D, Ricard J-D. Impact of high-flow nasal cannula oxygen therapy on intensive care unit patients with acute respiratory failure: a prospective observational study. Journal of critical care. 2012;27:324. e9–. e13. pmid:21958974
- 12. Roca O, Caralt B, Messika J, Samper M, Sztrymf B, Hernández G, et al. An index combining respiratory rate and oxygenation to predict outcome of nasal high-flow therapy. American journal of respiratory and critical care medicine. 2019;199:1368–76. pmid:30576221
- 13. Patel M, Chowdhury J, Mills N, Marron R, Gangemi A, Dorey-Stein Z, et al. Utility of the ROX Index in Predicting Intubation for Patients With COVID-19–Related Hypoxemic Respiratory Failure Receiving High-Flow Nasal Therapy: Retrospective Cohort Study. 2021;2:e29062. pmid:34548669
- 14. Hu M, Zhou Q, Zheng R, Li X, Ling J, Chen Y, et al. Application of high-flow nasal cannula in hypoxemic patients with COVID-19: a retrospective cohort study. BMC Pulm Med. 2020;20:324. pmid:33357219
- 15. Mellado-Artigas R, Mujica LE, Ruiz ML, Ferreyro BL, Angriman F, Arruti E, et al. Predictors of failure with high-flow nasal oxygen therapy in COVID-19 patients with acute respiratory failure: a multicenter observational study. Journal of Intensive Care. 2021;9:1–9. pmid:33673863
- 16. Goh KJ, Chai HZ, Ong TH, Sewa DW, Phua GC, Tan QL. Early prediction of high flow nasal cannula therapy outcomes using a modified ROX index incorporating heart rate. Journal of intensive care. 2020;8:1–14. pmid:32587703