Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Artificial intelligence augmented home sleep apnea testing device study (AISAP study)

  • Sunil Sharma ,

    Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

    sunil.sharma@hsc.wvu.edu

    Affiliation Division of Pulmonary, Critical Care and Sleep Medicine, West Virginia University, Morgantown, WV, United States of America

  • Kassandra Olgers,

    Roles Conceptualization, Data curation, Methodology, Visualization, Writing – original draft, Writing – review & editing

    Affiliation Division of Pulmonary, Critical Care and Sleep Medicine, West Virginia University, Morgantown, WV, United States of America

  • Scott Knollinger,

    Roles Conceptualization, Data curation, Methodology, Project administration, Visualization, Writing – original draft, Writing – review & editing

    Affiliation Department of Respiratory Care, Ruby Memorial Hospital, Morgantown, WV, United States of America

  • Saivenkat Somisetty,

    Roles Conceptualization, Data curation, Methodology, Software, Writing – review & editing

    Affiliation Montgomery Township High School, Skillman, NJ, United States of America

  • Calvin Seol,

    Roles Conceptualization, Data curation, Methodology, Project administration, Software, Writing – review & editing

    Affiliation Eberly College of Arts and Science, West Virginia University, Morgantown, WV, United States of America

  • Naveena Yanamala

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

    Affiliation Rutgers Robert Wood Johnson Medical School, Division of Cardiovascular Disease and Hypertension, New Brunswick, NJ, United States of America

Abstract

Study objective

This study aimed to prospectively validate the performance of an artificially augmented home sleep apnea testing device (WVU-device) and its patented technology.

Methodology

The WVU-device, utilizing patent pending (US 20210001122A) technology and an algorithm derived from cardio-pulmonary physiological parameters, comorbidities, and anthropological information was prospectively compared with a commercially available and Center for Medicare and Medicaid Services (CMS) approved home sleep apnea testing (HSAT) device. The WVU-device and the HSAT device were applied on separate hands of the patient during a single night study. The oxygen desaturation index (ODI) obtained from the WVU-device was compared to the respiratory event index (REI) derived from the HSAT device.

Results

A total of 78 consecutive patients were included in the prospective study. Of the 78 patients, 38 (48%) were women and 9 (12%) had a Fitzpatrick score of 3 or higher. The ODI obtained from the WVU-device corelated well with the HSAT device, and no significant bias was observed in the Bland-Altman curve. The accuracy for ODI > = 5 and REI > = 5 was 87%, for ODI> = 15 and REI > = 15 was 89% and for ODI> = 30 and REI of > = 30 was 95%. The sensitivity and specificity for these ODI /REI cut-offs were 0.92 and 0.78, 0.91 and 0.86, and 0.94 and 0.95, respectively.

Conclusion

The WVU-device demonstrated good accuracy in predicting REI when compared to an approved HSAT device, even in patients with darker skin tones.

Introduction

Obstructive sleep apnea is a highly prevalent and underdiagnosed condition. Recent estimates of its prevalence suggest over a billion people in the world may be suffering from it [1]. Undiagnosed and untreated sleep apnea is associated with significant cardiovascular morbidity including, hypertension, congestive heart failure (CHF), atrial fibrillation, stroke, and acute coronary syndrome (ACS) [24]. Simple, accurate, and reproducible screening and testing at home can play a crucial role in the early detection of this potentially fatal disease. While a complete polysomnography (PSG) recording remains the gold standard for diagnosing Obstructive Sleep Apnea (OSA), its reliance on laboratory facilities and trained technicians for manual setup and scoring renders it both expensive and time-consuming. Home sleep apnea testing (HSAT) on the other hand has high failure rates, cumbersome for patients and only captures single night data only [5]. As a result, there is a demand for innovative approaches that can facilitate the automatic diagnosis and evaluation of Obstructive Sleep Apnea (OSA) severity. Additionally, application of Artificial Intelligence (AI) has shown promise in predicting, diagnosing, scoring and classifying sleep apnea accurately and cost effective way [6].

The WVU Device Platform has three main parts: a finger-tip pulse oximeter that securely fits on your index finger, a charging cable, and patented AI-augmented software for biomarker discovery and prediction of OSA severity. The finger-tip sensor is a two-channel device and is designed to acquire pulse oximetry and PPG. signals from the proximal phalangeal radialis indicis artery of the index finger (Fig 1). It provides a user-friendly interface, simpler than the 4-channel devices commonly used in hospital settings. The AI-augmented algorithm, integrating time-series analysis with regression/classification approaches for preprocessing SPO2 data to detect ODI and other biomarkers and predict the severity of OSA, underwent training on our prior data from 365 patients in overnight sleep studies using the 4% desaturation hypopnea criteria [7]. The WVU device utilizes a combination of oxygen saturation changes and additional clinical information (e.g., demographics, comorbidities, risk factors) to identify desaturation events and automatically diagnose and assess the severity of OSA (Fig 1). This study evaluates the WVU device in consecutive hospitalized patients with significant comorbidities with simultaneous use of a 4-channel CMS-approved HSAT device.

thumbnail
Fig 1. Schematic overview of the process for classifying sleep apnea severity and detecting respiratory events using machine learning algorithms.

This figure provides a detailed view of the workflow used to diagnose sleep apnea and detect specific respiratory events through machine learning techniques. The process encompasses two primary phases, executed in a structured sequence: initially, the detection of specific respiratory events, followed by the classification of sleep apnea severity. Both phases employ a combination of classification and regression-based algorithms to ensure a comprehensive and accurate analysis.

https://doi.org/10.1371/journal.pone.0303076.g001

Methodology

This prospective study was conducted at a tertiary care university hospital between November 2022 and June 2023. The study was approved by the West Virginia University Institutional Review Board (WVU IRB) (IRB # 2205569959A001). The University Hospital has a formal sleep apnea screening program under which patients admitted to medical wards who have BMI ≥ 30 and STOP positive are offered a home sleep apnea test (HSAT) to screen for undiagnosed sleep apnea [8]. The HSAT device is a commercially available Type 3 device with 4 channels, including airflow, chest movement, pulse oximetry, and heart rate. The screening is performed once the patient has recovered from an acute condition and is on minimal or no oxygen support. Additional inclusion criteria for final analysis included a minimum data recording time of 2 hours or more for both the test (WVU device) and control device (HSAT). Consecutive adult patients admitted to the medical ward who were clinically undergoing an HSAT were approached and consented to simultaneously test with the WVU-device after informed consent was obtained.

The device was placed in the contra-lateral hand by a trained respiratory therapist. The device was retrieved by the respiratory therapist the following morning and data was downloaded. A board-certified sleep physician-reviewed and interpreted the data. The staff were blinded to the results of HSAT and WVU-device which were analyzed by a non-clinical scientist. The WVU device was calibrated to identify events with a 4% drop in oxygen. A Fitzpatrick classification scale [9] was used to accurately assess and include a proportion of patients with darker skin tones (IV-VI) as suggested by the FDA as the device was pulse-oximeter based [10].

Statistical analysis

Statistical analysis was performed using R (ref) to assess the accuracy of the WVU-device AI-enabled Platform in predicting respiratory events. Continuous variables are expressed as the mean ± standard deviation, and categorical variables are expressed as percentages. To assess the agreement between the WVU-device and the CMS-approved HSAT device, we performed a Pearson correlation coefficient, and Bland-Altman analysis. The significance threshold was set at a two-sided p-value <0.05. Finally, the Receiver Operating Characteristic (AUC-ROC) analysis was conducted to further assess the accuracy, precision, and reliability of the two measurements (i.e., WVU-device ODI and HSAT REI values) above specific clinically relevant thresholds (e.g., ODI ≥ 5, REI ≥ 5; AHI ≥ 15, REI ≥ 15; ODI ≥ 30, REI ≥ 30). The Area Under the ROC Curve (AUC) was calculated to quantify the overall discriminative ability of the WVU-device in predicting the sleep apnea severity thresholds. An AUC of 1 indicates perfect discrimination, while 0.5 suggests no discrimination beyond chance.

Results

A total of seventy-eight consecutive adult patients (age >18 years) admitted to the medical wards who met the inclusion criteria were studied and analyzed. Of the 78 patients, 40(52%) were males, with a mean BMI of 36.82 ± 8.22 and a mean neck circumference of 19 ± 1.41 inch. Of the 78 patients, 68 (87%) were Caucasians, 7 (9.1%) were African Americans. Of the total cohort 59(77%) had hypertension, 23(30%) had congestive heart failure (CHF), 27(35%) had type 2 diabetes mellitus, 22(29%) had coronary artery disease (CAD) and 17(22%) had chronic obstructive pulmonary disease (COPD). Of the total of 78 patients 51(66%) were classified as obstructive sleep apnea (OSA) with AHI of ≥ 5 and 16 (21%) were classified as severe OSA with AHI ≥ 30. A total of 9 patients (12%) had a darker skin tone (Fitzpatrick class IV to VI)

Sleep parameters revealed a mean HSAT AHI of 17.9 ± 20.15 and mean ODI derived from WVU-device of 19.83 ± 22.06 (Table 1).

thumbnail
Table 1. Baseline characteristics, sleep parameters and comorbidities for the group (N = 78).

https://doi.org/10.1371/journal.pone.0303076.t001

A strong and statistically significant correlation was observed between the ODI derived from the WVU device and the AHI measured by the 4-channel HSAT device (r = 0.885, p<0.001; Fig 2). This demonstrates a robust positive relationship between the two variables, reinforcing the reliability and validity of the relationship between the two measures.

thumbnail
Fig 2. Scatterplot comparing HSAT device AHI to WVU-device ODI.

AHI = apnea-hypopnea index, ODI = oxygen desaturation index.

https://doi.org/10.1371/journal.pone.0303076.g002

The Bland-Altman plot illustrating the difference between AHI and WVU-device ODI did not reveal any significant bias and showed a mean difference of -2.06 events/h between AHI and WVU-device ODI was −2.3 (Fig 3). The accuracy for ODI > = 5 and AHI > = 5 was 87%, for ODI> = 15 and AHI > = 15 was 89% and for ODI> = 30 and AHI of > = 30 was 95% (Fig 4). The sensitivity and specificity for these ODI/AHI cut-offs were 0.92 and 0.78, 0.91 and 0.86 and 0.94 and 0.95. The positive predictive values for these cut-offs were 0.89, 0.84, 0.83 and negative predictive values were 0.84, 0.93, and 0.98 (Table 2).

thumbnail
Fig 3. The Bland-Altman plot comparing the Apnea-Hypopnea Index (AHI) from the HSAT device with the Oxygen Desaturation Index (ODI) from the WVU device.

https://doi.org/10.1371/journal.pone.0303076.g003

thumbnail
Fig 4. Receiver operating characteristic curve (ROC) of WVU-device ODI for prediction of the HSAT-device derived AHI thresholds.

This figure displays ROC curves for the WVU-device Oxygen Desaturation Index (ODI) predicting Apnea-Hypopnea Index (AHI) thresholds of (A) 5, (B) 15, and (C) 30 events per hour. Each curve shows the balance between sensitivity and specificity for different thresholds. AUC values indicate the predictive accuracy of WVU-device ODI for each AHI threshold. Performance metrics including sensitivity, specificity and AUC for each AHI threshold, derived from ROC analysis on a dataset of n = 78 samples is also provided.

https://doi.org/10.1371/journal.pone.0303076.g004

thumbnail
Table 2. Performance metrics of WVU-device ODI at HSAT device AHI thresholds of 5, 15, and 30 events/h.

AUROC = area under the receiver operator curve, NPV = negative predictive value, PPV = positive predictive value.

https://doi.org/10.1371/journal.pone.0303076.t002

A subset analysis of patients with a Fitzpatrick score of 3 or more (n = 9) revealed an accuracy of 0.89, sensitivity of 1.00, specificity of 0.8, positive predictive value of 0.8, and negative predictive value of 1.00.

Discussion

Our findings reveal a strong correlation between the ODI derived from the WVU device and the AHI measured by the 4-channel HSAT device. The negative predictive values for clinically significant sleep apnea, defined as an AHI / ODI of 15 and 30 events per hour, were notably high at 93% and 98%, indicating a low likelihood of missing the disease.

Currently, available HSAT devices are often cumbersome to use due to their requirement for multiple channels (four or more) and have been shown to have a high failure rate of nearly 33% [5]. This highlights the need for a more user-friendly screening tool. Given the high prevalence of sleep apnea and its underdiagnosis, the WVU device could hold promise as an ideal tool for detecting sleep apnea in both hospitalized and ambulatory settings. Emerging data also suggest that screening for sleep apnea in hospitalized patients and early initiation of positive airway pressure therapy may lead to reduced readmissions and healthcare costs [1113].

There is a significant concern within the scientific community regarding the reliability of pulse oximeters in patients with darker skin tones [14]. Studies have consistently shown that the oximeters tend to overestimate oxygen saturations in individuals with darker skin tones [14]. However, the oxygen signal sampling and the proprietary algorithm utilized by the WVU device appear to overcome this barrier. Notably, our study is the first to include patients with a Fitzpatrick classification scale of IV or higher (12%) to specifically assess darker skin tones. An additional advantage of the WVU device is its multi-night data capability, which recent studies have found to be more accurate [15]. Compared to prior studies with high-resolution pulse-oximetry, the WVU-device demonstrates an overall improvement in PPV (89% vs 82%, 84% vs 76%, and 83% vs 73% for REI≥5, ≥15, and ≥30 respectively), specificity(78% vs 67%, 89% vs 84%, and 95% vs 93% for REI≥5, ≥15, and ≥30 respectively) and accuracy (87% vs 84%, 89% vs 87%, and 95% vs 91% for REI≥5, ≥15, and ≥30 respectively), indicating enhanced precision in its diagnostic capabilities [7]. Moreover, WVU-device’s algorithm also incorporates a robust self-learning ability, enabling continuous refinement and optimization of diagnostic performance. This self-learning capability holds the promise of enhancing precision, particularly for racial minorities, thus addressing healthcare disparities. AI augmented WVU device can also help in development of clinical prediction models that rely on accurate oxygen desaturation index (ODI) especially in patients planning to have surgical procedures such as CABG, bariatric surgery or arthroplasty among others. The device may help with better triaging of patients and reducing wait time for surgeries [16].

The limitations of the study include that despite its multi-night capability the device was analyzed for one night only. This was because these patients were undergoing a clinically indicated one-night HSAT study during their hospital stay and a multi-night study would have inadvertently extended their hospital stay. We performed the sleep studies in a hospitalized environment, although these studies were unattended to replicate conditions at home. This proof-of-concept study would have to be validated in a home environment in a multi-night protocol. Secondly, as with most hospitalized patients, our cohort had significant comorbid conditions. Testing the community with a lesser burden of comorbid conditions may be required to establish generalizability. Furthermore, machine learning algorithms have their own limitations including need for external validation on external data and potential for bias if the training data is not representative of the patient population to be studied. The latter has the risk of widening health disparities in ethnic minorities along with introducing regional bias.

Potential challenges to incorporate WVU device could be limited comfort levels of physicians with this technology and inertia of already established care pathways. Future research should focus on validating the device in a larger, more diverse population, along with evaluating its impact on patient outcomes.

Conclusion

WVU device is a simple, cost-effective screening tool that correlates well with approved, commercially available level 3 HSAT devices. Considering that a large proportion of patients with sleep apnea remain undetected, this tool may be beneficial to healthcare organizations to evaluate patients with a high risk of sleep apnea, develop clinical prediction models and help reduce healthcare disparities.

References

  1. 1. Benjafield AV, Ayas NT, Eastwood PR et al (2019) Estimation of the global prevalence and burden of obstructive sleep apnoea: a literature-based analysis. Lancet Respir Med 7(8):687–698. pmid:31300334
  2. 2. Yaggi HK, Concato J, Kernan WN et al (2005) Obstructive sleep apnea as a risk factor for stroke and death. N Engl J Med 353(19):2034–41. pmid:16282178
  3. 3. Javaheri S, Caref EB, Chen E et al (2011) Sleep apnea test- ing and outcomes in a large cohort of Medicare beneficiaries with newly diagnosed heart failure. Am J Respir Crit Care Med 183(4):539–546. pmid:20656940
  4. 4. Mehra R, Benjamin EJ, Shahar E, et al. Association of nocturnal arrhythmias with sleep-disordered breathing: the Sleep Heart Health Study. Am J Respir Crit Care Med. 2006;173(8):910–916. pmid:16424443
  5. 5. Lux L, Boehlecke B, Lohr KN. Effectiveness of portable monitoring devices for diagnosing obstructive sleep apnea: Update of a systematic review [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2004 Sep 1.
  6. 6. Bazoukis G, Bollepalli SC, Chung CT, Li X, Tse G, Bartley BL, et al. Application of artificial intelligence in the diagnosis of sleep apnea. J Clin Sleep Med. 2023 Jul 1;19(7):1337–1363. pmid:36856067; PMCID: PMC10315608.
  7. 7. Stansbury R, Badami V, Rojas E, Naqvi SF, Easterling J, Abdelfattah M, et al, Sharma S. Addressing rural health disparity with a novel hospital sleep apnea screening: Precision of a high-resolution pulse oximeter in screening for sleep-disordered breathing. Sleep Breath. 2022 Dec;26(4):1821–1828. Epub 2022 Jan 20. pmid:35050464.
  8. 8. Sharma S, Stansbury RC. Sleep-Disordered Breathing in Hospitalized Patients: A Game Changer? Chest. 2021 Oct 18:S0012–3692(21)04110-6. Epub ahead of print. pmid:34673024.
  9. 9. Fitzpatrick T. The validity and practicality of sun-reactive skin types I through VI. Arch Dermatol 1988;124:869–71. pmid:3377516
  10. 10. U.S.D.o.H.a.H. Services. Pulse oximeters—premarket notification submissions [510(k)s] guidance for industry and food and drug administration staff. Rockville: U.S. Department of Health and Human Services, 2013.
  11. 11. Sharma S, Mather P, Gupta A, et al. Effect of early intervention with positive airway pressure therapy for sleep disordered breathing on six-month readmission rates in hospitalized patients with heart failure. Am J Cardiol. 2016;17(6):940–945. pmid:26830259
  12. 12. Scalzitti NJ, O’Connor PD, Nielsen SW, Aden , Brock MS, Taylor DM, et al. Obstructive sleep apnea is an independent risk factor for hospital readmission. J Clin Sleep Med.2018;14(5):753–758. pmid:29734972
  13. 13. Wickwire EM, Tom SE, Vadlamani A, et al. Older adult US Medicare beneficiaries with untreated obstructive sleep apnea are heavier users of health care than matched control patients [published online January 15, 2020]. J Clin Sleep Med. pmid:31957657
  14. 14. Cabanas AM, Fuentes-Guajardo M, Latorre K, León D, Martín-Escudero P. Skin Pigmentation Influence on Pulse Oximetry Accuracy: A Systematic Review and Bibliometric Analysis. Sensors (Basel). 2022 Apr 29;22(9):3402. pmid:35591092; PMCID: PMC9102088.
  15. 15. Punjabi N.M., Patil S., Crainiceanu C., Aurora R.N. Variability and misclassification of sleep apnea severity based on multi-night testing. Chest, 158 (1) (2020), pp. 365–373. pmid:32081650
  16. 16. Ho G, Cozowicz C, Wong J, Singh M, Lam E, Mörwald EE, et al. Patient preference survey: are patients willing to delay surgery if obstructive sleep apnea is suspected? BMC Anesthesiol. 2018 Sep 12;18(1):128. pmid:30208964; PMCID: PMC6136217.