Severe obstructive sleep apnea (sOSA) and preoperative hypoxemia are risk factors of postoperative complications. Patients exhibiting the combination of both factors are probably at higher perioperative risk. Four scores (STOP-Bang, P-SAP, OSA50, and DES-OSA) are currently used to detect OSA patients preoperatively. This study compared their ability to specifically detect hypoxemic sOSA patients.
One hundred and fifty-nine patients scheduled for an overnight polysomnography (PSG) were prospectively enrolled. The ability of the four scores to predict the occurrence of hypoxemic episodes in sOSA patients was compared using sensitivity (Se), specificity (Sp), Youden Index, Cohen kappa coefficient, and the area under ROC curve (AUROC) analyses.
OSA50 elicited the highest Se [95% CI] at detecting hypoxemic sOSA patients (1 [0.89–1]) and was significantly more sensitive than STOP-Bang in that respect. DES-OSA was significantly more specific (0.58 [0.49–0.66]) than the three other scores. The Youden Index of DES-OSA (1.45 [1.33–1.58]) was significantly higher than those of STOP-Bang, P-SAP, and OSA50. The AUROC of DES-OSA (0.8 [0.71–0.89]) was significantly the largest. The highest Kappa value was obtained for DES-OSA (0.33 [0.21–0.45]) and was significantly higher than those of STOP-Bang, and OSA50.
In our population, DES-OSA appears to be more effective than the three other scores to specifically detect hypoxemic sOSA patients. However prospective studies are needed to confirm these findings in a perioperative setting.
Clinical trial registration
Citation: Deflandre E, Piette N, Bonhomme V, Degey S, Cambron L, Poirrier R, et al. (2018) Comparison of clinical scores in their ability to detect hypoxemic severe OSA patients. PLoS ONE 13(5): e0196270. https://doi.org/10.1371/journal.pone.0196270
Editor: James Andrew Rowley, Wayne State University, UNITED STATES
Received: June 8, 2017; Accepted: April 10, 2018; Published: May 7, 2018
Copyright: © 2018 Deflandre et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and Supporting Information files.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Obstructive Sleep Apnea (OSA) is a risk factor of perioperative complications. This risk increases with the severity of OSA , and becomes significant only in severe OSA patients (sOSA) . OSA is defined as severe when the Apnea Hypopnea Index (AHI), i.e. the number of apnea and hypopnea per hour of sleep, equals or exceeds 30 events per hour . AHI is classically obtained during an overnight polysomnography (PSG).
Hypoxic events in OSA patients may be responsible for several pathophysiological modifications [4–12]. Hypoxemia potentially contributes to the development of a coronary artery disease in OSA patients , and hence increases the risk of postoperative cardiovascular complications [4–8, 14, 15]. We recently reported that nocturnal episodes of hypoxemia occurred in some but not all sOSA patients [16–18]. Patients combining these two risk factors (severe OSA and hypoxemia) have probably a higher postoperative risk. This hypothesis is based however on data reported outside the postoperative setting. Accordingly, this combination increases the risk of kidney dysfunction and of diabetic retinal complications in humans [19, 20], as well as cardiovascular remodeling in a mouse model of sleep apnea . Consequently, preoperative screening to detect patients combining sOSA and nocturnal hypoxemia may be important for the anesthesiologists .
Several predictive scores have been proposed and validated to detect OSA patients preoperatively. The four scores most often used in anesthesia are the STOP-Bang, the P-SAP, the OSA50, and the DES-OSA scores [22–27]. These scores are illustrated in Fig 1. Prediction of hypoxemia is also clinically relevant. However, the STOP-Bang score does not predict hypoxemia in patients recovering from noncardiac surgery . The ability of the three other scores has never been tested in this respect. We therefore compared the efficiency of these four scores to predict hypoxemia particularly in sOAS patients who are the patients at highest postoperative risk.
This study was approved by the Ethics Committee of the Clinique Saint-Luc of Bouge (Clinique Saint-Luc of Bouge, Belgium, CE SLBO 2014/02, 20th of April 2014; chairperson: Dr. van der Rest). Because the study was observational, the Ethics Committee requested oral informed consent only. We assessed 168 adult patients scheduled for an overnight PSG at the Clinique Saint-Luc of Bouge, Belgium, between the 1st of July 2015 and the 15th of October 2015, and finally included 159 patients (see Fig 2 for CONSORT trial profile). Exclusion criteria included: chronic obstructive pulmonary disease GOLD 3 and 4, insofar as those patients may experience diurnal hypoxemia independently from OSA , chronic heart failure, illicit drug intake or smoking (e.g. cannabis), and pregnancy. This study was registered with Clinical Trials (ref: NCT02050685).
Before PSG, all patients were weighed and measured to determine the body mass index (BMI). Parameters to calculate the four scores compared in this study were recorded by a blind investigator. Electrophysiological parameters and capillary oxygen saturation (SpO2) were monitored using the polysomnographic DREAM® device (Medatec®, Brussels, Belgium). Signals were recorded on a dedicated computer using the Brainnet® for Windows Software (Medatec®, Brussels, Belgium). The analysis of PSG data included the determination of sleep stages according to the Rechtschaffen and Kales scoring rules . Periods of arousal were also considered, according to the American Sleep Disorders Association . Criteria for the definition of apnea and hypopnea agreed with the American Academy of Sleep Medicine rules . A hypopnea was defined as a reduction in airway flow ≥ 30% and associated with a SpO2 desaturation ≥ 4%. A second blind investigator collected the parameters derived from the PSG. As described in the Introduction, we only take into account severe OSA patients (patients with an AHI greater than 30 events per hour).
Hypoxemia is classically defined as a PaO2 less than 60 mmHg on room air . SpO2 recording is usually performed during PSG. To the best of our knowledge, there is no unanimous definition of hypoxemia based on mean SpO2 during the night of a PSG. We therefore combined criteria used by others authors [7, 33, 34]. Hypoxemic patients were defined as those with a mean peripheral saturation in oxygen (SpO2) during sleep < 90% or with more than 10% of their sleep time with a SpO2 < 90%.
The threshold values used for the present study were: a score ≥ 5 for the STOP-Bang score, a score ≥ 4 for the P-SAP score, a score ≥ 5 for the OSA50 score, and a score ≥ 7 for the DES-OSA score. The analyses of the PSG indicate that a positive test is the combination of severe OSA and nocturnal hypoxemia. The ability of the four scores to detect hypoxemic sOSA patients was compared using sensitivity (Se), specificity (Sp), Youden Index (YI) which was calculated as: [YI = sensitivity + specificity– 1] and varied in a range between -1 and 1, positive predictive value (PPV), negative predictive value (NPV), Positive Likelihood Ratio (+LR), Negative Likelihood Ratio (-LR), and area under ROC curve (AUROC) analyses. We also used the Cohen Kappa coefficient to assess the degree of concordance between clinical scores and the probability of hypoxemic sOSA. Se, Sp, PPV, NPV, +LR, -LR are expressed in percentages. AUROC and Cohen Kapa coefficient are expressed in values (range from 0 to 1). All the results are expressed with their 95% Confidence Intervals. The Mc Nemar test was applied to compare the sensitivities and specificities. A z-test was used to compare AUROC . A two-tailed P value < 0.05 was considered statistically significant. Data were analyzed using XLSTAT for Mac® (version 2015.5.01, Addinsoft SARL®, Paris, France) and MedCalc® (version 15.11.0 MedCalc Software bvba®, Ostend, Belgium). For our Se/Sp analyses, we used the threshold values defined by the authors of each score as being indicative of severe OSA: ≥ 5 for STOP-Bang, ≥ 4 for P-SAP, ≥ 5 for OSA50, and ≥ 7 for DES-OSA) [22–26]. We estimated that a sample size of 146 patients would provide a 90% power for detecting a 20% difference in Se or Sp between groups at a Bonferroni corrected alpha level of 0.008 (6 paired between-score comparisons). 159 patients were finally included in this study.
The patients’ characteristics are presented in Table 1. Complete results of Se, Sp, YI, PPV, NPV, +LR, -LR are provided in the S1 Appendix and summarized in Table 2. The McNemar tests revealed that the Se [95% CI] of STOP-Bang (0.8 [0.65–0.9]) at detecting hypoxemic patients was significantly lower as compared to P-SAP (0.97 [0.86–1]), and OSA50 (1 [0.89–1]). The McNemar tests showed that the Sp of DES-OSA (0.58 [0.49–0.66]) was significantly higher than the one of STOP-Bang (0.37 [0.29–0.46]), P-SAP (0.3 [0.23–0.39]), and OSA50 (0.09 [0.05–0.16]). The highest YI was obtained for DES-OSA (0.45 [0.38–0.53]). It was significantly greater than the YI of STOP-Bang (0.18 [0.12–0.34]), P-SAP (0.28 [0.21–0.35], and OSA50 (0.09 [0.05–0.14]). The best PPV and +LR were obtained for the DES-OSA score, while the best NPV and–LR were provided by the OSA50 score. S2 Appendix summarizes the results for mild (AHI between 5 and 14,9 events per hour) and moderate (AHI between 15 and 29.9 events per hour) OSA.
Efficiency is expressed in terms of sensitivity (Se), specificity (Sp), Youden Index (YI), positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (+LR), and negative likelihood ratio (-LR).
Complete results of AUROC analyses are detailed in the S1 Appendix, summarized in Table 3, and illustrated in Fig 3. The AUROC of DES-OSA (0.8 [0.71–0.89]) was significantly larger than the AUROC of the three other scores, namely STOP-Bang (0.65 [0.54–0.75]), P-SAP (0.69 [0.59–0.79]), and OSA50 (0.61 [0.5–0.71]). Results of Cohen Kappa coefficient analyses are presented in Table 3. The highest Kappa value was obtained for DES-OSA (0.33 [0.21–0.45]) and was significantly different as compared to STOP-Bang (0.11 [0.01–0.21]), and OSA50 (0.05 [0.02–0.08]).
Our findings suggest that the DES-OSA score is more efficient at identifying hypoxemic severe OSA patients than the STOP Bang, the OSA50, and the P-SAP scores. Whereas the STOP Bang, the OSA50, and the P-SAP scores were developed to detect all OSA patients (AHI ≥ 5) regardless of the potential risk of complications, the DES-OSA score more specifically identifies sOSA patients, who are at higher risk of postoperative complications, and for whom ambulatory surgery is questionable [2, 37, 38]. The better performance of DES-OSA in predicting the combination of sOSA and hypoxemia is, therefore, not surprising.
In our population, approximately half of the sOSA patients experienced nocturnal hypoxemia (sOSA = 51.2% of our population; combination sOSA and hypoxemia = 25.2% of our population). This confirms our previous findings . Hypoxemic sOSA who cumulate risks to develop postoperative complications might represent an important subgroup of patients to identify preoperatively. However, so far, no score was assessed as to its ability to detect hypoxemic sOSA patients. The four clinical scores predictive of OSA compared in this study were selected because of their speed and ease of completion in a preoperative setting. The ideal score must be sensitive and specific enough to maximally identify hypoxemic sOSA patients, with a minimum of false positives. In this respect, the Youden Index, which corresponds to the sum of Se and Sp, is particularly of interest to compare the performance of the different scores. The DES-OSA score was associated with the highest Youden Index, significantly higher than those of the STOP-Bang, the P-SAP, and the OSA50 scores. The OSA50 score exhibits the lowest Youden Index which is significantly lower as compared to the DES-OSA and the P-SAP scores. Although our study did not concern preoperative patients, the clinical performance and relevance of the OSA50 score to detect hypoxemic sOSA patients during a preoperative screening is probably questionable. In a recent study, Khanna et al. reported that the STOP-Bang score does not predict postoperative hypoxemia in a surgical population . Our study in patients addressed to a sleep clinic considers nocturnal hypoxemia outside any surgical setting. However, we confirm the poor efficiency of the STOP-Bang score at detecting hypoxemic sOSA patients.
Our study has some limitations. The study was performed in patients referred to a sleep center, which introduces a selection bias. Patients were explored for medical reasons, most often suspicion of OSA, but also other medical conditions such as insomnia or restless leg syndrome, and not necessarily in a perioperative setting. Consequently, we had a higher incidence of OSA in our population, as compared to a normal preoperative population. Most studies involving OSA screening had also the same limitation . Future studies are therefore required in surgical patients to extend and confirm our results. In addition to nocturnal hypoxemia evidenced during PSG, those studies should investigate the incidence of postoperative hypoxemia, like in the study of Khana et al. , which may contribute to several postoperative complications.
In conclusion, outside a perioperative setting, the DES-OSA score appears more efficient than the STOP-Bang, the P-SAP, and the OSA50 scores at detecting specifically hypoxemic sOSA patients. Identifying those patients is clinically relevant since they are probably at high risk of postoperative complications. This identification could prompt a more appropriate and intensive perioperative management using resources such as continuous positive airway pressure, postoperative monitoring in post-anesthesia care or intensive care unit, to improve outcome and safety of these patients, whether inpatients or outpatients. Future studies should extend our results in surgical patients and investigate the potential benefits of a tailored perioperative management for hypoxemic sOSA patients.
S1 Appendix. Complete results of sensitivity, specificity, Youden Index, positive predictive value, negative predictive value, positive likelihood ratio, negtaive likelihood ratio.
S2 Appendix. Statistics for mild and moderate OSA.
This report was previously presented, in part, at the SASM Annual Meeting (Chicago, 2016).
- 1. Mokhlesi B, Hovda MD, Vekhter B, Arora VM, Chung F, Meltzer DO. Sleep-disordered breathing and postoperative outcomes after elective surgery: analysis of the nationwide inpatient sample. Chest. 2013;144(3):903–14. pmid:23538745
- 2. Mutter TC, Chateau D, Moffatt M, Ramsey C, Roos LL, Kryger M. A matched cohort study of postoperative outcomes in obstructive sleep apnea: could preoperative diagnosis and treatment prevent complications? Anesthesiology. 2014;121(4):707–18. pmid:25247853
- 3. Adesanya A, Lee W, Greilich N, Joshi G. Perioperative management of obstructive sleep apnea. Chest. 2010;138(6):1489–98. pmid:21138886
- 4. Branson RD. The scientific basis for postoperative respiratory care. Respir Care. 2013;58(11):1974–84. pmid:24155356
- 5. Habre W, Petak F. Perioperative use of oxygen: variabilities across age. Br J Anaesth. 2014;113 Suppl 2:ii26–36.
- 6. Hwang D, Shakir N, Limann B, Sison C, Kalra S, Shulman L, et al. Association of Sleep-Disordered Breathing With Postoperative Complications*. Chest. 2008;133(5):1128–34. pmid:18339794
- 7. Chung F, Zhou L, Liao P. Parameters from preoperative overnight oximetry predict postoperative adverse events. Minerva Anestesiol. 2014;80(10):1084–95. pmid:24492663
- 8. Kabon B, Kurz A. Optimal perioperative oxygen administration. Curr Opn Anaesthesiol. 2006;19(1):11–8.
- 9. Wu J, Stefaniak J, Hafner C, Schramel JP, Kaun C, Wojta J, et al. Intermittent Hypoxia Causes Inflammation and Injury to Human Adult Cardiac Myocytes. Anesth Analg. 2016;122(2):373–80. pmid:26505576
- 10. Appleton SL, Vakulin A, McEvoy RD, Wittert GA, Martin SA, Grant JF, et al. Nocturnal Hypoxemia and Severe Obstructive Sleep Apnea are Associated with Incident Type 2 Diabetes in a Population Cohort of Men. J Clin Sleep Med. 2015;11(6):609–14. pmid:25766697
- 11. Jelic S, Padeletti M, Kawut SM, Higgins C, Canfield SM, Onat D, et al. Inflammation, Oxidative Stress, and Repair Capacity of the Vascular Endothelium in Obstructive Sleep Apnea. Circulation. 2008;117(17):2270–8. pmid:18413499
- 12. Deflandre E, Degey S, Opsomer N, Brichant JF, Joris J. Obstructive Sleep Apnea and Smoking as a Risk Factor for Venous Thromboembolism Events: Review of the Literature on the Common Pathophysiological Mechanisms. Obes Surg. 2016;26(3):640–8. pmid:26694212
- 13. Hayashi M, Fujimoto K, Urushibata K, Uchikawa S, Imamura H, Kubo K. Nocturnal oxygen desaturation correlates with the severity of coronary atherosclerosis in coronary artery disease. Chest. 2003;124(3):936–41. pmid:12970020
- 14. Long Y, Liu H-H, Yu C, Tian X, Yang Y-R, Wang C, et al. Pre-Existing Diseases of Patients Increase Susceptibility to Hypoxemia during Gastrointestinal Endoscopy. PloS one. 2012;7(5):e37614. pmid:22629430
- 15. Lecube A, Sampol G, Lloberes P, Romero O, Mesa J, Hernández C, et al. Diabetes Is an Independent Risk Factor for Severe Nocturnal Hypoxemia in Obese Patients. A Case-Control Study. PloS one. 2009;4(3):e4692. pmid:19262746
- 16. Deflandre EP, Bonhomme VL, Brichant JE, Joris JL. What mediates postoperative risk in obstructive sleep apnea: airway obstruction, nocturnal hypoxia, or both? Can J Anaest. 2016;63(9):1104–5.
- 17. Deflandre E. In reply: Perioperative nocturnal hypoxemia matters in surgical patients with obstructive sleep apnea. Can J Anaest. 2017;64(1):111.
- 18. Deflandre E, Kempeneers D, Degey S, Poirrier R, Legros P, Brichant JF, et al. Risk factors for nocturnal hypoxemia in severe OSA patients. Minerva Anestesiol. 2017 May;83(5):449–456. pmid:27922255
- 19. Marrone O, Battaglia S, Steiropoulos P, Basoglu OK, Kvamme JA, Ryan S, et al. Chronic kidney disease in European patients with obstructive sleep apnea: the ESADA cohort study. J Sleep Res. 2016;25(6):739–745. pmid:27191365
- 20. Banerjee D, Leong WB, Arora T, Nolen M, Punamiya V, Grunstein R, et al. The potential association between obstructive sleep apnea and diabetic retinopathy in severe obesity-the role of hypoxemia. PloS one. 2013;8(11):e79521. pmid:24260240
- 21. Castro-Grattoni AL, Alvarez-Buve R, Torres M, Farre R, Montserrat JM, Dalmases M, et al. Intermittent Hypoxia-Induced Cardiovascular Remodeling Is Reversed by Normoxia in a Mouse Model of Sleep Apnea. Chest. 2016;149(6):1400–8. pmid:26836908
- 22. Chung F, Yegneswaran B, Liao P, Chung SA, Vairavanathan S, Islam S, et al. STOP Questionnaire: A Tool to Screen Patients for Obstructive Sleep Apnea. Anesthesiology. 2008;108(5):812–21. pmid:18431116
- 23. Chung F, Subramanyam R, Liao P, Sasaki E, Shapiro C, Sun Y. High STOP-Bang score indicates a high probability of obstructive sleep apnoea. Br J Anaesth. 2012;108(5):768–75. pmid:22401881
- 24. Ramachandran SK, Kheterpal S, Consens F, Shanks A, Doherty TM, Morris M, et al. Derivation and Validation of a Simple Perioperative Sleep Apnea Prediction Score. Anesth Analg. 2010;110(4):1007–15. pmid:20357144
- 25. Chai-Coetzer CL, Antic NA, Rowland LS, Catcheside PG, Esterman A, Reed RL, et al. A simplified model of screening questionnaire and home monitoring for obstructive sleep apnoea in primary care. Thorax. 2011;66(3):213–9. pmid:21252389
- 26. Deflandre E, Degey S, Brichant JF, Poirrier R, Bonhomme V. Development and Validation of a Morphologic Obstructive Sleep Apnea Prediction Score: The DES-OSA Score. Anesth Analg. 2016;122(2):363–72. pmid:26599791
- 27. Deflandre E, Degey S, Brichant JF, Donneau AF, Frognier R, Poirrier R, et al. Pre-Operative Ability of Clinical Scores to Predict Obstructive Sleep Apnea (OSA) Severity in Susceptible Surgical Patients. Obesity surgery. 2017;27(3):716–729. pmid:27599985
- 28. Khanna AK, Sessler DI, Sun Z, Naylor AJ, You J, Hesler BD, et al. Using the STOP-BANG questionnaire to predict hypoxaemia in patients recovering from noncardiac surgery: a prospective cohort analysis. Br J Anaesth. 2016;116(5):632–40. pmid:27106966
- 29. Global Initiative for Chronic Obstructive Lung Disease (GOLD). Global strategy for the diagnosis m, and prevention of chronic obstructive pulmonary disease: revised 2011. Available from: http://www.goldcopd.org/uploads/users/files/GOLD_Report_2011_Jan21.pdf
- 30. Rechtshaffen A, Kales A. A manual of standardized terminology and scoring system for sleep stages of human subjects.: U.S. Government Printing Office, Washington, DC, NIH Publication No. 204.; 1968.
- 31. ASDA. EEG arousals: scoring rules and examples: a preliminary report from the Sleep Disorders Atlas Task Force of the American Sleep Disorders Association. Sleep. 1992;15(2):173–84. pmid:11032543
- 32. Berry RB, Budhiraja R, Gottlieb DJ, Gozal D, Iber C, Kapur VK, et al. Rules for scoring respiratory events in sleep: update of the 2007 AASM Manual for the Scoring of Sleep and Associated Events. Deliberations of the Sleep Apnea Definitions Task Force of the American Academy of Sleep Medicine. J Clin Sleep Med. 2012;8(5):597–619. pmid:23066376
- 33. Varol Y, Anar C, Tuzel OE, Guclu SZ, Ucar ZZ. The impact of active and former smoking on the severity of obstructive sleep apnea. Sleep Breath. 2015;19(4):1279–84. pmid:25801280
- 34. Zalucky AA, Nicholl DD, Hanly PJ, Poulin MJ, Turin TC, Walji S, et al. Nocturnal Hypoxemia Severity and Renin-Angiotensin System Activity in Obstructive Sleep Apnea. Am J Respir Crit Care Med. 2015;192(7):873–80. pmid:26102156
- 35. Landis JR, Koch GG. An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics. 1977;33(2):363–74. pmid:884196
- 36. Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology. 1983;148(3):839–43. pmid:6878708
- 37. Ankichetty S, Chung F. Considerations for patients with obstructive sleep apnea undergoing ambulatory surgery. Curr Opn Anaesthesiol. 2011;24(6):605–11.
- 38. Practice Guidelines for the Perioperative Management of Patients with Obstructive Sleep Apnea: An Updated Report by the American Society of Anesthesiologists Task Force on Perioperative Management of Patients with Obstructive Sleep Apnea. Anesthesiology. 2014;120(2):268–86. pmid:24346178
- 39. Chung F, Abdullah HR, Liao P. STOP-Bang Questionnaire: A Practical Approach to Screen for Obstructive Sleep Apnea. Chest. 2016;149(3):631–8. pmid:26378880
- 40. Yokhana SS, Gerst DG, Lee DS, Badr MS, Qureshi T, Mateika JH. Impact of repeated daily exposure to intermittent hypoxia and mild sustained hypercapnia on apnea severity. J Appl Physiol. 2012;112(3):367–77. pmid:22052874