https://orcid.org/0000-0002-4765-2917
Figures
Abstract
Introduction
An electronic nose (eNose) device has shown a high specificity and sensitivity to diagnose or rule out tuberculosis (TB) in the past. The aim of this study was to evaluate its performance in patients referred to INERAM.
Methods
Patients aged ≥15 years were included. A history, physical examination, chest radiography (CRX) and microbiological evaluation of a sputum sample were performed in all participants, as well as a 5-minute breath test with the eNose. TB diagnosis was preferably established by the gold standard and compared to the eNose predictions. Univariate and multivariate logistic regression analyses were performed to assess potential risk factors for erroneous classification results by the eNose.
Results
107 participants with signs and symptoms of TB were enrolled of which 91 (85.0%) were diagnosed with TB. The blind eNose predictions resulted in an accuracy of 50%; a sensitivity of 52.3% (CI 95%: 39.6–64.7%) and a specificity of 36.4% (CI 95%: 12.4–68.4%). Risk factors for erroneous classifications by the eNose were older age (multivariate analysis: OR 1.55, 95% CI 1.10–2.18, p = 0.012) and antibiotic use (multivariate analysis: OR 3.19, 95% CI 1.06–9.66, p = 0.040).
Conclusion
In this study, the accuracy of the eNose to diagnose TB in a tertiary referral hospital was only 50%. The use of antibiotics and older age represent important factors negatively influencing the diagnostic accuracy of the eNose. Therefore, its use should probably be restricted to screening in high-risk communities in less complex healthcare settings.
Citation: Coronel Teixeira R, Gómez L, González E, Jiménez de Romero N, González F, Aguirre S, et al. (2023) The accuracy of an electronic nose to diagnose tuberculosis in patients referred to an expert centre. PLoS ONE 18(2): e0276045. https://doi.org/10.1371/journal.pone.0276045
Editor: Tommaso Lomonaco, University of Pisa, ITALY
Received: January 17, 2022; Accepted: September 28, 2022; Published: February 7, 2023
Copyright: © 2023 Coronel Teixeira et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Although tuberculosis (TB) may seem a ‘silent pandemic’ compared to COVID-19, it is responsible, yearly, for 10 million cases and 1.2 million notified deaths [1, 2]. TB elimination is complex. On the one side due to different elements in the ‘cascade of care’ to establish the diagnosis but also because of factors concerning the bacteria itself as slow duplication and drug resistance which complicates treatment [1].
The “holy grail” to diagnose TB is a portable and rapid ‘point-of-care’ test providing a result within minutes [3]. Such a rapid test will never replace the current gold standard (positive culture of Mycobacterium tuberculosis complex) but may be used as a screening tool to either rule out TB or detect individuals with active disease. All conventional diagnostic methods, such as chest radiography (CXR), or sputum based test as Ziehl Neelsen (ZN) staining, GeneXpert MTB/RIF and culture of MTB are either expensive, time consuming, not widely available, and labour intensive [1].
An electronic nose (eNose) is a small, portable and a potentially rapid diagnostic ‘point-of-care’ device, that can detect gases in human breath and compares vectors derived from volatile organic compound (VOCs) from exhaled breath samples to a machine learning algorithm to diagnose a patient with TB [4]. Analysers for exhaled breath can be separated in two groups: 1. Systems for detecting and measuring a set of specific VOC’s. Usually, these systems comprise of a mobile unit for collecting the gas sample and another (fixed) unit for analysing it and separating the individual components. Separation techniques typically include GC-MS or IMS. 2. Electronic noses, often portable, possessing multiple different a-specific sensors (e.g. metal-oxide or conducting polymer) for measuring an integrated breath profile. In this case, Machine Learning techniques are applied for separating breath profiles of sick and healthy individuals. In order to accomplish this, a classifier (e.g. Artificial Neural Network, Random Forest, Support Vector Machine) has to be trained accordingly [5]. Until now, several studies have been performed with the eNose, which showed promising results [6–9]. In Paraguay, the eNose has been tested in adults (>18 years) with pulmonary TB, asthma or COPD and healthy controls, showing a high sensitivity (91%) and specificity (93%) to diagnose TB [10]. The eNose was also tested in an indigenous population, demonstrating its usefulness as a rule-out TB test in a remote area [11]. Additional studies testing the eNose in different settings are needed.
The aim of this study was to assess the accuracy of the eNose to classify individuals with signs and symptoms compatible with TB (in- and outpatient clinic) referred to the Paraguayan National Reference Center for respiratory diseases and TB (INERAM).
Methods
Study design and setting
A prospective study was performed in hospitalized and ambulatory patients from January 2016 to December 2017.
Participants
Participants, aged ≥ 15 years, presenting with respiratory symptoms for more than 15 days or having started anti-TB treatment already (<3 days before inclusion) were included. Authorization by parents and/or guardians was provided for participants under 18 years old. In Human Immunodeficiency Virus (HIV) positive individuals, the duration of respiratory symptoms was not considered. Exclusion criteria were respiratory failure or not willing or unable to sign informed consent.
Data collection
Patient demographics and clinical data (age, gender, BMI), information on co-morbidities (HIV, diabetes mellitus, asthma, chronic obstructive lung disease (COPD), or arterial hypertension), smoking and alcohol habits, drug abuse, co-medication (antibiotic use and/or TB treatment), food and beverage intake, and dental status (good condition: complete parts, no cavities or seals; regular: complete parts, presence of seals; bad: incomplete parts, presence of caries, incomplete seals) were recorded. Physical examination, a CXR, microbiological examination of sputum or biopsies and breath sampling with the eNose were performed in all participants. Microbiological examinations and breath tests conducted on the same day. TB symptoms (cough, fever, dyspnoea, night sweat, lymphadenopathy and/or haemoptysis) were noted. The study physicians assessed the CXRs for cavities, infiltrates, atelectasis and pleural effusion. All participants provided at least one sputum sample (produced spontaneously or induced by nebulization of 4 mL of hypertonic saline). ZN staining and mycobacterial culture (solid Ogawa-Kudoh medium) were performed in the laboratory of INERAM. In some cases, GeneXpert was performed according to the national guideline. The GeneXpert MTB/RIF® (Cepheid) samples were processed by the National Reference Laboratory (LCSP). The sputum samples were also evaluated for mycosis and common pathogens. Pulmonary TB (PTB) diagnosis was established by gold standard (positive culture of MTB complex). In case of a negative culture result, diagnosis was set by other strong supporting evidence (e.g. ZN positive, and/or MTB detected by GeneXpert [12]. In case no bacteriological prove was found a diagnosis was called a ‘clinical diagnosis’. In this last category, the improvement of clinical symptoms with anti-TB treatment was evaluated after two months (the initial phase of treatment). Extra-pulmonary TB (EPTB) patients were diagnosed by microbiological confirmation (positive culture of MTB) or with histopathological or chemical evidence supporting TB (in biopsies, pus or pleural fluid), or showing improvement of clinical condition after two months of anti-TB treatment. All participants were followed at the outpatient clinic. TB patients received anti-TB treatment according to the national guideline [13].
Breath sampling with the eNose
All participants underwent five-minute breath sampling using a nose clamp, through the eNose (AeonoseTM device, Zutphen, The Netherlands). The breath test measurements were done in the morning, always in the same place, in a room free of odours of gasses and alcohol and without dust. Cleaning of the eNose device on the outside was not performed unless it was needed because of stains on the device from a previous user. After every breath sampling a clean burn (approximately 10 minutes) was performed inside the device by heating of the metal sensors to 280 degrees Celsius, as part of the complete cycle process. Without this clean burn the eNose device cannot be used for another participant. Time of last meal/beverage in take/ medication or cigarette were recorded. All participants used a nose clamp and provided a breath sample by in- and exhaling for 5 min, through the AeonoseTM. The same device was used throughout the whole study. During sampling process, 36 measurement cycles, each containing 64 data points, were recorded per sensor. By performing this, each patient’s measurement comprised a data matrix with thousands of records. Proper reproducibility of the results is possible due to the sensor’s temperature control. However, even for sensors produced on the same wafer, thickness and ageing differences can cause small variations between sensors and AeonoseTM devices over time. This phenomenon is cope by normalizing the data. The data is then compressed using a Tucker 3-like algorithm. This results into a vector of 10 components per patient, redundant information and noise are remove. These resulting vectors and the results of the classification are used to train an artificial neural network (ANN). A double cross-validation was applied using the Leave-10%-Out method to minimize the risk of systematic errors [10].
The analysis of the breath samples was done by The eNose Company. The company did not take part in study design and logistics. To establish a new ‘training data set’ to train and optimize the previously used neural network (NN) from our calibration study [10], 31 de-blinded participants from this new cohort were added to the former training data set. For full description of the training and internal validation of the NN; see previous published research [6].
The data of the remaining study participants (n = 76) was kept blind to The eNose Company. These participants of the ‘blind data set’ were analysed in the optimized NN and then classified by the eNose Company as “TB yes” or “TB no”.
Statistical analysis
Statistical analysis was performed by using SPSS version 25.0 software (IBM Corporation, Armonk, NY, USA). Continuous variables were expressed as the mean (with standard deviation) or median (with range), and categorical variables were expressed as frequency (with percentage). Univariate testing of continuous variables was performed using Student t-test in case of normal distribution, and non-parametric tests for non-normal distribution. Categorical variables were analysed using X2 testing or Fisher’s exact test. Binary logistic regression analysis was performed to identify possible risk factors for wrong predictions in the NN algorithm. Variables with a p value ≤0.10 in univariate analysis were selected for multivariate logistic regression analysis. To prevent overfitting, the number of variables in the multivariate analysis was restricted by the number of patients with the outcome. All tests were two-sided, and a p value of ≤ 0.05 was considered statistically significant.
Inclusivity in global research
Additional information regarding the ethical, cultural, and scientific considerations specific to inclusivity in global research is included in the S1 Questionnaire.
Results
During the study period, 107 participants were enrolled with a mean age of 37 years and 70% being male (Table 1). TB was diagnosed in 91/107 (85%) of the participants. The most common TB characteristic was cough (87.9%), followed by fever (77.6%), dyspnoea (73.8%) and night sweat (72.0%). Four HIV positive TB patients were included and were on antiretroviral therapy. Fifty-six participants (52.3%) started TB treatment for ≤3 days when included in this study and providing the breath sample. Two participants received TB treatment in the past (>3 years ago). Table 2 shows microbiological outcomes of all participants. TB diagnosis was established by gold standard in 67 cases (74%). Twenty-six percent (24/91) were diagnosed differently; five PTB with ZN positive sputum only and another five were ‘clinical diagnosis’. Fourteen patients with EPTB (pleural or lymph node) also lacked bacteriological confirmation. The remaining group of 16/107 (15.0%) participants had an alternative diagnosis (e.g. pneumonia or chronic obstructive pulmonary disease (COPD) exacerbation).
The sensitivity and specificity of the NN in the ‘training data set’ (n = 137) were 80% (Confidence Interval (CI) 95%: 68.3–89.0%) and 84% (CI 95%: 74.0–91.0%), respectively. The positive predicted value (PPV) was 80% (CI 95%: 68.3–89.0%) and the negative predictive value (NPV) was 84% (CI 95%:74.0–89.0%). Fig 1A shows the ROC curve for this ‘training data set’. Twenty-four wrongly classified participants (24/137 (17.5%); 13/24 (54.2%) were receiving antibiotics and 5/24 (20.8%) were obese (Body Mass Index (BMI) ≥ 25). Fig 2 shows a raw sensor signals from the electronic nose.
The analysis of the NN with the ‘blind data set’ (n = 76 participants) resulted in 38 correct and 38 wrong classifications, yielding a sensitivity and specificity of 52.3% (CI 95%: 39.6–64.7%) and 36.4% (CI 95%: 12.4–68.4%) and an accuracy of 50%. The Positive Predicted Value (PPV) was 83.0% (CI 95%: 67.4–92.3%) with a Negative Predictive Value (NPV) of 11.4% (CI 95%: 3.7–27.7%). Fig 1B shows the ROC curve of the blind-prediction. Of the false positive cases (n = 7), three had comorbidities and six were under antibiotic treatment (other than anti-TB treatment) for at least 2 days. All false positive participants were admitted to the in-patient clinic and later discharged with a final diagnosis of pneumonia. Of the 31 false negative cases, 11/31 (35%) had comorbidities, 25/31 (81%) were under antibiotic treatment and 23/31 (74%) were under anti-TB treatment for at least 48 hours. Older age (35 years vs. 44 years, p = 0.028) and the use of antibiotics (60.5% vs. 81.8%, p = 0.043) were represented significantly different between the correct and wrongly classified participants (Table 3).
Table 4 shows the univariate and multivariate analysis of the possible risk factors associated with wrong predictions of TB diagnosis. Older age (univariate: OR 1.50, 95% CI 1.08–2.06, p = 0.016; multivariate: OR 1.55, 95% CI 1.10–2.18, p = 0.012) and the use of antibiotics (other than anti-TB treatment) for more than 24 hours (univariate: OR 2.89, 95% CI 1.01–8.23, p = 0.047; multivariate: OR 3.19, 95% CI 1.06–9.66, p = 0.040) were identified as risk factors associated with significant higher risk of erroneous classifications with the eNose.
Discussion
We evaluated the performance of the eNose in a referral hospital to diagnose TB in ill patients presenting with respiratory symptoms. We showed that in this study the accuracy of the eNose device was disappointingly only 50%. In this setting the eNose did not perform well enough according to the requisites for new screening tools or tests [3]. Older age and the use of antibiotics are significant risk factors for an incorrect prediction by the eNose with a very high OR for antibiotics use before TB diagnosis.
VOCs are unique to an individuals’ metabolism and may provide valuable information about the health condition of a subject. However, both host factors (genetics, co-morbidity, type of pathogen) and external factors (medication, toxic habits, food intake) may alter their breath prints, which could possibly result both in FN or FP classifications of the NN [14–16]. The use of antibiotics in a person with signs and symptoms compatible with TB is known to negatively influence the accuracy of conventional methods to diagnose TB, resulting in diagnostic delays in daily practice [17–22]. In our study, the use of antibiotics also significantly influenced the accuracy of the eNose device in a negative way. Even though there are several studies showing that co-medication influences breath prints, to the best of our knowledge, there are no publications specifically indicating the influence of antibiotics to an individuals’ VOCs.
We found that older age of the patients was also a risk factor for incorrect prediction by the eNose. Several studies have shown an association between age and breath prints [23–26]. The precise impact of ageing on metabolism is a topic that is widely discussed in the literature. It is not unreasonable to expect that this factor influences the concentrations of some metabolites and thereby altering breath prints [16].
In contrast to antibiotic use, anti-TB medication (< 3 days before inclusion of the participant) did not have a statistically significant impact on the predictive value of the eNose in this study. This may be explained because active TB is a disease that develops slowly over the course of weeks to months and requires a long duration of treatment to sterilize the patient from the bacteria. For that reason, it is understandable that a patients’ metabolism is not changing rapidly within the course of a few days. Therefore, the recent start of anti-TB treatment did not negatively influence the eNose predictions in our study cohort. We also did not find any association between BMI and incorrect predictions, even though there is evidence available that patients with a high BMI have different breath prints and a higher risk of false-positive test results compared to patients with a normal or low BMI [10, 23, 27, 28]. Dental status, time to last meal/ drinks, smoking and drug use neither appeared to be risk factors for a wrong prediction by the eNose in this study.
Our study has some limitations. Firstly, the sample size was small and consisted only of a small group of patients with an alternative diagnosis. Therefore, we might have introduced a design bias as the new NN ‘training data set’ might not have had enough pneumonia patients to train the new diagnostic algorithm correctly as the previous cohort consisted of PTB patients, patients with obstructive airway disease and healthy controls. Analysing larger cohorts will increase the accuracy of the eNose by establishing a more robust neural network algorithm. Secondly, the gold standard to establish TB diagnosis was not accomplished in all patients (mainly patients with EPTB) and it is possible that these patients in fact do not have active TB and the eNose classification was correctly made.
Conclusion
In this study, the accuracy of the eNose to diagnose TB in a tertiary referral hospital was only 50%. Factors associated with wrong predictions by the eNose were antibiotic use and older age of the participants. For complex referral patients who mostly have used antibiotics to rule out a simple pneumonia first, as recommended by the WHO, the eNose showed not to be useful. The eNose is likely to be more suitable as a TB rule-out test in less complex healthcare settings.
Acknowledgments
We would like to thank all participants and the staff of INERAM for their collaboration in collecting data and enabling us to perform this study.
References
- 1. World Health Organization (WHO). Global tuberculosis report 2020. 2020.
- 2. Coronel Teixeira R, Aguirre S, Pérez Bejarano D. Thinking about tuberculosis in times of COVID‐19. Journal of Internal Medicine 2021;289(4):589–90. pmid:33078450
- 3.
World Health Organization (WHO). Systematic screening for active tuberculosis: principles and recommendations: World Health Organization 2013.
- 4. Licht J-C, Grasemann H. Potential of the Electronic Nose for the Detection of Respiratory Diseases with and without Infection. International Journal of Molecular Sciences 2020;21(24):9416. pmid:33321951
- 5. Haworth J, Pitcher C, Ferrandino G, et al. Breathing new life into clinical testing and diagnostics: perspectives on volatile biomarkers from breath. Critical Reviews in Clinical Laboratory Sciences 2022;1(20):353–72. pmid:35188863
- 6. Zetola NM, Modongo C, Matsiri O, et al. Diagnosis of pulmonary tuberculosis and assessment of treatment response through analyses of volatile compound patterns in exhaled breath samples. Journal of Infection 2017;74(4):367–76. pmid:28017825
- 7. Bruins M, Rahim Z, Bos A, et al. Diagnosis of active tuberculosis by e-nose analysis of exhaled air. Tuberculosis 2013;93(2):232–38. pmid:23127779
- 8. Saktiawati AM, Stienstra Y, Subronto YW, et al. Sensitivity and specificity of an electronic nose in diagnosing pulmonary tuberculosis among patients with suspected tuberculosis. PLoS One 2019;14(6):e0217963. pmid:31194793
- 9. Mohamed E, Mohamed M, Moustafa M, et al. Qualitative analysis of biological tuberculosis samples by an electronic nose-based artificial neural network. The International Journal of Tuberculosis and Lung Disease 2017;21(7):810–17. pmid:28633707
- 10. Teixeira RC, Rodríguez M, de Romero NJ, et al. The potential of a portable, point-of-care electronic nose to diagnose tuberculosis. Journal of Infection 2017;75(5):441–47. pmid:28804027
- 11. Coronel Teixeira R, IJdema D, Gómez C, et al. The electronic nose as a rule‐out test for tuberculosis in an indigenous population. Journal of Internal Medicine 2021.
- 12.
World Health Organization (WHO). Definitions and reporting framework for tuberculosis–2013 revision: updated December 2014 and January 2020: World Health Organization, 2013.
- 13. Aguirre S, Román M, Gomez R, et al. Guía nacional para el manejo de la tuberculosis: Servicios de salud locales, distritales, regionales y unidades de salud de la familia: Asunción: Ministerio de Salud Pública y Bienestar Social. Dirección General, 2018.
- 14. Shirasu M, Touhara K. The scent of disease: volatile organic compounds of the human body related to disease and disorder. The Journal of Biochemistry 2011;150(3):257–66. pmid:21771869
- 15. Sun X, Shao K, Wang T. Detection of volatile organic compounds (VOCs) from exhaled breath as noninvasive methods for cancer diagnosis. Analytical and bioanalytical chemistry 2016;408(11):2759–80. pmid:26677028
- 16. Blanchet L, Smolinska A, Baranska A, et al. Factors that influence the volatile organic compound content in human breath. Journal of breath research 2017;11(1):016013. pmid:28140379
- 17. Belkina TV, Khojiev DS, Tillyashaykhov MN, et al. Delay in the diagnosis and treatment of pulmonary tuberculosis in Uzbekistan: a cross-sectional study. BMC infectious diseases 2014;14(1):1–8. pmid:25421106
- 18. Rush B, Wormsbecker A, Stenstrom R, et al. Moxifloxacin use and its association on the diagnosis of pulmonary tuberculosis in an inner city emergency department. The Journal of emergency medicine 2016;50(3):371–75. pmid:26416134
- 19. dos Santos VM, Martins RR, Fachinelli LR, et al. Unsuspected tuberculosis in COPD and use of levofloxacin: diagnostic challenges. Infez Med 2014;22(22):309–12. pmid:25551847
- 20. Wang M, Fitzgerald J, Richardson K, et al. Is the delay in diagnosis of pulmonary tuberculosis related to exposure to fluoroquinolones or any antibiotic? The International journal of tuberculosis and lung disease 2011;15(8):1062–68. pmid:21740669
- 21. Tattevin P, Che D, Fraisse P, et al. Factors associated with patient and health care system delay in the diagnosis of tuberculosis in France. The international journal of tuberculosis and lung disease 2012;16(4):510–15. pmid:22325560
- 22. Wang J-Y, Lee C-H, Yu M-C, et al. Fluoroquinolone use delays tuberculosis treatment despite immediate mycobacteriology study. European Respiratory Journal 2015;46(2):567–70. pmid:25900374
- 23. Bikov A, Lázár Z, Horvath I. Established methodological issues in electronic nose research: how far are we from using these instruments in clinical settings of breath analysis? Journal of breath research 2015;9(3):034001. pmid:26056127
- 24. Bikov A, Hernadi M, Korosi BZ, et al. Expiratory flow rate, breath hold and anatomic dead space influence electronic nose ability to detect lung cancer. BMC pulmonary medicine 2014;14(1):1–9. pmid:25510554
- 25. Kunos L, Bikov A, Lazar Z, et al. Evening and morning exhaled volatile compound patterns are different in obstructive sleep apnoea assessed with electronic nose. Sleep and Breathing 2015;19(1):247–53. pmid:24840212
- 26. Fens N, Van der Schee M, Brinkman P, et al. Exhaled breath analysis by electronic nose in airways disease. Established issues and key questions. Clinical & Experimental Allergy 2013;43(7):705–15. pmid:23786277
- 27. Alkhouri N, Eng K, Cikach F, et al. Breathprints of childhood obesity: changes in volatile organic compounds in obese children compared with lean controls. Pediatric obesity 2015;10(1):23–29. pmid:24677760
- 28. Saktiawati AM, Putera DD, Setyawan A, et al. Diagnosis of tuberculosis through breath test: a systematic review. EBioMedicine 2019;46:202–14. pmid:31401197