https://orcid.org/0000-0002-3456-9532
Figures
Abstract
Due to the high prevalence of patients attending with urinary tract infection (UTI) symptoms, the use of flow-cytometry as a rapid screening tool to avoid unnecessary cultures is becoming a widely used system in clinical practice. However, the recommended cut-points applied in flow-cytometry systems differ substantially among authors, making it difficult to obtain reliable conclusions. Here, we present FlowUTI, a shiny web-application created to establish optimal cut-off values in flow-cytometry for different UTI markers, such as bacterial or leukocyte counts, in urine from patients with UTI symptoms. This application provides a user-friendly graphical interface to perform robust statistical analysis without a specific training. Two datasets are analyzed in this manuscript: one composed of 204 urine samples from neonates and infants (≤3 months old) attended in the emergency department with suspected UTI; and the second dataset including 1174 urines samples from an elderly population attended at the primary care level. The source code is available on GitHub (https://github.com/GuillermoMG-HUVR/Microbiology-applications/tree/FlowUTI/FlowUTI). The web application can be executed locally from the R console. Alternatively, it can be freely accessed at https://covidiario.shinyapps.io/flowuti/. FlowUTI provides an easy-to-use environment for evaluating the efficiency of the urinary screening process with flow-cytometry, reducing the computational burden associated with this kind of analysis.
Citation: Martín-Gutiérrez G, Martín-Pérez C, Toledo H, Sánchez-Cantalejo E, Lepe JA (2022) FlowUTI: An interactive web-application for optimizing the use of flow cytometry as a screening tool in urinary tract infections. PLoS ONE 17(11): e0277340. https://doi.org/10.1371/journal.pone.0277340
Editor: Awatif Abid Al-Judaibi, University of Jeddah, SAUDI ARABIA
Received: August 11, 2022; Accepted: October 25, 2022; Published: November 8, 2022
Copyright: © 2022 Martín-Gutiérrez et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All data are fully available without restriction from: https://github.com/GuillermoMG-HUVR/Microbiology-applications/tree/FlowUTI/FlowUTI.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Urinary tract infections (UTIs) are among the most common bacterial infections in humans, representing the second most frequent community-acquired infection in women [1, 2]. That being so, it is no wonder that urine specimens are one of the most received samples in clinical microbiology laboratories [3]. Urine culture is the gold-standard method for UTI to detect the causative agent and initiate an appropriate antibiotic treatment. However, most of the samples have a negative result (over 60%) [4–6], consuming time and resources in their processing, and thus increasing workload and costs.
Several screening tests are used in clinical microbiology laboratories to rule out negative urine samples. In this sense, flow cytometry has become one of the most used screening tools, allowing to inform negative results earlier, reducing time, costs and unnecessary empirical antibiotic treatments [7–9]. Nevertheless, although the usefulness of flow-cytometry as a screening tool has been well established for patients with UTI symptoms, bacteria and leukocyte cut-off values applied vary significantly across studies [10]. This issue can be explained by two main facts: i) the basic measures to validate the diagnostic tests are expected to vary strongly depending on disease prevalence [11], and UTI prevalence varies greatly with age, gender and underlying diseases [2, 12, 13]; ii) Additionally, the interpretation of the quantitative results obtained depends on the threshold applied, which also varies according to the clinical characteristics of the patients [14]. Therefore, due to the heterogeneity of the patients analyzed in the different studies, it is difficult to compare the cut-off values applied or to derive reliable conclusions. Consequently, the best choice for microbiology laboratories is to set their own cut-off values according to patients attended in their health centers, in order to improve the reliability of the screening.
Well-designed interactive tools empower users to integrate information and discover new associations without requiring programming knowledge or data-analytic skills. Considering the powerful functions in R [15], a practical open language for statistical and graphical exploration of data set, and the versatility of Shiny [16], a free web application that does not require knowledge of any programming language, we developed FlowUTI, the first openly available shiny web-application created to determine optimal cut-off values for different UTI markers from flow cytometry systems. In addition, with the aim of evaluating FlowUTI, two datasets obtained with two different flow cytometry systems are analyzed in this manuscript.
Methods
FlowUTI development
The source code of the web application is written in the R language [15], developed by using the function “epi.test” of the R-package epiR [17] and the R-package pROC [18]. We used a specific code to compute the most consistent diagnostic parameters used in clinical studies. Data manipulation and data analysis were executed using R-scripts, which rely on CRAN packages including: ggplot2, dplyr, xtable, shinythemes, knitr and rmarkdown. A shiny web application [16] was created with the aim of providing an accessible tool for non-R users. We used the web application server Shiny to create a robust graphical user interface, which allows an interactive manipulation and interaction with the application.
Workflow
Fig 1 shows the workflow graph. First, urines from patients with a suspected UTI are cultured according to national or international guidelines for the diagnosis of UTIs [19, 20]. Additionally, urines are analyzed by flow-cytometry according to the manufacturer’s instructions. Once urines have been analyzed, the results are transferred to a spreadsheet software. The minimum data set required to run the application comprises the culture result, that must be a dichotomous variable (that hold precisely two distinct values, such as “Positive” and “Negative” results), and the variable of interest obtained by flow cytometry, such us bacteria count. The data set can be supplemented with additional variables (leukocyte, epithelial cells, yeasts, etc.). Finally, each spreadsheet is saved as its own Comma Separated Value (.csv) file. These .csv files are easily obtained from Excel or R software.
Flow cytometry figure was adapted from www.cytognos.com, and csv figure from www.shareicon.net.
Using the application
FlowUTI is able to work with data from any flow-cytometry analyzer, regardless of the brand or model. First, in the Upload data section, the users will upload to FlowUTI the .csv file. Additionally, in the Upload data section, two demo datasets can be uploaded as examples of the data format needed for running the model, where the users can learn and be trained to work with the application before introducing their own data. The information about the patients’ characteristics included in these datasets is described further on. Once the document is uploaded, the user can preview the dataset.
In the section Diagnostic accuracy, the user can explore the diagnostic parameters obtained according to the selected cut-off value (Fig 2). Firstly, the user selects the independent variable of concern (bacteria or leukocyte) and the dependent variable (culture result, the gold standard) (Fig 2A). After that, the cut-off values can be selected by clicking on the slider bar. Immediately, the application will run the computations and will display a table with all diagnostic parameters obtained according to the selected cut-off (Fig 2B). The lower and higher values for bacteria or leukocytes can be chosen to optimize the selection of the cut-off value (Fig 2C). In addition, the application generates a density distribution graph (Fig 2D), showing the positive and negative cultures distribution according to bacteria or leukocyte counts. In this interactive chart the users can zoom in and out, which will help visually to consider the best cut-off value. In this section, the users can also generate a summary document in .pdf format by clicking in the “Report” button (Fig 2E), gathering all the results according to the selected cut-off value, depending on the user’s choice.
Finally, in the section Tables of sensitivity and specificity, the discrimination power of bacteria and leukocyte counts are assessed by Receiver Operating Characteristic (ROC) curve analysis. As was performed in the last section, the user selects the independent and dependent variables of interest, and then the sensitivity, specificity, and the area under the curve (AUC), which is the measure of the ability of a classifier to distinguish between classes, used as a summary of the ROC curve (the higher the AUC, the better the performance of the model at distinguishing between the positive and negative classes), are immediately calculated using bootstrapping (n = 1000). Additionally, placed at the right side, the sensitivity-specificity and ROC curves are plotted.
Web application
We host the web application on the Shiny server at the web address https://covidiario.shinyapps.io/flowuti/. The user’s database is held temporarily in the application and will be discarded when the session is terminated. Each data upload is only viewable by the user during that single browser session. We will ensure that the FlowUTI app is online for five years following the publication date, with a monthly usage limit of 25 hours. The source code of FlowUTI is available on GitHub (https://github.com/GuillermoMG-HUVR/Microbiology-applications/tree/FlowUTI/FlowUTI), which can be downloaded into any computer to be used offline without limitations by using the R software.
Datasets
With the aim of evaluating FlowUTI, two datasets are analyzed in this manuscript. One of them (Neonate .csv) is composed of 204 urine samples from neonates and infants (≤ 3 months old) attended in the emergency department of the University Hospital Virgen del Rocío with suspected UTI during the years 2013–2015. All urine samples were obtained by bladder catheterization and held at 4°C until processed. All samples were processed within 2 hours after collection for culture and flow-cytometry analysis (UF-1000i, Sysmex). Counts above ≥102 colony-forming units (CFU)/ml were considered as positive cultures [19]. Regarding the second dataset (Elderly .csv), 1174 urine samples were included from an elderly population (≥65 years old) with UTI symptoms attended at the Primary Care Units of Virgen del Rocío University Hospital from January to December 2021. All samples were processed within 4–6 hours after collection for culture and flow-cytometry analysis (UF-5000, Sysmex). Counts above ≥105 CFU/ml were considered as positive cultures [21]. Both datasets are available on GitHub (https://github.com/GuillermoMG-HUVR/Microbiology-applications/tree/FlowUTI/FlowUTI/Demos).
Results
Examples of use
Below, we describe the use of FlowUTI for the screening of UTIs at the extreme-age groups: neonates and elderly. It is important to remark that the objective of this manuscript is not to determine a general cut-off value for these two groups of patients, but to highlight the importance of determining the cut-off values according to the patients analyzed. Therefore, the cut-offs evaluated by this study may not be useful in other centers.
Neonate dataset.
Once the Neonate .csv file is uploaded, we can observe a total of 204 records and two independent variables: bacteria and leukocytes. Table 1 summarizes the results obtained from the section Diagnostic accuracy. As can be observed, by selecting a cut-off value of 35 bacteria, a sensitivity of 100% is obtained, without false negative results, and with specificity and false positivity rates of 61% and 39%, respectively. Moreover, with this cut-off value we could avoid the culture of 44.11% of the urine samples. The higher the bacterial count selected as threshold, the higher the false negative rates. In the section Tables of sensitivity and specificity, an AUC of 0.948 was obtained, which means that bacterial count has an excellent discrimination capability for UTIs. On the other hand, leukocyte count appears not be as effective as the bacterial count as a screening of UTIs in this dataset, with an AUC of 0.808.
Elderly dataset.
In total, 1174 urine samples were included in the Elderly dataset (153 contaminated urines were discarded). We can observe that, with a bacteria cut-off value of 200 bacteria, a sensitivity of 95% is obtained, with a 5% of false negative results, a low value for sensitivity (56%) and a proportion of false positive of 44%. With these results, we could avoid the culture of 400 urine samples (34,07%). Again, the higher the bacterial count selected, the higher the false negative rates (Table 2). The leukocyte count presents low specificity values. Finally, the AUCs for bacterial and leukocytes counts are 0.924 and 0.79, respectively.
Discussion
In this manuscript we present FlowUTI, a new interactive shiny web-application addressed to physicians and researchers interested in implementing and optimizing the use of flow-cytometry analyzers as a UTI screening method. FlowUTI is a free tool that provides a user-friendly graphical interface to perform robust statistical analysis without a specific training, maximizing the efficiency of the urinary screening process.
The selection of appropriate cut-off values of a screening method is crucial to avoid erroneous diagnosis in clinical practice. Usually, the choice of cut-off values for bacteria or leukocytes are in accordance with the criterion formerly established by prior studies. However, there is a substantial heterogeneity of cut-off values applied among the studies, making it difficult to select one in particular. This issue was addressed in the meta-analysis performed by Shang and colleagues [22], where a dramatic heterogeneity (I2 > 90%) on sensitivity, specificity, likelihood ratios and diagnostic odds ratios for both bacteria and leukocytes counts was observed. Another problem to be solved is the statistical methodology required to establish a general cut-off value by means of a meta-analysis, which is complex and cumbersome. Only recently a new R-package has been developed for this purpose [23], but an optimal threshold across all urine flow-cytometry studies has not been achieved yet.
The selection of the best cut-off value depends on the prevalence of UTIs in the population, the diagnostic threshold selected, and the type of flow-cytometer used. Furthermore, the clinical relevance of the UTI is also a key factor. In this way, it’s only possible to determine the most appropriate cut-off value carefully considering the clinical cost of false-negative and false-positive results. UTIs in neonates and infants are associated with significant morbidity and long-term medical consequences [24, 25], and because antibiotic treatments have a profound impact on the gut microbiota [26], it’s crucial to promptly and accurately identify those patients who do not have the UTI. For this reason, high values of sensitivity and negative predictive values (NPV) are needed to limit false negative findings, even if the number of false-positive findings is high, resulting in additional cultures. Conversely, the utility of flow-cytometry in the elderly is challenging because of the high prevalence of bacteriuria and pyuria that may not be clinically relevant [27, 28]. For this reason, it’s important to perform a clinical evaluation about what is an acceptable number of false positive and false negative results. Moreover, the significant number of contaminated specimens in these patients [29] may cause erroneous interpretation of urine cultures, masking true infections, or leading to unnecessary treatments until diagnosis [30].
FlowUTI is an open interactive application, which allows users to include different variables related with the UTI screening. For instance, the presence of squamous epithelial cells in urine during the sample collection procedure has been related with contaminated culture results [31]. Hence, users can include the epithelial cell counts and the “contaminated” result in the database, and then study the discrimination power of this variable and its utility to reduce the number of false positive results. Additionally, higher yeast-like cell counts by UF-5000 have been related with the presence of Candida spp. in urine samples [32], so it could be also interesting to evaluate this marker in selected patients [33].
In conclusion, we expect that FlowUTI might be a useful tool for physicians and researchers, allowing them to interactively intervene on the choice of the best cut-off values, making it easier to analyze the data obtained from flow-cytometry and reducing the computational burden associated with this kind of analysis, without losing statistical reliability. We encourage clinical microbiology laboratories to establish their own cut-off values according to the patients attended. By doing so, they could decrease turnaround time of analysis and improving clinical decisions.
Acknowledgments
Guillermo Martín-Gutiérrez holds a Juan Rodés Contract (JR19/00039) from the Carlos III Health Institute.
References
- 1. Klein RD, Hultgren SJ. Urinary tract infections: microbial pathogenesis, host-pathogen interactions and new treatment strategies. Nat Rev Microbiol. 2020; 18:211–26. pmid:32071440
- 2. Foxman B. Epidemiology of urinary tract infections: incidence, morbidity, and economic costs. Am J Med. 2002;113 Suppl 1A:5S–13S.
- 3. Church D, Gregson D. Screening urine samples for significant bacteriuria in the clinical microbiology laboratory, Clin. Microbiol. Newsl. 2004; 23:179–83.
- 4. Schifman RB, Wieden M, Brooker J, Chery M, Delduca M, Norgard K, et al. Bacteriuria screening by direct bioluminescence assay of ATP. J Clin Microbiol. 1984; 20:644–8. pmid:6490851
- 5. Okada H, Sakai Y, Miyazaki S, Arakawa S, Hamaguchi Y, Kamidono S. Detection of significant bacteriuria by automated urinalysis using flow cytometry. J Clin Microbiol. 2000; 38:2870–2. pmid:10921941
- 6. Broeren MA, Bahçeci S, Vader HL, Arents NL. Screening for urinary tract infection with the Sysmex UF-1000i urine flow cytometer. J Clin Microbiol. 2011; 49:1025–9. pmid:21248088
- 7. Alvarez-Barrientos A, Arroyo J, Cantón R, Nombela C, Sánchez-Pérez M. Applications of flow cytometry to clinical microbiology. Clin Microbiol Rev. 2000; 13:167–95. pmid:10755996
- 8. Manoni F, Fornasiero L, Ercolin M, Tinello A, Ferrian M, Hoffer P, et al. Cutoff values for bacteria and leukocytes for urine flow cytometer Sysmex UF-1000i in urinary tract infections. Diagn Microbiol Infect Dis. 2009; 65:103–7. pmid:19748419
- 9. Jolkkonen S, Paattiniemi EL, Kärpänoja P, Sarkkinen H. Screening of urine samples by flow cytometry reduces the need for culture. J Clin Microbiol. 2010; 48:3117–21. pmid:20592157
- 10. Mejuto P, Luengo M, Díaz-Gigante J. Automated Flow Cytometry: An Alternative to Urine Culture in a Routine Clinical Microbiology Laboratory? Int J Microbiol. 2017; 2017:8532736. pmid:29090008
- 11. Brenner H, Gefeller O. Variation of sensitivity, specificity, likelihood ratios and predictive values with disease prevalence. Stat Med. 1997; 16:981–91. pmid:9160493
- 12. Hoberman A, Chao HP, Keller DM, Hickey R, Davis HW, Ellis D. Prevalence of urinary tract infection in febrile infants. J Pediatr. 1993; 123:17–23. pmid:8320616
- 13. Foxman B. Urinary tract infection syndromes: occurrence, recurrence, bacteriology, risk factors, and disease burden. Infect Dis Clin North Am. 2014; 28:1–13. pmid:24484571
- 14.
European association of urology. Guidelines on urological infections. 2021. https://uroweb.org/guidelines/urological-infections
- 15.
R Core Team. Vienna, Austria: R Foundation for statistical computing. 2021.
- 16.
Chang W, Cheng J, Allaire JJ, Sievert C, Schloerke B, Xie Y et al. 2021. shiny: Web Application Framework for R. R package version 1.6.0, URL https://CRAN.R-project.org/package=shiny.
- 17.
Stevenson M, Nunes T, Heuer C, Marshall C, Sanchez J, Thornton R, et al. epiR: Tools for the analysis of epidemiological data. CRAN repository. 2021.
- 18. Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez JC et al. pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics. 2011; 12:77. pmid:21414208
- 19.
European association of urology. 2018. https://uroweb.org/guidelines.
- 20. de Cueto M, Aliaga L, Alós JI, Canut A, Los-Arcos I, Martínez JA, et al. Executive summary of the diagnosis and treatment of urinary tract infection: Guidelines of the Spanish Society of Clinical Microbiology and Infectious Diseases (SEIMC). Enferm Infecc Microbiol Clin. 2017; 35:314–20. pmid:28017477
- 21. Nicolle L. Symptomatic urinary tract infection or asymptomatic bacteriuria? Improving care for the elderly. Clin Microbiol Infect. 2019; 25:779–81. pmid:30922927
- 22. Shang YJ, Wang QQ, Zhang JR, Xu YL, Zhang WW, Chen Y, et al. Systematic review and meta-analysis of flow cytometry in urinary tract infection screening. Clin Chim Acta. 2013; 424:90–5. pmid:23721948
- 23. Steinhauser S, Schumacher M, Rücker G. Modelling multiple thresholds in meta-analysis of diagnostic test accuracy studies. BMC Med Res Methodol. 2016; 16:97. pmid:27520527
- 24. Shortliffe LM, McCue JD. Urinary tract infection at the age extremes: pediatrics and geriatrics. Am J Med. 2002;113 Suppl 1A:55S–66S. pmid:12113872
- 25. Saadeh SA, Mattoo TK. Managing urinary tract infections. Pediatr Nephrol. 2011; 26:1967–76. pmid:21409431
- 26. Fjalstad JW, Esaiassen E, Juvet LK, van den Anker JN, Klingenberg C. Antibiotic therapy in neonates and impact on gut microbiota and antibiotic resistance development: a systematic review. J Antimicrob Chemother. 2018; 73:569–80. pmid:29182785
- 27. Mody L, Juthani-Mehta M. Urinary tract infections in older women: a clinical review. JAMA. 2014; 311:844–54. pmid:24570248
- 28. Schaeffer AJ, Nicolle LE. CLINICAL PRACTICE. Urinary Tract Infections in Older Men. N Engl J Med. 2016; 374:562–71. pmid:26863357
- 29. Hamilton-Miller JM. Issues in urinary tract infections in the elderly. World J Urol. 1999; 17:396–401. pmid:10654370
- 30. Kubik MJ, McCarter YS. Controversies in the diagnosis of urinary tract infections. Clin Microbiol Newsl. 2012; 34:185–91.
- 31. García-Coca M, Gadea I, Esteban J. Relationship between conventional culture and flow cytometry for the diagnosis of urinary tract infection. J Microbiol Methods. 2017; 137:14–8. pmid:28330780
- 32. Kim SY, Park Y, Kim H, Kim J, Koo SH, Kwon GC. Rapid Screening of Urinary Tract Infection and Discrimination of Gram-Positive and Gram-Negative Bacteria by Automated Flow Cytometric Analysis Using Sysmex UF-5000. J Clin Microbiol. 2018;56:e02004–17. pmid:29769277
- 33. Gajdács M, Dóczi I, Ábrók M, Lázár A, Burián K. Epidemiology of candiduria and Candida urinary tract infections in inpatients and outpatients: results from a 10-year retrospective survey. Cent European J Urol. 2019;72:209–14.