The development and validation of different decision-making tools to predict urine culture growth out of urine flow cytometry parameter

Objective Patients presenting with suspected urinary tract infection are common in every day emergency practice. Urine flow cytometry has replaced microscopic urine evaluation in many emergency departments, but interpretation of the results remains challenging. The aim of this study was to develop and validate tools that predict urine culture growth out of urine flow cytometry parameter. Methods This retrospective study included all adult patients that presented in a large emergency department between January and July 2017 with a suspected urinary tract infection and had a urine flow cytometry as well as a urine culture obtained. The objective was to identify urine flow cytometry parameters that reliably predict urine culture growth and mixed flora growth. The data set was split into a training (70%) and a validation set (30%) and different decision-making approaches were developed and validated. Results Relevant urine culture growth (respectively mixed flora growth) was found in 40.2% (7.2% respectively) of the 613 patients included. The number of leukocytes and bacteria in flow cytometry were highly associated with urine culture growth, but mixed flora growth could not be sufficiently predicted from the urine flow cytometry parameters. A decision tree, predictive value figures, a nomogram, and a cut-off table to predict urine culture growth from bacteria and leukocyte count were developed, validated and compared. Conclusions Urine flow cytometry parameters are insufficient to predict mixed flora growth. However, the prediction of urine culture growth based on bacteria and leukocyte count is highly accurate and the developed tools should be used as part of the decision-making process of ordering a urine culture or starting an antibiotic therapy if a urogenital infection is suspected.


Introduction
Urinary tract infections (UTI), ranging from uncomplicated cystitis to urosepsis, are amongst the most prevalent bacterial infections worldwide and are accountable for a large number of emergency consultations and hospitalizations [1,2]. The direct and indirect costs for all urinary tract infections in the Unites States of America in 2010 were estimated to be about 2.3 billion dollars [3]. In Europe, one study estimated the total ambulatory costs of UTI in France to be about 58 Million Euro annually-nearly one Euro per inhabitant [4].
A patient with a suspicion of UTI will be treated with an empirical antibiotic therapy in accordance with international guidelines [5,6]. As a result of the high incidence and this treatment recommendation, about 15% of all community-prescribed antibiotics are used for the treatment of UTI [7]. Considering the rising resistance rates, especially for Escherichia coli [8,9]-by far the most common species found in UTI-a false positive diagnosis and subsequent overtreatment with antimicrobial treatment have to be minimized.
The gold standard for the diagnosis of a UTI is a positive urine culture [10]. In clinical practice this leads to a problem, as a urine culture takes several days to grow, but a decision about antimicrobial treatment often cannot be postponed. In the diagnosis of an uncomplicated UTI, the criteria on which the decision for antimicrobial treatment is based are mainly patient reported symptoms and urine dipsticks [10,11]. Furthermore, microscopic examination of the urine sediment is possible. However, the frequently used urine dipstick suffers from a lack of sensitivity and specificity [12]; microscopic examinations are time consuming, expensive, and dependent on examiners' experience [13]. In patients presenting with non-specific symptoms such as fever, nausea, abdominal tenderness or back pain, screening methods for the prediction of urine culture growth are essential to rule out/in urological infection. Thus, better decision aids are needed to predict probable future urine culture growth.
Automated urine analysis with urine flow cytometry was recently developed as a valid, inexpensive and quick screening prior to microscopic examinations [14][15][16]. Urine flow cytometry is fully automated and can count and classify the different urine particles such as epithelial cells, erythrocytes, cylinders, leukocytes, yeasts and bacteria with high correlation to manual microscopy [17]. The number of bacteria and leukocytes per μL is highly accurate and it has been shown to be predictive of future urine culture growth [18]. However, many different cut-offs exist, leading to confusion. Clinically applicable tools for decision-making have not yet reached their full potential.
Thus, the aim of this study was to develop different aids for decision-making to i) predict negative culture, ii) positive culture, and iii) mixed culture growth. Such instruments might have the potential to avoid antibiotic overtreatment on the one hand, and unnecessary ordering of urine culture on the other. a retrospective single center study to evaluate the use of prediction rules developed out of urine flow cytometry in decision-making for the diagnosis of UTI in the emergency department.

Ethical considerations
The study was approved by the regional ethics committee of the Canton of Bern, Switzerland (KEK: 2016-01298).

Data collection
A comprehensive medical report of every patient who presented at the emergency department is electronically stored. The urine of patients presenting with suspected UTI is routinely analyzed with urine flow cytometry. Furthermore, a urine culture is usually obtained. This procedure might differ if an uncomplicated cystitis is suspected and the diagnosis is based on symptoms or urine flow cytometry only.
Eligible (see below) patients were identified through a key-word search for "urine culture" with different semantic combinations in the health records, stored in the emergency department's database (E-Care, ED 2.1.3.0, Turnhout, Belgium). The search was restricted to the period after the introduction of the urine flow cytometry to the time period starting on January 7 th , 2016 and ending July 31 st , 2016.

Urine flow cytometry
The UX-2000 (Sysmex Corporation, Kobe, Japan) is a fully automated urine analysis that quantifies different urine parameter via fluorescence flow cytometry such as: erythrocytes, leukocytes, epithelial cells, casts, bacteria, mucus, sperms, crystals, round epithelial cells, cylinders, and pathological cylinders.
At least 4 mL of urine is needed for analysis. The analysis takes four minutes and the results are available to the physician within 30 minutes, just after validation by the lab technician. The automated counts of the UX-2000 have shown a good correlation to manual microscopic counts [17]. All flow analyses were performed in an ISO 17025 accredited laboratory (Center of Laboratory Medicine, Inselspital).

Urine culture
Nurses and laboratory staff are regularly trained to ensure high quality standards to obtain 5mL of clean midstream/catheter urine in a vacutainer urine collection tube with boric acid (urine culture kit) and to send it to the laboratory within two hours.
In daily practice the urine culture is prepared directly until 4 pm with 5μL for CHROMagar and CNA-agar (colistin and nalidixic acid-agar) and incubated at 35˚C without CO 2. Antimicrobial bacterial activity is proven by Bacillus subtilis. Identification of the microorganism is realized with MALDI-TOF, resistance examination with the Kirby Bauer method. After 24 hours and also after 48 hours the results are taken and read off.

Eligibility criteria
All adult patients found through the key-word search were included when they had a urine culture and a urine flow cytometry obtained in the first 24h of their visit to the emergency department. Patients younger than 16 years old and those without a urine flow cytometry and/ or without a urine culture were excluded.

Study outcomes
The study outcome was urine culture growth. According to the European association of Urology, there is a large range of a colony forming unit (cfu) cut-offs defining a positive urine culture ranging from 10 2 cfu/mL in catheter urine samples of symptomatic patients to 10 5 cfu/mL in a spontaneously voided urine sample in asymptomatic patients [6].
Significant urine culture growth is defined here as at least 10 4 cfu/mL, because this limit i) represents the cut-off for significant bacterial growth in all complicated UTI, even in straight urine catheter samples [6], ii) is used in most of the urine flow cytometry studies [18], and iii) is often the minimum bacterial growth that is generally reported by clinical microbiology laboratories. A mixed culture was defined as a significant bacterial growth (!10 4 cfu/mL) with a mixed growth pattern.
Three outcome variables were defined. A categorical variable with the levels "no significant culture growth", "significant mixed flora growth" and "significant culture growth" was defined. Furthermore, two binary variables were created that classify the urine sample into i) "positive culture growth" (independently of mixed flora growth) vs. "no growth"and ii) "mixed flora growth" vs. "no mixed flora growth".

Data extraction
The following data for eligible patients were anonymized and extracted from the medical record of the emergency department into Microsoft Excel for Mac 2011 (Microsoft Corporation, USA): patient demographics such as age and sex, patient-reported data such as the presence of dysuria and urinary frequency, clinical findings such as suprapubic/flank/abdominal pain and fever, patient comorbidities, the discharge diagnosis group [19] as well as the urogenital diagnosis, if any, at discharge.
Urine flow cytometry results were automatically extracted and the number and species of an obtained urine culture were manually extracted from the laboratory database (Xserv.4, ixmid Software Technologie GmbH, Germany).

Statistical analysis
Statistical analysis was mainly performed using Stata 13.1 (StataCorp, College Station, Texas, USA). The whole sample was randomly divided into two group-sets: a training set and a validation set with a ratio of 70:30. Continuous variables (e.g. age) were presented with mean and standard deviation (SD) while categorical data were described as the absolute number and percent.
The association of mixed culture growth as well as positive culture growth with urine flow cytometry parameters as predictors were tested using logistic regression.
Different statistical approaches to predict a positive urine culture from the urine flow cytometry parameters bacteria and leukocytes were developed using the training set and validated with the validation set: • A colored scatter plot was generated out of the urine flow cytometry parameter to predict a positive urine culture (bacteria and leukocytes).
• A decision tree was development and its validation presented using SPSS (IBM Corp. Released 2016. IBM SPSS Statistics for Windows, Version 24.0. Armonk, NY: IBM Corp) with a Chi-square Automatic Interaction Detectors (CHAID) algorithm.
• A nomogram was created (training set) from a bootstrapped logistic regression and its predictive values for different predicted probabilities are presented (validation set).
• Predictive values of the validation set for different bacteria and leukocytes cut-offs for a positive urine culture test found by analysis of the training set (or published previously by other studies) are summarized.
Predictive values were presented with the associated 95% confidence interval (CI). A Pvalue of less than 0.05 was defined as statistically significant and P < 0.001 as highly significant.
The initial idea of predicting the categorical culture growth as 1) "no significant culture growth", 2) "significant, mixed-culture growth", 3) "significant, non-mixed culture growth" out of the urine flow cytometry parameter was discarded because the outcome mixed culture growth could not be adequately predicted (see below).

Patient characteristics
Six hundred and thirteen (n = 613) patients fulfilled the eligibility criteria and were included in the analysis. The sample was randomly divided into a ratio of 70:30 into a training set (n = 429) and a validation set (n = 184). The flowchart of the selection process is shown in Fig 1. The mean age was 59.5 (SD 19.6) years and 48.5% of the patients were female. Clinical and patient-reported data that were often found were fever (28.3%), abdominal or flank pain Tools to predict urine culture growth out of urine flow cytometry parameter (cumulative 33.2%), and dysuria (18.1%). A urogenital diagnosis at discharge was identified in 70.2% of the cases. Possible urogenital infection/urosepsis (29.5%) was the most frequently documented urogenital diagnosis. A detailed summary of the patients' characteristics is shown in Table 1.
Two hundred and forty-seven (40.6%) urine cultures met the criteria for a positive culture with at least 10 4 cfu/mL. Escherichia coli was found in 48.6% of the positive cultures, followed by Klebsiella pneumoniae (5.7%), and Staphylococcus aureus (4.5%). A mixed culture was found in 17.8% (see Table 2).
A majority of the patients (73.6%) were hospitalized.

Urine culture growth
The number of leukocytes and bacteria in urine flow cytometry showed a highly significant association (p<0.0001) in logistic regression with a positive urine culture (independently of the species or mixed flora) and all urine flow cytometry parameters as independent parameters. Cylinders (p = 0.016), yeasts (p = 0.027), and pathological cylinders (p = 0.006) in the urine flow cytometry presented significant associations with positive urine culture, but the association did not remain significant when restricting the analysis to positive urine culture without mixed flora and adding an interaction term between bacteria and leukocytes; hence they were not included further in the decision-making aids. The area under the receiver-operating curve (ROC) in the whole set to predict culture growth out of bacteria and leukocyte count was 0.93 (95% CI: 0.90, 0.94).  Table 3). The relationship between predictive values and different cut-offs of bacteria and leukocyte counts for a positive decision test is shown in Fig 3.

Mixed flora
Epithelial cells (p = 0.012), round epithelial cells (p = 0.012), and cylinder (p = 0.006) were associated with mixed flora growth. The area under the receiver operating characteristic curve (AUC) in the whole set to predict mixed flora growth out of the identified predictors was 0.66 (95% 0.61, 0.70).
The first attempt was to model the categorical outcome levels i) no growth, ii) mixed flora growth, and iii) positive culture growth out of the identified five urine flow cytometry parameters. The results of these models did not usefully predict mixed flora growth. Fig 2B illustrates the missing predictive value of epithelial cells to predict mixed culture growth. The illustration is similar in a three-dimensional plot additionally incorporating cylinders (see https://figshare. com/articles/Figure_pdf/5873799).
The decision-making tools presented below were therefore restricted to predicting positive urine culture growth (independently of mixed flora growth) vs. no growth out of the bacteria and leukocyte counts.

Decision tree
A decision tree using the Chi-square automatic interaction detection (CHAID) was build using the training set. Culture growth vs. no. growth (binary coded) was the dependent variable and Tools to predict urine culture growth out of urine flow cytometry parameter the counts of leukocytes and bacteria were the independent variables. The validation of the tree in the validation set is shown in Fig 4. The percent of the cultures that are correctly classified if the culture has shown bacterial growth was 81.2% (node 4 and 10) while no growth culture were classified correctly over all other nodes in 87.8% of the observations leading to an overall correct prediction of 85.3%.

Nomogram
The training set was used to create a nomogram from a bootstrapped logistic regression predicting positive urine culture from bacteria and leukocyte count. The area under the receiveroperating characteristic curve for these predictions was 0.92 (95% CI: 0.89, 0.95) in the training set and 0.93 (95% CI: 0.89, 0.96) in the validation set.    Fig 5 shows the obtained nomogram and predictive values for different probability cut-offs (validation set). For both, a given number of bacteria and leukocytes, a related score is assigned. Out of the sum of the scores, the total score is obtained. From the probability axis the probability for culture growth for the obtained total score can be read off. Each given probability of urine culture growth leads to different predictive values. For example, a test that is defined as positive if the predicted probability of urine culture growth is higher than 10% has a sensitivity of 98.6% (95% CI: 92.2%-100%) and a specificity of 57.4% (95%: 47.8%, 66.6%). For a sample calculation example see  Tools to predict urine culture growth out of urine flow cytometry parameter

Cut-off table
As illustrated in Fig 3, many different combinations of leukocytes and bacteria count cut-offs often have similar predictive values. Table 3 shows different cut-off values developed out of the training set or suggested in the literature and the corresponding predictive values validated with the validation set, or the whole sample in case of external suggested parameters.

Statement of principal findings
A retrospective analysis of patients presenting to an emergency department was performed to predict urine culture growth from urine flow cytometry parameters and different decisionmaking tools were developed and validated. While the number of leukocytes and bacteria were strongly associated with positive culture growth, mixed flora growth could not be sufficiently predicted from the urine flow cytometry parameters. To our knowledge, this is the first study that developed and validated different decision-making tools i.e. a decision tree, predictive value figures, a nomogram, and a cut-off table to predict urine culture growth out of bacteria and leukocyte count of urine flow cytometry.

Results in context
Polymicrobial bacteriuria or mixed flora is usually considered as contamination even though in special situations such as long-term catheterization it can be of significance [20]. In this trial, we tried to predict mixed flora growth out of epithelial and round epithelial cells. One Tools to predict urine culture growth out of urine flow cytometry parameter reason for the failure to predict mixed flora might be the fact that squamous epithelial cells, which were traditionally thought to have a higher predictive value than epithelial cells for instance [21], cannot be determined by UX-2000. The predictive performance of squamous cells with future generations of urine flow cytometer such as UF-4000 (Sysmex, Kobe, Japan), which allows the quantification of squamous cells, needs to be further studied. However, even squamous cells identified through microscopy yield a poor performance in predicting mixed flora [22]. Thus, a different approach might be required. Two trials used different patterns of bacteria fluorescent light to predict bacterial morphologies and mixed flora correctly using UF1000i (Sysmex, Kobe, Japan) [23,24]. While Yang, Yang [24] concluded that results of laser Tools to predict urine culture growth out of urine flow cytometry parameter flow cytometry predict growth of mixed flora, the results of Geerts, Jansz [23] showed underperformance in the category mixed flora. Further studies are needed to clarify the results of this promising approach and give recommendations for the use in clinical practice which is of special use in the emergency setting where the contamination rate is often found to be high [25]. Several studies focused on finding the "optimal" cut-off for different parameters that can be found by flow cytometry to predict urine culture growth [18]. Most of them include count of bacteria, leukocytes or the combination of both to predict urine culture growth; especially high sensitivity cut-off values to rule out future urine culture growth were presented using different cytometers such as UX-2000, UF-1000i, UF-100, Accuri C6 and others [15,[26][27][28][29][30]. With a cut-off value of 170 bacteria/μL and 150 leukocytes/μL, a sensitivity of 98.8%, a specificity of 76.5%, a negative predictive value of of 99.5% and four false negative results could be obtained (1.2%), avoiding the culture in 57.1% of samples [15]. The comparison with high-sensitivity cut-offs found in other studies is tricky as they often used different cut-off criteria for bacterial growth (e.g. 10 5 cfu/mL [16] or more complex criteria [31]), other study populations (e.g. including outpatient and general practitioner patients [30]), and other types of urine flow cytometer [15]. However, high sensitivity could be shown with our parameter too, even with a cut-off of 10 4 cfu/mL [15,16,31]. Recently, a Swedish study presented a linear discriminant analysis using bacteria and leukocytes on a log scale [29] similar to Fig 2A. The parameters were slightly different from the parameters presented in this article, which might be due to another cut-off of urine culture bacterial growth (!10 3 cfu/mL) and the use of Sysmex UF1000i. While such an approach is powerful by covering many different bacteria/leukocytecombinations, the equation might not be useful in clinical practice due to its complexity. Shang, Wang [18] concluded, in their systematic review on cut-off values for bacteria and leukocytes to predict urine culture growth focusing on UF-100 and UF-1000i, that the study populations were often not representative of UTI patients. This is a major limitation of their review as the disease prevalence and the characteristics of the population have to be taken into account, when interpreting the results [16]. In our study, the population consisted of patients presenting at the emergency department of a university hospital with a suspected UTI-a population that is heterogeneous, and also includes polymorbid, transplanted as well as immunosuppressed patients. One trial studied febrile patients in an emergency department. The authors presented a larger high-sensitivity bacteria cut-off compared to other trials to rule out UTI in febrile patients [32]. Further research on special subgroups of patients is required to improve the decision-making in specific scenarios.
Different tools were created and validated including a comprehensive nomogram that is detached from the "optimal" cut-off illusion and may be used for the interpretation of the results of the UX-2000 to evaluate a patient at the emergency department with a suspected urogenital infection. These tools are an aid for decision-making, when flow cytometry is used as one piece of the puzzle to lead to a diagnosis, treatment, or to decide if further diagnostic investigation is necessary. The decision about which tool to use is of individual preference.

Strengths and weaknesses of the study
This study is a retrospective study of laboratory data and health records. Information bias of the independent variable (urine flow cytometry parameter) and outcome variables (urine culture growth) is unlikely due to the use of laboratory tests that are regularly validated. Thus, high data quality in these variables can be assumed. However, clinical data that are used to describe the study sample are based on health records and completeness cannot be assured. Furthermore, selection bias might be a limitation of this trial, especially because more than 75% of the patients included in this study were subsequently hospitalized. Whether and how our results generalize to a healthier population remains to be investigated. The decision to order a urine culture was in the responsibility of the physician in charge. Thus, inter-individual variations might have led to selection bias.
An interesting question concerns the predictive value of urine flow bacteria and leucocytes for urine culture growth in specific subgroups of patients e.g. in which the clinical suspicion of a urosepsis was high. However, we are not able to analyze our data in that regard because the discharge diagnosis were made by the physician, thus the urine flow cytometry was taken into account in that diagnosis and all included patients were initially under the suspicion of having a UTI.
A broad search algorithm was used to identify all patients with an obtained urine culture to ensure a small number of missing eligible patients.
Despite frequent training of the nurses to educate a patient in the procedure of giving a clean urine sample, the quality of urine culture reflects the quality of taking urine cultures in an emergency department with an increased rate of mixed flora culture. External validity can only be assured with respect to a definition of a positive urine culture of at least 10 4 cfu/mL, the urine flow cytometer UX-2000 and to patient populations with a high number of complicated UTI and hospitalization rate.

Implications for clinicians
Medical decision-making aids such as scores, flow-charts, and algorithms are nowadays an essential element in daily routine and are thought to increase the quality of care and support evidence-based treatment [33]. This article provides the physician with different designed tools in tabularized form, in the form of a decision tree, as well as a graphical calculating device (nomogram) for use in clinical practice. Cut-off values with high sensitivity and negative predictive values were presented. Thus, the tools have the potential to reduce unnecessary prescription of antibiotics and to avoid initiating unnecessary urine cultures.

Unanswered questions and future research
Although studies have shown an economic benefit of the use of urine flow cytometry before urine culture [34], the impact on the prescription of antibiotics remains unknown.
Furthermore, there is a lack of studies that focus on urine flow cytometry cut-offs in specific clinical subgroups e.g. febrile [32] and especially immunosuppressed patients. In the setting of immunosuppressed patients predicting mixed flora growth is particular important. Thus, future research is needed to evaluate the predictive performance of new generation cytometer especially of squamous cells, which are quantified e.g. in UF-4000, or use other approaches to predict mixed flora culture.

Conclusions
Urine flow cytometry parameters fail to predict mixed flora growth. However, the prediction of urine culture growth from bacteria and leukocytes is highly accurate and several tools were presented that can be used in the decision process of initiating an urine culture or starting an antibiotic therapy for suspected urogenital infection.
Supporting information S1 File. Dataset of the study. (XLS)