Performance of Thirteen Clinical Rules to Distinguish Bacterial and Presumed Viral Meningitis in Vietnamese Children

Background and Purpose Successful outcomes from bacterial meningitis require rapid antibiotic treatment; however, unnecessary treatment of viral meningitis may lead to increased toxicities and expense. Thus, improved diagnostics are required to maximize treatment and minimize side effects and cost. Thirteen clinical decision rules have been reported to identify bacterial from viral meningitis. However, few rules have been tested and compared in a single study, while several rules are yet to be tested by independent researchers or in pediatric populations. Thus, simultaneous test and comparison of these rules are required to enable clinicians to select an optimal diagnostic rule for bacterial meningitis in settings and populations similar to ours. Methods A retrospective cross-sectional study was conducted at the Infectious Department of Pediatric Hospital Number 1, Ho Chi Minh City, Vietnam. The performance of the clinical rules was evaluated by area under a receiver operating characteristic curve (ROC-AUC) using the method of DeLong and McNemar test for specificity comparison. Results Our study included 129 patients, of whom 80 had bacterial meningitis and 49 had presumed viral meningitis. Spanos's rule had the highest AUC at 0.938 but was not significantly greater than other rules. No rule provided 100% sensitivity with a specificity higher than 50%. Based on our calculation of theoretical sensitivity and specificity, we suggest that a perfect rule requires at least four independent variables that posses both sensitivity and specificity higher than 85–90%. Conclusions No clinical decision rules provided an acceptable specificity (>50%) with 100% sensitivity when applying our data set in children. More studies in Vietnam and developing countries are required to develop and/or validate clinical rules and more very good biomarkers are required to develop such a perfect rule.


Introduction
Accurate and rapid diagnosis of acute bacterial meningitis (ABM) is essential as successful disease outcome is dependent on immediate initiation of appropriate antibiotic therapy [1,2]. Differentiating ABM from presumed acute viral meningitis (pAVM) often proves challenging for clinicians as their symptoms and laboratory tests are often similar and overlapping. Classical clinical manifestations of ABM in infants and children are usually difficult to recognize given the absence of meningeal irritation signs and delayed elevation of intracranial pressure. In addition, the various parameters examined in the cerebral spinal fluid (CSF) are less discriminative in children than in adults, especially in enterovirus meningitis where the CSF parameters may be similar to bacterial meningitis values. The vast majority of patients with acute meningitis are administered broad-spectral antibiotics targeting ABM while awaiting results of definitive CSF bacterial cultures. In the absence of ABM, this practice may enhance the local frequency of antibiotic resistance [3], cause adverse antibiotic effects [4], and high medical costs [5]. Thus, it is not only important to recognize ABM patients who promptly require antimicrobial therapy, but also pAVM patients who do not need antibiotics or hospital admission at all. An ideal diagnostic rule should demonstrate 100% sensitivity in detecting bacterial meningitis [6], while retaining a high specificity.

Identification of clinical rules
Two electronic databases including PubMed and Scopus were searched for suitable clinical rules. The search terms used were as follows: ''dengue AND (rule OR score)''. We supplemented these searches with a manual search of articles that developed and/or compared clinical rules. Since we aimed to find the clinical rule that could be applied in our hospital and test the generalizability of clinical rules [32], no restrictions were applied with respect to country, year, and language of studies that developed clinical rules. A total of 15 clinical rules were identified. Among them the Bonsu 2008 [22] and Dubos rules [21] were not tested as band leukocytes and procalcitonin were not available in our hospital.

Study design
The current study was performed at the Infectious Department of Pediatric Hospital Number 1, Ho Chi Minh City, Vietnam. The hospital is a tertiary pediatric hospital in southern Vietnam with 1200 beds. It was a retrospective cross-sectional analysis of the clinical signs and laboratory tests obtained from previously healthy children (#15 years) that were diagnosed with acute meningitis. Discharge diagnosis was reviewed to identify meningitis patients based on the International Classification of Diseases, 10 th Revision (ICD-10) with the following codes: G00, G00.x, G01*, G02.0*, G03, and G03.x. The study was approved in advance by the Ethical Review Committee of the Pediatric Hospital Number 1, Ho Chi Minh City, Vietnam. Written informed consent from the patients or their parents was waived by the Committee, because all data were retrospectively collected after the discharge of patients and numerically coded to ensure patient anonymity.
The entry criteria were as follows: children with proven acute bacterial meningitis (ABM) or presumed acute viral meningitis (PAVM), who had received a lumbar puncture between December 2003 and December 2008. Patients exhibiting blood-contaminated CSF (CSF erythrocyte count .10,000 cells/mL) [33], tuberculous meningitis, HIV infection, immune depression, and those found to have histories of pulmonary tuberculosis, liver diseases such as autoimmune disease, alcoholic liver disease and metabolic disease, kidney disease, neurosurgical disease or had undergone recent neurosurgery were excluded from the study. Neonates (less than 28 days old) and patients with missing laboratory variables listed in Table 1 were also excluded.
Proven ABM was diagnosed if the patient demonstrated CSF pleocytosis (CSF leukocyte count .7 cells/mL) [34,35] in addition to one of the following test results: (1) positive CSF culture for bacterial pathogens, (2) positive CSF latex agglutination test, or (3) positive blood culture. PAVM was defined as patients with a pleocytosis in the CSF (CSF leukocyte count .7 cells/mL) in addition to positive culture for viral pathogens or rapid remission without extensive antibiotic therapy combined with an absence of any four criteria of proven ABM [10,14,20,26,36,37].
Blood cultures were performed using 5% sheep blood agar before 2005 and a BACTEC 9240 system instrument (BD Biosciences, China) from 2005. CSF culture was done on 7% horse blood agar and 5% chocolate blood agar plates and incubated at 36uC for 24 h. Observed colonies were further identified by standard microbiological methods. Viral culture was not routinely performed, only five CSF samples were sent to Pasteur Institute (Ho Chi Minh City, Vietnam) for virus isolation.
At the time of admission, the relevant patient history regarding clinical symptoms and signs, and laboratory parameters listed in the Table 1 was collected. Clinical signs and symptoms that were not noted in the patient medical record were coded as normal.

Data analysis
All information was entered into a Microsoft Office Excel 2007 computerized database. Missing clinical signs and symptoms were not included and the number of patients per group was also adjusted before analysis. Our analysis showed that there were no significant differences in selected variables between patients with and without missing data.'' into the data analysis (page 6).
A score, judge, or probability of ABM (pABM) was calculated from each patient for each of the clinical decision rules according to the authors of the rules (Method S1). The overall accuracy of these rules represented by area under a receiver operating characteristic curve (ROC-AUC) was compared by the method of DeLong [38] using MedCalc statistical software (11.0, MedCalc Software bvba, Belgium). AUC values $0.5, 0.75, 0.93, or 0.97 were considered as fair, good, very good, or excellent accuracy [39]. The sensitivity and specificity of each rule was then calculated using our patient data set. To do so, we applied the thresholds indicated by the authors of the rules and by our own ROC analyses. The rules demonstrating 100% sensitivity were further analyzed to compare their specificity using the McNemar test [8].
The minimal required sample size and power of comparison were calculated using the MedCalc statistical software based on 5% type I error rate and 20% type II error rate. Assuming that ROC-AUCs of all clinical rules are at least 90% compared to the null hypothesis value 70% [22] , the required sample size was 48 subjects per group in this case.
In order to explain the limitation of Nigrovic's rule, we calculated the theoretical sensitivity and specificity of simple list of items rule with cut-off value at one item. Since selected variable demonstrated an independent predictor of ABM [14], the theoretical sensitivities and specificities of the simple list of items rule with cut-off value $1 can be derived from individual sensitivity and specificity of each variable as presented by equation 1 and 2, respectively ( Figure 1). The individual sensitivity and specificity of each variable were derived from the current study

Comparison of clinical rules
The overall accuracy of the rules was explored by calculation of the ROC-AUCs. All 13 clinical rules possessed AUC values between 0.75 and 0.94, indicating good accuracy (Table 3 and Figure S1) [39]. The Spanos rule had the highest AUC at 0.938. However, when comparing with the other four best rules (De Cauwer, Freedman, Nigrovic, and Thome), the Spanos rule was not significantly better by Delong method [38] (P.0.05, Figure 3).
When applying the thresholds indicated by the authors of the rules, no rule demonstrated 100% sensitivity, as prediction rules failed to identify six ABM patients by Thome    Our calculation showed that the theoretical sensitivity of Nigrovic's rule was 96.6% when computing the variables' sensitivity values observed in our study. The strength of the theoretical sensitivities was in the following order: Freedman = De Cauwer . Nigrovic . Schmidt . Brivet . Chavanet, which was almost identical to the order of real sensitivities performed in our data set (Table S1). The theoretical sensitivity of Nigrovic's rule was just slightly increased (98.1%) upon computing the variables' sensitivity values observed in Nigrovic's studies [14], further supporting that the rule is not perfect. Similarly, the strength of theoretical specificities was in the following order: Chavanet . Schmidt . Brivet . Nigrovic . De Cauwer . Freedman. These findings were similar to the order of real specificities in the data set. Furthermore, the correlation between the theoretical and real accuracy was analyzed by a Spearman rank test. Our results demonstrated that the theoretical sensitivity and specificity were highly correlated with real sensitivity and specificity, respectively ( Figure S2). Overall, there was no statistical difference between theoretical calculations and real values in data sets in regards to sensitivity and specificity, suggesting that our calculation was correct.

Discussion
To our knowledge, this is the first study that simultaneously tested more than ten prediction rules for clinical practice in meningitis. No clinical rule had superior overall accuracy compared to other rules. In addition, no rule provided 100% sensitivity with acceptable specificity (.50%). The overall accuracy of the two earliest rules (Spanos and Thomas rules) was not outperformed by recent developed rules, probably due to the similar epidemiology to the pre-vaccination era [10]. The high frequency of H. influenzae in our study could be explained by the lack of conjugate Hib vaccine in the Vietnamese national vaccination policy, and only a small number of children (0.5%) reportedly received conjugate Hib vaccine [40].
Among reported clinical decision models, the Nigrovic's rule [14] is the only rule that has been tested by more than three independent groups, and performed perfectly in several studies [8,26,27,28,29,30]. However, it only provided 96.3% sensitivity in our study, which is also in the range of other independent data sets [7,19,20,31] and well agreed with the theoretical sensitivity (96.64%) and specificity values (53.35%), explaining that the Nigrovic's rule could not identify all ABM patients in several data sets.
Based on these evidences and our equations, an ideal simple list of items clinical rule with theoretical sensitivity .99.99% and theoretical specificity .50% should include at least four independent variables that posses both sensitivity and specificity .85-90%. In addition, to improve the rule sensitivity without significantly reducing its specificity, we recommend adding additional variables with extremely high specificity (approximately 100%). We are not aware of more than three such conventional parameters to derive such an ideal rule. However, recent studies have proposed that blood procalcitonin [21,41], CSF lactate [42,43,44], and blood C-reactive protein (CRP) [45] are very good biomarkers for bacterial meningitis. Upon addition of procalcitonin test (99% sensitivity and 83% specificity [37]), the theoretical sensitivity of Nigrovic's rule would be significantly increased from 96.64% to 99.77% (Calculation: 12(120.9664)6(120.99) = 0.9997), while the theoretical specificity value would be dropped  from 53.35% to 44.28% (Calculation: 0.533560.83 = 0.4428). However, these three parameters have rarely been measured in the same study and their usefulness and independent contribution in the differential diagnosis of ABM from pAVM are rarely evaluated [46,47]. Thus, further studies are required to evaluate the contribution of these variables in the performance of clinical rules.
There were several limitations in our study. The first limitation was that the design was retrospective. Secondly, we only analyzed data from only one hospital. Therefore our results would be different from other hospitals, particularly in high-resources countries, where the epidemiology, clinical characteristics and outcome are different. Thirdly, our study focused on hospitalized patients in a big city. Therefore, further studies recruiting patients in clinics or local hospitals are required to further test these clinical rules. Fourthly, we could not confirm all pAVM patients as aseptic meningitis due to limited diagnosis in our hospital, which may affect the result. Another limitation is that the number of pAVM patients was much smaller than that of ABM, because several patients with extensive antibiotic therapy were excluded from criteria of pAVM. Finally, we were unable to include band leukocytes and blood procalcitonin, thus we could not test two promising Bonsu 2008 [22] and Dubos's [21] rules in the current study.
In conclusion, accurate bacterial meningitis is serious and the outcome is dependent on immediate initiation of appropriate antibiotic therapy. The best method for differentiating accurate bacterial meningitis from viral meningitis remains unclear. Several clinical decision rules have been derived to assist clinicians to distinguish between bacterial meningitis and viral meningitis, but barely tested and compared by independent studies. When applying our data set, no clinical rule provided an acceptable specificity (.50%) with 100% sensitivity. More studies in developing countries are required to confirm due to several limitations related to population and more accurate biomarkers are required to develop such a perfect rule. Figure S1 ROC curves of 13 clinical rules for differential diagnosis of ABM from PAVM when applying our data set. The AUCs of ROC curves were 0.812 for Bonsu 2004, 0