Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Automated Detection of External Ventricular and Lumbar Drain-Related Meningitis Using Laboratory and Microbiology Results and Medication Data

Automated Detection of External Ventricular and Lumbar Drain-Related Meningitis Using Laboratory and Microbiology Results and Medication Data

  • Maaike S. M. van Mourik, 
  • Rolf H. H. Groenwold, 
  • Jan Willem Berkelbach van der Sprenkel, 
  • Wouter W. van Solinge, 
  • Annet Troelstra, 
  • Marc J. M. Bonten
PLOS
x

Abstract

Objective

Monitoring of healthcare-associated infection rates is important for infection control and hospital benchmarking. However, manual surveillance is time-consuming and susceptible to error. The aim was, therefore, to develop a prediction model to retrospectively detect drain-related meningitis (DRM), a frequently occurring nosocomial infection, using routinely collected data from a clinical data warehouse.

Methods

As part of the hospital infection control program, all patients receiving an external ventricular (EVD) or lumbar drain (ELD) (2004 to 2009; n = 742) had been evaluated for the development of DRM through chart review and standardized diagnostic criteria by infection control staff; this was the reference standard. Children, patients dying <24 hours after drain insertion or with <1 day follow-up and patients with infection at the time of insertion or multiple simultaneous drains were excluded. Logistic regression was used to develop a model predicting the occurrence of DRM. Missing data were imputed using multiple imputation. Bootstrapping was applied to increase generalizability.

Results

537 patients remained after application of exclusion criteria, of which 82 developed DRM (13.5/1000 days at risk). The automated model to detect DRM included the number of drains placed, drain type, blood leukocyte count, C-reactive protein, cerebrospinal fluid leukocyte count and culture result, number of antibiotics started during admission, and empiric antibiotic therapy. Discriminatory power of this model was excellent (area under the ROC curve 0.97). The model achieved 98.8% sensitivity (95% CI 88.0% to 99.9%) and specificity of 87.9% (84.6% to 90.8%). Positive and negative predictive values were 56.9% (50.8% to 67.9%) and 99.9% (98.6% to 99.9%), respectively. Predicted yearly infection rates concurred with observed infection rates.

Conclusion

A prediction model based on multi-source data stored in a clinical data warehouse could accurately quantify rates of DRM. Automated detection using this statistical approach is feasible and could be applied to other nosocomial infections.

Introduction

Healthcare-associated infections (HAI) pose a great burden on current medical care and are increasingly viewed as preventable complications. HAI refers to the entire scope of infections associated with medical care and also includes nosocomial infections. The European burden of HAI has been estimated at 4.5 million infections contributing to 148,000 deaths [1]. Hospitals are encouraged to report HAI rates through surveillance organizations such as the National Healthcare Safety Network (NHSN) from the Centers for Disease Control and Prevention (CDC) in the United States [2] and the PREZIES network in the Netherlands [1], [3]. Efficient registration and feedback of infection rates to healthcare workers are considered essential elements to reduce infection rates and surveillance of infection rates is increasingly demanded by policy makers and the public [4]. However, manual registration of HAI rates is time-consuming and susceptible to error due to subjective interpretation of definitions and manual data handling [5], [6].

Therefore, there is an urgent need for more efficient and reliable surveillance methods. Automated classification algorithms using data stored in electronic medical records (i.e. clinical data warehouses [7]) have been developed both at the hospital-wide and procedure-specific level with varying success, in particular for surgical site infections and (catheter-related) bloodstream infections [8][18]. Most models use a classification approach based on the presence of (one or more) indicators of infection, such as positive microbiology results, antibiotic use, and discharge coding. Another, less used, method is development of a multivariable model with associated cut-off values to classify patients [11], [13]. Such (automated) models are of relatively low cost, time-saving, and facilitate standardized interpretation of infection criteria [10]. However, when case-finding is based on microbiological cultures, such classification algorithms have low sensitivity for culture-negative infections and specificity of these algorithms decreases when extending case-finding criteria. Furthermore, suboptimal positive predictive values still make manual confirmation of infection necessary. Given these potential drawbacks, most healthcare centers still use manual chart review as their primary method of surveillance.

The aim of this study was to develop a prediction model for HAI using data routinely stored in a clinical data warehouse (i.e. the Utrecht patient-oriented database, UPOD [19]) to retrospectively identify the presence of infection. In order to increase sensitivity for culture-negative infections while maintaining specificity and enable surveillance without the need for manual confirmation, an alternative approach to the classification algorithm was sought and data sources were extended to include not only microbiology results and antibiotic use but also results of clinical chemistry analysis. In clinical practice, such a model could eliminate or significantly reduce the workload of manual chart review and increase resources available for development and implementation of infection control measures. Drain-related meningitis (DRM) was selected as an example to investigate this general approach to automated evaluation of infection rates. This nosocomial infection, related to external cerebrospinal fluid (CSF) drainage through external ventricular (EVD) or lumbar drains (ELD), is sometimes also termed ventriculitis or meningoventriculitis and is one of the procedure-specific infections that has since 2004 been monitored by the department of hospital hygiene and infection control through labor-intensive manual chart review. The developed model achieved good discriminatory power at the level of the individual patient and group-level estimates of infection rates could be generated without any manual confirmation.

Methods

Ethics statement

The use of anonymous data through the UPOD has been exempted from review by the Institutional Review Board of the University Medical Center Utrecht as described previously [19].

Study design and outcome measure

Data collected as part of the hospital hygiene surveillance program were used to develop the prediction model for DRM. Results of routinely performed incidence surveys were considered as reference standard. Two infection control professionals assessed each patient for the development of DRM by chart review using modified NHSN/CDC criteria for healthcare-associated meningitis (figure 1) [20], [21]; in case of disagreement adjudication was performed through review. A surveillance episode was defined to start the day of drain placement up to seven days after drain removal of the last drain or up to discharge, whichever occurred first.

thumbnail
Figure 1. NHSN/CDC definition of healthcare-associated meningitis for patients >1 year of age [20], [21].

https://doi.org/10.1371/journal.pone.0022846.g001

Study population

All patients registered by the surveillance program to have received an external cerebrospinal fluid drain at the University Medical Centre Utrecht, a 1042-bed tertiary healthcare centre, were included in this study. Registration comprises all patients who received an EVD between January 1st 2004 and December 31st 2009 (with the exception of May–July 2004), and all patients receiving and ELD in 2004 to 2006. From January 2007 to December 2009 surveillance for ELD was only performed in patients who received the drain in operating theatres. Several infection control measures were implemented during the study period as described previously [20]. All EVDs are placed in operating theatres or, sometimes, under sterile conditions in the intensive care unit (ICU) by a neurosurgeon or trained resident. ELDs are either inserted in the operating theatre or in sterile conditions on the neurology ward. EVDs are tunneled five centimeters under the skin. All patients receive perioperative antibiotic prophylaxis. Drains are not exchanged on a prophylactic basis and CSF samples are collected for culture and biochemical analysis only when infection is clinically suspected; at this time empiric antibiotic therapy is initiated according to local protocol.

Throughout the study period, 742 patients received one or more drains. The following exclusion criteria were applied: death within one day after drain placement (40 patients), age under 18 years (n = 110), pre-existing central nervous system infection (n = 29), more than one simultaneous drain (n = 8), drain placement in a different centre (n = 4), admission duration less than one day (n = 3), second admission more than 30 days after discharge from a first drainage episode within the study (n = 7), and admission to the military hospital (n = 2). After application of exclusion criteria, 537 patients were available for analysis.

Data collection

The department of hospital hygiene provided outcome data along with the drain characteristics for each patient (drain type, duration, indication for placement). Prediction data was obtained through the Utrecht Patient Oriented Database (UPOD), a clinical data warehouse launched in 2004 for research purposes. The UPOD links results from laboratory analysis (clinical chemistry and hematology), microbiological cultures, and pharmacy dispensing data to information from the hospital admission and discharge system at the patient level [19]. Medication prescription data were obtained both from the UPOD as well as directly from the intensive care unit. Data were adapted to a standardized format and checked for inconsistencies. When necessary, original data sources were cross-referenced to exclude errors.

Predictor selection

Predictors were selected both on theoretical grounds and to best match the modified NHSN/CDC criteria. Duration of drainage, drain manipulation, subarachnoid hemorrhage, cerebrospinal fluid leakage, and other concomitant infections have been described as possible risk factors for meningitis previously [22]. Besides microbiological analysis of CSF and drains (after removal), biochemical markers of meningitis such as CSF leukocyte count, neutrophil count, glucose level, total protein level, and CSF/blood glucose ratio have been applied to the neurosurgical population with moderate success [23], [24] and were therefore considered as predictors. Unfortunately, Gram-stain results were not yet available and the UPOD does not contain information on drain manipulation and the occurrence of concomitant infections. If patients received both an EVD and an ELD, the EVD took priority in determining drain type. Culture results have been corrected for contamination by categorizing cultures growing coagulase-negative staphylococci as negative if no antibiotic therapy was initiated one day prior through three days after culture. Empiric antibiotic therapy was defined as the simultaneous use of vancomycine and ceftazidime (started four or more days after admission) or ceftriaxone and flucloxacillin (initiated within four days of admission) according to local protocol. The number of systemic antibiotics started throughout the surveillance episode was included as a surrogate marker for the presence of other concomitant infections.

Statistical analysis

Since the objective was to predict whether a patient had developed DRM during hospital stay, the value that was most indicative of infection measured throughout each patient's surveillance episode was taken for each predictor. Missing data were imputed using multiple imputation (ten imputations). For C-reactive protein (CRP), squared and cubic terms were included in the prediction model along with the linear term. The number of leukocytes in CSF was log-transformed prior to analyses.

Variables were selected for multivariate analysis based on theoretical considerations (as previously described) and results of univariate analysis (p<0.05 in the mean dataset). Using logistic regression analysis, a prediction model was then developed by means of manual backward selection (p<0.05). Although it is recommended to use higher p-values for selection of predictors in prediction research [25], this more stringent criterion was used due to the limited number of events (higher p-values would have resulted in too few events per predictor). Regression coefficients and standard errors were determined on each imputation set and pooled using Rubin's rule [26]. Subsequently, bootstrapping (100 samples per imputation set) was applied to correct for optimism.

Discrimination and calibration were determined for the final model. Discrimination refers to the ability to distinguish between patients with and without DRM; this was assessed by the area under the ROC curve. Calibration refers to the concordance between the predicted and observed probabilities of infection, which was assessed using a calibration plot. For clinical application, cut-off values for a predicted probability associated with high sensitivity and acceptable specificity were determined and associated sensitivity, specificity and predictive values were reported. Confidence intervals were determined using exact binomial methods. Finally, the summed predicted probabilities were used to investigate infection rates at the group level. All analyses were done using SPSS® 17 (SPSS Inc, Chicago IL) and R version 2.11.1 (www.r-project.org).

Results

A total of 691 drains were placed in 537 patients. DRM occurred in 82 patients (15.3%), or 13.5 infections per 1000 drainage days at risk. The most common causative micro-organisms were coagulase-negative staphylococci (33.8%), followed by Staphylococcus aureus (14.6%) and enterobacteriacaea (13.4%). Seventeen infections were culture negative (20.7%). Baseline characteristics are described in table 1. Median age of the included patients was 58.5 years, half (n = 263) received a CSF drainage system to treat secondary hydrocephalus following subarachnoid hemorrhage, intraventricular hemorrhage or (cerebellar) infarction and almost 60% (n = 312) of patients were admitted to the ICU during part of their stay. Patients were admitted for a median of 21 days (including readmissions within 30 days).

thumbnail
Table 1. Baseline characteristics of the patient population after multiple imputation of missing values and univariate association between variables and the risk of drain-related meningitis.

https://doi.org/10.1371/journal.pone.0022846.t001

Based on the results of the univariate analysis, the following variables were selected for multivariate analysis: indication for drain placement, duration of admission, number of drains placed, total drainage duration, duration of ICU admission, CRP, blood leukocytes, CSF leukocytes, CSF glucose, CSF protein, culture result (CSF and/or drain), total number of antibiotics started during admission, and whether empiric antibiotic therapy for drain-related meningitis was initiated.

Table 2 shows the predictors retained in the model and their associated p-values. Despite its high p-value (p = 0.230), the linear CRP term was kept in the model in order to allow the significant high-power terms to be included. The prediction rule can be used to calculate the probability of meningitis for each patient (figure 2). Discriminatory power of the model as determined by the area under the ROC curve was 0.970 (95% CI: 0.954–0.986). Calibration of the final model was good (figure 3).

thumbnail
Figure 2. Prediction rule for the development of drain-related meningitis.

Abbreviations: P(DRM) – probability of drain-related meningitis; LP – linear predictor; EVD – external ventricular drain; CRP – C-reactive protein; CSF – cerebrospinal fluid.

https://doi.org/10.1371/journal.pone.0022846.g002

thumbnail
Figure 3. Calibration plot of the model predicting drain-related meningitis.

The diagonal dashed line represents ideal prediction by the model, the pointed line predicted probabilities. Calibration, or the concordance between predicted and observed probability of infection, is adequate.

https://doi.org/10.1371/journal.pone.0022846.g003

thumbnail
Table 2. Outcome of backward stepwise logistic regression predicting the risk of drain-related meningitis.

https://doi.org/10.1371/journal.pone.0022846.t002

A cut-off in predicted probability of 0.107 resulted in 98.8% sensitivity, specificity of 87.9% and positive and negative predictive values of respectively 59.6% and 99.8% (table 3). The only missed infection was an infection with a coagulase-negative staphylococcus for which no antibiotics were started during admission and of which the patient recovered spontaneously. Selecting a cut-off probability of 0.175 missed three additional infections (sensitivity 95.1%), but only slightly improved specificity (91.0%) and positive predictive value (65.5%).

thumbnail
Table 3. Two-by-two contingency table for predicted probability (P(DRM)) in relation to drain-related meningitis.

https://doi.org/10.1371/journal.pone.0022846.t003

If a definite diagnosis is necessary at the patient level, application of the model reduced the number of charts to review manually to 25.3% (from 537 to 136 charts) while still identifying 98.8% of infections (81 out of 82). When interested in infection rates at the group level, the summed predicted probabilities reflect total infection percentages with good concordance (figure 4) and thus allow for surveillance without the need for manual confirmation.

thumbnail
Figure 4. Comparison of observed and predicted overall yearly infection rates.

Observed and predicted infection rates at the patient level (panel A) and expressed per 1000 drainage days at risk (panel B), including 95% confidence intervals. Predicted yearly infection rates are determined by the summed predicted probabilities and show good concordance with observed rates.

https://doi.org/10.1371/journal.pone.0022846.g004

Discussion

The results of this study show that information stored in clinical data warehouses can successfully be used to predict rates of DRM in patients receiving an external ventricular or lumbar drain. The combination of drain characteristics, microbiology and clinical chemistry results and antibiotic use achieved 98.8% sensitivity and 87.5% specificity in detecting drain-related meningitis when applying a predicted probability cut-off of 0.107. Negative and positive predictive values were 99.8% and 56.9% respectively. Performing chart review only for those patients identified by the model to have DRM would reduce the number of manual chart reviews by 74.7%. Monitoring of longitudinal infection rates at the group level, however, can be performed without manual confirmation, thereby providing an efficient surveillance tool. This study can be viewed as proof-of-concept for use of regression model-based systems to perform surveillance of nosocomial infections at the group level.

As opposed to most detection models described previously, the model presented here uses data from a multitude of sources in a multivariable model, and case-finding is based on the weighted combination of predictors from each source. As opposed to classification algorithms with case-finding based on broadly selected indicators, this weighted combination of predictors leads to high sensitivity for both culture-positive and culture-negative infections while maintaining acceptable positive predictive value. To the best of our knowledge, this is the first model also incorporating results of clinical chemistry and hematology analysis as predictors of healthcare-associated infections.

In comparison to conventional manual surveillance, this model reduces the time needed to perform surveillance, is less prone to error and less vulnerable to inter-rater variation. Furthermore, calculation of summed predicted probabilities for the at-risk population is an efficient surveillance tool to monitor changes in infection rates and determine when to perform in-depth analysis. Several studies have shown that although automated models using simplified and objective criteria may not always correctly predict absolute infection rates, such models may achieve reliable ranking of hospitals and accentuate differences between hospitals [12], [27], [28].

The large patient population included in this study allowed for the application of statistical methods as opposed to classification algorithms. Although the rule-of-thumb of ten events per predictor was violated, this does not necessarily lead to unreliable results [29]. Furthermore, the selected reference standard, the CDC/NHSN definition of healthcare-associated meningitis, has been measured consistently over time for purposes other than this research. Several other definitions of DRM have been used in literature [22], [30][32], however they mostly require positive culture results and therefore have low sensitivity for culture-negative infections which occurred in twenty percent of cases in this population. Even though the CDC/NHSN definition is only partially applicable to neurosurgical patients who are comatose or sedated, the other definitions of DRM will also face this problem as many require the presence of clinical symptoms to confirm the diagnosis. Although it can be argued that healthcare-associated meningitis is a different clinical entity than meningitis secondary to cerebrospinal fluid drainage, the selected reference standard has been measured consistently and reliably over the six-year period and contains many similarities to other definitions proposed for drain-related meningitis. Imputation of missing values was used to prevent the introduction of bias in deriving the model. Since it is not possible to impute missing values for individual patients, a probability of infection can not be computed for future patients with missing data. Out of the patients with an infection, only one had missing data for one predictor (CSF leukocyte count), thereby making underestimation of infection rates unlikely. Furthermore, predictors were only included if commonly determined in clinical practice. For this reason, parameters that have been described previously such as CSF lactate levels [33], [34], CSF cytokine levels [23], [35], and procalcitonin levels [36], [37] were not considered for inclusion. The calculation of the cell-index was considered as a tool to correct for blood-contaminated CSF [38]; however, since this measure could not be calculated in 65.3% of patients due to missing data, it was not included in the analyses. Finally, this model does not investigate infections occurring after discharge unless the patient is readmitted. However, contrary to surgical site infections, post-discharge surveillance is not as relevant since patients often remain in the hospital for a number of days after removal of the drain and it is customary for patients to return to their primary hospital when complications occur. These patients are then re-included in surveillance if readmission occurs within 30 days of discharge.

In summary, the model developed can accurately quantify rates of drain-related meningitis using multi-source data. The proposed model was developed using only retrospective data and although measures have been taken to prevent excessive optimism, prospective validation both within our centre and on a larger scale is necessary to assure applicability to other patient populations. This multivariable model-based approach can be applied to other types of nosocomial infections in the future. Also the development of methods to determine device utilization rates using data available through electronic healthcare records will further improve efficiency and reliability of surveillance.

Acknowledgments

The authors would like to thank all contributing data managers, H.E.M. Blok, H. den Breeijen and E. Mulder, for their contribution as well as the infection control professionals from the department of hospital hygiene and infection control (A.C.M. Gigengack-Baars, M.C.E. van der Jagt-Zwetsloot, S.M. van Dijk).

Author Contributions

Conceived and designed the experiments: MSMvM AT MJMB. Analyzed the data: MSMvM RHHG. Wrote the paper: MSMvM RHHG JWBvdS WWvS AT MJMB. Coordinated the clinical data warehouse: WWvS.

References

  1. 1. ECDC (2008) Annual Epidemiological Report on Communicable Disease in Europe 2008. Available: http://ecdc.europa.eu/en/publications/Publications/0812_SUR_Annual_Epidemiological_Report_2008.pdf. Accessed 2011 Feb 23.
  2. 2. Edwards JR, Peterson KD, Mu Y, Banerjee S, len-Bridson K, et al. (2009) National Healthcare Safety Network (NHSN) report: data summary for 2006 through 2008, issued December 2009. Am J Infect Control 37: 783–805.
  3. 3. van der Kooi TI, Mannien J, Wille JC, van Benthem BH (2010) Prevalence of nosocomial infections in The Netherlands, 2007–2008: results of the first four national studies. J Hosp Infect 75: 168–72.
  4. 4. Haley RW, Culver DH, White JW, Morgan WM, Emori TG, et al. (1985) The efficacy of infection surveillance and control programs in preventing nosocomial infections in US hospitals. Am J Epidemiol 121: 182–205.
  5. 5. Ehrenkranz NJ, Shultz JM, Richter EL (1995) Recorded criteria as a “gold standard” for sensitivity and specificity estimates of surveillance of nosocomial infection: a novel method to measure job performance. Infect Control Hosp Epidemiol 16: 697–702.
  6. 6. Gastmeier P, Kampf G, Hauer T, Schlingmann J, Schumacher M, et al. (1998) Experience with two validation methods in a prevalence survey on nosocomial infections. Infect Control Hosp Epidemiol 19: 668–73.
  7. 7. Trick WE (2008) Building a data warehouse for infection control. Am J Infect Control 36: S75–81.
  8. 8. Bellini C, Petignat C, Francioli P, Wenger A, Bille J, et al. (2007) Comparison of automated strategies for surveillance of nosocomial bacteremia. Infect Control Hosp Epidemiol 28: 1030–5.
  9. 9. Woeltje KF, Butler AM, Goris AJ, Tutlam NT, Doherty JA, et al. (2008) Automated surveillance for central line-associated bloodstream infection in intensive care units. Infect Control Hosp Epidemiol 29: 842–6.
  10. 10. Klompas M, Yokoe DS (2009) Automated surveillance of health care-associated infections. Clin Infect Dis 48: 1268–75.
  11. 11. Yokoe DS, Noskin GA, Cunnigham SM, Zuccotti G, Plaskett T, et al. (2004) Enhanced identification of postoperative infections among inpatients. Emerg Infect Dis 10: 1924–30.
  12. 12. Platt R, Kleinman K, Thompson K, Dokholyan RS, Livingston JM, et al. (2002) Using automated health plan data to assess infection risk from coronary artery bypass surgery. Emerg Infect Dis 8: 1433–41.
  13. 13. Sands K, Vineyard G, Livingston J, Christiansen C, Platt R (1999) Efficient identification of postdischarge surgical site infections: use of automated pharmacy dispensing information, administrative data, and medical record information. J Infect Dis 179: 434–41.
  14. 14. Bolon MK, Hooper D, Stevenson KB, Greenbaum M, Olsen MA, et al. (2009) Improved surveillance for surgical site infections after orthopedic implantation procedures: extending applications for automated data. Clin Infect Dis 48: 1223–9.
  15. 15. Graham PL III, San GP, Lutwick S, Haas J, Saiman L (2004) Validation of a multicenter computer-based surveillance system for hospital-acquired bloodstream infections in neonatal intensive care departments. Am J Infect Control 32: 232–4.
  16. 16. Trick WE, Zagorski BM, Tokars JI, Vernon MO, Welbel SF, et al. (2004) Computer algorithms to detect bloodstream infections. Emerg Infect Dis 10: 1612–20.
  17. 17. Pokorny L, Rovira A, Martin-Baranera M, Gimeno C, onso-Tarres C, et al. (2006) Automatic detection of patients with nosocomial infection by a computer-based surveillance system: a validation study in a general hospital. Infect Control Hosp Epidemiol 27: 500–3.
  18. 18. Brossette SE, Hacek DM, Gavin PJ, Kamdar MA, Gadbois KD, et al. (2006) A laboratory-based, hospital-wide, electronic marker for nosocomial infection: the future of infection control surveillance? Am J Clin Pathol 125: 34–9.
  19. 19. ten Berg MJ, Huisman A, van den Bemt PM, Schobben AF, Egberts AC, et al. (2007) Linking laboratory and medication data: new opportunities for pharmacoepidemiological research. Clin Chem Lab Med 45: 13–9.
  20. 20. Leverstein-Van Hall MA, Hopmans TEM, Van Der Sprenkel JWB, Blok HEM, Van Der Mark WAMA, et al. (2010) A bundle approach to reduce the incidence of external ventricular and lumbar drain-related infections: Clinical article. J Neurosurg 112: 345–53.
  21. 21. Horan TC, Andrus M, Dudeck MA (2008) CDC/NHSN surveillance definition of health care-associated infection and criteria for specific types of infections in the acute care setting. Am J Infect Control 36: 309–32.
  22. 22. Lozier AP, Sciacca RR, Romagnoli MF, Connolly ES Jr (2002) Ventriculostomy-related infections: a critical review of the literature. Neurosurgery 51: 170–81.
  23. 23. Schade RP, Schinkel J, Roelandse FW, Geskus RB, Visser LG, et al. (2006) Lack of value of routine analysis of cerebrospinal fluid for prediction and diagnosis of external drainage-related bacterial meningitis. J Neurosurg 104: 101–8.
  24. 24. Pfisterer W, Muhlbauer M, Czech T, Reinprecht A (2003) Early diagnosis of external ventricular drainage infection: results of a prospective study. J Neurol Neurosurg Psychiatry 74: 929–32.
  25. 25. Steyerberg EW (2008) Clinical Prediction Models: A practical approach to development, validation, and updating. New York: Springer. 497 p.
  26. 26. Rubin DB (1987) Multiple Imputation for Nonresponse in Surveys. Hoboken: J. Wiley & Sons. 288 p.
  27. 27. Huang SS, Livingston JM, Rawson NS, Schmaltz S, Platt R (2007) Developing algorithms for healthcare insurers to systematically monitor surgical site infection rates. BMC Med Res Methodol 7: 20.
  28. 28. Rubin MA, Mayer J, Greene T, Sauer BC, Hota B, et al. (2008) An agent-based model for evaluating surveillance methods for catheter-related bloodstream infection. AMIA Annu Symp Proc 631–5.
  29. 29. Vittinghoff E, McCulloch CE (2007) Relaxing the rule of ten events per variable in logistic and Cox regression. Am J Epidemiol 165: 710–8.
  30. 30. Mayhall CG, Archer NH, Lamb VA, Spadora AC, Baggett JW, et al. (1984) Ventriculostomy-related infections. A prospective epidemiologic study. N Engl J Med 310: 553–9.
  31. 31. Sundbarg G, Nordstrom CH, Soderstrom S (1988) Complications due to prolonged ventricular fluid pressure recording. Br J Neurosurg 2: 485–95.
  32. 32. Holloway KL, Barnes T, Choi S, Bullock R, Marshall LF, et al. (1996) Ventriculostomy infections: the effect of monitoring duration and catheter exchange in 584 patients. J Neurosurg 85: 419–24.
  33. 33. Leib SL, Boscacci R, Gratzl O, Zimmerli W (1999) Predictive value of cerebrospinal fluid (CSF) lactate level versus CSF/blood glucose ratio for the diagnosis of bacterial meningitis following neurosurgery. Clin Infect Dis 29: 69–74.
  34. 34. Wong GK, Poon WS, Ip M (2008) Use of ventricular cerebrospinal fluid lactate measurement to diagnose cerebrospinal fluid infection in patients with intraventricular haemorrhage. J Clin Neurosci 15: 654–5.
  35. 35. Lopez-Cortes LF, Marquez-Arbizu R, Jimenez-Jimenez LM, Jimenez-Mejias E, Caballero-Granado FJ, et al. (2000) Cerebrospinal fluid tumor necrosis factor-alpha, interleukin-1beta, interleukin-6, and interleukin-8 as diagnostic markers of cerebrospinal fluid infection in neurosurgical patients. Crit Care Med 28: 215–9.
  36. 36. Berger C, Schwarz S, Schaebitz WR, Aschoff A, Schwab S (2002) Serum procalcitonin in cerebral ventriculitis. Crit Care Med 30: 1778–81.
  37. 37. Martinez R, Gaul C, Buchfelder M, Erbguth F, Tschaikowsky K (2002) Serum procalcitonin monitoring for differential diagnosis of ventriculitis in adult intensive care patients. Intensive Care Med 28: 208–10.
  38. 38. Pfausler B, Beer R, Engelhardt K, Kemmler G, Mohsenipour I, et al. (2004) Cell index–a new parameter for the early diagnosis of ventriculostomy (external ventricular drainage)-related ventriculitis in patients with intraventricular hemorrhage? Acta Neurochir (Wien) 146: 477–81.