Repeated rotation of empiric antibiotic treatment strategies is hypothesized to reduce antibiotic resistance. Clinical rotation studies failed to change unit-wide prevalence of antibiotic resistant bacteria (ARB) carriage, including an international cluster-randomized crossover study. Unit-wide effects may differ from individual effects due to “ecological fallacy”. This post-hoc analysis of a cluster-randomized crossover study assesses differences between cycling and mixing rotation strategies in acquisition of carriage with Gram-negative ARB in individual patients.
This was a controlled cluster-randomized crossover study in 7 ICUs in 5 European countries. Clinical cultures taken as routine care were used for endpoint assessment. Patients with a first negative culture and at least one culture collected in total were included. Community acquisitions (2 days of admission or less) were excluded. Primary outcome was ICU-acquisition of Enterobacterales species with reduced susceptibility to: third- or fourth generation cephalosporins or piperacillin-tazobactam, and Acinetobacter species and Pseudomonas aeruginosa with reduced susceptibility for piperacillin-tazobactam or carbapenems. Cycling (altering first-line empiric therapy for Gram-negative bacteria, every other 6-weeks), to mixing (changing antibiotic type every empiric antibiotic course). Rotated antibiotics were third- or fourth generation cephalosporins, piperacillin-tazobactam and carbapenems.
For this analysis 1,613 admissions were eligible (855 and 758 during cycling and mixing, respectively), with 16,437 microbiological cultures obtained. Incidences of acquisition with ARB during ICU-stay were 7.3% (n = 62) and 5.1% (n = 39) during cycling and mixing, respectively (p-value 0.13), after a mean of 17.7 (median 15) and 20.8 (median 13) days. Adjusted odds ratio for acquisition of ARB carriage during mixing was 0.62 (95% CI 0.38 to 1.00). Acquired carriage with ARB were Enterobacterales species (n = 61), Pseudomonas aeruginosa (n = 38) and Acinetobacter species (n = 20), with no statistically significant differences between interventions.
There was no statistically significant difference in individual patients’ risk of acquiring carriage with Gram-negative ARB during cycling and mixing. These findings substantiate the absence of difference between cycling and mixing on the epidemiology of Gram-negative ARB in ICU.
This trial is registered with ClinicalTrials.gov, registered 10 January 2011, NCT01293071.
Citation: van Duijn PJ, Verbrugghe W, Jorens PG, Spöhr F, Schedler D, Deja M, et al. (2022) The effects of antibiotic cycling and mixing on acquisition of antibiotic resistant bacteria in the ICU: A post-hoc individual patient analysis of a prospective cluster-randomized crossover study. PLoS ONE 17(5): e0265720. https://doi.org/10.1371/journal.pone.0265720
Editor: Dafna Yahav, Rabin Medical Center, Beilinson Hospital, ISRAEL
Received: May 12, 2021; Accepted: March 5, 2022; Published: May 3, 2022
Copyright: © 2022 van Duijn et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: At the time of data collection, no measures were in place for public disclosure of data, nor was it a prerequisite for approval by the Internal Review Board or funding body. These data contain potentially identifying and sensitive values. When requested the data will be made available, if possible, with appropriate anonymization, depending on the specific data request assessing the restrictions of each participating study sites' Internal Review Board. These data are available via the Department of Data Management & Project Support, Julius Centre for Health Sciences and Primary Care, Utrecht, The Netherlands (F.R.Leus@umcutrecht.nl).
Funding: This work was supported financially by the European Commission under the Life Science Health Priority of the Seventh Framework Programme, under grant agreement number 241796 (SATURN project). Individual authors received no funding for this work, other than compensation for personnel for collecting study data. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: I have read the journal’s policy and the authors of this manuscript have the following competing interests: SH has received consulting fees from Sandoz. All other authors have declared that no competing interests exist
Abbreviations: ARB, Antibiotic resistant bacteria; ICU, Intensive Care Unit; ASP, Antibiotic stewardship programs; DDD, Defined Daily Dose; aOR, Adjusted odds-ratio
Treatment of critically ill patients admitted in Intensive Care Units (ICU) is frequently complicated by infections caused by antibiotic resistant Gram-negative bacteria. Patients in ICU often receive broad spectrum antibiotics which increases antibiotic resistance selective pressure and the chance of acquiring colonization with Gram-negative antibiotic resistant bacteria (ARB). To reduce this selective pressure, unit-wide antibiotic stewardship programs (ASP) have been advocated, sometimes advocating but also discouraging antibiotic rotation strategies [1–3]. These strategies aim to modulate the diversity of antibiotic exposure in a ward, rather than reducing overall antibiotic use. The increased heterogeneity of antibiotic exposure, hypothetically, reduces antibiotic resistance selection pressure and occurrence of antibiotic resistant bacteria [4–12].
Previous observational and quasi-experimental studies, however, yielded non-conclusive results for different pathogens, rotation schedules and outcomes [13–26]. In a multi-center cluster crossover study, two antibiotic rotation interventions, cycling and mixing, we found similar effects on the unit-wide ecology of Gram-negative ARB . In that study two different antibiotic rotation strategies for empiric treatment of patients with presumed Gram-negative infections were compared: During cycling the preferred antibiotic treatment changed every six weeks and during mixing it changed after every single patient. Effectiveness of the intervention was determined by measuring the prevalence of carriage with antibiotic resistant Gram-negative bacteria at the unit level, through monthly point-prevalence surveys. However, group-ecological and individual risks can differ, even within the same experimental study, due to what is called the “ecological fallacy” . We, therefore, performed a post-hoc analysis on study data of this previously performed study to investigate whether two antibiotic rotation schemes, cycling and mixing, yielded differences in the individual risk of acquiring Gram-negative ARB during ICU admission.
This was a post-hoc nested cohort analysis of a cluster-randomized cross-over study . Four months of standard care treatment preceded ICU randomization to two 9-month intervention periods. A one-month wash-out period separated the intervention periods. ICUs were cluster-allocated by randomization to perform either the cycling strategy followed by mixing or vice versa. Computer randomization of the allocation to interventions (in a 1:1 ratio) and randomization of the order of consecutively rotated antibiotics (in a random sequence per ICU) was performed by a person not involved in designing or performing the study. First-line empirical therapy for patients with assumed infections that required treatment of Gram-negative bacteria was rotated in 6-week periods between 3rd or 4th-generation cephalosporins, piperacillin-tazobactam and carbapenems (cycling). During mixing, the preferred empiric therapy was rotated for every new patient needing treatment. Patients could be treated with different antibiotic courses during admission. Readmissions were included. To safeguard optimal patient care, protocol allowed for physicians to change patient therapy on an individual basis at any time (e.g., de-escalation, combination therapy, allergic reactions or patient safety). There was no blinding of intervention allocation for physicians during admission, for those responsible for data collection or the patient. For the current analysis we had access to individual microbiological culture data from seven of eight participating European ICUs from Belgium, France, Germany, Portugal, and Slovenia. These were ICUs with mixed, medical or surgical patient populations.
Patients were included if a first culture was negative for Gram-negative ARB in cultures from the respiratory- or gastrointestinal tract (e.g. feces, rectum, perineum or gastric contents).
Enrolment of patients was preceded by approvement of the study and a waiver for individual informed consent, by all local Interne Review Boards of each participating center. This trial is registered with ClinicalTrials.gov, number NCT01293071. There were no changes to the study design after trial commencement.
The primary endpoint was the first clinical culture with Gram-negative ARB, defined as non-susceptibility to 3rd- or 4th generation cephalosporins and piperacillin-tazobactam in Enterobacterales species, and piperacillin-tazobactam or- meropenem resistance in Pseudomonas aeruginosa and Acinetobacter species. For the current analysis patients were eligible if microbiological cultures had been obtained from either the respiratory tract or gastrointestinal tract. Admissions with detected Gram-negative ARB carriage in the first two days of admission were excluded, as were admissions with positive endpoints on the same day as a first negative culture. Acquired carriage with Gram-negative ARB after day 2 of admission was assumed to be permanent for the duration of the study, i.e., readmissions of these patients were excluded from the analysis.
All clinical cultures taken during the study periods were included for endpoint analysis. If the primary endpoint was reached in an individual patient, subsequent culture results were excluded. Microbiological procedures were performed according to local laboratory practices, including local Minimum Inhibitory Concentration cutoff values. Participating microbiological laboratories did not change protocols for antibiotic susceptibility testing during the study. Likewise, infection prevention- and control measures did not change during the study period.
Systemic antibiotic use was collected at the individual or aggregate level. Consumed quantities of antibiotics were converted to WHO Defined Daily Dose (DDD). Use of different antibiotic groups were represented as subdivided 6-week periods for cycling, and divided over the 9-month intervention period for mixing.
Ethics approval and consent to participate
The study protocol was approved by each local Institutional Review Board (IRB) and all centers obtained a waiver for individual patient written informed consent.
Demographic- and infection prevention variables comparison between interventions were performed using bivariate tests. Dichotomous endpoints were tested using Pearsons’ chi-square test and for continuous outcomes using Students’ t-test. The primary outcome analysis was performed using chi-square test for binary endpoints. Odds ratios for acquisition between the two intervention periods (cycling and mixing) were calculated using mixed effects logistic regression modelling, with adjustment for clustering within each hospital and for confounders age, gender, length of stay, previous admission, origin of transfer to ICU and survival at ICU-discharge). Independent variables for this model were chosen based on potential correlation with the intervention and endpoint and being reasonably objectifiable.
Additionally, detection bias between intervention periods from clinical cultures was assessed by modeling the probability of having a culture taken during admission using a mixed effects model correcting for clustering within individual ICU. This analysis was performed on all admissions during the intervention period, including patients without clinical cultures obtained. To assess competing events bias, mean and median length of stay was calculated and linear regression modelling of the effect of intervention type on length of stay.
Carryover effects between first and second intervention period were assessed using the mixed effects model with an additional interaction term between intervention type and the sequence of performing mixing-then-cycling or vice-versa.
Post-hoc power calculations were performed using the pwr package, an effect size of 0.1 based on an arbitrary ‘small’ effect, significance level of 5% and assumed a two-sided alternative outcome. The calculated power to find a relevant difference was 88%. Analyses were performed using R software .
During the cycling and mixing intervention periods, there were 8,267 admissions overall in 7 ICUs in 5 countries (Fig 1). For this nested cohort study, 1,613 (19.5%) admissions were eligible. Data was collected from June 27, 2011, to February 16, 2014. Baseline demographics and ICU characteristics of interventions were comparable (Tables 1 and 2).
In these patients 16,437 microbiological cultures were collected (mean of 10.27 cultures per admission); upper- or lower respiratory materials (%), blood- or intravascular cultures (34.8%), enteric cultures (e.g. gastric fluid, bile, feces, rectum swabs, 29.9%) and urine (12.2%) (Appendix Table 1 in S1 Appendix). In total 8.3% of the cultures was taken as part of surveillance for carriage, not for diagnosis and treatment of infection (missings 12.7%). In cultures that grew Enterobacterales, most were Escherichia coli (22.5%), Pseudomonas species (21.2%), Klebsiella species (18.8%) and Enterobacter species (11.2%, Appendix Table 2 in S1 Appendix). The odds ratio (OR) of having a culture taken was lower during the mixing intervention (OR 0.83, 95% confidence interval (CI) 0.76 to 0.91, p-value <0.01) (Appendix Table 3 in S1 Appendix), which did not change after correction for confounders (age, gender, length of stay, previous admission and origin of transfer/referral): Adjusted odds ratio (aOR) 0.84 (95% CI 0.76 to 0.93).
In all, there were 855 and 758 admissions eligible for analysis during cycling and mixing, respectively. The primary endpoint was reached in 62 (7.3%) and in 39 (5.1%) patients during cycling and mixing, respectively, (p-value: 0.13), after a mean of 17.7 (median 15) and 20.8 (median 13) days, in cycling and mixing respectively (p-value: 0.43). Distributions of endpoint defined micro-organisms were: Enterobacterales species (n = 61), Pseudomonas aeruginosa (n = 38) and Acinetobacter species (n = 20), without statistically significant differences between interventions (Table 3).
Antibiotic use per interventions were similar, as expected by the protocol of equal use of each antibiotic (group) type over time (Appendix Table 4 in S1 Appendix).
In mixed effect logistic regression modelling the unadjusted odds ratio for acquiring Gram-negative ARB was 0.72 (CI 95% 0.47 to 1.10) during mixing compared to cycling (Table 4). After adjustment for age, gender, length of stay, previous admission and origin of referral (community or hospital), adjusted odds ratio was 0.62, 95% CI 0.38 to 1.00). Adding an interaction term for intervention strategy with the sequence of strategies to test for a carryover effect did not change results (Appendix Table 5 in S1 Appendix).
The culture probability OR was used for a sensitivity analysis. Here we assume the mixing population was indeed cultured less with missed endpoints. We assumed the size of this hypothetical mixing population was equal to the ‘real’ cultured cycling population. With an OR of 0.83, the mixing hypothetical population is roughly 120% of the real mixing population. To meet statistical significance in a comparison of the hypothetical mixing population and the real cycling population, the incidence in this missed group would need to be 17%. This is more than two times the prevalence found in the real mixing study population.
In this patient-level analysis of a randomized cluster design multi-center study, we found no difference in effect of two antibiotic rotation strategies on the individual risk of acquiring colonization with Gram-negative ARB. These findings are in line with the results of the previously reported ecological analysis of this trial .
Cycling and mixing are strategies that create opposite extremes of antibiotic diversity: Cycling maximizes homogeneity during a 6-week period, and mixing maximizes heterogeneity.
The cluster randomized, crossover design allows comparison of unit-wide interventions in ICUs. As compared to cohort studies it reduces the likelihood of differences in clinical practice between ICUs, and other differences such as case-mix that may affect endpoint risk estimates. Compared to individual randomization, cluster randomization prevents non-adherence to protocol in individual patients, but provides an estimate of intervention effect at the ward level.
With respect to patient characteristics there were statistically significant differences in mortality and illness severity scores. For mortality this was assumed to be a chance finding, as both intervention arms did not aim to influence mortality, and both arms received the same amounts and types of antibiotics. This difference is also reflected in the mortality scores APACHE II, SAPS II, TIS-28, which do not represent relevant differences: 2, 4.1 and 4.8 respectively points on these scores with maximum scores being from 55, 78 to 163.
It was not possible to compare the interventions to a non-intervention period due to high baseline variability of antibiotic use between ICUs. More specific, there is no parameter to measure antibiotic diversity, therefore it is not possible to quantitatively compare baseline periods, whether these are alike -or different- to their corresponding mixing or cycling periods.
This post-hoc analysis was motivated by previous findings of different results from ecological- and individual-based analyses . This study analyzed individual risks ratio of acquisition of ARB, in contrast to the previous ecological study that evaluated aggregated monthly point-prevalence surveys. Though ecological analyses offer useful information on group-wise interventions, such analyses can omit patient-level causal relations between exposure and outcome. This can lead to a form of bias called the ecological fallacy. Consequently, it has been advised to use both ecological as well as individual-level analyses for studies in antibiotic resistance when possible .
Ecological fallacy occurs when group-averaged risks lead to incorrect associations between exposure and outcome. For instance, when the exposed patients are not the same patients that acquire the endpoint. Or, when group analyses overlook that the outcome might have occurred before the exposure, distorting causal inference. Or when individual baseline and longitudinal risks differ over time, creating different subgroup risk profiles. For instance, when in time, enrolled patients have incrementally higher baseline-risk for adverse outcome at admission, (due to, for instance, stricter admission criteria), but individual treatment outcomes during admission improve over time. A mechanism that has been termed Simpsons Paradox . The first two mechanisms were prevented by the cluster design of the study, in which all patients underwent the same cluster-intervention from the time of admission, precluding a distorted exposure-outcome relation. Yet, the third mechanism for ecological fallacy–differences in individual baseline and longitudinal risks—could have occurred.
For this analysis, available clinical cultures were used as endpoint determinant. The inclusion criterion of having a first negative enteric or respiratory culture ascertained acquisition of colonization with Gram-negative antibiotic resistant bacteria. The intervention is still ecological, but the analysis aims to provide individual risk estimates for colonization in a ward where antibiotic rotation is applied.
The restricted availability of these cultures prompted us to assess the presence and size of detection bias, which could have resulted from differential inclusion between intervention arms, due to the absence of individual randomization. We found no indication that baseline demographics or confounders were different between interventions, and adjustment for this in regression analysis did not change effect estimates. Alternatively, there could be a direct effect of the intervention on diagnostic culture practices. We used the source population (all included patients with and without cultures) to perform two additional analyses to assess inclusion bias or an intervention effect on culture rates: 1) Assessing the differences between interventions of the probability of a culture being taken by regression analysis, and 2) a sensitivity analysis to contextualize the size of the potential intervention-effect on detection bias and outcome.
The probability of having a culture taken was lower during the mixing period, but there was no evidence that this was due to differences in patient characteristics, nor were there differences in intervention effects after adjusting for confounders in the primary analysis. Sensitivity analysis yielded that detection bias would have needed to have caused a relative decrease in incidence of ARB acquisition with 29.2% (absolute 2.8%) to achieve statistical significance of the primary analysis. Or, in other words, in the uncultured patients in the mixing population that should have been cultured, based on our model, endpoint incidence would have to have been 16.6%, compared to 7.3% in the included mixing population, to achieve statistical significance. We consider this unlikely and therefore conclude that detection bias did not substantially reduce the validity of our analysis.
Naturally, generalizability of our findings is shifted towards patients with a clinical culture taken, and thus patients with a relatively long stay in ICU. In fact, despite generally a low threshold for collecting clinical cultures in ICU patients in the participating ICUs, excluded patients were mostly short-stay patients, discharged after a median 2 days of admission. Furthermore, results would not be generalizable to ICUs with higher endemic prevalence of antibiotic resistant bacteria, where the antibiotics that were used in this study cannot be used for empiric therapy. These results however, are generalizable to most European ICUs, and any non-European ICU with similar technical capacities, staffing and resistance prevalence.
Our findings do not support superiority of effects of cycling over mixing or vice versa on acquisition of ARB in the participating ICUs. Based on current scientific evidence, including our study, antibiotic rotation should, therefore, not be recommended as standard care [13–27]. This study, however, rotated beta-lactam antibiotics exclusively, with fixed and pre-defined rotation schedules of per-patent rotation and 6-week periods. There are many variations of antibiotic rotation and effectiveness of some scheme is not excluded . Resistance acquisition under antibiotic rotation strategies has layered complexity from the microbiome, infection control measures, and collateral sensitivity . Further research is needed, and future clinical studies will benefit from a multi-disciplinary approach by including basic sciences and mathematical modeling. Ultimately the goal should be to provide tailor-made algorithms to guide ICU antibiotic policies, in order to optimize resistance selective pressure and patient safety.
S1 Checklist. CONSORT 2010 checklist of information to include when reporting a randomised trial*.
- 1. Barlam TF, Cosgrove SE, Abbo LM, Macdougall C, Schuetz AN, Septimus EJ, et al. Executive summary: Implementing an antibiotic stewardship program: Guidelines by the infectious diseases society of America and the society for healthcare epidemiology of America. Clinical Infectious Diseases. 2016. pmid:27118828
- 2. Lipsitch M, Samore MH. Antimicrobial use and antimicrobial resistance: A population perspective. Emerging Infectious Diseases. 2002. pmid:11971765
- 3. Livermore DM. Of stewardship, motherhood and apple pie. International Journal of Antimicrobial Agents. 2014. pmid:24630303
- 4. Bonhoeffer S, Lipsitch M, Levin BR. Evaluating treatment protocols to prevent antibiotic resistance. Proc Natl Acad Sci. 1997. pmid:9342370
- 5. Bergstrom CT, Lo M, Lipsitch M. From The Cover: Ecological theory suggests that antimicrobial cycling will not reduce antimicrobial resistance in hospitals. Proc Natl Acad Sci. 2004. pmid:15308772
- 6. Reluga TC. Simple models of antibiotic cycling. Math Med Biol. 2005. pmid:15778333
- 7. Beardmore R, Peña-Miller R. Rotating antibiotics selects optimally against antibiotic resistance, in theory. Math Biosci Eng. 2010;7: 527–552. pmid:20578784
- 8. Beardmore RE, Peña-Miller R. Antibiotic cycling versus mixing: The difficulty of using mathematical models to definitively quantify their relative merits. Math Biosci Eng. 2010. pmid:21077716
- 9. Kouyos RD, Abel zur Wiesch P, Bonhoeffer S. Informed switching strongly decreases the prevalence of antibiotic resistance in hospital wards. PLoS Comput Biol. 2011. pmid:21390265
- 10. Martínez JA, Nicolás JM, Marco F, Horcajada JP, Garcia-Segarra G, Trilla A, et al. Comparison of antimicrobial cycling and mixing strategies in two medical intensive care units. Crit Care Med. 2006. pmid:16424711
- 11. D’Agata EMC, Dupont-Rouzeyrol M, Magal P, Olivier D, Ruan S. The impact of different antibiotic regimens on the emergence of antimicrobial-resistant bacteria. PLoS One. 2008. pmid:19112501
- 12. Abel zur Wiesch P, Kouyos R, Abel S, Viechtbauer W, Bonhoeffer S. Cycling Empirical Antibiotic Therapy in Hospitals: Meta-Analysis and Models. PLoS Pathog. 2014. pmid:24968123
- 13. Gruson D, Hilbert G, Vargas F, Valentino R, Bui N, Pereyre S, et al. Strategy of antibiotic rotation: Long-term effect on incidence and susceptibilities of Gram-negative bacilli responsible for ventilator-associated pneumonia. Crit Care Med. 2003. pmid:12847382
- 14. Cobos-Trigueros N, Solé M, Castro P, Torres JL, Rinaudo M, De Lazzari E, et al. Evaluation of a Mixing versus a Cycling Strategy of Antibiotic Use in Critically-Ill Medical Patients: Impact on Acquisition of Resistant Microorganisms and Clinical Outcomes. PLoS One. 2016. pmid:26982807
- 15. Raymond DP, Pelletier SJ, Crabtree TD, Gleason TG, Hamm LL, Pruett TL, et al. Impact of a rotating empiric antibiotic schedule on infectious mortality in an intensive care unit. Crit Care Med. 2001. pmid:11395583
- 16. Van Loon HJ, Vriens MR, Fluit AC, Troelstra A, Van Der Werken C, Verhoef J, et al. Antibiotic rotation and development of Gram-negative antibiotic resistance. Am J Respir Crit Care Med. 2005. pmid:15516540
- 17. Raineri E, Crema L, Dal Zoppo S, Acquarolo A, Pan A, Carnevale G, et al. Rotation of antimicrobial therapy in the intensive care unit: Impact on incidence of ventilator-associated pneumonia caused by antibiotic-resistant Gram-negative bacteria. Eur J Clin Microbiol Infect Dis. 2010. pmid:20524138
- 18. Hedrick TL, Schulman AS, McElearney ST, Smith RL, Swenson BR, Evans HL, et al. Outbreak of resistant Pseudomonas aeruginosa infections during a quarterly cycling antibiotic regimen. Surg Infect. 2008. pmid:18426346
- 19. Ginn AN, Wiklendt AM, Gidding HF, George N, O’Driscoll JS, Partridge SR, et al. The ecology of antibiotic use in the ICU: Homogeneous prescribing of cefepime but not tazocin selects for antibiotic resistant infection. PLoS One. 2012. pmid:22761698
- 20. Sandiumenge A, Lisboa T, Gomez F, Hernandez P, Canadell L, Rello J. Effect of antibiotic diversity on ventilator-associated pneumonia caused by ESKAPE organisms. Chest. 2011. pmid:21659436
- 21. Gruson D, Hilbert G, Vargas F, Valentino R, Bebear C, Allery A, et al. Rotation and restricted use of antibiotics in a medical intensive care unit: Impact on the incidence of ventilator-associated pneumonia caused by antibiotic-resistant gram-negative bacteria. Am J Respir Crit Care Med. 2000. pmid:10988092
- 22. Damas P, Canivet JL, Ledoux D, Monchi M, Melin P, Nys M, et al. Selection of resistance during sequential use of preferential antibiotic classes. Intensive Care Med. 2006. pmid:16308683
- 23. Evans HL, Milburn ML, Hughes MG, Smith RL, Chong TW, Raymond DP, et al. Nature of gram-negative rod antibiotic resistance during antibiotic rotation. Surg Infect (Larchmt). 2005. pmid:16128629
- 24. Sandiumenge A, Diaz E, Rodriguez A, Vidaur L, Canadell L, Olona M, et al. Impact of diversity of antibiotic use on the development of antimicrobial resistance. J Antimicrob Chemother. 2006. pmid:16565158
- 25. Warren DK, Hill HA, Merz LR, Kollef MH, Hayden MK, Fraser VJ, et al. Cycling empirical antimicrobial agents to prevent emergence of antimicrobial-resistant Gram-negative bacteria among intensive care unit patients. Crit Care Med. 2004. pmid:15599150
- 26. Nijssen S, Fluit A, Van De Vijver D, Top J, Willems R, Bonten MJM. Effects of reducing beta-lactam antibiotic pressure on intestinal colonization of antibiotic-resistant gram-negative bacteria. Intensive Care Med. 2010. pmid:19921150
- 27. van Duijn PJ, Bonten MJM. Antibiotic rotation strategies to reduce antimicrobial resistance in Gram-negative bacteria in European intensive care units: Study protocol for a cluster-randomized crossover controlled trial. Trials. 2014. pmid:25011604
- 28. Harbarth S, Harris AD, Carmeli Y, Samore MH. Parallel Analysis of Individual and Aggregated Data on Antibiotic Exposure and Resistance in Gram-Negative Bacilli. Clin Infect Dis. 2001. pmid:11588690
- 29. van Duijn PJ, Verbrugghe W, Jorens PG, Spöhr F, Schedler D, Deja M, et al. The effects of antibiotic cycling and mixing on antibiotic resistance in intensive care units: a cluster-randomised crossover trial. Lancet Infect Dis. 2018. pmid:29396000
- 30. R Core Team (2018). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/.
- 31. Breslow MJ, Badawi O. Severity scoring in the critically ill: Part 1—Interpretation and accuracy of outcome prediction scoring systems. Chest. 2012. pmid:22215834
- 32. Blyth CR. On simpson’s paradox and the sure-thing principle. J Am Stat Assoc. 1972.
- 33. van Duijn PJ, Bonten M. It’s too soon to pull the plug on antibiotic cycling–Authors’ reply. Lancet Infect Dis. 2018;18.
- 34. Nichol D, Rutter J, Bryant C, Hujer AM, Lek S, Adams MD, et al. Antibiotic collateral sensitivity is contingent on the repeatability of evolution. Nat Commun. 2019. pmid:30659188