Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Benefits and unintended consequences of antimicrobial de-escalation: Implications for stewardship programs

  • Josie Hughes ,

    Contributed equally to this work with: Josie Hughes, Xi Huo

    Affiliation Centre for Disease Modelling, York University, Toronto, Ontario, Canada

    ORCID http://orcid.org/0000-0001-7875-9015

  • Xi Huo ,

    Contributed equally to this work with: Josie Hughes, Xi Huo

    Affiliations Centre for Disease Modelling, York University, Toronto, Ontario, Canada, Department of Mathematics, Ryerson University, Toronto, Ontario, Canada

  • Lindsey Falk,

    Current address: Health Quality Ontario, Toronto, Ontario, Canada

    Affiliation Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada

    ORCID http://orcid.org/0000-0002-9671-4045

  • Amy Hurford,

    Affiliation Department of Biology and Department of Mathematics and Statistics, Memorial University of Newfoundland, St. John’s, Newfoundland, Canada

  • Kunquan Lan,

    Affiliation Department of Mathematics, Ryerson University, Toronto, Ontario, Canada

  • Bryan Coburn,

    Affiliations Department of Laboratory Medicine & Pathobiology, University of Toronto, Toronto, Ontario, Canada, Department of Medicine, Sinai Health System & University Health Network, Toronto, Ontario, Canada, Department of Medicine, University of Toronto, Toronto, Ontario, Canada

  • Andrew Morris ,

    Andrew.Morris@sinaihealthsystem.ca (AM); wujh@yorku.ca (JW)

    Affiliations Department of Medicine, Sinai Health System & University Health Network, Toronto, Ontario, Canada, Department of Medicine, University of Toronto, Toronto, Ontario, Canada

  • Jianhong Wu

    Andrew.Morris@sinaihealthsystem.ca (AM); wujh@yorku.ca (JW)

    Affiliation Centre for Disease Modelling, York University, Toronto, Ontario, Canada

Benefits and unintended consequences of antimicrobial de-escalation: Implications for stewardship programs

  • Josie Hughes, 
  • Xi Huo, 
  • Lindsey Falk, 
  • Amy Hurford, 
  • Kunquan Lan, 
  • Bryan Coburn, 
  • Andrew Morris, 
  • Jianhong Wu
PLOS
x

Abstract

Sequential antimicrobial de-escalation aims to minimize resistance to high-value broad-spectrum empiric antimicrobials by switching to alternative drugs when testing confirms susceptibility. Though widely practiced, the effects de-escalation are not well understood. Definitions of interventions and outcomes differ among studies. We use mathematical models of the transmission and evolution of Pseudomonas aeruginosa in an intensive care unit to assess the effect of de-escalation on a broad range of outcomes, and clarify expectations. In these models, de-escalation reduces the use of high-value drugs and preserves the effectiveness of empiric therapy, while also selecting for multidrug-resistant strains and leaving patients vulnerable to colonization and superinfection. The net effect of de-escalation in our models is to increase infection prevalence while also increasing the probability of effective treatment. Changes in mortality are small, and can be either positive or negative. The clinical significance of small changes in outcomes such as infection prevalence and death may exceed more easily detectable changes in drug use and resistance. Integrating harms and benefits into ranked outcomes for each patient may provide a way forward in the analysis of these tradeoffs. Our models provide a conceptual framework for the collection and interpretation of evidence needed to inform antimicrobial stewardship.

Introduction

Strategies that reduce the costs associated with antimicrobial use while maintaining the benefits are urgently needed. Antimicrobial de-escalation aims to reduce costs by switching from antimicrobials that provide good empiric coverage to alternatives based on laboratory susceptibility results, stopping unecessary or redundant treatment, and switching from IV to oral therapy [1, 2]. Here, we consider the implications of sequential switching from empiric to alternative drugs, hereafter referred to as de-escalation. Though commonly practiced, the effects of de-escalation are not well understood [1, 3, 4]. De-escalation is associated with lower mortality in observational studies, but this relationship may be confounded by other determinants of improvement [3]. Non-inferiority trials have not found a change in mortality [5, 6], and have not been designed to assess effects of de-escalation on resistance [3]. Unintended increases in the emergence of multidrug-resistant (MDR) organisms [6] and superinfections [5] have been observed following antimicrobial de-escalation. Definitions of interventions and outcomes differ among studies, making comparison difficult [3].

To inform the design of future studies we develop a mathematical model of the ecological and evolutionary consequences of antimicrobial de-escalation in an intensive care unit (ICU). Existing models have informed our understanding of other stewardship and infection control interventions [714] but this is the first theoretical examination of de-escalation. We initially focus on the evolution and transmission of Pseudomonas aeruginosa in the ICU, then consider the wider implications of our results. In general, de-escalation differs among pathogens, infection types, patient groups, and treatment contexts, and the details matter. P. aeruginosa is primarily a nosocomial pathogen [15], and is particularly adept at developing resistance [1618]. Thus, the potential benefits of antimicrobial stewardship are high for this organism.

Materials and methods

Modeling the effects of antimicrobial de-escalation

We model the effects of antimicrobial de-escalation on P. aeruginosa transmission, infection, and evolutionary dynamics in an intensive care unit, and consequences for patients. Specifically, we consider the effects of three therapy adjustment strategies on the evolution of P. aeruginosa resistance to piperacillin-tazobactam, which is often used for empiric therapy of severe infections in the ICU because it provides good coverage of common pathogens [1921], and ciprofloxacin which is often used for definitive treatment of susceptible P. aeruginosa infections [2227]. Ciprofloxacin is the preferred alternative because it provides poorer empiric coverage of common Gram-positive pathogens such as Staphylococcus aureus [21], and can be given orally. We focus on the effects of drug switching, and do not consider drug combinations.

Infected patients are treated empirically with piperacillin-tazobactam until testing confirms the identity and susceptibilities of the infecting pathogen (row 1 of Fig 1). In the non-pseudomonal de-escalation scenario, patients confirmed to have a non-pseudomonal infection are switched to a non-pseudomonal drug, while patients with susceptible P. aeruginosa infections are switched to ciprofloxacin (row 3 of Fig 1). In the continuation scenario, alternative drugs are only given when testing confirms resistance to piperacillin-tazobactam (row 2 of Fig 1). In the ciprofloxacin de-escalation scenario, ciprofloxacin is used to treat all confirmed susceptible infections (row 4 row of Fig 1). The ciprofloxacin scenario is unrealistic, but provides insight into model behaviour. Infections resistant to both piperacillin-tazobactam and ciprofloxacin are treated with a last-resort drug such as a carbapenem or aminoglycoside (5th column of Fig 1).

thumbnail
Fig 1. Direct effects of antimicrobial de-escalation.

Infections can be caused by resistant or susceptible P. aeruginosa strains, or non-pseudomonal pathogens (circles and squares inside people, columns). Strains that are treated (pills inside people) effectively can be replaced by resistant strains (circles and squares outside people). All infections are treated empirically with piperacillin-tazobactam (row 1). In the continuation scenario, susceptible patients are also treated definitively with piperacillin-tazobactam (row 2). In the non-pseudomonal de-escalation scenario, patients with non-pseudomonal infections are treated definitively with non-pseudomonal drugs, while susceptible P. aeruginosa infections are treated with ciprofloxacin (row 3). In the ciprofloxacin scenario, both non-pseudomonal and susceptible P. aeruginosa infections are treated definitively with ciprofloxacin (row 4).

https://doi.org/10.1371/journal.pone.0171218.g001

Both de-escalation strategies (rows 3 and 4 of Fig 1) reduce the use of piperacillin-tazobactam. This should reduce selection for resistance to piperacillin-tazobactam, thus preserving the effectiveness of empiric therapy. The use of ciprofloxacin and alternative drugs will increase with de-escalation, but resistance to definitive drugs is less problematic than resistance to empiric therapy because definitive therapy is informed by laboratory susceptibility testing. De-escalation also leaves patients vulnerable to colonization and subsequent infection by P. aeruginosa strains that are susceptible to piperacillin-tazobactam but unaffected by ciprofloxacin (rows 3 and 4 of Fig 1) or non-pseudomonal alternatives (row 3 of Fig 1). We use mathematical models to better understand the implications of these tradeoffs.

Mathematical models always represent a compromise between realism and simplicity. To minimize complexity, we focus on the evolution of P. aeruginosa resistance to piperacillin-tazobactam and ciprofloxacin, ignoring resistance to alternative non-pseudomonal and last-resort drugs. Thus, we have not included all the possible costs of antimicrobial de-escalation. We have omitted MDR resistance mechanisms such as efflux pumps [2832], so the models represent a best-case scenario for the preservation of empiric therapy. Finally, we have omitted Clostridium difficile infection dynamics [3336]. Thus, our models are sufficiently complex to demonstrate some possible tradeoffs, but not to determine whether, and in what contexts, de-escalation is good policy.

Patient categorization and notation.

Patients are first categorized by their infection status: (U) uninfected; (E) infected, receiving empiric therapy, and expecting an accurate diagnosis; (F) infected, receiving empiric therapy, and expecting an inaccurate diagnosis because they have developed resistance during therapy; (D) infected and receiving definitive therapy; or (S) superinfected by P. aeruginosa. Infections can be caused by P. aeruginosa or other pathogens that cause similar symptoms (Fig 1).

A subscript denotes the P.aeruginosa strain carried by a patient, if any (Fig 1): (X) is a non-pseudomonal pathogen, (1) is a P.aeruginosa strain resistant to ciprofloxacin, (2) is strain resistant to piperacillin-tazobactam, (12) is a strain resistant to both drugs, and (0) is a strain susceptible to both drugs. Patients initially infected by species X and subsequently colonized by P.aeruginosa can develop superinfections; we mark these patients as (Xi) with i = 0, 1, 2, 12 referring to different P.aeruginosa strains (Fig 2).

thumbnail
Fig 2. Simplified flow diagram of non-pseudomonal de-escalation.

Subscripts denote the infecting or colonizing strain and superscripts denote the treatment status. See “Patient categorization and notation” for details. Orange text indicates inadequate treatment. Patients experiencing strain conversions during empiric therapy, discharge processes and death processes are not included in the simplified flow diagram. See S1S3 Figs for complete dynamics.

https://doi.org/10.1371/journal.pone.0171218.g002

Finally, the treatment status of patients is noted in a superscript (Fig 1): (1) ciprofloxacin indicates ciprofloxacin treatment, (2), indicates pipercillin-tazobactam treatment, (L) indicates treatment with an alternative drug for non-pseudomonal infections, and (3) indicates treatment for dual-resistant infections, (A) indicates prior drug exposure, and (N) indicates no prior drug exposure.

Fig 2 summarizes model dynamics in non-pseudomonal de-escalation scenario. Complete descriptions of the three treatment scenarios, with equations, can be found in S1 Appendix. Model parameters are summarized in Table 1. We assume large ranges for uncertain parameters, giving confidence that the full range of possible outcomes has been explored at the cost of oversampling unlikely outcomes.

Colonization, infection, and the emergence of resistance.

We distinguish infected patients and patients that are colonized by P. aeruginosa but do not have symptoms of a P. aeruginosa infection [9, 13]. Patients remain P. aeruginosa carriers for 12-25 days [43], which is longer than the normal stay in the ICU [5, 4446], so we assume that patients remain colonized for the duration of their stay [43].

Patients with a history of antimicrobial exposure are at higher risk of P. aeruginosa colonization [15, 47, 48]. For simplicity, we assume only exposed patients are colonized [13]. Patients with previous antimicrobial exposure can be colonized by resistant or susceptible P. aeruginosa strains, and patients receiving treatment can only be colonized by a strain that is resistant to that treatment. Transmission of P.aeruginosa strains among exposed patients is modeled by mass action with a transmission rate β [7, 49], without distinguishing among trasmission pathways [47, 50] or bacterial loads [51]. We examine sensitivity to a broad range of β (0.01 to 1 per day).

Colonized patients develop P. aeruginosa infections at a rate σc ∈ [0.05, 0.14] [13, 15]. Uncolonized patients develop infections with similar symptoms caused by other pathogens at a lower rate σx ∈ [0.013, 0.0203] [37]. See S1 Appendix for σc and σx estimation methods. Patients that become colonized while being treated for a non-pseudomonal infection develop P. aeruginosa superinfections with a probability η, which varies broadly between 0 and 100%.

The models allow resistant strains to replace dominant sensitive strains given antimicrobial selection pressure [7]. Resistance to ciprofloxacin and piperacillin can emerge within a patient at rates ϵ1 and ϵ2. We assume a broad range of emergence rates (0 to 3% per day).

Treatment and recovery.

On average, we assume phenotypic resistance testing takes 3 days (τ = 1/3) [5], failed treatment is corrected after 3 to 5 days (τ1 ∈ [1/5, 1/3]), and 7 days of treatment is sufficient to cure infection (τ2 = 1/4). Superinfection delays recovery by 0 to 11 days (τ3 ∈ [1/4, 1/15]). Pandrug-resistance is not modelled, so we penalize dual resistance by assuming infected patients recover more slowly at a rate ϑτ2, where ϑ is a hazard ratio that varies broadly between 0 and 1.

Death, discharge and admission.

Uninfected patients die at rate ν ∈ [0.005, 0.05] per day and are discharged at rate μ ∈ [0.025, 0.5] per day [52]. Infected patients recieving adequate treatment die at rate κνν, and are discharged at rate κμμ. In general, nosocomial infections are associated with higher death rates and longer durations of stay, but the association varies among infection types (κν ∈ [1, 2.3], κμ ∈ [0.49, 1]) [39, 40, 53]. The impact of inadequate empiric therapy on the probability of dying (δ) within ten days varies between 4 and 40% [41, 42]. Given δ and other rates we calculate the hazard ratio of death for inadequately treated patients κδ (S1 Appendix).

60% of admitted patients have prior exposure to antimicrobials (a = 0.6), and 0 to 10% of these are colonized with P. aeruginosa (m ∈ [0, 0.1]) [13]. Up to 70% of incoming P. aeruginosa strains are resistant to ciprofloxacin (r1 ∈ [0, 0.7]), and fewer are resistant to piperacillin-tazobactam (r2/r1 ∈ [0, 1]), while no MDR strains are admitted for simplicity.

The number of patients in the ICU denoted by N remains constant over time because that is generally the case in our ICU. To maintain a constant population we model the admission rate as a time-dependent variable λ(t) which is equal to the sum of death and discharge rates at time t.

Outcomes and analysis

Nearly all relevant studies measure the impact of de-escalation on all-cause mortality [3], so we report probability of death (pDeath), measured as the percentage of patients who die in the ICU over a 100 day period. However, effects of resistance and stewardship interventions on mortality can be small and difficult to detect [3, 5, 6, 54, 55]. Infection prevalence (pInfected) and the percentage of treatment that is ineffective (pMismatch) may be more sensitive measures of the effect of de-escalation on patients.

We include several additional outcome measures to yield insight into the causes of changes in death, infection prevalence and treatment failure. A primary goal of de-escalation is to preserve empiric therapy, so we measure the percentage of empiric therapy that covers the initial infection (pEmpiric). There are concerns about emergence of superinfections and MDR bacteria and superinfections [3], so we measure the percentage of patients that have non-pseudomonal infections and P. aeruginosa superinfections (pSuperinf), and the prevalence of resistant to both pseudomonal drugs (pRBoth). We also measure P. aeruginosa colonization prevalence (pColonized).

Antimicrobial resistance is often measured as the percentage of resistant first clinical isolates [56], but we measure prevalence (pRCipro, pRPipTazo, and pRBoth) because it more accurately reflects the burden of resistance [57]. Drug use studies are common [54, 58], so we measure the use of ciprofloxacin (UseCipro), piperacillin-tazobactam (UsePipTazo), and drugs used to treat dual-resistant P. aeruginosa infections (UseAlt) as the percentage of patients receiving each drug. See S1 Appendix for mathematical definitions of outcomes.

To assess sensitivity we use Jansen’s estimator of the contribution of each parameter to outcome variance [59, 60]. Bootstrapped 95% confidence bounds for the total sensitivity indices indicated that a base random sample of 5000 and a total of 95,000 parameter combinations are adequate. Sampling was done using quasi-random numbers and a radial design as recommended by Saltelli et al. [59]. Parameters are assumed to be uniformly distributed over the proposed intervals because we lack information about the distributions (Table 1). This sampling scheme gives confidence that the full range of possible outcomes has been explored, at the cost of oversampling unlikely outcomes.

Two parameter sets were randomly generated for the non-pseudomonal and ciprofloxacin scenarios. For each combination of parameters, the effect of non-pseudomonal de-escalation is the difference between outcomes of the non-pseudomonal de-escalation and continuation scenarios. The ciprofloxacin de-escalation scenario is similarly compared to the continuation scenario. Outcomes are averaged over the final 100 days of a 2000-day simulation. For each combination of parameter values we numerically confirmed that the model reached a steady state.

To understand when de-escalation increases or decreases the probability of death we also analyzed a Latin-Hypercube sample of 100,000 parameter combinations using classification and regression trees [6164]. We require a minimum of 2% of cases at a classification tree node in order to attempt a split, and pruned according to the “1-SE” rule [61]. Again, uniform distributions are assumed, and two parameter sets were randomly generated.

In our three academic ICUs over the past 14 years P. aeruginosa resistance to ciprofloxacin (oRCipro) and piperacillin-tazobactam (oRPipTazo) has not exceeded 70% and 50%, respectively. P. aeruginosa colonization prevalence ranges from 6% to 32% [65]. Outcomes outside of these ranges are considered unrealistic, and excluded from results unless otherwise specified. 34798/100000 and 33152/100000 parameter combinations meet the calibration criteria in the de-escalation and ciprofloxacin scenarios, respectively.

Results

In our models de-escalation preserves the effectiveness of empiric therapy (pEmpiric in Fig 3) by decreasing the use of and resistance to piperacillin-tazobactam (UsePipTazo and pRPipTazo in Fig 3), at the cost of greater ciprofloxacin use and resistance (UseCipro and pRCipro in Fig 3). For most parameter combinations, this leads to an overall reduction in the percentage of patients that receive inadequate treatment (pMismatch in Fig 3).

thumbnail
Fig 3. Benefits and unintended consequences of antimicrobial de-escalation.

Pale green indicates a benefit of de-escalation, and dark red indicates an unintended consequence. Numbers are the percentage of cases in which de-escalation is beneficial or detrimental. Continuation outcomes (c) are shown to put observed differences in context. pDeath is the probability (%) of death, pInfected is the infection prevalence, pMismatch is the % of inadequate treatment, and pEmpiric is the % of effective empiric therapy. pColonized and pSuperinf are P. aeruginosa colonization and superinfection prevalence, respectively. pRCipro/PipTazo/Both is the prevalence of resistance, and UseCipro/PipTazo/Alt is drug use.

https://doi.org/10.1371/journal.pone.0171218.g003

In the non-pseudomonal de-escalation scenario, the benefits of de-escalation are further offset by increases in P. aeruginosa colonization and superinfection (pColonized and pSuperinf in Fig 3a), which in turn leads to an increase the percentage of patients suffering from an infection (pInfected in Fig 3a). Effects of de-escalation on the probability of death are small (<1%), and can be either positive or negative depending on the parameter values (pDeath in Fig 3a). In sum, de-escalation increases the probability of infection while also increasing the effectiveness of treatment (pInfected and pMismatch in Fig 3a). This tradeoff is robust to parameter uncertainty, as it occurs in nearly all cases, over a very broad range of parameter values.

In the ciprofloxacin scenario de-escalated patients remain partially protected against pseudomonal infections, so de-escalation does not consistently lead to more colonization or superinfection (pColonized and pSuperinf in Fig 3b). Instead, the benefits of de-escalation are offset by increases in dual resistance and the use of alternative pseudomonal drugs (pRBoth and UseAlt in Fig 3b). Again, these tradeoffs are robust to parameter uncertainty. For 80% of parameter combinations de-escalation still increases the percentage of infected patients (pInfected in Fig 3b). Effects of de-escalation on the probability of death remain small (<1.2%), but are more consistently beneficial (pDeath in Fig 3b).

For many of our outcomes, de-escalation is consistently beneficial or harmful, regardless of the parameter values. The probability of death is a notable exception. De-escalation is most likely to have a substantial effect on death when discharge rate is low, transmission rate is moderate, and the impact of empiric therapy is high (Fig 4). However, other model parameters determine whether this effect is positive or negative (Fig 5).

thumbnail
Fig 4. Influence of important parameters on the effect of de-escalation on the probability of death.

Parameter importance was determined by sensitivity analysis (S5 Fig). Parameter values have been discretized to show variation in the distribution of outcomes—each point includes 10% of parameter combinations. Boxes show the median and interquartile range (IQR), whiskers include values within 150% of the IQR, and dashes show outliers.

https://doi.org/10.1371/journal.pone.0171218.g004

thumbnail
Fig 5. Classification trees distinguishing whether de-escalation decreases (blue) or increases (green) the probability of death.

The first number at each node is the probability that de-escalation decreases the probability of death, followed by the percentage of simulations included at that node. 10-fold cross validation classification error was 25% and 11% in the (a) non-pseudomonal and (b) ciprofloxacin scenarios, respectively. Important parameters include transmission rate (β), discharge rate (μ), superinfection rate (η), the rate of emergence of resistance to piperacillin-tazobactam (ϵ2), incoming resistance rates (r1 and r2), and death rate (ν).

https://doi.org/10.1371/journal.pone.0171218.g005

In the non-pseudomonal scenario, determinants of whether de-escalation decreases or increases the probability of death include transmission rate (β), discharge rate (μ), superinfection rate (η), the rate of emergence of resistance to piperacillin-tazobactam (ϵ2), incoming resistance rates (r1 and r2), and death rate (ν) (Fig 5a and S6a Fig). If transmission rate (β) is high and discharge rate (μ) is low then de-escalation is likely to be beneficial (right-most branch of Fig 5a). Conversely, de-escalation is likely to be detrimental when transmission rate (β) is low, discharge rate (μ) is high, and superinfection rate (η) is high (left-most branch of Fig 5a). Otherwise, a complex interaction among parameters determines whether de-escalation is good or bad (Fig 5a).

In the ciprofloxacin scenario the superinfection rate (η) is not an important parameter and model behaviour is simpler (Fig 5b and S6b Fig). De-escalation is likely to decrease the probability of death when relatively many patients are admitted colonized by piperacillin-tazobactam resistant P. aeruginosa (high r2/r1) (right-most branch of S6b Fig), or when admission of resistance (r1) is low and emergence of piperacillin-tazobactam resistance (ϵ2) is high (S6b Fig).

Discussion

We introduce novel population-level compartmental models for antimicrobial de-escalation in the ICU that distinguish between asymptomatic carriers and symptomatic patients, allow adjustment of empiric therapy, and allow the evolution of resistance to empiric and alternative drugs. A few models have allowed for adjustment of failed therapy [912], but none de-escalated successful empiric therapy. Our models provide insight into the potential tradeoffs associated with antimicrobial de-escalation, with implications for the design and interpretation of clinical studies.

In our models, de-escalation reduces resistance to piperacillin-tazobactam and preserves empiric therapy, as intended [1, 2, 66], at the cost of increasing resistance to ciprofloxacin. However, de-escalation to non-pseudomonal drugs also leaves patients vulnerable to P. aeruginosa colonization and superinfection, while de-escalation to ciprofloxacin selects for MDR. Overall, non-pseudomonal de-escalation increases the probability of adequate treatment while also increasing the probability of infection. These tradeoffs occur in nearly all the cases we investigated, across a broad range of parameter values.

Effects of de-escalation on mortality are small (<1.2%) in our models, consistent with observations [3, 5, 6]. In part this can be explained by dilution. If adequate empiric therapy can reduce 10-day mortality by up to 40%, and 17% more patients receive adequate empiric therapy, we expect a maximum impact of 6.8% which de-escalation trials have not so far been powered to detect [5, 6] (S1 Appendix). In our model, unintended consequences of de-escalation further temper the benefits of preserving empiric therapy; a 17% improvement in empiric coverage does not lead to a 6.8% reduction in the probability of death.

Increasing the probability of infection is bad for patients, while increasing the probability of adequate treatment is good for patients. To evaluate de-escalation policies it will be necessary to weigh these benefits and unintended consequences against one another. The Desirability of Outcome Ranking (DOOR) and Response Adjusted for Duration of Antibiotic Risk (RADAR) method was designed to handle competing risks, increase statistical power, and integrate harms and benefits at the patient level [58]. Integrating harms and benefits into a ranked outcome for each patient may provide a way forward in the analysis of de-escalation tradeoffs.

To date, there has been little consistency and some notable gaps in the selection of outcomes in de-escalation trials and observational studies [3, 4]. As an example, Leone et al. [5] did not measure the effectiveness of empiric therapy or MDR emergence, while Kim et al. [6] did not measure superinfections or C. difficile risk. No studies have adequately assessed impacts on resistance or the effectiveness of empiric therapy [3]. Proxy measures such as drug use and resistance are common in the antimicrobial stewardship literature [54, 58], but small changes in important clinical outcomes can outweigh more easily detected changes in proxy measures. We hope this study increases awareness of possible tradeoffs, and motivates more careful selection and comparison of outcomes.

In our models, the effects of de-escalation on mortality can be beneficial or detrimental. De-escalation is most likely to have a substantial effect when discharge rate is low, transmission rate is moderate, and the impact of inadequate empiric therapy is high. Superinfection rate, the rate of emergence of piperacillin-tazobactam resistance and the admission of resistant P. aeruginosa strains are also important. Intuitively, it makes sense that de-escalation has little effect when patient turnover is very high, when transmission is rare, when empiric therapy has little impact, or when there is little risk of resistance to empiric therapy.

Tradeoffs and parameter dependence in our results lead to a general expectation that effects of de-escalation will vary among species, contexts and patient groups. For example, most patients remain P. aeruginosa carriers for only 12-25 days [43], and the chance of re-infection is low. These patients may not suffer from MDR carriage after their initial infection is identified and appropriately treated. However, an important minority of immune-compromised and cystic fibrosis patients are persistent carriers of P. aeruginosa at high risk of reinfection, and the difficulties associated with MDR carriage are likely much higher for this group [31]. Future work might consider covering these patients with broad-spectrum antimicrobials in the high-risk ICU environment, while de-escalating lower risk patients.

Important drawbacks of ciprofloxacin de-escalation are not included in our single-species models. Ciprofloxacin is used for P. aeruginosa infections [5, 2227], and it may prevent P. aeruginosa colonization [50]. However it can also cause complications [67] and select for resistant strains [2830, 50]. De-escalating to a single drug would reduce prescribing heterogeneity and promote resistance among other pathogens. Ciprofloxacin and piperacillin-tazobactam are both associated with a high risk of Clostridium difficile infection [3336]. De-escalation to low risk alternatives [3336] may reduce C. difficile infections in patients without pseudomonal infections. Accounting for C. difficile risk would require a more complex model, and could tilt the balance in favour of non-pseudomonal de-escalation. The ciprofloxacin scenario clarifies model behaviour, but we do not recommend it.

The evolution of resistance is more complicated than our models. We assumed class-specific resistance mechanisms (e.g. beta-lactamases or DNA gyrase mutations) but MDR mechanisms (e.g. efflux pumps) are also common in patients with substantial antimicrobial exposure [2832]. Including these mechanisms would reduce the impact of de-escalation on the effectiveness of empiric therapy, and thus reduce the benefits of de-escalation. Empiric therapy can include alternatives to piperacillin-tazobactam and multiple drugs [19, 20]. Stress, variation in treatment timing, drug combinations, and treatment history can alter the rate of emergence of resistance [51, 6874]. P. aeruginosa persists in environmental reservoirs, and transmission dynamics are not well understood [47, 50].

It would not be difficult to build, on the basis of our compartment models, a complex individual-based model [8] that includes more drugs, more species, more infection types, more variation among patients, more transmission pathways, and more detailed evolutionary and in-host dynamics. The primary obstacle to such a model is lack of reliable parameter estimates. Even so, an individual-based model might yield some insight into C. difficile risk and treatment of vulnerable patients. It might also help guide the design and interpretation of studies of patient-level outcomes [58]. Our compartment models could easily be adapted for other hospital-acquired pathogens such as Acinetobacter baumannii. Consideration of dynamics outside the ICU will be required for species such as E. coli that belong to the normal human flora [75]. More complexity and realism is unlikely to alter our most important results; we expect both benefits and unintended consequences from antimicrobial de-escalation.

Supporting information

S1 Appendix. Model assumptions, equations, outcome measurements, and sample size calculation.

https://doi.org/10.1371/journal.pone.0171218.s001

(PDF)

S1 Fig. Flow diagram of non-pseudomonal de-escalation.

Compartments of patients under inadequate drug treatment are colored in orange. Discharge and death processes are not included in the diagram. See S1 Appendix for notation and equations.

https://doi.org/10.1371/journal.pone.0171218.s002

(TIF)

S2 Fig. Flow diagram of the continuation scenario.

Compartments of patients under inadequate drug treatment are colored in orange. Discharge and death processes are not included in the diagram. See S1 Appendix for notation and equations.

https://doi.org/10.1371/journal.pone.0171218.s003

(TIF)

S3 Fig. Flow diagram of ciprofloxacin de-escalation.

Compartments of patients under inadequate drug treatment are colored in orange. Discharge and death processes are not included in the diagram. See S1 Appendix for notation and equations.

https://doi.org/10.1371/journal.pone.0171218.s004

(TIF)

S4 Fig. Spearman’s rank correlations among effects of de-escalation in the (a) non-pseudomonal scenario and the (b) ciprofloxacin scenario.

https://doi.org/10.1371/journal.pone.0171218.s005

(TIF)

S5 Fig. Total sensitivity indices [59] for the effect of de-escalation on the probability of death in the (a) non-pseudomonal and (b) ciprofloxacin scenarios.

Results include parameter values that do not meet the calibration criteria.

https://doi.org/10.1371/journal.pone.0171218.s006

(TIF)

S6 Fig. Random Forest importance of each model parameter as a determinant of whether de-escalation decreases or increases the probability of death in the (a) non-pseudomonal and (b) ciprofloxacin scenario.

500 trees gave OOB classification error rates of 13% and 7.5% for the non-pseudomonal and ciprofloxacin scenarios, respectively.

https://doi.org/10.1371/journal.pone.0171218.s007

(TIF)

Acknowledgments

We thank members of the SHS-UHN Antimicrobial Stewardship Team and the Laboratory for Industrial and Applied Mathematics at York University for help translating medicine into math and vice versa. We thank Samuel Alizon, Christophe Fraser, and three anonymous reviewers for constructive criticism of an earlier version of this manuscript.

Author Contributions

  1. Conceptualization: JH XH JW AM AH BC KL.
  2. Data curation: JH XH.
  3. Formal analysis: JH XH.
  4. Funding acquisition: JW AM AH.
  5. Investigation: JH XH LF.
  6. Methodology: JH XH LF.
  7. Project administration: JW AM.
  8. Software: JH XH.
  9. Supervision: JW AM.
  10. Validation: JH XH.
  11. Visualization: JH XH.
  12. Writing – original draft: JH XH LF.
  13. Writing – review & editing: JH XH JW AM AH BC KL LF.

References

  1. 1. Timsit JF, Harbarth S, Carlet J. De-escalation as a potential way of reducing antibiotic use and antimicrobial resistance in ICU. Intensive Care Med. 2014;40(10):1580–1582. pmid:25227655
  2. 2. Dellit TH, Owens RC, McGowan JE, Gerding DN, Weinstein RA, Burke JP, et al. Infectious Diseases Society of America and the Society for Healthcare Epidemiology of America guidelines for developing an institutional program to enhance antimicrobial stewardship. Clin Infect Dis. 2007;44(2):159–177. pmid:17173212
  3. 3. Tabah A, Cotta M, Garnacho-Montero J, Roborts J, Lipman J, Tacey M, et al. A systematic review of the definitions, determinants and clinical outcomes of antimicrobial de-escalation in the intensive care unit. Clin Infect Dis. 2016;62(8):1009–1017. pmid:26703860
  4. 4. Silva BN, Andriolo RB, Atallah AN, Salomao R. De-escalation of antimicrobial treatment for adults with sepsis, severe sepsis or septic shock. Cochrane Database Syst Rev. 2013;3:CD007934. pmid:23543557
  5. 5. Leone M, Bechis C, Baumstarck K, Lefrant JY, Albanese J, Jaber S, et al. De-escalation versus continuation of empirical antimicrobial treatment in severe sepsis: a multicenter non-blinded randomized noninferiority trial. Intensive Care Med. 2014;40(10):1399–1408. pmid:25091790
  6. 6. Kim JW, Chung J, Choi SH, Jang HJ, Hong SB, Lim CM, et al. Early use of imipenem/cilastatin and vancomycin followed by de-escalation versus conventional antimicrobials without de-escalation for patients with hospital-acquired pneumonia in a medical ICU: a randomized clinical trial. Crit Care. 2012;16(1):R28. pmid:22336530
  7. 7. Spicknall IH, Foxman B, Marrs CF, Eisenberg JNS. A modeling framework for the evolution and spread of antibiotic resistance: Literature review and model categorization. AJE. 2013;178(4):508–520. pmid:23660797
  8. 8. van Kleef E, Robotham JV, Jit M, Deeny SR, Edmunds WJ. Modelling the transmission of healthcare associated infections: a systematic review. Bmc Infect Dis. 2013;13. pmid:23809195
  9. 9. zur Wiesch PA, Kouyos R, Abel S, Viechtbauer W, Bonhoeffer S. Cycling empirical antibiotic therapy in hospitals: meta-analysis and models. PLOS Pathog. 2014;10(6). pmid:24968123
  10. 10. Kouyos RD, zur Wiesch PA, Bonhoeffer S. Informed switching strongly decreases the prevalence of antibiotic resistance in hospital wards. Plos Comput Biol. 2011;7(3). pmid:21390265
  11. 11. Haber M, Levin BR, Kramarz P. Antibiotic control of antibiotic resistance in hospitals: a simulation study. Bmc Infect Dis. 2010;10. pmid:20738872
  12. 12. Obolski U, Stein GY, Hadany L. Antibiotic Restriction Might Facilitate the Emergence of Multi-drug Resistance. PLoS Comput Biol. 2015;11(6):e1004340. pmid:26110266
  13. 13. Hurford A, Morris AM, Fisman DN, Wu JH. Linking antimicrobial prescribing to antimicrobial resistance in the ICU: Before and after an antimicrobial stewardship program. Epidemics. 2012;4(4):203–210. pmid:23351372
  14. 14. Smith DL, Levin SA, Laxminarayan R. Strategic interactions in multi-institutional epidemics of antibiotic resistance. Proceedings of the National Academy of Sciences of the United States of America. 2005;102(8):3153–8. pmid:15677330
  15. 15. Boyer A, Doussau A, Thiebault R, Venier AG, Tran V, Boulestreau H, et al. Pseudomonas aeruginosa acquisition on an intensive care unit: relationship between antibiotic selective pressure and patients’ environment. Crit Care. 2011;15(1):R55. pmid:21306623
  16. 16. Poole K. Pseudomonas aeruginosa: resistance to the max. Front Microbiol. 2011;2. pmid:21747788
  17. 17. Morita Y, Tomida J, Kawamura Y. Responses of Pseudomonas aeruginosa to antimicrobials. Front Microbiol. 2014;4. pmid:24409175
  18. 18. Breidenstein EB, de la Fuente-Nunez C, Hancock RE. Pseudomonas aeruginosa: all roads lead to resistance. Trends Microbiol. 2011;19(8):419–426. pmid:21664819
  19. 19. Anonymous. Guidelines for the management of adults with hospital-acquired, ventilator-associated, and healthcare-associated pneumonia. Am J Resp Crit Care Med. 2005;171(4):388–416. pmid:15699079
  20. 20. Mermel LA, Allon M, Bouza E, Craven DE, Flynn P, O’Grady NP, et al. Clinical practice guidelines for the diagnosis and management of intravascular catheter-related infection: 2009 update by the Infectious Diseases Society of America. Clin Infect Dis. 2009;49(1):1–45. pmid:19489710
  21. 21. Hughes JS, Hurford A, Finley RL, Patrick DM, Wu J, Morris AM. How to measure the impacts of antibiotic resistance and antibiotic development on empiric therapy: new composite indices. BMJ Open. 2016;6(12):e012040. pmid:27986734
  22. 22. Braykov NP, Morgan DJ, Schweizer ML, Uslan DZ, Kelesidis T, Weisenberg SA, et al. Assessment of empirical antibiotic therapy optimisation in six hospitals: an observational cohort study. Lancet Infect Dis. 2014;14(12):1220–1227. pmid:25455989
  23. 23. Shime N, Kosaka T, Fujita N. De-escalation of antimicrobial therapy for bacteraemia due to difficult-to-treat Gram-negative bacilli. Infection. 2013;41(1):203–210. pmid:23254646
  24. 24. Khasawneh FA, Karim A, Mahmood T, Ahmed S, Jaffri SF, Tate ME, et al. Antibiotic de-escalation in bacteremic urinary tract infections: potential opportunities and effect on outcome. Infection. 2014;42(5):829–834. pmid:24874607
  25. 25. Kollef MH, Morrow LE, Niederman MS, Leeper KV, Anzueto A, Benz-Scott L, et al. Clinical characteristics and treatment patterns among patients with ventilator-associated pneumonia. Chest. 2006;129(5):1210–1218. pmid:16685011
  26. 26. Kaye KS. Antimicrobial de-escalation strategies in hospitalized patients with pneumonia, intra-abdominal infections, and bacteremia. J Hosp Med. 2012;7:S13–S21. pmid:23677630
  27. 27. Mokart D, Slehofer G, Lambert J, Sannini A, Chow-Chine L, Brun JP, et al. De-escalation of antimicrobial treatment in neutropenic patients with severe sepsis: results from an observational study. Intensive Care Med. 2014;40(1):41–49. pmid:24231857
  28. 28. Wolter DJ, Schmidtke AJ, Hanson ND, Lister PD. Increased expression of ampC in Pseudomonas aeruginosa mutants selected with ciprofloxacin. Antimicrob Agents Chemother. 2007;51(8):2997–3000. pmid:17517839
  29. 29. Rådberg G, Nilsson LE, Svensson S. Development of quinolone-imipenem cross resistance in Pseudomonas aeruginosa during exposure to ciprofloxacin. Antimicrob Agents Chemother. 1990;34(11):2142–7. pmid:2127345
  30. 30. Alyaseen SA, Piper KE, Rouse MS, Steckelberg JM, Patel R. Selection of cross-resistance following exposure of Pseudomonas aeruginosa clinical isolates to ciprofloxacin or cefepime. Antimicrob Agents Chemother. 2005;49(6):2543–5. pmid:15917569
  31. 31. Llanes C, Pourcel C, Richardot C, Plésiat P, Fichant G, Cavallo JD, et al. Diversity of β-lactam resistance mechanisms in cystic fibrosis isolates of Pseudomonas aeruginosa: a French multicentre study. J Antimicrob Chemother. 2013;68(8):1763–71. pmid:23629014
  32. 32. Fournier D, Richardot C, Müller E, Robert-Nicoud M, Llanes C, Plésiat P, et al. Complexity of resistance mechanisms to imipenem in intensive care unit strains of Pseudomonas aeruginosa. J Antimicrob Chemother. 2013;68(8):1772–80. pmid:23587654
  33. 33. Slimings C, Riley TV. Antibiotics and hospital-acquired Clostridium difficile infection: update of systematic review and meta-analysis. J Antimicrob Chemother. 2014;69(4):881–891. pmid:24324224
  34. 34. Owens RC, Donskey CJ, Gaynes RP, Loo VG, Muto CA. Antimicrobial-associated risk factors for Clostridium difficile infection. Clin Infect Dis. 2008;45(Suppl 1):S19–31.
  35. 35. Pépin J, Saheb N, Coulombe MA, Alary ME, Corriveau MP, Authier S, et al. Emergence of fluoroquinolones as the predominant risk factor for Clostridium difficile-associated diarrhea: a cohort study during an epidemic in Quebec. Clin Infect Dis. 2005;41(9):1254–1260. pmid:16206099
  36. 36. Brown KA, Fisman DN, Moineddin R, Daneman N. The magnitude and duration of Clostridium difficile infection risk associated with antibiotic therapy: a hospital cohort study. PLoS One. 2014;9(8):e105454. pmid:25157757
  37. 37. WHO: World Health Organization. Report on the burden of endemic health care-associated infection worldwide: A systematic review of the literature; 2011. Available from: http://apps.who.int/iris/bitstream/10665/80135/1/9789241501507_eng.pdf?ua=1.
  38. 38. Eagye KJ, Nicolau DP, Kuti JL. Impact of superinfection on hospital length of stay and costs in patients with ventilator-associated pneumonia. Semin Respir Crit Care Med. 2009;30(1):116–123. pmid:19199193
  39. 39. Schumacher M, Allignol A, Beyersmann J, Binder N, Wolkewitz M. Hospital-acquired infections–appropriate statistical treatment is urgently needed! Int J Epidemiol. 2013;42(5):1502–1508. pmid:24038717
  40. 40. Melsen WG, Rovers MM, Groenwold RH, Bergmans DC, Camus C, Bauer TT, et al. Attributable mortality of ventilator-associated pneumonia: a meta-analysis of individual patient data from randomised prevention studies. Lancet Infect Dis. 2013;13(8):665–671. pmid:23622939
  41. 41. Heyland DK, Cook DJ, Griffith L, Keenan SP, Brun-Buisson C. The attributable morbidity and mortality of ventilator-associated pneumonia in the critically ill patient. The Canadian Critical Trials Group. Am J Respir Crit Care Med. 1999;159(4 Pt 1):1249–1256. pmid:10194173
  42. 42. Tumbarello M, De Pascale G, Trecarichi EM, Spanu T, Antonicelli F, Maviglia R, et al. Clinical outcomes of Pseudomonas aeruginosa pneumonia in intensive care unit patients. Intensive Care Med. 2013;39(4):682–692. pmid:23370828
  43. 43. REUSSIR: Réseau épidémiologique des utilisateurs du système SIR. Le portage prolongé et la diffusion clonale interhospitalière des Pseudomonas aeruginosa multirésistants de sérotype O12 sont-ils liés? Étude multicentrique. Pathol Biol. 2001;49(8):620–623. pmid:11692749
  44. 44. Giantsou E, Liratzopoulos N, Efraimidou E, Panopoulou M, Alepopoulou E, Kartali-Ktenidou S, et al. De-escalation therapy rates are significantly higher by bronchoalveolar lavage than by tracheal aspirate. Intensive Care Med. 2007;33(9):1533–1540. pmid:17549458
  45. 45. Paskovaty A, Pastores SM, Gedrimaite Z, Kostelecky N, Riedel ER, Seo SK. Antimicrobial de-escalation in septic cancer patients: is it safe to back down? Intensive Care Med. 2015;41(11):2022–2023. pmid:26297120
  46. 46. De Bus L, Denys W, Catteeuw J, Gadeyne B, Vermeulen K, Boelens J, et al. Impact of de-escalation of beta-lactam antibiotics on the emergence of antibiotic resistance in ICU patients: a retrospective observational study. Intensive Care Med. 2016;
  47. 47. Agodi A, Barchitta M, Cipresso R, Giaquinta L, Romeo MA, Denaro C. Pseudomonas aeruginosa carriage, colonization, and infection in ICU patients. Intensive Care Med. 2007;33(7):1155–1161. pmid:17503016
  48. 48. Thuong M, Arvaniti K, Ruimy R, de la Salmoniere P, Scanvic-Hameg A, Lucet JC, et al. Epidemiology of Pseudomonas aeruginosa and risk factors for carriage acquisition in an intensive care unit. J Hosp Infect. 2003;53(4):274–282. pmid:12660124
  49. 49. Hurford A, Lin AL, Wu J. Determinants of the final size and case rate of nosocomial outbreaks. PLoS One. 2015;10(9):2–13.
  50. 50. Cobos-Trigueros N, Sole M, Castro P, Torres JL, Hernandez C, Rinaudo M, et al. Acquisition of Pseudomonas aeruginosa and its resistance phenotypes in critically ill medical patients: role of colonization pressure and antibiotic exposure. Crit Care. 2015;19:218. pmid:25936721
  51. 51. Opatowski L, Guillemot D, Boelle PY, Temime L. Contribution of mathematical modeling to the fight against bacterial antibiotic resistance. Curr Opin Infect Dis. 2011;24(3):279–287. pmid:21467930
  52. 52. Wolkewitz M, Cooper BS, Palomar-Martinez M, Alvarez-Lerma F, Olaechea-Astigarraga P, Barnett AG, et al. Multilevel competing risk models to evaluate the risk of nosocomial infection. Crit Care. 2014;18(2):R64. pmid:24713511
  53. 53. Wolkewitz M, Dettenkofer M, Bertz H, Schumacher M, Huebner J. Statistical epidemic modeling with hospital outbreak data. Stat Med. 2008;27(30):6522–6531. pmid:18759371
  54. 54. Morris AM. Antimicrobial stewardship programs: Appropriate measures and metrics to study their impact. Curr Treat Options Infect Dis. 2014;6(2):101–112. pmid:25999798
  55. 55. Lambert ML, Suetens C, Savey A, Palomar M, Hiesmayr M, Morales I, et al. Clinical outcomes of health-care-associated infections and antimicrobial resistance in patients admitted to European intensive-care units: a cohort study. Lancet Infect Dis. 2011;11(1):30–38. pmid:21126917
  56. 56. Hindler JF, Stelling J. Analysis and presentation of cumulative antibiograms: A new consensus guideline from the clinical and laboratory standards institute. Clin Infect Dis. 2007;44(6):867–873. pmid:17304462
  57. 57. Schwaber MJ, De-Medina T, Carmeli Y. Epidemiological interpretation of antibiotic resistance studies - what are we missing? Nat Rev Microbiol. 2004;2(12):979–983. pmid:15550943
  58. 58. Evans SR, Rubin D, Follmann D, Pennello G, Huskins WC, Powers JH, et al. Desirability of outcome ranking (DOOR) and response adjusted for duration of antibiotic risk (RADAR). Clinical Infectious Diseases. 2015;61(5):800–806. pmid:26113652
  59. 59. Saltelli A, Annoni P, Azzini I, Campolongo F, Ratto M, Tarantola S. Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index. Comput Phys Commun. 2010;181(2):259–270.
  60. 60. Pujol G, Iooss B, with contributions from Sebastien Da Veiga AJ, Fruth J, Gilquin L, Guillaume J, et al. sensitivity: Sensitivity Analysis; 2015. Available from: http://cran.r-project.org/package=sensitivity.
  61. 61. Harper EB, Stella JC, Fremier AK. Global sensitivity analysis for complex ecological models: a case study of riparian cottonwood population dynamics. Ecol Appl. 2011;21(4):1225–1240. pmid:21774426
  62. 62. Liaw A, Wiener M. Classification and Regression by randomForest. R News. 2002;2(3):18–22.
  63. 63. Therneau T, Atkinson B, Ripley B. rpart: Recursive Partitioning and Regression Trees; 2015. Available from: http://cran.r-project.org/package=rpart.
  64. 64. Carnell R. lhs: Latin Hypercube Samples; 2016. Available from: http://cran.r-project.org/package=lhs.
  65. 65. Venier AG, Leroyer C, Slekovec C, Talon D, Bertrand X, Parer S, et al. Risk factors for Pseudomonas aeruginosa acquisition in intensive care units: a prospective multicentre study. J Hosp Infect. 2014;88(2):103–108. pmid:25155240
  66. 66. Garnacho-Montero J, Escoresca-Ortega A, Fernandez-Delgado E. Antibiotic de-escalation in the ICU: how is it best done? Curr Opin Infect Dis. 2015;28(2):193–198. pmid:25692272
  67. 67. FDA: Food and Drug Administration. FDA Drug Safety Communication: FDA advises restricting fluoroquinolone antibiotic use for certain uncomplicated infections; warns about disabling side effects that can occur together; 2016. Available from: http://www.fda.gov/Drugs/DrugSafety/ucm500143.htm.
  68. 68. Gehring R, Schumm P, Youssef M, Scoglio C. A network-based approach for resistance transmission in bacterial populations. J Theor Biol. 2010;262(1):97–106. pmid:19747924
  69. 69. MacLean RC, Hall AR, Perron GG, Buckling A. The population genetics of antibiotic resistance: integrating molecular mechanisms and treatment contexts. Nat Rev Genet. 2010;11(6):405–414. pmid:20479772
  70. 70. Poole K. Stress responses as determinants of antimicrobial resistance in Pseudomonas aeruginosa: multidrug efflux and more. Can J Microbiol. 2014;60(12):783–791. pmid:25388098
  71. 71. Geli P, Laxminarayan R, Dunne M, Smith DL. “One-Size-Fits-All”? Optimizing treatment duration for bacterial infections. PLoS One. 2012;7(1). pmid:22253798
  72. 72. Nichol D, Jeavons P, Fletcher AG, Bonomo RA, Maini PK, Paul JL, et al. Steering evolution with sequential therapy to prevent the emergence of bacterial antibiotic resistance. PLoS Comput Biol. 2015;11(9):e1004493. pmid:26360300
  73. 73. Kouyos RD, Metcalf CJ, Birger R, Klein EY, zur Wiesch P, Ankomah P, et al. The path of least resistance: aggressive or moderate treatment? Proc Biol Sci. 2014;281(1794):20140566. pmid:25253451
  74. 74. Perron GG, Kryazhimskiy S, Rice DP, Buckling A. Multidrug therapy and evolution of antibiotic resistance: when order matters. Appl Environ Microbiol. 2012;78(17):6137–6142. pmid:22729549
  75. 75. Talaminos A, Lopez-Cerero L, Calvillo J, Pascual A, Roa LM, Rodriguez-Bano J. Modelling the epidemiology of Escherichia coli ST131 and the impact of interventions on the community and healthcare centres. Epidemiol Infect. 2016; p. 1–9.