Primary care antimicrobial stewardship interventions can improve antimicrobial prescribing, but there is less evidence that they reduce rates of resistant infection. This study examined changes in broad-spectrum antimicrobial prescribing in the community and resistance in people admitted to hospital with community-associated coliform bacteraemia associated with a primary care stewardship intervention.
Methods and findings
Segmented regression analysis of data on all patients registered with a general practitioner in the National Health Service (NHS) Tayside region in the east of Scotland, UK, from 1 January 2005 to 31 December 2015 was performed, examining associations between a primary care antimicrobial stewardship intervention in 2009 and primary care prescribing of fluoroquinolones, cephalosporins, and co-amoxiclav and resistance to the same three antimicrobials/classes among community-associated coliform bacteraemia. Prescribing outcomes were the rate per 1,000 population prescribed each antimicrobial/class per quarter. Resistance outcomes were proportion of community-associated (first 2 days of hospital admission) coliform (Escherichia coli, Proteus spp., or Klebsiella spp.) bacteraemia among adult (18+ years) patients resistant to each antimicrobial/class. 11.4% of 3,442,205 oral antimicrobial prescriptions dispensed in primary care over the study period were for targeted antimicrobials. There were large, statistically significant reductions in prescribing at 1 year postintervention that were larger by 3 years postintervention when the relative reduction was −68.8% (95% CI −76.3 to −62.1) and the absolute reduction −6.3 (−7.6 to −5.2) people exposed per 1,000 population per quarter for fluoroquinolones; relative −74.0% (−80.3 to −67.9) and absolute reduction −6.1 (−7.2 to −5.2) for cephalosporins; and relative −62.3% (−66.9 to −58.1) and absolute reduction −6.8 (−7.7 to −6.0) for co-amoxiclav, all compared to their prior trends. There were 2,143 eligible bacteraemia episodes involving 2,004 patients over the study period (mean age 73.7 [SD 14.8] years; 51.4% women). There was no increase in community-associated coliform bacteraemia admissions associated with reduced community broad-spectrum antimicrobial use. Resistance to targeted antimicrobials reduced by 3.5 years postintervention compared to prior trends, but this was not statistically significant for co-amoxiclav. Relative and absolute changes were −34.7% (95% CI −52.3 to −10.6) and −63.5 (−131.8 to −12.8) resistant bacteraemia per 1,000 bacteraemia per quarter for fluoroquinolones; −48.3% (−62.7 to −32.3) and −153.1 (−255.7 to −77.0) for cephalosporins; and −17.8% (−47.1 to 20.8) and −63.6 (−206.4 to 42.4) for co-amoxiclav, respectively. Overall, there was reversal of a previously rising rate of fluoroquinolone resistance and flattening of previously rising rates of cephalosporin and co-amoxiclav resistance. The limitations of this study include that associations are not definitive evidence of causation and that potential effects of underlying secular trends in the postintervention period and/or of other interventions occurring simultaneously cannot be definitively excluded.
In this population-based study in Scotland, compared to prior trends, there were very large reductions in community broad-spectrum antimicrobial use associated with the stewardship intervention. In contrast, changes in resistance among coliform bacteraemia were more modest. Prevention of resistance through judicious use of new antimicrobials may be more effective than trying to reverse resistance that has become established.
Why was this study done?
- Antimicrobial resistance is an increasing global public health threat, and antimicrobial use in humans is a key driver of resistance development. The majority of antimicrobial prescribing occurs in the community, and there is evidence that a significant proportion of such prescribing is unnecessary.
- Antimicrobial stewardship interventions aim to improve antimicrobial use. There is increasing evidence that they are effective in influencing rates of the targeted prescribing in both primary and secondary care, but there is much less reliable evidence on their effect on resistance.
- A primary care antimicrobial stewardship intervention in the Tayside region of Scotland, UK, in 2009 was intended to reduce broad-spectrum antimicrobial use in the community. We used this opportunity to examine changes associated with the intervention in prescribing of three individual targeted antimicrobials and changes in resistance to these antimicrobials among community-associated bloodstream infections using routinely collated population data.
What did the researchers do and find?
- We conducted segmented regression analyses of interrupted time series data on community prescribing of fluoroquinolones, cephalosporins, and co-amoxiclav between 2005 and 2012 and on resistance to the same antimicrobials among common gram-negative bloodstream infections (E. coli, Klebsiella spp., and Proteus spp.) identified on admission to hospital. We modelled immediate changes in rates and in trends associated with the stewardship intervention and estimated changes in prescribing (compared to predicted rates if the intervention had not occurred) 1 and 3 years later, and in resistance 1.5 and 3.5 years later (incorporating a prespecified expected 6-month delay between changes in prescribing and changes in resistance).
- We found large and sustained reductions in prescribing of all three antimicrobials associated with implementation of the stewardship intervention.
- We found reductions in coliform bacteraemia resistance rates associated with the intervention that were significant for fluoroquinolones and cephalosporins by 3.5 years but not for co-amoxiclav. The reductions in resistance took much longer than reductions in prescribing to become evident, and relative reductions were more modest. The overall pattern was of flattening rather than reversal of previously rising resistance rates.
What do these findings mean?
- Antimicrobial stewardship interventions in primary care have a role in containing antimicrobial resistance in serious infections.
- Very large changes in prescribing were associated with much smaller changes in resistance. Preventing development of resistance through careful use of new antimicrobials may be more effective than attempting to reverse established resistance.
Citation: Hernandez-Santiago V, Davey PG, Nathwani D, Marwick CA, Guthrie B (2019) Changes in resistance among coliform bacteraemia associated with a primary care antimicrobial stewardship intervention: A population-based interrupted time series study. PLoS Med 16(6): e1002825. https://doi.org/10.1371/journal.pmed.1002825
Academic Editor: Nicola Low, University of Bern, SWITZERLAND
Received: September 14, 2018; Accepted: May 13, 2019; Published: June 7, 2019
Copyright: © 2019 Hernandez-Santiago et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Aggregate quarterly rates used for the segmented regression analyses are provided as supporting information. The raw individual level data used to generate these rates cannot be shared publicly because of the associated legislation. Data can be made available by the Health Informatics Centre, University of Dundee (contact via firstname.lastname@example.org) with the HIC Safe Haven for researchers who meet the criteria for access to confidential data and subject to a data access fee.
Funding: VHS was supported by a Scottish Government Chief Scientist Office (https://www.cso.scot.nhs.uk/) Clinical Academic Training Fellowship (Grant reference number: CAF/12/07) for the duration of this work. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Abbreviations: AIC, Akaike Information Criterion; AMR, antimicrobial resistance; CHI, Community Health Index; CLSI, Clinical and Laboratory Standards Institute; EPOC, Effective Practice and Organisation of Care; HIC, Health Informatics Centre; ITSA, interrupted time series analysis; NHS, National Health Service; MIC, minimum inhibitory concentration; RECORD, REporting of studies Conducted using Observational Routinely collected health Data; SOP, Standard Operational Procedure; UBA, uncontrolled before–after
Increasing antimicrobial resistance (AMR) is a major global public health threat [1,2]. An estimated 2 million people have resistant infections in the United States annually, with associated longer hospital stays and more costly antimicrobial treatment; an estimated 25,000 associated deaths occur annually in the European Union, and 23,000 in the US [3,4]. Coliforms, principally E. coli, are the commonest cause of bacteraemia, with incidence rates rising internationally [5,6]. Resistant E. coli bacteraemia is associated with higher mortality and poorer health outcomes, at least partly due to increased risk of initial antimicrobial therapy being inappropriate [7–11].
Higher antimicrobial use is associated with increased resistance at both population  and individual  levels. Up to one-third of ambulatory care antimicrobial use is estimated to be inappropriate , with widespread inappropriate use of broad-spectrum antimicrobials . Antimicrobial use is therefore considered the most modifiable cause of AMR [12,16]. Antimicrobial stewardship interventions have been widely implemented , with demonstrated reductions in targeted prescribing, but the effect on AMR is as yet unclear because few studies of antimicrobial stewardship interventions robustly examine the effects on resistance .
The most recent systematic review and meta-analysis of the effect of stewardship on subsequent AMR included 32 studies and reported that stewardship interventions in hospitals were associated with reductions in colonisation and infection with multi-drug–resistant gram-negative bacteria . However, only two (6%) of the included studies were judged to be high-quality by the review authors, not least because almost all included studies had an uncontrolled before–after design, which is unreliable (particularly in this context, in which interventions are often triggered by outbreaks, which usually remit without any intervention, and include changes to infection control as well as prescribing). In the most recent systematic reviews of antimicrobial stewardship interventions selecting for higher-quality studies, effects on AMR were examined by only 15 (7%) included studies of hospital-based stewardship interventions, and only four (10%) included studies of community-based interventions [18,20]. All four community-based studies examined Streptococcus pneumoniae resistance, with only one showing declining macrolide resistance following reductions in community macrolide use . Although restricting the use of a new antimicrobial is expected to minimise the development of resistance, it therefore remains uncertain whether reductions in existing antimicrobial use will translate into lower resistance, particularly in the community, where most antimicrobials are prescribed [21,22].
A primary care stewardship intervention implemented across the National Health Service (NHS) Tayside region of Scotland, UK, in 2009 was associated with large (>50%) reductions in total aggregate primary care prescribing of four broad-spectrum antimicrobials in primary care (fluoroquinolones, cephalosporins, co-amoxiclav, and clindamycin; targeted primarily because all are associated with Clostridium difficile infection) . In the years since the intervention, serious gram-negative (particularly coliform) infection and antimicrobial resistance have been recognised as increasingly important problems, with three of the antimicrobials targeted in the intervention commonly used in treatment of coliform infections (coliform bacteria are intrinsically resistant to clindamycin). The aim of this study was to separately examine changes in prescribing of fluoroquinolones, cephalosporins, and co-amoxiclav over a longer period and to examine whether there were any associated changes in resistance to these antimicrobials in patients admitted to hospital with community-associated coliform bacteraemia.
Study context and antimicrobial stewardship intervention
In the Tayside region of Scotland, all healthcare is provided by the NHS with the exception of some less complicated elective surgery. All residents are registered with a single general practice providing primary medical care, and the regional NHS microbiology laboratory carries out all culture and sensitivity testing. Primarily in response to concerns about rising rates of C. difficile infection, in 2009, NHS Tayside implemented a multifaceted antimicrobial stewardship intervention to reduce the use of broad-spectrum antimicrobials in primary care. The intervention consisted of prescriber education, guideline development and dissemination, change of the primary care antimicrobial guideline format and content, feedback of comparative prescribing data at practice level with target setting in relation to formulary adherence, and small financial incentives to participate in improvement work (approximately £390/€441/US$497 for an average-sized practice with approximately 5,500 registered patients).
The educational component of the intervention comprised both written guidance and pharmacist-facilitated educational outreach (academic detailing) to practices. Written educational material included revised antimicrobial guidance and a special edition of the Tayside Prescriber newsletter focused on raising awareness about C. difficile infection, including avoiding broad-spectrum antimicrobials associated with it. Educational outreach included face-to-face dissemination of the new antimicrobial prescribing guidance to practices and general practitioners and discussion of the prescribing targets set for primary care. In parallel, practices’ antimicrobial prescribing was audited at both practice and individual prescriber level, with feedback of prescribing rates compared to other practices in the region. Feedback data informed target setting for each practice with the aim of increasing formulary adherence, involving practices agreeing to reduce fluoroquinolone use in general and for respiratory infections in particular and to reduce antimicrobial use without a documented indication. Practices also agreed to implement a practice policy for dealing with inappropriate or telephone requests for antimicrobials.
We have previously reported that the intervention was associated with a rapid, large, and sustained decrease in an aggregate measure of total targeted broad-spectrum antimicrobial prescribing (fluoroquinolones, cephalosporins, co-amoxiclav, and clindamycin) in all age groups .
The University of Dundee Health Informatics Centre (HIC) provided data from 1 January 2005 to 31 December 2015 on all blood culture isolates in the region (total including negative results = 157,229, total gram-negative bacteraemia = 4,701), community-dispensed antimicrobials (total 3,442,205 prescriptions), patient demography (mean population per study calendar quarter = 424,014), hospital admissions, and mortality. Linkage between data sets at individual patient level was done using the NHS Scotland unique patient identifier (the Community Health Index [CHI] number). The CHI number is used to identify patients during all NHS Scotland healthcare episodes and is consistent across all data sets. Details of the data sets and the cleaning and linkage processes are provided as supporting information (S1 Text).
Protocol, ethics, and reporting
This observational study was conducted as part of VHS’s doctoral research and did not have a published protocol or signed, dated analysis plan. However, all the primary analyses reported in the main paper were specified prior to examining the outcome data, with the sensitivity analyses designed post hoc informed by examination of the time trends on request of the statistical reviewer.
HIC Standard Operational Procedures (SOPs) have been approved by both the NHS East of Scotland Research Ethics Committee and the relevant Caldicott Guardians (who have legal responsibility for approving use of unconsented NHS patient data). Ethics committee review of individual studies is not required provided that HIC SOPs are followed, including that all data analysis is carried out using anonymised data held in the ISO27001 and NHS Scotland accredited HIC Safe Haven.
The population for the prescribing analysis included the whole population registered with a general practitioner in NHS Tayside from 1 January 2005 to 31 March 2012. For the resistance analysis, patients were included if they were aged 18+ years and had a hospital admission with community-associated coliform bacteraemia from 1 January 2005 to 31 December 2015. Coliform bacteraemia was defined as isolation of any of E. coli, Klebsiella spp., or Proteus spp. in a blood culture specimen. Community-associated was defined as an isolate from a blood culture taken on days 0, 1, or 2 of a hospital admission in patients with no previous hospital admission in the 30 days prior to index admission date. Only the first eligible isolate per patient per calendar quarter was included.
We examined changes in prescribing for each of the three targeted broad-spectrum antimicrobials that are used in gram-negative infection (fluoroquinolones, cephalopsorins, and co-amoxiclav), measured as the rate of patients per 1,000 population exposed to each antimicrobial in primary care in each calendar quarter. We then examined changes in resistance to these three antimicrobials/classes in community-associated coliform bacteraemia isolates, measured as the quarterly rate of resistance per 1,000 eligible isolates. Resistance was defined as in vitro resistance reported by the Tayside Microbiology laboratory, which used Vitek systems (Vitek until 2007, then Vitek 2 subsequently) and Clinical and Laboratory Standards Institute (CLSI)-recommended minimum inhibitory concentrations (MICs) during the study period. We classified isolates in which the MIC indicated intermediate susceptibility as resistant to that antimicrobial. Prescribing outcomes were measured quarterly from the first quarter of 2005 to the first quarter of 2012, inclusive, and resistance outcomes were measured quarterly from the first quarter of 2005 to the fourth quarter of 2015, inclusive.
Segmented regression analysis of interrupted time series data plotted quarterly was used to examine changes in both prescribing and resistance associated with implementation of the antimicrobial stewardship intervention. Time series analysis is generally acknowledged to be a robust quasiexperimental method for analysing the effect of interventions that have not been or cannot be randomised. It allows the statistical assessment of how an intervention is associated with change in an outcome of interest both immediately and over time . A time series is repeated observations (aggregate measures at each time point, for example, proportions or means) of a particular outcome over time, which is then divided (interrupted) into time periods before and after an intervention. Segmented regression analysis evaluates both changes in level (i.e., any step change) immediately after the intervention and changes in slope in the postintervention period compared to the preintervention period, which serves as the control. Segmented regression controls for the effect of secular time trends, avoiding bias commonly present in uncontrolled before-and-after comparison of mean rates.
The stewardship intervention was implemented at the start of the second quarter of 2009, and for the prescribing analysis, this was chosen to be the first time point after the intervention, giving 17 time points before the intervention and 12 time points in the postintervention segment, as prescribing data from 2005 to 2012 were included in the model. Constraining the postintervention time period in this way reduced the risks that assumptions of linearity will be violated and that other stewardship interventions will influence the outcome but includes sufficient time points to evaluate intervention effects, including their sustainability. For resistance analysis purposes, we a priori assumed that there would be a 6-month lag between intervention implementation and any associated change in resistance, based on evidence that individual exposure to antimicrobials is associated with subsequent increased resistance to those antimicrobials for up to 6 months [26–29]. The segmented regression model for resistance therefore assumed an interruption at the fourth quarter of 2009 (rather than the second quarter) and included data to the end of 2015, giving 19 preinterruption and 25 postinterruption time points. Models estimated the baseline level and trend in prescribing and resistance rates before the interruption and any step change or change in trend following the interruption.
We fitted an autoregressive model to analyse the changes in both prescribing and resistance. The core model fitted was as follows : where Yt is the outcome (for example, the rate per 1,000 coliform bacteraemia with resistance to a specified antimicrobial) at time t; time is a continuous variable, indicating the number of quarters from the start of the study period; interruption is an indicator for time points before and after the interruption; and time after interruption is a continuous variable, indicating the number of quarters after the interruption. β0 estimates the baseline level of the outcome at the start of the study, β1 estimates the baseline trend before the interruption, β2 estimates the immediate change in level of the outcome after the interruption, and β3 estimates the change in the trend after the interruption.
For prescribing outcomes, we estimated absolute and relative changes at 1 and 3 years after the intervention. For resistance, since the model interruption was a priori defined to be 6 months after the start of the antimicrobial stewardship intervention, results are reported as absolute and relative change at 1.5 and 3.5 years after the stewardship intervention (a 6-month lag, then 1 and 3 years’ follow-up from the modelled interruption). The absolute change was calculated as the difference between model estimates of the predicted outcome if prior trends had continued and the outcome given any intervention effect. The relative change is the absolute change as a proportion of the outcome rate that the model predicted would have occurred if prior trends had continued. We used the bootstrapping method proposed by Zhang , implementing an adaptation of Zhang’s published SAS macro to calculate the 95% CIs for both absolute and relative percentage change estimates.
For all models, we examined whether there was significant autocorrelation up to fourth-order autocorrelation using the Durbin–Watson statistic and visual inspection of autocorrelation and partial autocorrelation plots. Lag terms were fitted as required to adjust for significant autocorrelation and were included in the models if they were statistically significant and improved model fit. In a similar way, seasonality was explored and adjusted for in the prescribing models as required. Model outputs for lag and seasonal terms have no meaningful interpretation in terms of estimating changes associated with the intervention, so they are not reported.
Because of the shape of the resistance time series (Fig 1, right-hand panels), we conducted two types of post hoc sensitivity analysis for resistance to each of the three antimicrobials. First, we modelled the outcome against time without fitting any interruption and compared model fit using the Akaike Information Criterion (AIC)  with the model including the interruption for each antimicrobial. Next, we excluded the first year’s data from each resistance time series and refitted the models using data from 1 January 2006 to 31 December 2015 and compared model outputs and estimated absolute and relative changes with the results of the prespecified analyses.
A total of 3,442,205 oral antimicrobial prescriptions were dispensed in primary care in Tayside over the study period. Amoxicillin was by far the most prescribed antimicrobial, representing 30.1% of all antimicrobial prescriptions, followed by tetracyclines (11.8%), trimethoprim (11.4%), flucloxacillin (10.1%), and macrolides (9.5%). In total, the three targeted broad-spectrum antimicrobials included here accounted for 11.2% of all antimicrobial prescribing.
There were 2,143 eligible bacteraemia episodes from 2,004 patients over the whole period examined (mean age 73.7 [SD 14.8] years; 51.4% women), increasing from 136 in 2005 to 234 in 2015 (Table 1). There was a significant upwards trend in eligible bacteraemia episodes (increasing by 0.2 per 100,000 population per quarter, p = 0.01) prior to the intervention but no significant change in level or trend associated with the intervention (p = 0.26 and p = 0.53, respectively) (S1 Fig and S1 Table). Over the whole period, E. coli was isolated in 81.8% of eligible bacteraemia episodes, compared to Klebsiella spp. in 12.0% and Proteus spp. in 6.2%, with little change in these proportions over time. Mean length of stay fell slightly, and 30-day mortality did not change during the study period (Table 1). Across the entire time period, mean resistance rates were 9.0% for fluoroquinolones, 10.6% for cephalosporins, and 22.6% for co-amoxiclav. The proportions resistant to more than one of the three antimicrobials were low, with a maximum 12.8% (24 of 188) resistant to any two and 4.9% (10 of 203) resistant to all three in any study year, in 2009 and 2010, respectively (S2 Table).
Changes in prescribing
Table 2 shows the results of the interrupted time series analysis, and Table 3 reports the estimated changes associated with the intervention. There was significant autocorrelation in the prescribing time series for co-amoxiclav but not for fluoroquinolones or cephalosporins, according to the Durbin–Watson statistic and examination of autocorrelation plots. Adding a seasonal (in which calendar quarters 1 and 4 are winter and 2 and 3 are summer) term to the co-amoxiclav model improved model fit and accounted for autocorrelation (S3 Table). Consistent with our previous work examining an aggregate measure of broad-spectrum antimicrobial prescribing , there were large decreases in individual targeted antimicrobials used in gram-negative infection: fluoroquinolones, cephalosporins, and co-amoxiclav (Fig 1, left-hand panel). There were no significant trends prior to the intervention for any of these antimicrobials. However, following the intervention, there were statistically significant immediate reductions in prescribing of −3.8 (95% CI −4.7 to −3.0, p < 0.001) per 1,000 population dispensed per quarter for fluoroquinolones, of −1.2 (−1.9 to −0.5, p = 0.001) for cephalosporins, and of −1.9 (−2.6 to −1.3, p < 0.001) for co-amoxiclav (Table 2). The intervention was also associated with significant changes in trend following the intervention with further significant, additional decreases of −0.2 (95% CI −0.4 to −0.1, p < 0.001), −0.4 (95% CI −0.5 to −0.3, p < 0.001), and −0.4 (95% CI −0.5 to −0.3, p < 0.001) patients per 1,000 population dispensed per quarter for fluoroquinolones, cephalosporins, and co-amoxiclav, respectively. Given preintervention trends, fluoroquinolone prescribing (which was the main target of the intervention) at 1 and 3 years postintervention was estimated to be −53.4% (95% CI −58.9% to −47.7%) and −68.8% (−76.3% to −62.1%) lower than expected. These equate to absolute reductions of −4.7 (−5.5 to −4.0) and −6.3 (−7.6 to −5.2) people exposed per 1,000 population per quarter at 1 and 3 years postintervention, respectively. There were also large and statistically significant relative reductions in both cephalosporin and co-amoxiclav prescribing at 1 and 3 years postintervention (Table 3).
Changes in resistance
Table 4 shows the results of the interrupted time series analysis, and Table 5 reports the estimated changes associated with the intervention. There was significant autocorrelation in the cephalosporin resistance time series but not for fluoroquinolones or co-amoxiclav. Fitting a lag 4 term improved both model fit according to AIC and the Durbin–Watson statistic, accounting for autocorrelation (S4 Table). Resistance to all three targeted antimicrobials was significantly increasing before the intervention (Fig 1, Table 4), but there were no statistically significant immediate step changes in resistance at the time of the interruption (Table 4). Preintervention, fluoroquinolone resistance was statistically significantly increasing each quarter by 3.5/1,000 (95% CI 0.5 to 6.4, p = 0.02) bacteraemia episodes (from a baseline rate of 46.5/1,000) with a subsequent downwards change in trend postintervention of −4.4/1,000/quarter (−7.9 to −0.9, p = 0.01). Cephalosporin resistance was statistically significantly increasing each quarter by 8.2/1,000 bacteraemia episodes (3.5 to 12.9, p = 0.001), from baseline 49.9, with a downwards change in trend postintervention of −7.0/1,000/quarter (−12.3 to −1.8, p = 0.03). Co-amoxiclav resistance was nonsignificantly increasing each quarter by 4.6/1,000 bacteraemia episodes (−1.0 to 10.3, p = 0.11) with a nonsignificant downward change in slope of −3.3/1,000/quarter (−10.4 to 3.8, p = 0.33).
There was a relative reduction in fluoroquinolone resistance compared to predicted by 1.5 years after the stewardship intervention (−17.2%, 95% CI −38.4 to 0.04) that increased to −34.7% (−52.3 to −10.6) by 3.5 years. These equate to absolute reductions of −27.2 (−68.8 to 7.9) and −63.5 (−131.8 to −12.8) resistant bacteraemia episodes per 1,000 per quarter by 1.5 and 3.5 years postintervention, respectively (Table 5). The overall pattern was that a rising trend in resistance was reversed (Fig 1). Statistically significant relative reductions in cephalosporin resistance of −38.2% (95% CI −56.1 to −18.8) compared to predicted were observed by 1.5 years and −48.3% (95% CI −62.7 to −32.3) by 3.5 years after the stewardship intervention. Equivalent absolute reductions were −95.0 (−157.4 to −42.4) and −153.1 (−255.7 to −77.0) resistant bacteraemia episodes per 1,000 per quarter by 1.5 and 3.5 years postintervention, respectively (Table 5). The overall effect was that a rising trend essentially flattened. Differences between observed and predicted co-amoxiclav resistance were in the direction of a reduction, consistent with flattening of a previously rising trend, but did not reach statistical significance.
In the first post hoc sensitivity analyses comparing simple linear models of resistance outcomes over time to the primary models, including an interruption at 6 months after the stewardship intervention, AIC values indicated equivalent model fit (i.e., difference in AIC < 4) for all three antimicrobials (S5 Table). However, testing the specific hypothesis that the intervention was associated with changes in resistance requires fitting of an interruption, and model fit was no worse, supporting the primary analysis as reasonable, although interpretation should be cautious.
In the second post hoc sensitivity analyses comparing segmented regression resistance models that exclude 2005 data to the primary segmented regression resistance models (model outputs in S6 Table), the sensitivity analyses estimated relative changes at 1.5 and 3.5 years postintervention that for fluoroquinolones were larger but had wider 95% CI, for cephalosporins were slightly smaller with wider 95% CI, and for co-amoxiclav were slightly smaller with wider 95% CI and remained nonsignificant (S7 Table). These are in accordance with interpretation of the primary analyses, that the overall pattern is of reductions in resistance but the magnitude of the changes is modest compared to those for changes in prescribing, and not all changes are statistically significant. Again, this emphasises that interpretation of the resistance models should be cautious.
Summary of main findings
In this population-based study in Scotland, UK, large reductions in community prescribing of broad-spectrum antimicrobials following an antimicrobial stewardship intervention were associated with reductions in resistance among people admitted to hospital with community-associated coliform bacteraemia. Reductions in resistance from those predicted by prior trends were statistically significant by 1.5 years for cephalosporins (where rising rates flattened) and by 3.5 years after the stewardship intervention for fluoroquinolones (where rising rates were reversed). The changes in co-amoxiclav resistance were in the direction of flattening a rising trend, although they were not statistically significant. By 3.5 years after the stewardship intervention, the absolute changes were a reduction of 63.5 isolates resistant to fluoroquinolones per 1,000 bacteraemia episodes, a reduction of 153.1 per 1,000 bacteraemia episodes resistant to cephalosporins, and a nonsignificant reduction of 63.6 per 1,000 resistant to co-amoxiclav. Multidrug resistance was rare.
Strengths of the study
A key strength of this study is the availability of longitudinal data in a complete geographical population, allowing examination of long-term associations between a stewardship intervention and both targeted broad-spectrum antimicrobial prescribing and resistance to targeted broad-spectrum antimicrobials in serious community-associated infections. Further strengths include that the data analysed are all captured as part of routine care, meaning the likelihood of missing data is very low, and that Tayside has a very stable population, meaning observed changes are unlikely to reflect changes in the population at risk. In addition, there were no significant changes to the organisation of healthcare in Tayside over the study period, with the same two acute hospitals serving the whole population. We know that total antimicrobial prescribing in primary care did not change in association with the intervention [23,32], so it is unlikely that changes in general practitioner consultation rates or patient demand for antimicrobials influenced the findings. In addition, health-seeking behaviour is less likely to influence bacteraemia rates than positive community urine culture rates and cannot explain changes in resistance rates.
The interrupted time series analysis (ITSA) used is a robust design for evaluation of real-world interventions that cannot be randomised [33,34], and there were adequate time points before and after the intervention in our data series. In comparison with uncontrolled before–after (UBA) studies, which compare mean rates or proportions before and after an intervention, ITSA accounts for pre- and postintervention trends and changes in trend at the time of intervention. This is important for estimating intervention effects; for example, if the rate of the outcome of interest increases before an intervention and then decreases after (as happens with fluoroquinolone resistance in this analysis), then a UBA study may find that the mean outcome before and after does not change because time trends are ignored. Similarly, if the outcome was increasing preintervention and continued to increase at the same rate postintervention, a UBA study would erroneously report an increase associated with the intervention. Consistent with these limitations, UBA studies are not considered robust enough by the Cochrane Effective Practice and Organisation of Care (EPOC) Group to be included in systematic reviews, whereas ITSAs are eligible for inclusion .
The focus on community-associated bacteraemia is a further strength that minimises confounding from unmeasured exposure to antimicrobials in hospital. The data analysed are from one area of Scotland, but Tayside is representative (approximately 10%) of the wider Scottish population in terms of sociodemographic characteristics and not dissimilar to other regions, at least across the UK. There is no biological reason that an intervention elsewhere that is associated with similarly large changes in prescribing would not have similar associated changes in resistance, so these findings are likely to be widely generalisable.
Limitations of the study
One weakness of the analysis is that observational studies can only examine and report association, rather than causation, and it is impossible to exclude residual confounding, which, in the case of ITSA, includes not knowing what the postintervention trends in the outcomes would have been if the intervention had not occurred. Including a control group would have strengthened the analysis, but there was no suitable control group because this intervention was implemented across the entire health board, and all other Scottish health boards implemented prescribing interventions of various kinds at around the same time in response to a national C. difficile infection outbreak. More specific to ITSA and to this data series, a weakness is the relatively low number of bacteraemia episodes per time point (median 49.5 [IQR 37.7 to 59.2]), which increases variability between time points and reduces the power to detect a true change. ITSA may also be at risk of bias if there are other major interventions in the time period examined. There were no other major changes to community prescribing or infection control policies, but there was a parallel change in the NHS Tayside hospital prescribing policy implemented at the end of 2008. The hospital policy included removing the same targeted antimicrobials/classes (as the primary care policy) from first-line indicated therapy for almost all infections. The only remaining first-line therapeutic indications in the hospital policy for targeted antimicrobials were fluoroquinolones for prostatitis, ceftriaxone for bacterial meningitis (which are both uncommon), and co-amoxiclav for severe community-acquired pneumonia (which is relatively common). It is possible that these changes had some effect on resistance among coliforms circulating in the community, even though patients with a recent hospital admission were excluded from the analysis. This is more likely for co-amoxiclav, given that its use remained higher in hospital after the policy change, and may partly explain the lack of significant reduction in co-amoxiclav resistance observed. The size and temporal relationship of the changes we observed in prescribing are consistent with these being attributable to the intervention. Given the smaller changes in resistance observed and the findings of the sensitivity analyses, we are more cautious in our interpretation of the observed changes in resistance, which reinforces the conclusion that large changes in community prescribing are associated with more modest changes in resistance at best.
Comparison with the literature
Most antimicrobial prescribing is in the community, but the evidence base for antimicrobial stewardship in the community  is weaker than in the hospital , and few studies have robustly examined associations between community antimicrobial stewardship programs and resistance. A pilot study aiming to prospectively recruit patients admitted to hospital with urinary tract infection found an association between prior antimicrobial exposure and trimethoprim resistance in the index urine sample (crude odds ratio 3.58, 95% CI 1.18 to 10.9). The findings were limited by the small sample size, and the authors concluded that a study using routine data was required .
Gram-positive resistance has been shown to reduce in response to community-based stewardship. For example, pneumococcal macrolide resistance in Finland fell from 19.0% to 8.6% over 5 years following >40% reductions in macrolide use in the 1990s , and falling penicillin use in Iceland was associated with 25% reductions in pneumococcal penicillin resistance over 5 years . Reported associations between community antimicrobial use and gram-negative resistance are less consistent. A 25% reduction in ciprofloxacin use in Israel lasting 7 months was associated with a reported reduction in fluoroquinolone resistance in urinary E. coli isolates from 12% to 9% . In contrast, an 86% reduction in trimethoprim use in Sweden over 24 months was not associated with any change in resistance in urinary isolates , and sulphonamide resistance in the UK continued to rise in the 1990s despite the virtual ending of sulphonamide use .
The inconsistent relationship between antimicrobial stewardship and subsequent resistance observed in this study and others is likely related to the multiple mechanisms by which resistance arises and persists. Antimicrobial prescribing leads to selection pressures that increase resistance, but reducing antimicrobial use and therefore selection pressure will only rapidly reverse resistance if there is a significant associated fitness cost to maintaining it [40–42]. The cell wall structure in gram-positive bacteria results in higher fitness costs of resistance, consistent with observed reversibility of resistance after reductions in prescribing [21,36,43]. Fluoroquinolone resistance among E. coli has well-described associated fitness costs, but some resistance mutations confer little or no cost to bacterial fitness, and certain resistance traits may even be beneficial [42,43]. In addition, selection of resistance to a particular antimicrobial may occur in the absence of exposure to that particular antimicrobial because of mutations for resistance coexisting on the same plasmid and so being selected by other antimicrobial use [42,43]. Changes in resistance in response to stewardship interventions would therefore be predicted to have somewhat variable effects.
Implications of the findings
The findings are consistent with primary care antimicrobial stewardship targeting broad-spectrum antimicrobial prescribing being associated with worthwhile but delayed changes in resistance in coliform bacteraemia, a serious infection. However, this is not the only potential approach to containing resistance, and it may have limited impact if there is substitution with other antimicrobials because of the risk of developing cross-resistance such as that mediated by plasmids. Alternatives include reducing total antimicrobial prescribing and short-term cycling. Reducing total antimicrobial prescribing is increasingly the strategy of choice in primary care , but this is complicated because of the risk of harm from undertreating high-risk groups, including the elderly, if the treatment threshold is high . It will therefore be very important to collect balancing measures relating to infection outcomes in evaluating stewardship that reduces total antimicrobial use . Planned, short-term cycling of recommended antimicrobials is an alternative strategy that aims to avoid sustained selection pressure of any individual antimicrobial to limit the development of resistance. There is some evidence of effectiveness of this approach in the controlled environment of intensive care units, although reported associated changes in resistance are mixed in this context . Short-term cycling is limited by the small number of antimicrobials available for any indication, and its value in primary care remains to be established. Whatever strategy is chosen, the key problem remains avoiding antimicrobial prescribing when it is not necessary, and judicious use of new antimicrobials is key to reducing or containing resistance. Optimising these approaches will require improved clinical decision support and/or diagnostics at the point of care that overcome logistical, financial, and behavioural barriers to their implementation .
Further research is needed to better understand broader mechanisms of changes in resistance in response to antimicrobial stewardship interventions, including bacterial genomic studies in combination with population data. ITSA is an appropriate method for evaluating healthcare interventions that cannot be randomised, but the widespread application of ITSA in a timely manner requires more responsive surveillance systems that use continuous monitoring of routine data.
The known mechanisms involved in gram-negative resistance are consistent with our findings that very large (>60%) reductions in community antimicrobial use can help contain resistance in coliform bacteraemia in this context but may not lower it from established rates. Notably, the associated reduction in resistance was delayed, and rising resistance only truly reversed for fluoroquinolones. Minimising broad-spectrum antimicrobial use in the community is clearly of value in containing resistance in specific serious infections, but further research is required to examine the impact on resistance more generally and other outcomes of community stewardship interventions aiming to achieve large reductions in total antimicrobial use. The findings also emphasise the critical importance of careful use of new antibiotics to minimise the initial development of resistance.
S1 RECORD Checklist. RECORD, REporting of studies conducted using observational routinely collected health data.
S1 Data. Quarterly rates of prescribing and resistance.
S1 Text. Data sets and processes for data cleaning and linkage.
S1 Fig. Admission rate (per 100,000 population) with community-associated coliform bacteraemia in Tayside 2005–2015.
S1 Table. ITSA of rate of admission with community-associated coliform bacteraemia (modelled interruption is date of primary care antimicrobial stewardship intervention plus 6 months).
ITSA, interrupted time series analysis.
S2 Table. Rates of resistance to individual antimicrobials and combinations of targeted antimicrobials among eligible community-associated bacteraemia episodes per study year.
S3 Table. Assessment and accounting for autocorrelation in prescribing time series with final model selection.
S4 Table. Assessment and accounting for autocorrelation in resistance time series with final model selection.
S5 Table. Sensitivity analysis 1: Comparing model fit for simple linear regression models of resistance outcomes over time and segmented regression models of resistance outcomes incorporating the intervention (interruption at intervention plus 6 months).
S6 Table. Sensitivity analysis 2: Results of ITSA of changes in antimicrobial resistance to targeted antimicrobials among community-associated coliform bacteraemia associated with the stewardship intervention (modelled interruption is date of primary care antimicrobial stewardship intervention plus 6 months) with 2005 data removed (including 2006 to 2015).
ITSA, interrupted time series analysis.
S7 Table. Sensitivity analysis 2: Estimated absolute and relative changes, compared to levels predicted by prior trends, in resistance among community-associated coliform bacteraemia at 1.5 and 3.5 years after primary care antimicrobial stewardship intervention (modelled interruption is date of primary care antimicrobial stewardship intervention plus 6 months) with 2005 data removed (including 2006 to 2015).
We thank the HIC, University of Dundee, for providing the data and Safe Haven access. We also thank Ms Andrea Patton, University of Dundee, for statistical advice.
- 1. Spellberg B, Guidos R, Gilbert D, Bradley J, Boucher HW, Scheld WM, et al. The Epidemic of Antibiotic-Resistant Infections: A Call to Action for the Medical Community from the Infectious Diseases Society of America. Clin Infect Dis. 2008;46(2): 155–164. pmid:18171244
- 2. English Surveillance Programme for Antimicrobial Utilisation and Resistance (ESPAUR) Report 2017. [cited 2018 January 23] Public Health England. Available from: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/656611/ESPAUR_report_2017.pdf
- 3. ECDC/EMEA Joint Working Group, 2009. The bacterial challenge: time to react. [cited 2017 June 26] EMEA/576176, pp.13-42. Available from: http://www.ecdc.europa.eu/en/publications/Publications/0909_TER_The_Bacterial_Challenge_Time_to_React.pdf.
- 4. Centers for Disease Control and Prevention. Antibiotic resistance threats in the United States, 2013. [cited 2017 June 26] US Department of Health and Human Services. Centers for Disease Control and Prevention. Available from: http://www.cdc.gov/drugresistance/threat-report-2013/pdf/ar-threats-2013-508.pdf.
- 5. Kim Y-K, Pai H, Lee H-J, Park S-E, Choi E-H, Kim J, et al. Bloodstream Infections by Extended-Spectrum β-Lactamase-Producing Escherichia coli and Klebsiella pneumoniae in Children: Epidemiology and Clinical Outcome. Antimicrob Agents Chemother. 2002;46(5): 1481–1491. pmid:11959586
- 6. Melzer M, Petersen I. Mortality following bacteraemic infection caused by extended spectrum beta-lactamase (ESBL) producing E. coli compared to non-ESBL producing E. coli. J Infect. 55(3): 254–259. pmid:17574678
- 7. Schwaber MJ, Carmeli Y. Mortality and delay in effective therapy associated with extended-spectrum β-lactamase production in Enterobacteriaceae bacteraemia: a systematic review and meta-analysis. J Antimicrob Chemother. 2007;60(5): 913–920. pmid:17848376
- 8. Health Protection Scotland. Healthcare Associated Infections. Health Protection Scotland, 2014 Health Protection Scotland, Glasgow; 2015 [Report]. [cited 2017 June 26] Available from: http://www.hps.scot.nhs.uk/resourcedocument.aspx?id=4090.
- 9. Bou-Antoun S, Davies J, Guy R, Johnson AP, Sheridan EA, Hope RJ. Descriptive epidemiology of Escherichia coli bacteraemia in England, April 2012 to March 2014. Eurosurveillance. 2016;21(35): 30329.
- 10. Raman G, Avendano E, Berger S, Menon V. Appropriate initial antibiotic therapy in hospitalized patients with gram-negative infections: systematic review and meta-analysis. BMC Infect Dis. 2015;15(1): 395.
- 11. Peralta G, Sanchez MB, Garrido JC, De Benito I, Cano ME, Martinez-Martinez L, et al. Impact of antibiotic resistance and of adequate empirical antibiotic treatment in the prognosis of patients with Escherichia coli bacteraemia. J Antimicrob Chemother. 2007;60(4): 855–863. pmid:17644532
- 12. Bell BG, Schellevis F, Stobberingh E, Goossens H, Pringle M. A systematic review and meta-analysis of the effects of antibiotic consumption on antibiotic resistance. BMC Infect Dis. 2014;14:13. pmid:24405683
- 13. Costelloe C, Metcalfe C, Lovering A, Mant D, Hay AD. Effect of antibiotic prescribing in primary care on antimicrobial resistance in individual patients: systematic review and meta-analysis. BMJ. 2010 May 18;340:c2096. pmid:20483949
- 14. Fleming-Dutra KE, Hersh AL, Shapiro DJ, et al. Prevalence of inappropriate antibiotic prescriptions among us ambulatory care visits, 2010–2011. JAMA. 2016;315(17):1864–1873. pmid:27139059
- 15. Shapiro DJ, Hicks LA, Pavia AT, Hersh AL. Antibiotic prescribing for adults in ambulatory care in the USA, 2007–09. JAntimicrob Chemother. 2014;69(1): 234–240.
- 16. WHO. The evolving threat of antimicrobial resistance: options for action. Geneva, Switzerland: WHO. [cited 2017 June 26] Available from: http://apps.who.int/iris/bitstream/10665/44812/1/9789241503181_eng.pdf; 2012.
- 17. Davey P, Brown E, Charani E, Fenelon L, Gould IM, Holmes A, et al. Interventions to improve antibiotic prescribing practices for hospital inpatients. Cochrane Database Syst Rev. 2013 Apr 30;(4):CD003543. pmid:23633313
- 18. Davey P, Scott CL, Brown E, Charani E, Michie S, Ramsay CR, et al. interventions to improve antibiotic prescribing practices for hospital inpatients. Cochrane Database Syst Rev. 2017 Feb 9;2:CD003543. pmid:28178770
- 19. Baur D, Gladstone BP, Burkert F, Carrara E, Foschi F, Dobele S, et al. Effect of antibiotic stewardship on the incidence of infection and colonisation with antibiotic-resistant bacteria and Clostridium difficile infection: a systematic review and meta-analysis. Lancet Infect Dis 2017;17(9): 990–1001. pmid:28629876
- 20. Arnold SR, Straus SE. Interventions to improve antibiotic prescribing practices in ambulatory care. Cochrane Database Syst Rev. 2005;(4):CD003539. pmid:16235325
- 21. Seppala H, Klaukka T, Vuopio-Varkila J, Muotiala A, Helenius H, Lager K, et al. The effect of changes in the consumption of macrolide antibiotics on erythromycin resistance in group A streptococci in Finland. N Engl J Med 1997; 337: 441–446. pmid:9250845
- 22. Duffy E, Ritchie S, Metcalfe S, Van Bakel B, Thomas MG. Antibacterials dispensed in the community comprise 85%–95% of total human antibacterial consumption. J Clin Pharm Ther. 2017;43(1):59–64. pmid:28833324
- 23. Hernandez-Santiago V, Marwick CA, Patton A, Davey PG, Donnan PT, Guthrie B. Time series analysis of the impact of an intervention in Tayside, Scotland to reduce primary care broad-spectrum antimicrobial use. J Antimicrob Chemother. 2015;70(8): 2397–2404. pmid:25953807
- 24. Benchimol EI, Smeeth L, Guttmann A, Harron K, Moher D, Petersen I, et al. The REporting of studies conducted using observational routinely-collected health data (RECORD) statement. PLoS Med. 2015;12(10):e1001885. pmid:26440803
- 25. Wagner AK, Soumerai SB, Zhang F, Ross-Degnan D. Segmented regression analysis of interrupted time series studies in medication use research. J Clin Pharm Ther. 2002;27(4):299–309. pmid:12174032
- 26. Donnan PT, Wei L, Steinke DT, Phillips G, Clarke R, Noone A, et al. Presence of bacteriuria caused by trimethoprim resistant bacteria in patients prescribed antibiotics: multilevel model with practice and individual patient data. BMJ. 2004;328(7451):1297. pmid:15166067
- 27. Duffy MA, Hernandez-Santiago V, Orange G, Davey PG, Guthrie B. Trimethoprim prescription and subsequent resistance in childhood urinary infection: multilevel modelling analysis. Br J Gen Pract. 2013;63(609):e238–243. pmid:23540479
- 28. Malhotra-Kumar S, Lammens C, Coenen S, Van Herck K, Goossens H. Effect of azithromycin and clarithromycin therapy on pharyngeal carriage of macrolide-resistant streptococci in healthy volunteers: a randomised, double-blind, placebo-controlled study. Lancet. 2007;369(9560): 482–490. pmid:17292768
- 29. Chung A, Perera R, Brueggemann AB, Elamin AE, Harnden A, Mayon-White R, et al. Effect of antibiotic prescribing on antibiotic resistance in individual children in primary care: prospective cohort study. Bmj. 2007;335(7617):429. pmid:17656505
- 30. Zhang F, Wagner AK, Soumerai SB, Ross-Degnan D. Methods for estimating confidence intervals in interrupted time series analyses of health interventions. Journal of clinical epidemiology. 2009;62(2):143–148. pmid:19010644
- 31. Akaike H. A new look at the statistical model identification. IEEE Transactions on Automatic Control 1974;19(6): 716–723.
- 32. Rodriguez-Sanz A, Fuentes B, Martinez-Sanchez P, Prefasi D, Martinez-Martinez M, Correas E, et al. High-Density Lipoprotein: A Novel Marker for Risk of In-Hospital Infection in Acute Ischemic Stroke Patients? Cerebrovasc Dis. 2013;35(3):291–297. pmid:23595024
- 33. Fowler S, Webber A, Cooper BS, Phimister A, Price K, Carter Y, et al. Successful use of feedback to improve antibiotic prescribing and reduce Clostridium difficile infection: a controlled interrupted time series. J Antimicrob Chemother. 2007;59(5):990–995. pmid:17387117
- 34. Ansari F, Gray K, Nathwani D, Phillips G, Ogston S, Ramsay C, et al. Outcomes of an intervention to improve hospital antibiotic prescribing: interrupted time series with segmented regression analysis. J Antimicrob Chemother. 2003;52(5):842–848. pmid:14563900
- 35. Effective Practice and Organisation of Care (EPOC). What study designs should be included in an EPOC review? EPOC Resources for review authors. Oslo: Norwegian Knowledge Centre for the Health Services; 2016. [cited 2017 June 26] Available from: http://epoc.cochrane.org/epoc-specific-resources-review-authors.
- 36. Austin DJ, Kristinsson KG, Anderson RM. The relationship between the volume of antimicrobial consumption in human communities and the frequency of resistance. Proc Natl Acad Sci U S A. 1999;96(3): 1152–1156. pmid:9927709
- 37. Gottesman BS, Carmeli Y, Shitrit P, Chowers M. Impact of quinolone restriction on resistance patterns of Escherichia coli isolated from urine by culture in a community setting. Clin Infect Dis. 2009;49(6): 869–875. pmid:19686074
- 38. Sundqvist M, Geli P, Andersson DI, Sjolund-Karlsson M, Runehagen A, Cars H, et al. Little evidence for reversibility of trimethoprim resistance after a drastic reduction in trimethoprim use. J Antimicrob Chemother. 2010;65(2): 350–360. pmid:19900952
- 39. Enne VI, Livermore DM, Stephens P, Hall LM. Persistence of sulphonamide resistance in Escherichia coli in the UK despite national prescribing restriction. Lancet. 2001;357(9265):1325–1328. pmid:11343738
- 40. Johnsen PJ, Townsend JP, Bohn T, Simonsen GS, Sundsfjord A, Nielsen KM. Factors affecting the reversal of antimicrobial-drug resistance. Lancet Infect Dis. 2009;9(6):357–364. pmid:19467475
- 41. Holmes AH, Moore LSP, Sundsfjord A, Steinbakk M, Regmi S, Karkey A, et al. Understanding the mechanisms and drivers of antimicrobial resistance. Lancet. 2016;387(10014): 176–187. pmid:26603922
- 42. Andersson DI, Hughes D. Antibiotic resistance and its cost: is it possible to reverse resistance? Nat Rev Micro. 2010;8(4): 260–271.
- 43. Melnyk AH, Wong A, Kassen R. The fitness costs of antibiotic resistance mutations. Evol Appl. 2015;8(3):273–283. pmid:25861385
- 44. Optimising Antibiotic use 2018. Scottish Antimicrobial Prescribing Group. [Internet]. [cited 2019 February 26] Available from: https://www.sapg.scot/quality-improvement/primary-care/optimising-antibiotic-use/.
- 45. Gulliford MC, Moore MV, Little P, Hay AD, Fox R, Prevost AT, et al. Safety of reduced antibiotic prescribing for self limiting respiratory tract infections in primary care: cohort study using electronic health records. BMJ 2016;354:i3410. pmid:27378578
- 46. Toma M, Davey PG, Marwick CA, Guthrie B. A framework for ensuring a balanced accounting of the impact of antimicrobial stewardship interventions. J Antimicrob Chemother. 2017;72(12):3223–3231. pmid:28961725
- 47. Brown EM, Nathwani D. Antibiotic cycling or rotation: a systematic review of the evidence of efficacy. J Antimicrob Chemother. 2005;55(1): 6–9. pmid:15531594
- 48. UK 5-year action plan for antimicrobial resistance 2019–2024. 2019 [Internet]. [cited year month day] Available from: https://www.gov.uk/government/publications/uk-5-year-action-plan-for-antimicrobial-resistance-2019-to-2024.