The effectiveness of computerised decision support on antibiotic use in hospitals: A systematic review

Background Inappropriate antimicrobial use has been shown to be an important determinant of the emergence of antimicrobial resistance (AMR). Health information technology (HIT) in the form of Computerised Decision Support (CDS) represents an option for improving antimicrobial prescribing and containing AMR. Objectives To evaluate the evidence for CDS in improving quantitative and qualitative measures of antibiotic prescribing in inpatient hospital settings. Methods A systematic literature search was conducted of articles published from inception to 20th December 2014 using eight electronic databases: MEDLINE, EMBASE, PUBMED, Web of Science, CINAHL, Cochrane Library, HMIC and PsychINFo. An updated systematic literature search was conducted from January 1st 2015 to October 1st 2016 using PUBMED. The search strategy used combinations of the following terms: (electronic prescribing) OR (clinical decision support) AND (antibiotic or antibacterial or antimicrobial) AND (hospital or secondary care or inpatient). Studies were evaluated for quality using a 10-point rating scale. Results Eighty-one studies were identified matching the inclusion criteria. Seven outcome measures were evaluated: adequacy of antibiotic coverage, mortality, volume of antibiotic usage, length of stay, antibiotic cost, compliance with guidelines, antimicrobial resistance, and CDS implementation and uptake. Meta-analysis of pooled outcomes showed CDS significantly improved the adequacy of antibiotic coverage (n = 13; odds ratio [OR], 2.11 [95% CI, 1.67 to 2.66, p ≤ 0.00001]). Also, CDS was associated with marginally lowered mortality (n = 20; OR, 0.85 [CI, 0.75 to 0.96, p = 0.01]). CDS was associated with lower antibiotic utilisation, increased compliance with antibiotic guidelines and reductions in antimicrobial resistance. Conflicting effects of CDS on length of stay, antibiotic costs and system uptake were also noted. Conclusions CDS has the potential to improve the adequacy of antibiotic coverage and marginally decrease mortality in hospital-related settings.


Introduction
Antimicrobials have saved millions of lives since their introduction [1] however; antimicrobial resistance (AMR) has increased over the past four decades.[2] Evidence shows that 30%-50% of antimicrobial prescribing is sub-optimal.[3] Inappropriate antimicrobial use has been shown to be an important determinant of the emergence and persistence of AMR.[2] This pattern of irrational use in hospitals and the relative reduction in development of new antibiotic entities pose a challenge for clinicians, as their options to treat infections, especially those caused by resistant pathogens, become limited.
The use of health information technology (HIT) is one strategy to optimise antibiotic use in health care settings.Over the last twenty years, there have been rapid advances and investment in HIT, manifesting as an increased uptake of the use of computers in healthcare.The NHS embraces the role of HIT in optimising the quality of care and patient safety.In the UK, £12.8 billion has been invested in the National Programme for Information Technology (NPfIT) by the National Health Service (NHS).[4] Computerised Decision Support (CDS) represents a potential solution for improving antimicrobial prescribing and containing antimicrobial resistance by supporting clinical decision making [5,6] thus optimising antibiotic use and improving patient outcomes.It potentially plays an important role in guiding prescribing practices such as antibiotic selection and dosing suggestions, alerting potential adverse drug reactions and drug allergies.
Two previous systematic reviews focused on the impact CDS on antibiotic use in primary care [7] and included non-computerised decision support.[8] Another more recent systematic review addressed a similar research question and examined the impact of HIT interventions on antimicrobial prescribing.[9] The scope, design and timing of these reviews may have excluded relevant CDS studies that match the inclusion criteria in this review.The aim of the present study was to evaluate the current state of evidence for CDS interventions on antibiotic use in the hospital inpatient setting.Meta-analysis was conducted using odds ratio to assess the impact of CDS on the adequacy of antibiotic coverage and mortality and to assess the impact of CDS, using relative differences, on length of stay, volume of antibiotic use, antimicrobial resistance and compliance with guidelines.

Data source and study selection
A systematic literature search was conducted utilising eight online databases including MED-LINE, EMBASE, PUBMED, Web of Science, CINAHL, Cochrane Library, HMIC, and Psy-cINFO.The search was conducted from inception to 20 th December, 2014.An updated literature search was conducted from January 1 st 2015 to October 1 st 2016 using PUBMED.The searches were conducted using a strategy based upon combinations of the following terms: (electronic prescribing) OR (clinical decision support) AND (antibiotic or antibacterial or antimicrobial) AND (hospital or secondary care or inpatient).The search strategy appears in S1 Appendix, PRISMA search strategy details.This was supported by use of a checklist S1 Checklist PRISMA 2009 checklist to ensure that PRISMA principles were followed during the process.
Titles and abstracts from retrieved references were examined by two reviewers (FA and CEC) to determine the potential inclusion eligibility.Full texts of potential studies were examined for eligibility against the review inclusion criteria.Bibliographies of retrieved articles and previous systematic reviews were examined to identify additional articles that could have been missed by this search strategy.

Inclusion and exclusion criteria
Criteria for inclusion in the systematic review were: (i) conducted by health care providers in inpatient or ICU or emergency (ED) settings (ii) the intervention involved CDS aimed at improving antibiotic prescribing at the point of care and (iii) the intervention was compared to no intervention, non-CDS intervention (non-electronic decision support) or to an intervention with CDS of different features.For the purpose of the systematic review, CDS was defined as a computer-based system designed to help directly in clinical decision making in which characteristics of individual patients are utilised to generate recommendations presented to clinicians at the point of care in a passive or active format such as alerts, reminders and guidelines.[10][11][12] Non-electronic CDS studies, non-hospital based studies, qualitative studies, case reports, case series studies, conference abstracts, commentaries, and letters, papers examining the performance of the system as opposed to its impact on antibiotic prescribing were excluded.In the case where a study had an unclear inclusion status, conflicts were resolved and consensus was reached by a third reviewer (JFM).

Data extraction and quality assessment
A custom data extraction form was created to match the specific needs of the review.Data related to study design, participants, interventions, comparators, outcomes, and main findings were extracted by one reviewer (FA) and confirmed by another (CEC).Disagreements were resolved by consensus, with a third investigator (JFM).When studies did not report sufficient data to allow pooling for meta-analysis, results were summarised qualitatively using relative differences.Email requests for additional data were made to authors of papers containing insufficient information to be included in the meta-analyses.
The quality of included studies was assessed using a 10-point rating scale previously used to evaluate CDS studies (see Table 1).[9,[13][14][15] The scale included five domains (2 points per domain): method of allocation of study groups, unit of allocation, presence of baseline differences between groups, objectivity of outcome measures, and completeness of follow-up for appropriate unit of analysis.Assessment of the methodological quality of the eligible studies was undertaken independently by two reviewers (FA, CEC).Reviewer disagreements were resolved by a third reviewer (JFM).

Data analysis and statistical analysis
A defined set of outcomes essential in estimating the effect of CDS in optimising antibiotic use shaped the synthesis process.Meta-analysis was conducted when studies evaluated the same outcome and had sufficient data to allow pooling.All studies were eligible for consideration for inclusion in the meta-analysis as all assessed the impact of CDS on antibiotic prescribing in the hospital inpatient setting.The meta-analysis focused on two outcomes: adequacy of antibiotic coverage (13 studies) and mortality (20 studies).Odds ratios and 95% confidence intervals (CIs) were calculated for each trial by reconstructing tables based on the number of patients randomly allocated and the number of patients with the outcome of interest.Interstudy variance was assessed using the Tau 2 test.Inter-study heterogeneity was assessed using the Chi 2 test and the I 2 statistics.An I 2 value higher than 75% was regarded as 'significant heterogeneity' and a value less than 40% was considered 'not significant heterogeneity'.Study results were considered statistically significant if the p value was below 0.05.Summary estimates were calculated by using the Mantel Haenszel random-effects model [16] in Reviewer Manager ((RevMan).Version 5.3.Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration, 2014).This enabled an estimate of variation between studies to be made by comparing study results with a fixed effect meta-analysis result.
The calculated heterogeneity of included studies and outcome assessment precluded pooling of data for some outcomes, in which case percentage mean difference analyses of such outcomes were conducted.To facilitate comparison across studies, units of volume of antibiotic use were converted to defined daily doses per 1000 patient-days (DDD/1000 patient-days), while units for drug costs were left in the currency of the country of origin.Compliance with antibiotic guidelines was measured by percentage mean differences between intervention and control groups and length of stay was measured by differences in days between intervention and control groups.

Search results
For this systematic review, the PRISMA statement was adopted [17] as detailed in Figure A in S1 File, PRISMA checklist highlighting study selection, which shows the results of the search and selection process.After screening 2459 studies, the removal of 237 duplicates between databases, the addition of 18 studies from bibliographies of included studies and previous systematic reviews, and the addition of 10 studies from the second updated PUBMED search, a total 378 full-text studies were reviewed.Of these, 297 studies did not meet inclusion criteria for the following reasons: were not conducted in secondary or tertiary care settings, did not answer research questions, or had inadequate study design.The characteristics and a bibliography of the 81 included studies are summarized in S1

CDS interventions
The classification of CDS interventions by Baysari and co-workers was adopted in the present review.[9] CDS interventions found in this systematic review took four main forms: (1) standalone computerised decision support systems (CDSSs), (2) decision support embedded within a hospital's electronic medical record (EMR) or computerized provider order entry (CPOE) system, (3) computerized antimicrobial approval systems, and (4) antibiotic surveillance systems.Interventions were evaluated against usual care, no CDS, paper-based decision support or CDS.

Quality of studies
This systematic review indicates that the current state of evidence for CDS in optimising antibiotic use is poor and is limited to non-Cochrane study designs as there were few randomised studies found in the literature.The majority of studies identified used before-and-after designs with very few including a control group.The included studies achieved an average score of 5.7 of a possible total of 10 on the rating scale.Random allocation of health care professionals, patients or units to a CDS intervention was rare.The majority of studies assessed an objective outcome measure (length of stay) or used a subjective measure with blinded assessment.

Main findings
Evidence for the impact of CDS on antibiotic use in hospital inpatient settings has been reviewed systematically.Almost half of the studies included in the present systematic review did not appear in previous systematic reviews.This highlights the pace of the introduction and evaluation of health information technology in hospital settings.Therefore, this systematic review extends previous evidence, including studies never evaluated previously.
Studies were extremely variable in the types of CDS interventions and in the outcomes assessed.The most commonly assessed outcomes were mortality, length of stay, volume of antibiotic use and adequacy of antibiotic coverage.Other outcomes assessed included system uptake, antimicrobial resistance, cost and compliance with guidelines.Only a small number of studies of this systematic review assessed health outcomes (mortality and adequacy of antibiotic coverage) which may limit the strength of evidence needed to reflect on CDS design, selection and implementation.
The principal findings of the meta-analysis indicate evidence that some studies of CDS interventions were associated with improvements in adequacy of antibiotic coverage (by more than 100%) and patient mortality (reducing the risk of death by about 15%).However, these findings were likely to be driven by data from poor quality studies.Increases in compliance with guidelines have been noted in the present review.Drawing conclusions about the effects of CDS on length of stay and cost of antibiotics is difficult since results from the present review are conflicting.A meta-analysis by Baysari and co-workers (9) showed similar findings of the impact of CDS interventions on adequacy of antibiotic coverage The current systematic review indicated conflicting results on CDS uptake as some studies showed improved uptake while other showed poor adoption.A study by Demonchy and coworkers [19], has highlighted uptake and implementation issues of CDS as a major barrier.The impact of CDS interventions would have been greater if used regularly by prescribers.

Strengths
This systematic review provides a comprehensive, up-to-date overview of CDS interventions aimed at optimising antibiotic use in the hospital inpatient setting.A wide range of outcome measures was assessed including outcomes that have not been previously evaluated, such as cost, system uptake and antimicrobial resistance.It is noteworthy that non-randomised designs have been commonly utilised in evaluations of health informatics developments, as evidenced by this systematic review.The present review included studies that have not been included in other systematic reviews.
Given that the quality and study design of included studies were generally poor and the heterogeneity in respect of study quality and end points, the synthesis of the studies was problematic.However, it was possible to conduct meta-analysis and subgroup analysis which adds to the strength of this review.

Limitations and future research
The present systematic review is limited by the quality of studies included for analysis coupled with limitations inherent in the applied methods.All studies were eligible for inclusion in the meta-analysis but information contained in studies enabled meta-analysis to be conducted for two outcomes: adequacy of antibiotic coverage (n = 13 studies) and mortality (n = 20 studies).The number of studies that reported other outcome measures (e.g.volume of antibiotic use and cost) in a uniform way was not sufficient for other meta-analysis to be conducted.
Heterogeneity in study designs, CDS interventions, outcomes, implementation and contextual factors make it difficult to reach firm conclusions about the impact of CDS.Subgroup analysis was not successful in explaining or even reducing heterogeneity across subgroups.This indicates that heterogeneity was inherent in poor methodological and intervention designs.
There is a possibility that selective reporting may reduce the validity of some of the conclusions.A marginal reduction of mortality is a key finding of from this systematic review; however, this finding is based on a limited number of studies (n = 20).Selective reporting could not be controlled, as it is not clear how many studies that might have found an increase in mortality it would take to nullify or even reverse the findings here.Therefore, an assurance that there is no risk of an increase in mortality is not possible.Caution needs to be applied with regards to the possibility of publication bias or evaluation by developers.It should be clear that an external evaluation should be reported using accepted mixed methods research.There is a lack of literature about the impact of CPOE without explicit CDS from commercial vendors: this may be due to publication bias.
Future work should include conducting high quality systematic multi-site comparative studies of different CDS interventions for antibiotic prescribing.More qualitative work is required to highlight the barriers and facilitators of adopting CDS technology and better understanding users' perceptions and attitudes towards CDS interventions to trigger high adoption and uptake by providers.

Conclusion
This review indicates that CDS interventions can be effective in optimising antibiotic use in hospitals.The findings of this review can be used to enrich the debate around the impact of CDS on antibiotic optimisation.This review demonstrates the efficacy of CDS in optimising the adequacy of antibiotic coverage across different settings.However, evidence on the effect of CDS on clinical outcomes, economic outcomes and volume of antibiotic use was limited.CDS appears to be safe because the present review has not shown any significant risks such as worsening mortality or length of stay.CDS presents a promising future for optimising antibiotic use and improving patient care.However, in order to reach firm conclusions about the impact of CDS on antibiotic use, more high quality studies are needed within different settings and in different health systems.

Table 1 . 1 .
Detail of the 10-point Quality Assessment Scale used in the present study.Method of allocation of study groups 2 = Random, 1 = Quasi-random, ) = Selected concurrent controls 2. Unit of allocation 2 = Cluster (e.g.practice), 1 = Physician, 0 = Patients 3. Presence of baseline differences between groups 2 = No baseline differences present or appropriate statistical adjustments made 1 = Baseline differences present and no statistical adjustment made 0 = Baseline characteristics not reported 4. Objectivity of outcome measures 2 = Objective outcomes or subjective outcomes with blinded assessment 1 = Subjective outcomes with no blinding but clearly defined assessment criteria 0 = Subjective outcomes with no blinding and poorly defined assessment 5. Completeness of follow up for appropriate unit of analysis 2 = >90%, 1 = 80-90%, 0 = <80% https://doi.org/10.1371/journal.pone.0183062.t001antibiotic coverage, 15 assessed CDS uptake and use, 15 assessed cost of antibiotics, 10 assessed compliance with guidelines, and 4 assessed antimicrobial resistance.The majority of studies were conducted in the United States (45 studies).
Individual and pooled estimates are shown in Figure B in S1 File, Forest plot from individual studies and meta-analysis for adequacy of antibiotic coverage.).Overall, CDS interventions were associated with an increase in adequacy of antibiotic coverage based on the random effects model [OR = 2.11, 95% CI, 1.67 to 2.66, p < 0.00001].There was evidence of heterogeneity between studies (Chi 2 = 55.85,df = 15, I 2 = 73%, p < 0.00001) (Figure B in S1 File Forest plot from individual studies and meta-analysis for adequacy of antibiotic coverage.There was evidence of an effect of CDS interventions on the adequacy of antibiotic coverage for Cochrane compliant studies [OR = 1.47, 95% CI, 1.03 to 2.10, p = 0.03], and for non-Cochrane studies [OR = 2.18, 95% CI, 1.69 to 2.80, p < 0.00001] (Figure B in S1 File Forest plot from individual studies and metaanalysis for adequacy of antibiotic coverage).
Individual and pooled estimates are shown in Figure C in S1 File, Forest plot from individual studies and meta-analysis for mortality.Overall, results showed that CDS interventions had a marginal statistically significant effect on mortality based on the random effects model.[OR = 0.85, 95% CI, 0.75 to 0.96, p = 0.01].There was evidence of heterogeneity between studies (Chi 2 = 42.37,df = 20, I 2 = 53%, p = 0.01).There was no evidence of an effect of CDS interventions on mortality for Cochrane compliant studies (N = 5) [OR = 0.88, 95% CI, 0.75 to 1.04, p = 0.13].Based on non-Cochrane studies (N = 16), there was a marginal statistically significant effect of CDS interventions on mortality [OR = 0.84, 95% CI, 0.71 to 0.99, p = 0.04] (Figure C in S1 File, Forest plot from individual studies and meta-analysis for mortality.).
Table, Characteristics of included studies (Table A).References for included studies (List A).Twenty-six studies assessed mortality, 25 assessed length of stay, 19 assessed volume of antibiotic usage, 16 assessed adequacy of

Table 3 . Length of stay associated with CDS implementation. Study Length of stay in non-intervention group Length of stay in intervention group Difference P value
LRMs, local resistance maps; PMRTRs, preliminary microbiological reports with therapeutic recommendations; N/A, not reported; N/A, not stated https://doi.org/10.1371/journal.pone.0183062.t003

Table 4 . Antibiotic cost reductions associated with CDS interventions. Study Unit of measurement Antibiotic cost in non-intervention group Antibiotic cost in intervention group Difference P value
https://doi.org/10.1371/journal.pone.0183062.t004

Table 5 . Compliance associated with CDS. Study Compliance in non-intervention group Compliance in intervention group Difference P value
N/A, not reported; CDS, clinical decision support https://doi.org/10.1371/journal.pone.0183062.t005