Antibiotic prophylaxis in orthognathic surgery: A complex systematic review

Objective In orthognathic surgery, antibiotics are prescribed to reduce the risk of postoperative infection. However, there is lack of consensus over the appropriate drug, the dose and duration of administration. The aim of this complex systematic review was to assess the effect of antibiotics on postoperative infections in orthognathic surgery. Methods Both systematic reviews and primary studies were assessed. Medline (OVID), The Cochrane Library (Wiley) and EMBASE (embase.com), PubMed (non-indexed articles) and Health Technology Assessment (HTA) publications were searched. The primary studies were assessed using GRADE and the systematic reviews by AMSTAR. Results Screening of abstracts yielded 6 systematic reviews and 36 primary studies warranting full text scrutiny. In total,14 primary studies were assessed for risk of bias. Assessment of the included systematic reviews identified two studies with a moderate risk of bias, due to inclusion in the meta-analyses of primary studies with a high risk of bias. Quality assessment of the primary studies disclosed one with a moderate risk of bias and one with a low risk. The former compared a single dose of antibiotic with 24 hour prophylaxis using the same antibiotic; the latter compared oral and intravenous administration of antibiotics. Given the limited number of acceptable studies, no statistical analysis was undertaken, as it was unlikely to contribute any relevant information. Conclusion With respect to antibiotic prophylaxis in orthognathic surgery, most of the studies to date have been poorly conducted and reported. Thus scientific uncertainty remains as to the preferred antibiotic and the optimal duration of administration.

Introduction Antibiotic prophylaxis to reduce the risk of postoperative infection after orthognathic surgery is a broadly accepted practice. However, there is lack of consensus with respect to the preferred antibiotic and the dose and duration of administration: the available primary and composite data are contradictory [1][2][3][4]. Because of increasing antibiotic resistance, it is important to assess the potential benefits and risks associated with administration of antibiotics. The outcomes of numerous treatment procedures are dependent on effective infection control and treatment, hence antibiotic resistance poses a major threat to modern healthcare, [5]. In some geographic areas antibiotics can no longer be relied on, indicating a post-antibiotic era [6]. The development of antibiotic resistance is closely related to the extent of prescription and rational antibiotic usage is an increasingly important preventive measure [7][8][9]. The benefit of restrictive antibiotic policies is also evident in low consumption environments, where the levels of resistance are relatively low [10]. It has been shown that short-term use of antibiotics, even a single dose, may select for resistant viridans streptococci. Thus short-term utilisation, such as antibiotic prophylaxis, also needs to be reassessed [11,12].
A literature review on antibiotic prophylaxis in surgery conducted by The Swedish Council on Health Technology Assessment (SBU) concluded that prolonging antibiotic prophylaxis beyond the day of surgery provides no further benefit [13]. Whether this also applies to orthognathic surgery has not been determined.
Systematic reviews evaluate and summarize the state of knowledge on a defined topic. The methodology is strictly defined, with a reproducible literature search, independent literature analyses and meticulous statistical calculations [14][15][16]. Failure to adhere to the recommended approach may have untoward consequences: an incorrect conclusion may be drawn, giving the impression of solid evidence. Therefore assessment of primary studies as well as systematic reviews is equally important [17,18]. A complex systematic review involves quality assessment not only of systematic reviews, but also of the original research. This approach is recommended where reviews published to date show inconsistent results [19].
The aim of the present study was to compare the efficacy of short-and long-term antibiotic prophylaxis in orthognathic surgery, by means of a complex systematic review of the available scientific literature.

Objective
The objective of the study was to assess the effect of antibiotics on postoperative infections in orthognathic surgery. two different antibiotic protocols, or comparing antibiotics with alternative treatments, such as antibacterial rinsing, were also accepted. Exclusion criteria for systematic reviews were a more recent systematic review available by same author, non-systematic review, guidelines, letters, position papers, and consensus statements. Table 1 presents a summary of the predefined study population, intervention, comparison of therapies and the outcome parameters in the eligible studies.

Search strategies
The initial literature search was undertaken by two of the authors (ANA and AK) and two information specialists at the Karolinska Institutet University Library. The following databases were searched up to October 20, 2015: Medline (OVID), The Cochrane Library (Wiley) and EMBASE (embase.com), PubMed (non-indexed articles). The search terms used for the various databases are summarised in Table 2. The search was initially unfiltered, for the primary studies, then repeated, with a filter, for systematic reviews. Publications by the following Health Technology Assessment (HTA) organisations were searched until October 30, 2015,  The reference lists of all the eligible studies were hand-searched for potential complementary trials. Although there was no restriction according to language, retrieved papers in a language other than English were excluded. In order to detect more recent publications, complementary searches were undertaken in PubMed on November 24, 2016 and again on October 31, 2017. These additional searches did not use any filters and all new findings from both primary studies as well as systematic reviews were screened.

Study selection
Eligible studies were selected according to inclusion/exclusion criteria. ANA (first author) then screened the retrieved list, for initial exclusion of irrelevant publications, based on title. In case of uncertainty, a study was retained until the next selection step: examination of abstracts. The abstracts were read independently, in duplicate, by three reviewers, BL, AK and ST or MH, ANA and BK. Selected primary studies and systematic reviews were read in full-text, in duplicate, by three reviewers each, respectively. Any disagreement during the screening process was resolved by discussion in the project group. Studies excluded at this stage, and the reason for exclusion, were recorded.

Quality assessment
Systematic reviews (SR). The level of bias for systematic reviews was assessed by AMSTAR [20][21][22]. The reviews were classified as having low, moderate or high risk of bias according to the criteria shown in Table 3.
Primary studies. The quality of the included primary studies was assessed according to a protocol for assessment of randomized studies [23].

Quality of evidence
The scientific quality of the evidence in the primary studies was graded according to GRADE, as high, moderate, low, and very low [24], Table 4. Table 3. Criteria for low, moderate and high risk of bias according to Mejàre, 2015, based on AMSTAR assessment items.

Low
Predetermined research question and inclusion criteria established. 1 A minimum of two independent data extractors and consensus procedure reported. 2 At least the database MEDLINE/PubMed used. Reproducible search strategy reported. 3 Included and excluded studies reported. Ã 4 Relevant characteristics of included studies stated. 5 Scientific quality assessed and reported for each included study. 6 Alignment between scientific quality of included studies and formulating conclusions. 7 Rational and methods for pooling results reported. 8 Publication bias estimated. Can be omitted if publication bias was unlikely and not reported. 9 Conflict of interest stated. This item can be omitted if conflicts of interest were unlikely. 10

Moderate
A yes-answer to question 1-7 required.

High
A no-answer to any of the questions listed under moderate risk of bias (except question 1). Data extraction Systematic reviews. Data extracted from the systematic reviews covered objectives, main results, authors' estimated level of evidence, and knowledgegaps according to authors.
Primary studies. The following data were extracted from the primary studies: number of patients included, age, gender distribution, length of follow-up, type of intervention and type of control treatment and relevant treatment outcomes.

Assessment of publication bias and heterogeneity (primary studies)
Not applicable: there were insufficient studies for meta-analysis.

Literature search and study selection
The search for systematic reviews yielded 50 articles: 43 remained after exclusion of duplicates. The search strategy for primary studies yielded 470 articles: after exclusion of duplicates, 372 remained. The number of studies retrieved from each data base search is shown in Table 2. The search of publications by HTA organisations failed to identify any further studies. Flowcharts of the screening process for systematic reviews and primary studies are described in Figs 1 and 2, respectively.
The additional search on November 24, 2016 added 56 new articles and the search on October 31, 2017 added 38 new articles: all were screened for both systematic reviews and primary studies.
Systematic reviews. Screening of abstracts yielded six SR warranting retrieval of the fulltext for scrutiny. The additional searches did not yield any further systematic reviews for inclusion.
Primary studies. In all, 36 primary studies were read in full-text, 23 of which were excluded, leaving 13primary studies eligible for quality assessment. Table 5 presents primary studies read in full text but then excluded from quality assessment, along with the reason for exclusion. The additional search on November 24, 2016 yielded one more study to undergo quality assessment. Thus in all, 14 studies were assessed.

Quality assessment and data extraction of systematic reviews and metaanalyses
The quality assessment of the included systematic reviews identified two studies assessed as having a moderate risk of bias because of the inclusion in the meta-analyses of primary studies at high or unclear risk of bias [1,25], see Table 6. Four systematic reviews were considered to be at high risk of bias [2,3,26,27] (Table 7). The main shortcomings were inadequate We are very confident that the true effect lies close to that of the estimate of the effect

Moderate
We are moderately confident in the effect estimate: The true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different

Low
Our confidence in the effect estimate is limited: The true effect may be substantially different from the estimate of the effect

Very low
We have very little confidence in the effect estimate: The true effect is likely to be substantially different from the estimate of effect https://doi.org/10.1371/journal.pone.0191161.t004 Antibiotic prophylaxis in orthognathic surgery: A complex systematic review  consideration of quality assessment when formulating conclusions (n = 4), inadequate or unclear search strategy (n = 3), failure to include the characteristics and results of included studies (n = 3), study selection and data extraction had not been undertaken by two independent reviewers (n = 2), and failure to account for included and excluded studies (n = 2). No studies were considered to be at low risk of bias.

Quality assessment and data extraction of primary studies
The quality assessment identified one study with a moderate risk of bias [28] (Lindeboom et al. 2003) and one with a low risk of bias [29]  . A high risk of bias was found in 12 studies. The study with a moderate risk of bias compared a single dose of antibiotic with 24 hour prophylaxis using the same compound. The study with a low risk of bias compared oral and intravenous administration of antibiotics ( Table 8). The type of intervention and study population characteristics in studies classified as being at low or moderate risk of bias are shown in Table 8. The most common reason for high risk of bias was a poorly described, unclear randomization process (Table 9). Table 9 shows the eligible studies excluded due to a high risk of bias. The outcomes of the included primary studies with low or moderate risk of bias are presented in Table 10. Fig 3 presents a summary of the quality assessment of the included studies with low or moderate risk of bias. Because of the limited material, no statistical analysis was undertaken.

Discussion
This complex systematic review of prophylactic antibiotics in orthognathic surgery identified no systematic reviews at low risk of bias and two at moderate risk. The latter had somewhat

Brignardello-Petersenet al. 2015
Assess the effect of antibiotic prophylaxis in orthognathic surgery for prevention of SSI.
1) Long-term antibiotic prophylaxis probably reduces the risk of SSI.
2) Effects of shortterm versus single dose prophylaxis are uncertain.n = 11 1) Moderate quality of evidence for long-term prophylaxis 2) Low quality of evidence for short-term prophylaxis Antibiotics seem beneficial for reducing SSI.Preferred type of compound unknown.
Moderate.Studies with high risk of bias included in the meta-analyses.

Tan et al.2011
Investigate the efficacy of prophylactic antibiotics for prevention of SSI after orthognathic surgery.
1) Antibiotics reduce rate of SSI.
2) No significant difference between long-and short-term prophylaxis.n = 5 Low risk of bias concerning blinding (n = 4) and sequence generation (n = 4). Unclear risk for other sources of bias (n = 5).
Antibiotic prophylaxis significantly reduces SSI.
No further benefit of extended antibiotic prophylaxis.
ModerateStudies with high risk of bias included in meta-analyses.
Meta-analysis fixed model used despite heterogeneous studies discrepant results. Although both studies concluded that the use of antibiotics in orthognathic surgery was beneficial, Tan et al. concluded that prolonging antibiotic cover offered no further benefit and Brignardello-Petersen suggested the opposite. The main shortcoming in both of these systematic reviews is the inclusion in their metaanalyses of studies at either high risk [24], or unclear risk of bias [1]. There is an implied risk that the scientific shortcomings of the included primary studies may unduly influence the major conclusions of the systematic review. As the reader of the systematic review generally accepts the author's assessments of the included primary studies, such misuse of statistics is particularly problematic. The experienced reader can fairly easily detect the inclusion in the meta-analysis of primary studies with high risk of bias. Applying the description "unclear risk of bias" when lack of information about material and methods precludes assessment of scientific quality is per se acceptable. However, such studies should not be included in the metaanalysis as equivalent to studies at either low or moderate risk of bias: this is speculative and misleading. Such a systematic review is clearly at moderate to high risk of bias. In summary, Table 7. Systematic reviews excluded due to high risk of bias.

Reference
No answer to AMSTAR question number  Antibiotic prophylaxis in orthognathic surgery: A complex systematic review this highlights the importance of meticulous and stringent quality assessment of reviews before drawing conclusions, especially when the conclusions are relevant to clinical practice. The review of primary studies identified two studies, with different aims, at low or moderate risk of bias. These two studies have also been included in previous systematic reviews presented in this complex systematic review. There were no significant differences between any of the groups in the included studies, and the data were so limited that no conclusions should be drawn about potential beneficial effects in any treatment group. The differing aims of the two studies precluded meta-analysis.
A disappointing finding in this complex systematic review is that twelve studies could have been included in the results if they had met the required standards for randomized controlled Unclear randomization process and blinding Jansisyanont P, Sessirisombat S, Sastravaha P, Bamroong P. Antibiotic prophylaxis for orthognathic surgery: a prospective, comparative, randomized study between amoxicillin-clavulanic acid and penicillin. Antibiotic prophylaxis in orthognathic surgery: A complex systematic review trials. Despite the availability of guidelines for reporting RCTs, suboptimal study design, methods, and interpretation of results emerged as major weaknesses [30]. The findings presented following these shortcomings might not be guidelines for clinicians making decisions about the proper use of antibiotics in conjunction with orthognathic surgery, but more of a signal to researchers about the proper conduct of clinical trials and adherence to guidelines when reporting their results. Failure to adhere to a proper study protocol may also raise ethical issues, implying not only that the inexperienced reader might be misled by the results and that economic resources were being used inappropriately, but also that the subjects had undergone participation in a trial which lacked the potential to improve scientific knowledge.

Conclusion
In the field of orthognathic surgery, most studies of antibiotic prophylaxis to date have been poorly performed and reported. Scientific uncertainty remains with respect to the preferred antibiotic compound and the optimal range of the prophylaxis.