https://orcid.org/0000-0002-9749-8372
Figures
Abstract
Anxiety disorders, including panic disorder (PD), generalized anxiety disorder (GAD), social anxiety disorder (SAD), agoraphobia, and specific phobia, are among the most common psychiatric disorders. Although the traditional pharmacologic treatments for anxiety included barbiturates and then benzodiazepines, the introduction of tricyclic antidepressants, followed by the selective serotonin reuptake inhibitors (SSRIs), marked a tidal shift in the treatment of anxiety. Although not approved for treatment of anxiety disorders (with the exception of trifluoperazine) there is ongoing off-label, unapproved use of both first-generation “typical” antipsychotics (FGAs) and second-generation or “atypical” antipsychotics (SGAs) for anxiety. Although there have been systematic reviews and meta-analyses on the use of antipsychotics in anxiety disorders, most of these reviews focused on SGAs, primarily the use of quetiapine in GAD. Given that there is little known about the potential benefits and short-and long-term risks of using antipsychotics in anxiety, there is a need for an umbrella review of systematic reviews and meta-analyses of the use of both FGAs and SGAs in anxiety disorders. The specific aims of this study are as follows: (1) Evaluate the evidence of efficacy of FGAs and SGAs in anxiety disorders as an adjunctive treatment to SSRIs, serotonin norepinephrine reuptake inhibitors (SNRIs) and other non-antipsychotic medications; (2) Compare monotherapy with antipsychotics to first-line treatments for anxiety disorders in terms of effectiveness, risks, and side effects; and (3) Evaluate the short- and long-term risks and side effects of prescribing antipsychotics in anxiety disorders. The review is registered on PROSPERO (CRD42021237436). Since data extraction has not begun, there is not preliminary data to share.
Citation: Garakani A, Freire RC, Buono FD, Thom RP, Larkin K, Funaro MC, et al. (2022) An umbrella review on the use of antipsychotics in anxiety disorders: A registered report protocol. PLoS ONE 17(6): e0269772. https://doi.org/10.1371/journal.pone.0269772
Editor: Tina Hinton, University of Sydney, AUSTRALIA
Received: March 19, 2021; Accepted: May 27, 2022; Published: June 16, 2022
Copyright: © 2022 Garakani et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: This is a research review protocol, and no data have been collected yet, although when collected, all research data will be made accessible through the Figshare repository.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have read the journal’s policy and have the following competing interests: RT is receiving research support from the Williams Syndrome Association, Precidiag, Harvard Medical School Dupont Warren Fellowship, Louis V. Gerstner III Research Scholar Award, the American Academy of Child and Adolescent Psychiatry, and the Jerome Lejeune Foundation outside of the current study. MPR has received research grant funding from Neurocrine Biosciences, Inc., Millennium Pharmaceuticals, Takeda, Merck, and AI Cure; she is also an Advisory Board member for Neurocrine Biosciences, Inc. and Alkermes, Inc. outside of the current study. RF received research grant funding from Biohaven Pharmaceuticals outside of the current study. This does not alter our adherence to PLOS ONE policies on sharing data and materials. There are no patents, products in development or marketed products associated with this research to declare.
Introduction
Anxiety disorders represent the most common psychiatric disorders by prevalence in the United States [1]. In the DSM-5, the primary anxiety disorders affecting adults include panic disorder (PD), generalized anxiety disorder (GAD), social anxiety disorder (SAD), agoraphobia, and specific phobia. Although the traditional pharmacologic treatments for anxiety included barbiturates and then benzodiazepines, the introduction of tricyclic antidepressants, followed by the selective serotonin reuptake inhibitors (SSRIs), marked a tidal shift in the treatment of anxiety. Treatment guidelines recommend, as first-line pharmacotherapy, the use of SSRIs or venlafaxine for PD, GAD and SAD [2–4]. Although the use of benzodiazepines continue to be an option, there have been concerns raised about their long-term use, tolerance, dependence, and abuse potential, along with risks of falls; additionally, while there was initial evidence of dementia with chronic benzodiazepine use, a recent nationwide cohort and nesting case-control study of 235,465 patients reported no association between benzodiazepines and subsequent dementia, and allowed for speculation that they may even have protective effects [5]. As a result of negative perceptions, benzodiazepine prescribing has fallen out of favor, despite their clear efficacy especially early in treatment and as an as-needed option for anxiety [6].
Given the efficacy of psychotherapies such as cognitive behavioral therapy (CBT) and exposure therapy, as well as medications such as SSRIs, serotonin norepinephrine reuptake inhibitors (SNRIs), tricyclic antidepressants, pregabalin, and benzodiazepines, there would be ample therapeutic options for anxiety disorders. Despite the presence of treatment options, there has been limited research done on the management of patients with treatment-resistant or refractory anxiety [7]. Such patients are often considered for SNRIs, monoamine oxidase inhibitors (MAOs), other antidepressants, or combination treatments with other anxiolytic agents like the azapirones (e.g., buspirone). In spite of these therapeutic options, between 15–40% of patients with anxiety disorders do not respond (50% improvement to these first-line treatments) and roughly half of patients do not recover (defined as having minimal anxiety symptoms) [7]. There are also high rates of comorbidity between anxiety disorders and mood disorders [8], particularly major depressive disorder (MDD), and evidence that these patients are poorer responders to antidepressant treatment and have greater symptom severity including higher rates of suicide [9]. Antipsychotics represent a class of medications that have come under increased study for non-psychotic disorders over the years. The first-generation “typical” antipsychotics (FGAs) were prescribed primarily for schizophrenia but also were used for bipolar mania, agitation (psychotic and non-psychotic), sleep induction and Tourette’s syndrome. One antipsychotic, trifluoperazine, was also approved by the US Food and Drug Administration (FDA) in 2001 for non-psychotic anxiety [10], supported by a large randomized-double-blind, placebo-controlled study in 1986 [11]. As of 2020, there are no second-generation or “atypical” antipsychotics (SGAs) approved for any of the anxiety disorders. There is, however, ongoing use of both FGAs and SGAs for disorders other than psychotic and bipolar disorders, which is considered “off-label” (i.e., not approved by FDA or equivalent agency) [12,13]. Data collected on inpatients and outpatients in an academic setting suggest that prescriptions of antipsychotics for anxiety and anxiety disorders is common [14]. Previous systematic reviews report that there is evidence, in the form of randomized, double-blind, placebo-controlled trials, for the use of some SGAs in certain anxiety disorders [15,16]. Canadian guidelines [4] recommend antipsychotics only as a third-line option or adjunctive therapy for anxiety disorders, which may include olanzapine, quetiapine, risperidone, or aripiprazole, depending on the anxiety disorder. There is limited to poor support for the use of SGAs in PD [17]. These reviews suggest that the evidence is strongest for the use of quetiapine in GAD, but caution is advised given the poor tolerability and potential for serious adverse effects [18–20].
To date, despite the absence of a formal approval, antipsychotics continue to be prescribed off-label for various indications including insomnia and anxiety. There may be increasing use of antipsychotics in anxiety in cases where patients have been refractory to first- or second-line treatment options. The Canadian Network for Mood and Anxiety Treatments (CANMAT) Guidelines recommend SGAs both as treatment for bipolar disorder and (mostly as adjunctive for) MDD [21,22]. Patients with anxiety disorders and comorbidity with bipolar disorder or MDD are likely to receive SGAs as part of their treatment because of these recommendations. In addition, inferences that medications considered effective for MDD are also effective for anxiety disorders can increase the prescription of SGAs for patients with anxiety. The concern arises that there may not be sufficient evidence to support the use of this class of medications and there may not be sufficient consideration given to the short- and long-term risks of use of these medications, especially in vulnerable populations (e.g., children, adolescents, and those who are medically ill or immunocompromised, elderly and cognitively impaired). Clinically, medical providers prescribing antipsychotics for anxiety may choose to do so in cases where patients have not responded to first- and second-line treatments or have refractory symptoms, as long as they provide psychoeducation about the risks and benefits and potential adverse reactions such as tardive dyskinesia, neuroleptic malignant syndrome, metabolic syndrome (weight gain, elevated lipids, development of diabetes), extrapyramidal symptoms, and cardiac effects including arrhythmias and sudden death. Patients given such information may choose to forego treatment with SGAs or FGAs. That said, there is a widespread use of antipsychotics in primary care, less than a half of the prescriptions are for bipolar disorder or psychotic disorders [13]. A significant portion of antipsychotic prescriptions in primary care is for anxiety disorders [13], SGAs and FGAs are also prescribed for anxiety in other clinical settings. The concern arises about the risk of prescribing such medications to patients long-term and in particular higher-risk populations like children, those medically ill and the elderly. A recent study reported greater mortality in patients with dementia receiving both antipsychotics and benzodiazepines [23] and in patients with schizophrenia receiving this drug combination [24]. There is even evidence for increased mortality risk in otherwise-healthy adult patients with depression receiving antipsychotic augmentation of their antidepressant [25]. Additionally, the rates of tardive dyskinesia, while lower with SGAs (21%) compared to FGAs (30%), still remain high for such a serious and irreversible drug reaction [26].
There is a strong need for a review of the literature on the use of both FGAs and SGAs in anxiety disorders, as most of the current systematic reviews and meta-analyses exclude first-generation antipsychotics [15–20], with a focus on the most common DSM-5 anxiety disorders affecting adults: PD, agoraphobia, specific phobias, GAD and SAD.
Objectives
The objective of the current study is to conduct an umbrella review of systematic reviews and meta-analyses to evaluate the efficacy and safety of first and second-generation antipsychotics for the treatment of panic disorder, generalized anxiety disorder, specific phobia, social anxiety disorder (social phobia) and agoraphobia.
Specific aims.
Specific aim 1: Is there evidence of efficacy of first- and second-generation antipsychotics in anxiety disorders as an adjunctive treatment to SSRIs, SNRIs and other non-antipsychotic medications?
Specific aim 2: How does monotherapy with antipsychotics compare to first-line treatments for anxiety disorders (effectiveness, risks, and side effects)?
Specific aim 3: What are the short- and long-term risks and side effects of prescribing antipsychotics in anxiety disorders?
Methods
The umbrella review protocol is informed by the framework described by the Joanna Briggs Institute [27]. In addition, the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines will be used throughout the process [28]. A Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) checklist has been completed [29] (see S1 Checklist). This umbrella review protocol has been registered with the International Prospective Register of Systematic Reviews (PROSPERO) (registration number CRD42021237436).
Inclusion and exclusion criteria
Reviews will be considered even if they are part of a review of the use of antipsychotics in non-anxiety disorders or if the reviews are part of a larger study of pharmacological treatments of anxiety disorders. Reviews of antipsychotic monotherapy in anxiety disorders will be included. Reviews including adjunctive treatments of an antipsychotic with other medications (such as SSRIs, SNRIs, benzodiazepines, etc.) in anxiety disorders will be included as this reflects “real-life” clinical practice. Systematic reviews of adults over the age of 18 years will be included. Reviews in peer-reviewed journals in the English language will be included.
Reviews of use of antipsychotics in anxiety disorders comorbid with other disorders (such as depressive disorders, bipolar disorder, schizophrenia, substance use disorders, medical illnesses, etc.) will be excluded if the anxiety disorder is a secondary diagnosis. Reviews will be included if they comprise patients with comorbidities, but the anxiety disorder is the primary diagnosis. Reviews in children and adolescents under the age of 18 years will be excluded. Non-systematic reviews or narrative reviews or opinion-editorial pieces will be excluded. Reviews in non-peer-reviewed journals that are not in the English language will be excluded.
Information sources
A systematic search of the following databases will be completed: PubMed, MEDLINE, EMBASE, APA PsycInfo, CINAHL Complete; and the Cochrane Library from inception onwards. The search will be limited by study design using a search filter to identify systematic reviews and meta-analyses in MEDLINE, Embase, CINAHL, and APA PsycInfo following the SIGN recommended search terms for systematic reviews and meta-analyses [30]. Searches will be limited to the English language. No date limits will be imposed on the search.
An experienced medical librarian (MCF) will be consulted on methodology. A medical subject heading (MeSH) analysis of known key articles provided by the research team [mesh.med.yale.edu] will be done and scoping searches performed in each database. An iterative process will be used to translate and refine the searches. To maximize sensitivity, the formal search will use controlled vocabulary terms and synonymous free-text words to capture the concepts of “anxiety disorders” and “antipsychotic agents.” The search strategy will be peer reviewed by a second librarian, not otherwise associated with the project, using the PRESS standard. A draft MEDLINE search strategy is included in Table 1. Reviewers will check for additional relevant cited and citing articles using included studies. To capture recently published articles, a second database search will be rerun before publishing the paper.
Study selection
Search results will be pooled in EndNote 20 and de-duplicated [www.endnote.com]. This set will be uploaded to Covidence [www.covidence.org] for screening.
Two screeners will independently review the titles, abstracts and full-text of the eligible articles using Covidence. The studies will be graded on the measure of either having criteria “met” or “not met” with the latter studies being excluded. For cases where there is a lack of certainty, there will be an option for “not clear.” These studies will then be assessed by the third reviewer (AG) and a consensus will be reached after discussion between the reviewers after the initial review. Studies will be selected or excluded after a full-text review by two board-certified psychiatrists (AG and RCF) who are blinded to each other and working independently. Any discrepancy will be resolved by discussion between the two reviewers, or with a third person (RPT) if consensus cannot be reached.
Data collection
Data on study characteristics will be extracted independently, by two reviewers, using a standardized, pre-piloted JBI data extraction form. Disagreements will be resolved through discussion with a third reviewer to reach consensus. Reviewers will extract quantitative and qualitative data from each selected study, grouped by the type of intervention. Findings will be presented in tabular format with supporting text. Quantitative tabulation of results will include: first author name and year of publication; setting; characteristics of the study population; number of RCTs included in the systematic review; findings related to the intervention/s and comparator regimens used; the outcomes assessed.
To reduce bias, the methodological quality of the systematic reviews and meta-analysis will be evaluated using the AMSTAR-2 (A MeaSurement Tool to Assess systematic Reviews) scale [31], and a global score will be computed awarding a point for each domain that can be rated as yes/partially yes. We will also extract data about the funding of each systematic review/meta-analysis and the conflicts of interest of the authors, focused on any potential financial benefits associated to the intervention.
Data analysis
Each systematic review and meta-analysis will be re-analyzed with a random effects model, including estimations of heterogeneity (τ2 and I2) and prediction intervals. The unit of analysis will be conducted through qualitative review across the reviewers, a standard and validated approach [32]. Small study effects will be examined with the Egger test for each systematic review and meta-analysis with over 10 studies [33]. We will compute estimations of the aggregated effect which can be regarded as free from small sample bias by extrapolating the Egger regression line on a funnel plot to a theoretical study of infinite sample size and a standard error of 0 [34]. To ensure no overlapping reviews are in this analysis, we will implement a corrected covered analysis [35]. A CCA will be measure the overlap by the frequency of repeated occurrences of the indexed publication, to evaluate possible range of CCA for each review. We will also apply the excess significance test, which quantifies whether the observed number of studies with statistically significant results (“positive studies") differs from the expected number of positive studies, considering the estimates of statistical power for each study included in the systematic review and meta-analysis [36–38]. If articles are unpublished at the point of extraction, we will implement a Bayesian approach to account for uncertainty in the analysis. Using a procedure adapted from existent umbrella reviews [32], we will identify outcomes and interventions for which summary effects showed strong evidence of significance (p<0.001); were based on more than 1000 randomized participants; did not have large heterogeneity between studies (I2 <75%); their 95% prediction interval excluded the null; extrapolation of the Egger regression line for infinite sample size still suggested a nominally significant effect (i.e., 95% confidence interval did not include the null); had no evidence for excess significance; came from systematic reviews and meta-analysis with a score of at least 9 on the AMSTAR-2 scale; came from systematic reviews and meta-analysis not funded by the entity developing or marketing the intervention and free from authors with a financial conflict of interest associated to the intervention.
Acknowledgments
The authors would like to thank Vermetha Polite of the Cushing/Whitney Medical Library for technical support.
References
- 1. Kessler RC, Chiu WT, Demler O, Merikangas KR, Walters EE. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005;62(6): 617–627. pmid:15939839
- 2.
American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition. Arlington, VA: American Psychiatric Association; 2013.
- 3. National Institute for Health and Care Excellence (NICE). Social Anxiety Disorder: Recognition, Assessment and Treatment. Available from: https://www.nice.org.uk/guidance/cg159.
- 4. Katzman MA, Bleau P, Blier P, Chokka P, Kjernisted K, Van Ameringen M, et al. Canadian clinical practice guidelines for the management of anxiety, posttraumatic stress and obsessive-compulsive disorders. BMC Psychiatry. 2014;14 Suppl 1:S1.
- 5. Osler M, Jorgensen MB. Associations of Benzodiazepines, Z-Drugs, and Other Anxiolytics With Subsequent Dementia in Patients With Affective Disorders: A Nationwide Cohort and Nested Case-Control Study. Am J Psychiatry. 2020;177(6): 497–505. pmid:32252539
- 6. Balon R, Starcevic V. Role of Benzodiazepines in Anxiety Disorders. Adv Exp Med Biol. 2020;1191: 367–388. pmid:32002938
- 7. Bystritsky A. Treatment-resistant anxiety disorders. Mol Psychiatry. 2006;11(9):805–14. pmid:16847460
- 8. Saha S, Lim CCW, Cannon DL, Burton L, Bremner M, Cosgrove P, et al. Co-morbidity between mood and anxiety disorders: A systematic review and meta-analysis. Depress Anxiety. 2021;38(3):286–306. pmid:33225514
- 9. Pollack MH. Comorbid anxiety and depression. J Clin Psychiatry 2005;66: 22–29. pmid:16336033
- 10. Pies R. Should psychiatrists use atypical antipsychotics to treat nonpsychotic anxiety? Psychiatry (Edgmont). 2009;6(6): 29–37. pmid:19724759
- 11. Mendels J, Krajewski TF, Huffer V, Taylor RJ, Secunda S, Schless A, et al. Effective short-term treatment of generalized anxiety disorder with trifluoperazine. J Clin Psychiatry. 1986;47(4): 170–174. pmid:3514583
- 12. Citrome L, Kalsekar I, Guo Z, Laubmeier K, Hebden T. Diagnoses associated with use of atypical antipsychotics in a commercial health plan: a claims database analysis. Clin Ther. 2013;35(12): 1867–1875. pmid:24119767
- 13. Marston L, Nazareth I, Petersen I, Walters K, Osborn DP. Prescribing of antipsychotics in UK primary care: a cohort study. BMJ Open. 2014;4(12): e006135. pmid:25524544
- 14. Weber SR, Wehr AM, Duchemin AM. Prevalence of antipsychotic prescriptions among patients with anxiety disorders treated in inpatient and outpatient psychiatric settings. J Affect Disord. 2016;191: 292–299. pmid:26688499
- 15. Maher AR, Maglione M, Bagley S, Suttorp M, Hu JH, Ewing B, et al. Efficacy and comparative effectiveness of atypical antipsychotic medications for off-label uses in adults: a systematic review and meta-analysis. JAMA. 2011;306(12): 1359–1369. pmid:21954480
- 16. Perna G, Alciati A, Riva A, Micieli W, Caldirola D. Long-Term Pharmacological Treatments of Anxiety Disorders: An Updated Systematic Review. Curr Psychiatry Rep. 2016;18(3): 23. pmid:26830881
- 17. Perna G, Alessandra A, Raffaele B, Elisa M, Giuseppina D, Paolo C, et al. Is There Room for Second-Generation Antipsychotics in the Pharmacotherapy of Panic Disorder? A Systematic Review Based on PRISMA Guidelines. Int J Mol Sci. 2016;17(4): 551. pmid:27089322
- 18. Depping AM, Komossa K, Kissling W, Leucht S. Second-generation antipsychotics for anxiety disorders. Cochrane Database Syst Rev. 2010(12): CD008120. pmid:21154392
- 19. LaLonde CD, Van Lieshout RJ. Treating generalized anxiety disorder with second generation antipsychotics: a systematic review and meta-analysis. J Clin Psychopharmacol. 2011;31(3): 326–333. pmid:21508847
- 20. Maneeton N, Maneeton B, Woottiluk P, Likhitsathian S, Suttajit S, Boonyanaruthee V, et al. Quetiapine monotherapy in acute treatment of generalized anxiety disorder: a systematic review and meta-analysis of randomized controlled trials. Drug Des Devel Ther. 2016;10: 259–276. pmid:26834458
- 21. Kennedy SH, Lam RW, McIntyre RS, Tourjman SV, Bhat V, Blier P, et al. Canadian Network for Mood and Anxiety Treatments (CANMAT) 2016 Clinical Guidelines for the Management of Adults with Major Depressive Disorder: Section 3. Pharmacological Treatments. Can J Psychiatry. 2016;61(9): 540–560. pmid:27486148
- 22. Yatham LN, Kennedy SH, Parikh SV, Schaffer A, Bond DJ, Frey BN, et al. Canadian Network for Mood and Anxiety Treatments (CANMAT) and International Society for Bipolar Disorders (ISBD) 2018 guidelines for the management of patients with bipolar disorder. Bipolar Disord. 2018;20(2): 97–170. pmid:29536616
- 23. Norgaard A, Jensen-Dahm C, Gasse C, Wimberley T, Hansen ES, Waldemar G. Association of Benzodiazepines and Antidepressants With 180-Day Mortality Among Patients With Dementia Receiving Antipsychotic Pharmacotherapy: A Nationwide Registry-Based Study. J Clin Psychiatry. 2020;81(4): 19m12828. pmid:32526104
- 24. Tiihonen J, Suokas JT, Suvisaari JM, Haukka J, Korhonen P. Polypharmacy with antipsychotics, antidepressants, or benzodiazepines and mortality in schizophrenia. Arch Gen Psychiatry. 2012;69(5): 476–483. pmid:22566579
- 25. Gerhard T, Stroup TS, Correll CU, Setoguchi S, Strom BL, Huang C, et al. Mortality risk of antipsychotic augmentation for adult depression. PLoS One. 2020;15(9): e0239206. pmid:32997687
- 26. Widschwendter CG, Hofer A. Antipsychotic-induced tardive dyskinesia: update on epidemiology and management. Curr Opin Psychiatry. 2019;32(3): 179–84. pmid:30720484
- 27. Aromataris E, Fernandez R, Godfrey CM, Holly C, Khalil H, Tungpunkom P. Summarizing systematic reviews: methodological development, conduct and reporting of an umbrella review approach. Int J Evid Based Healthc. 2015;13(3): 132–140. pmid:26360830
- 28. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372: n71. pmid:33782057
- 29. Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ. 2015;350: g7647. pmid:25555855
- 30. SIGN. Scottish Intercollegiate Guidelines Network Search Filters 2015. Available from: https://www.sign.ac.uk/what-we-do/methodology/search-filters/.
- 31. Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358: j4008. pmid:28935701
- 32. Theodoratou E, Tzoulaki I, Zgaga L, Ioannidis JP. Vitamin D and multiple health outcomes: umbrella review of systematic reviews and meta-analyses of observational studies and randomised trials. BMJ. 2014;348: g2035. pmid:24690624
- 33. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315(7109): 629–634. pmid:9310563
- 34. Rucker DD, Preacher KJ, Tormala ZL, Petty RE. Mediation Analysis in Social Psychology: Current Practices and New Recommendations. Social and Personality Psychology Compass. 2011;5(6): 359–371.
- 35. Pieper D, Antoine SL, Mathes T, Neugebauer EAM, Eikermann M. Systematic review finds overlapping reviews were not mentioned in every other overview. J Clin Epidemiol. 2014;67(4): 368–375. pmid:24581293
- 36. Ioannidis JP, Trikalinos TA. An exploratory test for an excess of significant findings. Clin Trials. 2007;4(3): 245–253. pmid:17715249
- 37. Murad MH, Chu H, Lin L, Wang Z. The effect of publication bias magnitude and direction on the certainty in evidence. BMJ Evid-Based Med. 2018;23(3): 84–86. pmid:29650725
- 38. Turner NL, Dias S, Ades AE, Welton NJ. A Bayesian framework to account for uncertainty due to missing binary outcome data in pairwise meta-analysis. Stat Med. 2015;34(12): 2062–2080. pmid:25809313