Advertisement
  • Loading metrics

Selective publication of antidepressant trials and its influence on apparent efficacy: Updated comparisons and meta-analyses of newer versus older trials

  • Erick H. Turner ,

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

    turnere@ohsu.edu

    Affiliations Behavioral Health and Neurosciences Division, Veterans Affairs Portland Health Care System, Portland, Oregon, United States of America, Department of Psychiatry, Oregon Health & Science University, Portland, Oregon, United States of America

  • Andrea Cipriani,

    Roles Methodology, Writing – review & editing

    Affiliations Department of Psychiatry, University of Oxford, Oxford, United Kingdom, Oxford Health NHS Foundation Trust, Warneford Hospital, Oxford, United Kingdom, Oxford Precision Psychiatry Lab, NIHR Oxford Health Biomedical Research Centre, Oxford, United Kingdom

  • Toshi A. Furukawa,

    Roles Investigation, Methodology, Writing – review & editing

    Affiliation Department of Health Promotion and Human Behavior, Kyoto University Graduate School of Medicine/School of Public Health, Kyoto, Japan

  • Georgia Salanti,

    Roles Formal analysis, Methodology, Visualization, Writing – review & editing

    Affiliation Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland

  • Ymkje Anna de Vries

    Roles Data curation, Formal analysis, Investigation, Methodology, Validation, Writing – review & editing

    Affiliations Department of Psychiatry, Interdisciplinary Center Psychopathology and Emotion Regulation, University Medical Center Groningen, University of Groningen, Groningen, the Netherlands, Developmental Psychology, Department of Psychology, University of Groningen, Groningen, the Netherlands

Abstract

Background

Valid assessment of drug efficacy and safety requires an evidence base free of reporting bias. Using trial reports in Food and Drug Administration (FDA) drug approval packages as a gold standard, we previously found that the published literature inflated the apparent efficacy of antidepressant drugs. The objective of the current study was to determine whether this has improved with recently approved drugs.

Methods and findings

Using medical and statistical reviews in FDA drug approval packages, we identified 30 Phase II/III double-blind placebo-controlled acute monotherapy trials, involving 13,747 patients, of desvenlafaxine, vilazodone, levomilnacipran, and vortioxetine; we then identified corresponding published reports. We compared the data from this newer cohort of antidepressants (approved February 2008 to September 2013) with the previously published dataset on 74 trials of 12 older antidepressants (approved December 1987 to August 2002).

Using logistic regression, we examined the effects of trial outcome and trial cohort (newer versus older) on transparent reporting (whether published and FDA conclusions agreed). Among newer antidepressants, transparent publication occurred more with positive (15/15 = 100%) than negative (7/15 = 47%) trials (OR 35.1, CI95% 1.8 to 693). Controlling for trial outcome, transparent publication occurred more with newer than older trials (OR 6.6, CI95% 1.6 to 26.4). Within negative trials, transparent reporting increased from 11% to 47%.

We also conducted and contrasted FDA- and journal-based meta-analyses. For newer antidepressants, FDA-based effect size (ESFDA) was 0.24 (CI95% 0.18 to 0.30), while journal-based effect size (ESJournals) was 0.29 (CI95% 0.23 to 0.36). Thus, effect size inflation, presumably due to reporting bias, was 0.05, less than for older antidepressants (0.10).

Limitations of this study include a small number of trials and drugs—belonging to a single class—and a focus on efficacy (versus safety).

Conclusions

Reporting bias persists but appears to have diminished for newer, compared to older, antidepressants. Continued efforts are needed to further improve transparency in the scientific literature.

Author summary

Why was this study done?

  1. ➢ Clinicians and researchers depend on the peer-reviewed literature for accurate assessments of drug efficacy and safety, but this depends on whether the outcomes of all trials—negative, as well as positive—are reported transparently.
  2. ➢ In an earlier study, using Food and Drug Administration (FDA) review documents as a gold standard, we found that many negative trials had been misreported in the published literature as having positive outcomes or had simply not been published.
  3. ➢ Since then, reporting bias has been the subject of additional studies and policy changes, raising the question, Is the antidepressant literature now being reported more transparently?

What did the researchers do and find?

  1. ➢ Using FDA reviews on 4 newer antidepressants, we identified 30 trials, half with positive, and half with negative, outcomes.
  2. ➢ Among the 15 negative trials, 6 were unpublished and 2 others were misreported as positive. Seven other negative trials (47%) were reported transparently (as negative), an improvement over the low (11%) rate found earlier with the older antidepressants.
  3. ➢ Statistical comparison of the newer and older drug datasets indicated that transparent reporting had improved overall, mainly among negative trials. Yet compared to positive trials, the rate of transparent reporting for negative trials remains low.
  4. ➢ Using meta-analysis to compare drug efficacy based on FDA versus published data, we found less inflation of drug efficacy among newer, compared to older, antidepressants.

What do these findings mean?

  1. ➢ Reporting bias persists but appears to have diminished for newer, compared to older, antidepressants.
  2. ➢ We do not know whether these results extend to drugs beyond the antidepressants studied here, nor do we know whether they extend to drug safety, as opposed to efficacy.
  3. ➢ Reporting bias remains a significant impediment to researchers and medical decision-makers, so further efforts are needed to improve transparent reporting in the scientific literature.

Introduction

Reporting bias can lead to overestimates of efficacy and/or underestimates of harms and thus undermine the evidence base regarding drugs and other interventions. Reporting bias takes several forms, including study publication bias and outcome reporting bias [1]. With study publication bias, entire studies are published or not depending on their results; with outcome reporting bias, studies are published but their outcomes are reported selectively depending on their results.

One of the early studies on reporting bias in antidepressant trials was published by our group [2]. Examining 12 second-generation antidepressants approved by the Food and Drug Administration (FDA) between 1986 and 2004, we found strong evidence for both study publication bias and outcome reporting bias. Because drug companies must report results of all Phase II/III trials to the FDA in order to gain approval for a new drug, FDA review documents can be considered a gold standard, an unbiased sample of all studies undertaken. Compared to trial results in FDA review documents, results published in journals inflated the apparent efficacy of antidepressants over placebo both in terms of proportion of positive trials and effect size (ES).

Using similar methodology, evidence for reporting bias has also been found among drugs for the treatment of schizophrenia [3] and of anxiety disorders [4], although not to the extent observed in antidepressant trials. Nor is reporting bias limited to psychotropic drugs—it has been documented for both pharmacological and nonpharmacological interventions across medical indications [59], and it appears to exist in social, biological, and physical sciences, as well [10].

An examination of papers published in all disciplines between 1990 and 2007 suggested an increase in reporting bias over time [10]. Since then, however, there have been important transparency-related policy changes, such as requirements for registration of clinical trials in 2005 [11,12] and for reporting of the trial results mandated by the Food and Drug Administration Amendments Act (FDAAA) of 2007 [13], and recent work suggests that the level of transparency has improved [14,15].

This raises the question, has the level of transparency of clinical trials changed specifically for the drugs for which reporting bias has perhaps been best described, namely antidepressants? Since our 2008 publication, several new antidepressant drugs have entered the United States market. Using the earlier study of older antidepressants for comparison, the current study aims to similarly determine whether, and to what degree, the apparent efficacy of the newer drugs has been inflated in published journal articles. More specifically, it asks, does trial outcome (positive or not) still influence whether and how the trial is reported? And does reporting bias still inflate ES?

Methods

Data procurement

FDA-registered trials.

This study extends the methodology of a previously published study of reporting bias, which examined 74 trials of 12 older antidepressants [2]. Using medical and statistical reviews within FDA drug approval packages (https://www.accessdata.fda.gov/scripts/cder/daf/) [16], we identified all Phase II or Phase III double-blind placebo-controlled acute monotherapy trials for major depressive disorder on 4 newer antidepressants (Table 1).

thumbnail
Table 1. List of antidepressants included in the older and newer cohorts of RCTs.

https://doi.org/10.1371/journal.pmed.1003886.t001

Literature search

Having identified the inception cohort of premarketing trials registered with the FDA, we used PubMed to search for matching publications reasonably discoverable by clinicians. Example search syntax for one drug was “desvenlafaxine[title] placebo (“major depressive disorder” OR “major depression”).” From the search output, titles and abstracts were screened to include journal articles focused on the overall efficacy of the drug in question for major depressive disorder; thus, we excluded articles focused on other indications, subsets with specific comorbid conditions, particular symptom clusters, safety (as opposed to efficacy), specific demographic samples, trials lacking a parallel design (add-on, open-label, crossover), trials that were not placebo controlled, trials not involving acute treatment (long-term trials, including maintenance trials), and trials involving other routes of administration. A literature search for the FDA-registered trials was also carried out independently by author YD in the context of a separate publication [17]. Separately, TF and YO identified the trials in ClinicalTrials.gov using the “Other Study ID Numbers” field and identified corresponding publications using the “Publication of Results” field. Matching of FDA-registered trials to publications was confirmed using trial design, duration, drugs used (study drug, placebo, active comparator), and number of participants randomized to each treatment arm. The preferred publication type was a stand-alone article, an article reporting on a single trial, with exceptions allowed as previously described [3].

Confirmation of nonpublication

For desvenlafaxine and vilazodone, we were unable to identify publications corresponding to all of the FDA-registered trials, so following a method reported previously [3], we searched for bibliographic information on said trials within recent review articles [1823], whose authors made use of additional databases, including EMBASE [18,22], ClinicalTrials.gov [22,23], and Cochrane Central [18]. Additionally, the authors of one article contacted the sponsor [22], and the authors of another article were employees of the sponsor [20].

Data extraction

As with previous studies [2,3], we employed double data extraction and entry. Data were extracted and entered by 3 teams (ET with 3 assistants; YD; TF and YO), compared using Boolean formulas in Excel, and reconciled for any discrepancies. For each trial, we extracted the results on the primary outcome from the FDA reviews, including the summary statistics and the FDA reviewer’s judgment as to whether the trial was positive, i.e., whether it provided “substantial evidence of effectiveness” for purposes of marketing approval [24].

From the corresponding journal articles, consistent with our previous study of the apparent (to the average clinician-reader) efficacy of antidepressants [2],we extracted the summary statistics on the apparent primary outcome. This was defined as the drug–placebo comparison highlighted as the trial’s main result by virtue of its being reported first in the article’s results section.

Data analysis

Descriptive examination of trial results.

Many trials consisted of 2 or more treatment arms compared to a common placebo group, resulting in 2 or more P values. Treatment-arm-level P values reported by the FDA (PFDA) were compared to P values reported in corresponding journal articles (PJournal). Because of nonindependence (the same placebo group could be represented in 2 or more datapoints), they were examined descriptively in the form of scatterplots, one for each cohort of trials. The scatterplots necessarily excluded treatment arms whose results were not published (no PJournal values).

Transparent publication.

We considered a trial to be published transparently if the trial was published in a way that was consistent with the FDA report of that trial. Transparent publication was deemed absent when (a) the trial results were not published (study publication bias) or (b) the results were published but in a way that conflicted with the FDA report (outcome reporting bias). For example, if a trial was reported by the FDA to be negative (nonsignificant on the primary outcome), but the publication conveyed a positive overall result by emphasizing statistically significant results in the beginning of the results section (see above re apparent primary outcome) and the abstract, that trial was deemed not transparently published.

We examined 2 predictors of transparent publication. The first was trial cohort (i.e., older versus newer antidepressants). The second was trial outcome according to the FDA report—positive (study drug clearly statistically superior to placebo on the primary outcome) or not positive. The main model also included a third variable for the interaction between the first 2 predictors. To estimate the associations, within Stata 11 [25], we employed Firth (penalized) logistic regression using the module firthlogit [2527]. As a secondary analysis, we employed exact logistic regression [26] using the module exlogistic [25]. (These methods were chosen because, in the context of rare events, such as FDA-positive trials that are not transparently published, standard logistic regression fails. Please see S1 Text for elaboration.) We also undertook the following post hoc univariable analyses: Because transparent publication is arguably more likely to occur with positive than negative trials, we examined the effect of cohort within each of these subsets; similarly, we examined the effect of trial outcome within each cohort.

Meta-analysis of effect sizes.

The meta-analysis portion of this study is reported as per the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 guideline (S1 Checklist).

We examined whether reporting bias misinformed the public by comparing one meta-analysis (MA) using trial data obtained from FDA reviews to a second MA using data from the corresponding publications. The MAs were conducted using the metan module in Stata 11 [25], with random effects pooling and the DerSimonian–Laird estimator for heterogeneity. The resulting effect measures (standardized mean difference Hedges’ g ± 95% confidence interval) obtained by author ET were verified against those obtained independently by author YD. As in previous work [24], for each multiple-dose trial, we used fixed effects MA to obtain a single trial-level ES; to avoid a spuriously low standard error, each trial’s shared placebo n was counted once rather than redundantly for each dose group.

We then compared the results of the MAs, with effect size inflation (ESI), presumably due to reporting bias, calculated as journal-based ES (ESJournal) minus FDA-based ES (ESFDA). To facilitate visual comparison of these values, we exported the Stata-generated forest plots to vector-based graphics software (Intaglio version 3.9), which allowed corresponding ESFDA and ESJournal values to be placed alongside one another. Such pairwise forest plots were generated showing ES values at the level of trial, drug, and cohort.

Because journal-based ES and FDA-based ES are not independent (both derived from the same set of trials), we did not perform a formal statistical comparison through, for instance, meta-regression. As an exploratory method, we did perform multivariate MA, which is capable of handling such dependency, but it is limited in another respect. As explained further in the Supporting information, multivariate MA relies on the correlation between paired FDA-based and journal-based ES values, and complete pairs exist for published trials but not for unpublished trials. Because unpublished trials, compared to published trials, are much more likely to be negative and have systematically smaller ES values [2], journal-based ES values are missing not at random. Thus, the multivariate approach is less well suited to the examination of study publication bias than to outcome reporting bias. Because our dataset contains both of these forms of reporting bias, results of the multivariate MA are provided as Supporting information (S1 Text and S8 Fig).

We included only doses approved by the FDA, as reflected in the Dosage and Administration section of the product label. While this wording in this section was clear in many cases, in others, it was ambiguous; thus, for certain doses, arguments could be made for both inclusion and exclusion. We resolved this by conducting a primary MA using broad dose inclusion criteria and a sensitivity MA using narrow or restrictive dose inclusion criteria. For rationale and elaboration, please see legend to Table 2 and Table C in S1 Text. As explained in the latter, for dose reasons, one trial (vilazodone #244) was excluded from both MAs.

thumbnail
Table 2. FDA-registered trials and corresponding publications for cohort of 4 newer antidepressants.

https://doi.org/10.1371/journal.pmed.1003886.t002

Results

The FDA-registered trials and their corresponding publications are listed in Table 2. For the cohort of 4 newer antidepressants, there were 30 applicable trials with 13,747 participants, while for the cohort of older antidepressants, there were 74 applicable trials with 12,564 participants. Median trial sample sizes for the newer and older cohorts were 439.5 and 147.5, respectively (Z = 6.72, P < 0.001 by Wilcoxon rank-sum test), different by a factor of 3.

Results at level of treatment arm

The total number of treatment arms was 149, with 101 and 48 in the older and newer cohorts, respectively. Fig 1 plots the P values of all these arms against placebo as reported in the journals versus as confirmed in FDA reviews: It shows, among the 104 published treatment arms, a greater proportion of treatment arms lying along the Y = X (PJournal = PFDA) diagonal in the newer cohort, i.e., greater concordance between journal- and FDA-based data. The proportion of unpublished treatment arms (gray boxes) was 36% (36/101, CI95% 26% to 45%) for the older cohort versus 19% (9/48, CI95% 8% to 30%) for the newer cohort.

thumbnail
Fig 1. Reporting of P values in publications vs. FDA reviews for older and newer cohorts of antidepressant drugs.

P values reported by the FDA (PFDA, horizontal axes) are compared to those in corresponding journal articles (PJournal, vertical axes). The older and newer antidepressants are shown in the left and right plots, respectively. Each data point represents a drug treatment arm compared to placebo, with area proportional to the sum of their sample sizes. Dashed diagonal represents concordance between PFDA and PJournal, i.e., an absence of reporting bias. Cases of outcome reporting bias, where PFDA is NS but PJournal is reported as significant, are highlighted in yellow. (The 2 yellow circles far off the Y = X diagonal in the right-hand panel represent desvenlafaxine trials 309 and 317 (please see text).) Unpublished treatments arms, with PFDA values but no corresponding PJournal values, are shown in the gray boxes and highlighted in yellow. FDA, Food and Drug Administration; NS, not significant.

https://doi.org/10.1371/journal.pmed.1003886.g001

Transparent publication

For the 4 newer antidepressants, Table 2 and S1 Fig show each trial’s overall outcome, as determined by the FDA, and its corresponding publication status. Of these 30 trials, the FDA deemed 15 (50%) to be positive, i.e., statistically significant on the prespecified primary outcome, consistent with the proportion previously reported for the older cohort [2]. Among these 15 FDA-positive trials, all were published in agreement with the FDA (transparently reported as positive). Among the 15 not-positive trials, 7 (47%) were transparently published (as nonsignificant), a higher proportion than that observed for the older cohort (4/37 = 11%) [2]. The remaining 8 (53%) FDA-negative trials in the newer cohort were not transparently published.

Six of these were simply not published. One desvenlafaxine trial was referred to in one review publication as “an unpublished report with the code name Des 223” [18]; in a second review publication authored by employees of the sponsor, it was referred as “data on file” [20]. Regarding vilazodone, one review publication referred to 5 trials (#244 to 248) as “astonishingly unfavorable” and cited the FDA drug approval package and no publications; a second review publication listed them in a paragraph and a data table devoted to unpublished trials.

Two other FDA-negative trials—desvenlafaxine EU trial 309 and US trial 317—were published but classified as not transparently published for 2 reasons. First, they were published solely in the form of a single positive “pooled analysis” paper [27]. (This form of reporting bias has been previously described in the antidepressant literature [28].) To be classified as transparently reported, the 2 trials should have been published in separate stand-alone papers highlighting their nonsignificant results, or published in a combined article highlighting the 2 nonsignificant results. Second, in the journal article, a nonprimary method of handling dropouts (MMRM instead of LOCF; Table D in S1 Text) was used, leading to statistically significant pooled results. (Although pooling trials increases statistical power, this alone would not have yielded a statistically significant result. Via post hoc MA of the FDA-reported primary results for these 2 trials, we calculated Hedges’ g = 0.10 (CI95% −0.08 to 0.28, P = 0.27).) These significant results were highlighted in the abstract and beginning of the results section. Meanwhile, the nonsignificant results from the individual trials were reported beginning on the fifth page of the results section and not in the abstract.

The effects of trial outcome and cohort on transparent reporting were examined using logistic regression. Please refer to Fig 2 for all counts and proportions, as well as odds ratios. (Further logistic regression results are available in Table A in S1 Text.) With respect to the variable for trial outcome, transparent reporting occurred more often for FDA-positive than for FDA-negative trials (OR 181, CI95% 26.9 to 1,219, P < 0.001). Post hoc univariable analyses showed significant effects of trial outcome within the older cohort (OR 181 CI95% 26.9 to 1,219, P < 0.001), consistent with findings reported earlier [2], and within the newer cohort (OR 35.1, CI95% 1.8 to 693, P = 0.019). Within the newer cohort, 15 of 15 (100%) positive trials were reported transparently versus 7 of 15 (47%) negative trials.

thumbnail
Fig 2. Interaction plot illustrating the effects of trial outcome and cohort (older vs. newer antidepressants) on transparent publication.

For all trials regardless of outcome (dashed oblique line), the proportion of transparently reported trials increased from 54% (older drugs) to 73% (newer drugs). Within the subset of FDA-positive trials (blue line), transparent reporting, which was already nearly 100% for the older cohort, showed no further increase. By contrast, within FDA-negative trials (green line), transparent reporting increased from 11% to 47%. FDA, Food and Drug Administration.

https://doi.org/10.1371/journal.pmed.1003886.g002

With respect to the variable for cohort, the overall proportion of transparently reported trials increased from 54% to 73%. Controlling for trial outcome, trials in the newer cohort were 6.6 times more likely to be transparently reported than trials in the older cohort (OR 6.6, CI95% 1.6 to 26.4, P = 0.008).

Post hoc analyses suggested that the higher rate of transparent publication in the newer cohort was limited to negative trials, which increased from 11% to 47%. Negative trials in the newer cohort were 6.6 times more likely to be transparently reported than negative trials in the older cohort (OR 6.6, CI95% 1.6, 26.4, P = 0.008), equal to the abovementioned main effect of cohort. By contrast, positive trials were transparently reported approximately 100% of the time for both cohorts; thus, the post hoc univariable analysis showed no effect (OR 1.3, CI95% 0.05 to 33, P = 0.88).

As shown in Table A in S1 Text, the multivariable model’s interaction effect was nonsignificant (OR 0.19, CI95% 0.006 to 6.7, P = 0.36); omitting the interaction term had little impact on the multivariable models’ 2 main effects. For all of the abovementioned analyses, similar results were obtained using exact logistic regression (S2 Fig and Table A in S1 Text).

Meta-analysis

Dose groups and trials included and excluded in the primary and sensitivity MAs are listed in Table C in S1 Text. Meta-analytic trial-level results from Stata, including forest plots, based on data from the FDA and the published literature, and for both primary and sensitivity MAs, are shown in S3S6 Figs and Tables E-H in S1 Text.

S7 Fig is a forest plot comparing trial-level ES based on FDA versus journal data for each of the four newer antidepressants (using broad dose inclusion criteria). In S7 Fig, not-transparently published trials are highlighted for desvenlafaxine and vilazodone, which give rise to the observed (FDA- versus journal-based) ES differences at the level of drug (quantified below). For the other 2 drugs, levomilnacipran and vortioxetine, all trials were deemed transparently published (none highlighted otherwise); thus, their FDA- and journal-based ES values, at the level of both trial and drug, are virtually the same.

The abovementioned drug-level ES values are summarized and compared in Fig 3. In the primary MA (left panel), as mentioned above, ESI was largest for vilazodone (0.28 − 0.16 = 0.12), followed by desvenlafaxine (0.31 − 0.24 = 0.07). The overall FDA-based Hedges’ g for the 4 newer antidepressants was 0.24 (CI95% 0.18, 0.30), while the overall journal-based ES was 0.29 (CI95% 0.23, 0.36), for an ESI of +0.05.

thumbnail
Fig 3. Meta-analytic ES of 4 newer antidepressants derived from trial reports in FDA reviews vs. journal articles.

The primary (left panel) and sensitivity (right panel) MAs were based on broad and narrow/restrictive dose inclusion criteria, respectively, as described in text. For each antidepressant, 2 drug-level ES values are shown, one based on clinical trial data from FDA reviews and one based on data from the journal articles. ES, effect size; g, Hedges’s g; FDA, Food and Drug Administration; MA, meta-analysis.

https://doi.org/10.1371/journal.pmed.1003886.g003

In the sensitivity MAs (right panel of Fig 3),which employed restrictive/narrow dose inclusion criteria (see Methods), ES values were generally higher, especially for the FDA-based values, bringing them into closer alignment with the journal-based values. Thus, the abovementioned ESIs for vilazodone and desvenlafaxine decreased to nearly zero. The overall FDA-based Hedges’ g in these analyses was 0.33 (CI95%: 0.25, 0.41), while the overall journal-based value was 0.33 (CI95%: 0.25, 0.41), resulting in an ESI of approximately zero (+0.0).

Fig 4 compares the overall FDA- versus journal-based ES values for the newer versus the older cohort of antidepressants. As previously reported, the overall ESI for the older cohort was 0.10 (= 0.41 − 0.31), larger than the ESI found in the primary and sensitivity MA for the newer cohort. For additional context, ESI for individual drugs in the older cohort ranged from 0.03 (paroxetine controlled release) to 0.22 (mirtazapine), with a median of 0.10 [2].

thumbnail
Fig 4. Comparison of overall ES, derived from FDA reviews vs. journal articles, for older and newer antidepressants.

For the newer antidepressants, ESI was smaller than that previously reported for the older antidepressants. ES, effect size; ESI, effect size inflation; FDA, Food and Drug Administration; MA, meta-analysis.

https://doi.org/10.1371/journal.pmed.1003886.g004

Discussion

The data presented here suggest that reporting bias in the published literature on antidepressant drugs is still an important issue. Even within the cohort of newer antidepressants, statistical significance still has an undue influence on whether and how these trials are reported. Consistent with earlier work, we found that positive trials are much more likely to be transparently reported than negative trials, whether one looks within the newer cohort or at both cohorts combined. However, we also found evidence for improvement in reporting bias: Antidepressants trials, especially those deemed negative by the FDA, are more likely to be published transparently than they were previously. Regarding the meta-analytic results, though not the subject of formal statistical analysis, smaller ESI values for the newer, compared to the older, cohort also could be consistent with a decrease in reporting bias.

Comparison with previous findings

Our findings are consistent with at least 5 other recent studies: (1) A study [29] of Phase III randomized controlled trials (RCTs) in pediatric patients compared conference abstracts from 2008 to 2011 to subsequent publications and found evidence for “reduced but ongoing publication bias” as compared to a similar study from 15 years earlier [30]. (2) Another research group found evidence for improvements in some, though not all, measures of transparency (registration rates, results reporting, publication rates) for drugs approved in 2014, compared to drugs approved in 2012 [31]. (3) In an examination of trials of both pharmacological and nonpharmacological treatments for depression, the prevalence of proper registration and reporting was improved but still very low, despite the fact that registration and reporting had been mandatory for several years [32]. (4) Examining drugs approved for cardiovascular disease and diabetes mellitus [14], another group found a decrease in publication bias (as well as an increase in registration) among trials for drugs approved by the FDA after, compared to before, the FDAAA of 2007. (5) The same group applied similar methodology to drugs approved for several indications treated by neurologists and psychiatrists, as well as other indications (anesthesia, constipation, fibromyalgia, pain) [33]. The latter 2 studies were restricted to “pivotal” trials, a designation often assigned post hoc to trials with positive outcomes; by contrast, the current study covers all efficacy trials regardless of outcome, including so-called “failed trials” [34,35]. The current study differs from the 5 abovementioned studies in that it focuses on one drug class for one indication (major depressive disorder), thus enabling MA.

Possible explanations

How might we explain this apparent increase in transparency? There have long been many incentives to engage in reporting bias [36]. In the past, there was little awareness within the research and clinical communities that the problem existed, and pharmaceutical companies (and others) could engage in reporting bias without fear of detection. Since then, however, there has been a cultural change, and what was once standard practice is no longer considered acceptable. Numerous policy changes have been implemented, summarized elsewhere [37]. ClinicalTrials.gov was launched in 2000, but registrations initially lagged. In 2004—the year the FDA approved duloxetine, the newest drug within the older cohort of antidepressants [2]—the International Committee of Journal Editors (ICMJE) announced that prospective registration would be a precondition for publication. The following year saw a 73% increase in the registration rate over a span of just 5 months [38]. In 2005, the WHO International Clinical Trials Registry Platform (http://apps.who.int/trialsearch/Default.aspx) was launched. In 2007, the FDAAA was enacted [13], which legally mandated public registration of applicable clinical trials and called for the augmentation of ClinicalTrials.gov with a basic results database; in 2010, FDAAA was clarified and expanded in scope to include all Phase II to IV drug and device trials, adverse events, and basic results [39].

It seems reasonable to conclude that these policy changes played a major role in bringing about the increase in transparency suggested by the current study and the others mentioned above. However, given the level of attention directed toward reporting bias with antidepressants, in the form of lawsuits [39], numerous key publications [2,4043], and new incentives to increase transparency, for instance, the Good Pharma Scorecard [31], it is possible that substantial improvement would have occurred without these policy changes.

Implications, theoretical and practical

However, we must caution that, while the proverbial glass of transparency is now half full, it also remains half empty. Nothing less than full transparency should be considered acceptable in the realm of healthcare. Greater awareness of reporting bias is needed among researchers and clinicians so that they do not naively accept published research findings at face value.

The abovementioned policy changes should not be celebrated until compliance with them improves. In the case of FDAAA, apparently due to a lack of political will, enforcement has been lax, leading to over $5 billion in accrued fines remaining uncollected (http://fdaaa.trialstracker.net). Additionally, many journals that ostensibly support the ICMJE policy of preregistration continue to publish a substantial number of unregistered or belatedly registered trials [32].

Perhaps what is needed most is to eliminate reporting bias at its root. FDA reviews include the results of negative, as well as positive, studies, because the Agency receives study protocols before studies are undertaken, thus preventing drug companies from hiding the existence of studies or switching their outcomes. Although trial registries are intended to serve a similar purpose, they are separate from journals, in which the strength and direction of study results can continue to dictate submission and acceptance decisions. However, in an emerging peer review model known as Registered Reports [44], manuscripts are submitted and reviewed before studies are undertaken, leading to preliminary publication decisions based solely on the scientific question and methodological rigor. Registered Reports has been adopted, or offered as an option, by >300 journals in various fields (https://cos.io/rr/?_ga=2.192240618.1714708995.1570198509-367521697.1570198509, accessed November 20, 2021), but uptake among major medical journals has unfortunately lagged.

Limitations and suggestions for future research

Despite the above-referenced consistency with previous findings, this study has several limitations that may affect generalizability. The scope of this study is limited to drug efficacy, so future research could examine whether transparency has also improved with regard to safety issues. For instance, we previously found that reporting of serious adverse events in the older antidepressant trials is often incomplete and inconsistent with FDA information [45]. Another scope-related limitation is that, because this study is restricted to antidepressants, these findings may not extend to other drug classes, which likely vary in signal-to-noise ratio (efficacy), proportion of “negative” trials, “need” for reporting bias, and hence potential for increased transparency. Further, this report pertains to Phase II/III premarketing trials only; the extent of reporting biases for Phase IV postmarketing trials is yet to be examined.

The most recently approved drugs in our sample were approved by the FDA in 2013, making it difficult to draw inferences about more recent changes in transparency practices. On the other hand, this study is a follow-up to our earlier review, making the contrast before versus after 2008 the topic of research. In addition to 2008 being the year of approval for the newest antidepressant in the older cohort, it was the year FDAAA was enacted, a milestone event for addressing reporting bias [13]. In any case, it was not possible to include drugs approved after 2013. The sample of older drugs was limited to those FDA-approved as monotherapy for major depressive disorder, and newer such drugs do not exist. More recent FDA approvals exist only for drugs FDA-approved as adjunctive therapy and/or for other depression-related indications such as treatment-resistant depression and postpartum depression, and our findings may not extend to reporting practices on such drugs.

For the newer, compared to the older, cohort, although the sample size was larger in terms of overall number of patients, it was smaller in terms of the number of drugs (4 versus 12, respectively) and trials (30 versus 74). However, as noted above, the drugs and trials in our sample were dictated by the drug development/approval process. Additionally, selective reporting practices may be correlated among trials for the same treatment from the same sponsor. (This limitation also applies to our earlier study [2]; among the 12 older antidepressants, one company sponsored 3 drugs and 3 companies sponsored 2 each.)

There are also limitations with respect to the statistical approach. First, the dichotomization of drugs into older and newer cohorts is somewhat arbitrary in that it is based on the timing of the earlier publication on the older cohort [2]. Second, because pharmaceutical companies often have publication strategies [46], within any given company, decisions whether to transparently publish its trials likely do not occur independently of one another. Third, the logistic regression model’s interaction term was nonsignificant; however, due to a limited sample size, and because transparent reporting of positive trials in the older cohort was already approximately 100%, leaving no room for improvement, the model was underpowered to detect such an effect. Fourth, the nonindependence of journal-based and FDA-based ES precluded formally contrasting them via meta-regression, so we were unable to determine whether the observed ESI is statistically significant. Fifth and relatedly, in contrasting the proportion of unpublished treatment arms between the older and newer cohorts, because many studies compared 2 or more dose arms to a single placebo group, the assumption of independence was not fully met.

In the current study, we found no change in the (high) level of transparent publication within FDA-positive trials. This is unsurprising—regardless of cohort, it is hard to conceive of a positive trial going unreported or being reported as negative. Hence, it could be argued that positive trials are uninformative and may hamper reporting bias signal detection. Therefore, in future analyses of reporting bias datasets, researchers may wish to consider focusing primarily on (more informative) negative trials.

In the sensitivity MA, compared to the primary MA, the gap between ESJournals and ESFDA was diminished. The sensitivity MA employed narrow dose inclusion criteria, i.e., was limited to those doses unambiguously approved by the FDA. The FDA determines effective dose ranges based on how “successful” the various doses are in Phase II/III trials. In the narrowly defined dose range, the sample was limited to clearly successful (FDA-positive) data, which are associated with larger ES values and higher levels of transparent publication (less reporting bias).

Conclusions

In this study, we found a persistence of reporting bias within a cohort of newer antidepressants approved since 2008. However, compared to the cohort of older antidepressants, reporting bias in the newer cohort appeared to decrease, and neither study publication nor outcome reporting bias was found among trials for the 2 newest antidepressants (both approved in 2013). The observed improvement in transparency may be due to cultural and policy changes over the past decade. Further efforts and vigilance are needed to maintain and build upon these improvements.

Supporting information

S1 Fig. Overall trial outcome according to FDA and publication status for the 4 newer antidepressants.

Among 15 FDA-positive trials, all were published in agreement with the FDA (transparently reported as positive). Among 15 not-positive trials, 7 (47%) were transparently published (as nonsignificant), a higher proportion than that observed for the older antidepressants. FDA, Food and Drug Administration.

https://doi.org/10.1371/journal.pmed.1003886.s001

(EPS)

S2 Fig. Stata commands underlying results displayed in similar figure in main paper.

Above each line, the command for the primary method, penalized (Firth) logistic regression, is shown. Below each line, the command for the secondary method, exact logistic regression, is shown.

https://doi.org/10.1371/journal.pmed.1003886.s002

(EPS)

S3 Fig. Forest plot based on data from FDA reviews using broad dose inclusion criteria.

ES, effect size; FDA, Food and Drug Administration.

https://doi.org/10.1371/journal.pmed.1003886.s003

(EPS)

S4 Fig. Forest plot based on data from journal articles using broad dose inclusion criteria.

ES, effect size.

https://doi.org/10.1371/journal.pmed.1003886.s004

(EPS)

S5 Fig. Forest plot based on data from FDA reviews using narrow dose inclusion criteria.

ES, effect size; FDA, Food and Drug Administration.

https://doi.org/10.1371/journal.pmed.1003886.s005

(EPS)

S6 Fig. Forest plot based on data from journal articles using narrow dose inclusion criteria.

ES, effect size.

https://doi.org/10.1371/journal.pmed.1003886.s006

(EPS)

S7 Fig. Forest plot comparing trial-level ES based on FDA vs. journal data for the 4 newer antidepressants (broad dose inclusion criteria).

Red boxes indicate trials that were not transparently published. ES, effect size; FDA, Food and Drug Administration.

https://doi.org/10.1371/journal.pmed.1003886.s007

(EPS)

S8 Fig. Graphical depiction of results from the multivariate MA.

Blue asterisks and black plus signs represent trials of older and newer antidepressants, respectively. Blue and black ellipses indicate the 95% confidence regions for their respective summary effects. FDA, Food and Drug Administration; MA, meta-analysis; SMD, standardized mean difference.

https://doi.org/10.1371/journal.pmed.1003886.s008

(EPS)

S1 Text.

Table A. Results from penalized (Firth) logistic regression models. Effect of trial outcome and cohort on transparent reporting of trial outcome. Table B. Results from exact logistic regression models. Effect of trial outcome and cohort on transparent reporting of trial outcome. Table C. Rationale for inclusion of dosage groups in primary vs. sensitivity MA. Table D. Primary outcomes according to FDA review and journal articles. Table E. Numerical results (FDA-based MA; broad dose inclusion criteria). Table F. Numerical results (journal-based MA; broad dose inclusion criteria). Table G. Numerical results (FDA-based MA; narrow dose inclusion criteria). Table H. Numerical results (journal-based MA; narrow dose inclusion criteria). Table I. Results of the multivariate MAs. FDA, Food and Drug Administration; MA, meta-analysis

https://doi.org/10.1371/journal.pmed.1003886.s009

(DOCX)

S1 Data. Logistic regression data in Stata (.dta) format.

https://doi.org/10.1371/journal.pmed.1003886.s012

(DTA)

S2 Data. Logistic regression data in Excel (.xlsx) format.

https://doi.org/10.1371/journal.pmed.1003886.s013

(XLSX)

S3 Data. Data used for FDA-based MA, BROAD dose inclusion criteria, Stata (.dta) format.

FDA, Food and Drug Administration; MA, meta-analysis.

https://doi.org/10.1371/journal.pmed.1003886.s014

(DTA)

S4 Data. Data used for JOURNAL-based MA, BROAD dose inclusion criteria, Stata (.dta) format.

MA, meta-analysis.

https://doi.org/10.1371/journal.pmed.1003886.s015

(DTA)

S5 Data. Data used for FDA-based MA, NARROW dose inclusion criteria, Stata (.dta) format.

FDA, Food and Drug Administration; MA, meta-analysis.

https://doi.org/10.1371/journal.pmed.1003886.s016

(DTA)

S6 Data. Data used for JOURNAL-based MA, NARROW dose inclusion criteria, Stata (.dta) format.

MA, meta-analysis.

https://doi.org/10.1371/journal.pmed.1003886.s017

(DTA)

S7 Data. Data used for FDA- and JOURNAL-based MAs, BROAD and NARROW dose inclusion criteria, Excel (.xlsx) format.

FDA, Food and Drug Administration; MA, meta-analysis.

https://doi.org/10.1371/journal.pmed.1003886.s018

(XLSX)

Acknowledgments

Kathryn Burrows assisted with Table 2. The following individuals assisted with double data extraction and entry: Sepideh Alavi, Ilya Ivlev, Ryan McKenna. Yusuki Ogawa assisted with the search for FDA-registered trials and corresponding publications and with data extraction and entry.

References

  1. 1. Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan A-W, Cronin E, et al. Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS ONE. 2008;3:e3081. pmid:18769481
  2. 2. Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective Publication of Antidepressant Trials and Its Influence on Apparent Efficacy. N Engl J Med. 2008;358:252–60. pmid:18199864
  3. 3. Turner EH, Knoepflmacher D, Shapley L. Publication Bias in Antipsychotic Trials: An Analysis of Efficacy Comparing the Published Literature to the US Food and Drug Administration Database. PLoS Med. 2012;9:e1001189. pmid:22448149
  4. 4. Roest AM, Jonge d, Williams CD, Vries d YA, Schoevers RA, Turner EH. Reporting Bias in Clinical Trials Investigating the Efficacy of Second-Generation Antidepressants in the Treatment of Anxiety Disorders: A Report of 2 Meta-analyses. JAMA Psychiat. 2015;72:500–10. pmid:25806940
  5. 5. Hart B, Lundh A, Bero L. Effect of reporting bias on meta-analyses of drug trials: reanalysis of meta-analyses. BMJ. 2012;344:d7202–2. pmid:22214754
  6. 6. Omae K, Kataoka Y, Tsujimoto Y, Tsutsumi Y, Yamamoto Y, Fukuhara S, et al. Publication statuses of clinical trials supporting FDA-approved immune checkpoint inhibitors: a meta-epidemiological investigation. BMC Cancer. 2019;19:998. pmid:31651263
  7. 7. Rising K, Bacchetti P, Bero L. Reporting Bias in Drug Trials Submitted to the Food and Drug Administration: Review of Publication and Presentation. Ioannidis J, editor. PLoS Med. 2008;5:e217-discussion e217. pmid:19067477
  8. 8. McGauran N, Wieseler B, Kreis J, Schüler Y-B, Kölsch H, Kaiser T. Reporting bias in medical research—a narrative review. Trials. 2010;11:37. pmid:20388211
  9. 9. Ioannidis JPA, Karassa FB. The need to consider the wider agenda in systematic reviews and meta-analyses: breadth, timing, and depth of the evidence. BMJ. 2010;341:c4875. pmid:20837576
  10. 10. Fanelli D. Negative results are disappearing from most disciplines and countries. Scientometrics. 2012;90:891–904.
  11. 11. DeAngelis CD, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, et al. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. JAMA. 2004;292:1363–4. pmid:15355936
  12. 12. Zarin DA, Tse T, Williams RJ, Califf RM, Ide NC. The ClinicalTrials.gov results database—update and key issues. N Engl J Med. 2011;364:852–60. pmid:21366476
  13. 13. Zarin DA, Tse T. Moving toward transparency of clinical trials. Science. 2008;319:1340–2. pmid:18323436
  14. 14. Phillips AT, Desai NR, Krumholz HM, Zou CX, Miller JE, Ross JS. Association of the FDA Amendment Act with trial registration, publication, and outcome reporting. Trials. 2017;18:1518. pmid:28720112
  15. 15. Miller JE, Korn D, Ross JS. Clinical trial registration, reporting, publication and FDAAA compliance: a cross-sectional analysis and ranking of new drugs approved by the FDA in 2012. BMJ Open. 2015;5:e009758. pmid:26563214
  16. 16. Turner EH. How to access and process FDA drug approval packages for use in research. BMJ. 2013;347:f5992–2. pmid:24126858
  17. 17. Vries YA de, Roest AM, Jonge P de, Cuijpers P, Munafò MR, Bastiaansen JA. The cumulative effect of reporting and citation biases on the apparent efficacy of treatments: the case of depression. Psychol Med 2018;48:2453–2455. pmid:30070192
  18. 18. Laoutidis ZG, Kioulos KT. Desvenlafaxine for the acute treatment of depression: a systematic review and meta-analysis. Pharmacopsychiatry. 2015;48:187–99. pmid:26205685
  19. 19. Kornstein SG, McIntyre RS, Thase ME, Boucher M. Desvenlafaxine for the treatment of major depressive disorder. Expert Opin Pharmacother. 2014;15:1449–63. pmid:24914479
  20. 20. Tourian KA, Jiang Q, Ninan PT. Analysis of the Effect of Desvenlafaxine on Anxiety Symptoms Associated with Major Depressive Disorder: Pooled Data from 9 Short-Term, Double-blind, Placebo-Controlled Trials. CNS Spectr. 2010;15:187–93. pmid:20414167
  21. 21. Wang S-M, Han C, Lee S-J, Patkar AA, Masand PS, Pae C-U. Vilazodone for the Treatment of Depression: An Update. Chonnam Med J. 2016;52:91–100. pmid:27231672
  22. 22. McCormack PL. Vilazodone: a review in major depressive disorder in adults. Drugs. 2015;75:1915–23. pmid:26496736
  23. 23. Citrome L. Vilazodone for major depressive disorder: a systematic review of the efficacy and safety profile for this newly approved antidepressant—what is the number needed to treat, number needed to harm and likelihood to be helped or harmed? Int J Clin Pract. 2012;66:356–68. pmid:22284853
  24. 24. Katz R. FDA: evidentiary standards for drug development and approval. NeuroRx. 2004;1:307–16. pmid:15717032
  25. 25. Stata Statistical Software: Release 11. College Station, TX: StataCorp LP; 2009.
  26. 26. Mehta CR, Patel NR. Exact logistic regression: theory and examples. Stat Med. 1995;14:2143–60. pmid:8552893
  27. 27. Lieberman DZ, Montgomery SA, Tourian KA, Brisard C, Rosas G, Padmanabhan K, et al. A pooled analysis of two placebo-controlled trials of desvenlafaxine in major depressive disorder. Int Clin Psychopharmacol. 2008;23:188–97. pmid:18545056
  28. 28. Vries d YA, Roest AM, Turner EH, Jonge d PA. Hiding negative trials by pooling them: a secondary analysis of pooled-trials publication bias in FDA-registered antidepressant trials. Psychol Med. 2018;351:1–7. pmid:30261934
  29. 29. Crockett LK, Okoli GN, Neilson CJ, Rabbani R, Abou-Setta AM, Klassen TP. Publication of Randomized Clinical Trials in Pediatric Research: A Follow-up Study. JAMA Netw Open. 2018;1:e180156. pmid:30646048
  30. 30. Klassen TP, Wiebe N, Russell K, Stevens K, Hartling L, Craig WR, et al. Abstracts of randomized controlled trials presented at the society for pediatric research meeting: an example of publication bias. Arch Pediatr Adolesc Med. 2002;156:474–9. pmid:11980553
  31. 31. Miller JE, Wilenzick M, Ritcey N, Ross JS, Mello MM. Measuring clinical trial transparency: an empirical analysis of newly approved drugs and large pharmaceutical companies. BMJ Open. 2017;7:e017917. pmid:29208616
  32. 32. Shinohara K, Tajika A, Imai H, Takeshima N, Hayasaka Y, Furukawa TA. Protocol registration and selective outcome reporting in recent psychiatry trials: new antidepressants and cognitive behavioural therapies. Acta Psychiatr Scand. 2015;132:489–98. pmid:26367129
  33. 33. Zou CX, Becker JE, Phillips AT, Garritano JM, Krumholz HM, Miller JE, et al. Registration, results reporting, and publication bias of clinical trials supporting FDA approval of neuropsychiatric drugs before and after FDAAA: a retrospective cohort study. Trials. 2018;19:581. pmid:30352601
  34. 34. Ninan PT, Poole RM, Stiles GL. Selective publication of antidepressant trials. N Engl J Med. 2008;358:2181. PubMed PMID: author reply 2181–2. pmid:18494076
  35. 35. Otto MW, Nierenberg AA. Assay sensitivity, failed clinical trials, and the conduct of science. Psychother Psychosom. 2002;71:241–3. pmid:12207103
  36. 36. Turner EH. Publication bias, with a focus on psychiatry: causes and solutions. CNS Drugs. 2013;27:457–68. pmid:23696308
  37. 37. Dickersin K, Rennie D. The evolution of trial registries and their use to assess the clinical trial enterprise. JAMA. 2012;307:1861–4. pmid:22550202
  38. 38. Zarin DA, Tse T, Ide NC. Trial Registration at ClinicalTrials.gov between May and October 2005. N Engl J Med. 2005;353:2779–87. pmid:16382064
  39. 39. Gibson L. GlaxoSmithKline to publish clinical trials after US lawsuit. BMJ. 2004;328:1513–3. pmid:15217848
  40. 40. Melander H. Evidence b(i)ased medicine—selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications. BMJ. 2003;326:1171–3. pmid:12775615
  41. 41. Whittington CJ, Kendall T, Fonagy P, Cottrell D, Cotgrove A, Boddington E. Selective serotonin reuptake inhibitors in childhood depression: systematic review of published versus unpublished data. Lancet. 2004;363:1341–5. pmid:15110490
  42. 42. Jureidini J, McHenry L, Mansfield P. Clinical trials and drug promotion: Selective reporting of study 329. Int J Risk Saf Med. 2008;20:73–81.
  43. 43. Eyding D, Lelgemann M, Grouven U, Härter M, Kromp M, Kaiser T, et al. Reboxetine for acute treatment of major depression: systematic review and meta-analysis of published and unpublished placebo and selective serotonin reuptake inhibitor controlled trials. BMJ 2010;341:c4737. pmid:20940209
  44. 44. Chambers CD, Feredoes E, Muthukumaraswamy SD, Etchells P. Instead of “playing the game” it is time to change the rules: Registered Reports at AIMS Neuroscience and beyond. AIMS Neurosci. 2014;1:4–17.
  45. 45. Vries d YA, Roest AM, Beijers L, Turner EH, Jonge d P. Bias in the reporting of harms in clinical trials of second-generation antidepressants for depression and anxiety: A meta-analysis. Eur Neuropsychopharmacol. 2016;26:1752–9. pmid:27659240
  46. 46. Vedula SS, Goldman PS, Rona IJ, Greene TM, Dickersin K. Implementation of a publication strategy in the context of reporting biases. A case study based on new documents from Neurontin litigation. Trials. 2012;13:136. pmid:22888801
  47. 47. Liebowitz MR, Yeung PP, Entsuah R. A randomized, double-blind, placebo-controlled trial of desvenlafaxine succinate in adult outpatients with major depressive disorder. J Clin Psychiatry. 2007;68:1663–72. pmid:18052559
  48. 48. DeMartinis NA, Yeung PP, Entsuah R, Manley AL. A double-blind, placebo-controlled study of the efficacy and safety of desvenlafaxine succinate in the treatment of major depressive disorder. J Clin Psychiatry. 2007;68:677–88. pmid:17503976
  49. 49. Septien-Velez L, Pitrosky B, Padmanabhan SK, Germain J-M, Tourian KA. A randomized, double-blind, placebo-controlled trial of desvenlafaxine succinate in the treatment of major depressive disorder. Int Clin Psychopharmacol. 2007;22:338–47. pmid:17917552
  50. 50. Feiger AD, Tourian KA, Rosas GR, Padmanabhan SK. A placebo-controlled study evaluating the efficacy and safety of flexible-dose desvenlafaxine treatment in outpatients with major depressive disorder. CNS Spectr. 2009;14:41–50. pmid:19169187
  51. 51. Liebowitz MR, Manley AL, Padmanabhan SK, Ganguly R, Tummala R, Tourian KA. Efficacy, safety, and tolerability of desvenlafaxine 50 mg/day and 100 mg/day in outpatients with major depressive disorder. Curr Med Res Opin. 2008;24:1877–90. pmid:18507895
  52. 52. Boyer P, Montgomery S, Lepola U, Germain J-M, Brisard C, Ganguly R, et al. Efficacy, safety, and tolerability of fixed-dose desvenlafaxine 50 and 100 mg/day for major depressive disorder in a placebo-controlled trial. Int Clin Psychopharmacol. 2008;23:243–53. pmid:18703933
  53. 53. Montgomery SA, Mansuy L, Ruth A, Bose A, Li H, Li D. Efficacy and safety of levomilnacipran sustained release in moderate to severe major depressive disorder: a randomized, double-blind, placebo-controlled, proof-of-concept study. J Clin Psychiatry. 2013;74:363–9. pmid:23656841
  54. 54. Asnis GM, Bose A, Gommoll CP, Chen C, Greenberg WM. Efficacy and safety of levomilnacipran sustained release 40 mg, 80 mg, or 120 mg in major depressive disorder: a phase 3, randomized, double-blind, placebo-controlled study. J Clin Psychiatry. 2013;74:242–8. pmid:23561229
  55. 55. Gommoll CP, Greenberg WM, Chen C. A randomized, double-blind, placebo-controlled study of flexible doses of levomilnacipran ER (40–120 mg/day) in patients with major depressive disorder. J Drug Assess. 2014;3:10–9. pmid:27536449
  56. 56. Sambunaris A, Bose A, Gommoll CP, Chen C, Greenberg WM, Sheehan DV. A Phase III, Double-Blind, Placebo-Controlled, Flexible-Dose Study of Levomilnacipran Extended-Release in Patients With Major Depressive Disorder. J Clin Psychopharmacol. 2014;34:47–56. pmid:24172209
  57. 57. Bakish D, Bose A, Gommoll C, Chen C, Nunez R, Greenberg WM, et al. Levomilnacipran ER 40 mg and 80 mg in patients with major depressive disorder: a phase III, randomized, double-blind, fixed-dose, placebo-controlled study. J Psychiatry Neurosci. 2014;39:40–9. pmid:24144196
  58. 58. Khan A, Cutler AJ, Kajdasz DK, Gallipoli S, Athanasiou M, Robinson DS, et al. A randomized, double-blind, placebo-controlled, 8-week study of vilazodone, a serotonergic agent for the treatment of major depressive disorder. J Clin Psychiatry. 2011;72:441–7. pmid:21527122
  59. 59. Rickels K, Athanasiou M, Robinson DS, Gibertini M, Whalen H, Reed CR. Evidence for efficacy and tolerability of vilazodone in the treatment of major depressive disorder: a randomized, double-blind, placebo-controlled trial. J Clin Psychiatry. 2009;70:326–33. pmid:19284933
  60. 60. Jain R, Mahableshwarkar AR, Jacobsen PL, Chen Y, Thase ME. A randomized, double-blind, placebo-controlled 6-wk trial of the efficacy and tolerability of 5 mg vortioxetine in adults with major depressive disorder. Int J Neuropsychopharmacol. 2013;16:313–21. pmid:22963932
  61. 61. Mahableshwarkar AR, Jacobsen PL, Chen Y. A randomized, double-blind trial of 2.5 mg and 5 mg vortioxetine (Lu AA21004) versus placebo for 8 weeks in adults with major depressive disorder. Curr Med Res Opin. 2013;29:217–26. pmid:23252878
  62. 62. Henigsberg N, Mahableshwarkar AR, Jacobsen P, Chen Y, Thase ME. A randomized, double-blind, placebo-controlled 8-week trial of the efficacy and tolerability of multiple doses of Lu AA21004 in adults with major depressive disorder. J Clin Psychiatry. 2012;73:953–9. pmid:22901346
  63. 63. Mahableshwarkar AR, Jacobsen PL, Chen Y, Serenko M, Trivedi MH. A randomized, double-blind, duloxetine-referenced study comparing efficacy and tolerability of 2 fixed doses of vortioxetine in the acute treatment of adults with MDD. Psychopharmacology. 2015;232:2061–70. pmid:25575488
  64. 64. Jacobsen PL, Mahableshwarkar AR, Serenko M, Chan S, Trivedi MH. A randomized, double-blind, placebo-controlled study of the efficacy and safety of vortioxetine 10 mg and 20 mg in adults with major depressive disorder. J Clin Psychiatry. 2015;76:575–82. pmid:26035185
  65. 65. Mahableshwarkar AR, Jacobsen PL, Serenko M, Chen Y, Trivedi MH. A randomized, double-blind, placebo-controlled study of the efficacy and safety of 2 doses of vortioxetine in adults with major depressive disorder. J Clin Psychiatry. 2015;76:583–91. pmid:26035186
  66. 66. Alvarez E, Perez V, Dragheim M, Loft H, Artigas F. A double-blind, randomized, placebo-controlled, active reference study of Lu AA21004 in patients with major depressive disorder. Int J Neuropsychopharmacol. 2012;15:589–600. pmid:21767441
  67. 67. Baldwin DS, Loft H, Dragheim M. A randomised, double-blind, placebo controlled, duloxetine-referenced, fixed-dose study of three dosages of Lu AA21004 in acute treatment of major depressive disorder (MDD). Eur Neuropsychopharmacol. 2012;22:482–91. pmid:22209361
  68. 68. Boulenger J-P, Loft H, Olsen CK. Efficacy and safety of vortioxetine (Lu AA21004), 15 and 20 mg/day: a randomized, double-blind, placebo-controlled, duloxetine-referenced study in the acute treatment of adult patients with major depressive disorder. Int Clin Psychopharmacol. 2014;29:138–49. pmid:24257717