Reader Comments

Post a new comment on this article

other study that perhaps should have been included

Posted by turnere on 17 Sep 2008 at 22:48 GMT

Readers interested in this paper might also be interested in a paper I first-authored in the New England Journal of Medicine entitled Selective Publication of Antidepressant Trials and Its Influence on Apparent Efficacy (published Jan 17, 2008, available at

As I read the methods section of this paper, unless I missed something, it seems that our study of publication bias would have met inclusion criteria. Our study was based on a cohort of 74 trials registered with the FDA. At the time of registration with the FDA, drug companies are required to submit protocols. The FDA later analyzes the raw study data using the methods detailed in the original protocol.

Like this PLoS ONE paper, we also found ample evidence for selective reporting of entire studies and of outcomes within studies, and this depended heavily on whether the trial's outcome was statistically significant.

The studies included in this paper examined published study results and, by comparing them to study protocols, were able to determine whether outcome reporting bias had occurred. However, there was no gold-standard source of "real" study results to which claimed results could be compared. Thus, if a study reports P<.05, but the outcome reported on is different from the the outcome listed as primary in the original protocol, we can speculate that the result on the primary is nonsignificant, but we cannot know for sure. Having access to results from regulatory authorities such as the FDA allows us to know the actual results and thus quantify publication bias more fully.

RE: other study that perhaps should have been included

kdwan replied to turnere on 26 Sep 2008 at 15:09 GMT

Thank you for notifying us of this study. Our search strategy cut off was December 2007.

There is also another recent publication that may have been included, reference: Ramsey S, Scoggins J. Commentary: Practicing on the Tip of an Information Iceberg? Evidence of Underpublication of Registered Clinical Trials in Oncology. The Oncologist 2008;13:925–929.

This review gives a framework for updating the accumulating evidence!