Reader Comments

Post a new comment on this article

Title of paper is erroneous: most published studies are not false

Posted by plosmedicine on 30 Mar 2009 at 23:58 GMT

Author: Andrew Vickers
Position: Biostatistician
Institution: Memorial Sloan Kettering Cancer Center
Submitted Date: August 15, 2006
Published Date: August 18, 2006
This comment was originally posted as a “Reader Response” on the publication date indicated above. All Reader Responses are now available as comments.

I agree entirely with Dr Ioannidis' general conceptual framework: we undoubtedly need to focus on the relationship between the amount of evidence provide by a study and the prior probability of a hypothesis rather than on p values in isolation. However, the claim that "Most Published Research Findings Are False" is based on a false premise - that medical research is only about hypothesis testing, rather than estimation - and an unsubstantiated generalization about the typical distribution of prior probabilities in medical research.

I just looked up the five most recently published studies that I co-authored. Three were concerned with estimation, for example, the expected survival of patients with a particular type of cancer and the proportion of patients undergoing a surgical procedure that would be eligible for adjuvant chemotherapy. None of Dr Ioannidis' arguments have any bearing on the value of these studies. The fourth study examined whether FCGR2A polymorphism is associated with outcome of an antibody treatment that involving the FC receptor. This strikes me as having a high prior probability, and indeed, we did find that polymorphism predicted outcome. The final study examined risk factors for teratoma found at lymph node dissection for testicular cancer. We concluded that response to initial chemotherapy and teratoma in the orchiectomy specimen were predictive. Again, one would hardly give this a low prior probability.

In sum, I agree with Dr Ioannidis' methods, but not his conclusions.

No competing interests declared.