Table 1.
Prevalence of questionable research practices.
Fig 1.
Factors contributing to irreproducible TMS results.
When planning and executing a research study, the size of the investigated effect and the size of the sample directly influence a study’s statistical power—probability of correctly rejecting the null hypothesis when the null hypothesis is false—and the certainty of reported researcher results. Selecting sample size based on previous experience, published reports or power calculations based on inflated effect sizes from the literature often results in too few subjects being tested. In a study with low statistical power, significant results (i.e. p < 0.05) are biased towards extreme values(i.e. a large effect; study B). Independently, questionable research practices will also increase the rate of false discoveries and exaggerated effect sizes. Because these results meet the traditional level of statistical significance, they will likely become part of published literature. For the unlucky scientist who did not find statistically significant results (study A), the study may never be written up or it will be rejected by publishers because it presents uncertain, negative results. These studies become part of the cemetery of unpublished scientific research, the file-drawer.