Current Incentives for Scientists Lead to Underpowered Studies with Erroneous Conclusions
Parameter γ reflects the relative importance of published exploratory studies. The lines show predictions for two values of the probability that an effect is real (fE) and two values of the effect sizes rC and rE (solid: fE = 0.2, rC = rE = 0.21; dotted: fE = 0.3, rC = rE = 0.21; dashed: fE = 0.2, rC = rE = 0.32). The panels show (A) the optimal proportion of total sampling to spend on exploratory studies θ*, (B) the optimal sample size of exploratory studies SE*, (C) the resultant total number of published studies NE + NC, (D) the proportion of published studies that are confirmatory NC / (NE + NC), (E) the statistical power of exploratory studies WE, and (F) the proportion of published studies that draw incorrect conclusions (PF). Other values: SC = 120, T = 2,000, k = 20, α = 0.05, σ2 = 1, m = 3, and ϕ = 0.8. The chosen values for rC = rE reflect data reported by Richard and colleagues , where a correlation coefficient mode of 0.09 and a mean of 0.21 were observed. These values are in the middle of the range of effect sizes observed in meta-analyses across a number of biomedical research domains (range r ~ 0.15 to 0.50) .