Current Incentives for Scientists Lead to Underpowered Studies with Erroneous Conclusions
The figure shows the product of the number of published confirmatory studies, the number of published exploratory studies, and the proportion of published studies that are correct (red = high, blue = low). This measure is calculated for when all researchers are following the rational strategy given the values of γ and ϕ. The current emphasis on a small number of publications that report novel findings is characterised by high γ and high ϕ (top right). To improve scientific output according to this measure, we could reduce ϕ (i.e., make more published studies count for researchers’ careers) or reduce γ (i.e., reduce weighting of published exploratory studies). Interestingly, the ridge is flat, so any point along it has equal fitness. Therefore, a pragmatic compromise would be to reduce both γ and ϕ by a lesser amount. The panels show the VT for two values of the dependence of acceptance on sample size m and the Type I error rate α: (A) α = 0.05, m = 3, colour range: 2.0–3.18; (B) α = 0.05, m = 6, colour range: 2.0–2.82; (C) α = 0.03, m = 6, colour range: 2.0–2.065. Other values: SC = 120, T = 2000, k = 20, fE = 0.2, rC = rE = 0.21, and σ2 = 1.