Table 1.
Simulation parameters.
Fig 1.
Proportion of type-1 errors for overall effect size estimate d in the absence of publication bias and p-hacking for five heterogeneity estimators as a function of true heterogeneity (τ) and number of studies per meta-analysis (k), with α-level = 0.05.
Fig 2.
Bias in heterogeneity estimates (Tbias) in the absence of publication bias and p-hacking for five heterogeneity estimators as a function of true heterogeneity (τ) and number of studies in the meta-analysis (k).
Fig 3.
Root mean square error for heterogeneity estimates (TRMSE) in the absence of publication bias and p-hacking for five heterogeneity estimators as a function of true heterogeneity (τ) and number of studies in the meta-analysis (k).
Table 2.
The relative importance of design factors for Tbias.
Selected sum of squares from six-factorial ANOVA for five heterogeneity estimators.
Fig 4.
Bias in heterogeneity estimates (Tbias) for five heterogeneity estimators as a function of true heterogeneity (τ), true average effect size (θ), type of publication bias, and p-hacking environment, respectively.
Fig 5.
Bias in heterogeneity estimates (Tbias) for five heterogeneity estimators: Two-way interaction of true average effect size (θ) with type of publication bias.
Fig 6.
Bias in heterogeneity estimates (Tbias) for five heterogeneity estimators: Two-way interaction of true average effect size (θ) with strength of publication bias.
Table 3.
The relative importance of design factors for Trmse.
Selected sum of squares from six-factorial ANOVA for five heterogeneity estimators.
Fig 7.
Root mean square error for heterogeneity estimates (TRMSE) for five heterogeneity estimators as a function of number of studies in the meta-analysis (k), true heterogeneity (τ) true average effect size (θ), and p-hacking environment, respectively.
Fig 8.
Bias in effect size estimates (dbias) as a function of p-hacking environment, strength of publication bias, true average effect size (θ), and type of publication bias.
Data shown are for the DL estimator but are very similar for other estimators.
Table 4.
The relative importance of design factors for dbias (selected sum of squares from six-factorial between-subjects ANOVA).
Data are shown for DL but were very similar across all heterogeneity estimators.
Fig 9.
Type-1 error rates for d under 1-tailed publication bias as a function of strength of publication bias, level of p-hacking, and number of studies in the meta-analysis (k).
Data shown are for conditions with θ = 0 and are based on the DL estimator but are very similar for other estimators.
Fig 10.
Comparison of errors in estimation of effect size and heterogeneity.
Estimation errors for d are for the DL estimator, but virtually identical for the other estimators.