Fig 1.
Simulation study with a non-null treatment effect.
Theoretical distributions are used to draw the true treatment effects and success rates in the control group for only one trial with the conventional approach and for each of the 3 trials with the meta-experiment. The log odds ratio θ is drawn from a normal distribution with mean log(1.5) and SD 0.1. The success rate in the control group pc is drawn from a beta distribution with mean 30% and SD 10%. With the conventional approach, relative errors are simulated to deduce the postulated hypothesis in designing the trial. The sample size 2n is calculated to ensure 80% power. A trial of size 2n is simulated from the true treatment effect and success rate, and analyzed. With the meta-experiment approach, the same theoretical distributions are used to draw 3 treatments effects θ1, θ2 and θ3, and 3 success rates and
. 3 trials with sample size 100 each are simulated from the 3 treatment effects and success rates. Then the data are meta-analyzed in a random-effects model, allowing for variation between the results of the 3 trials.
Fig 2.
Box plot of the 10,000 calculated samples sizes based on the conventional approach.
These sample sizes are used for the situation of mean theoretical odds ratio = 1.5. The dotted line represents 300 patients, which corresponds to the number of patients involved in the meta-experiment.
Fig 3.
Statistical performance of conventional and meta-experiment approaches to determining sample size (1 trial vs 3 trials).
(a) For a treatment effect (mean theoretical odds ratio (OR) = 1.5), power and coverage of the 95% confidence interval (95% CI) are estimated. (b) For no treatment effect (mean theoretical OR = 1), the type I error rate is estimated. For both scenarios, box plots of 95% CIs are presented. Results were derived from a simulation study with 10,000 runs for an expected mean success of 30% for the control group (i.e., considering a Beta distribution with mean 30% and SD 10%), and SD = 0.1 for the theoretical log OR.