Finding Alternatives to the Dogma of Power Based Sample Size Calculation: Is a Fixed Sample Size Prospective Meta-Experiment a Potential Alternative?

Sample sizes for randomized controlled trials are typically based on power calculations. They require us to specify values for parameters such as the treatment effect, which is often difficult because we lack sufficient prior information. The objective of this paper is to provide an alternative design which circumvents the need for sample size calculation. In a simulation study, we compared a meta-experiment approach to the classical approach to assess treatment efficacy. The meta-experiment approach involves use of meta-analyzed results from 3 randomized trials of fixed sample size, 100 subjects. The classical approach involves a single randomized trial with the sample size calculated on the basis of an a priori-formulated hypothesis. For the sample size calculation in the classical approach, we used observed articles to characterize errors made on the formulated hypothesis. A prospective meta-analysis of data from trials of fixed sample size provided the same precision, power and type I error rate, on average, as the classical approach. The meta-experiment approach may provide an alternative design which does not require a sample size calculation and addresses the essential need for study replication; results may have greater external validity.

We assume that the true log odds ratio θ is normally distributed with mean µ th theta and standard deviation (SD) σ th θ . The success rate in the control group p C is assumed to be beta distributed. The success rate in the treatment group p T is deduced from p C and θ.

Theoretical values
For one simulation, the theoretical values are sampled from the theoretical distribution: In the situation with no treatment effect µ th θ = 0, in the situation with a non null treatment effect µ th θ = log(1.5). The SD σ th θ equals 0.1. p T is calculated from the formula

Postulated hypothesis
We then consider that the postulated hypothesis used for the sample size calculation differs from the theoretical values. We let ǫ theta be the relative error on the log OR and ǫ pC the relative error on the control group event rate after an angular transformation. We used data from a previously published review [1] to calibrate these errors: they are sampled from normal distributions with mean 0.082 and SD 5.583 for ǫ theta and mean 0.026 and SD 0.301 for ǫ pC .
Furthermore, the success rates are restricted to [5%; 95%] and the treatment effect is restricted to [0.2;2.22], the 95% range observed on the data.
The number of patients to be recruited in each group is calculated as: and Z q is the q-percentile of a standard normal distribution.

Data generation
The number of events in the control group X n C and the treatment group X n T are sampled from binomial distributions. The number of events are sampled considering the true parameter p C and p T rather than the postulated ones, p post C and p post The observed treatment effect is the log odd ratioθ defined aŝ with estimated variance V

Analysis
The 95% confidence interval for the log odd ratio is calculated as

Meta-experiment approach
The meta-experiment approach involves three trials of 100 subjects each.

Theoretical values
For one simulation, the 3 sets of theoretical values are sampled from the theoretical distributions: p i T are calculated from the formula The number of subjects to be recruited in each trial is 100.

Data generation
For trial i = 1, 2, 3, the number of events in the control group X i C and the treatment group X i T are sampled from binomial distributions and the observed treatment effects are the log odds ratiosθ i defined aŝ

Analysis
The observed effectθ i for any study is given by the mean µ, the deviation of the study's true effects from the mean, and the deviation of the study's observed effect from the study's true effect. That iŝ with ξ i ∼ N (0, σ 2th θ ) and ǫ i ∼ N (0, V i ). Under the random effect model the weight assigned to each study is w i = 1 V i +τ 2 where τ 2 is an estimate of σ 2th θ from the existing studies (eg the DerSimonian-Laird estimate [2]). The meta-analysis effect estimate isθ with approximate variance The 95% confidence interval for the log odd ratio is calculated as