https://orcid.org/0000-0001-9864-8443
Figures
Abstract
Objectives
Little is known about the integrity of randomization for randomized controlled trials (RCT) included in alteplase for stroke meta-analyses. If the RCTs were not properly randomized, the results could not be accepted at face value. The objective was to assess the integrity of randomization in individual patient data (IPD) meta-analyses supporting alteplase for acute ischemic stroke.
Methods
We assessed randomization reporting, performed qualitative risk of bias assessments arising from the randomization process, and performed fixed effects meta-analyses of baseline variables for which zero heterogeneity is expected if all included RCTs have unbiased randomization. Fixed-effects meta-analyses of baseline age, weight, and National Institute of Health Stroke Scale (NIHSS) score were performed. If heterogeneity was present (I2 > 0%), trials were systematically removed, starting with the RCT with the largest t-statistic until the I2 value was 0%.
Results
The NINDS rt-PA Stroke Study had a high risk of bias, the ECASS-3 RCT had some concerns, and all other trials were graded as low risk according to the Cochrane Risk of Bias (ROB-2) tool. The NINDS rt-PA Stroke Study contributed to heterogeneity in age and weight meta-analyses, and the ECASS-3 RCT contributed to heterogeneity in the NIHSS score meta-analysis. Removal of suspect trials resulted in the expected I2 value of 0%.
Citation: Garg R, Torrealba-Acosta G, Mickenautsch S, Berger VW (2025) A methodological assessment of randomization integrity in alteplase for acute ischemic stroke individual patient data meta-analyses. PLoS ONE 20(3): e0315342. https://doi.org/10.1371/journal.pone.0315342
Editor: Cem Bilgin,, Mayo Clinic, UNITED STATES OF AMERICA
Received: June 5, 2024; Accepted: February 2, 2025; Published: March 19, 2025
This is an open access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.
Data Availability: Data Availability: The IST-3 RCT data set can be accessed here: https://github.com/ravigarg415/IST_3 while the NINDS rt-PA Stroke Study data set can be requested from the National Institute of Neurological Disorders and Stroke here: https://www.ninds.nih.gov/current-research/research-funded-ninds/clinical-research/archived-clinical-research-datasets.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Meta-analyses of randomized controlled trials (RCTs) are considered the highest level of evidence for clinical practice guideline (CPG) recommendations [1]. Meta-analyses are observational by nature and retain all the biases of component RCTs [2]. Component RCTs rely on randomization to control for selection bias and ensure the validity of statistical testing. Selection bias may be difficult to assess after randomization and depends on methodological reporting. Baseline imbalances following randomization may be a symptom of selection bias or a chance finding. The Cochrane Risk of Bias 2 (RoB 2) tool recommends assessing for baseline imbalances but emphasizes that “baseline differences that are compatible with chance do not lead to a risk of bias” [3]. A chance finding can only be assured, however, if both the random generation of the allocation sequence and the actual randomization procedure were unbiased [4]. For RCTs that employ highly restrictive randomization procedures, such as permuted block randomization, or that have poor methods of allocation sequence concealment, the assumption of unbiased randomization and baseline differences due to chance is a common misconception [4, 5]. Unlike missing outcome data, for which sensitivity analyses can be performed on summary data to ensure study results are robust, selection bias is assessed primarily by a qualitative audit of the randomization process. Adequate self-reporting of the randomization process, however, does not necessarily equate to effective randomization [4].
Hicks and colleagues have popularized an agnostic analysis to assess for bias introduced by component RCTs included in meta-analyses by inadequate allocation sequence concealment and other sources of biased randomization [6–12]. This method relies on the principle that there should be zero heterogeneity in a fixed-effects meta-analysis of baseline continuous variables from component RCTs.
A recent acute ischemic stroke CPG recommends alteplase therapy within 4.5 hours of symptom onset based on meta-analyses of RCTs [1]. Of the eight component RCTs used in recent individual patient data (IPD) meta-analyses, two have suffered from substantial baseline imbalances in important prognostic factors following randomization: the NINDS rt-PA Stroke Study and ECASS-3 [13, 14]. In the NINDS rt-PA Stroke Study, there were imbalances in baseline aspirin use; baseline stroke severity measured by the National Institute of Health Stroke Scale (NIHSS) score; stroke sub-type; and baseline computed tomography (CT) all favoring the alteplase arm [14]. In the ECASS-3 RCT, there was similarly an imbalance in the baseline NIHSS score and additionally, an imbalance in prior stroke status both favoring the alteplase arm [13]. Although multiple reanalyses have been published focusing on covariate adjustments, latent covariate imbalances cannot be accounted for if the randomization is not valid [15].
The possibility that component RCTs of alteplase treatment for acute ischemic stroke introduced bias arising from the randomization process into meta-analyses has not been assessed. Therefore, the purpose of this study was to assess for bias arising from the randomization process in component RCTs included in alteplase for ischemic stroke IPD meta-analyses.
Methods
The publicly available NINDS rt-PA Stroke Study and IST-3 data sets were used for these analyses. The data was first accessed for research purposes on July 26th, 2022. The authors did not have access to information that could identify individual participants during or after data collection.
Study design and population
Institutional review board or ethics approval was not obtained (exempt determination) based on guidance at the authors institutions. Randomized controlled trials used in IPD meta-analyses by the Stroke Thrombolysis Trialists’ (STT) Collaborative Group were chosen for this analysis [16–21]. The STT Collaborative Group pooled IPD from nine RCTs for meta-analyses: NINDS A (N = 291); NINDS B (N = 333); ECASS I (N = 620); ECASS II (N = 800); ATLANTIS A (N = 142); ATLANTIS B (N = 613); ECASS III (N = 821); EPITHET (N = 101); and IST-3 (N = 3035) for a total of 6756 participants [22–29]. For this analysis, NINDS-A and NINDS-B were considered one trial as has been previously done given continuous, shared randomization between the two parts [15]. We excluded the ECASS 1 RCT, as has been previously done, to mimic treatment estimates used to support recommendations from clinical practice guidelines labeled “patients who would have met a 4.5 hour revised US label” by IPD meta-analysis authors [21]. The primary outcome of the meta-analysis was a modified Rankin Scale score of 0-1 at 3-6 months post-stroke. The Oxford Handicap Scale score (OHS) was treated as an equivalent to the mRS score.
Data analysis
Data was abstracted from the published RCTs independently by two researchers (R.G. and G.T.A.). Data from the NINDS rt-PA Stroke Study and IST-3 RCT were abstracted from the publicly available datasets. Key components for randomization methods were assessed for all component RCTs. We assessed the risk of bias of RCTs included in the meta-analysis using the RoB 2 tool [3]. We compared our risk of bias assessments to those in the included meta-analyses.
The technique proposed by Hicks et al. was used for the primary analysis [6]. Three continuous variables were abstracted from component RCTs using a standardized form: age, NIHSS score, and weight. Age and NIHSS score were chosen due to their status as being the most important prognostic variables for ischemic stroke outcomes for which any mean difference between groups, irrespective of the p-value from null hypothesis significance testing (NHST), would be considered relevant in the assessment of post-randomization confounding [30]. Weight was chosen as a validation variable to ensure the integrity of the analysis due to consistent reporting in alteplase for stroke RCTs.
The t-statistic was calculated from an independent two-sample t-test for the difference in baseline continuous variables, and the studies were ranked, in descending order, by the absolute value of their t-statistic for each variable. Trials expressing baseline variables as medians were excluded. A fixed-effects meta-analysis for each variable was performed to measure heterogeneity by I2 and trials were removed with subsequent iterations until the meta-analysis revealed no heterogeneity (I2 = 0%). To assess the assumptions of the procedure, it was repeated performing a random-effects meta-analysis (DerSimonian and Laird method) starting with the trial with the two smallest t-statistics and adding RCTs based on ascending order of their t-statistics until heterogeneity was observed. The point estimates and 95% confidence intervals for I2 and τ2 were assessed at each iteration. The confidence intervals for I2 and τ2 were determined using non-parametric bootstrapping and unequal tail probabilities [31]. All analyses were performed in R Studio version 2023.06.1 + 524. R code for the analysis can be provided for reasonable requests.
Results
Randomization methods reporting and risk-of-bias arising from the randomization process
The randomization methods and risk-of-bias for the included RCTs were reported in Table 1.
Based on ROB-2, the NINDS rt-PA Stroke Study was rated as having a high risk of bias arising from the randomization process (S1 Table); while the ECASS-3 was rated as having some concerns (S4 Table). The remaining five RCTs [ATLANTIS (S2 Table); ECASS-2 (S3 Table); EPITHET (S5 Table); and IST-3 (S6 Table)] were rated as having a low risk of bias arising from the randomization process. In addition to being the only trials that were not graded as a low risk of bias, the NINDS rt-PA Stroke Study and ECASS-3 RCT were the only RCTs with baseline imbalances in important prognostic variables favoring the treatment arm (S1 Table; S4 Table). The risk of bias arising from the randomization process was not reported in the five IPD meta-analyses published.
Heterogeneity in baseline variables
The differences in independent means of age, weight, and NIHSS score; and their associated t-statistics were reported in S7 Table. The EPITHET RCT only reported age as a mean ± SD while the NIHSS was reported as a median and weight was unreported [22]. The ECASS-2 RCT did not report continuous baseline characteristics as means ± standard deviations or report interquartile ranges associated with medians and was excluded from the analysis [25]. The NINDS rt-PA Stroke Study had the highest t-statistic for age (2.19) and weight (2.69). The ECASS-3 RCT had the highest t-statistic for NIHSS (2.24) (S7 Table).
A fixed-effects meta-analysis for age revealed the NINDS rt-PA Stroke Study was the only study for which the 95% confidence interval did not include the null. Heterogeneity was present (I2 = 15%, 95% CI 0-50%) [Fig 1].
The NINDS rt-PA Stroke Study (NINDS) is the only included study for which the 95% confidence interval did not include the null.
Exclusion of the NINDS rt-PA Stroke Study was required to achieve no heterogeneity (I2 = 0%, 95% CI 0-0%).
A fixed-effects meta-analysis for weight revealed the NINDS rt-PA Stroke Study was the only study for which the 95% confidence interval did not include the null. Heterogeneity was present (I2 = 45%, 95% CI 0-68%) [Fig 2]. Exclusion of the NINDS rt-PA Stroke Study was required to achieve no baseline heterogeneity (I2 = 0%, 95% CI 0-0%).
The NINDS rt-PA Stroke Study was the only included study for which the 95% confidence interval did not include the null.
A fixed-effects meta-analysis for NIHSS revealed the ECASS 3 RCT was the only study for which the 95% confidence interval did not include the null. Heterogeneity was present (I2 = 31%, 95% CI 0-55%) [Fig 3]. Exclusion of the ECASS 3 RCT was required to achieve no baseline heterogeneity (I2 = 0%, 95% CI 0-0%). The procedure for all three variables was validated in reverse (S1–S3 Figs). The results were robust to changes in sample size and were identical using τ2.
The ECASS-3 RCT was the only included study for which the 95% confidence interval did not include the null.
Quantification of selection bias
The pooled absolute risk difference, using the DerSimonian-Laird method, for all included trials was 3% (95% CI, -1% - 8%) (Fig 4). After the exclusion of the NINDS rt-PA Stroke Study and ECASS-3, the pooled absolute risk difference was reduced to 1% (95% CI, -4% - 6%) (Fig 5). The I2 estimate including all studies was 58% compared to 50% after removal of the NINDS rt-PA Stroke Study and ECASS-3.
Discussion
Qualitative risk of bias assessments arising from the randomization process, based on ROB 2, graded the NINDS rt-PA Stroke Study as a high risk of bias while the ECASS-3 RCT had some concerns. Both trials inadequately reported random sequence generation and randomization implementation, and both trials contributed to heterogeneity in the fixed-effects meta-analyses of baseline variables. Taken together, these trials are a source of systematic error in meta-analyses. We estimated that 2% of the average treatment effect (ATE) can be attributed to selection bias.
Our results found congruence between the qualitative risk of bias assessments and the evaluation of heterogeneity in the fixed-effects meta-analyses of baseline variables such that the two trials that were not graded a low-risk of bias (NINDS rt-PA Stroke Study and ECASS-3 RCT), also contributed to heterogeneity in the meta-analyses. Other authors, however, have reported discrepant results between the qualitative risk of bias assessments and the evaluation of heterogeneity in fixed-effects meta-analyses of baseline variables. For example, in a meta-analysis of RCTs of transversus abdominis plane block efficacy in hysterectomy, six RCTs were graded with a low risk of bias arising from the randomization process, and one was rated as having some concerns. Despite the favorable qualitative risk of bias assessments, meta-analyses of two baseline variables identified heterogeneity, while there should have been none if randomization was unbiased [6].
There are other limitations to full reliance on the qualitative risk of bias assessments. The ROB 2 tool is dependent on the self-reporting of RCTs. For instance, in a meta-analysis on the effect of vitamin K supplementation on bone mineral density and fractures, most trials had an unclear risk of bias in the random sequence generation or allocation sequence concealment domains [10]. A fixed-effects meta-analysis of baseline age revealed no heterogeneity providing reassurance about the integrity of randomization in these individual studies. These examples support previous recommendations on more detailed reporting in RCTs to control for selection bias (Table 2) [32].
The NINDS rt-PA Stroke Study and ECASS-3 RCT have similarities that are worth noting. Both RCTs had baseline imbalances in important prognostic variables (S1 and S4 Tables) and were also the only trials that contributed to heterogeneity in meta-analyses of baseline variables. The use of NHST to assess for differences at baseline is controversial [33]. A major argument against the routine use of NHST is that unbiased randomization produces an expected distribution of p values that does not necessarily apply to a single hypothesis test. Additionally, this distribution may be affected by the correlation between baseline characteristics and the number of hypothesis tests performed. Conversely, the premise of these arguments is that randomization, and not just random sequence generation, is unbiased which may be a foregone conclusion. Our results support the use of NHST at baseline and are consistent with other RCTs for which substantial baseline imbalances, coupled with flawed randomization methods, raised suspicion of selection bias [4].
Both RCTs employed permuted block randomization for which the potential for third-order selection bias has been thoroughly described [5]. Permuted blocks facilitate predictability in future allocations and may lead to more favorable patient selection in the experimental arm, and this has been referred to as the convergent strategy of guessing without certainty [34]. In the NINDS rt-PA Stroke Study, varying block sizes were used. Contrary to popular belief, varying the block size does not lessen the chance of randomization subversion [34, 35]. Rather, this strategy may be as problematic as using fixed block sizes. In a review of 179 open-label RCTs employing the same methodology chosen for the current report, the authors found baseline heterogeneity for age in all trials implementing a block randomization scheme, trials implementing a fixed block size of 4, and trials implementing varied block sizes including a block size of 2 [36]. In the ECASS-3 RCT, a fixed block size of four was used.
The principal issue with varied block sizes, which are smaller on average, and a fixed block size of four is the same: high predictability of future allocations. To illustrate, if the first allocation is identified as alteplase, the second allocation is twice as likely to be placebo as alteplase using a block size of 4. If a record of allocations is kept the last allocation is 100% predictable in 1/3rd of block sequence permutations [36]. Even if the allocation sequence is strictly concealed, the potential for accidental unblinding in alteplase trials is high by intracerebral hemorrhage, systemic bleeding, or angioedema. Additionally, neither the NINDS rt-PA Stroke Study nor the ECASS-3 RCT reported how the frothing reaction that occurs with alteplase reconstitution was mimicked in the placebo arm as is explicitly reported in more modern alteplase RCTs [37].
The EPITHET and ATLANTIS RCTs also employed blocked randomization. Still, they did not contribute to baseline heterogeneity in the age meta-analysis suggesting other elements of susceptible randomization may be required to precipitate randomization subversion. For example, randomization was decentralized in the NINDS rt-PA Stroke Study. Other elements that precipitate randomization subversion, such as the motivation of investigators or conflicts of interest, are unlikely to be fully substantiated post hoc.
Although permuted blocks facilitate the convergent strategy of guessing and may introduce third-order selection bias, the ability to directly subvert randomization is also possible; and this has been suspected in the NINDS rt-PA Stroke Study [14]. In this trial, sealed envelopes with the treatment assignments were attached to study pre-packs for purposes of emergent unblinding for adverse events. At the end of the trial, sixteen envelopes were opened for unblinding of which eight did not have listed safety reasons. Comparatively, only five envelopes were opened for unblinding in the ECASS-2 RCT which had a larger sample size. The NINDS rt-PA Stroke trial data revealed deviations from the expected 1:1 allocation ratio in most randomization strata; violation of a pre-specified randomization rule at 3 centers; and multiple baseline imbalances in prognostic variables favoring the alteplase arm when evaluated on exclusion criteria (S8 Table) [38]. A similarly detailed audit of the randomization process of other included trials, including the ECASS-3 RCT, cannot be performed due to a lack of publicly available trial data, study protocols, or product licensing applications.
Our results have important implications for meta-analyses and CPG recommendations regarding alteplase for acute ischemic stroke. First, we found that meta-analyses, including the NINDS rt-PA Stroke Study and ECASS-3, overestimated the ATE by 2%. Importantly, covariate adjustments do not adequately control this bias, as flawed randomization might lead to latent covariate imbalances. To perform a meta-analysis without removing suspect trials, the reverse propensity score (RPS) could be used to correct for selection bias [39]. The RPS is the conditional probability of a patient being assigned a specific treatment given prior allocations under block randomization. The randomization schedules are required to calculate the RPS, however, and are not publicly available.
As meta-analyses are widely considered the pinnacle of the evidence-based medicine pyramid, our findings have important implications for CPG recommendations. Currently, CPGs do not acknowledge the presence of selection bias in the NINDS rt-PA Stroke Study and ECASS-3 [1,40]. Aligned with the best methodological practices for CPG development, we believe the strength of the recommendation, currently “strong”, should be reconsidered based on limitations in the quality of the evidence [41].
Despite the inherent risk of selection bias, permuted block randomization remains a common randomization procedure in stroke trials and may be a more pervasive problem than appreciated. For example, a recent thrombolytic trial employing stratified block randomization resulted in unequal group allocations by the stratifying factor and ten baseline factors favoring the thrombolytic arm concerning for selection bias [42, 43]. Therefore, methods that are superior to permuted block randomization, such as the asymptotic maximal procedure, simple randomization, and minimization, should be used. Bayesian adaptive randomization designs have also recently been successfully employed in stroke clinical trials [44]. Alternatively, if permuted block randomization is performed, the integrity of the process should be verified using the RPS.
Limitations
The current report has limitations that are worth noting. The IPD meta-analyses include the ECASS-2 RCT which was excluded in the current report due to a lack of reporting of the chosen continuous variables. In ECASS-2, the median age and NIHSS were reported without their associated interquartile ranges. We believe this deviates from the best practices of baseline reporting. As previously noted, only trials that were not graded with a low risk of bias contributed to heterogeneity while ECASS-2 was graded with a low risk of bias. Given the heterogeneity contributions of the NINDS rt-PA Stroke Study and ECASS-3 RCT, it is unlikely that the exclusion of ECASS-2 meaningfully changes the conclusions as we also validated the procedure in reverse.
We found discrepancies in the trials that contributed to heterogeneity in meta-analyses: the NINDS rt-PA Stroke Study contributed to heterogeneity in age and weight meta-analyses while the ECASS-3 RCT only contributed to heterogeneity in the NIHSS score meta-analysis. It is notable in the meta-analysis of baseline NIHSS, that the 95% confidence interval begins to increase only after the addition of the NINDS rt-PA Stroke Study when the procedure was conducted in reverse, although the point estimate remained 0 (S3 Fig). Importantly, the chosen variables may not reflect the baseline characteristic that randomization was subverted on as this may be an unknown or categorical covariate. Rather, there may be an association between the variable for which heterogeneity was detected and the baseline characteristic(s) that randomization was subverted on. Alternatively, the discrepancies in heterogeneity may simply be a chance finding and this has been previously described using the current methodology in a systematic review of spinal manipulative therapy for chronic low back pain [7]. We agree with the original authors of the current method that any observed baseline heterogeneity is a cause for concern. As such, the reported results are best served in congruence, as opposed to isolation, with trial-level data. In the case of the NINDS rt-PA Stroke Study, the reported results are consistent with a risk of selection bias assessment using trial-level data [14]. A similar trial-level assessment of the ECASS-3 RCT could confirm or deny these findings.
We were unable to replicate multiple subgroup analyses published in IPD meta-analyses. Any subgroup, however, would retain systematic error found in the pooled estimate of all trials. The current report also only addressed bias arising from the randomization process. Other limitations of the IPD meta-analyses such as unblinding in the IST-3 RCT; handling of missing outcomes; and inclusion of RCTs with small sample sizes were not addressed in the current report and should also be considered [45–47].
Due to the lack of diversity in randomization methods used in the included trials, we could not substantiate if certain randomization procedures inherently impact the detection of baseline heterogeneity in the current method. Five out of six included trials used permuted block randomization, and one used minimization. Due to the predictability of future allocations, we believe this randomization method has a specifically higher risk of selection bias than others such as minimization with a random factor as used in the IST-3 RCT.
Finally, as previously noted, we did not have access to the randomization schedules and were unable to calculate an RPS to definitively detect and correct for selection bias; or assess for over-stratification.
A strength of the current report is the use of agnostic, validated analysis using summary data to assess randomization. The procedure has been reported in meta-analyses of tranexamic acid for post-partum hemorrhage; vitamin K supplementation on bone mineral density and fractures; transversus abdominis plane block efficacy in hysterectomy; interventions for the management of primary frozen shoulder; and atypical antipsychotics in dementia suggesting strong external validation [8–12]. The procedure was robust to assumptions, including changes in sample size, consistent with a recent simulation [48]. An important future area of meta-epidemiological research is determining, using the current method, whether certain randomization procedures portend a higher risk of selection bias than others.
Conclusions
Among RCTs included in IPD meta-analyses of alteplase for acute ischemic stroke, the NINDS rt-PA Stroke Study had a high risk of bias arising from the randomization process while the ECASS-3 RCT had some concerns. Both trials inadequately reported randomization and contributed to heterogeneity in fixed effects meta-analyses of baseline variables suggesting biased randomization. Results from these RCTs, and meta-analyses that include these RCTS, should be accepted cautiously.
Supporting Information
S1 Table. Risk of Bias Arising from the Randomization Process in the NINDS rt-PA Stroke Study.
https://doi.org/10.1371/journal.pone.0315342.s001
(DOCX)
S2 Table. Risk of Bias Arising from the Randomization Process in the ATLANTIS-A and ATLANTIS-B Randomized Clinical Trials.
https://doi.org/10.1371/journal.pone.0315342.s002
(DOCX)
S3 Table. Risk of Bias Arising from the Randomization Process in the ECASS-2 Randomized Clinical Trial.
https://doi.org/10.1371/journal.pone.0315342.s003
(DOCX)
S4 Table. Risk of Bias Arising from the Randomization Process in the ECASS-3 trial.
Abbreviations: NIHSS, National Institute of Health Stroke Scale.
https://doi.org/10.1371/journal.pone.0315342.s004
(DOCX)
S5 Table. Risk of Bias Arising from the Randomization Process in the EPITHET trial.
https://doi.org/10.1371/journal.pone.0315342.s005
(DOCX)
S6 Table. Risk of Bias Arising from the Randomization Process in the IST-3 trial.
https://doi.org/10.1371/journal.pone.0315342.s006
(DOCX)
S7 Table. Mean differences in age, weight, and National Institute of Health Stroke Scale scores; and the associated t-statistic for all included trials.
https://doi.org/10.1371/journal.pone.0315342.s007
(DOCX)
S8 Table. Overview of selection bias in the NINDS rt-PA Stroke Study.
https://doi.org/10.1371/journal.pone.0315342.s008
(DOCX)
S1 Fig. The I2 point estimate (orange dot) and associated 95% confidence intervals for the age meta-analysis
https://doi.org/10.1371/journal.pone.0315342.s009
(TIF)
S2 Fig. The I2 point estimate (orange dot) and associated 95% confidence intervals for the weight meta-analysis.
https://doi.org/10.1371/journal.pone.0315342.s010
(TIF)
S3 Fig. The I2 point estimate (orange dot) and associated 95% confidence intervals for the NIHSS meta-analysis.
https://doi.org/10.1371/journal.pone.0315342.s011
(TIF)
References
- 1. Powers WJ, Rabinstein AA, Ackerson T, Adeoye OM, Bambakidis NC, Becker K, et al. Guidelines for the early management of patients with acute ischemic stroke: 2019 update to the 2018 guidelines for the early management of acute ischemic stroke: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke. 2019;50(12):e344–418.
- 2.
Borzak S, Ridker PM. Discordance between meta-analyses and large-scale randomized, controlled trials: examples from the management of acute myocardial infarction. American College of Physicians. 1995, p. 873–7.
- 3. Sterne JA, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: a revised tool for assessing risk of bias in randomized trials. BMJ. 2019;366.
- 4. Berger VW, Weinstein S. Ensuring the comparability of comparison groups: is randomization enough?. Control Clin Trials. 2004;25(5):515–24. pmid:15465620
- 5.
Berger V. Selection bias and covariate imbalances in randomized clinical trials. John Wiley & Sons; 2007.
- 6. Hicks A, Fairhurst C, Torgerson DJ. A simple technique investigating baseline heterogeneity helped to eliminate potential bias in meta-analyses. J Clin Epidemiol. 2018;9555–62. pmid:29032245
- 7. Clark L, Fairhurst C, Cook E, Torgerson DJ. Important outcome predictors showed greater baseline heterogeneity than age in two systematic reviews. J Clin Epidemiol. 2015;68(2):175–81. pmid:25457029
- 8. López-Ruiz C, Orjuela JC, Rojas-Gualdrón DF, Jimenez-Arango M, Ríos JF de L, Vásquez-Trespalacios EM, et al. Efficacy of transversus abdominis plane block in the reduction of pain and opioid requirement in laparoscopic and robot-assisted hysterectomy: a systematic review and meta-analysis. Rev Bras Ginecol Obstet. 2022;44(1):55–66. pmid:35092960
- 9. Ker K, Shakur H, Roberts I. Does tranexamic acid prevent postpartum haemorrhage? A systematic review of randomised controlled trials. BJOG. 2016;123(11):1745–52. pmid:27558956
- 10. Mott A, Bradley T, Wright K, Cockayne ES, Shearer MJ, Adamson J, et al. Effect of vitamin K on bone mineral density and fractures in adults: an updated systematic review and meta-analysis of randomised controlled trials. Osteoporos Int. 2019;30(8):1543–59. pmid:31076817
- 11. Rex SS, Kottam L, McDaid C, Brealey S, Dias J, Hewitt CE, et al. Effectiveness of interventions for the management of primary frozen shoulder : a systematic review of randomized trials. Bone Jt Open. 2021;2(9):773–84. pmid:34555926
- 12. Hulshof T, Zuidema S, van Meer P, Gispen-de Wied C, Luijendijk H. Baseline imbalances and clinical outcomes of atypical antipsychotics in dementia: A meta-epidemiological study of randomized trials. Int J Methods Psychiatr Res. 2019;28(1):e1757.
- 13. Alper BS, Foster G, Thabane L, Rae-Grant A, Malone-Moses M, Manheimer E. Thrombolysis with alteplase 3-4.5 hours after acute ischaemic stroke: trial reanalysis adjusted for baseline imbalances. BMJ Evid Based Med. 2020;25(5):168–71. pmid:32430395
- 14. Garg R, Mickenautsch S. Risk of selection bias assessment in the NINDS rt-PA stroke study. BMC Med Res Methodol. 2022;22(1):172. pmid:35705913
- 15. Ingall TJ, O’Fallon WM, Asplund K, Goldfrank LR, Hertzberg VS, Louis TA, et al. Findings from the reanalysis of the NINDS tissue plasminogen activator for acute ischemic stroke treatment trial. Stroke. 2004;35(10):2418–24. pmid:15345796
- 16. Stroke Thrombolysis Trialists’ Collaborative Group. Details of a prospective protocol for a collaborative meta-analysis of individual participant data from all randomized trials of intravenous rt-PA vs. control: statistical analysis plan for the Stroke Thrombolysis Trialists’ Collaborative meta-analysis. Int J Stroke. 2013;8(4):278–83. pmid:23639145
- 17. Whiteley WN, Emberson J, Lees KR, Blackwell L, Albers G, Bluhmki E, et al. Risk of intracerebral haemorrhage with alteplase after acute ischaemic stroke: a secondary analysis of an individual patient data meta-analysis. Lancet Neurol. 2016;15(9):925–33. pmid:27289487
- 18. Emberson J, Lees KR, Lyden P, Blackwell L, Albers G, Bluhmki E, et al. Effect of treatment delay, age, and stroke severity on the effects of intravenous thrombolysis with alteplase for acute ischaemic stroke: a meta-analysis of individual patient data from randomised trials. Lancet. 2014;384(9958):1929–35. pmid:25106063
- 19. Lees KR, Emberson J, Blackwell L, Bluhmki E, Davis SM, Donnan GA, et al. Effects of Alteplase for Acute Stroke on the Distribution of Functional Outcomes: A Pooled Analysis of 9 Trials. Stroke. 2016;47(9):2373–9. pmid:27507856
- 20. Bluhmki E, Danays T, Biegert G, Hacke W, Lees KR. Alteplase for acute ischemic stroke in patients aged >80 years: pooled analyses of individual patient data. Stroke. 2020;51(8):2322–31. pmid:32611284
- 21. Hacke W, Lyden P, Emberson J, Baigent C, Blackwell L, Albers G, et al. Effects of alteplase for acute stroke according to criteria defining the European Union and United States marketing authorizations: Individual-patient-data meta-analysis of randomized trials. Int J Stroke. 2018;13(2):175–89. pmid:29171359
- 22. Davis SM, Donnan GA, Parsons MW, Levi C, Butcher KS, Peeters A, et al. Effects of alteplase beyond 3 h after stroke in the Echoplanar Imaging Thrombolytic Evaluation Trial (EPITHET): a placebo-controlled randomised trial. Lancet Neurol. 2008;7(4):299–309. pmid:18296121
- 23. Clark WM, Wissman S, Albers GW, Jhamandas JH, Madden KP, Hamilton S. Recombinant tissue-type plasminogen activator (Alteplase) for ischemic stroke 3 to 5 hours after symptom onset. The ATLANTIS Study: a randomized controlled trial. Alteplase Thrombolysis for Acute Noninterventional Therapy in Ischemic Stroke. JAMA. 1999;282(21):2019–26. pmid:10591384
- 24. Clark WM, Albers GW, Madden KP, Hamilton S. The rtPA (alteplase) 0- to 6-hour acute stroke trial, part A (A0276g) : results of a double-blind, placebo-controlled, multicenter study. Thromblytic therapy in acute ischemic stroke study investigators. Stroke. 2000;31(4):811–6. pmid:10753980
- 25. Hacke W, Kaste M, Fieschi C, von Kummer R, Davalos A, Meier D, et al. Randomised double-blind placebo-controlled trial of thrombolytic therapy with intravenous alteplase in acute ischaemic stroke (ECASS II). Second European-Australasian Acute Stroke Study Investigators. Lancet. 1998;352(9136):1245–51. pmid:9788453
- 26. Hacke W, Kaste M, Bluhmki E, Brozman M, Dávalos A, Guidetti D, et al. Thrombolysis with alteplase 3 to 4.5 hours after acute ischemic stroke. N Engl J Med. 2008;359(13):1317–29. pmid:18815396
- 27. IST-3 collaborative group, Sandercock P, Wardlaw JM, Lindley RI, Dennis M, Cohen G, et al. The benefits and harms of intravenous thrombolysis with recombinant tissue plasminogen activator within 6 h of acute ischaemic stroke (the third international stroke trial [IST-3]): a randomised controlled trial. Lancet. 2012;379(9834):2352–63. pmid:22632908
- 28. Group S-P, NIoN D. Tissue plasminogen activator for acute ischemic stroke. New Eng J Med. 1995;333(24):1581–8.
- 29. Hacke W, Kaste M, Fieschi C, Toni D, Lesaffre E, von Kummer R, et al. Intravenous thrombolysis with recombinant tissue plasminogen activator for acute hemispheric stroke. The European Cooperative Acute Stroke Study (ECASS). JAMA. 1995;274(13):1017–25. pmid:7563451
- 30. Mandava P, Kalkonde YV, Rochat RH, Kent TA. A matching algorithm to address imbalances in study populations: application to the National Institute of Neurological Diseases and Stroke Recombinant Tissue Plasminogen Activator acute stroke trial. Stroke. 2010;41(4):765–70. pmid:20167911
- 31. Jackson D, Bowden J. Confidence intervals for the between-study variance in random-effects meta-analysis using generalised heterogeneity statistics: should we use unequal tails? BMC Med Res Methodol. 2016;16(1):118. pmid:27604952
- 32. Berger VW, Christophi CA. Randomization technique, allocation concealment, masking, and susceptibility of trials to selection bias. J Mod App Stat Meth. 2003;2(1):80–6.
- 33. Stang A , Baethge C. Imbalance p values for baseline covariates in randomized controlled trials: a last resort for the use of p values? A pro and contra debate. Clinical Epidemiology. 2018:531-5.
- 34. Berger VW, He X. Concealing Block Sizes Is Not Sufficient. Clin Orthop Surg. 2015;7(3):422–3. pmid:26330971
- 35. Berger VW. Varying the block size does not conceal the allocation. J Crit Care. 2006;21(2):229; author reply 229-30. pmid:16769475
- 36. Clark L, Burke L, Margaret Carr R, Coleman E, Roberts G, Torgerson DJ. A review found small variable blocking schemes may not protect against selection bias in randomized controlled trials. J Clin Epidemiol. 2022;14190–8. pmid:34520850
- 37. Khatri P, Kleindorfer DO, Devlin T, Sawyer RN Jr, Starr M, Mejilla J, et al. Effect of alteplase vs aspirin on functional outcome for patients with acute ischemic stroke and minor nondisabling neurologic deficits: The PRISMS Randomized Clinical Trial. JAMA. 2018;320(2):156–66. pmid:29998337
- 38. Garg R, Mickenautsch S. Risk of selection bias assessment in the NINDS-rtPA stroke study: a reply to raised concerns. Research Gate2023 Available from: https://www.researchgate.net/publication/375960388_Risk_of_selection_bias_assessment_in_the_NINDS-rtPA_stroke_study_a_reply_to_raised_concerns
- 39. Berger VW. The reverse propensity score to detect selection bias and correct for baseline imbalances. Stat Med. 2005;24(18):2777–87. pmid:15981305
- 40. Cantrill SV, Brown MD, Brecher D, Diercks DB, Gemme SR, Gerardo CJ, et al. Clinical policy: use of intravenous tissue plasminogen activator for the management of acute ischemic stroke in the emergency department. Ann Emerg Med. 2015;66(3):322-333.e31. pmid:26304253
- 41. Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64(4):383–94. pmid:21195583
- 42. Xiong Y, Campbell BCV, Schwamm LH, Meng X, Jin A, Parsons MW, et al. Tenecteplase for Ischemic Stroke at 4.5 to 24 Hours without Thrombectomy. N Engl J Med. 2024;391(3):203–12. pmid:38884324
- 43.
Garg R. Tenecteplase for ischemic stroke at 4.5 to 24 hours without thrombectomy. PubPeer2024 Available from: https://pubpeer.com/publications/30D56099E89EF0E310B6D48C2679EE#
- 44. Lee JJ, Xuemin Gu, Suyu Liu. Bayesian adaptive randomization designs for targeted agent development. Clin Trials. 2010;7(5):584–96. pmid:20571130
- 45. Dechartres A, Trinquart L, Boutron I, Ravaud P. Influence of trial sample size on treatment effect estimates: meta-epidemiological study. BMJ. 2013;346:f2304. pmid:23616031
- 46. Zhang Z, Xu X, Ni H. Small studies may overestimate the effect sizes in critical care meta-analyses: a meta-epidemiological study. Crit Care. 2013;17(1):R2. pmid:23302257
- 47. Garg R. Methodological survey of missing outcome data in an alteplase for ischemic stroke meta-analysis. Acta Neurol Scand. 2022;146(3):252–7. pmid:35652287
- 48. Mickenautsch S, Yengopal V. Trial number and sample size do not affect the accuracy of the I2-point estimate for testing selection bias risk in meta-analyses. Cureus. 2024;16(4).author billing