Older adults are underrepresented in clinical research. To assess therapeutic efficacy in older patients, some randomized controlled trials (RCTs) include older adults only.
To compare treatment effects between RCTs including older adults only (elderly RCTs) and RCTs including all adults (adult RCTs) by a meta-epidemiological approach.
All systematic reviews published in the Cochrane Library (Issue 4, 2011) were screened. Eligible studies were meta-analyses of binary outcomes of pharmacologic treatment including at least one elderly RCT and at least one adult RCT. For each meta-analysis, we compared summary odds ratios for elderly RCTs and adult RCTs by calculating a ratio of odds ratios (ROR). A summary ROR was estimated across all meta-analyses.
We selected 55 meta-analyses including 524 RCTs (17% elderly RCTs). The treatment effects differed beyond that expected by chance for 7 (13%) meta-analyses, showing more favourable treatment effects in elderly RCTs in 5 cases and in adult RCTs in 2 cases. The summary ROR was 0.91 (95% CI, 0.77–1.08, p = 0.28), with substantial heterogeneity (I2 = 51% and τ2 = 0.14). Sensitivity and subgroup analyses by type-of-age RCT (elderly RCTs vs RCTs excluding older adults and vs RCTs of mixed-age adults), type of outcome (mortality or other) and type of comparator (placebo or active drug) yielded similar results.
Citation: Seegers V, Trinquart L, Boutron I, Ravaud P (2013) Comparison of Treatment Effect Estimates for Pharmacological Randomized Controlled Trials Enrolling Older Adults Only and Those including Adults: A Meta-Epidemiological Study. PLoS ONE 8(5): e63677. https://doi.org/10.1371/journal.pone.0063677
Editor: Ewout W. Steyerberg, University Medical Center Rotterdam, The Netherlands
Received: November 7, 2012; Accepted: April 5, 2013; Published: May 28, 2013
Copyright: © 2013 Seegers et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: Grant support was from the French Ministry of Health (Programme de recherche sur la performance du système des soins PREPS 2-002-0014) and Fondation pour la Recherche Médicale (Equipe « Espoirs de la recherche »). The funders had no role in the study design, analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
The number of people aged 60 years and older now is twice the number in 1980. By 2050, the number of people 80 years and older will be four-fold the current number . The life expectancy has increased during the last decades and thus, more older people may need medical interventions.
The care of older patients presents a challenge for evidence-based medicine : the body of evidence concerning the efficacy of therapeutic interventions in older adults is usually small because these patients are largely underrepresented in clinical trials , , , , , , . Nearly three-quarters of trials reported in major journals excluded older patients . This exclusion is often poorly justified despite the availability of methods for enhancing their recruitment , , ,  and several authorities highlighting the need for representative samples of older adults in randomized controlled trials (RCTs) , , ,  and even the need for randomized trials including older adults only , .
In the absence of evidence for older adults, pharmacologic results for younger adults are commonly extrapolated to older patients assuming that the efficacy is the same. However, this practice may not be valid because of physiological changes associated with aging that affect the pharmacokinetics of drugs, including multiple co-morbidities in older people and unpredictable treatment responses , . Indeed, the Randomized Aldactone Evaluation Study  demonstrated reduced mortality with spironolactone administration in patients with congestive heart failure, with average age 60 years. In contrast, data from several observational cohorts of patients with congestive heart failure who were 13 years older revealed that spironolactone was associated with an almost four-fold increase in number of deaths due to hyperkaliemia . We need to be able to draw on the results of good-quality research to inform best practice in the management of older adults.
In this study, we assessed empirically whether treatment effects differed between RCTs including older adults only (elderly RCTs) and RCTs including all adults (adult RCTs) that were performed for the same medical problem and with the same pharmacological therapeutic interventions, using a meta-epidemiological approach .
Materials and Methods
We performed a meta-epidemiological study to compare treatment effect estimates between elderly RCTs and adult RCTs . We identified a large number of meta-analyses that included at least one elderly RCT and at least one adult RCT and concerned a variety of conditions and therapeutic interventions. We recorded whether individual RCTs included older adults only or not. For each meta-analysis, we compared summary odds ratios (ORs) for elderly RCTs and adult RCTs by calculating a ratio of ORs (ROR). A summary ROR was estimated across all meta-analyses.
This methodology has been used previously to establish empirical evidence of bias in estimating treatment effects (e.g., double blinding, intent-to-treat analyses and allocation concealment , , , , single vs multi-center , , randomized vs non-randomized trials ) and recently to compare efficacy in trials of children and adults .
Selection of meta-analyses
One reviewer screened all systematic reviews published in the Cochrane Library in the Cochrane Database of Systematic Reviews (Issue 4, 2011) concerning all medical domains assessing pharmacological therapeutic interventions. Reviews for Cochrane Groups working on pregnancy, neonatality, fertility, cancer childhood, organization care and public health were excluded.
Eligible reviews reported at least one meta-analysis of a binary outcome for at least 3 RCTs, of which 1 was specific to older adults (elderly RCT) and 1 not specific to older adults (adult RCT). We focused on Cochrane reviews because some studies have shown that these have high methodological quality and are well reported .
One reviewer selected reviews in 3 steps: examining the title, abstract and full-text for potentially relevant reviews. For each eligible systematic review, we systematically screened the “Characteristics of included studies” table to identify elderly RCTs as follows: the authors clearly reported the population as “older”, “geriatric” or “elderly” or they defined the lower age limit as selection criteria or the minimum age of included patients as ≥60 years. If the previous information was not available, we also considered trials with mean patient age ≥75 years as elderly RCTs. For adult RCTs, we distinguished 4 categories: trials excluding older adults (defined as RCTs excluding older adults), trials including both older adults and non-older adults (RCTs of mixed-age adults), unclear or unspecified age recruitment and children-specific trials.
For each eligible meta-analysis, we extracted the following information from each Cochrane review: clinical domain, experimental and control interventions (placebo, no treatment, active drug), primary binary efficacy outcome, and number of included RCTs.
For each RCT, we extracted the age category reported in Table 1 of the Cochrane review, as described: elderly RCT or adult RCT (including RCTs excluding older adults, RCTs of mixed-age adults, unclear RCTs and children-specific RCTs). We also extracted the sample size and the number of patients and events per each study arm from the forest plot of the meta-analysis.
If more than 1 primary outcome was reported or if the primary outcome was not specified among several binary outcomes, we selected the first outcome presented in the Results section. When 2 active interventions were compared, identification of the experimental and control interventions was based on interpretation by the authors of the Cochrane review. If this identification was not clear, we identified which intervention was first discovered according to PubMed first indexing, and this was considered the control intervention. One of the authors extracted the data.
Data synthesis and analysis
We used a two-stage model to compare treatment effect estimates for elderly RCTs and adult RCTs.
Treatment effect estimates for elderly RCTs and adult RCTs
For each included meta-analysis, we estimated the 2 summary odds ratios (ORs) comparing the experimental and the control interventions for elderly RCTs and adult RCTs separately. In cases of many trials for an age group, we combined the trials by random-effects models . Heterogeneity across RCTs was assessed by means of the I2 statistic and the between-RCT variance τ2. Outcome events were re-coded so that an OR <1 indicated a beneficial effect of the experimental intervention. When appropriate, we used a continuity correction .
For each meta-analysis, we evaluated whether the difference in ORs between elderly RCTs and adult RCTs was larger than would be expected by chance alone using z scores. We also assessed in how many cases the summary OR for elderly RCTs was less than half the summary OR for adult RCTs and in how many cases the summary OR for elderly RCTs was twice the summary OR for adult RCTs.
Meta-analysis of ratios of odds ratios between elderly RCTs and adult RCTs
For each meta-analysis, we estimated an ROR and associated standard error  from the summary ORs estimated for elderly RCTs and adult RCTs. A ROR<1 indicated that elderly RCTs yielded larger estimates of the intervention effect than did adult RCTs. Then we combined all RORs across meta-analyses by using a random-effects meta-analysis model . Heterogeneity across meta-analyses was assessed by means of the I2 statistic and the between–meta-analyses variance τ2. We also computed a prediction interval for the summary ROR .
Age-group RCT analyses
In our main analysis, we compared elderly RCTs and adult RCTs. In a complementary analysis, we compared elderly RCTs to RCTs excluding older adults and to RCTs of mixed-age adults. Unclear RCTs were alternatively considered as RCTs excluding older adults or as RCTs of mixed-age adults.
Exploration of heterogeneity
We used a graphical method to identify meta-analyses that contributed considerably to the overall heterogeneity across RORs and that could strongly influence the overall summary ROR . In a sensitivity analysis, we excluded the most heterogeneous and influential meta-analyses and re-estimated the summary ROR and the heterogeneity statistics across meta-analyses. To further explore heterogeneity, we performed 2 pre-specified subgroup analyses by type of comparator treatment (active drug vs placebo and/or usual care or no treatment) and type of outcome (mortality vs other outcomes).
There is no validated method of power calculation for meta-epidemiological analyses. Statistical power of a meta-epidemiological analysis would depend on the true ROR, the number of meta-analyses and trials, the distribution in each meta-analysis of trials with and without the characteristic of interest, and the heterogeneity between and within studies. We arbitrarily prespecified a sample size of 50 meta-analyses.
The analyses involved the two-stage comparison described above and were consolidated with a one-stage multilevel model (ie, a hierarchical logistic regression model with mixed effects) . We used R v2.15.1  for all analyses and the packages meta , metafor  and lme4 .
Of the 3, 403 systematic reviews identified from the Cochrane Library, 55 were included (Figure S1). Their characteristics are in Table S1. Fifteen (27%) concerned psychiatry, 14 (25%) cardiovascular diseases and 9 (16%) neurology. In total, 35 (64%) meta-analyses compared pharmacologic treatment with placebo or no treatment. The 55 meta-analyses involved 524 RCTs overall (Figure S1): 90 (17%) were elderly RCTs, and 434 were adult RCTs, including 73 RCTs excluding older adults, 224 RCTs of mixed-age adults, 133 unclear RCTs and 4 children-only RCTs. The median sample size was 91 patients (min-max 18–6,706) for elderly RCTs and 82 (10–23,323) for adult RCTs. The total number of included patients was 199,760, with 21% for elderly RCTs and 79% for adult RCTs.
Treatment effect estimates for elderly RCTs and adult RCTs
Among the 55 meta-analyses, for 48, the summary ORs did not differ between elderly RCTs and adult RCTs. However, the summary ORs differed beyond chance for 7 (13%) meta-analyses (Figure S2), showing more favourable treatment effects in elderly RCTs for 5 meta-analyses (CD000096, CD00424, CD001886, CD003781, CD008120) and in adult RCTs for 2 meta-analyses (CD002747, CD007503). The characteristics of the 7 corresponding reviews are in Table S2.
When we considered the magnitude of treatment effect estimates, the summary OR for elderly RCTs was less than half the summary OR for adult RCTs for 9 (16%) meta-analyses. In contrast, the summary OR for elderly RCTs was at least twice the summary OR for adult RCTs for 8 (15%) meta-analyses.
Meta-analysis of RORs between elderly RCTs and adult RCTs
The results of the meta-analysis of RORs are in Figure S3. The summary ROR across all 55 included meta-analyses was 0.91 (95% CI, 0.77–1.08, p = 0.278), with substantial heterogeneity (I2 = 51.1% and τ2 = 0.137). The 95% prediction interval, which provides a predicted range for the true ROR in an individual meta-analysis, was 0.43 to 1.92.
Age-group RCT analyses
In comparing elderly RCTs with RCTs excluding older adults and RCTs of mixed-age adults, the summary ROR was 0.88 (95% CI 0.63–1.17, n = 29 reviews) and 0.93 (0.76–1.15, n = 38 reviews), respectively. The hierarchical multilevel modeling approach yielded similar summary RORs (elderly RCTs vs RCTs excluding older adults, 0.88, 0.70–1.09; elderly RCTs vs RCTs of mixed-age adults, 0.95, 0.83–1.09). Alternatively, when we considered trials for which the age category was unclear as RCTs excluding older adults or RCTs of mixed-age adults, we did not find any difference (Appendix S1).
Exploration of heterogeneity
Three meta-analyses (CD008120, CD003781, and CD002747) accounted for most of the heterogeneity (50.0%) and had a strong influence on the sROR (65% of the sum of the influences; Figure in Appendix S1). In a sensitivity analysis, the exclusion of these 3 meta-analyses strongly reduced the heterogeneity across RORs (I2 = 2%; τ2 = 0.0035), but the summary ROR was only slightly modified (0.99, 95% CI 0.90–1.10). The characteristics of these reviews are in Table S2.
Subgroup analysis by type of control group revealed no statistically significant difference, on average, in treatment effects estimates between elderly RCTs and adult RCTs with control groups of placebo, no treatment or usual care (summary ROR 0.87; 95% CI 0.68–1.11) or an active drug (0.95; 0.82–1.10). Subgroup analysis by type of outcome yielded similar results when assessing mortality (1.11; 0.90–1.36) and other outcomes (0.95; 0.82–1.10).
For a sample of 55 Cochrane systematic reviews, we performed an empirical evaluation of the relative treatment effect estimates of pharmacological therapeutic interventions between RCTs that included older adults only and RCTs of the general adult population. In 7 cases, treatment effect estimates differed significantly between elderly RCTs and adult RCTs, showing more favourable treatment effects in elderly RCTs for 5 meta-analyses and in adult RCTs for 2 meta-analyses. Treatment effects estimates did not differ significantly, on average, between elderly RCTs and adult RCTs.
For some elderly RCTs, the intervention efficacy would have been overestimated if based on adult RCTs (meta-analyses CD002747, CD007503). In these 2 meta-analyses, the experimental treatment may have suggested benefit in the global population, but for RCTs specific to older adults, the control treatment may have been more beneficial than the experimental treatment. The first identified situation concerned women with metastatic breast cancer. For adult RCTs, the results of the meta-analysis would have leaned toward chemotherapy alone, whereas for older women, the results would have favored endocrine therapy alone. This situation is unique in terms of menopausal status, which depends on the age of women. The second situation involved psychiatry (antidepressants for depression in physically ill people). However, in this meta-analysis, one elderly RCT involved patients in a general medical ward for older adults, whereas another included RCTs of mostly outpatients with chronic diseases (chronic prostatitis, Parkinson disease, multiple sclerosis and cancer). The type of depression could differ among these types of illnesses.
The lack of evidence from elderly RCTs may have 2 consequences: the use of medications with risks that are likely to out-weight benefits and undertreatment of older patients, which may result in a lack of improvement , , , . In practice, the allocation of a pharmacological treatment for an older patient is guided by determination of the absolute benefits and harm in individual patients. Unfortunately, many reports of RCTs fail to provide detailed adverse effects data, and the quality of those that do is poor , , , .
The definition of older patients is controversial. For example, the European Medicines Agency arbitrarily defines its scope as patients 65 years of age (standard retirement age) or older , the World Health Organization proposed a cut-off of 60 years or older , and several medical domains have a cut-off of 80 years and older . At the present and hopefully in the next decades, people in their 60 s or even 70 s will become healthier. Nevertheless, because of the physiological changes of aging (eg, weight, body composition, decreased glomerular filtration rate) and the increased risk of neurodegenerative diseases, diabetes, and cardiovascular events, for example, this age group differs from younger adults. Although aging is a progressive process, the definition of older adults in RCTs should be based on available and relevant indicators.
Our study contains some limitations. First, for consistency, we considered only reviews with binary efficacy outcomes only; however, this involves a large portion of clinically useful outcomes in clinical trials. Second, some Cochrane reviews did not provide information on the age of trial populations and, in theory, some of these meta-analyses might have been eligible if they included both elderly RCTs and adult RCTs. However, screening thousands of trials with unspecified age distribution would have been difficult, with uncertain yield; therefore, we depended on the information collected and recorded by the Cochrane authors of reviews to decide whether to include the review. The quality of Cochrane reviews is generally considered very good . Third, in our main analyses, we included RCTs with “unspecified/unclear” age group from each eligible meta-analysis, which accounted for 25% of the trials, but excluding or reclassifying them in sensitivity analyses with adult RCTs yielded similar results. Fourth, the lack of data did not allow for performing formal comparisons of different older age groups (eg, older vs very older). Fifth, the results of both elderly RCTs and adult RCTs may be biased for diverse reasons, and the meta-analysis results may also be affected. Some reviews perform quality and bias assessments, but these assessments can differ across reviews,  and associations of reported quality with treatment effects may be tenuous . However, biases are unlikely to differ in elderly RCTs and adult RCTs.
Our findings reflect only efficacy of treatment as measured in randomized trials. This efficacy may differ from effectiveness in clinical practice because patients included in trials may differ from patients in clinical practice. Some meta-confounding factors could bias our findings. These factors may be related to differences between included trials (concerning patient inclusion critera, the risk of bias of trials). Elderly patients included in elderly RCTs may be over-selected as compared with other adults included in general-age adult trials because of inclusion criteria in RCTs (eg, comorbities, illness severity, physiological functions, in- and outpatients). Taking such biases into account in the analyses may be difficult, but identifying such confounders may be useful. Therefore, we checked the co-morbidities used as selection criteria of trials included in meta-analyses. No trial had been excluded because of co-morbidities or physical or biological age-related dysfunctions. Nevertheless, we did not check patient exclusion criteria for individual elderly RCTs. Our exploratory framework does not allow for concluding that pharmacological treatment effects in elderly patients are similar to those in non-elderly patients because we explored trials that included only elderly patients as compared with trials that included adults without lower-age–inclusion criteria. Similar results of treatment effect estimates for pharmacologic RCTs including elderly patients only and adult RCTs were average estimates, but discrepancies in treatment effects can occur. The extent and direction of the difference were unpredictable, and extrapolation of evidence from adult RCTs to elderly RCTs can sometimes be tenuous.
As Scott and Guyatt state,  “It is wisest to assume similar relative treatment effects in older and younger patients unless there is compelling evidence of age-related differences.” Identification of interventions that do not provide any clinical benefit in older versus younger adults or are potentially harmful in older adults is important. Similarly, current European regulations require mandatory studies of older adults for all drugs,15 and the European Medicines Agency plans to make publicly available all results of clinical trials of older adults, but this move does not guarantee that sufficiently large trials will be conducted. Generation of large-scale evidence for geriatric indications is warranted.
Comparison of the summary odds ratios (ORs; and 95% confidence intervals [95% CIs]) for RCTs specifically including and not including older adults (elderly RCTs and adult RCTs). Summary ORs were estimated with random-effects meta-analysis. Data in blue indicate meta-analyses for which the difference between summary ORs in elderly RCTs and in adult RCTs was beyond what would be expected by chance alone. Labels in bold indicate meta-analyses for which the magnitude of the treatment effect estimates was ≥2 or ≤50%. OR<1 favors experimental treatment and OR >1 favors the control treatment. *One meta-analysis showed the experimental intervention to be significantly worse than the control (CD003348 “Patient controlled opioid analgesia versus conventional opioid analgesia for postoperative pain”).
Meta-analysis of ratios of ORs for elderly RCTs and adult RCTs. The width of the diamond is the 95% confidence interval for the true summary ROR and the dotted line is the prediction interval which indicates the possible ROR in an individual meta-analysis.
Description of meta-analyses including at least 1 elderly randomized controlled trial (RCT specific to older adults) and including at least 1 adult RCT (not specific to older adults).
Characteristics of meta-analyses with significantly different treatment effect estimates between elderly RCTs and adult RCTs. OR: odds ratio; 95% CI: 95% confidence interval, * the 3 reviews that accounted for most of the overall heterogeneity in ratio of ORs across all reviews.
The authors thank Laura Smales (BioMedEditing, Toronto, Canada) for editing the manuscript.
Conceived and designed the experiments: VS LT IB PR. Performed the experiments: VS. Analyzed the data: VS LT. Contributed reagents/materials/analysis tools: VS LT. Wrote the paper: VS. Revised the manuscript: VS LT IB PR.
- 1. WHO website. Available: http://www.who.int/features/factfiles/ageing/en/index.html. Accessed: 2012 May 15.
- 2. Scott IA, Guyatt GH (2010) Cautionary tales in the interpretation of clinical studies involving older persons. Arch Intern Med 170: 587–595.
- 3. Cherubini A, Oristrell J, Pla X, Ruggiero C, Ferretti R, et al. (2011) The persistent exclusion of older patients from ongoing clinical trials regarding heart failure. Arch Intern Med 171: 550–556.
- 4. Gurwitz JH, Col NF, Avorn J (1992) The exclusion of the elderly and women from clinical trials in acute myocardial infarction. JAMA 268: 1417–1422.
- 5. Gurwitz JH, Goldberg RJ (2011) Age-based exclusions from cardiovascular clinical trials: implications for elderly individuals (and for all of us): comment on “the persistent exclusion of older patients from ongoing clinical trials regarding heart failure”. Arch Intern Med 171: 557–558.
- 6. Heiat A, Gross CP, Krumholz HM (2002) Representation of the elderly, women, and minorities in heart failure clinical trials. Arch Intern Med 162: 1682–1688.
- 7. Hutchins LF, Unger JM, Crowley JJ, Coltman CA Jr, Albain KS (1999) Underrepresentation of patients 65 years of age or older in cancer-treatment trials. N Engl J Med 341: 2061–2067.
- 8. Konrat C, Boutron I, Trinquart L, Auleley GR, Ricordeau P, et al. (2012) Underrepresentation of elderly people in randomised controlled trials. The example of trials of 4 widely prescribed drugs. PLoS One 7: e33559.
- 9. Lewis JH, Kilgore ML, Goldman DP, Trimble EL, Kaplan R, et al. (2003) Participation of patients 65 years of age or older in cancer clinical trials. J Clin Oncol 21: 1383–1389.
- 10. Scott IA (2010) Cautionary Tales in the Interpretation of Clinical Studies Involving Older Persons. Arch Intern Med 170 587–595.
- 11. McMurdo ME, Roberts H, Parker S, Wyatt N, May H, et al. (2011) Improving recruitment of older people to research through good practice. Age Ageing 40: 659–665.
- 12. McMurdo ME, Witham MD, Gillespie ND (2005) Including older people in clinical research. BMJ 331: 1036–1037.
- 13. Murray MD, Callahan CM (2003) Improving medication use for older adults: an integrated research agenda. Ann Intern Med 139: 425–429.
- 14. Van Spall HG, Toren A, Kiss A, Fowler RA (2007) Eligibility criteria of randomized controlled trials published in high-impact general medical journals: a systematic sampling review. JAMA 297: 1233–1240.
- 15. PREDICT (2008) Increasing the participation of the elderly in clinical trials.
- 16. European Forum for Good Clinical Practice Geriatric research guidelines.
- 17. Pallis AG, Ring A, Fortpied C, Penninckx B, Van Nes MC, et al. (2011) EORTC workshop on clinical trial methodology in older individuals with a diagnosis of solid tumors. Ann Oncol 22: 1922–1926.
- 18. Sinha G (2006) Geriatric oncology aims for clinical trials. J Natl Cancer Inst 98: 1516–1518.
- 19. Watts G (2012) Why the exclusion of older people from clinical research must stop. BMJ 344: e3445.
- 20. Herrera AP, Snipes SA, King DW, Torres-Vigil I, Goldberg DS, et al. (2010) Disparate inclusion of older adults in clinical trials: priorities and opportunities for policy and practice change. Am J Public Health 100 Suppl 1S105–112.
- 21. Ridda I, Lindley R, MacIntyre RC (2008) The challenges of clinical trials in the exclusion zone: the case of the frail elderly. Australas J Ageing 27: 61–66.
- 22. Pitt B, Zannad F, Remme WJ, Cody R, Castaigne A, et al. (1999) The effect of spironolactone on morbidity and mortality in patients with severe heart failure. Randomized Aldactone Evaluation Study Investigators. N Engl J Med 341: 709–717.
- 23. Bozkurt B, Agoston I, Knowlton AA (2003) Complications of inappropriate use of spironolactone in heart failure: when an old medicine spirals out of new guidelines. J Am Coll Cardiol 41: 211–214.
- 24. Sterne JA, Juni P, Schulz KF, Altman DG, Bartlett C, et al. (2002) Statistical methods for assessing the influence of study characteristics on treatment effects in ‘meta-epidemiological’ research. Stat Med 21: 1513–1524.
- 25. Naylor CD (1997) Meta-analysis and the meta-epidemiology of clinical research. BMJ 315: 617–619.
- 26. Schulz KF, Chalmers I, Hayes RJ, Altman DG (1995) Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 273: 408–412.
- 27. Moher D, Pham B, Jones A, Cook DJ, Jadad AR, et al. (1998) Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 352: 609–613.
- 28. Pildal J, Hrobjartsson A, Jorgensen KJ, Hilden J, Altman DG, et al. (2007) Impact of allocation concealment on conclusions drawn from meta-analyses of randomized trials. Int J Epidemiol 36: 847–857.
- 29. Nuesch E, Reichenbach S, Trelle S, Rutjes AW, Liewald K, et al. (2009) The importance of allocation concealment and patient blinding in osteoarthritis trials: a meta-epidemiologic study. Arthritis Rheum 61: 1633–1641.
- 30. Dechartres A, Boutron I, Trinquart L, Charles P, Ravaud P (2011) Single-center trials show larger treatment effects than multicenter trials: evidence from a meta-epidemiologic study. Ann Intern Med 155: 39–51.
- 31. Bafeta A, Dechartres A, Trinquart L, Yavchitz A, Boutron I, et al. (2012) Impact of single centre status on estimates of intervention effects in trials with continuous outcomes: meta-epidemiological study. BMJ 344: e813.
- 32. Ioannidis JP, Haidich AB, Pappa M, Pantazis N, Kokori SI, et al. (2001) Comparison of evidence of treatment effects in randomized and nonrandomized studies. JAMA 286: 821–830.
- 33. Contopoulos-Ioannidis DG, Baltogianni MS, Ioannidis JP (2010) Comparative effectiveness of medical interventions in adults versus children. J Pediatr 157: 322–330 e317.
- 34. Higgins JP, Altman DG, Gotzsche PC, Juni P, Moher D, et al. (2011) The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ 343: d5928.
- 35. DerSimonian R, Laird N (1986) Meta-analysis in clinical trials. Control Clin Trials 7: 177–188.
- 36. Sweeting MJ, Sutton AJ, Lambert PC (2004) What to add to nothing? Use and avoidance of continuity corrections in meta-analysis of sparse data. Stat Med 23: 1351–1375.
- 37. Altman DG, Bland JM (2003) Interaction revisited: the difference between two estimates. BMJ 326: 219.
- 38. Golder S, Loke YK, Bland M (2011) Meta-analyses of adverse effects data derived from randomised controlled trials as compared to observational studies: methodological overview. PLoS Med 8: e1001026.
- 39. Riley RD, Higgins JP, Deeks JJ (2011) Interpretation of random effects meta-analyses. BMJ 342: d549.
- 40. Baujat B, Mahe C, Pignon JP, Hill C (2002) A graphical method for exploring heterogeneity in meta-analyses: application to a meta-analysis of 65 trials. Stat Med 21: 2641–2652.
- 41. Siersma V, Als-Nielsen B, Chen W, Hilden J, Gluud LL, et al. (2007) Multivariable modelling for meta-epidemiological assessment of the association between trial quality and treatment effects estimated in randomized clinical trials. Stat Med 26: 2745–2758.
- 42. R Development Core Team (2012) R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing,.
- 43. Guido Schwarzer (2012) meta: Meta-Analysis with R. R package version 2.1–0.
- 44. Wolfgang Viechtbauer (2010) Conducting meta-analyses in R with the metafor package. Journal of Statistical Software 36: 1–48.
- 45. Bates D, Maechler M, Bolker B (2012) lme4: Linear mixed-effects models using S4 classes. 0.999999–0 ed.
- 46. Simon SR, Gurwitz JH (2003) Drug therapy in the elderly: improving quality and access. Clin Pharmacol Ther 73: 387–393.
- 47. Gurwitz JH, Rochon P (2002) Improving the quality of medication use in elderly patients: a not-so-simple prescription. Arch Intern Med 162: 1670–1672.
- 48. Jencks SF, Cuerdon T, Burwen DR, Fleming B, Houck PM, et al. (2000) Quality of medical care delivered to Medicare beneficiaries: A profile at state and national levels. JAMA 284: 1670–1676.
- 49. Higashi T, Shekelle PG, Solomon DH, Knight EL, Roth C, et al. (2004) The quality of pharmacologic care for vulnerable older patients. Ann Intern Med 140: 714–720.
- 50. Pitrou I, Boutron I, Ahmad N, Ravaud P (2009) Reporting of safety results in published reports of randomized controlled trials. Arch Intern Med 169: 1756–1761.
- 51. Papanikolaou PN, Christidi GD, Ioannidis JP (2006) Comparison of evidence on harms of medical interventions in randomized and nonrandomized studies. CMAJ 174: 635–641.
- 52. Ioannidis JP, Lau J (2001) Completeness of safety reporting in randomized trials: an evaluation of 7 medical areas. JAMA 285: 437–443.
- 53. Cuervo LG, Clarke M (2003) Balancing benefits and harms in health care. BMJ 327: 65–66.
- 54. The Association of Anaesthetists of Great Britain and Ireland (2001) Anaesthesia and Peri-Operative Care of the Elderly. London.
- 55. Olsen O, Middleton P, Ezzo J, Gotzsche PC, Hadhazy V, et al. (2001) Quality of Cochrane reviews: assessment of sample from 1998. BMJ 323: 829–832.
- 56. Juni P, Witschi A, Bloch R, Egger M (1999) The hazards of scoring the quality of clinical trials for meta-analysis. JAMA 282: 1054–1060.
- 57. Ioannidis JP, Lau J (1998) Can quality of clinical trials and meta-analyses be quantified? Lancet 352: 590–591.