Systematic reviews can include cluster-randomised controlled trials (C-RCTs), which require different analysis compared with standard individual-randomised controlled trials. However, it is not known whether review authors follow the methodological and reporting guidance when including these trials. The aim of this study was to assess the methodological and reporting practice of Cochrane reviews that included C-RCTs against criteria developed from existing guidance.
Criteria were developed, based on methodological literature and personal experience supervising review production and quality. Criteria were grouped into four themes: identifying, reporting, assessing risk of bias, and analysing C-RCTs. The Cochrane Database of Systematic Reviews was searched (2nd December 2013), and the 50 most recent reviews that included C-RCTs were retrieved. Each review was then assessed using the criteria.
The 50 reviews we identified were published by 26 Cochrane Review Groups between June 2013 and November 2013. For identifying C-RCTs, only 56% identified that C-RCTs were eligible for inclusion in the review in the eligibility criteria. For reporting C-RCTs, only eight (24%) of the 33 reviews reported the method of cluster adjustment for their included C-RCTs. For assessing risk of bias, only one review assessed all five C-RCT-specific risk-of-bias criteria. For analysing C-RCTs, of the 27 reviews that presented unadjusted data, only nine (33%) provided a warning that confidence intervals may be artificially narrow. Of the 34 reviews that reported data from unadjusted C-RCTs, only 13 (38%) excluded the unadjusted results from the meta-analyses.
The methodological and reporting practices in Cochrane reviews incorporating C-RCTs could be greatly improved, particularly with regard to analyses. Criteria developed as part of the current study could be used by review authors or editors to identify errors and improve the quality of published systematic reviews incorporating C-RCTs.
Citation: Richardson M, Garner P, Donegan S (2016) Cluster Randomised Trials in Cochrane Reviews: Evaluation of Methodological and Reporting Practice. PLoS ONE 11(3): e0151818. doi:10.1371/journal.pone.0151818
Editor: Helge Bruns, University Hospital Oldenburg, GERMANY
Received: January 28, 2016; Accepted: March 4, 2016; Published: March 16, 2016
Copyright: © 2016 Richardson et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: MR, SD, and PG are supported by the Effective Health Care Research Consortium, which is funded by UKAid from the UK Government Department for International Development (Grant number, PO 5242; URL, https://www.gov.uk/government/organisations/department-for-international-development). SD is also funded by the Medical Research Council (Grant number, MR/K021435/1; URL, http://www.mrc.ac.uk/).
Competing interests: The authors have declared that no competing interests exist.
Systematic reviews summarise existing studies of interventions for a particular disease. Cochrane reviews are high-quality systematic reviews of primary research in human health care and health policy, and are conducted using standard methods by review groups within Cochrane . Randomised controlled trials are considered to be the highest quality of primary research study design and are therefore often included in such reviews. Individual-randomised trials (I-RCTs) and/or cluster-randomised trials (C-RCTs) can be included.
In an I-RCT, individual participants are randomly allocated to intervention groups. However, sometimes it is impractical, even impossible, to randomise individuals but it may be feasible to randomise clusters of individuals (e.g. schools, communities or clinics) to intervention groups . Therefore, in such trials, the unit of randomisation is the cluster rather than the individual. For example, to evaluate the effect of insecticidal spraying of a household on malaria prevalence, it would be impossible to randomise individuals to spraying or no spraying when more than one person lives in the same household because the whole household is usually sprayed; however, households could be randomised to spraying or no spraying. Consequently, C-RCTs are important in evaluating a variety of public health and health service interventions. Furthermore, C-RCTs can also be used when multiple outcome measurements are taken on the same individual (e.g. to evaluate the effectiveness of a topical cream for a skin condition, one measurement could be taken on each arm of the same individual); in such cases, individuals are randomised to interventions and each individual is considered to be a cluster.
The Cochrane Handbook  and other methodological publications [4–6] provide guidance regarding the inclusion of C-RCTs in reviews. This guidance includes details on how to assess the risk of bias, extract data, and analyse C-RCTs. If review authors do not follow the guidance, but instead, analyse C-RCTs in the same way as I-RCTs, the confidence interval (CI) for the treatment effect would be artificially narrow because clustering would not be taken into account. Interpreting such analyses that are not adjusted for clustering may lead to false conclusions being drawn from the review and result in patients being treated with inferior interventions. Review authors may be able to adjust treatment effect estimates for clustering themselves using estimates of the average cluster size and intracluster correlation coefficient (ICC), which quantifies the extent to which data from observations from the same cluster are correlated .
This report introduces assessment criteria that were developed based on the published guidance, and which were used to examine the methodological and reporting quality of Cochrane reviews that include C-RCTs. Specifically, this study assesses whether the following details are considered and/or reported in systematic reviews: C-RCTs are identified throughout the review; general cluster information is reported; risk of bias is assessed appropriately; and analyses are carried out correctly. The study also ascertained how often reviews incorrectly analyse C-RCTs in the same way as I-RCTs. This research updates and extends the previously published review by Laopaiboon et al. , which examined synthesis approaches of meta-analyses involving C-RCTs.
The Assessment Criteria
Criteria were developed to assess the quality of each review that included C-RCTs. The criteria were categorised under four themes: identifying C-RCTs, reporting C-RCTs, assessing risk of bias of C-RCTs, and analysing C-RCTs. Table 1 displays the criteria for each theme. Each criterion took the form of a question that could be answered “Yes” (representing current best practice), “No” (does not fulfil this criterion), or “N/A” (when it was not possible for the review to meet this criterion).
The criteria were based on recommendations for review authors given in the Cochrane Handbook  and methodological publications [2, 4–6] and our own experience as editors of a Cochrane Review Group. The CONSORT statement for C-RCTs , which provides reporting guidelines for C-RCT reports, was also consulted.
The 50 most recent Cochrane intervention reviews (new or updated) that reported the term “cluster” and/or “community” in the review (or references of included studies) and that included at least one C-RCT were eligible for inclusion in the current study. The presence of at least one C-RCT in the review was verified by obtaining the original trial articles for which the term “cluster” and “community” was used in the review. Review protocols and withdrawn reviews were excluded. Fifty reviews were considered sufficient to obtain a picture of current practice while keeping the study manageable. Only Cochrane Reviews were included because explicit guidance is available and promoted to review authors in the Cochrane handbook ; therefore, all review authors were expected to have followed this guidance (i.e. criteria 14–24 in Table 1), and it was hoped that some may also have met the additional criteria identified in the current study, which are believed to be important when including C-RCTs in Cochrane reviews.
Reviews were sought via the Cochrane Database of Systematic Reviews (29th November, 2013) using the search terms “cluster” OR “community”.
The full text was obtained for each review identified by the search. One author (MR) assessed the eligibility of reviews using a pre-prepared form. If eligibility was unclear, the review authors were contacted for more information.
One author (MR) extracted data from both the reviews and their included C-RCTs using two pre-piloted forms (one for review characteristics, one for trial characteristics). A second author (SD) independently extracted data for a sample of 10 reviews (20%); any differences in opinion between the two authors were resolved through discussion. The full trial report was retrieved for every C-RCT trial that was labelled as “community” or “cluster” in the review, and trials with “cluster” or “community” in the title. The original trial reports for each C-RCT were obtained by using the University of Liverpool electronic library and through inter-library loans. Data were extracted from the C-RCTs in order to assess the reviews using the criteria (e.g. if the C-RCT reported cluster-adjusted results, the review was also expected to report the adjusted results, whereas the review was not expected to report adjusted results if the C-RCT trial had not reported these data).
Information was extracted regarding review characteristics, and the C-RCT details (identifying C-RCTs, reporting C-RCTs, assessing risk of bias of C-RCTs, and analysing C-RCTs). See S1–S5 Tables for the type of information that was extracted. Criteria were assessed for all outcomes in the included reviews where necessary; for example, if a review did not perform analyses correctly for one outcome but performed analyses correctly for other outcomes, the review was assessed to have not met the criteria.
The characteristics of each review and C-RCTs were tabulated. Each review was assessed by MR against the proposed criteria using pre-piloted assessment forms. It was possible for one review to meet a criterion for some included C-RCTs but not meet the same criterion for other C-RCTs. For example for criterion 8, unit of randomisation may be reported for some C-RCTs but not for others. Therefore, the percentage of reviews meeting each criterion for all included C-RCTs was calculated and presented. For criteria 12, 18, 19–26, it was possible for a particular review to meet the criteria for one outcome but not for other outcomes; for these criteria, a review was judged to satisfy the criterion if the review satisfied the criterion for every outcome in the review. Reviews for which a criterion was not applicable were excluded from the calculation. The assessment result for each criterion for each review were tabulated.
A total of 276 reviews were assessed for eligibility, and 50 reviews were included [7–56]. S1 Table gives details of the included trials, participants, interventions and outcomes of each review. The 50 reviews included 1170 trials in total; the median number of trials included was 18 (interquartile range [IQR] 9–29, range 1–77).
A total of 201 C-RCTs were included in the included reviews, and references for these C-RCTs are provided in S1 File. The median number of C-RCTs included was 3 (IQR 1–6, range 1–23). The percentage of included C-RCTs ranged from 2% to 100%.
Forty-five reviews (90%) included both C-RCTs and I-RCTs; five reviews (10%) included C-RCTs alone. Forty-six reviews (92%) included C-RCTs that randomised groups of individuals to interventions, and four reviews (8%) included C-RCTs that randomised individuals to interventions, with each individual considered to be a cluster of observations (i.e. parts of the body, multiple pregnancies, or episodes of a condition).
The included reviews varied considerably in terms of disease area (S1 Table). Fig 1 shows the number of included reviews by Cochrane Review Group. Reviews were published by 26 Cochrane Review Groups with each group including between one and six reviews.
Interventions compared included: non-pharmacological (21 reviews); drugs (six reviews); screening (five reviews); training/education (four reviews); counselling/advice (four reviews); behavioural (two reviews); techniques (two reviews); psychosocial (one review); vaccination (one review); drugs/psychosocial/non-pharmacological (one review), drugs/non-pharmacological (one review), drugs/psychosocial (one review), drugs/psychosocial/behavioural (one review). A variety of outcomes were included in the reviews (S1 Table).
Table 1 shows the percentage of reviews that met each criterion. Assessment results for each criterion for each review are provided in S2 Table for “Identifying C-RCTs”, S3 Table for “Reporting C-RCTs”, S4 Table for “Assessing risk of bias of C-RCTs”, and S5 Table for “Analysing C-RCTs”.
About half of the abstracts of the reviews reported C-RCTs (24 reviews, 48%), and a little more than half were explicit that C-RCTs were eligible for inclusion (28 reviews, 56%). Most reviews described methods for analysis of C-RCTs (42 reviews, 84%). In results, 68% of reviews (34 reviews) identified C-RCTs in the description of the included studies (the “Description of studies” text), and most reviews (47 reviews, 94%) identified C-RCTs in the detailed “Characteristics of included studies” table. Only 24 reviews (48%) referred to C-RCTs in the risk of bias results tables, and fewer still (18 reviews, 36%) labelled C-RCTs in the efficacy results (i.e. text, tables, or forest plots).
One review was excluded from this analysis because none of the included C-RCT reports could be obtained. Of the remaining 49 reviews, systematic reporting varied across the criteria: 44 (90%) reported the unit of randomisation; 34 (69%) reported the study design; 22 (48%) reported the average cluster size; 29 (59%) reported whether the data were cluster-adjusted; eight (24%) reported the analysis method; and seven (30%) reported the ICC for each outcome of interest in each C-RCT.
The percentages of reviews that reported these characteristics for at least one of the included C-RCTs (as opposed to all included C-RCTs, as presented in Table 1) was also calculated. These results are provided in S6 Table.
Assessing Risk of Bias of C-RCTs
More than half of the reviews (26 reviews, 52%) assessed the risk of baseline imbalances, whereas only one review (2%) assessed the risk of non-comparability with RCTs. The assessment of comparability between I-RCTs and C-RCTs was not applicable to five reviews that did not include I-RCTs. A small percentage of reviews assessed the risk of recruitment bias (10%), the risk of loss of clusters and individuals (24%), and the risk of incorrect analysis (26%).
Only one review assessed all risk of bias criteria.
Only 26% of reviews (13 reviews) stated whether all C-RCT results were adjusted or unadjusted in the forest plots, text or tables.
The analysis of unadjusted data from C-RCTs was poor. Of the 27 reviews that presented unadjusted data, only nine (33%) provided a warning to say that CIs may be artificially narrow. Of the 34 reviews that reported unadjusted data, only 13 (38%) excluded the unadjusted results from the meta-analyses.
Of 27 reviews that included trials with unadjusted data that did not state that it would not be possible to adjust data themselves, six (22%) adjusted unadjusted cluster data using the average cluster size and ICC as reported in the C-RCT trial reports. In the case of 10 reviews that estimated an ICC (i.e. from similar trials, or elsewhere in the literature) to adjust cluster data, five (50%) carried out sensitivity analyses to investigate the effect of estimating ICC.
A total of 31 reviews included trials that adjusted for clustering, of which 21 (68%) correctly extracted adjusted data.
Of the 40 reviews that included both C-RCTs and I-RCTs in the same meta-analysis, only five reviews (13%) subgrouped the data according to study design to enable correct interpretation or investigation of heterogeneity. Furthermore, of 22 reviews that included C-RCTs with different units of randomisation in the same meta-analysis, no review subgrouped the data according to unit of randomisation.
In the current study, assessment criteria were devised and used to evaluate how well Cochrane reviews that included C-RCTs were carried out in practice. Findings showed that reviews could be substantially improved in terms of methodology and reporting.
The results of this study suggested that there is room for improvement in the identifying of C-RCTs for each of the assessed criteria. Perhaps the most concerning findings were that only 56% of reviews specified that C-RCTs were eligible for inclusion in the eligibility criteria (28 reviews), and that eight reviews did not describe any methods specific to C-RCTs (16%).
In order to ensure that readers of Cochrane review understand the study design of included C-RCTs and are able to interpret results correctly, review authors ought to report several details about the C-RCT. The results of the current study suggested that Cochrane review authors are unaware of the importance of reporting these details about included C-RCTs; reporting standards were low across all criteria, with the exception of “unit of randomisation”. Only 59% of the reviews reported whether C-RCT results were adjusted for clustering, and therefore readers of these reviews would be unable to assess whether the data had been analysed correctly.
The five additional risk-of-bias criteria for C-RCTs are listed in the Cochrane Handbook . Review authors ought to apply these assessments to ensure that the quality of each C-RCT trial has been adequately assessed; this is in addition to the criteria listed for assessing risk of bias for I-RCTs . Only one review assessed risk of bias for all additional criteria for included C-RCTs; hence, conclusions for the remaining reviews may have been drawn based on low quality evidence from studies that were not adequately appraised.
The importance of performing appropriate analyses for C-RCTs is crucial because errors in the analyses can affect results and influence healthcare decisions. Only 26% of included reviews stated whether C-RCT results were adjusted, and only 38% of the reviews excluded unadjusted results from the meta-analyses. When unadjusted results are presented without indicating that no adjustments for clustering have been performed, CIs are too narrow and these incorrect results may influence conclusions drawn from the review. When confidence intervals are too narrow for C-RCTs, the overall pooled effect is more precise than it should be and this may lead to incorrect conclusions (e.g. a significant difference may be found where none exists). Furthermore, if confidence intervals are too narrow for C-RCTs, this also implies that inverse variance weights for these trials are overestimated, and thus the C-RCTs obtain more weight in the meta-analysis than they should. This may cause bias in favour of the results generated by C-RCTs.
We believe that the current findings indicate that review authors do not understand or are not aware of appropriate methods and reporting in reviews including C-RCTs. The findings corroborate with research conducted in 2003; a study of 25 meta-analyses involving C-RCTs found that half (13/25) used inappropriate analysis methods that ignored the effect of clustering (i.e. review authors treated the included C-RCTs as though they were I-RCTs), and pooled these invalid results in meta-analyses . In the current study, only 38% of reviews excluded unadjusted results from C-RCTs in meta-analyses, suggesting that practice has not greatly improved since 2003. Furthermore, another study reviewed all reviews of the Cochrane Infectious Diseases Group without the use of search terms, and found that results were comparable to those in the current study . Review authors often used the same risk-of-bias assessment components for the assessment of C-RCTs and I-RCTs (86%, n = 92); this corroborates with the findings of this review (only one review applied all five cluster-specific risk-of-bias criteria), suggesting that very few authors perform risk-of-bias assessments that are specific to C-RCTs. Issues with incorrect analyses were also highlighted in the review of Cochrane Infectious Diseases Group reviews; for example, it was found that 57% of reviews (n = 92) combined non-adjusted C-RCTs with I-RCTs in the meta-analysis. 
Limitations and Further Work
The main limitation of this research is that only Cochrane reviews were included. Yet, the criteria are certainly applicable to other types of reviews, such as health technology assessment reviews and reviews published in journals. However, the results in the current study may not be generalisable to other types of reviews because guidance regarding the inclusion of C-RCTs is given in the Cochrane handbook and extra support is provided by Cochrane editorial teams. As further work, this project could be extended to assess the inclusion of C-RCTs in other types of reviews.
A second limitation of the current study is that only reviews that used the term “cluster” or “community” were included; therefore, any reviews that included C-RCTs but did not use these specific terms would not have been included. Identifying such reviews would involve obtaining the published article for each included trial in each review and would not be feasible. Such reviews may have been of poorer quality; therefore, the results reported in this article may be an underestimation of the methodological and reporting deficiencies identified in this study.
A further limitation of this study is the search date; the last search was carried out in November 2013 and the included reviews were published between June and December 2013. Although, many Cochrane reviews have been published since this date, they are unlikely to differ greatly from those included in the current study because, to the authors’ knowledge, no updated or new guidance has been published or presented at conferences.
In some respects, the quality of a review that includes C-RCTs is limited by the methodological and reporting quality of the included C-RCTs. For example, if a study does not adjust data for clustering, and does not report an ICC and average cluster size, it is difficult for review authors to obtain an estimate of treatment effect which takes the effects of clustering into consideration. The current study highlights that flaws exist in published C-RCTS, with many of the included C-RCTs not reporting key information, and not adjusting analyses for clustering. This could be due to trial authors not following the Consort statement for C-RCTs , or statistical peer reviewers not identifying lack of adjusted analyses, or even included trials being published before the Consort statement was published. Further work could explore how often trials are poorly reported and why this is the case.
Recommendations to Authors of Reviews
Table 2 provides a checklist that we recommend review authors use when including C-RCTs in reviews. The checklist is applicable to any systematic review incorporating C-RCTs. Review authors can use the checklist to carry out their review and editorial teams and users of reviews can use the checklist to check reviews for errors.
It is clear that a consolidated and comprehensive set of guidelines is needed that address the different aspects of including C-RCTs in systematic reviews to help improve the quality of published reviews including C-RCTs. An extension to existing systematic review and meta-analysis guidelines, such as the PRISMA statement , would meet this need; the criteria used in the current study could contribute to these extended guidelines. It is hoped that the current study will promote methods for including C-RCTs in reviews and help to improve the quality of future reviews.
S1 File. References for included C-RCTs in each review.
S2 File. PRISMA Checklist.
S1 Table. Summary of included reviews.
S2 Table. Assessment of “Identifying C-RCTs”.
S3 Table. Assessment of “Reporting C-RCTs”.
S4 Table. Assessment of “Assessing risk of bias of C-RCTs”.
S5 Table. Assessment of “Analysing C-RCTs”.
S6 Table. Additional results for “Reporting C-RCTs”.
We would like to thank Christianne Esparza for locating the included trial reports. We are also grateful to the review authors for publishing the reviews on which this research is based.
Conceived and designed the experiments: SD MR. Performed the experiments: MR. Analyzed the data: MR. Wrote the paper: MR SD PG.
- 1. The Cochrane Collaboration. Cochrane 2015. Available from: www.cochrane.org.
- 2. Campbell MK, Piaggio G, Elbourne DR, Altman DG. Consort 2010 statement: extension to cluster randomised trials. BMJ. 2012. doi: 10.1136/bmj.e5661 WOS:000308633800001.
- 3. Higgins J, Green S, (editors). Cochrane handbook for systematic reviews of interventions: Wiley Online Library; 2008.
- 4. Donner A, Klar N. Issues in the meta-analysis of cluster randomized trials. Stat Med. 2002;21(19):2971–80. doi: 10.1002/Sim.1301. WOS:000178357700022. pmid:12325113
- 5. Donner A, Piaggio G, Villar J. Statistical methods for the meta-analysis of cluster randomization trials. Stat Methods Med Res. 2001;10(5):325–38. WOS:000171793800002. pmid:11697225
- 6. Laopaiboon M. Meta-analyses involving cluster randomization trials: a review of published literature in health care. Stat Methods Med Res. 2003;12(6):515–30. pmid:14653354
- 7. Farrelly S, Brown GE, Flach C, Barley E, Laugharne R, Henderson C. User-held personalised information for routine care of people with severe mental illness. Cochrane Database Syst Rev. 2013;10:Cd001711. Epub 2013/10/08. doi: 10.1002/14651858.CD001711.pub2 pmid:24096715.
- 8. Stanton A, Grimshaw G. Tobacco cessation interventions for young people. Cochrane Database Syst Rev. 2013;8:Cd003289. Epub 2013/08/27. doi: 10.1002/14651858.CD003289.pub5 pmid:23975659.
- 9. Lopez LM, Tolley EE, Grimes DA, Chen M, Stockton LL. Theory-based interventions for contraception. Cochrane Database Syst Rev. 2013;8:Cd007249. Epub 2013/08/08. doi: 10.1002/14651858.CD007249.pub4 pmid:23921563.
- 10. Lavender T, Richens Y, Milan SJ, Smyth RM, Dowswell T. Telephone support for women during pregnancy and the first six weeks postpartum. Cochrane Database Syst Rev. 2013;7:Cd009338. Epub 2013/07/25. doi: 10.1002/14651858.CD009338.pub2 pmid:23881662.
- 11. Stead LF, Hartmann-Boyce J, Perera R, Lancaster T. Telephone counselling for smoking cessation. Cochrane Database Syst Rev. 2013;8:Cd002850. Epub 2013/08/13. doi: 10.1002/14651858.CD002850.pub3 pmid:23934971.
- 12. Hemmingsen B, Lund SS, Gluud C, Vaag A, Almdal TP, Hemmingsen C, et al. Targeting intensive glycaemic control versus targeting conventional glycaemic control for type 2 diabetes mellitus. Cochrane Database Syst Rev. 2013;11:Cd008143. Epub 2013/11/12. doi: 10.1002/14651858.CD008143.pub3 pmid:24214280.
- 13. Lazzerini M, Rubert L, Pani P. Specially formulated foods for treating children with moderate acute malnutrition in low- and middle-income countries. Cochrane Database Syst Rev. 2013;6:Cd009584. Epub 2013/06/25. doi: 10.1002/14651858.CD009584.pub2 pmid:23794237.
- 14. Van der Meer RM, Willemsen MC, Smit F, Cuijpers P. Smoking cessation interventions for smokers with current or past depression. Cochrane Database Syst Rev. 2013;8:Cd006102. Epub 2013/08/22. doi: 10.1002/14651858.CD006102.pub2 pmid:23963776.
- 15. Brocklehurst P, Kujan O, O'Malley LA, Ogden G, Shepherd S, Glenny AM. Screening programmes for the early detection and prevention of oral cancer. Cochrane Database Syst Rev. 2013;11:Cd004150. Epub 2013/11/21. doi: 10.1002/14651858.CD004150.pub4 pmid:24254989.
- 16. Manser R, Lethaby A, Irving LB, Stone C, Byrnes G, Abramson MJ, et al. Screening for lung cancer. Cochrane Database Syst Rev. 2013;6:Cd001991. Epub 2013/06/25. doi: 10.1002/14651858.CD001991.pub3 pmid:23794187.
- 17. Yonemoto N, Dowswell T, Nagai S, Mori R. Schedules for home visits in the early postpartum period. Cochrane Database Syst Rev. 2013;7:Cd009326. Epub 2013/07/25. doi: 10.1002/14651858.CD009326.pub2 pmid:23881661.
- 18. Sharma SK, Sharma A, Kadhiravan T, Tharyan P. Rifamycins (rifampicin, rifabutin and rifapentine) compared to isoniazid for preventing tuberculosis in HIV-negative people at risk of active TB. Cochrane Database Syst Rev. 2013;7:Cd007545. Epub 2013/07/06. doi: 10.1002/14651858.CD007545.pub2 pmid:23828580.
- 19. Foster C, Richards J, Thorogood M, Hillsdon M. Remote and web 2.0 interventions for promoting physical activity. Cochrane Database Syst Rev. 2013;9:Cd010395. Epub 2013/10/03. doi: 10.1002/14651858.CD010395.pub2 pmid:24085594.
- 20. Hajek P, Stead LF, West R, Jarvis M, Hartmann-Boyce J, Lancaster T. Relapse prevention interventions for smoking cessation. Cochrane Database Syst Rev. 2013;8:Cd003999. Epub 2013/08/22. doi: 10.1002/14651858.CD003999.pub4 pmid:23963584.
- 21. Schoonees A, Lombard M, Musekiwa A, Nel E, Volmink J. Ready-to-use therapeutic food for home-based treatment of severe acute malnutrition in children from six months to five years of age. Cochrane Database Syst Rev. 2013;6:Cd009000. Epub 2013/06/08. doi: 10.1002/14651858.CD009000.pub2 pmid:23744450.
- 22. Chamberlain C, O'Mara-Eves A, Oliver S, Caird JR, Perlen SM, Eades SJ, et al. Psychosocial interventions for supporting women to stop smoking in pregnancy. Cochrane Database Syst Rev. 2013;10:Cd001055. Epub 2013/10/25. doi: 10.1002/14651858.CD001055.pub4 pmid:24154953; PubMed Central PMCID: PMCPmc4022453.
- 23. Galappaththy GN, Tharyan P, Kirubakaran R. Primaquine for preventing relapse in people with Plasmodium vivax malaria treated with chloroquine. Cochrane Database Syst Rev. 2013;10:Cd004389. Epub 2013/10/29. doi: 10.1002/14651858.CD004389.pub3 pmid:24163057.
- 24. Dodd JM, Jones L, Flenady V, Cincotta R, Crowther CA. Prenatal administration of progesterone for preventing preterm birth in women considered to be at risk of preterm birth. Cochrane Database Syst Rev. 2013;7:Cd004947. Epub 2013/08/02. doi: 10.1002/14651858.CD004947.pub3 pmid:23903965.
- 25. Schaafsma FG, Whelan K, van der Beek AJ, van der Es-Lambeek LC, Ojajarvi A, Verbeek JH. Physical conditioning as part of a return to work strategy to reduce sickness absence for workers with back pain. Cochrane Database Syst Rev. 2013;8:Cd001822. Epub 2013/08/31. doi: 10.1002/14651858.CD001822.pub3 pmid:23990391.
- 26. Omidvari AH, Vali Y, Murray SM, Wonderling D, Rashidian A. Nutritional screening for improving professional practice for patient outcomes in hospital and primary care settings. Cochrane Database Syst Rev. 2013;6:Cd005539. Epub 2013/06/08. doi: 10.1002/14651858.CD005539.pub2 pmid:23744516.
- 27. Rice VH, Hartmann-Boyce J, Stead LF. Nursing interventions for smoking cessation. Cochrane Database Syst Rev. 2013;8:Cd001188. Epub 2013/08/14. doi: 10.1002/14651858.CD001188.pub4 pmid:23939719.
- 28. Van Ginneken N, Tharyan P, Lewin S, Rao GN, Meera SM, Pian J, et al. Non-specialist health worker interventions for the care of mental, neurological and substance-abuse disorders in low- and middle-income countries. Cochrane Database Syst Rev. 2013;11:Cd009149. Epub 2013/11/20. doi: 10.1002/14651858.CD009149.pub2 pmid:24249541.
- 29. Cramp F, Hewlett S, Almeida C, Kirwan JR, Choy EH, Chalder T, et al. Non-pharmacological interventions for fatigue in rheumatoid arthritis. Cochrane Database Syst Rev. 2013;8:Cd008322. Epub 2013/08/27. doi: 10.1002/14651858.CD008322.pub2 pmid:23975674.
- 30. Tusting LS, Thwing J, Sinclair D, Fillinger U, Gimnig J, Bonner KE, et al. Mosquito larval source management for controlling malaria. Cochrane Database Syst Rev. 2013;8:Cd008923. Epub 2013/08/30. doi: 10.1002/14651858.CD008923.pub2 pmid:23986463.
- 31. Sandall J, Soltani H, Gates S, Shennan A, Devane D. Midwife-led continuity models versus other models of care for childbearing women. Cochrane Database Syst Rev. 2013;8:Cd004667. Epub 2013/08/22. doi: 10.1002/14651858.CD004667.pub3 pmid:23963739.
- 32. Clement S, Lassman F, Barley E, Evans-Lacko S, Williams P, Yamaguchi S, et al. Mass media interventions for reducing mental health-related stigma. Cochrane Database Syst Rev. 2013;7:Cd009453. Epub 2013/07/25. doi: 10.1002/14651858.CD009453.pub2 pmid:23881731.
- 33. Kinnersley P, Phillips K, Savage K, Kelly MJ, Farrell E, Morgan B, et al. Interventions to promote informed consent for patients undergoing surgical and other invasive healthcare procedures. Cochrane Database Syst Rev. 2013;7:Cd009445. Epub 2013/07/09. doi: 10.1002/14651858.CD009445.pub2 pmid:23832767.
- 34. Dangour AD, Watson L, Cumming O, Boisson S, Che Y, Velleman Y, et al. Interventions to improve water quality and supply, sanitation and hygiene practices, and their effects on the nutritional status of children. Cochrane Database Syst Rev. 2013;8:Cd009382. Epub 2013/08/02. doi: 10.1002/14651858.CD009382.pub2 pmid:23904195.
- 35. Kahn SR, Morrison DR, Cohen JM, Emed J, Tagalakis V, Roussin A, et al. Interventions for implementation of thromboprophylaxis in hospitalized medical and surgical patients at risk for venous thromboembolism. Cochrane Database Syst Rev. 2013;7:Cd008201. Epub 2013/07/19. doi: 10.1002/14651858.CD008201.pub2 pmid:23861035.
- 36. Bath-Hextall FJ, Matin RN, Wilkinson D, Leonardi-Bee J. Interventions for cutaneous Bowen's disease. Cochrane Database Syst Rev. 2013;6:Cd007281. Epub 2013/06/25. doi: 10.1002/14651858.CD007281.pub2 pmid:23794286.
- 37. Civljak M, Stead LF, Hartmann-Boyce J, Sheikh A, Car J. Internet-based interventions for smoking cessation. Cochrane Database Syst Rev. 2013;7:Cd007078. Epub 2013/07/11. doi: 10.1002/14651858.CD007078.pub4 pmid:23839868.
- 38. Kruis AL, Smidt N, Assendelft WJ, Gussekloo J, Boland MR, Rutten-van Molken M, et al. Integrated disease management interventions for patients with chronic obstructive pulmonary disease. Cochrane Database Syst Rev. 2013;10:CD009437. doi: 10.1002/14651858.CD009437.pub2 pmid:24108523.
- 39. Thomas RE, Jefferson T, Lasserson TJ. Influenza vaccination for healthcare workers who care for people aged 60 or older living in long-term care institutions. Cochrane Database Syst Rev. 2013;7:CD005187. doi: 10.1002/14651858.CD005187.pub4 pmid:23881655.
- 40. Hughes C, Tunney M, Bradley MC. Infection control strategies for preventing the transmission of meticillin-resistant Staphylococcus aureus (MRSA) in nursing homes for older people. Cochrane Database Syst Rev. 2013;11:Cd006354. Epub 2013/11/21. doi: 10.1002/14651858.CD006354.pub4 pmid:24254890.
- 41. Marinho VC, Worthington HV, Walsh T, Clarkson JE. Fluoride varnishes for preventing dental caries in children and adolescents. Cochrane Database Syst Rev. 2013;7:Cd002279. Epub 2013/07/13. doi: 10.1002/14651858.CD002279.pub2 pmid:23846772.
- 42. Holme O, Bretthauer M, Fretheim A, Odgaard-Jensen J, Hoff G. Flexible sigmoidoscopy versus faecal occult blood testing for colorectal cancer screening in asymptomatic individuals. Cochrane Database Syst Rev. 2013;9:Cd009259. Epub 2013/10/03. doi: 10.1002/14651858.CD009259.pub2 pmid:24085634.
- 43. Alfirevic Z, Stampalija T, Gyte GM. Fetal and umbilical Doppler ultrasound in high-risk pregnancies. Cochrane Database Syst Rev. 2013;11:Cd007529. Epub 2013/11/14. doi: 10.1002/14651858.CD007529.pub3 pmid:24222334.
- 44. Rosendal M, Blankenstein AH, Morriss R, Fink P, Sharpe M, Burton C. Enhanced care by generalists for functional somatic symptoms and disorders in primary care. Cochrane Database Syst Rev. 2013;10:Cd008142. Epub 2013/10/22. doi: 10.1002/14651858.CD008142.pub2 pmid:24142886.
- 45. Hu H, Li C, Li F, Chen J, Sun J, Zou S, et al. Enamel etching for bonding fixed orthodontic braces. Cochrane Database Syst Rev. 2013;11:Cd005516. Epub 2013/11/26. doi: 10.1002/14651858.CD005516.pub2 pmid:24272130.
- 46. Gomes B, Calanzani N, Curiale V, McCrone P, Higginson IJ. Effectiveness and cost-effectiveness of home palliative care services for adults with advanced illness and their caregivers. Cochrane Database Syst Rev. 2013;6:Cd007760. Epub 2013/06/08. doi: 10.1002/14651858.CD007760.pub2 pmid:23744578.
- 47. Brocklehurst P, Price J, Glenny AM, Tickle M, Birch S, Mertz E, et al. The effect of different methods of remuneration on the behaviour of primary care dentists. Cochrane Database Syst Rev. 2013;11:Cd009853. Epub 2013/11/07. doi: 10.1002/14651858.CD009853.pub2 pmid:24194456.
- 48. Fellmeth GL, Heffernan C, Nurse J, Habibula S, Sethi D. Educational and skills-based interventions for preventing relationship and dating violence in adolescents and young adults. Cochrane Database Syst Rev. 2013;6:Cd004534. Epub 2013/06/20. doi: 10.1002/14651858.CD004534.pub3 pmid:23780745.
- 49. Moore ZE, Webster J. Dressings and topical agents for preventing pressure ulcers. Cochrane Database Syst Rev. 2013;8:Cd009362. Epub 2013/08/21. doi: 10.1002/14651858.CD009362.pub2 pmid:23955535.
- 50. Kredo T, Ford N, Adeniyi FB, Garner P. Decentralising HIV treatment in lower- and middle-income countries. Cochrane Database Syst Rev. 2013;6:Cd009987. Epub 2013/06/29. doi: 10.1002/14651858.CD009987.pub2 pmid:23807693.
- 51. Morag I, Ohlsson A. Cycled light in the intensive care unit for preterm and low birth weight infants. Cochrane Database Syst Rev. 2013;8:Cd006982. Epub 2013/08/06. doi: 10.1002/14651858.CD006982.pub3 pmid:23913547.
- 52. Gillaizeau F, Chan E, Trinquart L, Colombet I, Walton RT, Rege-Walther M, et al. Computerized advice on drug dosage to improve prescribing practice. Cochrane Database Syst Rev. 2013;11:Cd002894. Epub 2013/11/13. doi: 10.1002/14651858.CD002894.pub3 pmid:24218045.
- 53. Paul M, Dickstein Y, Schlesinger A, Grozinsky-Glasberg S, Soares-Weiser K, Leibovici L. Beta-lactam versus beta-lactam-aminoglycoside combination therapy in cancer patients with neutropenia. Cochrane Database Syst Rev. 2013;6:Cd003038. Epub 2013/07/03. doi: 10.1002/14651858.CD003038.pub2 pmid:23813455.
- 54. Shinohara K, Honyashiki M, Imai H, Hunot V, Caldwell DM, Davies P, et al. Behavioural therapies versus other psychological therapies for depression. Cochrane Database Syst Rev. 2013;10:Cd008696. Epub 2013/10/17. doi: 10.1002/14651858.CD008696.pub2 pmid:24129886.
- 55. Lopez LM, Otterness C, Chen M, Steiner M, Gallo MF. Behavioral interventions for improving condom use for dual protection. Cochrane Database Syst Rev. 2013;10:Cd010662. Epub 2013/10/29. doi: 10.1002/14651858.CD010662.pub2 pmid:24163112.
- 56. Zalmanovici Trestioreanu A, Fraser A, Gafter-Gvili A, Paul M, Leibovici L. Antibiotics for preventing meningococcal infections. Cochrane Database Syst Rev. 2013;10:CD004785. doi: 10.1002/14651858.CD004785.pub5 pmid:24163051.
- 57. Donegan S, Faragher B, Garner P. Cluster randomised trials in Cochrane reviews of infectious diseases. Abstracts of the 17th Cochrane Colloquium; 2009 11–14 Oct; Singapore, Singapore: John Wiley & Sons; 2009.
- 58. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151(4):264–9. pmid:19622511