Background

Systematic reviews of the literature occupy the highest position in currently proposed hierarchies of evidence [1] and occupy this top position for two fundamental premises. Firstly, a systematic review involves the application of scientific strategies which limit bias by systematic assembly, critical appraisal and synthesis of relevant studies on a particular topic [2,3]. Secondly, reviews that include a meta-analysis provide precise estimates of the association studied[4]. Because of the importance of systematic reviews in summarizing the advances of health care knowledge, their number is growing rapidly [4]. If systematic reviews in fact represent the best level of evidence, they are likely to have great clinical importance[4]. It follows that they may be cited often in the literature. The acknowledgement that one article gives to another is a reference; the acknowledgement that one article receives from another is a citation [5]. The number of citations an article receives after publication reflects its impact on the scientific community. There have been a few recent attempts to identify and analyze “the most cited articles” in various specialties [5–9]. However, an analysis of the most frequently cited systematic review and meta-analysis (SRM) articles has not yet been reported. Montori and colleagues examined whether systematic reviews receive more citations than narrative reviews [4]. They found that rigorous systematic reviews were cited significantly more often than narrative reviews. In this paper we sought to identify and examine the characteristics of the most cited SRM related articles, such as ranking, year of publication, publishing journal, topic and contribution of different world regions to most cited SRM articles.

In addition, we assessed whether there was an association between year of publication, number of authors, number of pages, journals’ impact factor, and average citations per year.

Methods

The Science Citation Index of the Institute for Scientific Information (ISI) was searched in October 2012 for systematic review and meta-analysis related articles. We searched for articles using validated keywords for identifying SRM [10]. To accredit an article to countries, the method of "absolute country counting" was adopted, in which each country contributing to an article received one paper credit based on the lead author’s correspondence or reprint address [11]. The 100 most-cited articles were selected for further descriptive analyses. Data collected included the year of publication, the topic covered, lead author’s correspondence country of origin, and number of citations.

We used a density-equalizing map to visualize the citation classics by the corresponding address of the author. We used Gastner and Newman's algorithm [12] in order to produce a map of the world in which countries were re-sized according to the number of most cited SRM articles. These calculations employ a diffusion equation in the Fourier domain borrowed from elementary physics, which allows variable resolution by tracking moving boundaries [12].

The impact factors and immediacy factors of journals listed in the 2012 Journal Citation Reports Science Edition were adopted as quantitative tools for evaluating journals in which these articles were published. A journal’s impact factor is a measure of the frequency with which the "average article" in the journal has been cited in a given period of time. The impact factor for a journal is calculated based on a three-year period, and can be considered to be the average number of times published papers are cited up to two years after publication. Non-parametric (Spearman’s) correlation was used to assess the association between years since publication (with reference to the year 2012), numbers of authors, article’s length (number of pages), journal’s impact factor and average citations per year.

Results

The 100 articles are listed in Table 1 in descending order, ranked according to the total number of citations since publication. Among the 100 citation classics, the most cited article received 7308 citations, and the least-cited 675 citations. The average citations per year ranged from 27.8 to 401.6. Figure 1 shows the density-equalizing map illustrating the number of contributions for each country in SRM citation classics. Density equalising mapping demonstrates that a relatively small number of countries were responsible for the majority of the top cited SRM articles (Figure 1). First authors from the USA produced the highest citation classics (n=46), followed by the UK (n=28) and Canada (n=15) (Figure 1). All the 100 most cited articles were published in the English Language.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 1. Spatial distribution of the 100 most frequently cited systematic reviews and meta-analyses related articles using density equalizing mapping.

The area of each country were re-sized in proportion to its total number of 100 most frequently cited systematic reviews and meta-analyses related articles.

https://doi.org/10.1371/journal.pone.0078517.g001

The year of publication with the relevant number of classics identified is shown in Figure 2. The oldest article was published in 1977 and the most recent article in 2008. Figure 3A shows correlation between average citations per year and year since publication (with reference to 2012). There was a statistically significant negative correlation between average citations per year and year since publication (Spearman’s rho = -0.636, 95% CI -0.739 to -0.501, p=0.0001), such that the average citations per year reduces with the number of years since publication.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 2. Graph demonstrating the time period of publication of the 100 most cited systematic reviews and meta-analyses related articles.

https://doi.org/10.1371/journal.pone.0078517.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 3. Correlation between since publication, number of authors, journals’ impact factor and average citation per year.

https://doi.org/10.1371/journal.pone.0078517.g003

The number of authors of the most cited articles ranged from one to 22. Four of the articles were authored by a single author and 18 articles by two authors. There was a statistically significant positive correlation between number of authors and average citations per year (rho=0.320, 95% CI 0.132 to 0.486, p=0.001), such that the greater the number of authors, the higher the average citation per year (Figure 3B). The median length of article was 10 pages (range: 2 to 75 pages). There was no statistically significant correlation between length and average citations per year (rho = -0.052, 95% CI -0.246 to 0.146, p=0.608). The most cited articles were published in 42 journals (Table 2), led by Journal of The American Medical Association (n=18) followed by the British Medical Journal (n=14), The Lancet (n=13), and the Annals of Internal Medicine (n=7). Journal impact factors ranged from 1.412 (for Biometrics) to 51.658 (for the New England Journal of Medicine).

There was a statistically significant positive correlation between average citations per year and journal impact factor (rho=0. 240, 95% CI 0.045 to 0.416, p=0. 016) (Figure 3C). General and internal medicine were the main topics covered by these highly cited articles (n=59). Considerable attention was also given to Psychology and Psychiatry (n=13).

The top-100 list contained landmark contributions dealing with methodological aspects of conducting systematic reviews and meta-analysis (n=17). At number 1, DerSimonian and Laird’s landmark article which introduced a novel simple random effects model for combining studies. Egger et al. (number-2) examined the prevalence of funnel plot asymmetry among published meta-analyses. Higgins et al. (number-4 and number-7) developed a new measure (I²) for quantifying heterogeneity between studies included in a meta-analysis. Stroup et al. (number-12) reported a proposal for reporting meta-analyses of observational studies. The list of the most cited articles also reflects major advances in the management of non-communicable diseases (n=40) and in the identification of their predisposing factors for such diseases over the last 30 years. Baigent and colleagues (number-8) examined the efficacy and safety of statins on cholesterol lowering. Abe and co-researchers examined effects of chemotherapy and hormonal therapy for early breast cancer recurrence (number-10) and Lewington et al. (number-13) examined age-specific relevance of usual blood pressure and vascular mortality.

Discussion

This study identified and characterised the 100 most cited SRM related articles published in the past three decades, providing an overview of the citation frequency of these most cited articles. The list of the most cited articles identifies first authors and topics which reflect advances in methodological techniques in meta-analysis, major advances in the management of chronic diseases, and identification of predisposing factors over the last 30 years. Some of the most frequently cited articles were methodological papers. As expected the most highly cited papers were more likely to be published in journals high on the impact factor list [13,14]. It is important to note that, at present, no Cochrane review is among the 100 most cited SRM related articles. Cochrane reviews are systematic reviews of primary research in human health care and health policy, and are internationally recognised as the highest standard in evidence-based health care [15,16]. They investigate the effects of interventions for prevention, treatment and rehabilitation [15,16]. They also assess the accuracy of a diagnostic test for a given condition in a specific patient group and setting. They are published online in The Cochrane Library [15,16]. The low citations received by Cochrane reviews may be due to improper citations of Cochrane reviews, and the relatively recent tracking of Cochrane reviews by ISI. In addition, ISI Science Citation Index database covers all new and substantially updated Cochrane reviews from January 2005, and the first impact factor for Cochrane Database of Systematic Review was released in June 2008.

We found that almost half of the most cited SRM related articles originated in the US. This Figure is comparable with the origin of citation classics in other fields [5–9]. The overwhelming influence the US has on medical research may be due to its large underlying population, enormous financial resources available to the scientific community in the country and its high population of active citing researchers compared to other countries [6,7]. Studies have demonstrated that biomedical research productivity worldwide is largely dependent on each country’s per capita gross national product and the expenditure allotted for research and development [17,18].

Our results support previous findings [5–9] that first authors from Africa, Asia, and South America had minimal or no contributions in the most cited articles. Scientific publishing activity worldwide over the past decades shows that most countries in these regions have low levels of publication [19]. The above finding is not a surprise because difficulties in research, publication, information access, and language barriers facing the least-developed countries are profound and seem almost intractable. Most information published in journals based in low and middle-income countries (LMICs) never leaves there home borders because these journals are largely excluded from major bibliographic databases. In addition, most of the reviews produced to date address health conditions that are priorities in the developed world[20]. Many major health concerns in LMICs have yet to be made the subject of a citation classic review, although there are signs that this may be changing[21].

In addition, the difficulties of conducting randomised controlled trials and other high quality studies in resource-poor situations result in the exclusion of many LMIC studies from systematic reviews [22]. However, there is a need to challenge the status quo. Scientists from these regions should forge multiple collaborations beyond historical, political, and cultural lines to share knowledge and expertise on SRM. In addition, there is a need to promote research in SRM in less developed regions of the world. This may involve but is not limited to the political will for research capacity development among LIMC health policymakers, the training of LMIC researchers to be competent in systematic review techniques, the development of infrastructure including research and academic institutes, the improvement of current collaborative partnerships with developed nations, increased sponsorship and support from world agencies such as the World Health Organization and the United Nations Organization.

Although we have tried to eliminate potential flaws in our citation analysis, some limitations were inevitable and are linked to the inherent problems of citation analysis [23,24]. The citation of a scientific article usually follows a time course, it is usually not cited until one to two years after publication, reaches a maximum after three to ten years, then declines [6]. Another problem is oriented or biased citing, including various types of conscious or unconscious biases, such as self-citation (bias towards one’s own work), in-house (bias towards friends or colleagues), journal or powerful person (bias towards reviewers, editors, members of grant awarding bodies), negative citation (bias towards potential negative credits), English language (bias towards publishing and referencing English articles), and omission bias (bias towards not referencing competitors or sources contradictory to one’s own results) [25]. Other limitations include the incorrect citation of origin for the authors. By using the author addresses listed in the bylines of research articles, one can only identify countries and organizations where the authors were employed when the research was done or where the article was written [18].

Conclusion

Since the late 1970s, the USA, UK, and Canada have taken leadership in the production of citation classic papers. No author from LMICs led any of the most cited 100 SRM. There is a need to challenge the status quo. Scientists from LMIC should forge multiple collaborations to share knowledge and expertise on SRM. In addition, there is a need to strengthen research capacity in these countries and more support should be provided for the advancement of research efforts.