Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

Articles by Latin American Authors in Prestigious Journals Have Fewer Citations

  • Rogerio Meneghini ,

    rogerio.meneghini@bireme.org

    Affiliations: BIREME-PAHO–WHO, Latin-American and Caribbean Center on Health Sciences Information, São Paulo, Brazil, DIS-Departamento de Informática Médica, Universidade Federal de São Paulo, São Paulo, Brazil

  • Abel L. Packer,

    Affiliations: BIREME-PAHO–WHO, Latin-American and Caribbean Center on Health Sciences Information, São Paulo, Brazil, DIS-Departamento de Informática Médica, Universidade Federal de São Paulo, São Paulo, Brazil

  • Lilian Nassi-Calò

    Affiliation: BIREME-PAHO–WHO, Latin-American and Caribbean Center on Health Sciences Information, São Paulo, Brazil

Articles by Latin American Authors in Prestigious Journals Have Fewer Citations

  • Rogerio Meneghini, 
  • Abel L. Packer, 
  • Lilian Nassi-Calò
PLOS
x
  • Published: November 25, 2008
  • DOI: 10.1371/journal.pone.0003804

Abstract

Background

The journal Impact factor (IF) is generally accepted to be a good measurement of the relevance/quality of articles that a journal publishes. In spite of an, apparently, homogenous peer-review process for a given journal, we hypothesize that the country affiliation of authors from developing Latin American (LA) countries affects the IF of a journal detrimentally.

Methodology/Principal Findings

Seven prestigious international journals, one multidisciplinary journal and six serving specific branches of science, were examined in terms of their IF in the Web of Science. Two subsets of each journal were then selected to evaluate the influence of author's affiliation on the IF. They comprised contributions (i) with authorship from four Latin American (LA) countries (Argentina, Brazil, Chile and Mexico) and (ii) with authorship from five developed countries (England, France, Germany, Japan and USA). Both subsets were further subdivided into two groups: articles with authorship from one country only and collaborative articles with authorship from other countries. Articles from the five developed countries had IF close to the overall IF of the journals and the influence of collaboration on this value was minor. In the case of LA articles the effect of collaboration (virtually all with developed countries) was significant. The IFs for non-collaborative articles averaged 66% of the overall IF of the journals whereas the articles in collaboration raised the IFs to values close to the overall IF.

Conclusion/Significance

The study shows a significantly lower IF in the group of the subsets of non-collaborative LA articles and thus that country affiliation of authors from non-developed LA countries does affect the IF of a journal detrimentally. There are no data to indicate whether the lower IFs of LA articles were due to their inherent inferior quality/relevance or psycho-social trend towards under-citation of articles from these countries. However, further study is required since there are foreseeable consequences of this trend as it may stimulate strategies by editors to turn down articles that tend to be under-cited.

Introduction

Scientists from developing countries seek arduously to publish their papers in prestigious mainstream international journals. Submission acceptance influences their career advancement and success in obtaining research grant funding. In particular, many Latin American research funding agencies and institutional committees responsible for deciding on promotions or selection of candidates to academic positions, frequently base their decisions on the impact factor (IF, produced by the Journal Citation Report, JCR, Thomson-Reuters) of the journals where the articles of the applicant have been published. More developed countries have also followed this procedure, like Italy [1], Nordic countries [2], Canada [3] and Hungary [4] among others. The IF of a journal indicates the average number of citations the articles of this journal received among all journals indexed in this database in a given period of time. For instance, the two-year based IF for a journal in the year 2006 is obtained by dividing the number of citations received in 2006 for articles published in 2004 and 2005 by the number of these articles published in 2004 and 2005. The IF is accepted as a reasonable measurement of the quality of a journal but IFs can only be compared if potential bias is taken into consideration e.g. the journals being compared must belong to the same area of investigation [5]. The use of the average IF of the journals in which an author is published as a direct measurement of his/her quality is, however dangerously misleading. Seglen, for instance [2], draws attention to the fact that the most cited half of journal articles are cited 10 times more often than the less cited half. Therefore, the possibility is not negligible that two scientists with the same pattern of publications in journals (and therefore with a similar weighted average IF) may have very distinct rates of citations per article. Distinct citation trends in sub-areas covered by a journal further prejudice the comparison of scientists in the same area [6].

When dealing with the collective group data, however, the use of citation analysis agrees significantly with peer opinions. This has been the case for assessment of research departments [7] and of national PhD programs [8]. Also, many studies refer to a good fit between the opinion of peers on the quality of the articles in a journal and its IF (e.g. chapter 5, reference 5).

The question of the national contribution to the impact of an international journal has been pointed out [9]. As far as we know, however, there have been no studies on the trend of impact of articles published in a given journal with respect to author's affiliation. We hypothesize that affiliation affects IF of entire subsets of articles when compared with the IF of all articles published by the journal.

Materials and Methods

We have chosen to analyze the output of the four LA countries with the highest number of publications in the Web of Science (WoS) data base: Argentina, Brazil, Chile and Mexico. In 2004 and 2005 the total contribution of these 4 countries represented 93.2% of the total LA WoS entries. This output was measured in six journals with a high reputation in their areas of knowledge and one multidisciplinary journal: Astrophysical Journal, Chemistry of Materials, Journal of the American Chemical Society, Journal of Biological Chemistry, Journal of Immunology, Physical Review Letters and Proceedings of the National Academy of Sciences of the USA. A total of 1244 articles published in the years 2004 and 2005 with the above country's authorships were found in the WoS database. These 1244 articles were divided in two subsets for each journal: one with LA authorship only (total of 219 articles) and another one including both non-collaborative and collaborative articles with other countries, virtually all being developed countries (219+1025 = 1244). The 2006 IFs of the two groups for each journal were calculated by the sum of citations given in 2006 to articles published in 2004 and 2005 divided by the number of articles published in these years. The same procedure was followed to calculate the IFs of groups of articles in these journals from five developed countries, namely, England, France, Germany, Japan and USA, for the purpose of comparison.

Attention must be given to the fact that the IF calculated according to this methodology does not correspond to the IF presented by the JCR, since the JCR and the WoS operate different journal collections. The reason for our using the WoS database is the availability of data to calculate the journal IF for countries which was not available in the standard JCR. This article and others present pitfalls in the standard calculation of IF [10] however, for the seven journals examined the overall WoS IF was 87.4±4.1% of the JCR IF, which would indicate that in the context of the present study the same conclusions would be achieved using the JCR IF.

Results

The trend of lower IF for LA countries' articles is readily noticed in Table 1. For 22 out of 26 journal/“total country” subsets the IF was inferior to the total IF of the journal. One should notice that on average, 77% of the total subsets represented international collaborations with a developed country. When only the non-collaborative articles are considered all of the measurable IFs are considerably lower. To provide a broad picture for comparison, the average for the overall IF of the seven journals was 5.93; for the total (collaborative and non-collaborative) LA articles it was 5.04 and for the non-collaborative LA articles it was 3.38.

thumbnail
Table 1. The 2006 impact factor of Web of Science specialized Journals and subsets of articles from Argentina, Brazil, Chile and Mexico.

doi:10.1371/journal.pone.0003804.t001

If we consider the collaborative LA articles only, the average IF raises to 5.25 (not shown in Table 1) approaching the average of the overall IFs of the seven journals (5.93).

Table 2 has an equivalent framework to Table 1, except that it refers to five developed countries: England, France, Germany, Japan, and USA. It permits an important comparison with the LA countries. In these cases the differences in IF for the non-collaborative articles of countries and the overall IF of the journals is not significant: the average for the overall IF of the seven journals was 5.93; for the total (collaborative and non-collaborative) articles from these five countries it was 6.36 and for the non-collaborative articles it was 5.73.

thumbnail
Table 2. The 2006 impact factor of Web of Science specialized Journals and subsets of articles from England, France, Germany, Japan and USA.

doi:10.1371/journal.pone.0003804.t002

Discussion

The visibility of scientific research, as a rule, benefits from increasing collaboration [13]. In the present study international collaboration had a minor effect on the IF when referred to scientifically developed countries. The exception was the Astrophysical Journal for which the non-collaborative articles had an average IF of 4.14 and the total articles an average IF of 6.77. Probably this has to do with the need of great telescopes for obtaining the most impacting results, located in Canarias, Hawaii, South Africa, Texas and Chile and to a privileged access to the Hubble telescope. In this case the USA would be less affected in a non-collaborative work since six of the ten biggest telescopes are located in their territory and the Hubble telescope belongs to NASA-USA.

The situation is different, however, in LA countries. An interesting point is the extent of collaboration in articles from these seven journals: the percentage of 77% is considerably higher than those of the collaborative effort of these four LA countries in the context of the whole WoS database: Argentina 40.3%, Brazil 26.7%, Chile 50.2% and Mexico 53.9%. This could be a measure of the effort required to publish in these seven prestigious journals and the importance of international collaboration to accomplish it.

The most important question in regard to these data is why the subsets of the non-collaborative articles of LA countries present such a low IF as compared to the overall IF of the journals? In principle one would expect a relatively homogenous review process for all manuscripts submitted to a given journal and the same rigor for their acceptance. However, the 219 non-collaborative articles from LA fall dramatically behind the average impact of these seven journals. The groups of non-collaborative articles that were closest to the overall IF were Argentina /J. Immunol. (4.50/5.71), Brazil / J. Am. Chem. Soc. (5.67/6.55) and Chile / Proc. Natl. Acad. Sci. USA (7.17/8.37). Brazil has strong research groups in chemistry [11] and Argentina has a strong tradition in immunology with one Nobel Prize in the category ‘Physiology or Medicine’ [12]. This would seem to indicate that LA articles are ignored with the possible exception of those centers of excellence. Presently, it is impossible to make a judgment as to whether this is due to the quality/relevance of the these articles or if articles with LA authors, without international collaboration, are destined to be under-cited due to social-psychological reasons. Although not yet an object of analysis, it is noticeable that many Brazilian authors envisaging publications in mainstream journals tend to produce reference lists containing a majority of prominent authors and prestigious journals and avoid citations of their compatriots, as if this would give more weight to their publications [14], [15]. One may wonder if a similar behavior also occurs with authors from developed countries, leading to a significant under citation of LA articles.

Regardless of the reason for the under citation of non-collaborative LA articles a drawback may be foreseeable in regard to the competition for editorial space in the high-status journals: would an Editor, concerned about the journal IF, consider that the acceptance of a LA article might weight against its value? After all, it is known that several strategies for increasing the IF are used by editors [16]. Why not consider strategies for protection against a decrease of IF value?

In conclusion, scientometric data render it possible to detect the under-citation trend of non-collaborative LA articles of specific prestigious journals but provides no elements to decide the basis for this phenomenon. Possible reasons could include psycho-social bias or real differences in scientific relevance of these articles. The only way to address this argument would be to conduct a detailed peer analysis of the articles, trying to establish a correlation between citation and quality, as has been done in similar circumstances [7], [8].

Acknowledgments

We thank Dr. Luiz R. Travassos for discussions and the thoughtful review of the manuscript.

Author Contributions

Conceived and designed the experiments: RM ALP. Performed the experiments: RM. Analyzed the data: RM ALP LNC. Wrote the paper: RM ALP LNC.

References

  1. 1. Calza L, Garbisa S (1995) Italian professorships. Nature 374: 492.
  2. 2. Seglen PO (1997) Why the impact factor of journals should not be used for evaluating research. British Medical Journal 314: 498–502. [PubMed].
  3. 3. Taubes G (1993) Measure for measure in science. Science 260: 884–886.
  4. 4. Vinkler P (1986) Evaluation of some methods for the relative assessment of scientific publications. Scientometrics 10: 157–177.
  5. 5. Moed HF (2005) Citation analysis in research evaluation. Springer.
  6. 6. Postma E (2007) Inflated impact factors? The true impact of evolutionary papers in non-evolutionary journals. PLoS One 10: e999. [PubMed].
  7. 7. Norris M, Oppenheim C (2003) Citation counts and the research assessment exercise V. J Documentation 59: 709–730.
  8. 8. Ostriker JP, Kuh CV (2003) Assessing Research-Doctorate Programs: A Methodology Study. Washington, DC: National Academy Press. http://books.nap.edu/openbook.php?record​_id10859&pageR1.
  9. 9. Schubert A, Glanzel W, Braun T (1989) Scientometric data files. A comprehensive set of indicators on 2649 journals and 96 countries in all major fields and subfields 1981–1985. Scientometrics 16: 3–478.
  10. 10. Moed HF, van Leeuwen THN, Reedijk J (1999) Towards appropriate indicators of journal impact. Scientometrics 46: 575–589.
  11. 11. Vieira CL (1999) Chemistry: Brazil Lobbies for First Nobel. Science 285: 1346.
  12. 12. Karpas A (2002) César Milstein (1927–2002): a somewhat personal reflection. Trends in Immunology 23: 321–322.
  13. 13. Glanzel W, Schubert A (2006) Analyzing scientific networks through co-authorship. In: Moed HF, Glanzel W, Schmoch U, editors. Handbook of quantitative science and technology research. Springer. In.
  14. 14. Pinto AC, de Andrade JB (1999) Impact factor of scientific journals: what is the meaning of this parameter? Quím. Nova 22: 448–453.
  15. 15. Meneghini R, Packer AL (2007) Is there science beyond English? Embo reports 8: 112–116.
  16. 16. The PLoS Medicine Editors (2006) The Impact Factor Game. PLoS Med 3(6): e291.