Introduction

Since the early times of citation indexes, self-citations have attracted the attention of bibliometricians [1, 2]. In evaluative bibliometrics, the main concern with self-citations is that they can potentially inflate impact metrics or distort their meaning [3–6]. It has thus been debated whether they should be removed from citation indicators [7–10]. In descriptive bibliometrics, on the other hand, self-citations have been studied from the point of view of scholarly communication. The motivations for self-citing have been classified [11, 12] and self-citations have been used to investigate how scientific authors relate with their own production [13, 14].

In the most general sense, self-citation occurs when an entity (e.g., an author, journal, institution, or country) receives a citation from a publication produced by the same entity [15]. Even if all self-citations derive from the act of self-referencing, self-citations and self-references should be distinguished [16]. For technical reasons shortly described below, the results derived from observing self-citations or self-references are different and so is their interpretation. This paper focuses on country self-citations.

Depending on the entity considered, self-citation can be classified into different types [6]. The most basic is the author self-citation, which occurs when the publications written by an author are cited in the following publications by the same author. For multi-authored publications, author self-citation can be defined narrowly or broadly, i.e., including or not citations generated by co-authors. Author self-citations intended in the extensive sense are sometimes called co-author self-citations [17] or all-author to all-author self-citations [6]. Journal self-citation occurs when an article published in a certain journal is cited by a subsequent publication in the same journal [18]; this form of self-citation has been mainly studied to understand how it can influence or even manipulate the journal Impact Factor [19–21]. Institution self-citation happens when the authors of the cited and of the citing publications share the same affiliation [22, 23]. By extending rather inappropriately the notion of self-citation, language self-citation refers to citations occurring among publications in the same language [24], and field self-citation for citations between publications belonging to the same academic field [25].

In the context of the study of the scientific performance of countries, the self-citations defined at the highest level of aggregation, i.e., country self-citations, have recently started to attract some attention, both from researchers [26–31] and in science policy reports [32]. A country self-citation, also called sometimes domestic citation (e.g., [33]), occurs when the publications produced by the researchers of a country are subsequently cited by researchers of the same country.

In this study, the trends of country self-citations in 50 countries worldwide are investigated in order to reveal groups of countries characterized by similar self-citation behavior in time. The main interest consists in individuating countries that deviate from standard trends because these anomalies may signal perturbations in the scientific development possibly induced by science policies. Italy is a case in point in this sense, as previous research has revealed an anomalous rise in the country’s self-citations after the introduction of pervasive bibliometric evaluation [26, 34–38].

The paper is structured as follows. In the next section, the bibliometric indicators based on country self-citations are reviewed. Then, the indicators, data, and analytical methods used in this study (time-series analysis) are presented. The main findings are shown in the Results section, whereas, in the Discussion section, we focus on countries characterized by anomalous self-citation trends. Based on the detailed reconstruction of the research policies adopted by these countries, it is argued that the anomalous trends are most likely explained by the adaptive response of scientists to the systems of incentives established by the policies themselves. Accordingly, Conclusions suggest managing bibliometric indicators in research policy contexts with extreme caution.

Review of the main indicators based on country self-citations

As anticipated, self-citations and self-references are defined, computed, and interpreted in different ways. To highlight these differences [39], for example, called the former diachronous or prospective self-citations and the latter synchronous or retrospective self-citations.

Self-citations can be defined as citations from citing sources to cited items that are both produced by (at least) the same entity E, i.e. the same author, journal, institution, or country. Different ways of defining these entities entail the use of different algorithms, which will be discussed in the next paragraph.

The computation of self-citations requires the definition of: (i) a publication window delimiting the cited items as the ones published in it. For the sake of simplicity, in what follows, the publication window is set at one year, hence the cited items considered are only the ones published in the year y; and (ii) a citation window delimiting the citing sources as the ones published in it. Let y be the reference year for calculation and also the publication window; T is the length of the observation period, expressed in years. Self-citations of an entity E in the year y can be defined as: (1) where s_E(y, y − 1 + i) is the number of self-citations received by the set of cited items published by the entity E in the publication window y, from citing sources produced by the same entity E in the citation window (y − 1 + i). The citation window includes the years (y − 1 + i) for i = k, …, T, where k ∈ {1, …, T} is chosen by the user for setting the desired time interval for observing citations. If k > 1 citation window and publication window are disjoint; for k = 1 and T > 1 they are partially overlapped, and for k = T = 1 they are completely overlapped. In the first case, citing sources are published in years following the publication windows; in the second case, the citation window includes the publication window and the following years; in the third one, citation window and publication window coincide. Analogously, the total citations received by the set of cited items published by the entity E in the year y can be defined as: (2) where c_E(y, y − 1 + i) is the number of citations received by the set of cited items published by E in the publication window y, from citing sources published in the citation window y − 1 + i.

The basic indicator is the self-citation rate (SR_E, y) for entity E in the year y, defined as the ratio between the self-citations and the total number of citations received by E [26, 28, 40, 41]: (3) where SR_E,y ∈ [0, 1]. C_E,y is usually interpreted as a proxy of the academic impact of the entity E. S_E,y is an indicator of academic impact generated by self-citations of E. Hence, a self-citation rate can be interpreted as the proportion of the academic impact of E due to its self-citation activity. It should be noted, that the denominator of the ratio is concretely produced by the citing choices of the whole scholarly community, and the ratio relates the self-citing choices of E with respect to the citing choices of the whole scholarly community.

Self-references indicate as well citations from citing sources to cited items that are both produced by (at least) the same entity, but the sets of citing sources and cited items are defined in different ways with respect to self-citations. Moreover, the computation of self-reference requires only a publication window for the set of citing sources. If the publication window is set also at one year, the cited sources are the publications produced by E in the year y, and cited items are (all or part of) the items listed in the bibliographies of the citing sources. SB_E,y, i.e. the number of self-references in the year y, is computed by summing up the citations from citing sources published in the year y by E to cited items previously produced by E. Analogously, R_E,y indicates the total number of citations given (i.e., references) by E in the year y to documents published before. The self-reference rate SBR_E,y for the year y is thus defined as: (4)

Note that a rigorous definition of this indicator would require the specification of a reference window, analogous to the citation window used in the self-citation rate. The studies based on the self-reference rate (e.g., [16]) usually assume a reference window that is extended to the oldest publication year of the references considered, but, in principle, other reference windows are possible.

When references are observed, the focus of the analysis is on the use of previous knowledge by the entity E, and not on the academic impact of the scientific production of E. It should be noted that the denominator of the ratio is, in this case, produced also by the citing choices of E. SBR_E should be interpreted as an indicator of how much E uses in its current work the knowledge that it had previously produced. For instance, for an author, a relatively high value of SBR_E,y may indicate that her/his work is largely based on her/his previous work. This in turn may be due to scarce attention to the work of other researchers in the field or to the fact that the author works in a little and specialized niche [39]. A relatively high value of SR_E,y, instead, indicates that a relatively high share of her/his academic impact is due to her/his self-citation activity.

Note that, in general, authors have more control over their self-reference rate than their self-citation rate, as they can manage more easily the citations they give (i.e., the denominator of the self-reference rate), than the citations that they receive (i.e., the denominator of the self-citation rate). In fact, authors interested in artificially inflating their citation metrics can effectively conceal strategic self-references within long reference lists and keep their self-reference rate low. Still, if they are not able to attract enough external citations to offset the impact of their strategic self-references, their self-citation rate will inevitably rise. This shows that the self-citation rate is more effective in revealing strategic self-citation behavior than the self-reference rate.

The basic indicator in the literature on country self-citations is the country self-citation rate. Hereafter N denotes the entity “country”; the self-citation rate of country N in the year y is the ratio between its country self-citations and the total number of citations received by that country [26, 28, 40, 41]: (5) where S_N,y is the raw number of self-citations of country N in the year y and C_N,y is the total number of citations received by N in the year y.

Computing the total of country self-citations for a country S_N,y is, however, less obvious than it may seem. As detailed in Methods and data, country self-citations can be in fact computed either narrowly or broadly, depending on how citations to international publications are considered. In [26], a variant of the self-citation rate indicator called “inwardness” is developed, characterized by a citation window variable in length. A broad definition of country self-citation is adopted, according to which a citation is considered a country self-citation when the intersection between the set of the countries of affiliation of the author(s) of the cited publication, and the set of the countries of affiliation of the author(s) of the citing publication, is not empty. This broad definition of country self-citation has the desirable property of ascribing to the world an inwardness value of inwardness 1, which makes therefore the inwardness an indicator normalized for the size of the country in terms of publications (see [26] for details). This property, however, does not hold for the SR_N,y when the country self-citations are computed narrowly (see Sec. Methods and data).

Inwardness is interpreted as a measure of the self-referentiality of a country: a higher level of inwardness suggests that the scientific publications produced by a country attract mainly the interest of the national community, whereas a lower level of inwardness suggests that the scientific production is cited mainly abroad. In the same sense, [42] related country self-citations to the degree of scientific insularity of nations. [42] also suggested several factors that may explain why developing countries show higher self-citation rates, among which a focus on applied scientific issues that respond to the perceived needs of national development, poor referencing practice, insufficient training of graduate students, preference for literature in national language than in English, and the proliferation of low-quality national journals. As [26] have shown, however, also developed countries may show anomalous raises in self-citation rates induced by research evaluation policies that reward raw citation metrics.

A variant of the self-citation rate is the citation domesticity indicator [43], in which citations coming from international publications are evenly distributed among the citing countries. Apart from [43], however, fractional counting of country self-citations has never been considered in the following literature. This may be explained by the fact that most more recent studies collect data on country self-citations from Elsevier’s SCImago or SciVal platforms, which do not apply fractional counting [44].

The complement of the self-citation rate is the foreign citation rate FR_N,y, also called “international scholarly impact of scientific research” by [45]: (6)

The country self-citation rate results to be positively correlated with the publication output of a country. In particular, [30] proposed a model where the country self-citation rate increases with the logarithm of the output: (7) where P_N,y is the number of publications of country N in the year y. This occurs because bigger countries have more domestic papers to cite and, hence, are more likely to attract citations from their own researchers than smaller countries [40, 41, 46]. By contrast, the average number of citations per document of a country is negatively correlated with self-citation rates [29]. Self-citation rates have increased over time: according to [29] estimates, the average self-citation rate of 62 countries raised of 28.9% from 1996 to 2009. [26] noted as well that the Inwardness of G10 countries increased during the period 2000–2016 with a mean increase of 5.2 percentage points.

To correct from the size-dependency of self-citation rates, [40] proposed to compare the self-citation rates with the world-share of publication of a country, based on the idea that if the publications from a country are cited as expected, then its share of country self-citations is proportional to its share of world publications. More recently, this indicator has been called “over-citation ratio” (OCR_N,y) by [27] and is defined as: (8) where P_w,y is the total number of publications in the world and α_N,y the proportion of publications of country N in the world. An over-citation ratio higher than 1 means that the country receives more citations from its own publications than expected based on its relative weight in the world scientific production. At the field level, [27] found that, as expected, over-citation ratio is higher for scientific fields of more national interest.

However, the over-citation ratio results to be size-dependent, as it is highly influenced by the denominator in the formula, i.e., the fraction of papers published by a country. For countries with high weight in the world publication output, such as the USA, the OCR_N,y will be always smaller than for small countries. For instance, for a country publishing one-third of the papers in the world (α_N,y = 33%), the OCR_N,y can never exceed a value of 1/0.33 ≈ 3, whereas a small country that published α_N,y = 3.3% of world papers, the maximum value of the OCR_N,y raises to 30 [24, 41]. In fact, [27] found that there is a negative power correlation between OCR_N,y and α_N,y: (9)

A further indicator, based on probability ratio, has thus been proposed, first in the study of language self-citations by [46] and then adapted to country self-citations by [41]. This indicator, called “odds-ratio” (O_N,y), relates two ratios: the numerator is the ratio of country self-citations to foreign citations, and the denominator is the ratio of domestic publication proportion to foreign publication proportion: (10)

The odds-ratio O_N,y measures to what extent the country relative preference to cite its own publications is greater or smaller than the existing ratio of its domestic publications to publications from other countries. Note that for small values of SR_N,y and α_N,y, the odds-ratio approaches the over-citation ratio.

The odds-ratio has three drawbacks as well, however [24]. First, if a country cites only its own publication, the measure is infinite. Second, it is oversensitive to small variations in the SR_N,y. Third, it is not normalized between 0 and 1. To fix these issues, [24] proposed the following indicator of relative self-citation rate: (11)

According to their interpretation, this formula considers the publication proportion α_N,y as a stimulus to the publication-citation system and the relative self-citation rate E_N,y as the subjective reaction of the system, which depends logarithmically on the intensity of the stimulus, as in Weber-Fechner equation. The function also expresses the law of diminishing returns: the larger α_N,y, the less important the changes in the relative self-citation rate. The relative self-citation rate has the advantages of being normalized and size-independent. Besides its mathematical merits, however, its meaning is less transparent compared to all the previous alternatives.

Methods and data

As anticipated, this work handles time series analysis of country self-citations: time-series clustering techniques are used for detecting countries whose self-citation behavior is similar [47, 48]. To build the time-series, a preliminary distinction between extensive (or broad) and restrictive (or narrow) country self-citations is introduced. The two counts are conceptually different and generate different estimates of country self-citations. Hence, two self-citation indicators are defined.

The relevant data from a citation database for the countries of interest are next retrieved and the time-series generated. The distance between the time-series is then calculated using a suitable distance measure and the structure of the distance matrix thus obtained is explored using multi-dimensional scaling (MDS). In the resulting MDS maps, countries characterized by similar trends will be placed close to each other, whereas countries with different trends far away [49].

Fig 1 sums up the phases of the present study. In the next paragraphs, the methodological and technical choices taken in each step are presented in detail and justified.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Phases of the analysis.

https://doi.org/10.1371/journal.pone.0294669.g001

Country self-citations of Type I and II

As said above, country self-citations can be computed in different ways depending on how self-citations to international publications, i.e. publications with authors from different countries, are considered. Analogously to author self-citations, the count of country self-citations for international publications can be done by adopting a publication-based or a author-based perspective [34].

The extensive publication-based perspective, adopted in the Inwardness indicator and implemented in the SciVal database, considers as country self-citations all citations coming from the collaborating countries. In the following, these extensively-intended country self-citations are referred to as country self-citation of Type I (SR^I), by omitting for simplicity the indexes of the publication window y and of the country N. The second way of counting country self-citations is author-based and more restrictive. Country self-citations of Type II (SR^II) are computed by considering only national author self-citations: a publication produced by (at least) an author of a given country receives a country self-citation of Type II if the citing publication is authored by one of the authors of the cited one, and this author is affiliated with the considered country.

The example in Fig 2 clarifies the computation of the two types. The Figure shows a citation network comprising publications labeled as 1, 2, …, 8, of which 3 are cited items and 5 are citing sources, and 8 citations indicated by the arrows linking the pairs of cited and the citing publications: (1, 4), (1, 5), …, (3, 8). The publications are authored by 5 authors labeled A, B, C, D, E from 3 countries: Italy (IT), Netherlands (NL) and Canada (CA). Inside each node, the authors of the relative publication and their countries of affiliation are reported by letters and acronyms. Three statistics for the two cited countries (Italy and Netherlands) are then calculated (Table 1): the total number of country citations, the total number of country self-citations Type I, and the total number of country self-citations Type II. For country citations, it is sufficient to count the citations landing on the publications produced by each country: the 3 Italian publications get 8 citations and the 1 Dutch publication 5 citations. For country self-citations Type I, we must compare for each citation the set of countries of the citing publications with that of the cited publications: when the intersection is not empty, the citation counts as a country self-citation Type I. Thus, Italy collects 7 country self-citations Type I, the Netherlands 4. Note, however, that the Dutch-Italian publication 1 receives only 1 citation from a publication with a Dutch author (publication 5), whereas the other three citing publications 6, 7, and 8 are in fact from Italian authors. This happens because country self-citations Type I of international publications include the citations coming from any of the collaborating country. In fact, a paper resulting from the collaboration of numerous countries is more likely to attract self-citations of Type I in comparison to a paper resulting from the collaboration of only a few countries. This is because there is a higher likelihood that the countries of the citing papers will overlap with the countries of the former rather than with the countries of the latter (see [26], sec. 3). In the extreme hypothetical scenario of a paper resulting from the collaboration of all countries in the world, any citation to this paper would be considered a self-citation of Type I, as the country of the citing papers will always overlap with at least one of the countries of the cited paper.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Toy citation network.

Nodes represent publications and arrows citation links. The authors of each publication, represented by capital letters, and their affiliation country, represented by acronyms, are shown inside each publication node.

https://doi.org/10.1371/journal.pone.0294669.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Classification of citations in Fig 2 based on the two definitions of country self-citation.

No fractional counting is applied.

https://doi.org/10.1371/journal.pone.0294669.t001

Country self-citations of Type II are introduced precisely to correct this somehow counter-intuitive property of country self-citations Type I. Country self-citations of Type II in fact include only the citations coming from the focal country. Then, if the focal country is the Netherlands, only the citation-link (1, 5) in the example will count as a country self-citation Type II for the Netherlands because it links two publications sharing the same Dutch author B. Citations (1, 6) and (1, 8), by contrast, will not count because they are generated by the Italian co-author of B. Symmetrically, citation (1, 5) will not count for Italy, whereas (1, 6) and (1, 8) will. Table 1 shows analytically in which relationship each citation-link stands with the two countries and reports the three statistics for each country. Note that no fractional counting was applied: citations from multi-author or international publications or landing on multi-author or international publications are not divided among the cited or citing countries.

Based on the previous definitions, the difference between the country self-citations of Type I and Type II of a country should be attributed mainly to the publications the country produces with other countries, i.e., to its international publications. A wider difference means that these international collaborations are widely cited not only by the focal country but also by the collaborating countries, increasing Type I but not Type II self-citations. A shorter difference, by contrast, means that the country has few international collaborations and/or that they do not attract citations from the collaborating countries. The difference between Type I and Type II is therefore related to the international citation impact of a country.

Part of the difference between the country self-citations of Type I and Type II of a country can be attributed also to citations generated by country publications that are not also author self-citations. In particular, the country self-citations of Type II do not capture citation exchanges between groups of national authors who cite each other but do not collaborate directly on writing papers.

Data

Scopus data were used for the computation of the indicators and data were provided by Elsevier through its ICSR Lab. The identification of author self-citations used in the calculation of Type II country self-citations relies on the Scopus author profiling procedure described in [52].

Countries with at least 100,000 publications indexed in the Scopus databases in the period 1996–2019 were considered (n = 50). The indicators were computed for each country on a yearly basis from 1996 to 2019 using, as said above, a 2-year citation window. Publications from all Scopus fields were aggregated, i.e., for each country, the entire scientific output was considered, with no distinction of the research area. Thus, for each indicator, 50 time-series (1 per country) with 24 observations each (1 per year) were computed. In the Supporting Information, the trends of the over-citation ratio, odds-ratio, and relative self-citation rate are available as well. All were calculated using both Types of country self-citations.

Tables 2 and 3 provides the descriptive statistics of the countries in the dataset.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Descriptive statistics 1996–2019.

The overline indicates the mean. Publications include only research articles, reviews, and conference papers. Citations () and self-citations ( and ) are computed on a 2-year citation window. Citing sources include all types of documents, cited publications include research articles, reviews, and conference papers.

https://doi.org/10.1371/journal.pone.0294669.t002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Average country self-citations of type I and type II and their average difference.

All values are multiplied by 100.

https://doi.org/10.1371/journal.pone.0294669.t003

Results

Fig 3 shows the trends of the 50 countries for the two indicators SR^I and SR^II. Three main observations can be made on these trends. First, the proportion of country self-citations of both types has decreased over time in most countries, following a linear pattern with more or less pronounced oscillations depending on the country. Three countries, however, deviate from this general behavior: in the case of Indonesia, Ukraine, and the Russian Federation, in fact, the trends of both indicators show an inversion from descending to ascending.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Country self-citation rate SR^I and SR^II by country.

Yearly data 1996–2019.

https://doi.org/10.1371/journal.pone.0294669.g003

Second, the SR^I and SR^II trends are highly correlated for most of the countries, with both indicators following similar trajectories over time. Again, there is a notable exception represented by China, which is the only country where the share of country self-citations of Type I has surged (China SR^I has increased by 24.1 p.p. between 1996 and 2018) while that of self-citations of Type II has substantially contracted (SR^II has decreased by 22 p.p. in the same period).

Third, the difference between SR^I and SR^II varies over time differently depending on the country (Fig 4 and Table 2).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Simple difference between SR^I and SR^II over time by country.

The solid red line represents yearly data (1996–2019), and the dotted black line is the average difference in the whole period.

https://doi.org/10.1371/journal.pone.0294669.g004

In particular, we can distinguish three groups of countries based on how the difference develops over time. The first group is characterized by an increasing difference, with the two curves of SR^I and SR^II progressively diverging. It includes Brazil, Chile, China, Colombia, Egypt, Hungary, India, Indonesia, Iran, Malaysia, Pakistan, Romania, Russian Federation, Slovakia, Turkey, and Ukraine. The second group shows a stable difference, meaning that the two curves follow parallel directions. It includes Argentina, Australia, Austria, Belgium, Canada, Croatia, Czech Republic, Denmark, Finland, France, Germany, Greece, Hong Kong, Ireland, Israel, Italy, Japan, Mexico, Netherlands, New Zealand, Norway, Poland, Portugal, Saudi Arabia, Singapore, South Africa, South Korea, Spain, Sweden, Switzerland, Taiwan, Thailand, and the United Kingdom. The last group includes only one country, the United States, where the difference reduces over time, i.e., the curves tend to converge. Note that all G10 countries, apart from the United States, belong to the second group and are all characterized by a significant difference between SR^I and SR^II, with the United States showing the highest difference of all countries considered (mean difference = 34.4).

Both indicators result to be highly correlated with the logarithm of the publication output for most of the countries (Fig 5). Notably, the relationship between the self-citation rate (of both Types) and scientific production is of inverse proportionality: across most of the countries observed, an increase in scientific output is associated with a decrease in the self-citation rate. This contrasts with the direct proportionality found in prior studies [30, 40, 41, 46]. Interestingly, however, a few countries deviate from this general pattern, exhibiting either positive correlation (China for SR^I, Indonesia on both SR^I and SR^II) or mild to low correlation coefficients (Colombia, Egypt, India, Iran, Italy, Malaysia, Pakistan, Romania, Russian Federation, Saudi Arabia, Thailand, Ukraine). The self-citation rate trend within this latter group of anomalous countries cannot be effectively predicted based solely on the trend of scientific production. (The scientific output trends of the 50 countries are reported in the Supporting Information).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. SR^I and SR^II versus publication output by country.

Each data point represents data for a year between 1996 and 2019. Pearson correlation coefficients (by country and indicator) are overlaid.

https://doi.org/10.1371/journal.pone.0294669.g005

To better investigate differences between groups of countries, the correlation-based distance was used to produce two matrices M^I and M^II of order 50 × 50, which represent the distances between the 50 countries’ trends respectively on the SR^I and SR^II.

The structures of the two matrices were visualized using Kruskal’s Non-metric Multidimensional Scaling in the two MDS maps in Figs 6 and 7, based respectively on SR^I and SR^II trends. As explained in Methods and data, the distance between the dots representing the countries on the MDS maps is inversely proportional to the similarity of their SR trends, so that countries characterized by similar trends will be placed closer and countries characterized by dissimilar trends far away. Note that, in both maps, the distances of the points on the 2-D map distort the original distances between the time-series only slightly, as shown by the low values of the stress of the MDS solutions (respectively, 9.18% and 7.45%). Note that the countries in the two latter zones are identical to those showing anomalous correlations with trends in scientific output.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. MDS solution for SR^I.

G10 countries are in bold. Stress = 9.18%.

https://doi.org/10.1371/journal.pone.0294669.g006

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. MDS solution for SR^II.

G10 countries are in bold. Stress = 7.45%. Note that the points of Malaysia and Indonesia are outside the plot.

https://doi.org/10.1371/journal.pone.0294669.g007

In the MDS map for SR^I (Fig 6), three zones can be distinguished. The first is the big cluster in the left area, where most of the countries are concentrated. The second is the belt that surrounds the cluster and includes Iran, Romania, Pakistan, Thailand, Colombia, Saudi Arabia, Ukraine, Egypt, India, and, most notably, Italy, the only G10 country that is not placed inside the big cluster. The third zone comprises the rest of the map, where countries characterized by very specific trends are scattered: Malaysia, China, Indonesia, and the Russian Federation.

The MDS map for SR^II (Fig 7) largely confirms this picture, showing significant overlap with the structural features of the previous map. Again, we find a big cluster including most of the countries, surrounded by a belt and, in the distant zones of the map, a scattering of anomalous countries. The belt includes the same countries as the belt in the previous map: Italy’s anomalous position with respect to G10 countries is confirmed. The most important difference between the two maps is China: under the profile of the SR^I trend, the country was placed in the distant zone, whereas under the profile of the SR^II trend, it is placed within the big cluster.

Discussion

The decreasing trend in both SR^I and SR^II which characterizes most of the countries shows that, in most of the cases, the overall citation impact of countries has grown more than the proportion of citation impact generated by domestic authors, i.e., that the denominator of both indicators has increased more than their numerator (see Section Indicator design). The faster increase in citations may be related in turn to the overall growth of scientific production and how it impacts the length of the reference lists of scientific publications. According to [57], in fact, the world’s scientific production exhibits 4% annual growth in publications and 1.8% annual growth in the number of references per publication. Combined, these dynamics produce a 12-year doubling period in the total amount of references, which results in turn in a generalized increase in citations [58]. The decreasing trends, thus, may be simply due to the different rates of growth of the numerator and the denominator of the indicators used here: country self-citations of Type I or Type II grow less than bibliographic references.

The observed decreasing trend in the SR^I, however, contradicts previous studies of the development of this indicator over time: [26], in fact, report an average increase of +5.2 p.p. in the SR^I of G10 countries between 2000 and 2016. Namely, the discrepancy between the present results and [26] depends on the different way of computing the SR^I indicator. Indeed, [26] used a non-fixed citation window, which included all the years from the publication year to 2016. For example, for the cited items published in the year 2000, a 17-year citation window was used, by summing up all citations from 2000 to 2016; whereas, for the year 2006, the citation window was 11 years long, including citations from 2006 to 2016; for 2016, the citation window included only 1 year, i.e., only citations from 2016 itself were counted. Since self-citations are in general younger than external citations [25, 59], they tend to represent a higher proportion of total citations for the years when the citation window is shorter. Hence, [26] registered inflation of SR^I for more recent years, as the citation window shortens. The present study, by contrast, does not suffer from this problem as it is based on a fixed citation window, i.e., only a fixed number of years after the target year is considered (see Section Indicator design).

Turning now to a more substantial interpretation of the results, there are two patterns that are worth highlighting. First, the emerging giant of science, China, is characterized by a unique behavior: China SR^I and SR^II show almost opposite trends, with the former significantly increasing and the latter significantly decreasing. If the increasing SR^I trend shows that Chinese scientists heavily rely on the scientific production of their own country, the decreasing SR^II trend indicates that author self-citations are diminishing, in line with Western countries. Interestingly, this divergence results in a growing difference over time between the two indicators (Fig 4), which can be interpreted as a sign of the rising international impact of Chinese publications. As noted above, in fact, the difference between SR^I and SR^II depends on the citations of the international publications of a country. A wider difference means that these international collaborations are widely cited not only by the focal country (China, in our case) but also by the collaborating countries. Results, therefore, seem to show that international collaborations of Chinese authors are increasingly cited by other countries as well, another sign of the new status of China as a scientific superpower [60]. Notably, India shows an increasing difference as well, which may be interpreted analogously as a sign of the rising scientific impact of this country.

The second key pattern, emerging from the MDS maps, is that there are several countries whose self-citation behavior stands out from that of the majority of countries. With a couple of exceptions (China and India), these countries are the same when the two indicators are considered: Colombia, Egypt, Indonesia, Iran, Italy, Malaysia, Pakistan, Romania, Russian Federation, Saudi Arabia, Thailand, and Ukraine. Note that the anomalous self-citation rates of these countries cannot be explained by the trend of their scientific production, as evidenced by the low correlation coefficients found (Fig 5). This suggests searching for specific perturbations of the citation behavior.

Interestingly, all these anomalous countries have adopted, in the recent past, specific research policies aiming at increasing publication output and citation impact of their national scientific community. In the following sections, the recent history of the research policies in each group of anomalous countries is reconstructed. It will be shown that all these policies are characterized by direct or indirect incentives that may create room for the strategic use of self-citations. SR^I and SR^II, therefore, seem to be sensitive to policy-induced perturbations of the citation habits.

Italy

Italy is among the top 10 producers of science in terms of global production and total citations (Table 2) and it has been considered the main European competitor of the United Kingdom for citation impact [133, 134]. However, data shows that Italy is the only G10 country exhibiting an anomalous trend of self-citations. The Italian anomaly is especially visible when compared with the trends of the other G10 countries: from 2010, Italy starts to diverge from the benchmark countries in both indicators (Fig 8). At the end of the observation period, it results to be the country with the highest SR^II, above Japan.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. G10 countries’ trends in SR^I and SR^II.

Italian trends are in bold green.

https://doi.org/10.1371/journal.pone.0294669.g008

The beginning of the change in the trend coincides with the wide process of reformation of the Italian university system started by the government in 2010 (Law 240/2010). A governmental agency (ANVUR) was established in charge of monitoring and evaluating the Italian research system and, in 2011, the first national research assessment exercise started, followed by a second round in 2015. In both, the evaluation was largely based on the automatic or semi-automatic use of algorithms fed by citation indicators [135, 136]. Universities started to be funded according to their performance in the research assessments. Moreover, the reform modified also the recruitment and advancement system for university professors by introducing a national scientific qualification (ASN). This qualification is mandatory both for hiring and promotion and, in order to obtain it, candidates in natural sciences, life sciences, and engineering must exceed publication and citation thresholds centrally defined by ANVUR [26]. As a result of these reforms, bibliometric performance has acquired a central role in the career of Italian scientists [137]. At the same time, anomalous rises in Italian self-citations have been documented by several studies [26, 34–38].

Conclusions

Since the early times of citation indexes, preoccupations with the opportunistic use of citations have been voiced by bibliometricians and the scientific community [138, 139]. The centrality acquired by metrics in the various ganglia of the research system, from the career of individual scientists to the ranking of institutions until the evaluation of the scientific performance of entire countries, has further sharpened these concerns [6, 140–143]. Self-citation, in particular, has persistently been indicated as among the easiest strategies available to scientists for artificially boosting their citation-related performance indicators [6, 144], raising the question of whether scientists, under pressure, do indeed attempt to game citation scores by increasing self-citation.

In the present study, we investigated how the propensity to self-citation changed in 50 countries all over the world from 1996 to 2019, using two different indicators based on country self-citations. The results show that, for most countries, self-citation rates have decreased over time following similar patterns. Tendency to self-citation, both at the country level and for individual scientists, seems in fact to be declining. Against this background, however, there are some countries that exhibit anomalous behavior, showing self-citation trends that are significantly different from those of “standard” countries.

The analysis of the research policies adopted by these anomalous countries in the last years has revealed that they all share a common trait, namely the introduction of direct or indirect rewards for the bibliometric performance of scientists. The temporal association we found in all anomalous countries between changes in policies on the one hand and changes in the self-citation behavior of the national scientific community on the other hand suggests that scientists do indeed respond to the new climate of incentives by altering, among other things, their citation habits. Policy pressure seems therefore capable of affecting rapidly and visibly the citation behavior of entire countries, possibly distorting global rankings of countries based on citations [31].

Clearly, we cannot offer a full-fledged causal explanation of our data, displaying the causal chains that start from the policy and end with the citation choices of individual authors. Neither we can demonstrate that the whole self-citation gain of anomalous countries is due to opportunistic adaptation to research policies. Still, the most likely high-level explanation of the macro-trends we observe is that the policies centered on or emphasizing citation performance do significantly affect the behavior of scientists.

Remarkably, our results show that merely having a performance-based funding system involving citation-based indicators is insufficient for a country to exhibit anomalous self-citation behavior. For example, countries like Belgium, Bulgaria, Croatia, and Sweden employ citation-based indicators in evaluating their university systems, yet they do not display anomalous self-citation tendencies [145]. It appears that the crucial factor influencing this anomalous self-citation behavior is the proximity of incentives based on citations to an individual researcher’s career and wage. It seems that the closer these incentives are to affecting researchers’ career progression and wages, the more likely they are to influence citation behavior. In Italy, for instance, the national scientific qualification, which is awarded based on citation indicators, has a direct impact on the career of researchers, as without it researchers cannot access tenure-track positions or advance to full professorships. By contrast, in countries where citation-based indicators contribute to complex funding formulas at the university level, the influence on citation behavior is more distant from individual researchers and, consequently, less significant.

In future research, it will be important to further investigate how varying architectures of performance-based funding systems have more or less impact on citation behavior, exploring in particular the extent to which the proximity of citation-based incentives to individual careers and wages induces strategic behavior. Other promising topics for future research include examining whether research fields characterized by varying epistemological, social, and institutional features respond differently to science policies and investigating whether citation incentives may affect countries’ mix of scientific output, inducing specialization in areas rewarded by citation metrics (e.g., the medical sciences) at the expense of less “performing” areas (e.g., the humanities).

On its part, this study contributes to the ongoing discussion on research evaluation systems by showing that when bibliometric indicators, and citation-based indicators in particular, are integrated into systems of incentives, they cease to be neutral measures to become active components in the research system. As such, they are able to modify the behavior of entire scientific communities. Hence, they should be handled by science policy makers with the most extreme caution.

Supporting information

Acknowledgments

This work uses Scopus data provided by Elsevier through the ICSR Lab. We are very grateful to Lucio Barabesi, Jeroen Bass for his help in retrieving the data, and Giuseppe De Nicolao for insightful discussions in the early stage of the research.