Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Open Access

Peer-reviewed

Research Article

Comparison of journal and top publisher self-citation rates in COVID-19 research

Abstract

Introduction

Journal self-citation contributes to the overall citation count of a journal and to some metrics like the impact factor. However, little is known about the extent of journal self-citations in COVID-19 research. This study aimed to determine the journal self-citations in COVID-19 research and to compare them according to the type of publication and publisher.

Methods

Data in COVID-19 research extracted from the Web of Science Core Collection 2020–2023 was collected and further analyzed with InCites. The journals with the highest self-citation rates and self-citation per publication were identified. Statistical comparisons were made according to the type of publication and publishers, as well as with other major infectious diseases.

Results

The median self-citation rate was 4.0% (IQR 0–11.7%), and the median journal self-citation rate was 5.9% (IQR 0–12.5%). 1,859 journals (13% of total coverage) had self-citation rates at or above 20%, meaning that more than one in five references are journal self-citations. There was a positive and statistically significant correlation of self-citations with the other indicators, including number of publications, citations, and self-citations per publication (p<0.001). Editorial materials contributed more to journal SC with a median self-citation rate of 5%, which was statistically higher than other documents such as articles, letters or reviews (p<0.001). Among the top twelve publishers, the Multidisciplinary Digital Publishing Institute had a median self-citation rate of 8.33% and was statistically higher than the rest (p<0.001). Self-citation rates for COVID-19 were lower than tuberculosis and HIV/AIDS, but self-citations per publication of these diseases were statistically lower than those for COVID-19 (p<0.001).

Conclusion

Some journals from the Web of Science Core Collection displayed exorbitant journal self-citation patterns during the period 2020–2023. Approximately, one in every five paper references in COVID-19 is a journal self-citation. Types of publication such as editorials engage in this practice more frequently than others, suggesting that in COVID-19 research, self-citing non-citable items could potentially contribute to inflate journal impact factors during the pandemic.

Citation: Quincho-Lopez A (2024) Comparison of journal and top publisher self-citation rates in COVID-19 research. PLoS ONE 19(12): e0314976. https://doi.org/10.1371/journal.pone.0314976

Editor: Tauseef Ahmad, Southeast University, CHINA

Received: March 19, 2024; Accepted: November 20, 2024; Published: December 5, 2024

Copyright: © 2024 Alvaro Quincho-Lopez. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the manuscript and its Supporting Information files.

Funding: The author(s) received no specific funding for this work.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Faced with the threat and uncertainty generated by the Coronavirus Disease 2019 (COVID-19), the scientific community responded quickly by generating new publications at an unprecedented level. Unsurprisingly, those publications have been cited by multiple other papers at incredibly high rates while the pandemic continues to be in the spotlight [1]. In fact, a fifth of the citations received to papers published in 2020 to 2021 were to COVID-19–related papers [2]. Subsequently, the citations received by said articles would play a very important role in the next metrics of different scoring systems to quantify journal’s impact [1].

Each year editors of journals enthusiastically await the release of journals’ metrics like the impact factor (IF) by the Journal Citation Reports (JCR). Some studies have revealed that journals that published more COVID-19-related articles experienced higher IF growth [3, 4]. Although, in all disciplines, journals experienced a higher IF increase during the period 2020–21 than previous years, this was much more noticeable in those journals with lower IF [4, 5]. Indeed, the massive covidization of research citations has distorted the metrics of scientific journals. However, the demanding race to publish was supported by both the authors who had their manuscripts published in a short time, as well as the journals and the editors who were eager to boost their metrics [2, 6].

Journal self-citations (SCs) occurs when a journal is cited by its own publications. Although SCs can occur regularly based on the relevance of a reference according to the author’s own criteria or in the belief that this would improve the probability of successfully passing the journal’s peer-review, there are also cases of coercive SC made by the editorial staff during the peer review process usually after a paper has been conditionally accepted for publication in the journal [7, 8]. Previous research raised concerns about the role of “forced” SC in the annual scientometric indicators of the journals or their impact on the position of the journal quartiles [8–11]. This takes on greater relevance in the context of COVID-19, where the race to publish eventually compromised the quality of articles and regular processes under express peer review [12, 13]. Previously, some studies have focused on how COVID-19 has impacted the IF due to the volume of papers and citations without considering self-citations as a mechanism to inflate metrics at the journal level [4, 5], or studies that have characterized the SCs of journals belonging to the JCR in general terms [14, 15].

Several studies have explored the journal SCs in different research areas. Although they have some similarities such as data sources, most frequently the JCR, they also have important differences in the methodology such as the sample size and sample selection. For example, there are some studies that selected for convenience certain specific journals [16–22], while others selected all journals under a JCR category as well as others that restricted it only to journals with an IF [23–26]. There are studies that analyzed results in a single year while others covered several periods [16, 19–22, 27–32]. The summary measure was also heterogeneous between studies. The most frequent was the mean SC rate, such as in the case of sports sciences and ecological sciences, of 6.7% and 11.8%, respectively [27, 28]; while in others a range of values was shown such as emergency medicine (4.8 to 8.1%), plastic and reconstructive surgery (1 to 4.5%), otolaryngology (4 to 11.9%) and anesthesia (4 to 30%) [17, 19, 21, 22]. In other areas such as soil sciences (10%), pediatrics (9%), dermatology (10.5%) and ophthalmology (11.6%) the median SC rate was reported [18, 33–35]. On the other hand, some articles focused on the impact of the journal SC on the IF, for example Sanfilippo et al. and the so-called "journal self-citation (JSC) rate", whose calculation is detailed as the IF including SC minus IF without SC divided by IF including SC, reported similar values in areas such as anesthesiology (4.8%), respiratory system (5.5%), general surgery (6.5%) and critical care (8.8%) [23–26]. Similarly, Opthof found that the % inflation IF by journal SC was 17% in a sample of the top 50 journals under the WoS category ‘Cardiac and Cardiovascular Systems’, even being higher than 33% in 8 journals [36]. Other relevant results can be found in Table 1.

thumbnail
Download:
Table 1. Brief review of the key literature.

https://doi.org/10.1371/journal.pone.0314976.t001

However, up to now, there are no studies that focus on journal SCs in COVID-19 research, despite the great impact it has had on science and the scientific community, considering its short-term evolutionary nature. Therefore, this study aimed to determine the journal self-citation rates (SCRs) for COVID-19 research in the Web of Science Core Collection (WoSCC), and to identify the journals with the highest SCRs and SC per publication. Additionally, it sought to analyze how SCs rates vary across different types of publications and publishers, and to compare these rates with those in other major infectious disease research. Concurrently, the study evaluated the correlation between SCs and other bibliometric indicators.

Methods

The WoSCC was used to retrieve all relevant documents in the present study on February 24th, 2024. The lack of self-citation analysis tools and lower availability of bibliometric indicators were the main reasons for not using PubMed or Scopus, respectively [37, 38]. Furthermore, there is a large overlap of journals between these databases, which would generate a large number of duplicates. Additionally, there are specific journal impact indicators for each database, therefore journals from one source cannot be judged using metrics from other sources [37, 39]. Different search terms were extracted from the Medical Subject Headings from PubMed and Emtree from Embase, and from a combination of both a complex search strategy was generated. Because only studies relevant to COVID-19 were sought to be recovered, the search was restricted to the title and keywords, but not the abstract. Then, the results were refined using the available WoSCC filters: (1) document type: exclude all types except for article, letter, review article, editorial material and early access; (2) year of publication: only results from 2020 to 2023 were included. The detailed search strategy can be found in the supporting information (S1 File).

The information collected from WoSCC was exported to InCites (from Clarivate) for further analysis. In this study, SCs were defined as instances where COVID-19 papers cite any previous work from the same journal, regardless of whether those cited papers are about COVID-19 or other topics. Three datasets were analyzed with information regarding COVID-19 (see S2 File): (i) the journal dataset with all journals publishing in COVID-19; (ii) the type of publications dataset; and (iii) the publisher dataset with the top twelve publishers with the highest number of publications on the topic. For the first dataset, the following information was collected from each journal: number of publications, citations, citations without SC, journal IF, and IF without SC, % documents cited, the Category Normalized Citation Impact (CNCI), the Journal Normalized Citation Impact (JNCI), immediacy index, % international collaboration, and the percentage of documents in the top 1% and 10% of highly cited papers in the WoSCC. SC were obtained indirectly by subtracting citations without SC from total citations. The SCR represents the proportion of a journal’s SCs to the total citations received multiplied by 100. The same reasoning was followed to calculate the JSC rate, which is detailed in another publication [26], but instead uses the IF with and without journal SC. Additionally, for a small sample of those journals that have more than 600 publications in COVID-19 papers, the number of categories assigned by the JCR and whether they were fully open access (OA) (non-hybrid nor subscription) were manually collected. The Mann–Whitney U-test for unrelated samples was performed separating journals according to their fully OA status and number of categories (1 vs. >1) to find differences according to the SCR and JSC rate.

With the same dataset, a correlation matrix was calculated and graphed based on the Spearman’s rank correlation coefficient to correlate the journal SC as an absolute value and in percentage with the previously mentioned indicators. Also, the same dataset of journals was filtered (> 100 publications and > 100 citations) to obtain those journals with the highest SCR and SC per publication values, and graph them as lollipop charts. The main journals were categorized into quartiles according to the JCR. When there were several quartiles according to the edition and/or categories, the best quartile was chosen. When a JCR quartile was unavailable, the Journal Citation Indicator (JCI) quartiles were used, since both metrics have a high correlation [40].

With the publisher dataset, violin plots were used to compare the SCR between the top 12 publishers with the highest number of publications in COVID-19. For this purpose, the ggstatsplot library was used [41], which allows comparing the medians using the Kruskal-Wallis test, also indicating the adjusted p-value after Holm’s correction as a post hoc test to conduct pairwise comparisons and identify specific group differences. A similar procedure was followed for the type of publication dataset comparing the SCR among articles, reviews, letters, and editorials. Because only up to 150,000 documents could be exported to InCites, the number of documents under the article category was sorted in decreasing order of citations while the content of the other publication types could be completely exported.

To compare the number of SCs for COVID-19 journals, I performed searches for other major infectious diseases such as HIV/AIDS and tuberculosis in the WoSCC (see S3 File). To ensure comparability, the same restrictions were used as for the COVID-19 search strategy, considering the type of publications and years (2020–2023). Nonetheless, in order to avoid any impact of the COVID-19 pandemic on publication dynamics, it was also analyzed in the period 2016–2023. Also, the Kruskal-Wallis test was applied between SC/publication of COVID-19 with the other diseases in both periods 2020–2023 and 2016–2023. All data analyses were conducted on RStudio software version 4.3.0 (Boston, MA, USA). A p-value less than 0.05 was considered statistically significant for all analyses. A flow chart of the search is shown in Fig 1.

thumbnail
Download:
Fig 1. Flow chart of the data retrieval, selection and analysis.

https://doi.org/10.1371/journal.pone.0314976.g001

Results

From more than 377,000 documents retrieved in the WoSCC, information of a total 14,690 journals was retrieved. Table 2 corresponds to the most productive journals that have more than 600 publications in the topic. In total, these 45 journals had 69,270 publications in COVID-19, obtaining 1,442,563 citations, of which 77,368 (5.36%) were SCs. Although it could be expected that the most productive journals would be those with the greatest number of citations, this was not the case, with the most cited within Table 2 being very high impact journals such as the NEJM, The Lancet and JAMA, all of which exceeded 100,000 citations. The largest number of SCs were for International Journal of Environmental Research and Public Health—IJERPH (MDPI), Vaccines (MDPI) and Science of the Total Environment (Elsevier). Journals that had a high SCR (>20%) were Sustainability (MDPI), Vaccines (MDPI), and Journal of Family Medicine and Primary Care (Wolters Kluwer). Science of the Total Environment (Elsevier) was the journal that had the highest number of SCs per publication (6 SCs), followed by other two journals, one from MDPI and Taylor & Francis (3 SCs, each). When calculating the JSC rate, three MDPI journals lead the indicator, namely Sustainability, Vaccines and Nutrients. No differences were found between the medians of fully OA journals (6.7% [3.6–9.1%]) and hybrid/subscription-based journals (3.9% [1.7–6.6%], p = 0.17), with respect to SCR. Neither for the JSC rate: 6.1% (3.4–8.5%) vs. 3.2% (1.6–7%), respectively (p = 0.14). Nonetheless, journals belonging to 2 or more JCR categories have higher SCR (7.7% [6.1–10.1%] vs. 4.7% [3.2–7.7%]; p = 0.04) and JSC rates (7.4% [5.4–8.8%] vs. 4.4 [2.4–8.3%]; p = 0.04) than those who belong to a specific category. For the full table, see the S2 File.

thumbnail
Download:
Table 2. Most productive journals with at least 600 COVID-19 publications indexed in the Web of Science 2020–2023.

https://doi.org/10.1371/journal.pone.0314976.t002

Based on the full dataset, the median SCR was 4.0% (IQR 0–11.7%), and the median JSC rate was 5.9% (IQR 0–12.5%). 10,822 journals (74% of total coverage) had SCR below 20%, whereas 1,859 journals (13% of total coverage) had SCR at or above 20%, meaning that more than one in five references are journal SC. The Spearman correlation matrix (Fig 2) shows that there was a positive and statistically significant correlation of SC with the other indicators, which means that when the SC increased, these indicators also increased (p<0.001). However, the degree of correlation varied from weak (% documents cited, % international collaboration) to moderate (IF, CNCI, JNCI, % documents in the top 1% and 10% of highly cited papers) in some cases, and even strong in others (publications, citations, and SC per publication).

thumbnail
Download:
Fig 2. Correlation matrix of journal self-citations in COVID-19 research with other indicators.

Note. SC, self-citation; Pub, publication; Cit, citations; SCPub, self-citation per publication; Pcited, % documents cited; IF, impact factor; CNCI, Category Normalized Citation Impact; JNCI, Journal Normalized Citation Impact; Top1, % documents in the top 1%; Top10, % documents in the top 10% of highly cited papers.

https://doi.org/10.1371/journal.pone.0314976.g002

To avoid exaggerated values of SCR and SC per publication in journals with few citations and/or publications, the journal dataset was filtered to only journals with at least 100 publications and citations, leaving 645 journals for analysis. Fig 3A shows the journals with the highest SCR, led by Bali Medical Journal and Orvosi Hetilap with more than 50%, followed by two others with >40%, five with >30% and eighteen with >21%. Also, Fig 3B shows that International Journal of Contemporary Hospitality Management had the highest number of SCs per publication (8 SC for each publication), followed by Science of Total Environment with 6 SC/publication.

thumbnail
Download:
Fig 3.

Lollipop charts of journals with the highest self-citation rates (A) and self-citation per publication (B) in COVID-19 research.

https://doi.org/10.1371/journal.pone.0314976.g003

When the publication type dataset was analyzed, it was observed that the editorial materials contributed more to journal SC with a median SCR of 5%, which was statistically higher than other documents such as articles, letters or reviews (p<0.001) (see Fig 4).

thumbnail
Download:
Fig 4. Violin plots of journal self-citation rates in COVID-19 research according to the type of publication.

https://doi.org/10.1371/journal.pone.0314976.g004

Fig 5 corresponds to the 12 most productive publishers in COVID-19 research distributed according to the level of journal SC (%). Fig 5A shows the 6 most productive publishers, led by Elsevier (1,648 journals) followed by Springer Nature (1,540 journals). However, MDPI was the one with the highest median SCR with 8.33% and was statistically higher than the other five publishers (p<0.001). Taylor & Francis also stands out in this group for its SCR (4.76%). Fig 5B shows the following most productive publishers, where SAGE stands out for its SCR (4.76%). Nature (0.97%) is the publisher with the lowest SCR, whose results were statistically lower than the other publishers, although no significant statistical differences were found when compared with PLOS (1.85%).

thumbnail
Download:
Fig 5.

Violin plots of journal self-citation rates in COVID-19 research according to the most productive publishers: ranking 1st-6th (A) and ranking 7th-12th (B).

https://doi.org/10.1371/journal.pone.0314976.g005

As shown in Table 3, the number of publications, journals and especially citations relevant to COVID-19 research exceeds that of other infectious diseases even in the last 8 years. As expected, SCs in COVID-19 were also much higher than the SCs generated in tuberculosis and HIV/AIDS research. When comparing the SCR, tuberculosis and especially HIV/AIDS research take the lead. By adjusting the number of SCs per publication, it was obtained that the result for COVID-19 was higher than other infectious diseases in the same timespan (2020–2023). Even when increasing the timespan to the last 8 years (2016–2023), COVID-19 SC per publication still remained higher than the others. In both periods, there was a significant statistical difference (p<0.001) when comparing journal SC per publication with those of COVID-19.

thumbnail
Download:
Table 3. Comparison of journal self-citations in COVID-19 with other infectious diseases.

https://doi.org/10.1371/journal.pone.0314976.t003

Discussion

This exploratory study of journal SC in COVID-19 research indicated that 1,859 journals (13% of total coverage) had SCRs at or above 20%, meaning that more than one in five references are journal SCs and could be considered as journals with "high self-citation rates", the latter as previously suggested by Clarivate Analytics [42]. Also, the median SCR was 4.0% and the median JSC rate was 5.9%. The SCR represents what percentage of the total citations of a given journal are self-citations, while the JSC rate indicates the percentage change in a journal’s IF when self-citations are not included compared to its baseline IF with self-citations. Those values were lower than those reported in other areas such as dentistry, ophthalmology, dermatology (median SCR of 13.7, 11.7 and 10.5%, respectively), critical care medicine, anesthesiology, and general surgery (median JSC rate of 8.8, 8.4 and 6.5%, respectively) [23, 24, 26, 31, 34, 35]. Given that the present study included information from more than 14,000 journals, these discrepancies could be explained by the different sample sizes (< 100 journals) of these studies or their interest in analyzing very specific research areas.

Journals that belong to 2 or more JCR categories have higher SCR (7.7% vs. 4.7%) and JSC rates (7.4% vs. 4.4) than those that belong to a specific category, although this was not the case for fully open access journals. These particular findings could not be extrapolated to all COVID-19 journals, but only to the most productive journals. However, COVID-19 research embodied the urgency of contemporary global challenges, demanding dissemination avenues that match its gravity. On that note, mega-journals, characterized by their broad scope, interdisciplinary nature, and commitment to open access, serve as ideal platforms for the dissemination of such research [43]. Their broad publishing scope with acceptance rates ranging from 20% to 70%, and usually rapid review process make them attractive to the vast number of authors in COVID-19, although some of these journals also maintain relatively high SCRs [44]. It is curious to highlight the case of IJERPH, which has had meteoric growth since 2020 regarding the number of papers, accounting for around 15% of the papers in SCImago Journal & Country Rank Public Health, Environmental and Occupational Health category in 2021 [45]. However, since March of 2023 it was delisted from Clarivate and did not receive the 2022 IF due to the fact that some papers were declared outside the scope of the journal [46], making it impossible to calculate the JSC rate of the most productive journal in COVID-19 after four years of the start of the pandemic. For the same reason, its true impact in 2023 could not be fully calculated.

Journals with higher IF that have published in COVID-19 tend to exhibit higher SCs. This finding is consistent with results reported by Miyamoto in the field of plastic and reconstructive surgery [19]. However, several other studies have reported contrary findings, with some observing an inverse correlation in areas like pediatrics, dermatology, and ecological sciences [27, 33, 34], while others, such as in otolaryngology and emergency medicine, found no significant correlation [17, 21]. These discrepancies may be attributed to methodological differences, including variations in sources, the number of journals analyzed, and the time periods studied as depicted in Table 1. Although there were several other statistically significant correlation results between the journal SC and various indicators, one must carefully interpret these relationships since the correlation does not assure that the relationship between two variables is causal [47]. On the other hand, journals that belong to the lower quartiles (Q3 and Q4, based on a lower IF) are the ones that benefit the most from SC, and therefore the most vulnerable to changes in position when SC are withdrawn [48], but they are also the ones with the highest SCR [14, 15]. Fig 3 shows that the journals with the highest SCR belong to Q4 or Q3, however, there are also journals from higher quartiles. Also, when ordering the journals according to the number of SC per publication, only high-impact journals (Q1 or Q2) are observed. I hypothesize that this difference is due to the denominator of the indicator, since in the SCR, despite a high number of SC, this is practically diluted in the Q1 or Q2 journals since their total citations are higher, while in the SC/publication only depends on the productivity of the journal, so they would be more comparable.

In the present study, editorials were the type of document that had the greatest number of SCs. Despite uncertainties about the ethics of allowing editors to publish in their own journals, this practice persists without defined regulations on the matter [49]. The IF of a journal for any specific year is calculated by dividing the total number of citations received by the articles published in the journal during the preceding 2 years (numerator), by the total number of articles published in the same 2 years (denominator) [50]. While all the published materials (original papers, reviews, editorials, letters to editor, news, book reviews, correspondence, etc.) are accepted in the numerator, only original papers and review articles are counted in denominator [50]. Therefore, having published several editorials with high percentages of SC could have inflated the 2022 IF since document types such as editorial materials and letters are not counted as ‘citable items’ in the denominator, but their citations are counted in the numerator of the IF calculation [14]. Although the present study cannot conclude that journal SC in COVID-19 caused by editorial materials have produced a dramatic change in the IF, it is undeniable that their SC levels are high. Previous research indicated that editorial materials from WoS did not seem to be extensively used to increase the IF by citing items published in the same journal [51]. Nonetheless, another study suggested that at least in other more specific fields, such as trauma and orthopedics, there is an increase in the IF score by 0.03 points, for every journal SC [10]. In the present study, the letters also had a large number of SCs. Manley demonstrated that the journal IF is reduced when letters are considered ’citable documents’ [52]. Although his data is limited to 6 journals related to scholarly publication and metrics, his study showed that the smaller the number of citable items, the greater the impact on the IF calculation generated by even a single letter [52]. Letters are easier to accept in various journals and do not take much time to prepare as an original article even though they may sometimes report original data [53]. Indeed, there is a recent trend among several journals to prioritize using letters for disseminating studies that are less robust or have a smaller sample size and therefore require less extensive discussion [54]. For example, some high-impact journals like JAMA have the ’research letter’ as a publication type that consists of focused reports of original research and can include any of the study types listed under original investigation. Similarly, The Lancet encourages the publication of some original results under the ’correspondence’ category. Previous research showed that the contribution of citations from the so-called ’non-citable items’ can increase the journal IF, especially in journals whose number of citable and non-citable items is unequal [55, 56]. Thus, the journal IF would decrease from the original if its calculation included all document types in the denominator or if citations of non-citable items were not counted in the numerator [14]. Although one can argue that the quality of an article’s research is the main reason why a given article is cited or not, there is a specific Matthew effect associated with journals and this gives articles published there an added value above of its intrinsic quality [57, 58]. It is possible that in this case journal SCs are the initial impulse for journals that are not high impact to overcome the Matthew effect, and thus be able to attract citations to their papers independently of their own impact but rather due to the journal’s status [14, 16]. However, this does not mean that high-impact journals do not have excessive SCs. Ultimately, like any other niche field, COVID-19 can have high SCs [24, 30, 34, 35].

Another interesting finding is the SCR at the publisher level, in which MDPI stands out significantly compared to the rest, although it had fewer journals (n = 209) with literature relevant to COVID-19 compared to other publishers with more than 1,000 journals. This may be partially explained by the fact that MDPI has several journals that are among the most productive in COVID-19 (Table 2), and since the relationship between the number of publications and SCs is proportional, it is expected to obtain these results. However, this publisher has already been questioned before, not only for high SCR in its journals, but also for an increasingly high rate of citations from other MDPI-journals, thus creating an intra-publisher citation network [59, 60], and also for its countless articles in special collections compared to other publishers [46]. The study by You et al. suggested that questionable publishers inflate the impact of their citations by including publisher-level SCs, which poses challenges for detection through traditional journal metrics [61]. This practice is also known as citation stacking [62], and it has become even more complex when intra- and inter-disciplinary publisher SCs come into play [63]. There is no better example of interdisciplinarity in recent years than COVID-19, mobilizing research in various areas of knowledge in a short time, which is associated with the impact of research based on its citations, and in turn with SCs [62, 64]. Exacerbated by a constant race against the clock to publish quickly and massively, this is also reflected when comparing COVID-19 with other long-standing diseases. In the present study, the level of journal SCRs was lower than the rest due to the exorbitant number of citations in the denominator, although at the same time, the number of SCs per publication stands out, which was significantly higher in both periods. One should approach with caution when comparing different infectious diseases, taking into account the rapid, pandemic spread of COVID-19 and the varying impact of these distinct disease entities.

Through a survey of more than 110,000 scholars from various disciplines and using a journal-based dataset from Scopus, a study concluded that higher-ranked journal editors seem to tend to force scholars to add citations that are not pertinent to their work and that commercial publishers are more likely to own journals that have coerced [65]. Although these results are definitely not conclusive, concerns are raised about how “relevant” the SC of some journals were. Journal SC are a natural part of the publication process, and though manipulation of citations is unethical [66], there is no clear limit as to how much is acceptable. This study does not intend to conclude that all SCs from journals and publishers correspond to coercive SCs, but rather its findings shed light on an even bigger question: how to detect coercive self-citations and how to stop them? Although some authors have developed sophisticated algorithms to detect journals or publishers SC [67, 68], it is still difficult to accurately distinguish between legitimate citations and those intended to artificially inflate impact metrics.

To address the issue of problematic self-citation practices, authors, reviewers, editors, and publishers can implement several specific measures:

  1. Exclude journal SCs from the IF calculation since this would eliminate the motivation to coerce [69].
  2. Adopt measures such as a transparent peer review, so that the reviewers’ reports, authors’ responses, and editors’ decision letters are all published alongside an article [70]. Transparency can discourage unethical practices and allow the community to monitor and report any irregularities.
  3. Avoid the inappropriate use of ’non-citable items’ for the calculation of IF, since all material published in a journal is subject to being cited, in this way all document types must be included in the denominator of the IF calculation [14].
  4. Report the classic journal impact metrics together with the level of journal SCs, similar to what was proposed for the authors’ h-index [71].
  5. Foster a community culture that values integrity over impact metrics along with publishers establishing ethical guidelines and impose sanctions to those who engage in such practices [65]. For instance, by imposing a ban on authors, removing editors from editorial roles, delisting reviewers from journals’ list of reviewers, and blacklisting journals from citation indexes [72].
  6. Rely on peer reviewers who are crucial in maintaining the scientific quality and integrity of published papers. However, due to the high volume of submissions and their busy schedules, they may not always verify every referenced source [72, 73]. Editors, who approve reviewers’ recommendations, are ultimately responsible for any inappropriate suggestions [73].

According to a document released by COPE (the Committee on Publication Ethics) in July 2019, citation manipulation refers to excessive citation of articles from the journal in which the author is publishing an article or another journal (’citation stacking’) as a means solely of increasing the number of citations of the journal in question [74]. There are many instances where citing researchers’ own work or that of colleagues are legitimate. This action becomes unethical only when the cited sources are not materially relevant to the manuscript’s scholarly content [72, 75]. Additionally, there are distorted publishing incentives that can lead scientists to publish for the wrong reasons, often chasing a journal IF score for a paper rather than producing well-structured and valuable research for their peers, regardless of where it’s published [73]. Unfortunately, metrics like the IF are used for formal evaluations and promotions of scientists in academia, particularly in developing nations, and are thus open to wide abuse by authors, editors and journals [50, 73]. Although, IF is often erroneously used to judge the quality of a paper, it was meant to exclusively judge the “popularity” of a journal [73]. Weale et al. suggested using the level of non-citation of articles within a journal as an alternative measure, as the ultimate goal of all research is to be cited in other authors’ work. This metric also provides a more comparable measure across different disciplines [76]. Finally, a new phenomenon is emerging where articles are cited in exchange for monetary compensation, aimed at manipulating reference lists to inflate a journal’s IF [72, 77].

This study has several limitations. First, preprints were not considered, even though they were a means of important and rapid dissemination of valuable information amid the threat of the pandemic, especially in the early phase [78]. However, not considering them allowed me to avoid duplicates and only select information that has already been peer-reviewed and published in a scientific journal. Second, the study did not cover journals not listed in the JCR. The JCR contains more than 21,000 journals and although it does not have many journals compared to other databases, its selectivity feature allows it to include only journals that met high-research standards and passed a strict process to continue their indexing in the JCR, and avoid being delisted each year [37, 39, 79]. Thus, the findings of the present study do not apply to other journals that may have published COVID-19 papers in other databases. Overall, the results of journal SC in COVID-19 research have a high degree of generalizability to all WoSCC journals due to the key characteristic of having used a complete sampling frame of journals that provides stronger and more reliable conclusions. Nonetheless, it is pertinent to recognize that including only the top twelve publishers based on their number of publications raises concerns that would limit the generalizability of publisher SC in COVID-19 research and could potentially result in misleading conclusions. Finally, there is a citation and indexing time lag of publications in the WoSCC that could have affected the findings. Future studies should aim to expand these datasets, particularly by completing the dataset for publishers in COVID-19 research, as well as to characterize the journal SC trends according to the various research fields involved in COVID-19 research.

Conclusions

In summary, this study has shown that the median SCR was 4% and that in 13% of the journals, one in five references was a journal self-citation. The number of publications, citations, citations per publication correlated with journal self-citations. The editorial materials contributed more to journal self-citations, suggesting that in COVID-19 research, self-citing non-citable items could potentially contribute to inflate journal impact factors. The MDPI publisher, despite having less journals in the topic, had higher self-citation rates compared to other publishers with more journals. The number of SC per publication of COVID-19 research was higher than those for tuberculosis and HIV/AIDS.

Supporting information

S1 File. Search strategy.

https://doi.org/10.1371/journal.pone.0314976.s001

(DOCX)

S2 File. Journals, type of publication and publisher datasets based in COVID-19 research.

https://doi.org/10.1371/journal.pone.0314976.s002

(XLSX)

S3 File. Tuberculosis and HIV/AIDS research.

https://doi.org/10.1371/journal.pone.0314976.s003

(XLSX)

References

  1. 1. Brandt MD, Ghozy SA, Kallmes DF, McDonald RJ, Kadirvel RD. Comparison of citation rates between Covid-19 and non-Covid-19 articles across 24 major scientific journals. PLoS One. 2022;17: e0271071. pmid:35895698
  2. 2. Ioannidis JPA, Bendavid E, Salholz-Hillel M, Boyack KW, Baas J. Massive covidization of research citations and the citation elite. Proc Natl Acad Sci. 2022;119: e2204074119. pmid:35867747
  3. 3. Maillard A, Delory T. Blockbuster effect of COVID-19 on the impact factor of infectious disease journals. Clin Microbiol Infect. 2022;28: 1536–1538. pmid:35998808
  4. 4. Uddin S, Khan A, Lu H. Impact of COVID-19 on Journal Impact Factor. J Informetr. 2023;17: 101458.
  5. 5. Delardas O, Giannos P. How COVID-19 Affected the Journal Impact Factor of High Impact Medical Journals: Bibliometric Analysis. J Med Internet Res. 2022;24: e43089. pmid:36454727
  6. 6. Clark J. How covid-19 bolstered an already perverse publishing system. Bmj. 2023;380: 689. pmid:36977517
  7. 7. Chorus CG. The practice of strategic journal self-citation: It exists, and should stop (a note from the editor-in-chief). Eur J Transp Infrastruct Res. 2015;15: 274–281.
  8. 8. Humphrey C, Kiseleva O, Schleicher T. A Time-Series Analysis of the Scale of Coercive Journal Self-Citation and its Effect on Impact Factors and Journal Rankings. Eur Account Rev. 2019;28: 335–369.
  9. 9. Campanario JM. Journals that rise from the fourth quartile to the first quartile in six years or less: Mechanisms of change and the role of journal self-citations. Publications. 2018;6: 47.
  10. 10. Jain A, Khor KS, Beard D, Smith TO, Hing CB. Do journals raise their impact factor or SCImago ranking by self-citing in editorials? A bibliometric analysis of trauma and orthopaedic journals. ANZ J Surg. 2021;91: 975–979. pmid:33559283
  11. 11. Straub DW, Anderson C. Communications of the Association for Information Systems Journal Self-Citation VI: Forced Journal Self-Citation–Common, Appropriate, Ethical? Commun Assoc Inf Syst. 2009;25: 6.
  12. 12. Ziemann S, Paetzolt I, Grüßer L, Coburn M, Rossaint R, Kowark A. Poor reporting quality of observational clinical studies comparing treatments of COVID-19 –a retrospective cross-sectional study. BMC Med Res Methodol. 2022;22: 23. pmid:35057739
  13. 13. Benjamens S, Pol RA, de Meijer VE, Haring MPD. Peer review during demanding times: maintain rigorous standards. Scientometrics. 2021;126: 6115–6117. pmid:33935329
  14. 14. Taşkın Z, Doğan G, Kulczycki E, Zuccala AA. Self-Citation Patterns of Journals Indexed in the Journal Citation Reports. J Informetr. 2021;15: 101221.
  15. 15. Gazni A, Didegah F. Journal self-citation trends in 1975–2017 and the effect on journal impact and article citations. Learn Publ. 2021;34: 233–240.
  16. 16. Deora H, Kraus KL, Couldwell WT, Garg K. Self-Citation Rates Among Neurosurgery Journals and Authors: Unethical or Misunderstood? World Neurosurg. 2023;178: e307–e314. pmid:37473867
  17. 17. Sri-Ganeshan M, Walker KP, Lines TJ, Neal-Williams TJL, Taylor DMD. Author and journal self-citation in Emergency Medicine original research articles. Am J Emerg Med. 2021;50: 481–485. pmid:34517173
  18. 18. Ma Y, McBratney A, Minasny B. Paper self-citation rates of leading soil science journals. Catena. 2021;202: 105232.
  19. 19. Miyamoto S. Self-citation rate and impact factor in the field of plastic and reconstructive surgery. J Plast Surg Hand Surg. 2018;52: 40–46. pmid:28486090
  20. 20. Kurmis TP, Kurmis AP. Self-citation rates among medical imaging journals and a possible association with impact factor. Radiography. 2010;16: 21–25.
  21. 21. Motamed M, Mehta D, Basavaraj S, Fuad F. Self citations and impact factors in otolaryngology journals. Clin Otolaryngol Allied Sci. 2002;27: 318–320. pmid:12383288
  22. 22. Fassoulaki A, Paraskeva A, Papilas K, Karabinis G. Self-citations in six anaesthesia journals and their significance in determining the impact factor. Br J Anaesth. 2000;84: 266–269. pmid:10743468
  23. 23. Sanfilippo F, Astuto M, Tigano S, La Rosa V, Morgana A, Zanghì A, et al. Author self-citation policies, the influence on journals’ impact factors, and self-citation rate in general surgery. Eur Surg. 2021;53: 329–334.
  24. 24. Sanfilippo F, Tigano S, Morgana A, Murabitol P, Astuto M. Self-citation policies in anaesthesiology journals. Br J Anaesth. 2021;126: e21–e25. pmid:33121748
  25. 25. Sanfilippo F, Crimi C, Morgana A, La Via L, Astuto M. The influence of policies limiting author self-citations on journals impact factor and self-citation rate in respiratory system. Multidiscip Respir Med. 2022;17: 871. pmid:36051889
  26. 26. Sanfilippo F, Tigano S, Morgana A, Murabito P, Astuto M. Self-citation policies and journal self-citation rate among Critical Care Medicine journals. J Intensive Care. 2021;9: 15. pmid:33499899
  27. 27. Krauss J. Journal self-citation rates in ecological sciences. Scientometrics. 2007;73: 79–89.
  28. 28. Bennett H, Singh B, Slattery F. Journal self-citations trends in sport sciences: an analysis of disciplinary journals from 2013 to 2022. Scientometrics. 2024; 1–10.
  29. 29. Blackledge KT, Ananth C V., Brandt JS. The influence of journal self-citations on impact factors in obstetrics and gynecology. Am J Obstet Gynecol. 2022;226: 736–737.e1. pmid:34968457
  30. 30. Sundaram K, Warren J, Anis HK, Klika AK, Piuzzi NS. Publication integrity in orthopaedic journals: the self-citation in orthopaedic research (SCOR) threshold. Eur J Orthop Surg Traumatol. 2020;30: 629–635. pmid:31858259
  31. 31. Livas C, Delli K. Journal Self-Citation Rates and Impact Factors in Dentistry, Oral Surgery, and Medicine: A 3-year Bibliometric Analysis. J Evid Based Dent Pract. 2018;18: 269–274. pmid:30514441
  32. 32. Landoni G, Pieri M, Nicolotti D, Silvetti S, Landoni P, Silvani P, et al. Self-citation in anaesthesia and critical care journals: Introducing a flat tax. Br J Anaesth. 2010;105: 386–387. pmid:20716571
  33. 33. Mimouni M, Ratmansky M, Sacher Y, Aharoni S, Mimouni-Bloch A. Self-citation rate and impact factor in pediatrics. Scientometrics. 2016;108: 1455–1460.
  34. 34. Reiter O, Mimouni M, Mimouni D. Analysis of self-citation and impact factor in dermatology journals. Int J Dermatol. 2016;55: 995–999. pmid:26695728
  35. 35. Mimouni M, Segal O. Self-citation rate and impact factor in ophthalmology. Ophthalmic Res. 2014;52: 136–140. pmid:25300716
  36. 36. Opthof T. Inflation of impact factors by journal self-citation in cardiovascular science. Netherlands Hear J. 2013;21: 163–165. pmid:23463445
  37. 37. Pranckutė R. Web of Science (WoS) and Scopus: The titans of bibliographic information in today’s academic world. Publications. 2021;9: 12.
  38. 38. Alryalat SAS, Malkawi LW, Momani SM. Comparing bibliometric analysis using PubMed, Scopus, and Web of Science databases. J Vis Exp. 2019; e58494. pmid:31710021
  39. 39. Birkle C, Pendlebury DA, Schnell J, Adams J. Web of Science as a data source for research on scientific and scholarly activity. Quant Sci Stud. 2020;1: 363–376.
  40. 40. Torres-Salinas D, Valderrama-Baca P, Arroyo-Machado W. Is there a need for a new journal metric? Correlations between JCR Impact Factor metrics and the Journal Citation Indicator-JCI. J Informetr. 2022;16: 101315.
  41. 41. Patil I. Visualizations with statistical details: The “ggstatsplot” approach. J Open Source Softw. 2021;6: 3167.
  42. 42. McVeigh M. Journal self-citation in the Journal Citation Reports. In: Clarivate Analytics [Internet]. 2009 [cited 16 Mar 2024]. Available: https://clarivate.com/webofsciencegroup/essays/journal-self-citation-jcr/#:~:text=A relatively high self-citation,incoming citations from other sources.
  43. 43. Siler K, Larivière V, Sugimoto CR. The diverse niches of megajournals: Specialism within generalism. J Assoc Inf Sci Technol. 2020;71: 800–816.
  44. 44. Ioannidis JPA, Pezzullo AM, Boccia S. The Rapid Growth of Mega-Journals: Threats and Opportunities. JAMA. 2023;329: 1253–1254. pmid:36939740
  45. 45. Gorman DM. The rise of a mega-journal in public health publishing. Learn Publ. 2023;36: 712–719.
  46. 46. Brainard J. Fast-growing open-access journals lose impact factors Web of Science delists some 50 journals, including one of the world’s largest. Science (80-). 2023;379: 1283–1284. pmid:36996220
  47. 47. Schober P, Schwarte LA. Correlation coefficients: Appropriate use and interpretation. Anesth Analg. 2018;126: 1763–1768. pmid:29481436
  48. 48. Campanario J-M. Los Journal Citation Reports (edición SCI) con y sin autocitas de revista. El Prof la Inf. 2018;27: 241–253.
  49. 49. Helgesson G, Radun I, Radun J, Nilsonne G. Editors publishing in their own journals: A systematic review of prevalence and a discussion of normative aspects. Learn Publ. 2022;35: 229–240.
  50. 50. Juyal D, Thawani V, Sayana A, Pal S. Impact factor: Mutation, manipulation, and distortion. J Fam Med Prim Care. 2019;8: 3475–3479. pmid:31803639
  51. 51. Campanario JM, González L. Journal self-citations that contribute to the impact factor: Documents labeled “editorial material” in journals covered by the Science Citation Index. Scientometrics. 2006;69: 365–386.
  52. 52. Manley S. Letters to the editor and the race for publication metrics. J Assoc Inf Sci Technol. 2022;73: 702–7.
  53. 53. Dkhar SA. Letter to editor: its importance and drawbacks. Int J Community Med Public Heal. 2018;5: 4634.
  54. 54. Kiesslich T, Weineck SB, Koelblinger D. Reasons for journal impact factor changes: Influence of changing source items. PLoS One. 2016;11. pmid:27105434
  55. 55. Xue-Li L, Ya-Hui W, Shuang-Shuang G. Citation characteristics of non-citable documents and contributions to journal impact factor. Curr Sci. 2018;114: 1423–1429.
  56. 56. Heneberg P. Parallel Worlds of Citable Documents and Others: Inflated Commissioned Opinion Articles Enhance Scientometric Indicators. J Am Soc Inf Sci Technol. 2014;65: 635–43.
  57. 57. Drivas K, Kremmydas D. The Matthew effect of a journal’s ranking. Res Policy. 2020;49: 103951.
  58. 58. Larivière V, Gingras Y. The Impact Factor’s Matthew Effect: A Natural Experiment in Bibliometrics. J Am Soc Inf Sci Technol. 2010;61: 424–7.
  59. 59. Copiello S. On the skewness of journal self-citations and publisher self-citations: Cues for discussion from a case study. Learn Publ. 2019;32: 249–258.
  60. 60. Ángeles Oviedo-García M. Journal citation reports and the definition of a predatory journal: The case of the Multidisciplinary Digital Publishing Institute (MDPI). Res Eval. 2021;30: 405–419a.
  61. 61. You T, Park J, Lee JY, Yun J, Jung WS. Disturbance of questionable publishing to academia. J Informetr. 2022;16: 101294.
  62. 62. Heneberg P. From excessive journal self-cites to citation stacking: Analysis of journal self-citation kinetics in search for journals, which boost their scientometric indicators. PLoS One. 2016;11: e0153730. pmid:27088862
  63. 63. Zhou Y. Self-citation and citation of top journal publishers and their interpretation in the journal-discipline context. Scientometrics. 2021;126: 6013–6040.
  64. 64. Okamura K. Interdisciplinarity revisited: evidence for research impact and dynamism. Palgrave Commun. 2019;5: 141.
  65. 65. Fong EA, Wilhite AW. Authorship and citation manipulation in academic research. PLoS One. 2017;12: e0187394. pmid:29211744
  66. 66. Moussa S. A bibliometric investigation of the journals that were repeatedly suppressed from Clarivate ‘ s Journal Citation Reports Reports. Account Res. 2023;30: 592–612. pmid:35469511
  67. 67. Yu T, Yu G, Wang MY. Classification method for detecting coercive self-citation in journals. J Informetr. 2014;8: 123–135.
  68. 68. Kojaku S, Livan G, Masuda N. Detecting anomalous citation groups in journal networks. Sci Rep. 2021;11: 14524. pmid:34267254
  69. 69. Fong EA, Patnayakuni R, Wilhite AW. Accommodating coercion: Authors, editors, and citations. Res Policy. 2023;52: 104754.
  70. 70. Petersen AM. Megajournal mismanagement: Manuscript decision bias and anomalous editor activity at PLOS ONE. J Informetr. 2019;13: 100974.
  71. 71. Bartneck C, Kokkelmans S. Detecting h-index manipulation through self-citation analysis. Scientometrics. 2011;87: 85–98. pmid:21472020
  72. 72. Mehregan M. Scientific journals must be alert to potential manipulation in citations and referencing. Res Ethics. 2022;18: 163–168.
  73. 73. Teixeira da Silva JA. The ethics of peer and editorial requests for self-citation of their work and journal. Med J Armed Forces India. 2017;73: 181–183. pmid:28924321
  74. 74. COPE Council. COPE discussion document: Citation manipulation. 2019 [cited 5 Aug 2024]. Available: https://publicationethics.org/node/44351
  75. 75. Mahian O, Wongwises S. Is it Ethical for Journals to Request Self-citation? Sci Eng Ethics. 2015;21: 531–533. pmid:24728611
  76. 76. Weale AR, Bailey M, Lear PA. The level of non-citation of articles within a journal as a measure of quality: A comparison to the impact factor. BMC Med Res Methodol. 2004;4: 14. pmid:15169549
  77. 77. Bohannon J. Apparent “pay to cite” offer sparks Internet outrage. In: Science News [Internet]. 2015 [cited 5 Aug 2014]. Available: https://www.science.org/content/article/apparent-pay-cite-offer-sparks-internet-outrage
  78. 78. Fraser N, Brierley L, Dey G, Polka JK, Pálfy M, Nanni F, et al. The evolving role of preprints in the dissemination of COVID-19 research and their impact on the science communication landscape. PLoS Biol. 2021;19: e3000959. pmid:33798194
  79. 79. Li K, Rollins J, Yan E. Web of Science use in published research and review papers 1997–2017: a selective, dynamic, cross-domain, content-based analysis. Scientometrics. 2018;115: 1–20. pmid:29527070