Introduction

Every year, industry spends an estimated $60 billion funding research on drugs, biotechnology, and medical devices [1]; commercial company expenditures on phase 1–3 clinical drug trials outstrip the National Institutes of Health’s spending on trials [2]. Disclosure of such funding is important in part because a Cochrane systematic review on industry sponsorship of research found that industry-funded studies tend to produce results favorable to their company sponsors more often than studies not funded by industry [3]. It concluded that this finding may be partly caused by the choice and dosing of comparators, selection of outcomes, and selective analysis, reporting, and publication. It is also possible that industry is successfully identifying research projects that are more likely to yield results favorable to their products.

The National Academy of Medicine defines a conflict of interest (COI) as “circumstances that create a risk that professional judgements or actions regarding a primary interest will be unduly influenced by a secondary interest [4].” In the case of research COIs, the primary interest is typically research integrity, while the secondary interest is generally financial gain. The conflict may not necessarily compromise research integrity, but conflicts increase that risk or create an appearance of risk [5]. Disclosing COIs in biomedical research may allow scientists, medical professionals, and the public to better assess the credibility of scientific findings and explicitly consider the potential impact of funding on the research.

PubMed is a free resource that provides access to a database of citations to published research, along with their abstracts, primarily in the biomedical and health fields, though it includes related disciplines like behavioral and life sciences and bioengineering. It was developed and is maintained by the National Center for Biotechnology Information at the National Library of Medicine (NLM), an institute within the National Institutes of Health [6].

In March 2016, the Center for Science in the Public Interest, along with five other organizations, 62 scientists and physicians, and 5 U.S. senators, asked the NLM to list researchers’ COIs in a standardized statement as part of the abstracts indexed on PubMed [7, 8]. Including COIs in the abstract page on PubMed is important because readers may not have access to the full article and many readers rely on the abstract alone to decide whether to read the full text of the article, to draw conclusions about a study, and/or to guide clinical care decisions [9, 10]. A year later, NLM announced it would begin including voluntary COI statements on PubMed, when these statements are provided by the journal, for inclusion in a “Conflict of Interest statement,” located just below the abstract on PubMed [11]. These statements may also include declarations of “no conflict.” The changes were implemented on March 8, 2017.

Although most journals have COI disclosure policies for authors [12, 13], the onus is on journals to ensure that information from each article is included on PubMed. When journals add their articles to PubMed electronically, they are required to submit and tag 13 pieces of information (e.g., article title, journal, and publisher), whereas 37 other tags are optional or required only if applicable (e.g., COI statement and abstract) [14]. The prevalence of use of the COI statement on PubMed remains unknown. To address this research gap, this study aims to describe the prevalence of the voluntary COI statement in articles indexed on PubMed.

Methods

Study sample

With assistance from a staff scientist at NLM, we first identified journals indexed on PubMed from January 1, 2016, through December 31, 2021, that published original research (e.g., clinical trials, observational studies, and validation studies), reviews (e.g., systematic reviews, meta-analyses, and narrative reviews), and comments (e.g., letters, editorials, consensus statements, case reports, and guidelines). We excluded books and documents, legislation and government publications, article preprints, news articles, and datasets on the ground that journal COI disclosure standards for these are less well developed. For each year, we recorded the total number of journals indexed and the number of journals with at least one article that included a COI statement on PubMed (hereafter, “posted COI statement”), indicating that the journal at least sometimes used the COI field. A posted COI statement included any stated COIs or any affirmative statement that there were no COIs in the paper. We also recorded the total number of published articles and the number of those that included a posted COI statement by year. The year 2016 was included to examine the change in use of the COI statement after it was added to PubMed in 2017. These data were collected in January 2023.

We then identified the 40 highest impact journals in 2021, based on Journal Citation Reports [15], that were indexed on PubMed and had published at least 50 articles meeting the study inclusion criteria from June 1, 2021, through May 31, 2022. We collected characteristics of these journals by searching journal, International Committee of Medical Journal Editors (ICMJE) [16], and Committee on Publication Ethics (COPE) websites [17]. Journal characteristics included whether the journal was associated with a professional society or not (society-based/other), journal type (predominantly basic science/predominantly clinical/review/other), journal publication country (U.S./U.K./other), impact factor (continuous), ICMJE and COPE membership (yes/no), and publisher (professional society/Elsevier/Springer/other).

We used a random number generator [18] to randomly select 100 articles from each of the 40 highest impact journals. We examined articles from the top 10 journals (993 articles, as one journal had only published 93 articles) to inform our final sample size, revealing that 33% included a COI in the article itself and 54% of those included a posted COI statement. With about 4,000 articles (40 journals) overall, we would be able to detect that prevalence of posted COI statement use among those with a published COI with a margin of error of +/- 3% at a 95% confidence level [19].

To determine what fraction of all COIs were reflected in posted COI statements, we downloaded and manually searched articles for COIs (a financial link with the topic under investigation) anywhere in the main text of the article, including in the funding, disclosures, and acknowledgements fields (hereafter, “published COIs”) and documented the location of all disclosures. We downloaded these articles between June 2022 and August 2022. Published COIs located in a disclosures field or COI statement in the article were coded as “COI statement” and those located in the acknowledgements or funding fields were coded separately. We included relevant published COIs regardless of when they took place. If the article declared “no conflict of interest” and there were no published COIs found in the main text of the article, we did not consider the article to have published COIs. We also recorded the study type of each article (comment/original research/review). We then assessed the PubMed entry corresponding to each article with published COIs to determine if the published COIs were described in a posted statement. A flow chart that shows how the samples were selected is included in Fig 1.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Article selection flow chart.

https://doi.org/10.1371/journal.pone.0308782.g001

Two members of the research team reviewed all articles in the first seven of the 40 journals to establish consistency in coding (about 700 articles). The reviewers then compared notes to identify inconsistencies in their identification of COIs and reached consensus through discussion. With the exception of about 10 articles, there was agreement between the two reviewers on identifying articles with published COIs and posted COIs, COI location, and study type. Thereafter, to conserve resources, one reviewer examined the remaining journals.

Results

1. Analysis of the full dataset (2016–2021)

Although the COI policy only went into effect in 2017, some posted COI statements were added retroactively (Jeff Beck (NLM), oral communication, April 8, 2022). The proportion of journals on PubMed with at least one article with a posted COI statement increased from 25.9% in 2016 (1,790/6,894 journals) to 33.2% in 2021 (2,511/7,550 journals), although that increase has not been monotonic (Fig 2). At the article level, the percentage with a posted COI statement steadily increased, from 9.0% in 2016 (31,718/350,587 articles) to 43.0% in 2021 (186,500/433,386 articles).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Percentage of all journals that included a posted COI statement for at least one article and the percentage of all articles with a posted COI statement, 2016–2021, as of 2023.

https://doi.org/10.1371/journal.pone.0308782.g002

2. Analysis of the 40 highest impact journals

Journal characteristics (June 2021 –May 2022).

Most of the 40 highest impact journals were not run by a society (72.5%) and most published predominantly clinical research (42.5%) and reviews (35.0%) (Table 1). All but one was based in the U.S. (32.5%) or the U.K. (65.0%) and the median 2021 impact factor was 65.9 (range: 43.4–202.7). Most journals were not ICMJE members (60.0%) but 95.0% of journals were members of COPE. Seventy-five percent of journals were published by Elsevier (32.5%) or Springer (42.5%).

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Characteristics of the 40 highest impact journals.

https://doi.org/10.1371/journal.pone.0308782.t001

Between June 1, 2021, and May 31, 2022, all of the 40 highest impact journals published at least one article with a posted COI statement, indicating widespread awareness and ability to use the COI field.

Prevalence of posted COI statements (2016–2021).

Of the 40 highest impact journals in 2021, the number that published at least one article with a posted COI statement ranged from 84.6% (33/39 journals) in 2016 (one journal did not publish any articles that year) to all 40 in 2021 (Fig 3). At the article level, the percentage with a posted COI statement increased over time from 5.9% in 2016 (706/11,936 articles) to 35.1% in 2021 (3,725/10,598 articles).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Among the 40 highest impact journals in 2021, percentage of all journals that included a posted COI statement for at least one article and the percentage of all articles that included a posted COI statement, 2016–2021, as of 2022.

https://doi.org/10.1371/journal.pone.0308782.g003

COI posting prevalence (June 2021 –May 2022).

Table 2 lists the 40 journals with the highest impact factors in 2021, along with the percentages of the randomly selected 100 articles per journal that contained published COIs and, among those, the percentage that included posted COIs. Six of the 40 journals published fewer than 100 articles between June 1, 2021, and May 31, 2022, for a total of 3,888 articles. The percentage of articles with published COIs by journal ranged from 5.0% to 83.0% [median: 26.0%; interquartile range: 12%-37.5%] and the percentage of articles with published COIs that also included a posted COI statement ranged from 0.0% (Science [7 articles with published COIs] and Nature Materials [5 articles with published COIs]) to 100.0% (BMJ, Lancet Psychiatry, Lancet Public Health, Cell, Cell Research, and Immunity) [median: 48.1%; interquartile range: 25.9%-94.0%]. A Spearman’s Rank Correlation Test indicated that there was no correlation between journal impact factor and the percentage of articles with published COIs that included a posted COI statement within this subset (r = -0.0854; p = 0.6001).

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Articles with published COIs and a posted COI statement among high impact journals, June 2021 through May 2022.

https://doi.org/10.1371/journal.pone.0308782.t002

Of all 3,888 randomly selected articles across the 40 highest impact journals, 69.8% [95% CI: 68.3%-71.2%] did not have published COIs. Of the 1,174 articles with published COIs, 63.3% [95% CI: 60.4%-66.0%] included a posted COI statement (Fig 4). (An analysis of posting and publication rates weighted by impact factor provided essentially identical results and is not presented).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Disclosure patterns of the articles in the 40 highest impact journals published from June 2021 through May 2022 and hand-searched for published COIs (n = 3,888), as of 2022.

https://doi.org/10.1371/journal.pone.0308782.g004

Characteristics and predictors of COI posting (June 2021 –May 2022).

Of the subset of articles with published COIs (n = 1,174), 27.0% were original research, 33.0% were review articles, and 39.9% were comments (Table 3). Most (n = 1,036, 88.2%) of these articles had published COIs in only the COI statement in the main text of the article, 7 (0.6%) had published COIs in only the acknowledgements, and 131 (11.2%) had published COIs in more than one location. None of the conflicts that appeared in an acknowledgements or funding section in the paper appeared in a posted COI statement.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. The association between article characteristics and inclusion of a posted COI statement among articles that disclosed a COI in high impact journals, June 2021 through May 2022.

https://doi.org/10.1371/journal.pone.0308782.t003

The bivariate regression results show that, among articles with published COIs, the odds of original research articles including a posted COI statement were over 3 times those odds for comment articles (OR: 3.74; 95% CI: 2.63, 5.32), an association that rose to over 5-fold (OR: 5.40; 95% CI: 3.51, 8.32) in the multivariate analysis. In the multivariate analysis, articles with published COIs in more than one location in the article text were 60% less likely than articles with published COIs in one location only to have a posted COI (OR: 0.40; 95% CI: 0.24, 0.68) (Table 3).

Discussion

This examination of the adoption of the COI statement on PubMed since its inception in 2017 found that, while the use of the posted COI statement on PubMed has increased fairly steadily, overall use of the statement could be improved. In 2021, only 33% of journals used the statement for at least one article. Among the 40 highest impact journals, use of the COI statement on PubMed was higher, with all journals posting COI statements for at least one article in 2021 but only 58% of articles with known COIs using the posted COI statement. Based on whether journals used the posted COI statement at least once per year, higher impact journals seem to be utilizing the posted statement more often than lower impact journals, but all journals could improve their consistent use of the posted COI statement.

The underutilization of the posted COI statement is a combination of journals that do not use the statement, journals that are using the statement inconsistently, and journals that are using the posted COI statement but failing to transfer conflicts from the acknowledgements or funding sections of their papers.

The first, and largest, problem is that most journals, particularly those with lower impact factors, do not use the posted COI statement at all. Journals have to transfer and tag over a dozen pieces of information for each article, yet many choose not to transfer COI information. Given that journals had five years to adopt the COI statement on PubMed at the time of our analysis, the relatively low adoption rate raises questions about whether all journals are aware of the option. According to the NLM, requiring the use of the posted COI statement as a mandatory condition of listing on PubMed is not advisable because it could limit submissions of journal articles, which does not align with PubMed’s mission as a comprehensive resource for finding and indexing publications (Jeff Beck (NLM), email communication, June 15, 2023).

NLM can, however, strongly encourage use of the statement. When the statement was first made available, the agency notified publishers through email and included information about the statement in a technical bulletin (Jeff Beck (NLM), oral communication, April 8, 2022) [13]. The group JATSforReuse, a voluntary collaboration of publishers and NLM [21], published technical recommendations for pulling COI statements from article metadata in 2020 [22], and, according to NLM, the agency continues to encourage publishers to include COI statements on PubMed (Jeff Beck (NLM), email communication, June 15, 2023). But NLM should go further and individually notify journals that did not use the posted COI statement in the most recent year.

We examined the second problem, that of inconsistent posted COI statement use, through our collection of a random subset of articles from the 40 highest impact journals. Journals varied in their use of the posted COI statement. In our sample, Science and Nature Materials never used the posted statement, although they have used the statement in articles not included in our sample, while only five journals used posted COI statements every time there was a published COI.

The third problem, which is much less common than the first two and the most difficult to remedy, relates to the transfer of conflicts located in an acknowledgements or funding section of an article to the posted COI statement. Of the articles in the top 40 journals, none with published COIs in an acknowledgements or funding section in the paper included those COIs in the posted COI statement, suggesting that journals pull data only from the COI statement in the article. In some instances, study funding was included within the abstract itself on the abstract page of PubMed. But this study was focused on the use of the COI statement, so these disclosures were not included in our analysis. Currently, including disclosures from acknowledgements and funding sections of published articles would require some degree of manual curation and may therefore be difficult to accomplish in practice. Standardization of disclosure practices to use a single field across all journals would greatly facilitate data transfer.

This study had several limitations. First, for the full dataset, we established the prevalence of posted COI statement use in part by determining whether the journal used the COI field at least once in a given year. Measuring the actual number of postings, as we also did, is dependent upon both the underlying prevalence of COIs and the rate of posting. In contrast, measuring whether journals posted at least once per year eliminates the first factor as long as journals are likely to include at least one published COI in a year. Most journals publish dozens of articles each year and our analysis of the 40 highest impact journals showed that all had at least five articles with a published COI in the year studied. Thus, all journals appear to have had ample opportunity to use the field at least once in any given year and failure to do so likely indicates non-use of the field. Second, we were only able to assess the prevalence of use of the posted COI statement by articles with published COIs in the 40 highest impact journals in a single year due to the large number of articles that would have been required individual assessment had we included more journals. While this provided sufficient precision for the purposes of this study, it precludes generalization of our findings on posting rate per conflict to other journals. Third, our study reflects posted COI statements as of 2022 when we collected the data, not as of the year on which we are reporting, because posted COI statements may have been added retroactively. Fourth, while we examined the 40 highest impact journals in 2021, our findings may not be generalizable to the highest impact journals in other years. Fifth, our regression only included two covariates; additional article-level variables that were not available in this study may also be associated with the posting of COIs.

Conclusions

Use of the PubMed COI statement has increased since it became available in 2017, but adoption is still limited, even among high impact journals. One consequence is that users of PubMed are currently unable to distinguish between a published COI that was not posted and an article with no COIs. NLM should carry out additional outreach to journals that are both not using the statement at all and using it erratically, and journals should use the statement for all COIs, regardless of where a COI is located in the paper. Until use of the statement reaches full adoption, NLM could improve the user experience through additional education of PubMed users about what is included and excluded from the COI statement and the importance of reviewing COI disclosures in the published articles.

Acknowledgments

We would like to thank Eva Greenthal for her work on an early draft of the study protocol and Kate Peglow for her assistance with data collection. We would also like to thank staff from the National Library of Medicine for their assistance in downloading data from PubMed and for added context on the voluntary COI statement.