Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

The use and acceptability of preprints in health and social care settings: A scoping review

  • Amanda Jane Blatch-Jones ,

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Supervision, Writing – original draft, Writing – review & editing

    A.J.Blatch-Jones@southampton.ac.uk

    Affiliation National Institute for Health and Care Research (NIHR) Coordinating Centre, School of Healthcare Enterprise and Innovation, University of Southampton, Southampton, Hampshire, United Kingdom

  • Alejandra Recio Saucedo,

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Writing – review & editing

    Affiliation National Institute for Health and Care Research (NIHR) Coordinating Centre, School of Healthcare Enterprise and Innovation, University of Southampton, Southampton, Hampshire, United Kingdom

  • Beth Giddins

    Roles Data curation, Formal analysis, Investigation, Writing – review & editing

    Affiliation National Institute for Health and Care Research (NIHR) Coordinating Centre, School of Healthcare Enterprise and Innovation, University of Southampton, Southampton, Hampshire, United Kingdom

Abstract

Background

Preprints are open and accessible scientific manuscript or report that is shared publicly, through a preprint server, before being submitted to a journal. The value and importance of preprints has grown since its contribution during the public health emergency of the COVID-19 pandemic. Funders and publishers are establishing their position on the use of preprints, in grant applications and publishing models. However, the evidence supporting the use and acceptability of preprints varies across funders, publishers, and researchers. The scoping review explored the current evidence on the use and acceptability of preprints in health and social care settings by publishers, funders, and the research community throughout the research lifecycle.

Methods

A scoping review was undertaken with no study or language limits. The search strategy was limited to the last five years (2017–2022) to capture changes influenced by COVID-19 (e.g., accelerated use and role of preprints in research). The review included international literature, including grey literature, and two databases were searched: Scopus and Web of Science (24 August 2022).

Results

379 titles and abstracts and 193 full text articles were assessed for eligibility. Ninety-eight articles met eligibility criteria and were included for full extraction. For barriers and challenges, 26 statements were grouped under four main themes (e.g., volume/growth of publications, quality assurance/trustworthiness, risks associated to credibility, and validation). For benefits and value, 34 statements were grouped under six themes (e.g., openness/transparency, increased visibility/credibility, open review process, open research, democratic process/systems, increased productivity/opportunities).

Conclusions

Preprints provide opportunities for rapid dissemination but there is a need for clear policies and guidance from journals, publishers, and funders. Cautionary measures are needed to maintain the quality and value of preprints, paying particular attention to how findings are translated to the public. More research is needed to address some of the uncertainties addressed in this review.

Background

The publication of research is slow, and accessibility of research outputs is often delayed due to manuscripts going through a lengthy process involving peer-review, revisions and getting published online [1]. Once published, the outputs of the research are not always widely accessible due to subscription charges, which ultimately places restrictions on who can access published manuscripts, especially those in resource-constrained countries or institutions, and the public. These barriers cause risk to the value of the research and its contribution to future scientific research and discovery being realised [25]. Preprints can offer a solution, and for some academic disciplines preprints have been part of the publication pathway for more than 20 years, providing instant open access to research via preprint servers (e.g., arXiv, bioRxiv, chemRxiv, and EarthArxiv) [1].

The first preprint server known as arXiv came into practice in the physics community in 1991 [6]. Posting a manuscript in a preprint server has its advantages particularly when it can take a long time to publish research findings in a peer review publication. Preprints allow researchers to post their research findings earlier and to allow for wider research community review that would otherwise only be afforded to a minimum number of peer reviewers (chosen by journal editors as part of the publication process) [2].

However, the acceptability and role of preprints can vary not only by discipline (including whether the research is biomedical, preclinical or implementation research) but also among researchers, publishers, and funding organisations.

What are preprints?

Preprints are open and accessible scientific manuscripts that are shared publicly, through a preprint server, before being submitted to a journal (or at the time of submission to a journal, as part of the submission process of the journal). Sharing a manuscript or report publicly prior to submission enables full access and, increases equity in the publication process (providing the ability to share research without paying access charges). However, it is important to note that some proponents of preprinting may only publish their work as a preprint and not as an article in a peer reviewed journal.

Use and acceptability of preprints

The role of preprints during the COVID-19 pandemic highlighted their importance in a public health emergency, and now with their continued use, it is important to understand how preprints contribute to the open research agenda [710]. However, the acceptability and use of preprints by some in the research community, can lead to hesitancy or resistance from researchers to submit their research to a preprint server [11, 12]. Concerns around the integrity and validity of submitting a manuscript to a preprint server, is also a challenge for articles published in a peer reviewed journal [13, 14]. Peer reviewed journal articles can also be retracted due to unreliable findings or errors in the published article. Funding organisations have different policy statements about how and where preprints are accepted as part of the application process. For example, whether preprints can be referenced as part of a grant application; included in an applicant’s curriculum vitae (CV); or form part of the dissemination process and publication of findings.

Understanding the position of funding organisations on preprints has important implications for how the research community is willing to engage in, and accept the added value and benefit of, publishing their research in a timely fashion, and, contribute to advancing scientific knowledge [12, 15, 16]. Some funding organisations, such as Wellcome and UK Research and Innovation (UKRI) have already initiated policy guidance on their position of preprints in grant applications and some journal publishers (e.g., Springer Nature, Wiley and SAGE) are beginning to implement statements around their publishing models.

The purpose of this systematic scoping review was to explore the position of publishers, funding organisations and the research community in terms of the use and acceptability of preprints in health and social care settings at the grant application stage and publication of findings (e.g., the use of preprints in grant applications or as part of the publishing model). To our knowledge, a review of this topic or at this scale has not been previously undertaken. The review was conducted to address the following question: What does the evidence say about the role and acceptance of preprints (challenges, benefits, value, hesitancy, impact) throughout the research lifecycle (e.g., grant applications and publication of findings)?

Methods

A previous non-systematic search identified several articles examining the role of preprints and how preprints contribute to the promotion of the open research agenda. This preliminary search was used to inform the design and conduct of this systematic scoping review (a proforma was written prior to conducting the review to ensure the rationale, objectives, design, and methodology were followed).

Due to the complexity, uncertainty, and nature of the available research (in terms of source, type, and audience) a scoping review methodology was undertaken. Scoping reviews are relevant to addressing research questions on priorities for research, clarification on concepts and definitions, and, providing research frameworks or background information in preparation for a systematic review. Scoping reviews typically identify evidence gaps or scope the body of literature rather than seeking to describe experiences or current practice [17]. Scoping reviews look to address ‘what has been done previously?’ and ‘what does the literature say?’ compared to systematic reviews that ask the question ‘does this intervention work for this particular group?’. Scoping review methodology does not judge the quality of the evidence but rather intends to map the evidence. Although scoping reviews present an overview of a broad research question or topic from a diverse body of evidence, they are conducted using methodological frameworks and guidance [18, 19]. The Joanna Briggs Institute (JBI) guidance was used to support the development, analysis and write up of the scoping review [2022].

Eligibility criteria

Context.

The context included UK and international settings within the academic environment.

Participants.

Publishers, research funding organisations and Higher Education Institutions (HEI’s) use of preprints were included (e.g., academia or research management focused). The role of preprints from a researcher’s perspective, placing emphasis on open science and open research, was also considered.

Inclusion criteria.

Evidence from research disciplines that focused on health and social care and all phases of research, from the point of testing a new treatment or intervention through to implementation and service delivery research were included. This is not an exhaustive list but included health and social care disciplines such as sociology, biology, health economics, social work, nutrition, chemistry, pharmacology, and psychology.

Exclusion criteria.

Evidence outside of the health and social care discipline, (e.g., engineering, physics, agriculture, arts, music) including industry and private businesses with no academic focus. The authors limited the review to only this discipline due to the variation of perspectives across other disciplines (including acceptability and how preprints are used to inform future scientific discovery) and to explore what challenges were specific to health and social care that may not be relevant to other fields of research. Non-English articles were excluded if no translation was available for the full article.

Types of sources

The scoping review considered all types of study designs for inclusion (e.g., randomised controlled trials, non-randomised controlled trials, before and after studies and interrupted time-series studies, analytical observational studies including prospective and retrospective cohort studies, case-control studies, analytical cross-sectional studies, descriptive observational study designs including case series, individual case reports and descriptive cross-sectional studies).

Qualitative studies that focused on, but not limited to, designs such as phenomenology, grounded theory, ethnography, qualitative description, action research and feminist research were also considered. In addition, systematic reviews, text, and opinion articles that met the inclusion criteria were also considered.

The review included several types of published material such as academic outputs through journal articles, commentaries, editorials, and opinion letters. Grey literature of policies, guidance and reports from funding organisations and publishers were also included. It was important to capture all types of literature due to the nature of the research question and to determine whether there were any gaps in the evidence (is the evidence anecdotal or were there research studies on the role and acceptance of preprints?).

Search strategy

The search strategy aimed to locate both peer reviewed and non-peer reviewed studies (through the inclusion of conference abstracts). An initial limited search of PubMed, Mendeley and Google Scholar was undertaken to identify articles on the topic. The text contained in the titles and abstracts of relevant articles and reports, and the index terms used to describe the articles were used to develop a full search strategy. The search strategy, including all identified keywords and index terms, was adapted for each database (See S1 Appendix: Search terms and keywords; S1 Table: Scopus and Web of Science searches). The reference lists of included articles were also screened to identify additional literature material.

There were no study or language limits applied during the information retrieval process. However, the search strategy was limited to the last five years (2017–2022), which enabled us to determine whether the COVID-19 pandemic accelerated the use and role of preprints in health and social care research. Two databases were searched: Scopus and Web of Science on 24 August 2022.

Data extraction and evidence selection

Following the search, all identified articles were collated and uploaded into EndNote version 20 (Clarivate Analytics, PA, USA) and duplicates removed. A pilot test of titles and abstracts were then screened for assessment against the inclusion criteria. Relevant articles were retrieved in full for screening. Ten articles were screened by all reviewers (ABJ, ARS, BG) to ensure continuity and agreement across the team. All articles were reviewed independently by two reviewers (ABJ reviewed all articles and BG and ARS reviewed 50% each) and notes were included in a separate Endnote library. Where the independent reviewer was unsure of the inclusion of the article, the record was annotated for discussion with the team and decisions for inclusion or exclusion were made by consensus. Disagreements between the reviewers at each stage of the selection process were also resolved through discussion.

After the final screening, the list of included articles was exported to an Excel spreadsheet where labelling of articles matched the evidence to:

  • the two processes of interest (funding application and/or publication of findings),
  • specific areas of interest (challenges, barriers, benefits, solutions, recommendations).

The results of the search and the study inclusion process are reported in full in a Preferred Reporting Items for Systematic Reviews and Meta-analyses extension for scoping review (PRISMA-ScR) flow diagram (see Fig 1).

Data were extracted from the articles included in the scoping review by using a data extraction tool developed by the reviewers, based on the research question. The tool included specific details about the participants, concept, context, study methods and key findings. The extraction included direct export from the included articles into the spreadsheet to avoid having to return to the original source (and prevent subjective or evaluative accounts of the included articles). Extraction from each full text article was divided between the three reviewers and ABJ reviewed and verified all extraction. Due to using a scoping review methodology, no risk of bias or assessment on quality was conducted. All the evidence was mapped and categorised to specific terms, which were discussed and agreed between team members at various stages throughout data extraction.

Funding organisations (e.g., Wellcome, UKRI, National Institute for Health and Care Research (NIHR), National Institutes of Health (NIH), Canadian Institutes of Health Research (CIHR), European Research Council)), including those organisations referenced in key articles, were explored as part of the grey literature search using Google. A search of publishers (e.g., Springer, Wiley, SAGE, PlosOne, F1000) were also conducted for additional information to understand the policy, position, and acceptability of preprints in their publication process. The policy, guidance and publications webpages were visited, and any preprint server details were recorded. The purposive sampling of funding organisations and publishers were drawn from discussions with NIHR Coordinating Centre staff to better understand the position of other funding organisations and publishers. A separate excel spreadsheet was used to capture the search results, specifically focusing on funding organisations and publishers preprint policies, guidance, and statements.

Results

A total of 587 articles were retrieved from the two databases. After deduplication (208 articles), 379 titles and abstracts and 193 full text articles were assessed for eligibility (12 articles were retrieved from references of those included articles). Of these 193, 98 articles met the eligibility criteria and were subsequently included in the review for full extraction. Fig 1 provides a full account of the records of identification flow diagram, including the reasons for the excluded articles. (See S2 Table: Full details of the 98 included articles).

Characteristics of the included studies

All articles were published after 2017. Most of the articles were published in a journal with a majority being perspective, editorial or commentaries (n = 69). There were 24 original research articles that consisted of surveys with researchers, clinicians, and editors; analyses of preprint servers and policies; analyses of funding organisations and publishers preprint policies; and comparisons of preprints to peer reviewed publications. Characteristics of the 98 included articles are presented in Table 1.

thumbnail
Table 1. Summary of the characteristics of the included articles (n = 98).

https://doi.org/10.1371/journal.pone.0291627.t001

As expected, much of the evidence focused on the publication stage of research with minimal evidence on the use of preprints at the grant application stage, apart from providing statements that some funding organisations accept preprints as part of a grant application. More than half of the evidence was published from 2020 onwards (n = 63) and most of the articles (based on first authors country) were based in Europe (n = 36) or Americas (n = 35).

Summarising the evidence

To summarise the evidence found in the database searches, key statements were derived from the articles and labelled under generic categories. Table 2 presents the statements, along with frequencies and source of evidence on the challenges and barriers associated to preprints. Table 3 presents the statements, along with frequencies and source of evidence on the benefits and value associated with the use of preprints.

thumbnail
Table 2. Challenges and barriers associated to preprint publications, ranked by theme and numerical frequency.

https://doi.org/10.1371/journal.pone.0291627.t002

thumbnail
Table 3. Benefits and value associated with the use of preprint publications, ranked by theme and numerical frequency.

https://doi.org/10.1371/journal.pone.0291627.t003

Fig 2 provides an overview of the benefit and value, and challenges and barriers statements associated to the use, acceptability, and implementation of preprints.

thumbnail
Fig 2. Benefits and value statements grouped under six themes and challenges and barriers statements grouped under four themes.

https://doi.org/10.1371/journal.pone.0291627.g002

All the statements associated with the use of preprints varied considerably depending on what perspective was taken (publishers, funding organisations or researchers) and when the article was published. There was a steady increase in the number of publications on preprints since 2020, which could be associated to the COVID-19 pandemic. However, any association between the effect of COVID-19 and the volume of preprints requires further research, which was out of remit for this scoping review (scoping methodology does not assess risk of bias or quality).

The remaining sections provide a summary of the evidence based on the research question: What does the evidence say about the role and acceptance of preprints throughout the research lifecycle, specifically in grant applications and publication of results?.

Barriers and challenges associated with preprints

Although preprints have been established in some disciplines for more than 20 years, the COVID-19 pandemic has revealed the rapid progress of preprints, both for their value and their shortcomings [8, 10, 2339]. Preprints are considered to lack scientific integrity, credibility and quality, and there is still reservation among editors, publishers, journals and researchers about the role of preprints and their contribution to science [4, 6, 35, 4043].

The evidence provided several process issues related to why preprints lack quality, screening, or quality checks during the submission of a manuscript to a preprint server [33, 44, 45]. It was reported that only three preprint servers (Research Square, bioRxiv and medRxiv) check if the content contains unfounded medical claims [33, 44]. Several articles suggested that preprints represent incomplete scrutinised documents that have not been validated by peer review [14], there is higher risk of retractions [39], and not all preprints end up getting published in a peer reviewed publication [6, 32].

The wider implications of preprints not having the same or similar quality checks or assurances means that information is subject to interpretation and could be used by the public and social media inaccurately [32]. This may result in the public reading a preprint as ‘established evidence’ when it has not been through any peer review [2, 30, 46] and the media sensationalising the research before it is vetted as sound science [32, 47]. The lack of understanding of the role of preprints in the research process could equate to potential harm and have immediate impact on medical and clinical practice, including patients [8, 26, 32, 33, 35, 45, 46, 4850]. Policy decisions may also be based on research where flaws have not yet been corrected as they typically would be during the peer review process [31, 51]. For researchers, the evidence suggested a high level of concern around the risks of intellectual ‘scooping’ [2, 12, 15, 47, 5254] and for some disciplines or fields of research, rapid dissemination of results should not be prioritised over quality or scrutiny [4, 46]. (See Table 2 for a summary of statements associated to the barriers and challenges of preprints).

Without the proper guidance or caveats on preprint platforms, stating what they are and what they are not, there is the potential for preprints to have a negative impact on the credibility and public perception towards research and scientific discovery [2, 24, 27]. This is in part due to the public interpreting the preprint as ‘established evidence’ when it has not been through any peer review, scrutiny or quality assurance process [32]. These cautionary measures also extend to the use of preprints by clinicians and policymakers, especially if the study involves medications, vaccines or medical devices. The results could have a negative impact on the treatment of patients (including public health and safety) [7, 8, 26, 32, 33, 35, 45, 46, 4851, 64] and result in the spreading of disinformation and distrust of research [24, 38].

Benefits and value of preprints

A general consensus by those supportive of preprints was the notion of how preprints should be celebrated for bringing research faster to the public [2, 3, 8, 39, 6973]. There was also acceptance that preprints are not a substitute for peer-reviewed publications, they are one element of the wider publication process of scientific discovery and sharing of research findings [3, 27, 74].

Most preprints do not differ substantially from the published peer-reviewed article, although there may be variation, there was no evidence that key sections of articles, such as results, changed between preprints and published versions [12, 24, 25, 36, 42, 75]. Brierley et al. (2021) found minimal changes were requested to preprint conclusions, suggesting that the entire publication pipeline is having a minimal but beneficial effect upon preprints and this was supported in the evidence [12, 24, 25, 36, 42, 75]. Moreover, the conclusions of 7.2% of non-COVID-19 related and 17.2% of COVID-19 related abstracts underwent only a discrete change (e.g., additional details on the funding statement) by the time of publication, but the majority of these changes do not qualitatively change the conclusions of the paper [10, 25].

As mentioned by Kaiser and others, intellectual ‘scooping’ and plagiarising have been regarded as a barrier to preprints [2, 12, 15, 47, 5254]. However, this is unlikely to happen as preprints provide a time-sensitive/time-stamp when submitted to a preprint server [15, 47, 70]. As explained by Tennant et al. (2019), most preprints have a permanent identifier, often a Digital Object Identifier (DOI) which makes it easy to cite, track and establish a proof of discovery [47]. There is virtually no evidence to suggest research scooping exists from preprint servers [2, 41, 47, 7678].

Key considerations found in the evidence suggested that preprints have the potential to offer free and immediate access to results, enabling the scientific community (including developed and emerging economies) to build upon research discovery [6, 26, 41, 50, 70, 7881]. Preprints can facilitate feedback from a wider peer community than the traditional two to four reviewers needed to assess manuscripts submitted to journals [59]. This allows a unique platform for scientific critique and feedback before the work is subjected to more lengthy review processes conducted by a journal [61]. It also enables research to be openly accessible, reducing research waste and speeding up opportunities for future funding (using preprints as part of the grant application) [58, 69]. This not only has important implications for Early Career Researchers (ECRs) [53, 69] but also allows for negative findings to be published and openly accessible [5, 15, 50, 58, 59, 69, 70]. (See Table 3 for a summary of statements associated to the benefits and value associated with the use of preprint publications).

The general consensus of those in support of preprints suggest that the focus has shifted to ‘how preprints should be cited’ rather than ‘whether preprints should be cited’ [35]. Furthermore, many of the potential risks associated with preprints reported not only from academia but also from publishers and funding organisations, exist within the traditional peer review publication system [98]. The COVID-19 pandemic may have initiated a cultural shift in the use of preprints by academia, publishers, funding organisations, the public and policymakers as a whole [10, 25, 29, 30, 81].

Academic community acceptability of preprints

One of the key challenges for the academic community is the lack of or no clear policies on the citation of preprints, from publishers and journals but also funding organisations. Klebel et al. (2020) found approximately 75% of journals do not have clear policies on co-reviewing, citation of preprints and publication of reviewers’ identities for preprints [57]. As the evidence suggested there were several competing interests for academics, and these were more relevant in some disciplines than others and not all research needs to implement all open science practices due to the nature of the research [4, 59].

A survey conducted by Soderberg et al. (2020) found that of 3759 researchers (with 13.25% non-completion) 69.73% felt slightly to strongly favourable towards preprints, 15.16% felt opposed to preprints and 14.95% felt neutral, although there was variation between disciplines and academic career stages [9]. However, assessing the credibility of preprints is an important factor to researchers, such as the transparency of research content, accessibility to the data (information on the open research content) and an independent review and validation of the research claims [60].

For many investigators, preprints may be considered an initial step along the scientific dissemination and publication pathway, just as an abstract or poster have a role in the early sharing and dissemination of research among specialist communities before submission to a peer reviewed journal [26, 64]. However, preprints do not come without their challenges that can sometimes be in direct conflict with the pressures of research findings having to be accessed openly and transparently as early as possible [30]. These initiatives unintentionally place greater responsibility onto the scientific community and raise concerns that preprints are substandard work [12, 76]. However, there was no evidence to support such claims in the literature reviewed. Researchers will only publish their best work as it reflects their abilities and expertise, and often their reputation [70]. So, the notion of low quality would not be expected from those using preprints to disseminate their research findings [7, 23, 53, 59, 70]. Existing evidence from Brierley et al. (2021) and Watson (2022) for example have highlighted how preprints do not differ substantially from peer reviewed journal articles and since the COVID-19 pandemic, there has been a notable increase in the number of preprints [24, 25].

Addressing the challenges of publication models

Despite the drawbacks and deadly consequences, there is little doubt that preprint publishing is here to stay. The question is how science will handle it. We are down a pathway of open science, and that pathway is going to accelerate. Our choice is not whether it occurs or not; our choice is whether it occurs responsibly.” (Watson, 2022) [24].

The evidence strongly indicated that the COVID-19 pandemic has changed the viewpoint and acceptance of preprints, at a time when rapid communication was needed and a greater visibility of effectively communicating the scientific advancements for healthcare [6, 8, 10, 99]. With more than half of the evidence being published during the pandemic suggests that in times of national emergency, initiatives such as open research and open access become even more under the spotlight [45]. In the first four months of the COVID-19 pandemic, 19,389 articles were available, with a third of these being preprints [24]. In fact, the media and government quoted from preprints as the traditional peer review system was proving to be too slow [3, 27]. There was a surge of data insights during the pandemic that have ultimately accelerated the pace and speed of research, and having direct influence on policy and practice [10, 24]. However, as discussed by Raynaud et al. (2021) fast tracking peer review at a time of crisis can have implications to the quality of the assessment and overall results [32]. Rapid publication and potential risk of unverified (not being through a peer reviewed process) findings in health related research for example, can result in misinformation and misrepresentation of findings [15]. Teixeira da Silva noted several concerns around retractions and the threat preprints can impose on the validity of scientific discovery, especially at the height of the COVID-19 pandemic [34, 35]. Ethical guidance and disclaimers are needed to indicate where results from a preprint have not been peer reviewed or quality assured. Informing the reader of these caveats, may help members of the public to stay informed of the status of the evidence, which is particularly relevant for clinical trials and experimental research (Box 1 provides an overview of the key considerations for best practice as summarised from the evidence).

Box 1. Considerations for best practice

• Quality: Preprints have quality controls including requirements for ethics approvals, consents, appropriate institutional archiving, and appropriate research reporting checklists

• Communications: Multiple forms of communication are required at the right time, at the right stage and are translational to the wider public. Communicating in diverse ways and to diverse audiences requires implementing several straightforward ideas that includes the use of multimedia and digital media platforms

• Publishing: Greater consistency and scrutiny across different journals or publishing platforms for medical scientific articles or manuscripts is needed to ensure quality and control are maintained even if different publishing models are used. Compatibility and appropriate checks on publication policies. Potential to reduce publication bias as not subject to publication selectivity.

• Open review: Open assessment from a larger community enables diverse groups to contribute, offers a democratic process and opportunities to accelerate research discovery, priority, and dissemination

• Open access and inclusivity: Open access of preprints immediately provides everyone with access to the research, including developing nations who often have limited funds (institutional or funding organisations) to publish, read, and subscribe to many publishing journals. Adopting preprints allows for enhancements to academic publishing and peer assessment to enhance scientific discovery and establish priority

• Diversifies Early Career Researchers: Preprints can accelerate training, enable ECRs to publish through open access routes

• Ethics: There is a need for principle-based regulations to achieve publication ethics for preprints. Gaining optimum accountability and transparency of research findings requires effort from both authors and preprint servers: (e.g., transparency in the time sequence of publication between preprints and subsequent peer-reviewed journal articles)

• Standards: Stricter common standards for preprints are required that cover issues such as screening submissions and retracting those that turn out to be seriously flawed or fraudulent.

Evidence from journals and publishers

There is growing acceptance from publishers recognising the value and necessity of preprints towards more open and transparent publication models [100]. Several articles included editorials or commentaries from journal publishers stating their position on preprints including changes to policy, and in the broader context of open access and open research [49, 71, 100102]. Evidence from the articles included in the review have shown that preprints are becoming recognised as part of the dissemination publication process in the wider open research initiative, but to varying degrees based on clinical relevance [7, 32, 45].

Teixeira da Silva and Dobránszki (2019) conducted a review of preprint policies from 14 publishers and found that in 2017 64% of publishers had a preprint policy in place but by February 2018 this had increased to 78% [43]. Further advancements have been seen by eLife, showing that preprints can work alongside peer reviewed publications by adopting a ‘publish, then review’ model replacing the traditional ‘review, then publish’ model [3]. Approximately 70% of the papers reviewed by eLife during May, June and July 2020 had already been posted as preprints indicating that their model was already being implemented into practice [45]. (See S3 Table: Publishers and Journals preprints policy or guidance).

There is a clear indication that publishers are reviewing their policies and amplifying the preprints model [3, 45] and these changes were also evident in the grey literature of publishers and journals. Several publishers are actively encouraging the research community to use preprints to publish their manuscripts (e.g., Lancet, BMC, Thorax and eLife); offering to upload the manuscript to a preprint server on their behalf following submission (e.g., PLOS); developing and introducing their own preprint server for specific research (e.g., Nature for COVID research); reviewing tools to reduce information overload (e.g., simultaneous multi-preprint server retrieval tools to aggregate preprints); and publishing the preprint version alongside the peer reviewed journal article on the website (e.g., PeerJ and F1000Research) [4, 6, 23, 49, 74, 83, 98, 101].

Publishers such as the Lancet and eLife are also disseminating their own preprint policies and guidance in editorials and commentaries on how the model of publishing is changing [64]. The PubMed database is also taking steps to index preprints from journals such as F1000 and PeerJ. However, despite these developments, the evidence suggests that preprint policies vary between publishing companies and in some instances within the same publishing company and across research disciplines, which could hinder the progress and integrity of preprints [35, 40, 41, 42, 43, 57, 59]. Box 2 provides a summary of the key considerations for publishers and journals as summarised from the evidence.

Box 2. Summary of the key considerations for publishers

• Operating principles and mechanisms to enhance the publishing process and manuscript version control

• Enhanced cooperation between scholarly publishers and preprint servers

• Consider a peer review procedure for preprints

• When preprints are cited in peer-reviewed journals, they must be clearly indicated as such and remain under press embargo until published

• Formulate principles of publication ethics for conduct (e.g., release and use of preprints to serve as references

• A balance between immediacy and quality control is required to ensure research findings and new discoveries are not compromised due to process driven inefficiencies

• Opportunities to develop alternative publishing models whereby a two-step approach could ensure open access is truly promoted and credible.

Evidence from funding organisations

Over the last five years several funding organisations have endorsed the use of preprints, encouraging researchers to use preprints as an interim to speed up the dissemination of the research [1]. There are also several funding organisations that allow the inclusion of preprints as part of a grant application, which are starting to be disseminated in published literature [70, 97]. However, the evidence was sparse, with most focusing on the publication aspect of research findings. (See S4 Table: Funding organisations position on preprints for grant applications and publications).

Some funding organisations have taken the initiative to develop their own exclusive platform to host preprints and other non-peer reviewed publications such as the Bill and Melinda Gates Foundation (platform to present findings as a future preprint server (Gates Open Research), using the F1000 Research technical platform that is leased for a fee [15]); Wellcome Trust (preprint server since 2016 (Wellcome Open Research)) [5]; and the National Institute for Health and Care Research (NIHR) (an Open Research platform for its researchers to publish, using prepublication checks and open peer review) [103].

The evidence also showed that many funding organisations are also assessing their policies and guidance or implementing new policies that take account of preprints in varying situations. A useful source to track funding organisations progress on these preprint policies is via the ASAPbio website [104, 105]. (see Box 3 for a summary of the key considerations for funding organisations based on the available evidence) [90].

Box 3. Summary of the key considerations for funding organisations

• To develop coherent guidance for the performance evaluation of preprints

• To consider responsible use of metrics and deter from affiliation-based citation bias, including metrics of scholarly achievements

• Ensure that reviewers (both grant and publication reviewers) adhere to guidance on preprints, ensuring fairness and impartiality in the review process when evaluating performance of research

• Citation bias exists more with preprints; therefore, funding organisations need to deter from using these measures when making decisions about funding recommendations

• Implement policies to promote openness and ensure researchers are transparent about their procedures and share all aspects of their funded research (going beyond just publications), following the Declaration On Research Assessment (DORA)

• Regulating the use of preprints through the Plan U approach, mandating that funding organisations request that all grant applicants post their research first on preprints servers.

Grant applications

There was limited evidence on the use of preprints in grant applications and most of the evidence was either in the grey literature or on the funding organisations webpages [15, 30, 70, 90].

Preprints can provide a funding organisation with evidence to substantiate recent productivity. However, there is wide variation in how funding organisations accept preprints, including in what circumstances they are acceptable in a grant proposal (e.g., researchers own preprints to demonstrate productivity or using preprints as part of the research proposal itself) [51, 70, 75]. Preprints are therefore said to have varying impact for researchers, grantees or applicants [2, 23, 41, 70, 75, 76, 78, 89, 90].

Several funding organisations (e.g., Wellcome Trust, NIH) now endorse the use of preprint servers for grant applications [106, 107]. Including preprints in the grant application process provides reviewers the opportunity to access and gain early insight to the research and data in its full format (i.e., not restricted by word limitation in the funding application) [3].

However, there have been concerns over the misuse of metrics, particularly with funding organisations using preprint citations [11, 60]. With the growing number of preprints there is a risk of author affiliation-based citation bias for those who use citations for scientific impact quantification (e.g., funding organisations, institutional promotion). This is particularly relevant to funding organisations who make recommendations to fund research based on citations. Although, we have seen a move away from this, with initiatives such as the Declaration on Research Assessment (DORA) [108], Contributor Roles Taxonomy (CRediT) [109] and Plan S/cOAlition S [110] this is not a global phenomenon [30, 60] and has important implications for diversity, equity and inclusion (including ECRs and Low-and-Middle Income Countries (LMICs)) [23, 30, 53, 59, 60].

Discussion

It was clear from the evidence that there was a general acceptance and understanding that preprints are not a substitution or replacement for a peer reviewed publication. Preprints complement and act as a mechanism within the publication process to disseminate the research findings faster, through openly accessible routes [23, 55, 92].

The distinction between a preprint and a peer reviewed publication becomes imperative when you consider the unintended consequences of their use on not only clinical practice but also on researchers to judge research performance, quality or productivity [2, 29, 64, 70, 91]. Whilst preprints can be part of the process to promote open research, by applying the principles of transparency and integrity, we also need to appreciate and acknowledge the associated caveats and limitations of their use [25, 26, 33, 64].

Although the COVID-19 pandemic has informed a greater appreciation for the scholarly landscape on the use and acceptability of preprints, there have been withdrawals of several preprints [10, 34]. However, there is no way of knowing whether peer review would have identified the problems resulting in the withdrawal of the manuscript at the point of publication. This is also true for peer reviewed journal articles, as articles are only withdrawn or retracted once published (issues raised by readers or the authors themselves). These challenges are not exclusive to preprints, however with careful consideration and acceptance of these challenges, alongside the limitations, preprints can become an integral part of scientific discovery if all journals support publishing on preprint servers as part of their process (not all journals ask authors to submit their manuscript on a preprint server as part of the submission process to a peer reviewed journal) [6]. Evidence comparing preprints to peer reviewed journal articles have found that the quality is similar for both, and the types of discrepancies (e.g., study characteristics, outcome reporting, numerical and statistical values) are common between both preprints and a peer reviewed journal article [10, 99, 111114]. Opportunities for rapid dissemination using preprints means that best practices through policies and guidance are required, from journals, publishers, and funding organisations, to ensure that preprints become embedded into practice albeit with caveats and cautionary measures to maintain quality, credibility, transparency and openness (as with peer reviewed articles submitted to journals). Initiatives such as Plan S (open access publishing) and cOAlition S (funders and stakeholders) advocate for immediate and open access to research publications without embargo [110], which over time may ultimately reassure the research community of the value and benefit of publishing on a preprint server prior to the submission of the manuscript to a journal.

The increasing demands and challenges to allow for the immediacy of results being made public places pressure on the open access agenda for not only publishers but also the research community [87]. Delays to the publication of manuscripts can also have far reaching implications for ECRs in terms of job opportunities and grant funding [53, 60, 61, 80]. Understanding the complexities that surround the use, acceptability, challenges and benefits of preprints goes far beyond the world of publishing research: policies and practices of funding organisations and university institutions have a direct impact on the longer-term impact of how research enhancement and discovery takes place [7, 23, 51, 59, 70].

Study strengths and limitations

The main strength of the review was the combination of a systematic database literature search with an extensive grey literature search of publishers and funding organisations position, policies, and guidance. However, as scoping reviews map the evidence rather than assess quality or risk of bias, this was a limitation to the current review. Restricting the evidence to only a health and social care discipline meant that articles from other disciplines were not included in the review, with the majority of evidence focused on publishing models and minimal evidence looking at the role of preprints in grant applications. Although, there were some articles from Asia (12.3%), most of the articles were from Europe and Americas (72.4%), which could have introduced regional biases (e.g., Western science). However, this could be due to a lack of evidence from these regions rather than missing literature from the systematic searches. Only searching in two databases could have meant we missed some articles (including preprints) on this subject matter. However, the role of scoping reviews are an overview of a broad topic or subject (identifying key concepts, gaps in the evidence and sources of evidence), rather than to evaluate or synthesise data on a focused research question or topic [18].

Due to the limited evidence found in the review on the acceptability and use of preprints in grant applications, research is needed to clearly understand what position funding organisations are taking and how it is perceived and accepted by the research community (with initiatives such as Plan S and cOAlition S [110]). There was a lack of evidence to clearly explain in what circumstances preprints are acceptable to cite in the grant application process. For example, to inform the research application or as part of an applicant’s CV, at what stage is it acceptable to include and what version of record, and how is it tracked when published or otherwise as a peer reviewed journal article (particularly for funding organisations).

Future recommendations

The recommendations arising from the review mainly focused on what measures are needed for preprints to become embedded in open research practices. These recommendations are important for publishers, funding organisations and researchers, by considering the role of preprints and where and in what circumstances preprints may not be feasible, such as

  • Accessibility and feasibility to submit and upload to preprint servers on behalf of the researcher (therefore ensuring continuity) [78]
  • Adopting the use of preprints in grant applications requires clear and ethical guidance
  • Applying a badge or signal to show whether the article complying with open data, data sharing, pre-registrations etc could also be adopted by preprint services (with clear guidance) similar to the Center for Open Science (COS) (a badge system has led to over 60 journals adopting this approach on published articles to indicate open data, data sharing, materials, and/or pre-registrations) [9, 33]
  • Preprint servers must have clear ethical and retraction policies in place, and these must be enforced and considered in publishers and funding organisations preprint policies [15, 34, 115]
  • Preprint policies should include ethical guidance in the event of misconduct and consider whether this should be the same process as peer-reviewed journal articles [34]
  • Preprints should not be posted when the reported information could be misapplied or misused, causing significant consequences to the health and safety of the public [116]

(See S5 Table: Future considerations and recommendations for publishers, funding organisations and the research community).

Clear, explicit, and transparent guidance by publishers, journals, and funding organisations (working collaboratively) are required to articulate to academia the role and purpose of preprints compared to a peer reviewed journal publication. There is also a need for publishers and journals to acknowledge how preprints fit into their publication model, ensuring that policies are aligned with preprints servers and are transparent about their use of preprints.

Methodologically, it would be important to consider preprints in other disciplines other than health and social care, to understand whether there are differences between disciplines and whether those that submit and have used preprint servers for several years, approach preprints with the same level of caution or not.

Conclusions

Further research is needed to address several key areas associated with preprints that may help to clarify the respective opportunities that preprints can offer to the publication process. It will be important to consider the views, opinions and expectations of researchers, funding organisations and publishers, to ensure that global challenges regarding access to research findings (e.g., budgetary constraints) does not inhibit ECRs or LMICs [23, 53, 84]. The lack of evidence on preprints at the grant application stage requires more research with funding organisations and the research community, such as screening of preprints submissions used in grant applications as part of the evidence to support the need of the research and as part of the applicants’ publications listing [15, 43, 70].

The question of quality, peer review, and integrity are paramount to the acceptability of preprints if they are to contribute to improving the reproducibility of research and further advance scientific discovery [50]. It is clear from the evidence that a clearer understanding of the role of preprints in the process of publishing research findings is needed [55]. Possible coordination among preprint servers for screening research and opportunities to work with funding organisations and researchers to increase their longer-term functionality [23, 66]. There is acceptance that preprints offer a way to present early findings of research into the public domain, openly and transparently, which can therefore help to foster awareness and reduce misinformation of health and social care research findings to the general public [112, 117]. The role of preprints is merely part of the process to reaching a peer reviewed publication. Preprints are therefore not a substitute for peer reviewed journal publications (and not all peer reviewed publications are published in a journal, and peer review can happen outside the context of journals such as Peer Community in [118]) but form part of the overall system and process of disseminating research rapidly using openly accessible routes to advance scientific discovery [12, 23].

Any changes to the processes and mechanisms of publishing research needs to ensure it meets current expectations of different audiences, and that research communities embrace the principles of quality, transparency and openness for the provision of innovation and new knowledge [31].

Supporting information

S1 Table. Scopus and Web of Science searches.

https://doi.org/10.1371/journal.pone.0291627.s003

(DOCX)

S2 Table. Full details of the 98 included studies.

https://doi.org/10.1371/journal.pone.0291627.s004

(DOCX)

S3 Table. Publishers and Journals preprints policy or guidance.

https://doi.org/10.1371/journal.pone.0291627.s005

(DOCX)

S4 Table. Funding organisations position on preprints for grant applications and publications.

https://doi.org/10.1371/journal.pone.0291627.s006

(DOCX)

S5 Table. Future considerations and recommendations for publishers, funding organisations and the research community.

https://doi.org/10.1371/journal.pone.0291627.s007

(DOCX)

Acknowledgments

We would like to thank the NIHR Journals Library centre staff and those members of the NIHR Open Research Oversight Group for their support during the development of the scoping review and throughout the write up of the article. We would also like to thank the NIHR Research on Research team for reviewing early draft versions of the article.

References

  1. 1. Sever R, Eisen M, Inglis J. Plan u: Universal access to scientific and medical research via funder preprint mandates. PloS Biol. 2019;17(6). pmid:31163026
  2. 2. Council COPE. COPE Discussion document: Preprints. March 2018.
  3. 3. Pearse RM, Ackland GL, Asai T, Hemmings HC Jr. Preprints in perioperative medicine: immediacy for the greater good. Br J Anaesth. 2021;126(5):915–8. pmid:33795134
  4. 4. Rodríguez EG. Preprints and preprint servers as academic communication tools. Revista Cubana de Informacion en Ciencias de la Salud. 2019;30(1):e1324–e.
  5. 5. Kaiser DE, Oliveira Crossetti MdG. Open Science and the emergence of preprints. Rev Gaucha Enferm. 2021;42:e20210030. pmid:33852673
  6. 6. Smart P. The Evolution, Benefits, and Challenges Of Preprints and their Interaction with Journals. Sci Ed. 2022;9(1):79–84.
  7. 7. Grant S, Wendt KE, Leadbeater BJ, Supplee LH, Mayo-Wilson E, Gardner F, et al. Transparent, Open, and Reproducible Prevention Science. Prev Sci. 2022;23(5):701–22. pmid:35175501
  8. 8. Koerber A. Is It Fake News or Is It Open Science? Science Communication in the COVID-19 Pandemic. J Bus Tech Commun. 2021;35(1):22–7.
  9. 9. Soderberg CK, Errington TM, Nosek BA. Credibility of preprints: an interdisciplinary survey of researchers. Royal Society Open Science. 2020;7(10). pmid:33204484
  10. 10. Fraser N, Brierley L, Dey G, Polka JK, Pálfy M, Nanni F, et al. The evolving role of preprints in the dissemination of COVID-19 research and their impact on the science communication landscape. PloS Biol. 2021;19(4):e3000959. pmid:33798194
  11. 11. Teixeira da Silva JA. Preprints: Ethical hazard or academic liberation? Kome. 2017;5(2):73–80.
  12. 12. Kaiser J. The preprint dilemma. Science. 2017;357(6358):1344–9. pmid:28963238
  13. 13. Baffy G, Burns MM, Hoffmann B, Ramani S, Sabharwal S, Borus JF, et al. Scientific Authors in a Changing World of Scholarly Communication: What Does the Future Hold? Am J Med. 2020;133(1):26–31. pmid:31419421
  14. 14. Teixeira da Silva JA. A Synthesis of the Formats for Correcting Erroneous and Fraudulent Academic Literature, and Associated Challenges. J General Philos Sci. 2022. pmid:35669840
  15. 15. Teixeira da Silva JA. The preprint debate: What are the issues? Medical journal, Armed Forces India. 2018;74(2):162–4. pmid:29692484
  16. 16. Callaway E. Heavyweight funders back central site for life-sciences preprints. Nature. 2017;542(7641):283–4. pmid:28202994
  17. 17. Pollock D, Davies EL, Peters MDJ, Tricco AC, Alexander L, McInerney P, et al. Undertaking a scoping review: A practical guide for nursing and midwifery students, clinicians, researchers, and academics. Journal of Advanced Nursing. 2021;77(4):2102–13. pmid:33543511
  18. 18. Arksey H, O’Malley L. Scoping studies: towards a methodological framework. International journal of social research methodology. 2005;8(1):19–32.
  19. 19. Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med. 2018;169(7):467–73. pmid:30178033
  20. 20. JBI. Scoping Review Network: https://jbi.global/scoping-review-network Last accessed 17/02/2023.
  21. 21. Peters MDJ, Marnie C, Tricco AC, Pollock D, Munn Z, Alexander L, et al. Updated methodological guidance for the conduct of scoping reviews. JBI Evidence Implementation. 2021;19(1):3–10. pmid:33570328
  22. 22. Munn Z, Pollock D, Khalil H, Alexander L, Mclnerney P, Godfrey CM, et al. What are scoping reviews? Providing a formal definition of scoping reviews as a type of evidence synthesis. JBI Evidence Synthesis. 2022;20(4):950–2. pmid:35249995
  23. 23. Subramanian K, Nalli A, Senthil V, Bhat A. Pharmaceutical industry–authored preprints: scientific and social media impact. Curr Med Res Opin. 2021;37(2):269–73. pmid:33206000
  24. 24. Watson C. Rise of the preprint: how rapid data sharing during COVID-19 has changed science forever. Nature Medicine. 2022;28(1):2–5. pmid:35031791
  25. 25. Brierley L, Nanni F, Polka JK, Dey G, Pálfy M, Fraser N, et al. Preprints in motion: tracking changes between preprint posting and journal publication during a pandemic. bioRxiv. 2021:2021.02.20.432090.
  26. 26. Flanagin A, Fontanarosa PB, Bauchner H. Preprints Involving Medical Research—Do the Benefits Outweigh the Challenges? JAMA. 2020;324(18):1840–3. pmid:33170226
  27. 27. Hurst P, Greaves S. COVID-19 Rapid Review cross-publisher initiative: What we have learned and what we are going to do next. Learn Publ. 2021;34(3):450–3. pmid:34230774
  28. 28. Lee AYS, Lin MW. Rapid publishing in the era of coronavirus disease 2019 (COVID-19). Medical Journal of Australia. 2020;212(11). pmid:32474929
  29. 29. Machin-Mastromatteo JD, Tarango J, Romo-González JR. Impact of the COVID-19 pandemic on scientific production. Libraries, Digital Information, and COVID: Practical Applications and Approaches to Challenge and Change: Elsevier; 2021. p. 183–98.
  30. 30. Jui-An Pan S. The Publication Ethics of Preprints and Preprints’ Influence on Knowledge Dissemination during the COVID-19 Pandemic. J Educ Media Libr Sci. 2022;59(1):35–71.
  31. 31. Peretz F, Bonini-Vuillod J, Grivaux M, Duracinsky M, Chassany O. COVID-19 and medical publications: How three articles have influenced the media and public decisions in France. Revue De Medecine Interne. 2021;42(8):583–90.
  32. 32. Raynaud M, Zhang H, Louis K, Goutaudier V, Wang J, Dubourg Q, et al. COVID-19-related medical research: a meta-research and critical appraisal. Bmc Medical Research Methodology. 2021;21(1). pmid:33397292
  33. 33. Strcic J, Civljak A, Glozinic T, Pacheco RL, Brkovic T, Puljak L. Open data and data sharing in articles about COVID-19 published in preprint servers medRxiv and bioRxiv. Scientometrics. 2022;127(5):2791–802. pmid:35370324
  34. 34. Teixeira da Silva JA. Silently withdrawn or retracted preprints related to Covid-19 are a scholarly threat and a potential public health risk: theoretical arguments and suggested recommendations. Online Info Rev. 2020;45(4):751–7.
  35. 35. Teixeira da Silva JA. Adjusting the use of preprints to accommodate the ’quality’ factor in response to COVID-19. Journal of Taibah University Medical Sciences. 2021;16(4):477–81. pmid:34408603
  36. 36. Wang Y, Cao Z, Zeng DD, Zhang Q, Luo T. The collective wisdom in the COVID-19 research: Comparison and synthesis of epidemiological parameter estimates in preprints and peer-reviewed articles. International Journal of Infectious Diseases. 2021;104:1–6. pmid:33352327
  37. 37. Weissgerber T, Riedel N, Kilicoglu H, Labbe C, Eckmann P, ter Riet G, et al. Automated screening of COVID-19 preprints: can we help authors to improve transparency and reproducibility? Nature Medicine. 2021;27(1):6–7. pmid:33432174
  38. 38. Gupta L, Gasparyan AY, Misra DP, Agarwal V, Zimba O, Yessirkepov M. Information and misinformation on COVID-19: A cross-sectional survey study. J Korean Med Sci. 2020a;35(27). pmid:32657090
  39. 39. Gupta L, Gasparyan AY, Zimba O, Misra DP. Scholarly publishing and journal targeting in the time of the Coronavirus Disease 2019 (COVID-19) pandemic: a cross-sectional survey of rheumatologists and other specialists. Rheumatol Int. 2020;40(12):2023–30. pmid:33048199
  40. 40. Barbour V. Openness, Integrity, Inclusion, and Innovation in Scholarly Communication: Competing or Complementary Forces? Frontiers in research metrics and analytics. 2021;6:767869–. pmid:35047723
  41. 41. Fry NK, Marshall H, Mellins-Cohen T. In praise of preprints. Microbial Genomics. 2019;5(4).
  42. 42. Itani D, Lababidi G, Itani R, El Ghoul T, Hamade L, Hijazi ARA, et al. Reporting of funding and conflicts of interest improved from preprints to peer-reviewed publications of biomedical research. J Clin Epidemiol. 2022;149:146–53. pmid:35738307
  43. 43. Teixeira da Silva JA, Dobránszki J. Preprint policies among 14 academic publishers. The Journal of Academic Librarianship. 2019;45(2):162–70.
  44. 44. Kirkham J, Moher D. Who and why do researchers opt to publish in post-publication peer review platforms?—findings from a review and survey of F1000 Research [version 1; referees: 2 approved, 1 approved with reservations]. F1000Research. 2018;7.
  45. 45. Kwon D. Open-access journal eLife announces ’preprint first’ publishing model. Nature. 2020. pmid:33319829
  46. 46. Leopold SS, Haddad FS, Sandell LJ, Swiontkowski M. Clinical Orthopaedics and Related Research, The Bone & Joint Journal, The Journal of Orthopaedic Research, and The Journal of Bone and Joint Surgery will not accept clinical research manuscripts previously posted to preprint servers. J Orthop Res. 2019;37(1):8–11.
  47. 47. Tennant JP, Crane H, Crick T, Davila J, Enkhbayar A, Havemann J, et al. Ten hot topics around scholarly publishing. Publ. 2019;7(2).
  48. 48. Musunuri S, Sandbrink JB, Monrad JT, Palmer MJ, Koblentz GD. Rapid Proliferation of Pandemic Research: Implications for Dual-Use Risks. mBio. 2021;12(5). pmid:34663091
  49. 49. Smyth AR, Rawlinson C, Jenkins G. Preprint servers: a ‘rush to publish’ or ‘just in time delivery’ for science? Thorax. 2020;75(7):532–3. pmid:32312737
  50. 50. Verma AA, Detsky AS. Preprints: a Timely Counterbalance for Big Data–Driven Research. J Gen Intern Med. 2020;35(7):2179–81. pmid:32141042
  51. 51. Sopinka NM, Coristine LE, DeRosa MC, Rochman CM, Owens BL, Cooke SJ. Envisioning the scientific paper of the future. Facet. 2020;5(1):1–16.
  52. 52. Bozzato V, Gnoato M, Vilia A, Apostolico M. Intellectual property protection for scholarly publishing in the Italian framework: a globally open approach for medical and life sciences authors. Italian Journal of Medicine. 2021;15(2):77–84.
  53. 53. Sarabipour S, Debat HJ, Emmott E, Burgess SJ, Schwessinger B, Hensel Z. On the value of preprints: An early career researcher perspective. PloS Biol. 2019;17(2). pmid:30789895
  54. 54. Yi HJ, Huh S. Korean editors’ and researchers’ experiences with preprints and attitudes towards preprint policies. Sci Ed. 2021;8(1):4–9.
  55. 55. Lombardi S, editor Every document is born "grey"-Some documents can become "open. 22nd International Conference on Grey Literature: Applications of Grey Literature for Science and Society, GL 2020; 2020: TextRelease.
  56. 56. Bernard R, Weissgerber TL, Bobrov E, Winham SJ, Dirnagl U, Riedel N. fiddle: a tool to combat publication bias by getting research out of the file drawer and into the scientific community. Clinical Science. 2020;134(20):2729–39. pmid:33111948
  57. 57. Klebel T, Reichmann S, Polka J, McDowell G, Penfold N, Hindle S, et al. Peer review and preprint policies are unclear at most major journals. PLoS ONE. 2020;15(10 October). pmid:33085678
  58. 58. Verma IM. Preprint servers facilitate scientific discourse. Proc Natl Acad Sci U S A. 2017;114(48):12630. pmid:29078419
  59. 59. McBee MT, Makel MC, Peters SJ, Matthews MS. A Call for Open Science in Giftedness Research. Gifted Child Q. 2018;62(4):374–88.
  60. 60. Nishioka C, Färber M, Saier T, editors. How does author affiliation affect preprint citation count? Analyzing citation bias at the institution and country level. 22nd ACM/IEEE Joint Conference on Digital Libraries, JCDL 2022; 2022: Institute of Electrical and Electronics Engineers Inc.
  61. 61. Bell SC, Flume PA, Castellani C. Seven P’s of publication practices. Journal of Cystic Fibrosis. 2020;19(3):333–5. pmid:32245683
  62. 62. Chiarelli A, Johnson R, Pinfield S, Richens E. Preprints and scholarly communication: An exploratory qualitative study of adoption, practices, drivers and barriers. F1000Research. 2019;8.
  63. 63. Vuong QH. Plan S, self-publishing, and addressing unreasonable risks of society publishing. Learn Publ. 2020;33(1):64–8.
  64. 64. Kleinert S, Horton R, Editors of the Lancet family of journals. Preprints with The Lancet: joining online research discussion platforms. Lancet. 2018;391(10139):2482–3.
  65. 65. Fleming JI, Cook BG. Open Access in Special Education: A Review of Journal and Publisher Policies. Remedial Spec Educ. 2022;43(1):3–14.
  66. 66. Smith JA, Sandbrink JB. Biosecurity in an age of open science. PloS Biol. 2022;20(4).
  67. 67. Alwine JC, Enquist LW, Dermody TS, Goodrum F. What is the price of science? mBio. 2021;12(2):1–3. pmid:33653889
  68. 68. Kullmann DM. Editorial. Brain. 2017;140(7):1817. pmid:29177499
  69. 69. Schapira M, The Open Lab Notebook Consortium, Harding RJ. Open laboratory notebooks: good for science, good for society, good for scientists. F1000Research. 2019;8:87–. pmid:31448096
  70. 70. Bourne PE, Polka JK, Vale RD, Kiley R. Ten simple rules to consider regarding preprint submission. Public Library of Science San Francisco, CA USA; 2017. p. e1005473. pmid:28472041
  71. 71. Communications Nature. Preprints under peer review. Nature Communications. 2017;8.
  72. 72. Eglen SJ, Mounce R, Gatto L, Currie AM, Nobis Y. Recent developments in scholarly publishing to improve research practices in the life sciences. Emerg Topics Life Sci. 2018;2(6):775–8. pmid:33530668
  73. 73. Poremski D, Falissard B, Fegert J, Witt A, Ordóñez AE, Martin A, et al. Moving from ‘personal communication’ to ‘available online at’: preprint servers enhance the timeliness of scientific exchange. Child and Adolescent Psychiatry and Mental Health. 2019;13(1):42. pmid:31695746
  74. 74. Choi YJ, Choi HW, Kim S. Preprint acceptance policies of Asian academic society journals in 2020. Sci Ed. 2021;8(1):10–7.
  75. 75. Brainard J. Do preprints improve with peer review? A little, one study suggests. Science. 2020.
  76. 76. Ferreira CM, Serpa S. The importance of preprint in scientific publication: Perspectives and challenges. J Soc Sci Res. 2018;4(12):642–7.
  77. 77. Carà PD, Ciriminna R, Pagliaro M. Has the Time Come for Preprints in Chemistry? ACS Omega. 2017;2(11):7923–8. pmid:31457346
  78. 78. Polka J. Preprints as a complement to the journal system in biology. Inf Serv Use. 2017;37(3):277–80.
  79. 79. Howat AM, Clark J. Converting Access Microbiology to an open research platform: community survey results. Access microbiology. 2021;3(9):000272–. pmid:34712910
  80. 80. Kowalczyk OS, Lautarescu A, Blok E, Dall’Aglio L, Westwood SJ. What senior academics can do to support reproducible and open research: a short, three-step guide. BMC Research Notes. 2022;15(1):116. pmid:35317865
  81. 81. Vlasschaert C, Topf JM, Hiremath S. Proliferation of Papers and Preprints During the Coronavirus Disease 2019 Pandemic: Progress or Problems With Peer Review? Adv Chron Kidney Dis. 2020;27(5):418–26. pmid:33308508
  82. 82. Berg J. Preprint ecosystems. Science. 2017;357(6358):1331. pmid:28963229
  83. 83. Clayson PE, Baldwin SA, Larson MJ. The open access advantage for studies of human electrophysiology: Impact on citations and Altmetrics. Int J Psychophysiol. 2021;164:103–11. pmid:33774077
  84. 84. Mwangi KW, Mainye N, Ouso DO, Esoh K, Muraya AW, Mwangi CK, et al. Open Science in Kenya: Where Are We? Frontiers in research metrics and analytics. 2021;6:669675–. pmid:34056516
  85. 85. Price R, Ozkan Y. Characteristics of Imperial College London’s COVID-19 research outputs. Learn Publ. 2021;34(3):358–69. pmid:33821101
  86. 86. Avissar-Whiting M, Bakker C, Heckner H, Massip S, Schneider J, Townsend R, et al. Addressing disorder in scholarly communication: Strategies from NISO plus 2021. Inf Serv Use. 2021;41(1–2):107–21.
  87. 87. Serpa S, Sá MJ, Santos AI, Ferreira CM. Challenges for the academic editor in the scientific publication. Acad J Interdiscip Stud. 2020;9(3):12–6.
  88. 88. McGuinness LA, Sheppard AL. A descriptive analysis of the data availability statements accompanying medRxiv preprints and a comparison with their published counterparts. PLoS ONE. 2021;16(5). pmid:33983972
  89. 89. Ettinger CL, Sadanandappa MK, Görgülü K, Coghlan KL, Hallenbeck KK, Puebla I. A guide to preprinting for early-career researchers. Biol Open. 2022;11(7). pmid:35876380
  90. 90. Heyard R, Hottenrott H. The value of research funding for knowledge creation and dissemination: A study of SNSF Research Grants. Hum Soc Sci Comm. 2021;8(1).
  91. 91. Moshontz H, Binion G, Walton H, Brown BT, Syed M. A Guide to Posting and Managing Preprints. Adv Method Pract Psychol Sci. 2021;4(2).
  92. 92. Triggle CR, MacDonald R, Triggle DJ, Grierson D. Requiem for impact factors and high publication charges. Account Res. 2022;29(3):133–64. pmid:33787413
  93. 93. Chaleplioglou A, Koulouris A. Preprint paper platforms in the academic scholarly communication environment. J Librariansh Inf Sci. 2021.
  94. 94. Staines H, Martone ME. Community feedback on scholarly content: why it is important and why it should be preserved. Insights-the Uksg Journal. 2018;31.
  95. 95. Green T. Is open access affordable? Why current models do not work and why we need internet-era transformation of scholarly communications. Learn Publ. 2019;32(1):13–25.
  96. 96. Schloss PD. Preprinting microbiology. mBio. 2017;8(3). pmid:28536284
  97. 97. Bosman J, de Jonge H, Kramer B, Sondervan J. Advancing open access in the Netherlands after 2020: From quantity to quality. Insights UKSG J. 2021;34.
  98. 98. Ibragimova I, Phagava H. Editorial: Preprints and peer-reviewed journals. International Journal of Health Governance. 2022;27(3):237–9.
  99. 99. Bero L, Lawrence R, Leslie L, Chiu K, McDonald S, Page M, et al. Cross-sectional study of preprints and final journal publications from COVID-19 studies: discrepancies in results reporting and spin in interpretation. BMJ Open. 2021;11(7):e051821. pmid:34272226
  100. 100. Abbasi K. A giant step for science: JRSM welcomes preprints in medical science. J R Soc Med. 2018;111(8):263. pmid:30102103
  101. 101. Celebrating Eysenbach G. 20 years of open access and innovation at JMIR publications. J Med Internet Res. 2019;21(12).
  102. 102. George CH, Alexander SPH, Cirino G, Insel PA, Izzo AA, Ji Y, et al. Editorial policy regarding the citation of preprints in the British Journal of Pharmacology (BJP). Br J Pharmacol. 2021;178(18):3605–10. pmid:34328225
  103. 103. National Institute for Health and Care Research. NIHR Open Research: https://openresearch.nihr.ac.uk/about Last accessed 17/02/2023
  104. 104. ASAPbio. Preprint resource center: https://asapbio.org/preprint-info Last accessed 17/02/2023.
  105. 105. ASAPbio. Preprint authors optimistic about benefits: preliminary results from the #bioPreprints2020 survey: https://asapbio.org/biopreprints2020-survey-initial-results Last accessed 17/02/2023.
  106. 106. Wellcome. We now accept preprints in grant applications: https://wellcome.org/news/we-now-accept-preprints-grant-applications Last accessed 17/02/2023.
  107. 107. Kaiser J. NIH enables investigators to include draft preprints in grant proposals Science. 2017.
  108. 108. DORA. Declaration on Research Assessment (DORA): https://sfdora.org/ Last accessed 17/02/2023.
  109. 109. Contributor Roles Taxonomy. CRediT: https://credit.niso.org/ Last accessed 17/02/2023.
  110. 110. PlanS. What is cOAlition S?: https://www.coalition-s.org/about/ Last accessed 11/07/2023.
  111. 111. Carneiro CFD, Queiroz VGS, Moulin TC, Carvalho CAM, Haas CB, Rayêe D, et al. Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature. Research Integrity and Peer Review. 2020;5(1):16. pmid:33292815
  112. 112. Fleerackers A, Riedlinger M, Moorhead L, Ahmed R, Alperin JP. Communicating Scientific Uncertainty in an Age of COVID-19: An Investigation into the Use of Preprints by Digital Media Outlets. Health Communication. 2022;37(6):726–38. pmid:33390033
  113. 113. Klein M, Broadwell P, Farb SE, Grappone T. Comparing published scientific journal articles to their pre-print versions. International Journal on Digital Libraries. 2019;20(4):335–50.
  114. 114. Nicholson DN, Rubinetti V, Hu D, Thielk M, Hunter LE, Greene CS. Examining linguistic shifts between preprints and publications. PloS Biol. 2022;20(2):e3001470. pmid:35104289
  115. 115. El-Menyar A, Mekkodathil A, Asim M, Consunji R, Rizoli S, Abdel-Aziz Bahey A, et al. Publications and retracted articles of COVID-19 pharmacotherapy-related research: A systematic review. Science progress. 2021;104(2):00368504211016936. pmid:33989091
  116. 116. Peiperl L, on behalf of the Plos Medicine Editors. Preprints in medical research: Progress and principles. PLoS Med. 2018;15(4). pmid:29659580
  117. 117. Sumner P, Vivian-Griffiths S, Boivin J, Williams A, Venetis CA, Davies A, et al. The association between exaggeration in health related science news and academic press releases: retrospective observational study. BMJ: British Medical Journal. 2014;349:g7015. pmid:25498121
  118. 118. Peer Community in: https://peercommunityin.org/ Last accessed 30/08/2023.