Unbundling the functions of publication

Science progresses only at the rate at which we can share information with one another. But as any author of a journal article can attest, formal mechanisms of scholarly communication do not always work efficiently and can be subject to biases [1–3]. Peer review takes time: not merely for the reviewer to compile a thorough assessment but also for the editor to find reviewers. In the swiftest case, a manuscript is accepted at the first journal, and the process to eventual publication may take approximately four months [4,5]. However, given that many researchers continue to be evaluated based on the reputation of the journals in which their work is published, authors are incentivised to “aim high” when they select a journal, and it can take several rounds of review (at a single or multiple journals) before the work is approved for publication. It is commonplace for a manuscript to have been submitted to at least two journals on its way to publication, and as a result, the overall peer review process can take years [6].

The sooner a piece of work can be read, evaluated, and built upon, the faster science moves. And by including a greater diversity of thought in the process of science, higher quality final products emerge. Yet, although our system of publication has superficially transitioned from physical print magazines to online websites, the mechanisms and processes of scientific communication are not much faster or more inclusive than they were in the 19th century.

Perhaps the underlying cause for this stasis is the fact that our system of evaluating scientific work—whether for deciding what to read or to whom to award grants and jobs—relies heavily on the reputation of journal titles, and in turn, journal article output is factored into university ranking calculations [7]. Experimenting with new forms of sharing science that are incompatible with publication in traditional venues therefore carries career risks. In addition, many open-science practices (posting lab notebooks, sharing data sets, or conducting replication studies) require significant extra effort for researchers, which is currently not universally prioritised. Therefore, researchers need efficient mechanisms of sharing research that align with current publishing practices while supporting a gradual evolution toward more transparent and efficient communication practices. One small step is to simply share manuscripts publicly at the time they are ready to send to a journal, i.e., by posting a preprint. Preprints are versions of manuscripts made public (often on a preprint server) before the conclusion of a formal (often journal-organized) editorial process. Using preprints to separate in-depth review from the initial act of sharing can increase efficiency while requiring minimal extra work for authors and presenting science in a format that is easily recognized by readers.

Here, we distill what we’ve learned from our work listening to concerns about, and investigating issues surrounding, preprints. We summarize the current state of support for preprinting in the life sciences, discuss extant needs and challenges, and put forth ideas for future developments.

Why now?

Posting preprints is standard practice in many fields in physics, mathematics, computer science, economics, and other disciplines. Preprints are only now becoming widespread in the life sciences (Table 1), despite a long history of sincere efforts to establish servers in biology by both public and private sectors dating back to the 1960s [6]. Why have they taken off in biology since 2015? We suspect that at least four factors have contributed.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Developments in preprinting across biomedical and life sciences since May 2018.

https://doi.org/10.1371/journal.pgen.1008565.t001

First, in today’s digital world, the idea of composing a manuscript in real-time using collaborative editing tools only to not share it with the community seems increasingly anachronistic.

Second, bioRxiv was positioned effectively within the existing publishing paradigm from the start. Founded by veterans of the publishing industry, John Inglis and Richard Sever, bioRxiv quickly established partnerships with a number of journals. These journals not only agreed to consider manuscripts posted as preprints but also established a direct submission pipeline enabling authors to submit to both with one click. Furthermore, perhaps driven by a competitive publishing environment, editors began to invite submission of manuscripts from preprint servers (discussed below). Preprints now represent an opportunity to publishers, in which previous efforts to share science in this way may have been seen as a commercial threat. Direct submission arrangements and anecdotes about manuscript recruitment offered researchers confidence that posting preprints would not endanger their chances of journal publication. Furthermore, the ownership of bioRxiv by Cold Spring Harbor Laboratory, a credible, nonprofit research institute, likely contributed to its resonance with the community of authors and readers.

Third, many funders have since provided active support and recognition for preprints. Although the NIH has been involved in preprinting through the Information Exchange Groups of the 1960s and Harold Varmus’s 1999 eBioMed proposal [6,37], only recently have many funders voiced support for preprints as a mechanism for applicants and grantees to demonstrate productivity. We discuss these policies in detail below.

Fourth, Twitter created a community that provided visibility to preprints and support to their authors [38]. All of the benefits of preprinting (including discussion, collaboration, visibility, and earlier disclosure) rely on active acknowledgment of preprints by the authors’ community. At the early stages of any movement, adopters will be relatively far and few between, limiting their ability to support one another. Twitter has allowed preprint enthusiasts to connect with one another across institutional boundaries, meaning that even a small number of early adopters can reap the benefits of increased exposure and feedback for their work by sharing preprints with one another.

Preprints in harmony with journals

In 1966, a cabal of journal editors “outlawed” Information Exchange Groups (the NIH’s photocopy and mail-based preprint exchange platform), fearing that preprints would damage their business model [6]. A representative of the American Association of Immunologists wrote that “Since the preprints are complete publications, there is a real danger that they will reduce the usefulness of existing journals in the field of Immunology and may ultimately supersede them” [39]. Indeed, reports that papers change little between their preprint version to the final published version have caused some to declare that preprints can be the end of the story [40]. Despite the irony that the article reporting this similarity added a section on bioRxiv before its publication in a journal, the more serious issue is that textual analysis may not accurately capture significant changes in meaning. And there is value in evaluation even if the manuscript stays exactly the same: peer review can provide validation as well as improvements.

Perhaps for these reasons, authors continue to use journals even in fields in which preprinting has long been common practice. For example, in physics, 73% of papers on the arXiv can be matched to an article that appears in a journal indexed by Web of Science [41]. Although bioRxiv is younger, the number is similar (67%, [42]), suggesting that neither archive is massively disrupting the journal business.

In fact, preprints are very much complementary to journals, and they offer several tangible benefits for editors and publishers. First, although there is no evidence that the relationship is causal, papers that have been preprinted garner more attention over time [43,44,45]. Preprints allow authors to receive feedback from a broader range of scientists than could be engaged in a typical peer review process. Although a small amount of this feedback appears in the commenting section [46], the majority is communicated elsewhere. For example, a survey of bioRxiv users found that over 40% of authors get feedback via social media, and private feedback from emails and other correspondence with colleagues is nearly as common [32]. In cases in which community feedback on a preprint is incorporated before or during revisions in a journal peer-review process (as we have done for this paper), the version of the paper that is ultimately accepted by the journal will have undergone more scrutiny, likely leading to a higher quality final product. Although the attention each individual paper garners may decline as more and more manuscripts are preprinted (note, for example, that median downloads per bioRxiv preprint in the first month peaked in 2016 [42]), we expect this form of feedback to continue, especially for highly interesting, time-critical, or controversial work that is in the greatest need of additional scrutiny (e.g., see Outbreak Science [47]).

Furthermore, preprints offer an efficient marketplace for papers [48]. Although many editors travel to conferences to invite submission of future manuscripts based on interesting presentations, preprint servers make the manuscripts themselves open to review by anyone in the world. This can enable editors to curate from a wider pool: all preprinted papers, rather than just those actively submitted to their journal. Therefore, it is no surprise that the practice of inviting journal submissions from preprint servers seems to be widespread [49]. PLOS Genetics has pioneered the formalization of this process with preprint editors [50] and Proc B has adopted the practice as well [51]. Unfortunately, many such invitations may be moot because it is common practice for authors to post the preprint version concurrently with submission to a journal, a process that is facilitated by integrations in both journal and bioRxiv submission systems [27,52]. In order to allow this marketplace of submission invitations to function efficiently, authors can post their preprint a few weeks before journal submission and allow their work to recruit feedback, attention, and editorial invitations. Doing so could help save both authors’ and editors’ time along the way.

Finally, preprints relieve pressure on journals. Authors generally would like their papers to be published as soon as possible, leading some journals to promise shorter peer review turnaround times, perhaps at the expense of allowing reviewers to be as thorough as they would like to be [53]. If authors can instead share a preprint immediately, they are likely to feel more comfortable waiting a bit longer for high-quality, journal-organized peer review.

Journal policies explicitly permitting or even encouraging preprinting have removed much lingering fear of rejection due to prior publication conflicts. Even some long-standing holdouts, notably Cell Press, JACS, and the American Association for Cancer Research [54] have updated their policies to be friendlier to preprints. A full list of basic journal policies on preprint archiving can be found at SHERPA/RoMEO [55], more informal lists can be found at Wikipedia [56], and detailed policies on preprint version, licensing, and media coverage policies can be found in Transpose [57].

Institutional and funder support

Preprints allow researchers to demonstrate their most recent work to prospective and current funders. It is becoming less acceptable to cite work that is “in submission” or “under review” in grant applications: when a manuscript is prepared, reviewers wish to see it and may request the applicant cites a preprinted version [58]. Practically, preprints allow reviewers to judge applicants for funding or promotion by the rigor of their latest science.

In comparison to journals, university policies for the assessment of applications for hiring, promotion, and tenure seem slower to change [59], but there have been bright spots for preprints. For example, in late 2016, NYU Langone Medical Center added language to their promotion and tenure guides to include preprints as a potential research output, and in early 2018, UC Davis added a “preprints” category to the their online faculty evaluation database [60]. UT Austin, The Rockefeller University, and UC Santa Cruz have all added language inviting job applicants for faculty positions to submit preprints as well [60]. Furthermore, preprints may hold value beyond what is codified in formal policies. A survey of hiring committee members conducted by “Future PI Slack” suggests that 10/15 of those surveyed find preprints useful in evaluating faculty candidates [61].

Perhaps the most proactive support for preprints has come from funders, who seemed poised to actively encourage the use of preprints in the life sciences. In May of 2016, the Simons Foundation Autism Research Initiative (SFARI) announced it would change its grant award letter to “strongly encourage” investigators to post preprints and that such papers would be taken into consideration in funding decisions [62]. On September 1 of the same year, these concepts became integrated into the overall Simons Foundation policy, and other funders followed suit, including The Leona M. and Harry B. Helmsley Charitable Trust, EMBO Long-Term fellowships and Young Investigator program, Human Frontiers Science Program, MRC, Wellcome Trust, HHMI, Cancer Research UK, BBSRC, UKRI Future Leaders Fellowship program, CNRS, and the European Research Council [63].

One influential funder policy has been NIH’s guide notice NOT-OD-17-050, which clarifies the NIH’s position on preprints and other interim research products: “The NIH encourages investigators to use interim research products, such as preprints, to speed the dissemination and enhance the rigor of their work … Interim research products can be cited anywhere other research products are cited” [36]. A notable exception, however, is in the use of preprints in post-submission materials [64], which are intended to accommodate events outside the control of the investigators.

Some private funders have gone beyond encouraging preprints to requiring them. Barring privacy concerns, the Chan Zuckerberg Initiative states a commitment to posting preprints prior to peer review [65]. As part of Wellcome’s updated open access policy, researchers working on fields of public health relevance will be required to preprint at the time of journal submission from 2020 [16].

As with all policies, their existence does not ensure they will be enacted. Funders also must develop mechanisms to monitor grantee reaction and compliance. The emergence of technological infrastructure (e.g., links between preprints and published papers, metadata about funding sources, and submission and posting dates), as well as continued dialogue between researchers and funders, is key to enabling these policies.

Some have argued that preprints should be used by funders to achieve open access to the literature in lieu of mandates for open access journal versions [66], but we believe this suggestion is not viable. Because peer review changes papers (often for the better) such a plan would create a two-tier system with some readers having privileged, paywalled access to the more definitive journal versions of papers, and the remainder left with access to other versions that might be outdated. Plan U could become an acceptable substitute for universal open access only if all journals and preprint servers permitted archiving postprints (i.e., versions of manuscripts accepted for journal publication) and/or all submitted version of manuscripts. Currently, bioRxiv does not allow preprint publication after acceptance, and many journals prohibit the publication of preprints that incorporate comments from the peer review process [67].

Technical issues

At present, preprint servers lack the technological infrastructure that could help them to realize their full potential. Addressing such challenges could make a large impact on how preprints are used and discovered.

For example, authors who have previously read a preprint often wish to quickly find out how it has changed upon the posting or publication of a subsequent version. Currently, neither preprint servers nor journals systematically present a summary of the changes made. Some users already make version notes when posting a revised manuscript to bioRxiv; making this more standard practice might involve enabling authors to submit a short piece of text to journals as well, similar to a conflict of interest disclosure or author CRediT declaration. Once this is complete, it would be natural for journals to provide a link back to the preprint version, which would present a more complete picture of how a manuscript evolved over time. Some journals already provide this backwards link—including Biophysical Journal, Plant Direct, and PLOS titles [68–70]. Formal inclusion of preprint information in the XML representing the manuscript’s version history would help: this recommendation for the JATS schema is currently pending decision [71]. Alternatively, researchers may wish to include a link to their own preprint in the final version (as we do here), where the journal policy permits this. Preprints could also be better supported by reference managers with features that would allow users to link preprints to later versions (whether revised preprints or a final journal version) and receive updates when subsequent versions are available online.

Change is needed in search tools, too. For example, preprints could be linked from PubMed and PubMed Central. (Note that this is effectively being done for papers in F1000 Research and associated platforms such as Wellcome Open Research. Once these papers pass peer review, they appear on PubMed Central along with their date-stamped first version.) This helps to establish a record of what work was done when, irrespective of delays imposed by the peer review process, which is key to determining priority of discovery. Europe PMC already indexes preprints and has implemented links between the preprint and published version of the same piece of work, though improved metadata could facilitate further search and tool development [11,72].

Beyond the basic metadata about a preprint, open access to the data detailing interactions with each preprint would enable innovation around how the latest science is discussed. For a recent effort to understand Twitter interactions with and downloads of preprints posted on bioRxiv, content metadata was derived by scraping the bioRxiv website [42]. In the absence of an official bioRxiv application programming interface (API), these authors and others had developed their own tools (including an API, command line tool and Python wrapper) to source and interact with bioRxiv content data. Since the publication of the analysis, bioRxiv released its own API [73].

Addressing the technical issues detailed above may help more people find and interact with preprints. As we will discuss in the next section, the low discoverability and perceived legitimacy of preprints is at the root of several more complex social problems.

The future of preprints

Although the rate of preprints appearing on bioRxiv is increasing exponentially, they still comprise less than 3% of the monthly output of PubMed [94]. Because the majority of the potential growth in the use of preprints lies ahead, research communities must ensure that such use is productive to both science and scientists.

Acknowledgments

We thank Anna Hatch, Daniela Saderi, Kristen Ratan, Maria Levchenko, Michael Parkin, Benjamin Schwessinger, Rich Abdill, Donald Forsdyke, Mario Malički, anonymous reviewers, and Tom Sheldon for helpful feedback on the manuscript.