Introduction

Reproducibility is crucial to the progress and impact of research and innovation, as it confirms or corrects the results of studies, resulting in more reliable outcomes and a reduction in research waste [1–3]. The reproducibility of scientific results has become a proxy term for many desirable attributes of science, including reliability, replicability, and efficiency, although there is still a certain degree of variation in its definition (Box 1). Therefore, considering reproducibility in the design and conduct of research should be a precondition to establishing the quality of research. Replication studies that aim to reproduce a study or experiment are scarce, the definition of a successfully reproduced study is ambiguous, and the empirical reproducibility rate is still overall modest [4,5]. To further complicate the scenario of defining and measuring reproducibility, a recent scoping review identified 50 different metrics used to quantify, assess, explain or predict reproducibility, reflecting considerable diversity in both their nature and the specific aspects of reproducibility they address [6].

There are several reasons for the lack of reproducibility, such as technical aspects (e.g., insufficient data collection, poorly described methods, or lack of transparent reporting) and barriers in the culture and ethics of research (e.g., the need for science to be continuously innovative, pressure to publish, and career evaluations based on quantity rather than quality, as well as a lack in funding for replication of studies) [9]. Some amount of non-replicability is normal in the scientific process, inherent in certain exploratory designs or because of expected variation in sampling leading to spurious results, even when methods are appropriate and robust. Studying complex systems with imperfect knowledge and tools and deliberate choices made by researchers may also increase the occurrence of non-replicable results [10]. More often, potential reproducibility of research and the actual replication of results are hampered by limitations in the study design, its conduct, and reporting.

Adhering to reproducibility standards and best practices enables researchers to better plan their studies and helps peer reviewers assess both proposed research projects and completed outputs. These tools range from general guidance outlining key steps to enhance good research and reproducibility before, during, and after analysis [11,12], to domain-specific checklists, such as those for molecular dynamics simulations [13], artificial intelligence studies [14], clinical metabolomics research [15], or World Bank research [16]. However, most of these initiatives remain fragmented across fields and are not developed within a unified framework that could bring together diverse disciplines sharing the need for common standards. Moreover, checklists often only address computational reproducibility [17], which implies obtaining the same computational results using the same input data, computational steps, methods, code, and analysis conditions [10].

Reproducibility checks are also part of journals’ instructions and editorial process, as transparent reporting is an essential component of reproducibility. Inevitably, the assessment of methodological rigor and reliability of results stands on complete and unbiased reporting, which can be enhanced by the adoption of reporting guidelines. For example, the ARRIVE checklist for reporting animal research aims to reduce poor reporting and limit irreproducible findings that can trigger inadequate clinical studies, with risks for participants and waste of resources [18,19]. More specific examples include the reporting standards and availability of data, materials, code, and protocols at Nature [20], the request to fill out a reproducibility checklist along with manuscript submission at the Journal of Artificial Intelligence Research [21], and the American Physiological Society Rigor And Reproducibility Checklist [22]. PLOS provides general guidance for reviewers and editors to assess whether the study methods report sufficient details to allow for their replicability or reproducibility, but without specifying which items should be checked [23]. However, although reproducibility checks introduced at the publication stage may support transparent reporting, they often come too late to resolve many reproducibility-related issues [24].

To address these challenges, the Open Science to Increase Reproducibility in Science (OSIRIS) project aims to develop and test tools that improve reproducibility at various levels [25]. Here, we present a consensus-based list of core items that outlines core expectations regarding planning, methods, data collection, management and analysis, and dissemination, with the aim of increasing research reproducibility.

The OSIRIS-Delphi study

Among the objectives of the OSIRIS project are the development and testing of tools to check compliance with reproducibility standards and best practice. Considering the increased interest in research transparency, reproducibility, and the variety of views and perspectives on how to monitor it by different interested parties, an online Delphi study was chosen as a simple and effective way to reach consensus on the core set of items to be included in a checklist for reproducibility of research projects [26,27]. The OSIRIS-Delphi study aimed at adopting a broad and global perspective, starting from a general concept of reproducibility (Box 1), that would be applicable at all stages, from planning to dissemination of a research project, and addressing the provision of sufficient details in methods to allow independent researchers to replicate an experiment and arrive at similar results.

Box 2 reports the key elements of the study methodology. Additional methodological details, including minor deviations from the protocol, are provided in the study protocol [28] and S1 File. Briefly, the Delphi study (two online surveys and one online meeting) led to the inclusion of 31 items out of the 44 items initially considered (S1 File). One item was split into two items after Round 1 (S2 File), resulting in 32 core items for reproducibility (Table 1 and S3 File). The international, multidisciplinary panel participating in this Delphi study identified these items as important for inclusion in a reproducibility checklist that would be applicable throughout the life cycle of a research project.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. The OSIRIS list of core items for reproducibility.

https://doi.org/10.1371/journal.pbio.3003726.t001

Consensus core items for reproducibility

The OSIRIS list of checks was developed as a consensus-based tool addressing different types of reproducibility and focusing on core items that are relevant to multiple disciplines. Since reproducibility can be influenced at various stages of the scientific process, the OSIRIS list of checks was designed in a structured way, capturing distinct—though interrelated—considerations that reflect the natural progression of a research project. It is a comprehensive tool designed to highlight practices in study planning, design, analysis, and dissemination that potentially affect the reproducibility of research projects. It includes field-independent elements that are broadly, though not entirely, discipline-agnostic and applicable across research areas, distinguishes it from other available checklists. The list is intended to define the minimum requirements needed to ensure reproducibility throughout all stages of a research project’s lifecycle. To enhance usability, the checklist is divided into four sections, with related practices grouped to help researchers and other stakeholders systematically identify and address reproducibility-related issues across the different stages of a project.

Challenges and future directions for the OSIRIS list of core items

Reaching consensus on a list of core items for reproducibility is an important step for future efforts in the field. However, it is important to recognize how our study characteristics may have affected the results and to what extent this tool can be adopted broadly by different users.

Although we adopted a broad and global perspective, both the development of the list of items and the process of identification and invitation of the members of the panel were inevitably influenced by the OSIRIS-Delphi Steering Committee perspective. S1 File reports the characteristics of the participants at each stage. At Round 1, 82 participants from 21 countries responded. About two-third were researchers, with a good case mix in terms of career stages. Half of the participants defined themselves as working in life sciences, but social sciences and humanities, physical sciences, and engineering were also represented. Researchers from European countries were overrepresented. These factors could have influenced the decisions to include some of the items, and we cannot exclude that different scientific communities would have selected different core items for reproducibility. However, the use of the Delphi approach assumes that consensus is a reasonable expectation, therefore some level of common interpretation of the issues under consideration was required.

The OSIRIS Delphi group was actively engaged, as denoted by the high response rate (91% at Round 1 and 93% at Round 2; Fig 1) and comments received (67% provided at least one comment in Round 1, 22% in Round 2). Comments received after Round 1 were not distributed before Round 2, but only before the consensus meeting; this may have limited the feedback during consensus process and was considered a weakness by some of the participants.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Summary of the level of consensus for each item at Rounds 1, 2, and final.

Round 1: blue and green vertical bands correspond to the pre-defined level of agreement for exclusion (1–3) and inclusion (7–9); the violin plot (horizontal feature) corresponds to the density of answers around median values. Circles report the level of consensus, i.e., 80% of responses being in the upper third (7–9), with green = inclusion; yellow = move to Round 2. Round 2: green = inclusion; yellow = move to the consensus meeting; purple = no skills to answer. Final (after the consensus meeting): green = inclusion; yellow = no consensus; blue = exclude. The full survey at Round 1 and its revisions, a short report of the consensus meeting, anonymized data, metadata, and the statistical code are openly available on the Open Science Framework (https://osf.io/z7xmy/files).

https://doi.org/10.1371/journal.pbio.3003726.g001

Sixteen panel members attended the online consensus meeting aimed at discussing the items that did not reach consensus and general themes that emerged from Round 1 and 2. Most of the comments received during the Delphi study focused on the item applicability and other general aspects about implementation. The debate on the applicability of some items to different stages of the research process, scientific field, and type (explorative, descriptive, explanatory, etc.) reflected how variable the interpretation of reproducibility can be and the set of accepted norms in each field. We recognized a degree of epistemic diversity (the systematic differences in fundamental concepts, problem formulation, empirical objects, methodological practices, and modes of judgments) across the different discipline represented [36]. From the Delphi study, the need for an implementation strategy to maximize uptake and use of these core items for reproducibility and make them an actionable and effective tool for checking reproducibility clearly emerged.

Implementation requires user testing across different research communities, as the concrete uptake of core reproducibility items depends on their interpretation and relevance for a specific stakeholder, research stage, or discipline. A collaborative pilot study that is currently underway aims to explore how researchers from diverse fields use and apply the core items recommended by the Delphi panel [37]. The first step of the pilot was a project conducted at the Responsible Research in Action Unconference (September 2025, Berlin). Researchers from various backgrounds were invited to apply the list of items to published papers in their disciplines and assess both feasibility and potential areas for adaptation. Participants appreciated the availability of a comprehensive checklist for reproducibility but reported challenges in interpreting some items—particularly those related to the planning phase, where possible overlap was noted among concepts such as study hypotheses, rationale and prior evidence, research questions, and objectives. They also observed that certain items might not be equally applicable across all types of research.

Participants highlighted that some refinements may be desirable when end-users wish to apply the checklist in their specific context. For instance, assessing whether authors “tracked and reported deviation(s) from the planned design” (item 24) would be more straightforward if pre-registration were in place—an item that did not reach consensus in the Delphi process. In the absence of pre-registration, this assessment necessarily relies on trust. They also provided valuable suggestions for revising the wording of some items that were heavily oriented toward experimental and quantitative science. A detailed description of the revised list of checks, incorporating this preliminary feedback, is available in S3 File.

The final output of the pilot phase will be an elaboration and explanation guidance to support the use of the list of checks, along with concrete examples of ideal practices. Recommendations for adapting the checks to specific disciplines and research types will also be provided, recognizing that the Delphi-derived core items were intentionally designed to address field-independent aspects of reproducibility, and that any attempt to define disciplinary agnosticism inevitably faces constraints given the diversity of scientific approaches.

Additional implementation efforts could involve editors and peer reviewers implementing the checks included in the dissemination section to assess reproducible research practices in manuscripts and provide constructive feedback to authors. Funders may find items in the planning section the most applicable to their reproducibility assessment, although other sections may be helpful in monitoring project progress. For instance, for projects funded by the European Commission, open access practices (i.e., open access to research documents and data) are required and monitored according to recommended standards [38]. Some of these reproducibility items can be embedded in this set of standards.

Conclusions

Alongside political and economic factors, a lack of reproducibility and openness in science can undermine the trustworthiness of scientific findings and, more broadly, public confidence in the integrity of science. Tools that enable researchers and other stakeholders to enhance rigor, self-criticism, and self-correction should not be viewed as admissions of failure, but rather as mechanisms for continuous innovation [39,40].

In this Delphi study, open science and transparency across most stages of a project emerged as key drivers of reproducibility. Every research project involves a series of choices and trade-offs between rigor and feasibility. Systematically documenting decisions, assumptions, processes, data and tools—and making them visible and inspectable—can reduce undisclosed flexibility, implicit analytical steps, omissions, and dependence on tacit knowledge.

Several other checklists that have been developed in specific fields, such as molecular dynamics simulations [13], artificial intelligence [14], metabolomics [15], physiology [22], and cognitive and neuropsychological studies [41], include items concerning open science, mainly relating to data and code availability. Only the latter explicitly requires authors to state whether the study was pre-registered before data collection and analysis took place. Other tools limit this requirement to situations where they are applicable (e.g., clinical trials or systematic reviews) [12], which reflects differing views on the applicability of pre-registration across various types of research. Tools designed to promote open science and accurate and transparent reporting are also meant to increase reproducibility [5,42–45]. Generic checklists may have broad applicability but lack depth, thus potentially limiting widespread uptake. More in-depth checklists will likely only appeal to a smaller group of domain specific users. In fact, reaching consensus on what constitutes the core items for reproducibility does not in itself guarantee effectiveness, as consensus may be incomplete or overlook important aspects. Implementation studies can help determine whether these core items meaningfully identify reproducibility issues. Evidence from interventional studies, such as randomized trials currently conducted within the OSIRIS project [25], may document and quantify the impact of reproducibility checks on research practices. Meta-research studies will likely also help expand this knowledge base.

The list of items presented in this Consensus View is applicable throughout the project life cycle and can serve as a systematic framework and knowledge base for building explanation documents, guidance, and teaching materials to support researchers and other stakeholders in the proper implementation of the given items. The systematic collection of already available, and the development of new, training and education materials has a foundational role in supporting researchers and in promoting a culture of reproducibility and open science [46], but evidence is needed to assess its impact. In addition to translating the theoretical and logical construct of reproducibility into simple and feasible tools, policy, incentives, and norms to improve research credibility should be continuously aligned and updated. To promote systemic cultural change, these efforts should involve everyone involved in the research ecosystem who can support and incentivize reproducibility in a harmonized action at different organizational levels [47,48].

In this Consensus View, we aimed to identify core items for reproducibility and provide a robust starting point to support implementation efforts. We encourage the scientific community to pilot-test the checklist, refine optimal methods for its application, and—most importantly—demonstrate whether assessing these items leads to tangible improvements. Generating evidence on these practices will be essential to move beyond the status quo and reliance on wishful thinking.

Supporting information

Acknowledgments

The OSIRIS-Delphi study group are Maria Josefina Ruiz Alvarez, Ilaria Cianchetta, Nicholas J DeVito, Joanna Diong, Bjoern Gerlach, Monica Gonzalez-Marquez, Kian Hématy, Amanda Kvarven, Jan Mertens, Fakhri Momeni, Gustav Nilsonne, Ulf Toelch, Jack Wilkinson, Xu Zhou, and Mircea Zloteanu. The authors would like to thank Andreas Scherer, Enrico Glerean, Vootele Võikar, Ákos Lencsés, Sara Garofalo, Valentina Parma, Isabelle Boutron, Catherine Coirault, Eva Furrer, Daniel Stekhoven, Bernhard Voelkl, Olivia Miske, Hans Van Eyghen, Frank Ostermann, Nathalie Theret, David Colquhoun, Julien Mayor, Agata Bochynska, Tamara Kalandadze, Lewend Mayiwar, David Allison, Lars Vilhuber, Julian Lange, Josefina Weinerova, Helen Macdonald, Frits Rosendaal, Maria Panagiotopoulou, Maria Rujano, Serge Horbach, Sven Arend Ulpts, Maha Said, Tony Ross-Hellauer, Elli Papadopoulou, Arianna Caporali, Astrid Chevance, Zoè Ancion, Maria Zalm, Raphael Levy, Aki MacFarlane, Sophie Ginolhac, Alex Adams, Marco Annoni, Robert Thibault, Michael Mende, Ling Li, Anna Ambrosini, Graciana Diez-Roux, Andrew Porter, Annemieke Aartsma-Rus, Esi Aduku, Xuan Zhang, Ioana Christea, Shai Silberberg, Anna Catharina Armond, Wainer Lusoli, Randy Ellis, Ted Rohr, Vinodh Ilangovan, Peter Clayson, Alison Abritis, Kelly Cobey, Lorraine Harbison, Guillaume Sescousse, Sarah Cohen Boulakia, Esther Plomp, Adam Pollack, Jennifer Heimberg, and Jan Mertens for participating in the Delphi study. They would also like to thank Eleonora Allocati, Cinzia Colombo, Ayu Dewi, Clément Palpacuer, Priya Sharma, and Maximilian Siebert for their help in the pilot of the online survey; Maddalena Fratelli, Hynek Roubik, István Kertesz, László Baranyai, Mariska M G Leeflang, Leonie A Dudda, and Inge Stegeman for comments on the draft protocol; and Omeralfaroug Ali, Leonhard Held, István Kertész, Marianna Laviola, Jessica Rohmann, Clara Sanchez-Izquierdo Lozano, Alfredo Sanchez-Tojar, and Susanne Schorr for participating in the Responsible Research in Action Unconference.