Community building is a key element to promoting good model-sharing practices. Individual model developers are embedded in larger communities and their behaviors are guided by community norms. You may shape those norms by involving your communities when sharing models.

Establishing community engagement and buy-in is crucial for the open modeling efforts of any domain, especially early in the process. Where no defacto community modeling standards exist, we recommend:

1. Surveying your community for requirements, establishing the community’s needs and their unique challenges, as well as current good practices to avoid “reinventing the wheel”;
2. Engaging with the community to brainstorm potential or “strawman” solutions to those open modeling challenges;
3. Selecting a subset of community members who represent both respected experts in the field and those most willing and available to lead modeling standards efforts;
4. Being prepared to prioritize, as complete agreement is rarely attainable, and available resources are often insufficient to fund everyone’s efforts; and
5. Allowing emerging standards to be adopted by different communities and adapted to their contexts.

We find some of these elements present in the history of the Overview, Design concepts and Details (ODD) protocol. The protocol originally suggested a standardized format for describing agent-based models (ABMs) in ecology [15]. The protocol resulted from a workshop conducted in 2004 [16] and the contributions from 28 co-authors—this roughly covers recommendations 1-to-3 above. What’s more, as the protocol was more widely adopted, it was adapted to new applications, including more complex models [17] and other disciplines [18]—this is recommendation number 5.

Community involvement strategies don’t all look the same, but we can draw inspiration from relevant guidelines, established open science communities, and the role of early career researchers (ECRs). Regarding guidelines, consider the CARE Principles for Indigenous Data Governance [19]. For models, CARE means allowing for the community to benefit from their own and others’ contributions, giving the community a voice in controlling use and distribution of relevant models, and considering the ethics that relate to the use of the models (e.g., by documenting assumptions, known biases, and guidance for mitigating against identified risks).

Regarding open science communities, we may learn from the likes of FORRT, a community of over 600 people “raising awareness of the pedagogical implications of open and reproducible science in higher education” [20]. Other example organizations foster inclusive communities that value collaboration are 2i2c, The Carpentries, the Center for Scientific Collaboration and Community Engagement, Invest in Open Infrastructure, MetaDocencia, and OLS [21]. These initiatives demonstrate the effectiveness of community-focused strategies in promoting openness and reproducibility in research, offering valuable frameworks that can be adapted and implemented across various scientific disciplines.

For the long-term promotion of model-sharing practices, special attention should be given to ECRs. ECRs constitute the largest researcher community in most countries [22] and represent a new generation of researchers who have been trained during the software technology boom. This boom has laid the groundwork for open science FAIR model-sharing practices. However, there is increasing evidence that suggests the ECR community is not embracing open science nor sharing scientific artifacts enough [23,24]. This reluctance is understandable considering the “publish or perish” culture of academia. In addition to publishing in journals, sharing models according to good practices requires additional resources, which ECRs often lack.

Despite the challenges, many young researchers are commiting to open science through their own communities—such as DSOS (Data Science and Open Science) and AEMON-J (Aquatic Ecosystem MOdeling Network—Juniour) in the aquatic sciences—or spaces created by institutions—such as the Open Modeling Foundation’s Early Career Scholars Working Group [25,26]. Ultimately, ECRs are the torchbearers of transformative change, setting the norms of science in the near future. Therefore, it is crucial to provide sufficient support and guidance to this key community.

Computational modeling is usually a multidisciplinary endeavor. Contributions may be of very different types, involving model developers, domain experts, archivists, and policy-makers. With this, you must be ready to acknowledge such a great variety of contributors, and the Contributor Roles Taxonomy (CRediT) is one approach towards this goal.

Often, modeling involves inputs from 2 types of parties: domain experts and model developers. While domain experts may provide the theory and data underpinning a model’s assumptions and an initial approach to formalizing a conceptual model, model developers would embed that domain expertise into software according to good practices. It would be unfair to publish a model in a way that did not attribute authorship to all those involved. However, generally, only those who write papers that are published in academic journals gain recognition through authorship. This matters because authorship is often the currency of the credit economy of science [27], and there are few journals that publish models [28].

One movement that has successfully challenged the status quo of academic publishing is the “Hidden REF” in the UK, which celebrates all research outputs, not disproportionately rewarding publications like the “research excellence framework” does [29]. This is a step in the right direction for model-sharing, as models are not generally peer-reviewed through academic publications. So, when sharing models, it is important to think carefully about whom to acknowledge and how.

Commonly, scientific contributions are recognized through authorship. Despite making scientific contributions, authorship is not typically given to RSEs or other people not contributing to publication texts. To resolve this issue and acknowledge such diverse contributions, publishers are progressively integrating the CRediT into their workflows and metadata systems [30]. Below, we outline 3 contributor roles that are particularly pertinent to the context of modeling:

• Data curation—which CRediT defines as activities related to training data annotation, cleaning, and maintenance—extremely time-consuming yet fundamental to model-production. Publicly available data sets for certain types of modeling (e.g., healthcare) are notably scarce. Furthermore, issues of missing or noisy data require preprocessing techniques that require special considerations for categorizing data (e.g., blood pressure data being “high,” “medium,” or “low” according to different clinical guidelines).
• Formal analysis necessitates computational or mathematical techniques for data analysis or synthesis. After data cleaning, analysis may include imputation (e.g., mean, median, or k-Nearest Neighbors) to fill in missing values based on extant data, data partitioning for training and testing, feature extraction and selection, hyperparameter optimization, dimensionality reduction, and performance metrics.
• Software development and related tasks are well covered by the “All Contributors” effort, which enables semi-automated contribution roles to be added for a person contributing to a GitHub repository [31]. However, modeling may require an additional contributor role for hardware, which can often be quite specific for the given model. Indeed, certain models can only run with sufficient computational power (e.g., a GPU as opposed to a CPU).

The structure required to recognize nontraditional contributions is growing. Initiatives such as CHAOSS are investigating methods to represent these contributions effectively for the benefit of research project health and individual recognition [32]. Repositories like Zenodo are incorporating contributor roles and increasingly adopting the CRediT taxonomy. It is now possible to easily add recognition for a person’s contribution to a GitHub repository using “All Contributors.” Additionally, specific communities are working to make such taxonomies more comprehensive, ensuring that all contributions receive appropriate credit (e.g., those in the geosciences) [33]. Even indicators of scientific credit are evolving. Consider DORA, a global initiative that aims to improve the ways in which scholar outputs are evaluated. DORA adopts a broad definition of scientific output that includes not only scholarly articles but also data sets, patents, and software [34].

Make your model reusable and more impactful by providing documentation alongside it. By “documentation,” we mean a collection of documents and additional written material that describe a computational model across its entire life cycle and along with its underlying assumptions and scientific bases [35]. This helps communicate to diverse stakeholders why a model is worth using.

Generally, models are more impactful where their users deem them to be (i) scientifically sound; (ii) relevant to the policy issue at hand; and (iii) the result of stakeholder engagement [36]. Accessible and thorough documentation is critical to model-sharing. In some cases, additional comments in documentation are needed to aid in understanding. Depending on the nature of the comments, they might be included in the code, in a public notes document where notes on the history of the model usage and stability are contributed, as structured metadata linked to the model’s metadata record, or as a combination of these. As a general rule, the more complex a model is, the more effort should be spent on creating its documentation, although metadata management technologies are becoming increasingly capable at streamlining such processes.

Documentation should contain valuable information for various audiences and be clearly organized. Efforts have already been made for making models more interpretable to different audiences. For example, “model cards” summarize, among other things, a model’s performance in different contexts and its intended scope [37]. They provide a helpful template for reaching audiences of different technical abilities. Meanwhile, the Model Openness Framework articulates the different components and their respective open licenses that are required for sharing deep learning models, how those objects interconnect, and the stakeholders that would be needed to support more robust model-sharing [38]. While such frameworks are comprehensive and require technical expertise to implement, they improve the model’s impact on downstream users.

At least 4 audiences benefit from tailored documentation: policy-makers, domain experts, archivists, and fellow model developers.

• Policy-makers gain from detailed documentation relating to potential policy impacts. Policy-informing models should be presented as such and accompanied by documentation that allow policy-makers to understand the science and articulate policy decisions based on the model [39]. Relevant documentation elicits assumptions, explains the contextual conditions and input parameters under which the model’s result should be considered valid, depicts the maturity of the model, and summarizes the underpinning science without using jargon.
• Domain experts benefit from documentation with more thorough explanations of the science underpinning a model, why it is a valid representation of its target, and why it is fit for purpose. Those explanations should include the jargon expected by experts in the science field but without any expectation of past modeling experience [40]. Indeed, research has shown that documentation targeted at domain experts may increase the trust they deposit in a model’s output [41].
• Archivists use model documentation and model metadata to determine the maturity of the model and, thus, what level of supporting resources to assign to the model. For example, a mature model’s documentation would include the high-level content described elsewhere in this section, detailed installation and execution notes, and recommendations on input datasets or settings for several example scenarios of importance. Assuming sufficiently rich model metadata—archivists might also provide support with intensive curation, user interface design, and dedicated funding for model executions at the model developer’s institution as requested by the community. Conversely, a less mature model may only be preserved for community reference (e.g., a Zenodo deposit).
• Other model developers benefit from documentation. For example, justifications of decisions that cannot be captured in metadata often help with a model’s replicability. Model developers may also find themselves working across different domains throughout their careers. For this purpose, more accessible documentation may help those who are new to a model’s domain to better understand the model’s nuances. An example is Earth System Documentation, which provides comprehensive documentation about the complex earth system models developed by more than 40 modeling groups worldwide, thereby allowing model developers to better understand the internationally coordinated effort. Additionally, integrating narrative and code through computational notebooks—such as Jupyter—can aid readers and users understand and reuse models. These notebooks interweave explanatory text with executable code, clarifying the model’s functionality and application.

Sharing models with accessible documentation is consistent with, and enabling of, common publication practices. Indeed, accessible documentation helps with a model’s assessment at the peer review stage before publication. Consider that there may be very few peer reviewers for any given submission who have both the relevant domain knowledge and the necessary model development expertise to evaluate a model. Therefore, it is helpful to additionally provide accessible documentation on a model’s parameters and dependencies—perhaps in a README file—that allows for its replicability [42].

The FAIR principles have gained significant traction throughout open science initiatives. It is important that you don’t try to reinvent the wheel when it comes to good model-sharing practices. With this, you can draw on the FAIR principles when sharing information about your model in its metadata.

The principles for creating FAIR metadata for data and software generally apply to how we share computational models. However, models present unique challenges due to their integration of conceptual constructs, metadata and software: models are distinct from the usual target of the FAIR principles. For example, a data set’s metadata does not provide insights about its individual components (i.e., data points), while a model’s metadata may provide insights about some components (e.g., an ML model’s metadata describing distinctions between its training and testing data). Furthermore, when sharing a model’s metadata, we are not necessarily sharing the data on which they were built, calibrated or validated. Rather, we are sharing information about the model.

These unique features render the FAIR principles a substantively complex topic for their implementation in the context of models. In what follows, we provide only simple considerations to improve a model’s findability, accessibility, interoperability and reusability, as well as its provenance.

• Findability: Models and their related artifacts (e.g., data sets and software) can be found and identified through relevant indexes and repositories if each are assigned persistent identifiers (PIDs) such as digital object identifiers (DOIs) for each version. This also helps situate models within complex knowledge graphs, supporting to findability from a variety of access points (e.g., publication and data set landing pages, internet browsers, and community-specific search interfaces). In many cases, the complex landscape of artifacts relevant to a given model is more properly addressed with a metadata container identifier. For this, we may use a Research Activity Identifier (RAiD) or a Research Object Crate (RO-Crate) [43], which effectively interlink a model’s heterogeneous research elements. Metadata underpinning the chosen PID may also make the model citable in publications or on websites, providing another important component of findability.
• Accessibility: When sharing models, a working link to access the complete model should be made available to the public, including code, executable files, and other items needed to use the model. However, items that are restricted by national or institutional policies or contracts should remain restricted until those conditions can be changed. Given the commonality of such situations, model accessibility should be planned ahead of time.
• Interoperability: Metadata produced in standard formats can be used to locate models within their broader research contexts, thereby allowing the model’s metadata to be interoperable with other domain-specific research projects. An example of this is at BioModels, EMBL-EBI, where the team of model curators enrich metadata and convert code into standard formats for their respective domains (e.g., systems biology markup language “SMBL” for systems biology, and Open Neural Network Exchange “ONNX” for ML-trained models) [44]. Ultimately, the curators ensure that models made available on the BioModels platform are interoperable with relevant systems and can be adequately indexed by other databases.
• Reusability: Model reusability can be enabled through 3 lines of action. Firstly, the usage license for the model must be included in the model’s metadata for the user to understand their legal rights to reuse the model, ideally indicated with a machine-actionable identifier (e.g., from spdx.org). Secondly, metadata can capture information needed to run the model. For example, in some domains, models may only be usable by interacting with code or tools that require complex workflows, such as specialized hardware, programming languages, and other specific dependencies. Consider a model trained to run on a GPU and that requires using the c++/CUDA programming language. In turn, this programming language’s functionality may depend on a certain programming package, such as cuda-toolkits [45]. Finally, models may contain sensitive items that are not publicly available. Such cases emphasize the importance of providing relevant model metadata for their potential reuse when said items become available.
• Provenance: Although provenance is a component of reusability (R1.3. [5]), we separate this concept out due to its complexity for models. Provenance refers to the need to (i) explain the history of components for the credibility of a model [46]; (ii) understand the process by which a model produces results in general and for a specific run; and (iii) link all relevant data sets, publications, and other software to the model, such as input and output data or training data [47]. Attribution metadata is useful to understand from where training data was derived (and, if relevant, where they were stored) and/or what base models may also be relevant. For example, image classification models may need to address questions of attribution, licensing (for images), and other provenance-related concerns. Documenting provenance is also a critical component of model validation studies where different versions or frameworks of the model may be used to predict various events with widely varying accuracies.

In models that produce research artifacts such as data, the practices described above must also be included in—or linked from—those artifacts. Although we have focused on making models FAIR, we should also seek the appropriate mechanisms to make model outputs just as FAIR.

Readability and actionability are enabled by the FAIR principles. Model metadata becomes machine readable and actionable when it is aligned with international and community-specific metadata structures. Said alignment enables a wider application of search tools to improve findability, and can improve the richness of the metadata, benefiting all interested stakeholders. The main international metadata structures are DataCite, Schema.org, and CITATION.cff, and should be used to structure the minimal metadata for models. Additional metadata beyond the minimum requirement for PID creation—such as those recommended by CodeMeta—are useful to support discoverability. We should also incorporate community-specific recommendations for FAIR implementation, which often relate to unique disciplines (e.g., the “Data, Optimization, Model and Evaluation” (DOME) [48] recommendations in the bioinformatics community).

RSEs can play important roles in modeling projects but have only garnered attention in recent years. With RSEs’ varied contributions to modeling, it is important to specifically recognize and reward their work.

Acknowledging diverse contributions is key to good model-sharing practices. However, there is one key role that is particularly crucial to the modeling process: the RSE. Computational models have always required software of some sort, but the term “RSE” was only coined in 2012, when a group of scholars met in Oxford, to ask: why is there no career for software developers in academia? [49]. Through campaigning and public outreach, the recognition of RSEs gained overwhelming support, and now encompasses about 10,000 professionals worldwide, and 9 region-specific communities [50]. But we must continue to celebrate the work of RSEs, and the value they bring to modeling is varied indeed.

In modeling, RSEs help domain experts work with the complex software they develop. They play a translational role between domain-specific expertise and software-specific tasks, such as data processing, model simulation, and software testing [51]. Thus, the RSE both applies their skills in software engineering, and comes to learn some of the intricacies of the domains with which they engage.

What’s more, RSEs provide a service to domain experts, who often have widely varying degrees of coding, software, and modeling skills. For RSEs, this means having to meet their different users’ specific needs. Some users may need complex cyberinfrastructure solutions to lead projects carried out by entire teams. These users will likely be more tech savvy. At the other end of the spectrum, we may have early career researchers with little technical expertise who require support with simpler tasks.

One approach that has been found to help RSEs tailor solutions to specific user needs is “design thinking,” which is user-centric and solution-based [52]. Software engineers in business settings have applied design thinking for over a decade [53], and it has already been found to relieve tensions in consortia involving both private companies and research institutions [54]. One specific instantiation of design thinking applied by RSEs is at the University of Notre Dame’s Center for Research Computing, where they specifically use this approach to deliver solutions that meet the needs of their users [55].

RSEs also help other stakeholders engage with model outputs in an accessible way. In clinical settings, it has been observed that RSEs remove technical barriers for clinicians and industry partners [56]. The different metadata and documentation resulting from good model-sharing practices also require RSEs’ input, and we have already seen the varied audiences supported by such practices.

With all this, RSEs play a critical role in modeling, and must be given the credit they deserve to attract and retain their talent for modeling purposes. Beyond giving RSEs credit, a good model-sharing practice is to standardize their roles and enable their career development [57]. This requires a more fundamental shift in how RSEs are evaluated in the workplace, as applying traditional academic norms to this emerging role is driving them away from research contexts and depleting the sciences of the skills and experience needed to tackle complex problems [58].

User-friendly tools, such as interfaces where users can request a model execution, are essential for models to be more seamlessly adopted by domain experts, including those who may lack the specialized computer science training [59]. Models often rely on complex cyberinfrastructure, and science gateways offer a solution for making modeling more accessible and straightforward.

In the early 2000s, several initiatives aimed to democratize cyberinfrastructure, making it available to researchers regardless of their geographical location. Through online portals, these initiatives enabled researchers to access various software applications, store and share large data sets, and even obtain training materials. This era marked the beginning of widespread accessibility to computational resources, laying the groundwork for the development of modern science gateways [60].

Today, science gateways provide intuitive interfaces that abstract the underlying technical complexities, enabling researchers to focus on their scientific problems rather than on the intricacies of the software. By offering integrated environments that support various workflows—from data analysis to simulation and visualization—science gateways streamline research processes, foster collaboration, and accelerate discovery. They also facilitate reproducibility and transparency in research by providing standardized tools and methodologies, making it easier for experts to validate and build upon each other’s work. Ultimately, user-friendly science gateways democratize access to advanced computational resources, empowering domain experts to leverage cutting-edge technologies to advance their fields. However, limitations still exist for science gateways to incorporate computationally complex models—such as earth system or forecasting models—requiring the participation of model developers to perform the set-up and execution of the model to create the desired output. Additionally, model coupling requires a human-in-the-loop interaction that needs RSEs to prepare the science gateway for such steps.

One example of a widely used science gateway is MyGeoHub [61]. MyGeoHub allows access to several science gateways for geospatial research and models. One of the platforms that are accessible via MyGeoHub is the WaterHub [62], which provides substantial benefits for Soil and Water Assessment Tool (SWAT) [63] modeling by enhancing open data access and reproducible workflows.

As a centralized repository for SWAT models, MyGeoHub simplifies the process by eliminating the need for complex local setups on a user’s computer for running the models. Furthermore, the science gateway fosters a collaborative environment by allowing users to share data either publicly or within projects they can configure and add members to. MyGeoHub allows for fine-grained security in the different science gateways that people can decide to keep data first private or to share data within a project or fully publicly.

MyGeoHub’s cloud-based scalability and high-performance capability allow users to run complex simulations on powerful resources, thus overcoming hardware limitations. Moreover, the potential integration with visualization tools within MyGeoHub enhances the clarity and communication of research findings. Additionally, MyGeoHub’s compatibility with SWATShare streamlines simulation processes and facilitates easy access to results.

SWATShare [64] is a web platform of the widely used, public domain hydrologic model SWAT. This semi-distributed, conceptual model simulates various processes, including rainfall-runoff, non-point source pollution, and the impacts of agricultural management practices on watersheds. Developed and maintained by the USDA Agricultural Research Service [65], SWAT is a valuable tool for researchers worldwide. The widespread adoption of SWAT is well-aligned with MyGeoHub’s collaborative framework, as SWAT’s open-source nature complements MyGeoHub’s mission to promote open science practices in hydrology. Fig 2 shows an example of a model created for hydrology research on the Tokwe Watershed by Enos Bahati and shared via SWATShare. This visualization is accessible via the WaterHub in MyGeoHub. In addition to the visualization, users have access to the metadata for this model including important research measures such as simulation time steps and simulation periods—this case, 30 years (Fig 2).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Screenshots from SWATShare.

The left panel shows available research projects in blue across most of Africa. The dot in red in Uganda has been selected, and relates with 2 models by Enos Bahati. The right panel shows metadata from the “New_tokwe” model. The figure is available under CC-BY 4.0 and can be replicated at https://mygeohub.org/groups/water-hub/swatshare?model_id=dc58f0b2b76ab9f799e35d47b400d4e.

https://doi.org/10.1371/journal.pcbi.1012702.g002

Supporting and training staff, including RSEs, and implementing practices like hallway testing (asking others to use your code to understand any usability issues) [66] are critical practices for ensuring the user-friendliness of services and tools. They help fill information gaps, connect users with complementary tools and services, and facilitate the enhancement and iterative improvement of model interfaces. User-friendly services encompass a range of elements, including effective documentation, comprehensive usage instructions, and demonstration resources. These components collectively enable users to understand and effectively integrate models into their own work, thereby enhancing overall usability and accessibility.

The development of user-friendly tools and the support structures surrounding them are pivotal in making advanced modeling and computational resources accessible to a broader range of researchers. By lowering the barriers to entry, science gateways not only democratize technology but also pave the way for significant scientific advancements, driving innovation and progress across various domains.

Of course, not all research domains have science gateways in place. In such cases, it is worth going back to basics by learning from the earlier initiatives; for example, by pursuing a distributed modeling network approach where modeling capabilities are hosted at different institutions but accessible through a single interface. You can also advocate for science gateways as a possible solution to your community’s needs.

As part of the research community, you have the power to influence what model-sharing practices are valued and adopted. Whichever your role in modeling, you should guide your peers and future scholars towards good model-sharing practices, and seek opportunities to inform the policies that publishers implement.

When publishing computational models, it is crucial to adhere to high standards for data- and model-linking by using appropriate PIDs. Detailed metadata, precise version control, and comprehensive documentation of data sources and model parameters are also essential components. These measures enable other scientists to understand and replicate models.

Furthermore, high standards in data- and model-linking facilitate meaningful comparisons across studies and foster collaborative advancements in the field. By maintaining robust links between data and models, researchers can more easily integrate their work with existing studies, leading to new insights and innovations. This approach enhances the credibility and impact of individual research projects, and contributes to the broader scientific community’s collective knowledge.

But who creates standards for model-linking? Generally, there are 2 approaches to developing standards: top-down and bottom-up. Top-down standards may come in the form of governmental or organizational policies. Policy-based approaches have been shown to be the most effective at increasing data-sharing [67], but the degree to which publishers require models to be shared is variable. For example, in a study of 7,500 articles on individual- and agent-based models that were published across 1,500 different journals, only 11% were found to share code [68]. Furthermore, data from DataCite shows that, out of over 17,000 models accessible through their platform, there are less than 1,000 citations in the literature to the models because they are not cited in their related metadata or publications [69].

One instance of a top-down policy can be found at the journal Springer Nature, which now has a policy that encourages code-sharing [70]. The policy’s implementation has relied on the human support and technological infrastructure made available to authors. Indeed, the policy requires journal staff to assist authors in making their code executable on CodeOcean. The policy also results from a pilot, where authors responded with little resistance [71]. This points to the second approach to developing standards.

A bottom-up approach to developing standards for model-linking relies on communities reaching a sort of “tipping point” [72]. At this tipping point, model developers and stakeholders cohesively adopt and disseminate some practice to such a degree that it becomes a standard. We already saw how this might be achieved with the case of the ODD protocol. In the case of Springer Nature’s policy, the research community’s readiness to follow open science practices was instrumental for its viability. With this, there is potential for model developers to promote model-sharing by influencing publication policies [73].

Researchers can shape model-sharing practices in various ways. Those who serve on journal boards can use their positions to influence journal policies. Where this isn’t possible, researchers in their role of peer reviewers can hold others to account and use their position to normalize model-sharing. In addition, community members in educational settings can train budding scientists to share models. The goal is for the next generation of scientists—who will go on to become future editors, reviewers, principal investigators, and teachers—to carry the torch of model-sharing [74].

With this, model-sharing isn’t just about enacting good practices, it’s also about promoting them. Whether you’re a mentor, professor, librarian, PhD candidate, or somebody else in the modeling world, you can inform how your networks approach model-sharing. Being part of academic publication processes is just one way to do this.

Working within silos is detrimental to scientific progress. With modeling being inherently multidisciplinary, you should value the role of collaborating across disciplines and organizations as central to all good model-sharing practices. After all, computational models are shared by people, for scientific and policy advancement, via platforms.

The concept of multidisciplinary collaboration in modeling is not new. For example, geoscience requires a diverse collection of expertise including physics, biology, chemistry, and social sciences to model the entire Earth system. This has led to a community-driven approach to model development and sharing called Community Earth System Model, which originated from the United States and the EC-Earth consortium in Europe. This approach is now co-developed by the international community and shared across the globe. Recent technological developments such as cloud computing also make sharing and international collaboration more streamlined.

We have already seen the various stakeholders who benefit from model-sharing practices—from policy makers to publishers and scientists (rule 4). The rule on accessible documentation proffered one approach to break down silos: it is a good model-sharing practice to enable diverse audiences to engage with your models by anticipating their needs through documentation. Indeed, when it comes to modeling for public policy purposes, research has found that involving different stakeholders from the outset increases the likelihood of a model being used and effective [75].

The rule on community-driven insights (rule 2) is also relevant: model developers should establish or be part of communities that encompass diverse disciplines and perspectives. Such communities may even bring together different modeling stakeholders. A key learning rule 2 is that breaking down silos does not happen automatically. Building communities takes effort and must be intentional. Having clear codes of conduct and welcoming different types of contributions are some ways to bring seemingly distinct communities closer together.

The rule on acknowledging diverse contributions (rule 3) further supports the community rule: we are not only engaging with diverse communities but also rewarding them. The CRediT taxonomy we saw there is predicated on the need for different staff—not only academics—to be acknowledged. In this regard, breaking down silos different departments across organizations working cohesively and being recognized in modeling efforts.

We already saw the role RSEs have to play (rule 6), and we can imagine how IT may provide support by making certain hardware and software available; how project managers, events coordinators and communications teams may help a project run smoothly and be effectively disseminated; and how cybersecurity experts may help teams establish policies on what excluded from model-sharing, from passkey values to dependencies on vulnerable packages [76].

Finally, we saw how science gateways are one approach to making cyberinfrastructure accessible to domain experts (rule 7). Increasing the accessibility of models using science gateways further breaks down silos, connecting a greater portion of a given community to advanced modeling capabilities without needing to know the right people.

Collaboration is essential for breaking down silos and fostering innovation in model development. Platforms such as GitHub and Hugging Face are widely used in facilitating the necessary interactions for advancing model-sharing practices [77,78]. The now-forming capability to execute complex models using a mix of cloud and on-premise resources also advances collaboration capabilities, further linking model developer communities together without having to rely on third party platforms to produce a model run [79]. These platforms and capabilities enable dynamic collaboration, allowing for continuous improvement, shared contributions, greater accessibility to both the models and their developers, and new ways to run models.

As the adage goes: perfection is the enemy of progress. We cannot be paralyzed by the desire to enact every good model-sharing practice each and every time. Rather, we must do the best we can, given the resources we have access to and the policies our institutions implement.

Making information about computational models publicly available is typically advantageous. However, there are many elements involved in the process; from clear licensing and different types of documentation to community-building and acknowledging contributions. Although there are challenges that inhibit model-sharing (e.g., a lack of technical skills to create user-friendly interfaces, or time constraints that do not allow for extensive documentation), it is important to realize that the act of sharing a model—in any format—holds more value than withholding it entirely [80–82].

Model-sharing takes place along a spectrum, from minimally making DOIs available to referencing some of a model’s components, to producing comprehensive documentation for diverse stakeholders to learn from a model. With this, it may be difficult to share comprehensive information about models, but we can always consider implementing those simpler, good practices that we can achieve.

Actively sharing models can help us be better prepared for the future of modeling. There is, after all, an increasing institutionalization of model-sharing. This is occurring at governmental and organizational levels. The European Union’s AI Act is an example at the government level of increased expectations to share models, whereby models that are components of artificial intelligence (AI) systems may trigger certain exemptions for the model developers if shared for free and open-source [83]. In the US, we find other efforts along these lines, with the White House seeking input on the risks and benefits of making AI models’ weights widely available [84].

At an organizational level, we find various examples of developing and implementing model-sharing policies:

• Wageningen University & Research (WUR) in the Netherlands has a team of model auditors. Their job is to assess the quality of models produced within WUR’s research departments [85,86]. WUR’s Research Modeling Group has also established clear standards for model developers to follow [87,88]. Through their published standards and auditors, model developers at WUR have access to support to ensure their models meet the institutionalized criteria for model-sharing. Moreover, WUR staff have access to their “good modeling practices Wiki,” where they can both learn about standards shared in academic literature and make their own contributions [89].
• The Channing Division of Network Medicine (CDNM) has institutionalized an approach to running code in clinical research settings. In this case, CDNM has developed “Data ID forms” for documenting the location and run date for the code that produces all figures, tables, parameters, and numbers reported in a paper [90]. It is worth noting that this form fits within a wider set of governance structures that promote research integrity.
• NASA, meanwhile, updated their policy regarding the models they fund in 2022. The policy requires models to be shared at the time of publication of the first related article, or at the end of the research award, as long as the software is not restricted. The policy additionally instructs such software to be developed openly on a version-controlled platform. This signals a long-awaited shift towards open science practices for models [91].

As with these 10 simple rules, we cannot expect perfection when sharing models. A variable’s provenance, a discipline’s perspective, or an assumption’s justification may always be missing. We continue to operate in a world where the establishment of standards and availability of peer reviewers for models are far from satisfying. But open modeling practices are becoming institutionalized as the open science movement continues to thrive. With this, you should feel encouraged to publicize your approach to modeling. Only by sharing may you receive community feedback, allow your models to be adopted and adapted by others, and promote good model-sharing practices.