2.1 Project management

Most software projects in a scientific context start with a single developer. However, over time the projects are likely to grow, be extended, and possibly taken over by other developers. Building a developer community is crucial for the success of the project in general and in particular for open-source projects [4]. Therefore, the project infrastructure should be able to handle multiple developers from the very start.

All of the guidelines in literature we have found mention the usage of a version control system (VCS). This is beneficial even for the single developer, as a VCS intrinsically features a backup solution and synchronization between different machines. Once more developers start working on the project, this enables transparent collaboration. By using a VCS, the “Make Incremental Changes” paradigm [5, 6] can be implemented easily and the intrinsically generated development history may serve as rudimentary documentation of design decisions [4].

In a more advanced scenario, the VCS is coupled with a project management tool that provides a means of communication within the developer team, and thereby further enhances transparency. As the communication logs are available for developers who join the team at a later stage, this also provides a certain form of documentation [4]. One essential element of a project management tool is a ticket system or issue tracker. Issues are requests for a certain change (such as a bug fix or feature implementation) and play a crucial role in modern iterative and incremental software development processes, such as feature-based development [14]. As the name suggests, issue trackers keep track of issues from their creation (by users or developers) to their completion in the form of an accepted solution by the developer [7]. Modern project management tools also include convenient mechanisms for code review. Similar to a scientific paper, a rigorous review process may be time-intensive and annoying, but eventually yields solutions of higher quality and wider acceptance [5].

2.2 Code quality

Just as we care about language style when writing a scientific article, so we should care about coding style when writing scientific software. Here, we should bear the mottos “Write Programs for People, Not Computers” [5] and “Don’t Repeat Yourself” [2, 5] in mind and produce easily readable and modular code. In developer teams, it is crucial to agree on a certain coding style at the beginning of the project. The coding style usually consists of two parts: rules for formatting code and best practices for programming in the respective language. Code formatting tools enable manual and automated checks to establish whether the source code is compliant with agreed code formatting rules [8]. Analogously, static code analysis tools check whether the agreed best practices are violated [7].

2.3 Independence

Some guidelines recommend that open standards, protocols, and file formats should be used wherever possible (e.g., the HDF5 format for large data sets [8]). Thereby, vendor lock-in situations are avoided which would arise, for example, if a certain source code can only be compiled using a certain compiler brand or version. Our general recommendation here is to provide solutions that work with the most widely used operating systems and compilers (and possibly combinations thereof) from the very start.

Following the advice that one should never reinvent the wheel, established software libraries and tools are often used to speed up development processes. Here, we recommend using open-source components unless there is a strong reason not to. This is in agreement with the interoperability and reusability part of the FAIR principle [15, 16].

2.4 Automation

We should “Let the Computer Do the Work” [2, 5] and automate repetitive tasks such as building the software, running tests, performing quality checks, and deploying the generated artifacts (typically, software in binary form and documentation) to a software repository. Otherwise, those tedious tasks are most likely postponed, not done at all, or performed only partially. Here, continuous integration (CI) tools are helpful as different jobs can be defined and grouped into stages, which are executed every time the developers push changes to the version control repository. Then, the developers receive feedback on the changes, which is an essential part of the “Make Incremental Changes” strategy [5].

The feedback typically consists of (at least) two parts, which are briefly outlined. First, the build process should run in an automated and platform-independent fashion. Here, it is particularly important that third-party dependencies are found without hard coded paths. The output of the build process tells the developers whether the build on different platforms was successful. This is especially beneficial as most developers develop on a certain platform and the code is not intrinsically tested on other platforms (different operating systems, different compiler versions, etc.). Second, test programs can be executed automatically on different platforms. For example, unit tests can help to verify the correct behavior of certain functions or modules of the software. Functional tests, on the other hand, help to gain more confidence in the overall function of the software [7].

It makes sense to define the continuous integration pipelines as early as possible, so that the developers benefit from the feedback from the very beginning. Thereby, bugs in the software (in particular regressions) can be detected early. Furthermore, the effectiveness of optimizations can be assessed while the correct operation of the software is ensured.

2.5 Documentation

In order to make scientific software reusable, providing documentation to users and developers is one of the most important steps [1–8]. Bangerth and Heister [4] list five items that the documentation should contain: traditional comments, function level documentation, class level documentation, overview of how modules interact, and complete examples in tutorial form. As to traditional comments, it is good practice to “Document Design and Purpose, Not Mechanics” [5] and avoid obvious comments. Function and class level documentation is typically generated based on comments in code using special annotation. The resulting reference manual is particularly interesting for developers and advanced users who need to know the details. On the other hand, the module overview documentation should inform new users about the big picture. This information is typically written into the files README (aim of the software, installation notes, list of dependencies), CHANGELOG (overview of releases, features, known bugs), CODE_OF_CONDUCT and CONTRIBUTING (guidelines for (potential) developers), as well as TUTORIAL (guide for (potential) users) [6].

2.6 Testing

As mistakes are natural and are bound to happen, we should plan for them and develop strategies on how to detect them as early as possible [5]. Automated testing, the importance of which has already been underlined in Section 2.4, is the cornerstone of such strategies. It should be noted that the effectiveness of tests should be monitored as well. Here, code coverage tools are useful as they are able to detect code parts which are not covered by the executed tests [7].

Again, we stress that certain measures, such as writing unit tests, should be carried out from the very beginning. Apart from their use in automated testing, unit tests may have a positive effect on the code design. Since modular code is usually testable, performing unit tests can be considered a necessary requirement for modular code [2].

2.7 Deployment

Whether or not a certain software project is used depends to a large degree on the ability to distribute it [4]. Hence, it is advisable to package the software and distribute it using an established software repository [1]. Similar to the practices discussed above, it is important that the deployment is carried out automatically and as early as possible [3].

3.1 Usage of a version control system (VCS) and appropriate workflow

A multitude of version control systems has been published and used over the last three decades. We stick to our criterion that the software must be open-source and note that git has received much attention since it was first released in 2005. It features distributed version control and a flexible branching model, rendering it perfectly suited for open-source projects. However, the flexible branching model might, at the same time, be a significant drawback. Each project should define a workflow to show how changes are developed, tested, and integrated. As usual, it makes sense to use something established, such as the GitLab Flow [19]. This workflow uses feature branches to develop and test new features or bug fixes. Once the changes on the feature branch fulfill the requirements and pass the automated tests and quality checks, the developer can open a merge request. A maintainer can subsequently merge the changes in the main development branch. Additionally, the GitLab Flow allows stable branches and different environments (such as production) in which further restrictions may apply. The latter features are not required at the initial stage of a project, but underline that the GitLab Flow is simple enough for small projects yet powerful enough for large and established projects.

3.2 Usage of a project management tool including issue tracking

There are several management tools and hosting platforms that can be combined with the git version control system with different strengths and drawbacks. Here, we would like to leave the choice to the developers and provide two possible solutions for the undecided.

Over the last decade, the GitHub platform has received significant attention. It provides free public git repositories and integrations with other services (such as the zenodo repository for storing research output). Due to its prominence, we have decided to provide a mirror repository of the project skeleton in GitHub [20]. This repository is marked as project template, which allows a new project to be instantiated with a few clicks. As to continuous integration, GitHub offers support for external CI providers such as Travis CI, AppVeyor, and Microsoft Azure. These services are typically free for open-source projects and configured using a YAML file, where CI jobs can be described. As an example, we have added a basic configuration file for Travis CI that triggers build and unit tests on Linux, Windows, and macOS platforms given that the user registers on Travis CI, where the CI operation can be activated for the repository in question.

Alternatively, the GitLab platform can be used, which is conceptually similar to GitHub, the main difference being the possibility of self-hosting the platform on a local server for free. While the concepts (such as Pages and Releases) are similar, there are slight differences. For example, the project template instantiation mechanism is different. At this point, it is not possible to create an instance of the project skeleton with a single click. However, we aim to provide that feature in the near future [21].

GitLab.com provides free hosting and internal continuous integration services for open-source projects. Currently, those internal CI services are restricted to the Linux operating system. It is possible, however, to install GitLab’s CI suite on a local machine and connect it to GitLab. Alternatively, an external service can be used for Windows or macOS operating systems. In the event that the project should not be open-source, the self-hosted operation mode may be selected. Here, the CI suite must be installed on local machines, which can subsequently be connected to the local GitLab installation.

It should be noted that we did not add configuration files for all options to the template in order to provide a lightweight skeleton. Instead, we included the configuration file for the GitLab internal CI, which calls the targets generated by the build systems. From this configuration file, corresponding files for other CI services can be derived.

3.3 Automated build system

In particular when the C++ programming language is involved, the CMake project provides well-established tools to build, test, and package software. The main advantage of CMake (compared to alternatives such as GNU make, Visual Studio, or Eclipse) is that a level of abstraction is introduced. The configuration files consist of directives such as add_library or find_package and are, therefore, quite easy to read and understand. Based on those configuration files, project files for the aforementioned alternatives (and many other build systems) can be generated. Thereby, the software project can be built for different operating systems or using different compilers. In addition, CMake features a mechanism for finding third-party libraries and tools. This feature is essential for cross-platform dependency management.

As a proof of concept, we have added a simple shared library written in C++ to our project skeleton. It features a simple class device with two member variables that represent its start and end coordinates, respectively. An instance of this class can be created using one of two constructors, where either the coordinates are specified directly, or the length can be set and the start coordinate is assumed to be at the origin. Finally, a method returns the length of the device.

For such a shared library, Python bindings can be generated conveniently using the SWIG project. It is fully supported by CMake and requires only a minimal configuration file, which basically specifies which C++ header files should be considered when the interface is created. SWIG scans the specified header files and automatically generates a Python module, which can be subsequently imported and used in a Python project.

3.4 Unit testing

Ideally, the software is designed so that each unit of software (e.g., a function) fulfills a certain, unique task (“Design by Contract” technique [2]). Furthermore, the implementation of each unit is flawless. While the first goal can be achieved by careful design and refactoring, the second statement is rarely true. As mentioned above, mistakes will happen and we have to test whether the implementations of each unit work correctly.

In the case of our simple C++ library, we have to check, for instance, whether the calculation of the length yields the correct result. This can be achieved by writing a unit test that creates an instance of the device class, calls its get_length method, and compares the result of the method to the expected value. Also, whenever the user specifies input data, the implementation should check whether those values are reasonable and deal with invalid values (most likely, by throwing an exception). Error handling code must be tested as well, for example by creating a unit test in which the error is provoked on purpose and checking whether the error handling code yields the correct behavior. As the number of unit tests is expected to be large for a real life project, it is recommended to use a unit test framework.

We chose the Catch2 library as it is open-source, lightweight and header-only. Based on this library, we added a test executable with several unit tests to our CMake build system. Here, we could rely on the CTest functionality of the CMake project. Whether or not the unit tests cover all possible situations can be assessed using code coverage tools. We have added the possibility of using the gcov tool to the project skeleton. This tool generates profiling information during the execution of tests. This information can be subsequently converted to a human-readable report, in which metrics such as line coverage are given on a per-file basis.

3.5 Automatic code formatting

Here, the clang-format tool constitutes a helpful and versatile instrument. It can be configured using a single file, in which the code formatting rules are specified. There are several predefined styles that can be used as-is, or alternatively serve as a basis. It is also possible to define a certain style from scratch, but we recommend using an existent style (with slight modifications, if required).

In our project skeleton, the clang-format tool is integrated into the CMake build system, making it easy for the user to format all source files automatically. This functionality is also used to check whether the source code conforms to the specified style in the scope of continuous integration.

3.6 Documentation generation

From the implementation point of view, we can separate the different types of documentation listed in Section 2.5 into two groups, namely the function reference and the overview documentation. The function reference is based on comments in the source code that use special annotation. The information in those comments can be extracted using the Doxygen tool. For the overview documentation, which provides the “big picture”, it makes sense to use a structured text format. Since Doxygen supports the Markdown language, we chose to write files such as README.md and CONTRIBUTING.md in this annotation. Both the overview documentation and function reference are then transformed into static HTML pages that can be viewed locally or uploaded to a web server.

We note that while Doxygen provides unchallenged support for in-source C++ documentation, the design of the generated HTML files appears a bit dated. More advanced workflows are available that use Doxygen as input parser and alternative tools to generate the static HTML pages. However, this is beyond the scope of the work at hand.

3.7 Automated packaging and deployment to a public repository

While many operating systems or programming languages feature a common repository for exchanging programs and libraries in binary form, it would be beneficial to have a language-agnostic repository that covers all operating systems. Fortunately, the conda system provides exactly this. Once a software project is in a stable state, a recipe can be created on conda-forge that defines the source of the project, the steps required to build it, and meta information such as the name of the responsible maintainer. Based on this recipe, the conda-forge build system automatically generates the binaries for different platforms. Then, on each platform the resulting package can be easily installed within a conda environment.

Most likely, the package has dependencies on other libraries. The conda system offers a vast number of third-party components and convenient methods of installing them. The environment approach already mentioned has a positive effect on the dependency management, as in Windows it is generally impossible to distinguish between different versions of a library (at least when considering unmanaged C++ code), dubbed the “DLL Hell”. Using conda environments, however, it is possible to separate different versions in a clean and convenient way.

The documentation generated could be included in a conda package as well. However, we found it more appropriate to publish it on a web server for visibility reasons. Both GitHub and GitLab offer the possibility of hosting static HTML pages, such as those generated by Doxygen. With a few lines of CI configuration, the documentation is automatically generated and uploaded. See [22] for an example.

4.1 Setup stage

At the beginning, it is important to define a meaningful name for the project and replace bertha with this name throughout the project (e.g., in the CMake build structure). Ensure that the name is not already used (e.g., in conda-forge) if the project is to be open-source. Then, the project team should agree on where to host the project (for internal use only or publicly available), on the license for the project, and on the workflow. The latter includes mainly the coding style and the version control workflow. Both should be documented as soon as possible.

4.2 Implementation stage

At this point, the software project has a solid initial state. Now it is time to add functionality. Here, consider writing the documentation first (the contract), then implementing the functionality, and at the same time writing unit tests. This approach will seem slow but improves the quality of the design and helps to detect mistakes early on. Also, the CMake build structure can be adjusted to add requirements (e.g., software libraries) or additional modules (besides the existing core library).

4.3 Publication stage

In the case of an open-source project, the code should be distributed and communicated as soon as it has some first functionality. For the distribution of the project in binary form, the conda recipe for bertha [18] may serve as reference.