PONE-D-20-37277

Free and Open Source Software a large scale analysis of code, comments, and commit frequency

PLOS ONE

Dear Dr. Jemielniak,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

As you can see below, both reviewers identified interesting aspects of your manuscript. However they also raised a number of concerns that prevent it from meeting PLOS publication criteria. In particular, when revision your article, pay special attention to the following issues:

- Discussion of the obtained results: Both reviewers asked for a revision of the conclusions reached (and the way they were expressed), considering teh limitations of the study.

- Methodology: Notice specially the requirements by Reviewer 1 for a more clear description of the applied methodology. Moreover, address the concerns by reviewer 2 on the collected dataset.

- English: Reviewer 1 also asked for a revision of the English used, as some parts are much less clear than others.

Please submit your revised manuscript by Mar 14 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Sergi Lozano

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. We suggest you thoroughly copyedit your manuscript for language usage, spelling, and grammar. If you do not know anyone who can help you do this, you may wish to consider employing a professional scientific editing service.  

Whilst you may use any professional scientific editing service of your choice, PLOS has partnered with both American Journal Experts (AJE) and Editage to provide discounted services to PLOS authors. Both organizations have experience helping authors meet PLOS guidelines and can provide language editing, translation, manuscript formatting, and figure formatting to ensure your manuscript meets our submission guidelines. To take advantage of our partnership with AJE, visit the AJE website (http://learn.aje.com/plos/) for a 15% discount off AJE services. To take advantage of our partnership with Editage, visit the Editage website (www.editage.com) and enter referral code PLOSEDIT for a 15% discount off Editage services.  If the PLOS editorial team finds any language issues in text that either AJE or Editage has edited, the service provider will re-edit the text for free.

Upon resubmission, please provide the following:

• The name of the colleague or the details of the professional service that edited your manuscript
• A copy of your manuscript showing your changes by either highlighting them or using track changes (uploaded as a *supporting information* file)
• A clean copy of the edited manuscript (uploaded as the new *manuscript* file)

3. Thank you for stating the following financial disclosure:

"no"

At this time, please address the following queries:

1. Please clarify the sources of funding (financial or material support) for your study. List the grants or organizations that supported your study, including funding received from your institution.
2. State what role the funders took in the study. If the funders had no role in your study, please state: “The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.”
3. If any authors received a salary from any of your funders, please state which authors and which funders.
4. If you did not receive any funding for this study, please state: “The authors received no specific funding for this work.”

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

4. Thank you for stating the following in your Competing Interests section: 

"no"

Please complete your Competing Interests on the online submission form to state any Competing Interests. If you have no competing interests, please state "The authors have declared that no competing interests exist.", as detailed online in our guide for authors at http://journals.plos.org/plosone/s/submit-now

 This information should be included in your cover letter; we will change the online submission form on your behalf.

Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: http://journals.plos.org/plosone/s/competing-interests

5. In your Data Availability statement, you have not specified where the minimal data set underlying the results described in your manuscript can be found. PLOS defines a study's minimal data set as the underlying data used to reach the conclusions drawn in the manuscript and any additional data required to replicate the reported study findings in their entirety. All PLOS journals require that the minimal data set be made fully available. For more information about our data policy, please see http://journals.plos.org/plosone/s/data-availability.

Upon re-submitting your revised manuscript, please upload your study’s minimal underlying data set as either Supporting Information files or to a stable, public repository and include the relevant URLs, DOIs, or accession numbers within your revised cover letter. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. Any potentially identifying patient information must be fully anonymized.

Important: If there are ethical or legal restrictions to sharing your data publicly, please explain these restrictions in detail. Please see our guidelines for more information on what we consider unacceptable restrictions to publicly sharing data: http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. Note that it is not acceptable for the authors to be the sole named individuals responsible for ensuring data access.

We will update your Data Availability statement to reflect the information you provide in your cover letter.

6. Please ensure that you refer to Figures 1, 2, 3, and 4 in your text as, if accepted, production will need this reference to link the reader to the figure.

7. Please include a copy of Tables 7, 8, and 9 which you refer to in your text on page 18.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: I Don't Know

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: No

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: My review is uploaded as an attachment. This following text is included to meet the minimum character count required by the PLOS ONE web form.

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

Reviewer #2: SUMMARY

The paper presents a large study of FOSS projects to better understand the growth of FOSS organization via basic quantitative indicators. The dataset built for the analysis includes projects being developed by 15 FOSS organizations, which are classified into 3 groups (i.e., big, medium, small) according to the number of projects they manage. The identified groups are used to conduct the study. Results reveal that collaborative efforts (measured in terms of commits, committed code, and comments) are lower than at the beginning of the current decade. Only projects from medium and small FOSS organizations seems to be less affected by this trend.

REVIEW

The paper addresses an interested topic of the field and helps to better understand the evolution of the contribution activity in FOSS projects, in particular, in the context of FOSS organizations. Although the analysis is purely quantitative, its large size helps to visualize how collaborative actions (i.e., commits and comments) have changed along the years. This is revealing and worth tracking.

I have two main concerns:

* The collection date to build the dataset looks old, and it should be better noted in the "Limitations of the research model, data and results" section. My main concern is the exponential explosion of activity in collaborative development platforms such as GitHub, which are becoming the de facto standard to develop FOSS projects, and where the organization dimension sometimes is blurry. This is also related to possible novel ways to manage FOSS projects which may be beyond traditional FOSS organizations. I understand that this idea may require additional research, but the text should note the situation.

* The discussion and conclusions of the paper are mainly hypotheses about FOSS organization evolution, as no empirical confirmation (i.e., interviews with developers) has been done. I wonder if they could be presented first for FOSS projects in general, and then extrapolate for FOSS organizations (being very cautious). This concern is also related to the previous one, as in the last years (2017-2021) FOSS organizations have been actively working on exploring new ways to support and help FOSS projects. 

All in all, I believe the paper makes an interesting contribution, but the conclusions are too focused on FOSS organizations and such a link is hard to visualize from the data gathered and analyzed.

DETAILED REVIEW

The first section introduces the problem and presents most of the related work. It also presents the main triggering idea to conduct the analysis presented in the paper.

In the second section, the authors briefly discuss the differences between free and open-source software, which helps to present some characteristics (and issues) typically found in FOSS software. This also serves to motivate the study conducted in the paper.

The third section presents the research methodology of the paper. Some indications for the subsections:

* Table 1 could be simplified indicating the starting year and its collected data-months. When needed, years with less than 12 collected data-months should be indicated (e.g., openstack in 2017)

* When describing CODPC, COMPC and COMPCOD and referring to either (or comments), is that the difference between the added and removed code (or comments)?

The fourth section presents the results of the study. Companion tables and figures illustrated the obtained results.

The fifth and last sections discuss the results and elaborates on some hypothesis to justify them.

MINOR ERRORS AND POSSIBLE TYPOS

- Normalize the use of "open-source" vs "open source"

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PONE-D-20-37277R1

Free and Open Source Software: A large-scale analysis of code, comments, and commit frequency

PLOS ONE

Dear Dr. Jemielniak,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

When revising your text, pay special attention to the following issues:

1.- Both reviewers pointed out that some aspects of the methodology are not properly justified and/or described. Notice PLOS ONE's publication criterion #3 (https://journals.plos.org/plosone/s/criteria-for-publication#loc-3).

2.- Reviewer 3 stressed the need for a better alignment between the results obtained and the conclusions reached. Notice also the journal's publication criterion #4 (https://journals.plos.org/plosone/s/criteria-for-publication#loc-4).

Moreover, before submitting your revised manuscript, please be sure that the comments by both reviewers are properly answered.

Please submit your revised manuscript by Jun 04 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Sergi Lozano

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: (No Response)

Reviewer #3: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #2: Yes

Reviewer #3: No

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: Yes

Reviewer #3: No

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: I have not found an answer to my comments in the revised version of the paper. I hold my decision and attach again my concerns.

The paper addresses an interesting topic of the field and helps to better understand the evolution of the contribution activity in FOSS projects, in particular, in the context of FOSS organizations. Although the analysis is purely quantitative, its large size helps to visualize how collaborative actions (i.e., commits and comments) have changed along the years. This is revealing and worth tracking.

I have two main concerns:

* The collection date to build the dataset looks old, and it should be better noted in the "Limitations of the research model, data and results" section. My main concern is the exponential explosion of activity in collaborative development platforms such as GitHub, which are becoming the de facto standard to develop FOSS projects, and where the organization dimension sometimes is blurry. This is also related to possible novel ways to manage FOSS projects which may be beyond traditional FOSS organizations. I understand that this idea may required additional research which is not included in the paper, but the text should note the situation.

* The discussion and conclusions of the paper are mainly hypotheses about FOSS organization evolution, as no empirical confirmation (or attempt to confirm) has been done. I wonder if they could be presented for FOSS projects in general, and then for FOSS organizations (being cautious). This concern is also related to the previous one, as in the last years (2017-2021) FOSS organizations have been actively working on exploring new ways to support and help FOSS projects.

All in all, I believe the paper makes an interesting contribution, but the conclusions are too focused on FOSS organizations and such a link is hard to visualize from the data gathered and analyzed.

DETAILED REVIEW

The first section introduces the problem and presents most of the related work. It also presents the main triggering idea to conduct the analysis presented in the paper.

In the second section, the authors briefly discuss the differences between free and open source software, which helps to present some characteristics (and issues) typically found in FOSS software. This also serves to motivate the study conducted in the paper.

The third section presents the research methodology of the paper. Some indications for the subsections:

* Table 1 could be simplified indicating the starting year and its collected data-months. When needed, years with less than 12 collected data-months should be indicated (e.g., GNOME in 2017)

* When describing CODPC, COMPC and COMPCOD and referring to either (or comments), is that the difference between the added and removed code (or comments)?

The fourth section presents the results of the study. Companion tables and figures illustrated the obtained results.

The fifth and last sections discuss the results and elaborates on some hypothesis to justify them.

MINOR ERRORS AND POSSIBLE TYPOS

- Normalize the use of "open-source" vs "open source"

Reviewer #3: My review exceeds the length limit and I am uploading the full review as an attachment. A summary of my review is below:

This article undertakes a longitudinal study of participation in FOSS projects, considering commit frequency, lines of code, and code comments. The work concludes that contributors are less active now, and that this decline is more strongly associated with large organizations than smaller ones. This finding is associated with three outcomes: the relationship between peer production and commercial production is maturing, the role of large organizations as intermediaries is declining, and the "likelihood of new big formal organizations" managing "emerging" projects is "systematically declining". The work concludes that this result suggests that open organizing is a "pipe dream".

This paper tackles an important concern with an ambitious scope. The writing is clear, and I appreciate the attention to a range of FOSS-related organizations. The work has strong potential to contribute to the field, with implications for computer science, social computing, and organizational studies.To the extent these organizations are in a state of failure, we need to know about it.

Overall, however, I find that this project faces a crossroads; there is a disconnection between the evidence developed and the argument made, and one side or the other needs to give way. In particular, there needs to be a clear connection between all of the core elements of the paper. To assist in tracking my argument in parallel with the paper's argument, I have broken my concerns into six themes below: the unit of analysis, the sample, the measures, the methods, the connection between the analysis and the discussion, and the ultimate conclusions. However, the gist of my concern (inconsistency between the design section, analysis section, and introduction/discussion/conclusions) is threaded throughout the paper, so some of my concerns as I describe them may seem to repeat/reprise previous concerns; for this I apologize in advance.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: No

Reviewer #3: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Attachments

Attachment

Submitted filename: Review PONE-D-20-37277R1.pdf

PONE-D-20-37277R2

Free and Open Source Software Organizations: A large-scale analysis of code, comments, and commit frequency using OpenHub.net data source

PLOS ONE

Dear Dr. Jemielniak,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

As you can see below, both reviewers are highly positive concerning your article. Nevertheless, Reviewer 3 provided a detailed list of issues (already included in a previous report) that should be addressed and/or properly discussed. In particular, your revision should pay special attention to point 2 in the reviewer's report (Evidence-Claims Link) as it is directly related to PLOS ONE's publication criterion #4 (https://journals.plos.org/plosone/s/criteria-for-publication#loc-4).

Please submit your revised manuscript by Aug 14 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Sergi Lozano

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: All comments have been addressed

Reviewer #3: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #2: Yes

Reviewer #3: Partly

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: I would like to thank authors for the detailed response and for addressing my comments. In my opinion, the new version of the paper has improved with regard to the previous one. I therefore recommend its acceptance.

Reviewer #3: I found myself even more excited about this paper having reviewed the latest version and I appreciate the revision efforts and response to comments. This is valuable and informative work that I am eager to see succeed. However, some of my core concerns remain, particularly with respect to the claims made versus the evidence presented.

Content-related concerns:

1. Abstract. I reprise my earlier comments here -- the final claim of the abstract is unsustained and needs to be amended. With apologies for the format of the below:

A. "as...the role of large FOSS organizations serving as platforms diminishes" -- it's not clear what is meant by "platforms" here, or whether these organizations seek to serve as such, or whether this service is indeed declining. What's presented is a decline in productivity and growth by various metrics, not a decline in "role as platform".

B. "the likelihood of new formal organizations" -- the analysis presented in this paper explicitly does not address new organizations nor does it measure likelihoods.

C. "managing the...projects emerging is systematically declining" -- the analysis does not address emerging projects; to claim a systematically declining likelihood, one would need evidence addressing the system rather than only its long-lived members (to capture 'systematically'), as well as measuring which organizations an emerging project goes on to join (to capture 'likelihood').

2. Evidence-Claims Link. The "organizations supplanted by platforms" argument is problematic and emblematic of the necessity to connect claims and evidence I cited in my previous review. The use of the term "platform" is unclear and the argument of platforms supplanting organizations is not sustained by the analysis presented.

A. In the abstract, organizations are said to have a role to serve as platforms; 1A has the opener for my points on this. Organizations create and use tools like GitHub to manage their work, but Mozilla for example is not Git nor does it frame itself as a Git provider.

B. The paper presents an argument that "platforms" are supplanting "organizations" (p. 16). This claim needs evidence. It may be the case that new project founders don't see a need to join an organization when a platform is sufficiently robust, but evidence for this is not presented because of the way the sample is constructed. Valid evidence here would perhaps look like joining rates of new projects through processes like the Apache Incubator versus similarly-successful projects staying independent. But in this case, there's still no supplanting per se because this is a false dichotomy: every project needs both an organization of some kind and an instance on github or similar, with sufficient value in terms of users and code already accumulated before they could hope to join any organization.

3. Sample. The response to my comments with respect to the sample are very much appreciated; a few more of these details should make their way into the paper, given the effort that was expended in this regard. I don't think it's necessary to include the data source in the title---my concern here is more about substance: reporting what is known about the sample, how it is constructed and vetted and verified, its limitations, and so on.

4. Collection of minor points:

A. The paper would benefit from a modest revision pass for typographical issues. I observed single-sentence paragraphs and some of the captions had typos. Y-axis labels on the figures would be useful.

B. Figure 4 is difficult to make sense of; it may be possible to improve it using a faceted plot or simply a repeat of the style of Figure 3.

C. I'm not sure it's necessary to publish the QQ plot; a direct visualization of the distribution or a measure of skewness would be reasonable.

D. "technological platforms" (p. 17) seems like an overbroad scope; I think what's meant here is only source control/collaboration platforms like GitHub.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: No

Reviewer #3: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Free and Open Source Software Organizations: A large-scale analysis of code, comments, and commits frequency.

PONE-D-20-37277R3

Dear Dr. Jemielniak,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Sergi Lozano

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments: