PONE-D-23-27406Gender differences in Dutch research funding over time: A statistical investigation of the Innovation Scheme 2012–2021PLOS ONE

Dear Dr. Albers,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Dec 11 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Julian D. Cortes

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please update your submission to use the PLOS LaTeX template. The template and more information on our requirements for LaTeX submissions can be found at http://journals.plos.org/plosone/s/latex.

3. Please upload a copy of Supporting Information Figure/Table/etc. which you refer to in your text on page 7,8 and 10.

Additional Editor Comments:

Dear author/s, thanks for submitting your work to PLoS ONE , I contrasted the core sections of your work with our seven criteria for publication. In line/Besides reviewers' suggestions, consider the following recommendations:  1 Experiments, statistics, and other analyses are performed to a high technical standard and are described in sufficient detail.Extend your argument on the potential relevance of institutional affiliation in the in the probability of success in your model (in “Model” or “Discussion” sections?)Following good practices standards in statistical practice (https://www.nature.com/articles/s41562-021-01211-8) an additional section aimed to visualizing the data, besides the “Observed success probabilities” figures, could be added to excel the descriptive/explicative interpretation of results in logistic regression models, such as: - Correlation Matrix- ROC Curve- Confusion Matrix2 Conclusions are presented in an appropriate fashion and are supported by the data.Extend your conclusion section with: - The paragraph in which it is stated that “more women than men leave academia before” is of great interest for the community of practice. Extend your conclusions on that. - The limitations of the study. - Further research agenda derivates from it. 3 The article is presented in an intelligible fashion and is written in standard English.Extend the abstract content with the following criteria: “Background or Introduction – What is currently known? Start with a brief, 2 or 3 sentence, introduction to the research area. Objectives or Aims – What is the study and why did you do it? Clearly state the research question you’re trying to answer. Methods – What did you do? Explain what you did and how you did it. Include important information about your methods, but avoid the low-level specifics. Some disciplines have specific requirements for abstract methods. CONSORT for randomized trials. STROBE for observational studies PRISMA for systematic reviews and meta-analyses. Results – What did you find? Briefly give the key findings of your study. Include key numeric data (including confidence intervals or p values), where possible. Conclusions – What did you conclude? Tell the reader why your findings matter, and what this could mean for the ‘bigger picture’ of this area of research.”4 The article adheres to appropriate reporting guidelines and community standards for data availability.Data and code for replication was submitted by the authors: https://osf.io/8bfaz/. Consider mention it in the manuscript body (“Data & materials” section?).  I hope you can incorporate the above suggestions to improve your already valuable work. Sincerely, Julián D. Cortés Associated Editor

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: As regards to form, the paper is fit for publication, and I have no comments whatsoever in that regard. However, I believe that two relatively important issues prevent it from being published in its actual form.

The first one, and probably the most important, relates to results interpretation. In the discussion section, the authors say: "In absence of any gender effects in the quality of applications and considerations of the assessment committee, the probability of finding at least the gender difference was smaller than 0.001, i.e. there is a very significant gender difference to the favour of women." While the first part of this sentence is adequate, the last part is plain wrong. P-values in logistic regression are to be interpreted in the same way as in any other frequentist statistical test, that is, as the probability of obtaining results at least extreme under the hypothesis we aim to nullify. If the probability is lower than a specified threshold, we can safely reject that hypothesis, and that's it. P-values do not say anything about the significance or the plausibility of any other hypothesis, since the hypothesis we aim to nullify is the only one being tested. In order to say anything more, you need to measure effect size. We invite the authors to look at how this can be done as regards to logistic regression; as a suggestion, a thorough analysis of odds ratios should be appropriate.

The second point relates to a comment made at the beginning of the article, where the authors mention that "in certain situations, such as childcare responsibilities, terms [for the different funding programs] can be extended". Given the pervasive an systemic gender imbalance all over the world as regards to childcare, it is reasonable to hypothesize that this childcare extension could have been a major factor in improving the gender imbalance situation as regards to the Veni grants, especially since it is probably during that period the necessities of child caring places the greatest strain on female scholars' careers. We suppose that data regarding such extensions is not available as of now; if it were, I don't think this research would be complete or publishable without a thorough analysis of grant extensions. But the unavailability of such data doesn't make the case of childcare extensions for Veni grants less crucial to the general study of gender imbalances in research funding. In light of this, we believe it is imperative that the authors discuss this matter thoroughly, at the very least as an unfortunate and (perhaps) unavoidable limitation of their research.

Reviewer #2: The manuscript aims at analyzing gender differences in award rates for research grants of the Dutch Research Council (NWO; Veni, Vidi, Vici). In particular, the focus is on temporal developments: Do gender inequalities have changed over time? To do so, the authors use publicly available NWO data on submissions and awarded grants (by gender) for the years 2012 to 2021. The analyses are mainly based on logistic regressions with interaction terms to detect temporal changes of the gender effect. The key finding is that throughout the observation period the success probabilities of female applicants have increased compared to success rates of male applicants.

The authors convincingly show how their work adds to the literature. I think that the research question can be answered well with the data. The analyzes are well comprehensible and well documented so that replication is possible. Although the authors mainly intend to provide descriptions of developments, they also shortly discuss possible explanations for the observed results. The discussion of their results reveals that they are aware of the limitations of their study. I really appreciate the clear writing style.

A few more detailed comments:

Lines 66-68: I would not say that all models indicate lower success rates of males. For example, model 3 shows a positive gender (main) effect, meaning that at year=0 males have an advantage over females. By the way, I would recommend to code the variable “year” slightly different, so that the value zero refers to the first year in the data (i.e., 2012=0, … , 2021=9)

Line 77: As far as I know Lutter & Schröder do find evidence for gender inequalities controlling for publication output (in favor of female researchers).

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Gender differences in Dutch research funding over time: A statistical investigation of the Innovation Scheme 2012–2021

PONE-D-23-27406R1

Dear Dr. Albers,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Stijn Michielse, PhD

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Thanks for the revisions. Two reviewers had reviewed the manuscript and came independently to the conclusion that the manuscript is acceptable for publication.

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: (No Response)

Reviewer #2: Thank you for the revision. I think the manuscript has benefited from the additions. In my view, it is now acceptable for publication.

However, please correct the following points before publication:

Line 275: "the black curve in 3" --> "the black curve in Figure 3"

Line 317: Include a reference at the end of the sentence

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

**********