PONE-D-22-13227Evaluating the fairness of the Boston Marathon qualifying timesPLOS ONE

Dear Dr. Albrecht,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

In order to resolve the differences between reviewers, I have read the paper and recommend significant revision. Please review each reviewers comments and address their concerns. In addition, I recommend the following:

• I believe reviewer 2 makes a legitimate case for the use of fairness. While I believe you have defined the word, I question whether fairness is relevant in a scientific paper. I believe you should reposition your paper as evidence-based for qualifying times. This is a more direct way to direct your argument; Are the current standards evidence-based? It doesn't appear so. Is your approach better? I can't say, but we should establish standards based on the evidence, not arbitrary fixed numbers. That is the case the authors should make and a suggestion on how to get there!  
• There is inconsistent grammar, language use (like many in areas where present not past tense is used). The paper feels disjointed as if written by different individuals and then combined without regard to flow.
• There are insufficient references for a paper of this magnitude. In particular, much of the (limited) discussion is more extended conclusions and speculation and lacks any meaningful literature to support it. I recommend several references to review:
  • Bagley DOI: 10.1111/sms.13404
  • Tanaka DOI: 10.1113/jphysiol.2007.141879
  • Nicolaidis doi: 10.3390/ijerph16101777
  • Waldvogel doi: 10.3390/ijerph16132377
  • https://runrepeat.com/state-of-running
  • https://runnerclick.com/marathon-finishing-times-study-and-statistics/
• The introduction is extensive and well written but should end with a clear purpose and hypotheses. 
• The Methods are very confusing and poorly structured, and should be written in the past tense. The general writing is completely different from the intro and much of the methods are actually results. I recommend streamlining and reorganizing to clearly identify who the participants were, where the data came from, how it was analyzed with details on stats, and specific criteria for exclusion of data. I envision much of the data with figures should move to results.
• Results are extensive but should be streamlined and flow based on the story you with to tell in the discussion. 
• Figures are at times confusing with superfluous text on the figures. Titles need to be concise where possible and clearly identify what the figure is showing without excess story or text. I'm just having a hard time following the story you're telling.
  • ​I recommend identifying the key 3 or 4 figures and moving the rest to supplements to tell your story. 
• The discussion is inadequate for the paper and data you present. Two big paragraphs do little to help the paper and largely just try to tell us how much better your optimized times are. However, you haven't really made that case, and really should leave it to the conclusion and future rec's for research. It also needs to lean into supporting references. I suggest:
  • Restating the purpose and hypotheses then reviewing what you found. Then each subsequent paragraph targets a single finding.
  • An additional paragraph can cover unexpected findings and then perhaps one more paragraph to note the limits of this study and future recommendations.
• Now hit the conclusion with the overarching findings and message. Avoid over speaking your results.

Please submit your revised manuscript by Aug 07 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Chris Harnish, PhD

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf.

2. In your Methods section, please include additional information about your dataset and ensure that you have included a statement specifying whether the collection and analysis method complied with the terms and conditions for the source of the data.

3. We note that you have stated that you will provide repository information for your data at acceptance. Should your manuscript be accepted for publication, we will hold it until you provide the relevant accession numbers or DOIs necessary to access your data. If you wish to make changes to your Data Availability statement, please describe these changes in your cover letter and we will update your Data Availability statement to reflect the information you provide.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: No

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: No

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This study presents the results of primary scientific research. To this reviewer’s knowledge, the results reported in this manuscript have not been published elsewhere. Statistics and other analysis are performed to a high technical standard and are described in sufficient detail. Conclusions are presented in an appropriate fashion and are supported by the data. This article is presented in an intelligible fashion and is written in standard English. In addition, the researchers meet all applicable standards for the ethics of experimentation and research integrity.

I am curious on why those ten races were selected for data analysis. It is not noted in the methods. When looking at the top qualify races for Boston, several of them are not even listed (e.g. Honolulu, Houston, LA). Top Qualifying Races | Boston Athletic Association (baa.org). Please provide more information on this.

This reviewer suggests providing a shorter summary after each Figure. The current summaries are very lengthy and are what is exactly in the text of the article. In addition, the color coding is helpful but many readers will be looking at this article in black/white (especially if print). This reviewer would suggest looking at different ways to express the data so can be viewed by all. Since, this study looked at data prior to the pandemic, do the authors feel that this has now changed the outcome of their data. Boston has now opened up the race for more athletes to be able to run. This might be something to mention in the discussion. Overall, I enjoyed reading this article. As a runner myself this is always great to see how investigators are looking at marathon data.

Reviewer #2: Dear authors,

The study aimed to investigate whether the Boston Marathon qualifying times are “Fair”. Although this is an interesting topic and deserves attention, in a way that future marathons' qualifying times seek to include a more homogenous number of qualifiers across genders, I have some concerns about the way the manuscript was presented and reported.

First, I believe that if authors want to evaluate fairness, they should include a proper definition of the term and how this affected their research question. Fairness in sports is a rather complicated matter, involving several other factors that were not considered in the study and not discussed properly. In my opinion, the study simply aimed to analyse whether the current Boston qualifying times include a similar percentage of women and men, and whether the times could be changed by applying a mathematical model in a way that the difference in proportion of qualifiers across genders is smaller (although the authors do not report how much this difference is).

Second, it is not clear from the introduction and the aims of the study what is the current differences in the number of women and men that qualify in each race. This is a piece of very important information, that would allow us to know if there is indeed a large difference in the proportion of qualifiers across genders.

Also, the methods section is quite complicated to follow and there is some information that is not relevant to the aims of the study. For example, the authors surprisingly collected data from 10 different other marathons, and it is not clear why they did that. I believe most of the data that were analysed and presented are not relevant to the aims of the study which was to analyse whether the Boston qualifying times include a similar proportion of runners across genders. I believe the authors should then focus on the “Proportion Analysis” assessing the Boston qualifying times and how changing it would minimize the difference in the proportion of qualifiers across each race. The authors also included in the methods sections, several results from their analysis that should be included only in the “Results” section.

Finally, the “Discussion” section does not represent all the data that was analysed. The authors previously presented the results from several different analyses, yet the discussion had only two paragraphs and only included one reference. The authors should aim to discuss the results of their study comparing it with previous published studies, making sure that all the data that was analysed, is discussed as well.

Kind regards.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PONE-D-22-13227R1Data-driven evaluation of the Boston marathon qualifying timesPLOS ONE

Dear Dr. Albrecht,

Thank you for submitting your revised manuscript to PLOS ONE. While I believe the manuscript has merit, reviewers could come to a consensus on this. Therefore, after careful review I enlisted a third non-academic reviewer with significant expertise in statistics and data analysis. It has been enlightening to read a fresh review from a unique point of view. I believe reviewer 3 has offered the authors a clear pathway forward while improving the overall value of the manuscript. Therefore, we invite you to submit a revised version of the manuscript that addresses the key areas recommended below.

• Please give careful consideration to the statistical methods provided.
• Consider carefully reviewing several databases on running, like that of run repeat. 
• Addressing the Fairness question; reviewers struggled with this concept and I would decide on one clear path and stay with it. Reviewer 3 offers some ideas.
• Be clear on what your messaging is on the results. Editor note: I believe you've overplayed the grand outcomes and missed some key points, like the Boston qualifier really breaks down as a specific age. 
  • ​I believe reviewer 3 has offered some of the best advice to tackle the results issues, but it is a bit of work.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Chris Harnish, PhD

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: (No Response)

Reviewer #3: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #2: Partly

Reviewer #3: Partly

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: Yes

Reviewer #3: No

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: Dear authors,

Thank you for considering my previous comments and making the necessary adjustments. Although I appreciate the authors’ efforts to address all points, there are still some important key points that need to be clarified and properly addressed.

I believe the manuscript is still a bit hard to follow and the message is still not clear. The aim of the study was to analyse how adjusting (not sure if “optimising” is adequate) the qualifying times of the Boston marathon would result in a more equal proportion of men and women across age groups. Yet, there is a lot of information that is not relevant to the aims of the study. For example, I believe most of the figures included in the supplemental material are irrelevant and should be excluded from the study. The introduction has improved and is properly written, clearly stating the problem and how you aim to solve it. However, the other sections are still quite confusing. In the methods section, I had a difficult time trying to understand how the demographics from the other 10 marathons would help you achieve your aim. The study adopted a pretty robust and complicated analysis, so every step of the data collection and analysis needs to be clearly explained. It is also important to report how many runners took part in each race you analysed and also split by age groups.

I still think there are too many figures that do little to help you deliver your message. The authors also did not attach figure 1 to their submission. In your results, you found that decreasing the qualifying times for the 35-39, 40-44, 45-49, 50-54 and 60-64 age groups would “optimise” the distribution of men and women across age groups. However, this approach, at the same time, substantially lowers the overall absolute number of qualifiers in the race, which will, in fact, exclude more runners. For this reason, I don’t think your approach will “optimise” the qualifying times, as you would essentially exclude more people from taking part in the race.

The discussion should start by stating the aims of the study and providing a comprehensive summary of the main findings. The following paragraphs should then expand the main findings comparing your results with previous studies. I think the discussion is quite speculative and not supported by adequate evidence. For example, reference number 17, although interesting, is a blog post that probably was not peer-reviewed. There are also a couple of paragraphs (lines 199 to 214) including information not relevant to the study.

Reviewer #3: This is an interesting piece of work that I think should be published with revisions. Please see attachment: the 'no' under #3 is because 'partly' was not option.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: No

Reviewer #3: Yes: Gregory P. Swinand

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Attachments

Attachment

Submitted filename: Review PONE-D-22-13227R1 GS.docx

Data-driven evaluation of the Boston marathon qualifying times

PONE-D-22-13227R2

Dear Dr. Albrecht,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. In addition, I ask that you address one minor suggestion by our reviewer to highlight the specific impacts on qualifying among women. This can be particularly helpful for those researchers who may wish to build on your research.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Chris Harnish, PhD

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #3: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #3: my main concerns were over the sample selection and how they dealt with that. The comparing to the larger database on what i presume are the only available covariates -- age, sex, and time, seems reasonable. the other concern was toning down the idea that this procedure was better or whatever and it is clearly explained that optimal is only with regards to minimising the objective function; which is fine. there is still a curious issue that the proposed new way would lead to fewer women qualifying for certain age categories. for future research, if they want to include, the final note that qualifying has become less difficult in the immediate aftermath of the pandemic is quite interesting, and how these things evolve over time or react to other such world-impacts; maybe a dynamic or prescriptive perspective could be considered next!

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #3: Yes: Dr Gregory P. Swinand

**********