Using machine learning to determine the nationalities of the fastest 100-mile ultra-marathoners and identify top racing events

Beat Knechtle; Katja Weiss; David Valero; Elias Villiger; Pantelis T. Nikolaidis; Marilia Santos Andrade; Volker Scheer; Ivan Cuk; Robert Gajda; Mabliny Thuany

doi:10.1371/journal.pone.0303960

Peer Review History

Original SubmissionDecember 29, 2023
12 Feb 2024 Decision Letter - Stevo Popovic, Editor PONE-D-23-43701Using Machine Learning to Determine the Nationalities of the Fastest 100-Mile Ultra-Marathoners and Identify Top Racing EventsPLOS ONE Dear Dr. Knechtle, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Mar 28 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Stevo Popovic, Ph.D. Academic Editor PLOS ONE Journal requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. Please note that PLOS ONE has specific guidelines on code sharing for submissions in which author-generated code underpins the findings in the manuscript. In these cases, all author-generated code must be made available without restrictions upon publication of the work. Please review our guidelines at https://journals.plos.org/plosone/s/materials-and-software-sharing#loc-sharing-code and ensure that your code is shared in a way that follows best practice and facilitates reproducibility and reuse. 3. In the online submission form you indicate that your data is not available for proprietary reasons and have provided a contact point for accessing this data. Please note that your current contact point is a co-author on this manuscript. According to our Data Policy, the contact point must not be an author on the manuscript and must be an institutional contact, ideally not an individual. Please revise your data statement to a non-author institutional point of contact, such as a data access or ethics committee, and send this to us via return email. Please also include contact information for the third-party organization, and please include the full citation of where the data can be found. 4. Your ethics statement should only appear in the Methods section of your manuscript. If your ethics statement is written in any section besides the Methods, please delete it from any other section. 5. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. Additional Editor Comments: I am so please to inform you I have collected two reviews and we are ready to go ahead with the evaluation process. It is your turn now to read the reviews and carefully revise the manuscript according to the requirements of the reviewers. I would appreciate if you prepare the comments with some arguments and adequate justifications for both of them. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Partly ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: N/A ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: Manuscript Number: PONE-D-23-43701 Manuscript Title: Using Machine Learning to Determine the Nationalities of the Fastest 100-Mile Ultra-Marathoners and Identify Top Racing Events The manuscript is interesting and well written with several strengths. I just have a few minor comments to make. - Overall, the different sections of the manuscript are well written. - Line 50: delete “(ML)”. - Line 60: Please, change “offered” to “occured”. - Lines 79 – 80: Missed reference. - Lines 103 – 105: Change “In this study, therefore, we sought to determine the country of origin of the fastest runners and the location of the fastest race courses.” to “In this context, we undertook this research to determine the country of origin of the fastest runners and the location of the fastest race courses.” - Line 142: Change “ML” to “machine learning (ML)” - Line 165: What is “MAE” - Line 257 - 269: The first paragraph of the discussion is too large, I would suggest creating a small paragraph. - Line 258 – 260: Replace “Based on a recent study, we assumed that the fastest athletes would be found to originate from Sweden, Hungary, Russia, Brazil, or Lithuania” to “Based on a recent study, we assumed that the fastest athletes would come from Sweden, Hungary, Russia, Brazil, or Lithuania.” - Line 374: delete “had”. - Line 376: delete “Better”. - Line 384: Change “offered” to “occurred”. - The limitations are properly discussed. - Figures 3, 4, 5, and 6: change “PDP” to “partial dependence plots (PDP)”. Reviewer #2: The work represents an interesting attempt to apply the XGBoost algorithm to determine the country of origin of the fastest runners and to identify the fastest races. The input characteristics under consideration are: athletes nationality, age group, gender, as well as the event country, while the output is the estimation of speed. The mentioned application has several limitations, the most important of which is the selection of input characteristics, i.e., the ability to obtain a good prediction from them. Another limitation is the dataset, which contains data from different types of races (championships, recreational races), undoubtedly resulting in a biased estimator. Additionally, for certain countries, there is a very small number of samples, and samples are with a high standard deviation. All of this does not lead to a high-quality estimator. However, my recommendation is for the work to be published, considering that the authors are aware of all the shortcomings of the estimator they have made, they have adequately addressed them in the paper and addressed potential risks, so the work can be a good guide for future research in this area. I suggest incorporating an analysis of the median as it often provides a more robust measure than the mean, particularly when dealing with datasets containing significant deviations in extreme values. ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: Yes: Nejmeddine Ouerghi Reviewer #2: No ******** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0303960.r001
Revision 1
31 Mar 2024 Author Response Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Partly ________________________________________ 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: N/A ________________________________________ 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ________________________________________ 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ________________________________________ 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: Manuscript Number: PONE-D-23-43701 Manuscript Title: Using Machine Learning to Determine the Nationalities of the Fastest 100-Mile Ultra-Marathoners and Identify Top Racing Events The manuscript is interesting and well written with several strengths. I just have a few minor comments to make. - Overall, the different sections of the manuscript are well written. - Line 50: delete “(ML)”. Answer: We agree with the expert reviewer and deleted as suggested. - Line 60: Please, change “offered” to “occured”. Answer: We agree with the expert reviewer and changed as suggested - Lines 79 – 80: Missed reference. Answer: We agree with the expert reviewer and added as suggested - Lines 103 – 105: Change “In this study, therefore, we sought to determine the country of origin of the fastest runners and the location of the fastest race courses.” to “In this context, we undertook this research to determine the country of origin of the fastest runners and the location of the fastest race courses.” Answer: We agree with the expert reviewer and changed as suggested - Line 142: Change “ML” to “machine learning (ML)” Answer: We agree with the expert reviewer and changed as suggested - Line 165: What is “MAE” Answer: We agree with the expert reviewer and changed to MAE (Mean Absolute Error) - Line 257 - 269: The first paragraph of the discussion is too large, I would suggest creating a small paragraph. Answer: We agree with the expert reviewer and changed as suggested to: The present study aimed to determine the country of origin of the fastest 100-mile runners and the countries hosting the fastest 100-mile race courses using an XG Boost regression model. We found that the event location (i.e. the country where the race is held) was the most important predictor for a fast 100-mile race time where the fastest race courses are offered in Romania, Israel, Switzerland, Finland, Russia, the Netherlands, France, Denmark, Czechia, and Taiwan. Regarding the first aim, the fastest athletes come mostly from Eastern European countries (i.e., Lithuania, Latvia, Ukraine, Finland, Russia, Hungary, and Slovakia). - Line 258 – 260: Replace “Based on a recent study, we assumed that the fastest athletes would be found to originate from Sweden, Hungary, Russia, Brazil, or Lithuania” to “Based on a recent study, we assumed that the fastest athletes would come from Sweden, Hungary, Russia, Brazil, or Lithuania.” Answer: since we had to reduce that section this sentence was deleted - Line 374: delete “had”. Answer: We agree with the expert reviewer and deleted as suggested. - Line 376: delete “Better”. Answer: We agree with the expert reviewer and deleted as suggested. - Line 384: Change “offered” to “occurred”. Answer: We agree with the expert reviewer and changed as suggested. - The limitations are properly discussed. Answer: no changes are required - Figures 3, 4, 5, and 6: change “PDP” to “partial dependence plots (PDP)”. Answer: We agree with the expert reviewer and changed as suggested. Reviewer #2: The work represents an interesting attempt to apply the XGBoost algorithm to determine the country of origin of the fastest runners and to identify the fastest races. The input characteristics under consideration are: athletes nationality, age group, gender, as well as the event country, while the output is the estimation of speed. The mentioned application has several limitations, the most important of which is the selection of input characteristics, i.e., the ability to obtain a good prediction from them. Another limitation is the dataset, which contains data from different types of races (championships, recreational races), undoubtedly resulting in a biased estimator. Additionally, for certain countries, there is a very small number of samples, and samples are with a high standard deviation. All of this does not lead to a high-quality estimator. However, my recommendation is for the work to be published, considering that the authors are aware of all the shortcomings of the estimator they have made, they have adequately addressed them in the paper and addressed potential risks, so the work can be a good guide for future research in this area. I suggest incorporating an analysis of the median as it often provides a more robust measure than the mean, particularly when dealing with datasets containing significant deviations in extreme values. Answer: We recognize the limitations of this analysis, but could only use the data available. As a predictive model, there is no question about our XGB model ability (R2=0.23 in-sample tested). So the main strength of our study resides in using the model interpretability tools (reverse engineering) such as the PDP charts and the prediction distribution charts, to understand the 23% of explained output variability. Interestingly, but not as a surprise, the XGB model mostly learnt the descriptive statistical structure of the dataset. On the comment about the median, I would like to emphasize that there is indeed a mix of mean and median values in the charts. So I´ll explain here in detail, so there are no doubts about this. The country ranking tables (sorted by number of race records) are created simply by aggregating race records and then calculating the different stats. So they show the basic descriptive statistics (mean, std, min and max and the counts). The PDP charts are produced after the model has been trained and represent the relative value of the model output for a predictor. The curve highlights the mean values. Finally, the called “model interpretability charts” show group average values (means) in the middle chart (red curve) and median values in the top chart (boxplot with blue labels). Actually they are nearly the same. Attachments Attachment Submitted filename: PONE-D-23-43701_Revision 1_Comments for Reviewers.docx https://doi.org/10.1371/journal.pone.0303960.r002
6 May 2024 Decision Letter - Stevo Popovic, Editor Using Machine Learning to Determine the Nationalities of the Fastest 100-Mile Ultra-Marathoners and Identify Top Racing Events PONE-D-23-43701R1 Dear Dr. Knechtle, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Stevo Popovic, Ph.D. Academic Editor PLOS ONE Additional Editor Comments (optional): Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #1: All comments have been addressed Reviewer #2: All comments have been addressed ******** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ****** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: (No Response) ****** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: (No Response) Reviewer #2: (No Response) ****** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: (No Response) ****** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: Manuscript Number: PONE-D-23-43701R1 Manuscript Title: Using Machine Learning to Determine the Nationalities of the Fastest 100-Mile Ultra-Marathoners and Identify Top Racing Events The manuscript is interesting and nicely written. The authors have been well corrected and modified the manuscript according to my comments. I recommend to accept the manuscript for publication. Reviewer #2: (No Response) ****** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: Yes: Nejmeddine Ouerghi Reviewer #2: No ******** https://doi.org/10.1371/journal.pone.0303960.r003
Formally Accepted
20 May 2024 Acceptance Letter - Stevo Popovic, Editor PONE-D-23-43701R1 PLOS ONE Dear Dr. Knechtle, I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now being handed over to our production team. At this stage, our production department will prepare your paper for publication. This includes ensuring the following: * All references, tables, and figures are properly cited * All relevant supporting information is included in the manuscript submission, * There are no issues that prevent the paper from being properly typeset If revisions are needed, the production department will contact you directly to resolve them. If no revisions are needed, you will receive an email when the publication date has been set. At this time, we do not offer pre-publication proofs to authors during production of the accepted work. Please keep in mind that we are working through a large volume of accepted articles, so please give us a few weeks to review your paper and let you know the next and final steps. Lastly, if your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. If we can help with anything else, please email us at customercare@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Professor Stevo Popovic Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0303960.r004

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .