Machine learning based prediction of recurrence after curative resection for rectal cancer

Youngbae Jeon; Young-Jae Kim; Jisoo Jeon; Kug-Hyun Nam; Tae-Sik Hwang; Kwang-Gi Kim; Jeong-Heum Baek

doi:10.1371/journal.pone.0290141

Peer Review History

Original SubmissionAugust 1, 2023
20 Oct 2023 Decision Letter - Chong-Chi Chiu, Editor PONE-D-23-21370Machine learning based prediction of recurrence after curative resection for rectal cancerPLOS ONE Dear Dr. Baek, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. The comments of two reviewers are listed below. Please submit your revised manuscript by Dec 04 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Chong-Chi Chiu Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. Please note that PLOS ONE has specific guidelines on code sharing for submissions in which author-generated code underpins the findings in the manuscript. In these cases, all author-generated code must be made available without restrictions upon publication of the work. Please review our guidelines at https://journals.plos.org/plosone/s/materials-and-software-sharing#loc-sharing-code and ensure that your code is shared in a way that follows best practice and facilitates reproducibility and reuse. 3. We note that you have stated that you will provide repository information for your data at acceptance. Should your manuscript be accepted for publication, we will hold it until you provide the relevant accession numbers or DOIs necessary to access your data. If you wish to make changes to your Data Availability statement, please describe these changes in your cover letter and we will update your Data Availability statement to reflect the information you provide. 4. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information. Additional Editor Comments: Please revise your article according to the suggestions of two reviewers. Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: I Don't Know Reviewer #2: I Don't Know ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: This study aimed to analyze factors related to rectal cancer recurrence after curative resection using different machine learning technique based on the analysis of 961 patients who underwent curative surgery for rectal cancer between 2004 and 2018 at Gil Medical Center. Overall, they found that SVM had best AUC, and the most influential factor across all machine learning methods except LR was found to be pT. Major comment 1. This finding of this study was based on the data collected between 2004 and 2018. Because anti-cancer treatment had great improvement with time, it is better to use more updated data to establish prediction model. 2. The discussion about the clinical implication may be added. Minor comment 1. When you introduce an abbreviation in the abstract and text, you should first provide the full term followed by the abbreviation in parentheses. 2. Please add a new figure to reveal the process of patient selection. 3. Please briefly state how to treat patients with rectal cancer in the study site. 3. Please discuss the strength of the present study. Reviewer #2: This study analyzed factors influencing rectal cancer recurrence after curative surgery using machine learning. It involved 961 patients who underwent surgery, excluding specific cases. Data imbalance was addressed with SMOTETomek. The top eight predictive variables included pT stage, sex, concurrent chemoradiotherapy, pN stage, age, postoperative chemotherapy, pTNM stage, and perineural invasion. Support Vector Machine (SVM) yielded the highest AUC (0.831) for recurrence prediction, with sensitivity, specificity, and accuracy of 0.692, 0.814, and 0.798. The study emphasizes vigilance for patients with high pT stages during postoperative follow-up in rectal cancer cases. Comments 1. Please spell full name of pT, pN, pTNM for the first time in the abstract section. 2. Please describe how to compare the AUC value among various model and define the best machine learning model using statistical analysis. 3. The underlying conditions and comorbidities of included patients should be added in the table 1. 4. Briefly describe how to manage the patients with rectal cancer in the study site. 5. Add some discussion about why SVM could be the best predicted model. ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review?** For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0290141.r001
Revision 1
4 Dec 2023 Author Response ● Reviewer #1: This study aimed to analyze factors related to rectal cancer recurrence after curative resection using different machine learning technique based on the analysis of 961 patients who underwent curative surgery for rectal cancer between 2004 and 2018 at Gil Medical Center. Overall, they found that SVM had best AUC, and the most influential factor across all machine learning methods except LR was found to be pT. Major comment 1. This finding of this study was based on the data collected between 2004 and 2018. Because anti-cancer treatment had great improvement with time, it is better to use more updated data to establish prediction model. Answer) First of all, we declare that every cancer patient except involving clinical trials received chemotherapy or radiotherapy according to the government guideline, because cancer treatment in South Korea is totally covered by national health insurance. As adjuvant chemotherapeutic regimens for colorectal cancer, 5-FU/leucovorin have covered by national health insurance before 2000s, 5-FU/leucovorin with oxaliplatin (FOLFOX) have covered in the mid-2000s, and capecitabine with oxaliplatin (CAPOX) have covered in the 2010s. Because our study included only non-metastatic rectal cancer, there was no need to consider changes in palliative chemotherapeutic regimen such as target agents or multikinase inhibitors, and there was no significant change in adjuvant chemotherapeutic agents during study periods (CAPOX is only covered in colon cancer). Furthermore, in rectal cancer, totally neoadjuvant treatment (TNT) is not covered by national health insurance. Exceptionally, 5-fluorouracil/leucovorin with oxaliplatin (FOLFOX) regimen has been covered by national health insurance since August 2016, after ADORE study suggested that FOLFOX showed better oncologic outcomes than the conventional 5-fluorouracil/leucovorin for the patients with ypStage Ⅱ,Ⅲ. In current study, because the patients who received FOLFOX chemotherapy after August 2016 were only 4.1% (n=40/961), it is thought to be a negligible impact to evaluate the predictive factors of recurrence. However, your comment is reasonable and very important. Therefore, we added above contents in discussion section (page 17, line 3). 2. The discussion about the clinical implication may be added. Answer) We added more clinical implication including answer of comment number 1 in discussion section. Minor comment 1. When you introduce an abbreviation in the abstract and text, you should first provide the full term followed by the abbreviation in parentheses. Answer) Including the above, all missing full names of the words have been filled in, and the meaning of the words have been descripted more clearly. 2. Please add a new figure to reveal the process of patient selection. Answer) We added a new figure and figure legends (Figure 1. Flowchart of the patient selection) in method section (page 7, line 4). Consequently, numbering of the following figures have been revised. 3. Please briefly state how to treat patients with rectal cancer in the study site. Answer) We added the process of treating patient with rectal cancer in method section (Patient selection and dataset, page 6, line 13). 3. Please discuss the strength of the present study. Answer) We added the strength of our study in the bottom of discussion section. ● Reviewer #2: This study analyzed factors influencing rectal cancer recurrence after curative surgery using machine learning. It involved 961 patients who underwent surgery, excluding specific cases. Data imbalance was addressed with SMOTETomek. The top eight predictive variables included pT stage, sex, concurrent chemoradiotherapy, pN stage, age, postoperative chemotherapy, pTNM stage, and perineural invasion. Support Vector Machine (SVM) yielded the highest AUC (0.831) for recurrence prediction, with sensitivity, specificity, and accuracy of 0.692, 0.814, and 0.798. The study emphasizes vigilance for patients with high pT stages during postoperative follow-up in rectal cancer cases. Comments 1. Please spell full name of pT, pN, pTNM for the first time in the abstract section. Answer) Including the above, all missing full names of the words have been filled in, and the meaning of the words have been descripted more clearly. 2. Please describe how to compare the AUC value among various model and define the best machine learning model using statistical analysis. Answer) In the discussion section, we statistically compared the AUC values among the models and added that SVM is the best recurrence prediction model. (page 15, line 17) 3. The underlying conditions and comorbidities of included patients should be added in the table 1. Answer) We collected the patients’ data from 15 years of electric medical records, and only clearly recorded variables were collected to reduce missing data. Although there was no information of comorbidity in old records, ASA (American Society of Anesthesiologists) classification was definitely recorded in all patients. ASA classification is a method to determine the patients’ systemic condition before surgery under general anesthesia, so we can evaluate the severity of major systemic diseases. Therefore, we did not adopt comorbidity, which had a lot of missing data, as a variable, but adopted ASA classification, which had no missing data. It is believed that ASA classification can generally indicate the patients’ underlying condition and comorbidities. 4. Briefly describe how to manage the patients with rectal cancer in the study site. Answer) We added the process of treating patient with rectal cancer in method section (Patient selection and dataset, page 6, line 13). 5. Add some discussion about why SVM could be the best predicted model. Answer) We added a discussion in the discussion section about why SVM is the best predictive model. (page 18, line 3) Thank you again for your insightful comments and suggestions. We appreciate you spending your valuable time in revising our paper. Sincerely, Attachments Attachment Submitted filename: Response to Reviewers_PLOSEONE.docx https://doi.org/10.1371/journal.pone.0290141.r002
6 Dec 2023 Decision Letter - Chong-Chi Chiu, Editor Machine learning based prediction of recurrence after curative resection for rectal cancer PONE-D-23-21370R1 Dear Dr. Baek, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Chong-Chi Chiu Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0290141.r003
Formally Accepted
8 Dec 2023 Acceptance Letter - Chong-Chi Chiu, Editor PONE-D-23-21370R1 Machine learning based prediction of recurrence after curative resection for rectal cancer Dear Dr. Baek: I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org. If we can help with anything else, please email us at customercare@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Professor Chong-Chi Chiu Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0290141.r004

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .