Feasibility of using a novel automatic cardiac segmentation algorithm in the clinical routine of lung cancer patients

Robert Neil Finnegan; Lucia Orlandini; Xiongfei Liao; Jun Yin; Jinyi Lang; Jason Dowling; Davide Fontanarosa

doi:10.1371/journal.pone.0245364

Peer Review History

Original SubmissionJuly 13, 2020
2 Sep 2020 Decision Letter - Dzung Pham, Editor PONE-D-20-20670 Feasibility of using a novel automatic cardiac segmentation algorithm in the clinical routine of lung cancer patients PLOS ONE Dear Dr. Finnegan, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. In particular, both reviewers suggested adding a more complete review of related literature, as well as a description of the clinical impact of the proposed methods. Reviewer 1 also had significant concerns about the evaluation. Please submit your revised manuscript by Oct 17 2020 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols We look forward to receiving your revised manuscript. Kind regards, Dzung Pham Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. In your Data Availability statement, you have not specified where the minimal data set underlying the results described in your manuscript can be found. PLOS defines a study's minimal data set as the underlying data used to reach the conclusions drawn in the manuscript and any additional data required to replicate the reported study findings in their entirety. All PLOS journals require that the minimal data set be made fully available. For more information about our data policy, please see http://journals.plos.org/plosone/s/data-availability. Upon re-submitting your revised manuscript, please upload your study’s minimal underlying data set as either Supporting Information files or to a stable, public repository and include the relevant URLs, DOIs, or accession numbers within your revised cover letter. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. Any potentially identifying patient information must be fully anonymized. Important: If there are ethical or legal restrictions to sharing your data publicly, please explain these restrictions in detail. Please see our guidelines for more information on what we consider unacceptable restrictions to publicly sharing data: http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. Note that it is not acceptable for the authors to be the sole named individuals responsible for ensuring data access. We will update your Data Availability statement to reflect the information you provide in your cover letter. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Partly ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ******** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: The authors proposed an atlas-based automatic heart segmentation method to delineate the heart in 4D-CT images acquired from lung cancer patients. Although there are merits of this work, there are some changes must be done before it can be accepted: 1. The major question of this study is that how the proposed method working in clinical practice? How about the reproducibility? The various image quality might be a problem for the proposed method. 2. The literature review of the work is highly insufficient, and the authors missed the most recent deep learning based segmentation methods. The authors should cite the following publications with proper discussions. Zhuang, Xiahai, et al. "Evaluation of algorithms for multi-modality whole heart segmentation: an open-access grand challenge." Medical image analysis 58 (2019): 101537. Yang, Guang, et al. "Simultaneous left atrium anatomy and scar segmentations via deep learning in multiview information with attention." Future Generation Computer Systems 107 (2020): 215-228. Chengjia Wang et al. A Two-stage U-Net Model for 3D Multi-class Segmentation on Full-resolution Cardiac Data. The 9th Workshop on Statistical Atlases and Computational Modelling of MICCAI 2018, pp. 191--199. Zenglin Shi et al. Bayesian VoxDRN: A Probabilistic Deep Voxelwise Dilated Residual Network for Whole Heart Segmentation from 3D MR Images. The 21st International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2018), pp. 569--577. Yuanhan Mo et al. Deep Poincare Map: A Novel Method Coupling Dynamical System with Deep Learning for Left Ventricle Segmentation. The 21st International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2018), pp. 561--568. 3. Ideally, the authors should consider to compare with some deep learning based methods or at least have some discussions. 4. Why the automated results represent lots of ragged artefacts? Reviewer #2: The proposed study investigates using an atlas-based cardiac segmentation method in the context of 4D-CT for delineation of the heart. The manuscript is clear, concise, and well written. The paper does not propose a novel method, but rather a somewhat different application of a previously published method. I have a couple of major concerns: 1) The auto-contouring method was used only for contouring on the zero-phase and propagation to other phases was done via deformable image registration. This method of contour propagation is acceptable and widely performed in RT, however since the authors are comparing the proposed method to manually contoured structures (contoured on each phase), this leads to a biased comparison. Furthermore, the authors are essentially testing the feasibility of using 4DCT and DIR to propagate contours (which has been widely published on) rather than using the automated segmentation framework. The proposed method should either be applied independently to each breathing phase or the manual contours on the 0% phase should be propagated via DIR to the other phases. Since deformation vector fields must typically satisfy a smoothness constraint, this may also explain why contours generated via the atlas-based and DIR method are smoother than manual contours. 2) Many recent studies have posited that dose to the whole heart may not be adequate for correlating radiation dose with cardiac toxicity outcomes: -Hoppe, Bradford S., et al. "The meaningless meaning of mean heart dose in mediastinal lymphoma in the modern radiation therapy era." Practical radiation oncology 10.3 (2020): e147-e154. -McWilliam, Alan, et al. "Novel Methodology to Investigate the Effect of Radiation Dose to Heart Substructures on Overall Survival." International Journal of Radiation Oncology* Biology* Physics (2020). -Stam, Barbara, et al. "Dose to heart substructures is associated with non-cancer death after SBRT in stage I–II NSCLC patients." Radiotherapy and Oncology 123.3 (2017): 370-375. Given that previous publications by the authors have shown that the auto-segmentation method can be applied to cardiac substructures, in addition to the whole heart, I would suggest extending the analysis in the manuscript to include contouring of cardiac substructures. 3) The manuscript is very brief, and the immediate impact of the proposed method in the clinic is not clearly highlighted. Additionally, visual results from only one or two patients are shown. It would be helpful to include the segmentation results of all 10 phases on one or two patient slices. Minor comments/concerns: 1) While some deep learning methods for automatic segmentation require large datasets, the method published by Morris et al: Morris, Eric D., et al. "Cardiac substructure segmentation with deep learning for improved cardiac sparing." Medical physics 47.2 (2020): 576-586. required only 25 patient datasets for training, so the authors' claim may be unfounded. 2) As the authors mention in the discussion - large datasets may need to be used to reach definitive conclusions on the dosimetric impact of the proposed method, some study on the dosimetric impact should be included. A comparison in heart DVH based on contours from the proposed method, and manual contours on the MIP and average CT would be helpful. 3) The cardiac contours are propagated via respiratory-correlated 4DCT, which may not be associated with or fully capture cardiac motion. Given this is standard practice in radiotherapy clinics, it is not a major concern, but the authors should at least mention this in the discussion section. ******** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: Yes:** Joe Harms [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0245364.r001
Revision 1
25 Nov 2020 Author Response Reviewer responses We would like to extend our sincere thanks to the reviewers for providing constructive comments and thoughtful suggestions, which we feel have contributed to an improved manuscript. Overall, this has produced a more detailed and thorough description of both our work and the clinical and research context in which it is placed. Below we address each comment individually. Reviewer 1 Comment 1. The major question of this study is that how the proposed method working in clinical practice? How about the reproducibility? The various image quality might be a problem for the proposed method. Response 1. Thank you for this question. We agree that this is a primary concern. We have provided additional details regarding how we will implement our method in clinical practice, and discuss the challenge faced with variable image quality (Discussion - paragraph 3). Comment 2. The literature review of the work is highly insufficient, and the authors missed the most recent deep learning based segmentation methods. The authors should cite the following publications with proper discussions. (citations follow) Response 2. We greatly appreciate the feedback and have taken this into consideration. We agree that many recent advanced in deep learning show promise for translation into the clinic, and have discussed the suggested published articles, along with several more, in detail in the revised manuscript (Discussion - paragraphs 6 and 7). Comment 3. Ideally, the authors should consider to compare with some deep learning based methods or at least have some discussions. Response 3. Thank you again for this suggestion. Following from comment 2, we have also provided a comparison to recent relevant research for cardiac delineation. Comment 4. Why the automated results represent lots of ragged artefacts? Response 4. Thank you for this question. Using a multi-atlas approach, the automatic segmentation is essentially spatially averaged in 3D. Since tomographic imaging is reconstructed in axial planes, these segmentations can look jagged when viewed in 2D slices as a results of these factors. In three dimensions, automatic segmentations often look much smoother (as shown in Figure 2), which was a point raised by reviewer 2. Reviewer 2 Comment 1. The auto-contouring method was used only for contouring on the zero-phase and propagation to other phases was done via deformable image registration. This method of contour propagation is acceptable and widely performed in RT, however since the authors are comparing the proposed method to manually contoured structures (contoured on each phase), this leads to a biased comparison. Furthermore, the authors are essentially testing the feasibility of using 4DCT and DIR to propagate contours (which has been widely published on) rather than using the automated segmentation framework. The proposed method should either be applied independently to each breathing phase or the manual contours on the 0% phase should be propagated via DIR to the other phases. Since deformation vector fields must typically satisfy a smoothness constraint, this may also explain why contours generated via the atlas-based and DIR method are smoother than manual contours. Response 1. We greatly appreciate this in depth comment and the suggestion to further validate our methodology. In response, we have applied our automatic segmentation software to each phase independently. Results are given in Figures 3 and 4, with additional description in the revised manuscript (methods - final paragraph, results - final 2 paragraphs, discussion - end of paragraph 2). Comment 2. Many recent studies have posited that dose to the whole heart may not be adequate for correlating radiation dose with cardiac toxicity outcomes [reference list]. Given that previous publications by the authors have shown that the auto-segmentation method can be applied to cardiac substructures, in addition to the whole heart, I would suggest extending the analysis in the manuscript to include contouring of cardiac substructures. Response 2. Thank you for this suggestion. Firstly, we agree that much more research is required to better understand the link between radiation dose to cardiac substructures and cardiotoxicities. This is an area of research we are actively involved with (see our recent work published in Radiother. And Oncol.: https://doi.org/10.1016/j.radonc.2020.09.004). The aim of this work was to demonstrate the feasibility of clinical implementation, and as current clinical practice does not consider cardiac substructures, we decided this extra information was not relevant for this study. With better cardiotoxicity models utilising cardiac substructure doses, it is likely this will change in the future, and we hope our tool will be able to be used to its full potential. Outside of clinical radiotherapy, this software might be applied for retrospective analysis of cardiac substructure doses more widely, as it has been already. We have discussed these points in the revised manuscript (Discussion - paragraph 5) Comment 3. The manuscript is very brief, and the immediate impact of the proposed method in the clinic is not clearly highlighted. Additionally, visual results from only one or two patients are shown. It would be helpful to include the segmentation results of all 10 phases on one or two patient slices. Response 3. Thank you for this suggestion. We believe that the revised manuscript presents a more complete analysis, and a more thorough investigation. Following your suggestion, we have included visual results for the best, median, and worst patient segmentation results (Figure 5). Minor comments/concerns. Comment 1. While some deep learning methods for automatic segmentation require large datasets, the method published by Morris et al required only 25 patient datasets for training, so the authors' claim may be unfounded. Response 1. We thank you for bringing this to our attention. In the revised manuscript, we have added a description of deep learning methods and a comparison between relevant studies and our work (Discussion - paragraphs 6 and 7). It is correct that the work by Morris et al required only 25 cases to train a network, however a direct comparison to our framework is difficult as the data used by their group comprised paired CT/MRI (which is typically not available). Despite this, we are excited by the feasibility of using smaller datasets to train deep learning models, and look forward to future work in this area, as typically most studies present training datasets comprising much numbers of patient images. Comment 2. As the authors mention in the discussion - large datasets may need to be used to reach definitive conclusions on the dosimetric impact of the proposed method, some study on the dosimetric impact should be included. A comparison in heart DVH based on contours from the proposed method, and manual contours on the MIP and average CT would be helpful. Response 2. We appreciate this suggestion. In the revised manuscript we have included a dosimetric investigation for a patient selected from our study dataset with the smallest heart-tumour separation. We present a qualitative analysis of the differences in dose (see Figure 6), which we discuss (Discussion - paragraph 4). Comment 3. The cardiac contours are propagated via respiratory-correlated 4DCT, which may not be associated with or fully capture cardiac motion. Given this is standard practice in radiotherapy clinics, it is not a major concern, but the authors should at least mention this in the discussion section. Response 3. Thank you for this astute observation. We have raised this point in our revised manuscript (Discussion - end of paragraph 8) Attachments Attachment Submitted filename: Reviewer responses.docx https://doi.org/10.1371/journal.pone.0245364.r002
29 Dec 2020 Decision Letter - Dzung Pham, Editor Feasibility of using a novel automatic cardiac segmentation algorithm in the clinical routine of lung cancer patients PONE-D-20-20670R1 Dear Dr. Finnegan, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Dzung Pham Academic Editor PLOS ONE Additional Editor Comments (optional): Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #2: All comments have been addressed ******** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #2: Yes ****** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #2: Yes ****** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #2: Yes ****** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #2: Yes ****** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #2: I am appreciative that the authors went to great lengths to fully address my comments. The manuscript now includes a full and thorough analysis and discussion and is more than acceptable for publication. My only minor comment is that the axial slices in figure 5 could be further zoomed in on the heart, but it is adequate in its current form. ****** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #2: Yes:** Joseph Harms https://doi.org/10.1371/journal.pone.0245364.r003
Formally Accepted
6 Jan 2021 Acceptance Letter - Dzung Pham, Editor PONE-D-20-20670R1 Feasibility of using a novel automatic cardiac segmentation algorithm in the clinical routine of lung cancer patients Dear Dr. Finnegan: I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org. If we can help with anything else, please email us at plosone@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr Dzung Pham Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0245364.r004

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .