Deep learning approach based on superpixel segmentation assisted labeling for automatic pressure ulcer diagnosis

Che Wei Chang; Mesakh Christian; Dun Hao Chang; Feipei Lai; Tom J. Liu; Yo Shen Chen; Wei Jen Chen

doi:10.1371/journal.pone.0264139

Peer Review History

Original SubmissionSeptember 6, 2021
23 Nov 2021 Decision Letter - Anwar P.P. Abdul Majeed, Editor PONE-D-21-28897Deep Learning Approach Based on Superpixel Segmentation Assisted Labeling for Automatic Pressure Ulcer DiagnosisPLOS ONE Dear Dr. Chang, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Jan 07 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Anwar P.P. Abdul Majeed Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. We note that your paper includes detailed descriptions of individual patients/participants. As per the PLOS ONE policy (http://journals.plos.org/plosone/s/submission-guidelines#loc-human-subjects-research) on papers that include identifying, or potentially identifying, information, the individual(s) or parent(s)/guardian(s) must be informed of the terms of the PLOS open-access (CC-BY) license and provide specific permission for publication of these details under the terms of this license. Please download the Consent Form for Publication in a PLOS Journal (http://journals.plos.org/plosone/s/file?id=8ce6/plos-consent-form-english.pdf). The signed consent form should not be submitted with the manuscript, but should be securely filed in the individual's case notes. Please amend the methods section and ethics statement of the manuscript to explicitly state that the patient/participant has provided consent for publication: “The individual in this manuscript has given written informed consent (as outlined in PLOS consent form) to publish these case details 3. Thank you for stating the following in the Acknowledgments Section of your manuscript: "This work was supported by the Innovation Project of Far Eastern Memorial Hospital (Grant No. PI20200002). We thank Shih-Chen Huang, who helped coordinate with the plastic surgeons in labeling the images of pressure ulcers" We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form. Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows: "This work was supported by the Innovation Project of Far Eastern Memorial Hospital (Grant No. PI20200002)" Please include your amended statements within your cover letter; we will change the online submission form on your behalf 4. We note that you have stated that you will provide repository information for your data at acceptance. Should your manuscript be accepted for publication, we will hold it until you provide the relevant accession numbers or DOIs necessary to access your data. If you wish to make changes to your Data Availability statement, please describe these changes in your cover letter and we will update your Data Availability statement to reflect the information you provide 5. In your Data Availability statement, you have not specified where the minimal data set underlying the results described in your manuscript can be found. PLOS defines a study's minimal data set as the underlying data used to reach the conclusions drawn in the manuscript and any additional data required to replicate the reported study findings in their entirety. All PLOS journals require that the minimal data set be made fully available. For more information about our data policy, please see http://journals.plos.org/plosone/s/data-availability. "Upon re-submitting your revised manuscript, please upload your study’s minimal underlying data set as either Supporting Information files or to a stable, public repository and include the relevant URLs, DOIs, or accession numbers within your revised cover letter. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. Any potentially identifying patient information must be fully anonymized. Important: If there are ethical or legal restrictions to sharing your data publicly, please explain these restrictions in detail. Please see our guidelines for more information on what we consider unacceptable restrictions to publicly sharing data: http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. Note that it is not acceptable for the authors to be the sole named individuals responsible for ensuring data access. We will update your Data Availability statement to reflect the information you provide in your cover letter Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: The authors proposed a workflow of labeling images of pressure ulcers as a way to use them as data for a deep learning algorithm to learn and perform detection and classification of pressure ulcers. The work presented is technically sound and should be useful to others interested in the field. Good introduction to the background and previous related work. While the sharing of their labeling techniques is appreciated, it is not necessarily "suitable for use with any images of pressure ulcers, and not just for our dataset" as claimed. 3.2 Total and breakdown of the data images collected should be mentioned here. Also were all these images taken with the same parameters such as distance from the wound to the camera, camera angle and such. Obviously, the same wound would look bigger and have more pixels if it is nearer to the camera than if it is further away. This should be clarified. 3.3 This section should be more concise. The division between inside and outside ulceration should be explained more directly. I think this can be explained better using either point form format or listing in down in a table. 3.4 What were the 4 models used to test the segmentation ? And why did the author use 4 different models for this ? It is not mentioned in this section. The text in Fig 2 is too small and unreadable. 3.5 Perhaps the surgeons found it difficult to agree on the regions because they were not specifically defined for them ? As in this paper, there was no specific definition on how the different tissues mentioned were defined. Figure 5 is more confusing than it should be. The flow of work should be clearer and any graphics that are not important should be removed. 5. Discussion - the discussion is not focused and doesnt expand on the findings and instead introduces new concept and results. There is also quite a lot of information that is repeated from previous sections. Overall formating needs to be rechecked and redone. For example, most paragraphs are not aligned correctly. Same goes for figures (which still have boxes around them) and equations. While the first half of this paper is quite good, the later half seems rushed and does not reach the quality expected after reading the first half. I hope that the authors can present their work (which seems like a lot of work) in a way that is easier to understand and appreciate. Reviewer #2: The authors explored the use of different DL models for the segmentation and classification of pressure ulcer. They also explored the efficacy different labelling technique, i.e. manual labelme as well as the SLIC. The paper methodological employed in the study is sound, whilst the results were well discussed. The following are my comments to improve the manuscript. 1. Section 3 - Methods - Naturally the evaluation metrics described in 3.1 should be moved at the end of the methods section. 2. Why was the ResNet101 used as the encoder for all models evaluated? Please substantiate the selection through literature, in the even a sensitivity test was not carried out. ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review?** For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0264139.r001
Revision 1
15 Dec 2021 Author Response Dear reviewer 1 3.2 Total and breakdown of the data images collected should be mentioned here. Also, were all these images taken with the same parameters such as distance from the wound to the camera, camera angle and such. Obviously, the same wound would look bigger and have more pixels if it is nearer to the camera than if it is further away. This should be clarified. Answer: Thank you for this critical observation. The images of pressure ulcers were download from our EMR, which were taken by various types of devices, including cellphones, pads and cameras. The conditions of taking pictures (illumination, background, distance, and angle) were also diverse. We had tried using a small number of images being taken under the standard condition to train models as in previous articles. Although these models had good results only in the training dataset, their performance was poor when facing images of other patients. That is unexpected but not surprised. It is important for deep learning models to comprehend information from any condition. Since the images were downloaded from the EMR of our hospital, we did not know the exact distance and camera angle of every image. However, in our dataset, the average ratio of wound area to the whole picture is around 44%. The distance too far or wound too small were difficult to be labeled and will be excluded by plastic surgeons. 3.3 This section should be more concise. The division between inside and outside ulceration should be explained more directly. I think this can be explained better using either point form format or listing in down in a table. Answer: We thank the reviewer for your insightful comment. To address this comment, we had condensed the definition and description of different types of tissues into Table 1. 3.4 What were the 4 models used to test the segmentation? And why did the author use 4 different models for this? It is not mentioned in this section. The text in Fig 2 is too small and unreadable. Answer: We apologize for not providing clarity while discussing the effect of classes combination. This paragraph demonstrated that a model has better sensitivity and DC for re-ep when re-ep training with ulceration than granulation + eschar + slough, although ulceration ≈ granulation + eschar + slough. In aspect of re-ep, the partitions will result four models, but only models with re-ep were used for comparison. Our description was slightly confusing for which we rewrote the paragraph to be clearer. 3.5 Perhaps the surgeons found it difficult to agree on the regions because they were not specifically defined for them? As in this paper, there was no specific definition on how the different tissues mentioned were defined. Figure 5 is more confusing than it should be. The flow of work should be clearer and any graphics that are not important should be removed. Answer: As your professional suggestions, we condensed the definition and description of tissues into Table 1. They have specific definitions but still hard to be labeled. Most tissues inside and peri-wound have a various range of colors and texture. It is unlikely to take every piece of tissue on the wounds for biopsy to confirm. Images can only be co-labeled by experienced plastic surgeons as “relative” ground truth. In our initial dataset, there were many excluding images because of no consensus on labeling. Even within the labeled images, there were still pixels difficult to be labeled and left undefined. The relation can be described as Ulceration = (Granulation ∩ Eschar ∩ Slough ∩ Undefined).We had revised Figure 5 to be more legible. We separated the labeling & training from automatic diagnosis to make new Figure 4 and S5 Figure. 5. Discussion- the discussion is not focused and does not expand on the findings, and instead introduces new concept and results. There is also quite a lot of information that is repeated from previous sections. Answer: We appreciate this critique from the reviewer. The first paragraph of discussion was redundant. We had trimmed down to make it more concise. There were two main parts in the discussion. The first part was the expanding of superpixel segmentation from the evolving of this type of ML. After superpixel segmentation processing, we input fewer pixel numbers but achieved better results in a manner of improving inter-rater consistency. In the second part, we tried to explain why different classes combination has an impact on the recall (sensitivity) of a specific class. Re-ep is an important class but has small number of pixels with irregular shape. 6. Overall formatting needs to be rechecked and redone. For example, most paragraphs are not aligned correctly. Same goes for figures (which still have boxes around them) and equations. While the first half of this paper is quite good, the latter half seems rushed and does not reach the quality expected after reading the first half. I hope that the authors can present their work (which seems like a lot of work) in a way that is easier to understand and appreciate. Answer: We apologize for not following the format of the journal. We are grateful for your general enthusiasm to our work. We had revised all our tables, figures, and supporting materials, including re-drawing some figures to make them legible. The discussion was also revised to be adhesive to the results. Best Regards and Merry Christmas Dr. Che Wei, Chang --- Dear reviewer 2 We apologize for not following the format of the journal. We are grateful for your general enthusiasm to our work. 1. Section 3 - Methods - Naturally the evaluation metrics described in 3.1 should be moved at the end of the methods section. Answer: We thank the reviewer for this thoughtful comment. We had moved the evaluation metrics before the results of different combination were presented. We apologize for not following the format of this great journal. We also revised all our tables, figures, and supporting materials, including re-drawing some figures to make them clearer. 2. Why was the ResNet101 used as the encoder for all models evaluated? Please substantiate the selection through literature, in the even a sensitivity test was not carried out. Answer: We are grateful for this insightful comment by the reviewer. ResNets are powerful encoder that can be compatible with any deep learning models and are easy to optimize. Our preliminary studies had tried ResNet50, ResNet101, and ResNet152 with U-Net. The U-Net with ResNet101 had much better performance than with ResNet50. It is interesting that U-Net with ResNet152 only had a marginal advantage than with ResNet101. ResNet152 had more likely to experience overfitting and consumed more computing resource. Based on the previous result, we applied ResNet101 for all deep learning models in our final presentation. We had added descriptions to Methods- Deep learning models regarding the choice of an encoder. Best Regards and Merry Christmas Dr. Che Wei, Chang Attachments Attachment Submitted filename: Response to Reviewers.docx https://doi.org/10.1371/journal.pone.0264139.r002
4 Feb 2022 Decision Letter - Anwar P.P. Abdul Majeed, Editor Deep Learning Approach Based on Superpixel Segmentation Assisted Labeling for Automatic Pressure Ulcer Diagnosis PONE-D-21-28897R1 Dear Dr. Chang, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Anwar P.P. Abdul Majeed Academic Editor PLOS ONE Additional Editor Comments (optional): NIL Reviewers' comments: The author(s) have addressed the queries raised by the reviewers https://doi.org/10.1371/journal.pone.0264139.r003
Formally Accepted
8 Feb 2022 Acceptance Letter - Anwar P.P. Abdul Majeed, Editor PONE-D-21-28897R1 Deep Learning Approach Based on Superpixel Segmentation Assisted Labeling for Automatic Pressure Ulcer Diagnosis Dear Dr. Chang: I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org. If we can help with anything else, please email us at plosone@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Anwar P.P. Abdul Majeed Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0264139.r004

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .