3D fluorescence microscopy data synthesis for segmentation and benchmarking

Dennis Eschweiler; Malte Rethwisch; Mareike Jarchow; Simon Koppers; Johannes Stegmaier

doi:10.1371/journal.pone.0260509

Peer Review History

Original SubmissionAugust 9, 2021
1 Oct 2021 Decision Letter - Kristen C. Maitland, Editor Transfer Alert This paper was transferred from another journal. As a result, its full editorial history (including decision letters, peer reviews and author responses) may not be present. PONE-D-21-255013D fluorescence microscopy data synthesis for segmentation and benchmarkingPLOS ONE Dear Dr. Eschweiler, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please consider the revisions suggested by the reviewers. Please submit your revised manuscript by Nov 15 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Kristen C. Maitland, Ph.D. Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: No ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: The authors present work addressing the need for automated generation of training data sets for deep-learning based segmentation of cell membranes and nuclei. Different methods for generating the synthetic training sets are evaluated qualitatively and quantitatively, as well as how effective they are when used to train a segmentation network that is applied to real data. This type of methods analysis is important for biologists to appreciate and understand the nuance and challenges still existing with deep-learning segmentation tools. It also highlights the importance of data quality and provides a way to asses optimal data quality for future acquisitions. The annotated data sets made available may be very useful to research groups that are still in the process of manually annotating their own data and don't have access to a useful already trained network. I recommend this work be published with a few suggestions for the author's consideration: My biggest suggestion is to consolidate the figures. I think they could be combined for a total of 10 figures at most that tell the story in a more succinct way. Line 5: the comma after "analyzed" should be removed Line 7: a comma could be included after "segmentation" When discussing spherical harmonics, it may be useful for a general audience to compare to Fourier Transform, a more familiar weighted sum of basis functions. Though, the presentation given is already done well. Line 202: The "in conclusion" seems unnecessary Line 209: It would be good to mention the accuracy measures that will be used here. Figure 5 legend: I would add commas so the text reads "Annotations can be obtained from manual, automated, and simulation approaches, and final cellular annotations are used to...." Figure 4: If there are these visual examples provided of the spherical harmonic models, it would be nice to see the statistical shape models also. Line 216: It would be useful to mention the imaging lens numerical aperture and physical pixel sizes here from ref 32. for quick understanding of the spatial resolution of the data sets. Line 222: It would be useful to mention this data set was acquired with a multi-photon fluorescence microscope (ref 33). Line 250: By integrating the z-dimension for the intensity profiles you lose the ability to asses the depth-dependent performance of the GAN. It would be interesting to have at least two -perhaps an "upper" and "lower" profile if not a variety of individual xy planes at different z positions. Line 252: The similarity between the real and synthetic xz intensity spectra could be qualified with the visually obvious discrepancies that happen in the high-frequency areas. What accounts for those discrepancies could be discussed. Line 262 and 263: If a few words describing the approaches in [34] and [35] could be used here it would improve the readability. Line 304: "allow to again conclude the generation of realistic 3D image data" is missing a subject, consider "allow us to again conclude..." Line 316: Please describe the approach in [37] for the audience's quick reference. Line 358: It would be interesting to also quantify the different "quality levels" of the synthesized data sets in terms of signal-to-noise ratio, contrast ratio, or some other normal image quality metric. Reviewer #2: The paper is overall well presented but there are a couple confusing aspects to the presentation. There is no top down explanation of the logic of the various experimental mixes of images and masks which makes things more challenging than necessary to understand There are a few grammatically problematic sentences e.g. "The obtained scores are similar to the scores obtained from the previous experiments and allow [us] to again conclude [success in] the generation of realistic 3D image data." It would be helpful to situate this work with respect to other CNN based training data simulation efforts e.g. NucleAIzer Hollandi et al Cell Systems 2020 ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: Yes:** Anthony Santella [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0260509.r001
Revision 1
3 Nov 2021 Author Response We thank you for considering our manuscript for publication as an article in PLOS ONE and we are excited to revise our manuscript based on the very helpful comments and suggestions from both reviewers. As one of the major changes made to our manuscript, we swapped sections 2 and 3 for improved readability and better coherence of the whole story. Further changes and detailed responses to the reviewers’ suggestions are listed in the following. We are looking forward to your assessment of the improvements made to the manuscript and hope all requests have been fulfilled. Reviewer #1: My biggest suggestion is to consolidate the figures. I think they could be combined for a total of 10 figures at most that tell the story in a more succinct way. - We totally agree that there have been to many figures impairing the reader’s experience. The total number of figures has been reduced by combining or removing previous figures, although we ended up at a quantity slightly above 10. Line 5: the comma after "analyzed" should be removed - Removed the comma, thanks for pointing this out. Line 7: a comma could be included after "segmentation" - Added the comma, thanks for the hint. When discussing spherical harmonics, it may be useful for a general audience to compare to Fou-rier Transform, a more familiar weighted sum of basis functions. Though, the presentation given is already done well. - We agree that the concept of the Fourier transform is very likely more familiar to the reader, and we added a hint to the similarities between both concepts to the spherical harmonic section. Thanks for pointing this out. However, we feel that a full comparison between both approaches would lead to an overly complex and lengthy explanation, that might lead to confusion, as Fourier transform is not used later on. Line 202: The "in conclusion" seems unnecessary - This phrase was removed. Line 209: It would be good to mention the accuracy measures that will be used here. - This should refer to the general quality of the generated structures. However, we think that this sentence is rather confusing and does not provide reasonable information, which is why we re-moved it. Figure 5 legend: I would add commas so the text reads "Annotations can be obtained from man-ual, automated, and simulation approaches, and final cellular annotations are used to...." - Commas were added to the legend. Figure 4: If there are these visual examples provided of the spherical harmonic models, it would be nice to see the statistical shape models also. - For consistency we added a few examples of shapes generated with statistical shape models. Line 216: It would be useful to mention the imaging lens numerical aperture and physical pixel sizes here from ref 32. for quick understanding of the spatial resolution of the data sets. - Good point, we missed this information entirely. The missing information was added to the data set description. Line 222: It would be useful to mention this data set was acquired with a multi-photon fluores-cence microscope (ref 33). - Microscopy technique and imaging resolution have been added to the data set description. Line 250: By integrating the z-dimension for the intensity profiles you lose the ability to assess the depth-dependent performance of the GAN. It would be interesting to have at least two -perhaps an “upper” and “lower” profile if not a variety of individual xy planes at different z positions. - Assessing the depth-dependent performance was the intention of these plots. To get a better overview of the distribution of intensities, we added further plots showing the distribution over YZ. To our understanding, these plots help to assess how well the cumulative intensity for differ-ent thicknesses of the specimen can be reconstructed. We think, XY profiles would be harder to interpret and would only show depth-dependent performance when considering quite a few z po-sitions, which would start to be cluttering. If you have any tips on how we could represent this in-formation in a more compact format, we would be happy to extend the figure. Line 252: The similarity between the real and synthetic xz intensity spectra could be qualified with the visually obvious discrepancies that happen in the high-frequency areas. What accounts for those discrepancies could be discussed. - One of our possible interpretations is that those discrepancies are caused by non-deterministic components like noise, and differences in signal intensities. We added further interpretations and explanations to Section 4. Line 262 and 263: If a few words describing the approaches in [34] and [35] could be used here it would improve the readability. - A short description of both approaches has been added, which hopefully helps to get a better un-derstanding of the experimental setup. Line 304: “allow to again conclude the generation of realistic 3D image data” is missing a subject, consider “allow us to again conclude…” - Thanks for pointing this out, we corrected the sentence. Line 316: Please describe the approach in [37] for the audience’s quick reference. - We added further explanations of the approach’s concept for a quick reference. Line 358: It would be interesting to also quantify the different “quality levels” of the synthesized data sets in terms of signal-to-noise ratio, contrast ratio, or some other normal image quality metric. - The PSNR was added as further metric. Due to the different value ranges of PSNR and the other metrics, we replaced the plot with a table. Reviewer #2: There is no top down explanation of the logic of the various experimental mixes of images and masks which makes things more challenging than necessary to understand. - A general explanation of the choice of our experiments was indeed missing and would help to clar-ify the whole evaluation section. A few sentences explaining our intention of the experiments was added to the beginning of Section 4. There are a few grammatically problematic sentences e.g. "The obtained scores are similar to the scores obtained from the previous experiments and allow [us] to again conclude [success in] the generation of realistic 3D image data." - Thanks for pointing this out, we corrected the sentence. It would be helpful to situate this work with respect to other CNN based training data simulation efforts e.g. NucleAIzer Hollandi et al Cell Systems 2020 - We think that a direct comparison to this approach would not be straightforwardly possible, as the generation of images in 2D and patch-based 3D faces significantly different challenges. However, we agree that referring to this approach would complement the state-of-the-art literature, which is why we added a citation of this approach to the introduction section. Attachments Attachment Submitted filename: Response to Reviewers.pdf https://doi.org/10.1371/journal.pone.0260509.r002
11 Nov 2021 Decision Letter - Kristen C. Maitland, Editor 3D fluorescence microscopy data synthesis for segmentation and benchmarking PONE-D-21-25501R1 Dear Dr. Eschweiler, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Kristen C. Maitland, Ph.D. Academic Editor PLOS ONE Additional Editor Comments (optional): Reviewers' comments: https://doi.org/10.1371/journal.pone.0260509.r003
Formally Accepted
23 Nov 2021 Acceptance Letter - Kristen C. Maitland, Editor PONE-D-21-25501R1 3D fluorescence microscopy data synthesis for segmentation and benchmarking Dear Dr. Eschweiler: I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org. If we can help with anything else, please email us at plosone@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Kristen C. Maitland Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0260509.r004

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .