Peer Review History
| Original SubmissionFebruary 10, 2020 |
|---|
|
PONE-D-20-03864 Mitochondrial event localiser (MEL) to quantitatively describe fission, fusion and depolarisation in the three-dimensional space PLOS ONE Dear Mr Theart, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. The study is of potential interest and I think it could provide a valuable tool. Still, there are some major concerns that need to be addressed sufficuently. In particular, you need to provide a better quantitative validation of the analysis including the issue raised about possible false positives. Moreover, the PLOS One rules concerning availabily of algorithms, also raised by one reviewer, has to be addressed. We would appreciate receiving your revised manuscript by May 16 2020 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols Please include the following items when submitting your revised manuscript:
Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out. We look forward to receiving your revised manuscript. Kind regards, Andreas S. Reichert, Dr. rer. nat. Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements: 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at http://www.plosone.org/attachments/PLOSOne_formatting_sample_main_body.pdf and http://www.plosone.org/attachments/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. Thank you for stating the following in the Competing Interests/Financial Disclosure * (delete as necessary) section: "This work was supported by the South African Medical Research Council (SAMRC), Cancer Association of South Africa (CANSA), and Telkom South Africa, and the South African National Research Foundation (NRF). Nvidia corporation sponsored the GPU that was used in this project. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript." We note that you received funding from a commercial source: Nvidia corporation Please provide an amended Competing Interests Statement that explicitly states this commercial funder, along with any other relevant declarations relating to employment, consultancy, patents, products in development, marketed products, etc. Within this Competing Interests Statement, please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials by including the following statement: "This does not alter our adherence to PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests). If there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared. Please include your amended Competing Interests Statement within your cover letter. We will change the online submission form on your behalf. Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: http://journals.plos.org/plosone/s/competing-interests [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Partly ********** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: No Reviewer #2: No ********** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: No Reviewer #2: Yes ********** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ********** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: The manuscript entitled "Mitochondrial event localiser (MEL) ..." by Theart et al. describes an automated analysis tool for 3D image stacks of mitochondrial fusion, fission and depolarisation. The manuscript is very well written and pleasent to read. The scientific topic of this manuscript is highly important as - to the best of my knowledge - an objective and automated analysis tool does not exist today; yet it is hihgly desirable, because the mitochondrial dynamics can only be accurately quantified in this way. The quantification of these processes is highly important, because it provides a firm basis for models of mitochondrial dynamics. I have a number of serious issues that need to be addressed before the manuscript may be considered for publication: 1) In line 166-169, the authors list the parameters that enter the image analysis. I suggest that the authors give here the range in which they have tested these parameters for each of them. Absolute numbers are required in order to understand statements that are otherwise meaningless, because one does not know what they compare to; e.g. "a time interaval that is too long" (line 112) - what is long and what not?; "less than some appropriately small number of voxels (in our case 20)" - what is appropriate? how does one decide about this? what does it mean in absolute terms, i.e. in µm? "overlapping volumes that account for less than 1% of the volume ... are eliminated" (caption of Fig. 2) - what makes 1% the magic number? "with a small percentage" (line 228) - what is small? what is large? how to decide this? 2) Line 305-306: I do not understand why the experiments and controls were recorded with different time steps. One would expect that every condition is recorded with identical settings and it would be the best if time steps would be chosen as small as possible. This would not only increase accuracy of the analysis, but would also allow to systematically study the impact of increasing time steps: for experiment and control the analysis could be repeated with larger time steps (leaving frames in between out) - but still having the exact time stepping in experiment and control. Otherwise, I do not see how to uniquely attribute changes in the analysis performance to the difference in the experiment versus control, because such differences may as well stem from technical issues like different time stepping in the imaging. While having such differences should be avoided in the experimental design, the reasoning of the authors for their data set with incommensurate time steps is not at all clear. 3) In line 348ff, the visualization in virtual reality is emphasized. I do not understand this point, may be; however, what I would have liked to see is a number of videos (e.g. with changing perspectives) that gives some idea how the dynamics looks in three spatial dimensions. Why did the author not opt for this possibility? 4) Related to the last issue, in general, I am missing a quantitative validation of the analyis tool. The Results section and the Discussion section contain a lot of statements that are not quantitatively supported by a rigorous validation. No comparison is made; neither to manually annotated data, nor by using another analysis tool (which may not have been developed for mitochondria, but for other data, e.g. for cell migration and interaction). Without any quantitative validation of this analysis tool, I cannot see how the value of the presented tool can be objectively judged. Without such an analysis, I think, this work can not be published. I would like to know, how many false positives and false negatives you have in detecting and not detecting events of fusion/fission/depolarisation; please, provide the numbers of your validation in terms of standard performance measures. 5) Once issue 4) has been done, the authors can remove statements like "it was often observed" (line 408), "From the analysis it seems that MEL is robust..." (line 426), "the false detection of these events is reduced" (line 432), "the best 3D quantification" (line 442), "it produces the most consistent results" (line 446) and replace them by quantiatively concrete statements. Please, check the whole manuscript for many of these kinds of statements, which all need to be removed and replaced by quantitative statements with clear reference as to how properties like "robust" and "best" are measured. 6) Table 2 contains a list of all tunable parameters, which need to be given in realistic units; the symbol \\sigma should not be used repeatedly for different quantitites. 7) The authors are not sufficiently motivating the need of their analysis tool: it is, in fact, mandatory to have such a tool in order to model the mitochondrial dynamics. Models of mitochondrial dynamics should be cited, e.g. a quick search revealed: https://doi.org/10.1371/journal.pcbi.1002576 https://doi.org/10.1002/bies.201200125 https://doi.org/10.1371/journal.pcbi.1003108 https://doi.org/10.1371/journal.pone.0168198 ... All these models (and may be others as well) will benefit from a tool like you have developed here, underlining the importance of it once it has been appropriately validated. 8) I somehow miss the information where the code can be downloaded and where the data can be downloaded. Did I overlook this? Reviewer #2: Overall, I think this is an interesting paper that with more rigorous analysis and experimental replicates with statistical analysis has the chance to contribute to the field of mitochondria morphometry. My most concerning critique is that the authors present the data of a single biological experimental replicate per condition. How can the community be assured of the repeatability of your method? This must be addressed prior to publication. Please describe and discuss the variance between accuracy (including false positive and false negative events as discussed in the critique below), results, and biological implications. —-How do you distinguish between events that are truly depolarization and loss of florescence signal from photobleaching? One way to describe this would be an overall signal to background fluorescence intensity from frame to frame, but discussion of the lower limit of detection of the algorithm is important given the assumption that a disappearing label is a depolarization event. An alternative approach could be the use of a covalently or otherwise permanently bound fluorophore (e.g MitoTracker Deep Red or fluorescent-tagged membrane bound protein). —- I would argue that the tunable parameter of minimum voxel volume or the otsu thresholding sensitivity should be re-tuned. In both the regions of interest image series displayed in figures 4 and 5, multiple incorrect categorizations of events can be observed by looking at the entire time lapse. For instance, in figure 5, a more dimly lit small spherical shaped mitochondrion is identified by the algorithm to undergo depolarization 3 times but by the naked eye it’s appearance changes very little and yet it exists across the entire image series. Similarly, the fission and fusion events the authors describe as kiss and run do not convincingly separate and join. If this is representative of the algorithm’s performance, then the reader must conclude that the false positive event rate is far too high to be acceptable. — To address the concerns just raised, the authors should (ideally under 3D visualization) manually determine the true positive, false positive and false negative rates of event localization and report these adjacent to the event rate graphs and discuss how this was used to tune their algorithm’s parameters. — One of the tenets of the scientific method is to alter only one variable during an experiment. By increasing the temporal density of imaging in the control versus toxicant treatment group, the authors have introduced differing levels of photobleaching and phototoxicity as a variable as well as varied the potential for missing events or mismatching labeled structures. Please address this time difference by repeating your experiments under similar conditions, address this bias in your discussion, or leave out any claims interpreting the changes observed as an effect of the toxicant. — What was the vehicle? We have observed empirically that mitochondria are exquisitely sensitive to dimethyl sulfoxide in any final concentration greater than 0.1%. Please list the vehicle and its concentration in your methods and ensure it is also administered at the same concentration in your control population of cells at an identical time prior to imaging. ********** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step. |
| Revision 1 |
|
PONE-D-20-03864R1 Mitochondrial event localiser (MEL) to quantitatively describe fission, fusion and depolarisation in the three-dimensional space PLOS ONE Dear Dr. Theart, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not yet fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. ============================== I think that the study has improved very much and many of the most critical points are resolved. Still, I kindly ask you to adress the remaining concerns and submit a revised version. Could you please respond specifically in which case biological replicates were generated and if not why the method is still sufficiently validated. ============================== Please submit your revised manuscript by Dec 10 2020 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript:
If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols We look forward to receiving your revised manuscript. Kind regards, Andreas S. Reichert, Dr. rer. nat. Academic Editor PLOS ONE [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #1: All comments have been addressed Reviewer #2: (No Response) ********** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Partly ********** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: No ********** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ********** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ********** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: The authors have fully addressed my concerns, including the extension of the data set and the execution of a quantitative validation. Reviewer #2: Major critiques: The revision addresses most of my minor critiques but fails to address my primary concern that the authors have not proven the repeatability of the biological findings they are claiming with repeat experimental replicates. While they do describe their n in terms of number of cells analyzed, I cannot find anywhere in the paper that suggests the biological experiment was repeated more than once under identical conditions for each condition. This unfortunately remains a fatal flaw that must be addressed prior to publication. If this is what was done for Fig. 10, please make it abundantly clear to the reader that these were independent experiments. Abstract: The high rate of error should be addressed as a limitation up front so as not to surprise the reader in the results. The authors make the argument that the error of the algorithm in correctly detecting fission fusion and depolarization events in their peroxide-induced stress experiment correlates with the ground truth data as determined by manual review but with false positive rates ranging from at best 84% to 2000% (taken by dividing the means on the left side by the right side on Table 2), this is a huge limitation. Please present the critical quantitative findings of the paper from Fig. 10 in the abstract or consider changing the title from quantitative to qualitative. E.g., “an average of x/x/x fusion/fission/depolarisation events per cell (verified/unverified?) were observed every 10 sec in control cells. With peroxide treatment, the rate initially rebalanced toward fusion to x/x/x, before returning to approximately the same as control levels x/x/x.” See first comment below on discussion. Consider removing the subcellular localization comments from the abstract. They distract from the main points of the paper. Results: Why is data collected on control cells for only ~250 seconds when it appears that the treated group is imaged for twice as long (combining pre treatment and post treatment)? This opens up the opportunity for confounding variables including photo bleaching and photooxidative stress. Was vehicle added to the control cells at the same time point as the treated cells? When it comes to statistical analysis, I’m not sure that lumping all of the many time-resolved events together via an average makes the most convincing argument for differences. While it is great and important to show the very high temporal resolution of the method, it would be more interesting and probably more statistically powerful to look at the summation of events over time (on a per cell basis) as opposed to averaging over time. Other alternatives would be analyzing specific time points or the derivative/rate of change. —— Minor critiques: Figure 6: The control and post treatment time lapse curves appear identical. Please verify. Secondly, the panels at the very bottom employ the use of a line graph that suggests a paired nature to the data. Bar graphs grouped by each event type would make the data easier to interpret to the reader. The error bars in the updated figure should be error between experimental replicates, not error between cells within an experimental replicate. Please add text to clarify that the number of events are per cell or per high powered field. Why are the control events so different from the pre-treated cells? It makes the reader wonder whether there is a high amount of biological variability (which could be addressed with repeat experiments) or a confounding variable has not been addressed in the experimental set up (temperature, CO2, pH?). Figure 7-9: Why is there such a difference in the total number of detected structures in the last frame of untreated versus the first frame of the hydrogen peroxide-treated cells? The appearance of the cell has significantly changed suggesting that time 0 is not exactly the time at which peroxide was administered. Furthermore, the overall cell morphology of the pre-treated and the control cell is completely different. I infer from the 2D representation that the pre-treated cell is more rounded and less adherent to the culture plate which is in contrast to the control which has the absence of mitochondria within the nuclear borders and spindle-shaped morphology and is adjacent to another cell. This suggests a different cell state which in and of itself can cause vastly different responses to treatment as well as baseline cell signaling characteristics. I would verify culture conditions are exactly the same between the groups. However, for representation of data, I would suggest picking cells that appear similar at baseline to show your reader. Discussion: I would refrain from making statements about patterns observed that aren’t rooted in multiple observations with quantification across biological replicates with statistical testing of your hypothesis. In particular, making claims about sub cellular localization of events in reference to peri nuclear versus periphery when no distance data is presented in the paper may make the reader question your findings. Instead, I would suggest alluding how your method could be used to test those hypotheses in future studies. Conclusion: Other methods (including those you referenced) have achieved deterministic automated qualification of mitochondrial morphology, albeit in in 2D not 3D. I would qualify your closing statements so as to not discount the work of others. ********** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: Yes: Anthony Presley Leonard [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. |
| Revision 2 |
|
Mitochondrial event localiser (MEL) to quantitatively describe fission, fusion and depolarisation in the three-dimensional space PONE-D-20-03864R2 Dear Dr. Theart, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Andreas S. Reichert, Dr. rer. nat. Academic Editor PLOS ONE Additional Editor Comments (optional): Reviewers' comments: |
| Formally Accepted |
|
PONE-D-20-03864R2 Mitochondrial event localiser (MEL) to quantitativelydescribe fission, fusion and depolarisation in thethree-dimensional space Dear Dr. Theart: I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org. If we can help with anything else, please email us at plosone@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Andreas S. Reichert Academic Editor PLOS ONE |
Open letter on the publication of peer review reports
PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.
We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.
Learn more at ASAPbio .