PONE-D-23-17568Machine learning evaluation for identification of M-proteins in human serumPLOS ONE

Dear Dr. Rotter Sopasakis,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Sep 24 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

John Adeoye

Academic Editor

PLOS ONE

Journal requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified (1) whether consent was informed and (2) what type you obtained (for instance, written or verbal, and if verbal, how it was documented and witnessed). If your study included minors, state whether you obtained consent from parents or guardians. If the need for consent was waived by the ethics committee, please include this information.

If you are reporting a retrospective study of medical records or archived samples, please ensure that you have discussed whether all data were fully anonymized before you accessed them and/or whether the IRB or ethics committee waived the requirement for informed consent. If patients provided informed written consent to have data from their medical records used in research, please include this information

3. Please note that PLOS ONE has specific guidelines on code sharing for submissions in which author-generated code underpins the findings in the manuscript. In these cases, all author-generated code must be made available without restrictions upon publication of the work. Please review our guidelines at https://journals.plos.org/plosone/s/materials-and-software-sharing#loc-sharing-code and ensure that your code is shared in a way that follows best practice and facilitates reproducibility and reuse.

4. We note that the grant information you provided in the ‘Funding Information’ and ‘Financial Disclosure’ sections do not match.

When you resubmit, please ensure that you provide the correct grant numbers for the awards you received for your study in the ‘Funding Information’ section.

5. Thank you for stating the following financial disclosure:

"This work is supported by the department of Clinical Chemistry, Sahlgrenska University Hospital, Gothenburg, Sweden (VRS, LMH, MN, FN, MA). There is no other specific funding for this work."         

Please state what role the funders took in the study.  If the funders had no role, please state: "The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."

If this statement is not correct you must amend it as needed.

Please include this amended Role of Funder statement in your cover letter; we will change the online submission form on your behalf.

6. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For more information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

    a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially sensitive information, data are owned by a third-party organization, etc.) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

   b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

7. We note that you have stated that you will provide repository information for your data at acceptance. Should your manuscript be accepted for publication, we will hold it until you provide the relevant accession numbers or DOIs necessary to access your data. If you wish to make changes to your Data Availability statement, please describe these changes in your cover letter and we will update your Data Availability statement to reflect the information you provide.

8. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information

Additional Editor Comments:

The reviewers both agree that the dataset used for ML model construction is valuable but both cited the poor methodological approach of the study. In view of this, the authors are advised to revise their ML modelling methodology extensively in line with contemporary standards to allow for further consideration of their manuscript in the Journal.

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: No

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Please see the attached file for my comments, in addition with the summary of my review below:

The authors performed capillary & gel electrophoresis and immunofixation for more to 60k serum samples. Based on their data, they classified their samples into either “absence of M-protein” or “presence of M-protein”, and classified the subtype of M-protein, when relevant. Next, they trained various ML classifiers to automate those classification tasks, by feeding those with raw numerical data of the SPEP (Serum Protein Electrophoresis) curves, i.e. the value of each point of the SPEP curve.

The authors present an extremely valuable dataset: more than 60k samples, annotated thanks to three biological assays: capillary SPEP, gel SPEP and immunofixation; with the interpretation from laboratory experts. This alone is a solid argument supporting the publication of their results. It should be noted that the data might be described better, for instance by 1) indicating the size of M-spikes in their curves and the proportion of small/medium/large M-spikes, for instance; and 2) the proportion of other abnormal patterns which may mask or be confounded with M-proteins (artifacts (fibrinogen, iodinated contrast agents…), beta-gamma bridging). Furthermore, it is not clear if all samples were analyzed with all 3 assays: capillary SPEP, gel SPEP and immunofixation. A table may be useful here. Finally, the choice of the authors to partition their data into a training and test sets only, without any validation set, despite their number of samples, is at major risk of data leakage: there is an urge in addressing or at least discussing this point in the manuscript.

The main downside of this study, in my opinion, is the machine learning methodology used here, which is far from state of the art standards. SPEP data, is, by definition, signal data. Multiple works have highlighted the superiority of DL (deep learning) methods, mostly but not limited to CNN & transformers, to such data. However, the word “signal” is never used throughout the manuscript and DL models are absent from the models trained by the authors. The question of why the authors chose to elude this major point stays unanswered at the end of this manuscript. The ML (meachine learning) knowledge of the authors does not seem to be an obstacle to the use of DL models, based on the expertise they demonstrate in ML and DL, largely discussing the upsides and downsides of various ML models including DL ones. Furthermore, training/inference speed should not be an issue either, since 60k samples times 300 points per curve is a relatively small amount of data to process (the authors state that 20 minutes were needed to train all 26 models in this study).

The use of SHAP values, here, further highlights this problem. The models seem to simply “look at high values” in the beta & gamma regions and deduce if there is an M-spike in the sample. However, M-spikes are not always characterized by high values, and small M-spikes may only be visually detected, by the expert, thanks to an abnormal qualitative pattern, rather than by observing high quantities in the beta/gamma fractions (e.g., shoulder in the beta fraction). This is only possible with ML if treating the data as a signal, e.g., by using convolutional layers. Since the authors do not inform about the exact patterns observed in their dataset (see my previous paragraph about data), it is impossible to predict their models’ expected behavior on such samples. Unfortunately, those samples are crucial, and highly responsible for the fact that SPEP is not yet fully automated in modern laboratories.

Finally, the authors imply that their methodology may have other applications. This is highly doubtful, as they are few examples of biological assays outputting highly standardized/aligned signal data, which may give such robust results with the methodology they use. Indeed, the previous studies the authors cite to support their work have been using this kind of ML tools to make predictions based on concentrations deducted from signal data, rather than the raw data themselves.

In total, this study seems highly promising, thanks notably to a highly valuable dataset, but which should be analyzed using state of the art machine learning tools to be considered in today’s literature.

Reviewer #2: This is a very interesting paper with a clear clinical question ( The major clinical question is the presence of monoclonal fraction(s) of antibodies (M-protein/paraprotein), which is essential for the diagnosis and follow-up of hematological diseases, such as multiple myeloma) . They also have a very nice dataset.

Regarding the adopted evaluation procedures, there is an important step that is missing: how did the authors choose the hyperparameters? Usually , one performs a gridsearch using k-fold in the training set as GridSeachCV form sk-learn does. If you use the test set to determine the best set of hyperparameters, the results woulb be biased.

it would be nice to compare the results provided by SHAP with standard feature importance that several tree-based algorithms can provide.

In the discussion , the authors say “Deep learning algorithms are highly suitable and powerful for image analysis but exhibit some disadvantages. One such weakness is the “black box” phenomenon, the inability to explain how the outcome result was achieved”. One can say the same thing about XGB or LGBM. For deep learning , there are several explanation methods that could be employed, including SHAP. Please correct this statement.

In the discussion the authors also say “The use of decision tree methods can be more transparent compared to deep learning algorithms, if implemented as we propose here, but work best for numerical series of data rather than data retrieved from images”. Decision trees are more transparent, but this is not true for the other algorithms derived from them as XGB. It is also very hard to tune XGB or LGBM hyperparameters, because there are several of them(see the documentation)

The authors say that the limitation of their approach was the relatively low accuracy

scores for identifying the specific isotype of M-protein present, probably due to the low number of certain M-protein isotypes in the population. What the authors had was an imbalanced dataset. This could be correct using SMOTE techniques.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Floris CHABRUN

Reviewer #2: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Attachments

Attachment

Submitted filename: Reviewing PONE.pdf

PONE-D-23-17568R1Machine learning evaluation for identification of M-proteins in human serumPLOS ONE

Dear Dr. Rotter Sopasakis,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Mar 25 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

John Adeoye

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

Reviewer #3: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: First, I want to thank the authors for carefully studying and replying to each and every one of my comments, and for the thorough work I believed they have undertaken for improving their manuscript. In my opinion, all major concerns regarding this study have been addressed.

Particularly, the current version of their manuscript now enables a detailed understanding of the exact data used by the authors and potentially accessible to the community, notably due to the addition of Tables 1 and 2 and confirmation that all samples were double checked with both CZE and gel SPEP. This confirms the high value of those data.

I still have a few minor comments to the authors, which may or may not trigger modifications in the final manuscript, to their discretion:

- New tables 1 and 2 are interesting and I feel are nice additions to the manuscript. In Table 2, I am not sure about the use of the word “tertiles”, which implies that selected cut-offs, namely 1g/L and 5g/L, would divide the dataset into three subsets of equal size (1 third of the samples), which is not the case here.

- Several arguments against the use of deep learning in this study seem fallacious in my opinion. For instance:

o the assumed ability of tree models to better handle class imbalance due to Gini index/entropy overlooks the fact that cross entropy is one of the most widely used loss functions in deep learning; for the very same reason.

o the authors imply that convolutional layers would not be suitable to the analysis of 306-point-wide traces reshaped to 17x18 images. Analyzing small images is perfectly performed by CNN, for instance on countless MNIST examples online (28x28 images). Furthermore, 306-wide traces can (and should here) be reshaped to (306,1) arrays/tensors to avoid jeopardizing spatial information, as described in several previous works analyzing 1-dimensional signal. Finally, authors could even use 1d-conv layers to avoid reshaping raw traces if reshaping itself is a concern.

o I can understand that the authors refuse to use DL compared to tree models, and I think the “no free lunch theorem” sufficiently justifies this strategy. But in my opinion, the specific arguments cited above are misleading and should be removed from the final version.

- Authors corrected the name of the python package they used from “shapely” to “Shapely” version 1.7.1. I would like to make sure this is not a mistake: Figure 3 highly resembles plots obtained with the “SHAP” package based on “ShapLEy” (not “ShapELy”) values. Furthermore, SHAP is also cited multiple times by the author. ShapELy v 1.7.1 seems to be a Python package related to geometric analysis rather than feature importance. Can the authors confirm there is no typo here?

- Knowing that the authors, even unknowingly, complied with recommendations such as STARD is extremely positive. I think the readership’s trust would highly benefit from citing this in the Methods section, though this is not mandatory.

- The table cited in the “Response to reviewers” file, depicting the performance of models according to the M-spike concentration (<1g/L, 1-5g/L, >5g/L) is really interesting, and in my opinion its addition in the manuscript or supplementary material, or at least a sentence in the Results would be of interest for the readers of this work.

Reviewer #3: This is an interesting study that demonstrates the analysis of a large set of electrophoresis data using machine learning for the identification of M-proteins in serum. Two referees have thoroughly assessed the paper and provided useful feedback which has been carefully addressed by the authors. While I agree that the ML algorithms used are not overly sophisticated, their applicability to the problem makes the procedure easy to implement. It would be good if this could be tested on 'new' electrophoresis data from various sources to test the transferability of the method, but I am aware this is beyond the scope of the current study. The dataset is extensive and relevant and I think the paper will make a good addition to the literature.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Floris Chabrun

Reviewer #3: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Machine learning evaluation for identification of M-proteins in human serum

PONE-D-23-17568R2

Dear Dr. Rotter Sopasakis,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

John Adeoye

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The authors have carefully updated their manuscript and addressed all interrogations I had in the previous versions, and in my opinion is suitable for publication in its current form.

I would like to thank the authors for their time and for carefully rewriting some parts of their manuscript that may have lacked clarity in the past.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Floris Chabrun

**********