PONE-D-19-34131

Development and validation of individualized miRNA signatures for sarcoma

PLOS ONE

Dear Dr. Zhu,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

We would appreciate receiving your revised manuscript by Apr 19 2020 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

David M Loeb

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements:

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at http://www.plosone.org/attachments/PLOSOne_formatting_sample_main_body.pdf and http://www.plosone.org/attachments/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. Please see http://www.bmj.com/content/340/bmj.c181.long for guidelines on how to de-identify and prepare clinical data for publication. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

3. Please include a copy of Table 1 and 2 which you refer to in your text on page 7.

4. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Partly

Reviewer #3: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: I Don't Know

Reviewer #2: No

Reviewer #3: I Don't Know

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: In this paper by Jin et al, the authors evaluated circulating microRNA signatures from patients with soft tissue sarcomas (STS) in order to develop predictive models of diagnosis. The investigators evaluated more than 880 sarcoma samples and determined 13 gene pairs that could be used for diagnosis with an AUC of 100% in their initial set, but only 68% in the external validation set. Of note, they also determined that the miRNA gene pairs showed enrichment for functional roles in cell cycle regulation. They concluded that their serum miRNA pairs can be used for the diagnosis of STS with high degree of accuracy and sensitivity. Overall, these authors have developed a non-invasive and potentially clinically applicable prediction model using serum microRNA signatures. However, there are some concerns regarding their approaches and conclusions, which are mentioned below.

Lines 59-60: Please further explain exactly what is meant by the statement: “detecting the level of miRNAs in body fluids, the state of somatic tumor cells can be inferred..”

Lines 106-107: What does it mean for the miRNAs to be “paired in the order from front to back”?

It is still not clear how the miRNAs were paired. It would be helpful to more of the general audience to further explain this process.

Lines 159-160: How were the 96 gene pairs, from more than 11,0000 pairs, selected to construct the prediction model.

But then in lines 163-164 it is stated that a total of 13 gene pairs, containing 16 miRNAs were used. All of this process is very confusing and difficult to follow for the reader.

Lines 173-174: It would be informative to provide more information regarding the other models used for prediction development and the comparison with this particular model.

Finally, all of the figures are extremely difficult to review. The clarity is extremely poor.

Reviewer #2: In this manuscript, the authors describe a miRNA gene pairs-based index to differentiate patients who have bony lesions from healthy controls. The concept is interesting and such biomarker development has tremendous promise in the sarcoma field. In particular, it is intriguing that the predictive signature was consistent across all stages of disease. However, there are critical weaknesses that need to be addressed for this manuscript to be considered for publication.

Major critiques:

1. The patient cohort is comprised of tumors listed as benign, intermediate, and malignant, all of which (individually and collectively) can be distinguished from healthy controls. Benign bone lesions (40+% in the cohort), by definition, are not sarcomas, and therefore the major conclusion that this model can distinguish sarcoma is not substantiated by the data. The signature can impressively distinguish all bone lesions from healthy controls in the test/training set, and there is only a minimal ability (figure 3C) to distinguish malignant vs benign. The authors may want to contend that a benign bone lesion may undergo malignant transformation, and therefore distinguishing benign vs intermediate vs malignant is not critical in their model.

2. The title, abstract, and body of the manuscript state that the approach has been validated, yet the validation in a very small (25 patient) external dataset did not perform nearly as well as the test/training set (65%). This is not unusual performance for such assays when there are differences in sample processing, chips, etc, though it fails to achieve the bar of validation as highlighted in the title and elsewhere. Of interest, the external dataset contained patients with varied sarcomas (i.e., malignant tumors), yet the test/training sets were composed of patients who had malignant and benign lesions. Separating out benign from malignant, and possibly eliminating patients with “intermediate” tumors from your analysis may lead to a more robust predictive model.

3. The broad title focused on miRNA signatures for sarcoma does not reflect the findings in the manuscript since it’s not only sarcoma that was differentiated from healthy controls, but rather bone lesions that may or may not be sarcoma.

Minor critique:

1. In the introductory background about the promise of early detection of patients with sarcoma from healthy controls, it would be more compelling to describe practical application, such as a patient presenting with a bone lesion and this model signature can differentiate benign from malignant. Further clinical application of such an assay could be as a post-treatment screening test for patients with sarcoma to identify early recurrence of disease. However, based on the major critiques above, this may not be a potential application of the described research.

Reviewer #3: This manuscript utilizes an in silico analysis of miRNA database to develop a predictive model for the detection of sarcomas from serum samples.

Some suggestions for the authors consideration:

1. Line 160, the authors states that 96 gene pairs were selected to construct the prediction model, however in the prediction model the authors state that a total of 13 gene pairs were used. The authors should consider in explaining further as to how they further narrowed down the gene pairs from 96 to 13 and perhaps also why they felt that was necessary.

2. In Line 178, the authors state that the gene pair prediction model showed an AUC of 68.4% in the outside data set. This is a large drop in the ability of the model to predict sarcoma when using an external data set. In some literature an AUC of 0.7-0.8 is considered acceptable and 0.8-0.9 is considered excellent. The authors conclusion that their model has high accuracy and sensitivity does not seem to be consistent when they tested their model to the external data set. The AUC of 100 in the internal data set, may simply denote that differences in the methodologies used to measure the serum miRNA may have a larger impact than the gene-pair model they developed. The authors should explain the implications of the drop in the AUC in regards to the broader applicability of the model.

3. The authors conclude that their test is very sensitive. It would be very important for the authors to better state the stage of the disease when the serum samples were obtained. If the serum samples are all obtained when the patients have active detectable disease, what would be the applicability of this model? Knowing that a patient had a "sarcoma" may not necessarily be practically beneficial, as a biopsy would still be required to define histological type. In lines 239-240, the authors state that this model "is promising in the diagnosis of sarcoma". However, that information, even if the AUC was better than 68% would not be sufficient for making treatment decisions. How well does this model work in the minimal residual disease state? This may have broader applicability to replace serial imaging? The authors may consider narrowing their conclusions to state that the model they have generated was outstanding in their internal test data set, but had significant reduction in the AUC when exposed to an internal data set in detecting sarcomas in patients with known active disease.

4. in the methods section for the sake of completeness, the authors may consider adding on which platform the statistical analysis was performed.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

PONE-D-19-34131R1

A novel serum miRNA-pair classifier for diagnosis of sarcoma

PLOS ONE

Dear Dr. Zhu,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please address the 2 minor concerns raised by Reviewer 2.

Please submit your revised manuscript by Jul 18 2020 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

David M Loeb

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: All comments have been addressed

Reviewer #3: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #2: Yes

Reviewer #3: Partly

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: Yes

Reviewer #3: I Don't Know

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: The authors have addressed reviewer critiques. An additional limitation of the study that was not addressed in the discussion is pooling of all stages of sarcoma, as this may skew results/accuracy/specificity in that higher stage patients may have more biologically aggressive disease and circulating tumor biomarkers. Subset analyses, if numbers were high enough, would be an additional important part of future investigation and validation of the signature.

Reviewer #3: the authors have addressed all the reviewer's comments. However, they have changed some language, which may require further clarification.

1. Statistical analysis: in the original manuscript, the authors used AUC. In this version, the authors changed the analysis to accuracy. They should explain in the statistical analysis how they define accuracy. They also should include that they are going to describe sensitivity, specificity, negative and positive predictive values in the statistical section.

2. Results: characteristic datasets. In the original manuscript the authors had included intermediate and low grade tumors. As per the authors, they have excluded these from this analysis. They may want to clarify in the characteristics of datasets section that they only included high-grade lesions.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: No

Reviewer #3: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

A novel serum miRNA-pair classifier for diagnosis of sarcoma

PONE-D-19-34131R2

Dear Dr. Zhu,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

David M Loeb

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments: