PONE-D-19-22935

First-line targeted therapies of advanced hepatocellular carcinoma: A Bayesian network analysis of randomized controlled trials

PLOS ONE

Dear Mr Xu,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

We would appreciate receiving your revised manuscript by Nov 10 2019 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Ivan D. Florez

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

http://www.journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and http://www.journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

1. Thank you for including your funding statement; "The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."

1. Please provide an amended Funding Statement that declares *all* the funding or sources of support received during this specific study (whether external or internal to your organization) as detailed online in our guide for authors at http://journals.plos.org/plosone/s/submit-now.  

1. Please state what role the funders took in the study.  If any authors received a salary from any of your funders, please state which authors and which funder. If the funders had no role, please state: "The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

Additional Editor Comments (if provided):

Dear Authors,

Your manuscript has been evaluated by two reviewers who have raised a number of points that need to be addressed before considering it for publication. Please provide a response to each one fo the points described below. In particular, make emphasis on the following points which may be the most problematic, and are esssential for any NMA:

- Protocol: Reviewer 1 has raised a crucial point regarding the protocol for this nma. One criterion for any systematic review to be of high quality is the presence of a protocol before starting the review. You haven't provided evidence that that protocol existed. At least a register in a database for Systematic reviews should have existed (e.eg., PROSPERO database). Please, either: provide evidence that the protocol/registration existed and was available before stariting the project or explicitly state in the mauscript that the review was not registered in PROSPERO database and did not have an online protocll available, and provide a brief explanation.

- Inconsistency. Reviewer #1 has raised the point on the inconsistency tests. You should explicitly provide the description of the global and the local tests performed to define the inconsistency (per loop), and provide the results of all the tests: the global and each one of the local tests perfomed for each outcome, and for each loop (fo the local). Depending on the tests used: P values, or ROR, should be shown, in the manuscript or in appendices

- Intransitivity: Reviewer #1 has pointed out that you made conclusions on transitivity, that are not supported in the manuscript, To conclude that you need to be clear about how this assumption was evaluated, considering the variables you describe that were used.

- Certainity of the evidence:Reviewer #1 has recommended to apply GRADE to determine what would be the certainity of the evidence (Also called quality of the evidence) with GRADE. We support that recommendation.

- Heterogeneity: Reviewer #1 pointed out as well that you didn't mention anything on the heterogeneity assessment. Heterogeneity should be evaluated and stataisitically assessed: Both: for the whole netwrok, and for pairwise comparisons. Also, in cases of high heterogeneity, special analyses such as Meta-regression and subgroup analyses shpuld have been performed.

- Reviewer #2 has described how your conclusions do not 100% reflect the results.

We expect you could provide appropriate answers to the concerns I have highlighted above along with the reviewers comments which are key to consider your manuscript.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Review for First-line targeted therapies of advanced hepatocellular carcinoma: A Bayesian

network analysis of randomized controlled trials

The authors evaluated first-line targeted therapies of advanced hepatocellular carcinoma via a network meta-analysis. This is the first NMA adding important information in the field of advanced hepatocellular carcinoma. However, I have concerns due to the lack of NMA methodological aspects and I think that authors should conduct a lot of things aiming to provide a comprehensive NMA analysis.

My comments focus as follows:

Abstract

I suggest to provide more information in methods section.

Please revise this sentence. “Direct and indirect evidence were 29 combined to time to progress (TTP), overall survival (OS), progress-free 30 survival (PFS), objective response rate (ORR), and the proportion of 31 Grade 3-5 adverse events (G3-5AE) and surface under the cumulative 32 ranking curve (SUCRA) of patients with advanced HCC.”

Provide the measure of outcomes and SUCRA values in a more informative and clear way in the sentence.

Please provide also that you implement the network meta-analysis model and the effect size used (OR and HR).

Materials and methods

Please refer about your NMA protocol. Have you a published protocol of the analysis?

Data synthesis and analysis

I advise authors to start this section reporting that they performed the NMA model and the measures used for treatment effects estimates.

Authors need to provide a global test for the assessment of inconsistency, such as the random-effects design-by-treatment interaction model and the local test (node-splitting). Then, authors could report in results section that it not applicable as they have star NMAs.

Missing details for Bayesian NMA. Authors report about posterior distributions but no further details for assumptions, for example prior distribution for model parameters. What priors for model parameters distributions? What prior distribution for heterogeneity? What assumption about heterogeneity, did they used a common between-study standard deviation across all treatment comparisons in each network? What about the assessment of transitivity? Authors need to compute measures for heterogeneity, for example I squared. Sensitivity analyses need to be conducted when it is needed. Rating the quality of the evidence in the estimates using GRADE criteria can also be conducted. Pairwise meta-analyses can also be conducted and I also suggest to conduct the comparison adjusted funnel plot.

Study characteristics

“There was no evidence that the transitivity assumption was violated in 177 any of the networks.” How authors conclude to this? Authors should refer about transitivity in later section of the analysis.

Results

I recommend to provide the results section taking into account all the methodological suggestions are given in Methods.

Reviewer #2: Authors perfored a network meta-analysis of the available RCTs comparing the efficacy of the various targeted therapes used in the management of HCC. This appeared to be a well conducted study with appropriately evaluted outcomes. I have a few comments:

1) Does the BCLC criteria of the included patients inflence the outcomes especially the overall survival? Is there a scope to perform sub-group analysis matching the patient and tumour charateristics in the current or future studies?

2) Are the conclusions reflecting the results appropriately? Soraferenib appears to remain superior from your analysis. THis should be one of the conclusions.

3)in the eligibility criteria: 'The key inclusion

85 criteria for study populations: more than 4 weeks since most recent local

therapy or no local therapy; no prior systemic therapy.' what does this mean?

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

PONE-D-19-22935R1

First-line targeted therapies of advanced hepatocellular carcinoma: A Bayesian network analysis of randomized controlled trials

PLOS ONE

Dear Mr Xu,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

We would appreciate receiving your revised manuscript by Jan 31 2020 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Ivan D. Florez

Academic Editor

PLOS ONE

Additional Editor Comments (if provided):

Thanks for submitting a revised version. Your revised manuscript has incorporated many of reviewers comments. However, in addition to reviewer #1 comments (which are below) there are a couple of points that hasn’t been completely addressed from reviewer #2, and additional issues identified by myself as Academic Editor:

Reviewer #2 points to be addressed:

1. Reviewer #2 pointed out how the BCLC criteria for the patients could have had an impact on the overall survival. You have pointed out that a subgroup analyses is not possible since the authors didn’t group patients by BCLC criteria. However, there are alternatives to this, considering that you agree that this stage could have an impact on the outcome, and therefore it can be a potential effect modifier.

- If authors provided a proportion of patients with a specific BCLC, you could group studies according to specific proportions

- If a subgroup analyses is not feasible at this point you should address this as a limitation in a paragraph at the end of the discussion section of your manuscript, and discuss there how were the tumor characteristics of the patients and how this may impact on the results.

2. Regarding reviewer#2 comment about the conclusions, you provide the next sentence: “For the moment, sorafenib was still as a first-line drug of first choice.” We think that according to the results, Srafenib should not be highlighted as the best approach. I think In conclusions you should emphasize in those interventions that were superior, BUT adding the quality of evidence for those. See the following sentence as an example:

For OS, Van (100 and 300mg), seem to be the best options with Low and moderate quality of evidence, respectively. For G3 5AE, Van 100 and 300mg), and play seem to be the best interventions, with low quality of evidence all of them (Just as an example, you need to apply GRADE methodology as I suggest below, and find the final quality assessment).

3. Also, regarding this comment from reviewer #2:

In the eligibility criteria: 'The key inclusion criteria for study populations: more than 4 weeks since most recent local therapy or no local therapy; no prior systemic therapy.' what does this mean?”
You have provided a response, However, it is not reflected in the paper. This part should be clear enough for reader so they could have the same question as Reviewer #2. Thus, please detail in this section, what does that mean and the reasons for that decision.

Major and minor Comments from Editor:

MAJOR:

You have explained your GRADE approach. However, major issues in your GRADE approach have been identified:

- You state that you used the methods recommended by: Puhan et al (2014). However, using only this method is not appropriate as it was the first approach and it has had at least 2 updates: Brignardello-Petersen et al. J Clin Epidemiol 2018, Brignardello-Petersen et al. J Clin Epidemiol 2019, Consider applying the full current approach to assess the quality of evidence.

- According to GRADE approach (Puhan’s and Brignardello-Petersen’s articles) you need to assess the quality of the direct, the indirect and the network evidence for each comparison and each outcome. Although you may present only your Network estimates and your network quality assessments in your main manuscript. However, you need to provide evidence that you conducted all the process appropriately and for that you need to provide all the direct, indirect and NMA estimates + quality of evidence. We cannot identify in your appendices these estimates (Except by some of them that are provided in the Forest plots, and without each GRADE assessment) for readers that would be interested in them.

- Also in page 35 you state:

“Therefore, in the present NMA, the GRADE score was downgraded universally due to the diversity of patient characteristics and the disunity of evaluation standard”. This sentence is very problematic, because this is not what GRADE guidance recommends. GRADE working group never recommends to downgrade comparisons “globally”. This shows a misunderstanding of the approach. If you apply GRADE, you should follow all the recommended steps by Puhan’s and Brignardello-Petersen’s articles which are the official GRADE working group Guidance.

- in the same page you state:

“Second, it was further downgraded in certain comparisons for no direct evidence”. That may be true. But again, the only way to identify what comparison were downgraded and for what reasons (GRADE Criteria that were use to downgrade in each direct and indirect and NMA estimates)), is to provide in appendices a full list of all the available estimates (D, I, and NMA) for all the outcomes, along with the corresponding GRADE assessment, following the GRADE guidance, and indicating the reasons for downloading each one of them, according to the 7 criteria recommended by the GRADE approach.

- Also, in the same page, you state: “In addition, it was further downgraded in some relevantcomparisons because of their IF below 50%”. Could you explain how this criterion match with the GRADE approach recommended by Puhan’s and Brignardello-Petersen 2018 and 2019, One more time, there should be evidence of what comparisons were downgraded by any particular criterion

- Finally, as you state that you calculated I2. There is no information in the manuscript nor the appendices that shows the i2 for all the direct comparisons. You should provide this data as you described you calculated them to assess the heterogeneity of direct comparisons

MINOR:

Additional minor changes to consider are:

- You excluded one study and the reason provided is that it was data deficient. please clearly explain what does that mean.

- Page 14, line 80, Please write PRISMA in Capital letters.

- Page 14, line 80. There is a specific PRISMA Guidance for NMA, it seems you followed the PRISMA for regular Sr and MA. You need to follow Guidance that is specific for NMA (PRISMA NMA). You also need to provide the appropriate reference for the PRISMA-NMA Guidance.

- Page 14, line 82. Since you cannot provide a PROSPERO registration number at this time, lease indicate the date you submitted the register to the PROSPERO database. Also indicate what was the state of this PROSPERO submission by the time you submitted this manuscript to PlosOne

- FIGURE 4 is of very low quality, you need to provide a better image

- Page 35: They “IF” acronym has not ben explained before and has not been detailed in the methods as a key approach

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: I am happy that the authors provided a registered protocol in PROSPERO database. I recommend authors to revise the sentence “It did not have an online protocol…..under review” with “The protocol has been registered in PROSPERO database and it is now under review”

The authors addressed most of the concerns and suggestions about the NMA methodology and the manuscript has been improved. Although, I have some comments and concerns for inconsistency checking.

Based on review comments, authors should have provided a global test for inconsistency assumption, eg. Design-by-treatment interaction model (DBT) and if inconsistency was detected they should have provided a node-splitting approach.

I don’t agree with the interpretation provided for the results of the Node-splitting method. Node-splitting approach checks if direct and indirect is in agreement (consistency). Authors concluded to “The result showed a robust and homogeneous network of evidence”. This should be revised. The results of all tests (P-values) for each outcome could be reported in the manuscript with S7, S13, S16 and S19 figures.

All the forest plots provided for NMA should be renamed in figures and tables with Comparison-adjusted funnel plot. Moreover, dashed lines in plots are missing and should be provided (Figure S20).

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

First-line targeted therapies of advanced hepatocellular carcinoma: A Bayesian network analysis of randomized controlled trials

PONE-D-19-22935R2

Dear Dr. Xu,

We are pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it complies with all outstanding technical requirements.

Within one week, you will receive an e-mail containing information on the amendments required prior to publication. When all required modifications have been addressed, you will receive a formal acceptance letter and your manuscript will proceed to our production department and be scheduled for publication.

Shortly after the formal acceptance letter is sent, an invoice for payment will follow. To ensure an efficient production and billing process, please log into Editorial Manager at https://www.editorialmanager.com/pone/, click the "Update My Information" link at the top of the page, and update your user information. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, you must inform our press team as soon as possible and no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

With kind regards,

Peter Starkel, M.D., Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

All comments have been addressed. No further remarks.

Reviewers' comments: