Peer Review History

Original SubmissionAugust 26, 2022
Decision Letter - Charlotte Beaudart, Editor

PONE-D-22-23846Development and preliminary validation of the Australia & New Zealand Musculoskeletal (ANZMUSC) Clinical Trials Network Research Question Importance Tool (RQITv2.0)PLOS ONE

Dear Dr. Taylor,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Nov 18 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Charlotte Beaudart

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. We note that the grant information you provided in the ‘Funding Information’ and ‘Financial Disclosure’ sections do not match. 

When you resubmit, please ensure that you provide the correct grant numbers for the awards you received for your study in the ‘Funding Information’ section.

3. Thank you for stating in your Funding Statement: 

"This study was partly funded by the NHMRC (Centre of Research Excellence: Australia & New Zealand Musculoskeletal (ANZMUSC) Clinical Trials Network; Chief Investigator Rachelle Buchbinder). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."

Please provide an amended statement that declares *all* the funding or sources of support (whether external or internal to your organization) received during this study, as detailed online in our guide for authors at http://journals.plos.org/plosone/s/submit-now.  Please also include the statement “There was no additional external funding received for this study.” in your updated Funding Statement. 

Please include your amended Funding Statement within your cover letter. We will change the online submission form on your behalf.

4. Thank you for stating the following in the Acknowledgments Section of your manuscript: 

"We gratefully acknowledge the help of Emeritus Professor Jane Latimer, Helen Ramsay (Executive Officer of ANZMUSC) and Tania Pocock (Health Research Council of New Zealand), Dr Chris Dalton (National Medical Officer BUPA Australia) for participation in the consensus workshop. RB is supported by an Australian National Health and Medical Research Council (NHMRC) Investigator Fellowship (APP1194483). RSH is supported by a NHMRC Senior Research Fellowship (#1154217). ANZMUSC is supported by a Centre of Research Excellence Grant by NHMRC (APP1134856)."

We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form. 

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows: 

"This study was partly funded by the NHMRC (Centre of Research Excellence: Australia & New Zealand Musculoskeletal (ANZMUSC) Clinical Trials Network; Chief Investigator Rachelle Buchbinder). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

5. In your Data Availability statement, you have not specified where the minimal data set underlying the results described in your manuscript can be found. PLOS defines a study's minimal data set as the underlying data used to reach the conclusions drawn in the manuscript and any additional data required to replicate the reported study findings in their entirety. All PLOS journals require that the minimal data set be made fully available. For more information about our data policy, please see http://journals.plos.org/plosone/s/data-availability.

Upon re-submitting your revised manuscript, please upload your study’s minimal underlying data set as either Supporting Information files or to a stable, public repository and include the relevant URLs, DOIs, or accession numbers within your revised cover letter. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. Any potentially identifying patient information must be fully anonymized.

Important: If there are ethical or legal restrictions to sharing your data publicly, please explain these restrictions in detail. Please see our guidelines for more information on what we consider unacceptable restrictions to publicly sharing data: http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. Note that it is not acceptable for the authors to be the sole named individuals responsible for ensuring data access.

We will update your Data Availability statement to reflect the information you provide in your cover letter.

6. One of the noted authors is a group or consortium ANZMUSC Clinical Trial Network. In addition to naming the author group, please list the individual authors and affiliations within this group in the acknowledgments section of your manuscript. Please also indicate clearly a lead author for this group along with a contact email address.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The topic is very important and some of the tools used by the authors are either inoovative or scientifically robust and the authors are to be commended for this. However, there are a number of problems that limit the validity of the research or that limit the understanding of the paper:

- The title is not very clear. Please update it so that it really reflects the work of the authors.

- The developers of this tool are all from Australia or New Zealand. If the goal is to provide a tool specifically for Australia and New Zealand, I wonder if this merits an international publication.

- The inclusion of "version 2.0" in the name of the tool is somewhat misleading as there is no real version 1. The "new" version is a simple and logical evolution of the tool during its development.

- All (or most) of the experts are in the field of musculoskeletal disorders and therefore the tool cannot be considered a generic tool. This should be reflected in the name of the tool so that it will not be used in a bad way.

- The main problem with this tool is how the initial validation is done. IF and citation counts are very poor and totally arbitrary proxies for research quality. This is acknowledged by the authors, but it does not solve the problem. At least, some sentitivity analyses with the use of other journals can be made (and not anly to 5 top and down journals).

- Surprisingly, contrary to current practice, patients were not considered in the development of the tool and the potential consequence of this is that the patient perspective is not considered a potential attribute.

- Much more information needs to be provided on the 14 hypothetical research questions. If these research questions are written in a way that makes scoring easy, the bias is too great. I strongly recommend using the previously submitted applications.

- The authors state that this is a preliminary validation but they do not clearly explain what their next steps are and why this was not done for this article.

- For full transparency, it is important to include in the paper the full final version of the tool or to explain how to access it and to explain very clearly how to score it.

Reviewer #2: The subject of the article: Development and preliminary validation of the Australia & New Zealand Musculoskeletal (ANZMUSC) Clinical Trials Network Research Question Importance Tool (RQITv2.0).

After reading the article, the reviewer found it to be an original and innovative manuscript. The manuscript is not characterized by significant formal deficiencies and factors that interfere with interpretation. The style and language are booth good, the structure is understandable, and the authors have devoted plenty of time to providing the necessary background knowledge.

The abstract summarizes the essence of the article well.

In the introduction, the reviewer is slightly missing additional data in a global context, or in comparison with other countries. According to the reviewer, it should be considered to correct this.

The method part has been precisely defined. The methods are well described, authentic and can be followed, the results presented in the tables are understandable and convincing. The description of the results and the use of tables are appropriate.

The discussion is satisfactory both in terms of content and scope.

In summary, all of these give the manuscript its significance, while I detailed above that the authors solved this excellently. the reviewer recommends the article for publication, which, in the reviewer's opinion, contains valuable data and information.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Revision 1

Re: PONE-D-22-23846

Development and preliminary validation of the Australia & New Zealand Musculoskeletal (ANZMUSC) Clinical Trials Network Research Question Importance Tool (RQITv2.0)

Charlotte Beaudart

Academic Editor

PLoS One

Dear Dr Beaudart

Many thanks for allowing us the opportunity of re-submission. I have tabulated our response to the reviewers’ and editor’s comments. Any references to the text refers to the re-submitted version of the manuscript.

The funding statement should state:

This study was funded by the Australian National Health and Medical Research Council (NHMRC) Australia & New Zealand Musculoskeletal (ANZMUSC) Clinical Trials Network Centre of Research Excellence (APP1134856). RB is supported by an NHMRC Investigator Fellowship (APP1194483). RSH is supported by an NHMRC Senior Research Fellowship (APP1154217). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Editorial comments

1. File naming and format requirements; Response - We have re-named files and formatted the manuscript to conform to style requirements Level 1 headings bold and 18pt font; level 2 headings bold and 16pt font, Figures and Table citations have been changed. Supplementary files have been created.

2. Mismatch of ‘funding information’ and ‘financial disclosure’ statements; Response - Correct grants numbers are now provided

3. Funding statement; Response - Amended funding statement to include all funding and sources of support, in cover letter above.

4. Acknowledgements section; Response - Funding information moved to the Funding Statement as suggested.

5. Data availability; Response - We have added data files to the repository to permit a minimal data-set.

6. Group authorship; Response - We wish to withdraw the group author as all authors are already individually included. Author list is amended.

Reviewer 1 comments

1. Unclear title; Response - We have changed the title to emphasize the context of the assessment tool. Title is now changed from: “Development and preliminary validation of the Australia & New Zealand Musculoskeletal (ANZMUSC) Clinical Trials Network Research Question Importance Tool (RQITv2.0)” to: “Which clinical research questions are the most important? Development and preliminary validation of the Australia & New Zealand Musculoskeletal (ANZMUSC) Clinical Trials Network Research Question Importance Tool”

2. The developers of this tool are all from Australia or New Zealand. If the goal is to provide a tool specifically for Australia and New Zealand, I wonder if this merits an international publication; Response - The goal is to provide a tool applicable in any geographic location. There are no aspects of the work that relate to local factors. No changes made.

3. Naming of the tool version; Response - We agree that this is potentially misleading, but we needed a means of referring to different versions of the tool as it developed during the study. We have removed 2.0 from the name of the final version of the tool and have used a different naming system within the manuscript to facilitate accurate understanding.

4. MSK specific tool; Response - We agree and have referred to the tool as Australia and New Zealand Musculoskeletal Clinical Trials Network Research Question Importance Tool (ANZMUSC-RQIT) to reflect this. ANZMUSC appended to final tool name in title and throughout manuscript.

5. Validation; Response - As noted in the limitations section of the manuscript, we agree that validation is challenging and bibliographic indices are far from ideal. However, the suggested sensitivity analysis will only compound this problem by continuing to employ bibliographic indices that will have even less certain meaning. A key step in our approach was to confirm a relationship exists between clinical trial quality as assessed by the Cochrane RoB tool and journal impact factor (see page 15, line 316-318 in methods and page 26 lines 462-464 in results (and also Table S1). We believe this is a better way to provide evidence to support our approach. No changes made.

6. Patient involvement; Response - We are surprised by the reviewer’s comment and wonder if our use of the term ‘consumer’ was somehow misunderstood. We have therefore clarified in the manuscript that consumers included patients and described their involvement in more detail. Additional text ln195-201 on pg 11: “Consumers (health-service users) were involved at every stage of the project, including generation of possible attributes of research importance (9% of participants), consensus development of the initial RQIT framework (18% of participants), weighting of specific elements of the framework using a novel best-work scaling method (12% of participants) and final authorship of this manuscript. Consumers were identified from membership of ANZMUSC and all had lived experience of musculoskeletal disorders.”

7. Hypothetical research questions; Response - These are now provided as a supplementary appendix. We do not believe it is necessary to repeat the reliability analysis. It is sufficiently clear that scoring by individuals is not reliable and only scoring by committee consensus is adequately reliable. New supplementary appendix.

8. Next steps for validation; Response - This is a fair comment. The main next step for assessing the tool will come from actual use in practice. We thought it was important that the tool be made available for other groups to use by having the work published. We argue that it is sufficiently ready for practical use but that further evaluation will only come from experience in practical settings. That could not be part of the current development phase which is described in this manuscript. Additional text in the Discussion section page 34, lines: 590-592 “Further validity evaluation should be undertaken by assessment of how the tool works prospectively in practice and comparison of scores with citations in subsequent clinical guidelines.”

9. Include the full final version and scoring instructions; Response - This was already provided as Table 4 and a worked example was provided. No changes made.

Reviewer 2 comments

1. Global context or comparison with other countries; Response - We appreciate that Australian and New Zealand data was used to illustrate MSK burden of disease, but do note that the introduction refers to the Global Burden of Disease project. We have added information from the US. New text added to introduction: “In the United States, funding for the National Institute of Arthritis and Musculoskeletal and Skin Diseases (NIAMS) have never been more than 2% of the National Institutes of Health annual disbursement (7)”.

2. Positive comments; Response - We appreciate the reviewer’s favourable opinion on the merits of the manuscript.

Attachments
Attachment
Submitted filename: Response to reviewers_final.docx
Decision Letter - Charlotte Beaudart, Editor

Which clinical research questions are the most important? Development and preliminary validation of the Australia & New Zealand Musculoskeletal (ANZMUSC) Clinical Trials Network Research Question Importance Tool (ANZMUSC-RQIT)

PONE-D-22-23846R1

Dear Dr. Taylor,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Charlotte Beaudart

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: ok

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

**********

Formally Accepted
Acceptance Letter - Charlotte Beaudart, Editor

PONE-D-22-23846R1

Which clinical research questions are the most important? Development and preliminary validation of the Australia & New Zealand Musculoskeletal (ANZMUSC) Clinical Trials Network Research Question Importance Tool (ANZMUSC-RQIT)

Dear Dr. Taylor:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Charlotte Beaudart

Academic Editor

PLOS ONE

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .