PONE-D-20-40628

Exploring the impact of regional variation on outcome prioritisation in core outcome set development: a case study in the field of gastric cancer surgery

PLOS ONE

Dear Dr. Alkhaffaf,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please revise the presentation of the study findings. Some guidance is reported below.

Please submit your revised manuscript by Aug 14 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Sandro Pasquali, M.D., Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments (if provided):

Overall the manuscript is very interesting as it investigates an underappreciated topic in surgical research. In this regard, authors should consider that most of readers are not familiar with the study topic and need some more guidance through the manuscript.

This is particularly important as the manuscript is well written, study hypothesis, aims and methodology are clear and conclusions robust. Also, the Discussion section stated “Ultimately, a COS will only achieve its stated goals if researchers use it.”thus supporting the dissemination of the results of this study.

Overall, I would request authors to improve their study presentation in order to make it more readable.

Title is difficult to get and there are repetitions. I would consider a simpler title such as “Development of a core outcome set for clinical trials in gastric cancer surgery: the GASTROS study”.

Abstract is not very informative of study findings. The result section should include the outcomes to be prioritized and report overall rates, otherwise it is very difficult to have an idea of what the manuscript analysed.

Result section is hard to get. For instance, outcomes should be grouped according to their main area, for instance: prognosis, post-operative morbidity, quality of life, long-term morbidity. This may make it easier for Authors to provide a more narrative reporting of their result section, which currently is just a reminder to tables, which is hard to follow for readers. Again , please consider taking the reader through the study results rather than ask him/her to look at tables.

Clearly authors want to reach surgeons and oncologists with this paper. How they plan to reach nurses? What about patients? It would be interesting to have authors’ strategy reported in the discussion.

A study limitation is the lack of medical oncologists and radiation oncologists. Can you please explain reasons for considering only surgeons for a disease that is treated multidisciplinary? This issue should be discussed.

In the study conclusion it should be reported that three stakeholders (surgeons, nurses and patients) were considered. There should at last be some examples of included outcomes, for instance those with the highest agreement.

I would appreciate some comment on the generalizability of these findings to other surgical oncology disease. Authors comment that gastric cancer has extensive differences compared to breast cancer. However these findings may be valuable, at least to some extents, for GI cancers. Please comment.

There is a typo in the first line of 4.4 – “is was”.

Journal Requirements:

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

3. Thank you for including your ethics statement:  "The study was given ethical approval by the North West - Greater Manchester East Research Ethics Committee (18/NW/0347) and governance approvals by Manchester University Hospitals NHS Foundation Trust.".   

Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified (1) whether consent was informed and (2) what type you obtained (for instance, written or verbal, and if verbal, how it was documented and witnessed). 

Once you have amended this/these statement(s) in the Methods section of the manuscript, please add the same text to the “Ethics Statement” field of the submission form (via “Edit Submission”).

For additional information about PLOS ONE ethical requirements for human subjects research, please refer to http://journals.plos.org/plosone/s/submission-guidelines#loc-human-subjects-research. 

4. Please include additional information regarding the survey or questionnaire used in the study and ensure that you have provided sufficient details that others could replicate the analyses. For instance, if you developed a questionnaire as part of this study and it is not under a copyright more restrictive than CC-BY, please include a copy, in both the original language and English, as Supporting Information. 

5. In your Methods section, please provide additional information about the participant recruitment method and the demographic details of your participants. Please ensure you have provided sufficient details to replicate the analyses such as: 

- the recruitment date range (month and year) 

- a description of any inclusion/exclusion criteria that were applied to participant recruitment

- a table of relevant demographic details

- a statement as to whether your sample can be considered representative of a larger population

- a description of how participants were recruited. 

6. Please list the name and version of any software package used for statistical analysis, alongside any relevant references. For more information on PLOS ONE's expectations for statistical reporting, please see https://journals.plos.org/plosone/s/submission-guidelines.#loc-statistical-reporting.

7. We note that the grant information you provided in the ‘Funding Information’ and ‘Financial Disclosure’ sections do not match. 

When you resubmit, please ensure that you provide the correct grant numbers for the awards you received for your study in the ‘Funding Information’ section. 

8. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For more information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. 

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially sensitive information, data are owned by a third-party organization, etc.) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

9.One of the noted authors is a group or consortium GASTROS International Working Group . In addition to naming the author group, please list the individual authors and affiliations within this group in the acknowledgments section of your manuscript. Please also indicate clearly a lead author for this group along with a contact email address.

10. We note that Figure 1 in your submission contain map image which may be copyrighted. All PLOS content is published under the Creative Commons Attribution License (CC BY 4.0), which means that the manuscript, images, and Supporting Information files will be freely available online, and any third party is permitted to access, download, copy, distribute, and use these materials in any way, even commercially, with proper attribution. For these reasons, we cannot publish previously copyrighted maps or satellite images created using proprietary data, such as Google software (Google Maps, Street View, and Earth). For more information, see our copyright guidelines: http://journals.plos.org/plosone/s/licenses-and-copyright.

We require you to either (1) present written permission from the copyright holder to publish these figures specifically under the CC BY 4.0 license, or (2) remove the figures from your submission:

a. You may seek permission from the original copyright holder of Figure 1 to publish the content specifically under the CC BY 4.0 license.  

We recommend that you contact the original copyright holder with the Content Permission Form (http://journals.plos.org/plosone/s/file?id=7c09/content-permission-form.pdf) and the following text:

“I request permission for the open-access journal PLOS ONE to publish XXX under the Creative Commons Attribution License (CCAL) CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). Please be aware that this license allows unrestricted use and distribution, even commercially, by third parties. Please reply and provide explicit written permission to publish XXX under a CC BY license and complete the attached form.”

Please upload the completed Content Permission Form or other proof of granted permissions as an "Other" file with your submission.

In the figure caption of the copyrighted figure, please include the following text: “Reprinted from [ref] under a CC BY license, with permission from [name of publisher], original copyright [original copyright year].”

b. If you are unable to obtain permission from the original copyright holder to publish these figures under the CC BY 4.0 license or if the copyright holder’s requirements are incompatible with the CC BY 4.0 license, please either i) remove the figure or ii) supply a replacement figure that complies with the CC BY 4.0 license. Please check copyright information on all replacement figures and update the figure caption with source information. If applicable, please specify in the figure caption text when a figure is similar but not identical to the original image and is therefore for illustrative purposes only.

The following resources for replacing copyrighted map figures may be helpful:

USGS National Map Viewer (public domain): http://viewer.nationalmap.gov/viewer/

The Gateway to Astronaut Photography of Earth (public domain): http://eol.jsc.nasa.gov/sseop/clickmap/

Maps at the CIA (public domain): https://www.cia.gov/library/publications/the-world-factbook/index.html and https://www.cia.gov/library/publications/cia-maps-publications/index.html

NASA Earth Observatory (public domain): http://earthobservatory.nasa.gov/

Landsat: http://landsat.visibleearth.nasa.gov/

USGS EROS (Earth Resources Observatory and Science (EROS) Center) (public domain): http://eros.usgs.gov/#

Natural Earth (public domain): http://www.naturalearthdata.com/

11. Please include a copy of Table 1 which you refer to in your text on page 18.

12. We note you have included a table to which you do not refer in the text of your manuscript. Please ensure that you refer to Table 31 and 32 in your text; if accepted, production will need this reference to link the reader to the Table.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: I Don't Know

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This is an interesting paper . I admire your efforts at recruitment for this Delphi.

In answer to the journal’s specific questions:

• Further data/evidence of analysis is offered in response to personal request.

• The manuscript is parsimonious, and well written and presented.

• It offers the tabular analysis I would expect to see to support the body of the paper, and the conclusions drawn seem sound in relation to the data presented. The sample sizes (you do point this out) are too small to make definitive conclusions, and there are many confounding factors (you also point this out) that may play a role in the variation you report.

As a reader:

I was initially not persuaded that subgroup analysis would be pragmatic in this context, but I inevitably felt drawn in by the tables and percentages, and was struck by Table 5 and how the results hang together here. You do state the key point that a COS is intended to allow synthesis, and to be useful in international trials – reminding the reader that the heart of COS methodology is the search for consensus rather than divergence. So the intention to analyse subgroups did raise some questions, but I think I’m persuaded that this was worth exploring (it made me think, always a good sign). Generally speaking, I felt that the paper was carefully worded so as not to stray into too much speculation.

Issues that came to mind as I read through (these are very general):

Section 2.3 Data analysis and interpretation - perhaps I'm misinterpreting this, but the first sentence suggests that participants included in the analysis did not complete the whole Delphi survey - in fact, some of them only completed 50% of the survey? It seemed to me that including surveys that were only partially filled in meant that you are not comparing like-with-like across the different outcomes, but I may be missing something here.

I would have appreciated a sentence or two summarising the concrete differences in pathology, treatment and outcome between regions (rather than the statement that there are differences), so that I could do some independent thinking around this. I don’t see why this paper shouldn’t flag up some potential issues around the consensus approach (rather than just reassuring us!), given that the same methodology for COS development is used across a variety of subject areas which are very different in shape. The COS development process is influenced by the nature of the condition being explored, whether you are trying to tie down outcomes for a single intervention or a range of interventions for the same condition, and the breadth of stakeholder roles involved (which is inevitably affected by the two previous issues). I wanted just a little more information about the background so that I could think about these issues without having to look them up in a separate paper.

It’s a benefit to this analysis to have a low number of stakeholder groups, and it’s a given that a study steering group should carefully think about which role perspectives (besides patients) they should include as respondents in a Delphi survey. I felt that the call to consider more finely grained characteristics of potential respondents when you plan recruitment was perhaps optimistic, as this is partly resource-dependent, and capacity in research institutions varies greatly. Not all COS projects would have funding to access demographic information about patients in order to guide recruitment (and adding demographic questions to the survey could be a burden and increase attrition). I’m not saying this isn’t a good idea for recruitment – just in an ideal world. I wonder whether the paper should touch on capacity issues and make some clear calls for investment in this kind of approach for COS studies that are likely to benefit from it, which would in turn feed back to funders of COS projects?

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

How are trial outcomes prioritised by stakeholders from different regions? Analysis of an international Delphi survey to develop a core outcome set in gastric cancer surgery.

PONE-D-20-40628R1

Dear Dr. Alkhaffaf,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Reviewer and editor's comments have been addressed and manuscript improved accordingly. 

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Sandro Pasquali, M.D., Ph.D.

Academic Editor

PLOS ONE