PONE-D-23-35166Mass media campaigns and the ‘file drawer problem’: a Delphi study of how to avoid campaign failurePLOS ONE

Dear Dr. Kite,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

As a reviewer I had concerns about how the methodology was described (see my review below), but with my hat on as Academic Editor, my view is that if it is going to be presented as a Delphi study, then the case for this needs to be clearly made in the paper, and the modifications need to be justified in methodological terms. This will also require additional detail in the methods section and the results section that are typically expected in Delphi studies (as summarised in the checklists that are available for Delphi studies, such as the CREDES checklist). However, on further reflection, it may be decided that the study would be better described simply as a sequential mixed methods study. This would be acceptable, and the study design would then suit the change in focus at each stage, as this would reflect a logical development of the study.

Please submit your revised manuscript by Mar 28 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Simon White

Academic Editor

PLOS ONE

Journal requirements:

1. When submitting your revision, we need you to address these additional requirements.

Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf.

2. Note from Emily Chenette, Editor in Chief of PLOS ONE, and Iain Hrynaszkiewicz, Director of Open Research Solutions at PLOS: Did you know that depositing data in a repository is associated with up to a 25% citation advantage (https://doi.org/10.1371/journal.pone.0230416)? If you’ve not already done so, consider depositing your raw data in a repository to ensure your work is read, appreciated and cited by the largest possible audience. You’ll also earn an Accessible Data icon on your published paper if you deposit your data in any participating repository (https://plos.org/open-science/open-data/#accessible-data).

3. We note that you have indicated that there are restrictions to data sharing for this study. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For more information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. 

Before we proceed with your manuscript, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information, data are owned by a third-party organization, etc.) and who has imposed them (e.g., a Research Ethics Committee or Institutional Review Board, etc.). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. For a list of recommended repositories, please see

https://journals.plos.org/plosone/s/recommended-repositories. You also have the option of uploading the data as Supporting Information files, but we would recommend depositing data directly to a data repository if possible.

We will update your Data Availability statement on your behalf to reflect the information you provide.

4. Please remove your figures from within your manuscript file, leaving only the individual TIFF/EPS image files, uploaded separately. These will be automatically included in the reviewers’ PDF.

5. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information. 

Additional Editor Comments:

Academic Editor acting as Reviewer 2 (due to difficulty finding a second reviewer for this topic):

This paper concerns mass media public health campaign failure, which is an important topic, but little appears to have been published about it. As such, this paper has the potential to make a valuable contribution to the field. The paper is generally well-written, and the results feel credible, from a qualitative point of view at least, but I have concerns about how the methodology is presented.

I agree that the study adopted a sequential mixed methods design, but it used such an atypical approach to a Delphi study that it is not clear whether it can still be classified as a Delphi study. Whilst definitions vary, the Delphi technique is generally regarded as a consensus methodology in which a Panel of experts individually consider their position on aspects of a topic and in subsequent Rounds, following feedback on the collective opinion of the Panel on these aspects from the previous Round, the (remaining) Panellists reconsider their previously declared position on these aspects, towards achieving (or not achieving) consensus. No explanation or justification (e.g. using methodological literature) is provided in the paper for such deviation from even accepted modifications to a ‘standard’ Delphi approach and the study seems instead to have broadly explored the topic and refined the findings, rather than specifically achieving a pre-defined consensus on aspects of the topic.

The paper reports that participants were asked to participate in two Rounds (either Round One and Round Two, or Round Two and Round Three) and that the aim of each Round was different (Round 1 focused on identifying common points of failure in campaigns; Round 2 focused on assessing which of the points of failure were most significant; and Round 3 focused on identifying solutions). Therefore, given that the stated aim of each Round was so different, it is not clear if or how Panellists could have reconsidered previously expressed views in the subsequent Round they participated in. This also begs the question of whether Round 1 can even be considered to be a Round in the Delphi study – it may be better described as preliminary work that informed the start of a 2-Round modified Delphi study (i.e. the output of the preliminary work informed the briefing for the first Round and this constituted the major modification of the Delphi approach).

However, Delphi studies typically start with identifying the range of views expressed by the Panel on a topic and then subsequent Rounds narrow the focus in pursuit of consensus. As such, ideally all Panellists will complete all Rounds, but whilst Panellist attrition in later Rounds is common (and good quality Delphi studies typically consider the impact of this), the addition of new Panellists in subsequent Rounds is an unusual feature that seems contrary to a process that funnels consensus development through Panellists’ completion of a series of Rounds.

Moreover, the task required in Round 3 (participation in a qualitative interview) in particular does not appear to have required reconsideration of Panellists’ previously expressed views or re-rating / re-scoring them on the basis of Panel opinion(s) from Round 2. As such, it is not clear whether or what consensus was achieved – the term is not mentioned or defined in the paper (e.g., in the methods section) and no measurements / assessments of it are presented in the results or are later discussed. This appears to be fundamentally incompatible with the claim of this study being a Delphi study.

In addition to this there are other issues that need to be amended as follows:

An explanation is needed in the main text of how the descriptive statistical analysis of quantitative data from Round 2 informed the analysis presented in the results section.

The description in the results of how the Round 3 data was analysed to generate the 4 major themes is a point of method and not a result.

The results need to be presented more succinctly. Even for a qualitative analysis, the results section is too long.

The discussion section would benefit from the implications of the work being better teased out (perhaps at the end).

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Thank you for inviting me to review this paper. I confirm that I have both, subject and methodological expertise. Below are my comments and recommendations for the authors’ considerations.

ABSTRACT

- Delete “surprisingly”

- The methods section of the abstract is very short

INTRODUCTION

- Well written overall.

METHODS

- Move lines 89 to 91 up under Methods (to become line 86).

- Heading on line 88 to be rephrased “recruitment.”

- Recruitment is unclear. Provide a breakdown as to how each “sub-population” was recruited/approached.

- It is unclear what the target sample size was, and why it was chosen.

- Description of the sample usually belongs under results. See checklists like SRQR to guide the reporting of this paper.

- It is unclear as to how this paper is mixed methods (as opposed to purely qualitative).

- Would be good to see a table summarising participant characteristics.

- It would be good to see a figure illustrating the various rounds with the various stakeholders (the process by which data were collected).

- There is Figure 2 but not Figure 1. Revise/proofread.

- It is unclear how analysis was done. Was there coding? What guided the choice of themes? Were there sub-themes? Which guidelines were used? What was done to enhance trustworthiness? How many people anaylsed data? Was there cross-checking to ensure accuracy?

RESULTS and DISCUSSION

- Quite lengthy yet informative. Great to see illustrative participant excerpts in the results.

FIGURE 2 (which is probably figure 1 since there is no figure 1) is pixelated and hard to read.

FIGURE 3 (which will become figure 2) captures the results very well.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Mass media campaigns and the ‘file drawer problem’: a mixed methods study of how to avoid campaign failure

PONE-D-23-35166R1

Dear Dr. Kite,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Simon White

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments: