PONE-D-24-46282Effectiveness of digital health interventions against COVID-19 misinformation: a systematic realist review of intervention trialsPLOS ONE

Dear Dr. Dickinson,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please find the reviewers comments below this email.

Please submit your revised manuscript by Mar 29 2025 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org . When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols . Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols .

We look forward to receiving your revised manuscript.

Kind regards,

Osmond Ekwebelem

Academic Editor

PLOS ONE

Journal requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. As required by our policy on Data Availability, please ensure your manuscript or supplementary information includes the following: 

A numbered table of all studies identified in the literature search, including those that were excluded from the analyses.  

For every excluded study, the table should list the reason(s) for exclusion.  

If any of the included studies are unpublished, include a link (URL) to the primary source or detailed information about how the content can be accessed. 

A table of all data extracted from the primary research sources for the systematic review and/or meta-analysis. The table must include the following information for each study: 

Name of data extractors and date of data extraction 

Confirmation that the study was eligible to be included in the review.  

All data extracted from each study for the reported systematic review and/or meta-analysis that would be needed to replicate your analyses. 

If data or supporting information were obtained from another source (e.g. correspondence with the author of the original research article), please provide the source of data and dates on which the data/information were obtained by your research group. 

If applicable for your analysis, a table showing the completed risk of bias and quality/certainty assessments for each study or outcome.  Please ensure this is provided for each domain or parameter assessed. For example, if you used the Cochrane risk-of-bias tool for randomized trials, provide answers to each of the signalling questions for each study. If you used GRADE to assess certainty of evidence, provide judgements about each of the quality of evidence factor. This should be provided for each outcome.  

An explanation of how missing data were handled. 

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

Reviewer #2: N/A

Reviewer #3: N/A

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Thank you for the opportunity to review this manuscript. While a realist review of interventions against COVID-19 misinformation is timely, I have significant concerns about the theoretical framework employed, which ultimately impacts the validity of the conclusions. Therefore, I recommend rejecting the paper in its current form.

Regarding the theoretical concerns, the introduction has many affirmations without proper citations. I won’t name each of them but I think it is remarkable that the first page includes just one citation. This of course has ramifications, and the idea that Precovid interventions against misinformation were based only in fact-checking and nudges show to some extent unfamiliarity with a literature that is broad in topics like vaccine hesitancy and HIV misinformation. Also related to this point, a realist review should clarify the underlying mechanisms that explain how an intervention produces its effects and in which contexts these effects take place. My impression is that although the paper explains the different kinds of interventions evaluated, it do not provide a theoretical framework to understand the different mechanisms and contexts that produce different outcomes.

The lack of a robust theoretical framework hinders the interpretation of findings. One example of this is the conclusion about addressing polarization and politics bias to understand vulnerability towards misinformation. The relationship of polarization and misinformation is now commonplace i the US literature, but it is not clear how this review raised flags about it. Moreover, it is not clear if in the literature this was a global finding or it is just applicable to the US case. Finally, the conclusions that no intervention is sufficient to counter misinformation and that games interventions are more effective should be further elaborate, to unravel which interventions are proper for which contexts and populations, as that is the ultimate goal of a realist review.

Finally, author should take into account that PlosONE have specific criteria for publication of systematic reviews (https://journals.plos.org/plosone/s/criteria-for-publication and https://journals.plos.org/plosone/s/submission-guidelines#loc-systematic-reviews-and-meta-analyses). The first problem I find with this submission is that while PRISMA flow diagram and checklist are included in the submission, the checklist items were not developed as a standalone document, but have a “location where item is reported” that do not seem to correspond with a page number of the manuscript. Also, the protocol submitted with PROSPERO should be included as supporting information, and it would be valuable to address RAMSES II standards for realist reviews. Finally, related to PlosONE criteria for publication, the manuscript does not follow the standard structure of the journal publications. For example, the methods section is presented as the last section, while in PlosONE it usually is after the introduction. This hinders comprehension, and seems like an editing error for the Study characteristic section mentions information given “before” that is actually presented at the end of the document.

Despite these concerns, I believe this review has the potential to make a valuable contribution to the understanding of misinformation interventions. I encourage the authors to significantly revise their manuscript before resubmitting for publication. I hope this feedback proves helpful in your research process.

Reviewer #2: Dear Robert,

Thank you for submitting this systematic review on interventions against COVID-19 misinformation. This is an important contribution to public health research, addressing a critical challenge that has had far-reaching consequences. While the review is commendable in its scope and aims, I have identified some areas that could benefit from further attention and refinement.

1.Scope of the Review:

Is the review exclusively focused on digital health interventions (i.e., internet/electronic device-based approaches)? While many of the interventions discussed in the paper include digital components, others—such as community engagement initiatives involving health promoters and outreach activities—do not appear to have a clear digital aspect. Similarly, some interventions categorized as “educational” or “message framing” may not inherently fall under the umbrella of digital health. As such, the current title of the article may inadvertently lead readers to believe that the review is limited to digital interventions. I recommend considering a title that better reflects the broader scope of the interventions analyzed.

2. Flow Diagram Discrepancy:

The flow diagram indicates that 65 studies were assessed for eligibility, with 45 excluded for reasons such as lack of intervention or relevance to COVID-19. However, the original review ultimately included 26 studies, which does not align with the arithmetic presented (65 - 45 = 20). Could you clarify or revise the flow diagram to ensure consistency and transparency in the study selection process?

3.Clarity and Focus on Intervention Types:

• Despite the review highlighting the eight types of interventions, the categorization and definitions of these types could be clearer. It is not always evident what distinguishes certain categories, and some appear to overlap conceptually. A clearer framework or typology of interventions would enhance the reader's understanding of the findings.

• The emphasis on game-type interventions is interesting, but it would be helpful to discuss why these interventions performed better in terms of impact and scalability. Are there specific features or mechanisms that make them more effective?

4. Quality Assessment:

While the review mentions quality assessment, it is not clear how the quality of the included studies influenced the synthesis of findings. The review indicates that no study was excluded based on poor quality. Were lower-quality studies given less weight, or were their limitations explicitly considered in the analysis? Clarifying this would enhance the rigor of the review.

5. Heterogeneity of Outcome Measures:

The review notes high heterogeneity in outcome measures and methodologies across the included studies, which complicates comparisons. While this is an important observation, the review could delve deeper into how future studies might standardize these measures or propose a framework for consistent evaluation of interventions.

6. Backfire Effects and Subgroup Analysis:

The intermittent reporting of backfire effects and subgroup-specific outcomes is a critical limitation. It would strengthen the review to provide more detail on the implications of these gaps. For instance, how might the variability in reporting backfire effects impact the development of tailored intervention packages?

7. Expansion of Realist Review Approaches:

The review mentions contributing to the expansion of realist review methodologies but does not fully explain how this contribution is achieved. Expanding on this would strengthen the review’s methodological impact.

8. Contextual Considerations and Policymaker Recommendations:

The recommendation for policymakers to create tailored intervention packages is valuable. However, the review could benefit from more specific guidance on how to operationalize these packages. Are there concrete examples or evidence-based principles that could guide the tailoring process for different contexts and subgroups?

In summary, this is a highly relevant and well-intentioned review that provides valuable insights into a pressing public health issue. Addressing the above points would further enhance the clarity, rigor, and applicability of the findings, making this work even more impactful.

Best regards,

Reviewer #3: I have with interest read the manuscript which aim to summarize research on covid-19 misinformation information. I am not a social or communication scientist, but working with covid-19, also in a public health perspective. From what I understand the research field of misinformation is new but rapidly expanding. I find the subject of this review of high importance. The manuscript is written as a realist review, trying to explore what works and for whom. The approach gives the authors a large amount of freedom how to analyze and present the findings.

A lot of words and phrases are a new to me. As a reader I would be help with a figure or “fact box” explaining all interventions available, and which are included in the analysis and together with a short example – to but everything in a context. As of now I have difficulty finding out what for example debunking really means, as some interventions are mentioned in the introduction and some are not.

From what I understand, even the definition of the word misinformation is controversial, and perhaps the author’s definition should be stated in the manuscript.

I would also appreciate a discussion of how the research concerning covid-19 misinformation differ from other misinformation research.

The references are used in an order I don’t often se, with references in the middle of a sentence and in some places an appropriate reference is missing (line 373 for example). I recommend looking over, and perhaps expanding the references.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean? ). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy .

Reviewer #1: No

Reviewer #2: Yes:  Tinkhani Mbichila

Reviewer #3: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/ . PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org . Please note that Supporting Information files do not need this step.

Interventions for combating COVID-19 misinformation: a systematic realist review

PONE-D-24-46282R1

Dear Mr. Dickinson,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager®  and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Osmond Ekwebelem

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments: