PONE-D-24-15369Enhancing Childhood Immunization Coverage in Mozambique and Malawi: Study Protocol of a mixed methods evaluation of the Let's Talk About Vaccines Multisite Community-Based Participatory ProjectPLOS ONE

Dear Dr. Shuro,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Aug 02 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Edison Arwanire Mworozi, M.D

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please provide a complete Data Availability Statement in the submission form, ensuring you include all necessary access information or a reason for why you are unable to make your data freely accessible. If your research concerns only data provided within your submission, please write "All data are in the manuscript and/or supporting information files" as your Data Availability Statement.

3. Your ethics statement should only appear in the Methods section of your manuscript. If your ethics statement is written in any section besides the Methods, please move it to the Methods section and delete it from any other section. Please ensure that your ethics statement is included in your manuscript, as the ethics statement entered into the online submission form will not be published alongside your manuscript. 

4. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information. 

5. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments:

Please revise this document as per reviewers comments!

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Does the manuscript provide a valid rationale for the proposed study, with clearly identified and justified research questions?

The research question outlined is expected to address a valid academic problem or topic and contribute to the base of knowledge in the field.

Reviewer #1: Partly

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

Reviewer #5: Yes

Reviewer #6: Partly

Reviewer #7: Partly

********** 

2. Is the protocol technically sound and planned in a manner that will lead to a meaningful outcome and allow testing the stated hypotheses?

The manuscript should describe the methods in sufficient detail to prevent undisclosed flexibility in the experimental procedure or analysis pipeline, including sufficient outcome-neutral conditions (e.g. necessary controls, absence of floor or ceiling effects) to test the proposed hypotheses and a statistical power analysis where applicable. As there may be aspects of the methodology and analysis which can only be refined once the work is undertaken, authors should outline potential assumptions and explicitly describe what aspects of the proposed analyses, if any, are exploratory.

Reviewer #1: Partly

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

Reviewer #5: Partly

Reviewer #6: Partly

Reviewer #7: Partly

********** 

3. Is the methodology feasible and described in sufficient detail to allow the work to be replicable?

Descriptions of methods and materials in the protocol should be reported in sufficient detail for another researcher to reproduce all experiments and analyses. The protocol should describe the appropriate controls, sample size calculations, and replication needed to ensure that the data are robust and reproducible.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

Reviewer #5: No

Reviewer #6: Yes

Reviewer #7: No

********** 

4. Have the authors described where all data underlying the findings will be made available when the study is complete?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception, at the time of publication. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

Reviewer #5: Yes

Reviewer #6: Yes

Reviewer #7: No

********** 

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

Reviewer #5: Yes

Reviewer #6: Yes

Reviewer #7: Yes

********** 

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above and, if applicable, provide comments about issues authors must address before this protocol can be accepted for publication. You may also include additional comments for the author, including concerns about research or publication ethics.

You may also provide optional suggestions and comments to authors that they might find helpful in planning their study.

(Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: General comments

Vaccinations remain a major advancement in global public health, despite challenges for these live-saving vaccines to reach a majority of children at need in hard-to-reach zones. It is important to understand community-level drivers on vaccine uptake needed to support health decision making about prioritization and cost of these interventions in one’s own setting.

This study protocol aims to evaluate the effectiveness (on under-two immunization coverage) and the extent of implementation (reach, adoption, implementation and maintenance) of the Project in Mozambique and Malawi.

Authors plan to integrate the theory of change approach with the RE-AIM framework, and hope findings will potentially provide evidence on mechanisms that explain observed improvements in under two immunization outcomes and toward resilient health systems as a result of the co-created community driven solution(s).

The paper will bring important addition to existing literature, especially from an area where data paucity remains a perennial issue. However, this reviewer finds that there are gaps to address in the current version of the paper before it is considered for publication.

SPECIFIC COMMENTS

Title: No need to capitalize each word, only names of places and persons here except that falls within the journal’s specifications.

- The manuscript lacks line numbering for easy reading and review. Also, there is need for authors to format the entire work appropriately.

ABSTRACT

- CBPR: Please after first use of an abbreviation, it should be used thereafter. No need spelling out in full.

- “There is a few example of evaluations of intervention created through human-centred design.” Authors can integrate this in the next sentence i.e., “… this approach...” or take it to the introduction.

INTRODUCTION

- “It is an approach that centres on creativity in an interactive process to bring human centred views … “Here, I think it gives a wrong impression that previous interventions have not been human-centred. This should be revised to community-directed/driven/centred.

- “… by evaluating the effectiveness and implementation of the project.” How will these be performed?

- Ensure consistency in reporting (e.g., RE-AIM or RE AIM)

- In figures/ Tables, authors should ensure that each abbreviations should be fully defined as footnotes if not within the figure (s)/Table(s) e.g., what is TOC, VR, HCD?

- DPT1-Penta and all abbreviations should be fully defined at first use before subsequent uses.

- Could the authors describe how data quality will be assessed considering the porous nature of data in resource-low settings?

- No need to put “long-term impacts” in bold!

- Figure 2 is very illustrative and it’s great! Except that some of the arrows are out of place or non-directional. Please, verify and adjust!

- What’s the rationale of targeting under-2 populations? Would this be sufficient to assess the long-term impacts?

- Ethics statement should only appear in the text (not abstract, except the journal demands that).

DISCUSSION/Limitation

- Incomplete records are a perennial problem in resource-low setting, how do the authors plan to ascertain data quality with this huge challenge?

- Additionally, authors need to explain clearly how they plan to overcome the stated limitation so as not to jeopardize potential findings.

REFERENCES

- Some are wrongly captured and should be verified at the level of the reference manager before uploading, so that articles should be referenced appropriately.

Reviewer #2: The standardization of the caretaker researchers and the control sites is missing. There is also need to start the criteria for the purposive sampling at the midline assessment. Study data quality control and assurance measures are wanting.

Reviewer #3: The manuscript outlines a detailed and systematic protocol for assessing the effectiveness and implementation of the "Let’s Talk About Vaccines" project. Drawing from my extensive experience in global health security and epidemic intelligence, I find the use of CBPR and HCD methods both innovative and highly appropriate for addressing the critical issue of low childhood immunization rates in Mozambique and Malawi. The methodology is robust and well-documented, ensuring that the study can be replicated and will yield valuable insights.

Here are a few minor recommendations for improvement:

1. Continuously review and update ethical considerations throughout the study, particularly concerning participant privacy and data sharing agreements, reflecting the dynamic nature of public health research.

2. Provide more specific examples of successful applications of the HCD approach in similar contexts to better justify its use in this study. My work with various health security initiatives has shown the importance of such context-specific illustrations.

3. Address potential limitations related to the interrupted time series analysis, especially concerning the quality and completeness of routine immunization data. Drawing from my experience in managing large-scale health data projects, ensuring data integrity is crucial for accurate outcomes.

Overall, the manuscript is comprehensive and well-prepared, setting a solid foundation for subsequent stages of the review process.

Reviewer #4: Background:Add more context:

1. Definition of vaccine coverage.

2. Highlight the challenges of vaccine coverage.

3. Mention the challenges that the methods under study are aimed to address.

Methods: Study population:

2. Which vaccines are being studied? Are the challenges of vaccine coverage the same for all vaccines?

Results/Dissemination:

Is health education/community dissemination in the plan?

Reviewer #5: Overall, relevant study area. The objectives can be explained more explicitly. The methodology, especially the sampling to be more detailed.

Reviewer #6: The author must explain rationale for the study more clearly, just referring to sub-optimal full immunization coverage and COVID-19 effect does not serve the purpose. It should define if there are some barriers affecting reaching to community, any particular beliefs etc. that hinder vaccination coverage.

Also what proposed intervention the Village Reach team expect, out of the HCD workshops planned? More clarity on possible intervention that may come out must be mentioned here. If this study has been conducted in other countries of LMIC/LIC, it would be useful to quote some examples.

Reviewer #7: Due to the process-oriented nature of the proposed analysis, the protocol as described in the current manuscript is not replicable. Given that replicability is the manuscript's primary objective, its contribution to the existing body of literature remains ambiguous.

A revision would be stronger if the methods were more specific and much more concise. The methodology may not lend itself to such a description. Remove didactic language. While many theories of evaluation are referenced it is unclear how they are all brought together into a cohesive analytic plan.

It is unlikely that the study is powered appropriately to detect significance of impact on a dropout rate of 5–6%. A sample size calculation to see a difference of about 5% coverage usually needs around N=250. You would be looking for a change of 2%. The study would not need a 2-sided test unless there is a reason why the proposed interventions would increase dropout rates? Is the "adequate sample size" for the study as a whole, for each country or each site?

The document does not describe the full timeline of the proposed study the timeline section states "until the end of 2024" when did the study start? How will the midpoint be determined?

Given that the current only baseline and two assessment points, how can you stratify the model by month?

Refine the terminology around assumptions: in the introduction and in the assumptions section you are putting forward hypotheses. Assumptions are caveats and determinations that are not part of the formal study outcomes.

Theory of change diagram: while rightly recognizing the complexity of healthcare utilization and delivery, the diagram and theory should reflect only the essential elements that are being tested or controlled in the study.

While a good amount of the proposed data is theorized to come from the files of Village Reach, it is unclear whether all the proposed data will be available, and more information is needed to provide assurance that access to the data of interest will be available.

********** 

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: Yes: Peter Babigumira Ahabwe

Reviewer #4: No

Reviewer #5: No

Reviewer #6: No

Reviewer #7: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Attachments

Attachment

Submitted filename: PLOS ONE_Linda Shuro et al., 2024_Lets talk about vaccines.docx

Enhancing childhood immunization coverage in Mozambique and Malawi: study protocol of a mixed methods evaluation of the 'Let's talk about vaccines' multisite community-based participatory project

PONE-D-24-15369R1

Dear Linda,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Edison Arwanire Mworozi, M.D

Academic Editor

PLOS ONE

Thank you.

I have no further additions to make towards this paper. 

Good luck and thank you.