Transfer Alert

This paper was transferred from another journal. As a result, its full editorial history (including decision letters, peer reviews and author responses) may not be present.

PONE-D-23-02103Effectiveness of a chatbot in improving the mental wellbeing of health workers in Malawi during the COVID-19 pandemic: A randomized, controlled trialPLOS ONE

Dear Dr. Kleinau,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

I deeply regret the unusual amount of time that has been necessary to ensure the appropriate revisions of this manuscript. While the study is intriguing, some revisions are necessary for clarity and robustness. Please enhance descriptions of research methodology, ensuring transparent reporting of allocation concealment and specifying the currency for financial figures. Clarify key elements in the statistical analysis, including outcome variable details and the rationale behind significance level choices. Ensure consistency and coherence in presenting figures, considering their placement in tables and addressing discrepancies in resilience change reporting. Additionally, respond to reviewer queries on the UCLA Loneliness scale, coding decisions for missing data, and justification for statistical significance thresholds. Attend to formatting errors and typos throughout the manuscript. Lastly, provide essential information, such as ethics committee protocol details, ISRCTN registry number, strategies for optimizing response rates, and insights into minimizing technical difficulties. I look forward to reviewing the updated version.

Please submit your revised manuscript by Jan 20 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

César González-Blanch, PhD

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf.

2. Thank you for stating in your Funding Statement:

‘Financial support for this RCT was provided by the United States Agency for International Development under the Cooperative Agreement No. AID-OAA-A-15-00046

https://www.usaid.gov

The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.”

Please provide an amended statement that declares *all* the funding or sources of support (whether external or internal to your organization) received during this study, as detailed online in our guide for authors at http://journals.plos.org/plosone/s/submit-now.  Please also include the statement “There was no additional external funding received for this study.” in your updated Funding Statement.

Please include your amended Funding Statement within your cover letter. We will change the online submission form on your behalf.

3. We note that you have stated that you will provide repository information for your data at acceptance. Should your manuscript be accepted for publication, we will hold it until you provide the relevant accession numbers or DOIs necessary to access your data. If you wish to make changes to your Data Availability statement, please describe these changes in your cover letter and we will update your Data Availability statement to reflect the information you provide.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: I Don't Know

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The study is quite interesting where it investigates whether the interactive chatbot, Vitalk, is more effective than the passive use of internet resources in improving mental wellbeing and resilience outcomes of health workers in Malawi.

However, the manuscript could be further improved.

Comments

Line 115, Line 206, the description of allocation concealment is to be mentioned.

Line 165, Line 168, for $10, the currency involved e,g in USD is to be clearly stated.

Line 167, for ‘web resources At the startup’ small a for At

Line 195, the outcome variable, 1 or 2-tailed test used in the sample size calculation, what was the ‘difference’ refers to is to be stated.

Line 365, the sentence ‘days every week; and I did not work at all (coded as missing)’ not clear.

Information on missing data (if any) e.g. percentage, and type of missing data is to be stated.

Line 381, difference-in-differences (DiD) estimator is to be defined.

Line 413, p ≤ 0.05 is preferable to p≤ 0.10 as indicative of statistical significance. Consideration of low P values (e.g., P<0.10) as trending toward statistical significance may be clinically relevant for improving practice, particularly in smaller studies.

Line 450, to state around 29.

Line 454, to state around 8.

Line 458, what ‘except Age’ refers to not clear.

Line 462, comma for figures to be replaced with dot

Line 462-472, figures to be presented in table.

Line 481, typo 11.4 (should be 11.04)

Line 462-472, median and range values are to be omitted since mean ± sd is used.

Line 554, resilience change is not the same for both groups.

Line 594, the statement is incorrect.

Line 599-600, Line 636 -637, Line 639-640, the data to be presented in table form.

Line 612, surgery (depression -1.55 points)

Line 613, special care (depression -2.05)

Table 4, median score to be omitted. Frequency to be replaced with ‘n’

Line 526, the sentence is unclear.

Line 543, either mean or median scores are to be reported.

The percentage figures in text are to be reported at least 1 decimal point.

Table 6, statistical test to be denoted in the table footnote.

Line 602, 659, symbol <= to be replaced with symbol ≤

Line 656, Line 659 the significant level 0.05 to be used.

Ensure all figures cited in the text are to be cited exactly from table(s).

95%CI figures presented in the text could be placed in the table instead.

Reviewer #2: Thank you for the paper. I found the study very interesting.

The need for such as tool is extremely high in Malawi given the very limited access to psychiatric care.. Evaluation of the value of chatbots in low-resource settings and specifically for helping health worker mental health is extremely important. I am pleased to see such a trial having been conducted and even more pleased that the results show the effectiveness of such an intervention.

The recruitment of over 1500 health workers and the retention rates are great achievements. The study has been well-designed, analysed and written up. The study conducted is well-designed and gathers data over an extended period (8 weeks) with mid-study data collection. A good number of participants were included.

Regardless of the intervention and its efficacy, the results that analyse, for example depression, anxiety and burnout, for different cadres is interesting information that may not be available otherwise.

Limitations are acknowledged.

Questions and further specific comments appear below.

It is great that the study provided funding to provide data package and transportation costs. I would hope that Vitalk could be deployed without disadvantage or bias due to cost of data packages. How much technical or other support was needed for participants to use the tool on their phones? I would hope that financial, technical and other support could be provided in the future from other sources to ensure the tool is able to be useful.

Do the authors believe the tool will continue to be useful in this COVID-normal and post-COVID period? What challenges and possible solutions are envisaged for the future maintenance, scalability, deployment and ongoing provision of Vitalk?

It was good to hear that there was a risk management process in terms of recruitment exclusion criteria and triggering via PHQ-9 responses. Also clarifying the chatbot does not seek to replace a human expert and help should be sought if needed.

I think removal of NLU is a wise move to avoid errors which may occur and increase the risk of harmful or inppropriate chatbot responses.

Viki is in Figure 3 not in Figure 2.

What was the motivation for including the UCLA Lonliness scale?

Why was 'I did not work at all' coded as missing?

The sample size used significance level of .05. Why was P ≤ 0.10 used for statistical significance? This is unusual. Most commonly p<=.05 is used.

Do you have any thoughts on the differences in numbers between the two groups at different study periods. The addition of percentages of the total might be useful to aid the comparisons. Why did the control group have fewer participants at midline and post-endline and more at baseline and endline?

Only participant who completed -->

Only participants who completed

I am surprised that almost all participants did not report that the pandemic had disrupted their work.

p. 23 Why wasn't mood status for the control group measured?

Control group is lower on baselines, what is the possible impact on interpretation of other results of these sig diffs. This means it would have been harder to show a difference following treatment.

So there is a significant difference for control as well, but no significant difference between groups at endline. This does not support that the intervention was useful.

Resilence is the exception.

p.24 deceases in depression -->

decreases in depression

p. 25 printing /formatting error at line 504.

Only DiD estimators seem to show statistically significant differences. Why is this the case, when t-tests do not show significant differences?

p.30 "None of the 95% CI contain zero, indicating that both study groups experienced a positive effect"

How do you account for this? Were the resources potentially useful, though they were not accessed very much. Or is being alerted to how you are feeling, via doing the baseline enough to help you feel better or feel that someone else cares to ask?

p.31 "very good (greater and 0.80)"

--> "very good (greater than 0.80)"

p. 31 "did not impart whether these were of clinical relevance."

impart??

Reviewer #3: This is an interesting paper reporting on the first RCT of a mental health app for healthcare workers during the COVID19 pandemic in Southern Africa combining multiple mental wellbeing outcomes, and measuring resilience and resilience-building activities. It is a well formulated and reported study which is of relevance to the mental health literature relating to the COVID-19 pandemic.

Reviewer #4: The writing is clear and straightforward. The author presents the material in an organized fashion, and the figures are well-chosen. However, I have some comments.

1. The University Research Co. (URC) Institutional Review Board and the University of Malawi Research Ethics Committee (UNIMAREC) approved the study protocol – Please kindly provide the protocol number given by ethics committee.

2. Trial was registered retrospectively with the ISRCTN registry-Please kindly provide this number.

3. How was missing data handled?

4. Strategies used to optimize the response rate were not mentioned.

5. The method of questionnaire administration was not specified.

6. There was no additional details regarding an incentive for questionnaire completion provided?

7. Response rate was not defined and also it was not reported. What was done with incomplete surveys if any?

8. Demographic data of the survey respondents was not clear. Though 8 categories of health care professionals was mentioned yet the figure only shows nurses.

9. Were the questionnaires pilot tested?

10. How can technical difficulties minimized for others who plan similar studies as there was a high attrition rate in the Treatment group due to this?

11. All the questionnaires used should be provided as appendices.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: Yes: Dr Sarah Markham

Reviewer #4: Yes: Khizra Sultana

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Attachments

Attachment

Submitted filename: Plos one clinical trial review.docx

Effectiveness of a chatbot in improving the mental wellbeing of health workers in Malawi during the COVID-19 pandemic: A randomized, controlled trial

PONE-D-23-02103R1

Dear Dr. Kleinau,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at http://www.editorialmanager.com/pone/ and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

César González-Blanch, PhD

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

I sincerely appreciate the effort you've put into addressing the concerns and suggestions raised by the reviewers. The feedback provided by the reviewers underscores the importance and relevance of the study, as well as the robustness of its design and analysis. Significant revisions have been made to the manuscript to address the raised concerns, including improvements in data presentation, clarity of writing, and justification of methodological decisions. The detailed response to the comments demonstrates a clear commitment to the quality and rigor of the research.

Reviewers' comments: