Transfer Alert

This paper was transferred from another journal. As a result, its full editorial history (including decision letters, peer reviews and author responses) may not be present.

PONE-D-19-30259

Impact of the extension of a performance-based financing scheme to nutrition services in Burundi on malnutrition prevention and management among children below five: a cluster-randomized control trial

PLOS ONE

Dear Dr. Korachais,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

We would appreciate receiving your revised manuscript by May 22 2020 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Seth Adu-Afarwuah

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

http://www.journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and http://www.journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. In your Methods section, please provide additional information about the participant recruitment method and the demographic details of your participants. Please ensure you have provided sufficient details to replicate the analyses such as: a) the recruitment date range (month and year), b) a description of any inclusion/exclusion criteria that were applied to participant recruitment, c) a table of relevant demographic details, d) a statement as to whether your sample can be considered representative of a larger population, e) a description of how participants were recruited, and f) descriptions of where participants were recruited and where the research took place.

3. Please correct your reference to "p=0.000" to "p<0.001" or as similarly appropriate, as p values cannot equal zero.

4. Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified (1) whether consent was informed and (2) what type you obtained (for instance, written or verbal). If your study included minors, state whether you obtained consent from parents or guardians. If the need for consent was waived by the ethics committee, please include this information.

5. Thank you for stating the following in the Competing Interests section:

"CK, SN and MN declare no conflict of interest.

BM has contributed to the emergence of PBF as a global health policy, through technical assistance, research and knowledge management. He is the lead facilitator of the PBF Community of Practice. He holds minority shares in Blue Square, a firm developing health system software solutions. "

Please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials, by including the following statement: "This does not alter our adherence to  PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests).  If there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

Please include your updated Competing Interests statement in your cover letter; we will change the online submission form on your behalf.

Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: http://journals.plos.org/plosone/s/competing-interests

6. Please remove your figures from within your manuscript file, leaving only the individual TIFF/EPS image files, uploaded separately.  These will be automatically included in the reviewers’ PDF.

7. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

Reviewer #3: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: I Don't Know

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This paper reported an important national study that aimed to address two research questions: (1) did the extension of performance-based financing to nutrition (PBF-N) scheme contributed to improve nutritional health services for children below five years old at HC level and; (2) did it reduce the prevalence of malnutrition among children below two years old at community level? Of the 193 eligible health centres providing nutrition services in Burundi, 100 HCs were randomised and 90 were selected to the study. Cross-sectional household surveys were conducted separately at baseline and at end-line about two years after the PBF-N intervention has started. In addition, health centre surveys were performed at both visits and routine data were obtained from the PBF-N program. The primary outcomes were the prevalence of chronic and acute malnutrition among children aged 6-23 months measured at the community level, and the MAM/SAM recovery rates among children below five years old at the HC level.

The study has followed a cluster-randomised controlled trial design, with half of the 90 selected HCs randomly allocated to intervention and the other half constituted the control group. Data were collected at both cluster and individual levels using two cross-sectional surveys. The methods were well documented; however, I have major concerns on the overall trial design, statistical analysis and results presented in this paper.

The CONSORT flow chart suggested that 193 HCs were assessed for eligibility, 100 were randomised, and 90 were selected and enrolled to the study. The authors did not explain why only half of the HCs were randomised, and the discrepancy between 100 and 90 HCs that were randomised. In principle, only eligible clusters should be randomised and included in final analysis.

The sample size of the household surveys was based on a reduction of 25% in acute malnutrition prevalence between HCs in the two groups. The number of children recruited in each HC was set to 72, from 12 sub-hills per cluster and 6 eligible households per sub-hill. This adds up to a total of 6,480 children aged 6-23 months surveyed at the community level. Was the same target used for baseline survey, which as reported included a total of 6,199 children?

For a cluster randomised trial, sample size calculation should be based on both the number of clusters and average cluster size with pre-specified statistical power (>80%) and significance level (normally 5%). Another important parameter to consider is the inflated design effect related to the intra-cluster correlation coefficient (ICC), which is the proportion of the total variance of the outcome that can be explained by the variation between clusters. The flow chart should also include information on both clusters and individuals, as recommended by the CONSORT 2010 statement with extension to cluster randomised trials.

For outcome analysis, the statistical model has included both time points while only the end-line survey data were potentially impacted by the intervention. The beta coefficients estimated the difference-in-difference, which would be hard to quantify the actual effect sizes between intervention and control groups on the defined outcomes. A standard approach is to analyse the end-line survey data only using the mixed models with a random cluster effect, with the assumption that the clusters were similar at baseline by randomisation and matching.

As required by the CONSORT guidelines, baseline characteristics at cluster and individual participant levels should be presented in a table for each group. Continuous variables can be summarised by the mean and standard deviation, and numbers and proportions should be reported for categorical variables. The supplementary table 1 needs to be modified following the guidelines. The outcome tables are hard to follow, and the numbers are inconsistent. For example, how were the total numbers of observation 1,254 and 1,549 obtained in Table 2? The column titles Mean and N were used for all outcomes regardless of their distributions. With an interaction term between group and period in the main model, which term was used to estimate the effects of intervention on different outcomes should be clearly stated so that the estimated coefficients, odds ratios, and incidence rate ratios can be interpreted appropriately. With more than one primary outcome and a large number of variables tested overall (386), any significant differences observed between intervention and control groups should be concluded with caution without controlling for overall Type 1 error rate.

Reviewer #2: 1. Abstract. The word “data” is plural so it should be “data were”.

2. p. 9. The last paragraph purports to explain the theory of change of the intervention. It does not do so. Having this explanation is essential especially for understanding why the authors expected health system improvements to have reduced chronic or acute malnutrition prevalence, an expectation that on the face of it does not seem warranted because health systems cannot remove most of the causes of the malnutrition (as the authors note in the Discussion). What the authors have provided here is useless for explaining the theory of change. The authors have basically said to the reader to go find out for yourself by going to the supplementary material, which is irritating and not acceptable. S2-S5 appear to have two files duplicated twice. The protocol is in French; PLoS One is an English language publication. Please revise to explain clearly in the main text of the manuscript the theory of change of the intervention for both recovery from and prevention of malnutrition.

3. p. 14, top. The authors write “We talk about shortness when LAZ is lower than minus two standard deviations (SD) and stunting when it’s lower than minus three SDs.” WHO (and just about everyone else) defines stunting as less than -2 SD, not -3 SD. Please revise and use the standard definitions to avoid confusion.

4. p.15. Where it says “The time varying variable PBFnutht is an an interaction term…” delete “time varying”. This term is not time varying in the common use of this descriptor.

5. p. 22. P-values, by definition, cannot be zero. All p-values given as 0.000 should be given as <0.001.

6. The manuscript requires careful technical editing and preparation. Overall, the writing is understandable, but many technical problems are throughout the manuscript. Furthermore, the manuscript was not carefully prepared, for example, line numbers in only part of the manuscript.

Reviewer #3: • General comments

This is a quality paper on an important nutrition issue using state-of-the art methodology (RCT) with particularly careful pairing of control and intervention HCs. It has often been assumed that health personnel and workers did not have enough incentives to devote more effort to nutrition services and therefore, it was hypothesized that performance-based monetary incentives would boost the nutrition performance and impact. There are some issues, however minor, that need to be addressed.

• Specific comments

1. Abstract

The objectives should be given. The results section is not very clear : 97% - 78% = 19%, not 14.7pp (what is “pp”?). Unless this is not the way the results should read.

2. Introduction :

a. Hunger and food insecurity are among the root causes of malnutrition. Malnutrition encompasses several forms including undernutrition. In addition to poverty and food insecurity, there are factors of child undernutrition which are in the health realm, beyond nutrition care and services, and management of undernutrition, that is, poor access to health care as well as WASH. This could be clearer in the paper.

b. Community-base management of undernutrition (and prevention) is now advocated, although it is clear that health personnel are involved, at community, as well as HC and hospital level. The expected nutrition activities and services at these three levels should be described, perhaps in general and in the Burundi context.

3. Methods

a. How many health centers in total for 200 providing nutrition services?

b. The rationale for the different age range in the health center and the community assessment should be given.

c. Fig. 1: The unit is the health center, please specify

d. Institut national de santé publiQUE

e. Who gets the subsidy and how is it shared in the case of hospitals, HCs and CHWs?

f. What is the MAM treatment supply? It should be explicit as it is often referred to.

g. Shortness and stunting: are these widely accepted cut-offs and terms? Above the age of 24 months, it should be stated that standing height is measured, not length.

h. Please clarify that the duration of the intervention was two years.

i. Is it possible to provide the list of the 19 indicators of quality of nutrition services, perhaps as supplementary data?

j. According to Table 2, CHWs are to screen and refer acute malnutrition cases to HCs. However, is this not accomplished through growth monitoring and promotion activities, which are HC’s activities?

k. The protocol is not needed even in appendix; besides, it is in French.

4. Results

a. On p. 18, there is a distinction of the intervention at hospital, HC, and community level. This is why the specific activities expected at these three levels have to be described in the methods section.

b. Fig. 3 is not very useful.

c. What about the % of defaulters in MAM management, in intervention vs control HCs?

d. It has to be clear that the improved SAM management performance only refers to uncomplicated SAM treatment outside the hospital since control hospitals also get PBF for the management of SAM cases with complications.

e. In Table 2, what is the meaning of **?

f. What about infant and child feeding practices in the community survey? It is referred to it in the discussion (p. 32, line 29) but the results are not shown.

5. Discussion and conclusion

a. Training of health personnel (although recent training apparently had no impact) and supervision of CHWs would appear to be major conditions of performance that the PBF strategy did not address. This should be discussed in more depth.

b. In the recommendations, the need for health centre staff training and CHW supervision should be emphasized, along with having personnel specifically for health promotion.

c. Is the fact the control hospitals and HCs also received subsidies not a limitation of the study, although this was intended to clearly identify the impact of the monetary incentives’?

d. The short duration of the intervention (2 years) may not have allowed to show an impact on stunting; this can be regarded as a limitation of the study.

e. P. 31, line 19: reference needed

f. P. 31, line 21: What equipment?

g. P. 32, line 27: also failures in WASH, not only food security

h. P. 32, line 40: “CHW strategy got a boost from PBF”: Please explain

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

PONE-D-19-30259R1

Impact of the extension of a performance-based financing scheme to nutrition services in Burundi on malnutrition prevention and management among children below five: a cluster-randomized control trial

PLOS ONE

Dear Dr. Korachais,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Sep 11 2020 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Seth Adu-Afarwuah

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #3: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: (No Response)

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: (No Response)

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: (No Response)

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: (No Response)

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: (No Response)

Reviewer #3: Only two minor points:

1. In the abstract, HC should be defined. Perhaps the conclusion should be clearer that performance-based financing had not substantive effect on the prevention of malnutrition and therefore alternative or additional strategies are required.

2. If the protocol is already published [17], one wonders why it is provided in supplementary files (S3-S6).

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #3: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Impact of the extension of a performance-based financing scheme to nutrition services in Burundi on malnutrition prevention and management among children below five: a cluster-randomized control trial

PONE-D-19-30259R2

Dear Dr. Korachais,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Seth Adu-Afarwuah

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #3: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #3: (No Response)

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #3: (No Response)

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #3: (No Response)

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #3: (No Response)

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #3: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #3: No