Response to reviewers:

Part 1: Editor comments and responses

1. Thank you for including your competing interests statement;

"The authors have declared no competing interests exist."

We note that one or more of the authors are employed by a commercial company: Save the Children Malawi, World Health Organization, Ministry of Health, The Joint TB/HIV Global Fund -ActionAid, and Partnership for Human Development -ABT Associates.

1. Please provide an amended Funding Statement declaring this commercial affiliation, as well as a statement regarding the Role of Funders in your study. If the funding organization did not play a role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript and only provided financial support in the form of authors' salaries and/or research materials, please review your statements relating to the author contributions, and ensure you have specifically and accurately indicated the role(s) that these authors had in your study. You can update author roles in the Author Contributions section of the online submission form.

Please also include the following statement within your amended Funding Statement.

“The funder provided support in the form of salaries for authors [insert relevant initials], but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section.”

If your commercial affiliation did play a role in your study, please state and explain this role within your updated Funding Statement.

2. Please also provide an updated Competing Interests Statement declaring this commercial affiliation along with any other relevant declarations relating to employment, consultancy, patents, products in development, or marketed products, etc.

Within your Competing Interests Statement, please confirm that this commercial affiliation does not alter your adherence to all PLOS ONE policies on sharing data and materials by including the following statement: "This does not alter our adherence to PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests) . If this adherence statement is not accurate and there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

Please include both an updated Funding Statement and Competing Interests Statement in your cover letter. We will change the online submission form on your behalf.

Response: Regarding the revised competing interest statement – we would like to clarify that Save the Children and ActionAid are not commercial companies– rather not-for-profit organizations; World Health Organization (WHO) is a UN affiliate organization; and Malawi Ministry of Health (MoH) is a government entity. We have included in the statement that some authors worked for these entities as requested, while also indicating that no competing interests exist.

We have amended the funding statement and competing interests statement in the cover letter as requested. Please note that since this paper was submitted I have changed positions and no longer work with Abt Associates in Timor-Leste. I currently work for Unitaid at the WHO and have updated my affiliation accordingly. Whereas at Save the Children I received small part of my salary for working on the paper as indicated in the revised funding statement, while at Abt Associates and Unitaid I worked on the paper in my personal time and received no further compensation.

Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: http://journals.plos.org/plosone/s/competing-interests

Additional Editor Comments:

This is a well done study about an important implementation topic. As an implementation study, the detail about the process of introducing the study is appreciated, as are the efforts to understand the context and challenges with uptake of a guideline.

The most important issue to address is that as Ntcheu was selected because of ongoing complementary programs, the authors must address the potential for BIAS, as Ntcheu may be quite different than other sites and uptake potentially much higher. Further, the implementation of district based teams, quality assurance efforts, and extension of the user-fee waiver, sounds very intensive, and the authors should address the feasibility that this could be replicated in a program setting.

Response: Thank you for raising this important point. The Community Based Maternal and Newborn Care (CBMNC) program was developed in Malawi in 2007 and many Health Surveillance Assistants (HSAs) across the country had been trained in the package since that time, although coverage of the pregnancy and postnatal home visits was low. In 2014, WHO and the MoH collaborated to update the CBMNC package to reflect the WHO recommendations on pregnancy and post-natal home visits and introduced the revised version in Ntcheu district, one of the 8 districts funded through the RAcE program – we have clarified this in the methods section (TC version, page 6, lines 127-131) and also modified the discussion section accordingly (TC version, page 24, lines 514-519). Regarding feasibility under normal program conditions, we have also further expanded the discussion section to address some of these issues (TC version, page 24, lines 508-525)

In addition to the review comments, can you please also address the following:

-In the abstract, can you please include if the finding that 3.4% of young infants clinically failing treatment is similar or different to the PSBI efficacy trial?

Response: Abstract revised to indicate that the treatment failure rate was less than those reported for the simplified regimens tested in the SATT (8-10%) and AFRINEST trials (5-8%) (TC version, page 2, lines 55-57). We also included reference to this in the discussion section (TC version, page 22, lines 455-457).

-The authors state that sepsis treatment in children is a large cost to the financial system; shouldn't this fall under the most basic primary care and not be subject to cost efficiency measures?

Response: Hospital treatment of sepsis, particularly complicated cases, can be costly as cases are admitted for inpatient treatment for multiple days. Hospital treatment is also expensive for most families, who may live far away and incur additional costs related to the treatment, such as transport, lodging, food, etc. Identifying cases earlier and managing them at outpatient level has potential to save costs for government-funded health systems and for families.

-Acceptability and uptake are not the same; the former should be measured through qualitative methods

Response: We agree that acceptability and uptake are not the same. As part of the qualitative component of the study we interviewed a small sample of mothers whose children had received outpatient treatment to better understand their perceptions of the treatment. We have modified the discussion to better distinguish (TC version, page 21, line 51)

-please define terms such as "ease of access" and "referral not feasible"

Referral not feasible refers to when families are unable to or unwilling to undertake a referral and we have added this definition to the introduction (TC version, page 4, lines 76-80). Ease of access referred to the proximity of health facilities to facilitate follow-up – we have revised the text in the methods section to clarify (TC version, page 7, line 165).

-Please add page numbers –

Response: added page numbers.

[Note: HTML markup is below. Please do not edit.]

Part 2: Reviewers comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This is a well written clear manuscript. The only section that could do with additional explanation is the 'secret mothers'. please add a few more sentences regarding their role within the context of PSBI. Are they widely used, already in existence? how do you know that they are effective?

Response: Thank you for your suggestion. Secret mothers are older women selected by their communities to serve as a link between women and the health care system, particularly during pregnancy and the postnatal period. The concept of secret mothers was started by the MoH in 2012 as part of the Safe Motherhood Initiative and secret mothers are operational in 13 districts of Malawi, including Ntcheu. Secret mothers were already been quite well established in Ntcheu district at the time of the PSBI study in 2017 and we were able to train existing secret mothers. However, to our knowledge their effectiveness has not been studied (we could find no published studies documenting this). We have included additional information on the role of secret mothers in the methods section and also a citation describing secret mothers to direct readers for further details (TC version, page 11, lines 240-245).

Reviewer #2: Dear Editors,

Thank you for inviting me to review this manuscript. it was my privilege to do so.

Overall, this is a superbly written manuscript. It is thorough and thoughtful, and the conclusions reached and next steps pragmatic and not-overreaching. A few minor thoughts/suggestions/questions.

1. It would be interesting to understand the reasons given by the parents who refused referral, in this setting. While there is some literature around this, for Malawi, and in particular for Ntcheu, it would be illustrative to understand these reasons. Such reasons may focus future policy and programmatic efforts.

Response: Thank you for this comment – as part of the qualitative component we asked a small number of mothers why they refused referral and the most common reasons cited were lack of transport to the referral hospital, the need to look after other children and the high costs associated with receiving inpatient treatment at the hospital. We also asked health providers why families refused referral. They also cited the same underlying issues (transport, financial costs, other responsibilities and preference to be treated closer to home). We have included some of the common reasons for refusal of referral in the introduction (TC version, page 4, lines 76-80).

2. Of the nine government and three CHAM facilities, were the results and follow-up similar? While statistical analysis may be precluded here due to small sizes, not infrequently there are differences in health care provision between CHAM and government facilities, and it will be interesting to examine whether follow-up and outcomes are comparable across both settings.

Response: Thank you for this – as noted the sample size was quite small – we have reviewed the treatment completion and follow-up rates between CHAM and government health facilities and the levels were similarly high in both types of facilities. For the research, participating CHAM facilities expanded free treatment to infants from up to six weeks to up to 8 weeks (aged 0-59 days). This would be an important area to track though as implementation expands outside of a research setting and we have noted this in the discussion (TC version, page 24, lines 521-525).

3. The incentives given to health facilities and HSA appeared to preclude the "secret mothers". How much of a role did the secret mothers play in this program? Might follow-up rates have been higher with their active engagement? Or incentivization?

Response: Incentives were only given to the formal providers in the system and primarily to facilitate communication between facility providers, HSAs and study staff (air time) and offset the additional burden of data collection due to the study forms that would not be present under normal conditions. Secret mothers were not required to collect any data. Based on information provided during the qualitative interviews, the contribution of secret mothers to the program was mixed, with some HSAs reporting they were helpful in linking them with families, while others indicated they were able to connect directly without secret mothers. As such, it is not clear whether follow-up rates could have been higher with incentivization of secret mothers. However, scale up of the program beyond the research will not involve incentives at any level of the system and that some districts will not have secret mothers in place. The possible role of secret mothers could be an area for further exploration and we have noted this is the discussion (TC version, page 23 lines 490-492).

4. While it was good to understand in absolute terms how much these incentives were, it would be helpful to the reader to have from a health systems/health economics standpoint some reference points so as to understand whether these incentives are truly sustainable by the Ministry of Health. For e.g. how much money is spent on health per person in that district?

Response: As noted above, monetary incentives were provided to health facility staff and HSAs only for study purposes to offset the additional communication costs and data collection burden. It is not anticipated that the MoH will continue to provide these incentives – we have noted this in the limitations (TC version, page 24, lines 510-511).

5. Given what seems like a successful program, is there a line of sight to line-item budget to continue or expand on this program? Has there been policy change yet?

Response: As mentioned in the discussion, the MoH has started to expand this approach and has developed costed plans for the roll-out and worked with funding and implementation partners to complete the roll-out in Ntcheu district and train providers in 11 additional districts. The MoH reports policy change on PSBI to the IMCI approach policy, and the MoH has updated guidelines for the management of sick young infants and children. The IMCI chart booklet was also updated from the generic WHO guidelines on cough, fast breathing and chest in-drawing pneumonia. Further, the Government has committed funds towards PSBI roll out to cover training and orientation of HSAs to ensure PSBI identification and follow up. We have updated the discussion to reflect this (TC version, page 25, lines 530-533).

6. Qualitative interviews - how were these chosen? Were they purposive? Was saturation reached with responses? More detailed methods in this section would make this well-written paper even more robust.

Response: Respondents for qualitative interviews were chosen purposively as described in the methods section. While we did not explicitly define the sample size in terms of saturation, we did find that respondents provided similar answers with regard to our main lines of inquiry, suggesting that data saturation had been reached even with a relatively small sample. We have modified the limitations section (TC version, page 24-25, lines 525-528).

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Charlotte E Warren

Reviewer #2: Yes: Cyril M Engmann, PATH & University of Washington

Attachments

Attachment

Submitted filename: response_to_reviewers_Jan 8.docx