Description of study sites for the design of the audit tool.

Pumwani Maternity Hospital (PMH) is one of the County referral hospitals in Kenya and was selected as the site for the design of the SSNB audit tool [22]. This hospital was selected as at the time of the study, the lead researcher (MO) had been a paediatrician in the hospital for approximately one year. It is also the largest referral maternity hospital in Kenya located in Nairobi County with approximately 100 deliveries per day [23]. The NBU admits approximately 350–400 newborns per month and provides intermediate-level care [24]. The hospital held monthly MPDSR meetings to discuss maternal near misses with a brief overview of perinatal morbidity and mortality statistics. In the event of a maternal death, the meeting was held within 48 hours of the death.

Description of study sites for usability testing.

The initial plan was to test the tools in PMH. However, Kenyatta National Hospital (KNH), the largest teaching and referral hospital in Kenya expressed interest in improving its newborn clinical audit process and was incorporated into the study as a testing site in January 2021. KNH caters for low-and-middle-income populations from Nairobi and its environs as well as referrals from other hospitals in the country and the greater Eastern Africa region [25]. The NBU of KNH admits approximately 250–300 newborns per month and offers intensive care to critically ill newborns [24]. The NBU staff do not attend MPDSR meetings, they however infrequently hold in-house meetings to discuss the NBU morbidity and mortality statistics.

Cognitive walkthrough and usability testing of prototype audit tools in the hospitals.

The SSNB audit tool was intended for use by a transdisciplinary team who directly provide care to the SSNB and were therefore involved in its design and testing its usability (Box 2).

Virtual design workshop.

The design workshop of the implementation guide involved 37 purposely selected participants (two neonatologists, 17 NBU nurse leaders, 17 paediatricians and an MoH official) and was held using the Zoom platform.

a). Design of draft zero of the prototype audit tool.

We studied the existing audit tools to inform the structure of draft zero of the SSNB prototype audit tool. These audit tools included: Kenyan MoH MPDSR Tool, WHO Stillbirth and Neonatal Death Case Review Form and WHO Child and Neonatal Death Review Form [2, 3, 10]. We also made reference to the modifiable factors in newborn care identified from a scoping review, Kenyan Basic Paediatric Protocols, Comprehensive Newborn Care Protocols and WHO guidelines that outline standards of care for the SSNB [12, 21, 27–30]. The outcome of this initial step was draft zero of the audit tool which was paper based.

b). Cognitive walkthrough of draft zero and subsequent modified prototypes of the audit tool.

The newborn unit audit meetings were initially held monthly then two-weekly.

Draft zero of the audit tool was subjected to a five-step iterative process during NBU audit meetings in PMH. The iteration involved the sequence of developing a prototype tool, evaluating it with the end users and modifying it based on their feedback to improve its efficiency and ensured that it could be used by the intended users with minimal coaching (Box 3). The tool went through 15 revisions during the study period until we reached a point of saturation on the feedback from the end users during each audit cycle. The outcome of this was a high-quality prototype audit tool.

a. Understanding the context through a review of literature.

Initial work involved understanding the context of use and user requirements. We reviewed literature that described the Kenyan health system context in terms of: (i)The organizational environment which include organizational culture, values, leadership [20, 31–33], (ii) physical environment with respect to structures, availability of equipment, medicines and materials [31, 34], (iii) health workers who are the end users in terms of their behaviours, attitudes and work tasks [34–37], and (iv) literature from other LMICs that described the facilitators and barriers to the maternal and perinatal audit process [38–45]. This information was used to develop the topic guide for the focus group discussions.

b. Understanding the context through focus group discussions and arriving at consensus on the audit implementation guide.

To further understand the context and user requirements, we conducted a consensus virtual workshop to engage the end users in the design of the audit implementation guide. The workshop consisted of three parts that included: (i) FGDs that allowed for in depth understanding of the facilitators and barriers to the newborn audit process within the Kenyan context (described in detail in S1 Appendix), (ii) a plenary session to discuss the audit implementation guide based on WHO recommendations that ensured all participants had the same understanding of an ideal implementation guide, and iii) a consensus meeting to agree on feasible SOPs for the implementation guide that would adapt the Geneva: World Health Organisation; 2018 manual with the context and users in mind [2]. The consensus meeting began with designing user personas to understand the attributes of the end users before arriving at consensus using the nominal group technique. The users were involved in every step of the design with a full description of the workshop provided in S1 Appendix. The outcome of which was the SSNB audit implementation guide.

a. Outcome of reviewing the available audit tools to understand the gaps and inform the structure of draft zero of the SSNB audit tool.

The review of the three maternal, perinatal and newborn audit tools informed the structure of draft zero of the prototype newborn audit tool. The Kenyan MPDSR tool and the WHO stillbirth and neonatal death case review form were focused on maternal and perinatal care. The WHO child and neonatal death review form had information on neonatal care but focused on care beyond the neonatal period [2, 3, 10].

The Kenyan MPDSR tool and the WHO stillbirth and neonatal death audit tool were similar in their structure. Both tools followed the same sequence to obtain patient information in a systematic manner. The sequence began with hospital details, biodata of mother and baby, mother’s antenatal care, labour and delivery details, resuscitation care of baby, cause of death, modifiable factors and action plans. The WHO child and neonatal death review form did not entirely focus on the perinatal and neonatal period, however it had content on the care of children beyond the neonatal period. The structure of the audit tool however followed the same sequence of patient care details from admission to death with more provision to discuss the care provided during hospital stay. A summary of the structure and content of the three studied tools has been presented on Table 1 below.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Summary of structure and content of available perinatal, neonatal and child clinical audit tools.

https://doi.org/10.1371/journal.pgph.0001577.t001

b. Structure of draft zero of the prototype newborn audit tool.

Draft zero of the prototype newborn audit tool was divided into four sections (S3 Appendix):

1. Newborn biodata which included the patient admission number, gender, date of birth, birth weight, gestation at birth, age at death (mortality audits) or age at review (near miss clinical audits), weight at death (mortality audits) or weight at review (near miss audits).
2. Mother’s details which included the ANC details and details of labour and delivery. ANC details included: Mother’s blood group, Human Immunodeficiency Virus (HIV) status, syphilis status, hypertension in pregnancy and diabetes in pregnancy.
   Labour and delivery details included: Mode of delivery, complications during labour and delivery and if the newborn was resuscitated after delivery. The complications of labour and delivery were coded at the bottom of the audit tool.
3. Review of the care provided during and after admission.
   This section of the audit tool began by reviewing the care provided at admission and was structured to allow the audit participants to discuss the quality of care provided; what was well done, what could have been done differently and the recommendations made for each step in care during admission (timely admission, adequacy and appropriate assessment -history, physical examination, and investigations, primary and secondary diagnoses, supportive and definitive management).
   The next section reviewed the post admission supportive and definitive management based on the progression of illness.
4. Details of death for mortality audits (primary and secondary cause of death) and the conditions that led to unfavourable outcomes for near miss audits.

c. Final outcome of the cognitive walkthrough of draft zero and the subsequent modified prototype audit tools.

The feedback received during the cognitive walkthrough phase was categorised into one of the three groups:

1. Usability—the extent in which the content of the audit tool allowed the users to comprehensively summarise the care provided to the small and sick newborns.
2. Human factors—the application of feedback on the structuring of the audit tool that enabled or limited the users’ interaction with the audit tool.
3. User experience—the perceptions and thoughts of the users based on their experience of using the audit tool.

A detailed description of the results and outcome of the modifications made based on user feedback during each audit cycle have been summarised in S4 Appendix.

The difference between the co-designed SSNB clinical audit tool and the available audit tools are summarised in Box 4 below.

Box 4. Differences between the co-designed SSNB audit tool and the available audit tools

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

https://doi.org/10.1371/journal.pgph.0001577.t002

Abbreviations: MPDSR, Maternal and Perinatal Death Surveillance and Response; NBU, Newborn unit; WHO, World Health Organisation.

a. Characteristics of the context and users that were perceived as facilitators and barriers to using the newborn clinical audit process based on the focus group discussions.

The focus group discussion results were categorised as facilitators or barriers to the audit process. The main themes arising from the FGDs and the participants quotes supporting the findings are provided in S5 Appendix. The themes arising include:

a. Outcome of the consensus meeting to design the implementation guide.

With an understanding of the user requirements through the creation of user personas (S6 Appendix), the team arrived at consensus on the standard operating procedure for conducting the SSNB clinical audit in Kenyan public hospitals using the nominal group technique. Table 2 below describes the seven components of the audit implementation guide and the proposed procedures by which they would be carried out to ensure the completion of each audit cycle.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Components of the audit implementation guide.

https://doi.org/10.1371/journal.pgph.0001577.t003

Organisational level factors.

In 2016, the perinatal component was included in the maternal death surveillance and response process in Kenya to improve the quality of newborn care in line with the sustainable development goals [49]. The MPDSR guidelines recommend that at the facility level, all maternal deaths should be reviewed within seven days of occurrence, with a set monthly meeting over and above each review meeting [9]. The guideline however recommends that monthly meetings should be convened to review all the perinatal deaths occurring in a facility within the period. The recommendation for multiple maternal death review meetings yet just one monthly perinatal meeting poses a challenge due to the high volume of perinatal and neonatal deaths [20]. The WHO recommends that the frequency of audit meetings should be based on the volume of cases with meetings to be held as frequently as weekly or two-weekly for high mortality areas with an in-depth review of one to two cases [2]. This emphasizes the need to have a separate neonatal clinical audit meeting that complements the MPDSR so that due attention can be placed on the newborn audit process.

The organizational culture has emerged as one that is capable of change evidenced by the support of the clinical audit process through having regular clinical audit meetings and adopting mechanisms to ensure that the audit process leads to change. Despite the presence of an environment that largely advocates for patient safety, the organizational environment is marred with a myriad of challenges that affect the impact of QI initiatives [25, 31, 34, 35, 50]. The emotional fatigue from such an environment and burnout from the high workloads may exacerbate a name and blame culture as identified from the FGDs [51]. Literature that highlights the facilitators and barriers to the audit process proposes mechanisms such as ensuring confidentiality, anonymity, respect and equality among meeting participants and the presence of a multidisciplinary audit committee to promote a no-blame environment [38, 41, 42].

The absence of organizational leadership from the audit meetings has been linked to the poor implementation of action plans arising from the meeting. Filippi et al. propose that there is a high likelihood that this reluctance to attend meetings may be due to lack of funds [40]. This may very well apply to the situation in Kenya as with the decentralization of health services, hospitals have lost autonomy to manage and use hospital funds. The result is delayed procurement of essential resources required to maintain a favourable work environment [32]. Therefore, requests are frequently made to the hospital managers which they are unlikely to implement and therefore prefer not to attend QI meetings.

Team level factors.

Clinical audits are a complex intervention that requires team collaboration to get the desired results. The hierarchical nature of the context has influenced the success of team collaboration in the clinical audit process. As evidenced from the results, the doctors determine if, when and how audit meetings will be conducted with minimal consultation from nursing and other cadres. The audit meetings have therefore been viewed as a doctors’ affair where the discussion will be dominated by the doctors and the focus will be on the medical aspects of care. Constituting an interdisciplinary team to manage the audits would be a solution to this problem [43]. In complex adaptive systems, each component must understand how important its contribution is to the whole [52]. This informed the selection of an interdisciplinary team whose actions are tightly linked and would influence the quality of newborn care. Successful clinical audit teams include those with well-defined roles for each member to prevent free riding whereby some members fail to contribute their fair share in team effort [53, 54]. Other characteristics of successful teams have been identified as those that have a committed chair who supports QI initiatives, are diverse and constitute the key decision-makers from departments involved in newborn care, have hospital managers as part of the team, promotes equality, open dialogue and confidentiality during the audit meetings [40, 43, 45].

Individual-level factors.

The poor implementation of audit recommendations has been widely recognized as a deterrent to the success of the audit process [42, 55]. Though this can be attributed to poor support from leadership, another important factor that plays into this is the poor skills in problem identification. As identified in the literature, a large proportion of the problems in newborn care in LMICs can be solved through the appropriate use of available resources and by improving knowledge and skills of the frontline health workers [56, 57]. These ‘simple’ measures added up play a significant role in improving the quality of newborn care as the health workers await implementation of the more resource-intensive solutions.