Study setting

Malawi is a country in SSA with a relatively high MMR and NMR (510/100,000 live births and 23/1,000 live births in 2013, respectively) [43]. The Ministry of Health currently implements a results-based financing program on MNH quality in Malawi (RBF4MNH Initiative). To determine the effect of this program, an impact evaluation was designed that also included a component to assess the clinical quality of routine MNH care [44]. To this end, we developed a process indicator selection rationale which will be further described below. In this article, we use findings from the impact evaluation’s baseline survey in April 2013, with the exclusive aim of illustrating the practicability of our conceptual process indicator selection approach to measuring processes of care at the MNH junction. This cross-sectional process of care assessment included 33 rural health facilities offering labor and delivery care. These 33 facilities are evenly distributed over four districts in the central (Dedza, Mchinji, Ntcheu) and southern regions (Balaka) of the country. We only included facilities providing EmOC (5 hospitals providing comprehensive EmOC (CEmOC) and 28 health centers providing basic EmOC (BEmOC)). All 33 facilities are also included in the overall impact evaluation design [44].

Data collection and analysis

The data on quality of routine care processes was collected through non-participatory clinical observation of delivery cases during the intrapartum- and immediate postpartum period. We only included women presenting without complications to ensure comparability between cases in terms of clinical acuteness. Data were collected by local research assistants with professional backgrounds in midwifery using structured clinical observation checklists. These paper-based checklists collected information on 51 clinical routine care processes identified by our selection approach (see below). For each observed clinical patient encounter, the research assistants documented whether a given clinical process was performed by the health staff involved in the case according to national clinical guidelines, which were considered ‘gold standard’ for the purpose of this study. To cover large parts of the MNH Junction, observations started with the initial arrival of a laboring woman to the maternity unit and lasted until two hours after the delivery of the placenta (defined as immediate postpartum period for the purpose of this study). In order to include a sufficient number of delivery cases in each site, research assistants spent a minimum of three days at each individual facility and observed every delivery that took place during this period. Only cases in which both mid-level birth attendants and patients gave written consent to be observed were included into the study. The collected paper-based data were entered manually into an electronic dataset and analyzed using Stata software, version 12. Descriptive statistics were used to summarize the data. Frequencies and performance indices were used to summarize the observed clinical performance patterns. Performance indices were computed by assigning a score of 1 to each procedure observed as performed according to standard; the score for a non-performed procedure was 0. For each observed case a total performance score was calculated by summing the scores of each performed procedure relevant to s specific index. Mean performance scores were then computed by summing the total performance scores and dividing them by the number of observed cases. To allow easier comparability of the resulting performance indices, we divided each mean performance score by the number of procedures included in the index and transferred them into a relative 5-point index scale ranging from 0 (no performance) to 5 (complete performance). We created two types of performance indices: a risk-based index for each of the six identified direct mortality causes and a function-based index for each of the three routine care main functions. Ethical clearance for this study was obtained from the Ethical Committee of the Medical Faculty at the University of Heidelberg and from the College of Medicine Research and Ethics Committee.

Selection of routine care indicators

We only focused on quality of care indicators assessing clinical care processes [45][46]. The selection process consisted of five steps and is summarized in Fig 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Overview of the steps leading to the selection of routine MNH care process indicators.

https://doi.org/10.1371/journal.pone.0123968.g002

In a first step, we identified the clinical presentations of major direct causes of maternal and newborn mortality (i.e. major obstetric and neonatal complications and emergencies) based on available epidemiological data [3,4,6,7,47]. We further classified possible clinical manifestations of the identified mortality causes based on their medical acuteness. For this purpose, we defined medical complications as any combination of abnormal clinical findings that constitute or expose a patient to a potential health risk. Medical emergencies were defined as the final clinical pathway of any unaddressed clinical complication leading to an acute life-threatening situation. We then identified those complications and emergencies most relevant to the MNH Junction, which included the following complications: PPH, pregnancy-induced hypertension or pre-eclampsia, infection of mother or newborn, forms of fetal or newborn distress; and the following emergencies: hemorrhagic shock, eclampsia, puerperal/neonatal sepsis, and birth asphyxia.

In a second step, we reviewed quality of care process indicators of different clinical MNH assessment tools and protocols developed for intrapartum- and immediate postpartum quality of care evaluation in LMICs [10,27–29,48] and further adjusted them, to the extent possible, to the Malawian context based on current national reproductive health quality assurance protocols and clinical training guidelines [49–53]. This allowed us to identify a total of 266 clinical process indicators relevant or commonly used in assessing MNH care quality.

In a third step, we reduced the list of clinical process indicators identified in step two by excluding indicators that addressed clinical aspects other than those identified in step one. The majority of indicators removed in this third step pertained to the prevention of mother-to-child transmission (PMTCT) of HIV, to aspects of patient centeredness, malaria prevention in pregnancy, and premature birth. The remaining 128 clinical process indicators are those directly addressing the epidemiologic profile of major maternal and neonatal causes of death (i.e. medical complications and emergencies) relevant to the MNH Junction.

In a fourth step, we used the definition of routine clinical care outlined above to further reduce the indicator list to contain only measures of obstetric and neonatal routine care processes. The majority of indicators removed in this fourth step pertained to the medical and surgical management of emergency conditions, such as treatment of eclamptic seizures, removal of retained products in hemorrhaging women, etc., as these processes would only be observed in a subset of women giving birth, and are also already covered in the EmOC signal functions. The resulting 51 indicators only measure clinical processes observable during every single provider-patient encounter at the MNH Junction regardless of the actual clinical case presentation.

In a last step, we assigned each of the 51 remaining clinical process indicators to one of three categories, each representing a main function of clinical routine care: 1) initial risk assessment; 2) continuous risk monitoring; and 3) potential risk prevention. These three functions are further explained below.

• The initial risk assessment function is essential to the beginning of a provider-patient encounter to allow a clinician an initial evaluation of the patient’s overall risk profile to further determine additional medical needs likely to be required in the later course. Generally, this function comprises indicators related to a patient’s medical history (i.e. assessment of clinical symptoms), as well as a focused physical examination (i.e. assessment of clinical signs). At the MNH Junction, this function includes the clinical assessment of a woman’s cardiovascular status (e.g. blood pressure, heart rate, anemia), her risk of infection or pregnancy-induced hypertension (e.g. time of membrane rupture, body temperature, chest auscultation, peripheral edema, urine protein), but also an initial evaluation of signs of fetal or neonatal distress (e.g. fetal heart rate check and intrauterine presentation, responsiveness and breathing effort of the newborn).
• As a clinical case continues, a patient’s risk profile requires repeated re-assessments—either in response to initially identified risk factors or in adherence to clinical protocols. Routine care processes related to the clinical re-evaluation of risk factors are include in the risk monitoring function. At the MNH Junction, this includes the monitoring of labor progression during the first labor stage (e.g. use of partographs to observe the dilation of the birth canal, the descent of the fetus, or the development of prolonged or obstructed labor), the continued monitoring of a woman’s cardiovascular status and infection risk (e.g. documenting maternal vital signs and temperature using a partograph, checking the appearance of amniotic fluid, immediate postpartum monitoring of maternal and neonatal vital and physical signs to detect PPH, immediate postpartum assessment of the effect of AMTSL), and the continued monitoring of the newborn during the immediate postnatal period (e.g. checking a newborn’s responsiveness and temperature).
• Elements of the risk prevention function occur throughout the provider-patient encounter and involve processes that actively reduce or control a patient’s risk of developing medical complications. At the MNH Junction, this includes processes of infection prevention (e.g. hand hygiene, use of sterile supplies and equipment, performance of certain examinations and other clinical procedures in a sterile manner) and PPH prevention (e.g. use of oxytocin and active delivery of the placenta during labor stage three as part of AMTSL), as well as the prevention of risk factors related to birth asphyxia and neonatal hypothermia (e.g. controlling fetal distress, removal of nuchal cord, suctioning airways, avoiding exposure of the newborn to a cool environment).

Routine care process indicator selection

Our five-step selection rationale resulted in a set of 51 routine care processes (see Table 1). Divided into the three routine care main functions of clinical routine care, we identified 23 indicators related to the initial assessment of patient risk, 17 to risk monitoring, and 11 to risk prevention. In relation to the direct causes of mortality, 15 routine care indicators targeted a woman’s bleeding risk, and 11 addressed the mothers’ and newborns’ risk of perinatal infections. The risk of complications due to obstruction, and fetal distress were addressed by each 5 routine care indicators, whereas 7 indicators were related to the risk of newborn distress, and 8 indicators to the risk of pre-eclampsia. Maternal blood pressure checks were identified as a routine care process common to the prevention and monitoring of both the risk of pregnancy-induced hypertension/pre-eclampsia and bleeding. No routine care indicators specific to MNH Junction were identified for the prevention of prolonged labor based on our selection process.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Final set of routine care process indicators by clinical core functions and major obstetric and neonatal complications.

https://doi.org/10.1371/journal.pone.0123968.t001

Testing of suggested process indicators

A total of 82 delivery cases were observed in 31 rural health facilities (45 observations in health centers, 37 observations in district hospitals). In two facilities no observations could be conducted due to the lack of deliveries during the research assistants’ stay. None of the 82 observed cases exhibited any obvious signs of clinical complications at initial presentation to the maternity ward. Given the different stages of labor at initial presentation and the variation in the sequence of clinical routine processes per each observed case, not all 51 identified processes could be observed in every single provider-patient encounter. This explains the differences in total numbers of observations for each process indicator. For the initial risk assessment function, the numbers of cases observed per indicator ranged from 66 to 78 (80–95% of all observed cases), for the risk monitoring function from 58 to 76 (71–93% of all observed cases), and for the risk prevention function from 61 to 76 (74–93% of all observed cases). The relatively low sub-sample of cases for the risk monitoring function is due to the fact that only 58 out of 82 cases (71%) were actively monitored by partograph during stage one labor, and only 76 out of 82 cases (93%) were kept in the maternity unit for direct immediate postpartum observation during the first two hours after delivery.

The results on single process indicator performance as well as details on indicator-specific sample sizes are summarized in Table 2. The scores of the performance indices for the examined risk factors and routine care functions are provided in Table 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Observed frequencies of routine care processes by clinical core function and complication risk.

https://doi.org/10.1371/journal.pone.0123968.t002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Overall performance indices (using relative scale ranging from 0 = no performance to 5 = complete performance).

https://doi.org/10.1371/journal.pone.0123968.t003

Examining the performance quality of each of the three main functions of routine care (i.e. initial risk assessment, risk monitoring, risk prevention), our data showed the following:

Identification of routine care process indicators

We identified a set of process indicators for the assessment of the quality of routine care in MNH by following a pre-defined selection strategy. In order to reduce the wide range of routine care processes currently available to an essential set of indicators, we relied on a selection process strongly oriented towards the direct causes of maternal and neonatal mortality. Aligning this focus on mortality causes with the strategic framework of the continuum of care, we identified the MNH Junction as the most critical point in time in the care of pregnant mothers and their newborns. Extracting MNH routine care processes within this defined scope resulted in a clearly framed set of key process indicators we consider essential to any MNH quality of care assessment.

Common methodologies used in the assessment of clinical care processes are medical record reviews, staff interviews, interviews with patients after having received care (i.e. exit interviews), equipment checks, and direct observation of care. Of these, direct observation is considered gold standard due to the relative high sensitivity and specificity this method offers in detecting poor clinical performance [54]. The identification of MNH ‘routine care signal functions’ feasible for direct observation was therefore our primary focus.

Some of the routine care indicators suggested here have also been shortlisted by other authors or are already included in tools used by initiatives like AMDD (Averting Maternal Death and Disability), DHS (Demographic and Health Surveys), or MCHIP [24,34–39], for instance correct partograph use and AMTSL. The difference between other approaches and ours is depth and/or breadth, as well as the underlying assessment strategy. The MCHIP surveys aim at providing a broad range of quality of MNH care measures with a high level of information detail. To do so comprehensively, not only direct observations but also inventory checklists, provider and patient interviews, and document reviews are necessary. The routine care signal functions suggested by Gabrysch et al. [24] are part of a comprehensive set of signal functions including not only routine care, but also emergency newborn care in addition to the established EmOC signal functions. The proposed six routine function are broadly framed clinical process indicators(e.g. monitoring and management of labour with partograph, using measures of infection prevention) that could be collected as part of in-depth facility assessments, but in particular lend themselves to inclusion in large-scale facility survey tools. The approaches to assess functionality can thus range from simple (e.g. reported provision) to sophisticated (direct observation).

In comparison, by taking direct observation as gold standard in our approach, we increased the depth of routine care measures by defining processes contained in broader care components in more detail (e.g. ‘hourly blood pressure checks’ as one process of the component ‘correct partograph use’). In addition, we reduced the assessment breadth by keeping focus on routine care processes relevant to the MNH Junction.

By outlining this approach and applying the resulting indicator set to an actual field setting we offer an example for a MNH quality of care assessment with focus on routine care processes relevant to the MNH Junction. We hope that this example encourages other MNH quality of care programs to include both the MNH Junction focus and the suggested routine care indicators more explicitly into future quality of care assessments. We expect that a wider application of our proposed approach will allow further refinement of the current indicator set and eventually achieve consensus on the use of routine care process indicators useful for MNH service evaluations in LMICs. Consistent assessments based on a core set of routine care processes across MNH programs will facilitate implementation and evaluation efforts at national and international levels. Commonly shared standards in assessing MNH routine care indicators across LMICs settings could thus contribute to successfully reducing maternal and neonatal mortality.

Routine care performance quality

The findings yielded in the Malawian setting highlighted several gaps in the performance of obstetric and neonatal routine care at the MNH Junction. At first glance, our quality of care assessment demonstrated that none of the major mortality risks (i.e. bleeding, infection, pre-eclampsia, prolonged labor, fetal and neonatal distress) were approached by birth attendants in a sufficiently consistent way. Risk assessment, risk monitoring, and risk prevention functions did not appear to be systematically aligned, and clinical performance often deviated from the underlying clinical standards. This is mainly reflected by the relatively low or at best intermediate risk-based performance indices ranging between 1.8 and 3.4 points on a 5-point scale. With bleeding, infection, and hypertensive disorders together representing more than half of all direct causes of maternal deaths in SSA [3], and PPH and postpartum infection having been reported most common direct causes of maternal mortality in Malawi [55], this inconsistent routine care performance at the MNH Junction is worrisome.

With respect to the main functions of clinical routine care (initial risk assessment, risk monitoring, and risk prevention), our assessment showed that risk prevention processes were generally more routinely performed than initial risk assessment or risk monitoring processes. This is reflected by the function-based performance indices where risk prevention functions are almost completely performed (index of 4.2 on a 5-point scale). The negative consequences on maternal health outcomes due to health providers’ incomplete performance in the assessment and monitoring of risk factors has been previously reported in Malawi [55]. Improvements in the quality of risk assessment and monitoring functions of routine MNH care are therefore critical in the Malawian context and beyond.

As demonstrated by the set of process indicators identified with our approach, the risk assessment function as part of high quality routine MNH care relies heavily on laboratory testing (e.g. urine and blood tests), thorough evaluation of a patient’s past medical history (e.g. pregnancy-related danger signs), and the focused assessment and interpretation of physical signs (e.g. chest auscultation, maternal and fetal vital signs). Therefore, systematic performance of the risk assessment function on a routine basis is challenged whenever healthcare providers face shortages of supplies (e.g. laboratory test kits, stethoscopes, thermometers). Similarly, systematic performance of the routine MNH care monitoring function is heavily challenged by inadequate supply (e.g. partograph forms), but also by shortcomings in infrastructure (e.g. dedicated postpartum beds within the maternity ward) and staff (e.g. time constraints, lack of supervision of inexperienced staff). The chronic staff shortage and overcrowding of maternity units in Malawi [56], is likely to be a main contributor to the poor performance quality in routine risk assessment and monitoring at the MNH Junction, especially during labor and early postpartum periods.

Structural limitations aside, the MNH Junction provides healthcare providers with a unique opportunity for uninterrupted patient care. Based on the relatively low risk-based indices, especially for pre-eclampsia, perinatal infection, and neonatal distress, our findings also indicate that mid-level birth attendants might not fully embrace this opportunity to their and their patients’ advantage. For example, incomplete assessment of a patient’s pregnancy history regarding symptoms of pregnancy-induced hypertension or fever, the lack of basic practices of hand hygiene, or the inadequate use of partographs cannot be attributed to structural deficits alone. Previous assessments of MNH service delivery in Malawi already pointed at the relatively weak knowledge base in respect to certain aspects of routine obstetric care among qualified health workers in Malawi [57][58]. Improving medical knowledge—especially achieving a comprehensive understanding and interpretation of clinical interrelations—remains certainly a key element of mortality reduction at the MNH Junction. Major obstetric and neonatal complications and emergencies can only be detected and prevented if risk assessment, risk monitoring and risk prevention processes specific to a given complication are well aligned and integrated into a provider’s clinical decision-making. We understand each identified routine care process at the MNH Junction as essential in addressing a specific mortality risk. In the Malawian example, we observed some inconsistencies in certain clinical performance patterns. For example, although the PPH prevention through AMTSL was very frequently performed according to standard, follow-up monitoring of the effectiveness of this preventive intervention during the postpartum period was extremely infrequent and misaligned. Similarly, although birth attendants commonly used sterile equipment for infection prevention purposes, other preventive measures to control infection spread, such as hand washing, were rarely part of this routine. Based on other studies, routine use of oxytocin in third stage of labor has been popular with healthcare providers as it expedites third stage labor and thus shortens the time a midwife has to spend with a woman after delivery [56]. Inadequate hand-washing practices have been found to be due to providers’ personal inconvenience, lack of additional sanitary supplies, or insufficient hygiene protocols [59]. The extent to which such rather organizational factors can be applied to this Malawian sample will require further qualitative exploration.

Based on discussions with Malawian healthcare providers in preparation of the data collection instrument, non-abidance by some of the selected processes was expected beforehand in the Malawian context. For example, due to frequent supply chain limitations, laboratory testing of hemoglobin and urine protein levels are currently rarely performed on a routine basis, but rather on an ‘as indicated’ basis, in spite of the existing guidelines. Additionally, while routine checks for bacteriuria are suggested by some international guidelines, this process is currently not part of national standards in Malawi. Such discrepancies between recommended process indicators related to specific diagnostic laboratory and imaging procedures and common practice are found in many LMICs settings and contribute to shortcomings in quality of care. Similarly, optimal adherence to some of the identified routine care processes might be affected by causes beyond the clinician’s control, such as logistical or political constraints. For example, shortages in essential medicines and equipment, poor infrastructure, and inequitable geographic distribution of health resources has been an ongoing challenge to healthcare service delivery in Malawi [60][61]. Such context-specific limitations remain a common challenge to the comparability of performance standards across settings.

Limitations

The set of process indicators suggested here is not intended to yield a comprehensive evaluation of a wide range of clinical indicators; neither is it sufficiently refined to provide a detailed assessment of MNH routine care aspects beyond the MNH Junction. What we attempt to achieve with the presented work is both to outline the need for a set of ‘signal functions’ for MNH routine care quality evaluations and to suggest one possible conceptual approach to address this need.

By prioritizing major causes of maternal and neonatal deaths, focusing on the MNH Junction within the continuum of care paradigm and aligning of processes along three main functions of clinical routine care, we attempted to identify process indicators that directly link non-emergency components of clinical care to mortality outcomes. With this focus we obviously excluded a number of other important routine care process indicators. We agree that, for instance, the role of patient-centeredness in modulating women’s health-seeking behavior [62], or the role of PMTCT in reducing the risk of intrapartum transmission of HIV in positive mothers [63] are essential to the quality of MNH care. We did not include these as we were not able to identify specific and strong direct links to the major mortality causes of women and their newborns in LMICs.

Although the identified process indicator set was sufficiently comprehensive to point out areas of low quality routine care performance at the MNH Junction, we faced limitations. For instance, when measuring the performance of monitoring functions, we rely on the providers’ documentation of process indicators, such as partograph entries. This is limited by the availability and use of clinical records in busy daily practice. Low scores on documentation-dependent indicators therefore need to be interpreted in the light of circumstances that might prevent healthcare providers from timely and complete record-keeping, and therefore might be underestimating actual clinical performance.

In respect to the Malawian example, the overall size of the observed sample in Malawi was relatively small. The purpose of this article, however, was to illustrate the application of our conceptual approach to routine MNH care, and therefore we considered this as secondary. Still, the Malawi findings outline a relatively clear pattern of shortfalls in the routine care provided at the MNH Junction, especially considering of the possibility of a Hawthorne effect and thus better performance than usual due to the observational assessment approach. For instance, the wide neglect of clinical monitoring or ineffective alignment of measures in PPH prevention provide some useful information to national and international health planners and policy makers in terms of clinical training and performance priorities. As long as there still persist relatively large deficits in MNH service delivery, concise but relevant sets of process indicators can provide sufficient information on the main shortcomings in addressing maternal and neonatal mortality.