Establishing a National Maternal Morbidity Outcome Indicator in England: A Population-Based Study Using Routine Hospital Data

Introduction As maternal deaths become rarer, monitoring near-miss or severe maternal morbidity becomes important as a tool to measure changes in care quality. Many calls have been made to use routinely available hospital administration data to monitor the quality of maternity care. We investigated 1) the feasibility of developing an English Maternal Morbidity Outcome Indicator (EMMOI) by reproducing an Australian indicator using routinely available hospital data, 2) the impact of modifications to the indicator to address potential data quality issues, 3) the reliability of the indicator. Methods We used data from 6,389,066 women giving birth in England from April 2003 to March 2013 available in the Hospital Episode Statistics (HES) database of the Health and Social care Information centre (HSCIC). A composite indicator, EMMOI, was generated from the diagnoses and procedure codes. Rates of individual morbid events included in the EMMOI were compared with the rates in the UK reported by population-based studies. Results EMMOI included 26 morbid events (17 diagnosis and 9 procedures). Selection of the individual morbid events was guided by the Australian indicator and published literature for conditions associated with maternal morbidity and mortality in the UK, but was mainly driven by the quality of the routine hospital data. Comparing the rates of individual morbid events of the indicator with figures from population-based studies showed that the possibility of false positive and false negative cases cannot be ruled out. Conclusion While routine English hospital data can be used to generate a composite indicator to monitor trends in maternal morbidity during childbirth, the quality and reliability of this monitoring indicator depends on the quality of the hospital data, which is currently inadequate.


Introduction
As maternal deaths become rarer, monitoring near-miss or severe maternal morbidity becomes important as a tool to measure changes in care quality. Many calls have been made to use routinely available hospital administration data to monitor the quality of maternity care. We investigated 1) the feasibility of developing an English Maternal Morbidity Outcome Indicator (EMMOI) by reproducing an Australian indicator using routinely available hospital data, 2) the impact of modifications to the indicator to address potential data quality issues, 3) the reliability of the indicator.

Methods
We used data from 6,389,066 women giving birth in England from April 2003 to March 2013 available in the Hospital Episode Statistics (HES) database of the Health and Social care Information centre (HSCIC). A composite indicator, EMMOI, was generated from the diagnoses and procedure codes. Rates of individual morbid events included in the EMMOI were compared with the rates in the UK reported by population-based studies.

Results
EMMOI included 26 morbid events (17 diagnosis and 9 procedures). Selection of the individual morbid events was guided by the Australian indicator and published literature for conditions associated with maternal morbidity and mortality in the UK, but was mainly driven by the quality of the routine hospital data. Comparing the rates of individual morbid events of the indicator with figures from population-based studies showed that the possibility of false positive and false negative cases cannot be ruled out.

Conclusion
While routine English hospital data can be used to generate a composite indicator to monitor trends in maternal morbidity during childbirth, the quality and reliability of this

Introduction
Most women in high resource settings give birth in hospitals or birth centres [1]. In both middle and low resource settings there is a focus on increasing the proportion of facility based deliveries [2][3][4], although there are concerns over the quality of care received in facilities as the number of women delivering in these settings increases [5,6]. It is thus increasingly important that we identify a meaningful and straightforward means of monitoring the quality of maternity care. Much of the focus has been on process measures such as caesarean section rates, induction of labour rates, neonatal unit admission rates and readmission rates. Monitoring any or all of these measures can be problematic, for example, there is no agreement on what the optimal caesarean delivery rate should be. While it is clear that caesarean delivery can be an essential obstetric intervention to save the life of both mother and baby, it is equally clear that high caesarean delivery rates may represent over-medicalisation and that unnecessary intervention can lead to long-term complications for both mother and baby [7][8][9][10]. Similarly monitoring readmission rates may lead to paradoxical responses; for example, readmission rates will decrease if women remain in hospital for longer after giving birth and thus longer stays may result, even though this may not be clinically necessary for, or desired by, most women. Similarly, readmission rates may increase if women are discharged home earlier, even when early discharge is being offered in response to women's wishes to improve their experience of care.
Outcome measures such as maternal and perinatal mortality are frequently monitored at both health centre and population levels. However as, in particular, maternal death becomes rarer, this becomes less meaningful as it is not very responsive to changes in care quality. The WHO introduced a complex maternal near miss/ morbidity indicator, which requires collection of detailed data including laboratory based parameters indicating organ system dysfunction [11]. However it has been found difficult to implement in some settings [12,13] and particularly, since it requires additional data collection, monitoring represents a burden that many healthcare environments cannot sustain. In response to concerns about data collection burden, many calls have been made to use routinely available hospital administration data in order to monitor outcomes [14][15][16].
In the UK the Royal College of Obstetrics and Gynaecology have developed a series of "maternity indicators" which include elements such as caesarean section rate [17]. However very few of the indicators are outcomes and the majority are process measures [18]. Importantly, the work identified major concerns over data quality. An approach undertaken by researchers in New South Wales (Australia) was to develop a maternal morbidity outcome indicator, which used a range of outcomes identified in routinely available maternity hospital discharge data to develop a single measure [19]. By including a number of outcomes to develop the single measure, concerns over false negative cases are mitigated, since most women with severe morbidity will have more than one eligible procedure or condition, and thus multiple appropriate codes in the data.
However, it is not clear whether data quality is sufficiently high in England to adopt the Australian approach. The aim of this study was to investigate the feasibility of reproducing the Australian Maternal Morbidity Outcome Indicator using routinely available English hospital maternity data; the impact of modifications to the Indicator to address potential data quality issues as well as known maternal health concerns in the UK; and the reliability of the indicator.

Materials and Methods
We used routine hospital data from the Hospital Episode Statistics (HES) database of the Health and Social care Information Centre (HSCIC) [20] to develop a single measure of maternal morbidity during childbirth. The HES is a dataset of inpatient, outpatient, and accident and emergency records from all National Health Services (NHS) hospitals in England [20]. In this study, we used a bespoke extract of anonymous inpatient data from childbirth episodes of all 6,389,066 women who gave birth in England from 1 April 2003 to 31 March 2013. Information was available on diagnoses and procedures at the time of childbirth and socio-demographic and pregnancy related characteristics of the women from the recorded hospital data. We constructed an English Maternal Morbidity Outcome Indicator (EMMOI) to measure maternal morbidity outcomes during childbirth using the method employed by Roberts et al to construct the Australian indicator [19] and examined the reliability of the EMMOI as a measure of morbid maternal events during childbirth in England by comparing with evidence from published population-based epidemiological studies.

Constructing the English Maternal Morbidity Outcome Indicator (EMMOI)
We generated a list of diagnoses and procedures to be included in the composite indicator for England (EMMOI) by reviewing both the initial and final lists of the components of the Australian Maternal Morbidity Outcome Indicator developed by Roberts et al [19] as well as published literature on conditions associated with maternal morbidity and mortality in the UK [21][22][23]. In the HES data, patient diagnoses are coded using the 10 th revision of the International Statistical Classification of Diseases and Related Health Problems (ICD-10) and procedures are coded using the OPCS Classification of Interventions and Procedures used by the NHS hospitals in the UK [24].

Statistical analysis
We calculated the frequency and rate of the individual morbid events (diagnoses and procedures) in the study population and their annual rates over the period of 10 years in order to examine potential variation in coding practice or data quality. We calculated the incidence rate and 95% confidence intervals (CI) of maternal morbidity outcomes per 1000 women giving birth in England using the EMMOI for each year from April 2003 to March 2013 and conducted tests for linearity to examine their trend over time. We also calculated the percentage change in incidence of maternal morbidity during childbirth in England in 2012-13 compared to 2003-04 and 95% CI.
We examined changes in the maternal sociodemographic and pregnancy characteristics by conducting χ2 tests for differences in proportions and χ 2 test for trend. This was followed by univariable and multivariable logistic regression analyses to examine whether the change in the odds of maternal morbidity outcomes over the period of 10 years were attributable to the changes in maternal and pregnancy characteristics in England. A core logistic regression model was built including the nine maternal and pregnancy characteristics shown in Table 1. Tests for correlation did not show any significant moderate to strong correlation between the variables. Maternal age was included as a continuous variable. We tested for plausible interactions by fitting interaction terms into the multivariable model followed by likelihood ratio testing (LR-test). No significant interactions were identified. The proportion of missing data was high for all the nine variables included in the model. We did not consider the data to be missing at random and a proxy variable was generated by categorising the missing data as a separate group for each variable. Sensitivity analysis was conducted by redistributing the missing observations into the different categories of the variables. We also performed a complete case analysis. The results of the sensitivity analysis and complete case analysis were not different from that of the main model. All analyses were performed using Stata version 13.1, SE (StataCorp, College Station, TX).

Results
The composite indicator, EMMOI, included all 14 diagnoses and nine out of 10 procedures from the final list of components of the Australian indicator as well as sepsis, eclampsia and cerebral venous thrombosis, which are important causes of maternal morbidity and mortality in the UK [21][22][23]. Diagnostic codes for sepsis, eclampsia and thrombosis were initially included in the Australian indicator, but were later removed at the stage of refining as these were poorly defined in the Australian population health datasets used to develop the indicator. Each diagnosis and procedure was examined before inclusion in the final EMMOI by calculating the incidence rate and examining the trend over time from 2003-13. We observed a 60-fold decrease in the estimated rate of transfusion of blood and blood products from 132 per 10,000 childbirth episodes in 2003-04 to 2 per 10,000 childbirth episodes in 2012-13, in contrast to a more than doubling of the estimated rate of postpartum haemorrhage (PPH) over the same period of time (Fig 1). Based on this observation we did not consider the transfusion or PPH data to be reliable and thus these codes were not included in the final EMMOI. We also observed a mismatch between diagnosis of 'uterine rupture' and the procedure 'repair of ruptured or inverted uterus' with substantial fluctuation in the rates across the years (Fig 2). The overall incidence of uterine rupture estimated using the diagnosis code was closer to the rate estimated by a population-based study in the UK (1.9 per 10,000 maternities, 95% CI = 1.6 to 2.2) [25], thus we included the diagnosis code and excluded the procedure code from the final indicator. The final list of diagnoses included in the EMMOI were acute abdomen, acute renal failure, acute psychosis, cardiac arrest/ failure or infarction, cerebral oedema or coma, disseminated intravascular coagulopathy, cerebrovascular accident, major complications of anaesthesia, obstetric embolism including amniotic fluid embolism, shock, sickle cell anaemia with crisis, status asthmaticus, status epilepticus, uterine rupture, eclampsia, sepsis, and cerebral venous thrombosis. The procedures included in the EMMOI were assisted ventilation including tracheostomy, curettage in combination with a general anaesthetic, dialysis, evacuation of haematoma, hysterectomy, procedures to reduce blood flow to uterus, re-closure of disrupted caesarean section wound, repair of bladder or cystostomy, and repair of intestine. Of the 6,389,066 women giving birth in England from 2003 to 2013, the EMMOI classified 24,427 women as having experienced one or more maternal morbidity during childbirth. There was an increase in the annual rate of maternal morbidity outcomes from 2003 to 2013 (Fig 3) with a linear trend (p<0.001), even after accounting for the changes in maternal and pregnancy characteristics overtime (shown in Table 1). The crude and adjusted odds ratios for each year compared to 2003-04 are shown in Table 2. The estimated rates using the EMMOI show an overall 29% (95% CI 25% to 33%) absolute increase in the incidence of maternal morbidity during childbirth in England in 2012-13 compared to 2003-04. The overall frequency and rate of the individual component diagnoses and procedures (along with their codes) included in the EMMOI are shown in Table 3 and their trends across 10 years are shown in Figs 4 and 5. The most common diagnosis coded was eclampsia (7.2 per 10,000 deliveries; 95% CI 6.9 to 7.4) followed by evacuation of haematoma (4.9 per 10,000 deliveries; 95% CI 4.8 to 5.2), uterine rupture (4.8 per 10,000 deliveries; 95% CI 4.6 to 5.0) and severe puerperal sepsis (4.4 per 10,000 deliveries; 95% CI 4.2 to 4.6). While the rate of eclampsia decreased over the 10 years, the rates of other morbid events either increased or remained unchanged, with notable increases being for sepsis and uterine rupture (Fig 4), assisted ventilation including tracheostomy and repair of bladder or cystostomy (Fig 5).

Discussion
Using routine hospital data on episodes of childbirth in England from April 2003 to March 2013 we were able to generate a composite indicator, EMMOI, to measure maternal morbidity outcomes during childbirth in England which included 26 morbid events (17 diagnoses and 9 procedures). Selection of individual morbid events was driven by the quality of the routine hospital data, which was variable and thus necessitated a number of adaptations. The composite indicator showed an increase in maternal morbidity outcomes during childbirth in England across the 10 years examined, with a 29% overall absolute increase in incidence. However, questions on reliability of the indicator remain due to mismatches between the rates of individual morbid events and the incidence of these conditions in the UK reported by population-based studies.
Comparing the overall rate of the individual component diagnoses and procedures included in the EMMOI with incidence of these morbid events in the UK reported by population-based studies and with the rates of the individual components of the Australian indicator estimated  in the Australian population health datasets [41], showed considerable inconsistency ( Table 3).
Rates of six out of the 17 component diagnoses, cardiac arrest/ failure or infarction, cerebrovascular accident, major complications of anaesthesia, sickle cell anaemia with crisis, sepsis, and cerebral venous thrombosis, were comparable with the incidence of these conditions in the UK reported in the literature. Of the nine procedures included in the EMMOI only the rate of hysterectomy was comparable with the reported incidence rates for the UK. Rates of 10 morbid events were comparable with the Australian indicator (Table 3). Estimated incidence rates for 12 morbid events during childbirth in the UK were not available from existing studies: acute abdomen, acute renal failure, cerebral oedema or coma, disseminated intravascular coagulopathy, status asthmaticus, status epilepticus, assisted ventilation including tracheostomy, curettage in combination with general anaesthesia, dialysis, evacuation of haematoma, re-closure of disrupted caesarean section wound, and repair of intestine. We thus cannot rule out the possibility of both false positive and false negative cases which reduces the validity and reliability of the EMMOI. Irrespective of the variations in definitions and indicators used to measure maternal morbidity, several studies from across the world show an increase in the incidence of maternal morbidity outcomes [41][42][43]. The observed increase in maternal morbidity outcomes during childbirth in England from 2003 to 2013 was mainly driven by an increase in sepsis, uterine rupture, assisted ventilation including tracheostomy and repair of bladder or cystostomy, although all morbid events showed a static or increasing trend except eclampsia. The incidence of sepsis in the UK between 1 June 2011 and 31 May 2012 was estimated as 4.7 per 10,000 maternities [21] and confidential enquiries into maternal deaths in the UK showed an overall decrease in the rates of genital tract sepsis, but with considerable fluctuations across the years from 1985-87 to 2009-11 [23]. Although a decreasing trend in eclampsia in this study conforms to the findings of the Confidential Enquiry into Maternal Deaths which suggest that deaths due to hypertensive disorders during pregnancy (pre-eclampsia and eclampsia) are at the lowest ever recorded [23], the routine hospital data showed higher rates of eclampsia compared to that estimated by a previous UK wide epidemiological study (2.7 cases per 10,000 births) [22]. This could be due to coding errors in the HES data as discussed below.
The HES data includes diagnosis of acute psychosis among women during the childbirth episode (0.53; 95% CI 0.48 to 0.59), which would be expected to be lower than the overall incidence of acute psychosis in the UK (10 per 10,000 women) [26] as this condition generally occurs during the postnatal period after discharge. We were not able to find any other comparable data. Compared to the incidence of uterine rupture calculated from the HES data, the low uterine rupture rates measured by the UK Obstetric Surveillance System (UKOSS) could be due to the exclusion of women in whom an incidental asymptomatic uterine dehiscence was noted at caesarean section [25]. These cases are likely to have been classified as uterine rupture in the HES data.
Incidence rates of 10 out of 26 morbid events constituting the EMMOI were in agreement with the rates of the individual morbid events included in the Australian indicator estimated in Establishing a National Maternal Morbidity Outcome Indicator in England the Australian population health datasets [41]. Incidence rates for three conditions, disseminated intravascular coagulopathy, cerebral oedema or coma, and sickle cell anaemia with crisis were not reported in the study by Roberts et al [41] that estimated the trends in maternal morbidity using the Australian indicator despite these being valid components of the Australian indicator [19]. It is not known whether this was due to lack of cases in the data or unreliability of the routine data used in the study. Retaining only the components that agree with the incidence figures in the UK or with the rates of the individual components of the Australian indicator would exclude 11 conditions from the English indicator, including eclampsia, obstetric embolism and cardiac arrest/ failure, which are associated with increased risk of maternal mortality in the UK [44]. The resultant composite indicator (EMMOI) would therefore be questionable in its ability to estimate accurately and reliably the incidence of maternal morbidity during childbirth in England.

Strengths and limitations of this study
This study was an attempt to create a composite indicator to measure maternal morbidity outcomes during childbirth using routine hospital data from 6,389,066 women giving birth in England from 2003 to 2013. Although we were able to demonstrate that a composite indicator to measure maternal morbidity outcomes during childbirth can be constructed using routine hospital data, we were not able to validate the accuracy of the composite indicator in classifying women as having suffered a morbidity event, since data protection and privacy laws allowed only for access to anonymised data. The use of a composite measure helps to avoid or reduce concerns about false negatives, but we cannot exclude the possibility of false positives.
It is important to acknowledge the limitation of not being able to use the data on blood transfusion, PPH and 'repair of ruptured or inverted uterus' which we did not find to be reliable. While the information collated from the NHS hospitals goes through a process of cleaning and quality checks, the quality of routine data collected by the HSCIC including HES has been questioned by a number of audits [45][46][47]. It has been noted that in most hospitals the diagnoses and procedures are coded by trained clinical coders from the hospital notes which are not structured or standardised, as a result errors and omissions occur [47]. In addition to errors in the data, we cannot rule out the play of chance, bias and confounding due to variations in clinical practice and the definitions of the morbid conditions used across the NHS hospitals. This substantially compromises the validity and reliability of the incidence estimates of maternal morbidity during childbirth in England measured using the EMMOI.
Beyond the coding limitations we describe, it is difficult to determine the reasons for the observed differences in codes and procedures relating to uterine rupture and postpartum haemorrhage. We could speculate that procedures, particularly where these require an operation under anaesthesia, such as repair of ruptured uterus, are more reliably coded than diseases/disorders when codes could be included in error simply from a list of differential diagnoses. Similarly, since blood transfusion is a more clearly defined procedure, requiring detailed checking, it may be more accurately recorded than postpartum haemorrhage; estimation of blood loss post-delivery is known to be inaccurate [48]. Conversely, blood transfusion is recorded in other databases, and it is possible that when the record of the transfusion is included in the National Health Service (NHS) Blood and Transplant database, it is less likely to be recorded in Hospital Episode Statistics. These databases are not linked, and therefore we are unable to assess whether this is the case.

Conclusion
A single measure of maternal morbidity could be a reliable indicator of quality of pregnancy care in some settings, and an indicator which uses routine data could be a cost-effective tool to monitor the quality of pregnancy care in England. However, while our study showed that routine English hospital data can be used to generate a composite indicator to monitor trends in maternal morbidity outcomes during childbirth, the quality and reliability of this monitoring indicator is dependent on the quality of the hospital data. Using the available data, we found that some codes were unusable due to major concerns about their validity. We could not confirm the reliability of the indicator due to mismatches between the rates of the individual components of the EMMOI calculated using the HES data and the rates reported by populationbased epidemiological studies in the UK. The ongoing efforts to improve the reporting and quality of NHS hospital data [49], could make this a valuable resource to measure incidence and trends of maternal morbidity outcomes in England, but currently the data are not sufficiently robust to do so.