Introduction

Severe maternal morbidity (SMM) is broadly defined as an unexpected and potentially life-threatening event associated with labor and delivery [1]. SMM is associated with prolonged [2] and increased costs of hospital stay [3], fetal deaths [4], preterm deliveries [5], and long-term consequences for maternal physical and mental health [1, 6, 7]. At the population level, SMM surveillance is commonly conducted using administrative hospital discharge data based on International Classification of Diseases (ICD) codes [8]. The Centers for Disease Control and Prevention (CDC) produced an index consisting of 21 indicators based on ICD diagnostic and procedure codes to identify SMM cases [9]. Combining the 21 indicators into a single index enables population-level surveillance of life-threatening events, interventions, and organ failure types given analyzing individual indicators may not be feasible due to rare occurrence at delivery hospitalization. Research shows combining multiple indicators into a single index offers consistency in monitoring trends and is more straightforward to both communicate and understand than are individual indicators described in complex medical terminology [10].

However, some individual SMM indicators have measurement concerns or may lack specificity for disease severity. Numerous studies have assessed measurement issues for individual SMM indicators (e.g., blood transfusion, disseminated intravascular coagulation [DIC], eclampsia) [11–13]. For example, blood transfusion and DIC are the most prevalent SMM indicators in the US [9], yet these indicators have low sensitivity (65% and 22%, respectively) [13] in validation studies using medical records and a substantial proportion of hospital discharge records with blood transfusion and DIC do not reflect severe morbidity [11, 13]. Similarly, a previous study has reported a low positive predictive value (41.7%) for ICD codes associated with eclampsia [14]. Composite measurements, like the SMM index, only perform well if individual indicators in the index meet specific criteria (reliability, validity, feasibility) [10].

To determine how well individual indicators perform in identifying severe morbidity, our study ranks individual indicators according to how they identify in-hospital mortality, a well-accepted measure of severity [2]. We rank each indicator hierarchically according to its contribution to the overall SMM index using its population attributable fraction (PAF) of in-hospital mortality and a novel signal-to-noise ratio (SNR), that compares the level of a signal (measurement) to the level of background noise (variability) [15]. To assess whether patterns and prevalence of SMM indices remained consistent across the transition from ICD-9 to ICD-10 coding, we estimated predicted prevalence for 2017–2019 using ICD-9 coded data and compared these to the observed prevalence using ICD-10 coded data.

Methods

We used the 1993–2019 National Inpatient Sample (NIS), the largest database of all-payer hospital discharge records in the US [16]. The NIS is a uniform, standardized dataset based on state in-patient data that are contributed to the Healthcare Cost and Utilization Project that is weighted by the Agency for Healthcare Research and Quality (AHRQ) to produce nationally representative estimates.

The NIS contains up to 40 diagnosis and 25 procedure codes per discharge including Diagnosis-Related Group (DRG) codes, as well as hospitalization characteristics, including patient demographics, admission source, discharge disposition, length of stay, and hospital charges. In-hospital mortality was identified using the variable “died during hospitalization.” To ensure comparability of data across all years, AHRQ created trend weights for 1993–2011 that are consistent with the 2012 and later redesigned NIS datasets [17]. The cross-sectional nature of NIS prevented identification of multiple delivery hospitalizations for the same woman; thus, the unit of analysis in this study was the delivery hospitalization, not the individual.

Due to the transition from ICD-9 to ICD-10 coding on October 2015, we first limited our analysis to delivery hospitalizations between January 1993 and September 2015 for comparability of codes across years and to obtain a sufficient sample size to produce reliable estimates of SMM and death, since these are rare events. We identified delivery hospitalizations using a previously described algorithm [18].

Results

The study population consisted of 18,198,934 hospitalizations representing 87,864,173 US delivery hospitalizations of women ages 12–55 years (1993–2015; S1 Table). There were 0.8 in-hospital deaths per 10,000 delivery hospitalizations (n = 6,685). The indicator-specific prevalence, in-hospital mortality rate, PAF, and SNR for the 21 current SMM indicators and the four recoded indicators (n = 25 indicators) are summarized in Table 1. The recoded indicator for severe pre-eclampsia, HELLP syndrome, or eclampsia was most common (160.9 per 10,000 delivery hospitalizations). However, its in-hospital mortality rate (0.1%) was the lowest of all indicators and 10 other indicators had a higher PAF (>9.8). Blood transfusion and the recoded blood transfusion without DIC were the second and third most common indicators at 70.1 and 65.6 per 10,000 delivery hospitalizations, respectively, but had low mortality rates (0.3% and 0.2%). Indicator-specific mortality rates were greatest for restoration of cardiac rhythm (37.1%), cardiac arrest (36.5%), and amniotic fluid embolism (15.9%).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Prevalence, in-hospital mortality, population attributable fraction (PAF), and signal-to-noise ratio (SNR) for severe maternal morbidity indicators (SMM), Nationwide Inpatient Sample, 1993–2015 (weighted sample = 87,864,173, unweighted sample = 18,198,934).

https://doi.org/10.1371/journal.pone.0294140.t001

Table 2 shows the order of the indicators established by our iterative process with hPAF and SNR values, excluding the three current indicators with measurement issues and adding four recoded indicators (n = 22 indicators). The indicators with the three highest hPAFs, mechanical ventilation (29.4%), restoration of cardiac rhythm (25.2%), and cardiac arrest (14.4%), were ranked third, first and second, respectively. Cardiac arrest had the highest SNR (69.4).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Results of the hierarchical population attributable fraction (hPAF) approach for severe maternal morbidity indicator selection, with the selection iteration (k) of each indicator, the cumulative hPAF, and the signal-to-noise ratio (SNR) selection criterion at iteration of selection.

https://doi.org/10.1371/journal.pone.0294140.t002

Among all delivery hospitalizations, the 22 SMM indicators identified 82% of in-hospital mortality. The top three ranked indicators (restoration of cardiac rhythm, cardiac arrest, mechanical ventilation) identified approximately 69% of in-hospital mortality. The top 15 ranked SMM indicators accounted for 80% of in-hospital mortality, while the bottom seven indicators together only identified an additional 2% of in-hospital mortality. Therefore, the top 15 indicators were combined into a ranked index for further analysis.

From 1993–2015, the overall SMM prevalence using the combined 21 current SMM indicators increased 143%, from 72.9 to 177.1 per 10,000 delivery hospitalizations (Fig 1). During this time, the prevalence of blood transfusion increased by almost 400%, driving a large portion of the increase in prevalence. The SMM prevalence of the 20 current SMM indicators, excluding blood transfusion, increased by 37% from 1993–2015. By contrast, the prevalence of the top 15 ranked SMM indicators increased by 79%, from 23.4 to 41.8 per 10,000 delivery hospitalizations from 1993–2015.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Prevalence of severe maternal morbidity (SMM) per 10,000 delivery hospitalizations for (1) all 21 SMM indicators, (2) 20 SMM indicators (excluding blood transfusion), (3) blood transfusion alone, and (4) the top 15 ranked SMM indicators that identify the most in-hospital mortality, NIS, 1993–2019.

The blue, dotted line represents the 21 current SMM indicators developed by the Centers for Disease Control and Prevention (CDC). The green, dashed line represents the current indicator for blood transfusion alone. The red, solid line represents 20 SMM indicators (excluding blood transfusion). The purple dashed line represents the top 15 ranked SMM indicators that identify the most in-hospital mortality. Shaded areas represent the 95% confidence intervals for the regression-based SMM prevalence predictions, using 2010–2015 (ICD-9 coding) data, in 2017–2019 (ICD-10 coding). Interval colors correspond to the respective line colors. Data from the last quarter of 2015 and for the year 2016 were omitted because of changes in coding practices during the first year after ICD-10 coding implementation.

https://doi.org/10.1371/journal.pone.0294140.g001

The observed cumulative SMM prevalence estimates are within the predicted 95% confidence intervals in 2017 and 2019 among delivery hospitalizations, but fell slightly outside in 2018 (Table 3, Fig 1). Eleven of the 15 indicators had observed prevalences within the predicted 95% confidence interval in 2017 and 2019, while nine did in 2018. In 2019, the estimated SMM prevalence using the top 15 ranked indicators that identified the most in-hospital mortality was 55 per 10,000 delivery hospitalizations (approximately SMM 21,800 [range 20,800 to 22,900] cases per year assuming 4 million delivery hospitalizations per year nationally).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Observed and predicted prevalences for years 2017–2019 per 10,000 deliveries of the 15 highest ranked severe maternal morbidity (SMM) indicators for in-hospital mortality, Nationwide Inpatient Sample (weighted sample = 11,001,554, unweighted sample = 2,200,312).

https://doi.org/10.1371/journal.pone.0294140.t003

Discussion

Among approximately 88 million delivery hospitalizations, 15 SMM indicators identified 80% of in-hospital mortality in 1993–2015. The overall prevalence of this 15-indicator ranked SMM index increased by approximately 80% from 1993–2015. In contrast, while the prevalence of the current 20 SMM indicators, excluding blood transfusion, only increased 37% from 1993–2015. Using predicted estimates for 2017–2019, the prevalence of this 15-indicator ranked SMM index is 54.6 per 10,000 delivery hospitalizations or approximately 22,000 SMM cases per year.

A major strength of this study is the use of NIS, a large, representative national all-payer database widely used in the US to provide generalizable estimates and trends. NIS provided a unique opportunity to determine which SMM indicators, though rare outcomes, identified the most in-hospital mortality at delivery hospitalization. Our study has several limitations. First, the SMM indicators include both diagnosis and procedure-based indicators from administrative data, for which coding rules change over time. Procedure-based indicators may be a proxy for multiple diagnosis-based indicators. For example, ventilation may be an indirect indicator of acute respiratory distress syndrome, pulmonary edema, and/or pulmonary embolism. Second, we used pre-defined conditions or procedures to ascertain SMM indicators from the hospital discharge records. The indicators identified 82% of in-hospital mortality, therefore approximately 18% is attributable to conditions not identified by these analyses. Conditions not included as indicators of severe morbidity that identify in-hospital mortality include those that clinicians encounter in the Intensive Care Unit (ICU; e.g., motor vehicle crashes, poisoning, suicides, malignancies). The NIS does not include ICU admission status for all reporting states [16]. Additionally, the SMM indicators do not capture information on comorbidities such as mental health conditions or substance use disorders which may also contribute to in-hospital mortality. Third, our study only examines in-hospital mortality at delivery hospitalization and does not look at antepartum or postpartum hospitalizations. This study does not capture the full spectrum of pregnancy related deaths that occur during pregnancy or up to one year postpartum. Fourth, we did not incorporate length of stay (LOS) in our algorithm. The SMM indicators with and without LOS considerations demonstrated excellent reliability (kappa statistic = 0.927) in a study from California [21]. Hence, its inclusion may add methodological complexity and may not contribute to meaningful interpretation of the association between SMM and in-hospital mortality. Fifth, the quality of documentation by physicians, coders’ training, and ongoing initiatives may affect coding accuracy [22]. Several validity studies reported high to moderate specificity but low sensitivity for some ICD-9 codes for SMM [11–13]. Although limited studies on the validity of ICD-10 codes are available, the sensitivity of these codes may be lower than for ICD-9 codes [13], though it may vary by indicator. Sixth, we were limited in examining pulmonary edema and acute congestive heart failure separately due to interchangeable coding practices for pulmonary edema and heart failure [23]. Pulmonary edema is labeled as a left ventricular failure in the coding manual and described as a left ventricular failure or heart failure in the medical chart [23], though ambiguity of medical diagnoses like this are inherent when using administrative data sets for analyses rather than individual medical records. Finally, we combined the top 15 SMM indicators into a ranked index for further analysis because these indicators identified 80% of in-hospital mortality and the remaining indicators only identified 2%; however, this threshold of 15 was arbitrary.

Our study offers an index for SMM population-level surveillance. While there are limitations to using administrative data to identify SMM, the ranked SMM index provides a measurement that is associated with the most severe outcome, in-hospital mortality, and can be used as a marker for maternal health.

The top three ranked indicators (restoration of cardiac rhythm, cardiac arrest, mechanical ventilation), all markers of acute cardiac and respiratory compromise, identified approximately 69% of in-hospital mortality. While cardiac arrest may result from a variety of cardiovascular and non-cardiovascular conditions, cardiovascular conditions are among the leading causes of maternal death [24]. This highlights the importance of rapid response teams, checklists, huddles, and debriefs to ensure successful and prompt resuscitation [25, 26].

The most prevalent indicators were the recoded severe type of hypertension of pregnancy indicator, the current blood transfusion indicator, and the recoded blood transfusion without DIC indicator; however, none of these indicators ranked among the top 15 indicators. Hypertensive disorders in pregnancy and postpartum hemorrhage are significant contributors to maternal morbidity and mortality and have increased in prevalence over the last two decades [24, 27]. In our analysis, we recoded the eclampsia indicator to include severe pre-eclampsia and HELLP syndrome in order to capture a broader spectrum of hypertensive disorders in pregnancy. While the severe pre-eclampsia, HELLP syndrome, or eclampsia indicator was last on our mutually exclusive, ranked SMM index, it is possible that an SMM code indicating a consequence of severe pre-eclampsia, HELLP syndrome, or eclampsia that is more strongly associated with in-hospital mortality (e.g., a puerperal cerebrovascular disorder; mechanical ventilation) would also be present. This would produce a lower hierarchical ranking of this indicator individually. The current blood transfusion indicator and the recoded blood transfusion without DIC indicator were the second and third most prevalent indicators. Obstetric transfusions are primarily associated with postpartum hemorrhage, and we sought to capture severe cases of postpartum hemorrhage by recoding the blood transfusion and DIC indicators. The NIS data do not quantify transfusion of blood products in units which might serve as a proxy for severity [28]. However, hospital discharge records with severe blood loss resulting in hemorrhagic shock are expected to have ICD codes for blood transfusion and multiple organ failure, including DIC. Thus, most hospitalizations with severe hemorrhage in our study should be captured by indicators of organ failure. The recoded indicator for both DIC and blood transfusion was among the 15 top ranked indicators. By making indicators mutually exclusive during the ranking process, we identified how many additional SMM deaths an indicator would capture, accounting for indicators already ranked. This process does not account for the true burden of the condition which informs population-based initiatives for prevention through organizations such as perinatal quality collaboratives [29].

Our methodology allowed the ranking of rare indicators with high in-hospital mortality and requires further testing. The strength and quality of evidence generated by its use could provide the basis for further refinement of SMM indicators, increasing potential applicability. Validation studies of the indicators used for the ranked SMM index are needed and may inform index utility.

Our results have important public health and surveillance implications. While this ranked SMM index results in a lower rate and total number of SMM cases compared with estimates of SMM using the combined 21 current indicators, it directly addresses the criticism of the combined 21 current SMM indicators: a lack of specificity for disease severity [11, 21]. While “severe” is difficult to define, linking diagnosis and procedure codes to in-hospital mortality underscores that selected conditions and events precede the most severe event, death. However, a ranking of indicators for the purpose of determining how many deaths could be prevented by sequentially eliminating indicators would need to use a different definition of PAF than we used (e.g., Benichou 2007) [30]; in particular, our hPAF is not appropriate for this purpose.

The ranked SMM index is for surveillance at the population level and was not developed for individual facility reviews [8]. Criteria for facility review of SMM are outlined by clinical organizations such as the American College of Obstetricians and Gynecologists and the Society for Maternal and Fetal Medicine [31]. The indicators included in the ranked SMM index could facilitate further refinement of SMM indicators for other uses (e.g., facility-based identification and review of SMM) and improve overall utility.

The ranked SMM index is unsuitable for comparison of individual health care institutions for multiple reasons. Health care institutions have access to the medical record and detailed and reliable data on comorbidities and severity, unavailable in the NIS, yielding more accurate, informative, and comparable case results. Comorbid conditions are strong predictors of overall SMM [32], but comorbid conditions are under-reported in hospital discharge data [33], with limited studies of reliability of ICD codes among hospitals [34], especially among delivery hospitalizations. The NIS data lack information on prenatal care and antepartum risk factors that are important determinants of SMM [35]. These factors are not considered in this ranked SMM index to adjust for differences in case-mix among institutions. Facilities with higher levels of maternal care are expected to have more high-risk deliveries and may have higher SMM rates compared with lower-level facilities [32]. Our approach of creating mutually exclusive indicators through iterative ranking may not be appropriate for care management goals in individual health care institutions. For instance, the risk of in-hospital death and thus, the condition severity and complexity of clinical management, increases with the growing number of SMM indicators [36]. Clinical reviews allow identification of preventable causes of SMM including determinants of clinical practices associated with SMM [31]. Only clinical reviews can guide design of hospital-level interventions and quality improvement initiatives to reduce SMM. A recent study of ICD-10 coding for blood transfusion, DIC, and acute renal failure confirmed wide variability of code sensitivity among hospitals in administrative discharge data and cautioned against using this type of data to compare inter-hospital safety and quality improvement initiatives at the facility level [13].

Conclusions

This study offers an index for reporting severe conditions using 15 indicators that identify 4 in 5 in-hospital deaths at delivery hospitalization. These indicators, when considered together, yield an index for population-level SMM surveillance based on in-hospital mortality. SMM surveillance is a promising method to further the understanding of maternity care and ultimately improve it [37]. Surveillance quality can be improved by reviewing medical records and corresponding ICD-10 codes to determine validity. Since a substantial proportion of pregnancy-related deaths and serious pregnancy-related complications occur outside of the delivery hospitalization, additional indices for antenatal and postpartum periods are needed [24]. Our findings underscore the need for continued testing of SMM indicators to refine metrics that provide simplified measurement of severe obstetric complications at the population level.

Supporting information

Acknowledgments

Disclaimer: The findings and conclusions in this paper are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.