Introduction

Massive transfusion (MT), defined as transfusion of ≥10 units of Whole Blood (WB) or packed red blood cells (pRBCs) within 24 hours, or 5 units within 4 hours, is required for approximately 3–5% of trauma patients [1]. With approximately 2.6 million trauma admissions per year in the US, there are 100,000 MTs for trauma each year -- 70% of all blood consumed at trauma centers [2,3]. Though standardization of Massive Transfusion Protocols (MTP) has led to reductions in both mortality, postoperative complications, and total volume of product transfused, these patients still face high mortality rates [4–8]. To make matters worse, the number of people donating blood has declined 40% in the last 2 decades [9]. Thus, judicious use of this limited resource is an important consideration, especially in resource-limited settings.

Both the Assessment of Blood Consumption (ABC) score and the Massive Transfusion Score have been applied in mortality prediction after MT but have only achieved 60–70% discrimination by area under the receiver operator curves (AUROC), likely because these scores, along with virtually every other MT-related scoring system, have been designed to predict who will require MTP, not who will survive once it has been initiated [10–12]. In current practice, the decision to initiate an MTP is typically made within minutes of patient arrival, often based on initial vital signs and clinical gestalt, with transfusion beginning almost immediately once activated. However, decisions about continuing transfusion and prognostication beyond the initial phase are more complex and rely on evolving physiologic parameters over the ensuing hours.

Artificial Intelligence (AI) and Machine Learning (ML) are rapidly growing fields that excel in their ability to integrate large amounts of data quickly to make real-time predictions. The ability of these models to use substantially more information in making predictions as compared to standard traditional techniques offers a unique advantage now that virtually all patient records are maintained electronically, making access to this data easy. ML algorithms have been developed for prediction of post-trauma complications such as surgical site infection, venous thromboembolism, and mortality in other populations as well as many other surgical and non-surgical populations [13–15]. These methods have also been refined to predict mortality in Critical Administration Threshold positive (CAT+) patients, those requiring ≥3 units of pRBCs in the first hour. However, the only model reported in the literature is based on < 2,000 CAT+ patients, not true MT patients, and its primary focus was prediction of need for MT [16].

To fill this gap, we aimed to utilize AI and ML techniques to predict survival in trauma patients undergoing MT using a large national dataset. We hypothesized that ML models trained on MT and ultramassive transfusion (UMT) patients could achieve similar, if not higher performance compared to pre-existing models for CAT+ patients, and that these predictions could be improved further with the incorporation of more clinical information that becomes available as the resuscitation progresses. These evidence-based mortality prediction tools have the potential to guide more informed blood allocation, especially in resource-limited settings where transfusion resources may be scarce.

Methods

Per recommendations by the EQUATOR network, the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD+AI) guidelines were followed, and the checklist is available in S1 Appendix [17]. This study was exempt from institutional review board approval because of use of a de-identified national database.

Results

Patient characteristics

Of 5,481,046 patients in TQIP from 2017 to 2021, 333,987 received at least one unit of any blood product. 47,744 patients received MT and of these, 20,337 received UMT. After excluding patients with missing data for 6-hour mortality (n = 1,675, of whom 800 were also in the UMT group), the study populations were comprised of 46,069 in the MT group and 19,537 in the UMT group.

The 46,069 MT patients had a 6-hour mortality rate of 21.9%, increasing to 41.2% at 30 days. Of these patients, 19,537 underwent UMT, with mortality rates at 6-hours and 30 days of 29.9% and 53.7% respectively. Tables 1 and 2 depict descriptive statistics for the MT (with UMT excluded) and UMT cohorts respectively, stratified by outcome.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Characteristics of patients receiving MT.

https://doi.org/10.1371/journal.pone.0335151.t001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Characteristics of patients receiving UMT.

https://doi.org/10.1371/journal.pone.0335151.t002

ML prediction performance

A GBDT showed the best performance for predicting 6-hour mortality for both the arrival and 4-hour timepoints. In both the MT and UMT models, performance improved at the 4-hour timepoint, from AUROC of 0.901 [95% CI 0.895–0.910] to 0.943 [95% CI 0.938–0.948] and 0.858 [95% CI 0.846–0.872] to 0.922 [95% CI 0.911–0.931] respectively (Fig 1). On paired bootstrap analysis, all GBDT models outperformed LASSO models (all p < 0.0001, Table 3).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Comparison of final models to L1 regularization regression models.

https://doi.org/10.1371/journal.pone.0335151.t003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Receiver-operator curves for prediction of 6-hour mortality in each model.

AUROC: area under the receiver-operator curve.

https://doi.org/10.1371/journal.pone.0335151.g001

All models demonstrated satisfactory calibration as depicted by the reliability diagrams in Fig 2. Decision curve analysis showed all models demonstrated net benefit over both “treat none” and “treat all” baseline strategies (Fig 3) for all decision thresholds under 0.95. Secondary performance metrics are displayed in Table 4.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Secondary metrics for each model developed and tested.

https://doi.org/10.1371/journal.pone.0335151.t004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Reliability diagrams to assess calibration of each model.

https://doi.org/10.1371/journal.pone.0335151.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Decision Curve Analysis to assess net benefit of each model.

https://doi.org/10.1371/journal.pone.0335151.g003

Evaluation with ShAP in all models identified the clinical factors having the strongest impact on model prediction, and were roughly similar between MT and UMT cohorts. On arrival, these were ED Glasgow Coma Scale (GCS), lowest systolic blood pressure, and patient weight/body mass index (BMI). At the four-hour timepoint, these determinant remained similar but total 4-hour transfusion volume became the single most important predictor in the MT group, and cryoprecipitate administration in the UMT group (Figs 4, 5).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Shapley additive explanation (SHAP) methods to assess feature importance for both massive transfusion models.

Left side: Beeswarm plot-each point represents an individual patient, the color of each point represents the value of that variable for that patient, and the horizontal displacement represents the effect of that value of that variable on the model’s outcome prediction for that individual patient. Right side: Top 20 most important features in relation to the model’s decision making, ranked by mean absolute SHAP value. ED: emergency department; SPB: systolic blood pressure; GCS: Glasgow Coma Scale; BMI: body mass index; Avg: average; EMS: emergency medical services.

https://doi.org/10.1371/journal.pone.0335151.g004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Shapley additive explanation (SHAP) methods to assess feature importance for all ultramassive transfusion models.

Left side: Beeswarm plot-each point represents an individual patient, the color of each point represents the value of that variable for that patient, and the horizontal displacement represents the effect of that value of that variable on the model’s outcome prediction for that individual patient. Right side: Top 20 most important features in relation to the model’s decision making, ranked by mean absolute SHAP value. ED: emergency department; GCS: Glasgow Coma Scale; SPB: systolic blood pressure; BMI: body mass index; Avg: average; EMS: emergency medical services.

https://doi.org/10.1371/journal.pone.0335151.g005

Discussion

To our knowledge, this is the first study to describe the use of AI and ML methods for real-time prediction of 6-hour mortality in patients undergoing MT and UMT. A GBDT performed the best for all models with AUROC of 0.901 [95% CI 0.895–0.910] in the arrival model, increasing to 0.943 [95% CI 0.938–0.948]in the 4-hour model (0.858 [95% CI 0.846–0.872] to 0.922 [95% CI 0.914–0.931] in UMT models). On descriptive feature analysis, the variables with the highest impact on the model’s prediction, on average, included ED GCS, lowest systolic blood pressure, patient weight/BMI, total 4-hour transfusion volume, and administration of cryoprecipitate. These results demonstrate that ML models can use variables available at the time of MTP initiation to accurately predict early mortality. These predictions outperform traditional regression-based approaches (p < 0.0001, Table 3) and can be refined by the inclusion of more data points as the resuscitation progresses. This information can serve as a useful clinical adjunct for clinicians facing difficult decisions about continued use of limited blood products and may be especially helpful in more resource-limited settings, such as smaller trauma centers or in cases of mass casualty incidents. This may also aid in difficult family discussions surrounding prognostication of this very sick cohort.

MTP is not a single event but an ongoing process, often over hours and involving variable transfusion volumes. While some patients may receive only a few units of blood, others can receive more than 100. Throughout this process, clinicians face multiple decision points where reassessment of resuscitation goals, patient trajectory, and futility becomes necessary. Our model is intended to support decision-making throughout the entire resuscitation process by providing real-time mortality prediction once MT has already begun and resuscitation is ongoing. While emergency and trauma physicians already incorporate evolving physiologic data into their decision-making, these decisions often rely heavily on clinical gestalt. The aim of our model is to quantify that expert intuition using data-driven approaches, creating a reproducible, evidence-based tool that mirrors expert reasoning. This is a goal supported by multiple recent studies citing the need to adapt advanced diagnostics, precision medicine, and patient-tailored strategies to inform clinical care in the context of massive transfusion [29,30].

We envision this model functioning as a decision-support tool for physicians during resuscitation, augmenting rather than replacing clinical judgment, particularly when time and resources are limited. This may be especially beneficial in lower-volume or lower-acuity trauma centers, where providers may not encounter massive transfusion cases frequently and where reliance on gestalt may be more variable. In these settings, a real-time, data-driven prediction model can help standardize decision-making, reduce uncertainty, and support more confident choices in the face of limited clinical exposure. Similarly, in mass casualty events, where clinical bandwidth and blood product availability can be rapidly overwhelmed, such a tool could aid in triaging ongoing transfusion efforts and prioritizing patients most likely to benefit.

Previous studies identifying when MT is futile in trauma patients have focused on identifying a threshold number of units of blood when continued resuscitation is considered “futile,” or have identified risk factors for mortality after MT and/or UMT. For instance, Louden et al identified ≥ 16 units in 4 hours as the point at which mortality rates eclipse 50%, labeling this “heroic,” and ≥ 36 units as the at which survival rates approach 0%, labeling this “near futile” [31]. Ang et al then went a step further to describe how patient age should influence these thresholds, with increasing patient age resulting in lower transfusion thresholds to maintain the same mortality rates [32]. A recent narrative review of this topic by Kim et al identified nine articles that examined this topic of when ongoing transfusion in trauma patients is futile, focusing on patients in the UMT range [33]. While the authors note that it is difficult, if not impossible, to decide on an absolute value a transfusion cutoff, they do note that “circumstances vary and the decision to continue transfusions should be individualized”. This widely shared view underscores the potential utility of ML-based solutions that provide highly individualized risk estimates, especially given the growing consensus that no single transfusion threshold can universally define futility in massive transfusion. Static numerical cutoffs have proven inadequate, and clinical decisions must instead account for patient-specific and situational factors [34–40].

There has been prior work on prediction models in MT patients, but virtually every scoring system and/or algorithm has been developed to predict which patients will require MTP activation in the first place, not who will survive once it has been initiated [11,12,41,42]. Additionally, while there are standardized metrics to quantify injury severity, such as Injury Severity Score and Trauma and Injury Severity Score, these are not available in real time and therefore not useful in mortality prediction or prognostication during the resuscitation. Benjamin et al developed the first ML-based algorithm for mortality risk-estimation in trauma patients requiring transfusions, and was the first proof-of-concept that highly individualized mortality prediction can outperform standard methods of mortality prediction, such as stratifying patients by ABC or Massive Transfusion Scores [16]. While their algorithm was based on a cohort of CAT+ patients, not MT patients, their Tier 2 model (using variables available on presentation as well as additional vital signs and adjuncts) was able to achieve an AUROC of 0.858 compared to the AUROC of 0.544 using just the ABC score.

Though our model’s performance metrics cannot be statistically compared to Benjamin’s given the different patient populations and variables in the datasets used in algorithm training, our AUROCs of 0.901 [95% CI 0.895–0.910] and 0.858 [95% CI 0.846–0.872] for our MT and UMT arrival models and 0.943 [95% CI 0.938–0.948]/ 0.922 [95% CI 0.914–0.931] for our 4-hour models suggest that our algorithm performs similarly if not better to existing ML methods. Another strength of our model is that it only uses structured variables that would be available within approximately 60 seconds of patient arrival. This can facilitate direct electronic medical record (EMR) integration where these structured variables could be pulled from the chart and would allow for the model to run directly within the EMR, automatically. Additionally, we specifically chose to train our model on a database that publishes new data every year to counteract model drift and remain stable over time [43].

Despite these strengths, our model is not without limitations. Given that these models were trained on a retrospective national database, it lacks the granularity of certain variables that may be important for the prediction of mortality after MT, such as the Focused Assessment with Sonography in Trauma exam, coagulation values, and hemoglobin: all of which were highly impactful features in Benjamin’s models for predicting CAT+ status. Because the dataset did not include the components necessary to calculate the ABC or Massive Transfusion Scores, we were unable to benchmark against those established tools. As with all retrospective modeling, the results are only as reliable as the training data. Additionally, though our data was taken from a national database, the model has not yet been prospectively validated, which may limit its generalizability. Our inclusion criterion of patients receiving at least 5 units of blood within the first four hours also introduces a degree of survival bias, as it excludes patients who died before reaching that threshold. Consequently, our model does not apply to patients who did not survive long enough to receive 5 units of blood.

While our model offers a high degree of accuracy in its predictions, it is certainly less interpretable than standard regression-based techniques, an issue common to all ML prediction algorithms. Though we can use ShAP to give us insights into why a given ML model makes the predictions it does, ShAP explanations do not depict independent associations like a regression would. Therefore, interpretation must be done cautiously with the knowledge that these are akin to univariate correlations. For instance, height appears consistently as an important predictor in our model. This does not imply that patient height is associated with mortality, however. It is more likely that height is unlikely to be measured accurately in critical patients and this “missingness” is a proxy for acuity or process-of-care dynamics, rather than a biologically meaningful signal. This phenomenon is consistent with findings from Donohue et al., who showed that missing thromboelastography data correlated with hypotension, low GCS, prehospital intubation, and increased 30-day mortality [44]. As a result, height in these critical patients may be defaulted to a standard value in the EMR, or is left missing and later imputed by our model. It is therefore critical to recognize that this feature analysis is provided to offer transparency into which variables the model is leveraging for its predictions, not to draw causal or explanatory conclusions about the biological or clinical importance of specific variables.

We chose to exclude explicit missingness indicator variables from the models for two reasons. First, to avoid overfitting to institution/database-specific documentation patterns that may not generalize during single-facility implementation, and second, to reduce the risk of the model overly relying on non-physiologic process artifacts. While missingness may carry real prognostic value, it is fragile as a predictor. Future institutional workflows changes, such as mandatory auto-filling of height/weight fields from prior encounters or license records, may result in decreased or entirely absent predictive utility of missingness. As a result, models that lean heavily on these artifacts may fail silently when deployed in environments with different charting practices. Though our model does not explicitly include missingness as a determinant of mortality, it may be able to pick up the imputed values as a sign of this missingness. To address this, separate models were run using random imputation for the height, weight, and temperature variables to assess model performance without this missingness heuristic. While performance of the models did drop slightly on admission (AUROC of 0.870 [95% CI 0.862–0.879] for MT and 0.814 [95% CI 0.801–0.830] for UMT), this drop was less pronounced in the 4-hour models (AUROC of 0.929 [95% CI 0.924–0.935] for MT and 0.904 [95% CI 0.895–0.914] for UMT. Relative feature importance remained similar across all models as well, except for weight and BMI becoming less impactful. Altogether, this suggests that these models are not overly reliant on missing data in their predictions, and that local implementation should take facility-specific practices into account when determining both whether to include explicit missingness and type of imputation method for missingness.

Most importantly though, this predictive tool is not designed to guide the initial activation of MTP or identify patients for whom it should be started. Rather, it is specifically designed to inform decisions about the continuation or termination of massive resuscitation once MTP has already been initiated. These are decisions that are currently based largely on clinician gestalt and experience rather than objective, data-driven methods, which may be especially problematic and variable in low-volume, low-acuity centers. Our model aims to supplement, not replace, clinical judgment in this complex setting. No matter how accurate prediction algorithms become, they certainly cannot replace the clinical decision-making of an experienced trauma surgeon. Importantly, while predictive accuracy is high, caution is warranted in interpreting model outputs to avoid a self-fulfilling prophecy, wherein high predicted mortality could lead to premature withdrawal of care. At the same time, the model may offer value in supporting continued resuscitation in patients who might otherwise be deemed futile, particularly in lower-volume trauma centers, where limited experience could lead to underestimation of survivability.

Prospective implementation of this tool must therefore be accompanied by ethical safeguards and institutional calibration. Threshold selection and model integration should reflect each institution’s resources, clinical culture, and tolerances for aggressive care. For example, a high predicted mortality score at one institution might trigger an early automated alert to the blood bank signaling imminent MTP activation, while at another, it could serve as an indicator to consider transfer to a higher level of care or an early alert to prepare either the operating room or an ICU bed. Ultimately, the model is intended to enhance, not constrain, individualized clinical decision-making, providing a data-informed method to augment trauma care, especially in resource-limited scenarios.

Our findings demonstrate that ML techniques can predict survival of trauma patients after both MT and UMT with outstanding discrimination and can refine this prediction over time as the resuscitation progresses. The next steps for this model will focus on two distinct areas: prospective validation and model extension using different data modalities. Prospective validation is essential to assess the real-time performance, clinical utility, and integration feasibility of this algorithm across diverse healthcare settings [45]. This will not only confirm the model’s predictive accuracy in practice but will also allow for iterative refinement based on institutional-specific workflows, thresholds, and patient populations.

Implementation trials could help evaluate how predictive outputs influence clinician decision-making, and whether they improve resource allocation, patient outcomes, patient communication/prognostication, or the efficiency of MTP continuation decisions. This will also allow for collection of other types of data that can extend the model, such as ED notes, operative reports, radiology reports, and time-series vitals data, as the use of unstructured data like this has been shown to improve predictive accuracy in other ML-based decision support tools [46]. Just as important as technical performance is end-user trust, usability, and buy-in. The successful integration of decision-support tools such as this will depend heavily on how frontline clinicians perceive their relevance, reliability, and ease of use. With this in mind, it will be important to include structured usability assessments such as those informed by the Health Information Technology Usability Assessment Scale framework to ensure that we are developing tools that actually help clinicians at the bedside [47,48]. As the use of ML and AI increases, so will the importance of databases that contain the necessary information for development of these tools. To support the development of robust ML models, future trauma databases should be structured to ensure high-quality, standardized, and clinically relevant variable capture.

Conclusions

Though ML models have been developed for prediction of mortality in CAT+ patients, this is the first reported ML model to predict 6-hour mortality trauma patients requiring both MT and UMT. A well-calibrated GBDT model performed best with AUROC of 0.901 [95% CI 0.895–0.910] in the arrival model, increasing to 0.943 [95% CI 0.938–0.948] in the 4-hour model (0.858 [95% CI 0.846–0.872] to 0.922 [95% CI 0.914–0.931] in UMT models). This model performs similarly to if not better than previously developed ML models for CAT+ patients. This demonstrates proof-of-concept that ML techniques can be leveraged for development of highly accurate prediction models that may aid clinicians in prognostication as well as making difficult decisions regarding futility and judicious use of blood products in the most critically injured patients.

Supporting information