Study design

This retrospective observational cohort study was conducted at the Peking University People’s Hospital, a tertiary teaching hospital of Peking University, China. The Peking University People’s Hospital Medical Ethics Board approved this study (2020PHB258−01). Informed consent for participation in this study was waived because no treatment interventions were mandated, and no protected health information was collected or analyzed. This study was conducted in accordance with the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement and the Declaration of Helsinki [9].

Patients

All adult patients (≥18 years old) who were consecutively diagnosed with THS and who stayed in the ICU for more than 24 h between May 1, 2013, and May 31, 2023, were included. The diagnostic criteria for THS at admission were as follows (meeting a,b,c or a,b, and d): a, obvious bleeding caused by trauma (estimated blood loss volume greater than 1200 ml); b, a hemoglobin concentration<100 g/L or >30 g/L and lower than that before trauma;c, a systolic blood pressure <90 mmHg (or shock index [heart rate/systolic blood pressure] >1) for three consecutive measurements; and/or d, a serum lactate concentration>2 mmol/L [10]. Patients who were pregnant or lactating, who were suffering from shock due to other causes, who were classified as non-traumatic, who were expected to die within 24 h of admission because of fatal trauma, or who had missing data were excluded. All patients who were evaluated, diagnosed, and treated for THS by the same team of critical care specialists followed the same treatment protocol derived from available recommendations [11–12].

Data collection

Data were collected consecutively and retrospectively. All the data were obtained from the Trauma-Specific Database, which is a real-world clinical database composed of data from more than 23,000 trauma patients registered between 2012 and May 2023. To ensure the highest data integrity and accuracy, we implemented a multilayered validation process. This included automated validation rules, logic and consistency checks, and regular audits and reconciliation. This rigorous approach was crucial for maintaining the database’s reliability and accuracy for both clinical decision-making and research analysis.Trained doctors and research nurses input the data. They were unaware of the study protocol and did not participate in the management or care of the patients. Ten percent of the sample data were randomly selected to assess the quality of the data. We started accessing the data on September 25, 2022.

Patient demographics, comorbidities, causes of trauma, main bleeding site, and other laboratory and clinical variables required for the assessments of ISS, AIS scores, APACHE II scores, SOFA scores, and CMPMIT-ICD-10scores were collected. The worst value of each score within 24 h after ICU admission was used in this study.

The China Mortality Prediction Model in Trauma based on the ICD-10-CM lexicon (CMPMIT-ICD-10) is a new trauma scoring system based on data from the Beijing Red Cross Emergency Center [8]. This scoring system accounted for sex, age, partial new injury codes, partial comorbidities, traumatic shock, and state of consciousness.The scores of this scale are shown in Table 1, and the ICD-10codes are listed in Table S1 in S1 File.The risk of death was graded as follows: 0–47,extremely low risk (risk of mortality is <10%); 48–60,low risk (risk of mortality is 11–30%); 61–73,medium risk (risk of mortality is 31–60%); 74–90,high risk (risk of mortality is 61–90%); and >90,extremely high risk (risk of mortality is >90%). The total scores ranged from 0 to 232. To facilitate the use of CMPMIT-ICD-10, the National Center for Trauma Medicine of China developed a WeChat Mini Program.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. CMPMIT-ICD-10 scores.

https://doi.org/10.1371/journal.pone.0340159.t001

Acute respiratory distress syndrome was defined according to the Berlin definition [13]. Acute kidney injury was defined as a serum creatinine increase of ≥0.3 mg/dL (≥26.5 µmol/L) within 48 h or a urine output volume <0.5 mL/(kg·h) for 6 h [14]. Acute liver injury was detected and confirmed by liver biochemical blood tests in addition to a lack of history of acute or chronic hepatitis or liver cirrhosis. The definitions for acute liver injury include one of the following thresholds: i) ≥5 × upper limit of normal(ULN) elevation in alanine aminotransferase, ii) ≥2×ULN elevation in alkaline phosphatase, or iii) ≥3×ULN elevation in alanine aminotransferase and simultaneous elevation of total bilirubin concentration exceeding 2× ULN [15]. Myocardial injury was defined as a cardiac troponin I(cTNI) concentration above the 99th percentile upper reference limit. Injury was considered acute if there was an increase and/or decrease in cTNI values [16]. cTNI levels were assessed via a high-sensitivity troponin I assay on a DxI800 (Beckman Coulter, Brea, CA, USA), wherein the 99th percentile for this test was 0.034 ng/mL.Trauma-induced coagulopathy was defined as the presence of significant coagulopathy upon arrival, a prothrombin time>18 seconds, an activated partial thromboplastin time>60 seconds, or a thrombin time>15 seconds (1.5 times the normal value). According to the British National Blood Transfusion Service and the American College of Pathologists,coagulopathy is characterized by the need for blood product replacement therapy in the presence of active or impending hemorrhage [17–19]. If such organ dysfunction occurred, effective treatment was administered according to the corresponding guidelines or standards of care.

Outcomes

The primary outcome was in-hospital death. The patients’ discharge data were retrieved from the trauma-specific database at the hospital and screened for in-hospital death.

Statistical analysis

Using power analysis and sample size (PASS)15.0 for sample size calculation, we calculated the required sample size on the basis of the area under the ROC curve of the model(AUC0 = 0.5 for the null hypothesis and AUC1 = 0.65 for the alternative hypothesis). The ratio of nonsurvival to survival was 1:9 (α = 0.05; power = 0.8). A total of 330 participants (no fewer than 33 nonsurviving patients) were included. Approximately 20% of the participants dropped out; thus, a total of 396 participants were included.

Continuous variables are presented as the mean (standard deviation, SD) or median (interquartile range, IQR), as appropriate, and categorical variables are presented as counts (percentages). Student’s t test, the nonparametric Mann–Whitney U test, and the Pearson χ2 test were used for comparisons of variables. The missing data rate of all variables was < 5%, missing continuous variables were inferred as the median of nonmissing values, and missing categorical covariates were inferred as the most frequent categorical values.

The discriminative power of the ISS, AIS, APACHEⅡ score, SOFA score, and CMPMIT-ICD-10 score was first assessed using the area under the receiver operating characteristic curve (AUC) for the prediction of death. The cutoff value of each score was determined using the Youden index. The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy of these five scores at the corresponding cutoff values were analyzed and compared. The DeLong test was used to calculate the 95% CI of the AUC, and the McNemar test was used to compare the AUCs between the scores.

Next, we examined the predictive values of the ISS, AIS score, APACHE II score, SOFA score, and CMPMIT-ICD-10 score using logistic regression analyses. Moreover, we conducted a subgroup analysis on the basis of the presence of multiple organ dysfunction syndrome (MODS) and length of ICU stay and a sensitivity analysis on the basis of mechanical ventilation. MODS was defined as the occurrence of dysfunction of two or more organs within one week of traumatic hemorrhagic shock, with a SOFA score ≥ 4 points [20].

Statistical analyses were conducted using SPSS (version 22.0) and R (version R4.3.2). Statistical thresholds were adjusted for multiple comparisons using the Bonferroni correction.

Advantages and disadvantages of several scores in trauma patients and the proposedCMPMIT-ICD-10 score

To date, several scoring systems have been developed in an attempt to predict outcomes in trauma patients, including anatomically based scoring systems, such as the ISS [22] and the NISS [23], which were derived from the AIS [24]; ICD-based scoring systems, such as the ICISS [25] and the TMPM-ICD-10 [26]; and general ICU scoring systems, such as the APACHE II [27] and the SOFA [28]. Unfortunately, all of these scores have limitations. The AIS score and ISS do not account for physiologic variables and require coders to be trained, which can be time consuming. In addition, accurate coding is often not possible in the initial stages of trauma. The APACHE II score and SOFA score are not indicators of anatomical injuries caused by trauma; thus, they cannot fully represent the severity of injury. Furthermore, these two scores were the worst in the first 24 hours after admission to the ICU and are susceptible to initial resuscitation and the level of care in the ICU. Because the current scoring systems cannot predict outcomes promptly and accurately, our team developed a CMPMIT-ICD-10 score specifically tailored for Chinese trauma patients. Therefore, in this study, we chose the CMPMIT-ICD-10 score and compared it with four other scores for the prediction of in-hospital death in THS patients.

ISS and SOFA had limited prognostic value in THS patients

The ISS is the preferred method for providing an overall score for patients with multiple injuries. Meredith et al. [29] conducted a retrospective analysis of data from the National Trauma Data Bank (NTDB) of the USA and revealed that the ISS performed well in predicting in-hospital death in trauma patients, with an average AUC of 0.876. Additionally, another large-scale retrospective study [30] in New Zealand revealed that the ISS had a high predictive value (AUC 0.847) for 60-day mortality due to trauma. However, Rutledge et al. [31] analyzed trauma data from the North Carolina Trauma Registry (NCTR) and reported that the ISS had limited predictive value for the prognosis of trauma patients, with an AUC of only 0.667, which was consistent with our findings (AUC of 0.660). Another study [32] revealed that for patients who were admitted to the ICU for longer than 24 hours, the ISS performed poorly in predicting in-hospital mortality (AUC 0.61). One possible reason for the differences in the AUC among different studies is the variation in patient selection. Additionally, our study focused on patients with THS who were admitted to the ICU and who typically have severe physiological disturbances, thereby limiting the predictive value of the ISS, which focuses only on anatomical injuries. Thomas et al. [33] reported that for patients with an ISS ≥ 16 without physiological risk factors, the mortality rate was very low, at 3.1%. However, with an increasing number of physiological factors, the mortality increased almost linearly, reaching 86%. Thus, incorporating physiological issues to sharpen an anatomically defined principle would help to identify critical trauma patients.

A multicenter study [34] showed that the SOFA score can reliably reflect organ dysfunction/failure in trauma patients. A cohort study conducted in Brazil on critically ill trauma patients [35] revealed that the SOFA score within 24 hours of admission performed well in predicting in-hospital mortality (AUC 0.807), significantly outperforming the ISS (AUC 0.616). These findings were consistent with the results of our study (AUC 0.767). Another prospective study of trauma patients in the ICU in Taiwan [36] revealed that the SOFA score had relatively poor predictive value for in-hospital mortality (AUC 0.707). Therefore, the SOFA score has limited value in predicting in-hospital mortality in trauma patients.

The APACHE II score has good prognostic value in THS patients

The APACHE II score is widely used to assess the severity of disease in critically ill patients. However, the APACHE II score does not reflect anatomical injuries; thus, it cannot fully represent the severity of injury. Therefore, some researchers [37–38] have reported that the APACHE II score is not effective at predicting outcomes for ICU trauma patients. However, subsequent studies [39–40] reported that the APACHE II score was a good predictor of mortality in ICU trauma patients. A prospective study by Wong et al. [41] revealed that the APACHE II score had a greater ability to predict mortality in 470 ICU trauma patients in Canada (AUC 0.92 ± 0.02). A 4-year retrospective study in South Korea involving 706 ICU trauma patients by Hwang et al. [42] reached comparable conclusions (AUC 0.950). These findings were consistent with our observations, where the APACHE II score demonstrated significantly better predictive value for prognosis (AUC 0.875) compared with the ISS (AUC 0.660). Therefore, the APACHE II score performed well in predicting in-hospital death in THS patients.

The ISS, SOFA, and APACHE II scores have low sensitivity, good specificity and good accuracy in predicting mortality in THS patients

Fueglistaler et al. [35] reported that the ISS had a low sensitivity (38.9%)for the prediction of 30-day mortality but a high specificity (96.2%) and accuracy (83.1%). A meta-analysis conducted by Deng et al. [43], which included 11 articles evaluating the performance of the ISS in predicting mortality, demonstrated that the sensitivity and specificity of the ISS varied widely across studies. This meta-analysis revealed a sensitivity of 0.64 and a specificity of 0.93 for the ISS. The sensitivity was lower than that of our study, but the specificity was greater. The possible reasons for these differences included differences in study quality, sample size, and economic characteristics across countries.

Additionally, a study by Hwang et al. [42] revealed that for ICU trauma patients, the sensitivity, specificity, and accuracy of the SOFA score were 74.1%, 97.1%, and 92.4%, respectively, and these values were greater than our results (63.4%, 80.5%, and 78.8%, respectively). However, another single-center study from Switzerland [35] revealed that the SOFA score had a sensitivity of only 20.4% for predicting 30-day mortality, while the specificity was relatively high at 94.0%, and the accuracy was slightly lower at 77.2%. The main reason for these differences might be related to the variation in cutoff values chosen in different studies. Interestingly, two different studies from different times and regions revealed that the APACHEII score has remarkably similar sensitivity, specificity, and accuracy—50.8%, 97.3%, and 91.1%, respectively—in a Canadian study [41] and 58.5%, 99.6%, and 91.1%, respectively, in a South Korean study [42]. In these two studies, the sensitivity of the APACHE II score was slightly lower than that in our study, but the specificity and accuracy were greater. This difference is likely related to the selection of criteria, as both studies analyzed the ability to predict group mortality by the probability of death (>0.5) calculated for each patient on the basis of the APACHE II score rather than the specific value. Therefore, the criteria were stricter. Although this scoring system effectively identified surviving patients and reduced unnecessary treatment and resource waste, the low sensitivity means that some patients who were at risk of dying were not identified promptly, potentially missing opportunities for intervention.

Advantages of CMPMIT-ICD-10 and its good performance in predicting mortality in THS patients

The establishment of trauma centers in China is still in its infancy, with the mortality and disability rates of severe trauma patients remaining higher than those in developed countries [44]. Currently, a proper method for evaluating the quality of care at these trauma centers is not available. The CMPMIT-ICD-10 was established on the basis of a trauma database in China and is important for strengthening trauma care capacities in China [8]. The CMPMIT-ICD-10 adequately considered the influence of comorbidities and posttraumatic physiological responses on mortality. Another notable advantage is that mortality prediction only requires baseline information such as age, sex, comorbidities, ICD-10-CM codes, and consciousness status, which may be more convenient for clinical application. The internal validation results of the CMPMIT-ICD-10 were good, with strong discrimination and calibration; as an external validation of the CMPMIT-ICD-10, this study further confirmed its robustness.

These results indicated that the CMPMIT-ICD-10 has advantages in specific groups while reflecting the severity of their condition and risk of mortality more accurately. Although the CMPMIT-ICD-10 score had a high AUC (0.9195) and sensitivity (90.0%) in patients with shorter hospital stays (<7 days), we also found that it may be limited in predicting in-hospital mortality in these patients, which requires further research and validation. Subgroup analysis further confirmed the robustness and reliability of the CMPMIT-ICD-10 in predicting in-hospital mortality among patients with THS. These findings are important for clinical decision-making, providing more specific guidance for clinicians to make more accurate prognostic assessments and treatment decisions on the basis of the particular situations of patients, thus improving patient outcomes.

Study limitations

Despite the great predictive performance of the CMPMIT-ICD-10 demonstrated by this study, several limitations should be considered. First, this study is limited by its retrospective design, and all of the patients were from the same hospital. We attempted to minimize this effect by consecutively recruiting patients and adjusting for confounding factors using a multivariate regression model. Second, although the CMPMIT-ICD-10 accounts for variables such as age, underlying illnesses, and post trauma physiological responses, it may not completely eliminate the impacts of the coding rules because of the possibility of not accurately quantifying the severity of injuries.Moreover, although sensitivity analysis and subgroup analysis were conducted to further confirm our findings, the impact of a relatively limited sample size on the results cannot be ruled out. In the future, we will conduct multicenter studies to expand the sample size and compare our results with those of other scoring systems, such as the Trauma and Injury Severity Score (TRISS) and TMPM-ICD-10, to enhance the reliability and applicability of our scoring system in clinical practice.