https://orcid.org/0000-0002-4709-0600
Figures
Abstract
Objectives
To establish and validate an individualized nomogram to predict the probability of death within 30 days in patients with sepsis-induced blood pressure drop would help clinical physicians to pay attention to those with higher risk of death after admission to wards.
Methods
A total of 1023 patients who were admitted to the Dongyang People’s Hospital, China, enrolled in this study. They were divided into model group (717 patients) and validation group (306 patients). The study included 13 variables. The independent risk factors leading to death within 30 days were screened by univariate analyses and multivariate logistic regression analyses and used for Nomogram. The discrimination and correction of the prediction model were assessed by the area under the Receiver Operating Characteristic (ROC) curve and the calibration chart. The clinical effectiveness of the prediction model was assessed by the Decision Curve Analysis (DCA).
Results
Seven variables were independent risk factors, included peritonitis, respiratory failure, cardiac insufficiency, consciousness disturbance, tumor history, albumin level, and creatinine level at the time of admission. The area under the ROC curve of the model group and validation group was 0.834 and 0.836. The P value of the two sets of calibration charts was 0.702 and 0.866. The DCA curves of the model group and validation group were above the two extreme (insignificant) curves.
Citation: Wang B, Chen J (2021) Establishment and validation of a predictive model for mortality within 30 days in patients with sepsis-induced blood pressure drop: A retrospective analysis. PLoS ONE 16(5): e0252009. https://doi.org/10.1371/journal.pone.0252009
Editor: Antonio Palazón-Bru, Universidad Miguel Hernandez de Elche, SPAIN
Received: June 27, 2020; Accepted: May 9, 2021; Published: May 20, 2021
Copyright: © 2021 Wang, Chen. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The minimal underlying data describing the original code in R software to establish our prediction model is available as Supporting Information. However, due to ethical restrictions, the clinical data from patients of Dongyang Hospital cannot be made publicly available. Instead, the data is accessible for non-commercial purposes with signed Confidentiality agreement by interested researchers who send data access requests to Mao Feng Wang (Head of Data Access committee of Affiliated Dongyang Hospital of Wenzhou Medical University, Email:13967995216@163.com).
Funding: The authors received no specific funding for this work.
Competing interests: The authors declare no conflict interest.
Introduction
Sepsis is a severe and life-threatening condition due to the response of the body to an uncontrolled infection, which affects millions of people worldwide each year [1–6]. In the United States, more than 970,000 patients suffering from sepsis are hospitalized each year, a number that increases every year [7]. A 20-year study on hospitalized patients in the United States revealed that the incidence of sepsis increases by 8.7% each year [8]. In addition, sepsis accounts for more than 50% of hospital deaths [9], demonstrating the impacts of sepsis on human health. Sepsis is associated with a very high mortality rate [8, 10–12]. Two multicenter studies showed that the in-hospital mortality rate of severe sepsis in China were 48.7% and 35.5%, respectively. Of course, these percentages were dependent on the level of medical care in each region [13, 14]. However, the mortality rate of sepsis and septic shock in China decreased thanks to medical advancements. The available current results revealed that the mortality rate of hospitalized patients with septic shock is approximately 40% [15]. Overall mortality of sepsis is approximately 17%, which is also high [3]. Among the patients with sepsis, some patients with decreased blood pressure did not meet the standard of septic shock, and owned a high mortality rate.
Currently, the clinical evaluation of the severity score and mortality estimation in patients with sepsis-induced blood pressure drop is based on the use of similar scoring tools such as Acute Physiology and Chronic Health Evaluation (APACHEII), or the Sequential Organ Failure Assessment (SOFA) [16, 17]. They all had certain deficiencies in the evaluation of patients with sepsis-induced blood pressure drop. The APACHEII score is applicable to critically ill patients and has a specific ability to predict the probability of death, but it is not specific to patients with sepsis-induced blood pressure drop and does not provide a specific risk assessment for patients with sepsis-induced blood pressure drop. In addition, early established APACHEII score would not be suitable for current updated treatment measures in sepsis patients as before [18]. SOFA score is not used to predict the outcome, but rather to describe the development of Multiple Organ Dysfunction Syndrome [17]. Hence, a lack of tools exists in clinical practice to predict the mortality in a short term of patients with sepsis-induced blood pressure drop.
For the above reasons, it is necessary to develop a tool to predict the mortality in the short term of patients with sepsis-induced blood pressure drop. Therefore, the purpose of this study was to establish a reliable and accurate risk prediction model for patients with sepsis-related blood pressure drop to determine the short-term progression of the disease to death. The prediction model was displayed as a nomogram [19], which could not only visually evaluate respective death risk based on individual variables, but also quantified the death risk of involved patients combining all the risk factors. Furthermore, this model might provide a basis for clinicians for an appropriate clinical evaluation of the patient, a better development of the interventional measures to apply and to promote the communication between clinicians and patients’ families.
Materials and methods
Patient selection
Inclusion criteria: (1) Patients with sepsis defined according to the definition of sepsis 3.0, such as: organ dysfunction triggered by infection that endangers the patient’s life. Rapid increase in the sequential (sepsis-related) organ failure assessment score (SOFA), with a total score 2 points. (2) The criteria to evaluate blood pressure reduction were a systolic blood pressure drop of 40 mm Hg from the original basal blood pressure or from the normal blood pressure, or a systolic blood pressure of less than 90 mm Hg at the time of admission to wards (excluding the emergency department). Exclusion criteria: (1) Patients with leukemia, liver cirrhosis, aplastic anemia, human immunodeficiency, autoimmune diseases (there were abnormal levels of white blood cell, red blood cell and platelet in patients with mentioned diseases above, resulting into different variables at baseline comparing to involved participants and higher risk of sepsis and septic shock. The patients with above diseases could be defined through examination results and counseling medical history, and then be excluded), primary peritonitis, other causes leaded shocks (hypovolemic hypotension; anaphylactic reaction due to drugs usage in emergency departments; hypotension due to analgesic drugs in emergency departments), pancreatitis or other diseases at admission; (2) Patients under the age of 18 years; (3) Patients with missing data (including two status on medical records we could not obtained the death results: transfer to another hospital and unhealed when discharged).
We started collecting data for the purpose of this study on April 1, 2020. A final number of 1023 of patients with blood pressure drop caused by sepsis and admitted to Dongyang People’s Hospital from January 2010 to February 2020 were enrolled in this study and retrospectively analyzed. The treatment was performed following standard procedures, such as the administration of anti-infective drugs, fluid infusion, nutritional support, drainage and other forms of treatment. The death was determined based on clinical records when patients were discharged from hospital. If patients were dead in hospital, the status at the time of discharging was “death”; if patients left hospital with improvement in clinical manifestations, their status was recorded as survivor in 30 days; if patients were not cured, they would be recorded as unhealed and excluded in this study.
Ethics statement
This study was approved by the Ethics Committee of Dongyang People’s Hospital, and written informed consent was obtained from the enrolled patients or their relatives. Also, written informed consent was obtained from guardians of the minors included in the study. The data were anonymously analyzed because the personal information was completely removed. This study was conducted in accordance with the principles of the Helsinki Declaration and its subsequent amendments.
Risk factors involvement and establishment of prediction model
We collected and the following factors of the subjects: Gender, age, and the level of the following indicators at admission: high-sensitivity C-reactive protein (mg/L), creatinine level (mmol/L), albumin level (g/L), platelet level (10*9/L), white blood cells number (10*9/L) and bilirubin (μmol/L); Cardiac insufficiency (cardiac function class III or above, considering the New York Heart Association classification as the standard), respiratory failure (oxygen partial pressure/oxygen concentration less than or equal to 300 mm Hg), peritonitis (diagnosed mostly in emergency department before admission into wards) requiring surgery (including intestinal perforation, intestinal obstruction, and appendicitis), malignant tumor history and consciousness disturbance (such as drowsiness, lethargy, coma) following admission in patients with sepsis-related blood pressure drop. Moreover, heart function and consciousness were evaluated at time of admission, when blood examination results were obtained in first test within 24 hours after admission.
All the evaluations were performed using the R software. The included patients were divided into the model group (n = 717) and the validation group (n = 306) by using createDataPartition function in R software [20]. In the model group, thirteen variables were analyzed by univariate analyses using twogrps function. The significant variables in univariate analysis were included in multivariate logistic regression analysis, before that linearity test was performed. A P greater than 0.1 was considered as linear by boxTidwell function. The forward stepwise method was used to select the variables and were eventually included in the model.
Based on the regression coefficients of independent variables, we established the individualized nomogram prediction model of the short-term mortality in patients with sepsis-related blood pressure drop [21, 22]. First, the individual point for involved risk factors could be obtained by matching the results of variables and point line. Then, all the respective points were summed as total point. Finally, results could be readout by matching the total point and the risk rate of death. The model was evaluated according to three parameters: discrimination, correction and clinical effect.
Evaluation of established prediction model
The discrimination of the predictive model was defined as the ability of the model to distinguish patients with sepsis-related blood pressure drop who are going to survive from the ones who are going to die. The area under the ROC curve (AUC) was used to assess the results [23]. The closer the AUC value was to 1, the better the discrimination ability of the predictive model. An AUC > 0.8 indicated that the model was good [24, 25].
The correction of a predictive model was defined as the agreement between the predicted probability and the actual observed probability. In this study, we use calibration chart to evaluate the goodness of fit, with a P value greater than 0.05 suggesting an adequate goodness of fit [26].
The clinical effectiveness of the prediction model was evaluated using the DCA curve. Its significance was found in a graph in which the ordinate was the benefit of the evaluation model (All curve) and the abscissa was the risk value of the disease (None curve). Accordingly, the model curve was the curve represented by the model, and the All curve and None curve (the two extreme curves) were used as the criteria. The more the distance of the model curve from the two extreme curves, the better the clinical significance and net benefit conferred to the clinic [25, 27].
The above evaluation was performed using the R software.
Statistical analysis
The measurement data in this study were not normally distributed tested by twogrps function in R software (except for albumin). The chi-square test was used to evaluate the categorical variables, and the Wilcoxon rank sum test (sample size less than 5000) was used to evaluate the continuous variables with abnormal distribution. For the continuous variable such as albumin, t-test was applied.
Results
Analysis and involvement of risk factors of death in patients
A total of 1023 patients were included in this study, divided into 717 in the model group (137 died) and 306 in the validation group (75 died). The global death rate in this study was 20.7% (212/ 1023). There was no significance in baseline characteristics between model group and validation (Table 1).
As regard the feasibility of the logistic regression analysis. The P of continuous variable were 0.227, 0.400, 0.593, there was linearity between the continuous variable and logit (p) (S1 Table). Moreover, the variance inflation factors (VIFs) were used to interpret the multicollinearity of all variables (S2 Table). The variance inflation factors (VIFs) were 1.183, 1.083, 1.111, 1.127, 1.126, 1.094, 1.046, 1.050,1.018 respectively, suggesting that there was no multiple-collinearity among the risk factors. All the involved variables could be used in further analysis.
Univariate analysis showed that presence of tumor history, peritonitis, heart failure, consciousness disturbance and respiratory failure had impact on whether survive within 30 days in patients with sepsis-induced drop in blood pressure (Table 2). Also, molecular expression of albumin, bilirubin, C-reaction protein and creatinine and the number of platelets in blood were significantly different between survivors and no survivors.
Then the significant variables in univariate analysis were tested in multivariate logistic regression analysis (Table 3), suggesting that albumin level, creatinine level, presence of cardiac insufficiency, respiratory failure, malignant tumor history, peritonitis requiring surgery and disturbance of consciousness were risk factors for death in patients with sepsis induced drop in blood pressure (Table 3) (P<0.05). Age, which was significant in univariable analysis, was not risk factor in multivariable logistic regression analysis. Thus, seven indexes would be included in the establishment of model after stepwise regression analysis (S3 Table).
Nomogram establishment for predicting risk of death
According to univariate analysis and multivariate logistic regression analysis and stepwise regression analysis, seven independent risk factors were included in the prediction model to develop an individualized nomogram prediction model (Fig 1), which should be interpreted as follows. A patient, for example, with sepsis-related blood pressure drop, cardiac function class III or above at the time of the onset (32 points), respiratory failure (47 points), without peritonitis requiring surgery (0 point), a creatinine level of 200 mmol/L (15.5 points), without tumor history (0 point), without consciousness disturbance (0 point), albumin level of 25g/L (57 points), had a total score of 151.5 points compared to the nomogram, corresponding to a 50% risk of death, thus having a high risk of death.
Arrows means matching results between the variables (from a patient as an example) and the point reference. The patients had no tumor history, no peritonitis, no consciousness disturbance, but had respiratory failure and heart failure. The levels of creatinine and albumin were 200 mmol/L and 25 g/L respectively.
Evaluation of predicting death in model group
Evaluation of the model discrimination: The area under the ROC curve of the model was 0.834 (Fig 2A), suggesting a good discrimination. At the optimal cut off point of 0.169, the model owned high specificity (73.8%) and sensitivity (79.6%). The correction plot (Fig 2B) illustrated that the fitting logistic curve (solid line) of the model group highly overlapped the standard curve, with a P value of 0.702, suggesting an adequate goodness of fit. The DCA curve (Fig 2C) was used for this evaluation. The curve of the model group was above the two extreme curves, without any intersection with the two extreme curves in the risk interval of 10% -80%, suggesting that the model this study possessed clinical effectiveness.
(A) ROC curves; (B) Calibration chart; (C) DCA curves.
Validation of prediction model in validation group
The model validation was also evaluated through the ability of discrimination (Fig 3A), correction (Fig 3B), and clinical significance (Fig 3C). In these figures, the ROC curve, correction diagram and DCA curve of the validation group were plotted. The area under the ROC curve was 0.836, suggesting a good discrimination. The P value of the correction diagram of the validation group was 0.866, suggesting that the model fitted the validation group. The DCA curve of the validation group had a good clinical value within the risk interval of 10% -90%.
(A) ROC curve; (B) Calibration chart; (C) The DCA curve.
Discussion
This study showed that the independent risk factors for death in patients with sepsis-related blood pressure drop were the presence of cardiac insufficiency class III or higher at the time of the onset, presence of respiratory failure, consciousness disturbance, malignant tumor history, peritonitis, albumin level, and creatinine level.
The real experience in clinical practice suggests that the blood pressure drops due to sepsis is associated with adverse clinical outcomes [28–31]. This pathology is more life-threatening than the sepsis alone, thus, a study evaluating this population would be of great significance. In this work, an individualized nomogram to predict the death within 30 days among patients with sepsis-related blood pressure drop was established. The ROC curve, correction curve and DCA curve were used to evaluate the discrimination, prediction accuracy and clinical effectiveness of the model, which were all significant. Furthermore, a good significance was demonstrated in the validation group, suggesting that this model might be beneficial in clinical practice.
Sepsis, caused by infection, usually results in disorders of immune response and clinical symptoms displaying dysfunction in kidney, heart and coagulation, consequently increasing the risk rate of death. In addition, the patients with sepsis induced drop in blood pressure show symptoms of hypoperfusion. However, the patients involved in this study include patients with sepsis shock and patients only with sepsis-induced drop in blood pression (could not meet the criteria of sepsis because of persistence duration of low blood pressure and lactic acid level) [15]. Theoretically, some patients in this study are milder than patients with sepsis shock.
Therefore, factors involved in sepsis shock mentioned above may be risk factors of death in patients with sepsis induced drop in blood pressure. As showed in our results, heart dysfunction is the death related risk factor, which is same to previous work [32]. In patients with sepsis-induced hypotension, the treatment requires fluid resuscitation, whereas in patients with cardiac insufficiency, fluid sensitivity increases, and improper fluid resuscitation leads to increased mortality. Peritonitis before admission to wards in this study, signs requiring surgery (Surgery for peritonitis), such as those generally caused by perforation, strangulated intestinal obstruction, necrotizing appendix and other surgical consequences, often leading to sepsis with blood pressure drop and requiring surgical treatment., Surgery is an appropriate treatment in patients with peritonitis, but it is evident that the injury from the disease itself, combined with the injury from surgery, lead to increased mortality [33, 34]. If no surgery is performed, the primary disease is not resolved, resulting in a dramatic deterioration of the patient condition. Furthermore, the survival rate of patients that do not undergo surgery is very low, based on the P values and ORs. In summary, the death risk of patients with sepsis induced hypotension increases in patients with peritonitis, especially in those patients without surgery involvement. “Tumor factors” is referred to patients with previous or current malignant tumors, regardless of the previous or current use of chemotherapy, surgery and other therapies. In patients, the effect of tumor itself and that of chemotherapy is already understood. Both tumors and chemotherapy affect the patient immunity. Existing studies demonstrated that tumors are one of the factors influencing the prognosis of patients with sepsis/septic shock [11]. As regard the role of albumin, some studies reported that albumin is an important factor affecting the critical prognosis of patients with sepsis/septic shock [35–38]. It is reasonable that such patients with respiratory failure will suffer from hypoxia and increased in mortality. Even if they were treated with ventilator, complications in the treatment will also led to increased mortality. The weakening of consciousness which means the decrease of self-protection function, may results in some complications, such as asphyxia and leads to the increase of mortality. In clinic, creatinine was a common index that was used to judge renal function, while patients with renal dysfunction had increased mortality.
Comparing to previous studies, not only the risk factors for short-term mortality in patients with sepsis-induced hypotension were found in this study, but these factors were further analyzed to derive specific scores for each of the influencing factors and present them on nomogram [17, 18]. The total score was calculated according to the specific scores of each influencing factor, and the death probability was deduced. The death risk was no longer an abstract fact, but more specific, which is an advantage not present in the other current clinical critical evaluation tools.
This study possesses some limitations: (1) it is a retrospective study, thus, a congenital deviation in the data collection is present; (2) this study is a single-centered study, thus lacking data verification with other centers; (3) some indicators were not included in this study due to data loss, such as lactate and procalcitonin level, of which more than 30% of the data was missing, leaving no room for interpolation; (4) although most patients with biliary obstruction, ureteral obstruction or those requiring drainage were subjected to a drainage procedures, a small number of patients refused drainage and their specific analyses were not performed. Hence, patients who required drainage but they were not subjected to it, might have a higher risk according to this model. Our plan is to continue to collect the data of patients with blood pressure drop caused by sepsis in subsequent studies in the next decade. Additionally, our future aim is to reduce data loss, thereby improving the collection of laboratory indicators in such patients and try to acquire additional data. Finally, it is of utmost importance to invite other centers to participate in our study.
Conclusion
This study developed a prediction model to evaluate the risk of short-term mortality in patients who suffer from sepsis-associated blood pressure drop. Through this model, the short-term mortality risk of these patients can be accurately evaluated, providing a more concrete and specific reference for clinicians.
Supporting information
S1 Table. The linear relationship analysis between continuous variables and logit (p).
https://doi.org/10.1371/journal.pone.0252009.s001
(DOCX)
S2 Table. Variance inflation factor of variables.
https://doi.org/10.1371/journal.pone.0252009.s002
(DOCX)
S3 Table. Stepwise regression analysis of involved variables.
https://doi.org/10.1371/journal.pone.0252009.s003
(DOCX)
S1 Checklist. PLOS ONE clinical studies checklist.
https://doi.org/10.1371/journal.pone.0252009.s004
(DOCX)
Acknowledgments
The authors thank the Le9 Health platform for the technical support, the integration of hospital data, and the specific data on patients with sepsis-induced blood pressure drop during the 10-year period in the hospital that was collected very efficiently and accurately, thus resulting in a great assistance in this research.
References
- 1. Degoricija V, Sharma M, Legac A, Gradiser M, Sefer S, Vucicevic Z. Survival analysis of 314 episodes of sepsis in medical intensive care unit in university hospital: impact of intensive care unit performance and antimicrobial therapy. Croat Med J. 2006;47:385–397. pmid:16758516
- 2. Dombrovskiy VY, Martin AA, Sunderram J, Paz HL. Rapid increase in hospitalization and mortality rates for severe sepsis in the United States: a trend analysis from 1993 to 2003. Crit Care Med. 2007;35:1244–1250. pmid:17414736
- 3. Fleischmann C, Scherag A, Adhikari NK, Hartog CS, Tsaganos T, Schlattmann P, et al. Assessment of Global Incidence and Mortality of Hospital-treated Sepsis. Current Estimates and Limitations. Am J Respir Crit Care Med. 2016;193:259–272. pmid:26414292
- 4. Kumar G, Kumar N, Taneja A, Kaleekal T, Tarima S, McGinley E, et al. Nationwide trends of severe sepsis in the 21st century (2000–2007). Chest. 2011;140:1223–1231. pmid:21852297
- 5. SepNet Critical Care Trials G. Incidence of severe sepsis and septic shock in German intensive care units: the prospective, multicentre INSEP study. Intensive Care Med. 2016;42:1980–1989. pmid:27686355
- 6. Whittaker SA, Fuchs BD, Gaieski DF, Christie JD, Goyal M, Meyer NJ, et al. Epidemiology and outcomes in patients with severe sepsis admitted to the hospital wards. J Crit Care. 2015;30:78–84. pmid:25128441
- 7. Angus DC, Linde-Zwirble WT, Lidicker J, Clermont G, Carcillo J, Pinsky MR. Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Crit Care Med. 2001;29:1303–1310. pmid:11445675
- 8. Martin GS, Mannino DM, Eaton S, Moss M. The epidemiology of sepsis in the United States from 1979 through 2000. N Engl J Med. 2003;348:1546–1554. pmid:12700374
- 9. Liu V, Escobar GJ, Greene JD, Soule J, Whippy A, Angus DC, et al. Hospital deaths in patients with sepsis from 2 independent cohorts. JAMA. 2014;312:90–92. pmid:24838355
- 10. Kaukonen KM, Bailey M, Suzuki S, Pilcher D, Bellomo R. Mortality related to severe sepsis and septic shock among critically ill patients in Australia and New Zealand, 2000–2012. JAMA. 2014;311:1308–1316. pmid:24638143
- 11. Miller RR 3rd, Dong L, Nelson NC, Brown SM, Kuttler KG, Probst DR, et al. Multicenter implementation of a severe sepsis and septic shock treatment bundle. Am J Respir Crit Care Med. 2013;188:77–82. pmid:23631750
- 12. Zimmerman JE, Kramer AA, Knaus WA. Changes in hospital mortality for United States intensive care unit admissions from 1988 to 2012. Crit Care. 2013;17:R81. pmid:23622086
- 13. Cheng B, Xie G, Yao S, Wu X, Guo Q, Gu M, et al. Epidemiology of severe sepsis in critically ill surgical patients in ten university hospitals in China. Crit Care Med. 2007;35:2538–2546. pmid:17828034
- 14. Zhou J, Qian C, Zhao M, Yu X, Kang Y, Ma X, et al. Epidemiology and outcome of severe sepsis and septic shock in intensive care units in mainland China. PLoS One. 2014;9:e107181. pmid:25226033
- 15. Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315:801–810. pmid:26903338
- 16. Baykara N, Akalin H, Arslantas MK, Hanci V, Caglayan C, Kahveci F, et al. Epidemiology of sepsis in intensive care units in Turkey: a multicenter, point-prevalence study. Crit Care. 2018;22:93. pmid:29656714
- 17. Vincent JL, Moreno R, Takala J, Willatts S, De Mendonca A, Bruining H, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med. 1996;22:707–710. pmid:8844239
- 18. Knaus WA, Draper EA, Wagner DP, Zimmerman JE. Prognosis in acute organ-system failure. Ann Surg. 1985;202:685–693. pmid:4073980
- 19. Timmers JM, Verbeek AL, IntHout J, Pijnappel RM, Broeders MJ, den Heeten GJ. Breast cancer risk prediction model: a nomogram based on common mammographic screening findings. Eur Radiol. 2013;23:2413–2419. pmid:23591619
- 20. Wang J, Zhang Z, Zhou Z, Gu H. Clinical Prediction Models: Model Validation. Chinese Journal of Evidence-Bases Cardiovascular Medicine. 2019;11:141–144.
- 21. Harrell FE Jr., Califf RM, Pryor DB, Lee KL, Rosati RA. Evaluating the yield of medical tests. JAMA. 1982;247:2543–2546. pmid:7069920
- 22. Iasonos A, Schrag D, Raj GV, Panageas KS. How to build and interpret a nomogram for cancer prognosis. J Clin Oncol. 2008;26:1364–1370. pmid:18323559
- 23. Morrow DA, Antman EM, Charlesworth A, Cairns R, Murphy SA, de Lemos JA, et al. TIMI risk score for ST-elevation myocardial infarction: A convenient, bedside, clinical score for risk assessment at presentation: An intravenous nPA for treatment of infarcting myocardium early II trial substudy. Circulation. 2000;102:2031–2037. pmid:11044416
- 24. Seymour CW, Liu VX, Iwashyna TJ, Brunkhorst FM, Rea TD, Scherag A, et al. Assessment of Clinical Criteria for Sepsis: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315:762–774. pmid:26903335
- 25. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. 2006;26:565–574. pmid:17099194
- 26. Niu XK, He WF, Zhang Y, Das SK, Li J, Xiong Y, et al. Developing a new PI-RADS v2-based nomogram for forecasting high-grade prostate cancer. Clin Radiol. 2017;72:458–464. pmid:28069159
- 27. Nattino G, Finazzi S, Bertolini G. A new test and graphical tool to assess the goodness of fit of logistic regression models. Stat Med. 2016;35:709–720. pmid:26439593
- 28. Bijker JB, van Klei WA, Kappen TH, van Wolfswinkel L, Moons KG, Kalkman CJ. Incidence of intraoperative hypotension as a function of the chosen definition: literature definitions applied to a retrospective cohort using automated data collection. Anesthesiology. 2007;107:213–220. pmid:17667564
- 29. Futier E, Lefrant JY, Guinot PG, Godet T, Lorne E, Cuvillon P, et al. Effect of Individualized vs Standard Blood Pressure Management Strategies on Postoperative Organ Dysfunction Among High-Risk Patients Undergoing Major Surgery: A Randomized Clinical Trial. JAMA. 2017;318:1346–1357. pmid:28973220
- 30. Lambden S, Creagh-Brown BC, Hunt J, Summers C, Forni LG. Definitions and pathophysiology of vasoplegic shock. Crit Care. 2018;22:174. pmid:29980217
- 31. Walsh M, Devereaux PJ, Garg AX, Kurz A, Turan A, Rodseth RN, et al. Relationship between intraoperative mean arterial pressure and clinical outcomes after noncardiac surgery: toward an empirical definition of hypotension. Anesthesiology. 2013;119:507–515. pmid:23835589
- 32. Ng PY, Sin WC, Ng AK, Chan WM. Speckle tracking echocardiography in patients with septic shock: a case control study (SPECKSS). Crit Care. 2016;20:145. pmid:27177587
- 33. Aller MA, Arias JI, Prieto I, Gilsanz C, Arias A, Yang H, et al. Surgical inflammatory stress: the embryo takes hold of the reins again. Theor Biol Med Model. 2013;10:6. pmid:23374964
- 34. Gentile LF, Cuenca AG, Efron PA, Ang D, Bihorac A, McKinley BA, et al. Persistent inflammation and immunosuppression: a common syndrome and new horizon for surgical intensive care. J Trauma Acute Care Surg. 2012;72:1491–1501. pmid:22695412
- 35. Artero A, Zaragoza R, Camarena JJ, Sancho S, Gonzalez R, Nogueira JM. Prognostic factors of mortality in patients with community-acquired bloodstream infection with severe sepsis and septic shock. J Crit Care. 2010;25:276–281. pmid:20149587
- 36. Franch-Arcas G. The meaning of hypoalbuminaemia in clinical practice. Clin Nutr. 2001;20:265–269. pmid:11407875
- 37. Moshage HJ, Janssen JA, Franssen JH, Hafkenscheid JC, Yap SH. Study of the molecular mechanism of decreased liver synthesis of albumin in inflammation. J Clin Invest. 1987;79:1635–1641. pmid:3584463
- 38. Vincent JL, Dubois MJ, Navickis RJ, Wilkes MM. Hypoalbuminemia in acute illness: is there a rationale for intervention? A meta-analysis of cohort studies and controlled trials. Ann Surg. 2003;237:319–334. pmid:12616115