Skip to main content
Browse Subject Areas

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

A risk-model for hospital mortality among patients with severe sepsis or septic shock based on German national administrative claims data

  • Daniel Schwarzkopf ,

    Roles Conceptualization, Data curation, Formal analysis, Methodology, Project administration, Visualization, Writing – original draft

    Affiliation Integrated Research and Treatment Center–Center for Sepsis Control and Care, Jena University Hospital, Jena, Germany

  • Carolin Fleischmann-Struzek,

    Roles Conceptualization, Methodology, Project administration, Writing – review & editing

    Affiliation Integrated Research and Treatment Center–Center for Sepsis Control and Care, Jena University Hospital, Jena, Germany

  • Hendrik Rüddel,

    Roles Conceptualization, Methodology, Writing – review & editing

    Affiliations Integrated Research and Treatment Center–Center for Sepsis Control and Care, Jena University Hospital, Jena, Germany, Department of Anesthesiology and Intensive Care Medicine, Jena University Hospital, Jena, Germany

  • Konrad Reinhart,

    Roles Conceptualization, Funding acquisition, Resources, Supervision, Writing – review & editing

    Affiliations Integrated Research and Treatment Center–Center for Sepsis Control and Care, Jena University Hospital, Jena, Germany, Department of Anesthesiology and Intensive Care Medicine, Jena University Hospital, Jena, Germany

  • Daniel O. Thomas-Rüddel

    Roles Conceptualization, Data curation, Formal analysis, Methodology, Project administration, Writing – review & editing

    Affiliations Integrated Research and Treatment Center–Center for Sepsis Control and Care, Jena University Hospital, Jena, Germany, Department of Anesthesiology and Intensive Care Medicine, Jena University Hospital, Jena, Germany



Sepsis is a major cause of preventable deaths in hospitals. Feasible and valid methods for comparing quality of sepsis care between hospitals are needed. The aim of this study was to develop a risk-adjustment model suitable for comparing sepsis-related mortality between German hospitals.


We developed a risk-model using national German claims data. Since these data are available with a time-lag of 1.5 years only, the stability of the model across time was investigated. The model was derived from inpatient cases with severe sepsis or septic shock treated in 2013 using logistic regression with backward selection and generalized estimating equations to correct for clustering. It was validated among cases treated in 2015. Finally, the model development was repeated in 2015. To investigate secular changes, the risk-adjusted trajectory of mortality across the years 2010–2015 was analyzed.


The 2013 deviation sample consisted of 113,750 cases; the 2015 validation sample consisted of 134,851 cases. The model developed in 2013 showed good validity regarding discrimination (AUC = 0.74), calibration (observed mortality in 1st and 10th risk-decile: 11%-78%), and fit (R2 = 0.16). Validity remained stable when the model was applied to 2015 (AUC = 0.74, 1st and 10th risk-decile: 10%-77%, R2 = 0.17). There was no indication of overfitting of the model. The final model developed in year 2015 contained 40 risk-factors. Between 2010 and 2015 hospital mortality in sepsis decreased from 48% to 42%. Adjusted for risk-factors the trajectory of decrease was still significant.


The risk-model shows good predictive validity and stability across time. The model is suitable to be used as an external algorithm for comparing risk-adjusted sepsis mortality among German hospitals or regions based on administrative claims data, but secular changes need to be taken into account when interpreting risk-adjusted mortality.


Sepsis is the major cause of death from infectious diseases [1]. Most of these deaths are considered to be preventable [24] and numerous quality improvement initiatives demonstrated that sepsis-related mortality can be considerably reduced [57]. Recognizing existing severe shortcomings in prevention and treatment of sepsis, a WHO resolution released in May 2017 urged the member states to develop national policies to improve sepsis care [4].

There is consensus that valid measurement of provider performance is central to improve quality of care [8]. Obtaining prospective clinical data for performance measurement is costly and possibly prevents hospitals from participating in quality improvement [9, 10]. Administrative or claims data are readily available, include all patients diagnosed with the condition of interest, and are widely used for public reporting of hospital performance [11, 12]. Administrative claims are already used to obtain inpatient quality indicators–among them sepsis-related mortality–within a voluntary quality collaborative involving more than 300 German hospitals [13].

Differences in case-mix between hospitals might bias comparisons of raw mortality rates. Thus, statistical risk-adjustment needs to be applied [14, 15]. Since the existing German voluntary sepsis quality indicator is only adjusted for age and gender, the validity of hospital comparisons is questionable. An US-American risk-model for inpatient sepsis mortality solely based on claims data has recently been developed [16], but as such risk-models are not valid if used outside the population they were designed for, a similar model needs to be developed for Germany. The aim of this study was therefore to develop a risk-model suitable for risk-adjusted comparison of sepsis-related mortality between German hospitals that is solely based on administrative claims data.

Materials and methods

The Agency for Health Care Research and Quality Improvement (AHRQ) of the US Department for Health and Human Services provides hospitals with a methodology for claims-based measurement and risk-adjusted comparison of mortality in several conditions [17]. Risk-models are developed based on the Nationwide Inpatient Sample [18] and can be used as an external risk adjustment algorithm by US hospitals to compare their performance against the national standard [19, 20]. We adopted this methodology to develop a risk-model on sepsis-related mortality using German national claims data [21].

Data source

This study uses German diagnosis related groups (DRG) data of patients treated between 2010 and 2015 [21]. By federal law on reimbursement of health care providers (§21 Krankenhausentgeltgesetz [KhEntgG]), all hospitals are obliged to annually provide their DRG data to a federal administrative body. With a time-lag of 1.5 years the data of all cases are made available for scientific analysis in anonymized form by the Federal Statistical Office (Deutsches Statistisches Bundesamt) via remote computing. German DRG data contain information on patient demographics, primary and secondary diagnoses (ICD-10-GM: International Classification of Diseases and Related Health Problems- German Modification), procedures and general medical measures (OPS: Operationen und Prozedurenschlüssel [German Procedure Classification]), treating departments, hospital length of stay, type of admission, and type of discharge. ICD-10 codes have no time-flag, OPS codes have a time-flag but this mostly indicates rather the time of coding than the time the procedure was performed. Hospitals are identified in the DRG data by a unique institutional identifier (IK: Institutionskennzeichen). More than one hospital site of the same institution might use the same IK, but typically these sites are organizationally linked and mutually dependent.

Study population

The target population were cases with severe sepsis or septic shock according to modified American College of Chest Physicians/Society of Critical Care Medicine criteria [22], identified by presence of the respective clinical ICD-10 codes (severe sepsis: R65.1, septic shock: R57.2), and of at least 15 years of age. We focused on severe sepsis and septic shock since sepsis without organ dysfunction is excluded from recent Sepsis-3 consensus definitions [1].

Derivation and validation cohorts.

Because of the time-lag of 1.5 years for availability of national DRG data, the validity of a risk-model for predicting mortality in cases treated two years after its development needed to be proven. Therefore, a risk-model was developed based on cases treated in 2013 (derivation cohort) and tested for its validity in 2014 and 2015 (validation cohorts). The final model was then developed based on cases treated in 2015. Additionally, secular trends in sepsis-related mortality were investigated using cases treated between 2010 and 2015.


The outcome was all-cause hospital mortality among patients with severe sepsis or septic shock. Hospital mortality could not be corrected for transfer to other hospitals since anonymized data did not allow identification of individual patients with several linked hospital stays. Cases with admission by transfer from another hospital and cases with discharge to another hospital were not excluded from the analysis.

Model derivation

Predictors of mortality.

Based on clinical reasoning and existing research, candidate variables were chosen from patient demographics [16, 23], hospital admission type [24], clinical characteristics of the infection [2325], comorbidities [16, 23], and specific procedures.

  1. Patient demographics included gender and age. To allow for non-linear effects of age, the quadratic and cubic polynomials were included. Age was transformed by mean-centering and standardization to decades (aget = [age– 70])/10).
  2. Hospital admission type was categorized in ‘emergency admission’, ‘referral by physician or dentist’, ‘transfer from another hospital with length of stay ≥ 24h’, and ‘transfer from rehabilitation clinic or another hospital with length of stay <24h’. Additionally, an indicator variable for admission by a surgical department was defined based on standardized codes for medical specialty of a department to identify the subgroup of surgical patients.
  3. Clinical characteristics of the infection included focus of infection defined by presence of specific ICD-10 codes, and presence of an explicit sepsis code as a primary diagnosis. Focus of infection codes were chosen based on existing international literature [2628] and clinical knowledge. Two junior physicians independently defined indicator variables for specific foci and consensus was created in discussions with two experienced intensivists. A primary diagnosis of sepsis was defined if an explicit sepsis code (A40.–A41., R57.2) was present as primary diagnosis. Finally, infection by multi-resistant pathogens was defined by presence of an OPS-code for treatment of multiresistant pathogens.
  4. Comorbidities were defined by the categories of the Charlson Comorbidity Index (CCI) and the Elixhauser Comorbidity Index (ECI) [29, 30] based on a German adaptation of a previously developed ICD-10 coding algorithm [31, 32]. Duplicate CCI and ECI categories were excluded. Additional categories for leukemia, congenital and acquired asplenia were introduced.
  5. Specific procedures defined by respective OPS codes were used to identify patient subgroups with especially high risk of dying: patients receiving palliative care, treatment of stroke, and chemotherapy.

Univariate relations between predictors were investigated, to avoid problems of multicollinearity. Highly correlated predictors were collapsed or one predictor was excluded based on clinical judgment and univariate relations to hospital mortality. Only conditions with incidence of at least 1% were considered as candidate variables [33, 34]. S1 Table presents the definitions of candidate variables.

Model development.

Risk factors were first selected from the set of candidate variables by a standard logistic regression model with backward elimination applied to the 2013 derivation cohort. Since patients with septic shock are a distinctive subgroup with higher mortality, we wanted to make the model suitable also for comparing hospital mortality within the subgroups of cases with or without septic shock. Risk-factors might have different effects within these respective subgroups, which can be modeled by statistical interaction effects. Therefore, interaction effects of the selected predictors with the presence of a diagnosis of septic shock were also included and backward-selected in a second modeling step. This was done to increase the validity of the model when using it within the subgroups of patients without septic shock or with septic shock. Given the large sample size, the exclusion criterion was set to p > 0.0003 based on a power-analysis (S1 Appendix). Resembling the approach of the AHRQ [20], the selected risk-model was refitted by a logistic model using generalized estimating equations (GEE) with an exchangeable covariance matrix to control for clustering of cases into hospitals [35]. After validating the 2013 risk-model, the full model development process was repeated based on cases treated in 2015 to obtain a model suitable for provider comparison based on most-recent data.

Model validation

Predictive performance of the model.

To evaluate the predictive performance of the model identified in the derivation cohort (“derivation model”), the area under the curve (AUC) was used as measure of discrimination, the distribution of observed mortality across the deciles of predicted risk was used as a measure of calibration, and the squared Pearson correlation (R2) between hospital mortality and the log-odds of predicted mortality was used as measure of explained variation [36].

Stability of the model.

Because of the 1.5 years time-lag in availability of national DRG data, the stability of the risk-models over time had to be tested. The validity of the derivation model was assessed in the 2014 and 2015 validation cohorts, respectively. This was done by using the identified risk-factors and coefficients estimates to predict mortality risk and log-odds of mortality risk among the cases of the validation cohorts. These predicted values were used to calculate AUC, distribution of observed mortality across deciles of risk, and R2 within the validation cohorts. Additionally, overfitting indices were calculated to assess if too many risk-factors were included in the models [33]. For this purpose, predicted log-odds of mortality risk were introduced as sole predictor in logistic regression models using GEE within each validation cohort. Coefficients estimates of intercepts and slopes were used to assess overfitting. Because of the known secular trend of decreasing sepsis mortality [3740], the intercepts were expected to deviate from the ideal value of 0. A substantial deviation of the slope from its ideal value of 1 would be an indicator of overfitting. Analyses were conducted using SAS® software, version 9.2 (SAS Institute, Cary NC).


Table 1 provides an overview of the study samples between 2010 and 2015. Incidence of severe sepsis/septic shock increased from 0.5% to 0.73% of all hospital episodes. After exclusion of cases with age <15 years, 113,750 cases in the derivation cohort of 2013 and 134,851 cases in the validation cohort of 2015 remained for analysis (S1 Fig). Proportion of cases with septic shock increased from 25.6% in 2010 to 30.2% in 2015. Hospital mortality among cases with severe sepsis or septic shock decreased from 48.4% to 42%.

Table 1. Sample of cases with severe sepsis or septic shock in German national diagnostic related groups data.

After univariate analyses, clinical review of candidate variables and backward selection in the 2013 derivation cohort, 42 risk-factors and 13 interaction effects with septic shock were identified (S2 Table). The model had good discrimination (AUC = 0.737; 95% CI: 0.734, 0.74; p < 0.001), calibration (mortality rate in lowest risk-decile and highest decile: 0.11–0.78) and fit (R2 = 0.16). Concerning stability, the model showed similar validity statistics in the two validation cohorts of 2014 and 2015 (Table 2). There was no indication of overfitting of the model. As expected based on known secular trends, the intercepts deviated from 0, but the slopes were close to 1. A secular trend is also apparent in the calibration plot (Fig 1), where mortality observed in 2015 is slightly lower than mortality predicted from the 2013 model. Beside this, the 2013 model was well calibrated across the deciles of risk also when used on data of 2015.

Fig 1. Calibration of the model derived in 2013 for predicting hospital mortality in cases with severe sepsis or septic shock treated in 2015.

The gap between predicted and observed mortality (M±SD = 2.7%±0.7%) results from a secular trend with decreasing mortality (compare Fig 2).

Table 2. Validity of logistic regression model fitted by generalized estimation equations for hospital mortality in cases with severe sepsis or septic shock in derivation and validation samples.

The model development was repeated for cases treated in 2015, identifying 40 risk-factors and 11 interaction effects with diagnosis of septic shock. Table 3 shows the risk-factors with coefficients estimates and odds ratios. Selected risk-factors and coefficient estimates closely resemble the model selected in 2013 (S2 Table).

Table 3. Coefficients estimates of logistic regression model using generalized estimation equations for hospital mortality in cases with severe sepsis or septic shock treated in 2015.

Fig 2 presents the secular trends between 2010 and 2015 in mortality for cases with severe sepsis and cases with septic shock. The decreasing trajectories remained significant also when controlled for differences in risk-factors.

Fig 2. Secular trends for hospital mortality between 2010–2015 in cases with severe sepsis or septic shock.

Dashed lines present odds ratios compared to 2010 without controlling for mortality risk-factors, solid lines present odds ratios compared to 2010 while controlling for risk-factors. Results based on a logistic regression using generalized estimating equations.


This study developed a risk-model based on German national DRG data, which is suitable for comparing sepsis-related mortality between German hospitals. To our knowledge this is the first European claims-based risk-model for sepsis-related mortality. It is also the first risk-model using a full national sample of inpatient cases, securing validity of its use among German hospitals. The model showed good discrimination, calibration and fit, comparable to previous claims-based models for mortality in sepsis and several other conditions [16, 23, 33, 34, 41, 42]. Additionally, the stability of the model across a time-lag of two years was proven.

We identified risk-factors among predispositions and characteristics of the infection. Beside these, severity of acute illness is considered to be a major risk-factor for sepsis-mortality [25, 43, 44]. Two previous claims-based risk-models included measures of presenting acute severity of illness, like use of vasopressors, early ventilation, and ICU care at admission [16, 23]. However, we did not include such indicators because procedure codes in German DRG data have no reliable time-flag to identify use of medication or procedures closely following hospital admission. Later use of organ support or ICU treatment cannot strictly be classified as patient-related risk-factors and might contain treatment-related complications biasing the risk-adjustment [45]. Yet, discrimination of our risk-model was only slightly lower compared to these previous models. Additionally, risk-models based on clinical data and incorporating acute illness severity did not show substantially higher discrimination compared to our new model [25, 44].

We observed some paradoxical protective effects of comorbidities like drug abuse, depression or complicated hypotension. Similar protective effects were also reported in the study by Ford et al [16] as well as in the developmental study of the Elixhauser comorbidity index [30]. Elixhauser et al. [30] explain these paradoxical effects by a coding bias caused by increased coding of less severe comorbidities in patients with few severe comorbidities and lower risk of mortality. Additionally, in patients that die quickly there might also be less coding of less severe diseases.

Since there is a time-lag of 1.5 years in availability of the national German DRG data, we needed to proof the stability of the model over time. A model developed using data of 2013 did not decrease in predictive validity when applied to the cases of 2015. This indicates that a model based on two years old national DRG data is suitable to be used for risk-adjustment in recently treated cases.

A secular trend of decreasing sepsis-related mortality has been reported for several developed countries [3739, 46]. These changes probably result from the so called Will Rogers phenomenon [47]. It states that decreased mortality might result from increased awareness for a disease and increased diagnosis of less severely ill cases. We could show that a trajectory of decreased mortality was also existent when an adjustment for risk-factors was done. This indicates that changes might not solely result from the Will Rogers phenomenon and that modest but true improvements in quality of sepsis care might have occurred in Germany.

Based on the identified risk-factors and their coefficients estimates, the standardized mortality ratio (“observed over expected ratio”) and risk standardized mortality rate along with their confidence limits can be obtained to measure provider performance in sepsis care [19, 48]. Additionally, a smoothed risk-adjusted rate can be calculated that incorporates a shrinkage factor which corrects for unreliability in estimates based on small case numbers [20]. Relying solely on readily available DRG data, risk-adjusted indicators can be obtained with minimal efforts. The developed risk-model is used for internal benchmark and public reporting of sepsis-related mortality of 75 hospitals participating in the German Quality Network Sepsis, a voluntary quality initiative founded in 2016. Likewise, every German hospital could use the risk-model to compare its performance to the German average. Additionally, the model can be used for risk-adjusted comparison of mortality between German counties or states. The risk-model is not suitable as a clinical score for predicting risk of death for individual patients, since it does not incorporate relevant clinical and laboratory variables.

This study has several limitations. Administrative data have certain shortcomings that might bias hospital comparisons, like payment-related incentives for coding, over or under-coding of conditions or risk-factors, and inconsistencies in coding between hospitals [11]. The problem of under-coding of cases with sepsis has been reported in several studies from the USA, Sweden, Canada, and Australia [49]. This was also shown in a recent pilot-study in a German university hospital [50]. It is currently unknown how this might affect risk-adjusted hospital comparisons. Since no appropriate German clinical data bases or registries for patients with sepsis exist, we could not validate our risk-model against a risk-model based on validated clinical data, as was done in previous studies [23, 33, 34, 41, 51]. Additional limitations are caused by specific shortcomings of German DRG data. Since patients are anonymized, related hospital admissions could not be identified. Additionally, only hospital mortality but no mortality measured after a given time period (e.g. 90-days) was available. Thus, differences in discharge strategies between hospitals might bias performance estimates [52, 53]. ICD-10 codes are not classified as present at admission, which leads to possible bias [45]. We focused on conditions that most likely occur as comorbidities by relying on established indices [29, 30]. Finally, using the model derived from two year old national DRG data on recently treated cases might result in underestimated–or less likely overestimated–risk-adjusted mortality because of unknown secular changes.


We developed a risk model for mortality in severe sepsis and septic shock based on claims data. The model showed good predictive validity, comparable to previous claims-based models as well as to clinical risk scores. The model can be used as an external risk-adjustment algorithm for evaluating and comparing sepsis-related mortality in German hospitals with minimal effort and costs. The availability of the model can promote measurement and improvement of quality of sepsis care in Germany. Future studies should investigate the validity of coding of risk-factors in German claims data and compare the validity of a claims-based model against models derived from validated clinical data.

Supporting information

S1 Appendix. Power calculation to determine the exclusion criterion in backward selection of risk-factors.


S1 Table. Definitions of candidate variables for the risk-model.


S2 Table. Coefficients estimates of logistic regression model using generalized estimation equations for hospital mortality in cases with severe sepsis or septic shock treated in 2013.



We thank Jürgen Stausberg for providing the definitions for Charlson and Elixhauser Comorbidity Indices adapted to the German ICD-10.


  1. 1. Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA-J Am Med Assoc. 2016;315(8):801–10. WOS:000370700500015. pmid:26903338
  2. 2. Goodwin APL, Srivastava V, Shotton H, Protopapa K, Butt A, Mason M. Just say sepsis! A review of the process of care received by patients with sepsis [PDF]. 2015 [cited 2017 15 January]. Available from:
  3. 3. Clinical Excellence Commission. Recognition and Management of Sepsis [PDF]. 2012 [cited 2017 January 15th]. Available from:
  4. 4. World Health Assembly Executive Board. EB140.R5 Improving the prevention, diagnosis and management of sepsis [PDF]. 2017 [cited 2017 October 26]. Available from:
  5. 5. Damiani E, Donati A, Serafini G, Rinaldi L, Adrario E, Pelaia P, et al. Effect of Performance Improvement Programs on Compliance with Sepsis Bundles and Mortality: A Systematic Review and Meta-Analysis of Observational Studies. PLOS ONE. 2015;10(5):e0125827. pmid:25946168
  6. 6. Levy MM, Rhodes A, Phillips GS, Townsend SR, Schorr CA, Beale R, et al. Surviving Sepsis Campaign: Association Between Performance Metrics and Outcomes in a 7.5-Year Study. Crit Care Med. 2015;43(1):3–12. WOS:000346409100023. pmid:25275252
  7. 7. Scheer CS, Fuchs C, Kuhn S-O, Vollmer M, Rehberg S, Friesecke S, et al. Quality Improvement Initiative for Severe Sepsis and Septic Shock Reduces 90-Day Mortality: A 7.5-Year Observational Study. Crit Care Med. 2016;Publish Ahead of Print. 00003246-900000000-96765. pmid:27661863
  8. 8. Brook RH, McGlynn EA, Cleary PD. Quality of health care .2. Measuring quality of care. N Engl J Med. 1996;335(13):966–70. WOS:A1996VK14800011. pmid:8782507
  9. 9. Khuri SF, Daley J, Henderson W, Hur K, Demakis J, Aust JB, et al. The Department of Veterans Affairs' NSQIP: the first national, validated, outcome-based, risk-adjusted, and peer-controlled program for the measurement and enhancement of the quality of surgical care. National VA Surgical Quality Improvement Program. Ann Surg. 1998;228(4):491–507. PMC1191523. pmid:9790339
  10. 10. Bloos F, Rüddel H, Thomas-Rüddel D, Schwarzkopf D, Pausch C, Harbarth S, et al. Effect of a multifaceted educational intervention for anti-infectious measures on sepsis mortality: a cluster randomized trial. Intensive Care Med. 2017;43(11):1602–12. pmid:28466151
  11. 11. Iezzoni LI. Risk adjustment. In: Smith PC, Mossialos E, Papanicolas I, Leatherman S, editors. Performance measurement for health system improvement: Experiences, challenges and prospects. New York: Cambridge University Press; 2009. p. 251–85.
  12. 12. Christianson JB, Volmar KM, Alexander J, Scanlon DP. A Report Card on Provider Report Cards: Current Status of the Health Care Transparency Movement. J Gen Intern Med. 2010;25(11):1235–41. PMC2947645. pmid:20625849
  13. 13. Initiative Qualitätsmedizin. Association Initiative Quality in Medicine [PDF]. 2017 [cited 2017 November 3]. Available from:
  15. 15. Jencks SF, Daley J, Draper D, Thomas N, Lenhart G, Walker J. INTERPRETING HOSPITAL MORTALITY DATA—THE ROLE OF CLINICAL RISK ADJUSTMENT. JAMA-J Am Med Assoc. 1988;260(24):3611–6. WOS:A1988R364400031.
  16. 16. Ford DW, Goodwin AJ, Simpson AN, Johnson E, Nadig N, Simpson KN. A Severe Sepsis Mortality Prediction Model and Score for Use With Administrative Data. Crit Care Med. 2016;44(2):319–27. 00003246-201602000-00010. pmid:26496452
  17. 17. Agency for Health Care Research and Quality Improvement. Inpatient Quality Indicators [PDF]. 2015 [cited 2017 November 1]. Available from:
  18. 18. Healthcare Cost and Utilization Project. Overview of the National (Nationwide) Inpatient Sample (NIS) 2017 [cited 2017 October 30]. Available from:
  19. 19. DeLong ER, Peterson ED, DeLong DM, Muhlbaier LH, Hackett S, Mark DB. Comparing risk-adjustment methods for provider profiling. Stat Med. 1997;16(23):2645–64. WOS:A1997YG63300002. pmid:9421867
  20. 20. Agency for Health Care Research and Quality Improvement. Quality indicator emprical methods—revised November 2014 [PDF]. 2015 [cited 2017 13 Oct]. Available from:
  21. 21. RDC of the Federal Statistical Office and Statistical Offices of the Länder. DRG-Statistics 2010–2015, own calculations.
  22. 22. Bone RC, Balk RA, Cerra FB, Dellinger RP, Fein AM, Knaus WA, et al. American College of Chest Physicians/Society of Critical Care Medicine consensus conference: Definitions for sepsis and organ failure and guidelines for use of innovative therapies in sepsis. Crit Care Med. 1992;20(6):864–74. WOS:A1992HX52000025. pmid:1597042
  23. 23. Lagu T, Lindenauer PK, Rothberg MB, Nathanson BH, Pekow PS, Steingrub JS, et al. Development and validation of a model that uses enhanced administrative data to predict mortality in patients with sepsis. Crit Care Med. 2011;39(11):2425–30. WOS:000296509500006. pmid:22005222
  24. 24. Mohr NM, Harland KK, Shane DM, Ahmed A, Fuller BM, Torner JC. Inter-hospital transfer is associated with increased mortality and costs in severe sepsis and septic shock: An instrumental variables approach. J Crit Care. 2016;36:187–94. WOS:000393078100033. pmid:27546770
  25. 25. Osborn TM, Phillips G, Lemeshow S, Townsend S, Schorr CA, Levy MM, et al. Sepsis Severity Score: An Internationally Derived Scoring System From the Surviving Sepsis Campaign Database*. Crit Care Med. 2014;42(9):1969–76. 00003246-201409000-00001. pmid:24919160
  26. 26. Angus DC, Linde-Zwirble WT, Lidicker J, Clermont G, Carcillo J, Pinsky MR. Epidemiology of severe sepsis in the United States: Analysis of incidence, outcome, and associated costs of care. Crit Care Med. 2001;29(7):1303–10. WOS:000169944400002. pmid:11445675
  27. 27. Wang HE, Shapiro NI, Angus DC, Yealy DM. National estimates of severe sepsis in United States emergency departments. Crit Care Med. 2007;35(8):1928–36. WOS:000248251600016. pmid:17581480
  28. 28. Wilhelms SB, Huss FR, Granath G, Sjoberg F. Assessment of incidence of severe sepsis in Sweden using different ways of abstracting International Classification of Diseases codes: Difficulties with methods and interpretation of results. Crit Care Med. 2010;38(6):1442–9. WOS:000278231700008. pmid:20400903
  29. 29. Charlson ME, Pompei P, Ales KL, Mackenzie CR. A new method of classifying prognostic co-morbidity in longitudinal studies—Development and validation. J Chronic Dis. 1987;40(5):373–83. WOS:A1987G855900002. pmid:3558716
  30. 30. Elixhauser A, Steiner C, Harris DR, Coffey RN. Comorbidity measures for use with administrative data. Med Care. 1998;36(1):8–27. WOS:000071181500004. pmid:9431328
  31. 31. Quan HD, Sundararajan V, Halfon P, Fong A, Burnand B, Luthi JC, et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care. 2005;43(11):1130–9. WOS:000233268500010. pmid:16224307
  32. 32. Hagn S. Vergleich verschiedener Komorbiditäts-Scores in Routinedaten der stationären Versorgung [PDF]. München, Germany: Institut für Medizinische Informationsverarbeitung, Biometrie und Epidemiologie der Universität München; 2014 [cited 2017 24 November]. Available from:
  33. 33. Krumholz HM, Wang Y, Mattera JA, Wang YF, Han LF, Ingber MJ, et al. An administrative claims model suitable for profiling hospital performance based on 30-day mortality rates among patients with an acute myocardial infarction. Circulation. 2006;113(13):1683–92. WOS:000236540700011. pmid:16549637
  34. 34. Bratzler DW, Normand SLT, Wang Y, O'Donnell WJ, Metersky M, Han LF, et al. An Administrative Claims Model for Profiling Hospital 30-Day Mortality Rates for Pneumonia Patients. Plos One. 2011;6(4). doi: ARTN e17401 ISI:000289404700004.
  35. 35. Liang K-Y, Zeger SL. Longitudinal data analysis using generalized linear models. Biometrika. 1986;73(1):13–22.
  36. 36. Mittlbock M, Schemper M. Explained variation for logistic regression. Stat Med. 1996;15(19):1987–97. WOS:A1996VK32500001. pmid:8896134
  37. 37. Fleischmann C, Thomas-Rueddel DO, Hartmann M, Hartog CS, Welte T, Heublein S, et al. Hospital Incidence and Mortality Rates of Sepsis An Analysis of Hospital Episode (DRG) Statistics in Germany From 2007 to 2013. Dtsch Arztebl Int. 2016;113(10):159-+. WOS:000374112100001. pmid:27010950
  38. 38. Kaukonen KM, Bailey M, Suzuki S, Pilcher D, Bellomo R. Mortality Related to Severe Sepsis and Septic Shock Among Critically III Patients in Australia and New Zealand, 2000–2012. JAMA-J Am Med Assoc. 2014;311(13):1308–16. WOS:000333645400014. pmid:24638143
  39. 39. Shankar-Hari M, Harrison DA, Rowan KM. Differences in Impact of Definitional Elements on Mortality Precludes International Comparisons of Sepsis Epidemiology-A Cohort Study Illustrating the Need for Standardized Reporting. Crit Care Med. 2016;Publish Ahead of Print. 00003246-900000000-96880. pmid:27352126
  40. 40. Lagu T, Rothberg MB, Shieh MS, Pekow PS, Steingrub JS, Lindenauer PK. Hospitalizations, costs, and outcomes of severe sepsis in the United States 2003 to 2007. Crit Care Med. 2012;40(3):754–61. WOS:000300532800006. pmid:21963582
  41. 41. Krumholz HM, Wang Y, Mattera JA, Wang YF, Han LF, Ingber MJ, et al. An administrative claims model suitable for profiling hospital performance based on 30-day mortality rates among patients with heart failure. Circulation. 2006;113(13):1693–701. WOS:000236540700012. pmid:16549636
  42. 42. Lindenauer PK, Grosso LM, Wang C, Wang Y, Krishnan JA, Lee TA, et al. Development, validation, and results of a risk-standardized measure of hospital 30-day mortality for patients with exacerbation of chronic obstructive pulmonary disease. J Hosp Med. 2013;8(8):428–35. pmid:23913593
  43. 43. Vincent JL, Moreno R, Takala J, Willatts S, DeMendonca A, Bruining H, et al. The SOFA (sepsis-related organ failure assessment) score to describe organ dysfunction/failure. Intensive Care Med. 1996;22(7):707–10. WOS:A1996UZ45200017. pmid:8844239
  44. 44. Moreno RP, Metnitz B, Adler L, Hoechtl A, Bauer P, Metnitz PGH, et al. Sepsis mortality prediction based on predisposition, infection and response. Intensive Care Med. 2008;34(3):496–504. WOS:000253200800016. pmid:18060541
  45. 45. Krumholz HM, Brindis RG, Brush JE, Cohen DJ, Epstein AJ, Furie K, et al. Standards for statistical models used for public reporting of health outcomes—An American Heart Association scientific statement from the quality of care and outcomes research interdisciplinary writing group—Cosponsored by the Council on Epidemiology and Prevention and the Stroke Council—Endorsed by the American College of Cardiology Foundation. Circulation. 2006;113(3):456–62. ISI:000234830900022. pmid:16365198
  46. 46. Gaieski DF, Edwards JM, Kallan MJ, Carr BG. Benchmarking the Incidence and Mortality of Severe Sepsis in the United States*. Crit Care Med. 2013;41(5):1167–74. 00003246-201305000-00002. pmid:23442987
  47. 47. Iwashyna TJ, Angus DC. Declining Case Fatality Rates for Severe Sepsis Good Data Bring Good News With Ambiguous Implications. JAMA-J Am Med Assoc. 2014;311(13):1295–7. WOS:000333645400011. pmid:24638109
  48. 48. Hosmer DW, Lemeshow S. Confidence-interval estimates of an index of quality performance based on logistic regression models. Stat Med. 1995;14(19):2161–72. WOS:A1995RX95900008. pmid:8552894
  49. 49. Jolley RJ, Sawka KJ, Yergens DW, Quan H, Jette N, Doig CJ. Validity of administrative data in recording sepsis: a systematic review. Crit Care. 2015;19:12.
  50. 50. Fleischmann C, Schettler A, Thomas-Rueddel DO, Schwarzkopf D, Stacke A, Haas C, et al. Validation of ICD code abstraction strategies for sepsis in administrative data. Infection; Aug2015. p. S39–S40.
  51. 51. Shahian DM, Silverstein T, Lovett AF, Wolf RE, Normand SLT. Comparison of clinical and administrative data sources for hospital coronary artery bypass graft surgery report cards. Circulation. 2007;115(12):1518–27. WOS:000245211300004. pmid:17353447
  52. 52. Drye E, Normand S-LT, Wang Y, Ross JS, Schreiner GC, Han L, et al. Comparison of Hospital Risk-Standardized Mortality Rates using In-hospital and 30-Day Models: Implications for Hospital Profiling. Ann Intern Med. 2012;156(1 Pt 1):19–26. pmid:22213491