Long-term survival outcome of critically ill patients is important in assessing effectiveness of new treatments and making treatment decisions. We developed a prognostic model for estimation of long-term survival of critically ill patients.
Methodology and Principal Findings
This was a retrospective linked data cohort study involving 11,930 critically ill patients who survived more than 5 days in a university teaching hospital in Western Australia. Older age, male gender, co-morbidities, severe acute illness as measured by Acute Physiology and Chronic Health Evaluation II predicted mortality, and more days of vasopressor or inotropic support, mechanical ventilation, and hemofiltration within the first 5 days of intensive care unit admission were associated with a worse long-term survival up to 15 years after the onset of critical illness. Among these seven pre-selected predictors, age (explained 50% of the variability of the model, hazard ratio [HR] between 80 and 60 years old = 1.95) and co-morbidity (explained 27% of the variability, HR between Charlson co-morbidity index 5 and 0 = 2.15) were the most important determinants. A nomogram based on the pre-selected predictors is provided to allow estimation of the median survival time and also the 1-year, 3-year, 5-year, 10-year, and 15-year survival probabilities for a patient. The discrimination (adjusted c-index = 0.757, 95% confidence interval 0.745–0.769) and calibration of this prognostic model were acceptable.
Age, gender, co-morbidities, severity of acute illness, and the intensity and duration of intensive care therapy can be used to estimate long-term survival of critically ill patients. Age and co-morbidity are the most important determinants of long-term prognosis of critically ill patients.
Citation: Ho KM, Knuiman M, Finn J, Webb SA (2008) Estimating Long-Term Survival of Critically Ill Patients: The PREDICT Model. PLoS ONE 3(9): e3226. doi:10.1371/journal.pone.0003226
Editor: Jeffrey A. Gold, Oregon Health & Science University, United States of America
Received: July 27, 2008; Accepted: August 25, 2008; Published: September 17, 2008
Copyright: © 2008 Ho et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: A grant was received from BUPA to provide support for the cost of linking the databases used in this study. The funding agency has no involvement in the study design, collection of data, analysis and interpretation of data, writing of the report, or in the decision to submit the article for publication.
Competing interests: The authors have declared that no competing interests exist.
Demand for intensive care unit (ICU) services is increasing , and at a rate that is higher than the average for all health care services . Increase in treatment and monitoring technology, patients' expectations, and ageing population all contribute to this increased demand for intensive care services . Indeed, intensive care is increasingly being provided to older and sicker patients, whom in the past were not treated in intensive care . Intensive care services accounted for 10% of the US$2.1 trillion total health expenditures on health care in the United States in 2006  and has been estimated to cost more than £700 million in the United Kingdom in 1999 . The cost of intensive care services coupled with increasing demand provides the rationale for improved modelling of outcomes of critically ill patients.
Long-term survival after critical illness is increasingly being recognized as an important outcome in assessing effectiveness of new therapy . In order to control for confounding and bias in assessing long-term survival of critically ill patients in a clinical trial, a risk adjustment tool that can objectively estimate long-term survival is essential. From a clinical perspective, many patients and clinicians are also interested in knowing the long-term survival outcome after critical illness, in addition to other information such as burden of treatment and quality of life after recovery, when making treatment decisions in the ICU. Although many clinicians may foretell patient hospital survival outcome more accurately than some objective prognostic models , treatment decisions made by clinicians do vary considerably with their practice style and work experience –. The strategy of continuing intensive treatment for all patients until death will reduce the need for patients and clinicians to make difficult treatment decisions and may improve the survival time of some. This strategy is, however, expensive and undesirable by imposing excessive burden of treatment on those who have a very poor prognosis . For example, initiating acute renal replacement therapy in critically ill patients with less than 10% probability of 6-month survival was estimated to cost US$274,000 (£137,000) per quality-adjusted life year saved .
The SUPPORT investigators from the United States and Wright et al. from the United Kingdom published two prognostic models that were based on age, severity of acute illness and admission diagnosis to estimate 6-month and 5-year survival of critically ill patients, respectively , . The utility of latter model is, however, limited by its ability to classify 5-year survival probabilities only into three risk categories when the calculated risk score is either <70, 70–80, or >80 . This model also did not consider the potential effect of detailed co-morbidity data on long-term survival of critically ill patients beyond the usual assessment included in the Acute Physiology and Chronic Health Evaluation (APACHE) score , . There is currently no prognostic model that is available to estimate the survival of critically ill patients beyond 5 years after the onset of critical illness. Furthermore, the relative importance of age, co-morbidity, and severity of acute illness in determining long-term prognosis of critically ill patients also remains unknown. In this study we examined the long-term survival of 11,930 critically ill adult patients who survived at least 5 days and developed a new prognostic model (Predicted Risk, Existing Diseases, and Intensive Care Therapy: the PREDICT model) to estimate their median survival time and long-term survival probabilities.
The characteristics of the cohort
This cohort study utilized the clinical database of the ICU at Royal Perth Hospital (RPH) in Western Australia. RPH is the largest tertiary university teaching hospital in Western Australia and the 22-bed ICU admits patients of all specialties except liver transplantation and captures over 40% of all critically ill patients in Western Australia. The database analyzed in this study includes details of all ICU admissions between 1989 and 2002, including demographic factors, admission diagnosis, admission source, severity of acute illness as measured by APACHE II scores based on the worst first 24-hour ICU data , daily organ failure assessment and supportive therapy required , and ICU and hospital survival outcome.
In this study the patients with a diagnosis excluded from the original APACHE II cohort (e.g. coronary artery graft surgery, burns, snake bite), those who resided outside Western Australia at the time of ICU admission (who could not be followed for survival), readmissions after the first ICU readmission, patients who were younger than 16 years old, and patients who did not survive more than 5 days during their hospitalization of the index ICU admission were excluded. The data were reviewed for internal consistency annually, and there were no patients with missing hospital mortality data. Some of the other details of this cohort have been described in our previous publications –.
The ICU clinical database was linked to the Western Australian hospital morbidity and mortality databases using probabilistic matching , providing information on patients' co-morbidities as recorded in all private and public hospital admissions including any prior ICU admissions up to 5 years before the index ICU admission. A relatively long five-year ‘look back’ period was chosen in this study to capture all existing co-morbidities of each patient. We ascertained the presence of co-morbidities in the Charlson co-morbidity index (Table 1) using the published ICD-9-CM and ICD-10-AM coding algorithms , . We did not assign a co-morbidity to a patient if that condition was diagnosed during the hospitalization of the index ICU admission. The proportions of invalid (false positive) and missed links (false negatives) between Western Australian hospital morbidity and mortality databases were evaluated several years ago, and both false positives and negatives were estimated to be 0.11% .
The survival status of the patients was assessed on 31 December 2003 and the length of follow up ranged from 1 year to 15 years with an average of 6 years. Western Australia is geographically isolated and has a very low rate of emigration (<0.03% in 2002), and as such, lost to long-term survival follow-up by the Western Australian mortality database is likely to be very low. The data analyzed had the patient name and address removed and the study was approved by the RPH Ethics Committee and the Western Australian Confidentiality of Health Information Committee (CHIC).
Development of the model
The prognostic model was fitted using Cox proportional hazards regression , restricting predictors to factors that were likely to be important predictors of long-term survival of hospitalized patients , , , . These pre-selected factors included age , , gender , APACHE II predicted mortality risk –, Charlson co-morbidity index , , days of mechanical ventilation, acute renal replacement therapy or hemofiltration, and vasopressor or inotropic therapy during the first 5 days of the index ICU admission . The APACHE II predicted mortality was chosen as a measure of severity of acute illness because it is widely used and summarizes the diagnosis, acute physiologic derangement within the first 24 hours of ICU admission, severe co-morbidities, and whether the ICU admission is after elective or emergency surgery. Our previous study also showed that the APACHE II predicted mortality has a very stable performance in this cohort over the past 10–15 years . Although age and severe co-morbidities are already used to calculate the APACHE II predicted mortality , these two factors may still have a profound effect on long-term survival over and beyond the weightings used in the APACHE II predicted mortality , , . As such, both age and Charlson co-morbidity index were used as separate predictors in additional to the APACHE II predicted mortality in this prognostic model. These seven predictors were also chosen because they are often recorded in the administrative databases of many ICUs, and as such, it is possible for other ICUs to validate this model using their data .
The proportional hazards assumption of the continuous predictors in the Cox model was assessed and found to be acceptable (Figure 1a, 1b, 1c). During the modelling process, we avoided categorizing continuous predictors ,  and allowed a non-linear relationship with hazard of death using a 6-knot restricted cubic spline function . The relative contribution of each predictor was assessed using the chi-square statistic minus the degrees of freedom . The discrimination performance of the model was assessed with the c-index, which is a generalization of the c-statistic or the area under the receiver-operating characteristic curve, allowing for censored data , . In this study, a c-index between 0.70 and 0.80 was regarded as acceptable and a c-index above 0.80 was regarded as excellent . Using the Design library in S-PLUS software (version 8.0, 2007. Insightful Corp., Seattle, Washington, USA), the c-index was computed and adjusted for optimism (arising from using the same data to develop the model and assess its performance) by a bootstrap technique to penalise for possible over-fitting, with 200 re-samples and at least 200 patients per risk group , . The bootstrapping technique was used in this study instead of splitting the data into development and validation data set because this method is regarded as most data ‘efficient’ and accurate in developing a prognostic model . Model calibration (similarity of predicted risks and proportions actually dying) was assessed graphically and used a bootstrap re-sampling to construct a bias-corrected calibration curve and its slope , . Nagelkerke's R2 (a generalized measure of the percentage of the variance in survival accounted for by the model) was computed to assess the overall performance of the model , . The performance of the model was assessed over the full 15 years of follow-up, when follow-up was restricted to a maximum of 5 years for each patient, and also when only patients admitted after 1997 were considered.
(a) Graph assessing the proportionality of hazards associated with severity of acute illness measured by the APACHE II predicted mortality risk categories (0–20%, 20–40%, 40–60%, 60–80%, 80–100%). (b) Graph assessing the proportionality of hazards associated with co-morbidities measured by Charlson co-morbidity index categories (0, 1, 2, 3, 4–5, >5). (c) Graph assessing the proportionality of hazards associated with age measured by age categories (16–30, 30–50, 50–60, 60–70, 70–80, >80 years old)
A nomogram was developed for the model that generates the median survival time and selected annual survival probabilities by adding up the scores for each of the seven predictors . The use of the nomogram and how each predictor may affect a patient's long-term prognosis is described for a selection of typical patient scenarios. In particular, these scenarios were selected to illustrate how the long-term prognosis of a patient can be different from the short-term prognosis. Nevertheless, the results of the nomogram should only be considered as an average estimate of patients with similar characteristics and not be used for individual patients.
Characteristics of the cohort
The study cohort consisted of a heterogeneous group of critically ill patients, with elective surgery including heart valve surgery, urology, gastrointestinal and spinal surgery accounting for 36.2% of all ICU admissions. The emergency admissions consisted of patients with multiple trauma (8.5%), isolated head trauma (2.5%), acute myocardial infarction, congestive heart failure, cardiac arrhythmias, or cardiogenic shock (7.4%), hypovolemic or hemorrhagic shock (0.8%), drug overdoses (7.2%), subarachnoid or intracranial hemorrhage (5.1%), sepsis (4.3%), pneumonia or aspiration (3.7%), obstructive airway diseases (2.1%), cardiorespiratory arrest (4.0%), gastrointestinal hemorrhage, perforation or obstruction (2.4%), and other medical and surgical emergencies. Details of this cohort including demographic factors, co-morbidities, severity of acute illness, and the length of ICU and hospital stay are described in Table 2.
Effect of the Predictors on Hazard of Death
Among all the seven pre-selected predictors in the model, age (50%), co-morbidity as measured by Charlson co-morbidity index (27%), and severity of acute illness as measured by the APACHE II predicted mortality (20%) made the strongest contributions in predicting survival time (Figure 2). After adjusting for other predictors, the log hazard of death increased linearly with age, Charlson co-morbidity index, and the number of days of vasopressor or inotropic therapy, mechanical ventilation, or hemofiltration therapy (Figure 3). The relationship between the APACHE II predicted mortality and log hazard of death was non-linear with a steep effect when the APACHE II predicted mortality was less than 10% and a smaller effect when it was more than 10%. The estimated (adjusted) hazard ratios for the seven predictors are summarized in Figure 4.
An increase of 20 years of age and an increase in Charlson co-morbidity index from 0 to 5 approximately doubled the risk of death. Doubling the APACHE II predicted mortality from 20% to 40% increased the relative risk of death by about 30 to 40%. Similarly, increased the number of days of intensive care therapy from 1 to 5 increased the relative risk of death by between 10% and 50%.
Clinical Application of the Model
Figure 5 presents the model in the form of a nomogram that provides the median survival time and long-term survival probabilities corresponding to a particular total score. The total score for a patient is obtained by adding up the scores for each of the seven predictors. We use the following hypothetical but typical patients to illustrate how the nomogram is used and how the short-term prognosis of a patient can be quite different from the long-term prognosis.
Note: gender: 2 = female, 1 = male. Predicted.mortality = APACHE II predicted mortality in %.
A 40-year old male, without pre-existing co-morbidities (ie Charlson co-morbidity index = 0), was admitted to the ICU because of septic shock with an APACHE II predicted mortality of 80%. He required vasopressor or inotropic therapy, mechanical ventilation, and hemofiltration therapy during the first 5 days in the ICU.
The gender of this patient scores 5 points, age scores 28 points, Charlson co-morbidity scores zero points, the APACHE II predicted mortality or risk scores 30 points, 5 days of vasopressor or inotropic therapy scores 7 points, 5 days of mechanically ventilation scores 15 points, and 5 days of hemofiltration scores 20 points. The total score of this patient is therefore 105 which gives an estimated median survival time of about 4 years, >70% 1-year survival probability, >50% 3-year survival probability, >45% 5-year survival probability, and >20% 10-year survival probability.
A 70-year old female, with chronic obstructive airway disease and non-insulin dependent diabetes mellitus with no end-organ damage (ie Charlson co-morbidity index = 2), was admitted to the ICU because of severe community acquired pneumonia with an APACHE II predicted mortality of 30%. She required vasopressor or inotropic therapy and mechanical ventilation but not hemofiltration during the first 5 days in the ICU.
The gender of this patient scores zero points, age scores 70 points, Charlson co-morbidity index scores 12 points, the APACHE II predicted mortality scores 16 points, 5 days of mechanical ventilation scores 15 points, and 5 days of vasopressor or inotropic therapy scores 7 points. The total score of this patient is therefore 120 which gives an estimated median survival time of about 2 years, 60% 1-year survival probability, 40% 3-year survival probability, 30% 5-year survival probability, and 10% 10-year survival probability.
A 80-year old male, with a history of myocardial infarction, congestive heart failure, peripheral vascular disease, cerebrovascular disease, and dementia (ie Charlson co-morbidity index = 5), was admitted to an ICU with bowel perforation and peritonitis with an APACHE II predicted mortality of 30%. He required vasopressor or inotropic therapy and mechanical ventilation but not hemofiltration during the first 5 days in the ICU.
The gender of this patient scores 5 points, age scores 85 points, Charlson co-morbidity index scores 30 points, the APACHE II predicted mortality scores 16 points, 5 days of mechanical ventilation scores 15 points, and 5 days of vasopressor or inotropic therapy scores 7 points. The total score of this patient is therefore 158 which gives an estimated median survival time of <0.5 years, 25% 1-year survival probability, and 10% 3-year survival probability.
Discrimination and Calibration of the Prognostic Model
The adjusted c-index for this prognostic model was 0.757 (95% confidence interval 0.745–0.769), Nagelkerke's R2 was 0.255 and the bias-corrected calibration of the model over a 15-year period was reasonable (slope of the calibration = 0.98)(Figure 6). The Nagelkerke's R2 remained unchanged and the adjusted c-index only increased marginally when the analysis was restricted to a maximum of 5 years follow up (c-index = 0.759, slope = 0.97) or data after 1997 (c-index = 0.762, slope of the calibration = 0.97).
This study showed that age, gender, co-morbidities (Charlson co-morbidity index), severity of acute illness (the APACHE II predicted mortality), and duration of intensive care therapy or organ support within the first 5 days of ICU admission are important prognostic factors for long-term survival of critically ill patients. To the best of our knowledge, this new prognostic model (Predicted Risk, Existing Diseases, and Intensive Care Therapy: the PREDICT model) is the first preliminary prognostic model that can be used to estimate the median survival time and long-term survival probabilities of critically ill patients up to 15 years after the onset of critical illness.
The current prognostic model has confirmed that age, gender, co-morbidities, severity of acute illness, and duration of intensive care therapy or organ failure are important predictors of 6 months to 5 years survival of hospitalized or critically ill patients , , , , . The current model is indeed built on the results of these previous studies but further extended the significance of these risk factors in predicting survival of critically ill patients beyond 6 months to 5 years. This current model also demonstrated that most of these predictors have a relatively linear relationship to the long-term survival probability. More importantly, our results also showed that age and co-morbidities are the most important determinants of long-term prognosis of critically ill patients. This latter finding has at least two significant clinical implications. First, the factors that determine long-term survival of a critically ill patient are different from those that affect short-term prognosis. Previous evidence suggested that diagnosis and acute physiological derangement of a patient are most important in determining hospital survival , . In our three hypothetical patients, Patient A has in fact the most severe form of acute critical illness and worst short-term prognosis. Nevertheless, because this patient is younger and has no co-morbidities, this patient has a very reasonable and better long-term prognosis than Patient B and C. If we use the prognostic model developed by Wright et al.  to estimate the long-term survival of our three hypothetical patients, Patient B will have the best 5-year prognosis (risk score is estimated to be 68) followed by Patient C (risk score 75) and then Patient A (risk score 87). The lack of detailed co-morbidity data and a heavy emphasis on severity of acute illness in the model developed by Wright et al. is the most likely explanation why our results are different from theirs.
Many clinicians may intuitively consider the intensity of organ failure as very important in affecting a patient's prognosis , . Our findings suggest that the effect of acute organ failure on long-term survival is not strong and mostly captured by age, co-morbidities, and the APACHE II predicted mortality on admission to ICU. Our previous studies have also showed that the intensity of organ failure alone is not as important as the APACHE II score in predicting hospital mortality , . Therefore, our findings suggest that clinicians should be very careful not to place undue emphasis on the severity of acute illness and intensity of organ failure when making long-term prognostications of critically ill patients.
Second, because the contributions by intensive care therapy are relatively small when compared to age, Charlson co-morbidity index, and the APACHE II predicted mortality, using the data after the first 24 to 48 hours of ICU stay is unlikely to underestimate the final total prediction score significantly (<20 points)(Figure 5). Therefore, early estimation of a slightly ‘optimistic’ long-term survival probability and median survival time is feasible after the first 24 to 48 hours of ICU stay; and in patients with either extremes of prognosis, this early estimation is unlikely to be significantly different from the final prediction by collecting all data after five days of intensive care therapy. Nevertheless, the current prognostic model utilizes the APACHE II predicted mortality after ICU admission as a predictor to estimate long-term survival, as such, the model cannot be used, in its current form, as a tool to triage ICU admission.
This study has significant limitations. First, patients' wishes and the anticipated quality of life before and after their critical illness are important factors in making treatment decisions , . The median survival time and long-term survival probabilities is only one of the many factors that patients and clinicians may consider in making treatment decisions. Furthermore, the c-statistics of this model is only about 0.76 and this leaves considerable uncertainty in its applicability in predicting long-term survival of individual patients. As such, the predicted survival probabilities of this prognostic model should only be considered as an average estimate of patients with similar characteristics and should not be used for individual patients. Second, evidence suggests that combining an objective prognostic model with physicians' intuition may improve the accuracy of outcome prediction . Whether combining this current prognostic model with physicians' intuition will improve its predictive performance further remains uncertain, but this merits further investigation. Third, although we studied a large cohort of critically ill patients, and also the case-mix, severity of illness, and in-hospital survival of this cohort is very similar to many other ICUs in Australia , validation of this model by other ICUs that have access to data linkage is essential to assess its generalizability. Finally, although the APACHE II prognostic model is still widely used for risk adjustment purposes in many ICUs , , it is possible that using newer prognostic models instead of the APACHE II prognostic model may improve our current model . Similarly, the performance of the current model may be improved if we consider more predictors in the model although this will also increase the complexity of the model. In this regard, we hope that the PREDICT model developed in this study will be of value to others who aim to develop a new prognostic model to enhance our understanding of long-term survival of critically ill patients.
In summary, Age, gender, co-morbidities, severity of acute illness, and the intensity and duration of intensive care therapy can be used to estimate long-term survival of critically ill patients. Age and co-morbidity are the most important determinants of the long-term prognosis of critically ill patients. The current prognostic model, the PREDICT model, provides a framework for prognostications and risk adjustment when long-term survival of critically ill patients is considered.
Conceived and designed the experiments: KMH MK JF SAW. Analyzed the data: KMH MK JF SAW. Contributed reagents/materials/analysis tools: KMH. Wrote the paper: KMH MK JF SAW.
- 1. Acute Health Division DoHS (1997) Review of intensive care in Victoria [Phase 1 report]. Melbourne: Department of Human Services.
- 2. Halpern NA, Bettes L, Greenstein R (1994) Federal and nationwide intensive care units and healthcare costs: 1986–1992. Crit Care Med 22: 2001–2007.
- 3. Kvåle R, Flaatten H (2002) Changes in intensive care from 1987 to 1997 - has outcome improved? A single centre study. Intensive Care Med 28: 1110–1116.
- 4. Poisal JA, Truffer C, Smith S, Sisko A, Cowan C, et al. (2007) Health spending projections through 2016: modest changes obscure part D's impact. Health Aff (Millwood) 26: w242–w253.
- 5. The Audit Commission (1999) Critical to Success. The place of efficient and effective critical care services within the acute hospital. London: Audit Commission for Local Authorities and the National Health Service in England and Wales.
- 6. Girard TD, Kress JP, Fuchs BD, Thomason JW, Schweickert WD, et al. (2008) Efficacy and safety of a paired sedation and ventilator weaning protocol for mechanically ventilated patients in intensive care (Awakening and Breathing Controlled trial): a randomised controlled trial. Lancet 371: 126–134.
- 7. Sinuff T, Adhikari NK, Cook DJ, Schünemann HJ, Griffith LE, et al. (2006) Mortality predictions in the intensive care unit: comparing physicians with scoring systems. Crit Care Med 34: 878–885.
- 8. Cook DJ, Guyatt GH, Jaeschke R, Reeve J, Spanier A, et al. (1995) Determinants in Canadian health care workers of the decision to withdraw life support from the critically ill. JAMA 273: 703–708.
- 9. Elstein AS, Christensen C, Cottrell JJ, Polson A, Ng M (1999) Effects of prognosis, perceived benefit and decision style upon decision making in critical care. Crit Care Med 27: 58–65.
- 10. Garland A, Connors AF (2007) Physicians' influence over decisions to forego life support. J Palliat Med 10: 1298–1305.
- 11. Connors AF Jr (1999) The influence of prognosis on care decisions in the critically ill. Crit Care Med 27: 5–6.
- 12. Hamel MB, Phillips RS, Davis RB, Desbiens N, Connors AF Jr, et al. (1997) Outcomes and cost-effectiveness of initiating dialysis and continuing aggressive care in seriously ill hospitalized adults. SUPPORT Investigators. Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments. Ann Intern Med 127: 195–202.
- 13. Knaus WA, Harrell FE Jr, Lynn J, Goldman L, Phillips RS, et al. (1995) The SUPPORT prognostic model. Objective estimates of survival for seriously ill hospitalized adults. Study to understand prognoses and preferences for outcomes and risks of treatments. Ann Intern Med 122: 191–203.
- 14. Wright JC, Plenderleith L, Ridley SA (2003) Long-term survival following intensive care: subgroup analysis and comparison with the general population. Anaesthesia 58: 637–642.
- 15. Knaus WA, Draper EA, Wagner DP, Zimmerman JE (1985) APACHE II: a severity of disease classification system. Crit Care Med 13: 818–829.
- 16. Williams TA, Dobb GJ, Finn JC, Knuiman M, Lee KY, et al. (2006) Data linkage enables evaluation of long-term survival after intensive care. Anaesth Intensive Care 34: 307–315.
- 17. Ho KM, Dobb GJ, Knuiman M, Finn J, Lee KY, et al. (2006) A comparison of admission and worst 24-hour Acute Physiology and Chronic Health Evaluation II scores in predicting hospital mortality: a retrospective cohort study. Crit Care 10: R4.
- 18. Williams TA, Dobb GJ, Finn JC, Knuiman MW, Geelhoed E, et al. (2008) Determinants of long-term survival after intensive care. Crit Care Med 36: 1523–1530.
- 19. Charlson ME, Pompei P, Ales KL, MacKenzie CR (1987) A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis 40: 373–383.
- 20. Holman CD, Bass AJ, Rouse IL, Hobbs MS (1999) Population-based linkage of health records in Western Australia: development of a health services research linked database. ANZ J Public Health 23: 453–459.
- 21. Cox DR (1972) Regression models and life tables (with discussion). Journal of the Royal Statistical Society 34: 187–220.
- 22. Lee SJ, Lindquist K, Segal MR, Covinsky KE (2006) Development and validation of a prognostic index for 4-year mortality in older adults. JAMA 295: 801–808.
- 23. Pompei P, Charlson ME, Ales K, MacKenzie CR, Norton M (1991) Relating patient characteristics at the time of admission to outcomes of hospitalization. J Clin Epidemiol 44: 1063–1069.
- 24. Wyatt JC, Altman DG (1995) Prognostic models: clinically useful or quickly forgotten? BMJ 311: 1539–1541.
- 25. Harrell FE Jr (2001) Regression Modeling Strategies. New York: Springer.
- 26. Hanley JA, McNeil BJ (1982) The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143: 29–36.
- 27. Feringa HH, Bax JJ, Hoeks S, van Waning VH, Elhendy A, et al. (2007) A prognostic risk index for long-term mortality in patients with peripheral arterial disease. Arch Intern Med 167: 2482–2489.
- 28. Efron B, Tibshirani R (1993) An Introduction to the Bootstrap. New York: Chapman & Hall.
- 29. Ho KM (2007) Forest and funnel plots illustrated the calibration of a prognostic model: a descriptive study. J Clin Epidemiol 60: 746–751.
- 30. Nagelkerke NJ (1991) A note on a general definition of the coefficient of determination. Biometrika 78: 691–692.
- 31. Ho KM, Finn J, Knuiman M, Webb SA (2007) Combining multiple comorbidities with Acute Physiology Score to predict hospital mortality of critically ill patients: a linked data cohort study. Anaesthesia 62: 1095–1100.
- 32. Cabré L, Mancebo J, Solsona JF, Saura P, Gich I, et al. and the Bioethics Working Group of the SEMICYUC (2005) Multicenter study of the multiple organ dysfunction syndrome in intensive care units: the usefulness of Sequential Organ Failure Assessment scores in decision making. Intensive Care Med 31: 927–933.
- 33. Nathens AB, Rivara FP, Wang J, Mackenzie EJ, Jurkovich GJ (2008) Variation in the rates of do not resuscitate orders after major trauma and the impact of intensive care unit environment. J Trauma 64: 81–88.
- 34. Ho KM, Lee KY, Williams T, Finn J, Knuiman M, et al. (2007) Comparison of Acute Physiology and Chronic Health Evaluation (APACHE) II score with organ failure scores to predict hospital mortality. Anaesthesia 62: 466–473.
- 35. Ho KM (2007) Combining sequential organ failure assessment (SOFA) score with acute physiology and chronic health evaluation (APACHE) II score to predict hospital mortality of critically ill patients. Anaesth Intensive Care 35: 515–521.
- 36. Ho KM, Liang J (2004) Withholding and withdrawal of therapy in New Zealand intensive care units (ICUs): a survey of clinical directors. Anaesth Intensive Care 32: 781–786.
- 37. Ho KM, English S, Bell J (2005) The involvement of intensive care nurses in end-of-life decisions: a nationwide survey. Intensive Care Med 31: 668–673.
- 38. Finfer S, Bellomo R, Lipman J, French C, Dobb G, et al. (2004) Adult-population incidence of severe sepsis in Australian and New Zealand intensive care units. Intensive Care Med 30: 589–596.
- 39. Harvey S, Harrison DA, Singer M, Ashcroft J, Jones CM, et al. PAC-Man study collaboration (2005) Assessment of the clinical effectiveness of pulmonary artery catheters in management of patients in intensive care (PAC-Man): a randomised controlled trial. Lancet 366: 472–477.
- 40. Finfer S, Bellomo R, Boyce N, French J, Myburgh J, et al. SAFE Study Investigators (2004) A comparison of albumin and saline for fluid resuscitation in the intensive care unit. N Engl J Med 350: 2247–2256.
- 41. Zimmerman JE, Kramer AA, McNair DS, Malila FM (2006) Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today's critically ill patients. Crit Care Med 34: 1297–1310.