Citation: Young NH, Andrews PJD (2008) Developing a Prognostic Model for Traumatic Brain Injury—A Missed Opportunity? PLoS Med 5(8): e168. https://doi.org/10.1371/journal.pmed.0050168
Published: August 5, 2008
Copyright: © 2008 Young and Andrews. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: The authors received no specific funding for this article.
Competing interests: NHY declares that he has no competing interests. PJDA declares that he is a researcher in traumatic brain injury and Chief Investigator of the EUROTHERM Trial (ISRCTN34555414; http://www.controlled-trials.com/ISRCTN34555414).
Abbreviations: CRASH, Corticosteroid Randomisation After Significant Head Injury; GCS, Glasgow coma scale; GOS, Glasgow outcome scale; IMPACT, International Mission on Prognosis and Analysis of Clinical Trials in TBI; TBI, traumatic brain injury
Traumatic brain injury (TBI) is a major cause of death and disability throughout the world. The annual incidence of TBI in the United States has been estimated at between 180 and 250 per 100,000 population per year . TBI is the cause of one third to one half of all trauma deaths, and the leading cause of disability in people under 40, severely disabling 15 to 20 per 100,000 population per year . Injuries, including TBI, are projected to account for 20% of the worldwide burden of death and disability by 2020 .
Research into new therapies for TBI has been disappointing. At least 21 multi-centre clinical trials have been conducted since 1985, none of which have shown a convincing benefit in the treatment of TBI . Clinical trials into TBI provide significant challenges: trauma is a neglected research topic worldwide, consent in unconscious patients requires careful ethical consideration, and the injuries are very heterogeneous in terms of mechanism and pathology [4,5].
Prognostic models are statistical models that combine data from patients to predict clinical outcome. Such models based on data collected soon after presentation could in theory be used to aid early clinical decision making and to allow more accurate counselling of patients' families. They could also have a pivotal role in the future design and analysis of randomised controlled trials, and assist in clinical audit by allowing adjustment for case mix .
However, while many prognostic models have been developed, none are widely used. A recent systematic review of prognostic models in TBI found that they were developed from small samples of patients, had poor methodology (for example, in over half of the models, loss to follow-up was not reported), were rarely externally validated, and were not clinically practical .
Linked Research Article
This Perspective discusses the following new study published in PLoS Medicine:
Steyerberg EW, Mushkudiani N, Perel P, Butcher I, Lu J, et al. (2008) Predicting outcome after traumatic brain injury: Development and international validation of prognostic scores based on admission characteristics. PLoS Med 5(8): e165. doi:10.1371/journal.pmed.0050165
Ewout Steyerberg and colleagues describe a prognostic model for the prediction of outcome of traumatic brain injury using data available on admission.
Developing a New Prognostic Model for TBI
In recent years, there have been two milestones that helped forward research into TBI: the formation of the IMPACT (International Mission on Prognosis and Analysis of Clinical Trials in TBI) database , and the CRASH (Corticosteroid Randomisation After Significant Head Injury) trial . The IMPACT database combined patient data from eight randomised controlled clinical trials and three observational studies to give a patient population of over 9,000. The CRASH trial was a randomised controlled trial of the effect of early steroid administration on outcome after TBI. The CRASH trial enrolled 10,008 patients and is the largest clinical trial ever conducted in patients with TBI.
This year, the CRASH trial collaborators published in the BMJ a series of prognostic models for predicting outcome after TBI, based on the 10,008 patients enrolled in the CRASH trial . As well as being the largest prognostic models developed for TBI, the new models display other key differences from previous models: they were externally validated, included patient data from low- and middle-income countries, and are publicly available. The external validation for these models took place against patient data in the IMPACT database.
In a paper published in this issue of PLoS Medicine, Ewout Steyerberg and colleagues describe the development and validation of new prognostic models for TBI . The authors review patient characteristics from 8,509 patients in the IMPACT database. From an initial examination of 26 potential predictors, the authors define a core prognostic model based on three clinical predictors: age, motor component of Glasgow coma score (GCS), and pupillary reactivity. They also develop an extended model, which includes the three core predictors plus information on secondary insults and computerized tomography characteristics, and a laboratory model, which also includes haemoglobin and glucose measurement.
The endpoint was Glasgow outcome scale (GOS) at six months (the GOS runs from 1 to 5, with 1 representing death and 5 representing recovery; see http://www.dundee.ac.uk/medther/Stroke/Scales/Gos.htm). For validation, the authors focus on prediction of death (GOS 1) versus survival (GOS 2–5) and of unfavourable (GOS 1–3) versus favourable (GOS 4–5) outcome. The model was externally validated on data from 6,272 patients from the CRASH trial. All predictors had statistically significant associations with six-month GOS. The discriminatory ability of the models was shown to increase with increasing complexity. External validation confirmed the discriminatory ability of the core model and a modified computerized tomography model with 6,681 and 5,309 patients, respectively, from the CRASH trial. The calibration was not as good, in that outcomes from the CRASH trial were systematically poorer; however, calibration was near perfect when only the patients from high-income countries were included.
Steyerberg and colleagues' paper in PLoS Medicine and the recent BMJ paper use very similar methodology to develop their predictive models (one used the IMPACT dataset as the training dataset and the other the CRASH dataset, externally validating against the other). It is debatable whether publishing two papers separately added value—they could have possibly been combined in a single publication.
What Are the Strengths and Limitations of This Model?
While the predictors of age, motor score, and pupillary reactivity have been included in many prognostic models for TBI, Steyerberg and colleagues' models  and the models produced by the CRASH collaborators  are the largest and most robustly validated that have been developed so far. They are also readily accessible to clinicians via a Web-based calculator [9,10].
Caution must be advised in applying population-based estimates of outcome to individual patients. While both of these groups of prognostic models have been externally validated, they have essentially been validated against the populations of each other. All of the patients from CRASH and most of the patients from IMPACT were enrolled in clinical trials. Work is needed to establish the accuracy of these models prospectively in patients not enrolled in clinical trials.
The CRASH dataset is more heterogeneous than many of the phase III neuroprotection studies because inclusion GCS included moderate and severe injury and included a high number of patients from low- and middle-income countries. Steyerberg and colleagues' models seemed less accurate at predicting outcome following TBI in low- and middle-income countries.
Logistic regression, used in the PLoS Medicine and BMJ papers [7,8], treats the patient cohorts as homogeneous and therefore has a number of limitations. In contrast, decision tree analysis takes a group of subjects and divides them into subgroups according to the best discriminating factor within the original group. The level at which each discriminating factor (blood pressure, age, GCS, gender, etc.) is important is given, and the process is repeated recursively producing subgroups with less variability or entropy than the original. If followed to the conclusion of the process, each patient would end up in a category of his or her own, a situation that is closer to clinical reality than treating all patients as a homogenous group and thus the same. Decision tree analysis requires a finite number of target categories to be of value. Logistic regression analysis was previously compared with decision tree analysis for predicting recovery from traumatic brain injury, using a small dataset from 124 adult head-injured patients in an intensive care unit .
Comparison between the logistic regression analysis and decision tree analysis shows a number of similarities, particularly that hypotension is a strong indicator of bad outcome and that age, gender, and brainstem syndromes are important indicators. Differences include the creation of smaller patient groups by the decision tree, each of which is characterized, whereas the regression analysis suggests factors that discriminate for the set of patients as a whole [12,13]. Decision tree analysis identifies more homogenous subgroups and predictors that are important to that subgroup. Such subgroups might include, for example, a good prognosis group with the following discriminating factors: young age, extradural haematoma, and initial good GCS. A possible poor prognosis group might be identified as: elderly patients, acute subdural haematoma, low GCS, and severe comorbidity. It is not plausible that the same prognostic factors will carry the same importance in such disparate categories of TBI.
The logistic regression analysis and decision tree analysis show age is an important co-variant in prediction of outcome, but the decision tree gives threshold values of 50 years for the prediction of death and 30 years for prediction of good or poor outcome. It might be clinically more informative to apply decision tree analysis to the IMPACT and CRASH datasets for prediction and hypothesis generation/revision.
We now have new prognostic models for TBI developed from large numbers of patients, which have been externally validated to allow prediction between favourable or unfavourable outcome at six months [7,8]. These models may provide useful additional information in regard to clinical decision making and the counselling of patients' relatives, but it must be remembered that their outcomes apply to populations—and so great caution is needed if applying them to individual patients. These models will undoubtedly be of great use in clinical audit, allowing comparison between different units and changes in management over time, while allowing for different case mix. The models should also allow better trial design and analysis: many patients with TBI previously included in clinical trials have expected outcomes that are so good or bad that no intervention could be expected to alter the outcome . Future clinical trials could focus on those patients with a more uncertain prognosis.
- 1. Bruns J Jr, Hauser WA (2003) The epidemiology of traumatic brain injury: A review. Epilepsia 44(Suppl 10): 2–10.J. Bruns JrWA Hauser2003The epidemiology of traumatic brain injury: A review.Epilepsia44Suppl 10210
- 2. Fleminger S, Ponsford J (2005) Long term outcome after traumatic brain injury. BMJ 331: 1419–1420.S. FlemingerJ. Ponsford2005Long term outcome after traumatic brain injury.BMJ33114191420
- 3. Finfer SR, Cohen J (2001) Severe traumatic brain injury. Resuscitation 48: 77–90.SR FinferJ. Cohen2001Severe traumatic brain injury.Resuscitation487790
- 4. Maas AIR, Marmarou A, Murray GD, Teasdale GM, Steyerberg EW (2007) Prognosis and clinical trial design in traumatic brain injury: The IMPACT study. J Neurotrauma 24: 232–238.AIR MaasA. MarmarouGD MurrayGM TeasdaleEW Steyerberg2007Prognosis and clinical trial design in traumatic brain injury: The IMPACT study.J Neurotrauma24232238
- 5. Perel P, Edwards P, Wentz R, Roberts I (2006) Systematic review of prognostic models in traumatic brain injury. BMC Med Inform Decis Mak 6: 38.P. PerelP. EdwardsR. WentzI. Roberts2006Systematic review of prognostic models in traumatic brain injury.BMC Med Inform Decis Mak638
- 6. Roberts I, Yates D, Sandercock P, Farrell B, Wasserberg J, et al. (2004) Effect of intravenous corticosteroids on death within 14 days in 10008 adults with clinically significant head injury (MRC CRASH trial): Randomised placebo-controlled trial. Lancet 364: 1321–1328.I. RobertsD. YatesP. SandercockB. FarrellJ. Wasserberg2004Effect of intravenous corticosteroids on death within 14 days in 10008 adults with clinically significant head injury (MRC CRASH trial): Randomised placebo-controlled trial.Lancet36413211328
- 7. Perel P, Arango M, Clayton T, Edwards P, et al., editors. (2008) Predicting outcome after traumatic brain injury: Practical prognostic models based on large cohort of international patients. BMJ 336: 425–429.MRC CRASH Trial Collaborators. P. PerelM. ArangoT. ClaytonP. Edwards2008Predicting outcome after traumatic brain injury: Practical prognostic models based on large cohort of international patients.BMJ336425429
- 8. Steyerberg EW, Mushkudiani N, Perel P, Butcher I, Lu J, et al. (2008) Predicting outcome after traumatic brain injury: Development and international validation of prognostic scores based on admission characteristics. PLoS Med 5: e165.EW SteyerbergN. MushkudianiP. PerelI. ButcherJ. Lu2008Predicting outcome after traumatic brain injury: Development and international validation of prognostic scores based on admission characteristics.PLoS Med5e165
- 9. Clinical Randomisation of an Antifibrinolytic in Significant Haemorrhage (2008) Prognostic model for predicting outcome after traumatic brain injury. Clinical Randomisation of an Antifibrinolytic in Significant Haemorrhage2008Prognostic model for predicting outcome after traumatic brain injury.Available: http://www.crash2.lshtm.ac.uk/Risk%20calculator/index.html. Accessed 2 July 2008. Available: http://www.crash2.lshtm.ac.uk/Risk%20calculator/index.html. Accessed 2 July 2008.
- 10. International Mission for Prognosis and Analysis of Clinical Trials in TBI (2008) Prognostic calculator. International Mission for Prognosis and Analysis of Clinical Trials in TBI2008Prognostic calculator.Available: http://www.tbi-impact.org/. Accessed 2 July 2008. Available: http://www.tbi-impact.org/. Accessed 2 July 2008.
- 11. Andrews PJD, Sleeman D, Satham PFX, McQuatt AM, Corruble V, et al. (2002) Predicting recovery after traumatic brain injury using intelligent data analysis of admission variables and time series physiological data—A comparison with logistic regression. J Neurosurg 97: 326–336.PJD AndrewsD. SleemanPFX SathamAM McQuattV. Corruble2002Predicting recovery after traumatic brain injury using intelligent data analysis of admission variables and time series physiological data—A comparison with logistic regression.J Neurosurg97326336
- 12. Signorini DF, Andrews PJD, Jones PA, Wardlaw JM, Miller JD (1999) Predicting survival using simple clinical variables: A case study in traumatic brain injury. J Neurol Neurosurg Psychiatry 66: 20–25.DF SignoriniPJD AndrewsPA JonesJM WardlawJD Miller1999Predicting survival using simple clinical variables: A case study in traumatic brain injury.J Neurol Neurosurg Psychiatry662025
- 13. Signorini DF, Andrews PJD, Jones PA, Wardlaw JM, Miller JD (1999) Adding insult to injury: The prognostic value of early secondary insults for survival after traumatic brain injury. J Neurol Neurosurg Psychiatry 66: 26–31.DF SignoriniPJD AndrewsPA JonesJM WardlawJD Miller1999Adding insult to injury: The prognostic value of early secondary insults for survival after traumatic brain injury.J Neurol Neurosurg Psychiatry662631
- 14. Menon D, Harrison D (2008) Prognostic modelling in prognostic brain injury can reliably estimate the probability of outcomes for groups but not for individuals. BMJ 336: 397–398.D. MenonD. Harrison2008Prognostic modelling in prognostic brain injury can reliably estimate the probability of outcomes for groups but not for individuals.BMJ336397398