Browse Subject Areas

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Linking Injury to Outcome in Acute Kidney Injury: A Matter of Sensitivity

  • John W. Pickering ,

    Affiliation Christchurch Kidney Research Group, Department of Medicine, School of Medicine and Health Sciences, Otago University, Christchurch, New Zealand

  • Zoltan H. Endre

    Affiliations Christchurch Kidney Research Group, Department of Medicine, School of Medicine and Health Sciences, Otago University, Christchurch, New Zealand, Department of Nephrology, Prince of Wales Clinical School, University of New South Wales, Sydney, Australia

Linking Injury to Outcome in Acute Kidney Injury: A Matter of Sensitivity

  • John W. Pickering, 
  • Zoltan H. Endre


Current consensus definitions of Acute Kidney Injury (AKI) utilise thresholds of change in serum or plasma creatinine and urine output. Biomarkers of renal injury have been validated against these definitions. These biomarkers have also been shown to be independently associated with mortality and need for dialysis. For AKI definitions to include these structural biomarkers, there is a need for an independent outcome against which to judge both markers of functional change and structural markers of injury. We illustrate how sensitivity to need for dialysis and death can be used to link functional and structural (biomarker) based definitions of AKI. We demonstrated the methodology in a representative cohort of critically ill patients, in which an increase of plasma creatinine of >26.4 µmol/L in 48 hours or >50% in 7 days (Functional-AKI) had a sensitivity of 62% for death or dialysis within 30 days. In a development sub-cohort the urinary neutrophil-gelatinase-associated-lipocalin threshold with a 62% sensitivity for death or dialysis was 140 ng/ml (Structural-AKI). Using these thresholds in a validation sub-cohort, the risk of death or dialysis relative to those with no AKI by either definition was, for combined Structural-AKI and Functional-AKI 3.11 (95% Confidence interval: 2.53 to 3.55), for those with Structural-AKI but not Functional-AKI 1.51 (1.26 to 1.62), and for those with Functional-AKI but not Structural-AKI 1.34 (1.16 to 1.42). Linking functional and structural biomarkers via sensitivity for death and dialysis is a viable method by which to define thresholds for novel biomarkers of AKI.


AKI is common and associated with increased in-hospital mortality, length of stay and subsequent development of chronic kidney disease [1], [2]. The absence of symptoms to herald AKI mandates the monitoring of biomarkers for diagnosis. Current consensus definitions utilise an increase in plasma creatinine and a reduction in urine output as surrogates of a reduction in glomerular filtration rate [3][6]. Injury biomarkers detect AKI up to 48-hours earlier than creatinine [7] and offer a ray of sunshine in the dark and largely negative history of prevention and treatment of AKI [8]. Higher injury biomarker concentrations are associated with increasing AKI functional severity class and hard outcomes [9][11]. It has also been demonstrated, in the case of the injury biomarker, neutrophil-gelatinase-associated-lipocalin (NGAL), that biomarker-positive creatinine-negative patients have increased need for dialysis and higher mortality than patients negative for both creatinine and NGAL [12].

A major limitation in the evaluation of injury biomarkers is that performance is usually only assessed by the ability to detect or predict increases in plasma creatinine, the same parameter used to diagnose AKI [13]. Thus, while an increase in biomarkers can predict an increase in creatinine, only the inevitably delayed change in creatinine can diagnose AKI. This ignores that creatinine is a marker of function, while injury biomarkers detect structural damage. This strategy is analogous to using a functional marker, such as echocardiography, to diagnose myocardial infarction instead of a biomarker of injury, such as troponin. It also questions whether an increase in biomarkers alone should be used to facilitate early intervention, and thus argues against the advantages for which novel biomarkers have been developed. Finally, it ignores that the current consensus creatinine-based definitions of AKI have selected thresholds, creating dichotomous outcomes (AKI or no-AKI) from a continuous variable.

Although we do not yet know how much structural damage is required to reduce GFR acutely, we need an independent outcome against which to judge both markers of functional change and structural injury. The outcomes on which consensus is most likely are the requirement for dialysis and short-term mortality. While only dialysis links easily to AKI, there is increasing support for both direct and indirect linkage to death [14], [15]. However, even with large multicentre cohorts to assess these outcomes, there is no objective strategy for validating the functional or biomarker performance thresholds required to demonstrate clinical utility and to avoid the circular arguments inherent in the current approach. Choosing dialysis alone as the outcome would precipitate such a circular argument given that in clinical practice dialysis inevitably follows a substantial increase in plasma creatinine and therefore 100% sensitivity for the AKI threshold. In this scenario there would be no Structural-AKI without Functional-AKI with the outcome effectively benchmarking the structural biomarker once more against a functional definition of AKI.

If structural biomarkers are to have clinical utility, then a suitable methodology needs to be developed to determine a clinically relevant threshold. We hypothesised that sensitivity for a significant outcome (in this case, the composite of dialysis and death) is the ideal parameter for defining the threshold required for diagnosis of AKI by either a single structural or functional biomarker or for any combination of structural or functional biomarkers. We suggest that sensitivity for dialysis and death could provide the benchmark against which further enhancements should be judged. The use of sensitivity provides an independent parameter, to link these potentially disparate biomarkers. We chose, by way of example, to assess this methodology for a single candidate biomarker, urinary NGAL, by retrospective analysis of the well-documented dataset from the EARLYARF (Early Acute Renal Failure) trial. However, the method can equally be applied using any other candidate biomarker for structural injury in AKI.

Materials and Methods

Patients from the EARLYARF trial in two intensive care units [16], [17] were randomly divided into development and internal validation cohorts stratified by AKI using the function-based Kidney Disease, Improving Global Outcomes (KDIGO) definition (Functional-AKI: an increase of plasma creatinine of either >26.4 µmol/L (0.3 mg/dl) in 48 hours or >50% in 7 days).

All biomarkers were measured at entry to ICU, and then at 12 and 24 hours. The maximum concentration for each patient was used in the analysis. Details of analysis have been reported previously [16], [17]. In brief, samples for alkaline phosphatase (AP), γ-glutamyl transpeptidase (GGT) were assayed immediately, by g-glutamyl-p- nitroanilide rate and p-nitrophenol rate reactions respectively (International Federation of Clinical Chemistry method). Samples for other assays were stored at −80°C until batch analysis. Urinary NGAL was measured using a NGAL ELISA Kit 036 (AntibodyShop, Grusbakken, Denmark) [18], Cystatin C (CysC) was measured with a BNII nephelometer (Dade Behring GmbH, Marburg, Germany) by particle-enhanced immunonephelometric assay [19], interleukin-18 (IL-18) using a human IL-18 ELISA kit (Medical and Biological Laboratories, Nagoya, Japan; see [20]), Kidney injury molecule-1 (KIM-1) using microsphere-based Luminex xMAP technology (Luminex, Austin, TX; see [21]), and α- and π-glutathione-S-transferase (α-GST and π-GST) using human ELISA test kits (Argutus Medical, Dublin, Ireland; see [22]).

The common outcome was need for dialysis or death in 30 days. In the development cohort the sensitivity of Functional-AKI was determined. The threshold concentration of the structural biomarker, urinary NGAL, which had the same sensitivity for this outcome was then determined. Patients with urinary NGAL above this threshold were deemed to have Structural-AKI. This threshold was then used to determine the proportions of patients with the outcome in both the development and validation cohorts. Urinary NGAL was chosen for this demonstration because it is the most studied of candidate biomarkers and has been evaluated to be independently associated with mortality and need for dialysis [12].

Results are presented as means ± standard deviation, medians (interquartile range), or n (%). Comparisons were made for normally distributed variables by Students t-test, for non-normally distributed by Mann Whitney U test, and for categorical variables by χ2 test. All confidence intervals are 95%. To compare the development and validation cohorts a χ2 Goodness-of-fit test was applied. This required a logistic regression model predicting the outcome of [Cohort + AKI + Cohort*AKI], where Cohort equals 0 for the development cohort and 1 for the validation cohort, and the AKI cell equals 0 for no-AKI, 1 for Structural-AKI only, 2 for Functional-AKI only, and 3 for both Structural-AKI and Functional-AKI.

The EARLYARF trial was approved by the multiregional ethics committee of New Zealand (MEC/050020029) and registered under the Australian and New Zealand Clinical Trials Registry (ACTRN012606000032550; Screening on entry to ICU was by presumptive consent, followed by written consent from the patient or family.


The clinical characteristics of the EARLYARF patients have been previously described [16], [17], [23]. Briefly, the 507 patients with available urinary NGAL data were; 39.4% female, 60±18 years of age, 18.1% with Chronic Kidney Disease, 18.9% with sepsis, with a median baseline plasma creatinine of 76 µmol/l (IQR: 60–92 µmol/l), and with a mean APACHE II score of 17.9±6.3. The development cohort had 254 patients and the validation cohort 253 patients. There were no differences in sex, age, weight, Chronic Kidney Disease, baseline plasma creatinine, estimated GFR, APACHE II score, SOFA score, sepsis or NGAL concentrations between the two cohorts (Table 1).

In the development cohort 110 of 254 (43%) patients had Functional-AKI and 28 of these needed dialysis or died within 30 days. Seventeen patients without Functional-AKI also needed dialysis or died. The sensitivity was thus 62%.

The threshold for urinary NGAL with 62% sensitivity for need for dialysis or death in 30 days was 140 ng/ml. We therefore defined Structural-AKI as NGAL >140 ng/ml. This threshold resulted in 26% (n = 38) of the no Functional-AKI (n = 144) being diagnosed as Structural-AKI (Table 2). Compared with the No-AKI reference group, the relative risk of dialysis or death within 30 days was significantly increased for those with Structural-AKI, Functional-AKI or both (Table 3).

Table 2. Patients in the Development cohort versus Validation cohort, n (% of total patients in each cohort).

Table 3. Patients having dialysis or death as an outcome, n (% of patients with each diagnosis), and relative risk, RR (95% Confidence interval), in each AKI category in the Development versus Validation cohorts.

The validation cohort was similar to the development cohort. There was no difference in the proportions of Structural-AKI (urinary NGAL>140 ng/ml) and Functional-AKI to the development cohort (p = 0.76, Table 2) or of those who needed dialysis or died (p = 0.56, Table 3). The relative risk of dialysis or death was greatest in the cohort with both Structural-AKI and Functional-AKI in both development and validation cohorts (Table 3). The risk of death or dialysis in the validation cohort was not different from that of the development cohort (p = 0.87). The sensitivity for AKI (either Structural or Functional) was 76% (31 died or needed dialysis with AKI, 10 without).

This method of determining thresholds may be extended to determine biomarker thresholds for each severity stage of Functional-AKI. Using the entire cohort we determined that the threshold equivalents for KDIGO Functional-AKI for NGAL were for: Stage 1, 140 ng/ml; Stage 2, 438 ng/ml; Stage 3, 2710 ng/ml. These are presented in Table 4 along with the thresholds for urinary alkaline phosphatase, γ-glutamyl transpeptidase, cystatin C, interleukin-18, kidney injury molecule-1, and α- and π-glutathione-S-transferase.

Table 4. Structural-AKI thresholds for severity stages based on equivalent sensitivity (62%) to Functional-AKI.


Literature thresholds for urinary NGAL diagnosis of AKI vary with cause and context of AKI and range from 72 ng/ml in children after cardio-pulmonary bypass surgery [24] to 680 ng/ml in adults after cardio-pulmonary bypass surgery [25]; Using these extremes in the two studies cited, the demonstrated sensitivities for the subsequent creatinine-based diagnosis of AKI ranged respectively from 42% to 62.5%. In our illustration, the threshold selected in a heterogenous ICU population using a sensitivity of 62% was also based on a currently accepted consensus definition of Functional-AKI. While this hypothesis clearly requires external validation in much larger datasets, the data appear robust compared with literature values and allows determination of subset relative risk. The present dataset contained only sufficient patients for a power of 73% at an α of 0.05 for comparison of the validation cohort to the development cohort. A data set of 750 patients would be needed to extend this to a power of 90%. Ideally, disaggregated data from multiple data sets would be used to determined thresholds for each biomarker.

As expected, the use of injury biomarkers of AKI resulted in more people being diagnosed with AKI. However, by linking the diagnosis of Functional-AKI with Structural-AKI through sensitivity to the same hard outcomes, clinicians can be confident in the clinical implications of the detected increase in injury biomarker. The technique will also enable the use of multiple biomarkers, each of which potentially reflect a different mechanism and time course of injury. The demonstration, that a low urinary biomarker (NGAL) concentration reduces risk, illustrates additional potential benefits from this strategy. We consider it important that when applying this methodology to determine thresholds the biomarker reflects injury specific to the kidney and not some other disease, such as sepsis that may also be independently related to mortality. This may mean different thresholds in different cohorts, such as for urinary Cystatin C [26] or NGAL [27], [28] in sepsis, as well as recognising that the kidney injury “signal” may be drowned out by other sources of the biomarker in some circumstances, thus rendering the biomarker not diagnostic of Structural-AKI.

We proposed sensitivity of Functional-AKI for determining thresholds because early stage AKI management choices are low-risk (see KDIGO guidelines, figure four [6]) and relationship with outcome is well established and similarly low risk. Alternatives include defining the thresholds using specificity, a predetermined sensitivity or specificity, or optimisation for both sensitivity and specificity. Specificity should be considered if interventions are high risk. In our illustration, the specificity of Functional-AKI was low (60.8%) resulting in threshold for NGAL of 127 ng/ml, similar to that determined using sensitivity. Choosing a pre-specified higher specificity will increase thresholds of both Functional-AKI and Structural-AKI, whereas choosing a higher sensitivity will produce decreases. As the definition of Functional-AKI has been determined by consensus and based on evidence that an increase in creatinine of as little as 0.3 mg/dl is associated with poor outcomes [14], we do not propose that a pre-specified specificity or sensitivity be used. It is highly unlikely that a biomarker threshold for Structural-AKI could be chosen with the identical sensitivity and specificity as Functional-AKI. An alternative strategy is to determine the nearest point on the biomarker receiver operator characteristic (ROC) curve (Sensitivity verse 1-Specificity) that is closest to the point on the creatinine ROC with the sensitivity and 1-specificity of Functional-AKI. While this avoids having to choose either sensitivity or specificity it would introduce an imbalance between the performance of Functional-AKI and Structural-AKI in terms of relative risk for mortality and dialysis need.

It is a point of debate whether mortality alone or mortality and dialysis need together should be the outcome measure used. We proposed the combined outcome of need for dialysis and mortality because of clinical relevance and the association of both Functional-AKI and kidney injury biomarkers with both dialysis and mortality. A rapidly increasing creatinine is clearly an indication for dialysis. Multiple biomarkers originating in the kidney have been shown to be associated with dialysis need [12], [17]. Functional-AKI is known to be associated with mortality, indeed the threshold of an increased creatinine of 0.3 mg/dl was based on the association with mortality [14] and multiple kidney injury biomarkers have been shown to be associated with mortality [12], [17]. For biomarkers that are associated with disease states other than kidney injury, it may be that there is some bias introduced because of the biomarker reflecting other illness in addition to kidney injury. This means that some of the mortality associated with the biomarker may not be because of the severity of kidney injury. These specific circumstances need to be identified and, if possible, the bias needs to be quantified, before application of the proposed methodology.

As suggested by Siew et al there is a need to examine large datasets to assess the agreement or disagreement with creatinine data [29]. Important for the development of thresholds are datasets, which contain sample biomarker concentrations at multiple time points. Utilising the maximum concentration rather than concentration at any one time point, as we have done in this illustration, will minimise miss-classifying patients merely because of the timing of the biomarker sample. Figure 1 illustrates how one patient may be classified to each of the four possible diagnostic classes if classification is made only on the basis of one sample at a single time point. When viewed in hindsight and with the benefit of all time points, the patient clearly had both a loss of GFR and structural injury caused by the cardiac arrest. The type of AKI diagnosed (Structural, Functional or both) is therefore merely a function of the timing of sampling, given the differences in temporal profiles of serum creatinine and urinary NGAL. This also highlights that if there is a clinical need for identifying both functional change and injury there is a need for serial sampling. Our method illustrates how these datasets may be used to establish injury biomarker thresholds. This will allow comparison of thresholds for different aetiologies of AKI and in different patient groups, for example those with and without sepsis.

Figure 1. Illustrative biomarker time course following a cardiac arrest.

Baseline creatinine was 96 µmol/l. Horizontal dotted lines represent the thresholds for Functional-AKI (26.4 µmol/l increase over baseline) and Structural-AKI (140 ng/ml). If the diagnosis of Structural-AKI and Functional-AKI were to be made at only one time point then the patient would be initially negative for both classifications before becoming positive for Structural-AKI for a short period whilst remaining negative for Functional-AKI. From 2 to 16 hours the patients is positive for both Structural and Functional-AKI before becoming negative again for Structural-AKI.

In conclusion, sensitivity to need for dialysis and death can be used to link and give equal weight to functional or structural biomarker-based definitions of AKI. This hypothesis awaits validation in large multicentre datasets.


Our thanks go to those who provided assays for the EARLYARF trial: Professor Prasad Devarajan (Urinary NGAL: Cincinnati Children's Hospital), Argutus Medical Ltd (GST: Dublin), Professor Joseph Bonventre (KIM-1: Brigham and Woman's Hospital, Harvard), Professor Charles Edelstein (IL-18: University of Colorado), and Canterbury Health Laboratories. Urinary NGAL kits for the data for figure 1 were provided by Abbott Diagnostics.

Author Contributions

Conceived and designed the EARLYARF trial: ZHE. Conceived the analysis: JWP. Interpreted the analysis: JWP ZHE. Wrote the paper: JWP ZHE.


  1. 1. Ricci Z, Cruz DN, Ronco C (2008) The RIFLE criteria and mortality in acute kidney injury: A systematic review. Kidney Int 73: 538–546
  2. 2. Coca SG, Yusuf B, Shlipak MG, Garg AX, Parikh CR (2009) Long-term Risk of Mortality and Other Adverse Outcomes After Acute Kidney Injury: A Systematic Review and Meta-analysis. Am J Kid Dis 53: 961–973
  3. 3. Bellomo R, Ronco C, Kellum JA, Mehta RL, Palevsky PM, et al. (2004) Acute renal failure – definition, outcome measures, animal models, fluid therapy and information technology needs: the Second International Consensus Conference of the Acute Dialysis Quality Initiative (ADQI) Group. Crit Care 8: R204–R212
  4. 4. Pickering JW, Endre ZH (2009) GFR shot by RIFLE: errors in staging acute kidney injury. Lancet 373: 1318–1319
  5. 5. Mehta RL, Kellum JA, Shah SV, Molitoris BA, Ronco C, et al. (2007) Acute Kidney Injury Network: report of an initiative to improve outcomes in acute kidney injury. Crit Care 11: R31
  6. 6. KDIGO (2012) Clinical Practice Guideline for Acute Kidney Injury Section 2: AKI Definition. Kidney Int Suppl 2: 19–36. doi:10.1038/kisup.2011.32.
  7. 7. Mishra J, Dent CL, Tarabishi R, Mitsnefes MM, Ma Q, et al. (2005) Neutrophil gelatinase-associated lipocalin (NGAL) as a biomarker for acute renal injury after cardiac surgery. Lancet 365: 1231–1238
  8. 8. Endre ZH, Pickering JW (2013) Acute kidney injury clinical trial design: old problems, new strategies. Pediatr Nephrol 28: 207–217
  9. 9. Kümpers P, Hafer C, Lukasz A, Lichtinghagen R, Brand K, et al. (2010) Serum neutrophil gelatinase-associated lipocalin at inception of renal replacement therapy predicts survival in critically ill patients with acute kidney injury. Crit Care 14: R9
  10. 10. Nejat M, Pickering JW, Devarajan P, Bonventre JV, Edelstein CL, et al. (2012) Some biomarkers of acute kidney injury are increased in pre-renal acute injury. Kidney Int 81: 1254–1262
  11. 11. Haase-Fielitz A, Bellomo R, Devarajan P, Bennett M, Story D, et al. (2009) The predictive performance of plasma neutrophil gelatinase-associated lipocalin (NGAL) increases with grade of acute kidney injury. Nephrol Dial Transpl 24: 3349–3354
  12. 12. Haase M, Devarajan P, Haase-Fielitz A, Bellomo R, Cruz DN, et al. (2011) The outcome of neutrophil gelatinase-associated lipocalin-positive subclinical acute kidney injury a multicenter pooled analysis of prospective studies. J Am Coll Cardiol 57: 1752–1761
  13. 13. Waikar SS, Betensky RA, Emerson SC, Bonventre JV (2012) Imperfect gold standards for kidney injury biomarker evaluation. J Am Soc Nephrol 23: 13–21
  14. 14. Chertow GM, Burdick E, Honour M, Bonventre JV, Bates D (2005) Acute kidney injury, mortality, length of stay, and costs in hospitalized patients. J Am Soc Nephrol 16: 3365–3370
  15. 15. Lassnigg A, Schmid ER, Hiesmayr M, Falk C, Druml W, et al. (2008) Impact of minimal increases in serum creatinine on outcome in patients after cardiothoracic surgery: do we have to revise current definitions of acute renal failure? Crit Care Med 36: 1129–1137
  16. 16. Endre ZH, Walker RJ, Pickering JW, Shaw GM, Frampton CM, et al. (2010) Early intervention with erythropoietin does not affect the outcome of acute kidney injury (the EARLYARF trial). Kidney Int 77: 1020–1030
  17. 17. Endre ZH, Pickering JW, Walker RJ, Devarajan P, Edelstein CL, et al. (2011) Improved performance of urinary biomarkers of acute kidney injury in the critically ill by stratification for injury duration and baseline renal function. Kidney Int 79: 1119–1130
  18. 18. Bennett MR, Dent CL, Ma Q, Dastrala S, Grenier F, et al. (2008) Urine NGAL predicts severity of acute kidney injury after cardiac surgery: A prospective study. Clin J Am Soc Nephro 3: 665–673
  19. 19. Erlandsen EJ, Randers E, Kristensen JH (1999) Evaluation of the Dade Behring N Latex Cystatin C assay on the Dade Behring Nephelometer II System. Scand J Clin Lab Invest 59: 1–8.
  20. 20. Shibata M, Hirota M, Nozawa F, Okabe A, Kurimoto M, et al. (2000) Increased concentrations of plasma IL-18 in patients with hepatic dysfunction after hepatectomy. Cytokine 12: 1526–1530
  21. 21. Liangos O, Perianayagam MC, Vaidya VS, Han WK, Wald R, et al. (2007) Urinary N-acetyl-beta-(D)-glucosaminidase activity and kidney injury molecule-1 level are associated with adverse outcomes in acute renal failure. J Am Soc Nephrol 18: 904–912
  22. 22. Westhuyzen J, Endre ZH, Reece G, Reith DM, Saltissi D, et al. (2003) Measurement of tubular enzymuria facilitates early detection of acute renal impairment in the intensive care unit. Nephrol Dial Transpl 18: 543–551.
  23. 23. Ralib AM, Pickering JW, Shaw GM, Devarajan P, Edelstein CL, et al. (2012) Test Characteristics of Urinary Biomarkers Depend on Quantitation Method in Acute Kidney Injury. J Am Soc Nephrol 23: 322–333
  24. 24. Parikh CR, Devarajan P, Zappitelli M, Sint K, Thiessen-Philbrook H, et al. (2011) Postoperative Biomarkers Predict Acute Kidney Injury and Poor Outcomes after Pediatric Cardiac Surgery. J Am Soc Nephrol. doi:10.1681/ASN.2010111163.
  25. 25. Haase M, Bellomo R, Devarajan P, Schlattmann P, Haase-Fielitz A, et al. (2009) Accuracy of neutrophil gelatinase-associated lipocalin (NGAL) in diagnosis and prognosis in acute kidney injury: a systematic review and meta-analysis. Am J Kid Dis 54: 1012–1024
  26. 26. Nejat M, Pickering JW, Walker RJ, Westhuyzen J, Shaw GM, et al. (2010) Urinary cystatin C is diagnostic of acute kidney injury and sepsis, and predicts mortality in the intensive care unit. Crit Care 14: R85
  27. 27. Bagshaw SM, Bennett M, Haase M, Haase-Fielitz A, Egi M, et al. (2010) Plasma and urine neutrophil gelatinase-associated lipocalin in septic versus non-septic acute kidney injury in critical illness. Intens Care Med 36: 452–461
  28. 28. Guo Y, Yan K-P (2011) Prognostic significance of urine neutrophil gelatinase-associated lipocalin in patients with septic acute kidney injury. Exp Ther Med 2: 1133–1139
  29. 29. Siew ED, Ware LB, Ikizler TA (2011) Biological markers of acute kidney injury. J Am Soc Nephrol 22: 810–820