Cabral et al state that the main finding of their meta-analysis is that PCT can be a simple and useful biomarker for early identification of sepsis in burn patients. This would be a clinically important result, but, unfortunately, it is based on the use of the flawed random effects model for meta-analysis. Two main issues with the random effects model are:
1) With increasing heterogeneity the random effects model moves from inverse variance study weights towards equal study weights. In practice this means that large studies get less weight (and small studies relatively more weight) as heterogeneity increases. There is no justification for this property of the random effects model.
In this case, the heterogeneity is so large that the study weights are virtually identical, as can be seen from Figure 5: with the exception of the very small Abdel Hafez study, all study weights are near 10%.
2) With increasing heterogeneity, the coverage of the random effects model drops below the nominal level, in this case 95%. So the confidence interval around the pooled effect estimate is too narrow.

We proposed and implemented in MetaXL an alternative to the random effects model, which we called the IVhet (inverse variance heterogeneity) model (Doi SA, Barendregt JJ, Khan S, Thalib L, Williams GM. Advances in the meta-analysis of heterogeneous clinical trials I: The inverse variance heterogeneity model. Contemp Clin Trials. 2015;45:130-8). The IVhet model keeps the study weights as in the inverse variance model, but uses a quasi-likelihood method to increase the confidence interval around the pooled estimate such that coverage remains on the nominal level.
When I re-calculate the data using the IVhet model, the pooled estimate becomes 1.04 (-.70, 2.77). So the effect size is less than half what it is with the random effects model, and it is no longer statistically significant. This is mostly because the large Cakir Madenci study now gets its proper inverse variance weight.


My conclusion is that the main finding of the Cabral et al study is likely to be not valid.


Jan J Barendregt, MA, PhD
Epigear International Pty Ltd
Email: j.barendregt@sph.uq.edu.au
Skype: janbarendregt
Phone: +61 7 3102 3093
Visit www.epigear.com: home of Ersatz, MetaXL, DisMod II, Prevent, and more!