Reader Comments

Post a new comment on this article

Authors' (Canutes?) respond

Posted by Hildabastian on 04 Oct 2010 at 07:44 GMT

A less utopian version of Cochrane’s challenge is unnecessary: rather only a precise reading of it. We need to concentrate on trials that are “relevant” [1]. The information needs of clinicians and patients will not be well served by removing the floodgates and admitting non-randomised studies of questionable relevance, any more than they are being well served by the plethora of randomised studies of questionable relevance.

Randomised and non-randomised studies frequently give different answers to the same question, and choices are required based on logic. It is a choice whether or not to recommend that postmenopausal women should take hormones to prevent cardiovascular disease, but the choice has mortal consequences.

We suggest that practice should be informed by non-randomised evidence in three main situations: (i) when estimates of the effects of an intervention are so large that it is inconceivable that they reflect bias [2]; (ii) when trials are of insufficient size to address questions about important possible uncommon adverse effects [3]; and (iii) that it really is inconceivable that randomised trials would ever be feasible, for example, to address uncertainties about how to treat white phosphorous burns sustained during an intensive bombing campaign.

As illustrated by numerous examples, acquiescing in lack of randomised trials when controlled experiments are, in principle, feasible, has resulted in unacceptable suffering and death. Cochrane’s suggestion offers protection against error – but also a vital and increasingly important triage principle for coping with information. The current bias and waste in the medical research enterprise has multiple layers [4]. But understanding causes and cures of our ignorance despite a flood of information is still worth more attention and effort than it currently gets. Like Canute, we are well aware that we cannot stop the tide. Getting more selective, though, and finding new ways to cooperate, might help us stay afloat.

1. Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med 7(9): e1000326.

2. Glasziou P, Chalmers I, Rawlins M, McCulloch P. When are randomised trials unnecessary? Picking signal from noise. BMJ. 2007 Feb 17;334(7589):349-351.

3. Vandenbroucke JP, Psaty BM. Benefits and risks of drug treatments: how to combine the best evidence on benefits with the best data about adverse effects. JAMA 2008; 300(20):2417-2419.

4. Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009 Jul 4;374(9683):86-89.

Competing interests declared: Authors' response.