Author: PJ Leonard
E-mail: coroebus@yahoo.com
Submitted Date: May 09, 2008
Published Date: May 12, 2008
This comment was originally posted as a “Reader Response” on the publication date indicated above. All Reader Responses are now available as comments.

Huedo-Medina et al report that decreasing placebo response with increasing baseline severity of depression is their paper’s “unique contribution”. However, it is clear that this is an artefact of using the standardised mean difference. Analyses using the raw HRSD scores reveal that mean placebo response remains constant with greater response to antidepressant treatment with increasing baseline severity [1].

Kirsch et al use a method designed to allow the inclusion of studies without placebo groups. This may be acceptable in some areas of psychology and education but is not in meta-analyses of double blind randomised controlled trials. Jim Young (March 27th) notes, "if it were possible to measure separate effects of treatment and placebo within each trial, then there would be no need for the placebo group at all." Kirsch et al must assume that the placebo response is uniform between studies [2] which is unlikely to be the case with this data (see Figure 2 and ref [1]) where “amounts of change for drug and placebo groups varied widely around their respective means”. This heterogeneity is particularly deleterious in this paper due to differing sample sizes between drug and placebo arms (PJ Leonard, March 17th).

Although more conventional analyses confirm the trend for greater effect size with increasing baseline severity (but see above), they also indicate that the methodology of Kirsch et al underestimates the effect size. Contrary to the reported 1.8 HRSD point effect, I [3] and Robert Waldmann (March 25th) find an effect of 2.7. It is important to accurately estimate the effect size rather than simply using NICE’s arbitrary ‘clinical significance’ threshold because there is no consensus as to what constitutes ‘clinical significance’ [4].

This paper states "there seems little evidence to support the prescription of antidepressant medication to any but the most severely depressed patients" but only analyses studies of the most severely depressed patients, with all but two studies in the analysis from subjects with mean baseline HRSD scores in the APA/NICE 'very severe' (>=23) range with one study in the 'severe' (19-22) range and one in the 'moderate' range (14-18) (note that these labels may be misleading with the 'severe' group better thought of as having moderate depression in clinical practice [4]). Thus roughly half the studies were in the range of severity exceeding the threshold for 'clinical significance' on the regression analysis [1] (consistent with Figure 4). We should be wary of extrapolating the regression line outside of the region of 'very severe' depression (clinically severe depression) and making recommendations about the 'clinical significance' of antidepressant drugs in 'severe' and 'moderate' depression (clinically moderate to mild depression) since the regression is derived from studies of 'very severe' patients. Therefore the FDA pre-licensing studies are not the place to look for data on the efficacy of new antidepressants in ‘severe’ and ‘moderate’ depression, and we should look elsewhere for this information.

Note that my 95% confidence intervals do not exclude an effect size of 3 points, similarly for the regression. It is worth noting that NICE are much more circumspect in dealing with statistically significant effects where the point estimates do not exceed 'clinical significance' thresholds but the confidence intervals do not exclude it: "There is evidence suggesting that there is a statistically significant difference between x and y but there is insufficient evidence to determine its clinical significance." [5].

[1] http://pyjamasinbananas.b...
[2] Morris & DeShon (2002) Psychological Methods 7:105-125.
[3] http://pyjamasinbananas.b...
[4] Moncrieff & Kirsch (2005) BMJ 331(7509):155-7.
[5] NICE (2004) National Clinical Practice Guideline Number 23.