Author: Jeanne Lenzer
Position: medical investigative journalist
Institution: New York
E-mail: jeanne.lenzer@gmail.com
Additional Authors: Shannon Brownlee, senior fellow, New America Foundation, Washington, DC
Submitted Date: March 04, 2008
Published Date: March 4, 2008
This comment was originally posted as a “Reader Response” on the publication date indicated above. All Reader Responses are now available as comments.

Despite Kirsch’s meta-analysis, there remains some question about which subset(s) of depressed patients benefit and to what degree, from antidepressants. No single study, not even Kirsch’s, can lay that question to rest. What is important is that Kirsch’s study, like Erick Turner’s,(1) show that the effect of these drugs has been inflated by publication bias. That alone is an important observation in an era when “disease mongering” is now so common it deserves its own DSM designation.

It is ironic that many critics of Kirsch’s meta-analysis say the studies he reviewed were too short to show benefit. GlaxoSmithKline’s news release of 26 February states that “regulatory bodies…have concluded that these medicines provide benefit to patients.”(2) But the studies that the regulatory bodies used were the same ones that Kirsch used for his analysis, and if the studies were too short for Kirsch’s analysis, why were they long enough to show benefit for approval? In other words, if the studies were too short to show benefit – then why were the drugs approved? This seems like Catch-22.

What is alarming to us as observers of clinical research is that the US Food and Drug Administration can approve a drug based on one or two good studies – while numerous prior studies failed to show efficacy or safety. This astonishing reality cannot be adequately explained away by claims that certain trials are “pivotal” and others not – a designation that has led to some questionable approvals. According to Turner,(3) roughly one-half of the studies of antidepressants have been negative – and one-half have been positive. Were the ones that led to approval “pivotal” trials?

Several respondents to Kirsch’s study indicate the writer’s belief that clinical observation trumps clinical research, saying, “millions of content patients can't be that wrong” (Werner) and “Any competent physician who has any experience treating depression will completely disagree with the findings [of Kirsch]” (Vargas). But history is instructive; overwhelming expert consensus and clinical observation have been proven wrong time and time again. For instance, during FDA hearings on the COX-2 painkillers, rofecoxib (Vioxx), valdecoxib (Bextra), and celecoxib (Celebrex), rheumatologists lined up to claim that COX-2 inhibitors were somehow superior to non-selective anti-inflammatory drugs, providing enhanced efficacy and safety.(4) Yet both claims have been definitively disproved. If anything, COX-2 inhibitors are slightly less powerful in effect and certainly more harmful in side effects, causing the excess deaths of some 39,000 to 60,000 people.(5)

This isn’t to dismiss clinical observation, but clinical observations must be reconciled with study data, and study data must be reconciled with clinical observation. What to do when the two are in conflict? Keep working at resolution. One of the fascinating outcomes of the COX-2 debacle was how few observations a clinician can make in their lifetime regarding patient deaths and COX-2 inhibitors, explaining why the problem was missed. According to Dr Garret Fitzgerald, a cardiologist at the University of Pennsylvania who testified in the FDA hearings on Vioxx, “The best estimates [of cardiovascular events among users of Vioxx] are from 1-2%. Although this sums up to a lot of people, given the population exposure, there is no way an individual practitioner could notice this against the high background incidence of [myocardial infarction], heart failure and stroke in the exposed population.” In other words, unless clinical observations are pooled, an individual clinician is unlikely to pick up relatively rare side-effects.

It is equally disingenuous for critics to point to the studies Kirsch did not include. GlaxoSmithKline states Kirsch analyzed only a “small subset” of the “more than 170 trials” they performed.(2) However, Kirsch states, “We included all the studies that GlaxoSmithKline sent to the FDA. Most of the 170 studies were post-licensing and so were not sent to the FDA. Also, most were not placebo controlled trials or were for indications other than depression.”(6) Certainly, had Kirsch included studies for indications other than depression, or without placebo arms, the meta-analysis would have been, rightfully, discredited as meaningless.

The take home message of Kirsch’s analysis is that it is difficult if not impossible to come to conclusions about the relative merits and risk of medications when only parts (usually positive parts) of the data are available. The problem of publication bias is so powerful that it has certainly distorted interventions besides antidepressants – a problem we discuss in our commentary in this week’s BMJ: http://www.bmj.com/cgi/co...

References

(1) Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 2008; 358(3):252-260.

(2) GlaxoSmithKline statement: Public Library of Science Medicine article on antidepressant medicines. GlaxoSmithKline . 2-26-2008. http://www.gsk.com/media/... accessed 3/3/08.

(3) Interview with Erick Turner. 2-26-2008.

(4) Joint Meeting of the Arthritis Advisory Committee and the Drug Safety and Risk Management Advisory Committee. FDA . 2005. http://www.fda.gov/ohrms/... accessed 3/3/08.

(5) Graham DJ, Campen D, Hui R, Spence M, Cheetham C, Levy G et al. Risk of acute myocardial infarction and sudden cardiac death in patients treated with cyclo-oxygenase 2 selective and non-selective non-steroidal anti-inflammatory drugs: nested case-control study. Lancet 2005; 365(9458):475-481.

(6) Interview with Irving Kirsch via email. 3-3-2008.