Citation: Nissen SE (2006) ADAPT: The Wrong Way to Stop a Clinical Trial. PLOS Clin Trial 1(7): e35. https://doi.org/10.1371/journal.pctr.0010035
Published: November 17, 2006
Copyright: © 2006 Steven E. Nissen. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: The author received no specific funding for this article.
Competing interests: SEN is the principal investigator for a large clinical trial comparing cardiovascular event rates for three nonsteroidal anti-inflammatory agents: naproxen, ibuprofen, and celecoxib. Although the study is supported by Pfizer, SEN and the Executive Committee for the trial have stipulated that no member of the trial leadership may accept any compensation from the sponsor or any other maker of drugs in this class.
Abbreviations: ADAPT, Alzheimer's Disease Anti-Inflammatory Prevention Trial; DSMB, data safety and monitoring board; FDA, Food and Drug Administration; NIH, National Institutes of Health; NSAID, nonsteroidal anti-inflammatory agent
In an accompanying article published in PLoS Clinical Trials, the Alzheimer's Disease Anti-Inflammatory Prevention Trial (ADAPT) Research Group report the cardiovascular outcomes from their study . The circumstances surrounding the termination of ADAPT were unusual and provide an important lesson for all clinical trialists, demonstrating the importance of following rigorous procedures for prematurely stopping clinical trials. In this case, stopping the trial before its intended completion resulted in data that cannot be reliably interpreted.
On September 30, 2004, Merck withdrew rofecoxib (Vioxx) from the market after the trial's data safety and monitoring board (DSMB) recommended termination of a placebo-controlled study of this agent in the prevention of colon polyps . The reason for study termination was a statistically significant increase in adverse cardiovascular outcomes. On December 17, 2004, Pfizer announced that termination of a trial of celecoxib (Celebrex) in colon polyp prevention, because it also showed statistically significant evidence for increased cardiovascular event rates . The results of these two trials were subsequently published in February 2005 in the New England Journal of Medicine [4,5]. The revelations about increased cardiovascular events with these “coxibs” generated enormous public attention and concern. However, in both cases, the decision-making leading to trial discontinuation was handled appropriately through the regular reviews conducted by an independent DSMB.
Three days following the announcement of the termination of the celecoxib colon polyp prevention study, the National Institute of Health (NIH), issued a press release entitled “Use of Non-Steroid Anti-Inflammatory Drugs Suspended in Large Alzheimer's Disease Prevention Trial” . The NIH press release stated that “data from the ADAPT trial indicated an apparent increase in cardiovascular and cerebrovascular events among participants taking naproxen when compared to placebo.” In the press release, NIH Director Dr. Elias Zerhouni stated that “this step is being taken as a precautionary measure to ensure the safety of the study's participants” and that “the investigators made their decision based on the risk/benefit analysis specific to this trial.” Shortly following the NIH announcement, the Food and Drug Administration (FDA) issued a public statement that “based on emerging information from a long-term prevention trial, the risk of cardiovascular and cerebrovascular events may increase among patients taking naproxen” .
These announcements generated front-page headlines such as “Heart Risk Seen in Naproxen” (Wall Street Journal), “Tough Choice: Pain or Risk” (USA Today), and “Patients, Doctors Agonize over Risks of Painkiller (Los Angeles Times) . Occurring immediately following the revelations about rofecoxib and celecoxib, the naproxen announcement generated considerable public apprehension . Physicians received many urgent calls from worried patients. However, there was a major problem with the naproxen warning: it was not based upon the application of standard procedures for stopping an ongoing clinical trial. During the subsequent FDA hearings to set policy on nonsteroidal anti-inflammatory agents (NSAIDs) and COX-2 inhibitors, I described the warnings about naproxen as “the medical equivalent of yelling ‘fire' in a crowded auditorium” .
The Data Finally Become Available
Now, nearly two years after the closure of the ADAPT study, we finally get to see the data that resulted in the public warning about naproxen . For the standard composite endpoint of cardiovascular death, myocardial infarction, and stroke, there were 17 events in celecoxib treatment group, 23 in the naproxen arm, and 22 in the placebo. The hazard ratio for naproxen compared with placebo was 1.57 with 95% confidence intervals of 0.87 to 2.81, p = 0.13. A broader composite outcome that added heart failure and transient ischemic attack yielded a marginally significant p value when comparing naproxen with placebo.
We must ask whether a DSMB would stop an ongoing clinical trial for such findings. The answer, it appears, was that the DSMB did not stop the trial; NIH officials did . There was no regularly scheduled DSMB safety review that resulted in study termination. It appears that NIH officials, concerned about revelations regarding the safety of coxibs in the colon polyps studies, simply decided to look at the cardiovascular event results in ADAPT [8,10]). Seeing a marginally significant difference between naproxen and placebo, they abruptly stopped the trial. Indeed, NIH officials went even further and issued the public warning .
Both the termination of this trial and the issuance of a public warning about naproxen were inappropriate and reflected faulty logic. These actions unnecessarily created public consternation and denied the participants in the trial the opportunity for their commitment to result in any benefit to themselves or society .
Reinforcing Rigorous Methods for Safety Monitoring of Ongoing Trials
Current NIH policies may have contributed to the decision to prematurely terminate ADAPT. Surprisingly, NIH rules allow the leadership of various scientific divisions to have unblinded access to data. Even more surprisingly, principal investigators are also often unblinded. This policy provides for considerable temptation to interfere with trial management. It is well recognized that during the conduct of randomized trials that hazard ratios are often unstable, sometimes drifting over time into marginal levels of “significance.” This is an artifact of statistics. If you repeatedly sample data, the multiplicity of “looks” at the data ensure that there will occasionally appear a transient “signal” of benefit or harm. These “signals” are not reliable.
We have many sources of evidence that refute the notion that naproxen increases the risk of adverse cardiovascular outcomes. A recent meta-analysis by McGettigan and Henry  examined the relative cardiovascular risks for 23 studies of NSAIDs and COX-2 inhibitors. The relative risk for naproxen was 0.97 with 95% confidence intervals ranging from 0.87 to 1.07. The FDA Advisory Panel meeting in February 2005 opined that naproxen was the most appropriate comparator with which to evaluate the relative risks of new anti-inflammatory agents .
To avoid inappropriate premature termination of trials, statisticians and clinical trialists long ago adopted rigorous “stopping rules” for safety monitoring of ongoing trials [13–17]. These rules require a much higher level of significance for harm or benefit earlier in the trial. The most common approach, originally described by O'Brien and Fleming , would have precluded early stopping of ADAPT and avoided the generation of spurious findings. Strict investigator discipline and appropriate prespecification of stopping rules are required to enforce this type of rigorous approach.
Who shares responsibility for the improper termination of ADAPT? In my opinion, a major factor was the inappropriate unblinding of the trial by leadership at the NIH. Allowing unrestricted access to the study data by NIH officials represents an unwise policy and can only lead to errors of this kind.
The circumstances surrounding the termination of ADAPT demonstrate the importance of following rigorous procedures for prematurely stopping clinical trials.
Futhermore, the principal investigator and the Steering Committee should have, when approached by NIH officials about premature examination of the trial data, said “no.” Similarly, the DSMB should have resisted efforts to improperly interrupt an ongoing trial. If rebuffed by NIH officials, both the Steering Committee and the DSMB should have publicly disavowed the decision to terminate the trial or, alternatively, explained that the reasons for termination did not include the finding of a hazard for naproxen treatment.
Management of clinical trials is a major public responsibility. It takes self-discipline and a precise understanding of statistical methods. For ADAPT, accepted scientific procedures were not followed, resulting in an inappropriate public warning. Accordingly, the trial results cannot be reliably interpreted.
- 1. ADAPT Research Group (2006) Cardiovascular and cerebrovascular events in the randomized, controlled Alzheimer's Disease Anti-inflammatory Prevention Trial (ADAPT). PLoS Clin Trials. 1. ADAPT Research Group2006Cardiovascular and cerebrovascular events in the randomized, controlled Alzheimer's Disease Anti-inflammatory Prevention Trial (ADAPT).PLoS Clin Trials1e33.
- 2. Merck Press Office (2004) Merck announces voluntary worldwide withdrawal of VIOXX. Merck Press Office2004Merck announces voluntary worldwide withdrawal of VIOXXAvailable: http://www.vioxx.com/vioxx/documents/english/vioxx_press_release.pdf#search=%22Merck%20withdrawal%20Vioxx%22. Accessed 19 October 2006. Available: http://www.vioxx.com/vioxx/documents/english/vioxx_press_release.pdf#search=%22Merck%20withdrawal%20Vioxx%22. Accessed 19 October 2006.
- 3. Pfizer Press Office (2004) Pfizer statement on new information regarding cardiovascular safety of Celebrex. Pfizer Press Office2004Pfizer statement on new information regarding cardiovascular safety of Celebrex.Available: http://www.pfizer.com/pfizer/are/investors_releases/2004pr/mn_2004_1217.jsp. Accessed 19 October 2006. Available: http://www.pfizer.com/pfizer/are/investors_releases/2004pr/mn_2004_1217.jsp. Accessed 19 October 2006.
- 4. Solomon SD, McMurray JJ, Pfeffer MA, Wittes J, Fowler R, et al. (2005) Cardiovascular risk associated with celecoxib in a clinical trial for colorectal adenoma prevention. N Engl J Med 352: 1071–1080.SD SolomonJJ McMurrayMA PfefferJ. WittesR. Fowler2005Cardiovascular risk associated with celecoxib in a clinical trial for colorectal adenoma prevention.N Engl J Med35210711080
- 5. Bresalier RS, Sandler RS, Quan H, Bolognese JA, Oxenius B, et al. (2005) Cardiovascular events associated with rofecoxib in a colorectal adenoma chemoprevention trial. N Engl J Med 352: 1092–1102.RS BresalierRS SandlerH. QuanJA BologneseB. Oxenius2005Cardiovascular events associated with rofecoxib in a colorectal adenoma chemoprevention trial.N Engl J Med35210921102[Erratum: N Engl J Med 355: 221.]. [Erratum: N Engl J Med 355: 221.].
- 6. NIH Office of Communications (2004) Use of non-steroidal anti-inflammatory drugs suspended in large Alzheimer's disease prevention trial. NIH Office of Communications2004Use of non-steroidal anti-inflammatory drugs suspended in large Alzheimer's disease prevention trial.Available: http://www.nih.gov/news/pr/dec2004/od-20.htm. Accessed 19 October 2006. Available: http://www.nih.gov/news/pr/dec2004/od-20.htm. Accessed 19 October 2006.
- 7. United States Food and Drug Administration (2004) FDA statement on naproxen. United States Food and Drug Administration2004FDA statement on naproxenAvailable: http://www.fda.gov/bbs/topics/news/2004/NEW01148.html. Accessed 19 October 2006. Available: http://www.fda.gov/bbs/topics/news/2004/NEW01148.html. Accessed 19 October 2006.
- 8. Woloshin S, Schwartz LM, Welch HG2005 Feb 1). Where the naproxen story went wrong. Washington Post. S. WoloshinLM SchwartzHG Welch200521)Where the naproxen story went wrong.Washington Postpage HE01. Available: http://www.washingtonpost.com/ac2/wp-dyn/A52625-2005Jan31?language=printer. Accessed 19 October 2006. page HE01. Available: http://www.washingtonpost.com/ac2/wp-dyn/A52625-2005Jan31?language=printer. Accessed 19 October 2006.
- 9. Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research ((2005) Joint meeting of the Arthritis Advisory Committeee and the Drug Safety and Risk Management Advisory Committee, Volume III. Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research(2005Joint meeting of the Arthritis Advisory Committeee and the Drug Safety and Risk Management Advisory Committee, Volume IIIAvailable: http://www.fda.gov/ohrms/dockets/ac/05/transcripts/2005–4090T3.htm. Accessed 19 October 2006. Available: http://www.fda.gov/ohrms/dockets/ac/05/transcripts/2005–4090T3.htm. Accessed 19 October 2006.
- 10. Herper M2005 Feb 18). The naproxen risk that wasn't. Forbes. M. Herper2005218)The naproxen risk that wasn't.ForbesAvailable: http://www.forbes.com/sciencesandmedicine/2005/02/18/cx_mh_0218naproxen.html. Accessed 19 October 2006. Available: http://www.forbes.com/sciencesandmedicine/2005/02/18/cx_mh_0218naproxen.html. Accessed 19 October 2006.
- 11. Zwillich T ((2005) Naproxen warning unjustified, say FDA experts: Report linking Aleve to heart attacks unnecessarily scared public. T. Zwillich(2005Naproxen warning unjustified, say FDA experts: Report linking Aleve to heart attacks unnecessarily scared publicAvailable: http://www.medicinenet.com/script/main/art.asp?articlekey=43623. Accessed 19 October 2006. Available: http://www.medicinenet.com/script/main/art.asp?articlekey=43623. Accessed 19 October 2006.
- 12. McGettigan P, Henry D (2006) Cardiovascular risk and inhibition of cyclooxygenase: A systematic review of the observational studies of selective and nonselective inhibitors of cyclooxygenase 2. JAMA 296: 1633–1644.P. McGettiganD. Henry2006Cardiovascular risk and inhibition of cyclooxygenase: A systematic review of the observational studies of selective and nonselective inhibitors of cyclooxygenase 2.JAMA29616331644
- 13. O'Brien PC, Fleming TR (1979) A multiple testing procedure for clinical trials. Biometrics 35: 549–556.O'Brien PC, Fleming TR1979A multiple testing procedure for clinical trials.Biometrics35549556
- 14. Pocock SJ (1982) Interim analyses for randomized clinical trials: The group sequential approach. Biometrics 38: 153–162.SJ Pocock1982Interim analyses for randomized clinical trials: The group sequential approach.Biometrics38153162
- 15. Goldman AI (1987) Issues in designing sequential stopping rules for monitoring side effects in clinical trials. Control Clin Trials 8: 327–337.AI Goldman1987Issues in designing sequential stopping rules for monitoring side effects in clinical trials.Control Clin Trials8327337
- 16. Emerson SS (1995) Stopping a clinical trial very early based on unplanned interim analyses: A group sequential approach. Biometrics 51: 1152–1162.SS Emerson1995Stopping a clinical trial very early based on unplanned interim analyses: A group sequential approach.Biometrics5111521162
- 17. Geller NL, Pocock SJ (1987) Interim analyses in randomized clinical trials: Ramifications and guidelines for practitioners. Biometrics 43: 213–223.NL GellerSJ Pocock1987Interim analyses in randomized clinical trials: Ramifications and guidelines for practitioners.Biometrics43213223