The authors have declared that no competing interests exist.
Literature search: AJB. Study management: AJB. Data abstraction: AJB WW. Resolution of disagreements: AJB WW CB. Interpretation of data: AJB, WW, PR, GA, CB. Conceived and designed the experiments: AJB WW PR GA CB. Performed the experiments: AJB WW CB. Analyzed the data: AJB. Wrote the paper: AJB WW PR GA CB.
A key priority in developing policies for providing affordable cancer care is measuring the value for money of new therapies using cost-effectiveness analyses (CEAs). For CEA to be useful it should focus on relevant outcomes and include thorough investigation of uncertainty. Randomized controlled trials (RCTs) of five years of aromatase inhibitors (AI) versus five years of tamoxifen in the treatment of post-menopausal women with early stage breast cancer, show benefit of AI in terms of disease free survival (DFS) but not overall survival (OS) and indicate higher risk of fracture with AI. Policy-relevant CEA of AI versus tamoxifen should focus on OS and include analysis of uncertainty over key assumptions.
We conducted a systematic review of published CEAs comparing an AI to tamoxifen. We searched Ovid MEDLINE, EMBASE, PsychINFO, and the Cochrane Database of Systematic Reviews without language restrictions. We selected CEAs with outcomes expressed as cost per life year or cost per quality adjusted life year (QALY). We assessed quality using the Neumann checklist. Using structured forms two abstractors collected descriptive information, sources of data, baseline assumptions on effectiveness and adverse events, and recorded approaches to assessing parameter uncertainty, methodological uncertainty, and structural uncertainty.
We identified 1,622 citations and 18 studies met inclusion criteria. All CE estimates assumed a survival benefit for aromatase inhibitors. Twelve studies performed sensitivity analysis on the risk of adverse events and 7 assumed no additional mortality risk with any adverse event. Sub-group analysis was limited; 6 studies examined older women, 2 examined women with low recurrence risk, and 1 examined women with multiple comorbidities.
Published CEAs comparing AIs to tamoxifen assumed an OS benefit though none has been shown in RCTs, leading to an overestimate of the cost-effectiveness of AIs. Results of these CEA analyses may be suboptimal for guiding policy.
There is growing concern over the ability, even in high income countries, to deliver affordable cancer care.
Cost-effectiveness analysis (CEA) is the comparative assessment of two or more interventions in terms of costs and benefits.
In trial-based CEAs, information on health care costs and health outcomes are collected during the course of a randomized controlled trial (RCT).
Cancer drug therapies are well studied in randomized controlled trails (RCTs) and RCTs are a key source of evidence incorporated into cancer CEAs. However, there are limitations to the data from RCTs. One issue is that they often use surrogate endpoints rather than overall survival (OS). Improvements in survival are a clear benefit to patients whereas the benefits of surrogate endpoints such as DFS are less clear. The studies evaluating five years of aromatase inhibitors compared to five years of tamoxifen in the treatment of post-menopausal women diagnosed with early stage hormone receptor positive breast cancer used disease free survival (DFS) rather than OS. Aromatase inhibitors reduced the risk of breast cancer recurrence when compared to tamoxifen.
Another well recognized problem is that RCTs of cancer drugs often do not provide detailed or complete data on adverse effects.
This paper provides a systematic review of CEA studies of AI versus tamoxifen. We describe the overall quality of these studies, focusing specifically on how uncertainty is assessed and its potential impact on study conclusions and policy implications.
We conducted a systematic review of the published literature to identify CEAs addressing first line hormonal therapy for early stage breast cancer in women. We searched the following electronic databases: MEDLINE (1996–March 9 2011), EMBASE (1996–February 2011), PsychINFO (1996–March Week 1 2011), and the Cochrane Database of Evidence-Based Medicine Reviews (1996–February 2011). (
Two reviewers (AJB, WW) independently screened titles and abstracts of identified citations for relevance. Full texts of potentially relevant articles were retrieved and independently screened by both reviewers. Disagreements were resolved through consultation with a third reviewer (CB).
We appraised the studies according to general guidelines for the conduct and reporting of cost-effectiveness analysis using the tool developed by Neumann et al.
We abstracted characteristics of the identified studies including publication year, country, comparators, outcomes (life years, QALYs, or both), model perspective, type of model, study sponsorship, time horizon and discount rate.
We abstracted incremental cost effectiveness ratios (ICERs) and the incremental cost-utility ratio (ICUR) from each study. To allow direct comparison across countries and years we converted the ICERs and ICURs to a common year and currency (2010 US Dollars). We first converted to US dollars using the Purchasing Power Parities (PPPs) for health from the World Bank.
We used a previously identified framework to characterize uncertainty in the context of CEA. Uncertainty can be divided into three categories : 1) parameter uncertainty, 2) structural uncertainty, and 3) methodological uncertainty.
We abstracted the source of data on breast cancer recurrence risk and harms associated with hormonal therapies (single RCT, meta-analysis, risk model, observational data, or a combination of sources). For example, relative risk estimates from RCTs could be combined with observational data on baseline risk that better reflects patients in practice.
We assessed the authors handling of structural uncertainty by abstracting what adverse events the authors incorporated into the CEA models and recording whether or not increased mortality following adverse events was incorporated into the models. For example, a systematic review of the literature demonstrated that for older patients mortality rates can increase more than five-fold in the three months following hip fracture and an elevated risk of death lasts for many years.
We assessed methodological uncertainty by determining whether or not authors conducted scenario analyses or sub-group analyses to look at cost-effectiveness in special populations, such as older women, women with co-morbidities or women at high risk of fracture. We also assessed methodological uncertainty by determining whether or not the authors performed sensitivity analysis on the discount rate or the method of extrapolating short term trial data over the long-term.
Two reviewers (AJB, WW) performed data abstraction and quality appraisal, with disagreements resolved through consultation with a third reviewer (CB).
We identified 1,622 non-duplicate citations, of which 40 were recognized as potentially relevant and the full text articles retrieved. (
PRISMA indicates Preferred Reporting Items for Systematic Reviews and Meta-Analyses.
A total of 18 articles were included in the final study, of which 16 were published in English, one in Spanish and one in Italian.
Study Characteristics | |
Country/Region | |
Euro zone | 5 (28%) |
United States | 3 (17%) |
United Kingdom | 3 (17%) |
Canada | 3 (17%) |
Brazil | 2 (11%) |
Colombia | 1 (6%) |
Korea | 1 (6%) |
Publication Year | |
2004–2007 | 11 (61%) |
2008–2010 | 7 (39%) |
Comparators | |
Tamoxifen, Anastrazole | 13 (72%) |
Tamoxifen, Anastrazole,Letrozole | 3 (17%) |
Tamoxifen, Letrozole | 2 (11%) |
Perspective | |
Health Care Payer | 16 (89%) |
Societal | 1 (6%) |
Multiple perspectives | 1 (6%) |
Type of Model | |
Markov cohort model | 17 (94%) |
Unclear | 1(6%) |
Sponsorship | |
Industry | 11 (61%) |
Government fundingagency | 3 (17%) |
Not Stated | 3 (17%) |
Other | 1 (6%) |
Outcomes | |
QALYs and Life Years | 10 (56%) |
QALYs | 6 (33%) |
Life years | 2 (11%) |
Time Horizon | |
Lifetime | 2 (11%) |
50 Years | 1 (6%) |
35 Years | 2 (11%) |
30 Years | 3 (17%) |
25 Years | 4 (22%) |
20 Years | 5 (28%) |
Less than 10 Years | 1 (6%) |
Discount Rate | |
3% | 9 (50%) |
3.5% | 3 (17%) |
5% | 3 (5%) |
Other |
2 (11%) |
Not Reported | 1 (6%) |
QALYs indicates quality adjusted life years.
The discount rates in these studies was different for costs and benefits.
Detailed information on study characteristics is available in
Detailed items on the Neumann critical appraisal instrument are available online in
When comparing anastrazole to tamoxifen, ICERs ranged from $77 to $97,202 per life year and ICURs ranged from $7,351 to $151,608 per QALY. (
No. | Author | Undiscounted life years gained | ANA vs TAM | LET vs TAM | |||
(ANAvs TAM) | (LET vs TAM) | ICER | ICUR | ICER | ICUR | ||
1 | Delea1 | 0·68 | − | − | $22,209 USD 2005 $24,797 USD 2010 | $23,743 USD 2005 $26,509 USD 2010 | |
2 | Delea2 | 0·77 | − | − | $22,038 CAD 2005 $24,109 USD 2010 | $23,662 CAD 2005 $25,886 USD 2010 | |
3 | Fonseca3 | R$27,327 BRL 2005 $42,870 USD 2010 | − | − | − | ||
4 | Gamboa4 | 0·49 | $37,071 COP 2007 $77 USD 2010 | − | − | − | |
5 | Gil5 | 0·54 | €33,282 EUR 2004 $80,763 USD 2010 | €62,477 EUR 2004 $151,608 USD 2010 | − | − | |
6 | Hillner6 | 0·17 | $40,600 USD 2002a $49,211 USD 2010 | $75,900 USD 2002a $91,998 USD 2010 | − | − | |
7 | Hind7 | 0·08 | 0·16 | £36,225 GBP 2004 $97,202 USD 2010 | £31,965 GBP 2004 $85,771 USD 2010 | £22,837 GBP 2004 $61,278 USD 2010 | £21,580 GBP 2004 $57,905 USD 2010 |
8 | Karnon8 | 0·42 | 0·59 | £11,703 GBP 2005 $30,373 USD 2010 | £11,428 GBP 2005 $29,660 USD 2010 | £10,502 GBP 2005 $27,256 USD 2010 | £10,379 GBP 2005 $26,937 USD 2010 |
9 | Lazzaro9 | − | €47,556 EUR 2005 $74,868 USD 2010 | − | − | ||
10 | Lee10 | − | − | ||||
11 | Locker11 | 0·22 | $23,541 USD 2003 $27,898 USD 2010 | $20,246 USD 2003 $23,993 USD 2010 | − | − | |
12 | Lux12 | €21,069 EUR 2008 $37,181 USD 2010 | − | − | |||
13 | Mansel13 | 0·23 | £18,702 GBP 2004 $50,183 USD 2010 | £17,656 GBP 2004 $47,376 USD 2010 | − | − | |
14 | Moeremans14 | 0·35 | €4,233 EUR 2004a $7,862USD 2010 | €3,958 EUR 2004a $7,351 USD 2010 | − | − | |
15 | Rocchi15 | 0·39 | $30,000 CAD 2004 $33,931 USD 2010 | $28,000 CAD 2004 $31,669 USD 2010 | − | − | |
16 | Sasse16 | R$ 32,403 BRL 2005 $50,834 USD 2010 | − | − | |||
17 | Skedgel17 | $27,622 CAD 2005 $30,218 USD 2010 | − | − | |||
18 | Skedgel18 | €19,982 EUR 2005 $35,897 USD 2010 | − | − |
The majority of CEA authors took estimates of the impact of hormonal therapies on breast cancer recurrence from a single RCT, without developing a natural history model based on other data sources. (n = 14) (
Category | |
Data sources | |
Recurrence rates | |
Single RCT | 14 (78%) |
Combination of data sources |
4 (22%) |
Meta-analysis | 0 (0%) |
Adverse event rates | |
Single RCT | 9 (50%) |
Meta-analysis | 0 (0%) |
Combination of data sources | 8 (44%) |
Other | 1 (6%) |
Handling of parameter uncertainty | |
Performed sensitivity analysis on the risk of breast cancer recurrence | 10 (56%) |
Performed sensitivity analysis on the adverse events | 12 (67%) |
Performed probabilistic sensitivity analysis | 11 (61%) |
Performed value of information analysis | 0 (0%) |
Handling of structural uncertainty | |
Incorporated increased mortality following any adverse event? | 11 (61%) |
Incorporated increased mortality following | |
Fracture | 6 (33%) |
Cardiovascular Events | 3 (17%) |
Stroke | 0 (0%) |
Thromboembolism | 4 (22%) |
Endometrial Cancer | 4 (22%) |
Handling of methodological uncertainty | |
Addressed the following sub-groups | |
Older cohorts of women | 6 (33%) |
Women at low risk of breast cancer recurrence | 2 (11%) |
Women at high risk of fracture | 1 (6%) |
Women with high risk of cardiovascular disease | 0 (0%) |
Women at high risk of stroke | 0 (0%) |
Women at high risk of thromboembolism | 0 (0%) |
Women at high risk of endometrial cancer | 0 (0%) |
Women with multiple co-morbid diseases | 1 (6%) |
Performed sensitivity analysis on the method of extrapolating breast cancer recurrence rates beyond thefollow-up time of available studies (n = 17) |
11 (65%) |
Performed sensitivity analysis on the discount rate | 13 (72%) |
Authors combined observational data or a risk model with RCT data.
One study did not extrapolate beyond the time horizon of the trial data used in construction of the model. (Lazzaro et al
Half of the studies took data on adverse events from a single RCT with the other half incorporating external information. (
A significant proportion of analyses (n = 7, 39%) assumed no additional mortality following any adverse event. (
Few CEAs performed sub-group or scenario analyses to address patient heterogeneity related to older women (n = 6, 33%), women at low risk of breast cancer recurrence (n = 2, 11%), and women with multiple co-morbid diseases (n = 1, 6%). The majority of CEAs conducted the analyses from the perspective of the health care system as payer, accounting for the costs attributable to provider organizations such as government payers or other health insurers. The perspective excludes patient costs such as lost productivity and out of pocket health expenditures which may affect CEA results. The time horizon modeled ranged from 20 years to lifetime. Despite the fact that 17 of 18 studies modeled breast cancer recurrence risk beyond the follow-up time from trial data, a large proportion did not assess the impact of uncertainty arising from extrapolating beyond the trial data. (n = 6, 35%) Discount rates were reported in 17 of 18 studies and ranged from 1.5% to 6%. Five studies (25%) did not vary the discount rate in sensitivity analysis.
Detailed information on handling of structural and methodological uncertainty is available in
Only two of the studies mentioned limitations associated with external validity of CEA findings based on RCT data, but discussion of this limitation was not extensive in either case. (
Our review identified 18 published CEAs that compared AIs with tamoxifen for the first line treatment of early breast cancer in post-menopausal women. We found a lack of sensitivity and sub-group analyses that could limit the relevance of the study findings. The CEA studies assumed that observed benefits of AI in terms of DFS observed in individual RCTs would lead to improved OS. Subsequent meta-analysis of RCTs and longer term follow up provide no evidence of an OS benefit with five years of AI.
The ICERs from the 18 published analyses appear to be generally consistent with other cost-effectiveness analyses of breast cancer related interventions. A systematic review identified 89 cost-effectiveness analyses for breast cancer related interventions with a median ICUR of $27 K (2008 USD $/QALY).
Our study provides the most recent and comprehensive view of CEAs addressing AIs in the first-line setting. Compared to other systematic reviews we employ no language restrictions.
Our paper adds to a growing body of literature elucidating the mechanisms through which CEAs can produce potentially misleading results.
Our analysis has some limitations. Jang et al demonstrated that industry sponsored CEAs were significantly more likely than CEAs conducted by independent academic centres to reach favourable conclusions about the cost-effectiveness of AIs.
Published CEAs comparing AIs to tamoxifen inadequately investigate uncertainty to overcome the limitations of translating RCT findings to real-world practice, potentially leading to suboptimal guidance for clinical and health policy decision-making. The implications of these findings extend beyond hormonal therapies for early stage breast cancer to other cancer therapies and drug therapies in general. Care must be taken when interpreting CEAs based on RCTs which employ surrogate endpoints with populations that differ from the real-world population.
Study characteristics.
(DOC)
Neumann appraisal.
(PDF)
Data sources and handling of parameter uncertainty.
(DOC)
Handling of structural and methodological uncertainty.
(DOC)
(DOC)
(DOC)
Carolyn Ziegler assisted with developing literature search strategies and ran the literature searches.