In July 2000, the province of Ontario, Canada, initiated a universal influenza immunization program (UIIP) to provide free seasonal influenza vaccines for the entire population. This is the first large-scale program of its kind worldwide. The objective of this study was to conduct an economic appraisal of Ontario's UIIP compared to a targeted influenza immunization program (TIIP).
Methods and Findings
A cost-utility analysis using Ontario health administrative data was performed. The study was informed by a companion ecological study comparing physician visits, emergency department visits, hospitalizations, and deaths between 1997 and 2004 in Ontario and nine other Canadian provinces offering targeted immunization programs. The relative change estimates from pre-2000 to post-2000 as observed in other provinces were applied to pre-UIIP Ontario event rates to calculate the expected number of events had Ontario continued to offer targeted immunization. Main outcome measures were quality-adjusted life years (QALYs), costs in 2006 Canadian dollars, and incremental cost-utility ratios (incremental cost per QALY gained). Program and other costs were drawn from Ontario sources. Utility weights were obtained from the literature. The incremental cost of the program per QALY gained was calculated from the health care payer perspective. Ontario's UIIP costs approximately twice as much as a targeted program but reduces influenza cases by 61% and mortality by 28%, saving an estimated 1,134 QALYs per season overall. Reducing influenza cases decreases health care services cost by 52%. Most cost savings can be attributed to hospitalizations avoided. The incremental cost-effectiveness ratio is Can$10,797/QALY gained. Results are most sensitive to immunization cost and number of deaths averted.
Universal immunization against seasonal influenza was estimated to be an economically attractive intervention.
Citation: Sander B, Kwong JC, Bauch CT, Maetzel A, McGeer A, Raboud JM, et al. (2010) Economic Appraisal of Ontario's Universal Influenza Immunization Program: A Cost-Utility Analysis. PLoS Med 7(4): e1000256. https://doi.org/10.1371/journal.pmed.1000256
Academic Editor: Joshua A. Salomon, Harvard School of Public Health, United States of America
Received: August 28, 2009; Accepted: February 25, 2010; Published: April 6, 2010
This is an open-access article distributed under the terms of the Creative Commons Public Domain declaration which stipulates that, once placed in the public domain, this work may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose.
Funding: This study was supported by an operating grant from the University of Toronto Research Program, which provided fellowship support for BA, and by an operating grant from the Public Health Agency of Canada. JCK was supported by a Canadian Institutes of Health Research Fellowship Award at the time of the study and is currently supported by an Ontario Ministry of Health and Long Term Care (MOHLTC) Career Scientist Award and a University of Toronto Department of Family and Community Medicine Research Scholar Award. This work was made possible with the support of the Institute for Clinical Evaluative Sciences (ICES), which is funded in part by the Ontario MOHLTC. The opinions, results and conclusions reported in this paper are those of the authors and are independent from the funding sources. No endorsement by ICES or the Ontario MOHLTC is intended or should be inferred. The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; or preparation, review, or approval of the manuscript. The researchers are independent from the funders.
Competing interests: BS has served as consultant to F. Hoffmann La-Roche for economic evaluations of Tamiflu (unrestricted funding). BS has received travel grants and honoraria for speaking at meetings on pandemic influenza mitigation strategies from F. Hoffmann La-Roche.
Abbreviations: CMG, Case Mix Group; DAD, Discharge Abstract Database; ED, emergency department; GBS, Guillain-Barré syndrome; ICUR, incremental cost-utility ratio; MOHLTC, Ministry of Health and Long-Term Care; NICE, National Institute for Health and Clinical Excellence; OHIP, Ontario Health Insurance Plan; QALY, quality-adjusted life year; RIW, resource intensity weight; RR, relative rate; SIR, Susceptible-Infected-Removed; TIIP, targeted influenza immunization program; UIIP, universal influenza immunization program
Annual outbreaks (epidemics) of influenza—a viral disease of the nose, throat, and airways—make millions of people ill and kill about 500,000 individuals every year. In doing so, they impose a considerable economic burden on society in terms of health care costs and lost productivity. Influenza epidemics occur because small but frequent changes in the viral proteins to which the immune system responds mean that an immune response produced one year by exposure to an influenza virus provides only partial protection against influenza the next year. Annual immunization with a vaccine that contains killed influenza viruses of the major circulating strains can boost this natural immunity and greatly reduce a person's chances of catching influenza. Consequently, many countries run seasonal influenza vaccine programs. These programs usually target people at high risk of complications from influenza and individuals likely to come into close contact with them, and people who provide essential community services. So, for example, in most Canadian provinces, targeted influenza immunization programs (TIIPs) offer free influenza vaccinations to people aged 65 years or older, to people with chronic medical conditions, and to health care workers.
Why Was This Study Done?
Some experts argue, however, that universal vaccination might provide populations with better protection from influenza. In 2000, the province of Ontario in Canada decided, therefore, to introduce a universal influenza immunization program (UIIP) to provide free influenza vaccination to everyone older than 6 months, the first large program of this kind in the world. A study published in 2008 showed that, following the introduction of the UIIP, vaccination rates in Ontario increased more than in other Canadian provinces. In addition, deaths from influenza and influenza-related use of health care facilities decreased more in Ontario than in provinces that continued to offer a TIIP. But is universal influenza vaccination good value for money? In this study, the researchers evaluate the cost-effectiveness of the Ontario UIIP by comparing the health outcomes and costs associated with its introduction with the health outcomes and costs associated with a hypothetical continuation of targeted influenza immunization.
What Did the Researchers Do and Find?
The researchers used data on TIIP and UIIP vaccine uptake, physician visits, emergency department visits, hospitalizations for influenza, and deaths from influenza between 1997 and 2004 in Ontario and in nine Canadian states offering TIIPs, and Ontario cost data, in their “cost-utility” analysis. This type of analysis estimates the additional cost required to generate a year of perfect health (a quality-adjusted life-year or QALY) through the introduction of an intervention. QALYs are calculated by multiplying the time spent in a certain health state by a measure of the quality of that health state. The researchers report that the cost of Ontario's UIIP was about twice as much as the cost of a TIIP for the province. However, the introduction of the UIIP reduced the number of influenza cases by nearly two-thirds and reduced deaths from influenza by more than a quarter compared with what would have been expected had the province continued to offer a TIIP, an overall saving of 1,134 QALYs. Furthermore, the reduction in influenza cases halved influenza-related health care costs, mainly because of reductions in hospitalization. Overall, this means that the additional cost to Ontario of saving one QALY through the introduction of the UIIP was Can$10,797, an “incremental cost-effectiveness ratio” of $10,797 per QALY gained.
What Do These Findings Mean?
In Canada, an intervention is considered cost-effective from the point of view of a health care purchaser if it costs less than Canadian $50,000 to gain one QALY. These findings indicate, therefore, that for Ontario the introduction of the UIIP is economically attractive. Indeed, the researchers calculate that even if the costs of the UIIP were to double, the additional cost of saving one QALY by introducing universal immunization would remain below $50,000. Other “sensitivity” analyses undertaken by the researchers also indicate that universal immunization is likely to be effective and cost-effective in Ontario if other key assumptions and/or data included in the calculations are varied within reasonable limits. Given these findings, the researchers suggest that a UIIP might be an appealing intervention in other Canadian provinces and in other high-income countries where influenza transmission and health-care costs are broadly similar to those in Ontario.
Please access these Web sites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1000256.
- A PLoS Medicine Research Article by Kwong and colleagues describes how the introduction of universal influenza immunization in Ontario altered influenza-related health care use and deaths in the province
- Wikipedia pages are available on QALYs and on cost-utility analysis (note that Wikipedia is a free online encyclopedia that anyone can edit; available in several languages)
- Bandolier, an independent online journal about evidence-based health-care, provides information about QALYs and their use in cost-utility analysis
- The UK National Institute for Health and Clinical Excellence has a webpage on Measuring effectiveness and cost-effectiveness: the QALY
Worldwide, annual epidemics of influenza cause significant morbidity and mortality and impose a considerable economic burden on society in terms of health care costs and lost productivity .
Seasonal prophylaxis with vaccines is the cornerstone of influenza management. Influenza vaccines are generally safe and effective. They reduce serologically confirmed influenza cases by 73% in healthy adults  and 58% in the elderly . Influenza vaccination in young children decreases the incidence of influenza infection , acute otitis media ,, and daycare absenteeism .
In most jurisdictions, seasonal prophylaxis with vaccines is currently recommended for people at high risk of complications, those capable of transmitting influenza to individuals at high risk of complications, and those who provide essential community services . Nevertheless, vaccine coverage rates among high-risk groups in Canada have been low in the past, substantially lower than the target coverage rate (70%) set by a national consensus conference on influenza in 1993 . In response to these findings, most provinces and territories in Canada offer free influenza vaccinations for those 65 y or older, those with chronic medical conditions, and health care workers (Targeted Influenza Immunization Program, TIIP) . The age-based recommendations in the United States are more broad, including individuals 50 y or older and children aged 6 mo to 18 y .
In 2000, the province of Ontario, Canada, initiated a universal influenza immunization program (UIIP)  to provide free influenza vaccines for the entire population (6 mo of age or older). Introduction of this program was associated with greater overall increases in influenza vaccination rates, particularly in children and working-age adults, and greater reductions in influenza-associated mortality and health care use in Ontario compared to other provinces that maintained targeted programs ,.
The cost-effectiveness of universal vaccination has not been evaluated, despite the investment of substantial financial resources. The objective of this study was to conduct an economic appraisal of Ontario's UIIP.
An economic evaluation was performed to estimate health outcomes and costs related to universal vaccination compared to the hypothetical continuation of a targeted program for the entire population of Ontario (12.16 million).
All the input data are age specific (Tables S1 and S2) and used as such in the calculations. The seven age groups are 4 y and under, 5–19 y, 20–49 y, 50–64 y, 65–74 y, 75–84 y, and 85 y and older. Most results are aggregated and reported for the total program; supplementary information is available by age group (Table S3).
Effectiveness of UIIP
This economic evaluation was informed by an earlier epidemiological study that compared outcomes in Ontario before and after universal immunization, using other provinces as controls . Introduction of Ontario's UIIP was associated with greater overall increases in influenza vaccination rates in Ontario (from 18% in 1996 to 42% in 2005) compared to other provinces (from 13% to 28%). Therefore, there was a 9-percentage-point incremental increase in Ontario, and the relative increases were most evident among those under 65 y. A different pattern was observed among the elderly; for those aged 75 y or older, there were greater relative increases among other provinces compared to Ontario, although Ontario still maintained higher rates at all times.
The impact of Ontario's UIIP on influenza-associated mortality, hospitalizations, and visits to emergency departments (EDs) and doctors' offices was estimated using data from 1997 to 2004 (3 y before and 4 y after UIIP implementation). To estimate influenza-associated outcomes, we used multivariate regression models to predict events with influenza viral activity in the model, and we subsequently removed the influenza terms (i.e., viral surveillance data) from the model to generate a baseline function that represented the hypothetical absence of influenza. To calculate influenza-associated events, we subtracted the expected baseline events from observed events during periods of influenza activity. The models controlled for age, sex, province, influenza surveillance data, and temporal trends. Influenza-associated event rates in the overall population decreased 40%–60% more in Ontario than in other provinces for all study outcomes. The relative reductions were particularly prominent among age groups younger than 65, consistent with the changes in vaccine uptake. However, despite greater increases in vaccination rates among the elderly in other provinces compared to Ontario, the reductions in influenza-associated events among the elderly were either less pronounced or the same in Ontario compared to other provinces. The robustness of these findings was confirmed by numerous sensitivity analyses . For a more detailed description of the regression model and results, see Text S1 and Table S6.
Side effects due to influenza vaccine were not included as they are usually mild. While there is a small risk of hospitalization because of Guillain-Barré syndrome (GBS), a population-based study found no statistically significant increase in hospital admissions because of GBS after introduction of UIIP .
Resource Use: Physician Services and Hospitalizations
For the economic evaluation, the mean number of events per season post-UIIP implementation as observed in Ontario was compared with the expected number of events under hypothetical continuation of TIIP (Table 1; more detailed information can be found in Tables S1 and S2). To calculate the expected number of events under hypothetical continuation of TIIP for Ontario, the relative change of mean number of events pre-2000 (1997/1998 to 1999/2000) to post-2000 (2000/2001 to 2003/2004) observed in other Canadian provinces was applied to the pre-UIIP event rates in Ontario. This assumes that in Ontario vaccination coverage rates would have increased and influenza-related events decreased by the same relative magnitude as in other provinces after 2000.
All data were obtained for seven influenza seasons covering the years from 1997 to 2004 . Physician services data were extracted from provincial datasets for Ontario (Ontario Health Insurance Plan, OHIP) and three other Canadian provinces (Quebec's Physician Claims Dataset, Alberta's Health Care Insurance Plan Dataset, and Manitoba's Medical Claims Dataset). Hospitalization data were obtained from Statistics Canada's Hospital Morbidity Database, a national hospital discharge dataset. Influenza-related events were identified using ICD 9/10 (International Statistical Classification of Diseases and Related Health Problems) service codes associated with pneumonia or influenza. These are complete datasets, covering all residents of the respective provinces.
Influenza-associated events decreased more in Ontario than other provinces: 75% versus 56% for hospitalizations, 69% versus 31% for ED use, and 79% versus 48% for doctors' office visits .
Quality-Adjusted Life Years (QALYs) Lost from Influenza
The administrative data used to estimate resource use do not yield a count of influenza cases. The number of cases of influenza requiring health care was estimated using the number of office visits and ED visits and excluding repeat visits by the same patient for the same diagnosis within a 21-d window following an initial visit. According to this approximation, 90% of all influenza-associated office visits, 85% of all influenza-associated ED visits, and all influenza-associated hospitalizations were deemed to be discrete new cases of influenza. The remainder of visits were considered to be repeat visits. Multiple visits across settings by the same patient were not removed since the number of those was expected to be small (for example, only 0.07% of all health care contacts were hospitalizations, some of which may have had an office and/or ED visit before being hospitalized). It was assumed that patients who die due to influenza had at least one health care contact before death. Therefore, these were also not separately counted as cases as they were presumed to have been accounted for.
Quality weights as reported by Turner  were used to estimate QALYs lost per case of influenza due to morbidity. In this recent assessment by the National Institute for Health and Clinical Excellence (NICE) of antiviral treatment of influenza, utilities for influenza were estimated from patient health state valuations reported daily for 21 d in oseltamivir clinical trials. Utilities were generated by recalibrating the Likert scores as obtained in the clinical trials to mean visual analogue scale scores, which were then transformed to time trade-off scores. Quality of life data were available for “otherwise healthy adults,” “adults with co-morbidities,” and the “elderly.” In the absence of utility data for children, it was assumed that children have similar utility weights as “otherwise healthy adults” as both groups are similar in other health outcome measures, such as time to return to normal activity . The utility weight for adults with co-morbidities as obtained in the clinical trials was applied to the 18.8% of the Ontario population aged 12 to 64 y deemed high risk because of chronic conditions . Patients in the trials were recruited from Europe and North America. We assume that the clinical trial population is sufficiently similar to the Ontario population (high income Western countries with similar life expectancy and population health status) so that we can apply the utility estimates to the population under consideration here. The QALY gain per influenza case prevented (Table 2) was calculated by multiplying the duration of the symptomatic period by the utility decrement associated with influenza-related illness.
To calculate QALYs lost due to premature death (Table 2), we estimated influenza-associated deaths using mortality data from Statistics Canada's Mortality Database . We considered deaths due to all respiratory and circulatory conditions for seven influenza seasons covering the years from 1997 to 2004. After UIIP introduction, influenza-associated mortality decreased 77% in Ontario compared to 46% in other provinces .
Quality-adjusted life expectancy was discounted at 3% per year in the base case analysis .
Unit Costs: UIIP Program, Physician Services, and Hospitalizations
All costs were obtained from Ontario sources and are expressed in 2006 Canadian dollars. Cost data are summarized in Table 2.
The total program cost for the universal program was $40 million per year (approximately $3.96 per dose and $7.55 in total per dose distributed) for each year since introducing the program (Nancy Peroff-Johnston, Ministry of Health and Long-Term Care [MOHLTC], personal communication, May 11, 2007). This included the cost of the vaccine, health care provider reimbursement for vaccination, communications strategies, and direct operating expenditures for the MOHLTC including staffing. Approximately 50% of the total budget was spent on the vaccine and the remaining 50% on all other cost items. To calculate the expected targeted program (TIIP) cost if universal immunization (UIIP) had not been implemented, the relative change in vaccine coverage (pre-2000 versus post-2000) observed in other provinces was applied to Ontario's pre-2000 vaccine coverage rate. Vaccine coverage rates for the population 12 y or older more than doubled (18% to 42%) in Ontario and other provinces (13% to 28%) between 1996/1997 and 2005. The Ontario pre-2000 TIIP program cost (Nancy Peroff-Johnston, MOHLTC, personal communication, May 11, 2007) per person vaccinated was used, inflated to 2006, and applied to the population who would have been covered under a targeted program . This assumes that the average cost per person (including vaccine, vaccine delivery, and other cost to the Ministry) remains the same but is applied to a larger number of persons receiving influenza immunization. For an expected TIIP coverage rate of 40%, the total program cost for targeted immunization was expected to be $20 million if TIIP had been continued.
Unit costs for physician services (office visits, ED visits, and in-hospital services) were obtained from the OHIP dataset. A location code in the dataset indicates whether the services were provided in the physician office, ED, or hospital.
Costs reflect the fee paid by condition type (pneumonia and influenza). Mean fees paid ($35 per office visit, $54 per ED visit) were used in the base case analysis; 95% CI defined the lower and upper limit for deterministic sensitivity analysis. For probabilistic analysis the primary data were sampled. An additional $166 per ED visit was added for non-physician costs to calculate total cost per ED visit .
Hospitalization costs were costed using the resource intensity weight (RIW) approach ,. The RIW is the ratio of the cost of a case in a Case Mix Group (CMG) to the average cost of all cases. The mean RIWs for pneumonia- or influenza-related hospitalizations were extracted from the acute care Discharge Abstract Database (DAD). The provincial cost per weighted case of $4,732 was applied to the mean RIW of influenza-associated hospitalizations of 1.33 . Mean physician charges for hospital inpatient services for pneumonia or influenza (obtained from the OHIP database using location code) were added. On average, there were 1.65 claims for physician services per hospitalization at a cost of $66.41 each. The average length of stay for pneumonia and influenza is 6 d . The total mean cost of a pneumonia- and influenza-related episode of hospitalization was therefore $6,418.
Cost-effectiveness of UIIP versus TIIP was measured by the incremental cost-utility ratio (ICUR), defined as the additional cost per QALY gained. The health benefits of universal immunization were estimated as number of influenza cases, health services utilization (physician visits, ED visits, hospitalizations), and deaths prevented in a typical influenza season. A time horizon of a lifetime was adopted to calculate QALYs lost attributable to influenza-related death. Time preference was incorporated by discounting QALYs lost at 3% annually in the base case analysis . Costs are not discounted because all costs occur within 1 y. The analysis was performed from the perspective of the health care payer, the Ontario MOHLTC.
The impact of data uncertainty was explored with one-way and probabilistic sensitivity analyses. One-way sensitivity analyses were performed for numbers of events, number of cases, event costs, immunization program cost, QALY penalty associated with influenza, and discount rate (undiscounted and 5%). Grouped sensitivity analysis was performed by simultaneously adjusting the values of (1) events (all events and deaths separate), (2) relative rate (RR) of events (office visits, ED visits, hospitalizations, and deaths, each event separate), (3) costs of events (office visits, ED visits, and hospitalizations, each event separate), and (4) QALY gain due to morbidity (illness) averted. This was achieved by the use of scaling parameters that link variables within a group and allow the simultaneous adjustment of their values over a defined range. The base case value was therefore multiplied with scaling parameter values ranging from 0 to 2. This range covers the 95% CI of all parameters except for deaths observed post-UIIP in persons under 65 y of age. The range for post-UIIP deaths is −4 to 6 for <65 y old. This age group (<65 y) accounts for approximately 2% of all deaths (7 out of 309 deaths per season). We therefore assumed the range from 0 to 2 for scaling parameters to be acceptable for varying post-UIIP deaths for all age groups combined (not individually). Two extreme (best and worst) case scenarios were also performed (Tables 1 and 2; more detailed information can be found in Tables S1 and S2) biasing the analysis for and against UIIP. The 95% CIs were used as extreme values for pre- and post-UIIP number of events (cases, physician services, ED visits, hospitalizations, deaths), RRs of events (to calculate number of events if TIIP would have been continued), unit costs, and QALY loss attributable to influenza morbidity.
For probabilistic analysis, distributions were assigned to key variables in the model (Tables S1 and S2). Distributions for events (office visits, ED visits, hospitalizations, and deaths), RRs, and unit cost for events were drawn from the original data. For unit costs for physician services, the actual administrative data were sampled using $10.00 increments, with the last bin being $220 per visit and greater (fees paid: $0 to <$10, $10 to <$20, $20 to <$30, … , $210 to <$220, $220 and greater). The midpoint for the top and bottom categories was adjusted by the minimum and maximum fee charged. The midpoint for the bottom category was $5.50 for office visits, $9.00 for ED visits, and $6.50 for inpatient services. The midpoint for the top category was $225 for office visits and inpatient services and $222 for ED visits. The midpoint of each bin was used for calculation. For hospitalization cost, the RIW 95% CI defined the lower and upper limit for deterministic sensitivity analysis. For probabilistic analysis, the primary data were sampled. Distributions for utility weights were based on confidence intervals in published reports . The probability of Ontario's UIIP being cost-effective for a range of willingness to pay thresholds ($0 to $100,000 per QALY gained) was calculated by sampling from these distributions running 1,000 trials. To ensure internal consistency, the same random number was used to sample from distributions for the different age brackets within the same group of variables: pre-UIIP events, post-UIIP events, RR, and QALY gain due to morbidity averted.
An additional analysis, using the diagnostic codes for all respiratory conditions to extract health care utilization data, is presented in the Text S2. This analysis was performed because many influenza-related health care interactions are coded under different diagnostic codes (e.g., common cold, asthma, COPD, otitis media, etc.), therefore underestimating the health care resource use due to influenza if using the diagnostic codes for pneumonia and influenza only (as in all other analyses).
In Ontario, 22,457 cases of influenza were observed on average per season after the introduction of the universal immunization program. If TIIP had been continued, the expected average number of cases per season was estimated to be 56,998. UIIP therefore prevented 34,541 influenza cases (61% of all cases each season).
Ontario's UIIP also prevented 111 deaths, a 28% reduction in mortality. This resulted in a projected 1,134 QALYs gained in total or 0.09 quality-adjusted life days per person vaccinated. Approximately half of all health gains (QALYs) were associated with a reduction of influenza mortality; the other half was associated with reduction in influenza-related morbidity.
The program costs of UIIP are high, approximately double that of a targeted program ($40 million versus $20 million). However, UIIP was estimated to prevent 786 influenza-related hospitalizations, 7,745 influenza-related ED visits, and 30,306 office visits per season. Preventing influenza cases effectively reduced influenza-related health care costs by 52%, saving the health care system approximately $7.8 million per season, so that the net cost of the UIIP program is $12.2 million, or $2.60 per person vaccinated.
Deterministic one-way sensitivity analysis (Figure 1, Table S4) revealed that results are highly sensitive to the RR of deaths if TIIP had been continued, the pre-UIIP and post-UIIP number of deaths, and moderately sensitive to UIIP and TIIP immunization program cost. However, the ICER remains below $50,000 per QALY under the following conservative individual conditions: (1) UIIP costs are twice the current program cost, (2) TIIP costs are zero, (3) the RR of death is not smaller than 45% of the base case value, (4) the pre-UIIP number of deaths is not smaller than 45% of the base case value, or (5) the post-UIIP number of deaths is not greater than 190% of the base case value.
The grey vertical line corresponds to all the uncertain parameters being at their respective base values. The width of the bars indicates the uncertainty associated with each parameter (ranging from lower to upper limit). The red segments of the bars correspond to result values increasing the base case ICER, and the blue segments of the bars correspond to result values decreasing the base case ICER. ED, emergency department; QALY, quality-adjusted life year; RR, relative rate; TIIP, targeted influenza immunization program; UIIP, universal influenza immunization program.
It is highly unlikely that these thresholds for key parameters are exceeded. Program costs were obtained from the Ontario Ministry of Health and thus reflect the true costs to the Ministry. A RR of death or a pre-UIIP number of deaths smaller than 45% of the base case value is far outside the 95% CI for these values. The 95% CIs range from 92% to 108% of the base case value for pre-UIIP deaths (all ages) and from 85% to 118% of base case value for RR of death (all ages). Similarly a post-UIIP number of deaths greater than 190% of the base case value is outside the 95% CI range for this variable (−260% to 120% of the base case value).
An analysis of extremes was also performed by examining best- and worst-case scenarios (Table S5). In the worst-case scenario (UIIP does not provide much health benefit over TIIP, health service resource unit costs are low), UIIP increases program costs by $20 million, saves only $1.8 million in health care service cost, and incurs a QALY loss. UIIP becomes unattractive because it is not associated with health gain, and also increased costs compared to TIIP.
In the best-case scenario (UIIP is highly effective, health service resource unit costs are high), UIIP increases program costs by $19.1 million, saves $40.4 million in health care services, and gains 5,619 QALYs (discount rate 3%). UIIP dominates TIIP in this scenario; i.e., UIIP is less costly and more effective than TIIP.
Probabilistic sensitivity analysis (Table 4, Figure 2) shows that the probability of UIIP being cost-effective at a willingness-to-pay threshold of $50,000 per QALY is exceeding 90%. The mean (95% CI) QALYs gained are 1,263 [−456; 3,082], and the mean (95% CI) net cost is $12.10 million [−$0.01 million; $16.11 million]. Finally, results for the additional analysis, using the diagnostic codes for all respiratory conditions to extract health care utilization data, are presented in Table S7.
Our analysis suggests that Ontario's UIIP is economically attractive. UIIP reduces the number of influenza cases and deaths and reduces health services resource use. The additional costs of UIIP are partly (39%) offset by savings in health care costs. Compared to TIIP, UIIP is cost-effective at an ICUR of $10,797 per QALY gained.
The main limitation in determining the impact of UIIP is the ecologic study design of the underlying effectiveness study. Causality cannot be established with this type of study design. The effect size may be exaggerated if there are province-specific epidemiological or health service factors that have in more recent years reduced influenza events in Ontario or increased it in other provinces. Similarly, since the outcome measure is influenza-like illness rather than confirmed influenza, it is possible that this could make estimates of outcomes sensitive to changes in the epidemiology of non-influenza respiratory illness. However, this design is appropriate for assessing the public health impact of a population-wide intervention . The results of Kwong's analysis of observational ecologic data in four provinces over 7 y are congruent with the results of randomized controlled trial data in targeted groups. Targeted immunization has been shown to be effective in preventing laboratory-confirmed influenza infection in healthy children 2 y and older , in healthy adults, especially when there is a good match and virus circulation is high , and older adults (65 y and older), especially in long-term care facilities . The weight of evidence seems to support the hypothesis that UIIP has been responsible for the decrease in cases and deaths in Ontario, in the absence of other obvious causal mechanisms. Stronger randomised clinical trial evidence of the effectiveness of universal vaccination in large populations is unlikely to be available in the future. Finally, to test whether the disproportionate decrease in influenza events from a 9% difference in coverage improvement may be attributable to herd immunity, we analyzed a simple Susceptible-Infected-Removed (SIR) compartmental model (for a description of the model and results, see Figures S1 and S2, Tables S7 and S8, and Text S3). This basic analysis suggests that the 40%–60% decrease in influenza events is plausible.
Other study limitations relate to the definition of cases and quality of life estimates used. Cases were defined based on health care resource use for pneumonia and influenza and therefore represent influenza-like illness cases. Symptomatic cases not requiring health care contacts are not included, potentially underestimating the benefit of the program. Utility weights to estimate QALYs were obtained from an analysis of clinical trial data from influenza patients in Europe and North America . While these data are not Canada-specific, the populations are similar. Furthermore, sensitivity analysis was performed and reported for utility weights used in the analysis.
This study has several strengths. The model is robust and is based on administrative health care resource use and cost data covering all residents of Ontario. Data were extracted and analyzed in detail to estimate influenza-associated events and costs as accurately as possible to assess the impact of this intervention on the population of interest rather than estimating parameter values from other, smaller subpopulations that may not be representative. Finally, extensive sensitivity analyses demonstrated the program to be effective and cost-effective under very conservative assumptions.
The cost-effectiveness of influenza immunization programs has been demonstrated by numerous economic evaluations of TIIPs, many of which are directly based on clinical trial data. TIIPs have been shown to be cost-effective in children 6 mo and older –, adults 50 y and older –, working adults –, working adult cancer patients , pregnant women , health care workers ,, high-risk individuals , and older adults (65 y and older) ,– from a health care payer perspective. Most economic evaluations found TIIPs to be not only cost-effective but cost-saving from a societal perspective.
Policy makers in many jurisdictions considering the implementation of universal immunization have expressed interest in an economic evaluation of Ontario's program. The program appears to offer some health benefits, but the relationship between cost and health benefits of universal immunization had not been evaluated. Our study provides evidence that a universal program is economically attractive in jurisdictions with influenza epidemiology and health care costs that are broadly similar to that of Ontario.
A UIIP may be an appealing intervention in high-income jurisdictions with comparable demographic characteristics (age distribution, risk profile, density) where influenza transmission can be expected to be reasonably similar to the population analyzed. A health care system similar to Ontario's (i.e., health care systems with one major payer), where the costs of the immunization program and the costs of treating influenza cases are both in the payer's budget, will enable the universal program costs to be partly offset by savings in health care cost.
Mean filtered relative reduction in Ontario for a range of vaccine coverage under UIIP.
(0.20 MB TIF)
Proportion of simulations with a relative reduction below 0.01%, 0.1%, 1%, and 5% in Ontario for a range of vaccine coverage under UIIP.
(0.26 MB TIF)
Mean annual influenza-related event rates by age group.
(0.14 MB DOC)
Relative rates by age groups.
(0.10 MB DOC)
Results (base case, disaggregated).
(0.12 MB DOC)
Deterministic sensitivity analysis.
(0.29 MB DOC)
Aggregated worst-case and best-case results.
(0.08 MB DOC)
Results of the regression analysis.
(0.35 MB DOC)
Results - sensitivity analysis using all respiratory conditions as outcome.
(0.05 MB DOC)
Results of empirical versus predicted relative reduction.
(0.03 MB DOC)
Effectiveness of Ontario's UIIP.
(0.06 MB DOC)
Additional analysis using all respiratory conditions as outcome.
(0.05 MB DOC)
We thank Deva Thiruchelvam and Brandon Zagorski for assistance with data preparation and analyses of Ontario cost data and Nancy-Peroff Johnston for providing data on Ontario's influenza immunization program. We also thank the Toronto Health Economics Network and fellow colleagues and students for comment on earlier drafts.
Previous presentation of the information reported in this manuscript:
Sander B, Kwong J, Bauch C, Maetzel A, McGeer A, Raboud J, Krahn, M. The Cost-Effectiveness of Ontario's Universal Influenza Immunization Program. Society for Medical Decision Making, 30th Annual Meeting. 2008.
Sander B, Kwong J, Bauch C, Maetzel A, McGeer A, Raboud J, Krahn, M. The Cost-Effectiveness of Ontario's Universal Influenza Immunization Program. The European Scientific Working Group on Influenza, 3rd European Influenza Conference. 2008.
ICMJE criteria for authorship read and met: BS JCK CTB AM AM JMR MDK. Agree with the manuscript's results and conclusions: BS JCK CTB AM AM JMR MDK. Designed the experiments/the study: BS CTB AM AM JMR MDK. Analyzed the data: BS. Collected data/did experiments for the study: BS JCK. Wrote the first draft of the paper: BS. Contributed to the writing of the paper: JCK CTB AM AM JMR MDK.
- 1. (2003) Influenza. 2003Influenza.Fact Sheet: World Health Organization. Fact Sheet: World Health Organization.
- 2. Jefferson TO, Rivetti D, Di Pietrantonj C, Rivetti A, Demicheli V (2007) Vaccines for preventing influenza in healthy adults. Cochrane Database Syst Rev. CD001269 p.TO JeffersonD. RivettiC. Di PietrantonjA. RivettiV. Demicheli2007Vaccines for preventing influenza in healthy adults.Cochrane Database Syst Rev.CD001269 p
- 3. Govaert TM, Thijs CT, Masurel N, Sprenger MJ, Dinant GJ, et al. (1994) The efficacy of influenza vaccination in elderly individuals. A randomized double-blind placebo-controlled trial. JAMA 272: 1661–1665.TM GovaertCT ThijsN. MasurelMJ SprengerGJ Dinant1994The efficacy of influenza vaccination in elderly individuals. A randomized double-blind placebo-controlled trial.JAMA27216611665
- 4. Neuzil KM, Dupont WD, Wright PF, Edwards KM (2001) Efficacy of inactivated and cold-adapted vaccines against influenza A infection, 1985 to 1990: the pediatric experience. Pediatr Infect Dis J 20: 733–740.KM NeuzilWD DupontPF WrightKM Edwards2001Efficacy of inactivated and cold-adapted vaccines against influenza A infection, 1985 to 1990: the pediatric experience.Pediatr Infect Dis J20733740
- 5. Heikkinen T, Ruuskanen O, Waris M, Ziegler T, Arola M, et al. (1991) Influenza vaccination in the prevention of acute otitis media in children. Am J Dis Child 145: 445–448.T. HeikkinenO. RuuskanenM. WarisT. ZieglerM. Arola1991Influenza vaccination in the prevention of acute otitis media in children.Am J Dis Child145445448
- 6. Clements DA, Langdon L, Bland C, Walter E (1995) Influenza A vaccine decreases the incidence of otitis media in 6- to 30-month-old children in day care. Arch Pediatr Adolesc Med 149: 1113–1117.DA ClementsL. LangdonC. BlandE. Walter1995Influenza A vaccine decreases the incidence of otitis media in 6- to 30-month-old children in day care.Arch Pediatr Adolesc Med14911131117
- 7. Colombo C, Argiolas L, La Vecchia C, Negri E, Meloni G, et al. (2001) Influenza vaccine in healthy preschool children. Rev Epidemiol Sante Publique 49: 157–162.C. ColomboL. ArgiolasC. La VecchiaE. NegriG. Meloni2001Influenza vaccine in healthy preschool children.Rev Epidemiol Sante Publique49157162
- 8. (2008) Statement on influenza vaccination for the 2008–2009 season. An Advisory Committee Statement (ACS). Can Commun Dis Rep 34: 1–46.2008Statement on influenza vaccination for the 2008–2009 season. An Advisory Committee Statement (ACS).Can Commun Dis Rep34146
- 9. (1993) Canadian Consensus Conference on Influenza. Can Commun Dis Rep 19: 136–142, 145–137.1993Canadian Consensus Conference on Influenza.Can Commun Dis Rep19136142, 145–137
- 10. Squires SG, Pelletier L (2000) Publicly-funded influenza and pneumococcal immunization programs in Canada: a progress report. Can Commun Dis Rep 26: 141–148.SG SquiresL. Pelletier2000Publicly-funded influenza and pneumococcal immunization programs in Canada: a progress report.Can Commun Dis Rep26141148
- 11. Fiore AE, Shay DK, Broder K, Iskander JK, Uyeki TM, et al. (2008) Prevention and control of influenza: recommendations of the Advisory Committee on Immunization Practices (ACIP), 2008. MMWR Recomm Rep 57: 1–60.AE FioreDK ShayK. BroderJK IskanderTM Uyeki2008Prevention and control of influenza: recommendations of the Advisory Committee on Immunization Practices (ACIP), 2008.MMWR Recomm Rep57160
- 12. (2000) Ontario invests $38 million to ease emergency room pressures with universal vaccination program. Ministry of Health and Long-Term Care. 2000Ontario invests $38 million to ease emergency room pressures with universal vaccination program. Ministry of Health and Long-Term Care.
- 13. Kwong JC, Stukel TA, Lim J, McGeer AJ, Upshur RE, et al. (2008) The effect of universal influenza immunization on mortality and health care use. PLoS Med 5: e211.JC KwongTA StukelJ. LimAJ McGeerRE Upshur2008The effect of universal influenza immunization on mortality and health care use.PLoS Med5e211
- 14. Moran K, Maaten S, Guttmann A, Northrup D, Kwong JC (2009) Influenza vaccination rates in Ontario children: implications for universal childhood vaccination policy. Vaccine. K. MoranS. MaatenA. GuttmannD. NorthrupJC Kwong2009Influenza vaccination rates in Ontario children: implications for universal childhood vaccination policy.Vaccine
- 15. Juurlink DN, Stukel TA, Kwong J, Kopp A, McGeer A, et al. (2006) Guillain-Barre syndrome after influenza vaccination in adults: a population-based study. Arch Intern Med 166: 2217–2221.DN JuurlinkTA StukelJ. KwongA. KoppA. McGeer2006Guillain-Barre syndrome after influenza vaccination in adults: a population-based study.Arch Intern Med16622172221
- 16. Turner D, Wailoo A, Nicholson K, Cooper N, Sutton A, et al. (2003) Systematic review and economic decision modelling for the prevention and treatment of influenza A and B. Health Technol Assess 7: iii–iv, xi–xiii, 1–170.D. TurnerA. WailooK. NicholsonN. CooperA. Sutton2003Systematic review and economic decision modelling for the prevention and treatment of influenza A and B.Health Technol Assess7iiiiv, xi–xiii, 1–170
- 17. Reisinger K, Greene G, Aultman R, Sander B, Gyldmark M (2004) Effect of influenza treatment with oseltamivir on health outcome and costs in otherwise healthy children. Clin Drug Investig 24: 395–407.K. ReisingerG. GreeneR. AultmanB. SanderM. Gyldmark2004Effect of influenza treatment with oseltamivir on health outcome and costs in otherwise healthy children.Clin Drug Investig24395407
- 18. Kwong JC, Stukel TA, McGeer AJ, Manuel DG (2007) Appropriate measures of influenza immunization program effectiveness. Vaccine 25: 967–969.JC KwongTA StukelAJ McGeerDG Manuel2007Appropriate measures of influenza immunization program effectiveness.Vaccine25967969
- 19. (2008) Mortality rate per 1,000 population. Canadian socio-economic information management system (CANSIM) Statistics Canada et al. 2008Mortality rate per 1,000 population. Canadian socio-economic information management system (CANSIM) Statistics Canada et al.
- 20. Mittmann N, Trakas K, Risebrough N, Liu BA (1999) Utility scores for chronic conditions in a community-dwelling population. PharmacoEconomics 15: 369–376.N. MittmannK. TrakasN. RisebroughBA Liu1999Utility scores for chronic conditions in a community-dwelling population.PharmacoEconomics15369376
- 21. (2006) HTA guidelines for the economic evaluation of health technologies. Ottawa..2006HTA guidelines for the economic evaluation of health technologies.Ottawa.
- 22. (2008) Consumer price index (CPI), Ontario, Health Care, 1979–2007. Canadian socio-economic information management system (CANSIM) Statistics Canada et al. 2008Consumer price index (CPI), Ontario, Health Care, 1979–2007. Canadian socio-economic information management system (CANSIM) Statistics Canada et al.
- 23. (2008) JPPC Rate Model results based on 2006/07 data. 2008JPPC Rate Model results based on 2006/07 data.Ontario Joint Policy and Planning Committee. Ontario Joint Policy and Planning Committee.
- 24. Pink GH, Bolley HB (1994) Physicians in health care management: 4. Case mix groups and resource intensity weights: physicians and hospital funding. CMAJ 150: 1255–1261.GH PinkHB Bolley1994Physicians in health care management: 4. Case mix groups and resource intensity weights: physicians and hospital funding.CMAJ15012551261
- 25. Pink GH, Bolley HB (1994) Physicians in health care management: 3. Case mix groups and resource intensity weights: an overview for physicians. CMAJ 150: 889–894.GH PinkHB Bolley1994Physicians in health care management: 3. Case mix groups and resource intensity weights: an overview for physicians.CMAJ150889894
- 26. OCCI (Ontario Case Costing Initiative) (2009) Costing analysis tool. OCCI (Ontario Case Costing Initiative)2009Costing analysis tool.
- 27. Jefferson T, Rivetti A, Harnden A, Di Pietrantonj C, Demicheli V (2008) Vaccines for preventing influenza in healthy children. Cochrane Database Syst Rev. CD004879.T. JeffersonA. RivettiA. HarndenC. Di PietrantonjV. Demicheli2008Vaccines for preventing influenza in healthy children.Cochrane Database Syst Rev.CD004879
- 28. Rivetti D, Jefferson T, Thomas R, Rudin M, Rivetti A, et al. (2006) Vaccines for preventing influenza in the elderly. Cochrane Database Syst Rev 3: CD004876.D. RivettiT. JeffersonR. ThomasM. RudinA. Rivetti2006Vaccines for preventing influenza in the elderly.Cochrane Database Syst Rev3CD004876
- 29. Esposito S, Marchisio P, Bosis S, Lambertini L, Claut L, et al. (2006) Clinical and economic impact of influenza vaccination on healthy children aged 2–5 years. Vaccine 24: 629–635.S. EspositoP. MarchisioS. BosisL. LambertiniL. Claut2006Clinical and economic impact of influenza vaccination on healthy children aged 2–5 years.Vaccine24629635
- 30. Hibbert CL, Piedra PA, McLaurin KK, Vesikari T, Mauskopf J, et al. (2007) Cost-effectiveness of live-attenuated influenza vaccine, trivalent in preventing influenza in young children attending day-care centres. Vaccine 25: 8010–8020.CL HibbertPA PiedraKK McLaurinT. VesikariJ. Mauskopf2007Cost-effectiveness of live-attenuated influenza vaccine, trivalent in preventing influenza in young children attending day-care centres.Vaccine2580108020
- 31. Li S, Leader S (2007) Economic burden and absenteeism from influenza-like illness in healthy households with children (5–17 years) in the US. Respiratory Medicine 101: 1244–1250.S. LiS. Leader2007Economic burden and absenteeism from influenza-like illness in healthy households with children (5–17 years) in the US.Respiratory Medicine10112441250
- 32. Marchetti M, Kuhnel UM, Colombo GL, Esposito S, Principi N (2007) Cost-effectiveness of adjuvanted influenza vaccination of healthy children 6 to 60 months of age. Human Vaccines 3: 14–22.M. MarchettiUM KuhnelGL ColomboS. EspositoN. Principi2007Cost-effectiveness of adjuvanted influenza vaccination of healthy children 6 to 60 months of age.Human Vaccines31422
- 33. Navas E, Salleras L, Dominguez A, Ibanez D, Prat A, et al. (2007) Cost-effectiveness analysis of inactivated virosomal subunit influenza vaccination in children aged 3–14 years from the provider and societal perspectives. Vaccine 25: 3233–3239.E. NavasL. SallerasA. DominguezD. IbanezA. Prat2007Cost-effectiveness analysis of inactivated virosomal subunit influenza vaccination in children aged 3–14 years from the provider and societal perspectives.Vaccine2532333239
- 34. Schmier J, Li S, King JC Jr, Nichol K, Mahadevia PJ, et al. (2008) Benefits and costs of immunizing children against influenza at school: an economic analysis based on a large-cluster controlled clinical trial. Health Affairs 27: w96–w104.J. SchmierS. LiJC King JrK. NicholPJ Mahadevia2008Benefits and costs of immunizing children against influenza at school: an economic analysis based on a large-cluster controlled clinical trial.Health Affairs27w96w104
- 35. Teufel IRJ, Basco WT Jr, Simpson KN (2008) Cost effectiveness of an inpatient influenza immunization assessment and delivery program for children with asthma. J Hosp Med 3: 134–141.IRJ TeufelWT Basco JrKN Simpson2008Cost effectiveness of an inpatient influenza immunization assessment and delivery program for children with asthma.J Hosp Med3134141
- 36. Salo H, Kilpi T, Sintonen H, Linna M, Peltola V, et al. (2006) Cost-effectiveness of influenza vaccination of healthy children. Vaccine 24: 4934–4941.H. SaloT. KilpiH. SintonenM. LinnaV. Peltola2006Cost-effectiveness of influenza vaccination of healthy children.Vaccine2449344941
- 37. Aballea S, Chancellor J, Martin M, Wutzler P, Carrat F, et al. (2007) The cost-effectiveness of influenza vaccination for people aged 50 to 64 years: an international model. Value in Health 10: 98–116.S. AballeaJ. ChancellorM. MartinP. WutzlerF. Carrat2007The cost-effectiveness of influenza vaccination for people aged 50 to 64 years: an international model.Value in Health1098116
- 38. Aballea S, De Juanes JR, Barbieri M, Martin M, Chancellor J, et al. (2007) The cost effectiveness of influenza vaccination for adults aged 50 to 64 years: a model-based analysis for Spain. Vaccine 25: 6900–6910.S. AballeaJR De JuanesM. BarbieriM. MartinJ. Chancellor2007The cost effectiveness of influenza vaccination for adults aged 50 to 64 years: a model-based analysis for Spain.Vaccine2569006910
- 39. Maciosek MV, Solberg LI, Coffield AB, Edwards NM, Goodman MJ (2006) Influenza vaccination. Health impact and cost effectiveness among adults aged 50 to 64 and 65 and older. Am J Prev Med 31: 72–79.MV MaciosekLI SolbergAB CoffieldNM EdwardsMJ Goodman2006Influenza vaccination. Health impact and cost effectiveness among adults aged 50 to 64 and 65 and older.Am J Prev Med317279
- 40. Newall AT, Scuffham PA, Kelly H, Harsley S, MacIntyre CR (2008) The cost-effectiveness of a universal influenza vaccination program for adults aged 50–64 years in Australia. Vaccine 26: 2142–2153.AT NewallPA ScuffhamH. KellyS. HarsleyCR MacIntyre2008The cost-effectiveness of a universal influenza vaccination program for adults aged 50–64 years in Australia.Vaccine2621422153
- 41. Turner DA, Wailoo AJ, Cooper NJ, Sutton AJ, Abrams KR, et al. (2006) The cost-effectiveness of influenza vaccination of healthy adults 50–64 years of age. Vaccine 24: 1035–1043.DA TurnerAJ WailooNJ CooperAJ SuttonKR Abrams2006The cost-effectiveness of influenza vaccination of healthy adults 50–64 years of age.Vaccine2410351043
- 42. Colombo GL, Ferro A, Vinci M, Zordan M, Serra G (2006) Cost-benefit analysis of influenza vaccination in a public healthcare unit. Ther Clin Risk Manag 2: 219–226.GL ColomboA. FerroM. VinciM. ZordanG. Serra2006Cost-benefit analysis of influenza vaccination in a public healthcare unit.Ther Clin Risk Manag2219226
- 43. Rothberg MB, Rose DN (2005) Vaccination versus treatment of influenza in working adults: a cost-effectiveness analysis. Am J Med 118: 68–77.MB RothbergDN Rose2005Vaccination versus treatment of influenza in working adults: a cost-effectiveness analysis.Am J Med1186877
- 44. Samad AH, Usul MHBHJ, Zakaria D, Ismail R, Tasset-Tisseau A, et al. (2006) Workplace vaccination against influenza in Malaysia: does the employer benefit? J Occup Health 48: 1–10.AH SamadMHBHJ UsulD. ZakariaR. IsmailA. Tasset-Tisseau2006Workplace vaccination against influenza in Malaysia: does the employer benefit?J Occup Health48110
- 45. Avritscher EB, Cooksley CD, Geraci JM, Bekele BN, Cantor SB, et al. (2007) Cost-effectiveness of influenza vaccination in working-age cancer patients. Cancer 109: 2357–2364.EB AvritscherCD CooksleyJM GeraciBN BekeleSB Cantor2007Cost-effectiveness of influenza vaccination in working-age cancer patients.Cancer10923572364
- 46. Roberts S, Hollier LM, Sheffield J, Laibl V, Wendel GD (2006) Cost-effectiveness of universal influenza vaccination in a pregnant population. Obstet Gynecol 107: 1323–1329.S. RobertsLM HollierJ. SheffieldV. LaiblGD Wendel2006Cost-effectiveness of universal influenza vaccination in a pregnant population.Obstet Gynecol10713231329
- 47. Burls A, Jordan R, Barton P, Olowokure B, Wake B, et al. (2006) Vaccinating healthcare workers against influenza to protect the vulnerable-Is it a good use of healthcare resources? A systematic review of the evidence and an economic evaluation. Vaccine 24: 4212–4221.A. BurlsR. JordanP. BartonB. OlowokureB. Wake2006Vaccinating healthcare workers against influenza to protect the vulnerable-Is it a good use of healthcare resources? A systematic review of the evidence and an economic evaluation.Vaccine2442124221
- 48. Chan AL, Shie HJ, Lee YJ, Lin SJ, Chan ALF, et al. (2008) The evaluation of free influenza vaccination in health care workers in a medical center in Taiwan. Pharm World Sci 30: 39–43.AL ChanHJ ShieYJ LeeSJ LinALF Chan2008The evaluation of free influenza vaccination in health care workers in a medical center in Taiwan.Pharm World Sci303943
- 49. Ryan J, Zoellner Y, Gradl B, Palache B, Medema J (2006) Establishing the health and economic impact of influenza vaccination within the European Union 25 countries. Vaccine 24: 6812–6822.J. RyanY. ZoellnerB. GradlB. PalacheJ. Medema2006Establishing the health and economic impact of influenza vaccination within the European Union 25 countries.Vaccine2468126822
- 50. Cai L, Uchiyama H, Yanagisawa S, Kamae I (2006) Cost-effectiveness analysis of influenza and pneumococcal vaccinations among elderly people in Japan. Kobe J Med Sci 52: 97–109.L. CaiH. UchiyamaS. YanagisawaI. Kamae2006Cost-effectiveness analysis of influenza and pneumococcal vaccinations among elderly people in Japan.Kobe J Med Sci5297109
- 51. Gilman BH, Bonito AJ, Eicheldinger C (2007) Impact of influenza immunization on medical expenditures among medicare elderly, 1999–2003. Am J Prev Med 32: 107–115.BH GilmanAJ BonitoC. Eicheldinger2007Impact of influenza immunization on medical expenditures among medicare elderly, 1999–2003.Am J Prev Med32107115
- 52. Hoshi SL, Kondo M, Honda Y, Okubo I (2007) Cost-effectiveness analysis of influenza vaccination for people aged 65 and over in Japan. Vaccine 25: 6511–6521.SL HoshiM. KondoY. HondaI. Okubo2007Cost-effectiveness analysis of influenza vaccination for people aged 65 and over in Japan.Vaccine2565116521
- 53. Wang ST, Lee LT, Chen LS, Chen THH (2005) Economic evaluation of vaccination against influenza in the elderly: an experience from a population-based influenza vaccination program in Taiwan. Vaccine 23: 1973–1980.ST WangLT LeeLS ChenTHH Chen2005Economic evaluation of vaccination against influenza in the elderly: an experience from a population-based influenza vaccination program in Taiwan.Vaccine2319731980
- 54. Postma MJ, Baltussen RM, Heijnen ML, de Berg LT, Jager JC (2000) Pharmacoeconomics of influenza vaccination in the elderly: reviewing the available evidence. Drugs Aging 17: 217–227.MJ PostmaRM BaltussenML HeijnenLT de BergJC Jager2000Pharmacoeconomics of influenza vaccination in the elderly: reviewing the available evidence.Drugs Aging17217227
- 55. Postma MJ, Jansema P, van Genugten ML, Heijnen ML, Jager JC, et al. (2002) Pharmacoeconomics of influenza vaccination for healthy working adults: reviewing the available evidence. Drugs 62: 1013–1024.MJ PostmaP. JansemaML van GenugtenML HeijnenJC Jager2002Pharmacoeconomics of influenza vaccination for healthy working adults: reviewing the available evidence.Drugs6210131024
- 56. Kunji K (2004) The Ontario experience with universal vaccination. K. Kunji2004The Ontario experience with universal vaccination.National Influenza Vaccine Summit. Atlanta. National Influenza Vaccine Summit. Atlanta.