Propensity Score Estimation to Address Calendar Time-Specific Channeling in Comparative Effectiveness Research of Second Generation Antipsychotics

Background Channeling occurs when a medication and its potential comparators are selectively prescribed based on differences in underlying patient characteristics. Drug safety advisories can provide new information regarding the relative safety or effectiveness of a drug product which might increase selective prescribing. In particular, when reported adverse effects vary among drugs within a therapeutic class, clinicians may channel patients toward or away from a drug based on the patient's underlying risk for an adverse outcome. If channeling is not identified and appropriately managed it might lead to confounding in observational comparative effectiveness studies. Objective To demonstrate channeling among new users of second generation antipsychotics following a Food and Drug Administration safety advisory and to evaluate the impact of channeling on cardiovascular risk estimates over time. Data Source Florida Medicaid data from 2001–2006. Study Design Retrospective cohort of adults initiating second generation antipsychotics. We used propensity scores to match olanzapine initiators with other second generation antipsychotic initiators. To evaluate channeling away from olanzapine following an FDA safety advisory, we estimated calendar time-specific propensity scores. We compare the performance of these calendar time-specific propensity scores with conventionally-estimated propensity scores on estimates of cardiovascular risk. Principal Findings Increased channeling away from olanzapine was evident for some, but not all, cardiovascular risk factors and corresponded with the timing of the FDA advisory. Covariate balance was optimized within period and across all periods when using the calendar time-specific propensity score. Hazard ratio estimates for cardiovascular outcomes did not differ across models (Conventional PS: 0.97, 95%CI: 0.81–3.18 versus calendar time-specific PS: 0.93, 95%CI: 0.77–3.04). Conclusions Changes in channeling over time was evident for several covariates but had limited impact on cardiovascular risk estimates, possibly due to unmeasured confounding. Although calendar time-specific propensity scores appear to improve covariate balance, the impact on comparative effectiveness results is limited in this setting.


Background
Comparative effectiveness research (CER) aims to evaluate the relative benefit or harms of treatment alternatives in patients who are representative of those treated in real-world practice [1], as opposed to the highly selected groups studied in randomized controlled trials. Observational studies of administrative claims are increasingly used for CER of pharmaceutical products. These studies need to address several important sources of bias, including confounding due to changes in the clinical use of treatments over time (i.e., channeling bias) [2]. Channeling occurs when users of a medication and its potential comparators are selectively prescribed a particular agent based on differences in underlying patient characteristics [3]. If these characteristics also affect the risk for the outcome of interest, channeling will lead to confounding.
The impact of calendar time-specific channeling is a particularly important consideration when assessing the comparative safety for drugs within a medication class targeted by a Food and Drug Administration (FDA) safety advisory. In such cases, when reported adverse effects vary among drugs within a therapeutic class, clinicians may respond by channeling patients toward or away from a particular treatment alternative based on the patient's underlying risk for an adverse outcome.
One way to address calendar time-specific channeling is through the use of calendar time-specific propensity scores [4] which estimate the probability of receiving treatment within carefully defined study time periods anchored around changes in policy or new safety or effectiveness information. This propensity score can also provide insight into changes in channeling over time and may provide better confounding control for treatment effect estimates.

Clinical Context
Second generation antipsychotics (SGAs) are effective medications for the treatment of psychosis, but also have been associated with metabolic adverse effects that increase the risks for cardiovascular morbidity and mortality with long term use [5]. In late 2003, the FDA issued a class-wide advisory regarding increased risks for metabolic adverse effects for patients using second generation antipsychotics [6]. Shortly after the class-wide warning was issued, members of the American Diabetes Association, the American Psychiatric Association, the American Association of Clinical Endocrinologists and the North American Association for the Study of Obesity developed a professional society consensus statement, identifying antipsychotic agents as being of higher or lower metabolic risk based on evidence available at that time [7]. This statement implicated clozapine and olanzapine as the agents with the highest known metabolic risk, with each of the other agents identified as being of either moderate (quetiapine or risperidone) or low/unknown (aripiprazole or ziprasidone) risk.
Previous studies identified large declines in olanzapine use among Medicaid enrollees following the FDA advisory and consensus statement publication [8][9][10]. Additional declines in olanzapine use have been documented in Florida during the years following the advisory, as Florida Medicaid temporarily implemented a prior authorization policy to restrict access to this product in July of 2005 [11]. It is unclear whether these observed decreases in olanzapine use are due to reductions in olanzapine use primarily among patients at higher risk for metabolic adverse events, or if decreases in olanzapine use were non-selective. If the former is true, channeling of patients at higher risk for adverse metabolic-related outcomes away from olanzapine could result in biased estimates of the comparative safety of second generation antipsychotic agents, particularly as related to adverse metabolic or cardiovascular related outcomes.
The objectives of this study are to evaluate (1) changes in channeling of patients away from olanzapine over time; (2) whether covariate balance is improved by matching within calendar time-specific strata; and (3) whether use of calendar time-specific propensity scores results in different cardiovascular risk estimates among second generation antipsychotic users as compared with unadjusted and traditional propensity score matched analyses.

Data Source
We used inpatient, outpatient and pharmacy claims from the Florida Medicaid program from 2000-2006 for this analysis. This data source contains inpatient, outpatient and prescription drug utilization data for all paid claims for Florida's fee-for-service Medicaid enrollees. Florida's Medicaid program is the fourth largest in the country and represents an ethnically and racially diverse population [12,13]. Furthermore, Medicaid insures a large proportion of patients with severe mental illness and paid for the majority of antipsychotic prescriptions in the US during this period [13,14].  Table S1 for details) [15][16][17][18][19][20][21].

Study Design and Cohort Identification
We created a retrospective cohort of new second generation antipsychotic users for our analysis. We selected adults aged 18-64 who were enrolled in Florida's fee-for-service Medicaid program. Enrollees were required to have a new SGA prescription fill between January 1, 2001 and December 31, 2005 and at least 6 months of continuous Medicaid enrollment prior to their index prescription fill date (N = 37,130). SGAs included: olanzapine, quetiapine, risperidone, ziprasidone, and aripiprazole. Clozapine was excluded due to its infrequent use (,1% of SGA fills). From this sample, we excluded individuals with codes for coronary artery disease (acute myocardial infarction, coronary artery revascularization, angina and chronic ischemic heart disease) during the 6 months prior to drug initiation (N = 2,370). Finally, we excluded enrollees who did not fill a second prescription for their index second generation antipsychotic medication within 91 days or those who experienced an outcome of interest prior to their second fill (N = 14,134). This resulted in a final sample of 20,626 Medicaid enrollees.

Follow Up
We followed these second generation antipsychotic initiators from their second prescription fill date until they experienced a cardiovascular outcome of interest, a gap in their Medicaid enrollment of over 2 months or until December 31, 2006. Enrollees who experienced a change in therapy (switching, augmenting or discontinuing their index prescription (see Table S2 for coding details)) were followed for up to 6 months after their treatment change to identify relevant cardiovascular outcomes allowing for a 6 months carry-over effect.

Dependent Variable
The primary outcome was coronary artery disease. We defined coronary artery disease as acute myocardial infarction, coronary artery revascularization (either percutaneous coronary intervention or coronary artery bypass grafting), angina or chronic ischemic heart disease (see Table S2 for coding details). Covariates Variables potentially associated with both second generation antipsychotic treatment selection and coronary artery disease, or coronary artery disease alone, were included in the propensity score model. These include: patient age (in years), sex, race (white, black, Hispanic or other), Medicaid eligibility (Supplemental Security Income or other), Medicaid region (categorized as 1-11 based on pre-defined regions within the Florida Medicaid program), prior health services utilization (number of inpatient visits, number of outpatient visits, presence of inpatient services for mental health treatment, receipt of care from a psychiatrist), metabolic or cardiovascular related comorbidities (diabetes, hyperlipidemia, hypertension, obesity, peripheral vascular disease, cerebrovascular disease, cardiac arrhythmias, heart failure), mental health related conditions (schizophrenia, bipolar disorder, psychosis, dementia, substance abuse, major depressive disorder, anxiety disorder), prior medication use indicating or increasing potential for cardiovascular risk (cox-2 inhibitors, NSAIDs, aspirin, antiplatelet medications) and measures of medical comorbidity (using the conditions identified in the Charlson comorbidity index separately) [22]. Variables were dichotomized unless otherwise indicated.

Analytic Methods-Propensity Score Estimation and Channeling Identification
Propensity scores were estimated using logistic regression by modeling the predicted probability of receiving olanzapine as a function of the covariates measured in the 6 months prior to the index prescription fill date. We created propensity score-matched cohorts in which patients initiating treatment with olanzapine were matched 1:1 to those initiating any other second generation antipsychotic medication. To create the propensity score matched cohorts we used a 5R1 propensity score digit matching algorithm [23]. In all, there were 47 parameters estimated in the PS model, providing approximately 26 events per predictor using the revised sample/time periods.
To evaluate changes in channeling away from olanzapine over time we estimated propensity scores separately in each pre-defined time period and constructed calendar time-specific propensity score-matched cohorts within each time period. We then combined the resulting datasets for the matched-pairs analysis. To evaluate the relative performance of the calendar time-specific propensity score, we also estimated a conventional propensity score (using all available data from January 2001-December 2005), while controlling for the year of initiation using dummy variables. As with the calendar time-specific propensity score, we used the same pre-treatment covariates in the conventional propensity score model and we utilized the 5R1 digit propensity score matching algorithm to generate matched pairs for analysis. For the conventional propensity score model, matches were made across all periods, rather than restricting to within-period matches.
Adjusted odds ratios from each of the propensity score estimation models were estimated overall and within each calendar time period to identify channeling of patients away from olanzapine over time. Channeling was investigated for covariates that were strong predictors of coronary artery disease and for conditions specifically identified in the metabolic risk advisory and consensus statement. Changes in channeling by covariate over time were assessed using the Cochran-Armitage test for trend. We calculated the absolute standardized mean differences to evaluate between-group covariate balance across key covariates overall and within period to assess the performance of each propensity score matching method [24].

Analytic Methods-Estimating Model-Specific Changes in Cardiovascular Outcomes
We estimated overall and period-specific hazard ratios for cardiovascular outcomes for patients using olanzapine versus those using other second generation antipsychotics using Cox Proportional Hazard Models. We compare estimates from the unadjusted model with estimates from the conventional propensity scorematched and calendar time-specific propensity score-matched cohorts. Because the propensity score matching techniques are used to balance characteristics of our treatment and control groups we do not include additional covariates in these models.

Sensitivity Analyses
In sensitivity analyses we reduced the post-censored observation time for enrollees who experienced a change in therapy (switching, augmenting or discontinuing their index prescription) from 6 months to 3 months to determine the extent to which the longer timeframe might bias our estimates of drug-related cardiovascular outcomes. Additionally, as sensitivity analysis for our primary ''as treated'' analysis, we used an intent-to-treat approach in which individuals were assumed to continue on their index prescription fill until they experienced a cardiovascular outcome of interest, a gap in their Medicaid enrollment of over 2 months or until December 31, 2006.

Cohort Characteristics
Before matching there were 7,082 new users of olanzapine and 13,544 new users of other second generation antipsychotic agents included in the sample (Table 1). Of those, 96% of olanzapine users were successfully matched in both the traditional propensity score model and the calendar time-specific model. In the calendar time-specific model, match percentages were 93.4% in period 1, 97.9% in period 2 and 99.8% in period 3. As compared with individuals initiating olanzapine, before matching, those initiating other second generation antipsychotic agents over the study period had less frequent health services utilization (average of 14.6 versus 16.1 outpatient visits in the prior 6 months), and were more likely to have diabetes (11.5% versus 8.5%) and hypertension (27.4% versus 23.9%) and less likely to have pulmonary disease (9.2% versus 11.8%).

Channeling of Patients Away from Olanzapine
We evaluated channeling by period for the top predictors of coronary artery disease in our sample (Table 2). Channeling was not evident for most predictors of coronary artery disease when comparing the odds ratios generated in period 1 to those generated in each subsequent period. However, for patients with prior diagnoses of hyperlipidemia and hypertension we see some evidence that those patients were less likely to receive olanzapine over time (p-value for trend: 0.02 and 0.05, respectively).

Impact of Estimation Strategy on Covariate Balance
We used the average standardized mean difference to assess within-and across-period covariate balance for the top predictors of coronary artery disease (Table 3). Covariate balance was improved in both the conventional and calendar time-specific propensity score matched cohorts as compared with the unmatched cohort. Overall, the calendar time-specific propensity score model produced smaller standardized differences than the conventional propensity score for 23 of 32 comparisons. In the remaining 9 comparisons, differences between the two models were small (usually less than 0.02).

Impact of Estimation Strategy on Hazard Ratio Estimates
We estimated the hazard ratio for coronary artery disease among individuals using olanzapine versus those using any other second generation antipsychotic agent (Table 4). Models were estimated for the entire study period (2001)(2002)(2003)(2004)(2005) and separately by period. In unadjusted models we observe a hazard ratio of 0.96 (95% CI: 0.82-3.05) for the full study period. These estimates were unchanged in both the conventional propensity score matched analysis and the calendar time-specific propensity score matched analysis (HR: 0.97, 95%CI: 0.81-3.18, conventional PS; HR: 0.93, 95%CI: 0.77-3.04, CTS PS). Similarly, we found no differences in hazard ratio estimates across any model or any period.

Sensitivity Analyses
Results from sensitivity analyses reducing post-censored followup time from 6 months to 3 months and utilizing an intent-totreat approach for classifying person-time of exposure were similar to results obtained in the primary analysis (not shown).

Discussion
Researchers intending to use administrative or secondary data sources for the purposes of comparative effectiveness and safety studies should consider the important role that policies and regulatory decisions play in shaping the use of treatments over time. In prescription drug-related research, the characteristics of populations receiving a particular treatment may vary over time with new drug approvals, new entries into a class, and regulatory warnings regarding emerging safety concerns. Here, we investigate whether concerns about increased metabolic risk for olanzapine shifted prescribing among second generation antipsychotic users.
We found some evidence of diagnosis-specific channeling of patients at higher cardio-metabolic risk away from olanzapine following an FDA advisory and subsequent consensus statement that highlighted these risks. In particular, individuals with prior hyperlipidemia and hypertension became less likely to receive olanzapine over time. However, channeling was not evident for other key risk factors, such as diabetes. Interestingly, in this sample patients with diabetes were less likely to receive olanzapine than other SGAs in each period, even those before the advisory. This suggests that clinicians were selectively prescribing non-olanzapine SGAs for patient with diabetes even before the advisory period.
Conventional propensity score models that control for time provide an overall estimate of the effect of a predictor on the likelihood of using a specific treatment but a ''null'' effect over a period may represent a higher likelihood of receiving the drug at one point and a lower likelihood of receiving the drug at another. Estimates of the odds of receiving olanzapine among patients with prior hyperlipidemia provide the best demonstration of this from our sample. Here we see that the overall point estimate is 1.17 for the study period, but period-to-period estimates range from 1.28 in period 1 to 0.97 in period 3. Note that the overall estimate is balanced on this measured covariate and would provide a valid estimate over the full study period. However, once we condition on each period (and thus misspecify the propensity score model in Table 4 at least some of the calendar time strata) covariate balance is no longer achieved using the matched pairs identified in the conventional model. We had hypothesized that olanzapine use would increase the risk for cardiovascular adverse events prior to the advisory, but that this association would fade over time even with proper control for measured confounders as individuals with higher baseline risk not captured by measured covariates were moved away from olanzapine. In our sample, estimates of the comparative effectiveness of second generation antipsychotics on cardiovascular outcomes did not appear to be influenced by estimation strategy. The small number of outcome events and unmeasured confounding appeared to influence our ability to detect any differences between olanzapine and other SGAs for adverse cardiovascular outcomes.
A key consideration when selecting a propensity scoring strategy is how well the resulting propensity score matched groups are balanced on measured characteristics. While we found only minor differences in the covariate balance between propensity score estimation strategies, we did find that within-period balance was improved by using a calendar time-specific propensity scoring model. While both matching strategies appeared to improve covariate balance overall, investigators who are concerned with investigating within-period differences would benefit from using a calendar time-specific propensity score method. This method is easy to implement and allows for within period comparisons without resulting in ''breaking'' the matches created in a conventional propensity score model. Additionally, the calendar time-specific propensity score allows appropriate changes in assignment of propensity for treatment receipt for each covariate and forces matches to be made within the investigator-specified time periods so that treatment and control groups are well balanced over each period when changes in channeling may be occurring.
Specific limitations include a lack of information on important unmeasured risk factors for cardiovascular disease (e.g., smoking, family history of cardiovascular disease) or poorly measured cardiovascular risk factors (e.g., potential under-coding of diabetes, hyperlipidemia and obesity) that may influence prescribing of olanzapine. It is important to realize, however, that due to our study design using an active comparator cohort, this only leads to confounding bias if these cardiovascular risk factors also affect channeling between olanzapine and other second generation antipsychotics. While we observed some evidence for bias due to unmeasured confounding, this does not reduce our ability to detect and control for changes in measured channeling over time. Second, the heterogeneous composition of our comparison group may influence our ability to detect differences in channeling and/ or cardiovascular risk (e.g., including all non-olanzapine second generation antipsychotic agents, which vary in metabolic risk). Further, as a result of the drug safety advisory, olanzapine use declined steadily over the study period. This reduced the sample size available for later periods. Additionally, other factors may have influenced the use of SGAs over our time period, including drug promotion and drug approvals within the class. Regarding the latter point, a low metabolic risk SGA (aripiprazole) was approved during our study period and may have influenced our estimates of channeling. Next, the best prediction model for detecting channeling may not be the best model to control for confounding, which is the primary goal of the PS model [25]. We implemented the propensity score using matching to simplify the presentation of our results. Propensity score matching allows us to estimate the treatment effect in the treated (olanzapine users). Other researchers may elect to use alternate propensity score implementation strategies (e.g., weighting or stratification), depending on the treatment effect of interest [26]. It will be important for future studies to estimate the impact of these different propensity scoring estimation and implementation methods on both covariate balance and on health outcomes.

Conclusions
Although calendar time-specific propensity scores appear to improve covariate balance, the impact on comparative effectiveness results is limited in this setting. Future work is needed to examine the utility of this method in observational comparative effectiveness research. Researchers should consider using calendar time-specific propensity scores to improve covariate balance and to identify and potentially reduce channeling bias in studies where prescription drug prescribing practices might have changed over time and calendar time-specific channeling is suspected.