Qualitative analysis of the dynamics of policy design and implementation in hospital funding reform

Background As in many health care systems, some Canadian jurisdictions have begun shifting away from global hospital budgets. Payment for episodes of care has begun to be implemented. Starting in 2012, the Province of Ontario implemented hospital funding reforms comprising three elements: Global Budgets; Health Based Allocation Method (HBAM); and Quality-Based Procedures (QBP). This evaluation focuses on implementation of QBPs, a procedure/diagnosis-specific funding approach involving a pre-set price per episode of care coupled with best practice clinical pathways. We examined whether or not there was consensus in understanding of the program theory underpinning QBPs and how this may have influenced full and effective implementation of this innovative funding model. Methods We undertook a formative evaluation of QBP implementation. We used an embedded case study method and in-depth, one-on-one, semi-structured, telephone interviews with key informants at three levels of the health care system: Designers (those who designed the QBP policy); Adoption Supporters (organizations and individuals supporting adoption of QBPs); and Hospital Implementers (those responsible for QBP implementation in hospitals). Thematic analysis involved an inductive approach, incorporating Framework analysis to generate descriptive and explanatory themes that emerged from the data. Results Five main findings emerged from our research: (1) Unbeknownst to most key informants, there was neither consistency nor clarity over time among QBP designers in their understanding of the original goal(s) for hospital funding reform; (2) Prior to implementation, the intended hospital funding mechanism transitioned from ABF to QBPs, but most key informants were either unaware of the transition or believe it was intentional; (3) Perception of the primary goal(s) of the policy reform continues to vary within and across all levels of key informants; (4) Four years into implementation, the QBP funding mechanism remains misunderstood; and (5) Ongoing differences in understanding of QBP goals and funding mechanism have created challenges with implementation and difficulties in measuring success. Conclusions Policy drift and policy layering affected both the goal and the mechanism of action of hospital funding reform. Lack of early specification in both policy goals and hospital funding mechanism exposed the reform to reactive changes that did not reflect initial intentions. Several challenges further exacerbated implementation of complex hospital funding reforms, including a prolonged implementation schedule, turnover of key staff, and inconsistent messaging over time. These factors altered the trajectory of the hospital funding reforms and created confusion amongst those responsible for implementation. Enacting changes to hospital funding policy through a process that is transparent, collaborative, and intentional may increase the likelihood of achieving intended effects.

1 QBP Semi-Structured Interview Guide Target Population: Questions will be administered to three levels of key informants specified in the Protocol: • Level 1: individuals who conceived of and designed QBPs • Level 2: individuals in organizations that are supporting QBP adoption • Level 3: individuals in hospitals responsible for implementing QBPs Reminder: no speaker phone Preamble to Interview: As part of health system funding reform, the Ontario government introduced a program called Quality-Based Procedures, or QBPs. Through this research, we hope to understand how QBPs are being implemented in Ontario hospitals, how this compares to the expectations of those responsible for developing the QBP policy, what would enable the best-practices described in QBP pathways to be successfully implemented, and what the outcomes of QBPs have been so far. The results will be shared in both publications and presentations.
Consent: I'll turn on the tape recorder now, read the consent script, and obtain your verbal consent.
Interviewer: Karen Palmer, Strategic Health Policy Lead, WCRI (WCH): For the purpose of the recording, this is Karen Palmer and I'm interviewing NAME who is the TITLE with ORGANIZATION, a LEVEL (1, 2, 3) key informant, on DATE at TIME (EST).
Thank you so much for agreeing to participate in this study about the development and implementation of Quality Based Procedures (QBPs). Our research is funded by CIHR and the MOHLTC.
Before we proceed with the interview, I'm going to read the consent script to obtain your verbal consent to participate in this study, and to record your responses.
The study does not involve significant risk to you and will take approximately an hour of your time. Your participation is completely voluntary and nobody else, other than those on the research team, will know whether or not you participate. We will not reveal names of institutions, or of individuals interviewed, and interview responses will be nonattributable. The information you provide will be completely de-identified. We may include direct non-attributable quotations in any publications or presentations. Data, including digital recording files and the transcribed interview data, will be appropriately de-identified and securely stored under lock and key at WCRI.
You may decline to answer questions as you see fit, you can ask to stop at any point, and you can request that your responses be excluded from our study data.
By agreeing, you are providing oral consent that you agree to participate in this study, that you agree to audio recording, and that you will answer the questions in this interview to the best of your ability.
If you agree to participate in the study please say, "I agree".
Participant: I agree Interviewer: Thank you for your oral consent. Now we will proceed with the interview.
Interview Questions: There are 2 parts to this interview. Part 1 is about QBPs in general, and the implementation process in particular. Part II is about the effects of QBPs.
PART I: These questions are about QBPs in general, and about the implementation process in particular.
A. These questions are about your general understanding of QBPs and health system funding reform (Administered to Level 3 only).

What do you understand about QBPs?
• Probe 1: How would you describe the components of QBPs? (e.g. Have you heard about changes in how we pay hospitals, or the QBP "handbooks" describing clinical pathways, or how money is going to flow according to procedures or numbers of patients? • Mechanisms to achieve goals (Probes 5-9) • Probe 5: What's your understanding of the mechanism by which QBPs are meant to achieve their goals? (i.e. step-by-step flow of how QBPs are supposed to unfold to achieve the goal) Did you do a gap analysis? How long does it take to implement a new QBP, with handbook, end-to-end?
• Probe 6: Do you think there have been changes over time in the mechanism of how QBPs are meant to achieve their goals? If so, you've already walked me through how QBPs were supposed to unfold, so can you now walk me through those changes as you understand them? (Don't read this: The intent is to build a flow chart showing respondents perception of QBP evolution.) • Probe 7: Why were QBPs the policy tool chosen to achieve the goal(s) you mentioned, rather than something else?
• Probe 8: Were options, other than the mechanism of QBPs, considered for achieving the goals you mentioned? (If yes: What were they? Why weren't they selected? If no: Why were other options not considered?
• Probe 9: Do you think that people's understanding of the mechanism behind QBPs is consistent across QBPs? Why or why not? Is it consistent across organizations? Why or why not?
Theory (i.e. the system of ideas and evidence supporting a belief that the goal can be achieved by that mechanism?) (Probes 10-11) • Probe 10: What's your understanding of the theory underpinning QBPs?
• • Probe 1: How do you think the implementation is going (within your organization, Levels 2, 3) compared to the intended or anticipated path?
• Probe 2: Do you think there is a shared impression of how implementation is going (stop here for Level 1) within your unit or institution or agency or division?
• • Probe 1: To whom is your organization accountable for the successful implementation of QBPs?
• Probe 2: At the personal level, to whom are you personally accountable for QBP implementation, and how do they measure your success in implementing QBPs?
• Probe 3: At the organizational level, how is QBP implementation being measured, or assessed, or evaluated? 7. Let's talk about the major barriers and facilitators, in terms of people and personalities, to the successful implementation of QBPs. (Administered to Levels 1, 2, 3, except as noted) • Probe 1: Speaking generally, how does organizational culture affect the successful implementation of QBPs? (Level 1 stop here) In your organization? Elsewhere in the system?
• Probe 2: Speaking generally, how does organizational leadership affect the success of QBPs? (Level 1 stop here) In your organization? Elsewhere in the system?
• Probe 3: Do you think different players in an organization have different tolerance for QBPs (e.g. administrators, clinicians)?
• Probe 4: How does your own personal capacity to manage change and analyze/improve performance affect the success of QBPs? (Levels 3 only) • Probe 5: Have the barriers and facilitators to success varied depending on the QBP? 8. What supports or enablers, in terms of tools and data, been important to the uptake or effectiveness of QBPs? (Administered to Levels 1, 2, 3) • Probe 1: Do you think any specific supports or enablers would help improve the speed, efficiency, effectiveness, and/or impact of QBP implementation, and why would they help?
• Probe 2: Have those supports or enablers been suggested at your organization? If so, have they been implemented? If not, why not?
• Probe 3: Who in your organization is likely to be most affected by the changes that will come as a result of QBPs? How likely are they to change their behavior to adapt to QBPs? How can they be encouraged to change? (Levels 2, 3) 9. I'm going to read you a list of adoption tools and supports. Which of these tools and supports would help to enable QBP adoption, and why do you think they would help? (Answer yes/no to each, and whether it would help if these were developed and delivered centrally (say by HQO, or another adoption agency, or provincially, or by the LHIN) or locally (by each provider organization or hospital). Who should develop and deploy these tools/supports? (Administered to Levels 1, 2, 3).
[Explain: Read HQO list of interventions in right column]