Foundational support.

All participating clinics receive foundational support, which allows collecting a minimum data set needed for the evaluation. Foundational support includes a baseline assessment (first three project months), SBIRT Oregon online toolkit access, and an exit assessment (final three project months). The SBIRT Oregon online toolkit provides multiple resources, including demonstration videos, role-play exercises, clinic workflows guides, pocket-sized readiness rulers, and patient testimonial videos [12]. Our team anticipates that with access to SBIRT Oregon resources, certain highly motivated clinics will be able to improve their SBIRT workflows and reporting.

Supplemental support.

Prior work highlights dedicated staff’s value in supporting and implementing guidelines and toolkit resources into clinical workflows and practices [15, 44, 45]. Thus, we also seek to provide supplemental implementation support, coordinated by a trained practice facilitator, to interested clinics. Supplemental support may include practice facilitation, HIT support, peer-to-peer learning collaboratives, and expert consultation via academic detailing. Academic detailing is an interactive educational outreach to providers and staff to disseminate evidence-based information about prescribing practices and therapeutic decision-making to improve patient care [46]. HIT support includes generating data to inform changes to workflows and reporting for Oregon’s CCO SBIRT metric [27]. If the Oregon CCO metric cannot be generated or is not desired by the clinic, the practice facilitator will explore alternative measures, including the National Quality Forum 2152/ Centers for Medicare & Medicaid Services 431 or help develop a custom EHR query [47]. See Table 3 and S2 Appendix for additional detail on the supplemental support strategies.

Key informant interviews.

We will conduct semi-structured qualitative interviews with three types of key informants: CCO staff, practice facilitators, and clinic primary points of contact. Interviews with CCO staff (N = 15), conducted in the first 12 months of the study, will address how CCOs work with clinics to improve their workflows and documentation for delivery of SBIRT and MAUD, informing a baseline understanding of regional context and supporting clinic recruitment. Every six months, individual semi-structured interviews will be conducted with practice facilitators to gather information about how they tailor implementation support to clinics and develop their expertise. Exit interviews with the clinic primary point of contact will be conducted following implementation. They will provide data about the clinic’s experience with implementation support and the status of their SBIRT performance at the conclusion of the study. All interviews will last 45–60 minutes and will be conducted by qualitative analysts in person or via videoconference. Transcripts will be professionally transcribed and validated for accuracy.

Clinic contact logs.

Following all clinic interactions, practice facilitators will record field notes in REDCap using structured forms containing open- and closed-ended prompts. Practice facilitators will use two types of structured note-reporting guides to record their interactions and clinic support. Brief communication logs will track short interactions with clinics (e.g., email outreach, scheduling), and in-depth intervention tracker forms will describe engagement, content, and the quality of scheduled facilitation meetings. The logs will produce evaluation data specified in the request for applications, such as the number and type of interactions and implementation strategies employed. The intervention tracker documents practice facilitator tailoring concerning clinic context and perceptions of clinic capacity for change. A qualitative analyst will regularly review the clinic contact logs for completeness.

Clinic baseline assessment.

The baseline assessment includes five data points: The clinic intake form, needs assessment, workflow observations, HIT capacity assessment, and SBIRT and MAUD performance data collection (Table 4). The clinic intake form is a survey administered to collect details on clinic size, ownership, and electronic health record use. The clinic intake form includes an SBIRT and MAUD implementation checklist. This checklist will help determine a clinic’s capacity to capture and report data on intervention outcomes and inform tailored EHR support. The needs assessment addresses clinic practice culture, external context, and QI capacity. The observation of practice workflows will involve shadowing primary care clinicians and clinic staff to understand clinic culture and workflows related to SBIRT and MAUD. The HIT capacity assessment is used to gauge a clinic’s screening and tracking processes and ability to report the CCO SBIRT metric. In tandem, we will collect SBIRT and MAUD performance data using aggregate quantitative data from clinic EHRs or via manual chart audits. These final two data sources will inform the level of HIT support clinics receive to produce study outcomes data and CCO SBIRT quality metric reporting.

Supplemental implementation support plan.

Practice facilitators use information from the baseline assessment to develop and refine a tailored supplemental support plan for each clinic in partnership with the clinic’s primary point of contact [50]. This plan draws on findings from the needs assessment. It specifies the anticipated supplemental implementation strategies that the clinic receives (e.g., practice facilitation, HIT support, and expert consultation for SBIRT or MAUD workflows) and whether the activities will occur remotely or in person. The supplemental support plan is delivered over a 9-month implementation period. Supplemental implementation support plans may continue to evolve based on the clinics’ needs and capacity during the active facilitation period; the initial plan and adaptations will be documented in the REDCap database by the practice facilitator.

Periodic reflections with practice facilitators.

Monthly, a qualitative analyst will facilitate 60-minute periodic reflections with the practice facilitator group via videoconference [51]. The qualitative analyst will solicit discussion topics from study leadership and practice facilitators before each session and develop a short interview guide. The group format will help identify patterns across clinics and facilitators related to clinic context and tailoring of implementation support. Sessions will be recorded, professionally transcribed, and validated for accuracy.

Exit assessment.

All clinics will participate in an exit consultation, coordinated by the practice facilitator, in the final quarter of the intervention (months 13–15). The closing activities allow solidifying program maintenance and updating the clinic intake form (including the SBIRT and MAUD implementation checklist). During the exit assessment, we will also collect the second set of SBIRT and MAUD performance data to compare SBIRT and MAUD delivery rates to eligible patients at baseline and post-intervention per the specifications of the request for applications and external evaluation. We anticipate that some clinics will develop the capacity for baseline reporting via participation in the intervention. After completing these activities, the practice facilitator will connect the clinic point of contact to the qualitative team to complete the study exit interview. Additional information on the clinic intake form (baseline assessment) and exit interview (key informant interviews) are provided above and in Table 4.

Quantitative analytic plan.

Quantitative data will be analyzed using Stata. We propose two primary performance outcomes. The first is the difference between baseline and exit assessment in the percentage of eligible patients who receive screening for UAU (and drug risk when using Oregon’s SBIRT protocol), brief intervention, and MAUD. Because baseline levels may be strongly correlated with change, we also plan to conduct a sensitivity analysis to scale the change as a proportion of the initial gap between baseline and target. The second outcome is whether the clinic can report the Oregon CCO SBIRT metric to its regional CCO by the end of the study. These outcomes will serve as the dependent variable of a linear (percentage difference) or logistic (ability to report CCO measure) regression model.

To evaluate the impact of tailored implementation support on SBIRT, MAUD, and QI capacity (Aim 2), we will combine data on clinics’ previous collaborations with ORPRN, how clinics were referred to ANTECEDENT and the number and types of contacts in the recruitment and implementation phases. We will use logistic regression to determine which factors increase the probability of successful recruitment and project completion. In addition to the predictors above, we will account for the possibility that the level of implementation success depends on the specific practice facilitator or CCO. We anticipate that higher-intensity contact will be negatively associated with clinic enrollment but positively associated with study completion (defined as participation in exit assessment). We further hypothesize that implementation support will be most effective when clinics have a QI lead and clinic champion.

Qualitative analytic plan.

All qualitative data will be maintained in ATLAS.ti and analyzed to identify patterns or themes according to Miller and Crabtree’s editing method [52, 53]. Our theoretical framework, a hybrid of i-PARIHS [33] and DSF [34] (Fig 1), will inform coding and analysis. Transcripts from the interviews with CCO staff will be coded and analyzed concurrently with data collection to identify how CCOs support clinics in achieving the SBIRT quality incentive metric (Aim 1). The remaining qualitative data (interviews with clinic contacts and PERCs, clinic contact logs, clinic surveys, baseline assessment, and periodic reflection transcripts) will be coded and analyzed on a rolling basis as clinics conclude participation in the intervention. Multiple coders will analyze, discuss data, and synthesize findings of clinic context, QI capacity, engagement with the intervention, level of implementation support provided, clinic disruptions, and ability to capture and report quantitative performance data on SBIRT and MAUD. In addition to clinic-level analysis, qualitative data about practice facilitator expertise and perspectives of practice transformation will be analyzed concerning study Aim 3. Preliminary themes at both levels will be identified and refined in consultation with the study team and ANTECEDENT Advisory Board as a form of member checking [52, 53].

Systems science analytic plan.

To examine how practice facilitators tailor implementation support based on clinic context, the SBIRT innovation, and expertise over time, we will systematically generate causal-loop diagrams from qualitative data to describe changes in practice facilitators’ perspectives of practice transformation (Aim 3). Causal-loop diagramming is a method from systems science that identifies feedback loops driving nonlinear behavior in complex systems [54, 55]. Diagrams corresponding to each facilitator’s perspective will allow for comparison between individuals over time. Once data collection is complete, we will use a group modeling approach [56] to synthesize the team’s mental model of effective practice transformation to support SBIRT and MAUD in primary care.

Mixed-methods convergence.

Finally, we will merge our quantitative, qualitative, and systems science data using joint displays to reconcile differences and enable the identification of patterns and themes across clinics [57]. To further examine implementation support provided to clinics (Aim 2), we will review descriptive statistics from the clinic contact logs, clinic intake form, and HIT capacity assessment in tandem with our qualitative data. Our goal is to classify clinics along three primary dimensions: (1) baseline capacity, (2) level of implementation support, and (3) observed improvement, and to examine details of the relationships among them. To define baseline capacity, we will categorize clinics as “high”, “medium”, or “low” based on QI infrastructure and workflows for SBIRT and MAUD during the baseline assessment. Qualitative and quantitative analysts will conduct a card sorting activity to classify the participating clinics based on the level of implementation needed to implement SBIRT and MAUD successfully. Analysts will use these resulting categories to create causal predictive matrices to describe clinic features and understand the factors associated with improvement [58]. We will use correspondence analysis to investigate relationships between categorical variables (for example, whether the distribution of implementation support activities varies with clinic baseline capacity) and latent class analysis to identify groups of clinics with similar characteristics. These methods represent high-dimensional data in a low-dimensional space so that it is easier to identify patterns and critical features [59]. We hypothesize that depending on the clinic context, varying levels of implementation support may be needed to support “successful implementation”. We will strive to identify case examples to inform studies using predictive models for tailored implementation support.

Sample size considerations.

Recruiting 150 clinics will ensure we retain a minimum of 120 clinics in the final analysis after an anticipated attrition rate of 20% based on prior studies. While we lack preliminary performance estimates for SBIRT for UAU to inform power calculations, for a similar measure—depression screening and follow-up—the OHA reported that CCO performance in the first year of tracking (2014) ranged from 3.3% to 68.1% [60]. Thus, we anticipate similar, significant variability in clinic-level performance outcomes. Our sample of 120–150 clinics ensures that even targeted sub-group analyses performed at the clinic level will have desirable “large” sample properties [59].