Interviews.

All SCOPE-MH hub sites were invited to participate in a virtual semi-structured interview. These interviews were conducted by the evaluation team which included a Registered Nurse with specialized knowledge in psychiatric and mental health care and qualitative methodology (CW), a Research Assistant who has supported the implementation of SCOPE-MH and with prior experience in conducting and supporting qualitative studies (ME), and a Psychiatrist who is the director for the SCOPE-MH program (BBA) at the site that acted to germinate the program dissemination. Interviews were completed using web-conferencing software (Microsoft Teams) between March and June 2023. All interviews were digitally recorded and transcribed with transcriptions checked against the original audio for accuracy.

Interview questions were developed by the evaluation team and mapped to the RE-AIM framework. Acknowledging the varied roles that interdisciplinary team members have in delivering the SCOPE-MH program, alongside a set of core questions, interview guides were adapted to each professional role. For example, primary care practitioners were asked about their access to the SCOPE-MH program, psychiatrists were asked about factors that influence their decision to accept referrals through the program, nurses and social workers were asked about their needs in implementing and developing the program, and navigators were asked about the referral pathway for patients of the program. All evaluation participants were asked about how they viewed their involvement in SCOPE-MH as contributing to the aims identified in the Quintuple Aims, [12] as well as how they felt that they could contribute to further development of the SCOPE-MH program.

Data analysis.

All anonymized interview transcripts and documents were uploaded to N.Vivo v12 for data management and analysis. Applying qualitative content analysis, a mix of both deductive coding, using the RE-AIM framework and set criteria and definitions of the framework, [13] and inductive analysis, was used. This analytical approach permitted an in-depth evaluation of the development, including adaptations of the program, across different sites as well as an identification of both challenges and opportunities for the implementation of the SCOPE-MH program. One evaluation team member (CW) independently analyzed all interview transcripts. Following this analysis, the full evaluation team (CW, ME, and BBA) met to review coding, identify relationships, discuss ideas, and select exemplar quotes to support established topic summaries.

Ethical considerations.

As this project was for the purposes of quality improvement, and was confirmed as such by the SCOPE host site (Women’s College Hospital Research Ethics Board) prior to project start, no institutional ethics approval was sought. As part of the original SCOPE project design, ongoing evaluation of the service was planned. All participating team members verbally consented to recording the discussions. This consent was documented in the transcript.

Participant characteristics.

For this project, participants included interdisciplinary staff members from each of the eight SCOPE-MH sites across the greater Toronto area. A total of 29 team members were interviewed with 44% (n = 13) of the attendees identifying as Program, Clinic, or Project Managers, 31% (n = 9) as Social Workers or Nurses, 17% (n = 5) as Primary Care Providers or Leads, 6% (n = 2) as Psychiatrists. On average, three team members joined each interview. An overview of interview attendees from each site are detailed in Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Interview attendees from the SCOPE-MH sites (n = 29).

https://doi.org/10.1371/journal.pone.0303750.t001

SCOPE-MH development and implementation.

Across all eight sites, the SCOPE-MH program has been fully implemented but with differing degrees of development. For some sites, the program has been operating for over a year, whereas with other sites, only a few months. Despite this difference, as well as differences in team membership and access to services due to geography, there were commonalities in how the model was developed, adapted, and in challenges and opportunities identified. Findings are reported and organized according to each of the components of the RE-AIM framework [13] with equity-related perspectives and considerations incorporated. Additionally, challenges to the uptake and implementation of SCOPE-MH are described.

Reach. Team members captured the reach of the SCOPE-MH program through their description of both the targeted population accessing the program, as well as in the uptake of the program at the level of the PCPs.

Common characteristics of patients who accessed the program included prevalent diagnostic needs and specific needs for care modalities. For example, team members across many of the sites identified adult attention deficit hyperactivity disorder as a prevalent referral reason for the program. The uptake of this specific condition reflects challenges in obtaining highly specialized psychiatric diagnoses in the community and emphasizes the need for both general and specialized psychiatric care. Beyond specific diagnoses, team members also identified that many of the patients who accessed the program had need for short-term therapy with reduced waiting lists. This included brief cognitive therapy, supportive therapy, and other counseling type services. By offering timely access to mental health services, the SCOPE-MH program was described as not only alleviating need, but also potentially mitigating the exacerbation of mental health conditions that may arise due to prolonged delays in care provision. However, in addition to these prevailing characteristics, most frequently, team members described that what was most common for those they supported through the SCOPE-MH program was instead paradoxically the “uncommon”.

“I would just say the uncommon. Simply because the way I describe my role and the SCOPE program is that the usual process works fine most of the time for most doctors, but they’re reaching out to SCOPE for support when they’re stuck. Earlier this week I needed to find a dietitian that offers counseling in-person for a child with autism and some education about avoidant food disorder. That’s uncommon. I’ve never been asked that before, might never get asked that again. There’s many one-off requests like that and I spend a lot of time finding resources to support them. There really is not a whole lot that is common.”–Nurse Navigator

In supporting the uncommon, the range of expertise across team members and the potential of the program to respond to the ever-evolving needs of complex patient needs and communities is exemplified. Demonstrating the patient-centeredness of the SCOPE-MH program, this ability to support complexity may contribute to the observed month-over-month increases in the number of PCPs accessing the program or the number of referrals made. Each of the SCOPE-MH sites described need for different services to fill different care gaps, and the varied ways that they approached these needs and gaps. For example, a more traditional uptake of the program involved the use of a social worker to navigate and coordinate care, while other sites identified more specific needs to address community-level gaps. Compared to other mental health models of care, this flexibility of how resources are utilized and what health professional roles are delivering the model is a unique facet of the SCOPE-MH program.

Effectiveness. While this evaluation did not explicitly measure effectiveness, across sites, team members described the impact of the SCOPE-MH program through the capacity and relationship building effect of the program. Expanding community capacity and relationships between PCPs, community supports, and other health adjacent professionals was described to occur continuously through engagement with the program. For example, team members identified that when a PCP would call to ask a specific question, they noted that the PCP would not contact them again when a similar need was identified:

“It’s really iterative because, you know, once I’ve had that conversation with the psychiatrist, I’m not going to call in to ask that same question again in the future.”–PCP Lead

Further supporting this capacity building finding is that several sites noted in interviews that while they may be engaging more PCPs, they were having fewer requests or referrals from existing connections. This signals that the relationships made available through the SCOPE-MH program can enhance skill and knowledge, potentially increasing the confidence of the PCP in addressing mental health issues. Similarly, it was noted that when resources were provided by the navigators, PCPs were quick to accept these resources, use them, and distribute them through their own networks. One navigator even reported later seeing resources provided to a PCP on their clinic website. This capacity and relationship building plays a pivotal role in strengthening the social fabric of the community. Team members identified that they felt that relationships were being forged where they had previously not existed, and that trust and interconnectedness were tangible bi-products of these relationships:

“So much about these partnerships is relational. I think we were a bit surprised with how slow it was in the beginning. We were expecting it to be a flood of referrals right away but it makes sense that it takes time to get to know [PCPs] and build that trust.”–Social Worker

Adoption. The uptake of the program by PCPs was described as pivotal to its success and to building momentum in the community. While there was an acknowledgement from team members that the program did not necessarily offer net-new resources, the program supported communities and PCP partners by creating efficiencies with existing resources. For many sites, this was evidenced by a steady increase in referrals being made and a consistent amount of time, including facetime, spent doing work for SCOPE-MH. While the team membership varied across sites, the expertise was consistently high and there were efforts being made to create communities of practice to support the teams. At one site, this expertise was described as a potential strategy to overcome their struggles with recruiting and retaining PCPs in their community:

“What is also true is we have lots of [community name] residents with no primary care physician. Those physicians that are here are saturated, they are not accepting new patients, and then, because of that, they come to our emergency [department]. We think it could be a benefit for physicians to practice here if they do have more supports like those offered with SCOPE. I think there’s no harm marketing the SCOPE program on a larger scale. It would be appetizing for physicians and nurses.”–Program Manager

At the individual settings, adoption was also demonstrated through the speed by which later sites were able to be up and running. For example, one of the sites was able to be fully operational within just 3 months. This was attributed to the mentorship, collaboration, and sharing across sites in addition to the expertise of the SCOPE team. This cross-site learning was facilitated through regular SCOPE team meetings where all sites shared progress and needs but was also described to have happened more organically. For example, one site described reaching out to a colleague at another site with questions and from this connection, a community of practice was established.

While adoption of the program was largely positive and demonstrated the scalability of SCOPE-MH, one challenge raised by team members was specific to a need to better promote, support, and encourage communication between PCPs and psychiatrists. For example, a recurring question from team members during interviews was about how PCPs could consult with psychiatrists. Across all sites, PCPs expressed a need to directly communicate with psychiatrists to have questions answered including those related to psychiatric medications, mental health forms, or ongoing follow-up, but not knowing how to initiate this or how this collaboration is compensated. These questions and concerns demonstrate a desire for intraprofessional collaboration, but highlight the limited support to do so in the existing health system.

Implementation. True to the spirit of the program, implementation of the SCOPE-MH program was tailored to the setting within its community. This included population- and community-specific adaptations, purpose-built interventions, and in the features of the program that were emphasized.

While each of the sites are geographically within the greater Toronto area, there is significant variation in demographics and in the availability and accessibility of services depending on site. For example, some sites interviewed are urban sites with populations that are highly ethnoculturally diverse. At these sites, team members described needing to find resources that were available in languages that are not the dominant ones. At other sites considered to be more rural, challenges with availability such as access to internet were present:

“Sometimes it’s really just connection to reliable internet that is the tough part. There are a couple of folks who don’t have a computer, but they have a cell phone, but it’s a flip phone so I cannot send them e-mails. The technology piece is really challenging because I may have to ask them to drive to the town to access WiFi and that is not very private if they have to be in a library or something.”–Social Worker

During these interviews, each site was able to clearly communicate the needs of their community, and the ways that they had purposefully adapted the SCOPE-MH program to fill gaps or address known challenges. For example, in one community, they had intentionally engaged youth and young adult supports while another site had recognized the extensive need for counseling in their community and, using SCOPE-MH funds, had contracted a private therapy clinic to provide wait-list relief. The ability of sites to adapt the program to fill their service needs using different features of the program was consistently described as a strength.

Maintenance. Delivering and sustaining the program over the long-term was top-of-mind for interviewed team members. This included strategies for marketing the program, consideration for the referral pathway, and careful thought specific to funding and staffing.

E-blasts, newsletters, and direct communication were all employed to recruit PCPs into SCOPE-MH. Team members described including testimonials, example resources, showcasing the low-barrier nature of the program, and even developing their own taglines:

“The recent e-blasts were much more of a sort of marketing flyer. It had testimonials, it had a catchy title. After we sent it out, I don’t know, I think we had like five replies within the first half hour and about ten by the end of the week.”–Program Manager

Team members described these marketing products and outreach attempts as not only bringing in new PCPs, but also spoke to the troubleshooting that occurred as a result of these conversations. For example, after one e-blast, one of the sites realized that in requiring their referral pathway to use a specific e-service provider, barriers to the program were created. Subsequent communication sought to clarify the pathway.

Equity. While not explicitly part of the RE-AIM framework, equity was a recurring theme identified in completed interviews and reflected throughout all discussion of the program. Despite the SCOPE-MH program not having been designed as a culturally-tailored program, the flexibility of the model has enabled sites to adapt the delivery of the care to community needs, including sites that were developed with input and feedback from community members and staff to adapt the program as well as sites that have added interpretation services and translated materials to ensure that language does not hinder access to mental health care

Across interviews, participants emphasized the ability of the SCOPE-MH program to enhance access to care. This included lowering the threshold to access (e.g., low barrier referrals or self-referral), bringing specialist services to less connected communities (i.e., distribution of resources), and better engaging existing community supports in an asset-based model (e.g., building collaborations with community-based organizations and other stakeholders to leverage resources). Aspirationally, in one of the interviews, a future state where SCOPE was able to eliminate barriers related to geography was described:

“I think SCOPE can actually remove barriers that are territorial in some ways. Like, often, unless you live in a certain area, you cannot access a certain doctor or specialist. You know the story. But, if SCOPE had a centralized resource pool, those barriers to access could be removed. It would be even more seamless for the patient and the PCPs. Like, if a psychiatrist has capacity, why not send them someone on a waitlist? It is an equity issue that where you live impacts what services you get.”—PCP Lead