Skip to main content
Advertisement
  • Loading metrics

Virtual opioid poisoning education and naloxone distribution programs: A scoping review

  • Bruna dos Santos ,

    Contributed equally to this work with: Bruna dos Santos, Rifat Farzan Nipun

    Roles Data curation, Formal analysis, Investigation, Methodology, Visualization, Writing – original draft, Writing – review & editing

    bruna.santos@mail.utoronto.ca

    Affiliation Dalla Lana School of Public Health, University of Toronto, Toronto, Canada

  • Rifat Farzan Nipun ,

    Contributed equally to this work with: Bruna dos Santos, Rifat Farzan Nipun

    Roles Data curation, Formal analysis, Investigation, Methodology, Visualization, Writing – original draft

    Affiliation Dalla Lana School of Public Health, University of Toronto, Toronto, Canada

  • Anna Maria Subic,

    Roles Conceptualization, Methodology, Project administration, Supervision, Writing – review & editing

    Affiliation Dalla Lana School of Public Health, University of Toronto, Toronto, Canada

  • Alexandra Kubica,

    Roles Conceptualization, Methodology, Project administration, Supervision, Writing – review & editing

    Affiliation Dalla Lana School of Public Health, University of Toronto, Toronto, Canada

  • Nick Rondinelli,

    Roles Conceptualization

    Affiliations Dalla Lana School of Public Health, University of Toronto, Toronto, Canada, Heart to Heart First Aid CPR Services Inc., Toronto, Canada

  • Don Marentette,

    Roles Conceptualization, Funding acquisition

    Affiliation Canadian Red Cross Opioid Harm Reduction Advisory Council, Canada

  • Joanna Muise,

    Roles Conceptualization, Funding acquisition

    Affiliation The Canadian Red Cross Society, Canada

  • Kevin Paes,

    Roles Conceptualization, Funding acquisition

    Affiliation The Canadian Red Cross Society, Canada

  • Meghan Riley,

    Roles Conceptualization, Funding acquisition

    Affiliation The Canadian Red Cross Society, Canada

  • Samiya Bhuiya,

    Roles Conceptualization, Funding acquisition

    Affiliation The Canadian Red Cross Society, Canada

  • Jeannene Crosby,

    Roles Conceptualization, Funding acquisition

    Affiliation The Canadian Red Cross Society, Canada

  • Keely McBride,

    Roles Conceptualization, Funding acquisition

    Affiliations Canadian Red Cross Opioid Harm Reduction Advisory Council, Canada, The Canadian Red Cross Society, Canada

  • Joe Salter,

    Roles Conceptualization, Funding acquisition

    Affiliations Canadian Red Cross Opioid Harm Reduction Advisory Council, Canada, The Canadian Red Cross Society, Canada

  • Aaron M. Orkin

    Roles Conceptualization, Funding acquisition, Methodology, Supervision, Writing – review & editing

    Affiliations Dalla Lana School of Public Health, University of Toronto, Toronto, Canada, Department of Family and Community Medicine, University of Toronto, Toronto, Canada, Li Ka Shing Knowledge Institute of Unity Health Toronto, Toronto, Canada, Department of Emergency Medicine, Unity Health Toronto, Toronto, Canada

Abstract

The global opioid poisoning crisis is a complex issue with far-reaching public health implications. Opioid Poisoning Education and Naloxone Distribution (OPEND) programs aim to reduce stigma and promote harm reduction strategies, enhancing participants’ ability to apply life-saving interventions, including naloxone administration and cardiopulmonary resuscitation (CPR) to opioid poisoning. While virtual OPEND programs have shown promise in improving knowledge about opioid poisoning response, their implementation and evaluation have been limited. The COVID-19 pandemic has sparked renewed interest in virtual health services, including OPEND programs. Our study reviews the literature on fully virtual OPEND programs worldwide. We analyzed 7,722 articles, 30 of which met our inclusion criteria. We extracted and synthesized information about the interventions’ type, content, duration, the scales used, and key findings. Our search shows a diversity of interventions being implemented, with different study designs, duration, outcomes, scales, and different time points for measurement, all of which hinder a meaningful analysis of interventions’ effectiveness. Despite this, virtual OPEND programs appear effective in increasing knowledge, confidence, and preparedness to respond to opioid poisoning while improving stigma regarding people who use opioids. This effect appears to be true in a wide variety of populations but is significantly relevant when focused on laypersons. Despite increasing efforts, access remains an issue, with most interventions addressing White people in urban areas. Our findings offer valuable insights for the design, implementation, and evaluation of future virtual OPEND programs.

Author summary

The opioid crisis is a global health issue that affects people of all backgrounds, leading to high death rates. Educational programs on opioid poisoning are emerging as life-saving and cost-effective solutions. This review focuses on these programs delivered virtually, removing the need for face-to-face interaction. We’ve gathered and summarized evidence on the outcomes of these programs, which may include naloxone distribution. Our findings provide valuable insights for planning, implementing, and evaluating such programs. We also point out areas where knowledge is lacking, setting the stage for future research.

Introduction

The opioid poisoning crisis is a global health challenge with high fatality rates [13]. The highest rates of opioid poisoning fatalities are seen in the United States, followed by Estonia, Canada, and Lithuania [2]. Opioid-related fatalities have been exacerbated by the increasingly toxic and unregulated drug supply market, contaminated by fentanyl and other substances such as benzodiazepines [2,4,5]. Beyond fatal opioid poisonings, the majority of opioid poisonings are non-fatal and contribute to the social, health, and economic costs of the global opioid poisoning epidemic [68]. Opioid-related harms can affect people in all communities, ages, and socioeconomic groups, including family members, friends, healthcare professionals, and community members of people with lived or living experience (PWLLE) of opioid use [9,10].

Opioid Poisoning Education and Naloxone Distribution (OPEND) programs are strategies to train and equip people to provide life-saving interventions, including naloxone administration, to potential opioid poisoning events [1114]. While existing evidence demonstrates that OPEND can save lives, improve knowledge about opioid poisoning response, improve attitudes, and reduce stigma toward people who use substances, a significant research gap exists [1116]. Specifically, OPEND programs have predominantly been focused on in-person interventions, which may inequitably inhibit access for those who are unable or unwilling to access in-person services, including rural and remote populations [13,14,17,18]. The COVID-19 pandemic prompted a shift away from in-person offerings, increasing interest in the implementation and evaluation of virtual OPEND programming [19,20]. However, there are no current knowledge syntheses on virtual OPEND programs.

Therefore, this scoping review reports on the existing literature on opioid poisoning education conducted entirely without in-person interaction. In compiling a cohesive overview of existing virtual OPEND programs and their outcomes reported in the literature, our goal is to support the development and evaluation of future programs.

Methods

Objective and review question

We synthesized the literature on virtual OPEND programming. We structured our scoping review question based on the Participants, Concept, Context framework for scoping reviews [21], as follows:

  • Participants: People at risk of opioid poisoning, likely to witness opioid poisoning, or otherwise interested in OPEND program participation.
  • Concept: Any opioid poisoning education program conducted entirely at a distance, without in-person interaction between participants and staff.
  • Context: Worldwide.

Language and terminology

Although ‘overdose’ is a more common term, we use the term ‘poisoning’ throughout this paper. The word ‘overdose’ suggests that the primary cause of the health emergency is an incorrect dose, which most of the time is not necessarily the case and can place stigmatizing blame on people who experience opioid poisoning. Therefore, we suggest using the term ‘poisoning’, which can shift the attention to the broader social context leading to these emergencies, such as the unregulated drug supply. We also prefer and refer here to the term ‘opioid poisoning education and naloxone distribution’ (OPEND) over ‘take-home naloxone’ because the latter does not fully acknowledge the circumstances of the many unhoused people and the educational and non-pharmacological elements that are essential to these programs.

Approach and protocol

The design and methods for this review are reported following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR) [22]. Further guidance was gained from an adapted version of Arksey and O’Malley’s methodological framework on scoping reviews and the Joanna Briggs Institute methodological guidance for conducting scoping reviews [23,24]. A preliminary search of MEDLINE and the Cochrane Database of Systematic Reviews was conducted, and no current or ongoing literature reviews on the topic were identified. We developed a scoping review protocol and registered it on Open Science Framework [https://doi.org/10.17605/OSF.IO/EW9RQ] [25].

Search strategy

An initial search was undertaken on MEDLINE to identify papers on the topic using terms such as ‘virtual’, ‘education’, ‘opioids’, and related terminology [21]. The text words contained in the titles and abstracts of relevant papers and index terms were used to develop a full search strategy for OVID MEDLINE, OVID Embase, EBSCO CINAHL, OVID PsycINFO, The Cochrane Library, SCOPUS, and ERIC (S2 Table). A University of Toronto health science librarian provided expertise in optimizing the search strategy. The search strategy, including all identified keywords and index terms, was adapted for each included database. Searches were conducted on June 1, 2023, except for the grey literature search, which was conducted on July 27, 2023. The grey literature search included the Canadian Agency for Drugs and Technologies in Health (CADTH) Grey Matters website, the System for Information on Grey Literature in Europe (OpenGrey), and TRIP Pro. Additionally, we screened the references of included papers to identify additional ones (reference tracking), but no other papers were identified. All search strategies are documented in S2 Table.

Source of evidence selection

Following deduplication in Covidence, titles and abstracts were screened independently by four reviewers (BDS, RFN, AK, and AMS) based on the inclusion criteria. The full-text review of selected studies was undertaken by BDS and RFN. Reasons for exclusion during the full-text review were recorded following PRISMA guidelines (Fig 1) [22]. Disagreements between the reviewers at any stage of the selection process were resolved through discussion until a consensus was reached.

thumbnail
Fig 1. PRISMA flowchart.

This figure depicts papers that were included in the scoping review at each stage of the review process, according to PRISMA guidelines.

https://doi.org/10.1371/journal.pdig.0000412.g001

Data extraction

Data were extracted in Covidence from included articles by two independent reviewers (BDS and RFN) using a data extraction tool modified from the JBI Manual for Evidence Synthesis [26]. The data extraction tool was first piloted with five papers per reviewer and modified as necessary. The data extracted included publication details (year of publication, country, and type of study), information about the target population (demographics such as age, gender, and race/ethnicity), intervention components (e.g. videos, telephone calls), topics addressed by the intervention, key findings, and other details relevant to the review question. All variables extracted are provided (S3 Table). Disagreements between the reviewers were resolved through discussion until a consensus was reached.

Data synthesis

The extracted data are synthesized and summarized in narrative and tabular format. In conducting our analysis, we categorized training into major topic areas, including prevention, response, laws and law enforcement, and developed subtopics to identify the most reported topics addressed in opioid poisoning and naloxone administration training. This analysis was conducted by examining the content of training materials reported in the relevant publication, its citations, or when possible, through a direct review of original educational resources.

Results

After deduplication, 7,722 out of the initial 10,003 papers were identified for further review. After title and abstract screening, 7,631 papers were excluded, and 91 were reviewed in full text (Fig 1). A full-text review elicited 31 papers for inclusion within this scoping review. Sisson et al., 2023 conducted a single study that was reported in two papers and was, therefore, reported here as a single study [27,28].

The characteristics of the included papers, such as type of study, number and type of participants, sex or gender, and age, are specified in Table 1. Of the papers included in our analysis, 29 (97%) were conducted in the United States of America and one in Australia [29].

Seventeen studies have a quasi-experimental pre-post design (57%); ten are descriptive (33%); two are randomized controlled trials (7%); and one is a non-randomized controlled trial (3%). The publication period extended from 2016 to 2023, peaking notably in 2022 (27%, n = 8).

Participant demographics

Eight of the included studies (27%) were designed for people with lived or living experience (PWLLE) of opioid use; six (20%) were for anyone interested, including laypersons; and other six (20%) focused on medical, nursing, and pharmacy students. The remaining studies involved health professionals (13%, n = 4), first responders (7%, n = 2), family members or friends of PWLLE (3%, n = 1), school workers (3%, n = 1), and law enforcement officers (3%, n = 1).

Fourteen studies (47%) reported mean participant ages. Participant ages ranged from 11 to 69 years, with the majority falling between 25 and 64 years. There was notable variation in participation based on biological sex, with thirteen studies of 21 (62%) primarily including female participants and nine (43%) male participants. Most studies (70%, n = 21) reported biological sex rather than gender, with 10 of these presenting only male and female options. Four studies (13%) allowed participants to report their gender, while six (20%) did not provide sex or gender information.

Data on race or ethnicity were not reported in thirteen (43%) of the studies. Among the studies reporting race/ethnicity, all of them had White or Caucasian participants as the majority [27,30,31,35,3740,42,45,4749,5558]. Specific racial identities were reported in a few studies, with Beiting et al. (2022) reporting only the percentage of Black individuals, while Yates et al. (2018) and Moses et al. (2021) reported only the percentage of White individuals.

No papers provided information on participants’ socioeconomic status. However, Huhn et al., 2018 reported that the median household income of participants was US$52,500, while Sisson 2023a reported between US$10,000 and US$49,000. Six studies (20%) provided data on participants’ educational level, with 3 having most participants with a high school degree [27,32,58], 2 with the majority having a college degree [38,54], and two having most with at least a graduate degree [36,42]. The geographical area in which participants resided was mentioned in six studies (20%). In four studies, most participants were from urban areas [33,39,40,42], while two studies specifically focused on rural regions [34,54].

Intervention approaches

Several strategies were employed for conducting virtual OPEND. A summary of intervention approaches is specified in Table 1, including the type of approach, duration, scales used to measure outcomes and key findings. A substantial number of the included studies (87%, n = 26) delivered their intervention over the internet, including videos (n = 9, with 2 virtual reality simulations), videoconferences (n = 6), courses (n = 5), slide presentations (n = 3), and websites (n = 3). Of the internet-based interventions, four additionally provided naloxone by mail. There were four telephone-based interventions (13%) coupled with mail-delivered naloxone.

Among the 22 studies that reported on duration, interventions ranged from 5 minutes to 2 hours, with most lasting approximately 20 minutes (n = 5, 17%). In four studies (13%), participants could complete the intervention in their own time [29,34,34,36,50]. Of these self-directed interventions [29,36,50], one reported the average time taken for participants to complete their training [36], while none reported drop-out rates.

The 30 studies analyzed in this review incorporated a variety of topics into their interventions. The predominant theme was how to respond to, recognize, and prevent opioid poisoning emergencies. Some commonly mentioned steps were naloxone administration and calling 911. Additional topics included history and statistics on opioid poisoning, laws and law enforcement, specifics on naloxone use, and stigma. An overview of the educational components can be seen in Table 2.

Most studies utilized a pre-post design or post-training questionnaires. 67% (n = 20) assessed participants’ knowledge of opioid poisoning response, followed by 11 (37%) that gathered qualitative feedback and measured satisfaction rates. Ten studies (33%) focused on attitudes, behaviour, and stigma related to opioid poisoning emergencies, and another 10 tracked naloxone kit distribution, prescriptions, and administration during or after the intervention. Other outcomes included confidence in responding to opioid poisoning, comparisons between different modalities, accessibility, barriers to access naloxone, preparedness, familiarity with virtual learning, and opioid use.

Regarding assessment tools, 33% (n = 10) of studies used the Opioid Overdose Knowledge Scale (OOKS) and the Opioid Overdose Attitudes Scale (OOAS). Most studies (n = 7) adapted the scales for their intervention, while 4 (13%) developed new assessment tools. The Brief Opioid Overdose Knowledge Questionnaire (BOOK) and the Medical Conditions Regard Scale (MCRS) were each used by two studies (7%). Other scales used were the New General Self-Efficacy Scale and the Naloxone-Related Risk Compensation Beliefs Scale.

Key findings

Findings from the included studies are summarized in Table 1.

Videos (n = 9).

Knowledge and confidence. Most studies reported an increase in knowledge about opioids, opioid use disorder (OUD), and opioid poisoning, as well as an increase in confidence in managing opioid-related situations [27,30,35,41,54,58]. While Adams et al. (2020) showed that the knowledge increase was sustained for most participants at the 6-month follow-up, in Cerles et al. (2021) this increase was not sustained at the 3-month follow-up.

Attitudes and stigma. There was an improvement in attitudes and a decrease in stigma around OUD and opioid poisoning in several studies [30,38,40,41]. Giordano et al. (2020), however, reported a significant decrease in attitude scores.

Training satisfaction. Participants generally found the training easy and interesting, and were highly satisfied with the content and methods, giving the videos a high satisfaction rating [27,27,30,35,54].

Previous experience or training. Many participants had personal experience with substance use or had proximity to opioid poisoning [37,38]. Most were not previously trained in naloxone administration or had personal experience administering or carrying naloxone [3841].

Videoconferences (n = 7).

Knowledge and confidence. Most studies reported an increase in knowledge about the opioid epidemic, opioid misuse, opioid poisoning risks, and naloxone [31,40,48,49]. There was also an increase in confidence in recognizing poisoning and administering naloxone [31,42]. However, Hohmann et al. (2022) found no difference in knowledge about naloxone and perceived barriers to its implementation.

Attitudes. Some studies reported improved attitudes toward naloxone use and distribution [40,49]. However, Hohmann et al. (2022) found no difference in attitudes toward naloxone, illicit opioid use, or prescribed opioid misuse.

Training and satisfaction. Two papers reported that most participants are comfortable with a virtual format but prefer in-person training [31,49].

Reach and diversity. Mathias et al. (2023) reported reaching both urban and rural populations, and substantial racial/ethnic diversity. Notably, Jensen et al. (2019) found that fewer people with substance use disorder were reached in the videoconference compared to the in-person group.

Courses (n = 5).

Knowledge, confidence and preparedness. All studies reported an increase in knowledge [33,36,46,52,53]. Additionally, Berland et al. (2019) reported increased preparedness, while Simmons et al. (2016) and (2018) reported increased confidence to intervene in opioid poisoning.

Attitudes and behaviour. Eukel et al. (2020) reported improved behaviours at the 12-month follow-up. However, Berland et al. (2019) reported no significant changes in attitude scores.

Training and satisfaction. Four studies reported high participant satisfaction rates with the courses [33,36,52,53].

Telephone calls (n = 4).

Reach and acceptance. All studies reported that a significant portion of potential participants were reached [43,51,55,57]. Most patients were reached within one, two or three attempts [51,55,57].

Slide presentations (n = 3).

Knowledge and confidence. Two studies reported increased knowledge and one increased confidence about opioid use and poisoning after the intervention [32,34,44]. Bergeria et al. (2019) noted a small decrease in knowledge at the 30-day follow-up.

Attitudes and stigma. One study found that the training was effective in decreasing stigma around prescribing naloxone [34]. Another reported fewer opioid poisoning-risk behaviours after the intervention [32].

Training and satisfaction. All studies reported high levels of satisfaction or acceptability for their interventions [32,34,44].

Websites (n = 3).

Knowledge, confidence and awareness. One study reported increased knowledge and confidence in discussing opioid poisoning [29], and another showed that most participants, with either a professional or personal interest in the training, correctly identified basic signs of opioid poisoning and appropriate responses [50].

Naloxone.

Of the interventions that offered naloxone by mail, four were telephone-based [43,51,55,57], two were videos [27,28,37], one was a videoconference [31], and another was a website [56]. Some studies, though not distributing naloxone, reported on naloxone-related outcomes.

Acceptability. Acceptability rates for naloxone kits distribution ranged from 73% to 93% [43,51,55,57]. Participants with a personal interest in the training were more likely to report being very likely to obtain naloxone [50]. The most common reason participants did not accept naloxone was because they felt it was unnecessary [51]. Castillo et al. (2022) reported an increase in nurses’ motivation and intent to prescribe naloxone to patients taking long-term opioids. Similarly, Hohmann et al. (2022) found that 66% of participants felt comfortable providing naloxone to people who use illicit opioids compared to 91% who felt comfortable providing it to people with prescribed opioids.

Barriers. The most frequently reported barriers to obtaining naloxone were related to the COVID-19 pandemic, lack of knowledge on how to access it, inability to get a prescription, cost, lack of availability, transportation, and stigma [37,53]. Despite perceived barriers not impacting the willingness to prescribe naloxone, the highest perceived barrier for nurses was time constraints [34].

Naloxone use and outcomes. Most or all opioid poisoning reversals resulted in survival, and the majority of kits were used on participants’ friends or family members [28,52,56]. Giordano et al. (2020) reported that, though they did not distribute naloxone kits, 12% of participants reported carrying naloxone in a 3-week follow-up, and one administered it. Sisson et al. (2023b) reported that 42% of individuals in the groups without naloxone distribution obtained their kit.

Comparison between different modalities.

Comparison groups. Four studies had a control group involving in-person training [33,41,45,49], and one study had a hybrid group [39]. Other comparison groups included reading-only or course, slide or slide + quiz, and virtual reality video or videoconference.

Differences between formats. Three studies with an in-person or hybrid group showed no significant differences in preparedness, attitudes, satisfaction, and stigma scores between formats [33,39,41], and one reported greater knowledge in the in-person group [49]. One study reported a significantly higher number of naloxone prescriptions obtained by participants in the virtual group [45]. For papers with other comparison groups, there were also no significant differences between groups [32,40,46]. Huhn et al. (2018) reported that the presentation-only group was more likely to complete the intervention compared to the group with the presentation coupled with a quiz [44]. Sisson et al. (2023a) reported higher satisfaction in the naloxone distribution group compared to the group without naloxone distribution [58]. Hohmann et al. (2022) showed an increase in the number of naloxone prescriptions obtained by participants only in the intervention group, but there was no difference related to other outcomes [42].

Limitations.

The included studies reported limitations in their designs. Twenty studies (67%) reported a lack of generalizability of their study results [27,28,3035,3942,4753,55,57]. Another frequent obstacle reported in 12 studies is the small sample size (40%), which ranged from six to 313 participants per group [3032,34,35,39,40,42,45,48,51,54]. Other limitations include the absence of longitudinal data [31,37,42,44,48,50,55], self-reported bias for questionnaires [27,28,49,52,53], the lack of comparison to an in-person intervention [38,41,44,47], no follow-up on the number of participants who responded to an opioid poisoning post-intervention [37,45,47,55], lack of validity of the tools used [30,34,36], low response rates of 21.2% and 33.7% [36,53], and the lack of randomization [46,47].

Discussion

This scoping review identified that there is considerable research and innovation underway in the implementation and evaluation of virtual OPEND, particularly since the start of the COVID-19 pandemic. Although this review did not include a critical appraisal, it is crucial to note that the diverse study designs, varying intervention durations, and different measurement time points complicate the analysis of intervention effectiveness.

Despite current challenges in analyzing intervention effectiveness, it is important to highlight the positive outcomes reported. Overall, virtual OPEND appears to be acceptable to participants and effective in increasing knowledge on how to respond to opioid poisoning, as well as improving attitudes and decreasing stigma about people who use opioids. Especially when coupled with naloxone distribution, programs appear to impact the motivation and willingness of people to respond to these emergencies. Notably, some studies demonstrated a high rate of successful resuscitations and decreased poisoning death rates [27,28,52,56]. Therefore, while the diverse study designs present analytical challenges, they do not detract from the potential life-saving impact of virtual OPEND.

Our synthesis indicates that most interventions are primarily targeted at PWLLE of opioid use and potential bystanders, defined as individuals who could potentially witness opioid poisoning. A cross-sectional internet-based survey and a mixed-methods study have demonstrated the unique position of bystanders in reversing opioid poisonings and saving lives when they are equipped with naloxone and the knowledge to administer it effectively [59,60].

The bulk of these studies are based in the USA, with only one study conducted in Australia [29]. However, opioid poisoning is a global concern, which demonstrates a pressing need to implement and evaluate virtual OPEND programs worldwide [2,13]. Most studies had a pre-post design, followed by descriptive studies, RCTs, and one non-randomized controlled trial, and did not include longitudinal data. Therefore, there is a need for more randomized and controlled trials that also assess the attrition of knowledge and the potential need or importance of retraining so more effective interventions can be designed and implemented in the future.

In addition, only 42% of studies utilized validated scales to measure outcomes related to opioid poisoning, with 62% of these studies using a modified version of the validated scales. These validated scales, especially OOKS and OOAS, were originally designed to assess illicit opioid poisoning risk factors and do not include knowledge specifically about naloxone [61,62]. Therefore, there is a critical demand for updated and validated survey instruments to effectively evaluate these interventions [61,62].

Few studies reported sociodemographic variables and focused on the inclusion of historically marginalized populations. Yet, those more likely to be impacted by the opioid poisoning crisis include people living in rural and remote areas, individuals experiencing homelessness, those living in poverty, incarcerated individuals, and Black, Indigenous, and People of Colour [63]. Without sociodemographic information, it is unknown whether sampling and selection biases and issues with representativeness have occurred, limiting the effectiveness of these interventions in addressing health inequities [64].

Moreover, most interventions did not make their training materials publicly available, and some of them did not report the educational content included in the intervention, hindering the potential for public dissemination and research replication. Among the studies that reported the content of their training, stigma reduction and policies on witness protection and naloxone use permission for opioid poisoning reversal were frequently not addressed, which if included, could have bolstered the outcomes [65,66].

This is the first scoping review on fully virtual OPEND interventions, highlighting the outcomes and potential impacts of these programs. Other reviews have explored opioid poisoning interventions and their effectiveness but did not focus on virtual programming. Pellegrino et al., (2021) conducted a scoping review on first aid educational interventions for opioid poisoning published until 2019, showing the potential of these programs with most studies showing improved educational and clinical outcomes, despite the sparsity and lack of quality of the studies reviewed [14]. Razaghizad et al. (2021) conducted an umbrella review on OPEND, demonstrating that there is credible evidence that opioid poisoning education programs improve knowledge and attitudes about opioid poisoning, enable participants to use naloxone safely and effectively, and reduce opioid-related mortality [13]. Additionally, Tas et al. (2022) reviewed technological interventions that prevent, detect, or respond to opioid poisoning emergencies, with promising approaches that were still under development, such as smartphone apps for alerting and recruiting first aid volunteers to act on nearby opioid poisoning events [67].

Our findings suggest the potential effectiveness of virtual OPEND interventions as a lifesaving tool, and can assist researchers and public health practitioners in designing, implementing, and evaluating future OPEND programming. We recommend providing more funding, personnel, and technology to amplify the implementation and evaluation of these programs. We believe coupling first-aid opioid poisoning training with a well-structured naloxone distribution system can notably increase the positive outcomes of this intervention. To address inequities and specific local challenges, we recommend sociodemographic information to be collected and interventions to be tailored by working closely with local communities and governments. It is also crucial to invest in the development or update and validation of relevant evaluation tools that can better assess OPEND’s effectiveness for diverse outcomes and in diverse settings. Additionally, we suggest conducting longitudinal studies to address the long-term outcomes of these interventions, including the duration of knowledge and the necessity for retraining.

Limitations

Our review has limitations. One such limitation is the lack of extensive and evidence-informed critical appraisal of the studies included. For this reason, it is not possible to indicate the quality of the studies or extract a meaningful conclusion. In the context of diverse study designs, different methodologies, intervention durations, and measurement time points can introduce variability that makes it challenging to compare results across studies. The variability of the studies makes it unclear whether the results synthesized in our review can be extrapolated to other settings, study designs or populations. Despite our best efforts to design a comprehensive search strategy without imposing any limitations, the constant updates and rapid publication of new research on technology-based interventions may result in the unintentional omission of important studies by the time of publication. We also acknowledge that many impactful programs addressing the opioid poisoning crisis are currently available online but did not involve a research design or share publicly available data on outcomes of these interventions and, therefore were not suitable for this review. Additionally, the lack of detailed information about the interventions in many studies may have impacted the accuracy of our synthesis, as we did not reach out to the authors for additional information.

Conclusion

This review synthesizes the literature on virtual Opioid Poisoning Education and Naloxone Distribution (OPEND) programs, and underscores gaps in program implementation and evaluation. Currently, it is difficult to draw substantial conclusions on OPEND effectiveness due to the variety of study designs and the lack of updated and validated evaluation methods. However, existing evidence points to the potential to increase knowledge, decrease stigma, and ultimately reduce morbidity and mortality due to opioid-related harms, which enhances its potential as a life-saving tool. This life-saving knowledge can have substantial and positive impacts on populations who are not currently being adequately reached by in-person OPEND interventions.

Acknowledgments

We acknowledge the time and efforts of Eden Kinzel, Liaison and Education Librarian at the University of Toronto, who provided expertise in the development of the research question and search strategies.

References

  1. 1. United Nations: Office on Drugs and Crime. Online World Drug Report 2023—Latest data and trend analysis [Internet]. [cited 2023 Oct 23]. Available from: //www.unodc.org/unodc/en/data-and-analysis/wdr-2023-online-segment.html
  2. 2. Krausz RM, Westenberg JN, Ziafat K. The opioid overdose crisis as a global health challenge. Curr Opin Psychiatry [Internet]. 2021 Jul [cited 2023 Aug 8];34(4):405. Available from: https://journals.lww.com/co-psychiatry/Abstract/2021/07000/The_opioid_overdose_crisis_as_a_global_health.13.aspx. pmid:33901060
  3. 3. Fincham JE. Global use and misuse of opioids. Int J Pharm Pract [Internet]. 2018 Apr 1 [cited 2024 Mar 12];26(2):91–2. Available from: pmid:29575538
  4. 4. Special Advisory Committee on the Epidemic of Opioid Overdoses. Opioid- and Stimulant-related Harms in Canada [Internet]. Ottawa: Public Heatlh Agency of Canada; 2022 Jun [cited 2022 Sep 15]. Available from: https://health-infobase.canada.ca/substance-related-harms/opioids-stimulants/
  5. 5. Cristiano N Fentanyl Contamination as a Risk Priority: the Impact of the Fentanyl Epidemic on Club Drug-Using Behaviours. Subst Use Misuse. 2022 May 12;57(6):975–82. pmid:35354372
  6. 6. Centers for Disease Control and Prevention. Annual Surveillance Report Of Drug-related Risks And Outcomes. Centers for Disease Control and Prevention, U.S. Department of Health and Human Services; 2019.
  7. 7. Darke S, Mattick RP, Degenhardt L. The ratio of non-fatal to fatal heroin overdose. Addict Abingdon Engl. 2003 Aug;98(8):1169–71.
  8. 8. Hong JS, Moran MT, Eaton LA, Grafton LM. Neurologic, Cognitive, and Behavioral Consequences of Opioid Overdose: a Review. Curr Phys Med Rehabil Rep [Internet]. 2019 Dec 1 [cited 2024 Mar 12];7(4):305–13. Available from:
  9. 9. Public Health Agency of Canada. Highlights from phase one of the national study on opioid and illegal substance-related overdose deaths: insights from coroners and medical examiners [Internet]. 2018 [cited 2022 Sep 29]. Available from: https://www.canada.ca/en/public-health/services/publications/healthy-living/highlights-phase-one-national-study-opioid-illegal-substance-related-overdose-deaths.html.
  10. 10. Monnat SM. Demographic and Geographic Variation in Fatal Drug Overdoses in the United States, 1999–2020. Ann Am Acad Pol Soc Sci [Internet]. 2022 Sep 1 [cited 2024 Mar 12];703(1):50–78. Available from: pmid:37366474
  11. 11. Riazi F, Toribio W, Irani E, Hughes TM, Huxley-Reicher Z, McBratney E, et al. Community Case Study of Naloxone Distribution by Hospital-Based Harm Reduction Program for People Who Use Drugs in New York City. Front Sociol [Internet]. 2021 [cited 2022 Oct 2];6. Available from: https://www.frontiersin.org/articles/10.3389/fsoc.2021.619683 pmid:34307540
  12. 12. Walley AY, Xuan Z, Hackman HH, Quinn E, Doe-Simkins M, Sorensen-Alawad A, et al. Opioid overdose rates and implementation of overdose education and nasal naloxone distribution in Massachusetts: interrupted time series analysis. BMJ [Internet]. 2013 Jan 31 [cited 2022 Oct 2];346:f174. Available from: https://www.bmj.com/content/346/bmj.f174. pmid:23372174
  13. 13. Razaghizad A, Windle SB, Filion KB, Gore G, Kudrina I, Paraskevopoulos E, et al. The Effect of Overdose Education and Naloxone Distribution: An Umbrella Review of Systematic Reviews. Am J Public Health [Internet]. 2021 Aug [cited 2022 Dec 15];111(8):e1–12. Available from: https://ajph.aphapublications.org/doi/full/10.2105/AJPH.2021.306306. pmid:34214412
  14. 14. Pellegrino JL, Krob JL, Orkin A, Pellegrino JL, Krob JL, Orkin A. First Aid Education for Opioid Overdose Poisoning: Scoping Review. Cureus [Internet]. 2021 Jan 3 [cited 2023 Apr 26];13(1). Available from: https://www.cureus.com/articles/46368-first-aid-education-for-opioid-overdose-poisoning-scoping-review. pmid:33552772
  15. 15. Winograd RP, Werner KB, Green L, Phillips S, Armbruster J, Paul R. Concerns that an opioid antidote could “make things worse”: Profiles of risk compensation beliefs using the Naloxone-Related Risk Compensation Beliefs (NaRRC-B) scale. Subst Abuse [Internet]. 2020 Apr 2 [cited 2022 Oct 4];41(2):245–51. Available from: pmid:31170030
  16. 16. Irvine MA, Buxton JA, Otterstatter M, Balshaw R, Gustafson R, Tyndall M, et al. Distribution of take-home opioid antagonist kits during a synthetic opioid epidemic in British Columbia, Canada: a modelling study. Lancet Public Health [Internet]. 2018 May 1 [cited 2022 Oct 2];3(5):e218–25. Available from: https://www.thelancet.com/journals/lanpub/article/PIIS2468-2667(18)30044-6/fulltext. pmid:29678561
  17. 17. Bardwell G, Lappalainen L. The need to prioritize research, policy, and practice to address the overdose epidemic in smaller settings in Canada. Can J Public Health Rev Can Santé Publique [Internet]. 2021 Mar 29 [cited 2022 Sep 22];112(4):733–6. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8225779/ pmid:33782915
  18. 18. Pijl EM, Alraja A, Duff E, Cooke C, Dash S, Nayak N, et al. Barriers and facilitators to opioid agonist therapy in rural and remote communities in Canada: an integrative review. Subst Abuse Treat Prev Policy [Internet]. 2022 Aug 26 [cited 2022 Sep 22];17:62. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9412807/ pmid:36028837
  19. 19. Canadian Centre on Substance Use and Addiction. Impacts of the COVID-19 Pandemic on People Who Use Substances. Ottawa, ON, CA: Canadian Centre on Substance Use and Addiction; 2020.
  20. 20. Husak L, Sovran V, Ytsma A, Comeau M. Impact of the COVID-19 Pandemic on Virtual Care: A Major Shift for Physicians and Patients. Healthc Q Tor Ont. 2022 Oct;25(3):11–3.
  21. 21. 11.2.4 Inclusion criteria—JBI Manual for Evidence Synthesis—JBI Global Wiki [Internet]. [cited 2023 Oct 17]. Available from: https://jbi-global-wiki.refined.site/space/MANUAL/4687714/11.2.4+Inclusion+criteria.
  22. 22. Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med [Internet]. 2018 Oct 2 [cited 2023 May 2];169(7):467–73. Available from: https://www.acpjournals.org/doi/10.7326/M18-0850. pmid:30178033
  23. 23. JBI Manual for Evidence Synthesis. Scoping reviews [Internet]. [cited 2023 Jun 26]. Available from: https://jbi-global-wiki.refined.site/space/MANUAL/4687342/Chapter+11%3A+Scoping+reviews.
  24. 24. Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol [Internet]. 2005 Feb 1 [cited 2023 May 2];8(1):19–32. Available from:
  25. 25. Dos Santos B, Nipun R farzan, Subic AM, Kubica A, Rondinelli N, Marentette D, et al. Remote Opioid Poisoning Education and Naloxone Distribution Programs: A Scoping Review Protocol. 2023 Jun 14 [cited 2023 Jun 26]; Available from: https://osf.io/ew9rq
  26. 26. JBI Manual for Evidence Synthesis. Data extraction [Internet]. [cited 2023 Jun 28]. Available from: https://jbi-global-wiki.refined.site/space/MANUAL/4687700.
  27. 27. Sisson ML, Azuero A, Chichester KR, Carpenter MJ, Businelle MS, Shelton RC, et al. Feasibility and acceptability of online opioid overdose education and naloxone distribution: Study protocol and preliminary results from a randomized pilot clinical trial. Contemp Clin Trials Commun [Internet]. 2023 Jun 1 [cited 2023 Apr 26];33:101131. Available from: https://www.sciencedirect.com/science/article/pii/S2451865423000777. pmid:37091508
  28. 28. Sisson ML, Azuero A, Chichester KR, Carpenter MJ, Businelle MS, Shelton RC, et al. Preliminary effectiveness of online opioid overdose and naloxone administration training and impact of naloxone possession on opioid use. Drug Alcohol Depend. 2023 Aug 1;249:110815. pmid:37336007
  29. 29. Farrugia A, Treloar C, Fraser S. Overdoselifesavers.org: A mixed-method evaluation of a public information website on experiences of overdose and using take-home naloxone to save lives. Drugs Educ Prev Policy. 2022;29(1):43–53.
  30. 30. Adams N. Beyond narcan: comprehensive opioid training for law enforcement. J Subst Use. 2021 Aug;26(4):383–90.
  31. 31. Beiting KJ, Molony J, Ari M, Thompson K. Targeted virtual opioid overdose education and naloxone distribution in overdose hotspots for older adults during COVID-19. J Am Geriatr Soc [Internet]. 2022 Sep 28 [cited 2022 Oct 27];jgs.18037. Available from: https://onlinelibrary.wiley.com/doi/10.1111/jgs.18037. pmid:36169217
  32. 32. Bergeria CL, Huhn AS, Dunn KE. Randomized comparison of two web-based interventions on immediate and 30-day opioid overdose knowledge in three unique risk groups. Prev Med. 2019;128(pm4, 0322116):105718. pmid:31078564
  33. 33. Berland N, Lugassy D, Fox A, Goldfeld KK, Oh SY, Tofighi B, et al. Use of Online Opioid Overdose Prevention Training for First Year Medical Students: A Comparative Analysis of Online vs In-Person training. Subst Abuse [Internet]. 2019 [cited 2022 Nov 14];40(2):240–6. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7257831/
  34. 34. Castillo E. Engaging healthcare providers to coprescribe naloxone to prevent opioid-induced mortality in Arizona. Diss Abstr Int Sect B Sci Eng. 2022;83(7-B):No-Specified.
  35. 35. Cerles AA, Dinh NNL, MacMillan L, Kemp DC, Rush MA. Development of Novel Video-Based First Responder Opioid Hazard Refresher Training. New Solut J Environ Occup Health Policy NS. 2021;31(3):298–306. pmid:34382476
  36. 36. Eukel H, Steig J, Frenzel O, Skoy E, Werremeyer A, Strand M. Opioid Misuse and Overdose: Changes in Pharmacist Practices and Outcomes. J Contin Educ Health Prof. 2020;40(4):242–7. pmid:33284175
  37. 37. French R, Favaro J, Aronowitz SV. A free mailed naloxone program in Philadelphia amidst the COVID-19 pandemic. Int J Drug Policy. 2021 Aug;94:103199. pmid:33765515
  38. 38. Galiher MV, Huffman M. Attitude changes following short-form opioid overdose video education: a pilot study. Harm Reduct J [Internet]. 2022 Oct 14 [cited 2022 Nov 21];19(1):114. Available from: pmid:36241995
  39. 39. Giordano NA, Whitney CE, Axson SA, Cassidy K, Rosado E, Hoyt-Brennan AM. A pilot study to compare virtual reality to hybrid simulation for opioid-related overdose and naloxone training. Nurse Educ Today. 2020 Feb 10;88:104365. pmid:32088524
  40. 40. Giordano NA, Febres-Cordero S, Baker H, Pfeiffer KM, Walsh LM, Gish A, et al. Opioid-involved overdose trainings delivered using remote learning modalities. Nurs Open. 2023;10(6):4132–6. pmid:36681653
  41. 41. Goss NC, Haslund-Gourley B, Meredith DM, Friedman AV, Kumar VK, Samson KR, et al. A Comparative Analysis of Online Versus in-Person Opioid Overdose Awareness and Reversal Training for First-Year Medical Students. Subst Use Misuse. 2021;56(13):1962–71. pmid:34355637
  42. 42. Hohmann L, Phillippe H, Marlowe K, Jeminiwa R, Hohmann N, Westrick S, et al. A state-wide education program on opioid use disorder: influential community members’ knowledge, beliefs, and opportunities for coalition development. BMC Public Health. 2022;22(1):886. pmid:35508988
  43. 43. Hughes TM, Kalicki A, Huxley-Reicher Z, Toribio W, Samuels DL, Weiss JJ, et al. A medical student-led model for telephone-based opioid overdose education and naloxone distribution during the COVID-19 pandemic. Subst Abuse. 2022;43(1):988–92. pmid:35435812
  44. 44. Huhn AS, Garcia-Romeu AP, Dunn KE. Opioid Overdose Education for Individuals Prescribed Opioids for Pain Management: Randomized Comparison of Two Computer-Based Interventions. Front Psychiatry [Internet]. 2018 [cited 2022 Nov 17];9. Available from: https://www.frontiersin.org/articles/10.3389/fpsyt.2018.00034. pmid:29483884
  45. 45. Jensen AN, Beam CM, Douglass AR, Brabson JE, Colvard M, Bean J. Description of a pharmacist-led clinical video telehealth group clinic for opioid overdose prevention and naloxone education. Ment Health Clin. 2019;9(4):294–7. pmid:31293850
  46. 46. Kim H, Heverling H, Cordeiro M, Vasquez V, Stolbach A. Internet Training Resulted in Improved Trainee Performance in a Simulated Opioid-Poisoned Patient as Measured by Checklist. J Med Toxicol Off J Am Coll Med Toxicol. 2016 Sep;12(3):289–94.
  47. 47. Mathias CW, Cavazos DM, McGlothen-Bell K, Crawford AD, Flowers-Joseph B, Wang Z, et al. Opioid overdose prevention education in Texas during the COVID-19 pandemic. Harm Reduct J. 2023 Mar 24;20(1):37. pmid:36964600
  48. 48. Millikan MD. A quality improvement project: Implementing the this is not about drugs program in southwest Kansas. Diss Abstr Int Sect B Sci Eng. 2021;82(8-B):No-Specified.
  49. 49. Moses TEH, Moreno JL, Greenwald MK, Waineo E. Training medical students in opioid overdose prevention and response: Comparison of In-Person versus online formats. Med Educ Online [Internet]. 2021 Jan 1 [cited 2022 Nov 21];26(1):1994906. Available from: https://www.tandfonline.com/doi/full/10.1080/10872981.2021.1994906. pmid:34727840
  50. 50. Roe SS, Banta-Green CJ. An Initial Evaluation of Web-Based Opioid Overdose Education. Subst Use Misuse [Internet]. 2016 Jan 28 [cited 2022 Nov 23];51(2):268–75. Available from: pmid:26709867
  51. 51. Rothbauer K, Genisot A, Frey T, JohnsonPharmacy William S Middleton Memorial Veterans Hospital D. Integration of Pharmacy Student Interns into a Naloxone Telephone Outreach Service. J Pain Palliat Care Pharmacother. 2022 Dec;36(4):208–15.
  52. 52. Simmons J, Rajan S, Goldsamt L, Elliott L. Implementation of online opioid overdose prevention, recognition and response trainings for professional first responders: Year 1 survey results. Drug Alcohol Depend [Internet]. 2016 Dec 1 [cited 2022 Nov 17];169:1–4. Available from: https://www.sciencedirect.com/science/article/pii/S0376871616309358. pmid:27744100
  53. 53. Simmons J, Rajan S, Goldsamt LA, Elliott L. Implementation of Online Opioid Prevention, Recognition and Response Trainings for Laypeople: Year 1 Survey Results. Subst Use Misuse [Internet]. 2018 Oct 15 [cited 2022 Dec 8];53(12):1997–2002. Available from: https://www.tandfonline.com/doi/full/10.1080/10826084.2018.1451891. pmid:29641944
  54. 54. Waldron AC. Opioid and naloxone training in a rural school district. Diss Abstr Int Sect B Sci Eng. 2022;83(4-B):No-Specified.
  55. 55. Wu S, Frey T, Wenthur CJ. Naloxone acceptance by outpatient veterans: A risk-prioritized telephone outreach event. Res Soc Adm Pharm RSAP. 2021;17(5):1017–20. pmid:32980236
  56. 56. Yang C, Favaro J, Meacham MC. NEXT Harm Reduction: An Online, Mail-Based Naloxone Distribution and Harm-Reduction Program. Am J Public Health [Internet]. 2021 Apr [cited 2023 Apr 26];111(4):667–71. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7958031/ pmid:33600254
  57. 57. Yates D, Frey T, Montgomery JC. Utilizing risk index for overdose or serious opioid-induced respiratory depression (RIOSORD) scores to prioritize offer of rescue naloxone in an outpatient veteran population: A telephone-based project. Subst Abuse. 2018;39(2):182–4. pmid:29578839
  58. 58. Sisson ML. Increasing naloxone access for persons who use opioids: An online recruitment and training approach to opioid overdose education and naloxone distribution. Diss Abstr Int Sect B Sci Eng. 2023;84(4-B):No-Specified.
  59. 59. Abdelal R, Raja Banerjee A, Carlberg-Racich S, Darwaza N, Ito D, Shoaff J, et al. Real-world study of multiple naloxone administration for opioid overdose reversal among bystanders. Harm Reduct J. 2022;19(1):49. pmid:35596213
  60. 60. Keane C, Egan JE, Hawk M. Effects of naloxone distribution to likely bystanders: Results of an agent-based model. Int J Drug Policy. 2018;55(9014759):61–9. pmid:29524734
  61. 61. Shoup JA, Mueller SR, Binswanger IA, Williams AV, Strang J, Glanz JM. Modifying and Evaluating the Opioid Overdose Knowledge Scale for Prescription Opioids: A Pilot Study of the Rx-OOKS. Pain Med Off J Am Acad Pain Med [Internet]. 2020 Aug 22 [cited 2023 Oct 20];21(10):2244–52. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7593800/ pmid:32827044
  62. 62. Franklin Edwards G, Mierisch C, Mutcheson B, Horn K, Henrickson Parker S. A review of performance assessment tools for rescuer response in opioid overdose simulations and training programs. Prev Med Rep [Internet]. 2020 Oct 21 [cited 2023 Aug 10];20:101232. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7610043/ pmid:33163333
  63. 63. Friesen EL, Kurdyak PA, Gomes T, Kolla G, Leece P, Zhu L, et al. The Impact of the COVID-19 Pandemic on Opioid-Related Harm in Ontario [Internet]. Ontario COVID-19 Science Advisory Table; 2021 Sep [cited 2022 Sep 15]. Available from: https://covid19-sciencetable.ca/sciencebrief/the-impact-of-the-covid-19-pandemic-on-opioid-related-harm-in-ontario.
  64. 64. Agency for Healthcare Research and Quality. Race, Ethnicity, and Language Data: Standardization for Health Care Quality Improvement [Internet]. 2018 [cited 2023 Aug 10]. Available from: https://www.ahrq.gov/research/findings/final-reports/iomracereport/reldata1.html.
  65. 65. Jakubowski A, Kunins HV, Huxley-Reicher Z, Siegler A. Knowledge of the 911 Good Samaritan Law and 911-calling behavior of overdose witnesses. LawAtlas GrantKJ , Banta Green CJ, Kuszler PC, Coffin PO, Schoeppe JA, Cerda M, Ransome Y, Keyes KM, et al, Darke S, Ross J, Zador D, Sunjic S, Davidson PJ, McLean RL, KralA H, Gleghorn AA, Edlin BR, Moss AR, Hernan MA, Hernan A, Jones JD, Campbell A, Metz VE, Comer SD, Lankenau SE, Wagner KD, Silva K, et al, Levy B, Spelke B, PaulozziL J, et al, Pollini RA, McCall L, Mehta SH, Celentano DD, Vlahov D, Strathdee SA, Rudd RA, Aleshire N, Zibbell JE, Gladden RM, Sherman SG, Gann DS, Scott G, Carlberg S, Bigg D, Heimer R, Siegler A, Huxley Reicher Maldjian L, Jordan R, Oliver C, Jakubowski A, Kunins HV, Siegler A, Tuazon E, Bradley O’Brien D, Paone D, TobinK E, Davey MA, Latkin CA, Tracy M, Piper TM, Ompad D, et al, editor. Subst Abuse. 2018;39(2):233–8. pmid:28972445
  66. 66. Sellen K, Markowitz B, Parsons JA, Leece P, Handford C, Goso N, et al. Considerations for the design of overdose education and naloxone distribution interventions: results of a multi-stakeholder workshop. BMC Public Health [Internet]. 2023 May 15 [cited 2023 May 24];23(1):888. Available from: https://bmcpublichealth.biomedcentral.com/articles/10.1186/s12889-023-15554-6. pmid:37189052
  67. 67. Tas B, Lawn W, Traykova EV, Evans RAS, Murvai B, Walker H, et al. A scoping review of mHealth technologies for opioid overdose prevention, detection, and response. [Internet]. PsyArXiv; 2022 [cited 2023 May 24]. Available from: https://psyarxiv.com/trfbc/