Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

A digital health game to prevent opioid misuse and promote mental health in adolescents in school-based health settings: Protocol for the PlaySmart game randomized controlled trial

  • Tyra M. Pendergrass Boomer ,

    Contributed equally to this work with: Tyra M. Pendergrass Boomer, Lily A. Hoerner

    Roles Conceptualization, Data curation, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

    Affiliation play2PREVENT Lab at the Yale Center for Health & Learning Games, Department of Internal Medicine, Yale School of Medicine, New Haven, CT, United States of America

  • Lily A. Hoerner ,

    Contributed equally to this work with: Tyra M. Pendergrass Boomer, Lily A. Hoerner

    Roles Data curation, Project administration, Writing – original draft, Writing – review & editing

    lily.hoerner@yale.edu

    Affiliation play2PREVENT Lab at the Yale Center for Health & Learning Games, Department of Internal Medicine, Yale School of Medicine, New Haven, CT, United States of America

  • Claudia-Santi F. Fernandes,

    Roles Conceptualization, Funding acquisition, Writing – original draft, Writing – review & editing

    Affiliations play2PREVENT Lab at the Yale Center for Health & Learning Games, Department of Internal Medicine, Yale School of Medicine, New Haven, CT, United States of America, Yale Child Study Center, New Haven, CT, United States of America, Department of Biomedical Informatics and Data Science, Yale School of Medicine, New Haven, CT, United States of America

  • Amber Maslar ,

    Roles Methodology, Software

    ‡ AM, SA and TCK also contributed equally to this work.

    Affiliation Memorial Sloan Kettering Cancer Center, New York, NY, United States of America

  • Sherry Aiudi ,

    Roles Methodology, Software

    ‡ AM, SA and TCK also contributed equally to this work.

    Affiliation Yale Center for Analytical Sciences, Yale School of Public Health, New Haven, CT, United States of America

  • Tassos C. Kyriakides ,

    Roles Methodology

    ‡ AM, SA and TCK also contributed equally to this work.

    Affiliation Yale Center for Analytical Sciences, Yale School of Public Health, New Haven, CT, United States of America

  • Lynn E. Fiellin

    Roles Conceptualization, Data curation, Funding acquisition, Investigation, Methodology, Supervision, Writing – original draft, Writing – review & editing

    Affiliations play2PREVENT Lab at the Yale Center for Health & Learning Games, Department of Internal Medicine, Yale School of Medicine, New Haven, CT, United States of America, Yale Child Study Center, New Haven, CT, United States of America, Yale School of Public Health, New Haven, CT, United States of America

Abstract

Adolescents who engage in non-opioid substance misuse and/or experience mental health symptoms are at greater risk of misusing opioids and/or developing opioid use disorder. Adolescence is a critical developmental period to both prevent the initiation of opioid misuse and target mental health. To date, there are no digital health games targeting both conditions. We describe the protocol for a randomized controlled trial designed to assess the efficacy of an original digital health game, PlaySmart. Five hundred and thirty-two adolescents aged 16–19 years old, who are at greater risk for initiating opioid misuse are recruited from 10 Connecticut school-based health sites. Participants are randomized to PlaySmart or a set of time/attention control videogames. Randomization was stratified by sex at birth and school grade. Participants play their assigned game or games for up to six weeks (300 minutes) and complete assessment questions over a 12-month period (baseline, post-gameplay, 3, 6, and 12 months). The primary outcome is perception of risk of harm of opioid misuse at 3 months. Secondary outcome measures specific to opioid misuse include intentions, self-efficacy, attitudes, knowledge, and perceived norms. Mental health outcomes include measures of depression (Patient Health Questionnaire-8), anxiety (Generalized Anxiety Disorder-7), help-seeking behaviors, stigma, measures of self-regulation, self-efficacy to seek professional help for mental health, and knowledge around coping skills. PlaySmart has the potential to significantly reduce the risk of initiation of opioid misuse, improve mental health outcomes, and given its high levels of engagement and accessibility, holds the promise for extensive reach, scale, and impact for adolescents.

Trial registration: ClinicalTrials.gov: NCT04941950. Registered on 23 June 2021.

Background

Opioids are a potentially addictive class of drugs that are legally available by prescription as pain relievers and illegally as drugs such as heroin [1]. Since the severity of the opioid epidemic has risen to a national public health emergency [2], prescribing opioids has become a cause for concern [3]. Reducing rates of opioid prescribing to adolescents helps to minimize the potential misuse of prescription opioids and associated health consequences, as adolescents who report opioid misuse often report medical use of opioids prior to misuse [4]. Forty percent of prescription opioid misuse and 68% of heroin misuse begins during adolescence [5], and the risk of developing opioid use disorder (OUD) and other substance use disorders (SUDs) is heightened during adolescence [6]. According to the 2019 Youth Risk Behavior Survey (YRBS) that included a sample of 13,872 participants from 136 different US schools, 14% of high school students reported lifetime misuse of opioids and half of those reported currently misusing opioids [7, 8]. Misuse of prescription opioids among adolescents is also associated with other adverse health outcomes and risk behaviors, including the use of alcohol and other drugs, youth violence, increased risk for HIV and other sexually transmitted diseases, and an increased risk of overdose [9]. Drug overdose is now the leading cause of death for those under age 50, and 75% of these 92,000 deaths were opioid-related in 2020 alone [10].

Studies have also demonstrated that opioid misuse or non-prescription use of opioids is highly correlated with mental health issues [11, 12]. Mental health and its relationship to opioid misuse must be examined and addressed through preventive interventions, as mental health issues in youth can precede a multitude of other problems, including but not limited to delinquency [13], suicide [14], mood and anxiety disorders [12], opioid misuse and OUD [11, 12], and other SUDs [13, 15]. Recently, the COVID-19 pandemic has exacerbated the already troubling youth mental health crisis, leading to a call for action from the Office of the U.S. Surgeon General [14, 16]. Even before the pandemic in 2019, one in three high school students had reported persistent feelings of sadness and hopelessness, representing a 40% increase from 2009 [15]. Reflecting the severity of the increase in mental health issues, suicide is now the second-leading cause of death for those ages 15–24, with 36% seriously considering attempting [17]. Adolescents are at a critical period in their development where education and interventions to address mental health issues are imperative and can potentially assist in preventing or reducing the risk for opioid misuse. Preventive interventions that target distinct issues that are often interrelated, such as mental health and opioid misuse, allow for a greater efficacy of addressing these issues and a wider overall impact.

In 2018, the National Institutes of Health launched the Helping to End Addiction Long-term (HEAL) Initiative to combat the opioid crisis, outlining 26 research priorities that target unmet needs for improving treatment for OUD and preventing opioid misuse in higher risk populations [6, 18]. Ten projects, including the study described here (1UG3DA050251-01/4UH3DA050251-03; PI: Fiellin), around the country are a part of the HEAL Prevention Cooperative (HPC) and are funded by the HEAL Initiative, and all of these target different populations that are at higher risk of opioid misuse in a variety of settings [6, 1821].

Videogames as interventions, or digital health games, have the advantage of meeting adolescents “where they are,” physically, educationally, and through means they are already familiar with–technology. Over 90% of adolescents play video games [22]. Serious games–games that have a primary purpose other than solely entertainment–are an ideal prevention platform, as they give adolescents the opportunity to practice healthy behaviors before transferring these skills to the real world [23, 24]. Unlike other interventions, serious games can be delivered with consistent fidelity; they also have the advantage of requiring minimal additional personnel and support while being easily delivered and distributed. The play2PREVENT® Lab has a wealth of experience in developing, evaluating, and disseminating easily accessible videogame interventions that target a range of health behaviors in adolescents [2537].

As a part of the HEAL Initiative, our research team developed and is evaluating a digital health game, PlaySmart, that aims to prevent the initiation of opioid misuse and promote mental health by increasing the perception of risk of harm from opioid misuse and increasing adaptive mental health strategies, respectively [6, 18]. The PlaySmart formative work incorporated input from adolescents to develop the intervention. There is strong evidence that opioid misuse and non-medical prescription opioid use are highly correlated with mental health issues [11]. This is further supported by the fact that the themes of mental health and seeking support emerged from our focus group discussions when gathering input on how to prevent the initiation of opioid misuse in adolescents. The subsequent pilot study used feedback from adolescents to establish the acceptability of the intervention with the target population.

The current randomized controlled trial (RCT) is evaluating the impact of the PlaySmart game, as compared with a control condition, on a range of outcomes related to opioid misuse and mental health in older adolescents at school-based health sites. Given its evidence-based development, mode of delivery, engagement, and accessibility, PlaySmart has the potential to both prevent opioid misuse and promote positive mental health behaviors and have considerable reach and impact on its target population.

Methods

Study overview

This study is a RCT designed to assess the efficacy of an original digital health game, PlaySmart, that targets opioid misuse and mental health in older adolescents. Five hundred and thirty-two adolescents, 16–19 years old, who are at higher risk for opioid misuse are recruited from 10 Connecticut high schools. These high schools partner with us through school-based health programs, working with School-Based Health Centers (SBHCs), school-based health staff, or other integral school personnel. Participants are randomized 1:1 to either the intervention group where they play PlaySmart or a set of time and attention control videogames that include games such as That Dragon Cancer, Can You Escape, and The Sims. These games have no relevant content to our target outcomes. For sites that have SHBCs, their staff and the research study team work together for recruitment and screening efforts. Recruitment efforts include the use of the lunch wave process (tabling and providing opt-outs to interested students), posters with QR codes, morning school announcements, and videos that detail the project.

Study timeline

Recruitment and enrollment for this RCT commenced in the Fall of 2021 on October 21st, 2021 and will continue through the Fall of 2023 ending on December 14th, 2023, and the 12-month follow-up will continue through the Fall of 2024. During the summers, the research team focuses its efforts on retention for the follow-up assessments, as no enrollment or intervention delivery takes place when school is not in session. The study timeline, an adaptation of the SPIRIT figure, can be found in Fig 1.

thumbnail
Fig 1. The study timeline, an adaptation of the SPIRIT figure.

https://doi.org/10.1371/journal.pone.0291298.g001

Recruitment, screening, eligibility, and informed consent/assent.

Interested participants are given a brochure about the study and an opt-out form. The opt-out form was created for the study to inform parents/guardians ahead of the eligibility screening process for the project, given that the screener includes questions about anxiety, depression, and substance misuse. Research study staff distribute these materials to interested potential participants during on-site recruitment sessions or partnered school-based health program staff distribute the form to students directly or electronically to parents/guardians of 16- and 17-year-olds who are enrolled in their health program. These forms are distributed ahead of all screenings, and the research team follows up with the potential participant via text to ensure that the form and brochure have been given to their parent/guardian. If a parent/guardian does not return the signed opt-out form indicating that their teen to be screened for the study, the school-based health program staff or research study team proceeds with screening the participant for study eligibility. If a student or parent/guardian does return the signed opt-out form, the research team documents this, and the student is never screened for eligibility. Students who are 18 or 19 years old receive a version of the opt-out form that is tailored to them and that does not require that it be shown to a parent/guardian.

The screening tool used to assess eligibility of potential participants consists of 14 questions that assess substance use and levels of anxiety and depression. Screeners are completed by potential participants independently under the supervision of school health staff or the research team. The composite screening tool was developed for this project and uses screening questions from CRAFFT N+ [38], Generalized Anxiety Disorder 2-item scale (GAD-2) [39], and Patient Health Questionnaire 2-item scale (PHQ-2) [39] which are commonly used in school-based health settings. Eligibility criteria for study participation are: 1) preferable enrollment in a SBHC; 2) aged 16–19 (grades 9–12); 3) report of no prior opioid misuse; 4) self-report past 30-day use of cigarettes, e-cigarettes, Juul, alcohol, marijuana (including synthetics), amphetamine, cocaine, benzodiazepines, ecstasy, bath salts, or any other misuse of non-opioid prescription drugs or use of non-opioid illicit drugs and/or have a score of ≥ 1 on the PHQ-2 [39] OR a score of ≥ 1 on the GAD-2 [39]; 5) willingness to sit for 60 minutes/session to play the game; and 6) ability to provide assent and parental/guardian consent (if under age 18).

If a potential participant is found to be eligible, they are given an enrollment packet that contains a study timeline and consent form that they must sign (if they are 18 years or older), or their parent/guardian must sign if they are under the age of 18. The research team follows up with the potential participant and parent/guardian (when appropriate) as a reminder to return the completed forms. Parent/guardian consent forms in Spanish, Portuguese, and French are provided when appropriate for the primary language spoken at home. A full ethical and human subjects research review was conducted and approved by the Pediatric Protocol Review Committee and the Institutional Review Board at the Yale School of Medicine (Protocol # 2000030553).

For our partner schools that have school-based health programs (including SBHCs), we work with clinical and non-clinical staff to screen and enroll students. Any mental health or substance use concerns that arise from participants taking the screener are addressed by clinicians in the school-based health programs. If any potential participant is found to be ineligible; they receive a packet that contains informational flyers developed by National Institute on Drug Abuse (NIDA), Substance Abuse and Mental Health Services Administration (SAMHSA), and our lab that provides information on drugs and the teen brain, mental health, school-based health programs, and school counselors. For schools without school-based health programs, we work with other integral school staff (e.g., health teachers, principals, and security guards) to enroll students. All students (those enrolled and not enrolled in the study) are provided with a packet that contains information mental health, substance use, and resources available to them in their school either after screening (if ineligible) or after baseline (if eligible). During the first baseline meeting (see below) prior to study enrollment and data collection, the components of the research project are explained again to participants and adolescent assent is obtained for those participants under 18 who turned in their parent/guardian consent form.

Baseline assessment meetings.

Following obtaining assent and consent, baseline assessment meetings are conducted in-person at the partner school sites and divided into two days to facilitate on the ground workflow. The research study team enters the participant’s contact information into the REDCap P11 database to set them up with a REDCapPRO account. The database sends a personalized link for the participant to set up an electronic account on REDCapPRO that is then used to complete their assessment questions at all timepoints associated with the study. REDCap (P11) is a highly secure version of REDCap, the 21 CFR Part 11 Validated REDCap Service that meets the required Food and Drug Administration (FDA) compliance guidelines for this study, and REDCapPRO is an extension of REDCap(P11) that allows participants to access their assessments so that data is populated into REDCap(P11) by the participant [40, 41].

During the second baseline assessment meeting, held one to two days later, participants complete baseline assessment questions electronically through their REDCapPRO account, which is the preferred method, or on paper and the research study team enters them into the study’s REDCap(P11) database [40, 41]. The study initially began with all participants completing baseline on paper as the database was being developed to fit the study’s workflow. During the 2022–2023 school year, the team made the transition to collecting all data electronically (except in extreme cases where technology prevented this, then paper assessments were still used).

Participants receive a $55 gift card upon completion of baseline assessments. Additionally, each participant who completes the baseline assessment receives a packet of informational flyers developed by NIDA, SAMHSA, and our lab that provides information on drugs and the teen brain, mental health, and school-based health programs and school counselors (similar to participants deemed to be ineligible for the study).

Randomization.

After completion of the baseline assessments and prior to participants’ first gameplay session, participants are randomized by research study staff. A single randomization scheme was generated and written in REDCap(P11). Randomization is stratified by sex at birth (male and female) and grade (dichotomized to 9–10 or 11–12). Eligible participants are assigned to play either PlaySmart or to a menu of control games on an iPad.

PlaySmart experimental condition.

Participants that are randomized to the intervention condition are assigned to play the digital health game PlaySmart. Components of PlaySmart are based on evidence-based behavior change theories; these are embedded throughout the six storylines and five minigames. Each storyline targets one or more unique themes that emerged during the development work with the target audience of adolescents and other stakeholders: exposure to opioids (both in a medical setting and in an informal, recreational setting), accessing mental health treatment, peer-to-peer support, boundaries with peers, reasons to misuse, and support systems.

Attention/Time control condition.

Participants that are randomized to the control condition have their choice of playing nine videogames such as That Dragon Cancer, Papers Please, and The Sims. These control games contain no relevant content related to the study goals.

Gameplay sessions.

Participants play on assigned iPads for gameplay sessions one to two times a week for 40 minutes to an hour over 3–6 weeks (depending on individual school site schedule). Gameplay sessions occur mostly after school, but at certain school sites, the school allows the study team to conduct gameplay sessions during participant study halls or free periods. The research study staff bring assigned iPads, headphones, and snacks to each gameplay session. The target gameplay time for the gameplay intervention is 300 minutes, regardless on the overall timeframe for gameplay sessions.

Follow-up time periods.

Outcome measures are being collected at baseline, post-gameplay (immediately following completion of gameplay), 3, 6 and, 12 months. The primary outcome is the perception of risk of harm of opioid misuse at the 3-month time-point. We also collect data on secondary outcomes and gameplay experience. Data on gameplay experience are only collected for experimental participants at the post-gameplay timepoint through a series of intervention-specific gameplay questions as well as qualitative interviews with those participants. Each assessment time-point has a specified window within which collection of the assessment data are considered “on time” as indicated: 6-week and 3-month assessments (+/-2 weeks); 6-month and 12-month assessments (+/- 1 month). Participants receive gift cards for completing assessments at each study visit: baseline ($55), post gameplay ($35), 3-month ($45), 6-month ($45), 12-month ($45). For follow-up timepoints during the school year, sessions are held in person at partner high school sites and are coupled with a pizza party for participants. For assessments that fall outside of the school year, research study staff meet with participants online via Zoom to be available for any questions that may arise as they answer their questions electronically.

Participant retention.

A total of 532 participants, aged 16–19 from 10 high schools including those that have school-based health programs are currently being recruited for this RCT to test the efficacy of PlaySmart. The research study team is highly engaged with participants to ensure text reminders are sent before and the day of all gameplay sessions and assessment completion days.

Data collection and management.

Study data is collected and managed using REDCap electronic data capture tools hosted at Yale University [40, 41]. REDCap (Research Electronic Data Capture) is a secure, web-based software platform designed to support data capture for research studies, providing 1) an intuitive interface for validated data capture; 2) audit trails for tracking data manipulation and export procedures; 3) automated export procedures for seamless data downloads to common statistical packages; and 4) procedures for data integration and interoperability with external sources. REDCap(P11) requires enrolled participants to set up accounts via REDCapPRO to answer their assessment questions associated with the study; this ensures that the intended participant is the only one who can answer the assessment questions. When this study began, we utilized the paper version of the study assessments to collect data, as the REDCap(P11) database was being completed and tailored to our study. After collecting participants’ assessment data on paper, the study team would enter the data into the backend of REDCap(P11). During the summer of 2022, the project transitioned to fully electronic versions of the assessments that can be emailed out to participants through the REDCapPRO extension of the REDCap(P11) system. This allows study participants to have their answers directly recorded in the database without the study team having to manually enter the data points.

We have established a Data and Safety Monitoring Board (DSMB) comprised of three experts in clinical trials in adolescents (one with expertise in substance use prevention), an expert in statistical analysis of clinical trials, and an expert in intervention design and implementation of RCTs related to opioids and other substances. The DSMB meets with the research team twice yearly to review study progress and review any specific challenges or adverse events.

Data measures.

Data from assessment measures help evaluate changes in the primary and secondary study outcomes. These measures are a compilation of both assessments that are used across all 10 HEAL project sites (“common measures” harmonized across all sites) and ones that are specific to our study site. A complete description of all assessment measures by timepoint can be found in Table 1.

thumbnail
Table 1. Timeline of assessment and corresponding data measures.

https://doi.org/10.1371/journal.pone.0291298.t001

Demographics. Demographic characteristics such as age, grade level, sex at birth, gender identity, sexual orientation, race, ethnicity, education level, and socio-economic status are collected through 12 questions.

Involvement with substances. Involvement with four different substances (e.g., illegal and legal opioids, alcohol, and marijuana) is assessed through four stem question asking about lifetime use inside and outside of medical settings for each substance. Measures of involvement with all substances are harmonized and the same instrument is used across all 10 HEAL sites [20, 21, 42, 43]. If participants answer yes to lifetime use for any of the substances, they are asked 11 questions about frequency and consequences.

Emotional regulation. Self-regulation in a specific moment (i.e. trait) is assessed only once at baseline with the Difficulties in Emotional Regulation Scale (DERS), containing 10 questions [44]. Self-regulation across time (i.e. state) is assessed at all timepoints and with use of The State Difficulties in Emotion Regulation Scale (S-DERS), a 10-item inventory [45]. Answer choices on the DERS range from not at all to completely, while answer choices on the S-DERS range from not at all to almost always. Measures of emotional regulation are not harmonized across all HEAL sites.

Self-efficacy. Self-efficacy to both do not misuse and to refuse prescription opioids and heroin are assessed through a total of 8 questions on two separate forms. Participants are given the stem, “Imagine that you have access to prescription opioids/heroin…” with branched questions asking about confidence to not misuse and refuse prescription opioids and heroin in the next 30 days. They are given 5 answer choices ranging from not at all to extremely. These measures were developed from the Drug Use Resistance Self-efficacy (DURSE) Scale [46]. The measure looking at the self-efficacy to misuse is harmonized and used across all HEAL sites [20]. Self-efficacy to refuse is a study-specific measure not harmonized across HEAL sites.

Mental health. Both anxiety and depression are measured at all timepoints through the use of the Patient-Health Questionnaire 8-item (PHQ-8) for depression [47] and the Generalized Anxiety Disorder 7-item inventory (GAD-7) for anxiety [48]. For each of measures, participants are given the stem question: “Over the last two weeks how often have you been bothered by any of the following problems…” with 8 question descriptors for depression (PHQ-8) and 7 question descriptors for anxiety (GAD-7). Measures of mental health are harmonized and used across all HEAL sites [20].

Pain. Experience with pain is assessed with the PROMIS Pain Interference Scale (PROMIS-PI) 2 item scale; answer choices range from Never to Almost Always [49, 50]. The PROMIS-PI measure is harmonized and used across all HEAL sites [20].

Social exposure and peer norms. Social exposure to substance misuse is assessed with the stem question: “What is the most often that someone who lives with you…” with 3 question descriptors, asking about three different substances: alcohol, marijuana, and heroin. Answer choices range from Never to 4–7 days a week [51]. Peer norms are assessed similarly through two questions related to the prevalence of opioids misuse in their school, with answer choices ranging from hardly any or none to all or most [52]. Peer norms is a study-specific measure not harmonized across all HEAL sites.

Perception of risk of harm of opioid misuse scale. Perception of risk of harm of opioid misuse is the study’s primary outcome. There is a well-established inverse relationship between perception of risk of harm from drug use and actual drug use [53]. This relationship in which low perceived harmfulness is strongly associated with a high risk of misuse has been demonstrated with non-medical use of prescription drugs [54], prescription opioids [5557], and heroin [58]. Individuals with low perceived harmfulness were almost 10 times more likely to use prescription opioids nonmedically, as compared to those with high perceived harmfulness (95% CI = 2.1–44.0) [55]. Similarly, results indicate a significant association between reporting great risk of trying heroin and lower likelihood of lifetime heroin use (OR = 0.38, 95% CI: 0.33, 0.44, p < 0.001) [58]. Given there is evidence that youth generally have a low perception of risk of harm around opioid misuse, we focus on perceived risk of harm from opioid misuse as a proximal target underlying the videogame intervention’s impact on the more distal outcome of initiation of opioid misuse.

Perception of risk of harm score (PRHS) is assessed with the stem question: “How much do you think people risk harming themselves (physically or in other ways), if they [branched question]?” with 8 different branched questions related to illegal opioids (heroin) and legally available opioids (e.g., Percocet, OxyContin, etc.). Examples of branched questions include “try heroin once or twice?” and “try any narcotic other than heroin (non-heroin narcotics such as prescription opioids such as Codeine, Vicodin, Oxycontin, Percocet, etc.) once or twice?” Answer choices are no risk, slight risk, moderate risk, and great risk, providing a score range of 1–4 [21, 53]. Perception of risk of harm is a study-specific measure not harmonized across HEAL sites.

Intentions to misuse opioids. Intentions to misuse opioids is assessed with the stem question: “How likely is it that you will [question descriptor] over the next 12 months?” with four different question descriptors. Examples of question descriptors include “use prescription opioids, even once or twice” and “use heroin, even once or twice.” Answer choices are definitely not, probably not, probably will, and definitely will [59]. Intentions to misuse opioids is a study-specific measure not harmonized across HEAL sites.

Attitudes toward misuse and its risks. Attitudes toward misuse and its risk are assessed by having participants indicate how much they agree or disagree with 15 different branched prompts with the following stem statement: “If I used/took a prescription opioid, or painkiller, or heroin, I would [branched prompt].” Examples of branched prompts included: feel good, get away from my problems, and feel less anxious. Five answer choices are presented that range from strongly disagree to strongly agree [60]. Attitudes towards misuse and its risks is a study-specific measure not harmonized across HEAL sites.

Decision-making skills. Decision-making skills are assessed by presenting participants with 9 statements such as: “I think about difficult situations that I might find myself in and make plans to deal with them” and “When there is a problem, I try to figure out what caused it.” For these statements, their answer choices are never, seldom, about half the time, often, and always [61]. Decision making skills is a study-specific measure not harmonized across HEAL sites.

Knowledge about opioid misuse and its risks. Knowledge is assessed with 30 statements such as “Prescription opioids can be just as deadly as heroin when they are misused”. Participants are given answer choices of true, false, or not sure. This instrument was adapted from the AIDS Risk Behavior Knowledge scale and modified to fit the outcomes around opioids and opioid misuse targeted in our study [62]. Knowledge about opioid misuse and its’ risks is a study-specific measure not harmonized across HEAL sites.

Help-seeking. Participants are asked both about their past help-seeking behaviors and general help-seeking. Past Help-Seeking is assessed by presenting the participant with a list of 13 different people that they might seek help or advice from; they check any of those people/resource that they have gone to for advice or help in the past six months. If they sought help from that person/resource, they are then asked to quantify the number of times in the past six months. Examples of people and resources listed include: partner, friend, school psychologist, phone/texting help line, internet [63, 64]. Participants are also given the General Help-Seeking Questionnaire to assess their tendencies to seek help in times of emotional distress [65]. This instrument contains 20 items related to how likely it is that they would seek help if (a) experiencing a personal or emotional problem and (b) if they were experiencing suicidal thoughts. The answer choices are given on a 7-point Likert scale with answers ranging from extremely unlikely to extremely likely. Both help-seeking measures are study-specific and not harmonized across HEAL sites.

Self-stigma of help-seeking scale. Stigma associated with help-seeking for mental health or substance misuse is assessed with the 10-item Self-Stigma of Seeking Help (SSOSH) [66]. Five answer choices are given ranging from strongly disagree to strongly agree. SSOSH is a study-specific measure not harmonized across HEAL sites.

Beliefs about psychological services. The Beliefs About Psychological Services (BAPS) scale is used to assess self-beliefs about seeking help and professional psychological services [67]. The answer choices provided are true or false. The BAPS scale is a study-specific measure not harmonized across HEAL sites.

Gameplay experience, usability, and engagement. For participants randomized to the experimental condition, gameplay experience, usability, and engagement is assessed with 15 questions such as: “This game helped me learn important things,” and “I felt frustrated with playing PlaySmart.” Answer choices range from strongly disagree to strongly agree [36]. Gameplay measures are study-specific and not harmonized across HEAL sites.

In-game data. In-game data is collected through the backend system of the game and available through the server that hosts the game. In-game data reports allow us to analyze the different decisions and choice paths that the player selected in the game.

Sample size.

The study is powered to detect a 15% absolute difference between control group and intervention group in terms of perceived risk of harm of misuse. The assumption is that at baseline, 32% of participants would report great perception of risk of harm, so at 3 months, it is assumed that the difference between the two study arms would be 15%: the difference in the proportion of participants achieving the primary outcome (reporting great risk of harm from misuse of prescription opioids and heroin/fentanyl) [50% (PlaySmart) vs. 35% (control)] with a power of 90% (p-value ≤ 0.05) (Power Analysis Statistical Software (To account for a 16% loss at 3 months based on our PlayForward study [28], the sample size is inflated to 532 participants (266 in each arm).PASS); 2008) [68]. A conservative estimate from the literature on drug use prevention interventions and using data from a study examining the impact of an intervention on perceptions of harm of drug use, indicates that a sample size of 454 participants will be required to detect a 15% absolute difference in the proportion of participants achieving the primary outcome. To account for a 16% loss at 3 months based on our PlayForward study [28], the sample size is inflated to 532 participants (266 in each arm).

Planned analyses.

Descriptive statistics of baseline characteristics (mean, median, IQR) and graphs will be used to assess randomization adequacy. Baseline characteristics that are determined not to be equally distributed among the two groups will be considered for covariate adjustment to determine their impact on intervention comparisons. The primary hypothesis is that, at three months, there will be a higher proportion of PlaySmart vs. control participants who report greater perception of risk of harm of misuse of prescription opioids AND heroin/fentanyl. The primary comparison will be the main effects of PlaySmart vs. control games in terms of the Perception of Risk of Harm Score (PRHS) that was constructed from a set of eight questions (range of score 1–4). The proportion of participants in the two study arms who achieve the primary outcome (PRHS = 32) at three months will be compared. The probability of achieving the primary outcome will be modeled using logistic regression. An unadjusted model (intervention variable only; stratified and non-stratified) and adjusted models (intervention plus other predictor variables) will be used in the analytical approach. In addition to the 3-month comparison, chi-square tests will be used to assess primary outcome proportion differences between the two study arms at 6 weeks, 6 months and 12 months. Additionally, the mean PRHS (continuous) will be compared at the various study time points (6 weeks/3 months/6 months/12 months) using t-tests. Mixed models will be used for longitudinal analysis of the PRHS to assess differences between the two study groups over time. Further analysis will assess differences in the direction of change (positive/negative delta) for each of the eight questions.

Differences in (a) intentions to misuse opioids and (b) self-efficacy for refusing opioids will be compared between the two groups. Means (standard errors) of intention and self-efficacy scores, by group, will be plotted at the various study time points (6 weeks, 3, 6 and 12 months). Change from baseline at each timepoint will also be compared between the two groups. Mixed Models (MM) will be used for longitudinal analysis to assess differences between the two groups over time in terms of (a) intention and (b) self-efficacy scores.

While the goal of this work is to prevent the initiation of opioid misuse in youth, it is not likely that a substantial proportion of even these adolescents at higher risk will initiate opioid misuse during the 12-month follow-up period. We will compare the proportions of participants initiating opioid misuse at all follow-up timepoints (with a focus on 12 months) using chi-square analyses. Given the observed rate of initiating opioid misuse in high school populations is estimated to be between 7–14% [15, 69], and we are focusing on higher risk adolescents who report current substance use and symptoms of depression or anxiety, we estimate a three-fold increased likelihood of initiating opioid misuse resulting in 42% of the population initiating opioid misuse. Given the relationship between perception of risk of harm and initiation of misuse and a 15% reduction in initiation of opioid misuse based on exposure to the videogame intervention, we expect an absolute decrease in initiation of 6% (36% in the PlaySmart arm vs. 42% in the control arm) at 12 months. The effective sample size of 532 will provide 29% power (two-sided alpha-level 0.05) to detect this difference. The probability of achieving this outcome will also be modeled using logistic regression analysis. Stratified (using randomization stratification variables; gender and grade) analysis and non-stratified analyses and adjusted models (e.g., intervention plus other predictor variables, mediator, and moderator variables) will be used in further logistic regression analyses.

Data will be collected on potential mediators and moderators. Since these variables might change over time during participation in the study, they will also be considered as time varying covariates.

Data from secondary outcomes will be presented descriptively at study time points at which they are collected. Numbers and proportions of participants will be presented for each of the components used to assess secondary outcomes. In addition, for standardized tools that have validated and published scoring algorithms, those algorithms will be used to compare the two study arms, both at various time points as well as over time. Additional analyses will be carried out for other subsets of questions in such tools using appropriate methodology to establish groups of questions to analyze together (e.g., Cronbach’s alpha analysis). As appropriate, repeated measures analysis will also be used to compare the two study arms in terms of secondary outcomes. Below the study’s secondary outcomes are classified as mediators or moderators.

Mediators. We predict that 1) knowledge, 2) perceived norms, and 3) attitudes will mediate the effects of the intervention on perception of risk of harm from misuse of opioids at 3 months. We will test the relationships among the outcomes and presumed mediators as proposed by MacKinnon et al. [70]. To test the significance of the mediation effect, we will calculate Sobel’s test for each proposed mediator. Sobel’s test determines the significance of the mediation effect by testing the null hypothesis that the indirect effect coefficient is zero (i.e., whether the indirect effect of the independent variable on the dependent variables is significantly different from zero) [71].

Moderators. Variables that predict the outcome variable of perceived risk of harm differently between the intervention and control groups will be considered to be moderators using the model outlined by Kraemer et al [72]. To be considered a moderator, the variable must be present prior to randomization and must not be related to the independent variable (PlaySmart vs. control games). Decision-making skills as a moderator will be compared between the two study arms at (a) the various time points; (b) change from baseline at various time points and overall study duration; (c) improvement (no/yes change) and (d) longitudinally. We will use t-tests for continuous variables, chi-square for comparing proportion of participants who showed improvement from baseline to each study time point and overall, and longitudinal modeling to include measures of moderators over the whole study duration.

Results

To date (as of 6/30/23), 380 participants have been enrolled (completing parent/guardian consent, adolescent assent and baseline assessments) and 373 participants have been randomized (7 did not continue with enrollment before randomization occurred). After randomization, three participants withdrew from the study and informed study staff. Currently, 370 participants are enrolled. The following have completed the follow-up protocol defined assessments: post-gameplay: 343 (92.7%); 3 months: 296 (80.0%); 6 months: 223 (60.3%); 12 months: 69 (18.7%). Currently 380 participants are in the assessment follow-up phase of the project, having completed the intervention (gameplay) phase of the project with assessment time windows opening soon for their upcoming assessments, and none actively in the intervention (gameplay) phase as the academic year has come to an end.

Data will be made available through the NIH Helping to End Addiction Long-term (HEAL) Initiative Public Access and Data Sharing Policy that provides guidance on an infrastructure for researchers, clinicians, and patients to share their information and knowledge; our study complies with this policy [73].

Discussion

The opioid epidemic is a growing critical national crisis. In adolescents, it is estimated that one in seven high school students have misused opioids in their lifetime and one in 14 high school students is currently misusing opioids [15]. Many of the current approaches to addressing opioid misuse in adolescents occur through monitoring and regulating entities that are able to prescribe opioids and statewide policies that limit the number of opioids being prescribed [74]. However, many of these approaches occur after their use has developed into opioid use disorder [16, 75]. Furthermore, prevention programs have been limited, with opioid misuse being rolled into prevention campaigns or school-based interventions having minimal dose, no rigorous evaluation for the efficacy of the program [76] or are not specifically designed for adolescents [77]. Therefore, there is a critical need for prevention-focused efforts and interventions to adequately address the opioid epidemic in this vulnerable population.

Moreover, in the area of digital health games that specifically focus on mental health, there have been some interventions that have been evaluated through randomized controlled trials that focus on specific topics like stress, anxiety and depression, but the PlaySmart game is the only intervention that addresses both mental health and opioid misuse [7882] and recently was recognized as an important resource for youth mental health by the Office of the U.S. Surgeon General [14].

There are potential limitations to this work including our assessments being self-report data; for example, we are not collecting urine samples to test for actual opioid misuse. However, our methods for data collection, that have been used and refined for a number of our prior trials, focus on optimizing privacy and confidentiality for our participants so they are encouraged to respond to questions accurately and honestly. Finally, while the COVID-19 pandemic intermittently introduced challenges in recruitment and data collection, the team was consistently pivoting, using online or virtual means to enroll and record participant responses, when needed. Despite this, we have demonstrated successful recruitment, enrollment, and collection of follow-up data and are very confident that that success will continue until the conclusion of the trial.

The comprehensive and rigorous approach to evaluating the efficacy of PlaySmart through the proposed RCT is critical in demonstrating its impact on both preventing opioid misuse and promoting mental health in older adolescents. As a highly accessible and engaging digital health game intervention, PlaySmart holds the promise of delivering its evidence-based knowledge/information and skill-building in a manner that has a high likelihood of “sticking” while having the potential of widespread dissemination and extensive reach to adolescent populations.

Supporting information

S1 Protocol. Complete approved institutional review board protocol.

https://doi.org/10.1371/journal.pone.0291298.s002

(PDF)

Acknowledgments

We thank Katherine Su (play2PREVENT lab) for her contributions to methodology and study execution; the team at Schell Games for their work developing the intervention; and our school partners across the state of Connecticut for their continued support and partnership.

References

  1. 1. National Institute on Drug Abuse; National Institute of Health; US Department of Health and Human Services [Internet]. [cited 2022 Sep 16]. Prescription Opioids DrugFacts. Available from: https://nida.nih.gov/publications/drugfacts/prescription-opioids
  2. 2. Merica D. Trump declares opioid epidemic a national public health emergency. CNN Politics. 2017 Oct 26. [Cited 2022 Sep 16]. Available from: https://www.cnn.com/2017/10/26/politics/donald-trump-opioid-epidemic/index.html
  3. 3. Rosenblum A, Marsch LA, Joseph H, Portenoy RK. Opioids and the Treatment of Chronic Pain: Controversies, Current Status, and Future Directions. Exp Clin Psychopharmacol. 2008 Oct;16(5):405–16. pmid:18837637
  4. 4. Hadland SE, Wharam JF, Schuster MA, Zhang F, Samet JH, Larochelle MR. Trends in Receipt of Buprenorphine and Naltrexone for Opioid Use Disorder Among Adolescents and Young Adults, 2001–2014. JAMA Pediatrics. 2017 Aug 1;171(8):747–55. pmid:28628701
  5. 5. Center for Behavioral Health Statistics and Quality. 2017 National Survey on Drug Use and Health: Detailed Tables. Substance Abuse and Mental Health Services Administration. 2018 Sep;Rockville, MD.
  6. 6. HEAL Initiative; National Institute of Health [Internet]. [cited 2022 Sep 16]. HEAL Initiative Research Plan. Available from: https://heal.nih.gov/about/research-plan
  7. 7. Gaither JR, Shabanova V, Leventhal JM. US National Trends in Pediatric Deaths From Prescription and Illicit Opioids, 1999–2016. JAMA Network Open. 2018 Dec 28;1(8):e186558. pmid:30646334
  8. 8. 1991–2019 High School Youth Risk Behavior Survey data [Internet]. Atlanta (GA): The Centers for Disease Control and Prevention (CDC) [cited 2022 Sep 16]. Available from: http://nccd.cdc.gov/youthonline/
  9. 9. Gaither JR, Leventhal JM, Ryan SA, Camenga DR. National Trends in Hospitalizations for Opioid Poisonings Among Children and Adolescents, 1997 to 2012. JAMA Pediatr. 2016 Dec 1;170(12):1195–201. pmid:27802492
  10. 10. Hedegaard H, Warner M. Drug Overdose Deaths in the United States, 1999–2020. 2021;(428):8. pmid:34978529
  11. 11. Martins SS, Keyes KM, Storr CL, Zhu H, Chilcoat HD. Pathways between nonmedical opioid use/dependence and psychiatric disorders: results from the National Epidemiologic Survey on Alcohol and Related Conditions. Drug Alcohol Depend. 2009 Jul 1;103(1–2):16–24. pmid:19414225
  12. 12. Martins SS, Fenton MC, Keyes KM, Blanco C, Zhu H, Storr CL. Mood and anxiety disorders and their association with non-medical prescription opioid use and prescription opioid-use disorder: longitudinal evidence from the National Epidemiologic Study on Alcohol and Related Conditions. Psychol Med. 2012 Jun;42(6):1261–72. pmid:21999943
  13. 13. Clayton HB, Bohm MK, Lowry R, Ashley C, Ethier KA. Prescription Opioid Misuse Associated With Risk Behaviors Among Adolescents. Am J Prev Med. 2019 Oct;57(4):533–9. pmid:31443955
  14. 14. Murthy VH. Protecting Youth Mental Health: The U.S. Surgeon General’s Advisory [Internet]. Surgeon General of the United States; 2021 [cited 2023 Apr 28] p. 1–53. Available from: https://www.hhs.gov/sites/default/files/surgeon-general-youth-mental-health-advisory.pdf
  15. 15. Jones CM, Clayton HB, Deputy NP, Roehler DR, Ko JY, Esser MB, et al. Prescription Opioid Misuse and Use of Alcohol and Other Substances Among High School Students—Youth Risk Behavior Survey, United States, 2019. MMWR Suppl. 2020 Aug 21;69(1):38–46. pmid:32817608
  16. 16. American Academy of Pediatrics [Internet]. AAP-AACAP-CHA Declaration of a National Emergency in Child and Adolescent Mental Health. [cited 2022 Sep 16]. Available from: http://www.aap.org/en/advocacy/child-and-adolescent-healthy-mental-development/aap-aacap-cha-declaration-of-a-national-emergency-in-child-and-adolescent-mental-health/
  17. 17. Centers for Disease Control and Prevention (CDC); National Center for Injury and Prevention Control [Internet]. [cited 2022 Sep 16]. Facts About Suicide. Available from: https://www.cdc.gov/suicide/facts/index.html
  18. 18. HEAL Initiative; National Institutes of Health [Internet]. [cited 2022 Sep 16]. Preventing At-Risk Adolescents from Developing Opioid Use Disorder. Available from: https://heal.nih.gov/research/new-strategies/preventing-opioid-use-disorder
  19. 19. Patel SV, Cance JD, Bonar EE, Carter PM, Dickerson DL, Fiellin LE, et al. Accelerating Solutions for the Overdose Crisis: an Effectiveness-Implementation Hybrid Protocol for the HEAL Prevention Cooperative. Prev Sci [Internet]. 2022 Nov 18 [cited 2022 Nov 30]; Available from: https://link.springer.com/10.1007/s11121-022-01465-2 pmid:36399222
  20. 20. Ridenour TA, Cruden G, Yang Y, Bonar EE, Rodriguez A, Saavedra LM, et al. Methodological Strategies for Prospective Harmonization of Studies: Application to 10 Distinct Outcomes Studies of Preventive Interventions Targeting Opioid Misuse. Prev Sci [Internet]. 2022 Aug 17 [cited 2022 Nov 30]; Available from: pmid:35976525
  21. 21. Bonar EE, Kidwell KM, Bohnert ASB, Bourque CA, Carter PM, Clark SJ, et al. Optimizing scalable, technology-supported behavioral interventions to prevent opioid misuse among adolescents and young adults in the emergency department: A randomized controlled trial protocol. Contemporary Clinical Trials. 2021 Sep 1;108:106523. pmid:34352386
  22. 22. Anderson M, Jiang J. Teens, Social Media and Technology 2018 [Internet]. Pew Research Center: Internet, Science & Tech. 2018 [cited 2022 Sep 16]. Available from: https://www.pewresearch.org/internet/2018/05/31/teens-social-media-technology-2018/
  23. 23. Peng W. Design and Evaluation of a Computer Game to Promote a Healthy Diet for Young Adults. Health Communication. 2009 Mar 10;24(2):115–27. pmid:19280455
  24. 24. Kato PM, Cole SW, Bradlyn AS, Pollock BH. A video game improves behavioral outcomes in adolescents and young adults with cancer: a randomized trial. Pediatrics. 2008 Aug;122(2):e305–317. pmid:18676516
  25. 25. Duncan LR, Hieftje KD, Culyba S, Fiellin LE. Game playbooks: tools to guide multidisciplinary teams in developing videogame-based behavior change interventions. Behav Med Pract Policy Res. 2014 Mar;4(1):108–16. pmid:24653781
  26. 26. Duncan LR, Hieftje KD, Pendergrass TM, Sawyer BG, Fiellin LE. Preliminary investigation of a videogame prototype for cigarette and marijuana prevention in adolescents. Substance Abuse. 2018 Jul 3;39(3):275–9. pmid:29425481
  27. 27. Fiellin LE, Hieftje KD, Fakhouri T, Duncan LR, Sawyer B, Fiellin D. PlayForward: A videogame that increases drug, alcohol and sexual risk knowledge in teens. Drug and Alcohol Dependence. 2015 Jan;146:e277–8.
  28. 28. Fiellin LE, Hieftje KD, Pendergrass TM, Kyriakides TC, Duncan LR, Dziura JD, et al. A Video Game Intervention for Sexual Risk Reduction in Minority Adolescents: Randomized Controlled Trial. J Med Internet Res. 2017 Sep 18;19(9):e314.
  29. 29. Fiellin LE, Hieftje KD, Duncan LR. Videogames, Here for Good. Pediatrics. 2014 Nov 1;134(5):849–51. pmid:25287452
  30. 30. Hieftje K, Rosenthal MS, Camenga DR, Edelman EJ, Fiellin LE. A Qualitative Study to Inform the Development of a Videogame for Adolescent Human Immunodeficiency Virus Prevention. Games for Health Journal. 2012 Aug;1(4):294–8.
  31. 31. Hieftje K, Fiellin L. A videogame Increases HIV Risk-related Knowledge in Adolescents. Journal of Adolescent Health. 2015 Feb;56(2):S15–6.
  32. 32. Hieftje K, Edelman EJ, Camenga DR, Fiellin LE. Electronic Media–Based Health Interventions Promoting Behavior Change in Youth: A Systematic Review. JAMA Pediatr. 2013 Jun 1;167(6):574. pmid:23568703
  33. 33. Hieftje K, Duncan LR, Fiellin LE. Novel Methods to Collect Meaningful Data From Adolescents for the Development of Health Interventions. Health Promotion Practice. 2014 Sep;15(5):714–22. pmid:24519998
  34. 34. Hieftje K, Fiellin LE, Pendergrass T, Duncan LR. Development of an HIV Prevention Videogame: Lessons Learned. International Journal of Serious Games [Internet]. 2016 Jun 28 [cited 2022 Sep 16];3(2). Available from: https://journal.seriousgamessociety.org/index.php/IJSG/article/view/110
  35. 35. Hieftje K, Pendergrass T, Kyriakides T, Gilliam W, Fiellin L. An Evaluation of an Educational Video Game on Mathematics Achievement in First Grade Students. Technologies. 2017 Jun 1;5(2):30.
  36. 36. Hieftje K, Pendergrass T, Montanaro E, Kyriakides T, Florsheim O, Fiellin L. “But do they like it?” Participant satisfaction and gameplay experience of a public health videogame intervention in adolescents. In: 2018 IEEE 6th International Conference on Serious Games and Applications for Health (SeGAH) [Internet]. Vienna: IEEE; 2018 [cited 2022 May 5]. p. 1–7. Available from: https://ieeexplore.ieee.org/document/8401349/
  37. 37. Pendergrass TM, Hieftje K, Crusto CA, Montanaro E, Fiellin LE. If We Build It, Will They Come? A Qualitative Study of Key Stakeholder Opinions on the Implementation of a Videogame Intervention for Risk Reduction in Adolescents. Games for Health Journal. 2016 Aug;5(4):279–85. pmid:27336205
  38. 38. Knight JR, Sherritt L, Shrier LA, Harris SK, Chang G. Validity of the CRAFFT substance abuse screening test among adolescent clinic patients. Arch Pediatr Adolesc Med. 2002 Jun;156(6):607–14. pmid:12038895
  39. 39. Staples LG, Dear BF, Gandy M, Fogliati V, Fogliati R, Karin E, et al. Psychometric properties and clinical utility of brief measures of depression, anxiety, and general distress: The PHQ-2, GAD-2, and K-6. Gen Hosp Psychiatry. 2019;56:13–8. pmid:30508772
  40. 40. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009 Apr;42(2):377–81. pmid:18929686
  41. 41. Harris PA, Taylor R, Minor BL, Elliott V, Fernandez M, O’Neal L, et al. The REDCap consortium: Building an international community of software platform partners. J Biomed Inform. 2019 Jul;95:103208. pmid:31078660
  42. 42. Miech RA, Johnston LD, Patrick ME, O’Malley PM, Bachman JG, Schulenberg JE. Monitoring the Future National Survey Results on Drug Use, 1975–2022: Secondary School Students. Ann Arbor. Ann Arbor, Michigan: Institute for Social Research, The University of Michigan; 2023 Jun.
  43. 43. David S, Hornik R, Maklan D. National Survey of Parents and Youth (NSPY), 1998–2004—Restricted Use Files. [Internet]. Interuniversity Consortium for Political and Social Research [distributor] Available from: https://www.icpsr.umich.edu/web/NAHDAP/studies/27868/versions/V2
  44. 44. Gratz KL, Roemer L. Multidimensional Assessment of Emotion Regulation and Dysregulation: Development, Factor Structure, and Initial Validation of the Difficulties in Emotion Regulation Scale. Journal of Pscyhopathology and Behavioral Assessment. 2004;26(1):41–54.
  45. 45. Lavendar JM, Tull MT, DiLillo D, Messman-Moore T, Gratz KL. Development and Validation of a State-Based Measure of Emotion Dysregulation: The State Difficulties in Emotion Regulation Scale (S-DERS). Assessment. 2017 Mar;24(2):197–209.
  46. 46. Carpenter CM, Howad D. Development of a Drug Use Resistance Self-efficacy (DURSE) Scale. American Journal of Health Behaviors. 2009;33(2):147–57. pmid:18844509
  47. 47. Kroenke K, Strine TW, Spitzer RL, Williams JBW, Berry JT, Mokdad AH. The PHQ-8 as a measure of current depression in the general population. Journal of Affective Disorders. 2009 Apr 1;114(1):163–73. pmid:18752852
  48. 48. Spitzer RL, Kroenke K, Williams JBW, Löwe B. A Brief Measure for Assessing Generalized Anxiety Disorder: The GAD-7. Archives of Internal Medicine. 2006 May 22;166(10):1092–7. pmid:16717171
  49. 49. Cunningham NR, Kashikar-Zuck S, Mara C, Goldschneider KR, Revicki DA, Dampier C, et al. Development and validation of the self-reported PROMIS pediatric pain behavior item bank and short form scale. PAIN. 2017 Jul;158(7):1323. pmid:28394851
  50. 50. Varni JW, Stucky BD, Thissen D, DeWitt EM, Irwin DE, Lai JS, et al. PROMIS Pediatric Pain Interference Scale: An Item Response Theory Analysis of the Pediatric Pain Item Bank. The Journal of Pain. 2010 Nov 1;11(11):1109–19. pmid:20627819
  51. 51. Keyes KM, Wall M, Cerdá M, Schulenberg J, O’Malley PM, Galea S, et al. How does state marijuana policy affect US youth? Medical marijuana laws, marijuana use and perceived harmfulness: 1991–2014. Addiction. 2016;111(12):2187–95. pmid:27393902
  52. 52. Day LE, Miller-Day M, Hecht ML, Fehmie D. Coming to the new D.A.R.E.: A preliminary test of the officer-taught elementary keepin’ it REAL curriculum. Addictive Behaviors. 2017 Nov 1;74:67–73. pmid:28595059
  53. 53. Bachman JG, Johnston LD, O’Malley PM, Humphrey RH. Explaining the recent decline in marijuana use: differentiating the effects of perceived risks, disapproval, and general lifestyle factors. J Health Soc Behav. 1988 Mar;29(1):92–112. pmid:3367032
  54. 54. Collins D, Abadi MH, Johnson K, Shamblen S, Thompson K. Non-medical use of prescription drugs among youth in an Appalachian population: prevalence, predictors, and implications for prevention. J Drug Educ. 2011;41(3):309–26. pmid:22125924
  55. 55. Arria AM, Caldeira KM, Vincent KB, O’Grady KE, Wish ED. Perceived harmfulness predicts nonmedical use of prescription drugs among college students: Interactions with sensation-seeking. Prev Sci. 2008 Sep;9(3):191–201. pmid:18633709
  56. 56. Ford JA, Rigg KK. Racial/Ethnic differences in factors that place adolescents at risk for prescription opioid misuse. Prev Sci. 2015 Jul;16(5):633–41. pmid:25344348
  57. 57. Voepel-Lewis T, Boyd CJ, McCabe SE, Zikmund-Fisher BJ, Malviya S, Grant J, et al. Deliberative Prescription Opioid Misuse Among Adolescents and Emerging Adults: Opportunities for Targeted Interventions. J Adolesc Health. 2018 Nov;63(5):594–600. pmid:30348282
  58. 58. Votaw VR, Wittenauer J, Connery HS, Weiss RD, McHugh RK. Perceived Risk of Heroin Use among Nonmedical Prescription Opioid Users. Addict Behav. 2017 Feb;65:218–23. pmid:27544695
  59. 59. Skenderian JJ, Siegel JT, Crano WD, Alvaro EE, Lac A. Expectancy Change and Adolescents’ Intentions to Use Marijuana. Psychology of Addicive Behavior. 2008;22(4):563–9. pmid:19071982
  60. 60. Callas PW, Flynn BS, Worden JK. Potentially modifiable psychosocial factors associated with alcohol use during early adolscence. Addictive Behaviors. 2004;29:1503–15.
  61. 61. Goldstein AP, McGinnis E. Skillstreaming the Adolescent: New Strategies and Perspectives for Teaching Prosocial Skills. Revised Edition. [Internet]. Research Press; 1997 [cited 2023 Mar 27]. Available from: https://eric.ed.gov/?id=ed416608
  62. 62. Kelly JA, St. Lawrence JS, Hood HV, Brasfield TL. An Objective Test of AIDS Risk Behavior Knowledge: Scale Development, Validation, and Norms. Journal of Behavior Therapy and Experimental Psychiatry. 1989;30(3):227–34. pmid:2632590
  63. 63. Rickwood D, Deane FP, Wilson CJ, Ciarrochi J. Young people’s help-seeking for mental health problems. Australian e-Journal for the Advancement of Mental Health. 2005;4(3, Supplement).
  64. 64. Cramer K. Mental Health Help Seeking in Schools: The Impact of Mental Health Literacy, Sitgma, and Barriers to Care [Dissertation]. University of California Riverside; 2016.
  65. 65. Wilson CJ, Deane FP, Ciarrochi J, Rickwood D. Measuring Help-Seeking Intentions: Properties of the General Help-Seeking Questionnaire. Canadian Journal of Counselling. 2005;39(1):15–28.
  66. 66. Vogel DL, Wade NG, Haake S. Measuring the Self-Stigma Associated with Seeking Psychological Help. Journal of Counseling Psychology. 2006;53(3):325–37.
  67. 67. Ægisdóttir S, Gerstein LH. Beliefs About Psychological Services (BAPS): development and psychometric properties. Counselling Psychology Quarterly. 2009;22(2):197–219.
  68. 68. Hintze J. (2008). PASS 2008. NCSS, LLC. Kaysville, Utah, USA. www.ncss.com.
  69. 69. Kann L. Youth Risk Behavior Surveillance—United States, 2017. MMWR Surveill Summ [Internet]. 2018 [cited 2022 Nov 30];67. Available from: https://www.cdc.gov/mmwr/volumes/67/ss/ss6708a1.htm
  70. 70. MacKinnon DP. Analysis of mediating variables in prevention and intervention research. NIDA Res Monogr. 1994;139:127–53. pmid:8742555
  71. 71. Sobel ME. Asymptotic Confidence Intervals for Indirect Effects in Structural Equation Models. Sociological Methodology. 1982;13:290–312.
  72. 72. Kraemer HC. A mediator effect size in randomized clinical trials. Int J Methods Psychiatr Res. 2014 Jun;23(4):401–10. pmid:24942819
  73. 73. HEAL Initiative; National Institutes of Health [Internet]. HEAL Public Access and Data Sharing. [cited 2023 Jul 5]. Available from: https://heal.nih.gov/data/public-access-data
  74. 74. National Conference of State Legislatures [Internet]. Prescribing Policies: States Confront Opioid Overdose Epidemic. [cited 2022 Dec 2]. Available from: https://www.ncsl.org/research/health/prescribing-policies-states-confront-opioid-overdose-epidemic.aspx
  75. 75. Davis JP, Prindle JJ, Eddie D, Pedersen ER, Dumas TM, Christie NC. Addressing the opioid epidemic with behavioral interventions for adolescents and young adults: A quasi-experimental design. J Consult Clin Psychol. 2019 Oct;87(10):941–51. pmid:31556670
  76. 76. Evans R, Widman L, Javidi H, Adams ET, Cacace S, Prinstein MJ, et al. Preliminary Evaluation of a Prescription Opioid Misuse Prevention Program Among Rural Middle School Students. J Community Health. 2020 Dec 1;45(6):1139–48. pmid:32785872
  77. 77. Patry E, Bratberg JP, Buchanan A, Paiva AL, Balestrieri S, Matson KL. Rx for addiction and medication safety: An evaluation of teen education for opioid misuse prevention. Res Social Adm Pharm. 2019 Aug;15(8):917–24. pmid:30076091
  78. 78. García-Carrión R, Villarejo-Carballido B, Villardón-Gallego L. Children and Adolescents Mental Health: A Systematic Review of Interaction-Based Interventions in Schools and Communities. Frontiers in Psychology [Internet]. 2019 [cited 2022 Dec 2];10. Available from: https://www.frontiersin.org/articles/10.3389/fpsyg.2019.00918
  79. 79. Manicavasagar V, Horswood D, Burckhardt R, Lum A, Hadzi-Pavlovic D, Parker G. Feasibility and effectiveness of a web-based positive psychology program for youth mental health: randomized controlled trial. J Med Internet Res. 2014 Jun 4;16(6):e140. pmid:24901900
  80. 80. Feiss R, Dolinger SB, Merritt M, Reiche E, Martin K, Yanes JA, et al. A Systematic Review and Meta-Analysis of School-Based Stress, Anxiety, and Depression Prevention Programs for Adolescents. J Youth Adolesc. 2019 Sep;48(9):1668–85. pmid:31346924
  81. 81. Saulsberry A, Marko-Holguin M, Blomeke K, Hinkle C, Fogel J, Gladstone T, et al. Randomized Clinical Trial of a Primary Care Internet-based Intervention to Prevent Adolescent Depression: One-year Outcomes. J Can Acad Child Adolesc Psychiatry. 2013 May;22(2):106–17. pmid:23667356
  82. 82. Garrido S, Millington C, Cheers D, Boydell K, Schubert E, Meade T, et al. What Works and What Doesn’t Work? A Systematic Review of Digital Mental Health Interventions for Depression and Anxiety in Young People. Frontiers in Psychiatry [Internet]. 2019 [cited 2022 Nov 30];10. Available from: https://www.frontiersin.org/articles/10.3389/fpsyt.2019.00759