https://orcid.org/0000-0003-1446-3017
Figures
Abstract
Background
Prescription Drug Monitoring Programs (PDMPs) are electronic databases that track controlled substance prescriptions in a state. They are underused tools in preventing opioid abuse. Most PDMP education research measures changes in knowledge or confidence rather than behavior.
Objective
To evaluate the impact of online case-based training on healthcare provider use of the Maryland (USA) PDMP.
Methods
We used e-mail distribution lists to recruit providers to complete a brief educational module. Using a pre-training and post-training survey in the module, we measured self-reported PDMP use patterns and perceived PDMP value in specific clinical situations and compared pre- and post-training responses. Within the module, we presented three fictional pain cases and asked participants how they would manage each, both before, and then after presenting prescription drug history simulating a PDMP report. We measured changes in the fictional case treatment plans before and after seeing prescription history. Finally, we measured and compared how often each participant accessed the Maryland PDMP database before and after completing the educational module. We used multivariate logistic regression to measure the effect of the intervention on actual PDMP use frequency.
Results
One hundred and fifty participants enrolled and completed the training module, and we successfully retrieved real-world PDMP use data of 137 of them. Participants’ decisions to prescribe opioids changed significantly after reviewing PDMP data in each of the fictional cases provided in the module. In the months following the training, the rate of PDMP use increased by a median of four use-cases per month among providers in practice for less than 20 years (p = 0.039) and two use-cases per month among infrequent opioid prescribers (p = 0.014).
Citation: Adenaiye OO, Zirpoli JB, Tan M, Day BF, Bolaji O, Mitchell CS, et al. (2022) Impact evaluation of a brief online training module on physician use of the Maryland, USA, Prescription Drug Monitoring Program. PLoS ONE 17(8): e0272217. https://doi.org/10.1371/journal.pone.0272217
Editor: Mukhtiar Baig, King Abdulaziz University Faculty of Medicine, SAUDI ARABIA
Received: September 6, 2021; Accepted: July 11, 2022; Published: August 9, 2022
Copyright: © 2022 Adenaiye et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: We have uploaded the data onto the Open Science framework. Here is the URL link: https://osf.io/75zux/.
Funding: This project was funded by the Maryland Department of Health (MDH) Occupational Health and Safety Surveillance project (CDC Cooperative Agreement 5 U60OH011154)) and Overdose Data to Action project (CDC Cooperative Agreement NU17CE924961). The views expressed do not necessarily represent those of the U.S. Centers for Disease Control and Prevention or the Maryland Department of Health. The Maryland Department of Health participated in the conceptualization of this project. We acknowledge the biostatistics support of the University of Maryland, Baltimore, Institute for Clinical & Translational Research (ICTR) and the National Center for Advancing Translational Sciences (NCATS) Clinical Translational Science Award (CTSA) grant number 1UL1TR003098.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Opioid prescribing has been declining since 2012 in the United States and in Maryland [1], but opioid overprescribing continues to contribute to the wicked public health problem of opioid overdose deaths [2–5]. The state of Maryland had decreases in prescription opioid-related deaths from 2016 through 2018, but no change from 2018 to 2019 [6].
Many studies have shown an increased risk for long-term work disability in patients prescribed opioids in workers’ compensation (WC) injuries, with associated increased costs [7–10]. The Workers’ Compensation Research Institute (WCRI) reviewed data from 28 states on low back injuries that had at least seven days away from work between 2008 and 2013 and found three times as much temporary disability in workers prescribed opioids as in similarly injured workers who did not receive any opioid prescriptions [11]. WCRI found that 21% of WC claims in Maryland for lost time injuries occurring between October 1, 2015, and September 30, 2016, received an opioid prescription [12]. States implementing opioid prescription guidelines in WC have seen improved outcomes in work disability and other metrics [13, 14].
Prescription drug monitoring programs (PDMPs) are statewide electronic databases that track prescriptions for controlled substances. They are intended to improve safe prescribing of opioids by identifying patients who may be misusing opioids or other controlled prescription drugs and those at increased risk for opioid overdose [15]. Maryland established its PDMP in 2014 and requires all clinicians who may prescribe controlled drugs to register for access. In July 2018, Maryland mandated that providers check the PDMP before prescribing a controlled drug to a patient [16]. Maryland opioid prescription rates had already started declining before the PDMP was launched [17]. Fig 1 illustrates opioid prescription trends in Maryland in relation to the launch of the PDMP in 2014 and the mandate implementation in 2018.
Maryland opioid prescription rate in relation to PDMP milestones [17].
Several studies of PDMPs have noted population-level benefits following PDMP implementation, including decreased opioid prescribing [18–20], opioid misuse [21], opioid-related deaths [20, 22], and patients seeing multiple prescribers for the same drug [20, 21]. However, others have failed to demonstrate a clear benefit [23–25] and even suggested potential unintended harm of increasing heroin use [23]. Studies have shown that providers feel PDMPs are beneficial [26, 27] but are often difficult to use [19, 27, 28]. Interventions to improve provider use of PDMPs have suggested some improved PDMP knowledge with educational interventions done face-to-face [29], through interactive webcasts [30], or online slide presentations [31, 32]. One recent study on the effectiveness of academic detailing, involving a one-to-one meeting between a trained pharmacist and primary care providers, demonstrated increased PDMP use in the 6-month follow-up period [33]. A recent systematic review on barriers to PDMP use found that the most reported system training barriers were lack of knowledge on how to use the PDMP, lack of access training, lack of training on how to interpret results and communicate results to patients and the lack of consistent guidelines on when to check the PDMP [34].
Research on the effectiveness of different physician education modalities, including online training, has often been limited to measuring increases in knowledge or confidence [35–37]. Assessing training impact on clinician behavior and patient outcomes is more challenging [38]. There is evidence to support the value of more active learning strategies, in which learners interact by making decisions, not just reading, viewing, or listening to content [39]. Simulations involving standardized patients are resource-intensive and logistically challenging for reaching clinical providers in practice. Researchers in medical education are hoping to replicate the success of simulated in-person training using more cost-effective and flexible online interactive training [40, 41].
We developed the current project to evaluate the impact of a brief, case-based module on Maryland prescribing providers’ medical decision making as well as PDMP attitudes, behaviors, and utilization. This module was based on a similar module published by one of the authors (MC) for the (now defunct) online clinician community of practice QuantiaMD in 2015. The module presents fictional case studies with medical decisions to be made before and after receiving PDMP data in each case, inviting learners to reflect on the impact of that information on their own medical decision making, rather than providing a “correct” answer, as does the design of most other online education. The previously published module is no longer available online, but it had a high level of provider engagement and expressed intent to change behavior (https://www.fiercepharma.com/sponsored/how-digital-content-helping-hcps-curb-opioid-epidemic.) The Maryland Department of Health was interested in testing a similar module, with a focus on WC care, in a way that would permit measurement of change in attitudes and behavior of participants. We hypothesized that this training module would increase participant recognition of the indications for using the PDMP and increase participant use of the PDMP in practice.
Materials and methods
Module development and launch
We developed the educational module and published it in Sharable Content Object Reference Model (SCORM) format, using the e-learning authoring tool iSpring Suite 9.0) [42]. We modified the module to include content about the Maryland PDMP and to use work-related condition scenarios in the cases. Maryland PDMP program managers reviewed the content about the PDMP for accuracy. University of Maryland School of Medicine faculty and residents beta-tested the module prior to launch.
The module included:
- A baseline questionnaire capturing minimal practice demographic information; self-reported WC services, opioid prescription frequency, and PDMP utilization; attitudes about the value of the PDMP in specific clinical situations; and self-reported PDMP use behavior in these clinical situations.
- Orientation to the Maryland PDMP.
- Three fictional cases featuring painful WC clinical situations, with clinical decisions (including opioid prescription) to be made before and after receiving PDMP data.
- A post-training questionnaire measuring attitudes about the value of the PDMP in specific clinical situations, and planned use of the PDMP in those situations.
- Optional provision of e-mail address if willing to complete a follow-up survey.
A link to the module and the survey questions are found in the Supplemental Materials.
Approvals
We obtained approval for human subjects’ research, with a waiver of signed consent, from the Institutional Review Boards of the University of Maryland Baltimore (HM-HP-00076603-2) and the Maryland Department of Health (MDH/BHA 17–75). We obtained continuing medical accreditation for this module from MedChi, the Maryland Medical Society.
Sample size calculations
Using G-power® [43], we estimated the needed sample size from results from a similar educational module administered to 16,861 clinicians in an online community of practice. The proportion of participants who said they would use the PDMP increased from 33% at baseline to 55% post-module. Hence, to detect a difference of δ = p1 – p0 = 0.22 (where p1 and p0 = proportions of subjects with an intention to use the PDMP post-training and pre-training respectively), with 80% power and 95% confidence level, we estimated we would need a sample of at least size n = 116 [44].
Recruitment
Eligibility criteria were having a license to practice/prescribe in Maryland and having a Maryland Controlled Dangerous Substances (CDS) registration. We recruited Maryland health care providers via e-mail using:
- Blast messages from the Maryland State Medical Society to its member list
- Announcements to the University of Maryland Medical System physician list
- Messages to members of Maryland medical societies
- Notices to members of the Maryland Workers’ Compensation Education Association
- Outreach via contacts at local hospitals to send notice to their clinical staff.
An e-mail message directed potential participants to the project landing page, which described the study, including planned use of data and protections. By clicking on a link that directed a participant to register for the training module, a participant consented to use of their data collected within the module and to the research team collecting information on their use of the PDMP before and after taking the training. We launched the training module on iSpring Learn (https://www.ispringsolutions.com/support/learn), a subscription online learning management system (LMS), which supported registration, delivery of the module, bookmarking, tracking progress and completion, and generation of a certificate for Continuing Medical Education upon completion. Participants could complete the module, which took about 30 minutes to complete including the questionnaires, at their discretion, using mobile technology if they wished. We sent out reminders to participants who started but did not complete the training. Recruitment continued for 10 months and we compensated participants with continuing medical education credits.
Data collection
The LMS collected all responses entered by participants in the module, including the baseline questionnaire, treatment decisions in the fictional cases, and the post-training survey. We began recruitment and data collection from the module in June 2018 and ended it in March 2019. We retrieved participants’ 6-month pre-training and up to 6-months post-training PDMP use data from the Maryland PDMP database of Chesapeake Regional Information System for our Patients (CRISP) using their CDS numbers. Other sources of data were publicly available registries of Maryland physicians and reports on registration for the Maryland PDMP [45] and responses to a post-training follow-up survey sent to a subset of participants who provided their e-mail addresses for this purpose. Fig 2 describes the collection of data used for analysis within and external to the module.
Data processing and statistical methods
We categorized the participants completing the module based on the self-reported frequency of opioid prescribing and PDMP use as “frequent” (several times per day; several times per week; about once a week) or “infrequent” (a few times a month; about once a month; a few times per year or less), as well as on self-reported delivery of WC related services in the previous year. We used McNemar’s test to compare participants’ responses to the survey before and after the module and their treatment plan responses in the fictional cases.
We used chi-square test to compare demographic differences in PDMP registration status between participants and licensed Maryland physicians and to compare participants’ responses to demographic questions.
For the participants with PDMP use data, we tested the differences between the rate of PDMP use up to six months before and six months after taking the module using Wilcoxon signed-rank test. To control for the effect of the mandate requiring providers to check the PDMP when prescribing controlled substances, we excluded from the analyses participants who took the training before the mandate implementation date. We re-computed the analyses on a subgroup of participants who took the training at least six months following the mandate. We ran multi-model logistic regression analyses to determine the odds of observing a change in the rate of PDMP use (i.e., an increase of at least one-use case per month) following training. We used a backward elimination model selection method to select the most predictive model.
Statistical significance was assessed using two-tailed tests at a significance level of 0.05. We used R software version 3.6.3 (RStudio, Inc., Boston, MA, USA) for data management and both R and Statistical Analysis Software (SAS) version 9.3 (Cary, NC, USA) for statistical analyses [46].
Results
Of the 178 eligible participants who started the module, 150 (87%) completed the training module. We retrieved the real-world PDMP use data for 137 (91%) of these 150 participants for the period of 6-months pre-training and up to 6-months post-training (Fig 3). We excluded the 13 participants whose PDMP use data we could not match from the analysis of changes in using the PDMP following the training.
One hundred forty-five (97%) were physicians (defined as physicians, dentists, or podiatrists); four were nurse practitioners and one was a physician assistant. Most of the participants (57%) reported that they had been in practice for over 20 years; 34% of these identified as frequent opioid prescribers. Twenty-one (14%) participants reported they had been in practice for up to five years; 15 (10%) reported they had been in practice from 6–10 years and 28 (19%) reported 11–20 years of medical practice. We observed no significant difference in the years of experience between participants in our study population and Maryland licensed physicians, using the date of first medical registration as a surrogate for date entering practice.
Of the 150 participants, 24 reported they had not yet registered to use the PDMP at baseline; of these, eight reported their application was pending approval. However, we retrieved PDMP use data for 137 (91%) of the 150 participants, suggesting that some participants did not recall or realize that they had registered. The proportion of our study population with PDMP registration was comparable to that of the overall Maryland physician population (91% vs 86.6%, p = 0.37) [45]. We found no significant difference between frequent and infrequent opioid prescribers based on their PDMP registration status (90% vs. 80%; p<0.17). However, frequent prescribers were more likely than infrequent prescribers to report having ever used the PDMP (88% vs. 53%; p<0.05). Also, frequent opioid prescribers were more likely to report checking the PDMP frequently (73% vs. 29%; p<0.05). WC providers were more likely to have ever used PDMP than those who did not provide WC services in the previous year (77% vs. 49%; p<0.05). The remaining characteristics of the participants are summarized in Table 1. We presented five different clinical situations and measured agreement with importance of checking the PDMP in those situations. We then asked about PDMP behavior in the same clinical situations. Less than 50% of the study population reported checking the PDMP in all the clinical circumstances that they agreed they should.
For each of the three fictional cases presented in the training module, participants’ opioid prescription treatment decisions changed significantly following the divulgence of the PDMP data (Fig 4 and S1 Table in S1 File).
We checked for differences between frequent and infrequent prescribers, and WC and non-WC providers, based on their opioid prescription decisions within the three specific fictional cases and found significant differences in both groups in case 1, a low back injury case. We found WC providers were also significantly more likely to revise their opioid prescription plan based on PDMP information in case 3, involving lost medication for a chronic pain condition (S2 Table in S1 File).
Based on the pre- and post-training surveys administered within the training module, there was a significant increase in the proportion of participants who considered it important to use the PDMP in most of the situations presented (Table 2 and S3 Table in S1 File). Post-training, 69% of all participants believed it was important to use the PDMP when seeing new patients (vs. 27% at baseline), and 126 (84%) indicated they would use the PDMP more frequently. Only four participants (2.7%) indicated that they did not routinely check the PDMP at baseline, and two (1.3%) indicated they did not plan to check the PDMP after the training. Nineteen participants (12.7%) reported checking the PDMP for current patients at least once at baseline, and 80 (53.3%) said they planned to do so at the end of training (p<0.001). Other baseline versus immediate-post-training self-reported changes are shown in Table 2.
Participants who did not access the PDMP at all in the six months before or after the training, based on PDMP access data, nevertheless showed a significant change in the intention to use the PDMP at the end of the module (S3 Table in S1 File).
Fifty-three (64%) of the 83 participants who rated the training module in the post-completion survey described it as excellent, 27 (33%) as good, and three (4%) as fair.
Three-month follow-up
In the follow-up survey three months after completion of training data collection, we received responses from 32 of the 59 participants who agreed to follow-up contact by providing their email in the module. Fifteen reported using the PDMP about as much as they had planned to, 12 reported using it less than they had planned, and five reported using it more than they had anticipated. Of the 12 who reported using it less than expected, three indicated that they were prescribing opioids less.
Actual use
We obtained actual use data on 137 of the 150 participants who completed the training. Of the 137, 13 completed the training early–before the PDMP mandate went into effect on July 1, 2018. Of the remaining 124, 30 (24%; 22 WC providers and 17 frequent opioid prescribers) participants used the PDMP frequently, while 59 (47.5%) participants did not use the PDMP in the six months before or the six months after the training. (Note that PDMP access data does not include potential access to PDMP data within organizational electronic health records.)
We ran a Wilcoxon signed-rank test on the 65 participants who took the training and used the PDMP after the mandate, for paired comparisons of use patterns. We excluded from this analysis the 13 (9.5%) participants who completed the training before the July 1, 2018 mandate and the 59 (47.5%) who never accessed the PDMP before or after the training. Of the 59 participants who never accessed the PDMP throughout the study period, 14 (24%) were self-reported frequent opioid prescribers, 28 (48%) were WC providers and 21 (36%) had been in practice for more than 20 years. Twelve of the 13 (92%) participants who took the training before the mandate were WC providers, three (23%) had been in practice for more than 20 years, and nine (69%) were frequent opioid prescribers.
The median rate of PDMP use for the 65 participants included was five per month in the pre-intervention period and 10 per month in the six-month post-intervention period. The rate of PDMP use increased in 39 of the 65 participants in the post-intervention period. The pre-training versus post-training analyses showed a significant increase in the rate of PDMP use in the months following the completion of the module compared to the months before the module in infrequent opioid prescribers (median increase: two per month; p = 0.014). Participants in practice less than 20 years showed a median increase of four use-cases per month (p = 0.039). In a sub-group analysis on participants (N = 20), who took the training after the state-implemented mandate had been in place for at least 6 months, there was a significant increase in the rate of PDMP use following the training (median increase:11/month; p = 0.003). However, the state mandate did not have a significant effect on the PDMP use in this sub-group of participants. The remaining results showing the difference in the rate of PDMP use pre- versus post-training are presented in Table 3.
Number of times PDMP was accessed per month before and after training based on participant self-reported characteristics.
Bivariate regression analyses showed a significant increase in the rate of PDMP usage among infrequent prescribers (OR:3.3, p = 0.027), and in participants who have been in practice for less than 20 years (OR: 3.9, p = 0.013) but no significant training impact was found on WC providers. This finding was confirmed in multi-model regression analysis with backward elimination in infrequent providers (OR:3.7, p = 0.041) and in participants in practice for less than 20 years (OR:9.4, p = 0.003). However, a significant training effect was also seen in the multi-model regression analysis for those who provide WC care (OR:6.6, p = 0.020), suggesting that for these providers the effect may have been related to some other factor or factors (S4 Table in S1 File).
Discussion
This study evaluated the effectiveness of a brief online educational intervention in increasing use of the Maryland PDMP. Overall, the largest impact of the training appears to be on infrequent opioid prescribers and providers with less than 20 years of practice. These findings are consistent with those in a recent study on the impact of academic detailing about the PDMP on primary care provider use patterns, with the largest effect on late adopters (those who had not yet registered or used the PDMP months after this was mandated) [33]. Delcher et al. found in their analysis of oxycodone related overdoses in Florida that an increase of one PDMP query per provider per month resulted in a decline of oxycodone-caused deaths by 0.229 persons per month [47]. Our study found an increase in PDMP use by a median of two queries per month in the months following the training among infrequent opioid prescribers (p = 0.014) and four queries per month among those in practice less than 20 years (p = 0.039). This suggests that such training can be effective in increasing familiarity and comfort with the PDMP in those who are less familiar with it.
The finding of training benefit observed in the sub-group of participants who enrolled at least six months after the state PDMP mandate had been implemented suggests that the mandate itself may not be sufficient to influence decisions to use the PDMP. Although the conclusions are limited by the small sample size, this finding is consistent with research on the effect of mandates, which showed variations among states that were thought to relate to differences among state laws, including exceptions to the mandate requirements [48].
From the baseline questionnaire, we learned that WC care providers and frequent opioid prescribers reported more frequent PDMP use at baseline, and that there was a large overlap between these categories, with 74% of the frequent opioid prescribers also reporting providing WC services. We expected that those who prescribe opioids more frequently would have more occasion to use the PDMP, a finding noted by Lin et al. in their 2015 Maryland survey of Maryland physicians about the PDMP [27]. At baseline, there was strong agreement in the value of checking the PDMP in all five types of clinical situations presented with no difference at baseline among our participant groups. An important finding is a universal discrepancy between positive baseline PDMP attitudes and self-reported baseline PDMP behavior in study participants, with a large gap between belief in the value of the PDMP and self-reported use patterns, which is consistent with the findings of Lin et al. [27]. This suggests that educational efforts directed toward PDMP attitudes may not be useful.
Researchers in the design of medical curricula have suggested that education should address three domains [49–51]:
- Qualification refers to giving students the knowledge, skills, and understanding needed to “qualify” as competent practitioners.
- Socialization refers to practice norms, values, expectations, and specialty or cultural context.
- Subjectification refers to the aim for learners to end up as subjects, able to draw their own conclusions and make their own decisions, and relates to the key educational ideas of agency, autonomy, and responsibility.
The educational module in this study was designed to demonstrate to the learner the value of PDMP information in clinical case scenarios. We presented three realistic fictional cases about painful work-related injuries and asked study participants to make hypothetical clinical decisions. We then provided participants additional information from the PDMP in each case and asked each participant how this new information impacted their decision. There were no specific right or wrong answers in this module. Using this approach, we found statistically significant differences in the clinical decisions that participants made about opioid prescribing in each case before and after reviewing the PDMP information. By inviting participants to adjust their own clinical decisions based on data provided, rather than providing right or wrong answers, our educational approach aligns with the medical education domain of “subjectification” described by Biesta, which is often neglected in online medical education that focuses on the acquisition of knowledge [49–51].
In the fictional cases, both frequent prescribers and WC provider participants were more likely to modify their opioid treatment decisions after seeing PDMP data in the case involving acute low back pain, the bread-and-butter injury of WC care (S2 Table in S1 File). Given the high risk of poor outcomes in patients given inappropriate opioids for acute musculoskeletal injuries in WC care, this finding supports using a similar educational strategy focused on preventing early opioid prescriptions when not clinically indicated, targeted toward WC providers.
Strengths
It is notoriously difficult to measure the actual impact of education on practice behaviors. We designed this study to also look at patterns of accessing the PDMP by participants before and after taking the training module, to measure any actual changes in PDMP use, one of the strengths of this study. This helped us overcome the well-documented propensity to overestimate compliance with best practices in self-reported behavior [52–56]. Participants acting as their own controls in the analyses involving comparisons before and after the module and in the case decision-making, is also a strength of the study design.
Limitations
Our PDMP data did not capture institutional use (where PDMP data is presented when accessing organizational electronic health records), and we did not capture which participants were able to see PDMP data this way, so it is possible that our PDMP use data missed these participants accessing the PDMP, and therefore missed any changes in their use patterns. There was no attempt to randomly sample participants, therefore self-selected participants may not be representative of Maryland prescribing providers. Medical decisions made in fictional cases may not accurately reflect decisions made in real cases and we did not have access to data on opioid prescribing before or after the training. Because our initial plan to recruit WC care providers (a population of particular interest) from an insurance contact file was unsuccessful, we modified our protocol to query about WC practice, so that we could analyze this variable. The single question about providing WC care in the last year may not have effectively categorized WC care providers. We did not receive enough follow-up survey responses for meaningful comparisons in self-reported use after the training.
We adjusted for the potential interaction of the mandate instituted by the State of Maryland on July 1, 2018 by excluding participants who took the training before the mandate from the analyses and re-computed the analyses on a subgroup of participants who took the training at least 6 months following the mandate. We also excluded from the final analyses, subjects who never used PDMP any time 6 months before the training or in the 6 months after the training. Although we did not identify any difference between the excluded participants and the remaining study population, we cannot be certain whether these participants differ in any important way from those included in the analyses. This exclusion potentially limits the external validity of our study.
Conclusion
This study demonstrated the effectiveness of a brief, engaging, online educational module on PDMP use by participants who were less familiar with the PDMP. This innovative “gradual reveal” method engages learners as their own teachers, incorporating additional information from the PDMP and revising their planned treatment in a way that promotes adherence to treatment guidelines that improve patient safety and clinical outcomes. Given that even modest increases in PDMP queries are associated with decreases in prescription opioid overdose deaths, the ability to shift the practice patterns of large numbers of prescribing providers using inexpensive online technology would have significant public health implications. This approach should be considered in future education aimed at decreasing early opioid prescribing in acute WC care, and for future educational strategies aimed at new PDMP registrants or physicians at an earlier phase in their careers. Different strategies may be needed for physicians who have been in practice for more than 20 years or who have already established specific practice patterns in using the PDMP. Online delivery of brief education demonstrating the value of PDMP data in clinical cases would be a cost-effective alternative to the more labor-intensive approach of academic detailing, which requires close to an hour of trainer time for each provider trained. Future research should address the feasibility, cost-effectiveness and impact of reaching larger groups of providers using online PDMP training, as well as methods for reaching providers who have more familiarity with the PDMP.
Supporting information
S1 File. Instruments used in this study, including module link, quiz questions and questionnaires.
Additional results tables.
https://doi.org/10.1371/journal.pone.0272217.s001
(DOCX)
Acknowledgments
We acknowledge the support of the Maryland Department of Health, Office of Provider Engagement and Regulation, as well as Rhonda Moody of the Chesapeake Regional Information System for our Patients (CRISP), which manages the Maryland PDMP, for providing data on the PDMP access patterns. We acknowledge the support of Maryland MedChi, which provided continuing medical education accreditation for the module and also supported recruitment efforts. We acknowledge the support of the University of Maryland Baltimore Institute for Clinical Translational Research, which provided biostatistics consultation.
References
- 1. Centers for Disease Control and Prevention. U.S. Opioid Dispensing Rate Maps [Internet]. 2021 [cited 2022 Jul 21]. Available from: https://www.cdc.gov/drugoverdose/maps/rxrate-maps.html
- 2. Guy GP, Zhang K, Schieber LZ, Young R, Dowell D. County-level opioid prescribing in the United States, 2015 and 2017. JAMA Intern Med. 2019;179(4):574–6. pmid:30742206
- 3. Centers for Disease Control and Prevention. Opioid Overdose: Understanding the Epidemic [Internet]. 2021 [cited 2022 Jul 21]. Available from: https://www.cdc.gov/drugoverdose/epidemic/index.html
- 4. Lee JC. The opioid crisis is a wicked problem. Am J Addict. 2018;27(1):51. pmid:29283484
- 5.
National Institute on Drug Abuse. Overdose Death Rates [Internet]. 2021 [cited 2022 Jul 21]. Available from: https://nida.nih.gov/research-topics/trends-statistics/overdose-death-rates
- 6. Centers for Disease Control and Prevention. Prescription Opioid Overdose Death Maps [Internet]. 2021 [cited 2022 Jul 21]. Available from: https://www.cdc.gov/drugoverdose/deaths/prescription/maps.html
- 7. Franklin GM, Stover BD, Turner JA, Fulton-Kehoe D, Wickizer TM. Early opioid prescription and subsequent disability among workers with back injuries: the Disability Risk Identification Study Cohort. Spine. 2008;33(2):199–204. pmid:18197107
- 8. Webster BS, Verma SK, Gatchel RJ. Relationship between early opioid prescribing for acute occupational low back pain and disability duration, medical costs, subsequent surgery and late opioid use. Spine. 2007;32(19):2127–32. pmid:17762815
- 9. Carnide N, Hogg-Johnson S, Koehoorn M, Furlan AD, Côté P. Relationship between early prescription dispensing patterns and work disability in a cohort of low back pain workers’ compensation claimants: a historical cohort study. Occup Env Med. 2019;76(8):573–81.
- 10. Volinn E, Fargo JD, Fine PG. Opioid therapy for nonspecific low back pain and the outcome of chronic work loss. Pain. 2009;142(3):194–201. pmid:19181448
- 11. Neumark D, Savych B, Lea R. The Impact of Opioid Prescriptions on Duration of Temporary Disability [Internet]. Workers Compensation Research Institute. 2018 [cited 2022 Jul 21]. Available from: https://www.wcrinet.org/reports/the-impact-of-opioid-prescriptions-on-duration-of-temporary-disability
- 12.
Wang D, Thumula V, Liu T-C. Interstate Variations in Dispensing of Opioids, 5th Edition [Internet]. Workers Compensation Research Institute. 2019 [cited 2022 Jul 21]. Available from: https://www.wcrinet.org/reports/interstate-variations-in-dispensing-of-opioids-5th-edition
- 13. Phillips AL, Thiese MS, Freeman M, Kartchner R, Hegmann KT. MD Implementation of an Opioid Guideline Impacts on Opioid Prescriptions, Adverse Outcomes, and an Association with a State Opioid-Related Fatalities. J Occup Environ Med. 2019;61(8):653–8. pmid:31348425
- 14. Sears JM, Haight JR, Fulton-Kehoe D, Wickizer TM, Mai J, Franklin GM. Changes in early high-risk opioid prescribing practices after policy interventions in Washington State. Health Serv Res. 2021;56(1):49–60. pmid:33011988
- 15. Centers for Disease Control and Prevention. Opioid Overdose: What Healthcare Providers Need to Know about PDMPs [Internet]. 2021 [cited 2022 Jul 21]. Available from: https://www.cdc.gov/drugoverdose/pdmp/providers.html
- 16. Maryland Department of Health. PDMP Use Mandate Information [Internet]. 2021 [cited 2022 Jul 21]. Available from: https://health.maryland.gov/pdmp/Pages/pdmp-use-mandate-information.aspx
- 17. Centers for Disease Control and Prevention. U.S. State Opioid Dispensing Rates, 2006 [Internet]. 2021 [cited 2022 Jul 21]. Available from: https://www.cdc.gov/drugoverdose/maps/rxstate2006.html
- 18. Wen H, Hockenberry JM, Jeng PJ, Bao Y. Prescription Drug Monitoring Program Mandates: Impact On Opioid Prescribing And Related Hospital Use. Health Aff (Millwood). 2020;38(9):1550–6.
- 19. Rutkow L, Turner L, Lucas E, Hwang C, Alexander GC. Most primary care physicians are aware of prescription drug monitoring programs, but many find the data difficult to access. Health Aff (Millwood). 2015;34(3):484–92. pmid:25732500
- 20. Centers for Disease Control and Prevention. Opioid Overdose: State Successes [Internet]. 2017 [cited 2022 Jul 21]. Available from: https://www.cdc.gov/drugoverdose/policy/successes.html
- 21. Ali MM, Dowd WN, Classen T, Mutter R, Novak SP. Prescription drug monitoring programs, nonmedical use of prescription drugs, and heroin use: Evidence from the National Survey of Drug Use and Health. Addict Behav. 2017;69:65–77. pmid:28152391
- 22. Patrick SW, Fry CE, Jones TF, Buntin MB. Implementation of prescription drug monitoring programs associated with reductions in opioid-related death rates. Health Aff (Millwood). 2016;35(7):1324–32. pmid:27335101
- 23. Fink DS, Schleimer JP, Sarvet A, Grover KK, Delcher C, Castillo-Carniglia A, et al. Association between prescription drug monitoring programs and nonfatal and fatal drug overdoses: A systematic review. Ann Intern Med. 2018;168(11):783–90. pmid:29801093
- 24. Brown R, Riley MR, Ulrich L, Kraly EP, Jenkins P, Krupa NL, et al. Impact of New York prescription drug monitoring program, I-STOP, on statewide overdose morbidity. Drug Alcohol Depend. 2017;178(May):348–54. pmid:28692945
- 25. Brady JE, Wunsch H, DiMaggio C, Lang BH, Giglio J, Li G. Prescription drug monitoring and dispensing of prescription opioids. Public Health Rep. 2014;129(2):139–47. pmid:24587548
- 26. Gershman JA, Gershman JA, Fass AD, Popovici I. Evaluation of Florida physicians’ knowledge and attitudes toward accessing the state prescription drug monitoring program as a prescribing tool. Pain Med. 2014;15(12):2013–9. pmid:24931295
- 27. Lin DH, Lucas E, Murimi IB, Jackson K, Baier M, Frattaroli S, et al. Physician attitudes and experiences with Maryland’s prescription drug monitoring program (PDMP). Addiction. 2016;112(2):311–9. pmid:27658522
- 28. Blum CJ, Nelson LS, Hoffman RS. A survey of physicians’ perspectives on the New York State mandatory prescription monitoring program (ISTOP). J Subst Abuse Treat. 2016;70:35–43. pmid:27692186
- 29. Barth KS, Ball S, Adams RS, Nikitin R, Wooten NR, Qureshi ZP, et al. Development and feasibility of an academic detailing intervention to improve prescription drug monitoring program use among physicians. J Contin Educ Health Prof. 2017;37(2):98–105. pmid:28562498
- 30. Finnell JT, Twillman RK, Breslan SA, Schultz J, Miller L. The Role of Continuing Medical Education in Increasing Enrollment in Prescription Drug Monitoring Programs. Clin Ther. 2017;39(9):1896–1902.e2. pmid:28865800
- 31. Martello J, Cassidy B, Mitchell A. Evaluating Emergency Department Opioid Prescribing Behaviors After Education About Mandated Use of the Pennsylvania Prescription Drug Monitoring Program. J Addict Nurs. 2018;29(3):196–202. pmid:30180006
- 32. Kellams J, Maye J. The Last State to Grant Nurse Practitioners DEA Licensure: An Education Improvement Initiative on the Florida Prescription Drug Monitoring Program. J Addict Nurs. 2017;28(3):135–42. pmid:28863055
- 33. Ball SJ, McCauley JA, Pruitt M, Zhang J, Marsden J, Barth KS, et al. Academic detailing increases prescription drug monitoring program use among primary care practices. J Am Pharm Assoc (2003). 2021 Jul-Aug;61(4):418–424. pmid:33812783
- 34. Martin HD, Modi SS, Feldman SS. Barriers and facilitators to PDMP IS Success in the US: A systematic review. Drug Alcohol Depend. 2021;219. pmid:33387937
- 35. Harris JM Jr, Kutob RM, Surprenant ZJ, Maiuro RD, Delate TA. Can Internet-based education improve physician confidence in dealing with domestic violence? Fam Med. 2002;34(4):287–92. pmid:12017143
- 36. Williams JG. Are online learning modules an effective way to deliver hand trauma management continuing medical education to emergency physicians? Plast Surg. 2014;22(2):75–8. pmid:25114616
- 37. Lee S, Young I, Colletti J. Online Module to Improve Emergency Department Observation Unit Practice. MedEdPORTAL. 2016 Jul 8;12:10423. pmid:31008203
- 38. Pisani AR, Cross WF, Gould MS. The assessment and management of suicide risk: state of workshop education. Suicide Life Threat Behav. 2011;41(3):255–76. pmid:21477093
- 39. Mansouri M, Lockyer J. A meta-analysis of continuing medical education effectiveness. J Contin Educ Health Prof. 2007;27(1):6–15. pmid:17385735
- 40. Carney PA, Dietrich AJ, Jr DHF, Mott LA. A standardized-patient assessment of a continuing medical education program to improve physicians’ cancer-control clinical skills. Acad Med. 1995;70(1):52–8. pmid:7826445
- 41. Müller N, Gschwendtner KM, Dwinger S, Bergelt C, Eich W, Härte M, et al. Study protocol of a randomized controlled trial on two new dissemination strategies for a brief, shared-decision-making (SDM) training for oncologists: web-based interactive SDM online-training versus individualized context-based SDM face-to-face training. Trials. 2019;20(18):1–11. pmid:30616653
- 42.
ISpring. iSpring Suite Max [Internet]. 2021 [cited 2022 Jul 21]. Available from: https://www.ispringsolutions.com/ispring-suite
- 43. Erdfelder E, Faul F, Buchner A. GPOWER: A general power analysis program. Behav Res Methods, Instruments, Comput. 1996;1–11.
- 44. Connor RJ. Sample Size for Testing Differences in Proportions for the Paired-Sample Design. Biometrics. 1987; 43(1), 207–211. pmid:3567305
- 45.
Maryland Department of Health. Health-General §21-2A-05(f)(3)–Annual Prescription Drug Monitoring Program Report, January 10, 2019. [Internet]. 2019 [cited 2022 Jul 21]. Available from: https://bha.health.maryland.gov/Documents/HG 21-2A-05(f)(3) - 2018 PDMP Annual Report.pdf
- 46.
SAS Institute I. Base SAS®9.3Procedures Guide Statistical Procedures. Cary, NC: SAS Institute Inc.; 2011.
- 47. Delcher C, Wagenaar AC, Goldberger BA, Cook RL, Maldonado-Molina MM, 2015. Abrupt decline in oxycodone-caused mortality after implementation of Florida’s prescription drug monitoring program. Drug Alcohol Depend. 2015; 150: 63–68. pmid:25746236
- 48. Strickler GK, Zhang K, Halpin JF, Bohnert ASB, Baldwin GT, Kreiner PW. Effects of mandatory prescription drug monitoring program (PDMP) use laws on prescriber registration and use and on risky prescribing. Drug and Alcohol Dependence. 2019; 199:1–9. pmid:30954863
- 49. Biesta GJJ, Braak M van. Beyond the Medical Model: Thinking Differently about Medical Education and Medical Education Research. Teach Learn Med. 2020;32(4):449–56. pmid:32799696
- 50. Biesta G. Risking ourselves in education: qualification, socialisation and subjectification revisited. Educ Theory. 2020;70(1):89–104.
- 51. Biesta G. Can the prevailing description of educational reality be considered complete? On the Parks-Eichmann paradox, spooky action at a distance, and a missing dimension in the theory of education. Pol Futur Educ. 2020; 18(8):1011–1025.
- 52. Saturno PJ, Palmer RH, Gascon JJ. Physician attitudes, self-estimated performance and actual compliance with locally peer-defined quality evaluation criteria. Int J Qual Heal Care. 1999;11(6):487–96.
- 53. Linder JA, Schnipper JL, Tsurikova R, Volk L A, Middleton B. Self-reported familiarity with acute respiratory infection guidelines and antibiotic prescribing in primary care. Int J Qual Heal Care. 2010;22(6):469–75. pmid:20935008
- 54. Rotshtein A, Karkabi K, Geyer O, Castel OC. Primary care physicians’ role perception and self-reported performance in glaucoma care: a survey study. BMC Res Notes. 2015;12(8):776. pmid:26653898
- 55. Theodossiades J, Myint J, Murdoch I, Edgar D, Lawrenson J. Does optometrists’ self-reported practice in glaucoma detection predict actual practice as determined by standardised patients? Ophtalmic Physiol Opt. 2012;32(3):234–41. pmid:22329780
- 56. Helmerhorst HJ, Schultz MJ, van der Voort PH, Bosman RJ, Juffermans NP, Jonge E de, et al. Self-reported attitudes versus actual practice of oxygen therapy by ICU physicians and nurses. Ann Intensive Care. 2014;4(23). pmid:25512878