Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Opioid prescription patterns among patients who doctor shop; Implications for providers

  • Todd Schneberk ,

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Writing – original draft, Writing – review & editing

    tschneberk@gmail.com

    Affiliation Department of Emergency Medicine, Los Angeles County +University of Southern California, Los Angeles, CA, United States of America

  • Brian Raffetto,

    Roles Conceptualization, Data curation, Formal analysis, Writing – original draft, Writing – review & editing

    Affiliation Department of Emergency Medicine, Los Angeles County +University of Southern California, Los Angeles, CA, United States of America

  • Joseph Friedman,

    Roles Conceptualization, Formal analysis, Writing – review & editing

    Affiliation David Geffen School of Medicine at University of California, Los Angeles, CA, United States of America

  • Andrew Wilson,

    Roles Conceptualization, Data curation, Formal analysis, Writing – review & editing

    Affiliations Department of Neurology, Greater Los Angeles VA, Los Angeles, CA, United States of America, Department of Neurology, University of California, Los Angeles, CA, United States of America

  • David Kim,

    Roles Data curation

    Affiliation Department of Emergency Medicine, University of California, Los Angeles, CA, United States of America

  • David L. Schriger

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Supervision, Writing – original draft, Writing – review & editing

    Affiliation Department of Emergency Medicine, University of California, Los Angeles, CA, United States of America

Abstract

Introduction

Patients who doctor shop for opioids are a vulnerable population that present a difficult dilemma for their health care providers regarding best methods of immediate treatment and how to manage their risk of harm from opioids. We aim to describe and compare opioid prescription patterns among high quantity prescription patients who doctor shopped, high quantity prescription patients who did not (doctor shopping eligible patients), and the remaining patients who received opioid prescriptions to guide population health policies for high risk opioid use patients.

Methods

We performed a cross-sectional descriptive analysis of opioid prescriptions during an 8-year period using California’s de-identified Controlled Substance Utilization Review and Evaluation System (CURES) database from years 2008–2015. We identified the prevalence of patients who doctor shopped and depicted their opioid prescription patterns including prescriber characteristics, in comparison to the aforementioned groups. Doctor shopping was defined by patients who received greater than 6 or more prescriptions from at least 6 different prescribers within 6 months of time.

Results

Among the 3 million individuals who received an opioid prescription during the 8-year period, 1.3% met the doctor shopper definition. These patients received high levels of chronic opioids with 82% and 33% averaging greater than 20 and 100 morphine milligram equivalents (MME) daily, respectively, in comparison to 72% and 18% in the doctor shopping eligible group. Patients who doctor shopped received a significant proportion of their MME from 1 main prescriber (54%) and only received 2–5% of their total MME from episodic care providers, despite 88% receiving a prescription from these providers.

Conclusions

Patients who doctor shop are at high risk of opioid use disorder but represent a small fraction of those with dangerous opioid use. Furthermore, these individuals do not receive substantial opioids from episodic providers, which challenges the utility of prescription reduction programs in curbing use among this population. These results suggest we re-evaluate physician roles in the care of these patients and focus on referral to treatment and harm reduction strategies.

Introduction

Prescriptions for opioids nearly quadrupled from 1999–2008, and overdoses followed a similar pattern, resulting in state efforts to identify individuals most at risk for opioid abuse and overdose [13]. The act of “doctor shopping”–receiving opioid prescriptions from multiple providers–was identified as a risk factor for opioid use disorder, overdose, diversion, and higher health risks [4, 5]. As a result, states developed Prescription Drug Monitoring Program (PDMP) databases in part, to flag patients who doctor shop [6]. Recommendations for the healthcare providers who find themselves caring for a patient who is doctor shopping are not clearly defined, and there is potential to increase stigmatization [7]. In addition, paradoxically, there is the potential to produce negative health outcomes as denied patients can pursue opioids through riskier channels or use more lethal forms of opioids [8]. At present, there is evidence that PDMPs may reduce overall opioid prescriptions depending on how they are operationalized, but there is insufficient evidence to determine if PDMPs curtail doctor-shopping behavior, or reduce negative patient centered outcomes, such as opioid overdoses [911].

The scale of the opioid crisis demands well informed, targeted efforts to reduce harm and find opportunities for intervention along the spectrum of opioid use. For this reason, we analyzed the California PDMP database from 2008 to 2015 to describe the opioid prescription pattern[s] of individuals who met criteria for doctor shopping and to compare them to those of individuals who did not meet doctor-shopping criteria. Our goal was to use these empiric data to contextualize the discussion around doctor-shopping behavior and to inform future initiatives to address the opioid epidemic.

Methods

Details about our acquisition and processing of the California’s Controlled Substance Utilization Review and Evaluation System (CURES) database can be found in our prior paper [12]. In brief, we utilized data obtained from CURES—part of the Prescription Drug Monitoring Program (PDMP)–during 2008 to 2015. Using Stata 14.2 (Stata Corp., College Station TX), we developed the following measures for each patient in the database: time between first and last prescription, time receiving opioid medications, mean and median morphine milligram equivalents (MME) per day, mean and median MME per day for the 90-day period of maximal use, and doctor shopping status (defined below). We also identified each patient’s primary opioid prescriber and the category of provider who wrote each prescription. Options for this category included: primary provider, 1-prescription provider (a provider who wrote a single prescription for the patient in the entire 8-year database), 2-prescription provider, and other. Operational definitions of each variable can be found in S1 Appendix.

We based our definition of doctor shopping behavior on California’s PDMP’s Patient Safety Alert criteria: any person who obtains prescriptions from 6 or more prescribers or 6 or more pharmacies during a 6-month period [13]. For this paper we included only those patients who met the provider definition (not multiple pharmacies) as this was our group of interest. We defined a “doctor shopping eligible” comparison group as those patients who could have doctor shopped (they had at least one 183-day period when they received 6 or more prescriptions) but never met the doctor shopping definition. That is to say, during that 183-day period, their 6 or greater prescriptions were not from 6 or more providers, indicating that their opioid prescription source was more uniform. A third category was defined as those who did not meet either definition, and by default had much less prescription opioid use. These categories allowed for comparison of doctor shoppers with those who had similar opioid utilization with a more constant prescription source. It also separated these high use groups from the lower utilization patients who are less likely to be analogous to those who doctor shopped.

In addition to describing mean and median usage we also categorically grouped patients into those who consumed greater than 20 MME/day and those who consumed great than 100 MME/day, overall and during each patient’s 90-day period of maximal use. These categories of use were associated with higher risk of overdose in prior literature [1416].

We used STATA 14.2 (Stata Corp., College Station, TX) to create graphical depictions of cross-tabulations of doctor shopping status and the relevant independent variables. Our purpose is descriptive. With such a large N, we could use formal testing to establish that a host of comparisons are significant. Supported by an increasing literature demonstrating the problems with statistical testing in observation studies, we opted to describe our findings–with appropriate measures of variance–rather than test them [1719]. The study was considered exempt by the UCLA Institutional Review Board.

Results

The 10% random sample from the CURES dataset included 17,954,968 opioid prescriptions written to 3,044,579 patients by 185,424 prescribers. Most prescriptions (87.7%) were for short-acting opioids; hydrocodone (62%) and oxycodone (10.4%) predominated (see S2 Appendix).

Prevalence of different patient categories based on prescription patterns

Of the ≈3 million individuals getting at least 1 opioid prescription during the 8-year study period, 37,333 patients (1.3%) met the doctor shopping definition at some time. Another 298,493 doctor shopping eligible patients (9.7%) received sufficient prescriptions—at least 6 in a 6-month period—to be doctor shopping but did not doctor shop. The remaining 2.7 million patients (89%) did not meet either definition (the “Neither” group). (Fig 1, left bar)

thumbnail
Fig 1. Patients, prescriptions, and Morphine Milligram Equivalent (MME) prescribed, by patients & prescriber characteristics.

https://doi.org/10.1371/journal.pone.0232533.g001

Opioid prescription patterns among patients who doctor shop

Over the 8-year data period, doctor shopping patients received 59 prescriptions during a 4.3-year span (1,563 days) between their first and last prescription (Table 1). They had opioid medication coverage for a mean of 1,216 days (81%) during that span (See S1 Appendix for methods and definitions). Their prescriptions came from a mean 14.7 prescribers. Median daily consumption on days when they had prescriptions for medications was 22.9 MME [IQR 9.9, 53.1] and daily median consumption during the 90-day period of maximum consumption was 59.6 MME [IQR 26.7, 138.5]. 82.1% of patients who doctor shopped had a maximum daily 90-day MME greater than 20; 33.4% exceeded 100 MME/day (Fig 2A). Doctor shopping patients received 83.7% short acting opioid prescriptions; 51.9% received only short acting opioids. Hydrocodone/acetaminophen was the most common (56.6%), followed by oxycodone/acetaminophen (9.1%) and morphine sulfate (6%). 18.9% of patients who doctor shopped received at least 1 prescription for buprenorphine or methadone (Fig 1E). These medications accounted for 32.6% of all long acting prescriptions and 5.4% of all prescriptions written for this group (Fig 1E).

thumbnail
Table 1. Patient characteristics (mean, median [IQR] unless otherwise indicated).

https://doi.org/10.1371/journal.pone.0232533.t001

Comparison of opioid medication patterns between active and inactive doctor-shopping periods

Of the 37,333 patients who met the doctor shopping definition, 3,571 (9.6%) met the doctor shopping definition for their entire time in the database (Table 2). The average doctor shopping patient had a mean 395 days that met the doctor shopping definition and 907 days that did not, doctor shopping for roughly 1/3 of their total time (1,216 days) in the database. These patients received a median 26.9 [IQR 12, 60] MME/day during shopping periods and 17.2 [6, 46] MME/day at other times. Despite median MME per prescription being lower (450 v. 675) during periods of doctor shopping, the increased number of prescriptions per patient per month (median 1.6 v. 0.9) resulted in the increase in daily MME. Short acting opioids comprised 86.1% of prescriptions during doctor shopping periods and 81.9% at other times. The percentage of patients receiving buprenorphine or methadone was similar during doctor shopping and non-doctor shopping periods (13.7% v. 14.2%). Taken together, these data suggest that prescription and MME consumption patterns differ slightly during patients’ doctor shopping and non-doctor shopping periods but both exhibit patterns consistent with chronic opioid use.

thumbnail
Table 2. Characteristics of doctor shopping patients during doctor shopping and non-doctor shopping periods (mean, median [IQR] unless otherwise indicated).

https://doi.org/10.1371/journal.pone.0232533.t002

Prescription patterns differed slightly based on prescriber type during doctor shopping and non-doctor shopping periods. When not doctor shopping, these patients received 8% of prescriptions and 3% of their total MME from one-time prescribers and 56% of their prescriptions and 64% of their total MME from their primary prescriber. When doctor shopping, they received 23% of prescriptions and 9% of their total MME from one-time prescribers and 28% of their prescriptions and 40% of their total MME from their primary provider. These proportions are illustrated in the split horizontal bar in Fig 3, which demonstrates that even when doctor shopping, patients receive a minority of the prescriptions (and MME) from their low frequency prescribers yet maintain a substantial proportion from primary prescribers.

thumbnail
Fig 3.

The above figure depicts absolute (top row) and relative (bottom row) distributions of prescriptions per patient, prescriptions per patient per time (months), MME per patient, and MME per patient per time (days), divided into opioids provided by primary prescribers (purple), 1 prescription prescribers (dark blue), 2 prescription prescribers (light blue) and other prescribers (gray) for the different categories of patients. The top row in each individual horizontal bar graph shows patients who doctor shopped divided into distributions when they met the definition of doctor shopping (yes) and when they did not meet the definition (no). The second row in each horizontal bar graph shows the distributions for patients who doctor shopped as the aggregate of all prescriptions during doctor shopping or not. The third row depicts the distributions of opioid prescriptions for those who were doctor shopper eligible and the fourth row is those who fit neither definition.

https://doi.org/10.1371/journal.pone.0232533.g003

Comparison of demographic and opioid prescription patterns across categories of patients

Doctor shopping patients were more often female (59.8%) than doctor shopping eligible (57.3%) and other patients (56.8%) (Table 1). The doctor shopping patients (median 51 years, IQR [40, 59]) were younger than doctor shopping eligible patients (57 [47, 67]) and slightly older than other patients (49 [33, 64]) (Table 1).

The 89% of patients who neither doctor shopped nor were doctor shopping eligible used opioids quite differently than these two groups (Table 1, Fig 1A). Over the 8-year course of the database, these individuals averaged 2.4 prescriptions, which they received from 1.7 providers. They averaged 3.9 MME per day when they had a prescription and their maximum 90-day daily MME was 5.4 MME/day. 4.3% of these patients had a maximum 90-day MME that was over 20 MME/day. Only 4.2% ever received a prescription for a long acting opioid and only 1% a prescription for buprenorphine or methadone.

The 9.7% of patients who were doctor shopping eligible and the 1.3% who doctor shopped showed generally similar patterns that were distinct from patients described in the prior paragraph. Compared to doctor shopping eligible patients, patients who doctor shopped had more active days in the database (median 1035 v. 667) and received more prescriptions (mean 58.9 v. 31.4) and more prescriptions per month (median 1.2 v. 0.9). They also had more prescribers (mean 14.7 v. 4.6) and more prescribers per month (median 0.42 v. 0.17). While doctor shopping patients received higher lifetime MME (median 22,070 v. 11,025) than doctor shopping eligible patients, this was largely due to their longer active time in the database, as their median MME/prescription was not appreciably higher (median 545 v. 529).

Slightly over 48% of doctor shopping patients received at least 1 prescription for a long acting opioid compared to 33.3% of doctor shopping eligible patients. A similar pattern was seen for opioids used to treat opioid use disorder (methadone, buprenorphine); 18.9% of patients who doctor shopped and 10.2% of doctor shopping eligible patients received at least one of these prescriptions.

The percentage of patients whose maximum 90-day MME exceeded 20 MME/day was higher for doctor shopping patients than doctor shopping eligible patients (82.1 v. 71.7%) as was the percentage who exceeded 100 MME/day (33.4% v. 18.2% (Fig 1B). The same comparison, using each patient’s mean daily MME over his/her time in the database reveals similar results (54.4% v. 43.2% for ≥ 20MME/day; 12.6% v. 9.0% for ≥ 100 MME/day).

The source of prescriptions was also similar between doctor shopping and doctor shopping eligible patients. Both patient populations received substantial quantities from their primary prescribers (prescriptions 44% v. 80%; MME 56% v. 86%, respectively), and much smaller percentages from the 1-prescription prescribers (prescriptions 15% v. 7%; MME 5% v. 3%, respectively)) (Fig 2). The main difference in prescription supply was from differential contribution from the “other” category. Fig 3 demonstrates the more unstable prescription supply of doctor shopping patients but also illustrates that when not doctor shopping, doctor shopping prescription patterns resemble those of doctor shopping eligible patients.

Comparison of prescriber characteristics across doctor shopping patient categories

We describe physician prescribing behavior in Table 3. Providers who only wrote prescriptions for doctor shopping eligible and doctor shopping patients wrote prescriptions for fewer patients in our sample (mean 12 and 6 respectively) than providers who wrote for at least one “neither” patient (mean 27). Though they wrote for fewer patients, these providers, wrote more prescriptions per patient (mean 6.7 v. 4.2 v. 1.3, respectively) and the prescriptions were for larger quantities of opioids (311 v. 300 v 189 median MME/prescription, respectively) and larger quantities of opioids per patient (680 v. 567 v. 228 median MME/Patient/Prescriber).

thumbnail
Table 3. Prescriber characteristics, N = 185,424 (mean, median [IQR] unless otherwise indicated).

https://doi.org/10.1371/journal.pone.0232533.t003

To understand the contribution of providers who had small prescription history with those patients like emergency, urgent care or other clinic providers, we employed two separate proxies likely to characterize their prescribing: the previously defined group of “episodic prescribers” and low frequency prescribers, defined as those who contributed 1 prescription to the patient [12]. Our data reflect that regardless of definition, both groups provided doctor shopping patients a small percentage of prescriptions (13.5% by the episodic and 15.0% by the 1 prescription definition) and an even smaller fraction of the total MME (1.7% and 4.9% respectively), despite having written at least one prescription for 88% (episodic) or 99% (1-prescription) of these patients. These prescribers were responsible for an even lower fraction of the opioids received by doctor shopping eligible patients: prescriptions 5.2% and 6.7%; MME 0.7% and 2.5%, respectively.

Limitations

Prescriptions from patients using a variety of names, addresses, and dates of birth may fail to link to a single patient identifier, artificially increasing the number of individuals in the database and decreasing their average use. However, per California law, patients need to establish identity in order to obtain a controlled substance from a pharmacy, thus limiting the magnitude of this phenomenon [20]. The CURES database also does not include opioids that may have been purchased through clandestine routes or procured via other forms of diversion.

Our analysis also assumes that the majority of opioids obtained via prescription are consumed by the patient and not diverted. While some of those included in the doctor shopping group may be actively diverting opioids, it is difficult to identify these patients in the clinical setting and an overemphasis upon this contribution would pose a threat to the provider-patient relationship. This is especially true for episodic prescribers, who represent the majority of clinicians.

Within the CURES database were a small number of patient (.04%) and provider identifiers (.03%) associated with exorbitant amounts of opioids. We cannot tell if these are real or artefactual, for example, pharmacy errors, drug rehabilitation centers that obtained all medications under the same patient name, or teaching hospitals that used one physician identifier for all prescriptions written by not yet licensed house staff. We accounted for these rare outliers by using medians in addition to means. Lastly, individuals who use a clinic as their PCP may see different physicians at each visit and appear to be doctor shopping despite having a regular source of care. This would lead to an overestimation of the number of patients who doctor shop, a bias that would not affect our conclusions.

Discussion

Our descriptive analysis of an 8-year sample of California’s PDMP provides several key insights into the prescription patterns of patients who doctor shop. First, those who doctor shop are relatively few in number (1.3% of all those receiving prescriptions). Second, they exhibit specific prescription acquisition patterns. They have the highest 90-day average daily MME use, putting them at higher risk of overdose [1416]. They also demonstrate the longest active time in the database compared to other groups, and they have the highest frequency of addiction related opioid prescriptions and overall long acting opioid use—use that may be indicative of treatment for a chronic painful medical disorder, an opioid use disorder, or both. Third, while those who doctor shopped obtained prescriptions from a larger number of prescribers, the majority of their opioid prescriptions came from a small number of providers who consistently prescribed large amounts of opioids over long periods of time (similar to doctor shopping eligible patients). In comparison, episodic and low frequency prescribers of opioids prescribed less than 10% of the MME obtained by doctor shopping patients, even when they were doctor shopping (top of upper right panel in Fig 2). These findings contradict the assumption that those who doctor shop depend on multiple one-time emergency department or clinic visits as a main source of opioid prescriptions.

An unexpected finding regarding prescription patterns for doctor shopping versus doctor shopping eligible patients is that there is less difference than similarity between the two groups. When not doctor shopping, doctor shopping patients exhibit the same prescription profile as doctor shopping eligible patients, whereas when they are doctor shopping they rely less on their primary opioid suppliers and more on episodic providers. Presumably, the switch to doctor shopping could be triggered by an event, such as a destabilization in access. These time periods of doctor shopping, inductively, may represent a period of heightened vulnerability in which to intervene with referral to treatment [21].

From a policy standpoint, opioid prescription reduction among opioid naïve or low use patients is projected to have a modest effect, while prescription reduction from the episodic settings for higher risk patient groups like those patients who doctor shopped promises to be ineffective [22]. Over 70% of both doctor shopping and doctor shopping eligible patients exhibited high risk opioid use (≥ 20 MME for 90 days). To focus on the doctor shopping patient is to miss 7 of 8 patients who engage in high risk prescription opioid use [23]. Given the similarity in behavior between doctor shopping and doctor shopping eligible patients, the currently promoted usage of PDMPs to flag only those patients who doctor shop is ambiguous and may be harmful if used simply for the policing purposes by providers. This is true especially in light of evidence that doctor shopping is not associated with overdoses [24]. Rather, there should be a more nuanced approach to ensure PDMPs empower providers to address the underlying problem present in the patient who doctor shops. An example of this could be including a list of local opioid treatment centers in the PDMP report when patients are flagged.

Furthermore, even if all contributions from episodic and low frequency prescribers were removed, doctor shopping patients would still receive large amounts of MME from primary prescribers and have dangerous levels of opioid use over long periods of time. The more pragmatic move for healthcare systems and providers who are faced with a patient who doctor shops in their emergency department, clinic or urgent care center may be to provide conduits to treatment [21]. Such referral pathways to opioid use disorder treatment improve addiction treatment retention and patient mortality [25, 26]. This strategy will not fit all patients who doctor shop, but it will ensure that patients with high risk opioid use disorder are offered evidence-based treatment and that the therapeutic alliance is preserved. Providers across the country are piloting different forms of linkage to treatment for patients with opioid use disorder, especially with regards to medication assisted treatment [2729]. Increasing treatment access and improving harm reduction programming represents a superior strategy for attenuating the opioid epidemic and should overshadow prescription reduction efforts in high risk populations.

Acknowledgments

California Department of Justice for use of the CURES PDMP database.

References

  1. 1. Katz N.P., Birnbaum H.G., and Castor A., Volume of prescription opioids used nonmedically in the United States. J Pain Palliat Care Pharmacother, 2010. 24(2): p. 141–4. pmid:20504136
  2. 2. Jones C.M., Mack K.A., and Paulozzi L.J., Pharmaceutical overdose deaths, United States, 2010. Jama, 2013. 309(7): p. 657–9. pmid:23423407
  3. 3. CDC. Wide-ranging online data for epidemiologic research (WONDER). 2016 [cited 2016 12/15/16]; Available from: http://wonder.cdc.gov.
  4. 4. Hall A.J., et al., Patterns of abuse among unintentional pharmaceutical overdose fatalities. Jama, 2008. 300(22): p. 2613–20. pmid:19066381
  5. 5. McDonald D.C. and Carlson K.E., Estimating the prevalence of opioid diversion by "doctor shoppers" in the United States. PLoS One, 2013. 8(7): p. e69241. pmid:23874923
  6. 6. Gugelmann H., Perrone J., and Nelson L., Windmills and pill mills: can PDMPs tilt the prescription drug epidemic? J Med Toxicol, 2012. 8(4): p. 378–86. pmid:23180357
  7. 7. Baldacchino A., et al., Guilty until proven innocent: a qualitative study of the management of chronic non-cancer pain among patients with a history of substance abuse. Addict Behav, 2010. 35(3): p. 270–2. pmid:19897313
  8. 8. Guarino H., et al., Young adults' opioid use trajectories: From nonmedical prescription opioid use to heroin, drug injection, drug treatment and overdose. Addict Behav, 2018.
  9. 9. Fink D.S., et al., Association Between Prescription Drug Monitoring Programs and Nonfatal and Fatal Drug Overdoses: A Systematic Review. Ann Intern Med, 2018. 168(11): p. 783–790. pmid:29801093
  10. 10. Haffajee R.L., et al., Four States With Robust Prescription Drug Monitoring Programs Reduced Opioid Dosages. Health Aff (Millwood), 2018. 37(6): p. 964–974.
  11. 11. Buchmueller T.C. and Carey C., The Effect of Prescription Drug Monitoring Programs on Opioid Utilization in Medicare. American Economic Journal: Economic Policy, 2018. 10(1): p. 77–112.
  12. 12. Schneberk T., et al., The Supply of Prescription Opioids: Contributions of Episodic-Care Prescribers and High-Quantity Prescribers. Ann Emerg Med, 2018. 71(6): p. 668–673.e3. pmid:29275945
  13. 13. CURES 2.0. 2016 [cited 2018 June 14th]; Available from: http://www.pdmpassist.org/pdf/TTAC_CURES_ROLLOUT_20160217.pdf.
  14. 14. Dowell D., Haegerich T.M., and Chou R., CDC Guideline for Prescribing Opioids for Chronic Pain—United States, 2016. Jama, 2016. 315(15): p. 1624–45. pmid:26977696
  15. 15. Dunn K.M., et al., Opioid prescriptions for chronic pain and overdose: a cohort study. Ann Intern Med, 2010. 152(2): p. 85–92. pmid:20083827
  16. 16. Gomes T., et al., Opioid dose and drug-related mortality in patients with nonmalignant pain. Arch Intern Med, 2011. 171(7): p. 686–91. pmid:21482846
  17. 17. Schriger D.L., Problems with current methods of data analysis and reporting, and suggestions for moving beyond incorrect ritual. Eur J Emerg Med, 2002. 9(2): p. 203–7. pmid:12131650
  18. 18. Ioannidis J.P.A., What Have We (Not) Learnt from Millions of Scientific Papers with P Values? The American Statistician, 2019. 73(sup1): p. 20–25.
  19. 19. Leppink J., O’Sullivan P., and Winston K., Evidence against vs. in favour of a null hypothesis. Perspectives on Medical Education, 2017. 6(2): p. 115–118. pmid:28210972
  20. 20. Pharmacy, B.o. 2018 LAWBOOK FOR PHARMACY. 2018 [cited 2018 July 12]; Available from: https://www.pharmacy.ca.gov/laws_regs/lawbook.pdf.
  21. 21. Samuels E.A., et al., A Quality Framework for Emergency Department Treatment of Opioid Use Disorder. Ann Emerg Med, 2019. 73(3): p. 237–247. pmid:30318376
  22. 22. Chen Q., et al., Prevention of Prescription Opioid Misuse and Projected Overdose Deaths in the United States. JAMA Netw Open, 2019. 2(2): p. e187621. pmid:30707224
  23. 23. Carey C.M., Jena A.B., and Barnett M.L., Patterns of Potential Opioid Misuse and Subsequent Adverse Outcomes in Medicare, 2008 to 2012. Ann Intern Med, 2018. 168(12): p. 837–845. pmid:29800019
  24. 24. Sun B.C., et al., Does Prescription Opioid Shopping Increase Overdose Rates in Medicaid Beneficiaries? Ann Emerg Med, 2018. 71(6): p. 679–687.e3. pmid:29174833
  25. 25. Sordo L., et al., Mortality risk during and after opioid substitution treatment: systematic review and meta-analysis of cohort studies. Bmj, 2017. 357: p. j1550. pmid:28446428
  26. 26. Duber H.C., et al., Identification, Management, and Transition of Care for Patients With Opioid Use Disorder in the Emergency Department. Ann Emerg Med, 2018.
  27. 27. D'Onofrio G., et al., Emergency department-initiated buprenorphine/naloxone treatment for opioid dependence: a randomized clinical trial. Jama, 2015. 313(16): p. 1636–44. pmid:25919527
  28. 28. Bernstein E., Bernstein J., and Levenson S., Project ASSERT: an ED-based intervention to increase access to primary care, preventive services, and the substance abuse treatment system. Ann Emerg Med, 1997. 30(2): p. 181–9. pmid:9250643
  29. 29. Neven D., et al., A Randomized Controlled Trial of a Citywide Emergency Department Care Coordination Program to Reduce Prescription Opioid Related Emergency Department Visits. J Emerg Med, 2016. 51(5): p. 498–507. pmid:27624507