Factors associated with opioid overdose: a 10-year retrospective study of patients in a large integrated health care system

Introduction

The prevalence of prescription opioid misuse has increased over the past decade.^1–5 Opioid overdose (OD) and mortality due to OD have also increased during this period.^1–6 One initiative is to expand access to naloxone for use in both licit and illicit narcotic overdose cases.^7,8 Naloxone, an antidote for opioids, has a history of safe and effective use by health care systems and providers in the treatment of OD and by paramedics, emergency medical technicians, and emergency medicine physicians.^9,10 For the past 2 decades, programs have offered overdose prevention education and have distributed naloxone for bystander administration among drug users, their family members, and their acquaintances.⁷ Currently, due to the increase in the prevalence of prescription opioid misuse and the wider use of heroin, there is growing interest in extending overdose education and naloxone distribution to conventional medical settings for patients who are prescribed pharmaceutical opioids.¹¹ To implement an effective policy to address issues associated with patients who are at risk of OD, it is important to understand the health care characteristics, including OD factors associated with risk, treatment options, and health care outcomes among those who experienced ODs in health care settings.¹²

Recently, the diagnostic criteria for opioid-use disorders have been redefined.¹³ Although research suggests that the prevalence of these disorders will likely decrease as a result of this revision, the key predictors for these disorders are essentially the same.¹⁴ A recent study suggested that primary care and internal medicine physicians prescribed the largest percentage of opioids to patients who died from overdose,¹⁵ pointing to the need for better surveillance tools for these providers. Additional research is needed to better detect the risk of OD in clinical practice and improve the dissemination of this information.¹⁶

In the research literature, specific demographic groups have been shown to be at greater risk of fatal OD, including men, those aged 45–54 years, and non-Hispanic white persons.¹² Rates are also reported to be higher in rural areas.⁷ A recent study by Meyer et al¹⁷ noted that receiving prescriptions for higher opioid doses are associated with an increased risk of OD death. An analysis using Veterans Health Administration data, as well as other data, has also confirmed the association between higher opioid consumption and increased risk of overdose death.^12,18,19 In addition, co-prescription of benzodiazepines, antidepressants, and antipsychotics was also associated with experiencing an overdose.¹² Research suggests that many patients taking opioids for chronic pain were often prescribed multiple psychoactive medications during the same period.^12,18 Concomitant illnesses often associated with overdose include opioid dependence, moderate or severe liver disease, metastatic solid tumor, and pancreatitis.¹² Illicit drug use and history of alcohol misuse are also associated with prescription opioid misuse.^4,20

The current study describes the clinical and demographic characteristics of patients who have overdosed on opioids and assesses key factors associated with negative health outcomes and high health care resource utilization among those admitted to a large Pennsylvania health care system (the Geisinger Health System [GHS]). This study examines the electronic health records (EHRs) for outpatients, inpatients, and emergency department patients for a period of up to 10 years. For more than a decade, Geisinger has used EHR data for population health management of chronic conditions, such as diabetes, asthma, and low back pain.^21–23 This EHR database can be leveraged to assess ODs and can be linked to prescription drug orders, other comorbid conditions, and health care utilization and costs. The GHS includes Geisinger Clinic, a hospital/provider network with more than 40 primary care clinics, a tertiary care teaching hospital, and ten other hospitals, as well as an insurance provider (the Geisinger Health Plan). Geisinger serves more than 3 million community residents throughout 45 counties in central, south-central, and northeast Pennsylvania and is the dominant health care provider in this region. In brief, Geisinger is a large physician-led health care system composed of ~1,600 primary and specialty care physicians, eleven hospital campuses, and 30,000 employees, making it one of the largest employers and health care providers in Pennsylvania (www.geisinger.org). Each hospital in the Geisinger system has an onsite emergency department and these departments see a combined annual volume of >200,000 patient visits per year.

This study examines prescribed medications in the context of medical conditions, and other factors, as these relate to a patient’s potential overdose risk. It is believed that this information will be instrumental in OD prevention, including in the identification of at-risk patients, in improving access to naloxone, and in informing more effective health system interventions.^18,19,24 Noteworthy is that this study represents a service-area-wide study, not just local area patients, which examines OD cases admitted to the Geisinger system. Significant is the fact that this diverse service area encompasses ~25,000 square miles in the state of Pennsylvania (Figure 1).

Figure 1 ODs in Geisinger service area. Abbreviations: GHS, Geisinger Health System; OD, opioid overdose.

Methods

Study sample

The eligible patients for this study included all patients identified in the EHR who met the following criteria between April 1, 2005, and March 31, 2015: had an International Classification of Diseases (ICD)-9 diagnosis code for OD (defined as ICD-9 codes 965.00, 965.01, 965.02, 965.09 or ICD-9 ‘E’ codes E850.0, E850.1, E850.2, E935.0, E935.1, E935.2) and entered any inpatient, emergency room, or had an outpatient visit during this time period. Patients included in this study were between the age of 10 years and 95 years and had one or more ODs based on these ICD-9 codes. We included patients as young as 10 years old in the current study, because we wanted to examine a broad spectrum of persons at risk of OD, as is common in national studies of drug misuse.²⁵ Since this was a service-area-wide investigation of OD cases, all ODs were included in this study, including prescription opioids, illicit opioids such as heroin, and synthetic opioids used in addiction treatment, such as methadone. Overall, health care encounter data were examined for 1,174,120 unique patients, among which a total of 2,039 patients were identified as having one or more ODs during the study time period.

Study measures

Using the EHR, medical demographic variables identified included age, sex, race, marital status, employment status, primary care physician status, health care insurance status, cigarette smoking, and body mass index. In addition, using ICD-9 codes, the Charlson comorbidity index was calculated for each patient,²⁶ as was the history of comorbid conditions, including cardiovascular disease (CVD), hypertension, chronic kidney disease, liver disease, pancreatitis, diabetes, rheumatoid arthritis, obesity, hepatitis C, HIV disease, sexually transmitted disease, chronic obstructive pulmonary disease (COPD), asthma, cancer, lower back pain, sleep apnea, and fibromyalgia.¹² Mental health disorders examined, which were based on ICD-9 codes, included history of depression, anxiety disorders, psychosis, personality disorders, and drug/alcohol use disorders. This study also examined the proximity of patients with OD to the GHS facilities. Thus, we included patient addresses for geospatial mapping and data analysis, as we describe later. It is noted that examination of the health care sites where patients typically received care indicated both outpatient primary care and specialty clinic locations were visited, although the OD encounter tended to take place at an inpatient or emergency department setting. Psychotherapeutic medications examined included not only prescription opioids but also antidepressant, anxiolytic, hypnotic, migraine, antipsychotic, buprenorphine, methadone, and benzodiazepine medications.¹² Prescription opioid use, typically a significant predictor of OD, was analyzed categorically by counting the number of prescription orders in the EHRs in the past 12 months before OD (0, 1–2, 3–8, 9+ orders), as was performed in earlier research.^23,27 This EHR method works well with patients prescribed opioids episodically and/or who are not part of a health plan.²³ Geisinger is an integrated, but open health care system that provides care for all payer groups in the region, including Medicare, Medicaid, those with private health insurance, and other groups. The EHR captures the health care encounters across all these payer groups, including the medications ordered by providers. We note that calculation of a “morphine equivalent dose” is limited with nonplan members, because these types of patients may receive opioids from multiple sources and prescription claims data are unavailable. Study outcomes examined included death, multiple ODs, inpatient hospitalization, emergency department use, outpatient visits, naloxone orders, and total costs 12 months after the index OD event. Patients with at least eight outpatient visits within 12 months after the first OD event (the 75th percentile among all OD cases) were defined as those with high outpatient visits. Likewise, the individuals whose total health care cost, including inpatient, outpatient, and emergency department charges during the 12 months post index OD period, was $36,169 or higher (the 75th percentile of the charges among all OD cases) were defined as those with high costs. For these estimates, we used health care charges from billing data, which may overestimate costs, but is a good relative measure of resource utilization.²⁸ Given the insurance status of the study sample (eg, Medicare, Medicaid, private insurance, and self-pay), these charge data represent a measure of overall resource utilization and are an important study outcome. Cost/charge data are typically not normally distributed and present challenges in linear analyses, rendering interpretation difficult. Consequently, we chose to use a percentile cut-point, as performed in earlier research.²⁷ We chose the 75th percentile to define high charges based on past research and the utility of these health economic end points.^27,29 Finally, Geisinger routinely collects the data on death and the date of death for patients for quality care and research purposes and uses multiple sources for these data, including the Pennsylvania Department of Health, social security records, and family/provider reports of death in the course of health care delivery. These data end points are then included in the EHRs and are available for research and quality improvement purposes.

Analytic approach

For our analyses, we used data from the patient’s EHR for a period of up to 10 years to examine factors associated with OD and factors that may lead to adverse outcomes post-OD. To examine the key variables associated with post-OD outcomes, we first describe demographic and clinical characteristics of patients and key study outcome experiences post-OD (mortality, repeated OD events, health care utilization, and cost) within 12 months after the index OD event (Table 1). For analysis purposes, the index event for this study was the occurrence of the first documented OD event. Histories of a chronic disease and/or mental disorders were coded as present, if they occurred any time within a year of the first OD, but not before 2005. Similarly, psychotherapeutic medication use was assessed in the 12-month window up to the first OD. Finally, to examine key patient characteristics related to the post-OD outcomes, we use stepwise multiple logistic regression analyses to identify the best predictors for OD outcomes of interest using backward elimination, after the examination of statistically significant bivariate associations. Age (in years) and male sex were retained in all these stepwise regressions to limit potential confounding. Since the outcomes of interest were defined as binary measures (eg, mortality, high utilization, and high cost), we used multiple logistic regression analyses. All analyses were conducted using Stata Version 13.1 software (Stata Corporation, College Station, TX, USA), with statistical differences of a P-value of <0.01 considered significant, due to the number of assessments made to account for the potential statistical bias caused by multiple comparisons.³⁰ The geospatial procedure used for mapping OD cases (Figure 1) included the use of ArcGIS for Desktop, Version 10.1 (2012); Environmental Systems Research Institute, Redlands, CA, USA. We used the estimated centroid for the patient’s five-digit residential zip code to define OD location. Finally, Geisinger’s Institutional Review Board approved this study. The review board did not require written consent to be obtained from the patients, as the study included de-indentified data only.

Table 1 Study demographic and outcome variables among patients with one or more ODs (N=2,039) Notes: All patients had one or more ICD-9-coded ODs during the study period (age range: 10–95 years). aNaloxone Rx order 1 year pre-OD =17.3 (95% CI =15.7–19.0). Abbreviations: CI, confidence interval; ICD-9, International Classification of Diseases-9; OD, opioid overdose; PCP, primary care physician; Rx, prescription.

Results

We identified a study sample of 2,039 patients, between the age of 10 years and 95 years, who had an ICD-9 diagnosis code for one or more ODs. Table 1 shows the baseline characteristics and study outcomes, and Figure 2 shows the overall age distribution of this sample. Patients were more often white (96.2%), had a mean age of 51.6 years, but were similar with respect to sex, with 53.7% female vs 46.3% male (Table 1). In addition, while the patients with OD were distributed widely across age groups, a significant number of OD cases occur in the age group of 20–40 years, as well as in the age group of 45–55 years (Figure 2).

Figure 2 Study age distribution among patients with one or more ODs. Abbreviation: OD, opioid overdose.

Furthermore, 36.1% of patients in the sample were currently married and over half had private health insurance (52.5%; Table 1). We also observed a high level of comorbidity in this study sample, with 36.3% of these patients having a Charlson comorbidity score of 4+ in the year prior to their overdose (Table 1). Notably, a large proportion of overdose subjects (86.7%) had a primary care provider within the Geisinger system, suggesting potential access to health care services. Noteworthy, however, was that 20.6% of OD cases overdosed on heroin or methadone (Table 1). Approximately 95% of the study sample only had one overdose, but 9.4% of these patients died within 1 year of the index overdose date (Table 1). We also found that 32.2% of patients with an overdose had an emergency room visit within 1 year after their overdose (38% had an emergency room visit 1 year before their overdose), and only 9.4% had naloxone prescribed to them by a Geisinger provider post-OD. Noteworthy is that 17.3% had naloxone prescribed to them through Geisinger the year before their OD (Table 1, notes section).

Table 2 shows significant increases (P<0.01) in the prevalence of death within 12 months post-OD among patients with comorbid medical conditions, including CVD, hypertension, chronic kidney disease, diabetes, COPD, and cancer. Table 2 also shows that 15.5% of the overdose sample had nine or more prescription orders for opioids in the year before their first overdose and that significant numbers of these patients were also receiving other psychoactive medications pre-OD, including anxiolytic (36.1%), hypnotic (27.7%), migraine (46.3%), and benzodiazepine (32.9%) medicines, all of which were associated with increased mortality risk.

Table 2 Significant bivariate results (P<0.01) for patient deaths within 1 year post-OD among patients with one or more ODs (N=2,039) Notes: All patients had one or more ICD-9-coded ODs during the study period (age range: 10–95 years). aSince no deaths occurred among the age group of 10–17 years, this group was collapsed into the age group of 18–35 years. bOpioid Rx orders 1 year post-OD: none =30.0%; 1–2 Rx orders =23.7%; 3–8 Rx orders =30.2%; nine or more Rx orders =16.1%. Abbreviations: CI, confidence interval; COPD, chronic obstructive pulmonary disease; CVD, cardiovascular disease; ICD-9, International Classification of Diseases-9; OD, opioid overdose; OR, odds ratio; Rx, prescription.

Tables 3–6 show the multiple logistic regression analyses assessing the determinants of death, multiple overdoses, increased utilization, and higher costs. The best predictors for death within 1 year of overdose were cancer (odds ratio [OR] =4.22), a Charlson score >4 (OR =2.31), diabetes (OR =2.18), any substance misuse disorders (OR =2.86), and 9+ opioid prescriptions in the past year (OR =2.29). Factors protective of death within a year of overdose included having a Geisinger primary care provider (OR =0.48), commercial or private insurance (OR =0.52), and somewhat counterintuitively, having a mental health disorder (OR =0.55; Table 3). The best predictor of multiple ODs was buprenorphine use pre-OD (OR =12.30; Table 4). However, more consistent with post-OD mortality, a pattern for higher patient utilization and higher costs post-OD was noted (Tables 5 and 6). For example, patients who had comorbid medical conditions, mental or substance use disorders, higher prescription opioid use, and used other psychotropic drugs the year before OD were more likely to have higher health care costs and higher service utilization post-OD (Tables 5 and 6). For example, higher outpatient service use post-OD was observed among those with pre-OD CVD (OR =1.62), with cancer (OR =1.70), with mental health disorders (OR =1.52), and those who had nine or more prescription orders for opioids pre-OD (OR =4.62; Table 5). Those with higher cost post-OD tended to have medical comorbidities, other psychotropic medication use, and higher prescription opioid use before OD (Table 6). Having a Geisinger primary care physician pre-OD was associated with higher cost and higher utilization post-OD (Tables 5 and 6).

Table 3 Significant multiple logistic regression results (P<0.01) for patient deaths 1 year post-OD among patients with one or more ODs (N=2,039) Notes: Area under the ROC curve =0.85; Hosmer–Lemeshow test = P>0.99. Results shown are for significant variables that were retained in the final model with a P-value of <0.01. All patients had one or more ICD-9-coded ODs during the study period (age range: 10–95 years). aAge (in years) and male sex were also included in the logistic regression, resulting in an adjusted age OR =1.01, P=0.036, and an adjusted male sex OR =1.56, P=0.014, in the final stepwise model. Abbreviations: CI, confidence interval; ICD-9, International Classification of Diseases-9; OD, opioid overdose; OR, odds ratio; PCP, primary care physician; ROC, receiver operating characteristic.

Table 4 Significant multiple logistic regression results (P<0.01) for multiple OD events among patients with one or more OD (N=2,039) Notes: Area under the ROC curve =0.71; Hosmer–Lemeshow test = P>0.71. Results shown are for the significant variables that were retained in the final model with a P-value of <0.01. All patients had one or more ICD-9-coded ODs during the study period (age range: 10–95 years). aAge (in years) and male sex were also included in the logistic regression, resulting in an adjusted age OR =1.00, P=0.230, and an adjusted male sex OR =1.40, P=0.136, in the final stepwise model. Abbreviations: CI, confidence interval; CVD, cardiovascular disease; ICD-9, International Classification of Diseases-9; OD, opioid overdose; OR, odds ratio; ROC, receiver operating characteristic; Rx, prescription.

Table 5 Significant multiple logistic regression results (P<0.01) for high number of outpatient visits (75th percentile) 1 year post-OD among patients with one or more ODs (N=2,039) Notes: Area under the ROC curve =0.74; Hosmer–Lemeshow test = P>0.79. Results shown are for significant variables that were retained in the final model with a P-value of <0.01. All patients had one or more ICD-9-coded ODs during the study period. aAge (in years) and male sex were also included in the logistic regression model, resulting in an adjusted age OR =1.01, P=0.004, and an adjusted male sex OR =0.76, P=0.017, in the final stepwise model. Abbreviations: CI, confidence interval; CVD, cardiovascular disease; ICD-9, International Classification of Diseases-9; OD, opioid overdose; OR, odds ratio; PCP, primary care physician; ROC, receiver operating characteristic; Rx, prescription.

Table 6 Significant multiple logistic regression results (P<0.01) for high total costs (75th percentile) 1 year post-OD among patients with one or more ODs (N=2,039) Notes: Area under the ROC curve =0.78; Hosmer–Lemeshow test = P>0.393. Results shown are for significant variables that were retained in the final model with a P-value of <0.01. All patients had one or more ICD-9-coded ODs during the study period. aAge (in years) and male sex were also included in the logistic regression model, resulting in age OR =0.99, P=0.023, and male sex OR =0.92, P=0.92, in the final stepwise model. Abbreviations: CI, confidence interval; CVD, cardiovascular disease; ICD-9, International Classification of Diseases-9; OD, opioid overdose; OR, odds ratio; PCP, primary care physician; ROC, receiver operating characteristic; Rx, prescription.

Figure 1 is a geographic map of Geisinger’s catchment area, with the residences of the patients experiencing overdoses plotted within this service area. While overdoses occur more frequently in urban areas, a large part of the study area consists of rural communities (Figure 1). This GHS catchment area encompasses ~45 counties in the state of Pennsylvania.

Finally, it is noted that while having a naloxone order post-OD was an outcome of interest in our study (Table 1), the prediction model for this outcome fit poorly (area under the receiver operating characteristic curve =0.69), and the best predictor of this outcome (OR =2.28) was having a naloxone order pre-OD. Consequently, these results are not shown in this article, but are available upon request from the corresponding author.

Discussion

Over the past decade, there has been an increase in the number of ODs in the US.^1–6 Results of the current study suggest that there may be opportunities to address the causes and consequences of OD from a health care system perspective. The observation that the majority of OD patients have a primary care provider and have private insurance supports the fact that there may be resources within the health care system to address this problem. Thus, opportunities for the prevention of OD, including the identification of at-risk patients, better health care system communications, improving access to naloxone in patient care settings, and follow-up care for substance use disorders, may have a significant impact on the reduction in ODs.⁸ As was shown, among our study sample of 2,039 patients with one or more ODs, only 9.4% of these patients had an order for naloxone in the EHR post-OD, suggesting a current naloxone access issue and opportunities for improvement (Table 1). A notable finding from this study is the use of prescription opioids.¹⁵ Based on EHR data, more than 60% of the patients had access to prescription opioids in the 12 months before their OD event. This supports the growing body of literature examining the increase over the past decade in the misuse of prescription opioids.^8,31

This study also shows significant variation for OD with regard to age. A large group of the OD sample is aged 20–30 years, which is in line with other research showing how this disorder affects younger adults.⁴ However, there are other noteworthy age groupings, including those patients in their 50s, 60s, and 70s. Another of note demographic finding is that the proportion of women in this sample of patients with OD is somewhat higher than men (54% vs 46%), which has been previously reported in the research literature related to prescription opioid misuse.¹⁴

Previous research has shown that ODs tend to be clustered in certain geographic areas,³² and this is certainly true for the current study (Figure 1). More effective interventions could be implemented by focusing on these OD “hotspots”.³² Primary care as well as specialty care physicians in these areas should be better informed related to OD interventions and the use of naloxone.

Although this study has its strengths, it also has limitations. One is that the data collected were from patients within a single health care system. Although this system does provide services to a large percentage of the population within a vast 25,000 square mile catchment area, there are other facilities whose data would not be included in this analysis. Thus, this study does not identify all overdose cases in the geographic area. Another issue includes the broad eligibility criteria applied to the study sample. Since we used only OD codes and age to select cases, we are limited with respect to the continuous eligibility for the study sample, which means health records for some patients may not be complete. However, the fact that ~90% of patients admitted for OD care had a Geisinger primary care physician suggests that this bias may be limited. In addition, the age range included in this study was broad (eg, 10–95 years old), thus further analysis may be required, since different age groups may have different underlying etiologies and outcomes. Another limitation is that our study did not have data on the “cause of death”, but only on the date of death. Thus, some patients may have died from natural causes unrelated to OD. Another limitation is that older patients on long-term opioids may have a host of medical problems and any one of these may have hastened their death, so this association is likely confounded. Thus, it is not surprising that there was an association with death up to 1 year post-OD. This association has been recently reported in the research literature as well.³³ Additional research is planned, including research to be conducted within the national Health Care Systems Research Network in the near future.³⁴ We also examined a number of predictor variables and consequently set the minimum P-value for statistical significance at 0.01, which may have biased some analysis. For example, the findings for buprenorphine use were limited, since only 37 patients (1.8%) had an order for this medication in the EHR pre-OD. The majority of clinicians, ~80%, did not prescribe this medication at Geisinger in the past year. Given the latter, while buprenorphine use was significant in predicting multiple ODs (OR =12.30), this prediction model was limited (Table 4). Furthermore, we used stepwise logistic regression, which is a straightforward statistical method to select among multiple predictor variables. However, this approach may have biased our study findings.³⁵ The strength of using the stepwise approach, nevertheless, is that it is empirically based and it requires few statistical assumptions.³⁵ Finally, the use of health care “charge” data as opposed to actual cost data is also a study limitation that needs to be addressed.

Conclusion

Although our research tends to validate previous findings, it is unique because it shows the extent and variation of ODs in a large geographic area dominated by a major health care system. Consequently, the findings of this study can be used to guide this health care system, as well as others, in implementing better policies for improving addiction medicine and OD care. One example that is currently in development is the “Escalation of Care Model” being implemented at Geisinger. For this program, the Pain Medicine Department is working in collaboration with clinical pharmacist and primary care physician groups to address the clinical appropriateness of long-term prescription opioid use and daily morphine equivalent dosing. This is being performed to develop and integrate longitudinal patient education, support, and monitoring programs and to promote increased use of non-opioid medication regimens. It is believed that this new “care model” will optimize the treatment of pain, while minimizing the potential side effects related to the long-term opioid use. There are other actions that health care systems can explore to manage this population more effectively. For example, the EHR may help health care systems to notify primary care physicians and specialists of high-risk patients in real time. The updating of guidelines and operation manuals focusing on distributing naloxone to those in need may also help. Further research is planned to assess these interventions among patients who are at high risk for OD.

Acknowledgments

The assistance of Joseph J Boscarino is gratefully acknowledged in the preparation of this manuscript. Support for this study included research funding from Indivior, Inc., Richmond, VA, USA. A version of this study was presented at the International Conference on Opioids, Boston, MA, USA, June 5, 2016. This study has not been previously published.

Author contributions

All authors contributed toward data analysis, drafting and critically revising the paper, gave final approval of the version to be published, and agree to be accountable for all aspects of the work.

Disclosure

JAB, HLK, JMP, MHF, and JJH are employees of Geisinger Clinic. VRN and NAR are employees of Indivior, Inc. The authors report no other conflicts of interest in this work.

References

1.		Manchikanti L, Fellows B, Ailinani H, Pampati V. Therapeutic use, abuse, and nonmedical use of opioids: a ten-year perspective. Pain Physician. 2010;13(5):401–435.
2.		Von Korff M. Commentary on Boscarino et al. (2010): understanding the spectrum of opioid abuse, misuse and harms among chronic opioid therapy patients. Addiction. 2010;105(10):1783–1784.
3.		Kuehn BM. SAMHSA: pain medication abuse a common path to heroin: experts say this pattern likely driving heroin resurgence. JAMA. 2013;310(14):1433–1434.
4.		Boscarino JA, Rukstalis M, Hoffman SN, et al. Risk factors for drug dependence among out-patients on opioid therapy in a large US health-care system. Addiction. 2010;105(10):1776–1782.
5.		Hasegawa K, Espinola JA, Brown DF, Camargo CA Jr. Trends in U.S. emergency department visits for opioid overdose, 1993–2010. Pain Med. 2014;15(10):1765–1770.
6.		Olsen Y. The CDC guideline on opioid prescribing: rising to the challenge. JAMA. 2016;315(15):1577–1579.
7.		Faul M, Dailey MW, Sugerman DE, Sasser SM, Levy B, Paulozzi LJ. Disparity in naloxone administration by emergency medical service providers and the burden of drug overdose in US rural communities. Am J Public Health. 2015;105(suppl 3):e26–e32.
8.		Humphreys K. An overdose antidote goes mainstream. Health Aff. 2015;34(10):1624–1627.
9.		Wermeling DP. Review of naloxone safety for opioid overdose: practical considerations for new technology and expanded public access. Ther Adv Drug Saf. 2015;6(1):20–31.
10.		Merlin MA, Saybolt M, Kapitanyan R, et al. Intranasal naloxone delivery is an alternative to intravenous naloxone for opioid overdoses. Am J Emerg Med. 2010;28(3):296–303.
11.		Mueller SR, Walley AY, Calcaterra SL, Glanz JM, Binswanger IAA. Review of opioid overdose prevention and naloxone prescribing: implications for translating community programming into clinical practice. Subst Abus. 2015;36(2):240–253.
12.		Zedler B, Xie L, Wang L, et al. Development of a risk index for serious prescription opioid-induced respiratory depression or overdose in veterans’ health administration patients. Pain Med. 2015;16(8):1566–1579.
13.		American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 5th ed. Arlington, DC: American Psychiatric Association; 2013.
14.		Boscarino JA, Hoffman SN, Han JJ. Opioid-use disorder among patients on long-term opioid therapy: impact of final DSM-5 diagnostic criteria on prevalence and correlates. Subst Abuse Rehabil. 2015;6:83–91.
15.		Lev R, Lee O, Petro S, et al. Who is prescribing controlled medications to patients who die of prescription drug abuse? Am J Emerg Med. 2016;34(1):30–35.
16.		Setnik B, Roland CL, Sommerville KW, et al. A multicenter, primary care-based, open-label study to identify behaviors related to prescription opioid misuse, abuse, and diversion in opioid-experienced patients with chronic moderate-to-severe pain. J Pain Res. 2015;8:361–373.
17.		Meyer R, Patel AM, Rattana SK, Quock TP, Mody SH. Prescription opioid abuse: a literature review of the clinical and economic burden in the United States. Popul Health Manag. 2014;17(6):372–387.
18.		Larochelle MR, Liebschutz JM, Zhang F, Ross-Degnan D, Wharam JF. Opioid prescribing after nonfatal overdose and association with repeated overdose: a cohort study. Ann Intern Med. 2016;164(1):1–9.
19.		Bohnert AS, Valenstein M, Bair MJ, et al. Association between opioid prescribing patterns and opioid overdose-related deaths. JAMA. 2011;305(13):1315–1321.
20.		Ives TJ, Chelminski PR, Hammett-Stabler CA, et al. Predictors of opioid misuse in patients with chronic pain: a prospective cohort study. BMC Health Serv Res. 2006;6:46.
21.		Berger Z, Kimbrough W, Gillespie C, et al. Lower copay and oral administration: predictors of first-fill adherence to new asthma prescriptions. Am Health Drug Benefits. 2009;2(4):174–180.
22.		Casey JA, Schwartz BS, Stewart WF, Adler NE. Using electronic health records for population health research: a review of methods and applications. Annu Rev Public Health. 2016;37:61–81.
23.		Stewart WF, Yan X, Boscarino JA, et al. Patterns of health care utilization for low back pain. J Pain Res. 2015;8:523–535.
24.		Lin LA, Bohnert AS, Ilgen MA, Pfeiffer PN, Ganoczy D, Blow FC. Outpatient provider contact prior to unintentional opioid overdose among VHA service users. Psychiatr Serv. 2015;66(11):1149–1154.
25.		Substance Abuse and Mental Health Services Administration. Results from the 2010 National Survey on Drug Use and Health: Summary of National Findings, NSDUH Series H-41, HHS Publication No. (SMA) 11-4658. Rockville, MD: Substance Abuse and Mental Health Services Administration; 2011.
26.		Sundararajan V, Henderson T, Perry C, Muggivan A, Quan H, Ghali WA. New ICD-10 version of the Charlson comorbidity index predicted in-hospital mortality. J Clin Epidemiol. 2004;57(12):1288–1294.
27.		Maeng DD, Stewart WF, Yan X, et al. Use of electronic health records for early detection of high-cost, low back pain patients. Pain Res Manag. 2015;20(5):234–240.
28.		Strom BL, Kimmel SE, Hennessy S, editors. Pharmacoepidemiology. 5th ed. Hoboken, NJ: John Wiley & Sons; 2012.
29.		Schulman KA, Glick HA, Polsky D, Reed SD. Pharmacoeconomics: economic evaluation of pharmaceuticals. In: Strom BL, Kimmel SE, Hennessy S, editors. Pharmacoepidemiology. 5th ed. Hoboken, NJ: John Wiley & Sons; 2012:678–708.
30.		Hulley SB, Cummings SR, Browner WS, Grady DG, Newman TB. Designing Clinical Research. 4th ed. Philadelphia, Baltimore, New York, London, Buenos Aires, Hong Kong, Sydney, Tokyo: Wolters Kluwer Health/Lippincott Williams & Wilkins; 2013.
31.		Von Korff M, Kolodny A, Deyo RA, Chou R. Long-term opioid therapy reconsidered. Ann Intern Med. 2011;155(5):325–328.
32.		Klimas J, O’Reilly M, Egan M, Tobin H, Bury G. Urban overdose hotspots: a 12-month prospective study in Dublin ambulance services. Am J Emerg Med. 2014;32(10):1168–1173.
33.		Ray WA, Chung CP, Murray KT, Hall K, Stein CM. Prescription of long-acting opioids and mortality in patients with chronic noncancer pain. JAMA. 2016;315(22):2415–2423.
34.		Coleman KJ, Stewart C, Waitzfelder BE, et al. Racial-ethnic differences in psychiatric diagnoses and treatment across 11 health care systems in the Mental Health Research Network. Psychiatr Serv. 2016;67:749–757.
35.		Draper NR, Smith H. Applied Regression Analysis. 3rd ed. New York, NY: John Wiley & Sons; 1998.