https://orcid.org/0000-0001-9466-880X
Figures
Abstract
Opiates used for acute pain are an established risk factor for chronic opioid use (COU). Patient characteristics contribute to progression from acute opioid use to COU, but most are not clinically modifiable. To develop and validate machine-learning algorithms that use claims data to predict progression from acute to COU in the Medicaid population, adult opioid naïve Medicaid patients from 6 anonymized states who received an opioid prescription between 2015 and 2019 were included. Five machine learning (ML) Models were developed, and model performance assessed by area under the receiver operating characteristic curve (auROC), precision and recall. In the study, 29.9% (53820/180000) of patients transitioned from acute opioid use to COU. Initial opioid prescriptions in COU patients had increased morphine milligram equivalents (MME) (33.2 vs. 23.2), tablets per prescription (45.6 vs. 36.54), longer prescriptions (26.63 vs 24.69 days), and higher proportions of tramadol (16.06% vs. 13.44%) and long acting oxycodone (0.24% vs 0.04%) compared to non- COU patients. The top performing model was XGBoost that achieved average precision of 0.87 and auROC of 0.63 in testing and 0.55 and 0.69 in validation, respectively. Top-ranking prescription-related features in the model included quantity of tablets per prescription, prescription length, and emergency department claims. In this study, the Medicaid population, opioid prescriptions with increased tablet quantity and days supply predict increased risk of progression from acute to COU in opioid-naïve patients. Future research should evaluate the effects of modifying these risk factors on COU incidence.
Author summary
Prescription opioids in the United States contribute to opioid-related overdose deaths. Evidence suggests the Medicaid population has a greater likelihood of opioid-related mortality. However, the current standard for postoperative pain management often doesn’t account for previous opioid use and/or misuse. Machine learning can help healthcare systems identify patients at risk of progression from opioid naïve to chronic opioid use (COU) in Medicaid patients. In this cohort study of 180,000 opioid naïve Medicaid patients, machine learning models accurately identified opioid naïve patients at risk of progression to COU. Features of the initial opioid prescription were strong predictors of progression to COU, including number of tablets, prescription length, and prescription of oxycodone. We found that prescription quantity and length are discrete features related to opioid prescribing that are predictive of COU incidence. This research suggests that stronger prescription guidelines are needed for opioid prescriptions among opioid naïve patients.
Citation: Johnson DG, Ho VT, Hah JM, Humphreys K, Carroll I, Curtin C, et al. (2022) Prescription quantity and duration predict progression from acute to chronic opioid use in opioid-naïve Medicaid patients. PLOS Digit Health 1(8): e0000075. https://doi.org/10.1371/journal.pdig.0000075
Editor: Heather Mattie, Harvard University T H Chan School of Public Health, UNITED STATES
Received: November 4, 2021; Accepted: June 13, 2022; Published: August 25, 2022
This is an open access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.
Data Availability: The raw data used in this study contain personal health information and therefore cannot be released to the public, readers may request access from Nilpa Shah (nilpa.shah@stanford.edu). However, all the code used to generate the analysis are available via GitHub (https://github.com/drgojo/mcaid-opioid).
Funding: Data and support for this work were provided by HMS (now part of Gainwell Technologies) as part of the Digital Health CRC (Cooperative Research Centre) Limited. The DHCRC is established and supported under the Australian Government’s Cooperative Research Centres Program. Research reported in this publication was also supported by the National Library of Medicine of the National Institutes of Health under Award Number R01LM013362. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Prescription opioids in the United States contribute to opioid-related overdose deaths and an annual economic burden exceeding $78 billion [1,2]. State-level policy interventions to address the opioid crisis have included prescription drug monitoring programs, prior authorization requirements, and limits on prescription length and dosage [3–5]. Reported outcomes are heterogenous; recent evidence suggests state-level drug policies may reduce prescription opioid misuse in the Medicare population when participation in prescription drug monitoring programs is mandatory.[6] Finding modifiable factors that can positively impact opioid-related endpoints is needed to guide these policies.
Identifying features of responsible opioid prescribing is particularly important in Medicaid patients, because they are at increased risk of developing chronic opioid use and substance use disorders [7–9]. Medicaid status has also been associated with a greater likelihood of non-medical prescription opioid use and opioid-related mortality [9–11]. In a cohort of opioid-naïve commercially insured patients including 1.15% in a Medicaid managed care program, Shah et al found that initial prescriptions with days supply over 10 days or cumulative dosage over 700 morphine milligram equivalents (MME) were associated with increased likelihood of continued opioid use at 1 and 3 years after index prescription [12]. Likewise, prescription of a long acting (LA) opioid and daily dosage exceeding 90 MME was associated with persistent opioid use in a cohort study of Canadian patients receiving opioids after dental procedures [13].
ML methods may predict opioid-related endpoints by identifying complex or non-linear relationships in large datasets [14–18]. At this time, use of these techniques to identify risk factors for chronic opioid use in Medicaid patients have been limited to single states [19–21]. Other single-state retrospective cohort studies have identified factors associated with conversion from acute to chronic opioid use including back pain, previous benzodiazepine prescription, and substance abuse disorders [20]. We hypothesized that ML models trained on multi-state data could accurately identify opioid naïve patients at high risk for progression to COU and that prescription features would be key drivers of the risk scores. We developed, validated, and compared five ML models to predict progression from acute to COU amongst adult Medicaid recipients from six anonymized states. The objective of this work is to identify modifiable features associated with conversion from acute to chronic opioid use. This work may provide evidence that can guide stakeholders in their strategic policy developments.
Methods
Data source
Deidentified Medicaid claims were made available by the Digital Health Cooperative Research Center from 3 southeastern states, 2 western states, and 1 midwestern state from 2015–2019. Of these states, two reported data for the entire Medicaid population, one reported only fee-for service claims, and three states reported claims for Medicaid Managed Care plan enrollees, which represented 15–53% of the state’s Medicaid population. (Table A in S1 Text). Our study followed the Minimal Information for Medical AI Reporting (MINIMAR) guideline for prediction model development and validation [22]. This project utilized a de-identified limited dataset.
Study population
Fig 1 illustrates cohort development in a CONSORT diagram. Our dataset included information from enrollment, inpatient, outpatient, dental and pharmacy Medicaid claims files. Patients aged 18 to 65 years with record of opioid prescription, 2 months of continuous enrollment prior to opioid prescription, 9 months of continuous enrollment following prescription, and opioid naïve status (defined as having 2 months without opioid prescription prior to the encounter of interest) were included. Exclusion criteria were incomplete or denied prescription claims, malignancy, history of opioid use disorder, and buprenorphine as an initial prescription given primary usage for addiction treatment. Out of 463,880 eligible patients, 30,000 were randomly sampled from each state to avoid overfitting the model on states with higher populations, effectively upsampling in states that had less than 30,000 patients. The final cohort consisted of 180,000 patients.
Features
Demographic data collected included age and sex. Race and ethnicity data were not included due to inconsistent reporting across states. Opioid prescriptions were categorized from a list of National Drug Codes (NDCs) provided by the Center for Disease Control (Table B in S1 Text). International Disease Classification (ICD-9 and ICD-10) codes for comorbidities drawn from the Elixhauser and Charlson comorbidity indices were used in addition to those for chronic opioid abuse (Table C in S1 Text) [23,24]. Patient ZIP codes were used to generate county-level data regarding income, unemployment, education, and urban-rural classification made available by the Economic Research Service of the United States Department of Agriculture. Other features were selected with input from clinical experts regarding potential factors that may affect outcomes.
Features were removed if they were not compatible between states, had high levels of missingness, or had low levels of variance (Table D in S1 Text). For all variables included, missingness was less than 5%. We used a one-year time window prior to initial opioid prescription to look for features in the datasets. Like the other time windows, this was chosen as a tradeoff between collecting sufficient data and standardizing between different individual’s lengths of eligibility records.
Indicator features were prior prescriptions of the common anti-neuropathic pain medications gabapentin and pregabalin, in addition to the comorbidities. Prescription-related features collected included daily MME, opioid type, prescription days supply, and visit type (elective, urgent, emergent, trauma).
Outcomes
The primary outcome of progression to COU was defined as receiving an additional opioid prescription 3 to 9 months after filling the initial prescription. Three months was chosen as the lower limit for the timeframe based on Center for Disease Control guidelines for opioid prescription, in which chronic pain was defined as pain lasting over 3 months [25]. In prior work, the upper limit for the timeframe for chronic opioid use ranged from 6 months to 1 year; 9 months was chosen as an intermediary value [26,27].
Machine learning models
The study population was randomly split into a training cohort (80%), in which the COU prediction models were derived, and a validation cohort (20%) in which the models were applied and tested. To predict progression from opioid naïve status to COU, we developed five predictive models. A dataset consisting of 80% of the available data was used for model training. Models representing a range of computational complexity with prior success in biomedical applications were selected for inclusion: principal component analysis (PCA), logistic regression, XGBOOST, and multilayer perceptron. Hyperparameters for all models were tuned using a grid search with 5-fold cross validation, optimizing for area under the ROC curve (auROC). Missing values were replaced using mean imputation from training results. Numeric features were standardized to a zero mean and unit variance. Code has been made publicly available on GitHub (https://scikit-learn.org/stable/modules/generated/sklearn.impute.SimpleImputer.html).
Model evaluation
One state was held out from all training data as the validation set and 20% of the remaining 5 states were held out as a test set. Model performance was evaluated by comparing mean 10-fold time-dependent incremental area under curve (iAUC) on the 20% test set. The percent of COU and non-CU in the training, testing, and validating set were similar to that in the total data set: Train: 0.292 COU, Test: 0.292 COU, Val: 0.317 COU. Precision-recall and receiver-operating characteristic curves were generated for each model using training and validation test sets.
Variable importance
Variable importance was determined by the selection frequency for the XGBoost model, which is described elsewhere [28].
Statistical analysis
Patient demographics were compared with unpaired t-tests for parametric data and analysis of variance (ANOVA) for parametric data, whereas the chi-square/Fisher’s exact tests were used for categorical variables. The Wilcoxon rank sum test for dependent samples was used to calculate the significance between the iAUCs. Analysis was performed using R software 4.0.3 (R Project for Statistical Computing), using two-sided tests and a significant threshold of p-value < 0.05. The parameters details can be found in the source code. All analyses were conducted using the scikit-learn machine learning module in Python and occurred between December 2018, and April 2021. Calibration curves were generated for each model in Figure A in S1 Text.
Results
Study population characteristics
In the cohort of 180,000 previously opioid naïve patients who were prescribed an opioid, 53,820 (29.9%) developed COU. Demographically, 113,960 (63.3%) were female with a mean age of 39.2 + 13.4 years (Table 1). COU patients were more likely to have a history of depression (16.7% vs. 11.6%, p<0.0001), chronic pulmonary disease (14.2% vs. 10.5%, p<0.0001), psychosis (8.8% vs. 7.3%, p<0.0001), diabetes (16.9% vs. 10.6%, p<0.0001), hypertension (19.0%, 10.8%, p<0.0001), and obesity (10.6% vs. 9.0% p<0.0001) compared to non-COU patients. Regarding utilization, COU patients had a greater number of mean prior Medicaid claims (46.7 vs. 42.8, p<0.0001), total prescriptions (7.5 vs 6.7, p<0.0001), days supply (26.6 vs 24.7, p<0.0001), and pills per prescription (62.4 vs. 55.6, p<0.0001) compared to non-COU patients. Prior gabapentinoids were prescribed more frequently in COU patients for pregabalin (7.0% vs. 3.4%, p<0.0001) and gabapentin (25.5% vs 15.2%, p<0.0001) compared to non-COU patients.
Characteristics of the initial opioid prescription identified for each patient are described in Table 2. Compared to patients with prescriptions limited to the acute pain period, COU patients received greater MME (33.2 vs. 23.2, p<0.0001), tablets per prescription (45.6 vs. 36.54, p<0.0001) and LA medications (2.55% vs 0.47%, p<0.0001). This difference persisted when the analysis was limited to patients with a chronic pain diagnosis (ICD-10 G89), with COU averaging 39.0 tablets per prescription and non-COU patients averaging 65.9 tablets per prescription (p<0.0001) Medications prescribed more frequently in COU patients included hydromorphone (0.56% vs. 0.42%, p = 0.0003), methadone (0.34% vs. 0.07%, p<0.0001), LA morphine (1.38% vs. 0.21%, p<0.0001), LA oxycodone (0.24% vs. 0.04%, p<0.0001), and tramadol (16.06% vs. 13.44%, p<0.0001) compared to non-COU patients. These patient demographic and prescription-level variables were incorporated as features used to train the machine learning models.
Hexagonal bin plots illustrating trends in prescribing practices are illustrated in Fig 2. The majority of prescriptions contained less than 40 tablets for a 10 day supply, with less up than 40 MME/day.
Model variable importance
The top 15 features in terms of variable importance for the XGBoost model are shown in Fig 3. Top predictors included characteristics of the initial opioid prescription including tablets per prescription, gabapentin prescription, and prescription length. The total list of variables can be found in Table E in S1 Text.
Model performance
Of the generated models, XGBOOST had the highest auROC, area under the precision recall curve (auPRC), and F1 score of 0.80, 0.68, and 0.61 respectively (Table 3). Precision recall curves and receiver operating characteristic curves for the XGBOOST model are displayed in Fig 4, with mean precision of 0.55 and auROC of 0.69 when using validation data. Fig 5 illustrates the relationship between tablet quantity and percentage of COU; notably, over 50% of COU patients in the cohort received at least 40 tablets in their initial opioid prescription.
Precision-Recall and ROC Curve for the XGBoost Model for the test dataset (A) and the validation dataset (B).
The XGBOOST model’s performance on validation data is included.
Discussion
In this cohort study, machine learning models based on Medicaid claims data accurately predicted progression from acute to COU among opioid naïve patients. Since Medicaid patients experience disproportionate rates of pain-related comorbidities, we hypothesized that the incidence of COU in the study cohort would be higher. Indeed, our cohort had a 29.9% rate of progression from acute to chronic opioid use, which is greater than published estimates of 10% in Medicare Part D beneficiaries [29]. COU incidence was even modestly higher than rates reported in disabled Medicare and injured Veterans Affairs populations, which ranged from 21.4% to 27% [30,31]. This may be partially explained by medical comorbidities, as patient features such as increased age, diabetes, depression, and obesity were associated with increased COU risk [32]. Social determinants of health were also relevant, as patients residing in counties with increased poverty and unemployment risk had a significantly higher risk of COU. These findings suggest that some of the risk factors previously identified in non-Medicaid populations generalized to the study cohort, although the overall incidence of COU was higher.
Within the cohort, COU risk varied by underlying chronic conditions, such as depression, diabetes, and obesity. On average, COU patients received more prescriptions, which contained a greater number of pills and MMEs for a longer length and days supply compared to non-COU patients. The ability to identify high-risk prescription features is important for policymakers and payers who currently target strategies for addressing the contribution of prescription opioids to the opioid epidemic.
In the best performing predictive model, relevant prescription-related features were tablets per prescription, previous gabapentin prescription, prescription length, and incident tramadol or LA oxycodone prescription in descending order of importance. XGBOOST, the best-performing model, is known for efficiency in large datasets and the ability to handle mixtures of numerical and categorical features. These traits may have been particularly advantageous for our cohort, which included states with diverse approaches to prescription limits: 1 state restricted tablet quantity, 4 states restricted days supply, and 1 state restricted days supply and MME.
Overall, tablet count was the most important feature in predicting COU. We demonstrate a dose-dependent relationship between tablets prescribed and COU risk; 40% of COU patients in our cohort received 20 or greater tablets in their initial prescription, while 60% of COU patients received at 50 or more tablets. This is consistent with prior studies of commercially insured opioid-naïve patients, in which increased tablet count was associated with increased risk of developing opioid use disorder [33]. A possible confounder is pain severity, as patients who present with more intense or long-lasting pain symptoms are likely to receive more initial tablets and request refills. However, our study showed that even among patients with a chronic pain diagnosis, patients progressing to COU had a higher tablet count compared to patients with chronic pain not progression to COU. Prior literature demonstrates that many patients receive excess narcotics [34], our findings suggest that reducing the quantity of opioid tablets prescribed to opioid-naïve patients may assist in reducing COU.
By using a discrete numerical cutoff, tablet quantity restrictions also have the benefit of being more easily integrated into clinical workflows. Healthcare systems could protocolize a tablet restriction or enable default tablet values within clinical decision support tools embedded in the electronic medical record, which have previously been shown to improve adherence to opioid-reduction strategies at the prescribing point of care [35–37].
Beyond individual healthcare institutions, our findings support limits on tablet quantity in opioid-related policymaking. Out of 28 states with prescribing limits for acute pain, only Colorado and Rhode Island explicitly restrict tablet quantities administered to 4 tablets per day and 20 doses overall, respectively [37,38]. The institution of tablet restrictions in Colorado was associated with a 24% decrease in mean total daily dose of opioids in patients who previously exceeded the limit [39]. Prescription length, the eighth most important feature in the model, is the most popular target for state-level opioid legislation within the cohort states and overall nationally. 24 states have enacted prescription length limits ranging from 3 to 30 days [38]. The majority of prescriptions in our cohort dispensed less than 20 day supply and were thus within compliance of at least five state limits on prescription length. Rising tablet counts may also represent compensatory behavior in prescribers faced with other dose-related restrictions; in disabled Medicare beneficiaries, Morden et al demonstrated that opioid tablet counts increased after 2011 while MME plateaued [30]. Daily MMEs are restricted in at least 3 states, but were not a relevant feature in the model.
Relevant medication-specific features including history of gabapentin prescription and incident tramadol prescription should also guide prescriber choice. Gabapentinoids have been increasingly promoted in opioid-sparing strategies in treatment of acute pain [40]. In the absence of further information regarding the clinical encounter that motivated prescription in this cohort, this finding raises questions about how gabapentinoids impact the pain trajectory of COU patients. As gabapentinoids have been associated with increased adverse events including sedation and respiratory depression when used in conjunction with opioids, further investigation is needed to evaluate the effects of co-prescribing opioids and gabapentinoids.
The higher prevalence of Tramadol prescriptions in patients progressing from acute opioid use to COU in our study merits further investigation. While Tramadol has been promoted as a safe opioid with a relatively low risk of addiction, it is notable that a weak opioid antagonist was prescribed more often in COU patients, while measures of opioid strength such as milligram morphine equivalents were not influential in the top-performing machine learning model. Taken in conjunction, our findings are aligned with recent reports that Tramadol may be a high-risk opioid for future abuse [41]. Similarly, Mudumbai et al demonstrated that postoperative time to opioid cessation was longer in veterans taking Tramadol preoperatively compared to those taking short-acting opioids [42]. These results contribute to recent literature suggesting that preferential prescribing of weak opioid agonists such as Tramadol is not equivalent to safer practice [43]. In some cases, tramadol prescription may serve as a ‘trojan horse,’ where its status as a relatively weak opioid receptor agonist may mask underlying risks regarding prolonged opioid use.
This study has important limitations. Our data represents a heterogenous collection of payer information from six states, which may not reflect the Medicaid population. However, the states include represent diverse geographical areas across the United States. Due to state policies, individual states cannot be named, which limits the interpretation of these results. However, given the size of the dataset, we generate important evidence that can guide policy decisions. Lastly, while comorbid features were selected from validated risk stratification tools, they are by no means exhaustive. Further evaluation of relevant features should be undertaken to improve the explanatory power and robustness of the model; features not easily captured in claims data such as patient access to non-opioid pain therapy may influence opioid prescribing habits. Without further data surrounding the clinical context at time of prescription, it is not possible to conclude whether an individual prescribing event was inappropriate or excessive. However, clear trends are identified through these analyses that provide insight to the prescription opioid problem amount Medicaid recipients. Our work demonstrates that machine learning models can be applied to opioid-related endpoints with good discrimination and performance; future work exploring the relationship between acute to chronic opioid use in subpopulations such as patients with acute pain conditions may further inform opioid-related policy.
Conclusion
In a machine learning model geared toward predicting progression from acute to chronic opioid use in an opioid-naive Medicaid cohort, prescriptions with increased tablet quantity and prescription length were predictive. Despite state-level legislation, there continues to be wide variation in prescriber practices, although the majority of analyzed prescriptions dispensed fewer than 40 tablets for a 20 day supply. Future research should evaluate the effects of restricting opioid tablet quantity on COU incidence.
Supporting information
S1 Text.
Table A. State Demographics. Table B. Opioids selected for inclusion, based on the National Drug Codes provided by the Center for Disease Control. Table C. ICD-9 and ICD-10 codes used for demographic data collection. Table D. Variable Missingness. Table E. Full List of Features by Code and Description. Figure A. Model Calibration Curves for PCA.
https://doi.org/10.1371/journal.pdig.0000075.s001
(DOCX)
Acknowledgments
The authors would like to thank Bill Lucia, Donna Price and Gary Call for their support in understanding the Medicaid datasets and variables of interest.
The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or the DHCRC
References
- 1. Stephenson J. Drug overdose deaths head toward record number in 2020, CDC warns. American Medical Association; 2020:e201318–e201318.
- 2. Florence CS, Zhou C, Luo F, Xu L. The Economic Burden of Prescription Opioid Overdose, Abuse, and Dependence in the United States, 2013. Med Care. 10 2016;54(10):901–6. pmid:27623005
- 3. Buchmueller TC, Carey C. The effect of prescription drug monitoring programs on opioid utilization in Medicare. American Economic Journal: Economic Policy. 2018;10(1):77–112.
- 4. Keast SL, Kim H, Deyo RA, et al. Effects of a prior authorization policy for extended-release/long-acting opioids on utilization and outcomes in a state Medicaid program. Addiction. Apr 20 2018; pmid:29679440
- 5. Lee B, Zhao W, Yang KC, Ahn YY, Perry BL. Systematic Evaluation of State Policy Interventions Targeting the US Opioid Epidemic, 2007–2018. JAMA Netw Open. 02 01 2021;4(2):e2036687. pmid:33576816
- 6. Buchmueller TC, Levinson ZM, Levy HG, Wolfe BL. Effect of the Affordable Care Act on Racial and Ethnic Disparities in Health Insurance Coverage. Am J Public Health. Aug 2016;106(8):1416–21. pmid:27196653
- 7. Chapel JM, Ritchey MD, Zhang D, Wang G. Prevalence and Medical Costs of Chronic Diseases Among Adult Medicaid Beneficiaries. Am J Prev Med. Dec 2017;53(6S2):S143–S154. pmid:29153115
- 8. Grogan CM, Andrews C, Abraham A, et al. Survey Highlights Differences In Medicaid Coverage For Substance Use Treatment And Opioid Use Disorder Medications. Health Aff (Millwood). 12 01 2016;35(12):2289–2296. pmid:27920318
- 9. Mack KA, Zhang K, Paulozzi L, Jones C. Prescription practices involving opioid analgesics among Americans with Medicaid, 2010. J Health Care Poor Underserved. Feb 2015;26(1):182–98. pmid:25702736
- 10. Han B, Compton WM, Jones CM, Cai R. Nonmedical Prescription Opioid Use and Use Disorders Among Adults Aged 18 Through 64 Years in the United States, 2003–2013. JAMA. Oct 13 2015;314(14):1468–78. pmid:26461997
- 11. Wen H, Hockenberry JM, Borders TF, Druss BG. Impact of Medicaid Expansion on Medicaid-covered Utilization of Buprenorphine for Opioid Use Disorder Treatment. Med Care. 04 2017;55(4):336–341. pmid:28296674
- 12. Shah A, Hayes CJ, Martin BC. Characteristics of Initial Prescription Episodes and Likelihood of Long-Term Opioid Use—United States, 2006–2015. MMWR Morb Mortal Wkly Rep. Mar 17 2017;66(10):265–269. pmid:28301454
- 13. Campbell TJ, Martins D, Tadrous M, et al. Dental Opioid Prescription Characteristics and the Risk of New, Persistent Use. Am J Prev Med. 06 2021;60(6):831–839. pmid:33744063
- 14. Ellis RJ, Wang Z, Genes N, Ma’ayan A. Predicting opioid dependence from electronic health records with machine learning. BioData Min. 2019;12:3. pmid:30728857
- 15. Katakam A, Karhade AV, Collins A, et al. Development of machine learning algorithms to predict achievement of minimal clinically important difference for the KOOS-PS following total knee arthroplasty. J Orthop Res. 04 2022;40(4):808–815. pmid:34275163
- 16. Rice JB, White AG, Birnbaum HG, Schiller M, Brown DA, Roland CL. A model to identify patients at risk for prescription opioid abuse, dependence, and misuse. Pain Med. Sep 2012;13(9):1162–73. pmid:22845054
- 17. White AG, Birnbaum HG, Schiller M, Tang J, Katz NP. Analytic models to identify patients at risk for prescription opioid abuse. Am J Manag Care. Dec 2009;15(12):897–906. pmid:20001171
- 18. Lo-Ciganic WH, Huang JL, Zhang HH, et al. Evaluation of Machine-Learning Algorithms for Predicting Opioid Overdose Risk Among Medicare Beneficiaries With Opioid Prescriptions. JAMA Netw Open. 03 01 2019;2(3):e190968. pmid:30901048
- 19. Meisel ZF, Lupulescu-Mann N, Charlesworth CJ, Kim H, Sun BC. Conversion to Persistent or High-Risk Opioid Use After a New Prescription From the Emergency Department: Evidence From Washington Medicaid Beneficiaries. Ann Emerg Med. 11 2019;74(5):611–621. pmid:31229392
- 20. Garg RK, Fulton-Kehoe D, Franklin GM. Patterns of Opioid Use and Risk of Opioid Overdose Death Among Medicaid Patients. Med Care. 07 2017;55(7):661–668. pmid:28614178
- 21. Dilokthornsakul P, Moore G, Campbell JD, et al. Risk Factors of Prescription Opioid Overdose Among Colorado Medicaid Beneficiaries. J Pain. Apr 2016;17(4):436–43. pmid:26721613
- 22. Hernandez-Boussard T, Bozkurt S, Ioannidis JPA, Shah NH. MINIMAR (MINimum Information for Medical AI Reporting): Developing reporting standards for artificial intelligence in health care. J Am Med Inform Assoc. Jun 2020; pmid:32594179
- 23. Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Med Care. Jan 1998;36(1):8–27. pmid:9431328
- 24. D’Hoore W, Sicotte C, Tilquin C. Risk adjustment in outcome assessment: the Charlson comorbidity index. Methods Inf Med. Nov 1993;32(5):382–7. pmid:8295545
- 25. Dowell D, Haegerich TM, Chou R. CDC Guideline for Prescribing Opioids for Chronic Pain—United States, 2016. JAMA. Apr 19 2016;315(15):1624–45. pmid:26977696
- 26. Sun EC, Darnall BD, Baker LC, Mackey S. Incidence of and Risk Factors for Chronic Opioid Use Among Opioid-Naive Patients in the Postoperative Period. JAMA Intern Med. Sep 01 2016;176(9):1286–93. pmid:27400458
- 27. Brummett CM, Waljee JF, Goesling J, et al. New Persistent Opioid Use After Minor and Major Surgical Procedures in US Adults. JAMA Surg. Jun 21 2017;152(6):e170504. pmid:28403427
- 28. Weng SF, Reps J, Kai J, Garibaldi JM, Qureshi N. Can machine-learning improve cardiovascular risk prediction using routine clinical data? PLoS One. 2017;12(4):e0174944. pmid:28376093
- 29. Axeen S. Trends in Opioid Use and Prescribing in Medicare, 2006–2012. Health Serv Res. 10 2018;53(5):3309–3328. pmid:29532477
- 30. Morden NE, Munson JC, Colla CH, et al. Prescription opioid use among disabled Medicare beneficiaries: intensity, trends, and regional variation. Med Care. Sep 2014;52(9):852–9. pmid:25119955
- 31. Riva JJ, Noor ST, Wang L, et al. Predictors of Prolonged Opioid Use After Initial Prescription for Acute Musculoskeletal Injuries in Adults: A Systematic Review and Meta-analysis of Observational Studies. Ann Intern Med. 11 03 2020;173(9):721–729. pmid:32805130
- 32. Stokes A, Lundberg DJ, Hempstead K, Berry KM, Baker JF, Preston SH. Obesity and Incident Prescription Opioid Use in the U.S., 2000–2015. Am J Prev Med. 06 2020;58(6):766–775. pmid:32229057
- 33. Cochran BN, Flentje A, Heck NC, et al. Factors predicting development of opioid use disorders among individuals who receive an initial opioid prescription: mathematical modeling using a database of commercially-insured individuals. Drug Alcohol Depend. May 01 2014;138:202–8. pmid:24679839
- 34. Sabatino MJ, Kunkel ST, Ramkumar DB, Keeney BJ, Jevsevar DS. Excess Opioid Medication and Variation in Prescribing Patterns Following Common Orthopaedic Procedures. J Bone Joint Surg Am. Feb 07 2018;100(3):180–188. pmid:29406338
- 35. Price-Haywood EG, Burton J, Burstain T, et al. Clinical Effectiveness of Decision Support for Prescribing Opioids for Chronic Noncancer Pain: A Prospective Cohort Study. Value Health. 02 2020;23(2):157–163. pmid:32113620
- 36. Delgado MK, Shofer FS, Patel MS, et al. Association between Electronic Medical Record Implementation of Default Opioid Prescription Quantities and Prescribing Behavior in Two Emergency Departments. J Gen Intern Med. 04 2018;33(4):409–411. pmid:29340937
- 37. Chiu AS, Jean RA, Hoag JR, Freedman-Weiss M, Healy JM, Pei KY. Association of Lowering Default Pill Counts in Electronic Medical Record Systems With Postoperative Opioid Prescribing. JAMA Surg. 11 01 2018;153(11):1012–1019. pmid:30027289
- 38. Davis CS, Lieberman AJ, Hernandez-Delgado H, Suba C. Laws limiting the prescribing or dispensing of opioids for acute pain in the United States: A national systematic legal review. Drug Alcohol Depend. 01 01 2019;194:166–172. pmid:30445274
- 39. Riggs CS, Billups SJ, Flores S, Patel RJ, Heilmann RM, Milchak JL. Opioid Use for Pain Management After Implementation of a Medicaid Short-Acting Opioid Quantity Limit. J Manag Care Spec Pharm. Mar 2017;23(3):346–354. pmid:28230448
- 40. Hah JM, Hilmoe H, Schmidt P, et al. Preoperative Factors Associated with Remote Postoperative Pain Resolution and Opioid Cessation in a Mixed Surgical Cohort: Post Hoc Analysis of a Perioperative Gabapentin Trial. J Pain Res. 2020;13:2959–2970. pmid:33239904
- 41. Dunn KE, Bergeria CL, Huhn AS, Strain EC. A Systematic Review of Laboratory Evidence for the Abuse Potential of Tramadol in Humans. Front Psychiatry. 2019;10:704. pmid:31616329
- 42. Mudumbai SC, Oliva EM, Lewis ET, et al. Time-to-Cessation of Postoperative Opioids: A Population-Level Analysis of the Veterans Affairs Health Care System. Pain Med. 09 2016;17(9):1732–43. pmid:27084410
- 43. Thiels CA, Habermann EB, Hooten WM, Jeffery MM. Chronic use of tramadol after acute pain episode: cohort study. BMJ. May 14 2019;365:l1849. pmid:31088782