Skip to main content

Advertisement

SpringerLink
Log in

Aiding the prescriber: developing a machine learning approach to personalized risk modeling for chronic opioid therapy amongst US Army soldiers

  • Published:
Health Care Management Science Aims and scope Submit manuscript

Abstract

The opioid epidemic is a major policy concern. The widespread availability of opioids, which is fueled by physician prescribing patterns, medication diversion, and the interaction with potential illicit opioid use, has been implicated as proximal cause for subsequent opioid dependence and mortality. Risk indicators related to chronic opioid therapy (COT) at the point of care may influence physicians’ prescribing decisions, potentially reducing rates of dependency and abuse. In this paper, we investigate the performance of machine learning algorithms for predicting the risk of COT. Using data on over 12 million observations of active duty US Army soldiers, we apply machine learning models to predict the risk of COT in the initial months of prescription. We use the area under the curve (AUC) as an overall measure of model performance, and we focus on the positive predictive value (PPV), which reflects the models’ ability to accurately target military members for intervention. Of the many models tested, AUC ranges between 0.83 and 0.87. When we focus on the top 1% of members at highest risk, we observe a PPV value of 8.4% and 20.3% for months 1 and 3, respectively. We further investigate the performance of sparse models that can be implemented in sparse data environments. We find that when the goal is to identify patients at the highest risk of chronic use, these sparse linear models achieve a performance similar to models trained on hundreds of variables. Our predictive models exhibit high accuracy and can alert prescribers to the risk of COT for the highest risk patients. Optimized sparse models identify a parsimonious set of factors to predict COT: initial supply of opioids, the supply of opioids in the month being studied, and the number of prescriptions for psychotropic medications. Future research should investigate the possible effects of these tools on prescriber behavior (e.g., the benefit of clinician nudging at the point of care in outpatient settings).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Data availability

The data used in this study is proprietary and will not be made available.

Notes

  1. Opioid use disorder is a medical condition specified in the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (2013). It is caused by a pattern of opioid use that results in eventual dependency. Symptoms include an inability to control or reduce use, use of larger amounts over time, and the development of tolerance (https://www.samhsa.gov/disorders/substance-use).

References

  1. Volkow ND, McLellan AT (2016) Opioid abuse in chronic pain—misconceptions and mitigation strategies. N Engl J Med 374(13):1253–1263

    Article  Google Scholar 

  2. U.S. Department of Health & Human Services, Centers for Disease Control and Prevention. Overdose Deaths Accelerating During COVID-19 (2020) https://emergency.cdc.gov/han/2020/han00438.asp. Accessed 27 Jan 2021

  3. Canan C, Polinski JM, Alexander GC et al (2017) Automatable algorithms to identify nonmedical opioid use using electronic data: a systematic review. J Am Med Inform Assoc 24(6):204–1210

    Article  Google Scholar 

  4. Johannes CB, Le TK, Zhou X et al (2010) The Prevalence of Chronic Pain in United States Adults: Results of an Internet-Based Survey. J Pain 11(11):1230–1239

    Article  Google Scholar 

  5. Institute of Medicine (2011) Relieving pain in America: a blueprint for transforming prevention, care, education and research. National Academies Press, Washington, DC

    Google Scholar 

  6. U.S. Department of Health & Human Services, Centers for Disease Control and Prevention, National Center for Injury Prevention and Control, Division of Unintentional Injury Prevention. U.S. Opioid Prescribing Rate Maps (2018) https://www.cdc.gov/drugoverdose/maps/rxrate-maps.html. Accessed 5 Feb 2019

  7. U.S. Department of Health & Human Services, Centers for Disease Control and Prevention Prescription Opioid Data (2018) https://www.cdc.gov/drugoverdose/data/prescribing.html. Accessed 5 Feb 2019

  8. U.S. Department of Health & Human Services, Centers for Disease Control and Prevention (2017) Opioid prescribing is still high and varies widely throughout the U.S. 2017 https://www.cdc.gov/media/releases/2017/p0706-opioid.html. Accessed 30 Jan 2018

  9. Chou R, Turner JA, Devine EB et al (2015) The effectiveness and risks of long-term opioid therapy for chronic pain: a systematic review for a National Institutes of Health Pathways to Prevention Workshop. Ann Intern Med 162(4):276–286

    Article  Google Scholar 

  10. Dowell D, Haegerich TM, Chou R (2016) CDC Guideline for Prescribing Opioids for Chronic Pain — United States, 2016. MMWR Recomm Rep. 65(No. RR-1):1–49

    Article  Google Scholar 

  11. Crofford LJ (2010) Adverse effects of chronic opioid therapy for chronic musculoskeletal pain. Nat Rev Rheumatol 6(4):191

    Article  Google Scholar 

  12. Larochelle MR, Liebschutz JM, Zhang F et al (2016) Opioid prescribing after nonfatal overdose and association with repeated overdose: a cohort study. Ann Intern Med 164(1):1–9

    Article  Google Scholar 

  13. Baldini A, Von Korff M, Lin EH (2012) A review of potential adverse effects of long-term opioid therapy: a practitioner’s guide. The primary care companion to CNS disorders 14(3)

  14. Finley EP, Schneegans S, Tami C et al (2018) Implementing prescription drug monitoring and other clinical decision support for opioid risk mitigation in a military health care setting: a qualitative feasibility study. J Am Med Inform Assoc 25(5):515–522

    Article  Google Scholar 

  15. Sharma M, Ugiliweneza B, Aljuboori Z, Nuno MA, Drazin D, Boakye M (2018) Factors predicting opioid dependence in patients undergoing surgery for degenerative spondylolisthesis: analysis from the MarketScan databases. J Neurosurg Spine 29(3):271–278

    Article  Google Scholar 

  16. Karhade AV, Cha TD, Fogel HA, Hershman SH, Tobert DG, Schoenfeld AJ, Bono CM, Schwab JH (2020) Predicting prolonged opioid prescriptions in opioid-naïve lumbar spine surgery patients. The Spine Journal 20(6):888–895

    Article  Google Scholar 

  17. Sabesan VJ, Chatha K, Goss L, Ghisa C, Gilot G (2019) Can patient and fracture factors predict opioid dependence following upper extremity fractures?: a retrospective review. J Orthop Surg Res 14(1):1–5

    Article  Google Scholar 

  18. Ciesielski T, Iyengar R, Bothra A et al (2016) A Tool to Assess Risk of De Novo Opioid Abuse or Dependence. Am J Med 129(7):699–705

    Article  Google Scholar 

  19. Rice JB, White AG, Birnbaum HG, Schiller M, Brown DA, Roland CL (2012) A model to identify patients at risk for prescription opioid abuse, dependence, and misuse. Pain Med 13(9):1162–73

    Article  Google Scholar 

  20. Cochran BN, Flentje A, Heck NC, Van Den Bos J, Perlman D, Torres J, Valuck R, Carter J (2014) Factors predicting development of opioid use disorders among individuals who receive an initial opioid prescription: mathematical modeling using a database of commercially-insured individuals. Drug Alcohol Depend 1(138):202–208

    Article  Google Scholar 

  21. Hylan TR, Von Korff M, Saunders K, Masters E, Palmer RE, Carrell D, Cronkite D, Mardekian J, Gross D (2015) Automated prediction of risk for problem opioid use in a primary care setting. J Pain 16(4):380–387

    Article  Google Scholar 

  22. Hastings JS, Howison M, Inman SE (2020) Predicting high-risk opioid prescriptions before they are given. Proc Natl Acad Sci 117(4):1917–1923

    Article  Google Scholar 

  23. Dufour R, Mardekian J, Pasquale MK, Schaaf D, Andrews GA, Patel NC (2014) Understanding predictors of opioid abuse: predictive model development and validation. Available at: https://www.pharmacytimes.com/view/understanding-predictors-of-opioid-abuse-predictive-model-developmentand-validation

  24. Ellis RJ, Wang Z, Genes N, Ma’ayan A (2019) Predicting opioid dependence from electronic health records with machine learning. BioData Mining. 12(1):1–9

    Article  Google Scholar 

  25. Butler SF, Fernandez K, Benoit C et al (2008) Validation of the Revised Screener and Opioid Assessment for Patients With Pain (SOAPP-R). J Pain 9(4):360–372

    Article  Google Scholar 

  26. Webster LR, Webster RM (2005) Predicting aberrant behaviors in opioid-treated patients: preliminary validation of the Opioid Risk Tool. Pain Med 6(6):432–442

    Article  Google Scholar 

  27. Bruehl S, Apkarian AV, Ballantyne JC et al (2013) Personalized medicine and opioid analgesic prescribing for chronic pain: opportunities and challenges. J Pain 14(2):103–113

    Article  Google Scholar 

  28. Volkow Nora D, Thomas McLellan A (2016) Opioid abuse in chronic pain—misconceptions and mitigation strategies. N Engl J Med. 374(13):1253–1263

    Article  Google Scholar 

  29. Harle CA, Bauer SE, Hoang HQ et al (2015) Decision support for chronic pain care: how do primary care physicians decide when to prescribe opioids? a qualitative study. BMC Fam Pract 16(1):48

    Article  Google Scholar 

  30. Pauly JP, Michailidis L, Kindred MG et al (2017) Predictors of Chronic Opioid Use in Newly Diagnosed Crohn’s Disease. Inflamm Bowel Dis 23(6):1004–1010

    Article  Google Scholar 

  31. Chou R, Fanciullo GJ, Fine PG et al (2009) Opioids for chronic noncancer pain: prediction and identification of aberrant drug-related behaviors: a review of the evidence for an American Pain Society and American Academy of Pain Medicine clinical practice guideline. J Pain 10(2):131–146

    Article  Google Scholar 

  32. Dunn KM, Saunders KW, Rutter CM et al (2010) Opioid prescriptions for chronic pain and overdose: a cohort study. Ann Intern Med 152(2):85–92

    Article  Google Scholar 

  33. Shah A, Hayes CJ, Martin BC (2017) Characteristics of Initial Prescription Episodes and Likelihood of Long-Term Opioid Use — United States, 2006–2015. MMWR Morb Mortal Wkly Rep 66:265–269

    Article  Google Scholar 

  34. Brenton A, Richeimer S, Sharma M et al (2017) Observational study to calculate addictive risk to opioids: a validation study of a predictive algorithm to evaluate opioid use disorder. Pharmgenomics Pers Med 10:187–195

    Google Scholar 

  35. Zedler B, Xie L, Wang L et al (2014) Risk factors for serious prescription opioid-related toxicity or overdose among Veterans Health Administration patients. Pain Med 15(11):1911–1929

    Article  Google Scholar 

  36. Webster LR (2017) Risk factors for opioid-use disorder and overdose. Anesth Analg 125(5):1741–1748

    Article  Google Scholar 

  37. Zhao S, Chen F, Feng A, Han W, Zhang Y (2019) Risk factors and prevention strategies for postoperative opioid abuse. Pain Res Manage 10:2019

    Google Scholar 

  38. Lawal OD, Gold J, Murthy A, Ruchi R, Bavry E, Hume AL, Lewkowitz AK, Brothers T, Wen X (2020) Rate and risk factors associated with prolonged opioid use after surgery: a systematic review and meta-analysis. JAMA network open. 3(6):e207367

    Article  Google Scholar 

  39. Park TW, Lin LA, Hosanagar A, Kogowski A, Paige K, Bohnert AS (2016) Understanding risk factors for opioid overdose in clinical populations to inform treatment and policy. J Addict Med 10(6):369–381

    Article  Google Scholar 

  40. Rudin C (2019) Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat Mach Intell 1(5):206–215

    Article  Google Scholar 

  41. Tibshirani R (1996) Regression shrinkage and selection via the lasso. J R Stat Soc Series B Stat Methodol 58:267–288

    Google Scholar 

  42. Meinshausen N (2007) Relaxed lasso. Comput Stat Data Anal 52(1):374–393

    Article  Google Scholar 

  43. Haiste T, Tibshrirani R, Friedman J (2008) Elements of Statistical Learning. Springer Series in Statistics, 2nd edn. Springer Science+Business Media, New York

    Google Scholar 

  44. Chen T, He T, Benesty M, et al. Package ‘xgboost’. 2021, https://cran.r-project.org/web/packages/xgboost/xgboost.pdf. Accessed 29 Nov 2021

  45. Culp M, Johnson K, Michailidis F. Package ‘ada’. 2021, https://cran.r-project.org/web/packages/ada/ada.pdf. Accessed 30 Nov 2021

  46. Lunardon N, Menardi G, Torelli N (2014) ROSE: a package for binary imbalanced learning. R J.6(1)

  47. Zeng J, Ustun B, Rudin C (2017) Interpretable classification models for recidivism prediction. J R Stat Soc Ser A Stat Soc 180(3):689–722

    Article  Google Scholar 

  48. Ustun B, Rudin C (2016) Supersparse linear integer models for optimized medical scoring systems. Mach Learn 102(3):349–391

    Article  Google Scholar 

  49. Souillard-Mandar W, Davis R, Rudin C et al (2016) Learning Classification Models of Cognitive Conditions from Subtle Behaviors in the Digital Clock Drawing Test. Mach Learn 102(3):393–441

    Article  Google Scholar 

  50. Cox J, Holden J, Sagovsky R (1987) Detection of Postnatal Depression: Development of the 10-item Edinburgh Postnatal Depression Scale. Br J Psychiatry 150(6):782–786

    Article  Google Scholar 

  51. Wilson PWF, D’Agostino RB, Levy D et al (1998) Prediction of coronary heart disease using risk factor categories. Circulation 97(18):1837–1847

    Article  Google Scholar 

  52. Knaus WA, Wagner DP, Draper EA et al (1991) The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest. 100(6):1619–36

    Article  Google Scholar 

  53. Rudin C. Optimized scoring systems for classification problems in MATLAB. https://github.com/ustunb/slim-matlab. Accessed 1 Aug 2017

  54. Embi PJ, Leonard AC (2012) Evaluating alert fatigue over time to EHR-based clinical trial alerts: findings from a randomized controlled study. J Am Med Inform Assoc 19:e145e–e1148

    Article  Google Scholar 

  55. Siwicki, B. Health system uses Epic EHR, communications tech to reduce sepsis mortality rate by 20%. Healthcare IT News. 2019, https://www.healthcareitnews.com/news/health-system-uses-epic-ehr-communications-tech-reduce-sepsis-mortality-rate-20. Accessed 28 Nov 2021

  56. Escobar GJ, Liu VX, Schuler A et al (2020) Automated identification of adults at risk for in-hospital clinical deterioration. N Engl J Med 383(20):1951–1960

    Article  Google Scholar 

  57. Chak E, Taefi A, Li CS et al (2018) Electronic medical alerts increase screening for chronic hepatitis B: a randomized, double-blind, controlled trial. Cancer Epidemiology and Prevention Biomarkers 27(11):1352–1357

    Article  Google Scholar 

  58. U.S. Department of Health & Human Services, Centers for Disease Control and Prevention. 2018. Implementing Clinical Decision Support Systems. https://www.cdc.gov/dhdsp/pubs/docs/Best_Practice_Guide_CDSS_508.pdf. Accessed 16 Jul 2018

  59. Weinstein MC, Russell LB, Gold MR,  Siegel JE (1996) Cost-effectiveness in health and medicine. Oxford University Press

Download references

Funding

The authors gratefully acknowledge the support of the National Institute for Healthcare Management (NIHCM) Foundation.

Author information

Authors and Affiliations

  1. Robert H. Smith School of Business, Decision, Operations and Information Technologies, University of Maryland, Van Munching Hall, College Park, MD, USA

    Margrét Vilborg Bjarnadóttir & Ritu Agarwal

  2. School of Business, Management and Operations, Villanova University, Villanova, PA, USA

    David B. Anderson

  3. Carey Business School, Johns Hopkins University, Baltimore, MD, USA

    Ritu Agarwal

  4. School of Medicine, Department of Medicine, Division of Primary Care and Population Health, Stanford University, Stanford, CA, USA

    D. Alan Nelson

Authors
  1. Margrét Vilborg Bjarnadóttir
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David B. Anderson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ritu Agarwal
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. D. Alan Nelson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Margrét Vilborg Bjarnadóttir.

Ethics declarations

Conflicts of interest/Competing interest

The authors have no competing interests to report.

Code availability

The authors will make their code available upon request.

Ethics approval

The study was approved by the IRB board of University of Maryland College Park.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix 1: Summary statistics comparing the training and testing samples

Table 4 Summary statistics for the training and testing populations, all numbers reported at the first month of opioid prescription
Full size table

Appendix 2: Misclassification Matrices

Misclassification matrices are a function of the selected risk cut-off. Below we therefore summarize the misclassification matrices for the testing data when the cut-offs are selected to classify 1% and 5% of the training data as at risk of COT.

Table 5 Misclassification matrices for LASSO labelling 5% of the population at risk
Full size table
Table 6 Misclassification matrices for LASSO labelling 1% of the population at risk
Full size table
Table 7 Misclassification matrices for SLIM labelling 5.4% of the population at risk in month 1 and 6.2% at risk of COT in moth 3
Full size table
Table 8 Misclassification matrices for SLIM labelling 1.0% of the population at risk in month 1 and 1.2% at risk of COT in moth 3
Full size table

Appendix 3: Complete set of data elements

Table 9 Demographic variables
Full size table
Table 10 Variables capturing employment including deployment
Full size table
Table 11 Other control variables
Full size table
Table 12 Variables capturing physical fitness and substance use
Full size table
Table 13 Variables capturing overall healthcare utilization and behavior
Full size table
Table 14 Variables capturing opioid and other prescription use. All variables contain information on the most recent month or the newest information available unless otherwise specified
Full size table
Table 15 Variables capturing pain
Full size table
Table 16 Variables capturing mental and behavioral health. All variables contain information on the most recent month (unless otherwise stated)
Full size table
Table 17 Variables capturing other health problems
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bjarnadóttir, M.V., Anderson, D.B., Agarwal, R. et al. Aiding the prescriber: developing a machine learning approach to personalized risk modeling for chronic opioid therapy amongst US Army soldiers. Health Care Manag Sci 25, 649–665 (2022). https://doi.org/10.1007/s10729-022-09605-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10729-022-09605-4

Keywords

Search

Navigation