Glanz et al. took on a very ambitious and important project—developing and validating a prediction model for opioid overdose among patients on long-term opioid therapy (LTOT)—and conducted a rigorous study that offers provocative results.1 Without question, the study and its findings should be considered foundational for any future efforts aiming to refine a clinically actionable prediction model. However, whether the study’s findings on their own should drive changes in treatment recommendations and policy, particularly with regard to naloxone distribution, is less clear.

First, the authors conducted a well-designed, well-executed, and clearly described study. The rationale for the use of predictive modeling in clinical risk scenarios was compelling, and the clear justifications and explanations for various design decisions are exemplary and worth emulating by others in the field. The development of a model that has five components that are typically readily available in a clinical encounter is also a welcome advance.

The potential for immediate clinical application is not as certain. Some of the model’s operating characteristics should give readers pause; to their credit, the authors highlight these issues. For example, while sensitivity of the model (i.e., proportion of those who overdosed that was predicted by the model) was 82%, the positive predictive value (PPV)—was low, ranging from 0.56 to 1.8% in the derivation and validation cohorts, respectively. This is especially concerning because PPV is more well-aligned with how predictive models are best used in clinical practice. In this case, for every 1000 patients that the model predicted were at high risk (i.e., had a positive test), only about 10 experienced an overdose. Positive predictive value varies as a function of prevalence, and the low PPV is in part due to the fact that overdoses, even in this epidemic era, are thankfully relatively rare events.

The authors predicated the importance of a clinically actionable predictive model on its value in helping LTOT prescribers decide who should receive the overdose reversal drug naloxone, an opioid receptor antagonist. There is no cost effectiveness data (or randomized controlled trial data) to date on the use of naloxone in primary care populations prescribed LTOT. It may be an over-reach to recommend applying this predictive model to determine whether to prescribe a $150–$4500 treatment that will not be needed ~ 99% of the time and can be administered by emergency medical responders who should be called in any case. That said, the larger issue here is not the authors’ potentially over-valuing naloxone among patients on LTOT for chronic pain; it is the unethically exorbitant cost of naloxone.2 In systems where naloxone costs are controlled (and dramatically lower), it is an easier clinical and policy decision to give naloxone’s effectiveness data in this population the benefit of the doubt.3 The predictive model may be at least as (or more) valuable in helping LTOT prescribers determine, for example, in whom it is critical to avoid benzodiazepine co-prescribing or to help motivate recipients of both classes of medications (opioids and benzodiazepines) to discontinue one or the other, although benzodiazepine use was not found to be predictive in this model.

A methodologic weakness of the study is that dose of LTOT was not time-updated in this analysis. Several studies have demonstrated that starting doses of LTOT are much more tightly clustered (and lower) than doses 6 months or certainly 2 years later.4 Thus, while the authors did not find an effect of dose based on more tightly clustered starting doses, there may well have been an effect of more widely distributed dose ranges at the time of overdose. The authors’ explanation that dose was not time-updated because prescribers do not have access to future data when they make treatment decisions implies that prescribers only have one opportunity—at the start of LTOT—to predict risk and make treatment decisions. In fact, prescribers can (and should) reassess this risk with every prescription they write, especially as the patient ages and/or dose is increased. This study’s finding on the lack of effect of opioid dose on risk of overdose therefore needs to be reexamined in future studies.

The relatively low specificity demonstrated by the model emphasizes, yet again, that there is much we do not understand in terms of immediate precipitants of opioid overdose among patients prescribed LTOT. Seminal case-cohort work by Bohnert et al. in the Veterans Health Administration5 demonstrated that there was no active prescription at the time of overdose death in nearly 40% of decedents, suggesting that the inferential logic that dose prescribed equals dose taken, which then equals the dose that immediately caused the overdose is frequently faulty. However, the predictors identified in the present study might offer some clues of how to increase the specificity of a refined model. Hepatitis C (an examined model component that did not make the final model) is a strong indicator of history of intravenous drug use (IVDU), and the most significant risk factor for current IVDU is a history of IVDU. Altering opioid formulations and insufflating or injecting them is perhaps the most biologically plausible pathway for recipients of pill prescriptions to experience overdose. Individuals with history of substance abuse/dependence are also probably more likely to alter formulations. Could it be that several of the factors identified were markers for future risk of altering formulations?

Glanz et al. took important steps forward towards the goal of a clinically actionable predictive model for opioid overdose among patients prescribed LTOT for chronic pain. Use of the predictive model itself and the accompanying rigor with which it was performed are important methodological advances. Winnowing to a short list of clinically available model components is also indispensable. Broadening the scope of potential outcomes to target, or clinical applications of the model are important future refinements.