Skip to main content

Advertisement

SpringerLink
Log in

Evaluating PROMIS® instruments and methods for patient-centered outcomes research: Patient and provider voices in a substance use treatment setting

  • Special Section: PROs in Non-Standard Settings (by invitation only)
  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Purpose

Our work as a primary research site of the Patient-Reported Outcomes Measurement Information System (PROMIS®), combined with support from the Patient-Centered Outcomes Research Institute, allowed us to evaluate the real-world applicability and acceptability of PROMIS measures in an addiction medicine setting.

Methods

As part of a 3-month prospective observational study, 225 outpatients at a substance abuse treatment clinic completed PROMIS item banks for alcohol use (as well as 15 additional item banks from 8 other PROMIS domains, including emotional distress, sleep, and pain), with assessments at intake, 1-month follow-up, and 3-month follow-up. A subsample of therapists and their patients completed health domain importance ratings and qualitative interviews to elicit feedback regarding the content and format of the patients’ assessment results.

Results

The importance ratings revealed that depression, anxiety, and lack of emotional support were rated highest of the non-alcohol-related domains among both patients and clinicians. General alcohol use was considered most important by both patients and clinicians. Based on their suggestions, changes were made to item response feedback to facilitate comprehension and communication.

Conclusions

Both therapists and patients agreed that their review of the graphical display of scores, as well as individual item responses, helped them to identify areas of greatest concern and was useful for treatment planning. The results of our pilot work demonstrated the value and practicality of incorporating a comprehensive health assessment within a substance abuse treatment setting.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. American Psychiatric Association. (2013). Assessment Measures. In Diagnostic and statistical manual of mental disorders (5th ed., pp. 733–748). Arlington, VA: American Psychiatric Association.

  2. U.S. Government Accountability Office (GAO). (2015). Comparative effectiveness research: HHS needs to strengthen dissemination and data-capacity-building efforts: GAO-15-280. www.gao.gov.

  3. Snyder, C. F., Jensen, R. E., Geller, G., Carducci, M. A., & Wu, A. W. (2010). Relevant content for a patient-reported outcomes questionnaire for use in oncology clinical practic: Putting doctors and patients on the same page. Quality of Life Research, 19, 1045–1055.

    Article  PubMed  Google Scholar 

  4. Wu, A. W., Kharrazi, H., Boulwar, L. E., & Snyder, C. F. (2013). Measure once, cut twice–adding patient-reported outcome measures to the electronic health record for comparative effectiveness research. Journal of Clinical Epidemiology, 66(8 Suppl), S12–S20.

    Article  PubMed Central  PubMed  Google Scholar 

  5. Marshall, S., Haywood, K., & Fitzpatrick, R. (2006). Impact of patient-reported outcome measures on routine practice: A structured review. Journal of Evaluation in Clinical Practice, 12(5), 559–568.

    Article  PubMed  Google Scholar 

  6. Rotgers, F. (2002). Clinically useful, research validated assessment of persons with alcohol problems. Behaviour Research and Therapy, 40(12), 1425–1441.

    Article  PubMed  Google Scholar 

  7. Cella, D., Gershon, R., Lai, J. S., & Choi, S. (2007). The future of outcomes measurement: Item banking, tailored short forms, and computerized adaptive assessment. Quality of Life Research, 16(Suppl 1), 133–144.

    Article  PubMed  Google Scholar 

  8. Pilkonis, P. A., Choi, S. W., Reise, S. P., Stover, A. M., Riley, W. T., & Cella, D. (2011). Item banks for measuring emotional distress from the patient-reported outcomes measurement information system: Depression, anxiety, anger. Assessment., 18(3), 263–283.

    Article  PubMed Central  PubMed  Google Scholar 

  9. Broderick, J. E., DeWitt, E. M., Rothrock, N., Crane, P. K., & Forrest, C. B. (2013). Advances in patient reported outcomes: The NIH PROMIS measures. eGEMs (Generating Evidence & Methods to improve patient outcomes), 1(1), Article 12, 1–7.

  10. Buysse, D., Yu, L., Moul, D. E., Germain, A., Stover, A., Dodds, N. E., et al. (2010). Development and validation of patient-reported outcome measures for sleep disturbance and sleep-related impairments. Sleep, 33(6), 781–792.

  11. Revicki, D., Chen, W., Harnam, N., Cook, K., Amtmann, D., Callahan, L. F., et al. (2009). Development and psychometric analysis of the PROMIS pain behavior item bank. Pain, 146(1–2), 158–169.

    Article  PubMed Central  PubMed  Google Scholar 

  12. Fries, J. F., Cella, D., Rose, M., Krishnan, E., & Bruce, B. (2009). Progress in assessing physical function in arthritis: PROMIS short forms and computerized adaptive testing. Journal of Rheumatology, 36(9), 2061–2066.

    Article  PubMed  Google Scholar 

  13. Pilkonis, P. A., Yu, L., Dodds, N. E., Johnston, K. L., Maihoefer, C., & Lawrence, S. M. (2014). Validation of the depression item bank from the Patient-Reported Outcomes Measurement Information System (PROMIS®) in a three-month observational study. Journal of Psychiatric Research, 56, 112–119.

    Article  PubMed Central  PubMed  Google Scholar 

  14. Reeve, B. B., Hays, R. D., Bjorner, J. B., Cook, K. F., Crane, P. K., Teresi, J. A., et al. (2007). Psychometric evaluation and calibration of health-related quality of life item banks: Plans for the Patient-Reported Outcome Measurement Information System (PROMIS). Medical Care, 45, S22–S31.

    Article  PubMed  Google Scholar 

  15. DeWitt, E. M., Stucky, B. D., Thissen, D., Irwin, D. E., Langer, M., Varni, J. W., et al. (2011). Construction of the eight item PROMIS pediatric physical function scales: Built using item response theory. Journal of Clinical Epidemiology, 64(7), 794–804.

    Article  PubMed Central  PubMed  Google Scholar 

  16. Varni, J. W., Stucky, B. D., Thissen, D., DeWitt, E. M., Irwin, D., Lai, J. S., et al. (2010). PROMIS Pediatric Pain Interference Scale: An item response theory analysis of the Pediatric Pain Item Bank. Journal of Pain, 11(11), 1109–1119.

    Article  PubMed Central  PubMed  Google Scholar 

  17. Amtmann, D., Cook, K., Jensen, M. P., Chen, W.-H., Choi, S. W., Revicki, D., et al. (2010). Development of a PROMIS item bank to measure pain interference. Pain, 150(1), 173–182.

    Article  PubMed Central  PubMed  Google Scholar 

  18. Flynn, K. E., Jeffrey, D. D., Keefe, F. J., Porter, L. S., Shelby, R. A., Fawzy, M. R., et al. (2011). Sexual functioning along the cancer continuum: focus group results from the Patient-Reported Outcomes Measurement Information System (PROMIS). Psycho-Oncology, 20(4), 378–386.

    Article  PubMed Central  PubMed  Google Scholar 

  19. Forrest, C. B., Bevans, K. B., Tucker, C., Riley, A. W., Ravens-Sieberer, U., Gardner, W., et al. (2012). Commentary: the Patient-Reported Outcome Measurement Information System (PROMIS) for Children and Youth: Application to pediatric psychology. Journal of Pediatric Psychology, 37(6), 614–621.

  20. Hahn, E. A., DeWalt, D. A., Bode, R. K., Garcia, S. F., DeVellis, R. F., Correia, H., et al. (2014). New English and Spanish social health measures will facilitate evaluating health determinants. Health Psychology, 33(5), 490–499.

  21. Hays, R. D., Bjorner, J. B., Revicki, R. A., Spritzer, K. L., & Cella, D. (2009). Development of physical and mental health summary scores from the Patient Reported Outcomes Measurement Information System (PROMIS) global items. Quality of Life Research, 18(7), 873–880.

    Article  PubMed Central  PubMed  Google Scholar 

  22. Patel, K. (2010). Health reform’s tortuous route to the Patient-Centered Outcomes Research Institute. Health Affairs, 29(10), 1777–1782.

    Article  PubMed  Google Scholar 

  23. Patient-Centered Outcomes Research Institute. (2011–2015). from www.pcori.org.

  24. Pilkonis, P. A., Yu, L., Colditz, J., Dodds, N. E., Johnston, K. L., Maihoefer, C., et al. (2013). Item banks for alcohol use from the Patient-Reported Outcomes Measurement Information System (PROMIS®): use, consequences, and expectancies. Drug and Alcohol Dependence, 130, 167–177.

    Article  PubMed Central  PubMed  Google Scholar 

  25. DeWalt, D. A., Rothrock, N., Yount, S., Stone, A. A., & on behalf of the PROMIS Cooperative Group. (2007). Evaluation of item candidates: The PROMIS qualitative item review. Medical Care, 45, S12–S21.

  26. Saunders, J. B., Aasland, O. G., Babor, T. F., Delafuente, J. R., & Grant, M. (1993). Development of the Alcohol-Use Disorders Identification Test (AUDIT): WHO collaborative project on early detection of persons with harmful alcohol consumption. Addiction, 88, 791–804.

    Article  CAS  PubMed  Google Scholar 

  27. Stinson, F. S., Grant, B. F., Dawson, D. A., Ruan, W. J., Huang, B., & Saha, T. (2005). Comorbidity between DSM-IV alcohol and specific drug use disorders in the United States: results from the National Epidemiologic Survey on Alcohol and Related Conditions. Drug and Alcohol Dependence, 80(1), 105–116.

    Article  PubMed  Google Scholar 

  28. Jensen, R. E., Rothrock, N. E., DeWitt, E. M., Spiegel, B., Tucker, C. A., Crane, H., et al. (2015). The role of technical advances in the adoption and integration of patient-reported outcomes in clinical care. Medical Care, 53(2), 153–159.

    Article  PubMed  Google Scholar 

  29. Lohr, K. N., & Zebrack, B. J. (2009). Using patient-reported outcomes in clinical practice: Challenges and opportunities. Quality of Life Research, 18, 99–107.

    Article  PubMed  Google Scholar 

  30. Rose, M., & Bezjak, A. (2009). Logistics of collecting patient-reported outcomes (PROs) in clinical practice: An overview and practical examples. Quality of Life Research, 18, 125–136.

    Article  PubMed  Google Scholar 

  31. Graff, F. S., Morgan, T. J., Epstein, E. E., McCrady, B. S., Cook, S. M., Jensen, N. K., & Kelly, S. (2009). Engagement and retention in outpatient alcoholism treatment for women. American Journal of Addictions, 18(4), 277–288.

    Article  Google Scholar 

  32. Loveland, D., & Driscoll, H. (2014). Examining attrition rates at one specialty addiction treatment provider in the United States: a case study using a retrospective chart review. Substance Abuse Treatment, Prevention, and Policy, 9(1), 41.

    Article  PubMed Central  PubMed  Google Scholar 

  33. Substance Abuse and Mental Health Services Administration Treatment Episode Data Set. (2010). Discharges from substance abuse treatment services. BHSIS Series S-67, HHS Publication No. (SMA) 14-4817. Rockville, MD: Substance Abuse and Mental Health Services Administration, 2013.

  34. Stark, M. J. (1992). Dropping out of substance abuse treatment: A clinically oriented review. Clinical Psychology Review, 12(1), 93–116.

    Article  Google Scholar 

  35. Babor, T. F., Higgins-Biddle, J. C., Saunders, J. B., & Monteiro, M. G. (2001). The alcohol use disorders identification test: Guidelines for use in primary care (No. WHO/MSD/MSB/01.6a). World Health Organization.

  36. Tracy, E. M. (2010). Social support: A mixed blessing for women in substance abuse treatment. Journal of Social Work Practice in the Addictions, 10(3), 257–282.

    Article  PubMed Central  PubMed  Google Scholar 

  37. Chou, K. L., Liang, K., & Sareen, J. (2011). The association between social isolation and DSM-IV mood, anxiety, and substance use disorders: Wave 2 of the National Epidemiologic Survey on Alcohol and Related Conditions. Journal of Clinical Psychiatry, 72(11), 1468–1476.

    Article  PubMed  Google Scholar 

  38. Warren, J. I., Stein, J. A., & Grella, C. E. (2007). Role of social support and self-efficacy in treatment outcomes among clients with co-occurring disorders. Drug and Alcohol Dependence, 89(2–3), 267–274.

    Article  PubMed Central  PubMed  Google Scholar 

  39. Richter, S. S., Brown, S. A., & Mott, M. A. (1991). The impact of social support and self-esteem on adolescent substance abuse treatment outcome. Journal of Substance Abuse, 3(4), 371–385.

    Article  CAS  PubMed  Google Scholar 

  40. Ciraulo, D. A., Piechniczek-Buczek, J., & Iscan, E. N. (2003). Outcome predictors in substance use disorders. Psychiatric Clinics of North America, 26(2), 381–409.

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

This project was funded by the Patient-Centered Outcomes Research Institute (1IP2PI000189; PI: Paul A. Pilkonis, PhD). We acknowledge the work of our colleagues within the NIDA Clinical Trials Network that collaborated in this work. Dennis Daley, PhD, is the principal investigator of the Tri-State Appalachian node, and clinical data from this node were collected in Pittsburgh, PA, where we are grateful for the help of Trey Ghee, LSW, Dorothy Sandstrom, MS, and Janis McDonald.

Author information

Authors and Affiliations

  1. Department of Psychiatry, Western Psychiatric Institute and Clinic, University of Pittsburgh School of Medicine, 3811 O’Hara Street, Pittsburgh, PA, 15213, USA

    Kelly L. Johnston, Suzanne M. Lawrence, Nathan E. Dodds, Lan Yu, Dennis C. Daley & Paul A. Pilkonis

  2. Department of Medicine, University of Pittsburgh School of Medicine, Pittsburgh, PA, 15213, USA

    Lan Yu

Authors
  1. Kelly L. Johnston
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Suzanne M. Lawrence
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nathan E. Dodds
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lan Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dennis C. Daley
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Paul A. Pilkonis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kelly L. Johnston.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical standard

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. This study was reviewed and approved by the University of Pittsburgh Institutional Review Board.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Johnston, K.L., Lawrence, S.M., Dodds, N.E. et al. Evaluating PROMIS® instruments and methods for patient-centered outcomes research: Patient and provider voices in a substance use treatment setting. Qual Life Res 25, 615–624 (2016). https://doi.org/10.1007/s11136-015-1131-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-015-1131-3

Keywords

Search

Navigation