Skip to main content
SpringerLink
Log in

Toward a national consumer survey: Evaluation of the CABHS and MHSIP instruments

  • Regular Articles
  • Published:
The Journal of Behavioral Health Services & Research Aims and scope Submit manuscript

Abstract

This article describes a study evaluating the Consumer Assessment of Behavioral Health Survey (CABHS) and the Mental Health Statistics Improvement Program (MHSIP) surveys. The purpose of the study was to provide data that could be used to develop recommendations for an improved instrument. Subjects were 3,443 adults in six behavioral health plans. The surveys did not differ significantly in response rate or consumer burden. Both surveys reliably assessed access to treatment and aspects of appropriateness and quality. The CABHS survey reliably assessed features of the insurance plan; the MHSIP survey reliably assessed treatment outcome. Analyses of comparable items suggested which survey items had greater validity. Results are discussed in terms of consistency with earlier research using these and other consumer surveys. Implications and recommendations for survey development, quality improvement, and national policy initiatives to evaluate health plan performance are presented.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Dickey B, Hermann RC, Eisen SV. Assessing the quality of psychiatric care: research methods and application in clinical practice.Harvard Review of Psychiatry. 1998;6(2):88–96.

    Google Scholar 

  2. Donabedian A.The Definition of Quality and Approaches to Its Assessment. Ann Arbor, MI: Health Administration Press; 1980.

    Google Scholar 

  3. Brook RH, McGlynn EA, Cleary PD. Measuring quality of care.New England Journal of Medicine. 1996;335(13):966–970.

    Google Scholar 

  4. Lee RI, Jones LW.The Fundamentals of Good Medical Care. Chicago: The University of Chicago Press; 1933.

    Google Scholar 

  5. American Psychiatric Association. Practice guideline for major depressive disorder in adults.American Journal of Psychiatry. 1993;150(suppl):1–26.

    Google Scholar 

  6. American Psychiatric Association. Practice guideline for eating disorders.American Journal of Psychiatry. 1993;150(suppl):212–228.

    Google Scholar 

  7. American Psychiatric Association. Practice guideline for the treatment of patients with bipolar disorder.American Journal of Psychiatry. 1994;151(suppl):1–36.

    Google Scholar 

  8. American Psychiatric Association. Practice guideline for the treatment of patients with substance use disorders: alcohol, cocaine, opioids.American Journal of Psychiatry. 1995;152(suppl):5–59.

    Google Scholar 

  9. American Psychiatric Association. Practice guideline for the treatment of patients with schizophrenia.American Journal of Psychiatry. 1997;154:1–63.

    Google Scholar 

  10. Lehman AF, Steinwachs DM. Translating research into practice: the Schizophrenia Patient Outcomes Research Team treatment recommendations.Schizophrenia Bulletin. 1998;24:1–10.

    Google Scholar 

  11. Brook RH, Chassin MR, Fink A, et al. A method for the detailed assessment of the appropriateness of medical technologies.International Journal of Technology Assessment in Health Care. 1986;2:53–63.

    Google Scholar 

  12. Davies AR, Ware JE Jr. Involving consumers in quality of care assessment.Health Affairs. 1988;7(1):33–48.

    Google Scholar 

  13. Eisen SV, Grob MC, Dill DL. Outcome measurement: tapping the patient's perspective. In: Mirin SM, Gossett J, Grob MC, eds.Psychiatric Treatment: Advances in Outcome Research. Washington, DC: American Psychiatric Press; 1991:213–235.

    Google Scholar 

  14. Kalman TP. An overview of patient satisfaction with psychiatric treatment.Hospital and Community Psychiatry. 1983;34:48–54.

    Google Scholar 

  15. Rosenheck R, Wilson NJ, Meterko M. Influence of patient and hospital factors on consumer satisfaction with inpatient mental health treatment.Psychiatric Services. 1997;48:1553–1561.

    Google Scholar 

  16. Ruggeri M. Patients' and relatives' satisfaction with psychiatric services: the state of the art of its measurement.Social Psychiatry and Psychiatric Epidemiology. 1994;29:212–227.

    Google Scholar 

  17. Joint Commission on Accreditation of Healthcare Organizations.2000 Hospital Accreditation Standards. Oakbrook, IL: Joint Commission; 2000.

    Google Scholar 

  18. Lebow J. Consumer satisfaction with mental health treatment.Psychological Bulletin. 1982;91:244–259.

    Google Scholar 

  19. Coker M, Sharp JH, Powell H, et al. Implementation of total quality management after reconfiguration of services on a general hospital unit.Psychiatric Services. 1997;48:231–236.

    Google Scholar 

  20. Johnson LD, Shaha S. Improving quality in psychotherapy.Psychotherapy. 1996;33:225–236.

    Google Scholar 

  21. Kaplan SH, Ware JE. The patient's role in health care and quality assessment. In: Goldfield N, Nash DB, eds.Providing Quality Care: The Challenge to Clinicians.: The American College of Physicians; 1989:25–67.

  22. Larsen DL, Attkisson CC, Hargraves WA, et al. Assessment of client/patient satisfaction: development of a general scale.Evaluation and Program Planning. 1979;2(3):197–207.

    Google Scholar 

  23. Holcomb WR, Parker JC, Leong GB, et al. Customer satisfaction and self-reported treatment outcomes among psychiatric inpatients.Psychiatric Services. 1998;49(7):929–934.

    Google Scholar 

  24. Fremont AM, Cleary PD, Hargraves JL, et al. Impact of patient-centered care processes and long-term myocardial infarction outcomes.Journal of General Internal Medicine. 1999;14(suppl 2):30.

    Google Scholar 

  25. Center for Mental Health Services.The MHSIP Consumer-Oriented Mental Health Report Card. The Final Report of the Mental Health Statistics Improvement Program (MHSIP) Task Force on a Consumer-Oriented Mental Health Report Card. Washington, DC: Center for Mental Health Services; 1996.

    Google Scholar 

  26. Dickey B. The development of report cards for mental health care. In: Sederer LI, Dickey B, eds.:Outcomes Assessment in Clinical Practice. Baltimore, MD: Williams & Wilkins; 1996:156–160.

    Google Scholar 

  27. Skolnick AA. A FACCT-filled agenda for public information.Journal of the American Medical Association. 1997;278:1558.

    Google Scholar 

  28. Ross EC. Managed behavioral health care premises, accountable systems of care and AMBHA's PERMS.Evaluation Review. 1997;21:318–321.

    Google Scholar 

  29. Forsyth BH, Lessler JT. Cognitive laboratory methods: a taxonomy. In: Biemer PP, Groves RM, Lyberg LE, Mathiowetz NA, Sudman S, eds.Measurement Errors in Surveys. New York: John Wiley & Sons; 1991.

    Google Scholar 

  30. Harris-Kojetin LD, Fowler FJ, Brown JA, et al. The use of cognitive testing to develop and evaluate CAHPS™ 1.0 core survey items.Medical Care. 1997;37(suppl 3):MS10-MS21.

    Google Scholar 

  31. Fowler FJ.Improving Survey Questions: Design and Evaluation. Thousand Oaks, CA: Sage; 1995.

    Google Scholar 

  32. Cleary PD. The increasing importance of patient surveys.British Medical Journal. 1999;319:720–721.

    Google Scholar 

  33. Ware JE, Snyder MK, Wright WR, et al. Defining and measuring patient satisfaction with medical care.Evaluation and Program Planning. 1983;6:247–263.

    Google Scholar 

  34. Cleary PD, Edgman-Levitan S, Roberts M, et al. Patients evaluate their hospital care: a national survey.Health Affairs. 1991;10(4):254–267.

    Google Scholar 

  35. National Committee for Quality Assurance.HEDIS 3.0 Volume 2, Technical Specifications. Washington, DC: NCQA; 1997.

    Google Scholar 

  36. Hays RD, Shaul JA, Williams VSL, et al. Psychometric properties of the CAHPS® 1.0 survey measures.Medical Care. 1999;37(3, suppl):MS22-MS31.

    Google Scholar 

  37. Eisen SV, Shaul JA, Clarridge BR, et al. Development of a consumer survey for behavioral health services.Psychiatric Services. 1999;50:793–798.

    Google Scholar 

  38. Fowler FJ, Gallagher PM, Nederend S. Comparing telephone and mail responses to the CAHPS® survey instrument.Medical Care. 1999;37(3, suppl):MS41-MS49.

    Google Scholar 

  39. Schnaier JA, Sweeny SF, Williams VSL, et al. Special issues addressed in the CAHPS® survey of Medicare managed care beneficiaries.Medical Care. 1999;37(3, suppl):MS69-MS78.

    Google Scholar 

  40. Eisen SE, Shaul JA, Clarridge B, et al.Consumer Assessment of Behavioral Health Services (CABHS)-Mental Health Statistics Improvement Program (MHSIP) Consumer Survey Evaluation Project. Technical report submitted to The Human Services Research Institute. Cambridge, MA: Human Services Research Institute; 2000.

    Google Scholar 

  41. Teague GB, Ganju V, Hornik JA, et al. The MHSIP mental health report card: a consumer oriented approach to monitoring the quality of mental health plans.Evaluation Review. 1997;21:330–341.

    Google Scholar 

  42. Ganju V, Wackwitz J, Trabin T.The Mental Health Statistics Improvement Program (MHSIP) Consumer Survey. Unpublished report submitted to the Committee on Performance Measurement, NCQA. 1998.

  43. Weidmer B, Brown J, Garcia L. Translating the CAHPS™ 1.0 survey instruments into Spanish.Medical Care. 1999;37(3, suppl):MS89-MS96.

    Google Scholar 

  44. Perrault M, Leichner P. Patient satisfaction with outpatient psychiatric services.Evaluation and Program Planning. 1993;16:109–118.

    Google Scholar 

  45. Stringfellow VL, Fowler FJ, Gallagher PM. An experiment with skip instructions: decreasing item non-response in a self-administered survey of Medicaid benificaries. Proceedings, Survey Methods Section, American Statistical Association. In press, 2001.

  46. Shaul JA, Eisen SV, Stringfellow VL, et al. Use of consumer ratings for quality improvement in behavioral health insurance plans.Joint Commission Journal on Quality Improvement. 2001;27(4):216–229.

    Google Scholar 

  47. McCorry F, Garnick DW, Bartlett J, et al. Developing performance measures for alcohol and other drug services in managed care plans.Joint Commission Journal on Quality Improvement. 2000;26:633–643.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mental Health Services Research, McLean Hospital, 115 Mill Street, 02478, Belmont, MA

    Susan V. Eisen PhD (Director)

  2. Department of Psychiatry, Harvard Medical School, USA

    Susan V. Eisen PhD (Director)

  3. Department of Health Care Policy, Harvard Medical School, USA

    James A. Shaul MHA (Project Director) & Paul D. Cleary PhD (Professor of Medical Sociology)

  4. The Evaluation Center, Human Services Research Institute, USA

    H. Stephen Leff PhD (Senior Vice President, Principal Investigator, and Director)

  5. Center for Survey Research, University of Massachusetts, Boston

    Vickie Stringfellow BA (Assistant Study Director) & Brian Clarridge PhD (Senior Research Fellow)

Authors
  1. Susan V. Eisen PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. James A. Shaul MHA
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. H. Stephen Leff PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Vickie Stringfellow BA
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Brian Clarridge PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Paul D. Cleary PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Susan V. Eisen PhD.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Eisen, S.V., Shaul, J.A., Leff, H.S. et al. Toward a national consumer survey: Evaluation of the CABHS and MHSIP instruments. The Journal of Behavioral Health Services & Research 28, 347–369 (2001). https://doi.org/10.1007/BF02287249

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02287249

Keywords

Search

Navigation