Report of the STS Quality Measurement Task Force
Quality Measurement in Adult Cardiac Surgery: Part 2—Statistical Considerations in Composite Measure Scoring and Provider Rating

https://doi.org/10.1016/j.athoracsur.2007.01.055Get rights and content

Section snippets

Executive Summary

There is increasing interest among payers, patients, regulators, and providers to measure and compare cardiac surgery quality. The Society of Thoracic Surgeons (STS) Quality Measurement Task Force (QMTF) was established to develop comprehensive, summary performance measures encompassing multiple domains of quality. This report describes statistical considerations relevant to combining multiple measures into an overall composite score and then using such scores to rate providers.

The QMTF

Performance Measure Definitions

The QMTF selected 11 individual CABG performance measures (five process and six outcome), all of which were endorsed by the National Quality Forum (NQF) and are available in the STS NCD (Table 1). Specific NQF inclusion/exclusion criteria were applied to these measures to the extent possible. The internal mammary artery (IMA) measure excluded patients undergoing repeat CABG surgery, the permanent stroke measure excluded patients with a previous cerebrovascular accident, and the three discharge

Distribution of Individual Performance Measures

The distribution of provider-specific performance on NQF measures was investigated by using random-effects modeling to identify overall performance levels and to quantify between-provider variation. In general, measures that vary widely yield high statistical power for discriminating among providers. Care is needed to ensure that the less variable measures contribute additional statistical information when they are combined with more variable measures in a composite.

The estimated distribution

Individual Performance Measure Correlation

The estimated correlation between pairs of NQF measures (true process usage rates and true risk-standardized outcome rates) is summarized in Table 2, Table 3. For process measures, the estimated Pearson correlation between pairs ranged from 0.10 (IMA versus discharge antiplatelets) to 0.50 (preoperative β-blockers versus discharge β-blockers). For risk-adjusted outcome measures, the estimated Pearson correlation between pairs ranged from 0.15 (prolonged ventilation versus permanent stroke) to

Composite Scoring Methodologies

The QMTF selected four quality-of-care domains for CABG, represented by 11 NQF-endorsed process and outcomes measures: (1) Perioperative Medical Care (preoperative and discharge β-blockers, discharge antiplatelets, and discharge antilipids), (2) Operative Care (IMA usage), (3) Risk-Adjusted Operative Mortality, and (4) Postoperative Risk-Adjusted Major Morbidity (stroke, prolonged ventilation, renal insufficiency, reexploration for any cause, and deep sternal wound infection). The QMTF

Performance Tier Determination

Having selected a methodology for composite scoring, the QMTF considered various options for assigning providers to performance tiers. For example, a number of different approaches have been used in the United Kingdom, including confidence intervals to determine high and low outliers, ranks based on percentiles, and absolute thresholds, which, unlike the first two options, is not a reflection of performance relative to other providers [19, 20].

Using actual STS data, the QMTF pilot tested the

Comment

Composite indicators are useful for summarizing and comparing the quality of care delivered by healthcare providers. In many areas of medicine, the number of acceptable quality indicators is large, and there is a need for summary measures that combine performance on multiple end points. Although quality improvement requires attention to each individual aspect of quality, there are many settings in which the users of quality measures are most interested in the bottom line. The comprehensive

Conclusion

This two-part report by the STS QMTF describes the development of a multidimensional CABG composite quality score that is scientifically rigorous, uses NQF-endorsed measures from the STS NCD, and is consistent with relevant national guidelines for performance measurement. The QMTF regards this as the first step in a process that will constantly evolve as new quality measures, statistical methodologies, and health care policy objectives are developed.

First page preview

First page preview
Click to open first page preview

References (25)

  • D.M. Shahian et al.

    Quality measurement in adult cardiac surgeryPart 1—Conceptual framework and measure selection

    Ann Thorac Surg

    (2007)
  • D.M. Shahian et al.

    Cardiac surgery report cards: comprehensive review and statistical critique

    Ann Thorac Surg

    (2001)
  • Performance measurement: accelerating improvement

    (2006)
  • S.-L.T. Normand et al.

    Statistical methods for profiling providers of medical care: issues and applications

    J Am Stat Assoc

    (1997)
  • C.L. Christiansen et al.

    Improving the statistical approach to health care provider profiling

    Ann Intern Med

    (1997)
  • H. Goldstein et al.

    League tables and their limitations: statistical issues in comparisons of institutional performance

    J R Stat Soc (series A)

    (1996)
  • A.H. Leyland et al.

    Multilevel modelling of health statistics

    (2001)
  • A.L. Shroyer et al.

    The Society of Thoracic Surgeons: 30-day operative mortality and morbidity risk models

    Ann Thorac Surg

    (2003)
  • D.J. Spiegelhalter et al.

    Bayesian approaches to clinical trials and health-care evaluation

    (2004)
  • B.P. Carlin et al.

    Bayes and empirical Bayes methods for data analysis

    (2000)
  • A. Gelman et al.

    Bayesian data analysis

    (2004)
  • T. Lied et al.

    Combining HEDIS indicators: a new approach to measuring plan performance

    Health Care Financ Rev

    (2002)
  • Cited by (201)

    • Measuring and reporting cardiac surgery quality: A continuing evolution

      2023, Journal of Thoracic and Cardiovascular Surgery
    View all citing articles on Scopus

    Dr Shahian is the Quality Measurement Task Force Chair and Writing Group Leader.

    View full text