Minimal important difference estimates for patient-reported outcomes: A systematic survey

doi:10.1016/j.jclinepi.2020.11.024

Journal of Clinical Epidemiology

Volume 133, May 2021, Pages 61-71

https://doi.org/10.1016/j.jclinepi.2020.11.024 Get rights and content

Abstract

Objectives

The objective of the study was to develop an inventory summarizing all anchor-based minimal important difference (MID) estimates for patient-reported outcome measures (PROMs) available in the medical literature.

Study Design and Setting

We searched MEDLINE, EMBASE, CINAHL, PsycINFO, and the Patient-Reported Outcome and Quality of Life Instruments Database internal library (January 1989–October 2018). We included primary studies empirically calculating an anchor-based MID estimate for any PROM in adults and adolescents. Pairs of reviewers independently screened and selected studies, extracted data, and evaluated the credibility of the MIDs.

Results

We identified 585 eligible studies, the majority conducted in Europe (n = 211) and North America (n = 179), reporting 5,324 MID estimates for 526 distinct PROMs. Investigators conducted their studies in the context of patients receiving surgical (n = 105, 18%), pharmacological (n = 85, 15%), rehabilitation (n = 65, 11%), or a combination of interventions (n = 194, 33%). Of all MID estimates, 59% (n = 3,131) used a global rating of change anchor. Major credibility limitations included weak correlation (n = 1,246, 23%) or no information regarding the correlation (n = 3,498, 66%) between the PROM and anchor and imprecision in the MID estimate (n = 2,513, 47%).

Conclusion

A large number of MIDs for assisting in the interpretation of PROMs exist. The MID inventory will facilitate the use of MID estimates to inform the interpretation of the magnitude of treatment effects in clinical research and guideline development.

Introduction

Outcomes that matter to patients have become a key focus in studies evaluating the effects of health care interventions. Patient-reported outcome measures (PROMs) provide information regarding a patient's health condition directly from the patient without interpretation by a clinician or anyone else [1]. Investigators have developed PROM measuring constructs such as function, pain, dyspnea, and fatigue. Many instruments measure a number of domains that bear on broader constructs, including functional status, emotional function, and health-related quality of life.

The evaluation and results of PROMs as outcomes in clinical trials, systematic reviews, clinical practice guidelines, although undeniably important, suffer from difficulties with intuitive understanding regarding the magnitude of change that patients have experienced [2]. The minimal important difference (MID), initially defined as “the smallest difference that patients perceive as beneficial and that would mandate, in the absence of troublesome side effects and excessive cost, a change in the patient's management” [3] is the most widely used approach to facilitating the interpretation of PROMs. An update of this definition includes the patient's perception not only of the benefits but also of harms and the possibility of an “informed proxy” as a valid informant when the patient is incapable of providing the information [4,5].

Investigators use two primary strategies to determine an MID: distribution and anchor-based. Distribution-based approaches rely on the statistical characteristics of the sample that fail to incorporate the patient perspective and yield MIDs that will vary widely depending on sample characteristics [6,7]. Anchor-based approaches relate a change in a PROM to an external criterion (i.e., the anchor) that is itself interpretable and provide meaning to the change experienced in the PROM [8]. Empirical evidence suggests that estimates from distribution-based approaches differ from one another and from anchor-based approaches and thus are of limited use [9,10].

Although widely accepted, the use of anchor-based MID estimates also presents challenges. Clinical trialists, systematic review authors, and guideline developers wishing to use MIDs to enhance PROM interpretability must conduct systematic searches to identify primary studies ascertaining MIDs. They will often find multiple MIDs and will often lack training and skills to choose the most credible and applicable to their context [[11], [12], [13]]. Therefore, to facilitate the interpretation of PROMs and to increase our understanding of and access to MIDs, we summarized all anchor-based MID estimates for PROMs available in the medical literature and evaluated their credibility.

Section snippets

Methods

Readers can find a detailed report of the methods of our review in a previously published protocol [14]. This report adheres to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses criteria that are relevant for this systematic survey [15].

Search results

Of 14,840 citations identified from our search, reviewers screened 10,469 titles and abstracts, of which 2,161 studies proved eligible for full-text evaluation. Of these, reviewers ultimately deemed eligible 585 studies reporting on 5,324 MID estimates for 526 distinct PROMs (Fig. 1).

Study-level characteristics

Most of the studies were conducted in Europe and North America. Many investigators conducted their studies in the context of patients receiving surgical, pharmacologic, rehabilitation, or a combination of

Main findings

This first systematic summary of all available anchor-based MID estimates for PROMs in the medical literature identified 585 primary studies reporting on more than 5,300 anchor-based MID estimates applicable to 526 distinct PROMs. Studies representing a wide variety of clinical disciplines, most frequently addressed disease/condition-specific PROMs and used longitudinal designs with self-reported global ratings of change. The credibility of the MID estimates varied substantially, and reporting

Acknowledgments

The authors would like to thank Tamsin Adams–Webber at the Hospital for Sick Children and Paul Alexander for their assistance with developing the initial literature search. We would also like to thank Shahrzad Motaghi Pisheh, Brittany Dennis, Marc Jacobs, Yuqing Zhang, Kevin Quach, Nigar Sekercioglu, Sean Kennedy, William Zhang, Samantha Craigie, Iván Flórez, Yutong Fei, Brian Younho Hong, Aran Tajika, Nozomi Takeshima, Naotsugu Iwakami, Yu Hayasaka, Angela Kaminski, Barbara Nussbaumer, and

References (28)

S.D. Glassman et al.
Thresholds for health-related quality of life measures: reality testing
Spine J
(2010)
R. Jaeschke et al.
Measurement of health status. Ascertaining the minimal clinically important difference
Control Clin trials
(1989)
R.D. Crosby et al.
Defining clinically meaningful change in health-related quality of life
J Clin Epidemiol
(2003)
G.H. Guyatt et al.
Methods to explain the clinical significance of health status measures
Mayo Clin Proc
(2002)
D. Turner et al.
The minimal detectable change cannot reliably replace the minimal important difference
J Clin Epidemiol
(2010)
C.B. Terwee et al.
Mind the MIC: large variation among populations and methods
J Clin Epidemiol
(2010)
J.G. Neely et al.
Practical guides to understanding sample size and minimal clinically important difference (MCID)
Otolaryngol Head Neck Surg
(2007)
F. Angst et al.
The minimal clinically important difference raised the significance of outcome effects above the statistical level, with methodological implications for future studies
J Clin Epidemiol
(2017)
D. Revicki et al.
Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes
J Clin Epidemiol
(2008)
FDA-NIH Biomarker Working Group

H.J. Schunemann et al.

Measurement properties and interpretability of the Chronic respiratory disease questionnaire (CRQ)

COPD

(2005)

H.J. Schunemann et al.

Commentary–goodbye M(C)ID! Hello MID, where do you come from?

Health Serv Res

(2005)

A. Wright et al.

Clinimetrics corner: a closer look at the minimal clinically important difference (MCID)

J Man Manip Ther

(2012)

M.A. Puhan et al.

Interpretation of treatment changes in 6-minute walk distance in patients with COPD

Eur Respir J

(2008)

Cited by (38)

Minimal important change and difference in health outcome: An overview of approaches, concepts, and methods
2024, Osteoarthritis and Cartilage
To provide an overview of approaches, concepts, and methods used to define and assess minimal important change and difference in health outcome.
A narrative review of the literature, guided by a conceptual framework.
We distinguish between (i) interpretation of health outcome in individuals versus groups, (ii) change within individuals or groups versus difference between change within individuals or groups; and (iii) the responder approach (based on the proportion of patients that obtain a defined response) versus the group average approach (based on the average amount of change in a group). We review approaches, concepts, and methods.
By bringing together and juxtaposing various approaches, concepts, and methods, we set a precursory step in the direction of consensus building in the field concerned with defining and assessing minimal important change and difference in health outcome. We emphasize the need for conceptual clarification and terminological standardization. We argue that assessing minimal importance of change and difference in health outcome is essentially a value judgment involving a range of considerations and perspectives.
Patient-Reported Outcome Measure Use in Guidelines Published by the American Academy of Ophthalmology: A Review
2023, Ophthalmology
We reviewed the use of patient-reported outcome measures (PROMs) in the treatment of ophthalmologic conditions as recommended by the Clinical Practice Guidelines (CPGs) published by the American Academy of Ophthalmology (AAO).
Patient-reported outcome measures are standardized instruments that provide information regarding a patient’s health status or health-related quality of life. Patient-reported outcome measures are increasingly used to inform study end points in ophthalmology studies. However, the extent to which PROMs are ultimately informing patient management recommendations in ophthalmology as part of CPGs remains an area of evidence gap.
We included all CPGs published by the AAO from inception to June 2022. We also included all primary studies and systematic reviews cited in the treatment sections of the CPGs evaluating treatment of an ophthalmic condition. The primary outcome was the frequency of PROMs discussed in CPGs and in cited studies evaluating treatment. Secondary outcomes included frequency of minimal important difference (MID) use to contextualize PROM results and percentage of strong and discretionary recommendations supported by PROMs. We published a study protocol a priori on PROSPERO (CRD42022307427). Reporting followed the Preferred Reporting Items for Systematic Review and Meta-analysis (PRISMA) guidelines. We assessed risk of bias using the Appraisal of Guidelines, Research and Evaluation II (AGREE II) instrument.
We identified 24 eligible CPGs, providing 2458 cited studies (2191 primary, 267 secondary) evaluating treatment of eye conditions. Ten CPGs (41.7%) reported consideration of PROMs. Of these, 31 of 94 (33%) recommendations were informed by studies evaluating a PROM as an outcome. Across all studies cited in the development of CPGs, 221 (9.0%) used PROMs as a primary or secondary outcome, of which 4 PROM results (1.8%) were interpreted using an empirically determined MID. Overall, the risk of bias was low for all CPGs.
Overall, outcomes of PROMs are seldom used in ophthalmology CPGs published by the AAO and in cited primary and secondary research on treatments. When PROMs were considered, their interpretation was seldom based on an MID. To improve patient care, guideline developers may consider incorporating PROMs and applicable MIDs to inform key outcomes when formulating treatment recommendations.
Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.
The difference between statistical significance and clinical relevance. The case of minimal important change, non-inferiority trials, and smallest worthwhile effect
2023, Injury
Clinical relevance and statistical significance are different concepts, linked via the sample size calculation. Threshold values for detecting a minimal important change over time are frequently (mis)interpreted as a threshold for the clinical relevance of a difference between groups. The magnitude of a difference between groups that is considered clinically relevant directly impacts the sample size calculation, and thereby the statistical significance in clinical study outcomes. Especially in non-inferiority trials the threshold for clinical relevance, i.e. the predefined margin for non-inferiority, is a crucial choice. A truly inferior treatment will be accepted as non-inferior when this margin is chosen too large. The magnitude of a clinically relevant difference between groups should be carefully considered, by determining the smallest effect for each specific study that is considered worthwhile. This means taking into account the (dis)advantages of both study interventions in terms of benefits, harms, costs, and potential side effects. This article clarifies common sources of confusion, illustrates the implications for clinical research with an example and provides specific suggestions to improve the design and interpretation of clinical research.
An extension minimal important difference credibility item addressing construct proximity is a reliable alternative to the correlation item
2023, Journal of Clinical Epidemiology
Minimal important difference (MID), the smallest change or difference that patients perceive as important, aids interpretation of change in patient-reported outcome measure (PROM) scores. A credibility instrument that assesses the methodological rigor of an anchor-based MID includes one core item addressing the correlation between the PROM and the anchor. However, the majority of MID studies in the literature fail to report the correlation. To address this issue, we extended the anchor-based MID credibility instrument by adding an item addressing construct proximity as an alternative to the correlation item.
Informed by an MID methodological survey, we added an alternative item—a subjective assessment of similarity of the constructs (i.e., construct proximity) between PROM and anchor—to the correlation item and generated principles for the assessment. We sampled 101 MIDs and analyzed the assessments performed by each pair of raters. By calculating weighted Cohen’s kappa, we assessed the reliability of the assessments.
Construct proximity assessment is based on the anticipated association between the anchor and PROM constructs: the closer the anticipated association, the higher the rating. Our detailed principles address the most frequently used anchors: transition ratings, measures of satisfaction, other PROMs, and clinical measures. The assessments showed acceptable agreement (weighted kappa 0.74, 95% CI 0.55–0.94) between raters.
In the absence of a reported correlation coefficient, construct proximity assessment provides a useful alternative in the credibility assessment of anchor-based MID estimates.
How much is enough? Considering minimally important change in youth mental health outcomes
2022, The Lancet Psychiatry
To make decisions in mental health care, service users, clinicians, and administrators need to make sense of research findings. Unfortunately, study results are often presented as raw questionnaire scores at different time points and regression coefficients, which are difficult to interpret with regards to their clinical meaning. Other commonly reported treatment outcome indicators in clinical trials or meta-analyses do not convey whether a given change score would make a noticeable difference to service users. There is an urgent need to improve the interpretability and relevance of outcome indicators in youth mental health (aged 12–24 years), in which shared decision making and person-centred care are cornerstones of an ongoing global transformation of care. In this Personal View, we make a case for considering minimally important change (MIC) as a meaningful, accessible, and user-centred outcome indicator. We discuss what the MIC represents, how it is calculated, and how it can be implemented in dialogues between clinician and researcher, and between youth and clinician. We outline how use of the MIC could enhance reporting in clinical trials, meta-analyses, clinical practice guidelines, and measurement-based care. Finally, we identify current methodological challenges around estimating the MIC and areas for future research. Efforts to select outcome domains and valid measurement instruments that resonate with youth, families, and clinicians have increased in the past 5 years. In this context, now is the time to define demarcations of changes in outcome scores that are clinically relevant, and meaningful to youth and families. Through the use of MIC, youth-centred outcome measurement, analysis, and reporting would support youth-centred therapeutic decision making.
Serious reporting deficiencies exist in minimal important difference studies: current state and suggestions for improvement
2022, Journal of Clinical Epidemiology
To evaluate reporting of minimal important difference (MID) estimates using anchor-based methods for patient-reported outcome measures (PROMs), and the association with reporting deficiencies on their credibility.
Systematic survey of primary studies empirically estimating MIDs. We searched Medline, EMBASE, PsycINFO, and the Patient-Reported Outcome and Quality of Life Instruments Database until October 2018. We evaluated study reporting, focusing on participants’ demographics, intervention(s), characteristics of PROMs and anchors, and MID estimation method(s). We assessed the impact of reporting issues on credibility of MID estimates.
In 585 studies reporting on 5,324 MID estimates for 526 distinct PROMs, authors frequently failed to adequately report key characteristics of PROMs and MIDs, including minimum and maximum values of PROM scale, measure of variability accompanying the MID estimate and number of participants included in the MID calculation. Across MID estimates (n = 5,324), the most serious reporting issues impacting credibility included infrequent reporting of the correlation between the anchor and PROM (66%), inadequate details to judge precision of MID point estimate (13%), and insufficient information about the threshold used to ascertain MIDs (16%).
Serious issues of incomplete reporting in the MID literature threaten the optimal use of MID estimates to inform the magnitude of effects of interventions on PROMs.

View all citing articles on Scopus

: Author’s contributions: A.C.L., T.D., B.C.J., G.N., S.E., and G.H.G. conceived the study idea; A.C.L., T.D., B.C.J., A.Q., M.P., and G.G. created the data extraction form for the MID inventory and led the development of the credibility instrument; A.C.L., T.D., A.Q., M.P., Y.W., N.D., D.Z., M.B., X.J., R.B.P., O.U., F.F., S.S., H.P.H., Q.H, V.W., Z.Y., L.Y., R.W.M.V., H.H., L.Z., Y.R., R.S., and L.L. extracted data and assessed the credibility of MIDs in our inventory; A.C.L. and T.D. wrote the first draft of the manuscript; A.C.L., T.D., A.Q., M.P., Y.W., N.D., B.C.J., D.Z., M.B., X.J., R.B.P., O.U., F.F., S.S., H.P.H., Q.H., V.W., ZY., L.Y., R.W.M.V., H.H., L.Z., Y.R., R.S., L.L., D.L.P., S.E., T.A.F., G.N., H.J.S., M.B., L.T., and G.H.G. interpreted the data analysis, critically revised and approved this manuscript. A.C.L. and T.D. are the guarantors.

: Conflict of interest statement: ACL, TD, and GHG hold the copyright of the credibility tool to evaluate minimal important difference estimates. (Devji T, Carrasco-Labra A, Qasim A, Phillips M, Johnston BC, Devasenapathy N, Zeraatkar D, Bhatt M, Jin X, Brignardello–Petersen R, et al. 2020. Evaluating the credibility of anchor-based estimates of minimal important differences for patient-reported outcomes: Instrument development and reliability study. BMJ (Clinical research ed). 369:m1714.)

: Funding/support: This research was funded in part by the Canadian Institutes of Health Research (CIHR), Knowledge Synthesis grant number DC0190SR.

: Role of the sponsor: The funding organization did not influence the design or conduct of the study; collection, management, analysis, or interpretation of the data; preparation, review, or approval of the manuscript; nor the decision to submit this manuscript for publication.

: Ethical approval statement: Not required. The Minimal Important Difference Inventory, authored by Dr Alonso Carrasco–Labra et al., is the copyright of McMaster University (Copyright © 2018, McMaster University, Hamilton, Ontario, Canada). The Minimal Important Difference Inventory has been provided under license from McMaster University and must not be copied, distributed, or used in any way without the prior written consent of McMaster University. Contact the McMaster Industry Liaison Office at McMaster University, email: [email protected] for licensing details.

^#: Co-first authorship.

View full text

ReviewMinimal important difference estimates for patient-reported outcomes: A systematic survey

Abstract

Objectives

Study Design and Setting

Results

Conclusion

Introduction

Section snippets

Methods

Search results

Study-level characteristics

Main findings

Acknowledgments

Spine J

Control Clin trials

J Clin Epidemiol

Mayo Clin Proc

J Clin Epidemiol

J Clin Epidemiol

Otolaryngol Head Neck Surg

J Clin Epidemiol

J Clin Epidemiol

Measurement properties and interpretability of the Chronic respiratory disease questionnaire (CRQ)

COPD

Commentary–goodbye M(C)ID! Hello MID, where do you come from?

Health Serv Res

Clinimetrics corner: a closer look at the minimal clinically important difference (MCID)

J Man Manip Ther

Interpretation of treatment changes in 6-minute walk distance in patients with COPD

Eur Respir J

Review
Minimal important difference estimates for patient-reported outcomes: A systematic survey