Abstract
Purpose
Treatment benefit as assessed using clinical outcome assessments (COAs), is a key endpoint in many clinical trials at both the individual and group level. Anchor-based methods can aid interpretation of COA change scores beyond statistical significance, and help derive a meaningful change threshold (MCT). However, evidence-based guidance on the selection of appropriately related anchors is lacking.
Methods
A simulation was conducted which varied sample size, change score variability and anchor correlation strength to assess the impact of these variables on recovering the simulated MCT for interpreting individual and group-level results. To assess MCTs derived at the individual-level (i.e. responder definitions; RDs), Receiver Operating Characteristic (ROC) curves and Predictive Modelling (PM) analyses were conducted. To assess MCTs for interpreting change at the group-level, the mean change method was conducted.
Results
Sample sizes, change score variability and magnitude of anchor correlation affected accuracy of the estimated MCT. For individual-level RDs, ROC curves were less accurate than PM methods at recovering the true MCT. For both methods, smaller samples led to higher variability in the returned MCT, but higher variability still using ROC. Anchors with weaker correlations with COA change scores had increased variability in the estimated MCT. An anchor correlation of around 0.50–0.60 identified a true MCT cut-point under certain conditions using ROC. However, anchor correlations as low as 0.30 were appropriate when using PM under certain conditions. For interpreting group-level results, the MCT derived using the mean change method was consistently underestimated regardless of the anchor correlation.
Conclusion
Sample size and change score variability influence the necessary anchor correlation strength when recovering individual-level RDs. Often, this needs to be higher than the commonly accepted threshold of 0.30. Stronger correlations than 0.30 are required when using the mean change method. Results can assist researchers selecting and assessing the quality of anchors.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Change history
09 February 2023
A Correction to this paper has been published: https://doi.org/10.1007/s11136-023-03356-7
References
Coon, C. D., & Cappelleri, J. C. (2016). Interpreting change in scores on patient-reported outcome instruments. Therapeutic Innovation & Regulatory Science, 50(1), 22–29.
Revicki, D. H. R., Cella, D., & Sloan, J. (2008). Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. Clinical Epidemiology, 61(2), 102–109.
Food and Drug Administration. (2009) Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims.
Coon, C. D., & Cook, K. F. (2018). Moving from significance to real-world meaning: Methods for interpreting change in clinical outcome assessment scores. Quality of Life Research, 27(1), 33–40.
Terwee, C. B., Peipert, J. D., Chapman, R., Lai, J. S., Terluin, B., Cella, D., Griffith, P., & Mokkink, L. B. (2021). Minimal important change (MIC): a conceptual clarification and systematic review of MIC estimates of PROMIS measures. Quality of Life Research, 30(10), 2729–2754.
Martin Bland, J., & Altman, D. (1986). Statistical methods for assessing agreement between two methods of clinical measurement. The Lancet, 327(8476), 307–310.
Food and Drug Administration. (2018) Patient-Focused Drug Development: Methods to Identify What is Important to Patients & Select, Develop or Modify Fit-for-Purpose Clinical Outcomes Assessments. Draft discussion document.
Cohen, J. (2013). Statistical power analysis for the behavioral sciences. Academic press.
Hays, R. D., Farivar, S. S., & Liu, H. (2005). Approaches and recommendations for estimating minimally important differences for health-related quality of life measures. COPD, 2(1), 63–67.
Guyatt, G. H., Norman, G. R., Juniper, E. F., & Griffith, L. E. (2002). A critical look at transition ratings. Journal of Clinical Epidemiology, 55(9), 900–908.
Escobar, A., Garcia Perez, L., Herrera-Espineira, C., et al. (2013). Total knee replacement; minimal clinically important differences and responders. Osteoarthritis Cartilage, 21(12), 2006–2012.
Devji, T., Carrasco-Labra, A., Qasim, A., Phillips, M., Johnston, B. C., Devasenapathy, N., Zeraatkar, D., Bhatt, M., Jin, X., Brignardello-Petersen, R., Urquhart, O., Foroutan, F., Schandelmaier, S., Pardo-Hernandez, H., Vernooij, R. W., Huang, H., Rizwan, Y., Siemieniuk, R., Lytvyn, L.,...Guyatt, G. H. (2020). Evaluating the credibility of anchor based estimates of minimal important differences for patient reported outcomes: instrument development and reliability study. BMJ, 2369, m1714.
Ousmen, A., Touraine, C., Deliu, N., Cottone, F., Bonnetain, F., Efficace, F., Brédart, A., Mollevi, C., & Anota, A. (2018). Distribution- and anchor-based methods to determine the minimally important difference on patient-reported outcome questionnaires in oncology: a structured review. Health Qual Life Outcomes, 16(1), 228.
Fayers, P. M., & Hays, R. D. (2014). Don’t middle your MIDs: Regression to the mean shrinks estimates of minimally important differences. Quality of Life Research, 23(1), 1–4.
Morris, T. P., White, I. R., & Crowther, M. J. (2019). Using simulation studies to evaluate statistical methods. Statistics in Medicine, 38(11), 2074–2102.
Terluin, B., Eekhout, I., Terwee, C. B., & de Vet, H. C. (2015). Minimal important change (MIC) based on a predictive modeling approach was more precise than MIC based on ROC analysis. Journal of Clinical Epidemiology, 68(12), 1388–1396.
Terluin, B., Griffiths, P., van der Wouden, J. C., Ingelsrud, L. H., & Terwee, C. B. (2020). Unlike ROC analysis, a new IRT method identified clinical thresholds unbiased by disease prevalence. Journal of Clinical Epidemiology, 124, 118–125.
Funding
The authors have no relevant financial or non-financial interests to disclose.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that no funds, grants, or other support were received during the preparation of this manuscript.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The original online version of this article was revised: Co-corresponding authorship for the author Abi Williams is removed.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Griffiths, P., Sims, J., Williams, A. et al. How strong should my anchor be for estimating group and individual level meaningful change? A simulation study assessing anchor correlation strength and the impact of sample size, distribution of change scores and methodology on establishing a true meaningful change threshold. Qual Life Res 32, 1255–1264 (2023). https://doi.org/10.1007/s11136-022-03286-w
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11136-022-03286-w