Abstract
Disease-specific quality indicators are used to compare institutions and health care providers in terms of processes or outcomes relevant to treatment of a particular condition. In the context of surgical cancer treatments, the performance variations can be due to hospital and/or surgeon level differences, creating a hierarchical clustering. We consider how the observed variation in care received at patient level can be decomposed into that causally explained by the hospital performance, surgeon performance within hospital, patient case-mix, and unexplained (residual) variation. For this purpose, we derive a four-way variance decomposition, with particular attention to the causal interpretation of the components. For estimation, we use inputs from a mixed-effect model with nested random hospital/surgeon-specific effects, and a multinomial logistic model for the hospital/surgeon-specific patient populations. We investigate the performance of our methods in a simulation study and demonstrate them through analysis of administrative data on kidney cancer care in Ontario.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
The data that support the findings of this study are available from ICES (https://www.ices.on.ca/). Restrictions apply to the access to these data, which were used under agreement for this study.
Code availability
R code to reproduce the simulation study will be made available at the Journal website.
References
Angrist, J.D., Imbens, G.W., Rubin, D.B.: Identification of causal effects using instrumental variables. J. Am. Stat. Assoc. 91(434), 444–455 (1996)
Bates, D., Mächler, M., Bolker, B., Walker, S.: Fitting linear mixed-effects models using lme4. J. Stat. Softw. 67(1), 1–48 (2015). https://doi.org/10.18637/jss.v067.i01
Bell, A., Fairbrother, M., Jones, K.: Fixed and random effects models: making an informed choice. Qual. Quant. 53(2), 1051–1074 (2019)
Bowsher, C.G., Swain, P.S.: Identifying sources of variation and the flow of information in biochemical networks. Proc. Natl. Acad. Sci. 109(20), E1320–E1328 (2012)
Chen, B., Lawson, K.A., Finelli, A., Saarela, O.: Causal variance decompositions for institutional comparisons in healthcare. Stat. Methods Med. Res. 29(7), 1972–1986 (2020)
Chen, B., Lawson, K.A., Finelli, A., Saarela, O.: Causal mediation analysis decomposition of between-hospital variance. Health Serv. Outcomes Res. Method. 22(1), 118–144 (2022)
Clarke, P., Crawford, C., Steele, F., Vignoles, A.: Revisiting fixed-and random-effects models: some considerations for policy-relevant education research. Educ. Econ. 23(3), 259–277 (2015)
Daignault, K., Lawson, K.A., Finelli, A., Saarela, O.: Causal mediation analysis for standardized mortality ratios. Epidemiology 30(4), 532–540 (2019)
Daniels, M.J., Gatsonis, C.: Hierarchical generalized linear models in the analysis of variations in health care utilization. J. Am. Stat. Assoc. 94(445), 29–42 (1999)
Díaz, I., van der Laan, M.: Population intervention causal effects based on stochastic interventions. Biometrics 68(2), 541–549 (2012)
Dieleman, J.L., Templin, T.: Random-effects, fixed-effects and the within-between specification for clustered data in observational health studies: A simulation study. PLoS One 9(10), e110257 (2014)
Donabedian, A.: The quality of care. How can it be assessed? JAMA 260(12), 1743–1748 (1988)
Farrell, P.J., Groshen, S., MacGibbon, B., Tomberlin, T.J.: Outlier detection for a hierarchical bayes model in a study of hospital variation in surgical procedures. Stat. Methods Med. Res. 19(6), 601–619 (2010)
Fisher, A., Rudin, C., Dominici, F.: All models are wrong, but many are useful: learning a variable’s importance by studying an entire class of prediction models simultaneously. J. Mach. Learn. Res. 20(177), 1–81 (2019)
Goldstein, H., Spiegelhalter, D.J.: League tables and their limitations: statistical issues in comparisons of institutional performance. J. R. Stat. Soc. Ser. A 159, 385–443 (1996)
Hawley, S.T., Hofer, T.P., Janz, N.K., Fagerlin, A., Schwartz, K., Liu, L., Deapen, D., Morrow, M., Katz, S.J.: Correlates of between-surgeon variation in breast cancer treatments. Med. Care 44, 609–616 (2006)
Hernán, M.A., Robins, J.M.: Estimating causal effects from epidemiological data. J. Epidemiol. Community Health 60, 578–586 (2006)
Lawson, K.A., Saarela, O., Abouassaly, R., Kim, S.P., Breau, R.H., Finelli, A.: The impact of quality variations on patients undergoing surgery for renal cell carcinoma: a national cancer database study. Eur. Urol. 72(3), 379–386 (2017)
Longford, N.T.: A fast scoring algorithm for maximum likelihood estimation in unbalanced mixed models with nested random effects. Biometrika 74(4), 817–827 (1987)
Merlo, J., Chaix, B., Ohlsson, H., Beekman, A., Johnell, K., Hjerpe, P., Råstam, L., Larsen, K.: A brief conceptual tutorial of multilevel analysis in social epidemiology: using measures of clustering in multilevel logistic regression to investigate contextual phenomena. J. Epidemiol. Community Health 60(4), 290–297 (2006)
Norberg, R.: Hierarchical credibility: analysis of a random effect linear model with nested classification. Scand. Actuar. J. 1986(3–4), 204–222 (1986)
Rabe-Hesketh, S., Skrondal, A., Pickles, A.: Maximum likelihood estimation of limited and discrete dependent variable models with nested random effects. J. Econom. 128(2), 301–323 (2005)
Racz, M.J., Sedransk, J.: Bayesian and frequentist methods for provider profiling using risk-adjusted assessments of medical outcomes. J. Am. Stat. Assoc. 105, 48–58 (2010)
Rubin, D.B., Rosenbaum, P.R.: The central role of the propensity score in observational studies for causal effects. Biometrika 70(1), 41–55 (1983)
Shahian, D.M., Normand, S.L.T.: Comparison of risk-adjusted hospital outcomes. Circ. J. Am. Heart Assoc. 117, 1955–1963 (2008)
Skrondal, A., Rabe-Hesketh, S.: Generalized latent variable modeling: Multilevel, longitudinal, and structural equation models. Crc Press, Florida (2004)
VanderWeele, T., Vansteelandt, S.: Conceptual issues concerning mediation, interventions and composition. Stat Interface 2, 457–468 (2009)
VanderWeele, T., Vansteelandt, S., Robins, J.: Effect decomposition in the presence of an exposure-induced mediator-outcome confounder. Epidemiology 25(2), 300–306 (2014)
Varewyck, M., Goetghebeur, E., Eriksson, M., Vansteelandt, S.: On shrinkage and model extrapolation in the evaluation of clinical center performance. Biostatistics 15(4), 651–664 (2014)
Wood, L., Bjarnason, G.A., Black, P.C., Cagiannos, I., Heng, D.Y., Kapoor, A., Kollmannsberger, C.K., Mohammadzadeh, F., Moore, R.B., Rendon, R.A., Soulieres, D., Tanguay, S., Venner, P., Jewett, M., Finelli, A.: Using the Delphi technique to improve clinical outcomes through the development of quality indicators in renal cell carcinoma. J. Oncol. Pract. 9(5), e262-267 (2013)
Yee, T.W., Stoklosa, J., Huggins, R.M.: The VGAM package for capture-recapture data using the conditional likelihood. J. Stat. Softw. 65(5), 1–33 (2015)
Acknowledgements
This work was supported by a Discovery Grant from the Natural Sciences and Engineering Research Council of Canada (to OS), a Catalyst Grant in Health Services and Economics Research from the Canadian Institutes of Health Research (to AF, KAL and OS) and the Ontario Institute for Cancer Research through funding provided by the Government of Ontario (to BC). This study contracted ICES Data & Analytic Services (DAS) and used de-identified data from the ICES Data Repository, which is managed by ICES with support from its funders and partners: Canada’s Strategy for Patient-Oriented Research (SPOR), the Ontario SPOR Support Unit, the Canadian Institutes of Health Research and the Government of Ontario. The opinions, results and conclusions reported are those of the authors. No endorsement by ICES or any of its funders or partners is intended or should be inferred. Parts of this material are based on data and information compiled and provided by CIHI. However, the analyses, conclusions, opinions and statements expressed herein are those of the author, and not necessarily those of CIHI. Parts of this material are based on data and information provided by Cancer Care Ontario (CCO). The opinions, results, view, and conclusions reported in this paper are those of the authors and do not necessarily reflect those of CCO. No endorsement by CCO is intended or should be inferred.
Author information
Authors and Affiliations
Contributions
The methodological research problem was devised by OS; the methods were formulated and implemented by BC, who wrote the first draft of the manuscript. KM, KAL and AF devised the applied research problem, and provided input into the creation of the analysis dataset, data analysis and interpretation of the results. All authors reviewed the manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Ethical approval
Our study was approved by the Research Ethics Board of the University Health Network, Toronto, Ontario.
Consent for publication
Please see https://www.ices.on.ca/Data-and-Privacy/ICES-data/Working-with-ICES-Data.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Chen, B., McAlpine, K., Lawson, K.A. et al. Hierarchical causal variance decomposition for institution and provider comparisons in healthcare. Health Serv Outcomes Res Method 23, 391–415 (2023). https://doi.org/10.1007/s10742-023-00301-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10742-023-00301-6