Abstract
Extended redundancy analysis (ERA), a generalized version of redundancy analysis (RA), has been proposed as a useful method for examining interrelationships among multiple sets of variables in multivariate linear regression models. As a limitation of the extant RA or ERA analyses, however, parameters are estimated by aggregating data across all observations even in a case where the study population could consist of several heterogeneous subpopulations. In this paper, we propose a Bayesian mixture extension of ERA to obtain both probabilistic classification of observations into a number of subpopulations and estimation of ERA models within each subpopulation. It specifically estimates the posterior probabilities of observations belonging to different subpopulations, subpopulation-specific residual covariance structures, component weights and regression coefficients in a unified manner. We conduct a simulation study to demonstrate the performance of the proposed method in terms of recovering parameters correctly. We also apply the approach to real data to demonstrate its empirical usefulness.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Anderson, T. W. (1951). Estimating linear restrictions on regression coefficients for multivariate normal distributions. Annals of Mathematical Statistics, 22(3), 327–351.
Benaglia, T., Chauveau, D., Hunter, D., & Young, D. (2009). mixtools: An R package for analyzing mixture models. Journal of Statistical Software, 32(6), 1–29.
Celeux, G., Forbes, F., Robert, C. P., & Titterington, D. M. (2006). Deviance information criteria for missing data models. Bayesian Analysis, 1(4), 651–673. https://doi.org/10.1214/06-BA122
Choi, J. Y., Kyung, M., Hwang, H., & Park, J.-H. (2019). Bayesian extended redundancy analysis: A Bayesian approach to component-based regression with dimension reduction. Multivariate Behavioral Research. https://doi.org/10.1080/00273171.2019.1598837
Dauvier, B., Chevalier, N., & Blaye, A. (2012). Using finite mixture of GLMs to explore variability in children’s flexibility in a task-switching paradigm. Cognitive Development, 27(4), 440–454. https://doi.org/10.1016/j.cogdev.2012.07.004
Davies, P. T., & Tso, M.K.-S. (1982). Procedures for reduced-rank regression. Applied Statistics, 31, 244–255. https://doi.org/10.2307/2347998
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society: Series B (Methodological), 39(1), 1–22. https://doi.org/10.1111/j.2517-6161.1977.tb01600.x
Diebolt, J., & Robert, C. P. (1994). Estimation of finite mixture distributions through Bayesian sampling. Journal of the Royal Statistical Society: Series B (Methodological), 56(2), 363–375. https://doi.org/10.1111/j.2517-6161.1994.tb01985.x
Drton, M., & Plummer, M. (2017). A Bayesian information criterion for singular models. Journal of the Royal Statistical Society: Series B (Methodological), 79(2), 323–380. https://doi.org/10.1111/rssb.12187
Ferguson, T. (1973). A Bayesian analysis of some nonparametric problems. Annals of Statistics, 1(2), 209–230.
Fritsch, A., & Ickstadt, K. (2009). Improved criteria for clustering based on the posterior similarity matrix. Bayesian Analysis, 4, 367–392. https://doi.org/10.1214/09-BA414
Fraley, C., & Raftery, A. E. (2002). Model-based clustering, discriminant analysis, and density estimation. Journal of the American Statistical Association, 458, 611–631. https://doi.org/10.1198/016214502760047131
Frühwirth-Schnatter, S. (2006). Finite Mixture and Markov Switching Models, Springer Series in Statistics. New York: Springer.
Frühwirth-Schnatter, S., & Malsiner-Walli, G. (2019). From here to infinity: Sparse finite versus Dirichlet process mixtures in model-based clustering. Advances in Data Analysis and Classification, 13, 33–64. https://doi.org/10.1007/s11634-018-0329-y
Hwang, H., DeSarbo, S. W., & Takane, Y. (2007). Fuzzy clusterwise generalized structured component analysis. Psychometrika, 72, 181–198. https://doi.org/10.1007/s11336-005-1314-x
Hwang, H., Suk, H. W., Lee, J.-H., Moskowitz, D. S., & Lim, J. (2012). Functional extended redundancy analysis. Psychometrika, 77(3), 524–542. https://doi.org/10.1007/S11336-012-9268-2
Hwang, H., Suk, H. W., Takane, Y., Lee, J.-H., & Lim, J. (2015). Generalized functional extended redundancy analysis. Psychometrika, 80, 101–125. https://doi.org/10.1007/S11336-013-9373-X
Hotelling, H. (1957). The relations of the newer multivariate statistical methods to factor analysis. British Journal of Mathematical and Statistical Psychology, 10(2), 69–79. https://doi.org/10.1111/j.2044-8317.1957.tb00179.x
Ishwaran, H., & Zarepour, M. (2000). Markov chain Monte Carlo in approximate Dirichlet and beta two-parameter process hierarchical models. Biometrika, 87, 371–390. https://doi.org/10.1093/biomet/87.2.371
Jolliffe, I. T. (1982). A note on the use of principal components in regression. Applied Statistics. https://doi.org/10.2307/2348005
Kamakura, W. A., Kim, B. D., & Lee, J. (1996). Modeling preference and structural heterogeneity in consumer choice. Marketing Science, 15(2), 152–172. https://doi.org/10.1287/mksc.15.2.152
Kamakura, W. A., & Russell, G. (1989). A probabilistic choice model for market segmentation and elasticity structure. Journal of Marketing Research, 26, 379–390. https://doi.org/10.1177/002224378902600401
Kok, B. C., Choi, J. S., Oh, H., & Choi, J. Y. (2019). Sparse extended redundancy analysis: Variable selection via the exclusive lasso. Multivariate Behavioral Research. https://doi.org/10.1080/00273171.2019.1694477
Liu, J., Wong, W., & Kong, A. (1994). Covariance structure of the Gibbs sampler with application to the comparisons of estimators and sampling schemes. Biometrika, 81(1), 27–40. https://doi.org/10.1093/biomet/81.1.27
Lovaglio, P. G., & Vittadini, G. (2014). Structural equation models in a redundancy analysis framework with covariates. Multivariate Behavioral Research, 49(5), 486–501. https://doi.org/10.1080/00273171.2014.931798
Malsiner-Walli, G., Frühwirth-Schnatter, S., & Grün, B. (2016). Model-based clustering based on sparse finite Gaussian mixtures. Statistics and Computing, 26, 303–324. https://doi.org/10.1007/s11222-014-9500-2
McCullagh, P., & Nelder, J. A. (1983). Generalized linear models. London: Chapman and Hall.
McLachlan, G. J., & Peel, D. (2000). Finite mixture models. New York: Wiley.
Mengersen, K. L., Robert, C. P., & Titterington, D. M. (2011). Mixtures: Estimation and applications. New York: Wiley.
Park, J.-H., Choi, J. Y., Lee, J., & Kyung, M. (2021). Bayesian approach to multivariate component-based logistic regression: analyzing correlated multivariate ordinal data. Multivariate Behavioral Research. https://doi.org/10.1080/00273171.2021.1874260
Pouwels, J. . L., & Cillessen, A. . H. . N. (2013). Correlates and outcomes associated with aggression and victimization among elementary-school children in a low-income urban context. Journal of Youth and Adolescence, 42, 190–205. https://doi.org/10.1007/s10964-012-9875-3
Raftery, A. E. (1995). Bayesian model selection in social research. Sociological Methodology, 25, 111–163.
Ramsay, J., Hooker, G., & Graves, S. (2009). Functional data analysis with R and Matlab. Berlin: Springer.
Richardson, S., & Green, P. . J. (1997). On Bayesian analysis of mixtures with an unknown number of components (with discussion). Journal of the Royal Statistical Society: Series B (Statistical Methodology), 59, 731–792. https://doi.org/10.1111/1467-9868.00095
Rousseau, J., & Mengersen, K. (2011). Asymptotic behaviour of the posterior distribution in overfitted mixture models. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 73(5), 689–710. https://doi.org/10.1111/j.1467-9868.2011.00781.x
Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6(2), 461–464. https://doi.org/10.1214/aos/1176344136
Scrucca, L., Fop, M., Murphy, T. . B., & Raftery, A. . E. (2016). mclust 5: Clustering, classification and density estimation using Gaussian finite mixture models. The R Journal, 8(1), 289–317.
Steele, R. J., & Raftery, A. E. (2010). Performance of Bayesian model selection criteria for Gaussian mixture models. Frontiers of Statistical Decision Making and Bayesian Analysis, 2, 113–130.
Stephens, M. (2000). Bayesian analysis of mixture models with an unknown number of components—An alternative to reversible jump methods. The Annals of Statistics, 28, 40–74.
Swait, J., & Sweeney, J. C. (2000). Perceived value and its impact on choice behavior in a retail setting. Journal of Retailing and Consumer Services, 7(2), 77–88. https://doi.org/10.1016/S0969-6989(99)00012-0
Takane, Y., & Hwang, H. (2005). An extended redundancy analysis and its applications to two practical examples. Computational Statistics & Data Analysis, 49(3), 785–808. https://doi.org/10.1016/j.csda.2004.06.004
Tanner, M. D., & Wong, W. (1987). The calculation of posterior distribution by data augmentation (with discussion). Journal of the American Statistical Association, 82, 528–550.
van den Wollenberg, A. L. (1977). Redundancy analysis an alternative for canonical correlation analysis. Psychometrika, 42(2), 207–219. https://doi.org/10.1007/BF02294050
Vehtari, A., Gelman, A. & Gabry, J. (2017). Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Statistics and Computing, 27, 1413–1432. https://doi.org/10.1007/s11222-016-9696-4
Viele, K., & Tong, B. (2002). Modeling with mixtures of linear regressions. Statistics and Computing, 12, 315–330. https://doi.org/10.1023/A:1020779827503
Wang, J. M., Duong, M., Schwartz, D., Chang, L., & Luo, T. (2014). Interpersonal and personal antecedents and consequences of peer victimization across middle childhood in Hong Kong. Journal of Youth and Adolescence, 43, 1934–1945. https://doi.org/10.1007/s10964-013-0050-2
Watanabe, S. (2013). A widely applicable Bayesian information criterion. Journal of Machine Learning Research, 14, 867–897.
Watanabe, S. (2021). WAIC and WBIC for mixture models. Behaviormetrika, 48, 5–21. https://doi.org/10.1007/s41237-021-00133-z
Wedel, M. (2000). Market segmentation: Conceptual and methodological foundations. Berlin: Springer.
Wold, H. (1966). Estimation of principal components and related methods by iterative least squares. In P. R. Krishnaiah (Ed.), Multivariate analysis (pp. 391–420). New York: Academic Press.
Wold, H. (1973). Nonlinear iterative partial least squares (NIPALS) modeling: some current developments. In: Krishnaiah, P.R. (Ed.), Multivariate Analysis (pp. 383–487). New York: Academic Press. https://doi.org/10.1016/B978-0-12-426653-7.50032-6.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Minjung Kyung and Ju-Hyun Park have contributed equally to this work.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Kyung, M., Park, JH. & Choi, J.Y. Bayesian Mixture Model of Extended Redundancy Analysis. Psychometrika 87, 946–966 (2022). https://doi.org/10.1007/s11336-021-09809-7
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11336-021-09809-7