Abstract
Gaussian mixture models are central to classical statistics, widely used in the information sciences, and have a rich mathematical structure. We examine their maximum likelihood estimates through the lens of algebraic statistics. The MLE is not an algebraic function of the data, so there is no notion of ML degree for these models. The critical points of the likelihood function are transcendental, and there is no bound on their number, even for mixtures of two univariate Gaussians.
Keywords
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Améndola, C., Faugère, J.-C., Sturmfels, B.: Moment varieties of Gaussian mixtures. J. Algebraic Stat. arXiv:1510.04654
Baker, A.: Transcendental Number Theory. Cambridge University Press, London (1975)
Belkin, M., Sinha, K.: Polynomial learning of distribution families. SIAM J. Comput. 44(4), 889–911 (2015)
Bishop, C.M.: Pattern Recognition and Machine Learning. Information Science and Statistics. Springer, New York (2006)
Buot, M., Hoşten, S., Richards, D.: Counting and locating the solutions of polynomial systems of maximum likelihood equations. II. The Behrens-Fisher problem. Stat. Sin. 17(4), 1343–1354 (2007)
Chang, E.-C., Choi, S.W., Kwon, D., Park, H., Yap, C.: Shortest paths for disc obstacles is computable. Int. J. Comput. Geom. Appl. 16, 567–590 (2006)
Choi, S.W., Pae, S., Park, H., Yap, C.: Decidability of collision between a helical motion and an algebraic motion. In: Hanrot, G., Zimmermann, P. (eds.) 7th Conference on Real Numbers and Computers, pp. 69–82. LORIA, Nancy (2006)
Drton, M., Sturmfels, B., Sullivant, S.: Lectures on Algebraic Statistics. Oberwolfach Seminars, vol. 39. Birkhäuser, Basel (2009)
Fraley, C., Raftery, A.E.: Enhanced model-based clustering, density estimation, and discriminant analysis software: MCLUST. J. Classif. 20, 263–286 (2003)
Ge, R., Huang, Q., Kakade, S.: Learning mixtures of Gaussians in high dimensions. In: STOC 2015, Proceedings of the Forty-Seventh Annual ACM on Symposium on Theory of Computing, pp. 761–770 (2015)
Gelfond, A.O.: Transcendental and Algebraic Numbers. Translated by Leo F. Boron, Dover Publications, New York (1960)
Gross, E., Drton, M., Petrović, S.: Maximum likelihood degree of variance component models. Electron. J. Stat. 6, 993–1016 (2012)
Huh, J., Sturmfels, B.: Likelihood geometry. In: Conca, A., et al. (eds.) Combinatorial Algebraic Geometry. Lecture Notes in Math., vol. 2108, pp. 63–117. Springer, Heidelberg (2014)
Moitra, A., Valiant, G.: Settling the polynomial learnability of mixtures of Gaussians. In: IEEE 51st Annual Symposium on Foundations of Computer Science, pp. 93–102 (2010)
Pearson, K.: Contributions to the mathematical theory of evolution. Philos. Trans. R. Soc. Lond. A 185, 71–110 (1894)
Redner, R.A., Walker, H.F.: Mixture densities, maximum likelihood and the EM algorithm. SIAM Rev. 26, 195–239 (1984)
Reeds, J.A.: Asymptotic number of roots of Cauchy location likelihood equations. Ann. Statist. 13(2), 775–784 (1985)
Srebro, N.: Are there local maxima in the infinite-sample likelihood of Gaussian mixture estimation? In: Bshouty, N.H., Gentile, C. (eds.) COLT. LNCS (LNAI), vol. 4539, pp. 628–629. Springer, Heidelberg (2007)
Sturmfels, B., Uhler, C.: Multivariate Gaussian, semidefinite matrix completion, and convex algebraic geometry. Ann. Inst. Statist. Math. 62(4), 603–638 (2010)
Teicher, H.: Identifiability of finite mixtures. Ann. Math. Stat. 34, 1265–1269 (1963)
Watanabe, S.: Algebraic Geometry and Statistical Learning Theory. Monographs on Applied and Computational Mathematics, vol. 25. Cambridge University Press, Cambridge (2009)
Watanabe, S., Yamazaki, K., Aoyagi, M.: Kullback information of normal mixture is not an analytic function. IEICE Technical report, NC2004-50 (2004)
Acknowledgements
CA and BS were supported by the Einstein Foundation Berlin. MD and BS also thank the US National Science Foundation (DMS-1305154 and DMS-1419018).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Améndola, C., Drton, M., Sturmfels, B. (2016). Maximum Likelihood Estimates for Gaussian Mixtures Are Transcendental. In: Kotsireas, I., Rump, S., Yap, C. (eds) Mathematical Aspects of Computer and Information Sciences. MACIS 2015. Lecture Notes in Computer Science(), vol 9582. Springer, Cham. https://doi.org/10.1007/978-3-319-32859-1_49
Download citation
DOI: https://doi.org/10.1007/978-3-319-32859-1_49
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-32858-4
Online ISBN: 978-3-319-32859-1
eBook Packages: Computer ScienceComputer Science (R0)