Abstract
Penalized likelihood is a general approach whereby an objective function is defined, consisting of the log likelihood of the data minus some term penalizing non-smooth solutions. Subsequently, this objective function is maximized, yielding a solution that achieves some sort of trade-off between the faithfulness and the smoothness of the fit.
In this paper we extend the penalized likelihood classification that we proposed in earlier work to the multi class case. The algorithms are based on using a penalty term based on the K-nearest neighbors and the likelihood of the training patterns’ classifications. The algorithms are simple to implement, and result in a performance competitive with leading classifiers.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Green, P.: Penalized Likelihood. In: Encyclopedia of Statistical Sciences Update, vol. 3. John Wiley Publishing, New Jersey (1999)
Gu, C., Kim, Y.J.: Penalized Likelihood Regression: General Formulation and Efficient Approximation. Can. J. Stat. 30, 619–628 (2002)
Green, P.J., Silverman, B.W.: Nonparametric Regression and Generalized Linear Models: A Roughness Penalty Approach. Chapman and Hall, London (1994)
Wahba, G.: Spline Models for Observational Data. SIAM, Philadelphia (1990)
O’Sullivan, F., Yandell, B., Raynor, W.: Automatic Smoothing of Regression Functions in Generalized Linear Models. J. Am. Stat. Assoc. 81, 96–103 (1986)
Gu, C.: Cross-validating Non-Gaussian Data. J. Comput. Graph. Stat. 1, 169–179 (1992)
Lu, F., Hill, G.C., Wahba, G., Desiati, P.: Signal Probability Estimation with Penalized Likelihood Method on Weighted Data. Department of Statistics, University of Wisconsin, Technical Report No. 1106 (2005)
Wahba, G.: Soft and Hard Classification by Reproducing Kernel Hilbert Space Methods. Proceedings of the National Academy of Sciences 99, 16524–16530 (2002)
Wahba, G., Gu, C., Wang, Y., Chappell, R.: Soft Classification, a.k.a. Risk Estimation, via Penalized Log Likelihood and Smoothing Spline Analysis of Variance. Department of Statistics, University of Wisconsin, Technical Report No. 899 (1993)
Cawley, G., Talbot, N.L., Girolami, M.: Sparse Multinomial Logistic Regression via Bayesian L1 Regularisation. Adv. Neural Inf. Process. Syst. 19, 209–216 (2007)
Atiya, A.F., Al-Ani, A.: A Penalized Likelihood Based Pattern Classification Algorithm. Pattern Recogn. 42, 2684–2694 (2009)
UCI Machine Learning Repository (2012), http://archive.ics.uci.edu/ml/
Chang, C.C., Lin, C.J.: LIBSVM toolbox (2012), http://www.csie.ntu.edu.tw/~cjlin/libsvm/
Vincent, P., Bengio, Y.: Manifold Parzen Windows. Adv. Neural Inf. Process. Syst. 15, 825–832 (2003)
Paris, S.: Parzen Windows Estimator Classifier (2008), http://www.mathworks.com/matlabcentral/fileexchange/17450-parzen-classifier
Goldberger, J., Roweis, S., Hinton, G., Salakhutdinov, R.: Neighbourhood Components Analysis. Adv. Neural Inf. Process. Syst. 17, 513–520 (2004)
Maaten, L.V.D.: NCA Toolbox for Dimensionality Reduction (2010), http://homepage.tudelft.nl/19j49/Matlab_Toolbox_for_Dimensionality_Reduction.html
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Talaat, A.S., Atiya, A.F., Mokhtar, S.A., Al-Ani, A., Fayek, M. (2012). Multiclass Penalized Likelihood Pattern Classification Algorithm. In: Huang, T., Zeng, Z., Li, C., Leung, C.S. (eds) Neural Information Processing. ICONIP 2012. Lecture Notes in Computer Science, vol 7665. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34487-9_18
Download citation
DOI: https://doi.org/10.1007/978-3-642-34487-9_18
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-34486-2
Online ISBN: 978-3-642-34487-9
eBook Packages: Computer ScienceComputer Science (R0)