Abstract
The purpose of this study is to propose a new online and nonlinear PCA(OL-NPCA) method for feature extraction from the incremental data. Kernel PCA(KPCA) is widely used for nonlinear feature extraction, however, it has been pointed out that KPCA has the following problems. First, applying KPCA to patterns requires storing and finding the eigenvectors of a kernel matrix, which is infeasible for a large number of data N. Second problem is that in order to update the eigenvectors with an another data, the whole eigenspace should be recomputed. OL-NPCA overcomes these problems by incremental eigenspace update method with a feature mapping function. According to the experimental results, which comes from applying OL-NPCA to a toy and a large data problem, OL-NPCA shows following advantages. First, OL-NPCA is more efficient in memory requirement than KPCA. Second advantage is that OL-NPCA is comparable in performance to KPCA. Furthermore, performance of OL-NPCA can be easily improved by re-learning the data. For classification extracted features are used as input for least squares support vector machine. In our experiments we show that proposed feature extraction method is comparable in performance to a Kernel PCA and proposed classification system shows a high classification performance on UCI benchmarking data and NIST handwritten data set.
This study was supported by a grant of the Korea Health 21 R&D Project, Ministry of Health & Welfare, Republic of Korea (02-PJ1-PG6-HI03-0004)
Chapter PDF
Similar content being viewed by others
References
Tipping, M.E., Bishop, C.M.: Mixtures of probabilistic principal component analysers. Neural Computation 11(2), 443–482 (1998)
Kramer, M.A.: Nonlinear principal component analysis using autoassociative neural networks. AICHE Journal 37(2), 233–243 (1991)
Diamantaras, K.I., Kung, S.Y.: Principal Component Neural Networks: Theory and Applications. John Wiley & Sons, Inc., Chichester (1996)
Kim, B.J., Shim, J.Y., Hwang, C.H., Kim, I.K.: Incremental Feature Extraction Based on Emperical Feature Map. In: Zhong, N., Raś, Z.W., Tsumoto, S., Suzuki, E. (eds.) ISMIS 2003. LNCS (LNAI), vol. 2871, pp. 440–444. Springer, Heidelberg (2003)
Softky, W.S., Kammen, D.M.: Correlation in high dimensional or asymmetric data set: Hebbian neuronal processing. Neural Networks 4, 337–348 (1991)
Gupta, H., Agrawal, A.K., Pruthi, T., Shekhar, C., Chellappa., R.: An Experimental Evaluation of Linear and Kernel-Based Methods for Face Recognition, accessible at http://citeseer.nj.nec.com
Suykens, J.A.K., Vandewalle, J.: Least squares support vector machine classifiers. Neural Processing Letters 9, 293–300 (1999)
Vapnik, V.N.: Statistical learning theory. John Wiley & Sons, New York (1998)
Hall, P., Marshall, D., Martin, R.: Incremental eigenalysis for classification. In: British Machine Vision Conference, September 1998, vol. 1, pp. 286–295 (1998)
Winkeler, J., Manjunath, B.S., Chandrasekaran, S.: Subset selection for active object recognition. In: CVPR, June 1999, vol. 2, pp. 511–516. IEEE Computer Society Press, Los Alamitos (1999)
Murakami, H., Kumar, B.V.K.V.: Efficient calculation of primary images from a set of images. IEEE PAMI 4(5), 511–515 (1982)
Scholkopf, B., Smola, A., Muller, K.R.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation 10(5), 1299–1319 (1998)
Tsuda, K.: Support vector classifier based on asymmetric kernel function. In: Proc. ESANN (1999)
Mika, S.: Kernel algorithms for nonlinear signal processing in feature spaces. Master’s thesis, Technical University of Berlin (November 1998)
Diamantaras, K.I., Kung, S.Y.: Principal Component Neural Networks: Theory and Applications. John Wiley&Sons, Inc., New York (1996)
Suykens, J.A.K., Vandewalle, J.: Multiclass Least Squares Support Vector Machines. In: Proc. International Joint Conference on Neural Networks (IJCNN 1999), Washington DC (1999)
Gestel, V., Suykens, T.J.A.K., Lanckriet, G., Lambrechts, D.M.A.B., Vandewalle, J.: A Bayesian Framework for Least Squares Support Vector Machine Classifiers. Internal Report 00-65, ESAT-SISTA, K.U. Leuven
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kim, B.J., Kim, I.K. (2004). Incremental Nonlinear PCA for Classification. In: Boulicaut, JF., Esposito, F., Giannotti, F., Pedreschi, D. (eds) Knowledge Discovery in Databases: PKDD 2004. PKDD 2004. Lecture Notes in Computer Science(), vol 3202. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30116-5_28
Download citation
DOI: https://doi.org/10.1007/978-3-540-30116-5_28
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-23108-0
Online ISBN: 978-3-540-30116-5
eBook Packages: Springer Book Archive