Mingrui Wu
ABSTRACT
This paper presents a Local Learning Projection (LLP) approach for linear dimensionality reduction. We first point out that the well known Principal Component Analysis (PCA) essentially seeks the projection that has the minimal global estimation error. Then we propose a dimensionality reduction algorithm that leads to the projection with the minimal local estimation error, and elucidate its advantages for classification tasks. We also indicate that LLP keeps the local information in the sense that the projection value of each point can be well estimated based on its neighbors and their projection values. Experimental results are provided to validate the effectiveness of the proposed algorithm.
- Belhumeur, P. N., Hespanha, J. P., & Kriegman, D. J. (1997). Eigenfaces vs fisherfaces: Recognition using class-specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19, 711--720. Google Scholar
Digital Library
- Belkin, M., & Niyogi, P. (2002). Laplacian eigenmaps and spectral techniques for embedding and clustering. In T. G. Dietterich, S. Becker and Z. Ghahramani (Eds.), Advances in neural information processing systems 14. Cambridge, MA: MIT Press.Google Scholar
- Bottou, L., & Vapnik, V. (1992). Local learning algorithms. Neural Computation, 4, 888--900. Google ScholarDigital Library
- Cook, R., & Weisberg, S. (1991). Discussion of "sliced inverse regression for dimension reduction". Journal of the American Statistical Association, 86, 328--332.Google Scholar
- Globerson, A., & Roweis, S. (2006). Metric learning by collapsing classes. In Y. Weiss, B. Schöölkopf and J. Platt (Eds.), Advances in neural information processing systems 18. Cambridge, MA: MIT Press.Google Scholar
- Goldberger, J., Roweis, S., Hinton, G., & Salakhutdinov, R. (2005). Neighbourhood components analysis. In L. K. Saul, Y. Weiss and L. Bottou (Eds.), Advances in neural information processing systems 17. Cambridge, MA: MIT Press.Google Scholar
- He, X., & Niyogi, P. (2004). Locality preserving projections. In S. Thrun, L. Saul and B. Schöölkopf (Eds.), Advances in neural information processing systems 16. Cambridge, MA: MIT Press.Google Scholar
- He, X., Yan, S., Hu, Y., Niyogi, P., & Zhang, H. (2005). Face recognition using laplacianfaces. IEEE Transactions on Pattern Analysis and machine Intelligence, 27, 328--340. Google ScholarDigital Library
- Li, K.-C. (1991). Sliced inverse regression for dimension reduction. Journal of the American Statistical Association, 86, 316--327.Google ScholarCross Ref
- Meinicke, P., & Ritter, H. (1999). Local PCA learning with resolution-dependent mixtures of gaussians. International Conference on Artifical Neural Networks. Edinburgh, UK: MIT Press.Google ScholarCross Ref
- Mukherjee, S., Wu, Q., & Zhou, D.-X. (2006). Learning gradients and feature selection on manifolds. submitted to Annals of Statistics.Google Scholar
- Nadaraya, E. A. (1989). Nonparametric estimation of probabiliry densities and regression curves. Dordrecht: Kluwer Academic.Google Scholar
- Roweis, S. T., & Saul, L. K. (2000). Nonlinear dimensionality reduction by locally linear embedding. Science, 290, 2323--2326.Google ScholarCross Ref
- Schölkopf, B., & Smola, A. J. (2002). Learning with kernels. Cambridge, MA: The MIT Press.Google Scholar
- Shawe-Taylor, J., & Cristianini, N. (2004). Kernel methods for pattern analysis. Cambridge, UK: Cambridge University Press. Google ScholarDigital Library
- Sugiyama, M. (2006). Local fisher discriminant analysis for supervised dimensionality reduction. In W. Cohen and A. Moore (Eds.), Proc. 23th international conference on machine learning. ACM Press. Google ScholarDigital Library
- Tipping, M. E., & Bishop, C. M. (1999). Mixtures of probabilistic principle component analysers. Neural Computation, 11, 443--482. Google ScholarDigital Library
- Wu, M., & Schöölkopf, B. (2007). A local learning approach for clustering. In B. Schölkopf, J. Platt and T. Hoffman (Eds.), Advances in neural information processing systems 19. Cambridge, MA: MIT press.Google Scholar
- Xia, Y., Tong, H., Li, W. K., & Zhu, L.-X. (2002). An adaptive estimation of dimension reduction space. Journal of Royal Statistical Society, 64, 363--410.Google ScholarCross Ref
- Local learning projections
Recommendations
Enhanced Locality Preserving Projections
CSSE '08: Proceedings of the 2008 International Conference on Computer Science and Software Engineering - Volume 01In pattern recognition, feature extraction techniques are widely employed to reduce the dimensionality of data. In this paper, a new manifold learning algorithm, called Enhanced Locality Preserving Projections, to identify the underlying manifold ...
Locality Preserving Projections
NIPS'03: Proceedings of the 16th International Conference on Neural Information Processing SystemsMany problems in information processing involve some form of dimensionality reduction. In this paper, we introduce Locality Preserving Projections (LPP). These are linear projective maps that arise by solving a variational problem that optimally ...
Comments