A survey of multilinear subspace learning for tensor data
Introduction
With the advances in data collection and storage capabilities, massive multidimensional data are being generated on a daily basis in a wide range of emerging applications, and learning algorithms for knowledge extraction from these data are becoming more and more important. Two-dimensional (2D) data include gray-level images in computer vision and pattern recognition [1], [2], [3], [4], multichannel EEG signals in biomedical engineering [5], [6], and gene expression data in bioinformatics [7]. Three-dimensional (3D) data include 3D objects in generic object recognition [8], hyperspectral cube in remote sensing [9], and gray-level video sequences in activity or gesture recognition for surveillance or human–computer interaction (HCI) [10], [11]. There are also many multidimensional signals in medical image analysis [12], content-based retrieval [1], [13], and space-time super-resolution [14] for digital cameras with limited spatial and temporal resolution. In addition, many streaming data and mining data are frequently organized as third-order tensors [15], [16], [17]. Data in environmental sensor monitoring are often organized in three modes of time, location, and type [17]. Data in social network analysis are usually organized in three modes of time, author, and keywords [17]. Data in network forensics are often organized in three modes of time, source, and destination, and data in web graph mining are commonly organized in three modes of source, destination, and text [15].
These massive multidimensional data are usually very high-dimensional, with a large amount of redundancy, and only occupying a subspace of the input space [18]. Thus, for feature extraction, dimensionality reduction is frequently employed to map high-dimensional data to a low-dimensional space while retaining as much information as possible [18], [19]. However, this is a challenging problem due to the large variability and complex pattern distribution of the input data, and the limited number of samples available for training in practice [20]. Linear subspace learning (LSL) algorithms are traditional dimensionality reduction techniques that represent input data as vectors and solve for an optimal linear mapping to a lower-dimensional space. Unfortunately, they often become inadequate when dealing with massive multidimensional data. They result in very high-dimensional vectors, lead to the estimation of a large number of parameters, and also break the natural structure and correlation in the original data [21], [2], [22].
Due to the challenges in emerging applications above, there has been a pressing need for more effective dimensionality reduction schemes for massive multidimensional data. Recently, interests have grown in multilinear subspace learning (MSL) [2], [21], [22], [23], [24], [25], [26], a novel approach to dimensionality reduction of multidimensional data where the input data are represented in their natural multidimensional form as tensors. Fig. 1 shows two examples of tensor data representations for a face image and a silhouette sequence. MSL has the potential to learn more compact and useful representations than LSL [21], [27] and it is expected to have potential future impact in both developing new MSL algorithms and solving problems in applications involving massive multidimensional data. The research on MSL has gradually progressed from heuristic exploration to systematic investigation [28] and both unsupervised and supervised MSL algorithms have been proposed in the past a few years [2], [21], [22], [23], [24], [25], [26].
It should be noted that MSL belongs to tensor data analysis (or tensor-based computation and modeling), which is more general and has a much wider scope. Multilinear algebra, the basis of tensor data analysis, has been studied in mathematics for several decades [29], [30], [31] and there are a number of recent survey papers summarizing recent developments in tensor data analysis. E.g., Qi et al. review numerical multilinear algebra and its applications in [32]. Muti and Bourennane [33] survey new filtering methods for multicomponent data modeled as tensors in noise reduction for color images and multicomponent seismic data. Acar and Yener [34] survey unsupervised multi-way data analysis for discovering structures in higher-order data sets in applications such as chemistry, neuroscience, and social network analysis. Kolda and Bader [35] provide an overview of higher-order tensor decompositions and their applications in psychometrics, chemometrics, signal processing, etc. These survey papers primarily focus on unsupervised tensor data analysis through factor decomposition. In addition, Zafeiriou [36] provides an overview of both unsupervised and supervised nonnegative tensor factorization (NTF) [37], [38] with NTF algorithms and their applications in visual representation and recognition discussed.
In contrast, this survey paper focuses on a systematic introduction to the field of MSL. To the best knowledge of the authors, this is the first unifying survey of both unsupervised and supervised MSL algorithms. For detailed introduction and review on multilinear algebra, multilinear decomposition, and NTF, the readers should go to [30], [31], [32], [33], [34], [35], [36], [39], [40] while this paper serves as a complement to these surveys. In the rest of this paper, Section 2 first introduces notations and basic multilinear algebra, and then addresses multilinear projections for direct mapping from high-dimensional tensorial representations to low-dimensional vectorial or tensorial representations. This section also provides insights to the relationships among different projections. Section 3 formulates a unifying MSL framework for systematic treatment of the MSL problem. A typical approach to solve MSL is presented and several algorithmic issues are examined in this section. Under the MSL framework, existing MSL algorithms are reviewed, analyzed and categorized into taxonomies in Section 4. Finally, MSL applications are summarized in Section 5 and future research topics are covered in Section 6. For easy reference, Table 1 lists several important acronyms used in this paper.
Section snippets
Fundamentals and multilinear projections
This section first reviews the notations and some basic multilinear operations [30], [31], [41] that are necessary in defining the MSL problem. The important concepts of multilinear projections are then introduced, including elementary multilinear projection (EMP), tensor-to-vector projection (TVP), and tensor-to-tensor projection (TTP), and their relationships are explored. Table 2 summarizes the important symbols used in this paper for quick reference.
The multilinear subspace learning framework
This section formulates a general MSL framework. It defines the MSL problem in a similar way as LSL, as well as tensor and scalar scatter measures for optimality criterion construction. It also outlines a typical solution and discusses related issues.
Taxonomy of multilinear subspace learning algorithms
This section reviews several important MSL algorithms under the MSL framework. Due to the fundamentality and importance of PCA and LDA, the focus is on the multilinear extensions of these two classical linear algorithms. Fig. 8, Fig. 9 depict taxonomies for these algorithms, one for multilinear extensions of PCA and the other for multilinear extensions of LDA, respectively, which will be discussed in detail in the following. The historical basis of MSL will also be described.
Multilinear subspace learning applications
Due to the advances in sensor and storage technology, MSL is becoming increasingly popular in a wide range of application domains involving tensor-structured data sets. This section will summarize several applications of MSL algorithms in real-world applications, including face recognition and gait recognition in biometrics, music genre classification in audio signal processing, EEG signal classification in biomedical engineering, anomaly detection in data mining, and visual content analysis in
Conclusions and future works
This paper presents a survey of an emerging dimensionality reduction approach for direct feature extraction from tensor data: multilinear subspace learning. It reduces the dimensionality of massive data directly from their natural multidimensional representation: tensors. This survey covers multilinear projections, MSL framework, typical MSL solutions, MSL taxonomies, and MSL applications. MSL is a new field with many open issues to be examined further. The rest of this section outlines several
Acknowledgment
The authors would like to thank the anonymous reviewers for their insightful comments, which have helped to improve the quality of this paper.
Haiping Lu received the B.Eng. and M.Eng. degrees in electrical and electronic engineering from Nanyang Technological University, Singapore, in 2001 and 2004, respectively, and the Ph.D. degree in electrical and computer engineering from University of Toronto, Toronto, ON, Canada, in 2008. Currently, he is a research fellow in the Institute for Infocomm Research, Agency for Science, Technology and Research, Singapore. His current research interests include pattern recognition, machine learning,
References (113)
- et al.
Tensor rank one discriminant analysis a convergent method for discriminative multilinear subspace selection
Neurocomputing
(2008) - et al.
Survey on tensor signal algebraic filtering
Signal Processing
(2007) Independent component analysis a new concept?
Signal Processing
(1994)- et al.
A theorem on the uncorrelated optimal discriminant vectors
Pattern Recognition
(2001) - et al.
Multilinear principal component analysis for face recognition with fewer features
Neurocomputing
(2010) - et al.
Incremental tensor biased discriminant analysis: a new color-based visual tracking method
Neurocomputing
(2010) - J. Ye, R. Janardan, Q. Li, GPCA: an efficient dimension reduction scheme for image compression and retrieval, in: The...
- et al.
Multilinear discriminant analysis for face recognition
IEEE Transactions on Image Processing
(2007) - et al.
Face recognition using kernel direct discriminant analysis algorithms
IEEE Transactions on Neural Networks
(2003) - et al.
A review on face and gait recognition: system, data and algorithms
A prior neurophysiologic knowledge free tensor-based scheme for single trial eeg classification
IEEE Transactions on Neural Systems and Rehabilitation Engineering
Using uncorrelated discriminant analysis for tissue classification with gene expression data
IEEE/ACM Transactions on Computational Biology Bioinformatics
A neural-network appearance-based 3-D object recognition using independent component analysis
IEEE Transactions on Neural Networks
Dimensionality reduction based on tensor modeling for classification methods
IEEE Transactions on Geoscience and Remote Sensing
Recognition of Humans and their Activities using Video
Quantifying and recognizing human movement patterns from monocular video images-part II: applications to biometrics
IEEE Transactions on Circuits and Systems for Video Technology
Decomposing the tensor kernel support vector machine for neuroscience data with structure labels
Machine Learning
Space-time super-resolution
IEEE Transactions on Pattern Analysis and Machine Intelligence
Less is more: sparse graph mining with compact matrix decomposition
Statistical Analysis and Data Mining
Incremental tensor analysis: theory and applications
ACM Transactions on Knowledge Discovery from Data
Face recognition in subspaces
Principal Component Analysis
Introduction
MPCA: multilinear principal component analysis of tensor objects
IEEE Transactions on Neural Networks
Supervised tensor learning
Knowledge and Information Systems
General tensor discriminant analysis and gabor features for gait recognition
IEEE Transactions on Pattern Analysis and Machine Intelligence
Uncorrelated multilinear principal component analysis for unsupervised multilinear subspace learning
IEEE Transactions on Neural Networks
Uncorrelated multilinear discriminant analysis with regularization and aggregation for tensor object recognition
IEEE Transactions on Neural Networks
Reconstruction and recognition of tensor-based objects with concurrent subspaces analysis
IEEE Transactions on Circuits and Systems for Video Technology
A taxonomy of emerging multilinear discriminant analysis solutions for biometric signal recognition
Multilinear Algebra
A multilinear singular value decomposition
SIAM Journal of Matrix Analysis and Applications
On the best rank-1 and rank-(R1,R2,…,RN) approximation of higher-order tensors
SIAM Journal of Matrix Analysis and Applications
Numerical multilinear algebra and its applications
Frontiers of Mathematics in China
Unsupervised multiway data analysis: a literature survey
IEEE Transactions on Knowledge and Data Engineering
Tensor decompositions and applications
SIAM Review
Algorithms for nonnegative tensor factorization
Multi-way Analysis
Algorithm 862: Matlab tensor classes for fast algorithm prototyping
ACM Transactions on Mathematical Software
Pattern Classification
Mathematical Methods and Algorithms for Signal Processing
Cited by (328)
Crop classification based on multi-temporal PolSAR images with a single tensor network
2023, Pattern RecognitionTensor completion via convolutional sparse coding with small samples-based training
2023, Pattern RecognitionDiscriminative subspace learning via optimization on Riemannian manifold
2023, Pattern RecognitionFacial age estimation using tensor based subspace learning and deep random forests
2022, Information SciencesSparse low-rank approximation of matrix and local preservation for unsupervised image feature selection
2023, Applied Intelligence
Haiping Lu received the B.Eng. and M.Eng. degrees in electrical and electronic engineering from Nanyang Technological University, Singapore, in 2001 and 2004, respectively, and the Ph.D. degree in electrical and computer engineering from University of Toronto, Toronto, ON, Canada, in 2008. Currently, he is a research fellow in the Institute for Infocomm Research, Agency for Science, Technology and Research, Singapore. His current research interests include pattern recognition, machine learning, biometrics, and biomedical engineering.
Konstantinos N. Plataniotis is a Professor with the Department of Electrical and Computer Engineering and the Director of the Knowledge Media Design Institute at the University of Toronto. He received his B.Eng. degree in Computer Engineering from University of Patras, Greece in 1988 and his M.S. and Ph.D. degrees in Electrical Engineering from Florida Institute of Technology in 1992 and 1994, respectively. His research interests include multimedia systems, biometrics, image and signal processing, communications systems and pattern recognition. He is a registered professional engineer in Ontario, and the Editor-in-Chief (2009–2011) for the IEEE Signal Processing Letters.
Anastasios N. Venetsanopoulos is a Professor of Electrical and Computer Engineering at Ryerson University, Toronto, and a Professor Emeritus with the Department of Electrical and Computer Engineering at the University of Toronto. He received the B.Eng. degree in electrical and mechanical engineering from the National Technical University of Athens, Greece in 1965, and the M.S., M.Phil., and Ph.D. degrees in Electrical Engineering from Yale University in 1966, 1968 and 1969, respectively. His research interests include multimedia, digital signal/image processing, telecommunications, and biometrics. He is a Fellow of the Engineering Institute of Canada, the IEEE, the Canadian Academy of Engineering, and the Royal Society Of Canada.