Semi-supervised hyperspectral classification from a small number of training samples using a co-training approach
Introduction
Rising interest in hyperspectral remote sensing has resulted in rapid development of methods for annotation, classification and segmentation of hyperspectral images (Bioucas-Dias et al., 2013). Many successful approaches (Li et al., 2013, Tilton et al., 2012, Wang et al., 2014) are based on semi-supervised learning and employ both spatial and spectral features of data. They aim to extend the initial training set with unlabelled samples in order to improve the classifier performance. These methods typically achieve better results than supervised approaches based only on spectral classifiers, as shown e.g. in Camps-Valls et al. (2007).
The paradigm of semi-supervised learning (Zhu and Goldberg, 2009) has been effectively applied beyond the field of hyperspectral imaging. One of the areas is long-term tracking (LTT) of unknown objects in a video stream. The efficient self-learning approach, based on co-training (Blum and Mitchell, 1998), was introduced in Kalal et al. (2012) in the form of the Tracking Learning Detection (TLD) framework.
The TLD approach decomposes the tracking problem into three sub-tasks: tracking, learning and detection. These sub-tasks were addressed by three simultaneously working components. The tracker was responsible for following the object position from frame to frame. The detector localised templates based on previously seen appearances. The learning component was, among others, responsible for estimation of detector errors and its update in order to avoid these errors in the future. The TLD used the P-N learning paradigm with two types of ‘experts’ (also called learners) evaluating the performance of the detector. P-expert was focused on missed detections while N-expert was focused on false positives. The analysis of stability performed in Kalal et al. (2012) has shown that well-designed experts can form a self-stabilising algorithm with the final error at an acceptable level.
We can find similarities between the tasks of hyperspectral classification (HC) and LTT. First, both problems have independent, multi-dimensional feature spaces. For the LTT the first feature space corresponds to visual similarity of objects, while the second one corresponds to their temporal behaviour, i.e. the position of the tracking window in consecutive frames. In HC the pixel similarity can be formulated as spatial (i.e. the distance between positions in the neighbourhood of each other) or spectral (i.e. the similarity of mixtures of materials). The second similarity lies in the fact that both LTT and HC include the detection component – LTT performs pattern matching for image fragments while HC performs spectral matching for individual image pixels. The third similarity is the assumption of predictability in the spatial dimension. LTT assumes that objects are moving in a predictable (e.g. locally linear) trajectory and their position can be estimated from previous frames using e.g. the Kalman filter (Harvey, 1990). Analogically, if we define an HC problem in terms of extending a small set of initial labels in a way similar to label propagation or region growing, the knowledge about class labels of pixels from the training set can be extended to their neighbourhood. It has been observed, e.g. in Dópido et al., 2013, Tan et al., 2015, that the class label is highly correlated with spatial similarity of pixels. In other words, we can expect that pixels located close to one another are likely to have the same label.
Based on those observations we implement the hyperspectral classification in a co-training framework with the P-N learning approach. Our P-expert assumes the same class labels for spatially close pixels. The N-expert detects pixels with similar spectra. Newly classified pixels are used to retrain the spectral classifier and to improve the spatial similarity model. Both experts are independent and while their limited scope of data makes them prone to errors, we show that their local accuracy is enough to stabilize the learning process. The schematics of the method are presented in Fig. 1.
The first contribution of this paper is the formulation of the hyperspectral classification problem in terms of the P-N learning paradigm. The second contribution is the implementation of the hyperspectral classifier based on proposed spatial and spectral experts. We present results on five data sets: the Indian Pines and Salinas Valley, University of Pavia, La Selva Biological Station and Madonna, Villelongue, France.
The rest of the paper is organised as follows: Section 2 presents the related work. Section 3 explains our method while Section 4 describes the experiment on real data and its results. Conclusions are provided in Section 5.
Section snippets
Related work
The classification of images acquired from an overhead perspective is a key component in information extraction for remote sensing applications, e.g. providing the details about land use, vegetation health or mineral concentrations (Campbell and Wynne, 2011). The complexity of this task results from diversity of objects and structures that are to be identified in images; in one case (Cheng et al., 2014) the requirement can be to detect instances from a broad set of objects that are only
Method
In this section we present the proposed co-training algorithm, inspired by the Tracking-Learning-Detection (TLD) (Kalal et al., 2012) approach. The TLD consists of three components. The first two are: the tracker that estimates the object motion between consecutive frames and the detector that treats every frame as independent, localising all appearances of the object in the image. The third component is a model that observes the performance of the tracker and the detector, identifies detection
Experiments
To evaluate the proposed algorithm, we perform several experiments with a number of publicly available hyperspectral datasets. The objective of those experiments is to evaluate its quantitative and qualitative performance in different conditions of: varying scene composition (e.g. agricultural, urban and forest areas), spatial layout (e.g. class area regularity, convexity, class count imbalances), spectral features (e.g. diversity, type of mixing model) and resolution (pixel size).
Conclusions
Our results show that the proposed approach based on co-training is very effective in semi-supervised hyperspectral classification. Our algorithm successfully employs the idea presented in the TLD framework. Classification results are comparable to or outperforming the state of art methods. The method is also robust in regards to parameter selection. The algorithm can be extended with a more complex spectral classifier e.g. SVM with dedicated kernels (Li et al., 2013) or an ensemble classifier (
Acknowledgements
This work has been partially supported by the project ‘Representation of dynamic 3D scenes using the Atomic Shapes Network model’ financed by the Polish National Science Centre, decision DEC-2011/03/D/ST6/03753. The authors would like to thank Prof. Matthew L. Clark and Prof. Nicolas Dobigeon for making available datasets La Selva Biological Station and Madonna respectively. Authors would also like to thank anonymous reviewers for their insightful comments and their help in improving this
References (44)
- et al.
Multi-class geospatial object detection and geographic image classification based on collection of part detectors
ISPRS J. Photogr. Remote Sensing
(2014) - et al.
Hyperspectral discrimination of tropical rain forest tree species at leaf to crown scales
Remote Sensing Environ.
(2005) - et al.
Image segmentation techniques
Comput. Vision Graphics Image Process.
(1985) - et al.
Recent advances in techniques for hyperspectral image processing
Remote Sensing Environ.
(2009) - et al.
A novel semi-supervised hyperspectral image classification approach based on spatial neighborhood information and classifier combination
ISPRS J. Photogr. Remote Sensing
(2015) - et al.
An efficient semi-supervised classification approach for hyperspectral imagery
ISPRS J. Photogr. Remote Sensing
(2014) - et al.
Semi-supervised classification for hyperspectral imagery based on spatial-spectral label propagation
ISPRS J. Photogr. Remote Sensing
(2014) - et al.
Manifold regularization: a geometric framework for learning from labeled and unlabeled examples
J. Machine Learning Res.
(2006) - et al.
Weighted k-nearest neighbor density estimates
A variable splitting augmented Lagrangian approach to linear spectral unmixing
Hyperspectral remote sensing data analysis and future challenges
IEEE Geosci. Remote Sensing Mag.
Combining labeled and unlabeled data with co-training
A novel transductive SVM for semisupervised classification of remote-sensing images
IEEE Trans. Geosci. Remote Sensing
Introduction to Remote Sensing
Semi-supervised graph-based hyperspectral image classification
IEEE Trans. Geosci. Remote Sensing
Spectral-spatial classification based on affinity scoring for hyperspectral imagery
IEEE J. Select. Topics Appl. Earth Observ. Remote Sensing
Effective and efficient midlevel visual elements-oriented land-use classification using vhr remote sensing images
IEEE Trans. Geosci. Remote Sensing
Semisupervised hyperspectral image classification via discriminant analysis and robust regression
IEEE J. Select. Topics Appl. Earth Observ. Remote Sensing
Two stage SVM classification for hyperspectral data
Kernel low-rank and sparse graph for unsupervised and semi-supervised classification of hyperspectral images
IEEE Trans. Geosci. Remote Sensing
Nonlinear unmixing of hyperspectral images: models and algorithms
IEEE Signal Process. Mag.
Semisupervised self-learning for hyperspectral image classification
IEEE Trans. Geosci. Remote Sensing
Cited by (52)
Multi-view learning for hyperspectral image classification: An overview
2022, NeurocomputingCitation Excerpt :And combined co-training and active learning to bring together the advantages of both to classify HSIs. Inspired by the Tracking-Learning-Detection (TLD) approach, a new semi-supervised co-training method was proposed by Romaszewski et al. [97]. Similarly to TLD, the HSI classification task is specified as a co-training problem so that it can be solved by a PN learning scheme.
Enhancing Hyperspectral Image Classification: Leveraging Unsupervised Information with Guided Group Contrastive Learning
2024, IEEE Transactions on Geoscience and Remote SensingPotential analysis and prospect of hyperspectral ground object recognition
2023, Cehui Xuebao/Acta Geodaetica et Cartographica Sinica