Semi-supervised hyperspectral classification from a small number of training samples using a co-training approach

https://doi.org/10.1016/j.isprsjprs.2016.08.011Get rights and content

Abstract

We present a novel semi-supervised algorithm for classification of hyperspectral data from remote sensors. Our method is inspired by the Tracking-Learning-Detection (TLD) framework, originally applied for tracking objects in a video stream. TLD introduced the co-training approach called P-N learning, making use of two independent ‘experts’ (or learners) that scored samples in different feature spaces. In a similar fashion, we formulated the hyperspectral classification task as a co-training problem, that can be solved with the P-N learning scheme. Our method uses both spatial and spectral features of data, extending a small set of initial labelled samples during the process of region growing. We show that this approach is stable and achieves very good accuracy even for small training sets. We analyse the algorithm’s performance on several publicly available hyperspectral data sets.

Introduction

Rising interest in hyperspectral remote sensing has resulted in rapid development of methods for annotation, classification and segmentation of hyperspectral images (Bioucas-Dias et al., 2013). Many successful approaches (Li et al., 2013, Tilton et al., 2012, Wang et al., 2014) are based on semi-supervised learning and employ both spatial and spectral features of data. They aim to extend the initial training set with unlabelled samples in order to improve the classifier performance. These methods typically achieve better results than supervised approaches based only on spectral classifiers, as shown e.g. in Camps-Valls et al. (2007).

The paradigm of semi-supervised learning (Zhu and Goldberg, 2009) has been effectively applied beyond the field of hyperspectral imaging. One of the areas is long-term tracking (LTT) of unknown objects in a video stream. The efficient self-learning approach, based on co-training (Blum and Mitchell, 1998), was introduced in Kalal et al. (2012) in the form of the Tracking Learning Detection (TLD) framework.

The TLD approach decomposes the tracking problem into three sub-tasks: tracking, learning and detection. These sub-tasks were addressed by three simultaneously working components. The tracker was responsible for following the object position from frame to frame. The detector localised templates based on previously seen appearances. The learning component was, among others, responsible for estimation of detector errors and its update in order to avoid these errors in the future. The TLD used the P-N learning paradigm with two types of ‘experts’ (also called learners) evaluating the performance of the detector. P-expert was focused on missed detections while N-expert was focused on false positives. The analysis of stability performed in Kalal et al. (2012) has shown that well-designed experts can form a self-stabilising algorithm with the final error at an acceptable level.

We can find similarities between the tasks of hyperspectral classification (HC) and LTT. First, both problems have independent, multi-dimensional feature spaces. For the LTT the first feature space corresponds to visual similarity of objects, while the second one corresponds to their temporal behaviour, i.e. the position of the tracking window in consecutive frames. In HC the pixel similarity can be formulated as spatial (i.e. the distance between positions in the neighbourhood of each other) or spectral (i.e. the similarity of mixtures of materials). The second similarity lies in the fact that both LTT and HC include the detection component – LTT performs pattern matching for image fragments while HC performs spectral matching for individual image pixels. The third similarity is the assumption of predictability in the spatial dimension. LTT assumes that objects are moving in a predictable (e.g. locally linear) trajectory and their position can be estimated from previous frames using e.g. the Kalman filter (Harvey, 1990). Analogically, if we define an HC problem in terms of extending a small set of initial labels in a way similar to label propagation or region growing, the knowledge about class labels of pixels from the training set can be extended to their neighbourhood. It has been observed, e.g. in Dópido et al., 2013, Tan et al., 2015, that the class label is highly correlated with spatial similarity of pixels. In other words, we can expect that pixels located close to one another are likely to have the same label.

Based on those observations we implement the hyperspectral classification in a co-training framework with the P-N learning approach. Our P-expert assumes the same class labels for spatially close pixels. The N-expert detects pixels with similar spectra. Newly classified pixels are used to retrain the spectral classifier and to improve the spatial similarity model. Both experts are independent and while their limited scope of data makes them prone to errors, we show that their local accuracy is enough to stabilize the learning process. The schematics of the method are presented in Fig. 1.

The first contribution of this paper is the formulation of the hyperspectral classification problem in terms of the P-N learning paradigm. The second contribution is the implementation of the hyperspectral classifier based on proposed spatial and spectral experts. We present results on five data sets: the Indian Pines and Salinas Valley, University of Pavia, La Selva Biological Station and Madonna, Villelongue, France.

The rest of the paper is organised as follows: Section 2 presents the related work. Section 3 explains our method while Section 4 describes the experiment on real data and its results. Conclusions are provided in Section 5.

Section snippets

Related work

The classification of images acquired from an overhead perspective is a key component in information extraction for remote sensing applications, e.g. providing the details about land use, vegetation health or mineral concentrations (Campbell and Wynne, 2011). The complexity of this task results from diversity of objects and structures that are to be identified in images; in one case (Cheng et al., 2014) the requirement can be to detect instances from a broad set of objects that are only

Method

In this section we present the proposed co-training algorithm, inspired by the Tracking-Learning-Detection (TLD) (Kalal et al., 2012) approach. The TLD consists of three components. The first two are: the tracker that estimates the object motion between consecutive frames and the detector that treats every frame as independent, localising all appearances of the object in the image. The third component is a model that observes the performance of the tracker and the detector, identifies detection

Experiments

To evaluate the proposed algorithm, we perform several experiments with a number of publicly available hyperspectral datasets. The objective of those experiments is to evaluate its quantitative and qualitative performance in different conditions of: varying scene composition (e.g. agricultural, urban and forest areas), spatial layout (e.g. class area regularity, convexity, class count imbalances), spectral features (e.g. diversity, type of mixing model) and resolution (pixel size).

Conclusions

Our results show that the proposed approach based on co-training is very effective in semi-supervised hyperspectral classification. Our algorithm successfully employs the idea presented in the TLD framework. Classification results are comparable to or outperforming the state of art methods. The method is also robust in regards to parameter selection. The algorithm can be extended with a more complex spectral classifier e.g. SVM with dedicated kernels (Li et al., 2013) or an ensemble classifier (

Acknowledgements

This work has been partially supported by the project ‘Representation of dynamic 3D scenes using the Atomic Shapes Network model’ financed by the Polish National Science Centre, decision DEC-2011/03/D/ST6/03753. The authors would like to thank Prof. Matthew L. Clark and Prof. Nicolas Dobigeon for making available datasets La Selva Biological Station and Madonna respectively. Authors would also like to thank anonymous reviewers for their insightful comments and their help in improving this

References (44)

  • J.M. Bioucas-Dias et al.

    Hyperspectral remote sensing data analysis and future challenges

    IEEE Geosci. Remote Sensing Mag.

    (2013)
  • A. Blum et al.

    Combining labeled and unlabeled data with co-training

  • L. Bruzzone et al.

    A novel transductive SVM for semisupervised classification of remote-sensing images

    IEEE Trans. Geosci. Remote Sensing

    (2006)
  • J.B. Campbell et al.

    Introduction to Remote Sensing

    (2011)
  • G. Camps-Valls et al.

    Semi-supervised graph-based hyperspectral image classification

    IEEE Trans. Geosci. Remote Sensing

    (2007)
  • Z. Chen et al.

    Spectral-spatial classification based on affinity scoring for hyperspectral imagery

    IEEE J. Select. Topics Appl. Earth Observ. Remote Sensing

    (2016)
  • G. Cheng et al.

    Effective and efficient midlevel visual elements-oriented land-use classification using vhr remote sensing images

    IEEE Trans. Geosci. Remote Sensing

    (2015)
  • G. Cheng et al.

    Semisupervised hyperspectral image classification via discriminant analysis and robust regression

    IEEE J. Select. Topics Appl. Earth Observ. Remote Sensing

    (2016)
  • M. Cholewa et al.

    Two stage SVM classification for hyperspectral data

  • F. de Morsier et al.

    Kernel low-rank and sparse graph for unsupervised and semi-supervised classification of hyperspectral images

    IEEE Trans. Geosci. Remote Sensing

    (2016)
  • N. Dobigeon et al.

    Nonlinear unmixing of hyperspectral images: models and algorithms

    IEEE Signal Process. Mag.

    (2014)
  • I. Dópido et al.

    Semisupervised self-learning for hyperspectral image classification

    IEEE Trans. Geosci. Remote Sensing

    (2013)
  • Cited by (52)

    • Multi-view learning for hyperspectral image classification: An overview

      2022, Neurocomputing
      Citation Excerpt :

      And combined co-training and active learning to bring together the advantages of both to classify HSIs. Inspired by the Tracking-Learning-Detection (TLD) approach, a new semi-supervised co-training method was proposed by Romaszewski et al. [97]. Similarly to TLD, the HSI classification task is specified as a co-training problem so that it can be solved by a PN learning scheme.

    • Potential analysis and prospect of hyperspectral ground object recognition

      2023, Cehui Xuebao/Acta Geodaetica et Cartographica Sinica
    View all citing articles on Scopus
    View full text