Hyperspectral image classification using an extended Auto-Encoder method

https://doi.org/10.1016/j.image.2020.116111Get rights and content

Highlights

  • A modified Auto-Encoder based on MM technique is proposed for Hyper-Spectral Image classification.

  • Three main modifications are made. First, SAM is used as a regularization term to construct the weights of the Auto-Encoder.

  • Second, Fuzzy weighting is used to fine-tune the parameters.

  • Third, multi-scale features (MSF) are used to improve the performance of the Auto-encoder resulting in the proposed method - MSF-EAEMM.

  • MSF-EAEMM achieves high accuracy and reduces the time complexity using lower orientation and scales of Gabor filter.

Abstract

This article proposes a spectral–spatial method for classification of hyperspectral images (HSIs) by modifying traditional Auto-Encoder based on Majorization Minimization (MM) technique. The proposed method consists of suggesting three main modifications. First, to construct weights of Auto-Encoder, similarity angle map(SAM) criterion is used as regularization term. It is useful to extract spectral similarity of initial features. Second, to enhance the classification accuracy, fuzzy mode is used to estimate parameters. These modifications lead to create an extended Auto-Encoder based on MM (EAEMM). Third, to improve the performance of Auto-Encoder, multi-scale features (MSF) are extracted. In comparison with some of the state-of-the-art methods, the experimental results obtained using the proposed method (MSF-EAEMM) show that it significantly improves the classification accuracy of HSI classification.

Introduction

Nowadays, hyperspectral image (HSI) classification is used in many applications such as agriculture, medical science, etc. However, themes such as the limited training sampling and the presence of mixed pixels and noise in the data pose various challenges for HSI classification. To classify HSIs, diverse algorithms [1], [2], [3], [4], [5], [6] have been recently proposed. Auto-Encoder (AE) based techniques are very effective in classifying images.

Some of the AE methods for HSI classification are: deep network model based on Gabor filtering (GFDN) [1], Stacked Auto-Encoder (SAE) [7] and Sparse Auto-Encoder (SSAE) [8]. The main problem of AE for HSI classification is that the information based on pyramid filters has been neglected, and spectral angle similarity and spectral distortion have not been considered. In this research, an extended Auto-Encoder based on Majorization Minimization (MM) method using multi-scale features (MSF-EAEMM) is proposed.

The specific contributions of this proposed method are the following:

  • A model is designed based on multi-scale features. The main reason is that the weights of Auto-Encoder are constructed based on them. In fact, it is useful to solve the problem of extracting initial spatial information in Auto-Encoder models.

  • An Auto-Encoder based on SAM criterion is proposed. Using SAM as regularization term to construct Auto-Encoder weights is important to extract spectral similarity in Auto-Encoder models. It increases the performance of HSI classification.

  • Using fuzzy weighting is important to correct the values of parameters. It helps to improve the classification accuracy.

  • In the proposed model, lower orientation and scales of Gabor filter are selected based on the distribution of data and kurtosis criterion. This makes the proposed model efficient and reduces the time complexity.

Finally, a comparative study was also conducted with traditional Auto-Encoder techniques [1], [9] and other spectral–spatial methods [2], [3]. The main ideas of the proposed work include the following two aspects: First, considering the full benefits of the structural information of HSI; second, creating a network model based on SAM criterion and fuzzy mode.

The remainder of the paper is organized as follows. In Section 2, a comprehensive overview of the related works is presented. In Section 3, the background of related methods is provided. In Section 4, the proposed method is presented. Experimental results and discussion are presented in Section 5. Section 6 concludes the paper summarizing the contributions and achievements.

Section snippets

Related works

HSI classification has been a very popular research area in recent years. In this section the existing Auto-Encoder based methods are discussed.

Autoencoders (AE) contain two networks and effort to learn high-level features. The first network transmits the input (training data) to the feature space. While the second one maps the feature space to the output (training data). Different objective functions and their solutions can create various Auto-Encoders. A case in point is a semi-supervised

Background of Nonsubsampled Pyramid (NSP)

NSP filters [18] create a multiscale decomposition of the original image into low-frequency subbands (APˆ)and high-frequency subbands (BPˆ,Pˆ=1,,J) containing the same size of the original image. J represents the number of levels.

Fig. 1 indicates a three-scale (J = 3) decomposition for ZRa×b×n1. a×b is the number of pixels and n1 is the number of bands. As such, AJ=Ψ1(Z) and Ψ1 depict lowpass filter of the NSP transformation.

The multiscale property is created from a shift-invariant

Proposed extended Auto-Encoder (MSF-EAEMM)

A Multi-Scale Feature-Extended Auto-Encoder based on Majorization Minimization (MSF-EAEMM) is proposed in this paper. The proposed method (MSF-EAEMM) considers spectral and spatial information simultaneously. A block diagram of the proposed method is shown in Fig. 3.

First of all, initial stacked features are provided by applying 2-D low-pass filters of NSP (Apˆ), Gabor filters (Gu,v) and spectral features (I). Using low-pass images by pyramid filters of NSP are significant. Since pyramid

Data description

The hyperspectral datasets are two regions taken by airborne visible infrared imaging spectrometer (AVIRIS) sensor and reflective optics system imaging spectrometer (ROSIS-3) hyperspectral sensor (see Fig. 4).

Indian Pines dataset: It consists of 224 spectral bands in the wavelength range 0.42.5 μm and 145 × 145 pixels. This dataset contains spatial resolution of 20 m per pixel and is of 16 classes.

Pavia University: It employs 103 spectral bands in a range from 0.43 to 0.86 μm and 610 × 340

Conclusion

In this article, a spectral–spatial classification method based on Auto-Encoder classifier using MM was introduced. For Auto-Encoder classifier, weights based on SAM criterion, fuzzy mode and multi-scale features were suggested. These weights were valuable to consider the impact of the spectral and spatial information simultaneously. Intrinsic properties of HSI data such as low frequency features and local features are simultaneously used to improve the performance of the network. Experimental

CRediT authorship contribution statement

Elham Kordi Ghasrodashti: Conceptualization, Methodology, Software, Investigation, Writing - review & editing. Nabin Sharma: Investigation, Writing - review & editing.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

The authors would like to thank Professor P. Gamba from the University of Pavia, Pavia, Italy, for providing the Pavia dataset and the referee committee members of Signal Processing: Image Communication Journal for their constructive, pertinent, and invaluable comments/suggestions.

References (24)

  • XuY. et al.

    Hyperspectral image classication via a random patches network

    ISPRS J. Photogramm. Remote Sens.

    (2018)
  • KruseF.A. et al.

    The spectral image processing system(sips)—interactive visualization and analysis of imaging spectrometer data

    Remote Sens. Environ.

    (1993)
  • HaghighatM. et al.

    Cloudid: Trustworthy cloud-based and cross-enterprise biometric identification

    Expert Syst. Appl.

    (2015)
  • LiC. et al.

    Gabor filtering based deep network for hyperspectral image classification

    IGARSS

    (2017)
  • SunL. et al.

    Supervised spectral–spatial hyperspectral image classification with weighted markov random fields

    IEEE Trans. Geosci. Remote Sens.

    (2015)
  • SunL. et al.

    Adjacent superpixel-based multiscale spatial-spectral kernel for hyperspectral classification

    IEEEJSTARS

    (2019)
  • SunL. et al.

    Low rank component induced spatial-spectral kernel method for hyperspectral image classification

    IEEE TCSVT

    (2020)
  • SunL. et al.

    Weighted nonlocal low-rank tensor decomposition method for sparse unmixing of hyperspectral images

    IEEE JSTARS

    (2020)
  • ChenY. et al.

    Spectral–spatial classification of hyperspectral data based on deep belief network

    IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.

    (2014)
  • Hoo-ChangS. et al.

    Stacked autoencoders for unsupervised feature learning and multiple organ detection in a pilot study using 4d patient data

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2013)
  • J. Mehta, Robust Autoencoders (Master of Technology thesis), (2016) pp....
  • MajumdarA. et al.

    Semi-supervised stacked label consistent autoencoder for reconstruction and analysis of biomedical signals

    IEEE Trans. Biomed. Eng.

    (2017)
  • Cited by (23)

    • A deep autoencoder approach for detection of brain tumor images

      2022, Computers and Electrical Engineering
      Citation Excerpt :

      They achieve 96% accuracy exhausting 68% of images out of total images. Ghasrodashti et al. [22] proposed a spectral-spatial technique for the taxonomy of HSIs. They developed an outmoded autoencoder with the help of the majorization minimization method.

    • Morphologically dilated convolutional neural network for hyperspectral image classification

      2022, Signal Processing: Image Communication
      Citation Excerpt :

      To resolve such issues, deep learning models provide an effective option since it can pull out hierarchical features, which are invariant and robust [25]. Initially, the deep learning models proposed for HSI classification were stacked autoencoder (SAE) [26,27] and deep belief network (DBN) [28]. In terms of feature extraction power and classification precision, these models were able to extract more robust features and outperform other conventional machine learning approaches.

    • Ramp sparse support matrix machine and its application in roller bearing fault diagnosis[Formula presented]

      2021, Applied Soft Computing
      Citation Excerpt :

      Considering the advantages of the matrix as the modeling element, many models based on matrix classification have been developed in recent years. For example, Extended auto-encoder based on majorization minimization and multi-scale features (MSF-EAEMM) is a kind of hyperspectral image classification method, which uses fuzzy mode for parameter estimation and extracts multi-scale features of images to improve image classification accuracy [11]. To solve the classification problem of occluded images, Feng et al. put forward a convolutional neural network (CNN) based on the feature of depth enhancement, which uses a set of enhanced depth feature vectors (DFVS) to improve the classification accuracy of occlusion images [12].

    View all citing articles on Scopus
    View full text