Elsevier

Neural Networks

Volume 163, June 2023, Pages 205-218
Neural Networks

Collaborative-guided spectral abundance learning with bilinear mixing model for hyperspectral subpixel target detection

https://doi.org/10.1016/j.neunet.2023.02.002Get rights and content

Abstract

Detecting subpixel targets is a considerably challenging issue in hyperspectral image processing and interpretation. Most of the existing hyperspectral subpixel target detection methods construct detectors based on the linear mixing model which regards a pixel as a linear combination of different spectral signatures. However, due to the multiple scattering, the linear mixing model cannot​ illustrate the multiple materials interactions that are nonlinear and widespread in real-world hyperspectral images, which could result in unsatisfactory performance in detecting subpixel targets. To alleviate this problem, this work presents a novel collaborative-guided spectral abundance learning model (denoted as CGSAL) for subpixel target detection based on the bilinear mixing model in hyperspectral images. The proposed CGSAL detects subpixel targets by learning a spectral abundance of the target signature in each pixel. In CGSAL, virtual endmembers and their abundance help to achieve good accuracy for modeling nonlinear scattering accounts for multiple materials interactions according to the bilinear mixing model. Besides, we impose a collaborative term to the spectral abundance learning model to emphasize the collaborative relationships between different endmembers, which contributes to accurate spectral abundance learning and further help to detect subpixel targets. Plentiful experiments and analyses are conducted on three real-world and one synthetic hyperspectral datasets to evaluate the effectiveness of the CGSAL in subpixel target detection. The experiment results demonstrate that the CGSAL achieves competitive performance in detecting subpixel targets and outperforms other state-of-the-art hyperspectral subpixel target detectors.

Introduction

Hyperspectral remote sensing system can acquire and record the radiance of the Earth’s surface objects or scenes within each pixel area in vast spectral wavelength bands, providing hyperspectral images (HSIs) containing a large number of contiguous spectral bands (Bioucas-Dias et al., 2013, Manolakis et al., 2003, Wu et al., 2018). These contiguous spectral bands present rich spectral information that can be used to distinguish various ground cover materials (Axelsson et al., 2016, Goetz, 2009, Landgrebe, 2002, Su et al., 2020, Xie et al., 2018). With this characteristic, hyperspectral images have been widely used in many fields, covering agriculture, military, and mineralogy (Manolakis and Shaw, 2002, Yue et al., 2021). In these fields, the detection of specific materials or objects of interest, such as precision agriculture, military and civilian target detection, and mineral exploration, attracts considerable attention due to real-world applications (Boubanga-Tombet et al., 2018, Eismann et al., 2009, Lu et al., 2020). Such detection tasks make hyperspectral target detection become an important topic in remote sensing image processing (Dong, Du et al., 2015, Nasrabadi, 2014, Zhu et al., 2019).

Target detection in hyperspectral imagery is to identify whether the target of interest exists for each pixel based on the known spectrum of the target (Bioucas-Dias et al., 2013, Du and Zhang, 2014, Xie et al., 2019). In general, the known spectrum is acquired from the spectral library or selected from the training sample, usually, one spectrum of the target signature is enough (Healey and Slater, 1999, Li et al., 2022, Manolakis and Shaw, 2002). Target detection in hyperspectral imagery is quite different from that in other optical remote sensing images. For example, target detection in high-resolution remote sensing images detects specific ground objects with the bounding box boxing out the targets, which is a box-level target detection task (Li et al., 2020, Xia et al., 2018). Conversely, target detection in hyperspectral imagery is more like a pixel-level detection task that labels every pixel as the background or target, which is essentially a binary classification problem (Du et al., 2016, Manolakis et al., 2013, Zhao et al., 2021). However, compared with a large number of background pixels, the number of target pixels only occupies a very small fraction, which does not allow the overall classification error for a good tool to measure the hyperspectral target detection performance. Consequently, most existing methods that perform target detection in hyperspectral imagery are formed according to the Neyman–Pearson criterion, maximizing the detection probability in the fixed false alarm rate (Chang and Du, 2004, Nasrabadi, 2014).

As early as the 1980s, exploiting hyperspectral images to detect specific targets of interest emerged. In the early stage, people construct target detectors based on simple spectrum matching where the pixel with higher spectral similarity to the prior target spectrum is more likely to be labeled as the target. Spectral angle mapper (SAM) (Kruse et al., 1993), Spectral information divergence (SID) (Chang, 1999), and spectral matched filter (SMF) (Nasrabadi, 2008) are the detectors of this category. After that, machine learning theory has been applied to target detection in hyperspectral images, and extensive robust hyperspectral target detectors based on machine learning theory emerged, such as kernel-based algorithms (Jiao and Chang, 2008, Kwon and Nasrabadi, 2004), representation-based target detectors (Bitar et al., 2019, Chen et al., 2011, Li et al., 2015, Zhang et al., 2015), and metric and manifold learning-based methods (Dong, Zhang et al., 2015, Zhang et al., 2013). In recent years, exploiting deep learning methods to solve the target detection problem in hyperspectral imagery is mainstream. These methods design deep networks to extract spectral and spatial features to distinguish the difference between targets and background, thus allowing for target pixel identification. For example, HTD-net design a multi-layer convolutional neural network to distinguish target-background and target–target pixel pairs (Zhang et al., 2020), while TSCNTD constructs an effective deep learning-based target detection method using two-stream convolutional networks (Zhu et al., 2020).

The aforementioned detection methods are mainly designed for full-pixel target detection in hyperspectral images, where target pixels are considered as pixels that contain single target spectral information. However, due to the limitation of spatial resolution and target size, targets in the image scene often appear as subpixel targets which are the mixed pixels containing the target of interest and other background materials. In this case, the detection performance of the above full-pixel target detection methods could be limited, and the process of detection turns out to be the subpixel target detection task in hyperspectral imagery. In the open literature, different methods have been investigated to solve the subpixel target detection problem in hyperspectral images. According to different background description models, structural background description methods and nonstructural background methods are investigated. Matched subspace detector (MSD) (Scharf & Friedlander, 1994), adaptive matched subspace detector (AMSD) (Manolakis et al., 2001), and orthogonal subspace projector (OSP) (Chang, 2005) are the category of structural background description methods, while adaptive coherence/cosine estimator (ACE) (Kraut & Scharf, 1999), hybrid unstructured detector (HUD) (Broadwater & Chellappa, 2007), and non-parametric adaptive matched filter (Matteoli et al., 2020) are the category of nonstructural background description detectors. Moreover, to achieve robust performance, more attempts have been made at hyperspectral subpixel target detection. For example, to improve the background reconstruction, a pixel reconstruction-based subpixel target detection model (Song et al., 2021) is presented, it tries to obtain accurate background endmembers and reconstruct the input hyperspectral image to detect subpixel targets. A double dictionary-based nonlinear representation method (Wang et al., 2022) constructs double dictionaries to consider the spatial characteristics of the target and background for subpixel detection. Two subpixel target detectors are presented based on a multiple-instance learning framework, which aims to learn a discriminative prime target signature by maximizing the posterior detection statistics of subpixel hyperspectral targets (Jiao et al., 2022). It should be noted that some sparse and collaborative-based detectors, such as the sparsity-based target detector (STD) (Chen et al., 2011) and the combined sparse and collaborative representation detector (CSCR) (Li et al., 2015), although they are designed for the full-pixel target detection, they still have a certain capability of detecting subpixel targets.

Despite great advances in subpixel target detection for hyperspectral images, most of the existing subpixel target detectors are constructed based on the linear mixing model (LMM), assuming that different endmembers interact with the incident radiation in a linear way at a subpixel level (Gao et al., 2022, Ling et al., 2022, Manolakis et al., 2001, Yang et al., 2018). However, due to the multiple scattering during hyperspectral imaging, the linear mixing model cannot illustrate the multiple materials interactions that are nonlinear and widespread in real-world hyperspectral images, which could result in unsatisfactory performance in real-world hyperspectral image subpixel target detection. To alleviate this problem, we develop a new collaborative-guided spectral abundance learning model (denoted as CGSAL) for subpixel target detection in hyperspectral images based on a bilinear mixing model (BMM) in this paper. BMM is capable of illustrating multiple materials interactions by modeling virtual endmembers and their abundance, which can help to achieve good accuracy for modeling nonlinear scattering accounts for multiple materials interactions and contribute to more accurate model learning. Moreover, to emphasize the contributions of different endmembers and the collaborative relationships between them, we impose a collaborative constraint term on the proposed model, which is beneficial to spectral abundance learning of the target of interest and allows for precise subpixel target detection. The main contributions of this study are summarized as follows:

  • (1)

    We present a novel collaborative-guided spectral abundance learning method with the bilinear mixing model for hyperspectral subpixel target detection. The proposed CGSAL detector can model multiple materials interactions with regard to the nonlinear scattering in the process of hyperspectral imaging, so as to reach good accuracy for spectral abundance learning, which can help detect subpixel targets accurately.

  • (2)

    We propose to add a collaborative term to the spectral abundance learning model to emphasize the collaborative relationships between different endmembers. Under the condition of the limited number of samples in the endmember dictionary, the collaborative term can demonstrate the contributions of different endmembers, thus allowing for precise spectral abundance learning of the target.

The remainder of this work is organized as follows. Section 2 presents the proposed CGSAL method for hyperspectral subpixel target detection, including the collaborative-guided spectral abundance learning model, and its numerical solution algorithm. In Section 3, Experiments and analyses conducted on three real-world hyperspectral image datasets are provided, including detection performance evaluation and comparison, and parameters analyses. Finally, the conclusion of this article is drawn in Section 4.

Section snippets

Proposed CGSAL method

In this section, the proposed CGSAL method for hyperspectral subpixel target detection is detailedly described. We first introduce the collaborative-guided spectral abundance learning approach, in which we learn the spectral abundance of the target of interest to detect subpixel targets based on the bilinear mixing model and we add a collaborative term to emphasize the collaborative relationships of endmembers for accurate spectral abundance learning. We then provide the numerical solution

Experimental results and analysis

To evaluate the subpixel target detection performance and test the robustness and effectiveness of the proposed CGSAL method, extensive experiments and analyses conducted on three real-world and one synthetic hyperspectral image datasets are provided in this section. First of all, we introduce the four datasets and provide the experiment setting details. We then present the detection performance comparison with respect to several valid hyperspectral target detectors and our proposed CGSAL

Conclusion

In this work, we propose a new subpixel target detector with a novel collaborative-guided spectral abundance learning model based on the bilinear mixing model for hyperspectral images. To model the multiple materials interactions, the proposed CGSAL presents the virtual endmember term for denoting nonlinear scattering in the processing of hyperspectral imaging. To further emphasize the collaborative relationships that contribute to illustrating the virtual endmember term between different

Declaration of Competing Interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Bo Du reports financial support was provided by Wuhan University.

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China under Grants 62225113, 61871299, and the Natural Science Foundation of Hubei Province under Grants 2018CFA050.

References (63)

  • WuC. et al.

    Hyperspectral anomalous change detection based on joint sparse representation

    ISPRS Journal of Photogrammetry and Remote Sensing

    (2018)
  • XieW. et al.

    Spectral constraint adversarial autoencoders approach to feature representation in hyperspectral anomaly detection

    Neural Networks

    (2019)
  • XieW. et al.

    Trainable spectral difference learning with spatial starting for hyperspectral image denoising

    Neural Networks

    (2018)
  • ZhaoX. et al.

    Hyperspectral target detection based on transform domain adaptive constrained energy minimization

    International Journal of Applied Earth Observation and Geoinformation

    (2021)
  • ZhongY. et al.

    Blind spectral unmixing based on sparse component analysis for hyperspectral remote sensing imagery

    ISPRS Journal of Photogrammetry and Remote Sensing

    (2016)
  • Bioucas-DiasJ.M. et al.

    Hyperspectral remote sensing data analysis and future challenges

    IEEE Geoscience and Remote Sensing Magazine

    (2013)
  • BitarA.W. et al.

    Sparse and low-rank matrix decomposition for automatic target detection in hyperspectral imagery

    IEEE Transactions on Geoscience and Remote Sensing

    (2019)
  • Boubanga-TombetS. et al.

    Thermal infrared hyperspectral imaging for mineralogy mapping of a mine face

    Remote Sensing

    (2018)
  • BoydS. et al.

    Distributed optimization and statistical learning via the alternating direction method of multipliers

    Foundations and Trends ®in Machine Learning

    (2011)
  • BroadwaterJ. et al.

    Hybrid detectors for subpixel targets

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    (2007)
  • ChangC.-I.

    Spectral information divergence for hyperspectral image analysis

    IEEE International Geoscience and Remote Sensing Symposium

    (1999)
  • ChangC.-I.

    Orthogonal subspace projection (osp) revisited: A comprehensive study and analysis

    IEEE Transactions on Geoscience and Remote Sensing

    (2005)
  • ChangC.-I. et al.

    Estimation of number of spectrally distinct signal sources in hyperspectral imagery

    IEEE Transactions on Geoscience and Remote Sensing

    (2004)
  • ChenY. et al.

    Sparse representation for target detection in hyperspectral imagery

    IEEE Journal of Selected Topics in Signal Processing

    (2011)
  • DongY. et al.

    Target detection based on random forest metric learning

    IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing

    (2015)
  • DuB. et al.

    Beyond the sparsity-based target detector: A hybrid sparsity and statistics-based detector for hyperspectral images

    IEEE Transactions on Image Processing

    (2016)
  • EismannM.T. et al.

    Automated hyperspectral cueing for civilian search and rescue

    Proceedings of the IEEE

    (2009)
  • FanW. et al.

    Comparative study between a new nonlinear model and common linear model for analysing laboratory simulated-forest hyperspectral data

    International Journal of Remote Sensing

    (2009)
  • GaoL. et al.

    Using low-rank representation of abundance maps and nonnegative tensor factorization for hyperspectral nonlinear unmixing

    IEEE Transactions on Geoscience and Remote Sensing

    (2022)
  • HealeyG. et al.

    Models and methods for automated material identification in hyperspectral imagery acquired under unknown illumination and atmospheric conditions

    IEEE Transactions on Geoscience and Remote Sensing

    (1999)
  • Herwegab, J. A., Kerekesa, J. P., Weatherbeec, O., Messingera, D., van Aardta, J., Ientiluccia, E., Ninkova, Z.,...
  • Cited by (0)

    View full text