Abstract
Residual network which can effectively overcome gradient disappearance in convolutional neural networks has been successfully applied to hyperspectral classification. Yet simply accumulating residual units does not improve the model performance. Besides, the lack of training samples seriously restricts the hyperspectral classification accuracy. In attempt to solve the abovementioned problems, residual networks with multi-attention mechanism (RNMA) is proposed. The spectral-spatial attention module (SSAM) in accordance with the structure characteristics of hyperspectral images is introduced into the RNMA method. The SSAM is embedded after the input which can strengthen bands relevant to ground object and spatial salient regions of ground object while weaken less informative features. Meanwhile, channel attention module is introduced into residual network unit to form residual network unit with channel attention module (RUCA). Based on RUCA, the features of the spectral dimension and the spatial dimension are sequentially extracted. Besides, sample augmentation method with spectral and spatial constraints is applied to expand sample size. Experiments results on three hyperspectral datasets show that the RNMA model outperforms than other methods in classification accuracy. The RNMA model achieved superior classification result compared with other state-of-the-art methods in the small training sample size.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abdollahi A, Pradhan B, Shukla N, Chakraborty S, Alamri A (2020) Deep learning approaches applied to remote sensing datasets for road extraction: a state-of-the-art review. Remote Sensing 12(9):1444. https://doi.org/10.3390/rs12091444
Acosta I, Khodadadzadeh M, Tusa L, Ghamisi P, Gloaguen R (2019) A machine learning framework for drill-core mineral mapping using hyperspectral and high-resolution mineralogical data fusion. IEEE J Sel Top Appl Earth Obser Remote Sens 99:1–14. https://doi.org/10.1109/JSTARS.2019.2924292
Ball JE, Anderson DT, Chan CS (2017) Comprehensive survey of deep learning in remote sensing: theories, tools, and challenges for the community. J Appl Remote Sens 11:11–54. https://doi.org/10.1117/1.JRS.11.042609
Chen YS, Lin ZH, Zhao X, Wang G, Gu YF (2014) Deep learning based classification of hyperspectral data. IEEE J Sel Top Appl Earth Obs Remote Sens 7(6):2094–2107. https://doi.org/10.1109/JSTARS.2014.2329330
Chen Y, Zhao X, Jia X (2015) Spectral–spatial classification of hyperspectral data based on deep belief network. IEEE J Sel Topics Appl Earth Obs Remote Sens 8(6):2381–2392. https://doi.org/10.1109/JSTARS.2015.2388577
Chen YS, Jiang H, Li C, Jia X, Ghamisi P (2016a) Deep feature extraction and classification of hyperspectral images based on convolutional neural networks. IEEE Trans Geosci Remote Sens 54:6232–6251. https://doi.org/10.1109/TGRS.2016.2584107
Chen J, Xia J, Du P, Chanussot J, Xue Z, Xie X (2016b) Kernel supervised ensemble classifier for the classification of hyperspectral data using few labeled samples. Remote Sensing. 8(7):601. https://doi.org/10.3390/rs8070601
Ding HY, Xu LM, Wu Y, Shi WZ (2020) Classification of hyperspectral images by deep learning of spectral-spatial features. ARAB J GEOSCI. 13(12):464. https://doi.org/10.1007/s12517-020-05487-4
Fang L, Li S, Kang X (2014) Spectral-spatial hyperspectral image classification via multiscale adaptive sparse representation. IEEE Trans Geosci Remote Sens 52(12):7738–7749. https://doi.org/10.1109/TGRS.2014.2318058
Hang R, Liu Q, Song H, Sun Y (2016) Matrix-based discriminant subspace ensemble for hyperspectral image spatial-spectral feature fusion. IEEE Trans Geosci Remote Sens 54(2):783–794. https://doi.org/10.1109/TGRS.2015.2465899
Haq QS, Tao L, Sun F, Yang S (2012) A fast and robust sparse approach for hyperspectral data classification using a few labeled samples. IEEE Trans Geosci Remote Sens 50(6):2287–2302. https://doi.org/10.1109/TGRS.2011.2172617
He K, Zhang X, Ren S (2016) Deep residual learning for image recognition. IEEE Computer Society. Las Vegas.:770–778. https://doi.org/10.1109/CVPR.2016.90
Hu W, Huang Y, Wei L, Zhang F, Li H (2015) Deep convolutional neural networks for hyperspectral image classification. J Sensors 2015:1–12. https://doi.org/10.1155/2015/258619
Hughes G (1968) On the mean accuracy of statistical pattern recognizers. IEEE Transactions on Information Theory 14(1):55–63. https://doi.org/10.1109/TIT.1968.1054102
Jia K, Liang S, Gu X, Baret F, Wei X, Wang X (2016) Fractional vegetation cover estimation algorithm for Chinese GF-1 wide field view data. Remote Sens Environ 177:184–191. https://doi.org/10.1016/j.rse.2016.02.019
Kang X, Li C, Li S, Lin H (2018) Classification of hyperspectral images by gabor filtering based deep network. IEEE J Sel Top Appl Earth Obs Remote Sens 11(4):1166–1178. https://doi.org/10.1109/JSTARS.2017.2767185
Karaca AC (2020) Domain transform filter and spatial-aware collaborative representation for hyperspectral image classification using few labeled samples. IEEE Geoscience and Remote Sensing Letters.:1–5. https://doi.org/10.1109/LGRS.2020.2998605
Kunkel B, Blechinger F, Lutz R, Doerffer R, Van der Piepen H, Schroder M (1998) ROSIS (Reflective Optics System Imaging Spectrometer)—a candidate instrument for polar platform missions. optoelectronic technologies for remote sensing from space. https://spie.org/Publications/Proceedings/Paper/10.1117/12.943611?SSO=1.Accessed on 12 November 2019
Li W, Wu G, Zhang F, Du Q (2017) Hyperspectral image classification using deep pixel-pair features. IEEE Trans Geosci Remote Sens. 55(2):844–853. https://doi.org/10.1109/TGRS.2016.2616355
Liu J, Zhang J, Gao Y, Zhang C, Li Z (2012) Enhancing spectral unmixing by local neighborhood weights. IEEE J Sel Top Appl Earth Obs Remote Sens 5:1545–1552. https://doi.org/10.1109/JSTARS.2012.2199282
Liu Q, Zhou F, Hang R et al (2017) Bidirectional-convolutional LSTM based spectral-spatial feature learning for hyperspectral image classification. Remote Sens. 9(12):1330–1347. https://doi.org/10.3390/rs9121330
Liu L, Shi Z, Pan B, Zhang N, Luo H, Lan X (2020) Multiscale deep spatial feature extraction using virtual RGB image for hyperspectral imagery classification. Remote Sens 12(2):280. https://doi.org/10.3390/rs12020280
Lu T, Li ST, Fang L, Ma YY (2016) Spectral-spatial adaptive sparse representation for hyperspectral image denoising. IEEE Trans Geosci Remote Sens 54(10):373–385. https://doi.org/10.1109/TGRS.2015.2457614
Mei S, Ji J, Geng Y, Zhang Z (2019) Unsupervised spatial-spectral feature learning by 3d convolutional autoencoder for hyperspectral classification. IEEE Trans Geosci Remote Sens 57(9):6808–6820. https://doi.org/10.1109/TGRS.2019.2908756
Melgani F, Bruzzone L (2004) Classification of hyperspectral remote sensing images with support vector machines. IEEE Transactions on Geoscience and Remote Sensing 42(8):1778–1790. https://doi.org/10.1109/TGRS.2004.831865
Niu B, Lan J, Shao Y, Zhang H (2020) A dual-branch extraction and classification method under limited samples of hyperspectral images based on deep learning. Remote Sensing12(3):536. https://doi.org/10.3390/rs12030536
Pan B, Shi Z, Xu X (2017) R-VCANet: A new deep learning-based hyperspectral image classification method. IEEE J Sel Top Appl Earth Obser Remote Sens 10(5):1975-1986.10.1109/JSTARS.2017.2655516
Paoletti M, Haut J, Plaza J, Plaza A (2018) A new deep convolutional neural network for fast hyperspectral image classification. ISPRS J Photogramm Remote Sens 145:120–147. https://doi.org/10.1016/j.isprsjprs.2017.11.021
Pei W, Dibeklio glu H, Baltrušaitis T, Tax DM (2020) Attended end-to-end architecture for age estimation from facial expression videos. IEEE Trans Image Processing 29:1972–1984. https://doi.org/10.1109/TIP.2019.2948288
Pradhan B, Al-Najjar HAH, Sameen MI, Tsang I, Alamri AM (2020) Unseen land cover classification from high-resolution orthophotos using integration of zero-shot learning and convolutional neural networks. Remote Sensing 12(10):1676. https://doi.org/10.3390/rs12101676
Roy SK, Krishna G, Dubey SR, Chaudhuri BB (2020) HybridSN: Exploring 3-D -2-D CNN feature hierarchy for hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters 17(2):277–281. https://doi.org/10.1109/LGRS.2019.2918719
Samaniego L, Bardossy A, Schulz K (2008) Supervised classification of remotely sensed imagery using a modified k-NN technique. IEEE Transactions on Geoscience and Remote Sensing 46(7):2112–2125. https://doi.org/10.1109/TGRS.2008.916629
Sekandari M, Masoumi I, Beiranvand Pour A, Muslim MA, Rahmani O, Hashim M, Zoheir B, Pradhan B, Misra A, Aminpour SM (2020) Application of Landsat-8, Sentinel-2, ASTER and WorldView-3 spectral imagery for exploration of carbonate-hosted Pb-Zn deposits in the Central Iranian Terrane (CIT). Remote Sensing 12(8):1239. https://doi.org/10.3390/rs12081239
Sellami A, Farah M, Farah IR, Solaiman B (2019) Hyperspectral imagery classification based on semi-supervised 3-d deep neural network and adaptive band selection. Expert Syst Appl 129:246–259. https://doi.org/10.1016/j.eswa.2019.04.006
Shao Y, Lan J (2019) A spectral unmixing method by maximum margin criterion and derivative weights to address spectral variability in hyperspectral imagery. Remote Sens 11(9):1045–1073. https://doi.org/10.3390/rs11091045
Shao Y, Lan J, Zhang Y, Zou J (2018) Spectral unmixing of hyperspectral remote sensing imagery via preserving the intrinsic structure invariant. Sensors. 18:3528. https://doi.org/10.3390/s18103528
Shao Y, Lan J, Niu B (2020) Dual-channel networks with optimal-band selection strategy for arbitrary cropped hyperspectral images classification. IEEE Geoscience and Remote Sensing Letters.:1–5. https://doi.org/10.1109/LGRS.2020.3023103
Sun L, Wu ZB, Liu JJ, Xiao L, Wei ZH (2015) Supervised spectral-spatial hyperspectral image classification with weighted Markov random fields. IEEE Trans Geosci Remote Sens 53(3):1490–1503. https://doi.org/10.1109/TGRS.2014.2344442
Tang Y, Lu Y, Yuan H (2015) Hyperspectral image classification based on three-dimensional scattering wavelet transform. IEEE Trans Geosci Remote Sens 53(5):2467–2480. https://doi.org/10.1109/TGRS.2014.2360672
Tao G, Jia K, Zhao X, Wei X, Xie X, Zhang X, Wang B, Yao Y, Zhang X (2019) Generating high spatio-temporal resolution fractional vegetation cover by fusing GF-1 WFV and MODIS Data. Remote Sens 11:2324. https://doi.org/10.3390/rs11192324
Transon J, Andrimont R, Maugnard A, Defourny P (2018) Survey of hyperspectral earth observation applications from space in the sentinel-2 context. Remote Sens 10(2):157. https://doi.org/10.3390/rs10020157
Vincent P, Larochelle H, Lajoie I, Bengio Y (2010) Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. J Mach Learn Res 11(12):3371–3408. https://doi.org/10.1016/j.mechatronics.2010.09.004
Wang Q, Meng Z, Li X (2017) Locality adaptive discriminant analysis for spectral-spatial classification of hyperspectral images. IEEE Geosci Remote Sens Lett 14(11):2077–2081. https://doi.org/10.1109/LGRS.2017.2751559
Wu H, Prasad S (2017) Convolutional recurrent neural networks for hyperspectral data classification. Remote Sens 9:298. https://doi.org/10.3390/rs9030298
Yang J, Zhao YQ, Chan JCW (2017) Learning and transferring deep joint spectral-spatial features for hyperspectral classification. IEEE Trans Geosci Remote Sens 55:4729–4742. https://doi.org/10.1109/TGRS.2017.2698503
Zeng Y, Ritz C, Zhao J, Lan J (2019) Scattering transform framework for unmixing of hyperspectral data. Remote Sens 11:2868. https://doi.org/10.3390/rs11232868
Zhang L, Zhang L, Tao D, Huang X (2012) On combining multiple features for hyperspectral remote sensing image classification. IEEE Trans Geosci Remote Sens 50(3):879–893. https://doi.org/10.1109/TGRS.2011.2162339
Zhao W, Du S (2016) Spectral spatial feature extraction for hyperspectral image classification: a dimension reduction and deep learning approach. IEEE Trans Geosci Remote Sens 54(8):4544–4554. https://doi.org/10.1109/TGRS.2016.2543748
Zhao Y, Yuan Y, Wang Q (2019) Fast spectral clustering for unsupervised hyperspectral image classification. Remote Sens. 11:399. https://doi.org/10.3390/rs11040399
Zhong Z, Li J, Luo Z, Chapman M (2018) Spectral-spatial residual network for hyperspectral image classification: a 3-d deep learning framework. IEEE Transactions on Geoscience and Remote Sensing 56(2):847–858. https://doi.org/10.1109/TGRS.2017.2755542
Acknowledgments
We would like to thank the editor and reviewers for their reviews that improved the content of this paper.
Conflict of interest
The authors declare that they have no conflict of interest.
Funding
This research was funded by the funding 41416040203, the funding FRF-GF-18-008A, and the funding FRF-BD-19-002A
Author information
Authors and Affiliations
Corresponding author
Additional information
Responsible Editor: Amjad Kallel
Rights and permissions
About this article
Cite this article
Shao, Y., Lan, J., Liang, Y. et al. Residual networks with multi-attention mechanism for hyperspectral image classification. Arab J Geosci 14, 252 (2021). https://doi.org/10.1007/s12517-021-06516-6
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s12517-021-06516-6