Elsevier

Pattern Recognition Letters

Volume 56, 15 April 2015, Pages 45-51
Pattern Recognition Letters

Object recognition in hyperspectral images using Binary Partition Tree representation

https://doi.org/10.1016/j.patrec.2015.01.003Get rights and content

Highlights

  • New object detection technique by using hierarchical region-based image representations.

  • Binary Partition Tree is proposed as a structured search space in order to incorporate the spectral and the spatial information.

  • The strategy is applied on several datasets of hyperspectral images of urban areas.

  • The obtained results show the interest of studying the objects of the scene with a region-based perspective.

Abstract

In this work, an image representation based on Binary Partition Tree is proposed for object detection in hyperspectral images. This hierarchical region-based representation can be interpreted as a set of hierarchical regions stored in a tree structure, which succeeds in presenting: (i) the decomposition of the image in terms of coherent regions and (ii) the inclusion relations of the regions in the scene. Hence, the BPT representation defines a search space for constructing a robust object identification scheme. Spatial and spectral information are integrated in order to analyze hyperspectral images with a region-based perspective. For each region represented in the BPT, spatial and spectral descriptors are computed and the likelihood that they correspond to an instantiation of the object of interest is evaluated. Experimental results demonstrate the good performances of this BPT-based approach.

Introduction

Automatic object recognition to map areas has received a lot of attention thanks to the advance of remote sensing technology [2]. In this context, the spatial and the spectral resolutions of the new sensors have played a fundamental role. Specifically, the improvement of the spatial information has been essential for this high-level image understanding task. Accordingly, morphological approaches have received an important interest in gray value or color images [4], [5], [17].

In the hyperspectral literature, object detection techniques have been mainly developed in the context of pixel-wise spectral classification. In this approach, spectra having a high similarity with the material describing the reference object are individually detected. The drawbacks of pixel-wise analysis is well-known in classical [8] and hyperspectral [12], [16] remote sensing images. A major problem is the important semantic gap due to the lack of concordance between the low level and reduced information provided by a single pixel and the human interpretation. Despite this, traditional algorithms characterize objects only by their spectral signatures.

Because of the pixel-based model limitations, research on region-based object detection algorithms has recently received much attention. Region-based representations allow in particular spatial features such as shape, area or orientation to be computed. These features can significantly contribute to the definition of robust object detection algorithms. In this context, the ECognition software [10] was developed. It relies on hierarchical segmentation and produces an image partition on which various region descriptors can be computed. These descriptors are then used as region features for the recognition of objects in the image. One of the main limitations of this strategy is that it assumes that the best partition corresponds to one level of the previously computed hierarchical segmentation. Unfortunately, this assumption is rarely true and, very often, coherent objects can be found at different levels of the hierarchy. Ideally, a robust strategy should study the features in the complete hierarchy to detect the best regions representing the object. As a result, recent works have tried to investigate how different spectral, spatial and joint spectral/spatial features computed on regions evolve from one level to another in a segmentation hierarchy [15]. This study proposes to study the regions at different scales, however, no methodology is proposed to automatically select the regions forming the objects.

Instead of using a classical hierarchical segmentation approach which produces a single partition, a solution to address the need of multiscale analysis relies on image representations based on regions trees. These representations are useful because besides allowing the study of internal region properties (color, texture, shape, etc.), they also permit the study of external relations such as adjacency, inclusion, similarity of properties, etc. Furthermore, a tree is essentially a hierarchical structure and therefore supports multiscale analysis of regions. The multiscale nature of trees provides flexibility to situations where a given image has to be studied at different scales depending on the processing purpose. In this context, the work presented in [1] proposes the use of component trees resulting from the iterative application of morphological opening and closing on individual PCA spectral bands. The main limitations of this approach are twofold: First, the component tree mainly describes the structure of extrema of the spectral bands and, in hyperspectral images, there is no particular reason why objects of interest should be limited to extrema of spectral bands. Furthermore, in [1], the approach consists in pruning the tree to create a partition before performing the object recognition. The pruning essentially extracts the largest homogenous regions. Once the partition is defined, the search for objects is performed. As in [10], one of the drawbacks is that the object detection task is done after a segmentation step producing a partition.

The work presented here proposes to initially generate a hierarchical region-based representation of the image and, then, to use this representation as search space for the object detection (therefore avoiding the creation of a partition on which objects are searched as in [1], [10]. A Binary Partition Tree (BPT) as in [3] is used as hyperspectral image representation. BPTs are less limited than component trees [1] as they do not focus on the description of extrema of spectral bands. They perform a hierarchical grouping based on pixel homogeneity and can directly take into the correlation between spectral bands. For object detection, the use of BPT has been introduced in [19] where a simple top-down analysis of the tree branches was done. During this analysis, the objects were detected by selecting the largest nodes having the appropriate features. Therefore, the detected object was the first node in the branch and the rest of BPT branches were not studied. However, the best region representing the object is not always the largest one with the appropriate features. Here, we present a more robust strategy that studies all BPT nodes to detect the best ones representing the sought object. The paper organization is as follows: Section 2 introduces the BPT and its construction. The BPT analysis for object detection is discussed in Section 3. Experimental results are reported in Section 4. Finally, conclusions are drawn in Section 5.

Section snippets

BPT construction

The BPT is a structured representation of a set of hierarchical partitions which is usually obtained through an iterative bottom-up region merging algorithm. Starting from individual pixels or any other initial partition, the tree is constructed by iteratively merging the pair of most similar neighboring regions. Each iteration requires three different tasks: (1) the pair of most similar neighboring regions is identified, (2) a new region corresponding to the union of the region pair is formed,

Object detection strategy

As instantiations of the object of interest, O, may have many different visual appearances, the detection relies on a set of features, ΩF, characterizing O. Based on these features, the likelihood of each BPT node P(O|Ri) to be an instantiation of O is assessed and assigned to the node. Once the BPT has been populated with these likelihood, a search is performed to detect the most probable instantiations of the object of interest.

Experimental results

This section addresses the evaluation of the object detection strategy proposed in Section 3. The goal of the experiments is to compare the results of the proposed strategy with a classical pixel-wise method such as SVM classification. In order to perform this evaluation, detection examples of two different urban objects: roads and buildings, are discussed. The experimental evaluation is carried out using two different hyperspectral images captured by two different sensors.

The first studied

Conclusions

An automatic hyperspectral object detection methodology using a BPT image representation has been detailed in this work. It has been illustrated how BPT can be a powerful image representation which provides a hierarchically structured search space for object recognition applications where the spectral and the spatial information can be incorporated in the search of a reference object. The obtained results show the interest of studying the objects of the scene with a region-based perspective and

References (19)

There are more references available in the full text version of this article.

Cited by (21)

  • Supervised quality evaluation of binary partition trees for object segmentation

    2021, Pattern Recognition
    Citation Excerpt :

    In other words, when considering BPT-based image processing, a good segmentation algorithm has to be applied on a good image tree data structure. A wide literature has been devoted to BPT-based segmentation, mostly in the challenging context of remote sensing [13–22]. Various metrics were investigated (spectral, spatial, geometric, etc.) for building BPTs.

  • Binary Partition Tree construction from multiple features for image segmentation

    2018, Pattern Recognition
    Citation Excerpt :

    The basic criteria used in most of segmentation approaches are generally radiometric or geometric region similarities (or their fusion into a single metric). Except for a few contributions in the field of object recognition [33–35], BPTs were mainly used for segmentation / classification tasks, where such total partitions make sense from a semantic point of view. More precisely, the wider application field of BPTs is remote sensing [36].

  • Learning spatial relations and shapes for structural object description and scene recognition

    2018, Pattern Recognition
    Citation Excerpt :

    In a second time, we evaluated the robustness of our approach when considering another input segmentation algorithm for the extraction of primitive object subparts. We relaunched our experiments following the same protocol but using as input segmentation the results obtained from the binary partition tree (BPT) algorithm [41], which has been involved successfully in various segmentation tasks [47,48]. The BPT is a hierarchical representation of an image, organized as a binary tree, that models the content of an image in a multiscale way.

  • Processing Hyperspectral Images

    2016, Optical Remote Sensing of Land Surface: Techniques and Methods
  • RGB-Induced Feature Modulation Network for Hyperspectral Image Super-Resolution

    2023, IEEE Transactions on Geoscience and Remote Sensing
View all citing articles on Scopus

This paper has been recommended for acceptance by N. Sladoje.

View full text