Improving image segmentation quality through effective region merging using a hierarchical social metaheuristic

https://doi.org/10.1016/j.patrec.2005.07.022Get rights and content

Abstract

This paper proposes a new evolutionary region merging method in order to efficiently improve segmentation quality results. Our approach starts from an oversegmented image, which is obtained by applying a standard morphological watershed transformation on the original image. Next, each resulting region is represented by its centroid. The oversegmented image is described by a simplified undirected weighted graph, where each node represents one region and weighted edges measure the dissimilarity between pairs of regions (adjacent and non-adjacent) according to their intensities, spatial locations and original sizes. Finally, the resulting graph is iteratively partitioned in a hierarchical fashion into two subgraphs, corresponding to the two most significant components of the actual image, until a termination condition is met. This graph-partitioning task is solved by a variant of the min-cut problem (normalized cut) using a hierarchical social (HS) metaheuristic. We have efficiently applied the proposed approach to brightness segmentation on different standard test images, with good visual and objective segmentation quality results.

Introduction

Image segmentation is one of the most complex stages in image analysis. It becomes essential for subsequent image description and recognition tasks. The problem consists of partitioning an image into its constituent regions, objects or labels (Gonzalez and Woods, 2002). The level of division depends on the specific problem being solved. This partition is accomplished in such a way that the pixels belonging to homogeneous regions, regarding to one or more features (i.e., brightness, texture or colour), share the same label, and regions of pixels with significantly different features have different labels. There must be considered four objectives for developing an efficient generalized segmentation algorithm (Ho and Lee, 2003): continuous closed contours, non-oversegmentation, independence of threshold setting and short computation time. In particular, the oversegmentation error, which occurs when a single semantic object is divided into several regions, is a tendency of some segmentation methods like watershed (Beucher and Lantuejoul, 1979, Haris et al., 1998, Hernández and Barner, 2000). Therefore, some subsequent region merging process is needed to improve the segmentation results.

Metaheuristic search algorithms are high-level general strategies for finding high quality solutions that have some mechanism for escaping from local optima (Glover and Kochenberger, 2002, Michalewicz and Fogel, 2000, Voss, 2001). The relevance of metaheuristics is reflected in their application to solve many different real-world complex problems, mainly combinatorial. Since the initial proposal of Glover about tabu search in 1986, many metaheuristics have emerged to design good general heuristics methods for solving different domain application problems. Genetic programming, GRASP, simulated annealing or ant colony optimization, are some other well-known examples of metaheuristics. Their relevance is reflected in the publication of many books and papers on this research area. There can be found a recent survey in (Glover and Kochenberger, 2002).

The applications of evolutionary techniques to Image Processing and Computer Vision problems have increased mainly due to the robustness of the approach (Parker, 1996). Many image analysis tasks like image enhancement, feature selection and image segmentation, have been effectively solved using genetic programming (Poli, 1996). Among these tasks, segmentation is one of the most difficult one. The standard linear segmentation methods are insufficient for a reliable object classification. The usage of some non-linear approaches like neural networks or mathematical morphology methods has usually provided better results (Sonka et al., 1999). However, the inherent complexity of many scenes (i.e., images with non-controlled illumination conditions or textured images) makes it very difficult to achieve an optimal pixel classification into regions, due to the combinatorial nature of the task. Evolutionary image segmentation (Ho and Lee, 2003, Poli, 1996, Yoshimura and Oe, 1999) has reported a good performance in relation to more classical segmentation methods.

Many segmentation approaches have been proposed in the literature (Gonzalez and Woods, 2002, Parker, 1996, Sarkar et al., 2000, Sonka et al., 1999). In this paper, we present a region merging method based on a new evolutionary metaheuristic called hierarchical social (HS) algorithms. This metaheuristic uses as evolutionary strategies the cooperation and competition mechanism, instead of classical crossover and mutation operators. Note that other well-known evolutionary metaheuristics present a similar type of cooperation and competition mechanism, like ant colony optimization (Dorigo et al., 1996), swarm intelligence (Kennedy et al., 2003) and cultural algorithms (Reynolds, 1999).

The proposed segmentation method can be considered as region-based, and it pursuits a high-level extraction of the image structures. After an oversegmentation of the initial image, our method produces a hierarchical top-down region-based decomposition. This way, the oversegmented image is represented by a weighted undirected graph, called modified region adjacency graph (MRAG). The MRAG structure is similar to the region adjacency graph (RAG) (Haris et al., 1998, Hernández and Barner, 2000, Sarkar et al., 2000) but MRAG also enables to add edges between pairs of non-adjacent regions (nodes).

In the MRAG model, nodes are represented by the centroid of each region as a result of the initial oversegmentation, and weighted edges are defined taking into account the spatial distance between nodes, their corresponding brightness mean value, and the corresponding region sizes. We consider that the minimum semantic information is given by each watershed region, so the mapping from the original oversegmented image to the MRAG can be made in a straight way (that is, each region mapped by one node).

Next, an optimal bipartition that minimizes the normalized cut value (Shi and Malik, 2000) for the image graph is computed. This process is successively repeated for each of the two resulting regions (image subgraphs) using a binary splitting scheme until a termination condition is achieved. An evident computational advantage is obtained when describing the image as a set of regions instead of pixel in the MRAG structure. It enables a faster region merging in oversegmented images with higher spatial resolution than reported in the literature (Haris et al., 1998, Hernández and Barner, 2000, Sarkar et al., 2000).

The definition of MRAG and the application of a hierarchical social (HS) metaheuristic to solve efficiently the normalized cut problem is the core of the proposed method. Our approach for modelling and solving image segmentation as a graph-partitioning problem is related to Shi and Malik’s work (2000). They use a computational technique based on a generalized eigenvalue problem for computing the segmentation regions. Instead of that, we found that higher quality segmentation results can be obtained when applying HS Algorithms, through an iterative solution of a normalized cut problem.

Section snippets

Modified region adjacency graph

Several approaches have been proposed to decrease the effect of severe oversegmentation of watershed-based segmentation approaches (Haris et al., 1998, Hernández and Barner, 2000). These methods usually involve a preprocessing of the original image. Many of them are based on the region adjacency graph (RAG) which is a usual data structure used to represent region neighbourhood relations in a segmented image (Sonka et al., 1999). RAG is a weighted undirected graph G = {V, E, W}, where each node

Image partitioning via graph cuts

The earliest image graph-based segmentation method used fixed neighbourhood structures and local measures in computing segmentation. In (Zahn, 1971) is presented a segmentation method based on minimum spanning tree of the image graph. This technique has also been successfully used in clustering applications. It leads to high quality segmentation in quite simple images but, for real images with high variability, the resulting segmentation is not acceptable. As an improvement, other authors (

Hierarchical social (HS) algorithms

This section presents the general features of a new evolutionary metaheuristic called hierarchical social (HS) algorithms. In order to get a general description of this metaheuristic, the reader is pointed to references (Duarte et al., 2004a, Duarte et al., 2004b, Duarte, 2004, Fernández et al., 2003). This metaheuristic have been successfully applied to several problems as critical circuit computation (Fernández et al., 2003), scheduling (Duarte et al., 2004c, Duarte, 2004), max-cut problem (

Method overview

Three major stages are considered in the proposed evolutionary segmentation approach. A high level description of the algorithm is shown in Fig. 6. First, we create an oversegmented image, as a preprocessing stage, applying a watershed segmentation to the initial brightness image. In particular, Vincent and Soille (1991) algorithm is used. The oversegmented image, which contains a very high number of regions of little significance, will be the input for the image graph construction.

In the

Experimental results

Computational experiments were evaluated in a 3 GHz Intel Pentium 4, 1 GB DRAM. All algorithms were coded by the same programmer in C++ without code optimization. With respect to the implemented HS algorithm, the selected number of groups is Nodes/100 and the number of autonomous iterations is 20.

First, the efficiency of the algorithm is tested. We compare our proposal with a pixel-based approach (Duarte et al., 2004b, Shi and Malik, 2000). In order to get comparable results we use a set of

Conclusion

This paper has introduced the HS metaheuristic, as a region merging technique to efficiently improve the image segmentation quality results. First, the original image is oversegmented with a standard watershed algorithm. Next, the associated MRAG structure is built to model the segmentation problem as a graph. MRAG may consider as neighbour regions those ones that are not adjacent. This new model allows the processing of larger spatial resolution images than other typical graph-based

References (35)

  • R. Urquhart

    Graph theoretical clustering based on limited neighborhood sets

    Pattern Recognition

    (1982)
  • M. Yoshimura et al.

    Evolutionary segmentation of texture image using genetic algorithms

    Pattern Recognition

    (1999)
  • Barbu, A., Zhu, S.-C., 2003. Graph partitioning by Swendsen–Wang cuts. In: Ninth IEEE International Conference on...
  • Beucher, S., Lantuejoul, C., 1979. Use of watersheds in contour detection. In: Internat. Workshop on Image Processing,...
  • Ding, C., He, X., Zha, H., Gu, M., Simon, H., 2001. A min-max cut algorithm for graph partitioning and data clustering....
  • M. Dorigo et al.

    Ant system: optimization by a colony of cooperating agents

    IEEE Trans. Systems Man Cybernet. Part B

    (1996)
  • Duarte, A., 2004. Algoritmos Sociales Jerárquicos: Una metaheurística basada en la hibridación entre métodos...
  • A. Duarte et al.

    A hierarchical social metaheuristic for the max-cut problem

  • A. Duarte et al.

    Top-Down Evolutionary Image Segmentation using a Hierarchical Social Metaheuristic

    (2004)
  • Duarte, A., Fernández, F., Sánchez, A., 2004c. Software Pipelining using a Hierarchical Social Metaheuristic. In:...
  • P.F. Felzenszwalb et al.

    Efficient graph-based image segmentation

    Internat. J. Computer Vision

    (2004)
  • Fernández, F., Duarte, A., Sánchez, A., 2003. A software pipelining method based on a hierarchical social algorithm....
  • R.C. Gonzalez et al.

    Digital Image Processing

    (2002)
  • Gothandaraman, A., 2004. Hierarchical image segmentation using the watershed algorithm with a streaming implementation....
  • K. Haris

    Hybrid Image Segmentation Using Watersheds and Fast Region Merging

    IEEE Trans. Image Process.

    (1998)
  • Hernández, S.E., Barner, K.E., 2000. Joint region merging criteria for watershed-based image segmentation. In:...
  • Cited by (36)

    • Rice and wheat grain counting method and software development based on Android system

      2017, Computers and Electronics in Agriculture
      Citation Excerpt :

      The major deficiency of the traditional watershed algorithm is over-separation. Recently many scholars have conducted studies to improve the watershed algorithm to reduce over-separation (Duarte et al., 2006; Wang and Paliwal, 2006). The active contour model is a classical image segmentation algorithm based on partial differential equations (Zheng et al., 2009).

    • A survey of graph theoretical approaches to image segmentation

      2013, Pattern Recognition
      Citation Excerpt :

      In the partial differential equations (PDE) based methods [17–21], the segmentation of a given image is calculated by evolving parametric curves in the continuous space such that an energy functional is minimized for a desirable segmentation. Region splitting and merging is another popular category of segmentation methods, where the segmentation is performed in an iterative manner until some uniformity criteria [22,23] are satisfied. The reviews of various segmentation techniques can be found for image thresholding methods [24], medical image segmentation [25,26], statistical level set segmentation [27], 3D image segmentation [28], edge detection techniques [29] and so on.

    • Grouping objects in multi-band images using an improved eigenvector-based algorithm

      2010, Mathematical and Computer Modelling
      Citation Excerpt :

      The first, the multi-scale segmentation method, has been addressed in [12–14] by using different strategies for constructing pyramids. The second, the region-based method, has been developed by using different strategies for merging pixels into a set of regions [15–19]. These two aspects have largely decreased the time complexity of the spectral relaxation algorithm, even to become linear.

    View all citing articles on Scopus
    View full text