Elsevier

Pattern Recognition

Volume 47, Issue 4, April 2014, Pages 1764-1776
Pattern Recognition

Unified detection of skewed rotation, reflection and translation symmetries from affine invariant contour features

https://doi.org/10.1016/j.patcog.2013.11.016Get rights and content

Highlights

  • A unified framework is proposed for detecting different symmetries on real-world images.

  • Affine invariant contour matching is used to find correspondence under skewed imaging.

  • A sign change criterion is proposed to classify matching pairs for different symmetries.

  • Novel schemes such as that based on short line segments are proposed for symmetry voting.

  • Different symmetries are detected simultaneously or separately with desired performance.

Abstract

Symmetry detection is significant for object detection and recognition since symmetries are salient cues for distinguishing geometrical structures from cluttered backgrounds. This paper proposes a unified framework to detect rotation, reflection and translation symmetries simultaneously on unsegmented real-world images. The detection is performed based on affine invariant contour features, and is applicable under skewed imaging with distortions. Contours on a natural image are first matched to each other to find affine invariant matching pairs, which are then classified into three groups using a sign change criterion corresponding to the three types of symmetries. The three groups are used to vote for the corresponding symmetries: the voting for rotation utilizes a scheme of short line segments; the voting for reflection is based on a parameterization of axis equation, and the voting for translation uses a cascade-like approach. Experimental results of real-world images are provided with quantitative evaluations, validating that the proposed framework achieves desired performance.

Introduction

Detections of rotation, reflection and translation symmetries increasingly find their use in pattern recognition, 2D image processing, and 3D object reconstruction. Their applications include but not limited to: photo editing [1] (using reflection), object recognition [2] (using reflection and rotation), image segmentation [3] (using reflection), scene perception [4] (using translation), digital inspection [3] (using reflection and rotation), scene reconstruction [5] (using translation), and 3D object reconstruction [6] (using reflection).

This paper aims to detect the three common symmetries simultaneously for a potentially wide variety of applications. The contribution of this paper is that it detects the three types of symmetries of natural images under skewed imaging in a unified framework, which is novel in the aspects that: (1) it uses contour features on unsegmented real-world images to detect all the symmetries; (2) it is based upon affine invariant contour matching; (3) it proposes a sign change criterion to group the matching pairs; and (4) it adopts novel voting schemes to detect different symmetries.

Though detection methods based on ideal contours on synthetical images are well studied, those for cluttered contours on natural images are few to be researched. Using affine invariant contour matching on natural images we can unify the detection of rotation, reflection and translation in the same way, since they are special cases of the affine transform. By the grouping scheme we classify the single affine transform to three types for different voting process. To simplify the work we only vote for the major aspects of symmetries: for rotation symmetries we detect their rotation centers; for reflection symmetries we detect their symmetry axes; and for translation symmetries we detect their near-regular lattices. The pipeline of the algorithm thus includes four main stages: contour detection, contour matching, contour grouping and contour voting for symmetries.

In the following sections, we first analyze related work on symmetry detection in Section 2, then propose the theoretical formulation and algorithm framework in 3 Symmetry formulations, 4 Framework for symmetry detection, respectively. The algorithm implementation and evaluation methods are given in Section 5, and experimental results are provided in Section 6. The conclusion are given in the last section.

Section snippets

Related work

Recent surveys for symmetry detection [7], [8] indicate that after several decades of effort, symmetry detection in real-world images still remains a challenging problem. We shall analyze existing work for detecting rotation, reflection, translation symmetries and multiple of them, with remarks on contour-based methods.

Rotation symmetry detection has been well researched [9], [10], [11], [12], [13], [14], [15], [16], [17], [18]. Some previous work only applied to non-skewed rotation symmetry

Symmetry formulations

In this section we discuss from the original formulation of symmetry to the proposed formulation by affine transformations to detect symmetries under distortions.

Framework for symmetry detection

To detect the three types of symmetries, our scheme is to use a single matching process to find affine matching of contours (see Section 4.1), then classify the matching to different groups of matches (Section 4.2), finally use different voting process to vote for different symmetries (Section 4.3). The diagram of the complete algorithm is shown in Fig. 1.

Implementation, evaluation and parameter analysis

This section discusses some issues for implementing the proposed algorithm and evaluating the performance of the algorithm. Parameter analysis for the algorithm is also given in this section.

Experimental results

In this section we provide experimental results of the proposed algorithm. The algorithm is designed for detecting symmetries in general skewed images with distortions, including non-skewed images as well. Detection results for some of non-skewed images are shown in Fig. 9 as a best-case-validation of the algorithm. We shall quantitatively evaluate the detections for rotation, reflection and translation symmetries separately, based on commonly used data sets for skewed (and non-skewed) images

Conclusion

We proposed a framework for detecting skewed rotation, reflection and translation symmetries from real-world images using contour features. The affine invariant contour matching is a key step, which is followed by contour grouping and contour voting to achieve the successful detections. This is one of the few work to address the problems of detecting three types of symmetries simultaneously. The algorithm can also be used to detect each of the symmetries individually; in this case, the

Conflict of interest statment

None declared.

Acknowledgments

We thank the anonymous reviewers for helpful suggestions to improve our work. We thank G. Loy, J.-O. Eklundh, M. Park and Y. Liu for providing their code for research and evaluation. This work was supported by the National Natural Science Foundation of China under Grant 60803071, and by the Research Grants Council of Hong Kong under Project CityU117106.

Zhaozhong Wang received his Ph.D. degree in computer science from Shanghai Jiao Tong University, Shanghai, China, in 2005. He is currently an assistant professor at Image Processing Center, Beihang University, Beijing, China. His research interests include image and video processing, pattern recognition, and mathematical methods in computer vision.

References (50)

  • W.H. Li, A.M. Zhang, L. Kleeman, Real time detection and segmentation of reflectionally symmetric objects in digital...
  • Y. Sun, B. Bhanu, Symmetry integrated region-based image segmentation, in: IEEE Conference on Computer Vision and...
  • P. Minwoo, K. Brocklehurst, R.T. Collins, Y. Liu, Translation-symmetry-based perceptual grouping with applications to...
  • A.Y. Yang, S. Rao, K. Huang, W. Hong, Y. Ma, Geometric segmentation of perspective images based on symmetry groups, in:...
  • T. Xue, J. Liu, X. Tang, Symmetric piecewise planar object reconstruction from a single image., in: IEEE Conference on...
  • Y. Liu et al.

    Computational symmetry in computer vision and computer graphics

    Found. Trends Comput. Graph. Vis.

    (2009)
  • M. Park, S. Lee, P. Chen, S. Kashyap, A. Butt, Y. Liu, Performance evaluation of state-of-the-art discrete symmetry...
  • V. Prasad, L. Davis, Detecting rotational symmetries, in: International Conference on Computer Vision, 2005, pp. II:...
  • B. Johansson, G. Granlund, Fast selective detection of rotational symmetries using normalized inhibition, in: European...
  • S. Lee, R. Collins, Y. Liu, Rotation symmetry group detection via frequency analysis of frieze-expansions, in: IEEE...
  • S. Lee et al.

    Skewed rotation symmetry group detection

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2010)
  • H. Cornelius, G. Loy, Detecting rotational symmetry under affine projection, in: International Conference on Pattern...
  • S. Carlsson, Symmetry in perspective, in: European Conference on Computer Vision, 1998, p. I:...
  • Y. Liu, J. Hays, Y. Xu, H. Shum, Digital papercutting, in: Proceedings of SIGGRAPH Technical Sketch,...
  • M. Venkatesh, S. Cheung, Symmetric shape completion under severe occlusions, in: International Conference on Image...
  • Cited by (9)

    • Mirror symmetry detection in curves represented by means of the Slope Chain Code

      2019, Pattern Recognition
      Citation Excerpt :

      Michaelsen et al. [8] developed a mirror symmetry detection algorithm for real 2D images based on the Scale-Invariant Feature Transform algorithm which detects a combinatorial set of higher-order symmetric Gestalten. Wang et al. [9] proposed a unified detection of skewed rotation, reflection and translation symmetries from affine invariant contour features. Elawady et al. [10] proposed a global reflectional symmetry detection scheme based on an edge-based feature extraction using Log-Gabor filters and a voting scheme parameterized by corresponding textural and color neighborhood information of the image.

    • Reflection and Rotation Symmetry Detection via Equivariant Learning

      2022, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
    • Detection of the main symmetry axis based on the clustering analysis

      2017, Proceedings of the 29th Chinese Control and Decision Conference, CCDC 2017
    View all citing articles on Scopus

    Zhaozhong Wang received his Ph.D. degree in computer science from Shanghai Jiao Tong University, Shanghai, China, in 2005. He is currently an assistant professor at Image Processing Center, Beihang University, Beijing, China. His research interests include image and video processing, pattern recognition, and mathematical methods in computer vision.

    Lianrui Fu received the B.Eng. degree in pattern recognition and intelligent system from Beihang University, Beijing, China, in 2008. He is currently working toward the Ph.D. degree at Image Processing Center, Beihang University. His research interests include object detection, object recognition, and embedded vision system.

    Y.F. Li obtained his Ph.D. degree in robotics from the Department of Engineering Science, Oxford University, UK, in 1993. From 1993 to 1995, he worked as a research staff in the Department of Computer Science at the University of Wales, Aberystwyth, UK. He joined City University of Hong Kong in 1995. His research interests include robot vision, visual tracking, robot sensing and sensor based control.

    View full text