Skip to main content
SpringerLink
Log in

TILT: Transform Invariant Low-Rank Textures

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

In this paper, we propose a new tool to efficiently extract a class of “low-rank textures” in a 3D scene from user-specified windows in 2D images despite significant corruptions and warping. The low-rank textures capture geometrically meaningful structures in an image, which encompass conventional local features such as edges and corners as well as many kinds of regular, symmetric patterns ubiquitous in urban environments and man-made objects. Our approach to finding these low-rank textures leverages the recent breakthroughs in convex optimization that enable robust recovery of a high-dimensional low-rank matrix despite gross sparse errors. In the case of planar regions with significant affine or projective deformation, our method can accurately recover both the intrinsic low-rank texture and the unknown transformation, and hence both the geometry and appearance of the associated planar region in 3D. Extensive experimental results demonstrate that this new technique works effectively for many regular and near-regular patterns or objects that are approximately low-rank, such as symmetrical patterns, building facades, printed text, and human faces.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Baker, S., & Matthews, I. (2004). Lucas-Kanade 20 years on: A unifying framework. International Journal of Computer Vision, 56(3), 221–255.

    Article  Google Scholar 

  • Becker, S., Candès, E., & Grant, M. (2011). Templates for convex cone problems with applications to sparse signal recovery. Mathematical Programming Computation, 3(3), 165–218.

    Article  MathSciNet  Google Scholar 

  • Bertsekas, D. P. (2004). Nonlinear programming. Belmont: Athena Scientific.

    Google Scholar 

  • Cai, J., Candès, E., & Shen, Z. (2010). A singular value thresholding algorithm for matrix completion. SIAM Journal on Optimization, 20(4), 1956–1982.

    Article  MathSciNet  MATH  Google Scholar 

  • Candès, E., & Recht, B. (2009). Exact matrix completion via convex optimization. Foundations of Computational Mathematics, 9(6), 717–772.

    Article  MathSciNet  MATH  Google Scholar 

  • Candès, E., & Tao, T. (2010). The power of convex relaxation: Near-optimal matrix completion. IEEE Transactions on Information Theory, 56(5), 2053–2080.

    Article  Google Scholar 

  • Candès, E., Li, X., Ma, Y., & Wright, J. (2011). Robust principal component analysis? Journal of the ACM, 58(3), 11:1–11:37.

    Article  Google Scholar 

  • Canny, J. (1986). A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8(6), 679–698.

    Article  Google Scholar 

  • Chandrasekaran, V., Sanghavi, S., Parrilo, P., & Willsky, A. (2011). Rank-sparsity incoherence for matrix decomposition. SIAM Journal on Optimization, 21(2), 572–596.

    Article  MathSciNet  MATH  Google Scholar 

  • Chang, S., Davis, L., Dunn, S., Eklundh, J., & Rosenfeld, A. (1987). Texture discrimination by projective invariants. Pattern Recognition Letters, 5, 337–342.

    Article  Google Scholar 

  • Chen, J. L., & Kundu, A. (1994). Rotation and gray scale transform invariant texture identification using wavelet decomposition and hidden Markov model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(2), 208–214.

    Article  Google Scholar 

  • Cohen, F., Fan, Z., & Patel, M. (1991). Classification of rotated and scaled texture images using Gaussian Markov random field models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13, 192–202.

    Article  Google Scholar 

  • Cromme, L. (1978). Strong uniqueness: A far-reaching criterion for the convergence analysis of iterative procedures. Numerische Mathematik, 29, 179–193.

    Article  MathSciNet  MATH  Google Scholar 

  • Do, M., & Vetterli, M (2002). Rotation invariant texture characterization and retrieval using steerable wavelet-domain hidden Markov models. IEEE Transactions on Multimedia, 4(4), 517–527.

    Article  Google Scholar 

  • Eckstein, J., & Bertsekas, D. (1992). On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators. Mathematical Programming, 55, 293–318.

    Article  MathSciNet  MATH  Google Scholar 

  • Frey, B., & Jojic, N. (1999). Transformed component analysis: Joint estimation of spatial transformations and image components. In Proc. of IEEE international conference on computer vision.

    Google Scholar 

  • Gabay, D., & Mercier, B. (1976). A dual algorithm for the solution of nonlinear variational problems via finite element approximations. Computers and Mathematics with Applications, 2, 17–40.

    Article  MATH  Google Scholar 

  • Ganesh, A., Lin, Z., Wright, J., Wu, L., Chen, M., & Ma, Y. (2009). Fast algorithms or recovering a corrupted low-rank matrix. In Proceedings of third international workshop on computational advances in multi-sensor adaptive processing.

    Google Scholar 

  • Garding, J., & Lindeberg, T. (1996). Direct computation of shape cues using scale-adapted spatial derivative operators. International Journal of Computer Vision, 17(2), 163–191.

    Article  Google Scholar 

  • Glowinski, R., & Marroco, A. (1975). Sur l’approximation, par elements finis d’ordre un, et la resolution, par penalisation-dualite, d’une classe de problemes de Dirichlet non lineares. Revuew Francaise d’Automatique, Informatique et Recherche Operationelle, 9, 41–76.

    Google Scholar 

  • Greenspan, H., Belongie, S., Goodman, R., & Perona, P. (1994). Rotation invariant texture recognition using a steerable pyramid. In Proc. of international conference on pattern recognition.

    Google Scholar 

  • Haley, G., & Manjunath, B. (1999). Rotation-invariant texture classification using a complete space-frequency model. IEEE Transactions on Image Processing, 8, 255–269.

    Article  Google Scholar 

  • Harris, C., & Stephens, M. (1988). A combined corner and edge detector. In Proc. of 4th alvey vision conference.

    Google Scholar 

  • He, B. (2009). Parallel splitting augmented Lagrangian methods for monotone structured variational inequalities. Computational Optimization and Applications, 42(2).

  • He, B., Tao, M., & Yuan, X. (2011). Alternating direction method with Gaussian back substitution for separable convex programming. SIAM Journal on Optimization, under-revision.

  • Jittorntrum, K., & Osborne, M. (1980). Strong uniqueness and second order convergence in nonlinear discrete approximation. Numerische Mathematik, 34, 439–455.

    Article  MathSciNet  MATH  Google Scholar 

  • Kondepudy, R., & Healey, G. (1994). Using moment invariants to analyze 3-D color textures. In Proc. of IEEE international conference on image processing.

    Google Scholar 

  • Kosecka, J., & Zhang, W. (2005). Extraction, matching, and pose recovery based on dominant rectangular structures. CVGIP. Image Understanding, 100(3), 274–293.

    Google Scholar 

  • Lee, S., & Liu, Y. (2010). Skewed rotation symmetry group detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(9), 1659–1672.

    Article  Google Scholar 

  • Leung, T., & Malik, J. (2001). Representing and recognizing the visual appearance of materials using three-dimensional textons. International Journal of Computer Vision, 43(1), 29–44.

    Article  MATH  Google Scholar 

  • Levina, E., & Bickel, P. J. (2006). Texture synthesis and non-parametric resampling of random fields. Annals of Statistics, 34(4), 1751–1773.

    Article  MathSciNet  Google Scholar 

  • Lin, Z., Chen, M., Wu, L., & Ma, Y. (2009). The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices (UIUC Technical Report UILU-ENG-09-2215).

  • Liu, R., Lin, Z., Wei, S., & Su, Z. (2011). Solving principal component pursuit in linear time via l 1 filtering (Preprint).

  • Liu, Y., & Collins, R. (2001). Skewed symmetry groups. In Proc. of IEEE conference on computer vision and pattern recognition.

    Google Scholar 

  • Liu, Y., Collins, R., & Tsin, Y. (2004). A computational model for periodic pattern perception based on Frieze and wallpaper groups. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(3), 354–371.

    Article  Google Scholar 

  • Liu, Y., Hel-Or, H., Kaplan, C. S., & Gool, L. V. (2010). Computational symmetry in computer vision and computer graphics. Foundations and Trends in Computer Graphics and Vision, 5(1–2), 1–195.

    MATH  Google Scholar 

  • Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60, 91–110.

    Article  Google Scholar 

  • Ma, Y., Soatto, S., Kosecka, J., & Sastry, S. (2004). An invitation to 3D vision. Berlin: Springer.

    Google Scholar 

  • Madiraju, S., & Liu, C. (1994). Rotation invariant texture classification using covariance. In Proc. of IEEE international conference on image processing.

    Google Scholar 

  • Mikolajczyk, K., & Schmid, C. (2004). Scale and affine invariant interest point descriptors. International Journal of Computer Vision, 60(1), 63–86.

    Article  Google Scholar 

  • Mikolajczyk, K., & Schmid, C. (2005). A performance evaluation of local descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27, 1615–1630.

    Article  Google Scholar 

  • Mobahi, H., Zhou, Z., Yang, A., & Ma, Y. (2011). Holistic 3D reconstruction of urban structures from low-rank textures. In Third international IEEE workshop on 3D representation and recognition.

    Google Scholar 

  • Morel, J. M., & Yu, G. (2009). ASIFT: A new framework for fully affine invariant image comparison. SIAM Journal on Imaging Sciences, 2(2), 438–469.

    Article  MathSciNet  MATH  Google Scholar 

  • Park, M., Lee, S., Chen, P., Kashyap, S., Butt, A., & Liu, Y. (2008). Performance evaluation of state-of-the-art discrete symmetry detection algorithms. In Proc. of IEEE conference on computer vision and pattern recognition.

    Google Scholar 

  • Park, M., Brocklehurst, K., Collins, R., & Liu, Y. (2010). Translation-symmetry-based perceptual grouping with applications to urban scenes. In Asian conference on computer vision.

    Google Scholar 

  • Peng, Y., Ganesh, A., Wright, J., Xu, W., & Ma, Y. (2010). RASL: Robust alignment by sparse and low-rank decomposition for linearly correlated images. In Proc. of IEEE conference on computer vision and pattern recognition.

    Google Scholar 

  • Peng, Y., Ganesh, A., Wright, J., Xu, W., & Ma, Y. (2011). RASL: Robust alignment by sparse and low-rank decomposition for linearly correlated images. IEEE Transactions on Pattern Analysis and Machine Intelligence, to appear.

  • Recht, B., Fazel, M., & Parillo, P. (2010). Guaranteed minimum rank solution of matrix equations via nuclear norm minimization. SIAM Review, 52(3), 471–501.

    Article  MathSciNet  MATH  Google Scholar 

  • Schindler, G., Krishnamurthy, P., Lublinerman, R., Liu, Y., & Dellaert, F. (2008). Detecting and matching repeated patterns for automatic geo-tagging in urban environments. In Proc. of IEEE conference on computer vision and pattern recognition.

    Google Scholar 

  • Sundaramoorthi, G., Petersen, P., Varadarajan, V. S., & Soatto, S. (2009). On the set of images modulo viewpoint and contrast changes. In Proc. of IEEE conference on computer vision and pattern recognition.

    Google Scholar 

  • Toh, K., & Yun, S. (2010). An accelerated proximal gradient algorithm for nuclear norm regularized least squares problems. Pacific Journal of Optimization, 6, 615–640.

    MathSciNet  MATH  Google Scholar 

  • Winder, S., & Brown, M. (2007). Learning local image descriptor. In Proc. of IEEE conference on computer vision and pattern recognition.

    Google Scholar 

  • Wu, W. R., & Wei, S. C. (1996). Rotation and gray-scale transform-invariant texture classification using spiral resampling, subband decomposition and hidden Markov model. IEEE Transactions on Image Processing, 5, 1423–1434.

    Article  Google Scholar 

  • Yang, A., Huang, K., Rao, S., & Ma, Y. (2005). Symmetry-based 3-D reconstruction from perspective images. Computer Vision and Image Understanding, 99(2), 210–240.

    Article  Google Scholar 

  • Yuan, X., & Tao, M. (2011). Recovering low-rank and sparse components of matrices from incomplete and noisy observations. SIAM Journal on Optimization, 21(1), 57–81.

    Article  MathSciNet  MATH  Google Scholar 

  • Zhang, Z., Liang, X., & Ma, Y. (2011a). Unwrapping low-rank textures on generalized cylindrical surfaces. In Proc. of IEEE international conference on computer vision.

    Google Scholar 

  • Zhang, Z., Matsushita, Y., & Ma, Y. (2011b). Camera calibration with lens distortion from low-rank textures. In Proc. of IEEE conference on computer vision and pattern recognition.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Technology, Tsinghua University, Beijing, China

    Zhengdong Zhang

  2. Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, Urbana, IL, USA

    Arvind Ganesh & Yi Ma

  3. Institute for Advanced Study, Tsinghua University, Beijing, China

    Xiao Liang

  4. Visual Computing Group, Microsoft Research Asia, Beijing, China

    Yi Ma

Authors
  1. Zhengdong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arvind Ganesh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiao Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yi Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Arvind Ganesh.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zhang, Z., Ganesh, A., Liang, X. et al. TILT: Transform Invariant Low-Rank Textures. Int J Comput Vis 99, 1–24 (2012). https://doi.org/10.1007/s11263-012-0515-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-012-0515-x

Keywords

Search

Navigation