Rao Muhammad Anwer
ABSTRACT
Recognizing materials and textures in realistic imaging conditions is a challenging computer vision problem. For many years, local features based orderless representations were a dominant approach for texture recognition. Recently deep local features, extracted from the intermediate layers of a Convolutional Neural Network (CNN), are used as filter banks. These dense local descriptors from a deep model, when encoded with Fisher Vectors, have shown to provide excellent results for texture recognition. The CNN models, employed in such approaches, take RGB patches as input and train on a large amount of labeled images. We show that CNN models, which we call TEX-Nets, trained using mapped coded images with explicit texture information provide complementary information to the standard deep models trained on RGB patches. We further investigate two deep architectures, namely early and late fusion, to combine the texture and color information. Experiments on benchmark texture datasets clearly demonstrate that TEX-Nets provide complementary information to standard RGB deep network. Our approach provides a large gain of 4.8%, 3.5%, 2.6% and 4.1% respectively in accuracy on the DTD, KTH-TIPS-2a, KTH-TIPS-2b and Texture-10 datasets, compared to the standard RGB network of the same architecture. Further, our final combination leads to consistent improvements over the state-of-the-art on all four datasets.
- Timo Ahonen, Jiri Matas, Chu He, and Matti Pietikainen. 2009. Rotation Invariant Image Description with Local Binary Pattern Histogram Fourier Features. In SCIA. Google Scholar
Digital Library
- Joan Bruna and Stephane Mallat. 2013. Invariant Scattering Convolution Networks. TSE 35, 8 (2013), 1872--1886. Google ScholarDigital Library
- Barbara Caputo, Eric Hayman, and P Mallikarjuna. 2005. Class-Specific Material Categorisation. In ICCV. Google ScholarDigital Library
- Tsung-Han Chan, Kui Jia, Shenghua Gao, and Yi Ma. 2014. "PCANet: A Simple Deep Learning Baseline for Image Classification" TIP 24, 12 (2014), 5017--5032.Google ScholarDigital Library
- Ken Chatfield, Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2014. Return of the Devil in the Details: Delving Deep into Convolutional Nets. In BMVC.Google Scholar
- Jie Chen, Shiguang Shan, Chu He, Guoying Zhao, Matti Pietikainen, Xilin Chen, and Wen Gao. 2010. WLD: A Robust Local Image Descriptor. PAMI 32, 9 (2010), 1705--1720. Google ScholarDigital Library
- Guilhem Cheron, Ivan Laptev, and Cordelia Schmid. 2015. P-CNN: Pose-Based CNN Features for Action Recognition. In ICCV. Google ScholarDigital Library
- Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. 2014. Describing Textures in the Wild. In CVPR. Google ScholarDigital Library
- Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, and Andrea Vedaldi. 2016. Deep Filter Banks for Texture Recognition, Description, and Segmentation. IJCV 118, 1 (2016), 65--94. Google ScholarDigital Library
- G. Csurka, C. Bray, C. Dance, and L. Fan. 2004. Visual categorization with bags of keypoints. In Workshop on Statistical Learning in Computer Vision, ECCV.Google Scholar
- Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei-Fei Li. 2009. Ima- geNet: A large-scale hierarchical image database. In Proc. CVPR.Google ScholarCross Ref
- Andreas Eitel, Jost Tobias Springenberg, Luciano Spinello, Martin Riedmiller, and Wolfram Burgard. 2015. Multimodal Deep Learning for Robust RGB-D Object Recognition. In IROS.Google Scholar
- Abdolhossein Fathi and Ahmad Nilchi. 2012. Noise tolerant local binary pattern operator for efficient texture analysis. PRL 33, 9 (2012), 1093--1100. Google ScholarDigital Library
- Yimo Guo, Guoying Zhao, and Matti Pietikainen. 2012. Discriminative features for texture description. PR 45, 10 (2012), 3834--3843. Google ScholarDigital Library
- Zhenhua Guo, Lei Zhang, and David Zhang. 2010. A Completed Modeling of Local Binary Pattern Operator for Texture Classification. TIP 19, 6 (2010), 1657--1663. Google ScholarDigital Library
- Zhenhua Guo, Lei Zhang, and David Zhang. 2010. Rotation invariant texture classification using LBP variance (LBPV) with global matching. PR 43, 3 (2010), 706--719. Google ScholarDigital Library
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In CVPR.Google Scholar
- Fahad Shahbaz Khan, Rao Muhammad Anwer, Joost van de Weijer, Andrew Bagdanov, Antonio Lopez, and Michael Felsberg. 2013. Coloring Action Recognition in Still Images. IJCV 105, 3 (2013), 205--221. Google ScholarDigital Library
- Fahad Shahbaz Khan, Rao Muhammad Anwer, Joost van de Weijer, Andrew D. Bagdanov, Maria Vanrell, and Antonio M. Lopez. 2012. Color attributes for object detection. In CVPR.Google Scholar
- Fahad Shahbaz Khan, Rao Muhammad Anwer, Joost van de Weijer, Michael Felsberg, and Jorma Laaksonen. 2015. Compact color-texture description for texture classification. PRL 51 (2015), 16--22. Google ScholarDigital Library
- Fahad Shahbaz Khan, Joost van de Weijer, Sadiq Ali, and Michael Felsberg. 2013. Evaluating the Impact of Color on Texture Recognition. In CAIP.Google Scholar
- Fahad Shahbaz Khan, Joost van de Weijer, Rao Muhammad Anwer, Andrew Bagdanov, Michael Felsberg, and Jorma Laaksonen. 2016. Scale Coding Bag of Deep Features for Human Attribute and Action Recognition. arXiv preprint arXiv:1612.04884 (2016).Google Scholar
- Fahad Shahbaz Khan, Joost van de Weijer, Rao Muhammad Anwer, Michael Felsberg, and Carlo Gatta. 2014. Semantic Pyramids for Gender and Action Recognition. TIP 23, 8 (2014), 3633--3645.Google ScholarCross Ref
- Fahad Shahbaz Khan, Joost van de Weijer, and Maria Vanrell. 2009. Top-Down Color Attention for Object Recognition. In ICCV.Google Scholar
- Fahad Shahbaz Khan, Joost van de Weijer, and Maria Vanrell. 2012. Modulating Shape Features by Color Attention for Object Recognition. IJCV 98, 1 (2012), 49--64. Google ScholarDigital Library
- Yann LeCun, Bernhard Boser, John Denker, Donnie Henderson, R Howard, Wayne Hubbard, and Lawrence Jackel. 1989. Handwritten Digit Recognition with a Back-Propagation Network. In NIPS. Google ScholarDigital Library
- Seung Ho Lee, Jae Young Choi, Yong Man Ro, and Konstantinos Plataniotis. 2012. Local Color Vector Binary Patterns From Multichannel Face Images for Face Recognition. TIP 21, 4 (2012), 2347--2353. Google ScholarDigital Library
- Thomas Leung and Jitendra Malik. 1996. Detecting, localizing and grouping repeated scene elements from an image. In ECCV. Google ScholarDigital Library
- Thomas Leung and Jitendra Malik. 2001. Representing and Recognizing the Visual Appearance of Materials using Three-dimensional Textons. IJCV 43, 1 (2001), 29--44. Google ScholarDigital Library
- Gil Levi and Tal Hassner. 2015. Emotion Recognition in the Wild via Convolu- tional Neural Networks and Mapped Binary Patterns. In ICMI. Google ScholarDigital Library
- Li Liu, Paul Fieguth, Yulan Guo, Xiaogang Wang, and Matti Pietikainen. 2017. Local binary features for texture classification: Taxonomy and experimental study. PR 62 (2017), 135--160. Google ScholarDigital Library
- Li Liu, Paul Fieguth, Xiaogang Wang, Matti Pietikainen, and Dewen Hu. 2016. Evaluation of LBP and Deep Texture Descriptors with a New Robustness Bench- mark. In ECCV.Google Scholar
- Li Liu, Songyang Lao, Paul Fieguth, Yulan Guo, Xiaogang Wang, and Matti Pietikainen. 2016. Median Robust Extended Local Binary Pattern for Texture Classification. TIP 25, 3 (2016), 1368--1381. Google ScholarDigital Library
- Lingqiao Liu, Chunhua Shen, and Anton van den Hengel. 2015. The Treasure beneath Convolutional Layers: Cross-convolutional-layer Pooling for Image Classification. In CVPR.Google Scholar
- Li Liu, Lingjun Zhao, Yunli Long, and Paul Fieguth. 2012. Extended local binary patterns for texture classification. IMAVIS 30, 2 (2012), 86--99. Google ScholarDigital Library
- Li Liu, Lingjun Zhao, Yunli Long, Gangyao Kuang, and Paul Fieguth. 2012. Extended local binary patterns for texture classification. IVC 30, 2 (2012), 86--99. Google ScholarDigital Library
- Topi Maenpaa and Matti Pietikainen. 2004. Classification with color and texture: jointly or separately? PR 37, 8 (2004), 1629--1640.Google Scholar
- Timo Ojala, Matti Pietikainen, and Topi Maenpaa. 2002. Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns. PAMI 24, 7 (2002), 971--987. Google ScholarDigital Library
- Ville Ojansivu, Esa Rahtu, and Janne Heikkila. 2009. Rotation Invariant Local Phase Quantization for Blur Insensitive Texture Analysis. In ICPR.Google Scholar
- Florent Perronnin and Christopher Dance. 2007. Fisher Kernels on Visual Vocab- ularies for Image Categorization. In CVPR.Google Scholar
- Gaurav Sharma, Sibt ul Hussain, and Frederic Jurie. 2012. Local Higher-Order Statistics (LHS) for Texture Categorization and Facial Analysis. In ECCV. Google ScholarDigital Library
- Karen Simonyan and Andrew Zisserman. 2014. Two-Stream Convolutional Networks for Action Recognition in Videos. In NIPS. Google ScholarDigital Library
- Karen Simonyan and Andrew Zisserman. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. In ICLR.Google Scholar
- Milan Sulc and Jiri Matas. 2014. Fast Features Invariant to Rotation and Scale of Texture. In ECCV Workshops.Google Scholar
- Xiaoyang Tan and Bill Triggs. 2007. Fusing Gabor and LBP Feature Sets for Kernel-Based Face Recognition. In AMFG. Google ScholarDigital Library
- Xiaoyang Tan and Bill Triggs. 2010. Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting Conditions. TIP 19, 9 (2010), 1635--1650. Google ScholarDigital Library
- Radu Timofte and Luc Van Gool. 2012. A Training-free Classification Framework for Textures, Writers, and Materials. In BMVC.Google Scholar
- Sibt ul Hussain and Bill Triggs. 2012. Visual Recognition Using Local Quantized Patterns. In ECCV.Google Scholar
- Xiaoyu Wang, Tony Han, and Shuicheng Yan. 2009. An HOG-LBP Human Detector with Partial Occlusion Handling. In ICCV.Google Scholar
- Matthew Zeiler and Rob Fergus. 2014. Visualizing and Understanding Convolutional Networks. In ECCV.Google Scholar
- Junge Zhang, Kaiqi Huang, Yinan Yu, and Tieniu Tan. 2011. Boosted local structured HOG-LBP for object localization. In CVPR. Google ScholarDigital Library
- Jun Zhang, Jimin Liang, and Heng Zhao. 2013. Local Energy Pattern for Texture Classification Using Self-Adaptive Quantization Thresholds. TIP 22, 1 (2013), 31--42. Google ScholarDigital Library
- J. Zhang, M. Marszalek, S. Lazebnik, and C. Schmid. 2007. Local features and kernels for classification of texture and object catergories: A Comprehensive Study. IJCV 73, 2 (2007), 213--218. Google ScholarDigital Library
- Jun Zhang, Heng Zhao, and Jimin Liang. 2013. Continuous rotation invariant local descriptors for texton dictionary-based texture classification. CVIU 117, 1 (2013), 56--75. Google ScholarDigital Library
- Guoying Zhao, Timo Ahonen, Jiri Matas, and Matti Pietikainen. 2012. Rotation- Invariant Image and Video Description With Local Binary Pattern Features. TIP 21, 4 (2012), 1465--1477 Google ScholarDigital Library
Index Terms
- TEX-Nets: Binary Patterns Encoded Convolutional Neural Networks for Texture Recognition
Recommendations
Description of interest regions with local binary patterns
This paper presents a novel method for interest region description. We adopted the idea that the appearance of an interest region can be well characterized by the distribution of its local features. The most well-known descriptor built on this idea is ...
Feature extraction based on co-occurrence of adjacent local binary patterns
PSIVT'11: Proceedings of the 5th Pacific Rim conference on Advances in Image and Video Technology - Volume Part IIIn this paper, we propose a new image feature based on spatial co-occurrence among micropatterns, where each micropattern is represented by a Local Binary Pattern (LBP). In conventional LBP-based features such as LBP histograms, all the LBPs of ...
Scale- and rotation-invariant texture description with improved local binary pattern features
Local Binary Pattern (LBP) is an effective image descriptor based on joint distribution of signed gray level differences. Simplicity, discriminative power, computational efficiency and robustness to illumination changes are main properties of LBP. ...
Comments