ABSTRACT
This paper presents a person search system by uncertain attributes. Attribute-based person search aims at finding person images that are the best matched with a set of attributes specified by a user as a query. The specified query attributes are inherently uncertain due to many factors such as the difficulty of retrieving characteristics of a target person from brain-memory and environmental variations like light and viewpoint. Also, existing attribute recognition techniques typically extract confidence scores along with attributes. Most of state-of-art approaches for attribute-based person search ignore the confidence scores or simply use a threshold to filter out attributes with low confidence scores. Moreover, they do not consider the uncertainty of query attributes. In this work, we resolve this uncertainty by enabling users to specify a level of confidence with each query attribute and consider uncertainty in both query attributes and attributes extracted from person images. We define a novel matching score to measure the degree of a person matching with query attribute conditions by leveraging the knowledge of probabilistic databases. Furthermore, we propose a novel definition of Critical Point of Confidence and compute it for each query attribute to show the impact of confidence levels on rankings of results. We develop a web-based demonstration system and show its effectiveness using real-world surveillance videos.
Supplemental Material
- Alex Bewley, Zongyuan Ge, Lionel Ott, Fabio Ramos, and Ben Upcroft. 2016. Simple Online and Realtime Tracking. In Proc. IEEE International Conference on Image Processing (ICIP).Google ScholarCross Ref
- Alexey Bochkovskiy, Chien-Yao Wang, and Hong-Yuan Mark Liao. 2020. YOLOv4: Optimal Speed and Accuracy of Object Detection. ArXiv abs/2004.10934 (2020).Google Scholar
- Yu-Tong Cao, Jingya Wang, and Dacheng Tao. 2020. Symbiotic Adversarial Learning for Attribute-Based Person Search. In Proc. European Conference on Computer Vision (ECCV).Google ScholarDigital Library
- Di Chen, Shanshan Zhang, Jian Yang, and Bernt Schiele. 2020. Norm-Aware Embedding for Efficient Person Search. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarCross Ref
- Mark de Berg, Otfried Cheong, Marc J. van Kreveld, and Mark H. Overmars. 2008. Computational Geometry: Algorithms and Applications, 3rd Edition. Springer.Google ScholarDigital Library
- Qi Dong, Xiatian Zhu, and Shaogang Gong. 2019. Person Search by Text Attribute Query As Zero-Shot Learning. In Proc. IEEE/CVF International Conference on Computer Vision (ICCV).Google ScholarCross Ref
- Tingting Dong, Shoji Nishimura, and Jianquan Liu. 2019. Refining Image Search Results using Multiple Attributes. In Proc. IEEE International Conference on Big Data (Big Data).Google ScholarCross Ref
- Xiao Han, Sen He, Li Zhang, and Tao Xiang. 2021. Text-Based Person Search with Limited Data. CoRR abs/2110.10807 (2021). arXiv:2110.10807Google Scholar
- Boseung Jeong, Jicheol Park, and Suha Kwak. 2021. ASMR: Learning Attribute-Based Person Search With Adaptive Semantic Margin Regularizer. In Proc. IEEE/CVF International Conference on Computer Vision (ICCV).Google ScholarCross Ref
- Xu Lan, Xiatian Zhu, and Shaogang Gong. 2018. Person Search by Multi-Scale Matching. In Proc. European Conference on Computer Vision (ECCV).Google ScholarDigital Library
- Dangwei Li, Xiaotang Chen, and Kaiqi Huang. 2015. Multi-Attribute Learning for Pedestrian Attribute Recognition in Surveillance Scenarios. In Proc. IEEE Asian Conference on Pattern Recognition (ACPR).Google ScholarCross Ref
- Shuang Li, Tong Xiao, Hongsheng Li, Bolei Zhou, Dayu Yue, and Xiaogang Wang. 2017. Person Search with Natural Language Description. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarCross Ref
- Bharti Munjal, Sikandar Amin, Federico Tombari, and Fabio Galasso. 2019. Query-Guided End-To-End Person Search. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarCross Ref
- Joseph Redmon, Santosh Kumar Divvala, Ross B. Girshick, and Ali Farhadi. 2016. You Only Look Once: Unified, Real-Time Object Detection. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarCross Ref
- Dan Suciu. 2018. Probabilistic Databases. In Encyclopedia of Database Systems, Second Edition, Ling Liu and M. Tamer Özsu (Eds.). Springer.Google Scholar
- Chufeng Tang, Lu Sheng, Zhaoxiang Zhang, and Xiaolin Hu. 2019. Improving Pedestrian Attribute Recognition With Weakly-Supervised Multi-Scale Attribute-Specific Localization. In Proc. IEEE/CVF International Conference on Computer Vision (ICCV).Google ScholarCross Ref
- Xiao Wang, Shaofei Zheng, Rui Yang, Aihua Zheng, Zhe Chen, Jin Tang, and Bin Luo. 2021. Pedestrian Attribute Recognition: A Survey. Pattern Recognition (2021), 108--220.Google Scholar
- Yichao Yan, Jinpeng Li, Jie Qin, Song Bai, Shengcai Liao, Li Liu, Fan Zhu, and Ling Shao. 2021. Anchor-Free Person Search. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarCross Ref
Index Terms
- Person Search by Uncertain Attributes
Recommendations
Visual query attributes suggestion
MM '12: Proceedings of the 20th ACM international conference on MultimediaQuery suggestion is an effective solution to help users deliver their search intent. While many query suggestion approaches have been proposed for test-based image retrieval with query-by-keywords, query suggestion for content-based image retrieval (...
Person resolution in person search results: WebHawk
CIKM '05: Proceedings of the 14th ACM international conference on Information and knowledge managementFinding information about people on the Web using a search engine is difficult because there is a many-to-many mapping between person names and specific persons (i.e. referents). This paper describes a person resolution system, called WebHawk. Given a ...
Supporting personalized ranking over categorical attributes
This paper studies how to enable an effective ranked retrieval over data with categorical attributes, in particular, by supporting personalized ranked retrieval of highly relevant data. While ranked retrieval has been actively studied lately, existing ...
Comments