Document Type : Original Article
Authors
1 Faculty of Computer Engineering, Shahrood University of Technology, Shahrood, Iran
2 Institut Galilée, Université Sorbonne Paris Nord, Villetaneuse, France
Abstract
Video surveillance systems are widely used in the public and private sectors for maintaining security and healthcare purposes. Performance of surveillance systems directly depends on their accuracy in re-identification. There are three regions in a camera view, including person’s body, background, and possible carried object by the person. Background, in existing approaches, is either overlooked or treated like a person’s body in re-identification. In this paper, these three regions are considered in re-identification but with different importance. In our proposed technique, first, the input image is semantically segmented into the three regions using a deep semantic segmentation approach. Then, the effect of each region on characteristic features of people is tuned depending on the region’s importance in re-identification. The proposed technique, leveraging robust descriptors, such as the Gaussian of Gaussian (GOG) and Hierarchical Gaussian Descriptors (HGD), can enhance existing methods in dealing with the challenging issues such as partial occlusion caused by carried objects and background in re-identification. Experimental results on commonly used people re-identification datasets demonstrate effectiveness of the proposed technique in improving performance of existing re-identification methods.
Keywords
Main Subjects
- Hammoudi, K., Abu Taha, M., Benhabiles, H., Melkemi, M., Windal, F., El Assad, S., and Queudet, A., 2020. Image-Based Ciphering of Video Streams and Object Recognition for Urban and Vehicular Surveillance Services. In Fourth International Congress on Information and Communication Technology, pp 519–527. Doi: 10.1007/978-981-32-9343-4_42
- Goyal, A., Anandamurthy, S.B., Dash, P., Acharya, S., Bathla, D., Hicks, D., Bhan, A., and Ranjan, P., 2020. Automatic Border Surveillance Using Machine Learning in Remote Video Surveillance Systems. In Emerging Trends in Electrical, Communications, and Information Technologies, pp.751-760. Springer, Singapore. Doi:10.1007/978-981-13-8942-9_64
- Huang, Z., Liu, Y., Fang, Y., and Horn, B.K.P., 2018. Video-based Fall Detection for Seniors with Human Pose Estimation. In: 2018 4th International Conference on Universal Village (UV). IEEE, pp 1–4. Doi: 10.1109/UV.2018.8642130
- Gawande, U., Hajari, K., and Golhar, Y., 2020. Pedestrian detection and tracking in video surveillance system: issues, comprehensive review, and challenges. Recent Trends in Computational Intelligence, pp.1–24. London, United Kingdom: IntechOpen
- Li, W., Zhao, R., and Wang, X., 2013. Human Reidentification with Transferred Metric Learning. Asian conference on computer vision, Berlin, Heidelberg, pp: 31-44. Doi: 10.1007/978-3-642-37331-2_3
- Matsukawa, T., Okabe, T., Suzuki, E., and Sato, Y., 2016. Hierarchical Gaussian Descriptor for Person Re-identification. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp: 1363-1372. Doi: 10.1109/CVPR.2016.152
- Matsukawa, T., Okabe, T., Suzuki, E., and Sato, Y., 2020. Hierarchical Gaussian Descriptors with Application to Person Re-Identification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(9), pp.2179–2194. Doi: 10.1109/TPAMI.2019.2914686
- Li, P., Wang, Q., and Zhang, L., 2013. A Novel Earth Mover’s Distance Methodology for Image Matching with Gaussian Mixture Models. In: 2013 IEEE International Conference on Computer Vision. pp: 1689–1696. Doi: 10.1109/ICCV.2013.212.
- Liao, S., Hu, Y., Xiangyu Zhu, and Li, S.Z., 2015. Person re-identification by Local Maximal Occurrence representation and metric learning. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp: 2197-2206. Doi: 10.1109/CVPR.2015.7298832
- Liao, S., Zhao, G., Kellokumpu, V., Pietikainen, M., and Li, S.Z., 2010. Modeling pixel process with scale invariant local patterns for background subtraction in complex scenes. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE, pp. 2197-2206. Doi: 10.1109/CVPR.2015.7298832
- Vishwakarma, D.K., and Upadhyay, S., 2018. A Deep Structure of Person Re-Identification Using Multi-Level Gaussian Models. IEEE Transactions on Multi-Scale Computing Systems, 4(4), pp.513–521. Doi: 10.1109/TMSCS.2018.2870592
- Prates, R., and Schwartz, W.R., 2019. Kernel cross-view collaborative representation based classification for person re-identification. Journal of Visual Communication and Image Representation, 58, pp.304–315. Doi: 10.1016/j.jvcir.2018.12.003
- Prates, R., and Schwartz, W.R., 2019. Matching People Across Surveillance Cameras. In: Anais Estendidos da Conference on Graphics, Patterns and Images (SIBGRAPI). Sociedade Brasileira de Computação - SBC, pp. 84-90. Doi: 10.5753/sibgrapi.est.2019.8306
- Zhou, Q., Fan, H., Zheng, S., Su, H., Li, X., Wu, S., and Ling, H., 2018. Graph Correspondence Transfer for Person Re-Identification | Proceedings of the AAAI Conference on Artificial Intelligence. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1). Doi: 10.5555/3504035.3504966
- Zhou, Q., Fan, H., Yang, H., Su, H., Zheng, S., Wu, S., and Ling, H., 2021. Robust and Efficient Graph Correspondence Transfer for Person Re-Identification. IEEE Transactions on Image Processing, 30, pp.1623–1638. Doi: 10.1109/TIP.2019.2914575
- Cao, Z., Simon, T., Wei, S.-E., and Sheikh, Y., 2017. Realtime multi-person 2d pose estimation using part affinity fields. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 7291–7299.
- Fang, J., Zhang, R., and Jiang, F., 2019. Sample Specific Multi-Kernel Metric Learning for Person Re-identification. In: 2nd International Conference on Electrical and Electronic Engineering (EEE 2019). Atlantis Press, pp 226–241. Doi: 10.2991/eee-19.2019.38
- Liu, Y., Liu, S., and Wang, Z., 2015. Multi-focus image fusion with dense SIFT. Information Fusion, 23, pp.139–155. Doi: 10.1016/j.inffus.2014.05.004
- An, L., Kafai, M., Yang, S., and Bhanu, B., 2013. Reference-based person re-identification. In: 2013 10th IEEE International Conference on Advanced Video and Signal Based Surveillance. IEEE, pp 244–249. Doi: 10.1109/AVSS.2013.6636647
- Yang, Y., Yang, J., Yan, J., Liao, S., Yi, D., and Li, S.Z., 2014. Salient Color Names for Person Re-identification. pp 536–551. Doi: 10.1007/978-3-319-10590-1_35
- Ahmed, E., Jones, M., and Marks, T.K., 2015. An improved deep learning architecture for person re-identification. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 3908–3916. Doi: 10.1109/CVPR.2015.7299016
- Jia, J., Ruan, Q., Jin, Y., An, G., and Ge, S., 2020. View-specific subspace learning and re-ranking for semi-supervised person re-identification. Pattern Recognition, 108, pp.107568. Doi: 10.1016/j.patcog.2020.107568
- Zhao, C., Wang, X., Zuo, W., Shen, F., Shao, L., and Miao, D., 2020. Similarity learning with joint transfer constraints for person re-identification. Pattern Recognition, 97, pp.107014. Doi: 10.1016/j.patcog.2019.107014
- Xu, Y., Fang, X., Wu, J., Li, X., and Zhang, D., 2016. Discriminative Transfer Subspace Learning via Low-Rank and Sparse Representation. IEEE Transactions on Image Processing, 25(2), pp.850–863. Doi: 10.1109/TIP.2015.2510498
- Layne, R., Hospedales, T.M., and Gong, S., 2012. Towards Person Identification and Re-identification with Attributes. pp 402–412. . Doi: 10.1007/978-3-642-33863-2_40
- Zhao, R., Ouyang, W., and Wang, X., 2013. Unsupervised Salience Learning for Person Re-identification. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp 3586–3593. Doi: 10.1109/CVPR.2013.460
- Martinel, N., Micheloni, C., and Foresti, G.L., 2015. Kernelized Saliency-Based Person Re-Identification Through Multiple Metric Learning. IEEE Transactions on Image Processing, 24(12), pp.5645–5658. Doi: 10.1109/TIP.2015.2487048
- Harel, J., Koch, C., and Perona, P., 2006. Graph-based visual saliency. Advances in neural information processing systems, 19, pp. 545-552.
- Heller, K., Svore, K., Keromytis, A.D., and Stolfo, S., 2003. One class support vector machines for detecting anomalous windows registry accesses. In: ICDM Workshop on Data Mining for Computer Security. Doi: 10.7916/D85M6CFF
- Martinel, N., Micheloni, C., and Foresti, G.L., 2015. Saliency Weighted Features for Person Re-identification. pp 191–208. Doi: 10.1007/978-3-319-16199-0_14
- Sun, Y., Zheng, L., Deng, W., and Wang, S., 2017. SVDNet for Pedestrian Retrieval. In: 2017 IEEE International Conference on Computer Vision (ICCV). IEEE, pp 3820–3828. Doi: 10.1109/ICCV.2017.410
- Zheng, Z., Zheng, L., and Yang, Y., 2019. Pedestrian Alignment Network for Large-scale Person Re-Identification. IEEE Transactions on Circuits and Systems for Video Technology, 29(10), pp.3037–3045. Doi: 10.1109/TCSVT.2018.2873599
- Yu, Y., Liang, C., Ruan, W., and Jiang, L., 2020. Crowdsourcing-Based Ranking Aggregation for Person Re-Identification. In: ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, pp 1933–1937. Doi: 10.1109/ICASSP40776.2020.9053496
- Sun, Y., Zheng, L., Yang, Y., Tian, Q., and Wang, S., 2018. Beyond Part Models: Person Retrieval with Refined Part Pooling (and A Strong Convolutional Baseline). pp 501–518. Doi: 10.1007/978-3-030-01225-0_30
- He, K., Zhang, X., Ren, S., and Sun, J., 2016. Deep Residual Learning for Image Recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 770–778. Doi: 10.1109/CVPR.2016.90
- Chen, L., Yang, H., Xu, Q., and Gao, Z., 2021. Harmonious attention network for person re-identification via complementarity between groups and individuals. Neurocomputing, 453, pp.766–776. Doi: 10.1016/j.neucom.2020.07.118
- Mortezaie, Z., Hassanpour, H., and Beghdadi, A., 2021. A Color-Based Re-Ranking Process for People Re-Identification : Paper ID 21. In: 2021 9th European Workshop on Visual Information Processing (EUVIP). IEEE, pp 1–5. Doi: 10.1109/EUVIP50544.2021.9484056
- Parraga, C.A., and Akbarinia, A., 2016. NICE: A Computational Solution to Close the Gap from Colour Perception to Colour Categorization. PLOS ONE, 11(3), pp.e0149538. Doi: 10.1371/journal.pone.0149538
- Mortezaie, Z., Hassanpour, H., and Beghdadi, A., 2022. People re-identification under occlusion and crowded background. Multimedia Tools and Applications, pp.1–21. Doi: 10.1007/s11042-021-11868-y
- Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., and Adam, H., 2018. Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation. Proceedings of the European conference on computer vision (ECCV), pp: 801-818. Doi: 10.1007/978-3-030-01234-2_49
- Gray, D., and Tao, H., 2008. Viewpoint Invariant Pedestrian Recognition with an Ensemble of Localized Features. pp 262–275. Doi: 10.1007/978-3-540-88682-2_21
- Roth, P.M., Hirzer, M., Köstinger, M., Beleznai, C., and Bischof, H., 2014. Mahalanobis Distance Learning for Person Re-identification. In: Person Re-Identification. Springer London, London, pp 247–267. Doi: 10.1007/978-1-4471-6296-4_12
- Moghaddam, B., Jebara, T., and Pentland, A., 2000. Bayesian face recognition. Pattern Recognition, 33(11), pp.1771–1782. Doi: 10.1016/S0031-3203(99)00179-X
- Kostinger, M., Hirzer, M., Wohlhart, P., Roth, P.M., and Bischof, H., 2012. Large scale metric learning from equivalence constraints. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp 2288–2295. Doi: 10.1109/CVPR.2012.6247939
- Alipanahi, B., Biggs, M., and Ghodsi, A., 2008. Distance metric learning vs. fisher discriminant analysis. In: Proceedings of the 23rd national conference on Artificial intelligence. pp 598–603. Doi: 10.5555/1620163.1620164
- Zhang, K., Zhang, L., and Yang, M.-H., 2012. Real-Time Compressive Tracking. European conference on computer vision (ECCV), pp 864–877. Doi: 10.1007/978-3-642-33712-3_62
- Ojala, T., Pietikainen, M., and Maenpaa, T., 2002. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(7), pp.971–987. Doi: 10.1109/TPAMI.2002.1017623
- mortezaie, zahra, and Hassanpour, H., 2019. A survey on age invariant face recognition methods. Jordanian Journal of Computers and Information Technology, 5(2), pp.87–96. Doi: 10.5455/jjcit.71-1554841475
- Li, W., Zhao, R., Xiao, T., and Wang, X., 2014. DeepReID: Deep Filter Pairing Neural Network for Person Re-identification. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp 152–159. Doi: 10.1109/CVPR.2014.27