Skip to main content
SpringerLink
Log in
Book cover

European Conference on Computer Vision

ECCV 2022: Computer Vision – ECCV 2022 pp 93–109Cite as

Optimal Transport for Label-Efficient Visible-Infrared Person Re-Identification

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13684))

Abstract

Visible-infrared person re-identification (VI-ReID) has been a key enabler for night intelligent monitoring system. However, the extensive laboring efforts significantly limit its applications. In this paper, we raise a new label-efficient training pipeline for VI-ReID. Our observation is: RGB ReID datasets have rich annotation information and annotating infrared images is expensive due to the lack of color information. In our approach, it includes two key steps: 1) We utilize the standard unsupervised domain adaptation technique to generate the pseudo labels for visible subset with the help of well-annotated RGB datasets; 2) We propose an optimal-transport strategy trying to assign pseudo labels from visible to infrared modality. In our framework, each infrared sample owns a label assignment choice, and each pseudo label requires unallocated images. By introducing uniform sample-wise and label-wise prior, we achieve a desirable assignment plan that allows us to find matched visible and infrared samples, and thereby facilitates cross-modality learning. Besides, a prediction alignment loss is designed to eliminate the negative effects brought by the incorrect pseudo labels. Extensive experimental results on benchmarks demonstrate the effectiveness of our approach. Code will be released at https://github.com/wjm-wjm/OTLA-ReID.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Asano, Y.M., Rupprecht, C., Vedaldi, A.: Self-labelling via simultaneous clustering and representation learning. In: ICLR (2020)

    Google Scholar 

  2. Chen, Y., Wan, L., Li, Z., Jing, Q., Sun, Z.: Neural feature search for RGB-infrared person re-identification. In: CVPR, pp. 587–597 (2021)

    Google Scholar 

  3. Choi, S., Lee, S., Kim, Y., Kim, T., Kim, C.: Hi-CMD: hierarchical cross-modality disentanglement for visible-infrared person re-identification. In: CVPR, pp. 10257–10266 (2020)

    Google Scholar 

  4. Cuturi, M.: Sinkhorn distances: lightspeed computation of optimal transport. In: NIPS, vol. 26, pp. 2292–2300 (2013)

    Google Scholar 

  5. Dai, P., Ji, R., Wang, H., Wu, Q., Huang, Y.: Cross-modality person re-identification with generative adversarial training. In: IJCAI, vol. 1, p. 6 (2018)

    Google Scholar 

  6. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: CVPR, pp. 248–255. IEEE (2009)

    Google Scholar 

  7. Deng, W., Zheng, L., Ye, Q., Kang, G., Yang, Y., Jiao, J.: Image-image domain adaptation with preserved self-similarity and domain-dissimilarity for person re-identification. In: CVPR, pp. 994–1003 (2018)

    Google Scholar 

  8. Ding, Y., Fan, H., Xu, M., Yang, Y.: Adaptive exploration for unsupervised person re-identification. TOMM 16(1), 1–19 (2020)

    Article  Google Scholar 

  9. Fu, Y., Wei, Y., Wang, G., Zhou, Y., Shi, H., Huang, T.S.: Self-similarity grouping: a simple unsupervised cross domain adaptation approach for person re-identification. In: CVPR, pp. 6112–6121 (2019)

    Google Scholar 

  10. Ganin, Y., Lempitsky, V.: Unsupervised domain adaptation by backpropagation. In: ICML, pp. 1180–1189. PMLR (2015)

    Google Scholar 

  11. Ge, Y., Chen, D., Li, H.: Mutual mean-teaching: pseudo label refinery for unsupervised domain adaptation on person re-identification. In: ICLR (2020)

    Google Scholar 

  12. Ge, Y., Zhu, F., Chen, D., Zhao, R., Li, H.: Self-paced contrastive learning with hybrid memory for domain adaptive object re-id. In: NIPS (2020)

    Google Scholar 

  13. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2016)

    Google Scholar 

  14. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: ICML, pp. 448–456. PMLR (2015)

    Google Scholar 

  15. Li, D., Wei, X., Hong, X., Gong, Y.: Infrared-visible cross-modal person re-identification with an x modality. In: AAAI, vol. 34, pp. 4610–4617 (2020)

    Google Scholar 

  16. Liang, W., Wang, G., Lai, J., Xie, X.: Homogeneous-to-heterogeneous: unsupervised learning for RGB-infrared person re-identification. TIP 30, 6392–6407 (2021)

    MathSciNet  Google Scholar 

  17. Lin, Y., Dong, X., Zheng, L., Yan, Y., Yang, Y.: A bottom-up clustering approach to unsupervised person re-identification. In: AAAI, vol. 33, pp. 8738–8745 (2019)

    Google Scholar 

  18. Liu, H., Tan, X., Zhou, X.: Parameter sharing exploration and hetero-center triplet loss for visible-thermal person re-identification. TMM 23, 4414–4425 (2020)

    Google Scholar 

  19. Lu, Y., et al.: Cross-modality person re-identification with shared-specific feature transfer. In: CVPR, pp. 13379–13389 (2020)

    Google Scholar 

  20. Luo, H., et al.: A strong baseline and batch normalization neck for deep person re-identification. TMM 22(10), 2597–2609 (2019)

    Google Scholar 

  21. Mekhazni, D., Bhuiyan, A., Ekladious, G., Granger, E.: Unsupervised domain adaptation in the dissimilarity space for person re-identification. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12372, pp. 159–174. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58583-9_10

    Chapter  Google Scholar 

  22. Nguyen, D.T., Hong, H.G., Kim, K.W., Park, K.R.: Person recognition system based on a combination of body images from visible light and thermal cameras. Sensors 17(3), 605 (2017)

    Article  Google Scholar 

  23. Pan, S.J., Yang, Q.: A survey on transfer learning. TKDE 22(10), 1345–1359 (2009)

    Google Scholar 

  24. Park, H., Lee, S., Lee, J., Ham, B.: Learning by aligning: visible-infrared person re-identification using cross-modal correspondences. In: CVPR, pp. 12046–12055 (2021)

    Google Scholar 

  25. Ristani, E., Solera, F., Zou, R., Cucchiara, R., Tomasi, C.: Performance measures and a data set for multi-target, multi-camera tracking. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9914, pp. 17–35. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-48881-3_2

    Chapter  Google Scholar 

  26. Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: CVPR, pp. 815–823 (2015)

    Google Scholar 

  27. Tian, X., Zhang, Z., Lin, S., Qu, Y., Xie, Y., Ma, L.: Farewell to mutual information: variational distillation for cross-modal person re-identification. In: CVPR, pp. 1522–1531 (2021)

    Google Scholar 

  28. Wang, D., Zhang, S.: Unsupervised person re-identification via multi-label classification. In: CVPR, pp. 10981–10990 (2020)

    Google Scholar 

  29. Wang, G.A., et al.: Cross-modality paired-images generation for RGB-infrared person re-identification. In: AAAI, vol. 34, pp. 12144–12151 (2020)

    Google Scholar 

  30. Wang, G., Zhang, T., Cheng, J., Liu, S., Yang, Y., Hou, Z.: RGB-infrared cross-modality person re-identification via joint pixel and feature alignment. In: CVPR, pp. 3623–3632 (2019)

    Google Scholar 

  31. Wang, Z., Wang, Z., Zheng, Y., Chuang, Y.Y., Satoh, S.: Learning to reduce dual-level discrepancy for infrared-visible person re-identification. In: CVPR, pp. 618–626 (2019)

    Google Scholar 

  32. Wei, L., Zhang, S., Gao, W., Tian, Q.: Person transfer GAN to bridge domain gap for person re-identification. In: CVPR, pp. 79–88 (2018)

    Google Scholar 

  33. Wu, A., Zheng, W.S., Yu, H.X., Gong, S., Lai, J.: RGB-infrared cross-modality person re-identification. In: ICCV (2017)

    Google Scholar 

  34. Wu, Q., et al.: Discover cross-modality nuances for visible-infrared person re-identification. In: CVPR, pp. 4330–4339 (2021)

    Google Scholar 

  35. Yang, F., et al.: Joint noise-tolerant learning and meta camera shift adaptation for unsupervised person re-identification. In: CVPR, pp. 4855–4864 (2021)

    Google Scholar 

  36. Ye, M., Ruan, W., Du, B., Shou, M.Z.: Channel augmented joint learning for visible-infrared recognition. In: ICCV, pp. 13567–13576 (2021)

    Google Scholar 

  37. Ye, M., Shen, J., J. Crandall, D., Shao, L., Luo, J.: Dynamic dual-attentive aggregation learning for visible-infrared person re-identification. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12362, pp. 229–247. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58520-4_14

    Chapter  Google Scholar 

  38. Ye, M., Shen, J., Lin, G., Xiang, T., Shao, L., Hoi, S.C.: Deep learning for person re-identification: a survey and outlook. TPAMI 44, 2872–2893 (2021)

    Article  Google Scholar 

  39. Ye, M., Wang, Z., Lan, X., Yuen, P.C.: Visible thermal person re-identification via dual-constrained top-ranking. In: IJCAI, vol. 1, p. 2 (2018)

    Google Scholar 

  40. Zhai, Y., Ye, Q., Lu, S., Jia, M., Ji, R., Tian, Y.: Multiple expert brainstorming for domain adaptive person re-identification. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12352, pp. 594–611. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58571-6_35

    Chapter  Google Scholar 

  41. Zhang, Z., Xie, Y., Li, D., Zhang, W., Tian, Q.: Learning to align via wasserstein for person re-identification. TIP 29, 7104–7116 (2020)

    MATH  Google Scholar 

  42. Zhang, Z., Xie, Y., Zhang, W., Tang, Y., Tian, Q.: Tensor multi-task learning for person re-identification. TIP 29, 2463–2477 (2019)

    MathSciNet  MATH  Google Scholar 

  43. Zheng, K., Liu, W., He, L., Mei, T., Luo, J., Zha, Z.J.: Group-aware label transfer for domain adaptive person re-identification. In: CVPR, pp. 5310–5319 (2021)

    Google Scholar 

  44. Zheng, L., Shen, L., Tian, L., Wang, S., Wang, J., Tian, Q.: Scalable person re-identification: a benchmark. In: ICCV, pp. 1116–1124 (2015)

    Google Scholar 

  45. Zheng, Y., et al.: Online pseudo label generation by hierarchical cluster dynamics for adaptive person re-identification. In: CVPR, pp. 8371–8381 (2021)

    Google Scholar 

  46. Zhong, Z., Zheng, L., Li, S., Yang, Y.: Generalizing a person retrieval model hetero-and homogeneously. In: ECCV, pp. 172–188 (2018)

    Google Scholar 

  47. Zhong, Z., Zheng, L., Luo, Z., Li, S., Yang, Y.: Invariance matters: exemplar memory for domain adaptive person re-identification. In: CVPR, pp. 598–607 (2019)

    Google Scholar 

  48. Zhong, Z., Zheng, L., Luo, Z., Li, S., Yang, Y.: Learning to adapt invariance in memory for person re-identification. TPAMI 43(8), 2723–2738 (2020)

    Google Scholar 

Download references

Acknowledgements

This work is supported by grants from the National Key Research and Development Program of China (2021ZD0111000), National Natural Science Foundation of China No.62106075, 62176092, Shanghai Science and Technology Commission No.21511100700, Natural Science Foundation of Shanghai (20ZR1417700), CAAI-Huawei MindSpore Open Fund.

Author information

Authors and Affiliations

  1. East China Normal University, Shanghai, China

    Jiangming Wang, Zhizhong Zhang & Yuan Xie

  2. Shanghai Development Center of Computer Software Technology, Shanghai, China

    Mingang Chen

  3. ZheJiang Lab, Hangzhou, China

    Yi Zhang

  4. Huawei Technologies, Hangzhou, China

    Cong Wang

  5. Shanghai Jiao Tong University, Shanghai, China

    Bin Sheng

  6. Xiamen University, Fujian, China

    Yanyun Qu

Authors
  1. Jiangming Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhizhong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mingang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Cong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Bin Sheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yanyun Qu
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Yuan Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Zhizhong Zhang or Yuan Xie .

Editor information

Editors and Affiliations

  1. Tel Aviv University, Tel Aviv, Israel

    Shai Avidan

  2. University College London, London, UK

    Gabriel Brostow

  3. Google AI, Accra, Ghana

    Moustapha Cissé

  4. University of Catania, Catania, Italy

    Giovanni Maria Farinella

  5. Facebook (United States), Menlo Park, CA, USA

    Tal Hassner

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 5484 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, J. et al. (2022). Optimal Transport for Label-Efficient Visible-Infrared Person Re-Identification. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13684. Springer, Cham. https://doi.org/10.1007/978-3-031-20053-3_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-20053-3_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-20052-6

  • Online ISBN: 978-3-031-20053-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation