Karim M. Ibrahim
ABSTRACT
The problem of multi-label classification with missing labels (MLML) is a common challenge that is prevalent in several domains, e.g. image annotation and auto-tagging. In multi-label classification, each instance may belong to multiple class labels simultaneously. Due to the nature of the dataset collection and labelling procedure, it is common to have incomplete annotations in the dataset, i.e. not all samples are labelled with all the corresponding labels. However, the incomplete data labelling hinders the training of classification models. MLML has received much attention from the research community. However, in cases where a pre-trained model is fine-tuned on an MLML dataset, there has been no straightforward approach to tackle the missing labels, specifically when there is no information about which are the missing ones. In this paper, we propose a weighted loss function to account for the confidence in each label/sample pair that can easily be incorporated to fine-tune a pre-trained model on an incomplete dataset. Our experiment results show that using the proposed loss function improves the performance of the model as the ratio of missing labels increases.
- Thierry Bertin-Mahieux, Douglas Eck, and Michael Mandel. 2011. Automatic tagging of audio: The state-of-the-art. In Machine audition: Principles, algorithms and systems. IGI Global, 334--352.Google Scholar
- Wei Bi and James T Kwok. 2014. Multilabel classification with label correlations and missing labels. In Proceedings of 28th AAAI Conference on Artificial Intelligence.Google ScholarCross Ref
- Matthew R Boutell, Jiebo Luo, Xipeng Shen, and Christopher M Brown. 2004. Learning multi-label scene classification. Pattern recognition, Vol. 37, 9 (2004), 1757--1771.Google Scholar
- Serhat Selcuk Bucak, Rong Jin, and Anil K. Jain. 2011. Multi-label learning with incomplete class assignments. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE Computer Society.Google Scholar
- Ricardo S Cabral, Fernando Torre, Jo ao P Costeira, and Alexandre Bernardino. 2011. Matrix completion for multi-label image classification. In Proceedings of the Advances in neural information processing systems.Google Scholar
- Gang Chen, Yangqiu Song, Fei Wang, and Changshui Zhang. 2008. Semi-supervised multi-label learning by solving a sylvester equation. In Proceedings of the 2008 SIAM International Conference on Data Mining.Google ScholarCross Ref
- Minmin Chen, Alice Zheng, and Kilian Weinberger. 2013. Fast image tagging. In Proceedings of the International conference on machine learning.Google Scholar
- Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, and Yantao Zheng. 2009. NUS-WIDE: a real-world web image database from National University of Singapore. In Proceedings of the ACM international conference on image and video retrieval.Google ScholarDigital Library
- Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In Proceedings of the IEEE conference on computer vision and pattern recognition.Google ScholarCross Ref
- Jia Deng, Olga Russakovsky, Jonathan Krause, Michael S Bernstein, Alex Berg, and Li Fei-Fei. 2014. Scalable multi-label annotation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems.Google ScholarDigital Library
- Thibaut Durand, Nazanin Mehrasa, and Greg Mori. 2019. Learning a Deep ConvNet for Multi-label Classification with Partial Labels. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Google ScholarCross Ref
- Charles Elkan and Keith Noto. 2008. Learning Classifiers from Only Positive and Unlabeled Data.Google Scholar
- Mingchen Gao, Ulas Bagci, Le Lu, Aaron Wu, Mario Buty, Hoo-Chang Shin, Holger Roth, Georgios Z Papadakis, Adrien Depeursinge, Ronald M Summers, et al. 2018. Holistic classification of CT attenuation patterns for interstitial lung diseases via deep convolutional neural networks. Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization, Vol. 6, 1 (2018), 1--6.Google ScholarCross Ref
- Eva Gibaja and Sebastián Ventura. 2015. A tutorial on multilabel learning. ACM Computing Surveys (CSUR), Vol. 47, 3 (2015), 52.Google ScholarDigital Library
- Zhi-Fen He, Ming Yang, Yang Gao, Hui-Dong Liu, and Yilong Yin. 2019. Joint multi-label classification and label correlations with missing labels and feature selection. Knowledge-Based Systems, Vol. 163 (2019), 145--158.Google ScholarCross Ref
- Mengying Hu, Hu Han, Shiguang Shan, and Xilin Chen. 2018. Multi-label Learning from Noisy Labels with Non-linear Feature Transformation. In Proceedings of the Asian Conference on Computer Vision.Google Scholar
- Jun Huang, Feng Qin, Xiao Zheng, Zekai Cheng, Zhixiang Yuan, Weigang Zhang, and Qingming Huang. 2019. Improving multi-label classification with missing labels by learning label-specific features. Information Sciences, Vol. 492 (2019), 124--146.Google ScholarDigital Library
- Simon Kornblith, Jonathon Shlens, and Quoc V Le. 2019. Do better imagenet models transfer better?. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Google ScholarCross Ref
- Yuncheng Li, Yale Song, and Jiebo Luo. 2017. Improving pairwise ranking for multi-label image classification. In Proceedings of the IEEE conference on computer vision and pattern recognition.Google ScholarCross Ref
- Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2017. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision.Google ScholarCross Ref
- Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In Proceedings of the European conference on computer vision.Google ScholarCross Ref
- Jan Margeta, Antonio Criminisi, R Cabrera Lozoya, Daniel C Lee, and Nicholas Ayache. 2017. Fine-tuned convolutional neural nets for cardiac MRI acquisition plane recognition. Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization, Vol. 5, 5 (2017), 339--349.Google ScholarCross Ref
- Andrew McCallum. 1999. Multi-label text classification with a mixture model trained by EM. In Proceedings of the AAAI workshop on Text Learning.Google Scholar
- Olivier Petit, Nicolas Thome, Arnaud Charnoz, Alexandre Hostettler, and Luc Soler. [n.d.]. Handling Missing Annotations for Semantic Segmentation with Deep ConvNets. Technical Report.Google Scholar
- Guo-Jun Qi, Xian-Sheng Hua, Yong Rui, Jinhui Tang, Tao Mei, and Hong-Jiang Zhang. 2007. Correlative multi-label video annotation. In Proceedings of the 15th ACM international conference on Multimedia.Google ScholarDigital Library
- Marina Sokolova and Guy Lapalme. 2009. A systematic analysis of performance measures for classification tasks. Information processing & management, Vol. 45, 4 (2009), 427--437.Google Scholar
- Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander A Alemi. 2017. Inception-v4, inception-resnet and the impact of residual connections on learning. In Proceedings of the 31st AAAI Conference on Artificial Intelligence.Google ScholarCross Ref
- Nima Tajbakhsh, Jae Y Shin, Suryakanth R Gurudu, R Todd Hurst, Christopher B Kendall, Michael B Gotway, and Jianming Liang. 2016. Convolutional neural networks for medical image analysis: Full training or fine tuning? IEEE transactions on medical imaging, Vol. 35, 5 (2016), 1299--1312.Google Scholar
- Grigorios Tsoumakas, Anastasios Dimou, Eleftherios Spyromitros, Vasileios Mezaris, Ioannis Kompatsiaris, and Ioannis Vlahavas. 2009a. Correlation-based pruning of stacked binary relevance models for multi-label learning. In Proceedings of the 1st international workshop on learning from multi-label data.Google Scholar
- Grigorios Tsoumakas, Ioannis Katakis, and Ioannis Vlahavas. 2009b. Mining multi-label data. In Data mining and knowledge discovery handbook. Springer, 667--685.Google Scholar
- Arash Vahdat. 2017. Toward robustness against label noise in training deep discriminative neural networks. In Proceedings of the Advances in Neural Information Processing Systems.Google Scholar
- Byron C Wallace, Kevin Small, Carla E Brodley, and Thomas A Trikalinos. 2011. Class imbalance, redux. In Proceedings of the IEEE 11th international conference on data mining.Google ScholarCross Ref
- Jiang Wang, Yi Yang, Junhua Mao, Zhiheng Huang, Chang Huang, and Wei Xu. 2016. Cnn-rnn: A unified framework for multi-label image classification. In Proceedings of the IEEE conference on computer vision and pattern recognition.Google ScholarCross Ref
- Baoyuan Wu, Zhilei Liu, Shangfei Wang, Bao-Gang Hu, and Qiang Ji. 2014. Multi-label learning with missing labels. In Proceedings of 22nd International Conference on Pattern Recognition.Google ScholarDigital Library
- Baoyuan Wu, Siwei Lyu, and Bernard Ghanem. 2015. Ml-mg: Multi-label learning with missing labels using a mixed graph. In Proceedings of the IEEE international conference on computer vision.Google ScholarDigital Library
- Miao Xu, Rong Jin, and Zhi-Hua Zhou. 2013. Speedup matrix completion with side information: Application to multi-label learning. In Proceedings of the Advances in neural information processing systems.Google Scholar
- Miao Xu, Gang Niu, Bo Han, Ivor W Tsang, Zhi-Hua Zhou, and Masashi Sugiyama. 2018. Matrix Co-completion for Multi-label Classification with Missing Features and Labels. arXiv preprint arXiv:1805.09156 (2018).Google Scholar
- Hsiang-Fu Yu, Prateek Jain, Purushottam Kar, and Inderjit Dhillon. 2014a. Large-scale multi-label learning with missing labels. In International conference on machine learning.Google Scholar
- Hsiang-Fu Yu, Prateek Jain, Purushottam Kar, and Inderjit Dhillon. 2014b. Large-scale multi-label learning with missing labels. In Proceedings of the International conference on machine learning.Google Scholar
- Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. 2016. Understanding deep learning requires rethinking generalization. arXiv preprint arXiv:1611.03530 (2016).Google Scholar
- Min-Ling Zhang and Zhi-Hua Zhou. 2013. A review on multi-label learning algorithms. IEEE transactions on knowledge and data engineering, Vol. 26, 8 (2013), 1819--1837.Google Scholar
- Zongwei Zhou, Jae Shin, Lei Zhang, Suryakanth Gurudu, Michael Gotway, and Jianming Liang. 2017. Fine-tuning convolutional neural networks for biomedical image analysis: actively and incrementally. In Proceedings of the IEEE conference on computer vision and pattern recognition.Google ScholarCross Ref
Index Terms
- Confidence-based Weighted Loss for Multi-label Classification with Missing Labels
Recommendations
Improving multi-label classification with missing labels by learning label-specific features
AbstractExisting multi-label learning approaches mainly utilize an identical data representation composed of all the features in the discrimination of all the labels, and assume that all the class labels are observed for each training sample. ...
Enhancing Label Correlations in multi-label classification through global-local label specific feature learning to Fill Missing labels
AbstractIn multi-label classification, challenges arise from missing labels due to subjective analysis or label ambiguity. This makes it difficult to accurately capture label correlations and enhance classifier performance. Previous research has ...
Low rank label subspace transformation for multi-label learning with missing labels
Highlights- An integrated framework to recover missing labels and train the multi-label classifier by learning label correlations and transforming label subspace.
AbstractMulti-label datasets often contain label information with missing values and recovering them is a non-trivial challenge. Several methods augment the observed label matrix by constructing auxiliary labels and learning high order label ...
Comments