Abstract
Multi-label classification has attracted much interest due to its wide applicability. Modeling label interactions and investigating their impact on classifier quality are crucial aspects of multi-label classification. In this paper, we propose a multi-structure SVM (called MSSVM) which allows the user to hypothesize multiple label interaction structures and helps to identify their importance in improving generalization performance. We design an efficient optimization algorithm to solve the proposed MSSVM. Extensive empirical evaluation provides fresh and interesting insights into the following questions: (a) How do label interactions affect multiple performance metrics typically used in multi-label classification? (b) Do higher order label interactions significantly impact a given performance metric for a particular dataset? (c) Can we make useful suggestions on the label interaction structure? and (d) Is it always beneficial to model label interactions in multi-label classification?
A. Kasinikota—This work was done when the author was at IISc, Bangalore, India.
A long version of this paper along with supplementary material is available at https://sites.google.com/site/pbalamuru/home/mssvm.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Balamurugan, P., Shevade, S., Sundararajan, S., Keerthi, S.S.: A sequential dual method for structural svms. In: SDM (2011)
Beck, A.: The 2-coordinate descent method for solving double-sided simplex constrained minimization problems. JOTA 162(3), 892–919 (2014)
Bi, W., Kwok, J.T.: Multilabel classification with label correlations and missing labels. In: AAAI (2014)
Dembczyński, K., Jachnik, A., Kotłlowski, W., Waegeman, W., Hüllermeier, E.: Optimizing the f-measure in multi-label classification: plug-in rule approach versus structured loss minimization. In: ICML (2013)
Dembszynski, K., Waegeman, W., Cheng, W., Hüllermeier, E.: On label dependence and loss minimization in multi-label classification. Mach. Learn. 88(1–2), 5–45 (2012)
Engelbrecht, H.A.: Efficient decoding of high-order hidden markov models. Ph.D. thesis, University of Stellenbosch (2007)
Finley, T., Joachims, T.: Training structural svms when exact inference is intractable. In: ICML (2008)
Forney, G.D.: The Viterbi algorithm. Proc. IEEE 61, 268–278 (1973)
Ghamrawi, N., McCallum, A.: Collective multi-label classification. In: CIKM (2005)
Godbole, S., Sarawagi, S.: Discriminative methods for multi-labeled classification. In: PAKDD (2004)
Gonçalves, A., Zuben, F.J.V., Banerjee, A.: Multi-label structure learning with ising model selection. In: IJCAI (2015)
Guo, Y., Gu, S.: Multi-label classification using conditional dependency networks. In: IJCAI (2011)
Guzman-Rivera, A., Batra, D., Kohli, P.: Multiple choice learning: learning to produce multiple structured outputs. In: NIPS (2012)
Hariharan, B., Vishwanathan, S.V., Varma, M.: Efficient max-margin multi-label classification with applications to zero-shot learning. Mach. Learn. 88(1–2), 127–155 (2012)
Huang, S.J., Zhou, Z.H.: Multi-label learning by exploiting label correlations locally. In: AAAI (2012)
Kruskal, J.B.: On the shortest spanning subtree of a graph and the traveling salesman problem. Proc. AMS 7, 48–50 (1956)
Kyriakopoulou, A., Kalamboukis, T.: Using clustering to enhance text classification. In: SIGIR (2007)
Lafferty, J., McCallum, A., Pereira, F.C.N.: Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: ICML (2001)
Lampert, C.H.: Maximum margin multi-label structured prediction. In: NIPS (2011)
Liu, W., Tsang, I.: On the optimality of classifier chain for multi-label classification. In: NIPS (2015)
Manning, C.D., Raghavan, P., Schütze, H.: Introduction to Information Retrieval. Cambridge University Press, New York (2008)
Marchand, M., Su, H., Morvant, E., Rousu, J., Shawe-Taylor, J.S.: Multilabel structured output learning with random spanning trees of max-margin markov networks. In: NIPS (2014)
Mirzazadeh, F., Ravanbakhsh, S., Ding, N., Schuurmans, D.: Embedding inference for structured multilabel prediction. In: NIPS (2015)
Platt, J.C.: Fast training of support vector machines using sequential minimal optimization. In: Advances in Kernel Methods, pp. 185–208 (1999)
Read, J., Martino, L., Olmos, P.M., Luengo, D.: Scalable multi-output label prediction: from classifier chains to classifier trellises. Pattern Recogn. 48(6), 2096–2109 (2015)
Read, J., Pfahringer, B., Holmes, G., Frank, E.: Classifier chains for multi-label classification. In: Buntine, W., Grobelnik, M., Mladenić, D., Shawe-Taylor, J. (eds.) ECML PKDD 2009. LNCS (LNAI), vol. 5782, pp. 254–269. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-04174-7_17
Su, H., Rousu, J.: Multilabel classification through random graph ensembles. In: ACML (2013)
Tan, M., Shi, Q., van den Hengel, A., Shen, C., Gao, J., Hu, F., Zhang, Z.: Learning graph structure for multi-label image classification via clique generation. In: CVPR (2015)
Tsoumakas, G., Katakis, I.: Multi-label classification: an overview. Int. J. Data Warehous. Min. 3, 1–13 (2007)
Tsoumakas, G., Vlahavas, I.: Random k-Labelsets: An Ensemble Method for Multilabel Classification. In: Kok, J.N., Koronacki, J., Mantaras, R.L., Matwin, S., Mladenič, D., Skowron, A. (eds.) ECML 2007. LNCS (LNAI), vol. 4701, pp. 406–417. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-74958-5_38
Vapnik, V.N.: Statistical Learning Theory. Wiley-Interscience, New York (1998)
Wu, Q., Tan, M., Song, H., Chen, J., Ng, M.K.: Ml-forest: a multi-label tree ensemble method for multi-label classification. IEEE TKDE 28, 2665–2680 (2016)
Yu, C.N.J., Joachims, T.: Learning structural SVMs with latent variables. In: ICML (2009)
Yu, H.F., Jain, P., Kar, P., Dhillon, I.S.: Large-scale multi-label learning with missing labels. In: ICML (2014)
Zhai, S., Zhao, C., Xia, T., Wang, S.: A multi-label ensemble method based on minimum ranking margin maximization. In: ICDM (2015)
Zhang, M.L., Zhang, K.: Multi-label learning by exploiting label dependency. In: ACM SIGKDD (2010)
Zhang, M.L., Zhou, Z.H.: ML-KNN: a lazy learning approach to multi-label learning. Pattern Recogn. 40, 2038–2048 (2007)
Zhang, M.L., Zhou, Z.H.: A review on multi-label learning algorithms. IEEE TKDE 26(8), 1819–1837 (2014)
Zhang, X., Graepel, T., Herbrich, R.: Bayesian online learning for multi-label and multi-variate performance measures. In: AISTATS (2010)
Acknowledgments
The authors thank anonymous reviewers of the current and earlier versions of the paper for their useful comments. The second author thanks Prof. Francis Bach for the discussion.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Kasinikota, A., Balamurugan, P., Shevade, S. (2018). Modeling Label Interactions in Multi-label Classification: A Multi-structure SVM Perspective. In: Phung, D., Tseng, V., Webb, G., Ho, B., Ganji, M., Rashidi, L. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2018. Lecture Notes in Computer Science(), vol 10937. Springer, Cham. https://doi.org/10.1007/978-3-319-93034-3_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-93034-3_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-93033-6
Online ISBN: 978-3-319-93034-3
eBook Packages: Computer ScienceComputer Science (R0)