Abstract
This paper is concerned with Relational Support Vector Machines, at the intersection of Support Vector Machines (SVM) and Inductive Logic Programming or Relational Learning. The so-called phase transition framework, originally developed for constraint satisfaction problems, has been extended to relational learning and it has provided relevant insights into the limitations and difficulties thereof. The goal of this paper is to examine relational SVMs and specifically Multiple Instance (MI) Kernels along the phase transition framework. A relaxation of the MI-SVM problem formalized as a linear programming problem (LPP) is defined and we show that the LPP satisfiability rate induces a lower bound on the MI-SVM generalization error. An extensive experimental study shows the existence of a critical region, where both LPP unsatisfiability and MI-SVM error rates are high. An interpretation for these results is proposed.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Andrews, S., Tsochantaridis, I., Hofmann, T.: Support Vector Machines for Multiple-Instance Learning. In: NIPS Proc. of 15th, pp. 561–568 (2002)
Blockeel, H., Page, D., Srinivasan, A.: Multi-Instance Tree Learning. In: ICML, pp. 57–64 (2005)
Botta, M., Giordana, A., Saitta, L., Sebag, M.: Relational Learning as Search in a Critical Region. Journal of Machine Learning Research 4, 431–463 (2003)
Cheeseman, P., Kanefsky, B., Taylor, W.: Where the Really Hard Problems are. In: IJCAI, pp. 331–337 (1991)
Chen, Y., Wang, J.Z.: Image Categorization by Learning and Reasoning with Regions. Journal of Machine Learning Research 5, 913–939 (2004)
Chevaleyre, Y., Zucker, J.-D.: Solving Multiple-Instance and Multiple-Part Learning Problems with Decision Trees and Rule Sets. Application to the Mutagenesis Problem. In: Canadian Conference on Artificial Intelligence, pp. 204–214 (2001)
Collobert, R., Bengio, S., Mariéthoz, J.: Torch: A Modular Machine Learning Software Library. Technical Report IDIAP-RR 02-46 (2002)
Cuturi, M., Vert, J.-P.: Semigroup Kernels on Finite Sets. In: NIPS, pp. 329–336 (2004)
Dietterich, T., Lathrop, R., Lozano-Perez, T.: Solving the Multiple-Instance Problem with Axis-Parallel Rectangles. Artificial Intelligence 89(1-2), 31–71 (1997)
Gärtner, T., Flach, P.A., Kowalczyk, A., Smola, A.J.: Multi-Instance Kernels. In: ICML, pp. 179–186 (2002)
Giordana, A., Saitta, L.: Phase Transitions in Relational Learning. Machine Learning 41, 217–251 (2000)
Hogg, T., Huberman, B.A., C., Williams, C.P.: Phase Transitions and the Search Problem. Artificial intelligence 81(1-2), 1–15 (1996)
Kearns, M., Li, M.: Learning in the Presence of Malicious Errors. SIAM J. Comput. 22, 807–837 (1993)
Kramer, S., Lavrac, N., Flach, P.: Propositionalization Approaches to Relational Data Mining. In: Dzeroski, S., Lavrac, N. (eds.) Relational data mining, pp. 262–291 (2001)
Kwok, J., Cheung, P.-M.: Marginalized Multi-Instance Kernels. In: Kwok, J., Cheung, P.-M. (eds.) IJCAI, pp. 901–906 (2007)
Mahé, P., Ralaivola, L., Stoven, V., Vert, J.-P.: The Pharmacophore Kernel for Virtual Screening with Support Vector Machines. Journal of Chemical Information and Modeling 46, 2003–2014 (2006)
Maron, O., Lozano-Pérez, T.: A Framework for Multiple-Instance Learning. In: NIPS, pp. 570–576 (1997)
Muggleton, S., De Raedt, L.: Inductive Logic Programming: Theory and Methods. Journal of Logic Programming 19, 629–679 (1994)
Pernot, N., Cornuéjols, A., Sebag, M.: Phase Transitions Within Grammatical Inference. In: Pernot, N., Cornuéjols, A., Sebag, M. (eds.) IJCAI, pp. 811–816 (2005)
Vapnik, V.N.: Statistical Learning Theory. Wiley-Interscience, Chichester (1998)
Weidmann, N., Frank, E., Pfahringer, B.: A Two level Learning Method for Generalized Multi-Instance Problems. In: Lavrač, N., Gamberger, D., Todorovski, L., Blockeel, H. (eds.) ECML 2003. LNCS (LNAI), vol. 2837, pp. 468–479. Springer, Heidelberg (2003)
Zhang, Q., Goldman, S.A.: EM-DD: A Improved Multiple-Instance Learning Technique. In: NIPS Proc of the 14th, pp. 1073–1080 (2001)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gaudel, R., Sebag, M., Cornuéjols, A. (2008). A Phase Transition-Based Perspective on Multiple Instance Kernels. In: Blockeel, H., Ramon, J., Shavlik, J., Tadepalli, P. (eds) Inductive Logic Programming. ILP 2007. Lecture Notes in Computer Science(), vol 4894. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-78469-2_14
Download citation
DOI: https://doi.org/10.1007/978-3-540-78469-2_14
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-78468-5
Online ISBN: 978-3-540-78469-2
eBook Packages: Computer ScienceComputer Science (R0)