Abstract
Markov logic networks (MLNs) are an expressive representation for statistical relational learning that generalizes both first-order logic and graphical models. Existing discriminative weight learning methods for MLNs all try to learn weights that optimize the Conditional Log Likelihood (CLL) of the training examples. In this work, we present a new discriminative weight learning method for MLNs based on a max-margin framework. This results in a new model, Max-Margin Markov Logic Networks (M3LNs), that combines the expressiveness of MLNs with the predictive accuracy of structural Support Vector Machines (SVMs). To train the proposed model, we design a new approximation algorithm for loss-augmented inference in MLNs based on Linear Programming (LP). The experimental result shows that the proposed approach generally achieves higher F 1 scores than the current best discriminative weight learner for MLNs.
Chapter PDF
References
Getoor, L., Taskar, B. (eds.): Statistical Relational Learning. MIT Press, Cambridge (2007)
Richardson, M., Domingos, P.: Markov logic networks. MLJ 62, 107–136 (2006)
Lowd, D., Domingos, P.: Efficient weight learning for Markov logic networks. In: Kok, J.N., Koronacki, J., Lopez de Mantaras, R., Matwin, S., Mladenič, D., Skowron, A. (eds.) PKDD 2007. LNCS (LNAI), vol. 4702, pp. 200–211. Springer, Heidelberg (2007)
Poon, H., Domingos, P.: Joint inference in information extraction. In: AAAI 2007, pp. 913–918 (2007)
Singla, P., Domingos, P.: Discriminative training of Markov logic networks. In: AAAI 2005, pp. 868–873 (2005)
Huynh, T.N., Mooney, R.J.: Discriminative structure and parameter learning for Markov logic networks. In: ICML 2008, pp. 416–423 (2008)
Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines and Other Kernel-based Learning Methods. Cambridge University Press, Cambridge (2000)
Joachims, T.: A support vector method for multivariate performance measures. In: ICML 2005, pp. 377–384 (2005)
Taskar, B., Guestrin, C., Koller, D.: Max-margin Markov networks. In: NIPS 2003 (2003)
Tsochantaridis, I., Joachims, T., Hofmann, T., Altun, Y.: Large margin methods for structured and interdependent output variables. JMLR 6, 1453–1484 (2005)
Joachims, T., Finley, T., Yu, C.N.: Cutting-plane training of structural SVMs. MLJ (2009), http://www.springerlink.com/content/h557723w88185170
Kautz, H., Selman, B., Jiang, Y.: A general stochastic approach to solving problems with hard and soft constraints. In: Dingzhu Gu, J.D., Pardalos, P. (eds.) The Satisfiability Problem: Theory and Applications, AMS, pp. 573–586 (1997)
Riedel, S.: Improving the accuracy and efficiency of MAP inference for Markov logic, 468–475 (2008)
Poon, H., Domingos, P.: Sound and efficient inference with probabilistic and deterministic dependencies. In: AAAI 2006, Boston, MA (July 2006)
Singla, P., Domingos, P.: Lifted first-order belief propagation. In: AAAI 2008, 1094–1099 (2008)
Collins, M.: Discriminative training methods for hidden Markov models: Theory and experiments with perceptron algorithms. In: CONLL 2002, pp. 1–8 (2002)
Domingos, P., Kok, S., Lowd, D., Poon, H., Richardson, M., Singla, P.: Markov logic. In: De Raedt, L., Frasconi, P., Kersting, K., Muggleton, S.H. (eds.) Probabilistic Inductive Logic Programming. LNCS (LNAI), vol. 4911, pp. 92–117. Springer, Heidelberg (2008)
Kok, S., Singla, P., Richardson, M., Domingos, P.: The Alchemy system for statistical relational AI. Technical report, Department of Computer Science and Engineering, University of Washington (2005), http://www.cs.washington.edu/ai/alchemy
Lafferty, J., McCallum, A., Pereira, F.: Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: ICML 2001, Williamstown, MA, pp. 282–289 (2001)
Collins, M.: Parameter estimation for statistical parsing models: Theory and practice of distribution-free methods. In: Harry Bunt, J.C., Satta, G. (eds.) New Developments in Parsing Technology. Kluwer, Dordrecht (2004)
Finley, T., Joachims, T.: Training structural SVMs when exact inference is intractable. In: ICML 2008, pp. 304–311 (2008)
Asano, T., Williamson, D.P.: Improved approximation algorithms for MAX SAT. J. of Algorithms 42(1), 173–202 (2002)
Asano, T.: An improved analysis of Goemans and Williamson’s LP-relaxation for MAX SAT. Theoretical Computer Science 354(3), 339–353 (2006)
Boros, E., Hammer, P.L.: Pseudo-Boolean optimization. Discrete Applied Mathematics 123(1-3), 155–225 (2002)
Hinton, G.E.: Training products of experts by minimizing contrastive divergence. Neural Computation 14(8), 1771–1800 (2002)
Anguelov, D., Taskar, B., Chatalbashev, V., Koller, D., Gupta, D., Heitz, G., Ng, A.: Discriminative learning of Markov random fields for segmentation of 3D scan data. In: CVPR 2005, pp. 169–176 (2005)
Taskar, B., Lacoste-Julien, S., Jordan, M.I.: Structured prediction, dual extragradient and Bregman projections. JMLR 7, 1627–1653 (2006)
Collins, M., Globerson, A., Koo, T., Carreras, X., Bartlett, P.L.: Exponentiated gradient algorithms for conditional random fields and max-margin Markov networks. JMLR 9, 1775–1822 (2008)
Taskar, B., Abbeel, P., Koller, D.: Discriminative probabilistic models for relational data. In: UAI 2002, Edmonton, Canada, pp. 485–492 (2002)
Rückert, U., Kramer, S.: Margin-based first-order rule learning. MLJ 70(2-3), 189–206 (2008)
Szummer, M., Kohli, P., Hoiem, D.: Learning CRFs using graph cuts. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 582–595. Springer, Heidelberg (2008)
Biba, M., Ferilli, S., Esposito, F.: Discriminative structure learning of Markov logic networks. In: Železný, F., Lavrač, N. (eds.) ILP 2008. LNCS (LNAI), vol. 5194, pp. 59–76. Springer, Heidelberg (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Huynh, T.N., Mooney, R.J. (2009). Max-Margin Weight Learning for Markov Logic Networks. In: Buntine, W., Grobelnik, M., Mladenić, D., Shawe-Taylor, J. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2009. Lecture Notes in Computer Science(), vol 5781. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04180-8_54
Download citation
DOI: https://doi.org/10.1007/978-3-642-04180-8_54
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-04179-2
Online ISBN: 978-3-642-04180-8
eBook Packages: Computer ScienceComputer Science (R0)