ABSTRACT
Reasoning is essential for the development of large knowledge graphs, especially for completion, which aims to infer new triples based on existing ones. Both rules and embeddings can be used for knowledge graph reasoning and they have their own advantages and difficulties. Rule-based reasoning is accurate and explainable but rule learning with searching over the graph always suffers from efficiency due to huge search space. Embedding-based reasoning is more scalable and efficient as the reasoning is conducted via computation between embeddings, but it has difficulty learning good representations for sparse entities because a good embedding relies heavily on data richness. Based on this observation, in this paper we explore how embedding and rule learning can be combined together and complement each other's difficulties with their advantages. We propose a novel framework IterE iteratively learning embeddings and rules, in which rules are learned from embeddings with proper pruning strategy and embeddings are learned from existing triples and new triples inferred by rules. Evaluations on embedding qualities of IterE show that rules help improve the quality of sparse entity embeddings and their link prediction results. We also evaluate the efficiency of rule learning and quality of rules from IterE compared with AMIE+, showing that IterE is capable of generating high quality rules more efficiently. Experiments show that iteratively learning embeddings and rules benefit each other during learning and prediction.
- Molood Barati, Quan Bai, and Qing Liu. 2016. SWARM: An Approach for Mining Semantic Association Rules from Semantic Web Data. In PRICAI(Lecture Notes in Computer Science), Vol. 9810. Springer, 30-43. Google ScholarDigital Library
- Kurt D. Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: a collaboratively created graph database for structuring human knowledge. Proceedings of SIGMOD(2008), 1247-1250. Google ScholarDigital Library
- Antoine Bordes, Nicolas Usunier, Alberto García-Durán, Jason Weston, and Oksana Yakhnenko. 2013. Translating Embeddings for Modeling Multi-relational Data. Proceedings of NIPS(2013), 2787-2795. Google ScholarDigital Library
- Wanyun Cui, Yanghua Xiao, Haixun Wang, Yangqiu Song, Seung-won Hwang, and Wei Wang. 2017. KBQA: Learning Question Answering over QA Corpora and Knowledge Bases. PVLDB10, 5 (2017), 565-576. Google ScholarDigital Library
- Rajarshi Das, Shehzaad Dhuliawala, Manzil Zaheer, Luke Vilnis, Ishan Durugkar, Akshay Krishnamurthy, Alexander J. Smola, and Andrew McCallum. 2017. Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning. CoRRabs/1711.05851(2017).Google Scholar
- Thomas Demeester, Tim Rocktäschel, and Sebastian Riedel. 2016. Lifted Rule Injection for Relation Embeddings. In EMNLP. The Association for Computational Linguistics, 1389-1399.Google Scholar
- Tim Dettmers. 2018. Convolutional 2D Knowledge Graph Embeddings. Proceddings of AAAI(2018).Google Scholar
- Boyang Ding, Quan Wang, Bin Wang, and Li Guo. 2018. Improving Knowledge Graph Embedding Using Simple Constraints. In ACL (1). Association for Computational Linguistics, 110-121.Google Scholar
- Luis Galárraga, Christina Teflioudi, Katja Hose, and Fabian M. Suchanek. 2015. Fast rule mining in ontological knowledge bases with AMIE+. VLDB J.24, 6 (2015), 707-730. Google ScholarDigital Library
- Luis Antonio Galárraga, Christina Teflioudi, Katja Hose, and Fabian M. Suchanek. 2013. AMIE: association rule mining under incomplete evidence in ontological knowledge bases. In 22nd International World Wide Web Conference, WWW '13, Rio de Janeiro, Brazil, May 13-17, 2013. 413-422. Google ScholarDigital Library
- Alberto García-Durán, Antoine Bordes, and Nicolas Usunier. 2015. Composing Relationships with Translations. Proceedings of EMNLP(2015), 286-290.Google ScholarCross Ref
- Shu Guo, Quan Wang, Bin Wang, Lihong Wang, and Li Guo. 2015. Semantically Smooth Knowledge Graph Embedding. Proceedings of ACL(2015), 84-94.Google ScholarCross Ref
- Shu Guo, Quan Wang, Lihong Wang, Bin Wang, and Li Guo. 2016. Jointly Embedding Knowledge Graphs and Logical Rules. In EMNLP. The Association for Computational Linguistics, 192-202.Google Scholar
- Shu Guo, Quan Wang, Lihong Wang, Bin Wang, and Li Guo. 2018. Knowledge Graph Embedding With Iterative Guidance From Soft Rules. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, Louisiana, USA, February 2-7, 2018.Google ScholarCross Ref
- Víctor Gutie´rrez-Basultoand Steven Schockaert. 2018. From Knowledge Graph Embedding to Ontology Embedding? An Analysis of the Compatibility between Vector Space Representations and Rules. In KR. AAAI Press, 379-388.Google Scholar
- Vinh Thinh Ho, Daria Stepanova, Mohamed H. Gad-Elrab, Evgeny Kharlamov, and Gerhard Weikum. 2018. Rule Learning from Knowledge Graphs Guided by Embedding Models. In International Semantic Web Conference (1)(Lecture Notes in Computer Science), Vol. 11136. Springer, 72-90.Google ScholarDigital Library
- Petr Hájek. 1998. The metamathematics of fuzzy logic. (1998).Google Scholar
- Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, and Jun Zhao. 2015. Knowledge Graph Embedding via Dynamic Mapping Matrix. Proceedings of ACL(2015), 687-696.Google ScholarCross Ref
- Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. CoRRabs/1412.6980(2014).Google Scholar
- Ni Lao, Tom M. Mitchell, and William W. Cohen. 2011. Random Walk Inference and Learning in A Large Scale Knowledge Base. In EMNLP. ACL, 529-539. Google ScholarDigital Library
- Yankai Lin, Zhiyuan Liu, Huan-Bo Luan, Maosong Sun, Siwei Rao, and Song Liu. 2015. Modeling Relation Paths for Representation Learning of Knowledge Bases. Proceedings of EMNLP(2015), 705-714.Google ScholarCross Ref
- Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. 2015. Learning Entity and Relation Embeddings for Knowledge Graph Completion. Proceedings of AAAI(2015), 2181-2187. Google ScholarDigital Library
- Hanxiao Liu, Yuexin Wu, and Yiming Yang. 2017. Analogical Inference for Multi-relational Embeddings. In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017. 2168-2178. Google ScholarDigital Library
- Pasquale Minervini, Luca Costabello, Emir Mu&ntiled;oz, Vít Novácek, and Pierre-Yves Vandenbussche. 2017. Regularizing Knowledge Graph Embeddings via Equivalence and Inversion Axioms. In ECML/PKDD (1)(Lecture Notes in Computer Science), Vol. 10534. Springer, 668-683.Google Scholar
- Arvind Neelakantan, Benjamin Roth, and Andrew McCallum. 2015. Compositional Vector Space Models for Knowledge Base Completion. Proceedings of ACL(2015), 156-166.Google ScholarCross Ref
- Maximilian Nickel, Kevin Murphy, Volker Tresp, and Evgeniy Gabrilovich. 2016. A Review of Relational Machine Learning for Knowledge Graphs. Proc. IEEE104, 1 (2016), 11-33.Google ScholarCross Ref
- Maximilian Nickel, Lorenzo Rosasco, and Tomaso A. Poggio. 2016. Holographic Embeddings of Knowledge Graphs. Proceedings of AAAI(2016), 1955-1961. Google ScholarDigital Library
- Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. 2011. A Three-Way Model for Collective Learning on Multi-Relational Data. Proceedings of ICML(2011), 809-816. Google ScholarDigital Library
- Pouya Ghiasnezhad Omran, Kewen Wang, and Zhe Wang. 2018. Scalable Rule Learning via Learning Representation. In IJCAI. ijcai.org, 2149-2155. Google ScholarDigital Library
- Petar Ristoski and Heiko Paulheim. 2016. RDF2Vec: RDF Graph Embeddings for Data Mining. Procedings of ISWC(2016), 498-514.Google Scholar
- Yelong Shen, Po-Sen Huang, Ming-Wei Chang, and Jianfeng Gao. 2017. Modeling Large-Scale Structured Relationships with Shared Memory for Knowledge Base Completion. In Rep4NLP@ACL. Association for Computational Linguistics, 57-68.Google Scholar
- Richard Socher, Danqi Chen, Christopher D. Manning, and Andrew Y. Ng. 2013. Reasoning With Neural Tensor Networks for Knowledge Base Completion. Prodeddings of NIPS(2013), 926-934. Google ScholarDigital Library
- Fabian M. Suchanek, Gjergji Kasneci, and Gerhard Weikum. 2007. Yago: a core of semantic knowledge. Proceedings of WWW(2007), 697-706. Google ScholarDigital Library
- Kristina Toutanova and Danqi Chen. 2015. Observed versus latent features for knowledge base and text inference. In Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Compositionality. 57-66.Google ScholarCross Ref
- The´o Trouillon, Johannes Welbl, Sebastian Riedel, E&Acute;ric Gaussier, and Guillaume Bouchard. 2016. Complex Embeddings for Simple Link Prediction. Proceedings ICML (2016), 2071-2080. Google ScholarDigital Library
- Guanying Wang, Wen Zhang, Ruoxu Wang, Yalin Zhou, Xi Chen, Wei Wang, Hai Zhu, and Huajun Chen. 2018. Label-Free Distant Supervision for Relation Extraction via Knowledge Graph Embedding. In EMNLP. The Association for Computational Linguistics.Google Scholar
- Quan Wang, Bin Wang, and Li Guo. 2015. Knowledge Base Completion Using Embeddings and Rules. In Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI 2015, Buenos Aires, Argentina, July 25-31, 2015. 1859-1866. Google ScholarDigital Library
- Zhichun Wang and Juan-Zi Li. 2015. RDF2Rules: Learning Rules from RDF Knowledge Bases by Mining Frequent Predicate Cycles. CoRRabs/1512.07734(2015).Google Scholar
- Zhigang Wang and Juan-Zi Li. 2016. Text-Enhanced Representation Learning for Knowledge Graph. Proceedings of IJCAI(2016), 1293-1299. Google ScholarDigital Library
- Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge Graph and Text Jointly Embedding. Proceedings of EMNLP(2014), 1591-1601.Google ScholarCross Ref
- Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge Graph Embedding by Translating on Hyperplanes. Proceedings of AAAI(2014), 1112-1119. Google ScholarDigital Library
- Han Xiao, Minlie Huang, Lian Meng, and Xiaoyan Zhu. 2017. SSP: Semantic Space Projection for Knowledge Graph Embedding with Text Descriptions. Proceedings of AAAI(2017), 3104-3110. Google ScholarDigital Library
- Ruobing Xie, Zhiyuan Liu, Jia Jia, Huanbo Luan, and Maosong Sun. 2016. Representation Learning of Knowledge Graphs with Entity Descriptions. Proceedings of AAAI(2016), 2659-2665. Google ScholarDigital Library
- Ruobing Xie, Zhiyuan Liu, Huanbo Luan, and Maosong Sun. 2017. Image-embodied Knowledge Representation Learning. In IJCAI. ijcai.org, 3140-3146. Google ScholarDigital Library
- Ruobing Xie, Zhiyuan Liu, and Maosong Sun. 2016. Representation Learning of Knowledge Graphs with Hierarchical Types. Proceedings of IJCAI(2016), 2965-2971. Google ScholarDigital Library
- Wenhan Xiong, Thien Hoang, and William Yang Wang. 2017. DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning. In EMNLP. Association for Computational Linguistics, 564-573.Google Scholar
- Bishan Yang, Wen tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2015. Embedding Entities and Relations for Learning and Inference in Knowledge Bases. Proceedings of ICLR(2015).Google Scholar
- Fan Yang, Zhilin Yang, and William W. Cohen. 2017. Differentiable Learning of Logical Rules for Knowledge Base Reasoning. In NIPS. 2316-2325. Google ScholarDigital Library
- Fuzheng Zhang, Nicholas Jing Yuan, Defu Lian, Xing Xie, and Wei-Ying Ma. 2016. Collaborative Knowledge Base Embedding for Recommender Systems. In KDD. ACM, 353-362. Google ScholarDigital Library
- Wen Zhang. 2017. Knowledge Graph Embedding with Diversity of Structures. Proceedings WWW Companion(2017), 747-753. Google ScholarDigital Library
Recommendations
Active knowledge graph completion
AbstractEnterprise and public Knowledge Graphs (KGs) are known to be incomplete. Methods for automatic completion, sometimes by rule learning, scale well. While previous rule-based methods learn closed (non-existential) rules, we introduce ...
Enhancing Rule Learning on Knowledge Graphs Through Joint Ontology and Instance Guidance
Pattern Recognition and Computer VisionAbstractRule learning is a machine learning method that extracts implicit rules and patterns from data, enabling symbol-based reasoning in artificial intelligence. Unlike data-driven approaches such as deep learning, using rules for inference allows for ...
Incorporating logic rules with textual representations for interpretable knowledge graph reasoning
AbstractReasoning on knowledge graphs (KGs) is significant for downstream applications, such as question answering and information extraction. On the basis of using factual triples in KGs, learning logic rules has been explored to handle KG ...
Highlights- An interpretable knowledge graph reasoning method is proposed.
- It employs a ...
Comments