Skip to main content

Advertisement

Springer Nature Link
Log in

Triple confidence measurement in knowledge graph with multiple heterogeneous evidences

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

Knowledge graph (KG) is a representative technique of knowledge engineering, and it is often used in various intelligence applications, which assume that all triples in knowledge graphs (KGs) are correct. However, due to the noise brought by automatic KG construction techniques and the fuzziness of knowledge in specific fields, measuring uncertainty of KGs (i.e., the confidence of each triple being true) is important to the tasks of error detection and fact verification. Existing studies on triple confidence measurement either only relies on explicit evidences or merely depends on embedding evidences, which causes the resulting confidences are not precise enough. To solve this problem, in this paper, we propose a new triple confidence measurement (TCM) method, which combines multiple heterogeneous evidences including explicit evidences (i.e., concept paths and neighbor concept subgraphs) and different embedding evidences acquired by large language model, KG embedding models, contrastive learning, and graph convolutional network. Experiments on different real-world datasets demonstrate not only the superiority of TCM in the tasks of error detection and link prediction, but also the effectiveness of all proposed explicit evidences and embedding evidences.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data Availability

No datasets were generated or analysed during the current study.

Notes

  1. https://github.com/yao8839836/kg-llm

References

  1. Lehmann, J., Isele, R., Jakob, M., Jentzsch, A., Kontokostas, D., Mendes, P.N., Hellmann, S., Morsey, M., Van Kleef, P., Auer, S., et al.: DBpedia-a large-scale, multilingual knowledge base extracted from Wikipedia. Semantic Web. 6(2), 167–195 (2015)

    Article  Google Scholar 

  2. Wu, T., Wang, H., Li, C., Qi, G., Niu, X., Wang, M., Li, L., Shi, C.: Knowledge graph construction from multiple online encyclopedias. World Wide Web: Internet and Web Inf Syst 23, 2671–2698 (2020)

    Article  Google Scholar 

  3. Zhang, N., Jia, Q., Deng, S., Chen, X., Ye, H., Chen, H., Tou, H., Huang, G., Wang, Z., Hua, N., et al.: Alicg: fine-grained and evolvable conceptual graph construction for semantic search at alibaba. In: Proc. of SIGKDD, pp. 3895–3905 (2021)

  4. Wu, T., Cao, X., Zhu, Y., Wu, F., Gong, T., Wang, Y., Jing, S.: AsdKB: A Chinese Knowledge Base for the Early Screening and Diagnosis of Autism Spectrum Disorder. In: Proc. of ISWC, Part II, pp. 59–75 (2023)

  5. Chen, Y., Li, H., Qi, G., Wu, T., Wang, T.: Outlining and filling: hierarchical query graph generation for answering complex questions over knowledge graphs. IEEE Trans. Knowl. Data Eng. 35(8), 8343–8357 (2023)

  6. Liu, Z., Xiong, C., Sun, M., Liu, Z.: Entity-Duet Neural Ranking: Understanding the Role of Knowledge Graph Semantics in Neural Information Retrieval. In: Proc. of ACL, pp. 2395–2405 (2018)

  7. Gao, F., Hou, J., Gu, J., Zhang, L.: Knowledge Graph based Mutual Attention for Machine Reading Comprehension over Anti-Terrorism Corpus. Data Intelligence 5(3), 685–706 (2023)

  8. Carlson, A., Betteridge, J., Kisiel, B., Settles, B., Hruschka, E., Mitchell, T.: Toward an Architecture for Never-Ending Language Learning. In: Proc. of AAAI, pp. 1306–1313 (2010)

  9. Speer, R., Chin, J., Havasi, C.: ConceptNet 5.5: An Open Multilingual Graph of General Knowledge. In: Proc. of AAAI, pp. 4444–4451 (2017)

  10. Ji, L., Wang, Y., Shi, B., Zhang, D., Wang, Z., Yan, J.: Microsoft concept graph: mining semantic concepts for short text understanding. Data Intelligence. 1(3), 238–270 (2019)

    Article  Google Scholar 

  11. Shi, B., Weninger, T.: Discriminative predicate path mining for fact checking in knowledge graphs. Knowl.-Based Syst. 104, 123–133 (2016)

    Article  Google Scholar 

  12. Lin, P., Song, Q., Shen, J., Wu, Y.: Discovering Graph Patterns for Fact Checking in Knowledge Graphs. In: Proc. of DASFAA, pp. 783–801 (2018)

  13. Belth, C., Zheng, X., Vreeken, J., Koutra, D.: What is Normal, What is Strange, and What is Missing in a Knowledge Graph: Unified Characterization via Inductive Summarization. In: Proc. of The Web Conference, pp. 1115–1126 (2020)

  14. Xie, R., Liu, Z., Lin, F., Lin, L.: Does William Shakespeare Really Write Hamlet? Knowledge Representation Learning with Confidence. In: Proc. of AAAI, pp. 4954–4961 (2018)

  15. Jia, S., Xiang, Y., Chen, X., Wang, K.: Triple Trustworthiness Measurement for Knowledge Graph. In: Proc. of WWW, pp. 2865–2871 (2019)

  16. Guo, A., Tan, Z., Zhao, X.: Measuring Triplet Trustworthiness in Knowledge Graphs via Expanded Relation Detection. In: Proc. of KSEM, Part I, pp. 65–76 (2020)

  17. Zhang, Q., Dong, J., Duan, K., Huang, X., Liu, Y., Xu, L.: Contrastive Knowledge Graph Error Detection. In: Proc. of CIKM, pp. 2590–2599 (2022)

  18. Dong, J., Zhang, Q., Huang, X., Tan, Q., Zha, D., Zihao, Z.: Active Ensemble Learning for Knowledge Graph Error Detection. In: Proc. of WSDM, pp. 877–885 (2023)

  19. Wang, Y., Ma, F., Gao, J.: Efficient Knowledge Graph Validation via Cross-Graph Representation Learning. In: Proc. of CIKM, pp. 1595–1604 (2020)

  20. Huang, J., Zhao, Y., Hu, W., Ning, Z., Chen, Q., Qiu, X., Huo, C., Ren, W.: Trustworthy Knowledge Graph Completion based on Multi-Sourced Noisy Data. In: Proc. of the Web Conference, pp. 956–965 (2022)

  21. Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating Embeddings for Modeling Multi-Relational Data. In: Proc. of NIPS, pp. 2787–2795 (2013)

  22. Sun, Z., Deng, Z.-H., Nie, J.-Y., Tang, J.: RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space. In: Proc. of ICLR (2019)

  23. Yao, L., Peng, J., Mao, C., Luo, Y.: Exploring Large Language Models for Knowledge Graph Completion. (2023) arXiv:2308.13916

  24. Gao, X., Xiao, B., Tao, D., Li, X.: A survey of graph edit distance. Pattern Anal. Appl. 13, 113–129 (2010)

    Article  MathSciNet  Google Scholar 

  25. Toutanova, K., Chen, D., Pantel, P., Poon, H., Choudhury, P., Gamon, M.: Representing text for joint embedding of text and knowledge bases. In: Proc. of EMNLP, pp. 1499–1509 (2015)

  26. Dettmers, T., Minervini, P., Stenetorp, P., Riedel, S.: Convolutional 2d knowledge graph embeddings. In: Proc. of AAAI, pp. 1811–1818 (2018)

  27. Bollacker, K., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase: a collaboratively created graph database for structuring human knowledge. In: Proc. of SIGMOD, pp. 1247–1250 (2008)

  28. Yang, B., Yih, W.-t., He, X., Gao, J., Li, D.: Embedding entities and relations for learning and inference in knowledge bases. In: Proc. of ICLR, pp. 1–13 (2015)

  29. Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., Bouchard, G.: Complex Embeddings for Simple Link Prediction. In: Proc. of ICML, pp. 2071–2080 (2016)

  30. Wang, J., Wu, T., Chen, S., Liu, Y., Zhu, S., Li, W., Xu, J., Qi, G.: unKR: A Python Library for Uncertain Knowledge Graph Reasoning by Representation Learning. In: Proc. of SIGIR, pp. 2822–2826 (2024)

Download references

Funding

This work is supported by the NSFC (Grant No. 62376058, 52378009, 62276063), the Fundamental Research Funds for the Central Universities (2242022R40045), ZhiShan Young Scholar Program of Southeast University, and the Big Data Computing Center of Southeast University.

Author information

Authors and Affiliations

  1. Southeast University, Nanjing, Jiangsu, China

    Tianxing Wu, Kai Yao, Wei Li & Guilin Qi

  2. Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications, (Southeast University), Ministry of Education, Nanjing, Jiangsu, China

    Tianxing Wu & Guilin Qi

  3. Huawei Technologies Co., Ltd., Shenzhen, China

    Yijun Yu, Nengwen Zhao & Renyou Zhang

  4. Monash University, Melbourne, Australia

    Peibo Duan

Authors
  1. Tianxing Wu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Kai Yao
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Wei Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Guilin Qi
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Yijun Yu
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Nengwen Zhao
    View author publications

    You can also search for this author inPubMed Google Scholar

  7. Renyou Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  8. Peibo Duan
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

T.W. and G.Q. proposed the idea of this work and designed the method. K.Y. finished a part of the experiments. T.W. and W.L. wrote the paper. Y.Y., N.Z., R.Z., and P.D. revised this paper and gave a lot of suggestions. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Tianxing Wu.

Ethics declarations

Competing Interests

The authors declare no competing interests.

Ethical Approval

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article belongs to the Topical Collection: Special Issue on Neuro-Symbolic Intelligence: Large Language Model Enabled Knowledge Engineering

Guest Editors: Haofen Wang, Arijit Khan, Jun Liu and Michael Witbrock

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, T., Yao, K., Li, W. et al. Triple confidence measurement in knowledge graph with multiple heterogeneous evidences. World Wide Web 27, 70 (2024). https://doi.org/10.1007/s11280-024-01307-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11280-024-01307-x

Keywords