Abstract
In recent years several measures for the gold standard based evaluation of ontology learning were proposed. They can be distinguished by the layers of an ontology (e.g. lexical term layer and concept hierarchy) they evaluate. Judging those measures with a list of criteria we show that there exist some measures sufficient for evaluating the lexical term layer. However, existing measures for the evaluation of concept hierarchies fail to meet basic criteria. This paper presents a new taxonomic measure which overcomes the problems of current approaches.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Cimiano, P., Pivk, A., Schmidt-Thieme, L., Staab, S.: Learning taxonomic relations from heterogenous sources of evidence. In: Ontology Learning from Text: Methods, Applications, Evaluation. IOS Press, Amsterdam (2005)
Spyns, P., Reinberger, M.L.: Lexically evaluating ontology triples generated automatically from texts. In: Proc. of the Second European Conference on the Semantic Web (2005)
Sabou, M., Wroe, C., Goble, C., Stuckenschmidt, H.: Learning domain ontologies for semantic web service descriptions. Journal of Web Semantics 3(4) (2005)
Sabou, M., Wroe, C., Goble, C., Mishne, G.: Learning domain ontologies for web service descriptions: an experiment in bioinformatics. In: Proc. of WWW 2005 (2005)
Reinberger, M.L., Spyns, P.: Unsupervised text mining for the learning of dogma-inspired ontologies. In: Ontology Learning from Text: Methods, Applications and Evaluation
Maedche, A.: Ontology Learning for the Semantic Web. Kluwer, Boston (2002)
Maedche, A., Staab, S.: Measuring similarity between ontologies. In: Gómez-Pérez, A., Benjamins, V.R. (eds.) EKAW 2002. LNCS, vol. 2473. Springer, Heidelberg (2002)
Cimiano, P., Hotho, A., Staab, S.: Learning concept hierarchies from text corpora using formal concept analysis. JAIR – Journal of AI Research 24, 305–339 (2005)
Maynard, D., Peters, W., Li, Y.: Metrics for evaluation of ontology-based information extraction. In: Proc. of the EON 2006 Workshop (2006)
Hahn, U., Schnattinger, K.: Towards text knowledge engineering. In: Proc. of the 15th National Conference on Artificial Intelligence (AAAI 1998) (1998)
Brank, J., Mladenic, D., Grobelnik, M.: Gold standard based ontology evaluation using instance assignment. In: Proc. of the EON 2006 Workshop (2006)
Hartmann, J., Spyns, P., Maynard, D., Cuel, R.: Carmen Suarez de Figueroa, M., Sure, Y.: Methods for ontology evaluation. Deliverable D1.2.3, Knowledge Web (2004)
van Rijsbergen, C.: Information Retrieval. Butterworths, London (1979)
Dellschaft, K.: Measuring the similiarity of concept hierarchies and its influence on the evaluation of learning procedures. Diploma thesis, Universität Koblenz-Landau (2005), http://www.uni-koblenz.de/FB4/Institutes/IFI/AGStaab/Theses/2005/DADellschaft.pdf
Hearst, M.: Automatic aquisition of hyponyms from large text corpora. In: Proc. of the 14th International Conference on Computational Linguistics (1992)
Staab, S., Braun, C., Bruder, I., Düsterhöft, A., Heuer, A., Klettke, M., Neumann, G., Prager, B., Pretzel, J., Schnurr, H.-P., Studer, R., Uszkoreit, H., Wrenger, B.: GETESS - searching the web exploiting german texts. In: Klusch, M., Shehory, O., Weiss, G. (eds.) CIA 1999. LNCS, vol. 1652, pp. 113–124. Springer, Heidelberg (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Dellschaft, K., Staab, S. (2006). On How to Perform a Gold Standard Based Evaluation of Ontology Learning. In: Cruz, I., et al. The Semantic Web - ISWC 2006. ISWC 2006. Lecture Notes in Computer Science, vol 4273. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11926078_17
Download citation
DOI: https://doi.org/10.1007/11926078_17
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-49029-6
Online ISBN: 978-3-540-49055-5
eBook Packages: Computer ScienceComputer Science (R0)