Abstract
With the outburst of available data, Definition Extraction has emerged as an important technique as it is a precursor to many other tasks like ontology generation, glossary creation, and question answering. Definition Extraction is most commonly treated as a binary classification problem of definitional and non-definitional sentences. Traditional techniques for definition extraction involve rule-based approaches, which did not yield good results because of the overwhelming complexity of natural language. Incorporating linguistic information via syntactic dependencies turned out to be useful in identifying sentences containing a definition. In this paper, we explore the performance of Transformer based architectures, like Bidirectional Encoder Representations from Transformers (BERT), which produce state-of-the-art results on many Natural Language Processing (NLP) tasks. Experiments on an annotated dataset of definitional sentences prove that BERT obtains results comparable to the state-of-the-art benchmark. In further experiments, we look under the hood of BERT, trying to figure out the reason for its success. Analyzing the outputs of the attention heads reveals that BERT captures not only syntactic dependencies but many other relevant dependencies within the words of the sentence, which proves beneficial in Definition Extraction.
The authors Priyanshu Kumar and Aadarsh Singh have equal contribution in the work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Anke, L.E.: Towards definition extraction using conditional random fields. In: Proceedings of the Student Research Workshop Associated with RANLP, pp. 63–70 (2013)
Anke, L.E., Schockaert, S.: Syntactically aware neural architectures for definition extraction. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pp. 378–385 (2018)
Borg, C., Rosner, M., Pace, G.: Evolutionary algorithms for definition extraction. In: Proceedings of the 1st Workshop on Definition Extraction, pp. 26–32. Association for Computational Linguistics (2009)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Espinosa-Anke, L., Saggion, H.: Applying dependency relations to definition extraction. In: Métais, E., Roche, M., Teisseire, M. (eds.) Natural Language Processing and Information Systems. LNCS, vol. 8455, pp. 63–74. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-07983-7_10
Espinosa-Anke, L., Saggion, H., Delli Bovi, C.: Definition extraction using sense-based embeddings. In: Gupta, P., Banchs, R.E., Rosso, P. (eds.) International Workshop on Embeddings and Semantics (IWES 2015), Alicante, Spain, 15 September 2015. CEUR (2015). 6 p
Jin, Y., Kan, M.Y., Ng, J.P., He, X.: Mining scientific terms and their definitions: a study of the ACL anthology. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 780–790 (2013)
Kobyliński, Ł., Przepiórkowski, A.: Definition extraction with balanced random forests. In: Nordström, B., Ranta, A. (eds.) GoTAL 2008. LNCS (LNAI), vol. 5221, pp. 237–247. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-85287-2_23
Li, S.L., Xu, B., Chung, T.L.: Definition extraction with LSTM recurrent neural networks. In: Sun, M., Huang, X., Lin, H., Liu, Z., Liu, Y. (eds.) CCL/NLP-NABD -2016. LNCS (LNAI), vol. 10035, pp. 177–189. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-47674-2_16
Meyer, I.: Extracting knowledge-rich contexts for terminography. In: Recent Advances in Computational Terminology, vol. 2, p. 279 (2001)
Navigli, R., Velardi, P.: Learning word-class lattices for definition and hypernym extraction. In: Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pp. 1318–1327. Association for Computational Linguistics (2010)
Navigli, R., Velardi, P., Ruiz-Martínez, J.M., et al.: An annotated dataset for extracting definitions and hypernyms from the web. In: LREC (2010)
Saggion, H., Gaizauskas, R.J.: Mining on-line sources for definition knowledge. In: FLAIRS Conference, pp. 61–66 (2004)
Seppälä, S.: A proposal for a framework to evaluate feature relevance for terminographic definitions. In: Proceedings of the 1st Workshop on Definition Extraction, pp. 47–53. Association for Computational Linguistics (2009)
Sierra, G., Alarcón, R., Aguilar, C., Barrón, A.: Towards the building of a corpus of definitional contexts. In: Proceeding of the 12th EURALEX International Congress, Torino, Italy, pp. 229–240 (2006)
Taylor, W.L.: “Cloze procedure”: a new tool for measuring readability. Journal. Bull. 30(4), 415–433 (1953)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Westerhout, E., Monachesi, P.: Extraction of Dutch definitory contexts for elearning purposes. LOT Occas. Ser. 7, 219–234 (2007)
Wolf, T., et al.: Transformers: state-of-the-art natural language processing. arXiv preprint arXiv:1910.03771 (2019)
Wu, Y., et al.: Google’s neural machine translation system: bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144 (2016)
Zhang, C., Jiang, P.: Automatic extraction of definitions. In: 2009 2nd IEEE International Conference on Computer Science and Information Technology, pp. 364–368. IEEE (2009)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Kumar, P., Singh, A., Kumar, P., Kumar, C. (2020). An Explainable Machine Learning Approach for Definition Extraction. In: Bhattacharjee, A., Borgohain, S., Soni, B., Verma, G., Gao, XZ. (eds) Machine Learning, Image Processing, Network Security and Data Sciences. MIND 2020. Communications in Computer and Information Science, vol 1241. Springer, Singapore. https://doi.org/10.1007/978-981-15-6318-8_13
Download citation
DOI: https://doi.org/10.1007/978-981-15-6318-8_13
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-6317-1
Online ISBN: 978-981-15-6318-8
eBook Packages: Computer ScienceComputer Science (R0)