Abstract
Encoding structural information in low-dimensional vectors is a recent trend in natural language processing that builds on distributed representations [14]. However, although the success in replacing structural information in final tasks, it is still unclear whether these distributed representations contain enough information on original structures. In this paper we want to take a specific example of a distributed representation, the distributed trees (DT) [17], and analyze the reverse problem: can the original structure be reconstructed given only its distributed representation? Our experiments show that this is indeed the case, DT can encode a great deal of information of the original tree, and this information is often enough to reconstruct the original object format.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
The circular convolution between \(\mathbf {a}\) and \(\mathbf {b}\) is defined as the vector \(\mathbf {c}\) with component \(c_i = \sum _j a_j b_{i-j\text { mod } d}\). The shuffled circular convolution is the circular convolution after the vectors have been randomly shuffled.
References
Aiolli, F., Da San Martino, G., Sperduti, A.: Route kernels for trees. In: ICML 2009 Proceedings of the 26th Annual International Conference on Machine Learning, pp. 17–24. ACM, New York, NY, USA (2009). http://doi.acm.org/10.1145/1553374.1553377
Baroni, M., Lenci, A.: Distributional memory: a general framework for corpus-based semantics. Comput. Linguist. 36(4), 673–721 (2010). http://dx.doi.org/10.1162/coli_a_00016
Baroni, M., Zamparelli, R.: Nouns are vectors, adjectives are matrices: representing adjective-noun constructions in semantic space. In: Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pp. 1183–1193. Association for Computational Linguistics, Cambridge, MA, October 2010. http://www.aclweb.org/anthology/D10-1115
Clark, S., Coecke, B., Sadrzadeh, M.: A compositional distributional model of meaning. In: Proceedings of the Second Symposium on Quantum Interaction (QI-2008), pp. 133–140 (2008)
Collins, M., Duffy, N.: Convolution kernels for natural language. In: Dietterich, T.G., Becker, S., Ghahramani, Z. (eds.) Advances in Neural Information Processing Systems 14, pp. 625–632. MIT Press, Cambridge (2001)
Collins, M., Duffy, N.: New ranking algorithms for parsing and tagging: kernels over discrete structures, and the voted perceptron. In: Proceedings of ACL 2002 (2002)
Dagan, I., Glickman, O., Magnini, B.: The PASCAL RTE challenge. In: PASCAL Challenges Workshop, Southampton, U.K. (2005)
Ferrone, L., Zanzotto, F.M.: Towards syntax-aware compositional distributional semantic models. In: Proceedings of COLING 2014, The 25th International Conference on Computational Linguistics: Technical Papers, pp. 721–730. Dublin City University and Association for Computational Linguistics, Dublin, Ireland, August 2014. http://www.aclweb.org/anthology/C14-1068
Grefenstette, E., Sadrzadeh, M.: Experimental support for a categorical compositional distributional model of meaning. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP 2011, pp. 1394–1404. Association for Computational Linguistics, Stroudsburg, PA, USA (2011). http://dl.acm.org/citation.cfm?id=2145432.2145580
Kimura, D., Kuboyama, T., Shibuya, T., Kashima, H.: A subpath kernel for rooted unordered trees. In: Huang, J.Z., Cao, L., Srivastava, J. (eds.) PAKDD 2011, Part I. LNCS, vol. 6634, pp. 62–74. Springer, Heidelberg (2011)
Li, X., Roth, D.: Learning question classifiers. In: Proceedings of the 19th International Conference on Computational Linguistics, COLING 2002, vol. 1, pp. 1–7. Association for Computational Linguistics, Stroudsburg, PA, USA (2002). http://dx.doi.org/10.3115/1072228.1072378
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. CoRR abs/1301.3781 (2013). http://arxiv.org/abs/1301.3781
Mitchell, J., Lapata, M.: Vector-based models of semantic composition. In: Proceedings of ACL 2008: HLT, pp. 236–244. Association for Computational Linguistics, Columbus, Ohio, June 2008. http://www.aclweb.org/anthology/P/P08/P08-1028
Plate, T.A.: Distributed Representations and Nested Compositional Structure. Ph.D. thesis (1994). http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.48.5527
Turney, P.D., Pantel, P.: From frequency to meaning: vector space models of semantics. J. Artif. Intell. Res. (JAIR) 37, 141–188 (2010)
Vishwanathan, S.V.N., Smola, A.J.: Fast kernels for string and tree matching. In: Becker, S., Thrun, S., Obermayer, K. (eds.) NIPS, pp. 569–576. MIT Press, Cambridge (2002)
Zanzotto, F.M., Dell’Arciprete, L.: Distributed tree kernels. In: Proceedings of International Conference on Machine Learning, 26 June–1 July 2012
Zanzotto, F.M., Dell’Arciprete, L.: Transducing sentences to syntactic feature vectors: an alternative way to “parse”? In: Proceedings of the Workshop on Continuous Vector Space Models and Their Compositionality, pp. 40–49. 8 August 2013. http://www.aclweb.org/anthology/W13-3205
Zanzotto, F.M., Korkontzelos, I., Fallucchi, F., Manandhar, S.: Estimating linear models for compositional distributional semantics. In: Proceedings of the 23rd International Conference on Computational Linguistics (COLING), August 2010
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Ferrone, L., Zanzotto, F.M., Carreras, X. (2015). Decoding Distributed Tree Structures. In: Dediu, AH., Martín-Vide, C., Vicsi, K. (eds) Statistical Language and Speech Processing. SLSP 2015. Lecture Notes in Computer Science(), vol 9449. Springer, Cham. https://doi.org/10.1007/978-3-319-25789-1_8
Download citation
DOI: https://doi.org/10.1007/978-3-319-25789-1_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-25788-4
Online ISBN: 978-3-319-25789-1
eBook Packages: Computer ScienceComputer Science (R0)