Abstract
Modern applications of Latent Semantic Analysis (LSA) must deal with enormous (often practically infinite) data collections, calling for a single-pass matrix decomposition algorithm that operates in constant memory w.r.t. the collection size. This paper introduces a streamed distributed algorithm for incremental SVD updates. Apart from the theoretical derivation, we present experiments measuring numerical accuracy and runtime performance of the algorithm over several data collections, one of which is the whole of the English Wikipedia.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Berry, M.: Large-scale sparse singular value computations. The International Journal of Supercomputer Applications 6(1), 13–49 (1992)
Brand, M.: Fast low-rank modifications of the thin singular value decomposition. Linear Algebra and its Applications 415(1), 20–30 (2006)
Comon, P., Golub, G.: Tracking a few extreme singular values and vectors in signal processing. Proceedings of the IEEE 78(8), 1327–1343 (1990)
Deerwester, S., Dumais, S., Furnas, G., Landauer, T., Harshman, R.: Indexing by Latent Semantic Analysis. Journal of the American Society for Information Science 41(6), 391–407 (1990)
Golub, G., Van Loan, C.: Matrix computations. Johns Hopkins University Press, Baltimore (1996)
Gorrell, G., Webb, B.: Generalized hebbian algorithm for incremental Latent Semantic Analysis. In: Ninth European Conference on Speech Communication and Technology (2005)
Griffiths, T., Steyvers, M., Tenenbaum, J.: Topics in semantic representation. Psychological Review 114(2), 211–244 (2007)
Halko, N., Martinsson, P., Tropp, J.: Finding structure with randomness: Stochastic algorithms for constructing approximate matrix decompositions. arXiv 909 (2009)
Levy, A., Lindenbaum, M.: Sequential Karhunen–Loeve basis extraction and its application to images. IEEE Transactions on Image Processing 9(8), 1371 (2000)
Salton, G.: Automatic text processing: the transformation, analysis, and retrieval of information by computer. Addison-Wesley Longman Publishing Co., Inc., Boston (1989)
Vigna, S.: Distributed, large-scale latent semantic analysis by index interpolation. In: Proceedings of the 3rd International Conference on Scalable Information Systems, pp. 1–10. ICST (2008)
William, H., Teukolsky, S., Vetterling, W.: Numerical Recipes in C: The art of scientific computing. Cambridge University Press, New York (1988)
Zha, H., Marques, O., Simon, H.: Large-scale SVD and subspace-based methods for Information Retrieval. Solving Irregularly Structured Problems in Parallel, 29–42 (1998)
Zha, H., Simon, H.: On updating problems in Latent Semantic Indexing. SIAM Journal on Scientific Computing 21, 782 (1999)
Zha, H., Zhang, Z.: Matrices with low-rank-plus-shift structure: Partial SVD and Latent Semantic Indexing. SIAM Journal on Matrix Analysis and Applications 21, 522 (2000)
Řehůřek, R., Sojka, P.: Software Framework for Topic Modelling with Large Corpora. In: Proceedings of LREC 2010 Workshop on New Challenges for NLP Frameworks, pp. 45–50 (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Řehůřek, R. (2011). Subspace Tracking for Latent Semantic Analysis. In: Clough, P., et al. Advances in Information Retrieval. ECIR 2011. Lecture Notes in Computer Science, vol 6611. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-20161-5_29
Download citation
DOI: https://doi.org/10.1007/978-3-642-20161-5_29
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-20160-8
Online ISBN: 978-3-642-20161-5
eBook Packages: Computer ScienceComputer Science (R0)