Abstract
We describe a compression model for semistructured documents, called Structural Contexts Model, which takes advantage of the context information usually implicit in the structure of the text. The idea is to use a separate semiadaptive model to compress the text that lies inside each different structure type (e.g., different XML tag). The intuition behind the idea is that the distribution of all the texts that belong to a given structure type should be similar, and different from that of other structure types. We test our idea using a word-based Huffman coding, which is the standard for compressing large natural language textual databases, and show that our compression method obtains significant improvements in compression ratios. We also analyze the possibility that storing separate models may not pay of if the distribution of different structure types is not different enough, and present a heuristic to merge models with the aim of minimizing the total size of the compressed database. This technique gives an additional improvement over the plain technique. The comparison against existing prototypes shows that our method is a competitive choice for compressed text databases.
This work was partially supported by CYTED VII.19 RIBIDI project (all authors) and Fondecyt Project 1-020831 (second author).
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
J. Bentley, D. Sleator, R. Tarjan, and V. Wei. A locally adaptive data compression scheme. Communications of the ACM, 29:320–330, 1986.
J. Cheney. Compressing XML with multiplexed hierarchical PPM models. In Proc. Data Compression Conference (DCC 2001), pages 163–, 2001.
D. Harman. Overviewof the Third Text REtrieval Conference. In Proc. Third Text REtrieval Conference (TREC-3), pages 1–19, 1995. NIST Special Publication 500-207.
H. S. Heaps. Information Retrieval — Computational and Theoretical Aspects. Academic Press, 1978.
D. A. Huffman. A method for the construction of minimum-redundancy codes. Proc. Inst. Radio Engineers, 40(9):1098–1101, 1952.
H. Liefke and D. Suciu. XMill: an efficient compressor for XML data. In Proc. ACM SIGMOD 2000, pages 153–164, 2000.
A. Moffat. Word-based text compression. Software — Practice and Experience, 19(2): 185–198, 1989.
E. Silva de Moura, G. Navarro, N. Ziviani, and R. Baeza-Yates. Fast and flexible word searching on compressed text. ACM Transactions on Information Systems, 18(2):113–139, 2000.
G. Navarro, E. Silva de Moura, M. Neubert, N. Ziviani, and R. Baeza-Yates. Adding compression to block addressing inverted indexes. Information Retrieval, 3(1):49–77, 2000.
Ian H. Witten, Timothy C. Bell, John G. Cleary. Text Compression. Prentice Hall, Englewood Cliffs, N. J., 1990.
P. Tolani and J. R. Haritsa. XGRIND: A query-friendly XML compressor. In ICDE, 2002. http://citeseer.nj.nec.com/503319.html.
I. H. Witten, A. Moffat, and T. C. Bell. Managing Gigabytes. Morgan Kaufmann Publishers, Inc., second edition, 1999.
N. Ziviani, E. Moura, G. Navarro, and R. Baeza-Yates. Compression: A key for next-generation text retrieval systems. IEEE Computer, 33(11):37–44, November 2000.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Adiego, J., Navarro, G., de la Fuente, P. (2003). Compressing Semistructured Text Databases. In: Sebastiani, F. (eds) Advances in Information Retrieval. ECIR 2003. Lecture Notes in Computer Science, vol 2633. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-36618-0_34
Download citation
DOI: https://doi.org/10.1007/3-540-36618-0_34
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-01274-0
Online ISBN: 978-3-540-36618-8
eBook Packages: Springer Book Archive