Abstract.
Due to their expressive power, regular expressions (REs) are quickly becoming an integral part of language specifications for several important application scenarios. Many of these applications have to manage huge databases of RE specifications and need to provide an effective matching mechanism that, given an input string, quickly identifies the REs in the database that match it. In this paper, we propose the RE-tree, a novel index structure for large databases of RE specifications. Given an input query string, the RE-tree speeds up the retrieval of matching REs by focusing the search and comparing the input string with only a small fraction of REs in the database. Even though the RE-tree is similar in spirit to other tree-based structures that have been proposed for indexing multidimensional data, RE indexing is significantly more challenging since REs typically represent infinite sets of strings with no well-defined notion of spatial locality. To address these new challenges, our RE-tree index structure relies on novel measures for comparing the relative sizes of infinite regular languages. We also propose innovative solutions for the various RE-tree operations including the effective splitting of RE-tree nodes and computing a "tight" bounding RE for a collection of REs. Finally, we demonstrate how sampling-based approximation algorithms can be used to significantly speed up the performance of RE-tree operations. Preliminary experimental results with moderately large synthetic data sets indicate that the RE-tree is effective in pruning the search space and easily outperforms naive sequential search approaches.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
1. Altinel M, Franklin MJ (2000) Efficient filtering of XML documents for selective dissemination of information. In: Proceedings of VLDB, Cairo, September 2000, pp 53--64
2. Baeza-Yates R, Gonnet G (1996) Fast text searching for regular expressions or automaton searching on a trie. J ACM 43(6):915--936
3. Baeza-Yates R, Ribeiro-Neto B, Navarro G (1999) Indexing and searching. In: Modern information retrieval. ACM Press, New York, pp 191--228
4. Beckmann N, Kriegel H-P, Schneider R, Seeger B (1990) The R*-Tree: an efficient and robust access method for points and rectangles. In: Proceedings of SIGMOD, Atlantic City, NJ, May 1990, pp 322--331
5. Chan C-Y, Felber P, Garofalakis M, Rastogi R (2002) Efficient filtering of XML documents with XPath expressions. In: Proceedings of the 18th international conference on data engineering, San Jose, February 2002, pp 235--244
6. Chan CY, Garofalakis M, Rastogi R (2002) RE-Tree: an efficient index structure for regular expressions. In: Proceedings of VLDB, Hong Kong, August 2002, pp 251--262
7. Clark J, DeRose S (1999) XML path language (XPath) v. 1.0. W3C Recommendation, http://www.w3.org./TR/xpath
8. Cochran WG (1977) Sampling techniques. Wiley, New York
9. Intel Corporation (2000) Intel NetStructure XML accelerators, http://www.intel.com/netstructure/products/xml/accelerators.htm
10. Deutsch A, Fernandez M, Suciu D (1999) Storing semistructured data with STORED. In: Proceedings of the ACM SIGMOD conference on management of data, Philadelphia, June 1999, pp 431--442
11. Diao Y, Fischer P, Franklin MJ, To R (2002) YFilter: efficient and scalable filtering of XML documents. In: Proceedings of the 18th international conference on data engineering, San Jose, February 2002, pp 341--342
12. Garey MR, Johnson DS (1979) Computers and intractability: a guide to the theory of NP-Completeness. Freeman, San Francisco
13. Garofalakis M, Gionis A, Rastogi R, Seshadri S, Shim K (2000) XTRACT: a system for extracting document type descriptors from XML documents. In: Proceedings of SIGMOD, Dallas, May 2000, pp 165--176
14. Guttman A (1984) R-Trees: a dynamic index structure for spatial searching. In: Proceedings of SIGMOD, Boston, June 1984, pp 47--57
15. Hopcroft JE, Ullman JD (1979) Introduction to automata theory, languages, and computation. Addison-Wesley, Reading, MA
16. Ishikawa Y, Kitagawa H, Ohbo N (1993) Evaluation of signature files as set access facilities in OODBs. In: Proceedings of SIGMOD, Washington, DC, May 1993, pp 247--256
17. Kannan S, Sweedyk Z, Mahaney S (1995) Counting and random generation of strings in regular languages. In: Proceedings of SODA, San Francisco, January 1995, pp 551--557
18. Kilpelainen P, Mannila H, Ukkonen E (1995) MDL learning of unions of simple pattern languages from positive examples. In: Proceedings of EuroCOLT, Barcelona, March 1995, pp 252--260
19. Lakshmanan LVS, Parthasarathy S (2002) On efficient matching of streaming XML documents and queries. In: Proceedings of EDBT, Prague, March 2002, pp 142--160
20. Quinlan JR, Rivest RL (1989) Inferring decision trees using the minimum description length principle. Information and Computation 80:227--248
21. Rissanen J (1978) Modeling by shortest data description. Automatica 14:465--471
22. Stewart JW (1998) BGP4 inter-domain routing in the Internet. Addison-Wesley, Reading, MA
23. Wu S, Manber U (1992) Fast text searching allowing errors. Commun ACM 35(10):83--91
Author information
Authors and Affiliations
Corresponding author
Additional information
Received: 16 September 2002, Published online: 8 July 2003
Edited by R. Ramakrishnan
Rights and permissions
About this article
Cite this article
Chan, CY., Garofalakis, M. & Rastogi, R. RE-tree: an efficient index structure for regular expressions. VLDB 12, 102–119 (2003). https://doi.org/10.1007/s00778-003-0094-0
Issue Date:
DOI: https://doi.org/10.1007/s00778-003-0094-0