Skip to main content
SpringerLink
Log in

Answering linear optimization queries with an approximate stream index

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

We propose a SAO index to approximately answer arbitrary linear optimization queries in a sliding window of a data stream. It uses limited memory to maintain the most “important” tuples. At any time, for any linear optimization query, we can retrieve the approximate top-K tuples in the sliding window almost instantly. The larger the amount of available memory, the better the quality of the answers is. More importantly, for a given amount of memory, the quality of the answers can be further improved by dynamically allocating a larger portion of the memory to the outer layers of the SAO index.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Abadi DJ, Carney D, Çetintemel U et al (2003) Aurora: a new model and architecture for data stream management. VLDB J 12(2): 120–139

    Article  Google Scholar 

  2. Agarwal PK, Har-Peled S, Varadarajan KR (2004) Approximating extent measures of points. JACM 51(4): 606–635

    Article  MathSciNet  Google Scholar 

  3. Babcock B, Babu S, Datar M et al (2002) Models and issues in data stream systems. PODS, pp 1–16

  4. Bruno N, Chaudhuri S, Gravano L (2002) Top-k selection queries over relational databases: mapping strategies and performance evaluation. TODS 27(2): 153–187

    Article  Google Scholar 

  5. Barber CB, Dobkin DP, Huhdanpaa H (1996) The quickhull algorithm for convex hulls. ACM Trans Math Softw 22(4): 469–483

    Article  MATH  MathSciNet  Google Scholar 

  6. Bangolae SL, Jayasumana AP, Chandrasekar V (2003) Gigabit Networking: digitized radar data transfer and beyond. ICC, pp 684–688

  7. Böhm C, Kriegel HP (2001) Determining the convex hull in large multidimensional databases. DaWaK, pp 294–306

  8. Chang YC, Bergman LD, Castelli V et al (2000) The onion technique: indexing for linear optimization queries. SIGMOD, pp 391–402

  9. Chandrasekaran S, Cooper O, Deshpande A et al (2003) TelegraphCQ: continuous dataflow processing for an uncertain world. CIDR

  10. Chang KC, Hwang SW (2002) Minimal probing: supporting expensive predicates for top-k queries. SIGMOD, pp 346–357

  11. Carey MJ, Kossmann D (1997) On Saying “Enough Already!” in SQL. SIGMOD, pp 219–230

  12. Cormode G, Muthukrishnan S (2003) Radial histograms for spatial streams. Technical Report 2003-11 DIMACS

  13. Clarkson KL, Mehlhorn K, Seidel R (1993) Four results on randomized incremental constructions. Comput Geom 3: 185–212

    Article  MATH  MathSciNet  Google Scholar 

  14. Dantzig GB (1963) Linear programming and extensions. Princeton University Press, NJ

    MATH  Google Scholar 

  15. DeWitt DJ, Gray J (1992) Parallel database systems: the future of high performance database systems. CACM 35(6): 85–98

    Google Scholar 

  16. Donjerkovic D, Ramakrishnan R (1999) Probabilistic optimization of top N queries. VLDB, pp 411–422

  17. Fagin R (1996) Combining fuzzy information from multiple systems. PODS, pp 216–226

  18. Fagin R, Lotem A, Naor M (2001) Optimal aggregation algorithms for middleware. PODS, pp 102–113

  19. Gibbons PB, Matias Y (1999) Synopsis data structures for massive data sets. SODA, pp 909–910

  20. Hristidis V, Koudas N, Papakonstantinou Y (2001) PREFER: a system for the efficient execution of multi-parametric ranked queries. SIGMOD, pp 259–270

  21. Hardy GH, Littlewood JE, Polya G (1934) Inequalities. Cambridge University Press, London

    Google Scholar 

  22. Hershberger J, Suri S (2004) Adaptive sampling for geometric problems over data streams. PODS, pp 252–262

  23. Ilyas IF, Aref WG, Elmagarmid AK (2004) Supporting top-k join queries in relational databases. VLDB J 13(3): 207–221

    Article  Google Scholar 

  24. Li CS, Chang YC, Bergman LD et al (2000) Model-based multi-modal information retrieval from large archives. ICDCS International Workshop of Knowledge Discovery and Data Mining in the World-Wide Web

  25. Li CS, Chang YC, Smith JR et al (2001) SPIRE/EPI-SPIRE model-based multi-modal information retrieval from large archives. MMCBIR

  26. Marian A, Bruno N, Gravano L (2004) Evaluating top-k queries over web-accessible databases. TODS 29(2): 319–362

    Article  Google Scholar 

  27. Mouratidis K, Bakiras S, Papadias D (2006) Continuous monitoring of top-k queries over sliding windows. SIGMOD, pp 635–646

  28. O’Rourke J (1998) Computational geometry in C, 2nd edn. Cambridge University Press, London

    MATH  Google Scholar 

  29. Preparata FP, Shamos MI (1985) Computational geometry—an introduction. Springer, Berlin

    Google Scholar 

  30. Sullivan DG, Seltzer MI (2000) Isolation with flexibility: a resource management framework for central servers. USENIX General Track, pp 337–350

  31. 2005 UC Data Mining Competition Homepage. http://mill.ucsd.edu

  32. Waldspurger CA, Weihl WE (1994) Lottery scheduling: flexible proportional-share resource management. OSDI, pp 1–11

  33. Yi K, Yu H, Yang J et al (2003) Efficient maintenance of materialized top-k views. ICDE, pp 189–200

  34. Luo G, Wu K, Yu PS (2007) SAO: a stream index for answering linear optimization queries. ICDE, pp 1302–1306

  35. Gedik B, Wu K, Yu PS et al (2007) CPU load shedding for binary stream joins. KAIS 13(3): 271–303

    Article  Google Scholar 

  36. Cho M, Pei J, Wang K (2007) Answering ad hoc aggregate queries from data streams using prefix aggregate trees. KAIS 12(3): 301–329

    Article  Google Scholar 

  37. Agarwal D (2007) Detecting anomalies in cross-classified streams: a bayesian approach. KAIS 11(1): 29–44

    Article  Google Scholar 

  38. Kalousis A, Prados J, Hilario M (2007) Stability of feature selection algorithms: a study on high-dimensional spaces. KAIS 12(1): 95–116

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. IBM T.J. Watson Research Center, 19 Skyline Drive, Hawthorne, NY, 10532, USA

    Gang Luo, Kun-Lung Wu & Philip S. Yu

Authors
  1. Gang Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kun-Lung Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Philip S. Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gang Luo.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Luo, G., Wu, KL. & Yu, P.S. Answering linear optimization queries with an approximate stream index. Knowl Inf Syst 20, 95–121 (2009). https://doi.org/10.1007/s10115-008-0157-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-008-0157-z

Keywords

Search

Navigation