Skip to main content
SpringerLink
Log in

Online stochastic optimization under time constraints

  • Published:
Annals of Operations Research Aims and scope Submit manuscript

Abstract

This paper considers online stochastic combinatorial optimization problems where uncertainties, i.e., which requests come and when, are characterized by distributions that can be sampled and where time constraints severely limit the number of offline optimizations which can be performed at decision time and/or in between decisions. It proposes online stochastic algorithms that combine the frameworks of online and stochastic optimization. Online stochastic algorithms differ from traditional a priori methods such as stochastic programming and Markov Decision Processes by focusing on the instance data that is revealed over time. The paper proposes three main algorithms: expectation E, consensus C, and regret R. They all make online decisions by approximating, for each decision, the solution to a multi-stage stochastic program using an exterior sampling method and a polynomial number of samples. The algorithms were evaluated experimentally and theoretically. The experimental results were obtained on three applications of different nature: packet scheduling, multiple vehicle routing with time windows, and multiple vehicle dispatching. The theoretical results show that, under assumptions which seem to hold on these, and other, applications, algorithm E has an expected constant loss compared to the offline optimal solution. Algorithm R reduces the number of optimizations by a factor |R|, where R is the number of requests, and has an expected ρ(1+o(1)) loss when the regret gives a ρ-approximation to the offline problem.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Benoist, T., Bourreau, E., Caseau, Y., & Rottembourg, B. (2001). Towards stochastic constraint programming: A study of online multi-choice knapsack with deadlines. In Proceedings of the seventh international conference on principles and practice of constraint programming (CP’01), London, UK (pp. 61–76). Berlin: Springer.

    Google Scholar 

  • Bent, R., & Van Hentenryck, P. (2004a). A two-stage hybrid local search for the vehicle routing problem with time windows. Transportation Science, 8(4), 515–530.

    Article  Google Scholar 

  • Bent, R., & Van Hentenryck, P. (2004b). Online stochastic and robust optimization. In Proceeding of the 9th Asian computing science conference (ASIAN’04), Chiang Mai University, Thailand, December 2004.

  • Bent, R., & Van Hentenryck, P. (2004c). Regrets only. Online stochastic optimization under time constraints. In Proceedings of the 19th national conference on artificial intelligence (AAAI’04), San Jose, CA, July 2004.

  • Bent, R., & Van Hentenryck, P. (2004d). Scenario based planning for partially dynamic vehicle routing problems with stochastic customers. Operations Research, 52(6).

  • Bent, R., & Van Hentenryck, P. (2004e). The value of consensus in online stochastic scheduling. In Proceedings of the 14th international conference on automated planning & scheduling (ICAPS 2004), Whistler, British Columbia, Canada, 2004.

  • Bent, R., & Van Hentenryck, P. (2005). Online stochastic optimization without distributions. In Proceedings of the 15th international conference on automated planning & scheduling (ICAPS 2005), Monterey, CA, 2005.

  • Bent, R., Katriel, I., & Van Hentenryck, P. (2005). Sub-optimality approximation. In Eleventh international conference on principles and practice of constraint programming, Stiges, Spain, 2005.

  • Bertsekas, D., & Tsitsiklis, J. (1996). Neuro-dynamic programming. Belmont: Athena Scientific.

    Google Scholar 

  • Birge, J. R., & Louveaux, F. (1997). Introduction to stochastic programming. New York: Springer.

    Google Scholar 

  • Blair, C. E., & Jeroslow, R. G. (1979). On the value function of an integer program. Mathematical Programming, 23.

  • Borodin, A., & El-Yaniv, R. (1998). Online computation and competitive analysis. Cambridge: Cambridge University Press.

    Google Scholar 

  • Borodin, A., Kleinberg, J., Raghawan, P., Sudan, M., & Williamson, D. (2001). Adversarial queuing theory. Journal of the Association for Computing Machinery, 48(1), 13–38.

    Google Scholar 

  • Campbell, A., & Savelsbergh, M. (2002). Decision support for consumer direct grocery initiatives (Report TLI-02-09). Georgia Institute of Technology.

  • Chang, H., Givan, R., & Chong, E. (2000). On-line scheduling via sampling. In Artificial intelligence planning and scheduling (AIPS’00) (pp. 62–71).

  • Choi, J., Realff, M., & Lee, J. (2004). Dynamic programming in a heuristically confined state space: a stochastic resource-constrained project scheduling application. Computers and Chemical Engineering, 28, 2039–2058.

    Article  Google Scholar 

  • Csirik, J., Johnson, D. S., Kenyon, C., Orlin, J. B., Shor, P. W., & Weber, R. R. (2000). On the sum-of-squares algorithm for bin packing. In Proc. 32nd annual ACM symp. on theory of computing (STOCS’2000) (pp. 208–217).

  • de Farias, D., Van Roy, B. (2002). Approximate linear programming for average-cost dynamic programming. In S. Becker, S. Thrun, & K. Obermayer (Eds.), NIPS (pp. 1587–1594). Cambridge: MIT Press.

    Google Scholar 

  • Dean, B., Goemans, M. X., & Vondrak, J. (2004). Approximating the stochastic knapsack problem: the benefit of adaptivity. In Proceedings of the 45th annual IEEE symposium on foundations of computer science (pp. 208–217). Rome, Italy, 2004.

  • Fiat, A., & Woeginger, G. (1998). Online algorithms: the state of the art.

  • Flaxman, A., Frieze, A., & Krivelevich, M. (2005). On the random 2-stage minimum spanning tree. In ACM-SIAM symposium on discrete algorithms (SODA-2005).

  • Higle, J. L., & Sen, S. (1996). Stochastic decomposition. Dordrecht: Kluwer Academic.

    Google Scholar 

  • Kall, P., & Wallace, S. W. (1994). Stochastic programming. Chichester: Wiley.

    Google Scholar 

  • Karlin, A., Manasse, M., Rudolph, L., & Sleator, D. (1988). Competitive snoopy caching. Algorithmica, 3, 79–119.

    Article  Google Scholar 

  • Kesselman, A., Lotker, Z., & Mansour, Y. (2004). Buffer overflow management in QoQ switches. SIAM Journal on Computing, 33, 563–583.

    Article  Google Scholar 

  • Kleywegt, A., & Shapiro, A. (1999). The sample average approximation method for stochastic discrete optimization. SIAM Journal on Optimization, 15(1–2), 1–30.

    Google Scholar 

  • Koutsoupias, E., & Papadimitriou, C. H. (2000). Beyond competitive analysis. SIAM Journal of Computing, 30(1), 300–317.

    Article  Google Scholar 

  • Larsen, A., Madsen, O., & Solomon, M. (2002). Partially dynamic vehicle routing-models and algorithms. Journal of Operational Research Society, 53, 637–646.

    Article  Google Scholar 

  • Moehring, R. H., Radermacher, F. J., & Weiss, G. (2004). Stochastic scheduling problems. I: General strategies. ZOR—Zeitschrift für Operations Research, 28, 193–260.

    Article  Google Scholar 

  • Moehring, R. H., Schulz, A., & Uetz, M. (1999). Approximation in stochastic scheduling: the power of LP-based priority policies. Journal of the Association for Computing Machinery, 46(6), 924–942.

    Google Scholar 

  • Nikovski, D., & Branch, M. (2003). Marginalizing out future passengers in group elevator control. In Uncertainty in Artificial Intelligence (UAI’03), 2003.

  • Powell, W. B., & van Roy, B. (2004). Approximate dynamic programming for high-dimensional dynamic resource allocation problems. In Handbook of learning and approximate dynamic programming (pp. 261–279).

  • Puterman, M. (1994). Markov decision processes. New York: Wiley.

    Book  Google Scholar 

  • Romisch, W., & Schultz, R. (1993). Stability of solutions for stochastic programs with complete recourse. Mathematics of Operations Research, 18, 590–609.

    Article  Google Scholar 

  • Ross, S. (1997). A first course in probability. Englewood Cliffs: Prentice Hall.

    Google Scholar 

  • Schultz, R. (1993). Continuity properties of expectation functions in stochastic integer programming. Mathematics of Operations Research, 18(3), 578–589.

    Article  Google Scholar 

  • Schultz, R. (1995). On structure and stability in stochastic programs with random technology matrix and complete integer recourse. Mathematical Programming, 70(1), 73–89.

    Article  Google Scholar 

  • Schultz, R., Stougie, L., & van der Vlerk, M. H. (1996). Two-stage stochastic integer programming: a survey. Statistica Neerlandica. Journal of the Netherlands Society for Statistics and Operations Research, 50(3), 404–416.

    Google Scholar 

  • Schultz, R., Stougie, L., & van der Vlerk, M. H. (1998). Solving stochastic programs with integer recourse by enumeration: a framework using Gröbner basis reductions. Mathematical Programming, 83, 229–252.

    Google Scholar 

  • Shapiro, A. (2005). On complexity of multistage stochastic programs. www.optimization-online.org.

  • Shapiro, A., & Homem-de Mello, T. (1998). A simulation-based approach to two-stage stochastic programming with recourse. Mathematical Programming, Ser. A, 81(3), 301–325.

    Article  Google Scholar 

  • Shaw, P. (1998). Using constraint programming and local search methods to solve vehicle routing problems. In Proceedings of fourth international conference on the principles and practice of constraint programming (CP’98) (pp. 417–431). Pisa, October 1998.

  • Shmoys, D., & Swamy, C. (2004). Stochastic optimization is (almost) as easy as deterministic computation. In IEEE symp. foundations of computer science (FOCS-2004) (pp. 228–237).

  • Skutella, M., & Uetz, M. (2005). Stochastic machine scheduling with precedence constraints. SIAM Journal of Computing, 34(4), 788–802.

    Article  Google Scholar 

  • Solomon, M. M. (1987). Algorithms for the vehicle routing and scheduling problems with time window constraints. Operations Research, 35(2), 254–265.

    Article  Google Scholar 

  • Stougie, L. (1985). Design and analysis of methods for stochastic integer programming. PhD thesis, University of Amsterdam.

  • Stougie, L., & van der Vlerk, M. H. (1997). Stochastic integer programming. In M. Dell’Amico et al. (eds.) Annotated bibliographies in combinatorial optimization (pp. 127–141). New York: Wiley.

    Google Scholar 

  • van der Vlerk, M. H. (1995). Stochastic programming with integer recourse. PhD thesis, University of Groningen, The Netherlands.

Download references

Author information

Authors and Affiliations

  1. Brown University, Box 1910, Providence, RI, 02912, USA

    Pascal Van Hentenryck, Russell Bent & Eli Upfal

Authors
  1. Pascal Van Hentenryck
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Russell Bent
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eli Upfal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pascal Van Hentenryck.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Van Hentenryck, P., Bent, R. & Upfal, E. Online stochastic optimization under time constraints. Ann Oper Res 177, 151–183 (2010). https://doi.org/10.1007/s10479-009-0605-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10479-009-0605-5

Keywords

Search

Navigation