Skip to main content
SpringerLink
Log in

Combining Trust-Region Techniques and Rosenbrock Methods to Compute Stationary Points

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

Rosenbrock methods are popular for solving a stiff initial-value problem of ordinary differential equations. One advantage is that there is no need to solve a nonlinear equation at every iteration, as compared with other implicit methods such as backward difference formulas or implicit Runge–Kutta methods. In this article, we introduce a trust-region technique to select the time steps of a second-order Rosenbrock method for a special initial-value problem, namely, a gradient system obtained from an unconstrained optimization problem. The technique is different from the local error approach. Both local and global convergence properties of the new method for solving an equilibrium point of the gradient system are addressed. Finally, some promising numerical results are also presented.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Botsaris, C.A.: A class of methods for unconstrained minimization based on stable numerical integration techniques. J. Math. Anal. Appl. 63, 729–749 (1978)

    Article  MathSciNet  Google Scholar 

  2. Brown, A.A., Bartholomew-Biggs, M.C.: Some effective methods for unconstrained optimization based on the solution of systems of ordinary differential equations. J. Optim. Theory Appl. 62, 211–224 (1989)

    Article  MathSciNet  Google Scholar 

  3. Evtushenko, Yu.G.: Numerical Optimization Techniques. Optimization Software Publications, New York (1985)

    Google Scholar 

  4. Luo, X.L., Liao, L.-Z., Tam, H.W.: Convergence analysis of the Levenberg–Marquardt method. Optim. Methods Softw. 22, 659–678 (2007)

    Article  MathSciNet  Google Scholar 

  5. Schropp, J.: Using dynamical systems methods to solving minimization problems. Appl. Numer. Math. 18, 321–335 (1995)

    Article  MathSciNet  Google Scholar 

  6. Han, Q.M., Liao, L.-Z., Qi, H.D., Qi, L.Q.: Stability analysis of gradient-based neural networks for optimization problems. J. Glob. Optim. 19, 363–381 (2001)

    Article  MathSciNet  Google Scholar 

  7. Liao, L.-Z., Qi, H.D., Qi, L.Q.: Neurodynamical optimization. J. Glob. Optim. 28, 175–195 (2004)

    Article  MathSciNet  Google Scholar 

  8. Helmke, U., Moore, J.B.: Optimization and Dynamical Systems. Springer, Berlin (1994)

    Google Scholar 

  9. Łojasiewicz, S.: Sur les trajectoires de gradient d’une fonction analytique. Seminari di Geometria 1982–1983 (Lecture notes), pp. 115–117. Dipartemento di Matematica, Universita di Bologna (1984)

  10. Absil, P.-A., Mahony, R., Andrews, B.: Convergence of the iterates of descent methods for analytic cost functions. SIAM J. Optim. 16(2), 531–547 (2005)

    Article  MathSciNet  Google Scholar 

  11. Lageman, C.: Konvergenz reell-analytischer gradientenähnlicher Systeme. Diplomarbeit im Fach Mathematik, Mathematisches Institut, Universität Würzburg, Germany (2002)

  12. Grantham, W.J.: Trajectory following optimization by gradient transformation differential equations. In: Proceedings of the 42nd IEEE Conference on Decision and Control, Maui, HI, 9–12 December 2003, pp. 5496–5501

  13. Hairer, E., Nørsett, S.P., Wanner, G.: Solving Ordinary Differential Equations I: Nonstiff Problems, 2nd edn. Springer, Berlin (1993)

    Google Scholar 

  14. Hairer, E., Wanner, G.: Solving Ordinary Differential Equations II: Stiff and Differential-Algebraic Problems, 2nd edn. Springer, Berlin (1996)

    Google Scholar 

  15. Humphries, A.R., Stuart, A.M.: Runge–Kutta methods for dissipative and gradient dynamical systems. SIAM J. Numer. Anal. 31, 1452–1485 (1994)

    Article  MathSciNet  Google Scholar 

  16. Stuart, A.M., Humphries, A.R.: The essential stability of local error control for dynamical systems. SIAM J. Numer. Anal. 32, 1940–1971 (1995)

    Article  MathSciNet  Google Scholar 

  17. Conn, A.R., Gould, N., Toint, Ph.L.: Trust-Region Methods. SIAM, Philadelphia (2000)

    Google Scholar 

  18. Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, Berlin (1999)

    Google Scholar 

  19. Coffey, T.S., Kelley, C.T., Keyes, D.E.: Pseudotransient continuation and differential-algebraic equations. SIAM J. Sci. Comput. 25, 553–569 (2003)

    Article  MathSciNet  Google Scholar 

  20. Fowler, K.R., Kelley, C.T.: Pseudo-transient continuation for nonsmooth nonlinear equations. SIAM J. Numer. Anal. 43, 1385–1406 (2005)

    Article  MathSciNet  Google Scholar 

  21. Rosenbrock, H.H.: Some general implicit processes for the numerical solution of differential equations. Comput. J. 5, 329–331 (1963)

    Article  MathSciNet  Google Scholar 

  22. Shampine, L.F.: Implementation of Rosenbrock methods. ACM Trans. Math. Softw. 8, 93–113 (1982)

    Article  MathSciNet  Google Scholar 

  23. Shampine, L.F., Reichelt, M.W.: The Matlab ODE suite. SIAM J. Sci. Comput. 18, 1–22 (1997)

    Article  MathSciNet  Google Scholar 

  24. Kelley, C.T., Liao, L.-Z., Qi, L.Q., Chu, M.T., Reese, J.P., Winton, C.: Projected pseudotransient continuation. SIAM J. Numer. Anal. 46, 3071–3083 (2008)

    Article  MathSciNet  Google Scholar 

  25. Higham, D.J.: Trust region algorithms and timestep selection. SIAM J. Numer. Anal. 37, 194–210 (1999)

    Article  MathSciNet  Google Scholar 

  26. Luo, X.L.: Singly diagonally implicit Runge–Kutta methods combining line search techniques for unconstrained optimization. J. Comput. Math. 23, 153–164 (2005)

    MathSciNet  Google Scholar 

  27. Kelley, C.T., Keyes, D.E.: Convergence analysis of pseudo-transient continuation. SIAM J. Numer. Anal. 35, 508–523 (1998)

    Article  MathSciNet  Google Scholar 

  28. Deuflhard, P.: Newton Methods for Nonlinear Problems: Affine Invariance and Adaptive Algorithms. Springer, Berlin (2004)

    Google Scholar 

  29. Fletcher, R.: Practical Methods of Optimization. Wiley, New York (1987)

    Google Scholar 

  30. Levenberg, K.: A method for the solution of certain problems in least squares. Q. Appl. Math. 2, 164–168 (1944)

    MathSciNet  Google Scholar 

  31. Marquardt, D.: An algorithm for least-squares estimation of nonlinear parameters. SIAM J. Appl. Math. 11, 431–441 (1963)

    Article  MathSciNet  Google Scholar 

  32. Yuan, Z.D., Fei, J.G., Liu, D.G.: The Numerical Solutions for Initial Value Problems of Stiff Ordinary Differential Equations. Science Press, Beijing (1987) (in Chinese)

    Google Scholar 

  33. Luo, X.L.: Unconstrained optimization methods based on dynamic systems (in Chinese). Ph. D. thesis, Institute of Computational Mathematics and Scientific Engineering/Computing, Chinese Academy of Sciences (2001) (in Chinese)

  34. Luo, X.L.: Convergence analysis of methods based on BDF for unconstrained optimization. Math. Numer. Sinica 25, 177–184 (2003) (in Chinese)

    Google Scholar 

  35. Powell, M.J.D.: Convergence properties of a class of minimization algorithms. In: Mangasarian, O.L., Meyer, R.R., Robinson, S.M. (eds.) Nonlinear Programming, vol. 2, pp. 1–27. Academic Press, New York (1975)

    Google Scholar 

  36. Dennis, J.E., Moré, J.J.: A characterization of superlinear convergence and its application to quasi-Newton methods. Math. Comput. 28, 549–560 (1974)

    Article  Google Scholar 

  37. Moré, J.J., Garbow, B.S., Hillstrom, K.E.: Testing unconstrained optimization software. ACM Trans. Math. Softw. 7, 17–41 (1981)

    Article  Google Scholar 

  38. MATLAB 6.5: The MathWorks Inc., http://www.mathworks.com (2003)

Download references

Author information

Authors and Affiliations

  1. School of Information Engineering, Beijing University of Posts and Telecommunications, P. O. Box 101, 100876, Peking, People’s Republic of China

    X.-L. Luo

  2. Key Laboratory of Information Processing and Intelligent Technology, Ministry of Information Industry, P. O. Box 101, 100876, Peking, People’s Republic of China

    X.-L. Luo

  3. Center for Research in Scientific Computation and Department of Mathematics, North Carolina State University, Box 8205, Raleigh, NC, 27695-8205, USA

    C. T. Kelley

  4. Department of Mathematics, Hong Kong Baptist University, Kowloon Tong, Kowloon, Hong Kong

    L.-Z. Liao

  5. Department of Computer Science, Hong Kong Baptist University, Kowloon Tong, Kowloon, Hong Kong

    H. W. Tam

Authors
  1. X.-L. Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. C. T. Kelley
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. L.-Z. Liao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. H. W. Tam
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to L.-Z. Liao.

Additional information

Communicated by T.L. Vincent.

This research was supported in part by Grant 2007CB310604 from National Basic Research Program of China, and #DMS-0404537 from the United States National Science Foundation, and Grant #W911NF-05-1-0171 from the United States Army Research Office, and the Research Grant Council of Hong Kong.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Luo, XL., Kelley, C.T., Liao, LZ. et al. Combining Trust-Region Techniques and Rosenbrock Methods to Compute Stationary Points. J Optim Theory Appl 140, 265–286 (2009). https://doi.org/10.1007/s10957-008-9469-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-008-9469-0

Keywords

Search

Navigation