Convergence analysis of adaptive trust region methods

Zhen-Jun Shi; Xiang-Sun Zhang; Jie Shen

RAIRO - Operations Research (2007)

  • Volume: 41, Issue: 1, page 105-121
  • ISSN: 0399-0559

Abstract

top
In this paper, we propose a new class of adaptive trust region methods for unconstrained optimization problems and develop some convergence properties. In the new algorithms, we use the current iterative information to define a suitable initial trust region radius at each iteration. The initial trust region radius is more reasonable in the sense that the trust region model and the objective function are more consistent at the current iterate. The global convergence, super-linear and quadratic convergence rate are analyzed under some mild conditions. Numerical results show that some special adaptive trust region methods are available and efficient in practical computation.

How to cite

top

Shi, Zhen-Jun, Zhang, Xiang-Sun, and Shen, Jie. "Convergence analysis of adaptive trust region methods." RAIRO - Operations Research 41.1 (2007): 105-121. <http://eudml.org/doc/250107>.

@article{Shi2007,
abstract = { In this paper, we propose a new class of adaptive trust region methods for unconstrained optimization problems and develop some convergence properties. In the new algorithms, we use the current iterative information to define a suitable initial trust region radius at each iteration. The initial trust region radius is more reasonable in the sense that the trust region model and the objective function are more consistent at the current iterate. The global convergence, super-linear and quadratic convergence rate are analyzed under some mild conditions. Numerical results show that some special adaptive trust region methods are available and efficient in practical computation. },
author = {Shi, Zhen-Jun, Zhang, Xiang-Sun, Shen, Jie},
journal = {RAIRO - Operations Research},
keywords = {Adaptive trust region method; unconstrained optimization; global convergence; super-linear convergence.},
language = {eng},
month = {6},
number = {1},
pages = {105-121},
publisher = {EDP Sciences},
title = {Convergence analysis of adaptive trust region methods},
url = {http://eudml.org/doc/250107},
volume = {41},
year = {2007},
}

TY - JOUR
AU - Shi, Zhen-Jun
AU - Zhang, Xiang-Sun
AU - Shen, Jie
TI - Convergence analysis of adaptive trust region methods
JO - RAIRO - Operations Research
DA - 2007/6//
PB - EDP Sciences
VL - 41
IS - 1
SP - 105
EP - 121
AB - In this paper, we propose a new class of adaptive trust region methods for unconstrained optimization problems and develop some convergence properties. In the new algorithms, we use the current iterative information to define a suitable initial trust region radius at each iteration. The initial trust region radius is more reasonable in the sense that the trust region model and the objective function are more consistent at the current iterate. The global convergence, super-linear and quadratic convergence rate are analyzed under some mild conditions. Numerical results show that some special adaptive trust region methods are available and efficient in practical computation.
LA - eng
KW - Adaptive trust region method; unconstrained optimization; global convergence; super-linear convergence.
UR - http://eudml.org/doc/250107
ER -

References

top
  1. R.H. Byrd, J. Nocedal and Y.X. Yuan, Global convergence of a class of quasi-Newton methods on convex problems. SIAM J. Numer. Anal.24 (1987) 1171–1190.  Zbl0657.65083
  2. A.R. Conn, N.I.M. Gould and P.L. Toint, Trust-Region Methods, Society for Industrial and Applied Mathematics (SIAM), Philadelphia PA (2000).  Zbl0958.65071
  3. G. Corradi, A trust region algorithm for unconstrained optimization. Int. J. Comput. Math.65 (1997) 109–119.  
  4. Y.H. Dai, D.C. Xu, A new family of trust region algorithms for unconstrained optimization. J. Comput. Math.21 (2003) 221–228.  Zbl1028.65071
  5. R. Fletcher, Practical Method of Optimization, Unconstrained Optimization, Vol. 1, John Wiley, New York (1980).  Zbl0439.93001
  6. J.Y. Fan, W.B. Ai and Q.Y. Zhang, A line search and trust region a lgorithm with trust region radius converging to zero. J. Comput. Math.22 (2004) 865–872.  Zbl1068.65074
  7. J.H. Fu and W.Y. Sun, Nonmonotone adaptive trust-region method for unconstrained optimization problems. Appl. Math. Comput.163 (2005) 489–504.  Zbl1069.65063
  8. E.M. Gertz, A quasi-Newton trust-region method. Math. Program. Ser. A100 (2004) 447–470.  Zbl1068.90108
  9. N.I.M. Gould, D. Orban, A. Sartenaer and P.L. Toint, Sensitivity of trust-region algorithms to their parameters. 4OR3 (2005) 227–241.  Zbl1086.65060
  10. N.I.M. Gould, D. Orban and P.L. Toint, Numerical methods for large-scale nonlinear optimization. Acta Numer.14 (2005) 299–361.  Zbl1119.65337
  11. L. Hei, A self-adaptive trust region algorithm. J. Comput. Math.21 (2003) 229–236.  Zbl1028.65072
  12. J.J. Moré, Recent developments in algorithms and software for trust region methods, in Mathematical Programming: The State of the Art, edited by A. Bachem, M. Grotchel and B. Korte, Springer- Verlag, Berlin (1983) 258–287.  
  13. J.J. Moré, B.S. Grabow and K.E. Hillstrom, Testing Unconstrained Optimization software. ACM Trans. Math. Software7 (1981) 17–41.  Zbl0454.65049
  14. D. Mauricio and N. Maculan, A trust region method for zero-one nonlinear programming. RAIRO Rech. Opér.31 (1997) 331–341.  Zbl0888.90122
  15. J. Nocedal and S.J. Wright, Numerical Optimization. Springer-Verlag, New York (1999).  Zbl0930.65067
  16. Q. Ni, A globally convergent method of moving asymptotes with trust region technique. Optim. Methods Softw.18 (2003) 283–297.  Zbl1042.90042
  17. M.J.D. Powell, Convergence properties of a class minimization algorithms, in Nonlinear Programming 2, edited by O.L. Mangasarian, R.R. Meyer and S.M. Robinson, Academic Press, New York (1975) 1–27.  
  18. M.J.D. Powell, On the global convergence of trust region algorithms for unconstrained optimization. Math. Prog.29 (1984) 297–303.  Zbl0569.90069
  19. A. Sartenaer, Automatic determination of an initial trust region in nonlinear programming. SIAM J. Sci. Comput.18 (1997) 1788–1803.  Zbl0891.90151
  20. G.A. Shultz, R.B. Schnabel and R.H. Byrd, A family of trust-region-based algorithms for unconstrained minimization with strong global convergence. SIAM J. Numer. Anal.22 (1985) 47–67.  Zbl0574.65061
  21. W.Y. Sun, Nonmonotone trust region method for solving optimization problems. Appl. Math. Comput.156 (2004) 159–174.  Zbl1059.65055
  22. J. Sun, On piecewise quadratic Newton and trust region problems. Math. Programming, Ser. B76 (1997) 451–467.  Zbl0872.90088
  23. Z.J. Shi and H.Q. Wang, A new self-adaptive trust region method for unconstrained optimization. Technical Report, College of Operations Research and Management, Qufu Normal University (2004).  
  24. J.Z. Zhang and C.X. Xu, Trust region dogleg path algorithms for unconstrained minimization, Management science in China (Kowloon, 1996). Ann. Oper. Res.87 (1999) 407–418.  Zbl0924.90124
  25. X.S. Zhang, Trust region method in neural network. Acta Math. Appl. Sinica12 (1996) 1–10.  
  26. X.S. Zhang, Trust region method in neural network. Acta Math. Appl. Sinica (English Ser.) 13 (1997) 342–352.  Zbl0885.68127
  27. X.S. Zhang, J.L. Zhang and L.Z. Liao, An adaptive trust region method and its convergence. Science in China45 (2002) 620-631.  Zbl1105.90361
  28. X.S. Zhang, Z.W. Chen and J.L. Zhang, A self-adaptive trust region method for unconstrained optimization. OR Transactions5 (2001) 53–62.  
  29. Y.X. Yuan, A review of trust region algorithms for optimization, in ICIAM 99 (Edinburgh), Oxford Univ. Press, Oxford (2000) 271–282.  Zbl0992.65061
  30. Y.X. Yuan, Trust region algorithms for nonlinear equations. Information1 (1998) 7–20.  Zbl1006.65048

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.