Convergence analysis of adaptive trust region methods

Zhen-Jun Shi; Xiang-Sun Zhang; Jie Shen

RAIRO - Operations Research (2007)

  • Volume: 41, Issue: 1, page 105-121
  • ISSN: 0399-0559

Abstract

top
In this paper, we propose a new class of adaptive trust region methods for unconstrained optimization problems and develop some convergence properties. In the new algorithms, we use the current iterative information to define a suitable initial trust region radius at each iteration. The initial trust region radius is more reasonable in the sense that the trust region model and the objective function are more consistent at the current iterate. The global convergence, super-linear and quadratic convergence rate are analyzed under some mild conditions. Numerical results show that some special adaptive trust region methods are available and efficient in practical computation.

How to cite

top

Shi, Zhen-Jun, Zhang, Xiang-Sun, and Shen, Jie. "Convergence analysis of adaptive trust region methods." RAIRO - Operations Research 41.1 (2007): 105-121. <http://eudml.org/doc/250107>.

@article{Shi2007,
abstract = { In this paper, we propose a new class of adaptive trust region methods for unconstrained optimization problems and develop some convergence properties. In the new algorithms, we use the current iterative information to define a suitable initial trust region radius at each iteration. The initial trust region radius is more reasonable in the sense that the trust region model and the objective function are more consistent at the current iterate. The global convergence, super-linear and quadratic convergence rate are analyzed under some mild conditions. Numerical results show that some special adaptive trust region methods are available and efficient in practical computation. },
author = {Shi, Zhen-Jun, Zhang, Xiang-Sun, Shen, Jie},
journal = {RAIRO - Operations Research},
keywords = {Adaptive trust region method; unconstrained optimization; global convergence; super-linear convergence.},
language = {eng},
month = {6},
number = {1},
pages = {105-121},
publisher = {EDP Sciences},
title = {Convergence analysis of adaptive trust region methods},
url = {http://eudml.org/doc/250107},
volume = {41},
year = {2007},
}

TY - JOUR
AU - Shi, Zhen-Jun
AU - Zhang, Xiang-Sun
AU - Shen, Jie
TI - Convergence analysis of adaptive trust region methods
JO - RAIRO - Operations Research
DA - 2007/6//
PB - EDP Sciences
VL - 41
IS - 1
SP - 105
EP - 121
AB - In this paper, we propose a new class of adaptive trust region methods for unconstrained optimization problems and develop some convergence properties. In the new algorithms, we use the current iterative information to define a suitable initial trust region radius at each iteration. The initial trust region radius is more reasonable in the sense that the trust region model and the objective function are more consistent at the current iterate. The global convergence, super-linear and quadratic convergence rate are analyzed under some mild conditions. Numerical results show that some special adaptive trust region methods are available and efficient in practical computation.
LA - eng
KW - Adaptive trust region method; unconstrained optimization; global convergence; super-linear convergence.
UR - http://eudml.org/doc/250107
ER -

References

top
  1. R.H. Byrd, J. Nocedal and Y.X. Yuan, Global convergence of a class of quasi-Newton methods on convex problems. SIAM J. Numer. Anal.24 (1987) 1171–1190.  
  2. A.R. Conn, N.I.M. Gould and P.L. Toint, Trust-Region Methods, Society for Industrial and Applied Mathematics (SIAM), Philadelphia PA (2000).  
  3. G. Corradi, A trust region algorithm for unconstrained optimization. Int. J. Comput. Math.65 (1997) 109–119.  
  4. Y.H. Dai, D.C. Xu, A new family of trust region algorithms for unconstrained optimization. J. Comput. Math.21 (2003) 221–228.  
  5. R. Fletcher, Practical Method of Optimization, Unconstrained Optimization, Vol. 1, John Wiley, New York (1980).  
  6. J.Y. Fan, W.B. Ai and Q.Y. Zhang, A line search and trust region a lgorithm with trust region radius converging to zero. J. Comput. Math.22 (2004) 865–872.  
  7. J.H. Fu and W.Y. Sun, Nonmonotone adaptive trust-region method for unconstrained optimization problems. Appl. Math. Comput.163 (2005) 489–504.  
  8. E.M. Gertz, A quasi-Newton trust-region method. Math. Program. Ser. A100 (2004) 447–470.  
  9. N.I.M. Gould, D. Orban, A. Sartenaer and P.L. Toint, Sensitivity of trust-region algorithms to their parameters. 4OR3 (2005) 227–241.  
  10. N.I.M. Gould, D. Orban and P.L. Toint, Numerical methods for large-scale nonlinear optimization. Acta Numer.14 (2005) 299–361.  
  11. L. Hei, A self-adaptive trust region algorithm. J. Comput. Math.21 (2003) 229–236.  
  12. J.J. Moré, Recent developments in algorithms and software for trust region methods, in Mathematical Programming: The State of the Art, edited by A. Bachem, M. Grotchel and B. Korte, Springer- Verlag, Berlin (1983) 258–287.  
  13. J.J. Moré, B.S. Grabow and K.E. Hillstrom, Testing Unconstrained Optimization software. ACM Trans. Math. Software7 (1981) 17–41.  
  14. D. Mauricio and N. Maculan, A trust region method for zero-one nonlinear programming. RAIRO Rech. Opér.31 (1997) 331–341.  
  15. J. Nocedal and S.J. Wright, Numerical Optimization. Springer-Verlag, New York (1999).  
  16. Q. Ni, A globally convergent method of moving asymptotes with trust region technique. Optim. Methods Softw.18 (2003) 283–297.  
  17. M.J.D. Powell, Convergence properties of a class minimization algorithms, in Nonlinear Programming 2, edited by O.L. Mangasarian, R.R. Meyer and S.M. Robinson, Academic Press, New York (1975) 1–27.  
  18. M.J.D. Powell, On the global convergence of trust region algorithms for unconstrained optimization. Math. Prog.29 (1984) 297–303.  
  19. A. Sartenaer, Automatic determination of an initial trust region in nonlinear programming. SIAM J. Sci. Comput.18 (1997) 1788–1803.  
  20. G.A. Shultz, R.B. Schnabel and R.H. Byrd, A family of trust-region-based algorithms for unconstrained minimization with strong global convergence. SIAM J. Numer. Anal.22 (1985) 47–67.  
  21. W.Y. Sun, Nonmonotone trust region method for solving optimization problems. Appl. Math. Comput.156 (2004) 159–174.  
  22. J. Sun, On piecewise quadratic Newton and trust region problems. Math. Programming, Ser. B76 (1997) 451–467.  
  23. Z.J. Shi and H.Q. Wang, A new self-adaptive trust region method for unconstrained optimization. Technical Report, College of Operations Research and Management, Qufu Normal University (2004).  
  24. J.Z. Zhang and C.X. Xu, Trust region dogleg path algorithms for unconstrained minimization, Management science in China (Kowloon, 1996). Ann. Oper. Res.87 (1999) 407–418.  
  25. X.S. Zhang, Trust region method in neural network. Acta Math. Appl. Sinica12 (1996) 1–10.  
  26. X.S. Zhang, Trust region method in neural network. Acta Math. Appl. Sinica (English Ser.) 13 (1997) 342–352.  
  27. X.S. Zhang, J.L. Zhang and L.Z. Liao, An adaptive trust region method and its convergence. Science in China45 (2002) 620-631.  
  28. X.S. Zhang, Z.W. Chen and J.L. Zhang, A self-adaptive trust region method for unconstrained optimization. OR Transactions5 (2001) 53–62.  
  29. Y.X. Yuan, A review of trust region algorithms for optimization, in ICIAM 99 (Edinburgh), Oxford Univ. Press, Oxford (2000) 271–282.  
  30. Y.X. Yuan, Trust region algorithms for nonlinear equations. Information1 (1998) 7–20.  

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.