New hybrid conjugate gradient method for nonlinear optimization with application to image restoration problems

Youcef Elhamam Hemici; Samia Khelladi; Djamel Benterki

Kybernetika (2024)

  • Issue: 4, page 535-552
  • ISSN: 0023-5954

Abstract

top
The conjugate gradient method is one of the most effective algorithm for unconstrained nonlinear optimization problems. This is due to the fact that it does not need a lot of storage memory and its simple structure properties, which motivate us to propose a new hybrid conjugate gradient method through a convex combination of β k R M I L and β k H S . We compute the convex parameter θ k using the Newton direction. Global convergence is established through the strong Wolfe conditions. Numerical experiments show the superior efficiency of our algorithm to solve unconstrained optimization problem compared to other considered methods. Applied to image restoration problem, our algorithm is competitive with existing algorithms and performs even better when the level of noise in the image is significant.

How to cite

top

Hemici, Youcef Elhamam, Khelladi, Samia, and Benterki, Djamel. "New hybrid conjugate gradient method for nonlinear optimization with application to image restoration problems." Kybernetika (2024): 535-552. <http://eudml.org/doc/299389>.

@article{Hemici2024,
abstract = {The conjugate gradient method is one of the most effective algorithm for unconstrained nonlinear optimization problems. This is due to the fact that it does not need a lot of storage memory and its simple structure properties, which motivate us to propose a new hybrid conjugate gradient method through a convex combination of $\beta _\{k\}^\{RMIL\}$ and $\beta _\{k\}^\{HS\}$. We compute the convex parameter $\theta _\{k\}$ using the Newton direction. Global convergence is established through the strong Wolfe conditions. Numerical experiments show the superior efficiency of our algorithm to solve unconstrained optimization problem compared to other considered methods. Applied to image restoration problem, our algorithm is competitive with existing algorithms and performs even better when the level of noise in the image is significant.},
author = {Hemici, Youcef Elhamam, Khelladi, Samia, Benterki, Djamel},
journal = {Kybernetika},
keywords = {unconstrained optimization; conjugate gradient method; descent direction; line search; image restoration},
language = {eng},
number = {4},
pages = {535-552},
publisher = {Institute of Information Theory and Automation AS CR},
title = {New hybrid conjugate gradient method for nonlinear optimization with application to image restoration problems},
url = {http://eudml.org/doc/299389},
year = {2024},
}

TY - JOUR
AU - Hemici, Youcef Elhamam
AU - Khelladi, Samia
AU - Benterki, Djamel
TI - New hybrid conjugate gradient method for nonlinear optimization with application to image restoration problems
JO - Kybernetika
PY - 2024
PB - Institute of Information Theory and Automation AS CR
IS - 4
SP - 535
EP - 552
AB - The conjugate gradient method is one of the most effective algorithm for unconstrained nonlinear optimization problems. This is due to the fact that it does not need a lot of storage memory and its simple structure properties, which motivate us to propose a new hybrid conjugate gradient method through a convex combination of $\beta _{k}^{RMIL}$ and $\beta _{k}^{HS}$. We compute the convex parameter $\theta _{k}$ using the Newton direction. Global convergence is established through the strong Wolfe conditions. Numerical experiments show the superior efficiency of our algorithm to solve unconstrained optimization problem compared to other considered methods. Applied to image restoration problem, our algorithm is competitive with existing algorithms and performs even better when the level of noise in the image is significant.
LA - eng
KW - unconstrained optimization; conjugate gradient method; descent direction; line search; image restoration
UR - http://eudml.org/doc/299389
ER -

References

top
  1. Andrei, N., , Adv. Model. Optim. 10 (2008), 147-161. MR2424936DOI
  2. Andrei, N., Nonlinear conjugate gradient methods for unconstrained optimization., Springer Optimization and its Applications, Romania 2020. MR4179461
  3. Hanachi, S. Ben, Sellami, B., Belloufi, M., , RAIRO - Oper. Res. 56 (2022), 2315-2327. MR4458849DOI
  4. Cai, J. F., Chan, R., Morini, B., Minimization of an edge-preserving regularization functional by conjugate gradient type methods., Image Processing Based on Partial Differential Equations. Mathematics and Visualization. Springer, Berlin, Heidelberg 2007. MR2424224
  5. Chan, R. H., Ho, C. W., Nikolova, M., , IEEE Trans. Image Process. 14 (2005), 10, 1479-1485. MR2170264DOI
  6. Dai, Y. H., Yuan, Y., , SIAM J. Optim. 10 (1999), 1, 177-182. Zbl0957.65061MR1740963DOI
  7. Dai, Y. H., Yuan, Y., 10.1023/A:1012930416777, Ann. Oper. Res. 103 (2001), 33-47. MR1868442DOI10.1023/A:1012930416777
  8. Delladji, S., Belloufi, M., Sellami, B., New hybrid conjugate gradient method as a convex combination of FR and BA methods., J. Inform. Optim. Sci. 42 (2021), 3, 591-602. 
  9. Djordjevic, S., , Acta Math. Scientia 39B (2019), 1, 214-228. MR4064244DOI
  10. Djordjevic, S., New hybrid conjugate gradient method as a convex combination of HS and FR methods., J. Appl. Math. Comput. 2 (2018), 9, 366-378. MR4064244
  11. Dolan, E. D., Moré, J. J., , Math. Program. 91 (2002), 201-213. Zbl1049.90004MR1875515DOI
  12. Fletcher, R., Practical Methods of Optimization., Unconstrained Optimization. Wiley, New York 1987. MR0955799
  13. Fletcher, R., Reeves, C. M., , Comput. J. 7 (1964), 2, 149-154. Zbl0132.11701MR0187375DOI
  14. Hager, W. W., Zhang, H., A survey of nonlinear conjugate gradient methods., Pacific J. Optim. 2 (2006), 35-58. MR2548208
  15. Hestenes, M. R., Steifel, E., , J. Res. Natl. Bur. Stand. 49 (1952), 6, 409-436. MR0060307DOI
  16. Jiang, X., Liao, W., Yin, J., Jian, J., , Numer. Algor. 91 (2022), 161-191. MR4466147DOI
  17. Liu, Y., Storey, C., , Part 1: Theory. J. Optim. Theory Appl. 69 (1991), 1, 129-137. MR1104590DOI
  18. Ma, G., Lin, H., Jin, W., Han, D., , J. Appl. Math. Comput. 68 (2022), 4733-4758. MR4519772DOI
  19. Malik, M., Sulaiman, I. M., Abubakar, A. B., Ardaneswari, G., Sukono, A., , AIMS Math. 8 (2023), 1-28. MR4501065DOI
  20. Mtagulwa, P., Kaelo, P., A convergent modified HS-DY hybrid conjugate gradient method for unconstrained optimization problems., J. Inform. Optim. Sci. 40 (2019), 1, 97-113. MR3895187
  21. Polak, E., Ribiere, G., Note sur la convergence des mé thodes de directions conjuguées., Rev. Française Inform. Recherche Opertionelle 16 (1969), 35-43. MR0255025
  22. Polyak, B. T., 10.1016/0041-5553(69)90035-4, U.S.S.R. Comput. Math. Phys. 9 (1969), 94-112. DOI10.1016/0041-5553(69)90035-4
  23. Powell, M. J. D., 10.1007/BF01593790, Math. Program. 2 (1977), 241-254. MR0478622DOI10.1007/BF01593790
  24. Rivaie, M., Mustafa, M., Leong, W. J., Ismail, M., , Appl. Math. Comput. 218 (2012), 22, 11323-11332. MR2942413DOI
  25. Rivaie, M., Mustafa, M., Abdelrhaman, A., , Appl. Math. Comput. 268 (2015), 1152-1163. MR3399494DOI
  26. Sellami, B., Chaib, Y., , Ann. Oper. Res. Springer 241 (2016), 497-513. MR3509428DOI
  27. Sellami, B., Chaib, Y., , RAIRO Oper. Res. 50 (2016), 1013-1026. MR3570546DOI
  28. Yang, X., Luo, Z., Dai, X., A global convergence of LS-CD hybrid conjugate gradient method., Adv. Numer. Anal. 2013 (2013), 5 pp. MR3125068
  29. Zoutendijk, G., Nonlinear programming, computational methods., In: Integer and Nonlinear Programming (J. Abadie, ed.), 1970, pp. 37-86. MR0437081

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.