A decent three term conjugate gradient method with global convergence properties for large scale unconstrained optimization problems

Ibtisam A. Masmali; Zabidin Salleh; Ahmad Alhawarat; Ibtisam A. Masmali; Zabidin Salleh; Ahmad Alhawarat

doi:10.3934/math.2021624

AIMS Mathematics

2021, Volume 6, Issue 10: 10742-10764. doi: 10.3934/math.2021624

Previous Article Next Article

Research article

A decent three term conjugate gradient method with global convergence properties for large scale unconstrained optimization problems

1.
Department of Mathematics, College of Science, Jazan University, Jazan, Saudi Arabia
2.
Department of Mathematics, Faculty of Ocean Engineering Technology and Informatics, Universiti Malaysia Terengganu, Kuala Nerus 21030, Terengganu, Malaysia
3.
Department of Mathematics, Faculty of Ocean Engineering Technology and Informatics, Universiti Malaysia Terengganu, Terengganu, Malaysia

Received: 04 May 2021 Accepted: 14 July 2021 Published: 26 July 2021
MSC : 49M37, 65K05, 90C3

The conjugate gradient (CG) method is a method to solve unconstrained optimization problems. Moreover CG method can be applied in medical science, industry, neural network, and many others. In this paper a new three term CG method is proposed. The new CG formula is constructed based on DL and WYL CG formulas to be non-negative and inherits the properties of HS formula. The new modification satisfies the convergence properties and the sufficient descent property. The numerical results show that the new modification is more efficient than DL, WYL, and CG-Descent formulas. We use more than 200 functions from CUTEst library to compare the results between these methods in term of number of iterations, function evaluations, gradient evaluations, and CPU time.
- conjugate gradient method,
- inexact line search,
- global convergence
Citation: Ibtisam A. Masmali, Zabidin Salleh, Ahmad Alhawarat. A decent three term conjugate gradient method with global convergence properties for large scale unconstrained optimization problems[J]. AIMS Mathematics, 2021, 6(10): 10742-10764. doi: 10.3934/math.2021624

Related Papers:

Abstract

The conjugate gradient (CG) method is a method to solve unconstrained optimization problems. Moreover CG method can be applied in medical science, industry, neural network, and many others. In this paper a new three term CG method is proposed. The new CG formula is constructed based on DL and WYL CG formulas to be non-negative and inherits the properties of HS formula. The new modification satisfies the convergence properties and the sufficient descent property. The numerical results show that the new modification is more efficient than DL, WYL, and CG-Descent formulas. We use more than 200 functions from CUTEst library to compare the results between these methods in term of number of iterations, function evaluations, gradient evaluations, and CPU time.

References

[1]	P. Wolfe, Convergence conditions for ascent methods, SIAM Rev., 11 (1969), 226-235. doi: 10.1137/1011036
[2]	P. Wolfe, Convergence conditions for ascent methods. Ⅱ: Some corrections, SIAM Rev., 13 (1971), 185-188. doi: 10.1137/1013035
[3]	M. R. Hestenes, E. Stiefel, Methods of conjugate gradients for solving linear systems, J. Res. Nat. Bur. Standards, 49 (1952), 409-435. doi: 10.6028/jres.049.044
[4]	E. Polak, G. Ribiere, Note sur la convergence de méthodes de directions conjuguées, ESAIM: Math. Modell. Numer. Anal., 3 (1969), 35-43.
[5]	Y. Liu, C. Storey, Efficient generalized conjugate gradient algorithms, part 1: Theory, J. Optim. Theory Appl., 69 (1991), 129-137. doi: 10.1007/BF00940464
[6]	R. Fletcher, C. M. Reeves, Function minimization by conjugate gradients, Comput. J., 7 (1964), 149-154. doi: 10.1093/comjnl/7.2.149
[7]	R. Fletcher, Practical methods of optimization, Vol. 1, Unconstrained Optimization, A Wiley-Interscience Publication, 1997.
[8]	Y. H. Dai, Y. Yuan, A nonlinear conjugate gradient method with a strong global convergence property, SIAM J. Optim., 10 (1999), 177-182. doi: 10.1137/S1052623497318992
[9]	M. J. Powell, Nonconvex minimization calculations and the conjugate gradient method, In: Numerical analysis, Springer, Berlin, Heidelberg, (1984), 122-141.
[10]	J. C. Gilbert, J. Nocedal, Global convergence properties of conjugate gradient methods for optimization, SIAM J. Optim., 2 (1992), 21-42. doi: 10.1137/0802003
[11]	G. Zoutendijk, Nonlinear programming, computational methods, Integer Nonlinear Program., (1970), 37-86.
[12]	M. Al-Baali, Descent property and global convergence of the Fletcher-Reeves method with inexact line search, IMA J. Numer. Anal., 5 (1985), 121-124. doi: 10.1093/imanum/5.1.121
[13]	Y. H. Dai, L. Z. Liao, New conjugacy conditions and related nonlinear conjugate gradient methods, Appl. Math. Optim., 43 (2001), 87-101. doi: 10.1007/s002450010019
[14]	W. W. Hager, H. Zhang, A new conjugate gradient method with guaranteed descent and an efficient line search, SIAM J. Optim., 16 (2005), 170-192. doi: 10.1137/030601880
[15]	W. W. Hager, H. Zhang, The limited memory conjugate gradient method, SIAM J. Optim., 23 (2013), 2150-2168. doi: 10.1137/120898097
[16]	Z. Wei, S. Yao, L. Liu, The convergence properties of some new conjugate gradient methods Appl. Math. Comput., 183 (2006), 1341-1350.
[17]	A. Alhawarat, Z. Salleh, M. Mamat, M. Rivaie, An efficient modified Polak-Ribière-Polyak conjugate gradient method with global convergence properties, Optim. Methods Software, 32 (2017), 1299-1312. doi: 10.1080/10556788.2016.1266354
[18]	P. Kaelo, P. Mtagulwa, M. V. Thuto, A globally convergent hybrid conjugate gradient method with strong Wolfe conditions for unconstrained optimization, Math. Sci., 14 (2020), 1-9. doi: 10.1007/s40096-019-00310-y
[19]	S. Yao, L. Ning, H. Tu, J. Xu, A one-parameter class of three-term conjugate gradient methods with an adaptive parameter choice, Optim. Methods Software, 35 (2020), 1051-1064. doi: 10.1080/10556788.2018.1510926
[20]	G. Yuan, J. Lu, Z. Wang, The PRP conjugate gradient algorithm with a modified WWP line search and its application in the image restoration problems, Appl. Numer. Math., 152 (2020), 1-11. doi: 10.1016/j.apnum.2020.01.019
[21]	G. Yuan, T. Li, W. Hu, A conjugate gradient algorithm for large-scale nonlinear equations and image restoration problems, Appl. Numer. Math., 147 (2020), 129-141. doi: 10.1016/j.apnum.2019.08.022
[22]	J. Liu, S. Du, Y. Chen, A sufficient descent nonlinear conjugate gradient method for solving M-tensor equations, J. Comput. Appl. Math., 371 (2020), 112709. doi: 10.1016/j.cam.2019.112709
[23]	S. Yao, Y. Wu, J. Yang, J. Xu, A three-term gradient descent method with subspace techniques, Math. Probl. Eng., 2021 (2021), 8867309.
[24]	A. Alhawarat, N. Trung, Z. Salleh, Conjugate gradient method: A developed version to resolve unconstrained optimization problems, J. Comput. Sci., 16 (2020), 1220-1228. doi: 10.3844/jcssp.2020.1220.1228
[25]	I. Bongartz, A. R. Conn, N. Gould, P. L. Toint, CUTE: Constrained and unconstrained testing environment, ACM Trans. Math. Software, 21 (1995), 123-160. doi: 10.1145/200979.201043
[26]	E. D. Dolan, J. J. Moré, Benchmarking optimization software with performance profiles, Math. Program., 91 (2002), 201-213. doi: 10.1007/s101070100263

Reader Comments

Your name:*

Email:*
© 2021 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)