A new three-term conjugate gradient algorithm with modified gradient-differences for solving unconstrained optimization problems

Jie Guo; Zhong Wan; Jie Guo; Zhong Wan

doi:10.3934/math.2023128

AIMS Mathematics

2023, Volume 8, Issue 2: 2473-2488. doi: 10.3934/math.2023128

Previous Article Next Article

Research article

A new three-term conjugate gradient algorithm with modified gradient-differences for solving unconstrained optimization problems

Jie Guo ¹,
Zhong Wan ^{2
,
,}

1.
School of Mathematical Sciences, Changsha Normal University, Changsha 410081, China
2.
School of Mathematics and Statistics, Central South University, Changsha 410083, China

Received: 05 September 2022 Revised: 05 October 2022 Accepted: 13 October 2022 Published: 04 November 2022
MSC : 90C25, 90C30

Unconstrained optimization problems often arise from mining of big data and scientific computing. On the basis of a modified gradient-difference, this article aims to present a new three-term conjugate gradient algorithm to efficiently solve unconstrained optimization problems. Compared with the existing nonlinear conjugate gradient algorithms, the search directions in this algorithm are always sufficiently descent independent of any line search, as well as having conjugacy property. Using the standard Wolfe line search, global and local convergence of the proposed algorithm is proved under mild assumptions. Implementing the developed algorithm to solve 750 benchmark test problems available in the literature, it is shown that the numerical performance of this algorithm is remarkable, especially in comparison with that of the other similar efficient algorithms.
- optimization problems,
- conjugate gradient method,
- global convergence,
- line search,
- numerical simulation
Citation: Jie Guo, Zhong Wan. A new three-term conjugate gradient algorithm with modified gradient-differences for solving unconstrained optimization problems[J]. AIMS Mathematics, 2023, 8(2): 2473-2488. doi: 10.3934/math.2023128

Related Papers:

Abstract

Unconstrained optimization problems often arise from mining of big data and scientific computing. On the basis of a modified gradient-difference, this article aims to present a new three-term conjugate gradient algorithm to efficiently solve unconstrained optimization problems. Compared with the existing nonlinear conjugate gradient algorithms, the search directions in this algorithm are always sufficiently descent independent of any line search, as well as having conjugacy property. Using the standard Wolfe line search, global and local convergence of the proposed algorithm is proved under mild assumptions. Implementing the developed algorithm to solve 750 benchmark test problems available in the literature, it is shown that the numerical performance of this algorithm is remarkable, especially in comparison with that of the other similar efficient algorithms.

References

[1]	J. Y. Tang, Z. Wan, Orthogonal dual graph-regularized nonnegative matrix factorization for co-clustering, J. Sci. Comput., 87 (2021), 1–37. https://doi.org/10.1007/s10915-021-01489-w doi: 10.1007/s10915-021-01489-w
[2]	T. Li, Z. Wan, New adaptive barzilar-borwein step size and its application in solving large scale optimization problems, ANZIAM J., 61 (2019), 76–98. https://doi.org/10.1017/S1446181118000263 doi: 10.1017/S1446181118000263
[3]	C. M. Hu, Z. M. Wan, S. Zhu, Z. Wan, An integrated stochastic model and algorithm for constrained multi-item newsvendor problems by two-stage decision-making approach, Math. Comput. Simul., 193 (2022), 280–300. https://doi.org/10.1016/j.matcom.2021.10.018 doi: 10.1016/j.matcom.2021.10.018
[4]	Z. Wan, C. Q. Yu, Reutilization of the existent sales network for recycling unwanted smartphones by nonlinear optimization, J. Cleaner Prod., 350 (2022), 131349. https://doi.org/10.1016/j.jclepro.2022.131349 doi: 10.1016/j.jclepro.2022.131349
[5]	S. Huang, Z. Wan, A new nonmonotone spectral residual method for nonsmooth nonlinear equations, J. Comput. Appl. Math., 313 (2017), 82–101. http://doi.org/10.1016/j.cam.2016.09.014 doi: 10.1016/j.cam.2016.09.014
[6]	S. Deng, Z. Wan, X. Chen, An improved spectral conjugate gradient algorithm for nonconvex unconstrained optimization problems, J. Optim. Theory Appl., 157 (2013), 820–842. http://doi.org/10.1007/s10957-012-0239-7 doi: 10.1007/s10957-012-0239-7
[7]	W. W. Hager, H. C. Zhang, A new conjugate gradient method with guaranteed descent and an efficient line search, SIAM J. Optim., 16 (2005), 170–192. http://doi.org/10.1137/030601880 doi: 10.1137/030601880
[8]	N. Andrei, A simple three-term conjugate gradient algorithm for unconstrained optimization, J. Comput. Appl. Math., 241 (2013), 19–29. http://doi.org/10.1016/j.cam.2012.10.002 doi: 10.1016/j.cam.2012.10.002
[9]	N. Andrei, On three-term conjugate gradient algorithms for unconstrained optimization, Appl. Math. Comput., 219 (2013), 6316–6327. http://doi.org/10.1016/j.amc.2012.11.097 doi: 10.1016/j.amc.2012.11.097
[10]	L. Zhang, W. Zhou, D. H. Li, A descent modified Polak-Ribi$\grave{e}$re-Polyak conjugate gradient method and its global convergence, IMA J. Numer. Anal., 26 (2006), 629–640. http://doi.org/10.1093/imanum/drl016 doi: 10.1093/imanum/drl016
[11]	W. L. Liu, Z. M. Wan, Z. Wan, B. Gong, Sustainable recycle network of heterogeneous pharmaceuticals with governmental subsidies and service-levels of third-party logistics by bi-level programming approach, J. Cleaner Prod., 249 (2020), 119324. http://doi.org/10.1016/j.jclepro.2019.119324 doi: 10.1016/j.jclepro.2019.119324
[12]	J. Guo, Z. Wan, A new three-term spectral conjugate gradient algorithm with higher numerical performance for solving large scale optimization problems based on Quasi-Newton equation, Int. J. Model. Simul. Sci. Comput., 12 (2021), 2150053. http://doi.org/10.1142/S1793962321500537 doi: 10.1142/S1793962321500537
[13]	M. K. Riahi, I. A. Qattan, On the convergence rate of Fletcher-Reeves nonlinear conjugate gradient methods satisfying strong Wolfe conditions: application to parameter identification in problems governed by general dynamics, Math. Methods Appl. Sci., 45 (2022), 3644–3664. http://doi.org/10.1002/mma.8009 doi: 10.1002/mma.8009
[14]	M. Powell, Restart procedures for the conjugate gradient method, Math. Program., 12 (1977), 241–254. http://doi.org/10.1007/BF01593790 doi: 10.1007/BF01593790
[15]	L. Nazareth, A conjugate direction algorithm without line searches, J. Optim. Theory Appl., 23 (1977), 373–387. http://doi.org/10.1007/BF00933447 doi: 10.1007/BF00933447
[16]	N. Andrei, A modified Polak-Ribi$\grave{e}$re-Polyak conjugate gradient algorithm for unconstrained optimization, Optimization, 60 (2011), 1457–1471. http://doi.org/10.1080/02331931003653187 doi: 10.1080/02331931003653187
[17]	L. Zhang, W. Zhou, D. H. Li, Some descent three-term conjugate gradient methods and their global convergence, Optim. Methods Software, 22 (2007), 697–711. http://doi.org/10.1080/10556780701223293 doi: 10.1080/10556780701223293
[18]	J. K. Liu, Y. M. Feng, L. M. Zou, Some three-term conjugate gradient methods with the inexact line search condition, Calcolo, 55 (2018), 16. http://doi.org/10.1007/s10092-018-0258-3 doi: 10.1007/s10092-018-0258-3
[19]	D. Touati-Ahmed, C. Storey, Efficient hybrid conjugate gradient techniques, J. Optim. Theory Appl., 64 (1990), 379–397. http://doi.org/10.1007/bf00939455 doi: 10.1007/bf00939455
[20]	M. Al-Baali, Descent property and global convergence of the Fletcher-Reeves method with inexact line search, J. Numer. Anal., 5 (1985), 121–124. http://doi.org/10.1093/imanum/5.1.121 doi: 10.1093/imanum/5.1.121
[21]	J. C. Gilbert, J. Nocedal, Global convergence properties of conjugate gradient methods for optimization, SIAM J. Optim., 2 (1992), 21–42. http://doi.org/10.1137/0802003 doi: 10.1137/0802003
[22]	Y. F. Hu, C. Storey, Global convergence result for conjugate gradient methods, J. Optim. Theory Appl., 71 (1991), 399–405. http://doi.org/10.1007/bf00939927 doi: 10.1007/bf00939927
[23]	S. Huang, Z. Wan, J. Zhang, An extended nonmonotone line search technique for large-scale unconstrained optimization, J. Comput. Appl. Math., 330 (2018), 586–604. http://doi.org/10.1016/j.cam.2017.09.026 doi: 10.1016/j.cam.2017.09.026
[24]	H. C. Zhang, W. W. Hager, A nonmonotone line search technique and its application to unconstrained optimization, SIAM J. Optim., 14 (2004), 1043–1056. http://doi.org/10.1137/S1052623403428208 doi: 10.1137/S1052623403428208
[25]	N. Andrei, An unconstrained optimization test functions collection, Adv. Model. Optim., 10 (2008), 147–161.
[26]	I. Bongartz, A. R. Conn, N. Gould, P. L. Toint, Cute: Constrained and unconstrained testing environment, ACM Trans. Math. Software, 21 (1995), 123–160. http://doi.org/10.1145/200979.201043 doi: 10.1145/200979.201043
[27]	E. D. Dolan, J. J. Mor$\acute{e}$, Benchmarking optimization software with performance profiles, Math. Progra., 91 (2002), 201–213. http://doi.org/10.1007/s101070100263 doi: 10.1007/s101070100263
[28]	J. Guo, Z. Wan, A modified spectral PRP conjugate gradient projection method for solving large-scale monotone equations and its application in compressed sensing, Math. Probl. Eng., 2019 (2019), 5261830. http://doi.org/10.1155/2019/5261830 doi: 10.1155/2019/5261830
[29]	A. Althobaiti, J. Sabi'u, H. Emadifar, P. Junsawang, S. K. Sahoo, A scaled Dai-Yuan projection-based conjugate gradient method for solving monotone equations with applications, Symmetry, 14 (2022), 1401. http://doi.org/10.3390/sym14071401 doi: 10.3390/sym14071401
[30]	J. Sabi'u, K. O. Aremu, A. Althobaiti, A. Shah, Scaled three-term conjugate gradient methods for solving monotone equations with application, Symmetry, 14 (2022), 936. http://doi.org/10.3390/sym14050936 doi: 10.3390/sym14050936
[31]	J. Sabi'u, A. Shah, M. Y. Waziri, A modified Hager-Zhang conjugate gradient method with optimal choices for solving monotone nonlinear equations, Int. J. Comput. Math., 99 (2022), 332–354. http://doi.org/10.1080/00207160.2021.1910814 doi: 10.1080/00207160.2021.1910814
[32]	N. Ullah, J. Sabi'u, A. Shah, A derivative-free scaling memoryless Broyden-Fletcher-Goldfarb-Shanno method for solving a system of monotone nonlinear equations, Numer. Linear Algebra Appl., 28 (2021), 2374. http://doi.org/10.1002/nla.2374 doi: 10.1002/nla.2374
[33]	A. B. Abubakar, J. Sabi'u, P. Kumam, A. Shah, Solving nonlinear monotone operator equations via modified SR1 update, J. Appl. Math. Comput., 67 (2021), 343–373. http://doi.org/10.1007/s12190-020-01461-1 doi: 10.1007/s12190-020-01461-1
[34]	J. Sabi'u, A. Shah, M. Y. Waziri, K. Ahmed, Modified Hager-Zhang conjugate gradient methods via singular value analysis for solving monotone nonlinear equations with convex constraint, Int. J. Comput. Methods, 18 (2021), 2050043. http://doi.org/10.1142/S0219876220500437 doi: 10.1142/S0219876220500437
[35]	J. Sabi'u, K. Muangchoo, A. Shah, A. B. Abubakar, K. O. Aremu, An inexact optimal hybrid conjugate gradient method for solving symmetric nonlinear equations, Symmetry, 13 (2021), 1829. http://doi.org/10.3390/sym13101829 doi: 10.3390/sym13101829

Reader Comments

Your name:*

Email:*
© 2023 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)