A new family of hybrid three-term conjugate gradient method for unconstrained optimization with application to image restoration and portfolio selection

Maulana Malik; Ibrahim Mohammed Sulaiman; Auwal Bala Abubakar; Gianinna Ardaneswari; Sukono; Maulana Malik; Ibrahim Mohammed Sulaiman; Auwal Bala Abubakar; Gianinna Ardaneswari; Sukono

doi:10.3934/math.2023001

AIMS Mathematics

2023, Volume 8, Issue 1: 1-28. doi: 10.3934/math.2023001

Previous Article Next Article

Research article

A new family of hybrid three-term conjugate gradient method for unconstrained optimization with application to image restoration and portfolio selection

1.
Department of Mathematics, Faculty of Mathematics and Natural Sciences, Universitas Indonesia, Depok 16424, Indonesia
2.
Institute of Strategic Industrial Decision Modelling, School of Quantitative Sciences, Universiti Utara Malaysia, Kedah 06010, Malaysia
3.
Numerical Optimization Research Group, Department of Mathematical Sciences, Faculty of Physical Sciences, Bayero University, Kano, Kano 700241, Nigeria
4.
Department of Mathematics and Applied Mathematics, Sefako Makgatho Health Sciences University, Ga-Rankuwa, Pretoria, Medun 204, South Africa
5.
Department of Mathematics, Faculty of Mathematics and Natural Sciences, Universitas Padjadjaran, Sumedang 45363, Indonesia

Received: 29 July 2022 Revised: 08 September 2022 Accepted: 15 September 2022 Published: 27 September 2022
MSC : 65K10, 90C52, 90C26

The conjugate gradient (CG) method is an optimization method, which, in its application, has a fast convergence. Until now, many CG methods have been developed to improve computational performance and have been applied to real-world problems. In this paper, a new hybrid three-term CG method is proposed for solving unconstrained optimization problems. The search direction is a three-term hybrid form of the Hestenes-Stiefel (HS) and the Polak-Ribiére-Polyak (PRP) CG coefficients, and it satisfies the sufficient descent condition. In addition, the global convergence properties of the proposed method will also be proved under the weak Wolfe line search. By using several test functions, numerical results show that the proposed method is most efficient compared to some of the existing methods. In addition, the proposed method is used in practical application problems for image restoration and portfolio selection.
- conjugate gradient method,
- unconstrained optimization,
- sufficient descent condition,
- global convergence,
- image restoration,
- portfolio selection
Citation: Maulana Malik, Ibrahim Mohammed Sulaiman, Auwal Bala Abubakar, Gianinna Ardaneswari, Sukono. A new family of hybrid three-term conjugate gradient method for unconstrained optimization with application to image restoration and portfolio selection[J]. AIMS Mathematics, 2023, 8(1): 1-28. doi: 10.3934/math.2023001

Related Papers:

Abstract

The conjugate gradient (CG) method is an optimization method, which, in its application, has a fast convergence. Until now, many CG methods have been developed to improve computational performance and have been applied to real-world problems. In this paper, a new hybrid three-term CG method is proposed for solving unconstrained optimization problems. The search direction is a three-term hybrid form of the Hestenes-Stiefel (HS) and the Polak-Ribiére-Polyak (PRP) CG coefficients, and it satisfies the sufficient descent condition. In addition, the global convergence properties of the proposed method will also be proved under the weak Wolfe line search. By using several test functions, numerical results show that the proposed method is most efficient compared to some of the existing methods. In addition, the proposed method is used in practical application problems for image restoration and portfolio selection.

References

[1]	A. B. Abubakar, P. Kumam, M. Malik, P. Chaipunya, A. H. Ibrahim, A hybrid FR-DY conjugate gradient algorithm for unconstrained optimization with application in portfolio selection, AIMS Math., 6 (2021), 6506–6527. https://doi.org/10.3934/math.2021383 doi: 10.3934/math.2021383
[2]	A. B. Abubakar, P. Kumam, M. Malik, A. H. Ibrahim, A hybrid conjugate gradient based approach for solving unconstrained optimization and motion control problems, Math. Comput. Simulat., 201 (2022), 640–657. https://doi.org/10.1016/j.matcom.2021.05.038 doi: 10.1016/j.matcom.2021.05.038
[3]	A. B. Abubakar, M. Malik, P. Kumam, H. Mohammad, M. Sun, A. H. Ibrahim, et al., A Liu-Storey-type conjugate gradient method for unconstrained minimization problem with application in motion control, J. King Saud Univ., Sci., 34 (2022). https://doi.org/10.1016/j.jksus.2022.101923
[4]	M. Al-Baali, Descent property and global convergence of the Fletcher-Reeves method with inexact line search, J. Inst. Math. Appl., 5 (1985), 121–124. https://doi.org/10.1093/imanum/5.1.121 doi: 10.1093/imanum/5.1.121
[5]	N. Andrei, Nonlinear optimization with financial applications, Cham, Switzerland: Springer, 2020.
[6]	A. M. Awwal, I. M. Sulaiman, M. Malik, M. Mamat, P. Kumam, K. Sitthithakerngkiet, A spectral RMIL+ conjugate gradient method for unconstrained optimization with applications in portfolio selection and motion control, IEEE Access, 9 (2021), 75398–75414. https://doi.org/10.1109/ACCESS.2021.3081570 doi: 10.1109/ACCESS.2021.3081570
[7]	S. Babaie-Kafaki, N. Mirhoseini, Z. Aminifard, A descent extension of a modified Polak-Ribière-Polyak method with application in image restoration problem, Optim. Lett., 2021, 1–17. https://doi.org/10.1007/s11590-022-01878-6
[8]	S. Babaie-Kafaki, R. Ghanbari, Two modified three-term conjugate gradient methods with sufficient descent property, Optim. Lett., 8 (2014), 2285–2297. https://doi.org/10.1007/s11590-014-0736-8 doi: 10.1007/s11590-014-0736-8
[9]	M. Bartholomew-Biggs, Nonlinear conjugate gradient methods for unconstrained optimization, Springer Science & Business Media, 2006.
[10]	J. Cao, J. Wu, A conjugate gradient algorithm and its applications in image restoration, Appl. Numer. Math., 152 (2020), 243–252. https://doi.org/10.1016/j.apnum.2019.12.002 doi: 10.1016/j.apnum.2019.12.002
[11]	Y. H. Dai, Y. Yuan, A nonlinear conjugate gradient method with a strong global convergence property, SIAM J. Optim., 10 (1999), 177–182. https://doi.org/10.1137/S1052623497318992 doi: 10.1137/S1052623497318992
[12]	J. Deepho, A. B. Abubakar, M. Malik, I. K. Argyros, Solving unconstrained optimization problems via hybrid CD-DY conjugate gradient methods with applications, J. Comput. Appl. Math., 405 (2022). https://doi.org/10.1016/j.cam.2021.113823
[13]	S. Devila, M. Malik, W. Giyarti, A new hybrid PRP-MMSIS conjugate gradient method and its application in portfolio selection, JRAM, 5 (2021), 47–59. https://doi.org/10.26740/jram.v5n1 doi: 10.26740/jram.v5n1
[14]	E. D. Dolan, J. J. Moré, Benchmarking optimization software with performance profiles, Math. Program., 91 (2002), 201–213. https://doi.org/10.26740/jram.v5n1.p47-59 doi: 10.26740/jram.v5n1.p47-59
[15]	F. J. Fabozzi, H. M. Markowitz, F. Gupta, Handbook of finance, Hoboken, NJ, USA: Wiley, 2008. https://doi.org/10.1002/9780470404324.hof002057
[16]	R. Fletcher, Practical methods of optimization, Hoboken, NJ, USA: Wiley, 2013.
[17]	R. Fletcher, C. M. Reeves, Function minimization by conjugate gradients, Comput. J., 7 (1964), 149–154. https://doi.org/10.1093/comjnl/7.2.149 doi: 10.1093/comjnl/7.2.149
[18]	G. T. Friedlob, F. J. Jr. Plewa, Understanding return on investment, John Wiley & Sons, 1996.
[19]	M. R. Hestenes, E. Stiefel, Methods of conjugate gradients for solving linear systems, J. Res. Nat. Bur. Stand., 49 (1952), 409–438.
[20]	R. V. Hogg, A. T. Craig, Introduction to mathematical statistics, Pearson, 2018.
[21]	R. V. Hogg, S. A. Klugman, Loss distributions, John Wiley & Sons, 2009.
[22]	A. H. Ibrahim, M. Kimiaei, P. Kumam, A new black box method for monotone nonlinear equations, Optimization, 2021, 1–19. https://doi.org/10.1080/02331934.2021.2002326
[23]	A. H. Ibrahim, P. Kumam, W. Kumam, A family of derivative-free conjugate gradient methods for constrained nonlinear equations and image restoration, IEEE Access, 8 (2020), 162714–162729. https://doi.org/10.1109/ACCESS.2020.3020969 doi: 10.1109/ACCESS.2020.3020969
[24]	A. H. Ibrahim, P. Kumam, A. Kamandi, A. B. Abubakar, An efficient hybrid conjugate gradient method for unconstrained optimization, Optim. Method. Softw., 8 (2022), 1–14. https://doi.org/10.1080/10556788.2021.1998490 doi: 10.1080/10556788.2021.1998490
[25]	J. Jian, W. Chen, X. Jiang, P. Liu, A three-term conjugate gradient method with accelerated subspace quadratic optimization, J. Appl. Math. Comput., 68 (2021), 2407–2433. https://doi.org/10.1007/s12190-021-01622-w doi: 10.1007/s12190-021-01622-w
[26]	J. Jian, L. Yang, X. Jiang, P. Liu, M. Liu, A spectral conjugate gradient method with descent property, Mathematics, 8 (2020). https://doi.org/10.3390/math8020280
[27]	X. Jiang, W. Liao, J. Yin, J. Jian, A new family of hybrid three-term conjugate gradient methods with applications in image restoration, Numer. Algorithms, 91 (2022), 161–191. https://doi.org/10.1007/s11075-022-01258-2 doi: 10.1007/s11075-022-01258-2
[28]	A. Kamandi, K. Amini, A globally convergent gradient-like method based on the armijo line search, J. Math. Model., 9 (2021), 665–676. https://doi.org/10.22124/JMM.2021.18854.1612 doi: 10.22124/JMM.2021.18854.1612
[29]	Y. Liu, C. Storey, Efficient generalized conjugate gradient algorithms, part 1: Theory, J. Optimiz. Theory. App., 69 (1991), 129–137. https://doi.org/10.1007/BF00940464 doi: 10.1007/BF00940464
[30]	M. Malik, S. S. Abas, M. Mamat, Sukono, I. S. Mohammed, A new hybrid conjugate gradient method with global convergence properties, Int. J. Adv. Sci. Techn., 29 (2020), 199–210.
[31]	M. Malik, A. B. Abubakar, S. M. Ibrahim, M. Mamat, S. S. Abas, S. Firman, A new three-term conjugate gradient method for unconstrained optimization with applications in portfolio selection and robotic motion control, IAENG Int. J. Appl. Math., 51 (2021), 471–486.
[32]	M. Malik, I. M. Sulaiman, M. Mamat, S. S. Abas, Sukono, A new class nonlinear conjugate gradient method for unconstrained optimization models and its application in portfolio selection, Nonlinear Funct. An. Appl., 26 (2021), 811–837.
[33]	M. Malik, M. Mamat, S. S. Abas, I. M. Sulaiman, Sukono, Performance analysis of new spectral and hybrid conjugate gradient methods for solving unconstrained optimization problems, IAENG Int. J. Comput. Sci., 48 (2021), 66–79.
[34]	H. M. Markowitz, G. P. Todd, Mean-variance analysis in portfolio choice and capital markets, John Wiley & Sons, 2000.
[35]	H. B. Mayo, Investments: An introduction, Cengage Learning, 2020.
[36]	S. K. Mishra, B. Ram, Introduction to unconstrained optimization with R, Springer Nature, 2019.
[37]	J. J. Moré, B. S. Garbow, K. E. Hillstrom, Testing unconstrained optimization software, ACM T. Math. Software, 7 (1981), 17–41.
[38]	Y. Narushima, H. Yabe, J. A. Ford, A three-term conjugate gradient method with sufficient descent property for unconstrained optimization, SIAM J. Optimiz., 21 (2011), 212–230. https://doi.org/10.1137/080743573 doi: 10.1137/080743573
[39]	J. Nocedal, Updating quasi-Newton matrices with limited storage, Math. Comp., 35 (1980), 773–782. https://doi.org/10.1090/S0025-5718-1980-0572855-7 doi: 10.1090/S0025-5718-1980-0572855-7
[40]	E. Polak, G. Ribiere, Note sur la convergence de méthodes de directions conjuguées, Math. Model. Numer. Anal., 3 (1969), 35–43.
[41]	B. T. Polyak, The conjugate gradient method in extremal problems, USSR Comput. Math. Phys., 9 (1969), 94–112. https://doi.org/10.1016/0041-5553(69)90035-4 doi: 10.1016/0041-5553(69)90035-4
[42]	M. J. D. Powell, Nonconvex minimization calculations and the conjugate gradient method, Lect. Notes. Math., 1984,122–141. https://doi.org/10.1007/BFb0099521
[43]	M. Rivaie, M. Mamat, L. W. June, I. Mohd, A new class of nonlinear conjugate gradient coeficients with global convergence properties, Appl. Math. Comput., 218 (2012), 11323–11332. https://doi.org/10.1016/j.amc.2012.05.030 doi: 10.1016/j.amc.2012.05.030
[44]	Z. Salleh, G. Alhamzi, I. Masmali, A. Alhawarat, A modified liu and storey conjugate gradient method for large scale unconstrained optimization problems, Algorithms, 14 (2021), 227. https://doi.org/10.3390/a14080227 doi: 10.3390/a14080227
[45]	D. F. Shanno, Conjugate gradient methods with inexact searches, Math. Oper. Res., 3 (1978), 244–256. https://doi.org/10.1287/moor.3.3.244 doi: 10.1287/moor.3.3.244
[46]	R. Steven, Introduction to the mathematics of finance: From risk management to options pricing, Springer Science & Business Media, 2004.
[47]	I. M. Sulaiman, M. Malik, A. M. Awwal, P. Kumam, M. Mamat, S. Al-Ahmad, On three-term conjugate gradient method for optimization problems with applications on COVID-19 model and robotic motion control, Adv. Cont. Discr. Mod., 2022 (2022), 1–22. https://doi.org/10.1186/s13662-021-03638-9 doi: 10.1186/s13662-021-03638-9
[48]	I. M. Sulaiman, M. Mamat, A new conjugate gradient method with descent properties and its application to regression analysis, J. Numer. Anal. Ind. Appl. Math., 14 (2020), 25–39.
[49]	Q. Tian, X. Wang, L. Pang, M. Zhang, F. Meng, A new hybrid three-term conjugate gradient algorithm for large-scale unconstrained problems, Mathematics, 9 (2021), 1353. https://doi.org/10.3390/math9121353 doi: 10.3390/math9121353
[50]	G. Yu, J. Huang, Y. Zhou, A descent spectral conjugate gradient method for impulse noise removal, Appl. Math. Lett., 23 (2010), 555–560. https://doi.org/10.1016/j.aml.2010.01.010 doi: 10.1016/j.aml.2010.01.010
[51]	J. Yin, J. Jian, X. Jiang, M. Liu, L. Wang, A hybrid three-term conjugate gradient projection method for constrained nonlinear monotone equations with applications, Numer. Algor., 88 (2021), 389–418. https://doi.org/10.1007/s11075-020-01043-z doi: 10.1007/s11075-020-01043-z
[52]	L. Zhang, W. Zhou, D. Li, A descent modified Polak-Ribière-Polyak conjugate gradient method and its global convergence, J. Inst. Math. Appl., 26 (2006), 629–640. https://doi.org/10.1093/imanum/drl016 doi: 10.1093/imanum/drl016
[53]	X. Zheng, J. Shi, A modified sufficient descent Polak-Ribiére-Polyak type conjugate gradient method for unconstrained optimization problems, Algorithms, 11 (2018), 133. https://doi.org/10.3390/a11090133 doi: 10.3390/a11090133

Reader Comments

Your name:*

Email:*
© 2023 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)