A hybrid approach to conjugate gradient algorithms for nonlinear systems of equations with applications in signal restoration

Xuejie Ma; Songhua Wang; Xuejie Ma; Songhua Wang

doi:10.3934/math.20241717

AIMS Mathematics

2024, Volume 9, Issue 12: 36167-36190. doi: 10.3934/math.20241717

Previous Article Next Article

Research article Special Issues

A hybrid approach to conjugate gradient algorithms for nonlinear systems of equations with applications in signal restoration

Xuejie Ma ¹,
Songhua Wang ^{2
,
,}

1.
School of Artificial Intelligence, Guangzhou Huashang College, Guangzhou 511300, China
2.
School of Mathematics, Physics and Statistics, Baise University, Baise 533099, China

Received: 16 September 2024 Revised: 05 November 2024 Accepted: 11 November 2024 Published: 27 December 2024
MSC : 65K05, 90C56

This paper proposes a novel hybrid PRP-HS-LS-type conjugate gradient algorithm for solving constrained nonlinear systems of equations. The proposed algorithm presents several significant advancements and key features: (i) the conjugate parameter is constructed by utilizing the hybrid technique; (ii) the search direction, designed with the conjugate parameter, possesses sufficient descent and trust region properties without the need for a line search mechanism; (iii) the global convergence is rigorously established under general assumptions, notably without the requirement of the Lipschitz continuity condition; (vi) numerical experiments demonstrate the algorithm's efficiency, particularly in solving large-scale constrained nonlinear systems of equations and addressing the sparse signal restoration problem.
- constrained nonlinear systems of equations,
- conjugate gradient method,
- large-scale,
- global convergence,
- signal reconstruction
Citation: Xuejie Ma, Songhua Wang. A hybrid approach to conjugate gradient algorithms for nonlinear systems of equations with applications in signal restoration[J]. AIMS Mathematics, 2024, 9(12): 36167-36190. doi: 10.3934/math.20241717

Related Papers:

Abstract

This paper proposes a novel hybrid PRP-HS-LS-type conjugate gradient algorithm for solving constrained nonlinear systems of equations. The proposed algorithm presents several significant advancements and key features: (i) the conjugate parameter is constructed by utilizing the hybrid technique; (ii) the search direction, designed with the conjugate parameter, possesses sufficient descent and trust region properties without the need for a line search mechanism; (iii) the global convergence is rigorously established under general assumptions, notably without the requirement of the Lipschitz continuity condition; (vi) numerical experiments demonstrate the algorithm's efficiency, particularly in solving large-scale constrained nonlinear systems of equations and addressing the sparse signal restoration problem.

References

[1]	W. Xue, P. Wan, Q. Li, An online conjugate gradient algorithm for large-scale data analysis in machine learning, AIMS Mathematics, 6 (2021), 1515–1537. https://doi.org/10.3934/math.2021092 doi: 10.3934/math.2021092
[2]	J. Chorowsk, J. M. Zurada, Learning understandable neural networks with nonnegative weight constraints, IEEE Trans. Neural. Netw. Learn. Syst., 26 (2015), 62–69. https://doi.org/10.1109/TNNLS.2014.2310059 doi: 10.1109/TNNLS.2014.2310059
[3]	Y. H. Zheng, B. Jeon, D. H. Xu, Q. M. Wu, H. Zhang, Image segmentation by generalized hierarchical fuzzy C-means algorithm, J. Intell. Fuzzy Syst., 28 (2015), 961–973. https://doi.org/10.3233/IFS-141378 doi: 10.3233/IFS-141378
[4]	Y. Li, C. Li, W. Yang, A new conjugate gradient method with a restart direction and its application in image restoration, AIMS Mathematics, 12 (2023), 28791–28807. https://doi.org/10.3934/math.20231475 doi: 10.3934/math.20231475
[5]	S. Aji, P. Kumam, A. M. Awwal, An efficient DY-type spectral conjugate gradient method for system of nonlinear monotone equations with application in signal recovery, Aims Mathematics, 6 (2021), 8078–8106. https://doi.org/10.3934/math.2021469 doi: 10.3934/math.2021469
[6]	K. Ahmed, M. Y. Waziri, A. S. Halilu, Sparse signal reconstruction via Hager-Zhang-type schemes for constrained system of nonlinear equations, Optimization, 73 (2024), 1949–1980. https://doi.org/10.1080/02331934.2023.2187255 doi: 10.1080/02331934.2023.2187255
[7]	D. D. Li, S. H. Wang, Y. Li, J. Q. Wu, A projection-based hybrid PRP-DY type conjugate gradient algorithm for constrained nonlinear equations with applications, Appl. Numer. Math., 195 (2024), 105–125. https://doi.org/10.1016/j.apnum.2023.09.009 doi: 10.1016/j.apnum.2023.09.009
[8]	M. Dehghan, A. Shirilord, Three-step iterative methods for numerical solution of systems of nonlinear equations, Eng. Comput., 38 (2022), 1015–1028. https://doi.org/10.1007/s00366-020-01072-1 doi: 10.1007/s00366-020-01072-1
[9]	M. Dehghan, G. Karamali, A. Shirilord, An iterative scheme for a class of generalized Sylvester matrix equations, AUT J. Math. Comut., 5 (2024), 195–215.
[10]	K. Meinijes, A. P. Morgan, A methodology for solving chemical equilibrium systems, Appl. Math. Comput., 22 (1987), 333–361.
[11]	X. Tong, S. Zhou, A smoothing projected Newton-type method for semismooth equations with bound constraints, J. Ind. Manag. Optim., 1 (2005), 235–250. https://doi.org/10.3934/jimo.2005.1.235 doi: 10.3934/jimo.2005.1.235
[12]	M. Dehghan, M. Hajarian, Fourth-order variants of Newton's method without second derivatives for solving non-linear equations, Eng. Comput., 29 (2012), 356–365. https://doi.org/10.1108/02644401211227590 doi: 10.1108/02644401211227590
[13]	C. Wang, Y. Wang, C. Xu, A projection method for a system of nonlinear monotone equations with convex constraints, Math. Meth. Oper. Res., 66 (2007), 33–46. https://doi.org/10.1007/s00186-006-0140-y doi: 10.1007/s00186-006-0140-y
[14]	G. Yuan, Z. Wei, M. Zhang, An active-set projected trust region algorithm for box constrained optimization problems, J. Syst. Sci. Complex., 28 (2015), 1128–1147. https://doi.org/10.1007/s11424-014-2199-5 doi: 10.1007/s11424-014-2199-5
[15]	J. Fan, On the levenberg-marquardt methods for convex constrained nonlinear equations, J. Ind. Manag. Optim., 9 (2013), 227–241. https://doi.org/10.3934/jimo.2013.9.227 doi: 10.3934/jimo.2013.9.227
[16]	J. Yin, J. Jian, G. Ma, A modified inexact Levenberg-Marquardt method with the descent property for solving nonlinear equations, Comput. Optim. Appl., 87 (2024), 289–322. https://doi.org/10.1007/s10589-023-00513-z doi: 10.1007/s10589-023-00513-z
[17]	Y. Ding, Y. Xiao, J. Li, A class of conjugate gradient methods for convex constrained monotone equations, Optimization, 66 (2017), 2309–2328. https://doi.org/10.1080/02331934.2017.1372438 doi: 10.1080/02331934.2017.1372438
[18]	M. Sun, J. Liu, New hybrid conjugate gradient projection method for the convex constrained equations, Calcolo, 53 (2016), 399–411. https://doi.org/10.1007/s10092-015-0154-z doi: 10.1007/s10092-015-0154-z
[19]	H. Zheng, J. Li, P. Liu, An inertial Fletcher-Reeves-type conjugate gradient projection-based method and its spectral extension for constrained nonlinear equations, J. Appl. Math. Comput., 70 (2024), 2427–2452. https://doi.org/10.1007/s12190-024-02062-y doi: 10.1007/s12190-024-02062-y
[20]	Z. Yu, J. Lin, J. Sun, Y. Xiao, L. Liu, Z. Li, Spectral gradient projection method for monotone nonlinear equations with convex constraints, Appl. Numer. Math., 59 (2009), 2416–2423. https://doi.org/10.1016/j.apnum.2009.04.004 doi: 10.1016/j.apnum.2009.04.004
[21]	M. Dehghan, R. Mohammadi-Arani, Generalized product-type methods based on bi-conjugate gradient (GPBiCG) for solving shifted linear systems, Comput. Appl. Math., 36 (2017), 1591–1606. https://doi.org/10.1007/s40314-016-0315-y doi: 10.1007/s40314-016-0315-y
[22]	J. K. Liu, Y. M. Feng, A derivative-free iterative method for nonlinear monotone equations with convex constraints, Numer. Algor., 82 (2019), 245–262. https://doi.org/10.1007/s11075-018-0603-2 doi: 10.1007/s11075-018-0603-2
[23]	W. J. Hu, J. Z. Wu, G. L. Yuan, Some modified Hestenes-Stiefel conjugate gradient algorithms with application in image restoration, Appl. Numer. Math., 158 (2020), 360–376. https://doi.org/10.1016/j.apnum.2020.08.009 doi: 10.1016/j.apnum.2020.08.009
[24]	G. Ma, J. Jin, J. Jian, D. Han, A modified inertial three-term conjugate gradient projection method for constrained nonlinear equations with applications in compressed sensing, Numer. Algor., 92 (2023), 1621–1653. https://doi.org/10.1007/s11075-022-01356-1 doi: 10.1007/s11075-022-01356-1
[25]	J. H. Yin, J. B. Jian, X. Z. Jiang, M. X. Liu, L. Wang, A hybrid three-term conjugate gradient projection method for constrained nonlinear monotone equations with applications Numer. Algor., 88 (2021), 389–418.
[26]	P. T. Gao, C. J. He, Y. Liu, An adaptive family of projection methods for constrained monotone nonlinear equations with applications, Appl. Math. Comput., 359 (2019), 1–16. https://doi.org/10.1016/j.amc.2019.03.064 doi: 10.1016/j.amc.2019.03.064
[27]	M. Li, A modified Hestense-Stiefel conjugate gradient method close to the memoryless BFGS quasi-Newton method, Optim. Methods Softw., 33 (2018), 336–353. https://doi.org/10.1080/10556788.2017.1325885 doi: 10.1080/10556788.2017.1325885
[28]	M. Li, A three term Polak-Ribière-Polyak conjugate gradient method close to the memoryless BFGS quasi-Newton method, J. Ind. Manag. Optim., 16 (2018), 245–260.
[29]	A. B. Abubakar, P. Kumam, H. Mohammad, A note on the spectral gradient projection method for nonlinear monotone equations with applications, Comput. Appl. Math., 39 (2020), 129. https://doi.org/10.1007/s40314-020-01151-5 doi: 10.1007/s40314-020-01151-5
[30]	J. K. Liu, S. J. Li, A projection method for convex constrained monotone nonlinear equations with applications, Comput. Math. Appl., 70 (2015), 2442–2453. https://doi.org/10.1016/j.camwa.2015.09.014 doi: 10.1016/j.camwa.2015.09.014
[31]	E. D. Dolan, J. Jorge, Benchmarking optimization software with performance profiles, Math. Program., 91 (2001), 201–213. https://doi.org/10.1007/s101070100263 doi: 10.1007/s101070100263
[32]	D. D. Li, J. Q. Wu, Y. Li, S. H. Wang, A modified spectral gradient projection-based algorithm for large-scale constrained nonlinear equations with applications in compressive sensing, J. Comput. Appl. Math., 424 (2023), 115006. https://doi.org/10.1016/j.cam.2022.115006 doi: 10.1016/j.cam.2022.115006
[33]	M. A. T. Figueiredo, R. D. Nowak, S. J. Wright, Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems, IEEE J. Sel. Top. Signal Process., 1 (2007), 586–597. https://doi.org/10.1109/JSTSP.2007.910281 doi: 10.1109/JSTSP.2007.910281
[34]	Y. Xiao, Q. Wang, Q. Hu, Non-smooth equations based method for $\ell1$-norm problems with applications to compressed sensing, Nonlinear Anal. Theory Methods Appl., 74 (2011), 3570–3577.

Reader Comments

Your name:*

Email:*
© 2024 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)