An efficient DY-type spectral conjugate gradient method for system of nonlinear monotone equations with application in signal recovery

Sani Aji; Poom Kumam; Aliyu Muhammed Awwal; Mahmoud Muhammad Yahaya; Kanokwan Sitthithakerngkiet; Sani Aji; Poom Kumam; Aliyu Muhammed Awwal; Mahmoud Muhammad Yahaya; Kanokwan Sitthithakerngkiet

doi:10.3934/math.2021469

AIMS Mathematics

2021, Volume 6, Issue 8: 8078-8106. doi: 10.3934/math.2021469

Previous Article Next Article

Research article

An efficient DY-type spectral conjugate gradient method for system of nonlinear monotone equations with application in signal recovery

1.
KMUTT Fixed Point Research Laboratory, Room SCL 802 Fixed Point Laboratory, Science Laboratory Building, Department of Mathematics, Faculty of Science, King Mongkut's University of Technology Thonburi (KMUTT), 126 Pracha-Uthit Road, Bang Mod, Thung Khru, Bangkok 10140, Thailand
2.
Center of Excellence in Theoretical and Computational Science (TaCS-CoE), Faculty of Science, King Mongkut's University of Technology Thonburi (KMUTT), 126 Pracha Uthit Rd., Bang Mod, Thung Khru, Bangkok 10140, Thailand
3.
Department of Medical Research, China Medical University Hospital, China Medical University, Taichung 40402, Taiwan
4.
Department of Mathematics, Faculty of Science, Gombe State University, Gombe 760214, Nigeria
5.
Intelligent and Nonlinear Dynamic Innovations Research Center, Department of Mathematics, Faculty of Applied Science, King Mongkut's University of Technology North Bangkok (KMUTNB), 1518, Wongsawang, Bangsue, Bangkok 10800, Thailand

Received: 21 March 2021 Accepted: 08 May 2021 Published: 24 May 2021
MSC : 65K05, 65L09, 90C30

Many problems in engineering and social sciences can be transformed into system of nonlinear equations. As a result, a lot of methods have been proposed for solving the system. Some of the classical methods include Newton and Quasi Newton methods which have rapid convergence from good initial points but unable to deal with large scale problems due to the computation of Jacobian matrix or its approximation. Spectral and conjugate gradient methods proposed for unconstrained optimization, and later on extended to solve nonlinear equations do not require any computation of Jacobian matrix or its approximation, thus, are suitable to handle large scale problems. In this paper, we proposed a spectral conjugate gradient algorithm for solving system of nonlinear equations where the operator under consideration is monotone. The search direction of the proposed algorithm is constructed by taking the convex combination of the Dai-Yuan (DY) parameter and a modified conjugate descent (CD) parameter. The proposed search direction is sufficiently descent and under some suitable assumptions, the global convergence of the proposed algorithm is proved. Numerical experiments on some test problems are presented to show the efficiency of the proposed algorithm in comparison with an existing one. Finally, the algorithm is successfully applied in signal recovery problem arising from compressive sensing.
- nonlinear monotone equations,
- conjugate gradient method,
- spectral conjugate gradient method and large-scale problems
Citation: Sani Aji, Poom Kumam, Aliyu Muhammed Awwal, Mahmoud Muhammad Yahaya, Kanokwan Sitthithakerngkiet. An efficient DY-type spectral conjugate gradient method for system of nonlinear monotone equations with application in signal recovery[J]. AIMS Mathematics, 2021, 6(8): 8078-8106. doi: 10.3934/math.2021469

Related Papers:

Abstract

Many problems in engineering and social sciences can be transformed into system of nonlinear equations. As a result, a lot of methods have been proposed for solving the system. Some of the classical methods include Newton and Quasi Newton methods which have rapid convergence from good initial points but unable to deal with large scale problems due to the computation of Jacobian matrix or its approximation. Spectral and conjugate gradient methods proposed for unconstrained optimization, and later on extended to solve nonlinear equations do not require any computation of Jacobian matrix or its approximation, thus, are suitable to handle large scale problems. In this paper, we proposed a spectral conjugate gradient algorithm for solving system of nonlinear equations where the operator under consideration is monotone. The search direction of the proposed algorithm is constructed by taking the convex combination of the Dai-Yuan (DY) parameter and a modified conjugate descent (CD) parameter. The proposed search direction is sufficiently descent and under some suitable assumptions, the global convergence of the proposed algorithm is proved. Numerical experiments on some test problems are presented to show the efficiency of the proposed algorithm in comparison with an existing one. Finally, the algorithm is successfully applied in signal recovery problem arising from compressive sensing.

References

[1]	M. Sun, J. Liu, Y. Wang, Two improved conjugate gradient methods with application in compressive sensing and motion control, Math. Probl. Eng., 2020, (2020), 9175496.
[2]	S. P. Dirkse, M. J. Ferris, A collection of nonlinear mixed complementarity problems, Optim. Method. Softw., 5 (1995), 319–345. doi: 10.1080/10556789508805619
[3]	A. J. Wood, B. F. Wollenberg, G. B. Sheblé, Power generation, operation, and control, John Wiley & Sons, 2013.
[4]	K. Meintjes, A. P. Morgan, A methodology for solving chemical equilibrium systems, Appl. Math. Comput., 22 (1987), 333–361. doi: 10.1016/0096-3003(87)90076-2
[5]	N. A. Iusem, V. M. Solodov, Newton-type methods with generalized distances for constrained optimization, Optimization, 41 (1997), 257–278. doi: 10.1080/02331939708844339
[6]	M. Fukushima, Equivalent differentiable optimization problems and descent methods for asymmetric variational inequality problems, Math. program., 53 (1992), 99–110. doi: 10.1007/BF01585696
[7]	Y. B. Zhao, D, Li, Monotonicity of fixed point and normal mappings associated with variational inequality and its application, SIAM J Optimization, 11 (2001), 962–973. doi: 10.1137/S1052623499357957
[8]	Y. H. Xiao, Q. Y. Wang, Q. J. Hu, Non-smooth equations based method for $\ell_1$-norm problems with applications to compressed sensing, Nonlinear Anal.: Theory, 74 (2011), 3570–3577. doi: 10.1016/j.na.2011.02.040
[9]	J. E. Dennis, J. J. Moré, A characterization of superlinear convergence and its application to quasi-newton methods, Math. Comput., 28 (1974), 549–560. doi: 10.1090/S0025-5718-1974-0343581-1
[10]	D. H. Li, M. Fukushima, A globally and superlinearly convergent gauss–newton-based bfgs method for symmetric nonlinear equations, SIAM J. Numer, Anal., 37 (1998), 152–172.
[11]	A. M. Awwal, P. Kumam, A. B. Abubakar, Spectral modified Polak–Ribiére–Polyak projection conjugate gradient method for solving monotone systems of nonlinear equations, Appl. Math. Comput., 362 (2019), 124514. doi: 10.1016/j.amc.2019.06.028
[12]	G. L. Zhou, K. C. Toh, Superlinear convergence of a newton-type algorithm for monotone equations, J. Optimiz. Theory App., 125 (2005), 205–221. doi: 10.1007/s10957-004-1721-7
[13]	W. J. Zhou, D. H. Li, A globally convergent bfgs method for nonlinear monotone equations without any merit functions, Math. Comput., 77 (2008), 2231–2240. doi: 10.1090/S0025-5718-08-02121-2
[14]	M. V. Solodov, B. F. Svaiter, A globally convergent inexact newton method for systems of monotone equations, In: Reformulation: Nonsmooth, piecewise smooth, semismooth and smoothing methods, 1998,355–369.
[15]	C. W. Wang, Y. J. Wang, C. L. Xu, A projection method for a system of nonlinear monotone equations with convex constraints, Math. Meth. Oper. Res., 66 (2007), 33–46. doi: 10.1007/s00186-006-0140-y
[16]	W. Y. Cheng, A PRP type method for systems of monotone equations, Math. Comput. Model., 50 (2009), 15–20. doi: 10.1016/j.mcm.2009.04.007
[17]	Y. H. Xiao, H. Zhu, A conjugate gradient method to solve convex constrained monotone equations with applications in compressive sensing, J. Math. Anal. Appl., 405 (2013), 310–319. doi: 10.1016/j.jmaa.2013.04.017
[18]	J. K. Liu, S. J. Li, A projection method for convex constrained monotone nonlinear equations with applications, Comput. Math. Appl., 70 (2015), 2442–2453. doi: 10.1016/j.camwa.2015.09.014
[19]	Y. H. Dai, Y. X. Yuan. A nonlinear conjugate gradient method with a strong global convergence property, SIAM J. Optimiz., 10 (1999), 177–182. doi: 10.1137/S1052623497318992
[20]	J. K. Liu, S. J. Li, Spectral DY-type projection method for nonlinear monotone systems of equations, J. Comput. Math, 33 (2015), 341–355. doi: 10.4208/jcm.1412-m4494
[21]	J. K. Liu, S. J. Li, Multivariate spectral DY-type projection method for convex constrained nonlinear monotone equations, J. Ind. Manag. Optim., 13 (2017), 283–295.
[22]	J. K. Liu, Y. M. Feng. A derivative-free iterative method for nonlinear monotone equations with convex constraints, Numer Algorithms, 82 (2019), 245–262.
[23]	A. M. Awwal, P. Kumam, L. Wang, S. Huang, W. Kumam, Inertial-based derivative-free method for system of monotone nonlinear equations and application, IEEE Access, 8 (2020), 226921–226930. doi: 10.1109/ACCESS.2020.3045493
[24]	J. Guo, Z. Wan, A modified spectral PRP conjugate gradient projection method for solving large-scale monotone equations and its application in compressed sensing, Math. Probl. Eng., 2019 (2019), 5261830.
[25]	A. M. Awwal, P. Kumam, H. Mohammad, W. Watthayu, A. B. Abubakar, A Perry-type derivative-free algorithm for solving nonlinear system of equations and minimizing $\ell_1$ regularized problem, Optimization, 70 (2021), 1231–1259. doi: 10.1080/02331934.2020.1808647
[26]	A. M. Awwal, L. Wang, P. Kumam, H. Mohammad, W. Watthayu, A projection Hestenes–Stiefel method with spectral parameter for nonlinear monotone equations and signal processing, Math. Comput. Appl., 25 (2020), 27.
[27]	A. B. Abubakar, P. Kumam, H. Mohammad, A. M. Awwal, A Barzilai-Borwein gradient projection method for sparse signal and blurred image restoration, J. Franklin I., 357 (2020), 7266–7285. doi: 10.1016/j.jfranklin.2020.04.022
[28]	A. B. Abubakar, P. Kumam, H. Mohammad, A note on the spectral gradient projection method for nonlinear monotone equations with applications, Comp. Appl. Math., 39 (2020), 129. doi: 10.1007/s40314-020-01151-5
[29]	S. Aji, P. Kumam, P. Siricharoen, A. B. Abubakar, M. M. Yahaya, A modified conjugate descent projection method for monotone nonlinear equations and image restoration, IEEE Access, 8 (2020), 158656–158665. doi: 10.1109/ACCESS.2020.3020334
[30]	S. Aji, P. Kumam, A. M. Awwal, M. M. Yahaya, W. Kumam. Two hybrid spectral methods with inertial effect for solving system of nonlinear monotone equations with application in robotics. IEEE Access, 9 (2021), 30918–30928.
[31]	A. H. Ibrahim, P. Kumam, A. B. Abubakar, U. B. Yusuf, S. E. Yimer, K. O. Aremu, An efficient gradient-free projection algorithm for constrained nonlinear equations and image restoration, AIMS Mathematics, 6 (2020), 235–260.
[32]	A. M. Awwal, L. Wang, P. Kumam, H. Mohammad, A two-step spectral gradient projection method for system of nonlinear monotone equations and image deblurring problems, Symmetry, 12 (2020), 874. doi: 10.3390/sym12060874
[33]	Z. S. Yu, J. Lin, J. Sun, Y. H. Xiao, L. Y. Liu, Z. H. Li, Spectral gradient projection method for monotone nonlinear equations with convex constraints, Appl. Numer. Math., 59 (2009), 2416–2423. doi: 10.1016/j.apnum.2009.04.004
[34]	A. M. Awwal, P. Kumam, A. B. Abubakar, A modified conjugate gradient method for monotone nonlinear equations with convex constraints, Appl. Numer. Math., 145 (2019), 507–520. doi: 10.1016/j.apnum.2019.05.012
[35]	W. J. Zhou, D. H. Li. A globally convergent BFGS method for nonlinear monotone equations without any merit functions, Math. Comput., 77 (2008), 2231–2240. doi: 10.1090/S0025-5718-08-02121-2
[36]	W. La Cruz, A spectral algorithm for large-scale systems of nonlinear monotone equations, Numer. Algor., 76 (2017), 1109–1130. doi: 10.1007/s11075-017-0299-8
[37]	W. La Cruz, J. M. Martínez, M. Raydan, Spectral residual method without gradient information for solving large-scale nonlinear systems of equations, Math. Comput., 75 (2006), 1429–1448. doi: 10.1090/S0025-5718-06-01840-0
[38]	Y. Bing, G. Lin, An efficient implementation of merrills method for sparse or partially separable systems of nonlinear equations, SIAM J. Optim., 1 (1991), 206–221. doi: 10.1137/0801015
[39]	E. D. Dolan, J. J. Moré, Benchmarking optimization software with performance profiles. Math. Program., 91 (2002), 201–213.
[40]	M. A. T. Figueiredo, R. D. Nowak, An EM algorithm for wavelet-based image restoration, IEEE T. Image Processing, 12 (2003), 906–916. doi: 10.1109/TIP.2003.814255
[41]	E. T. Hale, W. T. Yin, Y. Zhang, A fixed-point continuation method for $\ell_1$-regularized minimization with applications to compressed sensing, CAAM Technical Report TR07-07, Rice University, 43 (2007), 44.
[42]	A. Beck, M. Teboulle, A fast iterative shrinkage-thresholding algorithm for linear inverse problems, SIAM J. Imaging Sci., 2 (2009), 183–202. doi: 10.1137/080716542
[43]	M. A. T. Figueiredo, R. D. Nowak, S. J. Wright, Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems, IEEE J.-STSP, 1 (2007), 586–597.
[44]	E. Van Den Berg, M. P. Friedlander, Probing the pareto frontier for basis pursuit solutions, SIAM J. Sci. Comput., 31 (2009), 890–912. doi: 10.1137/080714488
[45]	E. G. Birgin, J. M. Martínez, M. Raydan, Nonmonotone spectral projected gradient methods on convex sets, SIAM J. Optimiz., 10 (2000), 1196–1211. doi: 10.1137/S1052623497330963
[46]	J. S. Pang, Inexact Newton methods for the nonlinear complementarity problem, Math, Program, 36 (1986), 54–71. doi: 10.1007/BF02591989
[47]	M. M. Yahaya, P. Kumam, A. M. Awwal, S. Aji, A structured quasi–Newton algorithm with nonmonotone search strategy for structured NLS problems and its application in robotic motion control, J. Comput. Appl. Math., 395 (2021), 113582. doi: 10.1016/j.cam.2021.113582

Reader Comments

Your name:*

Email:*
© 2021 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)