An efficient augmented memoryless quasi-Newton method for solving large-scale unconstrained optimization problems

Yulin Cheng; Jing Gao; Yulin Cheng; Jing Gao

doi:10.3934/math.20241231

AIMS Mathematics

2024, Volume 9, Issue 9: 25232-25252. doi: 10.3934/math.20241231

Previous Article Next Article

Research article

An efficient augmented memoryless quasi-Newton method for solving large-scale unconstrained optimization problems

Yulin Cheng ,
Jing Gao ^,

School of Mathematics and Statistics, Beihua University, Jilin 132013, China

Received: 15 May 2024 Revised: 22 July 2024 Accepted: 21 August 2024 Published: 29 August 2024
MSC : 65K05, 90C31, 90C53

In this paper, an augmented memoryless BFGS quasi-Newton method was proposed for solving unconstrained optimization problems. Based on a new modified secant equation, an augmented memoryless BFGS update formula and an efficient optimization algorithm were established. To improve the stability of the numerical experiment, we obtained the scaling parameter by minimizing the upper bound of the condition number. The global convergence of the algorithm was proved, and numerical experiments showed that the algorithm was efficient.
- unconstrained optimization,
- quasi-Newton method,
- BFGS update,
- secant equation,
- global convergence
Citation: Yulin Cheng, Jing Gao. An efficient augmented memoryless quasi-Newton method for solving large-scale unconstrained optimization problems[J]. AIMS Mathematics, 2024, 9(9): 25232-25252. doi: 10.3934/math.20241231

Related Papers:

Abstract

In this paper, an augmented memoryless BFGS quasi-Newton method was proposed for solving unconstrained optimization problems. Based on a new modified secant equation, an augmented memoryless BFGS update formula and an efficient optimization algorithm were established. To improve the stability of the numerical experiment, we obtained the scaling parameter by minimizing the upper bound of the condition number. The global convergence of the algorithm was proved, and numerical experiments showed that the algorithm was efficient.

References

[1]	W. Y. Sun, Y. X. Yuan, Optimization theory and methods: nonlinear programming, New York: Springer, 2006. https://doi.org/10.1007/b106451
[2]	J. Z. Zhang, N. Y. Deng, L. H. Chen, New quasi-Newton equation and related methods for unconstrained optimization, J. Optim. Theory Appl., 102 (1999), 147–167. https://doi.org/10.1023/A:1021898630001 doi: 10.1023/A:1021898630001
[3]	J. Z. Zhang, C. X. Xu, Properties and numerical performance of quasi-Newton methods with modified quasi-Newton equations, J. Comput. Appl. Math., 137 (2001), 269–278. https://doi.org/10.1016/S0377-0427(00)00713-5 doi: 10.1016/S0377-0427(00)00713-5
[4]	S. Babaie-Kafaki, On optimality of the parameters of self-scaling memoryless quasi-Newton updating formulae, J. Optim. Theory Appl., 167 (2015), 91–101. https://doi.org/10.1007/s10957-015-0724-x doi: 10.1007/s10957-015-0724-x
[5]	S. Babaie-Kafaki, Z. Aminifard, Two-parameter scaled memoryless BFGS methods with a nonmonotone choice for the initial step length, Numer. Algorithms, 82 (2019), 1345–1357. https://doi.org/10.1007/s11075-019-00658-1 doi: 10.1007/s11075-019-00658-1
[6]	Z. Aminifard, S. Babaie-Kafaki, S. Ghafoori, An augmented memoryless BFGS method based on a modified secant equation with application to compressed sensing, Appl. Numer. Math., 167 (2021), 187–201. https://doi.org/10.1016/j.apnum.2021.05.002 doi: 10.1016/j.apnum.2021.05.002
[7]	S. Babaie-Kafaki, Z. Aminifard, S. Ghafoori, A hybrid quasi-Newton method with application in sparse recovery, Comput. Appl. Math., 41 (2022), 249. https://doi.org/10.1007/s40314-022-01962-8 doi: 10.1007/s40314-022-01962-8
[8]	M. Jourak, S. Nezhadhosein, F. Rahpeymaii, A new self-scaling memoryless quasi-Newton update for unconstrained optimization, 4OR, 22 (2024), 235–252. https://doi.org/10.1007/s10288-023-00544-6 doi: 10.1007/s10288-023-00544-6
[9]	C. G. Broyden, The convergence of a class of double-rank minimization algorithms 1. General considerations, IMA J. Appl. Math., 6 (1970), 76–90. https://doi.org/10.1093/imamat/6.1.76 doi: 10.1093/imamat/6.1.76
[10]	C. G. Broyden, The convergence of a class of double-rank minimization algorithms 2. The new algorithm, IMA J. Appl. Math., 6 (1970), 222–231. https://doi.org/10.1093/imamat/6.3.222 doi: 10.1093/imamat/6.3.222
[11]	R. Fletcher, A new approach to variable metric algorithms, Comput. J., 13 (1970), 317–322. https://doi.org/10.1093/comjnl/13.3.317 doi: 10.1093/comjnl/13.3.317
[12]	D. Goldfarb, A family of variable-metric methods derived by variational means, Math. Comput., 24 (1970), 23–26.
[13]	D. F. Shanno, Conditioning of quasi-Newton methods for function minimization, Math. Comput., 24 (1970), 647–656.
[14]	S. S. Oren, E. Spedicato, Optimal conditioning of self-scaling variable metric algorithms, Math. Program., 10 (1976), 70–90. https://doi.org/10.1007/BF01580654 doi: 10.1007/BF01580654
[15]	S. S. Oren, D. G. Luenberger, Self-scaling variable metric (SSVM) algorithms: Part Ⅰ: Criteria and sufficient conditions for scaling a class of algorithms, Manag. Sci., 20 (1974), 845–862. https://doi.org/10.1287/mnsc.20.5.845 doi: 10.1287/mnsc.20.5.845
[16]	D. S. Watkins, Fundamentals of matrix computations, John Wiley & Sons, 2004.
[17]	S. Babaie-Kafaki, A modified scaled memoryless BFGS preconditioned conjugate gradient method for unconstrained optimization, 4OR, 11 (2013), 361–374. https://doi.org/10.1007/s10288-013-0233-4 doi: 10.1007/s10288-013-0233-4
[18]	J. Nocedal, S. J. Wright, Numerical optimization, 2 Eds., New York: Springer, 2006. https://doi.org/10.1007/978-0-387-40065-5
[19]	E. D. Dolan, J. J. More, Benchmarking optimization software with performance profiles, Math. Program., 91 (2002), 201–213. https://doi.org/10.1007/s101070100263 doi: 10.1007/s101070100263
[20]	G. Zoutendijk, Nonlinear programming, computational methods, In: Integer and nonlinear programming, Amsterdam: North-Holland, 1970, 37–86.
[21]	N. I. M. Gould, D. Orban, P. L. Toint, CUTEr and SifDec: A constrained and unconstrained testing environment, revisited, ACM Trans. Math. Software, 29 (2003), 373–394. https://doi.org/10.1145/962437.962439 doi: 10.1145/962437.962439
[22]	Y. H. Dai, A perfect example for the BFGS method, Math. Program., 138 (2013), 501–530. https://doi.org/10.1007/s10107-012-0522-2 doi: 10.1007/s10107-012-0522-2
[23]	N. Andrei, An adaptive scaled BFGS method for unconstrained optimization, Numer. Algorithms, 77 (2018), 413–432. https://doi.org/10.1007/s11075-017-0321-1 doi: 10.1007/s11075-017-0321-1
[24]	B. A. Hassan, I. A. R. Moghrabi, A modified secant equation quasi-Newton method for unconstrained optimization, J. Appl. Math. Comput., 69 (2023), 451–464. https://doi.org/10.1007/s12190-022-01750-x doi: 10.1007/s12190-022-01750-x
[25]	G. L. Yuan, X. Zhao, K. J. Liu, X. X. Chen, An adaptive projection BFGS method for nonconvex unconstrained optimization problems, Numer. Algorithms, 95 (2024), 1747–1767. https://doi.org/10.1007/s11075-023-01626-6 doi: 10.1007/s11075-023-01626-6
[26]	X. M. Lu, C. F. Yang, Q. Wu, J. X. Wang, Y. H. Wei, L. Y. Zhang, et al., Improved reconstruction algorithm of wireless sensor network based on BFGS quasi-Newton method, Electronics, 12 (2023), 1–15. https://doi.org/10.3390/electronics12061267 doi: 10.3390/electronics12061267
[27]	V. Krutikov, E. Tovbis, P. Stanimirović, L. Kazakovtsev, D. Karabašević, Machine learning in quasi-Newton methods, Axioms, 13 (2024), 1–29. https://doi.org/10.3390/axioms13040240 doi: 10.3390/axioms13040240
[28]	A. B. Abubakar, P. Kumam, H. Mohammad, A. H. Ibrahim, T. Seangwattana, B. A. Hassan, A hybrid BFGS-Like method for monotone operator equations with applications, J. Comput. Appl. Math., 446 (2024), 115857. https://doi.org/10.1016/j.cam.2024.115857 doi: 10.1016/j.cam.2024.115857
[29]	Y. Narushima, S. Nakayama, M. Takemura, H. Yabe, Memoryless quasi-Newton methods based on the spectral-scaling Broyden family for Riemannian optimization, J. Optim. Theory Appl., 197 (2023), 639–664. https://doi.org/10.1007/s10957-023-02183-7 doi: 10.1007/s10957-023-02183-7
[30]	J. R. Rice, J. J. Moré, B. S. Garbow, K. E. Hillstrom, Testing unconstrained optimization software, ACM Trans. Math. Software, 7 (1981), 17–41. https://doi.org/10.1145/355934.355936 doi: 10.1145/355934.355936
[31]	N. Andrei, An unconstrained optimization test functions collection, Adv. Model. Optim., 10 (2008), 147–161.
[32]	P. J. Liu, X. Y. Wu, H. Shao, Y. Zhang, S. H. Cao, Three adaptive hybrid derivative-free projection methods for constrained monotone nonlinear equations and their applications, Numer. Linear Algebra Appl., 30 (2023), e2471. https://doi.org/10.1002/nla.2471 doi: 10.1002/nla.2471
[33]	W. J. Zhou, D. M. Shen, Convergence properties of an iterative method for solving symmetric non-linear equations, J. Optim. Theory Appl., 164 (2015), 277–289. https://doi.org/10.1007/s10957-014-0547-1 doi: 10.1007/s10957-014-0547-1
[34]	X. W. Fang, Q. Ni, M. L. Zeng, A modified quasi-Newton method for nonlinear equations, J. Comput. Appl. Math., 328 (2018), 44–58. https://doi.org/10.1016/j.cam.2017.06.024 doi: 10.1016/j.cam.2017.06.024

Reader Comments

Your name:*

Email:*
© 2024 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)