An accelerated adaptive two-step Levenberg–Marquardt method with the modified Metropolis criterion

Dingyu Zhu; Yueting Yang; Mingyuan Cao; Dingyu Zhu; Yueting Yang; Mingyuan Cao

doi:10.3934/math.20241199

AIMS Mathematics

2024, Volume 9, Issue 9: 24610-24635. doi: 10.3934/math.20241199

Previous Article Next Article

Research article

An accelerated adaptive two-step Levenberg–Marquardt method with the modified Metropolis criterion

School of Mathematics and Statistics, Beihua University, Jilin 132013, China

Received: 11 June 2024 Revised: 11 August 2024 Accepted: 15 August 2024 Published: 22 August 2024
MSC : 65K05, 90C30

In this paper, aiming at the nonlinear equations, a new two-step Levenberg–Marquardt method was proposed. We presented a new Levenberg–Marquardt parameter to obtain the trial step. A new modified Metropolis criterion was used to adjust the upper bound of the approximate step. The convergence of the method was analyzed under the H$ \ddot{\rm o} $lderian local error bound condition and the H$ \ddot\rm o $lderian continuity of the Jacobian. Numerical experiments showed that the new algorithm is effective and competitive in the numbers of functions, Jacobian evaluations and iterations.
- nonlinear equations,
- Levenberg–Marquardt method,
- Metropolis criterion,
- H$ \ddot{\rm o} $lderian local error bound,
- H$ \ddot{\rm o} $lderian continuity
Citation: Dingyu Zhu, Yueting Yang, Mingyuan Cao. An accelerated adaptive two-step Levenberg–Marquardt method with the modified Metropolis criterion[J]. AIMS Mathematics, 2024, 9(9): 24610-24635. doi: 10.3934/math.20241199

Related Papers:

Abstract

In this paper, aiming at the nonlinear equations, a new two-step Levenberg–Marquardt method was proposed. We presented a new Levenberg–Marquardt parameter to obtain the trial step. A new modified Metropolis criterion was used to adjust the upper bound of the approximate step. The convergence of the method was analyzed under the H$ \ddot{\rm o} $lderian local error bound condition and the H$ \ddot\rm o $lderian continuity of the Jacobian. Numerical experiments showed that the new algorithm is effective and competitive in the numbers of functions, Jacobian evaluations and iterations.

References

[1]	G. D. A. Moura, S. D. T. M. Bezerra, H. P. Gomes, S. A. D. Silva, Neural network using the Levenberg–Marquardt algorithm for optimal real-time operation of water distribution systems, Urban Water J., 15 (2018), 692–699. https://doi.org/10.1080/1573062X.2018.1539503 doi: 10.1080/1573062X.2018.1539503
[2]	Y. J. Sun, P. P. Wang, T. T. Zhang, K. Li, F. Peng, C. G. Zhu, Principle and performance analysis of the Levenberg–Marquardt algorithm in WMS spectral line fitting, Photonics, 9 (2022), 999. https://doi.org/10.3390/photonics9120999 doi: 10.3390/photonics9120999
[3]	A. Alloqmani, O. Alsaedi, N. Bahatheg, R. Alnanih, L. Elrefaei, Design principles-based interactive learning tool for solving nonlinear equations, Comput. Syst. Sci. Eng., 40 (2022), 1023–1042. https://doi.org/10.32604/csse.2022.019704 doi: 10.32604/csse.2022.019704
[4]	Z. W. Liao, F. Y. Zhu, W. Y. Gong, S. J. Li, X. Y. Mi, AGSDE: Archive guided speciation-based differential evolution for nonlinear equations, Appl. Soft Comput., 122 (2022), 108818. https://doi.org/10.1016/j.asoc.2022.108818 doi: 10.1016/j.asoc.2022.108818
[5]	Z. Seifi, A. Ghorbani, A. Abdipour, Time-domain analysis and experimental investigation of electromagnetic wave coupling to RF/microwave nonlinear circuits, J. Electromagnet Wave., 35 (2021), 51–70. https://doi.org/10.1080/09205071.2020.1825994 doi: 10.1080/09205071.2020.1825994
[6]	A. Rothwell, Numerical methods for unconstrained optimization, In: Optimization methods in structural design, Cham: Springer, 2017, 83–106. https://doi.org/10.1007/978-3-319-55197-5
[7]	G. L. Yuan, M. J. Zhang, A three-terms Polak-Ribière-Polyak conjugate gradient algorithm for large-scale nonlinear equations, J. Comput. Appl. Math., 286 (2015), 186–195. https://doi.org/10.1016/j.cam.2015.03.014 doi: 10.1016/j.cam.2015.03.014
[8]	G. L. Yuan, Z. X. Wei, X. W. Lu, A BFGS trust-region method for nonlinear equations, Computing, 92 (2011), 317–333. https://doi.org/10.1007/s00607-011-0146-z doi: 10.1007/s00607-011-0146-z
[9]	J. H. Zhang, Y. Q. Wang, J. Zhao, On maximum residual nonlinear Kaczmarz-type algorithms for large nonlinear systems of equations, J. Comput. Appl. Math., 425 (2023), 115065. https://doi.org/10.1016/j.cam.2023.115065 doi: 10.1016/j.cam.2023.115065
[10]	J. N. Wang, X. Wang, L. W. Zhang, Stochastic regularized Newton methods for nonlinear equations, J. Sci. Comput., 94 (2023), 51. https://doi.org/10.1007/s10915-023-02099-4 doi: 10.1007/s10915-023-02099-4
[11]	R. Behling, D. S. Gonçalves, S. A. Santos, Local convergence analysis of the Levenberg–Marquardt framework for Nonzero–Residue nonlinear least-squares problems under an error bound condition, J. Optim. Theory Appl., 183 (2019), 1099–1122. https://doi.org/10.1007/s10957-019-01586-9 doi: 10.1007/s10957-019-01586-9
[12]	E. H. Bergou, Y. Diouane, V. Kungurtsev, Convergence and complexity analysis of a Levenberg–Marquardt algorithm for inverse problems, J. Optim. Theory Appl., 185 (2020), 927–944. https://doi.org/10.1007/s10957-020-01666-1 doi: 10.1007/s10957-020-01666-1
[13]	K. Levenberg, A method for the solution of certain non-linear problems in least squares, Quart. Appl. Math., 2 (1944), 164–168. https://doi.org/10.1090/qam/10666 doi: 10.1090/qam/10666
[14]	D. W. Marquardt, An algorithm for least-squares estimation of nonlinear parameters, J. Soc. Ind. Appl. Math., 11 (1963), 431–441. https://doi.org/10.1137/0111030 doi: 10.1137/0111030
[15]	N. Yamashita, M. Fukushima, On the rate of convergence of the Levenberg–Marquardt method, In: Topics in numerical analysis, computing supplementa, Vienna: Springer, 2001,239–249. https://doi.org/10.1007/978-3-7091-6217-0_18
[16]	J. Y. Fan, Y. X. Yuan, On the convergence of a new Levenberg–Marquardt method, Report, Institute of Computational Mathematics and Scientific/Engineering Computing, Beijing: Chinese Academy of Science, 2001.
[17]	J. Y. Fan, A Modified Levenberg–Marquardt algorithm for singular system of nonlinear equations, J. Comput. Math., 21 (2003), 625–636.
[18]	K. Amini, F. Rostami, G. Caristi, An efficient Levenberg–Marquardt method with a new LM parameter for systems of nonlinear equations, Optimization, 67 (2018), 637–650. https://doi.org/10.1080/02331934.2018.1435655 doi: 10.1080/02331934.2018.1435655
[19]	C. F. Ma, L. H. Jiang, Some research on Levenberg–Marquardt method for the nonlinear equations, Appl. Math. Comput., 184 (2007), 1032–1040. https://doi.org/10.1016/j.amc.2006.07.004 doi: 10.1016/j.amc.2006.07.004
[20]	J. Y. Fan, J. Y. Pan, A note on the Levenberg–Marquardt parameter, Appl. Math. Comput., 207 (2009), 351–359. https://doi.org/10.1016/j.amc.2008.10.056 doi: 10.1016/j.amc.2008.10.056
[21]	J. Y. Fan, The modified Levenberg–Marquardt method for nonlinear equations with cubic convergence, Math. Comp., 81 (2012), 447–466. https://doi.org/10.1090/S0025-5718-2011-02496-8 doi: 10.1090/S0025-5718-2011-02496-8
[22]	J. Y. Fan, J. L. Zeng, A Levenberg–Marquardt algorithm with correction for singular system of nonlinear equations, Appl. Math. Comput., 219 (2013), 9438–9446. https://doi.org/10.1016/j.amc.2013.03.026 doi: 10.1016/j.amc.2013.03.026
[23]	J. Y. Fan, Accelerating the modified Levenberg–Marquardt method for nonlinear equations, Math. Comp., 83 (2014), 1173–1187. https://doi.org/10.1090/S0025-5718-2013-02752-4 doi: 10.1090/S0025-5718-2013-02752-4
[24]	X. D. Zhu, G. H. Lin, Improved convergence results for a modified Levenberg–Marquardt method for nonlinear equations and applications in MPCC, Optim. Method. Softw., 31 (2016), 791–804. https://doi.org/10.1080/10556788.2016.1171863 doi: 10.1080/10556788.2016.1171863
[25]	H. Y. Wang, J. Y. Fan, Convergence rate of the Levenberg–Marquardt method under Hölderian local error bound, Optim. Method. Softw., 35 (2020), 767–786. https://doi.org/10.1080/10556788.2019.1694927 doi: 10.1080/10556788.2019.1694927
[26]	M. L. Zeng, G. H. Zhou, Improved convergence results of an efficient Levenberg–Marquardt method for nonlinear equations, J. Appl. Math. Comput., 68 (2022), 3655–3671. https://doi.org/10.1007/s12190-021-01599-6 doi: 10.1007/s12190-021-01599-6
[27]	L. Chen, Y. F. Ma, A modified Levenberg–Marquardt method for solving system of nonlinear equations, J. Appl. Math. Comput., 69 (2023), 2019–2040. https://doi.org/10.1007/s12190-022-01823-x doi: 10.1007/s12190-022-01823-x
[28]	N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, E. Teller, Equation of state calculations by fast computing machines, J. Chem. Phys., 21 (1953), 1087–1092. https://doi.org/10.1063/1.1699114 doi: 10.1063/1.1699114
[29]	R. Behling, A. Iusem, The effect of calmness on the solution set of systems of nonlinear equations, Math. Program., 137 (2013), 155–165. https://doi.org/10.1007/s10107-011-0486-7 doi: 10.1007/s10107-011-0486-7
[30]	G. W. Stewart, J. G. Sun, Matrix perturbation theory, New York: Academic Press, 1990.
[31]	R. B. Schnabel, P. D. Frank, Tensor methods for nonlinear equations, SIAM J. Numer. Anal., 21 (1984), 815–843. https://doi.org/10.1137/0721054 doi: 10.1137/0721054
[32]	J. J. Moré, B. S. Garbow, K. E. Hillstrom, Testing unconstrained optimization software, ACM T. Math. Software, 7 (1981), 17–41. https://doi.org/10.1145/355934.355936 doi: 10.1145/355934.355936
[33]	N. I. M. Gould, D. Orban, P. L. Toint. CUTEr and SifDec: A constrained and unconstrained testing environment, revisited, ACM T. Math. Software, 29 (2003), 373–394. https://doi.org/10.1145/962437.962439 doi: 10.1145/962437.962439
[34]	E. D. Dolan, J. J. Moré, Benchmarking optimization software with performance profiles, Math. Program., 91 (2002), 201–213. https://doi.org/10.1007/s101070100263 doi: 10.1007/s101070100263

Reader Comments

Your name:*

Email:*
© 2024 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)