In this paper, aiming at the nonlinear equations, a new two-step Levenberg–Marquardt method was proposed. We presented a new Levenberg–Marquardt parameter to obtain the trial step. A new modified Metropolis criterion was used to adjust the upper bound of the approximate step. The convergence of the method was analyzed under the H$ \ddot{\rm o} $lderian local error bound condition and the H$ \ddot\rm o $lderian continuity of the Jacobian. Numerical experiments showed that the new algorithm is effective and competitive in the numbers of functions, Jacobian evaluations and iterations.
Citation: Dingyu Zhu, Yueting Yang, Mingyuan Cao. An accelerated adaptive two-step Levenberg–Marquardt method with the modified Metropolis criterion[J]. AIMS Mathematics, 2024, 9(9): 24610-24635. doi: 10.3934/math.20241199
In this paper, aiming at the nonlinear equations, a new two-step Levenberg–Marquardt method was proposed. We presented a new Levenberg–Marquardt parameter to obtain the trial step. A new modified Metropolis criterion was used to adjust the upper bound of the approximate step. The convergence of the method was analyzed under the H$ \ddot{\rm o} $lderian local error bound condition and the H$ \ddot\rm o $lderian continuity of the Jacobian. Numerical experiments showed that the new algorithm is effective and competitive in the numbers of functions, Jacobian evaluations and iterations.
[1] |
G. D. A. Moura, S. D. T. M. Bezerra, H. P. Gomes, S. A. D. Silva, Neural network using the Levenberg–Marquardt algorithm for optimal real-time operation of water distribution systems, Urban Water J., 15 (2018), 692–699. https://doi.org/10.1080/1573062X.2018.1539503 doi: 10.1080/1573062X.2018.1539503
![]() |
[2] |
Y. J. Sun, P. P. Wang, T. T. Zhang, K. Li, F. Peng, C. G. Zhu, Principle and performance analysis of the Levenberg–Marquardt algorithm in WMS spectral line fitting, Photonics, 9 (2022), 999. https://doi.org/10.3390/photonics9120999 doi: 10.3390/photonics9120999
![]() |
[3] |
A. Alloqmani, O. Alsaedi, N. Bahatheg, R. Alnanih, L. Elrefaei, Design principles-based interactive learning tool for solving nonlinear equations, Comput. Syst. Sci. Eng., 40 (2022), 1023–1042. https://doi.org/10.32604/csse.2022.019704 doi: 10.32604/csse.2022.019704
![]() |
[4] |
Z. W. Liao, F. Y. Zhu, W. Y. Gong, S. J. Li, X. Y. Mi, AGSDE: Archive guided speciation-based differential evolution for nonlinear equations, Appl. Soft Comput., 122 (2022), 108818. https://doi.org/10.1016/j.asoc.2022.108818 doi: 10.1016/j.asoc.2022.108818
![]() |
[5] |
Z. Seifi, A. Ghorbani, A. Abdipour, Time-domain analysis and experimental investigation of electromagnetic wave coupling to RF/microwave nonlinear circuits, J. Electromagnet Wave., 35 (2021), 51–70. https://doi.org/10.1080/09205071.2020.1825994 doi: 10.1080/09205071.2020.1825994
![]() |
[6] | A. Rothwell, Numerical methods for unconstrained optimization, In: Optimization methods in structural design, Cham: Springer, 2017, 83–106. https://doi.org/10.1007/978-3-319-55197-5 |
[7] |
G. L. Yuan, M. J. Zhang, A three-terms Polak-Ribière-Polyak conjugate gradient algorithm for large-scale nonlinear equations, J. Comput. Appl. Math., 286 (2015), 186–195. https://doi.org/10.1016/j.cam.2015.03.014 doi: 10.1016/j.cam.2015.03.014
![]() |
[8] |
G. L. Yuan, Z. X. Wei, X. W. Lu, A BFGS trust-region method for nonlinear equations, Computing, 92 (2011), 317–333. https://doi.org/10.1007/s00607-011-0146-z doi: 10.1007/s00607-011-0146-z
![]() |
[9] |
J. H. Zhang, Y. Q. Wang, J. Zhao, On maximum residual nonlinear Kaczmarz-type algorithms for large nonlinear systems of equations, J. Comput. Appl. Math., 425 (2023), 115065. https://doi.org/10.1016/j.cam.2023.115065 doi: 10.1016/j.cam.2023.115065
![]() |
[10] |
J. N. Wang, X. Wang, L. W. Zhang, Stochastic regularized Newton methods for nonlinear equations, J. Sci. Comput., 94 (2023), 51. https://doi.org/10.1007/s10915-023-02099-4 doi: 10.1007/s10915-023-02099-4
![]() |
[11] |
R. Behling, D. S. Gonçalves, S. A. Santos, Local convergence analysis of the Levenberg–Marquardt framework for Nonzero–Residue nonlinear least-squares problems under an error bound condition, J. Optim. Theory Appl., 183 (2019), 1099–1122. https://doi.org/10.1007/s10957-019-01586-9 doi: 10.1007/s10957-019-01586-9
![]() |
[12] |
E. H. Bergou, Y. Diouane, V. Kungurtsev, Convergence and complexity analysis of a Levenberg–Marquardt algorithm for inverse problems, J. Optim. Theory Appl., 185 (2020), 927–944. https://doi.org/10.1007/s10957-020-01666-1 doi: 10.1007/s10957-020-01666-1
![]() |
[13] |
K. Levenberg, A method for the solution of certain non-linear problems in least squares, Quart. Appl. Math., 2 (1944), 164–168. https://doi.org/10.1090/qam/10666 doi: 10.1090/qam/10666
![]() |
[14] |
D. W. Marquardt, An algorithm for least-squares estimation of nonlinear parameters, J. Soc. Ind. Appl. Math., 11 (1963), 431–441. https://doi.org/10.1137/0111030 doi: 10.1137/0111030
![]() |
[15] | N. Yamashita, M. Fukushima, On the rate of convergence of the Levenberg–Marquardt method, In: Topics in numerical analysis, computing supplementa, Vienna: Springer, 2001,239–249. https://doi.org/10.1007/978-3-7091-6217-0_18 |
[16] | J. Y. Fan, Y. X. Yuan, On the convergence of a new Levenberg–Marquardt method, Report, Institute of Computational Mathematics and Scientific/Engineering Computing, Beijing: Chinese Academy of Science, 2001. |
[17] | J. Y. Fan, A Modified Levenberg–Marquardt algorithm for singular system of nonlinear equations, J. Comput. Math., 21 (2003), 625–636. |
[18] |
K. Amini, F. Rostami, G. Caristi, An efficient Levenberg–Marquardt method with a new LM parameter for systems of nonlinear equations, Optimization, 67 (2018), 637–650. https://doi.org/10.1080/02331934.2018.1435655 doi: 10.1080/02331934.2018.1435655
![]() |
[19] |
C. F. Ma, L. H. Jiang, Some research on Levenberg–Marquardt method for the nonlinear equations, Appl. Math. Comput., 184 (2007), 1032–1040. https://doi.org/10.1016/j.amc.2006.07.004 doi: 10.1016/j.amc.2006.07.004
![]() |
[20] |
J. Y. Fan, J. Y. Pan, A note on the Levenberg–Marquardt parameter, Appl. Math. Comput., 207 (2009), 351–359. https://doi.org/10.1016/j.amc.2008.10.056 doi: 10.1016/j.amc.2008.10.056
![]() |
[21] |
J. Y. Fan, The modified Levenberg–Marquardt method for nonlinear equations with cubic convergence, Math. Comp., 81 (2012), 447–466. https://doi.org/10.1090/S0025-5718-2011-02496-8 doi: 10.1090/S0025-5718-2011-02496-8
![]() |
[22] |
J. Y. Fan, J. L. Zeng, A Levenberg–Marquardt algorithm with correction for singular system of nonlinear equations, Appl. Math. Comput., 219 (2013), 9438–9446. https://doi.org/10.1016/j.amc.2013.03.026 doi: 10.1016/j.amc.2013.03.026
![]() |
[23] |
J. Y. Fan, Accelerating the modified Levenberg–Marquardt method for nonlinear equations, Math. Comp., 83 (2014), 1173–1187. https://doi.org/10.1090/S0025-5718-2013-02752-4 doi: 10.1090/S0025-5718-2013-02752-4
![]() |
[24] |
X. D. Zhu, G. H. Lin, Improved convergence results for a modified Levenberg–Marquardt method for nonlinear equations and applications in MPCC, Optim. Method. Softw., 31 (2016), 791–804. https://doi.org/10.1080/10556788.2016.1171863 doi: 10.1080/10556788.2016.1171863
![]() |
[25] |
H. Y. Wang, J. Y. Fan, Convergence rate of the Levenberg–Marquardt method under Hölderian local error bound, Optim. Method. Softw., 35 (2020), 767–786. https://doi.org/10.1080/10556788.2019.1694927 doi: 10.1080/10556788.2019.1694927
![]() |
[26] |
M. L. Zeng, G. H. Zhou, Improved convergence results of an efficient Levenberg–Marquardt method for nonlinear equations, J. Appl. Math. Comput., 68 (2022), 3655–3671. https://doi.org/10.1007/s12190-021-01599-6 doi: 10.1007/s12190-021-01599-6
![]() |
[27] |
L. Chen, Y. F. Ma, A modified Levenberg–Marquardt method for solving system of nonlinear equations, J. Appl. Math. Comput., 69 (2023), 2019–2040. https://doi.org/10.1007/s12190-022-01823-x doi: 10.1007/s12190-022-01823-x
![]() |
[28] |
N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, E. Teller, Equation of state calculations by fast computing machines, J. Chem. Phys., 21 (1953), 1087–1092. https://doi.org/10.1063/1.1699114 doi: 10.1063/1.1699114
![]() |
[29] |
R. Behling, A. Iusem, The effect of calmness on the solution set of systems of nonlinear equations, Math. Program., 137 (2013), 155–165. https://doi.org/10.1007/s10107-011-0486-7 doi: 10.1007/s10107-011-0486-7
![]() |
[30] | G. W. Stewart, J. G. Sun, Matrix perturbation theory, New York: Academic Press, 1990. |
[31] |
R. B. Schnabel, P. D. Frank, Tensor methods for nonlinear equations, SIAM J. Numer. Anal., 21 (1984), 815–843. https://doi.org/10.1137/0721054 doi: 10.1137/0721054
![]() |
[32] |
J. J. Moré, B. S. Garbow, K. E. Hillstrom, Testing unconstrained optimization software, ACM T. Math. Software, 7 (1981), 17–41. https://doi.org/10.1145/355934.355936 doi: 10.1145/355934.355936
![]() |
[33] |
N. I. M. Gould, D. Orban, P. L. Toint. CUTEr and SifDec: A constrained and unconstrained testing environment, revisited, ACM T. Math. Software, 29 (2003), 373–394. https://doi.org/10.1145/962437.962439 doi: 10.1145/962437.962439
![]() |
[34] |
E. D. Dolan, J. J. Moré, Benchmarking optimization software with performance profiles, Math. Program., 91 (2002), 201–213. https://doi.org/10.1007/s101070100263 doi: 10.1007/s101070100263
![]() |