An adaptive simple model trust region algorithm based on new weak secant equations

Yueting Yang; Hongbo Wang; Huijuan Wei; Ziwen Gao; Mingyuan Cao; Yueting Yang; Hongbo Wang; Huijuan Wei; Ziwen Gao; Mingyuan Cao

doi:10.3934/math.2024413

AIMS Mathematics

2024, Volume 9, Issue 4: 8497-8515. doi: 10.3934/math.2024413

Previous Article Next Article

Research article

An adaptive simple model trust region algorithm based on new weak secant equations

School of Mathematics and Statistics, Beihua University, Jilin 132013, China

Received: 09 January 2024 Revised: 10 February 2024 Accepted: 22 February 2024 Published: 28 February 2024
MSC : 90C06, 90C30

In this work, we proposed a new trust region method for solving large-scale unconstrained optimization problems. The trust region subproblem with a simple form was constructed based on new weak secant equations, which utilized both gradient and function values and available information from the three most recent points. A modified Metropolis criterion was used to determine whether to accept the trial step, and an adaptive strategy was used to update the trust region radius. The global convergence and locally superlinearly convergence of the new algorithm were established under appropriate conditions. Numerical experiments showed that the proposed algorithm was effective.
- trust region method,
- weak secant equation,
- modified Metropolis criterion,
- adaptive strategy
Citation: Yueting Yang, Hongbo Wang, Huijuan Wei, Ziwen Gao, Mingyuan Cao. An adaptive simple model trust region algorithm based on new weak secant equations[J]. AIMS Mathematics, 2024, 9(4): 8497-8515. doi: 10.3934/math.2024413

Related Papers:

Abstract

In this work, we proposed a new trust region method for solving large-scale unconstrained optimization problems. The trust region subproblem with a simple form was constructed based on new weak secant equations, which utilized both gradient and function values and available information from the three most recent points. A modified Metropolis criterion was used to determine whether to accept the trial step, and an adaptive strategy was used to update the trust region radius. The global convergence and locally superlinearly convergence of the new algorithm were established under appropriate conditions. Numerical experiments showed that the proposed algorithm was effective.

References

[1]	Y. Ji, Y. Ma, The robust maximum expert consensus model with risk aversion, Inf. Fusion, 99 (2023), 101866. https://doi.org/10.1016/j.inffus.2023.101866 doi: 10.1016/j.inffus.2023.101866
[2]	S. Qu, S. Li, A supply chain finance game model with order-to-factoring under blockchain, Syst. Eng. Theory Pract., 43 (2023), 3570–3586. https://doi.org/10.12011/SETP2022-2888 doi: 10.12011/SETP2022-2888
[3]	Y. Ji, Y. Yuan, Z. Peng, A novel robust flexible minimum cost consensus model with consensus granule, Group Decis. Negot., 2024. https://doi.org/10.1007/s10726-023-09869-3 doi: 10.1007/s10726-023-09869-3
[4]	N. Eslami, B. Najafi, S. M. Vaezpour, A trust region method for solving multicriteria optimization problems on riemannian manifolds, J. Optim. Theory Appl., 196 (2022), 212–239. https://doi.org/10.1007/s10957-022-02142-8 doi: 10.1007/s10957-022-02142-8
[5]	V. A. Ramirez, G. N. Sottosanto, Nonmonotone trust region algorithm for solving the unconstrained multiobjective optimization problems, Comput. Optim. Appl., 81 (2022), 769–788. https://doi.org/10.1007/s10589-021-00346-8 doi: 10.1007/s10589-021-00346-8
[6]	H. H. Dwail, M. A. K. Shiker, Using a trust region method with nonmonotone technique to solve unrestricted optimization problem, J. Phys. Conf. Ser., 1664 (2020), 012128. https://doi.org/10.1088/1742-6596/1664/1/012128 doi: 10.1088/1742-6596/1664/1/012128
[7]	L. Zhao, W. Sun, R. J. B. de Sampaio, Nonmonotone adaptive trust region method based on simple conic model for unconstrained optimization, Front. Math. China, 9 (2014), 1211–1238. https://doi.org/10.1007/s11464-014-0356-8 doi: 10.1007/s11464-014-0356-8
[8]	M. Ahookhosh, K. Amini, M. R. Peyghami, A nonmonotone trust-region line search method for large-scale unconstrained optimization, Appl. Math. Modell., 36 (2012), 478–487. https://doi.org/10.1016/j.apm.2011.07.021 doi: 10.1016/j.apm.2011.07.021
[9]	X. T. Zhu, M. Xi, W. Y. Sun, A new nonmonotone BB-TR method based on simple conic model for large scale unconstrained optimization, Numer. Math. A J. Chin. Univ., 38 (2016), 173–192.
[10]	H. Zhu, Q. Ni, J. Jiang, C. Dang, A new alternating direction trust region method based on conic model for solving unconstrained optimization, Optimization, 70 (2020), 1555–1579. https://doi.org/10.1080/02331934.2020.1745793 doi: 10.1080/02331934.2020.1745793
[11]	Q. Zhou, C. Zhang, A new nonmonotone adaptive trust region method based on simple quadratic models, J. Appl. Math. Comput., 40 (2012), 111–123. https://doi.org/10.1007/s12190-012-0572-x doi: 10.1007/s12190-012-0572-x
[12]	Q. Zhou, J. Chen, Z. Xie, A nonmonotone trust region method based on simple quadratic models, J. Comput. Appl. Math., 272 (2014), 107–115. https://doi.org/10.1016/j.cam.2014.04.026 doi: 10.1016/j.cam.2014.04.026
[13]	Q. Sun, L. Duan, B. Cui, A nomonotone trust region algorithm with simple quadratic models, J. Syst. Sci. Math. Sci., 29 (2009), 470–483.
[14]	X. Li, W. Dong, Z. Peng, A new nonmonotone trust region Barzilai-Borwein method for unconstrained optimization problems, Acta Math. Appl. Sin. Engl. Ser., 37 (2021), 166–175. https://doi.org/10.1007/s10255-021-0997-9 doi: 10.1007/s10255-021-0997-9
[15]	Y. Liu, X. Liu, Trust region BB methods for unconstrained optimization, Math. Numer. Sin., 38 (2016), 96–112. https://doi.org/10.12286/jssx.2016.1.96 doi: 10.12286/jssx.2016.1.96
[16]	Q. Zhou, W. Sun, H. Zhang, A new simple model trust-region method with generalized Barzilai-Borwein parameter for large-scale optimization, Sci. China Math., 59 (2016), 2265–2280. https://doi.org/10.1007/s11425-015-0734-2 doi: 10.1007/s11425-015-0734-2
[17]	Q. Z. Yang, Optimization method, Beijing: Science Press, 2015.
[18]	M. J. Ebadi, A. Fahs, H. Fahs, R. Dehghani, Competitive secant (BFGS) methods based on modified secant relations for unconstrained optimization, Optimization, 72 (2023), 1691–1701. https://doi.org/10.1080/02331934.2022.2048381 doi: 10.1080/02331934.2022.2048381
[19]	J. Zhang, C. Xu, Properties and numerical performance of quasi-Newton methods with modified quasi-Newton equations, J. Comput. Appl. Math., 137 (2001), 269–278. https://doi.org/10.1016/S0377-0427(00)00713-5 doi: 10.1016/S0377-0427(00)00713-5
[20]	J. E. Dennis, H. Wolkowicz, Sizing and least-change secant methods, SIAM J. Numer. Anal., 30 (1993), 1291–1314. https://doi.org/10.1137/0730067 doi: 10.1137/0730067
[21]	S. Zhao, T. Yan, K. Wang, Y. Zhu, Adaptive trust-region method on Riemannian manifold, J. Sci. Comput., 96 (2023), 67. https://doi.org/10.1007/s10915-023-02288-1 doi: 10.1007/s10915-023-02288-1
[22]	S. Lior, E. Yonathan, M. Shie, Adaptive trust region policy optimization: global convergence and faster rates for regularized MDPs, Proceedings of the AAAI Conference on Artificial Intelligence, 34 (2020), 5668–5675. https://doi.org/10.1609/aaai.v34i04.6021 doi: 10.1609/aaai.v34i04.6021
[23]	A. Kamandi, K. Amini, A new nonmonotone adaptive trust region algorithm, Appl. Math., 67 (2022), 233–250. https://doi.org/10.21136/AM.2021.0122-20 doi: 10.21136/AM.2021.0122-20
[24]	X. Zhang, J. Zhang, L. Liao, An adaptive trust region method and its convergence, Sci. China Ser. A, 45 (2002), 620–631. https://doi.org/10.1360/02ys9067 doi: 10.1360/02ys9067
[25]	Z. Shi, J. Guo, A new trust region method with adaptive radius, Comput. Optim. Appl., 41 (2008), 225–242. https://doi.org/10.1007/s10589-007-9099-8 doi: 10.1007/s10589-007-9099-8
[26]	S. Rezaee, S. Babaie-Kafaki, An adaptive nonmonotone trust region algorithm, Optim. Methods Software, 34 (2017), 264–277. https://doi.org/10.1080/10556788.2017.1364738 doi: 10.1080/10556788.2017.1364738
[27]	N. Ghalavand, E. Khorram, V. Morovati, Two adaptive nonmonotone trust-region algorithms for solving multiobjective optimization problems, Optimization, 2023. https://doi.org/10.1080/02331934.2023.2234920 doi: 10.1080/02331934.2023.2234920
[28]	X. Ding, Q. Qu, X. Wang, A modified filter nonmonotone adaptive retrospective trust region method, PLoS ONE, 16 (2021), e0253016. https://doi.org/10.1371/journal.pone.0253016 doi: 10.1371/journal.pone.0253016
[29]	M. Yousefi, A. M. Calomardo, A stochastic nonmonotone trust-region training algorithm for image classification, International IEEE Conference on Signal-Image Technologies and Internet-Based System, 2022,522–529. https://doi.org/10.1109/SITIS57111.2022.00084 doi: 10.1109/SITIS57111.2022.00084
[30]	Q. Zhou, D. Hang, Nonmonotone adaptive trust region method with line search based on new diagonal updating, Appl. Numer. Math., 91 (2015), 75–88. https://doi.org/10.1016/j.apnum.2014.12.009 doi: 10.1016/j.apnum.2014.12.009
[31]	N. I. Gould, D. Orban, P. L. Toint, UTEr and SifDec: a constrained and unconstrained testing environment, revisited, ACM Trans. Math. Software, 29 (2003), 373–394. https://doi.org/10.1145/962437.962439 doi: 10.1145/962437.962439
[32]	N. Andrei, Introduction: overview of unconstrained optimization, In: Nonlinear conjugate gradient methods for unconstrained optimization, Springer, 2020. https://doi.org/10.1007/978-3-030-42950-8_1
[33]	N. Andrei, An unconstrained optimization test functions collection, Adv. Model. Optim., 10 (2008), 147–161.
[34]	E. D. Dolan, J. J. Moré, Benchmarking optimization software with performance profiles, Math. Program., 91 (2002), 201–213. https://doi.org/10.1007/s101070100263 doi: 10.1007/s101070100263

Reader Comments

Your name:*

Email:*
© 2024 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)