A recent proximal gradient algorithm for convex minimization problem using double inertial extrapolations

Suparat Kesornprom; Papatsara Inkrong; Uamporn Witthayarat; Prasit Cholamjiak; Suparat Kesornprom; Papatsara Inkrong; Uamporn Witthayarat; Prasit Cholamjiak

doi:10.3934/math.2024917

AIMS Mathematics

2024, Volume 9, Issue 7: 18841-18859. doi: 10.3934/math.2024917

Previous Article Next Article

Research article

A recent proximal gradient algorithm for convex minimization problem using double inertial extrapolations

1.
Research Center in Optimization and Computational Intelligence for Big Data Prediction, Department of Mathematics, Faculty of Science, Chiang Mai University, Chiang Mai 50200, Thailand
2.
Office of Research Administration, Chiang Mai University, Chiang Mai 50200, Thailand
3.
School of Science, University of Phayao, Phayao 56000, Thailand

Received: 14 March 2024 Revised: 11 May 2024 Accepted: 21 May 2024 Published: 05 June 2024
MSC : 46E20, 46N40, 65K05, 68T07, 90C25

In this study, we suggest a new class of forward-backward (FB) algorithms designed to solve convex minimization problems. Our method incorporates a linesearch technique, eliminating the need to choose Lipschitz assumptions explicitly. Additionally, we apply double inertial extrapolations to enhance the algorithm's convergence rate. We establish a weak convergence theorem under some mild conditions. Furthermore, we perform numerical tests, and apply the algorithm to image restoration and data classification as a practical application. The experimental results show our approach's superior performance and effectiveness, surpassing some existing methods in the literature.
- forward-backward algorithm,
- minimization problem,
- inertial extrapolation,
- weak convergence
Citation: Suparat Kesornprom, Papatsara Inkrong, Uamporn Witthayarat, Prasit Cholamjiak. A recent proximal gradient algorithm for convex minimization problem using double inertial extrapolations[J]. AIMS Mathematics, 2024, 9(7): 18841-18859. doi: 10.3934/math.2024917

Related Papers:

Abstract

In this study, we suggest a new class of forward-backward (FB) algorithms designed to solve convex minimization problems. Our method incorporates a linesearch technique, eliminating the need to choose Lipschitz assumptions explicitly. Additionally, we apply double inertial extrapolations to enhance the algorithm's convergence rate. We establish a weak convergence theorem under some mild conditions. Furthermore, we perform numerical tests, and apply the algorithm to image restoration and data classification as a practical application. The experimental results show our approach's superior performance and effectiveness, surpassing some existing methods in the literature.

References

[1]	H. H. Bauschke, P. L. Combettes, Convex analysis and monotone operator theory in Hilbert spaces, New York: Springer, 2013. https://doi.org/10.1007/978-1-4419-9467-7
[2]	A. Beck, M. Teboulle, A fast iterative shrinkage-thresholding algorithm for linear inverse problems, SIAM J. Imaging Sci., 2 (2009), 183–202. https://doi.org/10.1137/080716542 doi: 10.1137/080716542
[3]	J. Y. Bello Cruz, T. T. A. Nghia, On the convergence of the forward–backward splitting method with linesearches, Optim. Method. Softw., 31 (2016), 1209–1238. https://doi.org/10.1080/10556788.2016.1214959 doi: 10.1080/10556788.2016.1214959
[4]	R. S. Burachik, A. N. Iusem, Set-valued mappings and enlargements of monotone operators, New York: Springer, 2008. https://doi.org/10.1007/978-0-387-69757-4
[5]	P. L. Combettes, L. E. Glaudin, Quasi-nonexpansive iterations on the affine hull of orbits: from Mann's mean value algorithm to inertial methods, SIAM J. Optim., 27 (2017), 2356–2380. https://doi.org/10.1137/17M112806X doi: 10.1137/17M112806X
[6]	P. L. Combettes, V. R. Wajs, Signal recovery by proximal forward-backward splitting, Multiscale Model. Sim., 4 (2005), 1168–1200. https://doi.org/10.1137/050626090 doi: 10.1137/050626090
[7]	Q. L. Dong, J. Z. Huang, X. H. Li, Y. J. Cho, T. M. Rassias, MiKM: multi-step inertial Krasnosel'skiǐ–Mann algorithm and its applications, J. Glob. Optim., 73 (2019), 801–824. https://doi.org/10.1007/s10898-018-0727-x doi: 10.1007/s10898-018-0727-x
[8]	J. C. Dunn, Convexity, monotonicity, and gradient processes in Hilbert space, J. Math. Anal. Appl., 53 (1976), 145–158. https://doi.org/10.1016/0022-247X(76)90152-9 doi: 10.1016/0022-247X(76)90152-9
[9]	O. Guler, On the convergence of the proximal point algorithm for convex minimization, SIAM J. Control Optim., 29 (1991), 403–419. https://doi.org/10.1137/0329022 doi: 10.1137/0329022
[10]	J. Han, J. Pei, H. Tong, Data mining: concepts and techniques, 4 Eds., Morgan kaufmann, 2022.
[11]	A. Hanjing, S. Suantai, A fast image restoration algorithm based on a fixed point and optimization method, Mathematics, 8 (2020), 378. https://doi.org/10.3390/math8030378 doi: 10.3390/math8030378
[12]	Y. Ho, S. Wookey, The real-world-weight cross-entropy loss function: modeling the costs of mislabeling, IEEE Access, 8 (2019), 4806–4813. https://doi.org/10.1109/ACCESS.2019.2962617 doi: 10.1109/ACCESS.2019.2962617
[13]	P. Inkrong, P. Cholamjiak, Modified proximal gradient methods involving double inertial extrapolations for monotone inclusion, Math. Method. Appl. Sci., in press. https://doi.org/10.1002/mma.10159
[14]	P. Inkrong, P. Cholamjiak, On multi-inertial extrapolations and forward-backward-forward algorithms, Carpathian J. Math., 40 (2024), 293–305.
[15]	L. O. Jolaoso, Y. Shehu, J. C. Yao, R. Xu, double inertial parameters forward-backward splitting method: applications to compressed sensing, image processing, and SCAD penalty problems, J. Nonlinear Var. Anal., 7 (2023), 627–646. https://doi.org/10.23952/jnva.7.2023.4.10 doi: 10.23952/jnva.7.2023.4.10
[16]	K. Kankam, N. Pholasa, P. Cholamjiak, Hybrid forward-backward algorithms using line search rule for minimization problem, Thai J. Math., 17 (2019), 607–625.
[17]	K. Kankam, N. Pholasa, P. Cholamjiak, On convergence and complexity of the modified forward‐backward method involving new line searches for convex minimization, Math. Method. Appl. Sci., 42 (2019), 1352–1362. https://doi.org/10.1002/mma.5420 doi: 10.1002/mma.5420
[18]	J. Liang, Convergence rates of first-order operator splitting methods, PhD thesis, Normandie Université, GREYC CNRS UMR 6072, 2016.
[19]	P. Martinet, Régularisation d'inéquations variationelles par approximations successives, Rev. fr. autom. inform. rech. opér., 4 (1970), 154–159.
[20]	Y. Nesterov, A method of solving a convex programming problem with convergence rate $\mathcal{O}(1/k^2)$, Dokl. Akad. Nauk SSSR, 269 (1983), 543.
[21]	Z. Opial, Weak convergence of the sequence of successive approximations for nonexpansive mappings, Bull. Amer. Math. Soc., 73 (1967), 591–597. https://doi.org/10.1090/S0002-9904-1967-11761-0 doi: 10.1090/S0002-9904-1967-11761-0
[22]	N. Parikh, S. Boyd, Proximal algorithms, Foundations and Trends® in Optimization, 1 (2014), 127–239. http://doi.org/10.1561/2400000003 doi: 10.1561/2400000003
[23]	B. T. Polyak, Introduction to optimization, New York: Optimization Software Inc., Publications Division, 1987.
[24]	B. T. Polyak, Some methods of speeding up the convergence of iteration methods, USSR Comput. Math. Math. Phys., 4 (1964), 1–17. https://doi.org/10.1016/0041-5553(64)90137-5 doi: 10.1016/0041-5553(64)90137-5
[25]	C. Poon, J. Liang, Trajectory of alternating direction method of multipliers and adaptive acceleration, Advances in neural information processing systems 32 (NeurIPS 2019), Vancouver, Canada, 2019, 7325–7333.
[26]	R. T. Rockafellar, Monotone operators and the proximal point algorithm, SIAM J. Control Optim., 14 (1976), 877–898. https://doi.org/10.1137/0314056 doi: 10.1137/0314056
[27]	W. Takahashi, Introduction to nonlinear and convex analysis, Yokohama Publishers, 2009.
[28]	K. H. Thung, P. Raveendran, A survey of image quality measures, 2009 international conference for technical postgraduates (TECHPOS), Kuala Lumpur, Malaysia, 2009, 1–4. https://doi.org/10.1109/TECHPOS.2009.5412098
[29]	C. Wang, N. Xiu, Convergence of the gradient projection method for generalized convex minimization, Comput. Optim. Appl., 16 (2000), 111–120. https://doi.org/10.1023/A:1008714607737 doi: 10.1023/A:1008714607737
[30]	Z. Wang, A. C. Bovik, H. R. Sheikh, E. P. Simoncelli, Image quality assessment: from error visibility to structural similarity, IEEE Trans. Image Process., 13 (2004), 600–612. https://doi.org/10.1109/TIP.2003.819861 doi: 10.1109/TIP.2003.819861
[31]	H. K. Xu, Averaged mappings and the gradient-projection algorithm, J. Optim. Theory Appl., 150 (2011), 360–378. https://doi.org/10.1007/s10957-011-9837-z doi: 10.1007/s10957-011-9837-z

Reader Comments

Your name:*

Email:*
© 2024 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)