An accelerated forward-backward algorithm with a new linesearch for convex minimization problems and its applications

1.   Introduction

Throughout this article, we suppose that $\mathcal{H}$ is a real Hilbert space with an inner product $\langle \cdot, \cdot \rangle$ and the induced norm $\| \cdot \|$. We are interested in studying the following unconstrained convex minimization problem:

where $f_{1}, f_{2} : \mathcal{H}\to \mathbb{R}\cup \{\infty\}$ are two proper, lower semi-continuous and convex functions such that $f_{1}$ is differentiable on an open set containing the domain of $f_{2}$. Problem (1.1) has been widely studied due to its applications which can be used in various real-world applications such as in signal and image processing, in regression problems, and in classification problems, etc., see ^{[3,5,8,10,11,13]} and the references therein. One of the important topics of studying Problem (1.1) is to invent some efficient procedures for approximating minimizers of $f_{1}+f_{2}$. Various optimization methods were introduced and developed by many researchers, see ^{[3,5,6,7,8,12,14,15,16,25,26,27,32]}, for instance.

If a minimizer $x^{*}$ of $f_{1}+f_{2}$ exists, it is known that $x^{*}$ is characterized by the fixed point equation of the forward-backward operator

where $\alpha > 0$, $\operatorname{prox} _{f_2}$ is the proximity operator of $f_{2}$ and $\nabla f_{1}$ stands for the gradient of $f_{1}$. The above equation leads to the following iterative method:

Method 1. Let $x_{1} \in \operatorname{dom} f_{2}$. For $k \geq 1$, let

where $0 < \alpha_{k} < \frac{2}{L}$ and $L$ is a Lipschitz constant of $\nabla f_{1}$.

This method is well known as the forward-backward splitting algorithm ^[8,15], which includes the proximal point algorithm ^[17,24], the gradient method ^[4,9] and the $CQ$ algorithm ^[1] as special cases. It is observed from Method 1 that we need to assume the Lipschitz continuity condition on the gradient of $f_{1}$ and the stepsize $\alpha_{k}$ depends on the Lipschitz constant $L$, which is not an easy task to find in general practice (see ^[3,5,8,12] for other relevant methods).

In the sequel, we set the standing hypotheses on Problem (1.1) as follows:

$\textbf{(H1)}$ $f_{1}, f_{2}:\mathcal{H}\to \mathbb{R}\cup \{\infty\}$ are two proper, lower semi-continuous and convex functions with $\operatorname{dom} f_{2} \subseteq \operatorname{dom} f_{1},$ and $\operatorname{dom} f_{2}$ is nonempty, closed and convex;

$\textbf{(H2)}$ $f_{1}$ is differentiable on an open set containing $\operatorname{dom} f_{2}$. The gradient $\nabla f_{1}$ is uniformly continuous on any bounded subset of $\operatorname{dom} f_{2}$ and maps any bounded subset of $\operatorname{dom} f_{2}$ to a bounded set in $\mathcal{H}$.

We note that the second part of (H2) is a weaker assumption than the Lipschitz continuity assumption on $\nabla f_{1}$.

Cruz and Nghia ^[7] proposed a technique for selecting the stepsize $\alpha_{k}$ which is independent of the Lipschitz constant $L$ by using the following linesearch process.

Linesearch 1 is a particular case of the linesearch proposed in ^[29] for inclusion problems and it was shown that this linesearch is well defined, that is, it stops after finitely many steps, see [7, Lemma 3.1] and [29, Theorem 3.4(a)]. Cruz and Nghia ^[7] employed the forward-backward iteration where the stepsize $\alpha_{k}$ is generated by Linesearch 1.

Method 2. Let $x_{1} \in \operatorname{dom} f_{2}$, $\sigma > 0$, $\theta\in (0, 1)$ and $\delta \in (0, \frac{1}{2}).$ For $k \geq 1$, let

where $\alpha_{k}: =$ Linesearch 1$(x_{k}, \sigma, \theta, \delta).$

In optimization theory, to speed up the convergence of iterative methods, many mathematicians often use the inertial-type extrapolation ^[20,22] by adding the technical term $\beta_{k}(x_{k}-x_{k-1})$. The control parameter $\beta_{k}$ is called an inertial parameter, which controls the momentum $x_{k}-x_{k-1}$. Using Linesearch 1, Cruz and Nghia ^[7] also introduced an accelerated algorithm with the inertial technical term as follows.

Method 3. Let $x_{0} = x_{1} \in \operatorname{dom} f_{2}$, $\alpha_{0} = \sigma > 0$, $\theta\in (0, 1)$, $\delta \in (0, \frac{1}{2})$, and $t_{1} = 1$. For $k \geq 1$, let

where $\alpha_{k}: =$ Linesearch 1$(y_{k}, \alpha_{k-1}, \theta, \delta).$

The technique of choosing $\beta_{k}$ in Method 3 was first mentioned in the fast iterative shrinkage-thresholding algorithm (FISTA) by Beck and Teboulle ^[5]. Weak convergence results of Methods 2 and 3 were obtained for solving Problem (1.1) with (H1) and (H2).

Recently, Kankam et al. ^[14] proposed a modification of Linesearch 1 as follows.

They showed the well-definedness of Linesearch 2 and introduced the following double forward-backward algorithm whose stepsize is generated by Linesearch 2.

Method 4. Let $x_{1} \in \operatorname{dom} f_{2}$, $\sigma > 0$, $\theta\in (0, 1)$ and $\delta \in (0, \frac{1}{8}).$ For $k \geq 1$, let

where $\alpha_{k}: =$ Linesearch 2$(x_{k}, \sigma, \theta, \delta).$

A weak convergence theorem of Method 4 was proved and an application in signal recovery was illustrated, see ^[14].

In this paper, inspired and motivated by the results of Cruz and Nghia ^[7] and Kankam et al. ^[14], and other related researches, we aim to improve Linesearches 1 and 2 and introduce a new accelerated algorithm using our proposed linesearch for the convex minimization problem of the sum of two convex functions. The paper is organized as follows. Basic definitions, notations and some useful tools for proving our convergence results are given in Section 2. Our main result is in Section 3. In this section, we introduce a new modification of Linesearches 1 and 2 and present a double forward-backward algorithm by using an inertial technique for solving Problem (1.1) with Hypotheses (H1) and (H2). After that, a weak convergence theorem of the proposed method is proved. The complexity of our reduced algorithm is also discussed. In Section 4, we apply the convex minimization problem to an image restoration problem and a regression problem. We analyze and illustrate the convergence behavior of our method, and also compare its efficiency with the well-known methods in the literature.

2.   Notations and tools

The mathematical symbols used throughout this paper are as follows. $\mathbb{R}$, $\mathbb{R}_{+}$ and $\mathbb{N}$ are the set of real numbers, the set of nonnegative real numbers, and the set of positive integers, respectively. $I_{d}$ stands for the identity operator on $\mathcal{H}$. Denote weak and strong convergence of a sequence $\{x_{k}\} \subset \mathcal{H}$ to $x \in \mathcal{H}$ by $x_{k} \rightharpoonup x$ and $x_{k} \rightarrow x$, respectively. The set of all weak-cluster points of $\{x_{k}\}$ is denoted by $\omega_{w}(x_{k})$. If $C$ is a nonempty closed convex subset of $\mathcal{H}$, then $P_{C}$ stands for the metric projection from $\mathcal{H}$ onto $C$, i.e., for each $x \in \mathcal{H}$, $P_{C}x$ is the unique element in $C$ such that $\| x - P_{C}x \| = \operatorname{dist}(x, C) : = \inf_{y \in C } \| x - y \|.$

Let us recall the concept of the proximity operator which extends the notion of the metric projection. Let $f : \mathcal{H} \to \mathbb{R}\cup \{\infty\}$ be a proper, lower semi-continuous and convex function. The proximity (or proximal) operator ^[2,18] of $f$, denoted by $\operatorname{prox}_{f}$ is defined for each $x \in \mathcal{H}$, $\operatorname{prox}_{f}x$ is the unique solution of the minimization problem:

If $f : = i_{C}$ is an indicator function on $C$ (defined by $i_{C}(x) = 0$ if $x \in C$; otherwise $i_{C}(x) = \infty$), then $\operatorname{prox}_{f} = P_{C}$.

The proximity operator can be formulated in the equivalent form

where $\partial f$ is the subdifferential of $f$ defined by

Moreover, we have the following useful fact:

The following is a property of the subdifferential operator.

Lemma 2.1 (^[23]). If $f : \mathcal{H} \to \mathbb{R}\cup \{\infty\}$ is a proper, lower semi-continuous and convex function, then the graph of $\partial f$ defined by $Gph(\partial f): = \{(x, y)\in \mathcal{H}\times \mathcal{H} : y\in \partial f(x)\}$ is demiclosed, i.e., if the sequence $\{(x_{k}, y_{k})\} \subset Gph(\partial f)$ satisfies that $x_{k}\rightharpoonup x$ and $y_{k}\to y,$ then $(x, y) \in Gph(\partial f).$

We end this section by providing useful tools for proving our main results.

Fact. Let $x, y \in \mathcal{H}$. The following inequalities hold on $\mathcal{H}$:

Lemma 2.2 (^[12]). Let $\{a_{k}\}$ and $\{t_{k}\}$ be two sequences of nonnegative real numbers such that

Then the following holds

Moreover, if $\sum_{k = 1}^{\infty}t_{k} < \infty,$ then $\{a_{k}\}$ is bounded.

Lemma 2.3 (^[31]). Let $\{a_{k}\}$ and $\{b_{k}\}$ be two sequences of nonnegative real numbers such that $a_{k+1}\leq a_{k}+b_{k}$ for all $k \in \mathbb{N}$. If $\sum_{k = 1}^{\infty}b_{k} < \infty,$ then $\lim_{k\to\infty}a_{k}$ exists.

Lemma 2.4 (Opial ^[19]). Let $\{x_{k}\}$ be a sequence in $\mathcal{H}$ such that there exists a nonempty set $\Omega \subset \mathcal{H}$ satisfying:

(i) For every $p\in \Omega,$ $\lim_{k\to\infty}\|x_{k}-p\|$ exists;

(ii) $\omega_{w}(x_{k})\subset\Omega.$

Then, $\{x_{k}\}$ converges weakly to a point in $\Omega.$

3.   Methods and convergence analysis

In this section, using the idea of Linesearches 1 and 2, we introduce a new linesearch and present an inertial double forward-backward algorithm with the proposed linesearch for solving the convex minimization problem of the sum of two convex functions without any Lipschitz continuity assumption on the gradient. A weak convergence result of our proposed method is analyzed and established.

We now focus on Problem (1.1) and assume that $f_{1}$ and $f_{2}$ satisfy Hypotheses (H1) and (H2). The solution set of (1.1) is denoted by $\Omega$. Also, suppose that $\Omega \neq \emptyset$. For simplicity, we let $\mathcal{F} : = f_{1} + f_{2}$ and denote by $FB_{\alpha}$ the forward-backward operator of $f_{1}$ and $f_{2}$ with respect to $\alpha$, that is, $FB_{\alpha} : = \operatorname{prox}_{\alpha f_{2}}(I_{d}-\alpha \nabla f_{1})$. Here, our linesearch is designed as follows.

Remark 3.1. The loop termination condition of Linesearch 3 is weaker than that of Linesearch 2. Indeed, since

it follows from the well-definedness of Linesearch 2 that our linesearch also stops after finitely many steps, see [14, Lemma 3.2] for more details.

Using Linesearch 3, we propose the following iterative method with the inertial technical term.

To verify the convergence of Method 5, the following result is needed.

Lemma 3.2. Let $\{x_{k}\}$ be the sequence generated by Method 5. For each $k\geq 1$ and $p \in \operatorname{dom} f_{2},$ we have

Proof. Let $k\geq 1$ and $p \in \operatorname{dom} f_{2}.$ By (2.3), (3.1) and (3.2), we get

Then, by the definition of the subdifferential of $f_{2},$ we have

and

By the assumptions on $f_{1}$ and $f_{2}$, we have the following fact:

From (3.6), we get

and

Summing (3.4), (3.5), (3.7) and (3.8) yields

Using (3.6) again, the above inequality becomes

Since $\alpha_{k}: =$ Linesearch 3$(x_{k}, \sigma, \theta, \mu, \delta)$, we have

It follows from (3.9) and (3.10) that

From (2.4), we get

and

Therefore, we conclude from (3.11)-(3.13) that

Now we are ready to prove a convergence result of Method 5, which is the main theorem of this paper.

Theorem 3.3. Let $\{x_{k}\}$ be the sequence generated by Method 5. If $\alpha_{k}\geq \alpha > 0,$ $\beta_{k}\geq 0$ for all $k\in \mathbb{N}$ and $\sum_{k = 1}^{\infty}\beta_{k} < \infty,$ then $\{x_{k}\}$ converges weakly to a point in $\Omega.$

Proof. Let $p^{*} \in \Omega,$ $\alpha_{k}\geq \alpha > 0,$ $\beta_{k}\geq 0$ for all $k\in \mathbb{N}$ and $\sum_{k = 1}^{\infty}\beta_{k} < \infty.$ By applying Lemma 3.2, we have

which implies

Let $x_{0} = x_{1}.$ By (3.3) and (3.15), we have

By Lemma 2.2, we have $\|x_{k+1}-p^{*}\|\leq K \cdot\prod_{j = 1}^{k}(1+2\beta_{j}),$ where $K: = \max\{\|x_{1}-p^{*}\|, \|x_{2}-p^{*}\|\},$ and $\{x_{k}\}$ is bounded. Thus, $\sum_{k = 1}^{\infty}\beta_{k}(\|x_{k}-p^{*} \|+\|x_{k-1}-p^{*} \|) < \infty.$ In view of (3.16), it follows from Lemma 2.3 that $\lim_{k\to\infty}\|x_{k}-p^{*}\|$ exists. Also, we have $\lim_{k\to\infty}\|x_{k}-p^{*}\| = \lim_{k\to\infty}\|y_{k}-p^{*}\|.$ Then, by (3.14), we get

Next, we show that $\omega_{w}(x_{k})\subset \Omega.$ Pick $\bar{p} \in \omega_{w}(x_{k}),$ then there is a subsequence $\{x_{k_{i}} \}$ of $\{x_{k}\}$ such that $x_{k_{i}}\rightharpoonup \bar{p}.$ Thus, $z_{k_{i}}\rightharpoonup \bar{p}.$ By (H2), we have

From (2.3), we get

Here, by $z_{k_{i}}\rightharpoonup \bar{p}$ and (3.17)-(3.19), it follows from Lemma 2.1 that $0\in \partial \mathcal{F}(\bar{p}),$ that is, $\bar{p}\in \Omega.$ So, $\omega_{w}(x_{k})\subset \Omega.$ Using Lemma 2.4, we obtain that $x_{k} \rightharpoonup p$ for some $p \in \Omega$. This completes the proof.

The following method is obtained by reducing the inertial step in Method 5.

We also analyze the convergence and the complexity of Method 6.

Theorem 3.4. Let $\{x_{k}\}$ be the sequence generated by Method 6. If $\alpha_{k}\geq \alpha > 0$ for all $k\in \mathbb{N},$ then the following statements hold:

(i) $\{x_{k}\}$ converges weakly to a point in $\Omega;$

(ii) If $\delta\in (0, \frac{\mu}{8}),$ then $\{\mathcal{F}(x_{k})\}$ is decreasing and

Proof. Setting $\beta_{k} : = 0$ in Method 5, (i) is obtained directly from Theorem 3.3. To prove (ii), let $\delta\in (0, \frac{\mu}{8}).$ We first show that $\{ \mathcal{F}(x_{k})\}$ is decreasing. By Lemma 3.2, we get

Taking $p: = z_{k}$ in (3.23), we have

Also, taking $p: = x_{k}$ in (3.23), we have

Combining (3.24) and (3.25), we get

which implies

This together with $\delta\in (0, \frac{\mu}{8})$ yields $\mathcal{F}(x_{k+1})\leq \mathcal{F}(x_{k}),$ that is, $\{\mathcal{F}(x_{k})\}$ is decreasing. Next, we will show that (3.22) is true. Pick any $p^{*}\in \Omega.$ From (3.23), we have

Hence

It follows that

By the decreasing property of $\{\mathcal{F}(x_{k})\}$, we get

It follows from (3.26) that

We note that the stepsize condition on $\{\alpha_{k}\}$ in Theorems 3.3 and 3.4 needs the boundedness from below by a positive real number. Next, we show that this condition can be guaranteed by the Lipschitz continuity assumption on the gradient.

Let $C \subset \mathcal{H}$ be a nonempty closed convex set. Recall that an operator $T : C \rightarrow \mathcal{H}$ is said to be Lipschitz continuous if there exists $L > 0$ such that $\|Tx - Ty\| \leq L \|x - y\|$ for all $x, y \in C.$

Proposition 3.5. Let $\{\alpha_{k}\}$ be the sequence generated by Linesearch 3 of Method 5 (or Method 6). If $\nabla f_{1}$ is Lipschitz continuous on $\operatorname{dom} f_{2}$ with a coefficient $L > 0,$ then $\alpha_{k}\geq \min \left\{ \sigma, \frac{\delta\theta}{L}\right\}$ for all $k \in \mathbb{N}.$

Proof. Let $\nabla f_{1}$ be $L$-Lipschitz continuous on $\operatorname{dom} f_{2}$. From Linesearch 3, we know that $\alpha_{k} \leq \sigma$ for all $k \in \mathbb{N}$. If $\alpha_{k} < \sigma$, then $\alpha_{k} = \sigma \theta^{m_{k}}$ where $m_{k}$ is the smallest positive integer such that

Let $\bar{\alpha_{k}}: = \frac{\alpha_{k}}{\theta} > 0$. By the Lipschitz continuity of $\nabla f_{1}$ and the above expression, we have

which implies that $\alpha_{k} > \frac{\delta\theta}{L}$. Consequently, $\alpha_{k}\geq \min \left\{ \sigma, \frac{\delta\theta}{L}\right\}$ for all $k \in \mathbb{N}.$

Remark 3.6. It is worth mentioning again that the Lipschitz continuity assumption on the gradient of $f_{1}$ is sufficient for Hypothesis (H2). However, if we assume this assumption further, the calculation of the stepsize $\alpha_{k}$ generated by Linesearch 3 is still independent of the Lipschitz constant.

4.   Applications

In this section, we apply the convex minimization problem, Problem (1.1), to an image restoration problem and a regression problem. To slove these problems, we analyze and illustrate the convergence behavior of our method (Method 5) and compare its efficiency with Methods 1-4 and 6. We also consider the following method with another linesearch strategy ^[21]:

Method 7. Let $x_{1} = y_{0} \in \operatorname{dom} f_{2}$, $\bar{\sigma} > 0$, $\bar{\theta} \in (0, 1)$, $\rho_{k} > 0$, and $t_{1} = 1$. For $k \geq 1$, let

where $\alpha_{k}: =$ Linesearch 4$(x_{k}, \bar{\sigma}, \bar{\theta}).$

Note that Method 7 is well known as the FISTA with backtracking, see ^[5,25].

All experiments and visualizations are done with MATLAB and are performed on a laptop computer (Intel Core-i5/4.00 GB RAM/Windows 8/64-bit).

4.1.   Image restoration problems

Many problems rising in image and signal processing, especially the image restoration problem, can be formulated as the following equation:

where $x\in \mathbb{R}^{N}$ is an original image, $y \in \mathbb{R}^{M}$ is the observed image, $\varepsilon$ is an additive noise and $A \in \mathbb{R}^{M\times N}$ is the blurring operation. To approximate the original image, we need to minimize the value of $\varepsilon$ by using the LASSO model ^[28]:

where $\lambda > 0$ is a regularization parameter, $\|\cdot\|_{1}$ is the $l_{1}$-norm, and $\|\cdot\|_{2}$ is the Euclidean norm. In this situation, we apply Problem (1.1) to the LASSO model (4.2) by setting

where $A = RW$, $R$ is the matrix representing the blur operator and $W$ is the inverse of a three stage Haar wavelet transform. We use the peak signal-to-noise ratio (PSNR) in decibel (dB) ^[30] and the structural similarity index metric (SSIM) ^[33] as the image quality measures, which are formulated as follows:

where $MSE(k) = \frac{1}{M}\|x_{k}- x\|^{2}_{2},$ $M$ is the number of image samples, and $x$ is the original image and

where $\{\nu_{x_{k}}, \xi_{x_{k}}\}$ and $\{\nu_{x}, \xi_{x}\}$ denote the mean intensity and standard deviation set of the deblerring image $x_{k}$ and the original image $x,$ respectively, $\xi_{x_{n}x}$ denotes its cross correlation, and $C_{1}$, $C_{2}$ are small constants value to avoid instability problem when the denominator is too close to zero.

Example 4.1. Consider two grayscale images, Cameraman (see Figure 1 (a)) and Boy (see Figure 2 (a)), with size of $256\times 256$, which are contaminated by Gaussian blur of filter size $9\times 9$ with standard deviation $\hat{\sigma} = 4$, noise $10^{-5}$ (see Figure 1 (b) and Figure 2 (b)).

Firstly, we test the efficiency of Method 5 for recovering the Caneraman image at the 300th iteration by choosing various linesearch parameters and different starting points as shown in Table 1.

It is observed from Table 1 that the linesearch parameter $\sigma$ influences the image recovery performance of Method 5, while choosing the starting point $x_{1}$ has no significant impact on the convergence behavior of our method.

Next, we test and analyze the image recovery performance of Methods 1-7 for recovering the images (Cameraman and Boy) by setting the parameters as in Table 2 and by choosing the blurred images as the starting points. The maximum iteration number for all methods is fixed at 500. The comparative experiments for recovering the Cameraman and Boy images are as follows. Original images, blurry image contaminated by Gaussian blur and restored images by Methods 1-7 are shown in Figures 1 and 2. The PSNR and SSIM results are shown in Figures 3 and 4. It can be seen that Method 5 gives the higher values of PSNR and SSIM than the others, meanwhile Method 6 gives the better results than the other methods without the inertial step. Therefore, our method has the highest image recovery efficiency comparing with the studied methods.

4.2.   Regression problems

We first introduce a brief concept of extreme learning machine (ELM) for regression problems. Let $S = \{(x_{k}, t_{k}) \; :\; x_{k}\in \mathbb{R}^{n}, t_{k}\in \mathbb{R}^{m}, k = 1, 2, ..., N \}$ be a training set of $N$ distinct samples where $x_{k}$ is an input data and $t_{k}$ is a target. For any single hidden layer of ELM, the output of the $i$-th hidden node is

where $G$ is an activation function, $a_{i}$ and $b_{i}$ are parameters of the $i$-th hidden node. The output function of ELM for a single-hidden layer feedforward neural networks (SLFNs) with $M$ hidden nodes is

where $\xi_{i}$ is the output weight of the $i$-th hidden node. The hidden layer output matrix $H$ is defined by

The main goal of ELM is to find $\xi = [\xi_{1}^{\top}, ..., \xi_{M}^{\top}]^{\top}$ such that

where $T = [t_{1}^{\top}, ..., t_{N}^{\top}]^{\top}.$ From (4.3), we can estimate the weight $\xi$ by $\xi = H^{\dagger}T$ where $H^{\dagger} = (H^{\top}H)^{-1}H^{\top}$ is the pseudo-inverse matrix of $H$.

Example 4.2. This example aims to predict the sine function by using the ELM method defined as follows:

$\bullet$ Create randomly 10 distinct points $y_{1}, y_{2}, ..., y_{10}\in [-4, 4];$

$\bullet$ Create a training matrix $R = [y_{1} \; y_{2}\; ... \; y_{10}]^{\top}$ and a training target matrix $S = [\sin(y_{1}) \; \sin(y_{2}) \; ...\; \sin(y_{10})]^{\top}$;

$\bullet$ Create a testing matrix $V = [-4 \; -3.99\; -3.98\; ...\; 4]$ and a target matrix $T = [\sin(-4) \; \sin(-3.99)\; ...\; \sin(y_{10})]^{\top}$;

$\bullet$ Using sigmoid as an activation function generates the hidden layer output matrix $H_{1}$ of the training matrix $R$ with the hidden node $M = 100;$

$\bullet$ Choose a regularization parameter $\lambda = 10^{-5}.$ Formulate a convex minimization problem via the LASSO model that

and find the optimal weight $\xi_{k}$ of this problem by employing Methods 1-5 with certain number of iterations;

$\bullet$ Using sigmoid as an activation function generates the hidden layer output matrix $H_{2}$ of the testing matrix $V$ with the hidden node $M = 100;$

$\bullet$ Calculate the output $O_{k} = H_{2}\xi_{k}$ and calculate the mean squared error (MSE) at $k$-th iteration by

where $N$ is the number of distinct samples.

The parameters for each prediction method are set as in Table 3.

Now a graph of predicting the sine function by Methods 1-7 at the 500th iteration is shown in Figure 5. We show a comparative result of iteration numbers, mean squared error values and times in Table 4 when the stopping criterion is set as: $MSE\leq 10^{-3}$ or at the 10000th iteration. Also, a graph of the mean squared error is shown in Figure 6. It can be observed that Method 5 gives a better performance for predicting the sine function than the other tested methods.

5.   Conclusions

In this paper, we discuss the convex minimization problem of the sum of two convex functions in a Hilbert space. The challenge of removing the Lipschitz continuity assumption on the gradient of the function attracts us to study the concept of the linesearch process. We introduce a new linesearch and propose an inertial forward-backward algorithm whose stepsize does not depend on any Lipschitz constant for solving the considered problem without any Lipschitz continuity condition on the gradient. It is shown that the sequence generated by our proposed method converges weakly to a minimizer of the sum of those two convex functions under some mild control conditions. As applications, we employ our method to recover blurry images in an image restoration problem and predict the sine function in a regression problem. The results of the experiments show that our method has a higher efficiency than the well-known methods in ^{[5,7,14,15,25]}.

Acknowledgments

This research was supported by Thailand Science Research and Innovation under the project IRN62W0007 and Chiang Mai University.

Conflict of interest

The authors declare no conflict of interest.