$V$-Moreau envelope of nonconvex functions on smooth Banach spaces

1.   Introduction and preliminaries

Let $X$ be a Banach space with dual space $X^*$. The duality pairing between $X$ and $X^*$ will be denoted by $\langle\cdot, \cdot\rangle$. We denote by $\mathbb B$ and $\mathbb B_*$ the closed unit ball in $X$ and $X^*$, respectively. The normalized duality mapping $J:X\rightrightarrows X^*$ is defined by

where $\|\cdot\|$ stands for both norms on $X$ and $X^*$. Similarly, we define $J^*$ on $X^*$. Many properties of $J$ and $J^*$ are well known and we refer the reader, for instance, to ^[15].

Definition 1.1. For a fixed closed subset $S$ of $X$, a fixed function $f:S\to \mathbb R\cup\{\infty\}$, and a fixed $\lambda > 0$, we define the following functional: $G^V_{\lambda, f}:X^*\times S \to \mathbb R\cup\{\infty\}$

where $V(x^*, x) = \|x^*\|^2-2\langle x^*, x\rangle +\|x\|^2$. Clearly, the functional $V$ has the form $V (x^*; x) = \|x^*-x\|^2$, whenever $X$ is a Hilbert space (i.e., $X^* = X$). This remark highlights the significance of using the functional $V$ rather than the square of the norm, as the latter cannot generally be expressed in the form of $V$ in Banach spaces.

Using the functional $G^V_{\lambda, f}$, we define the $V$-Moreau envelope of $f$ associated with $S$ as follows:

We also define the generalized $(f, \lambda)$-projection on $S$ as follows:

$(\bullet)$ When $f = 0$, $\lambda = \frac{1}{2}$, the generalized $(f, \lambda)$-projection $\pi^{f, \lambda}_S$ on $S$ coincides with the generalized projection $\pi_S$ over $S$.

$(\bullet)$ When $\lambda = \frac{1}{2}$, the generalized $(f, \lambda)$-projection $\pi^{f, \lambda}_S$ on $S$ coincides with the $f$-generalized projection $\pi^f_S$ introduced and studied in ^[16,17].

We need some important results that we gather in the following proposition (see for instance ^[1,5,6]).

Proposition 1.1. Let $X$ be a Banach space.

$1)$ If $X$ is $q$-uniformly convex, then for any $\alpha > 0$, there exists some constant $K_\alpha > 0$ such that

$2)$ If $X$ is $p$-uniformly smooth, then the dual space $X^*$ is $p'$-uniformly convex with $p' = \frac{p}{p-1}$.

$3)$ Assume that $X$ is $q$-uniformly convex and let $\alpha > 0$. Then, for any $x^*\in \alpha \mathbb{B}_*$ and any $y\in \alpha \mathbb{B}$,

where $c > 0$ is the constant given in the definition of $q$-uniform convexity of $X$.

We also recall many concepts and definitions as follows:

Definition 1.2.

$1)$ Let $f:X\to {\mathbb R}\cup \{+\infty\}$ be a lower semi-continuous function (l.s.c. in short) and $x\in X$, where $f$ is finite. The $V$-proximal subdifferential (see ^[8]) $\partial^\pi f$ of $f$ at $x$ is defined by $x^*\in \partial^\pi f(x)$ if and only if there exist $\sigma > 0, \delta > 0$ such that

We notice that $\partial^\pi f(\bar x)\subset L\mathbb{B}_*$, whenever $f$ is locally Lipschitz at $\bar x$ (see ^[4]).

$2)$ The $V$-proximal normal cone of a nonempty closed subset $S$ in $X$ at $x\in S$ is defined as the $V$-proximal subdifferential of the indicator function of $S$, that is, $N^\pi(S; x) = \partial^\pi \psi_S(x)$. Note that $N^\pi$ is also characterized (see ^[4]) via $\pi_S$ as follows:

$3)$ The Fréchet subdifferential and Fréchet normal cone (see for instance ^[3,14]) are defined as follows: $x^*\in \partial^F f(\bar x)$ if and only if for all $\epsilon > 0$, there exists $\delta > 0$ such that

The Fréchet normal cone $N^F(S; x)$ of a nonempty closed subset $S$ in $X$ at $\bar x\in S$ is defined as $N^F(S; \bar x) = \partial^F \psi_S(\bar x)$.

$4)$ The limiting $V$-proximal normal cone is defined as follows (see ^[7]):

Before starting our study, we state some special cases showing the importance of the study of $e^V_{\lambda, S}f$ and $\pi_S^{f, \lambda}$.

Case 1. If $X$ is a Hilbert space and $S = X$, the functional $e^V_{\lambda, S}f$ coincides with the Moreau envelope of $f$ with index $\lambda > 0$ and the generalized $(f, \lambda)$-projection $\pi^{f, \lambda}_S$ coincides with the proximal mapping $P_\lambda(f)$ (see for instance ^[13]).

Case 2. If $X$ is a Hilbert space and $f\equiv0$, the generalized $(f, \lambda)$-projection $\pi^{f, \lambda}_S$ coincides with the metric projection on $S$ (see for instance ^[9,10]).

Case 3. If $X$ is a reflexive Banach space, the generalized $(f, \lambda)$-projection $\pi^{f, \lambda}_S$ coincides with the generalized projection $\pi_S$ on $S$ introduced for closed convex sets in ^[2,11,12] and for closed nonconvex sets in ^[4,6].

Motivated by the special cases presented above and their relevance (as seen in ^{[1,2,3,4,11,12,16,17]} and their references), we initially introduced and started investigating the generalized $(f, \lambda)$-projection $\pi^{f, \lambda}_S$ in ^[5]. There, we laid the groundwork for understanding its basic properties and potential applications. In this paper, we aim to expand upon that foundation by delving into more advanced properties of $\pi^{f, \lambda}_S$, particularly in relation to the differentiability of the functional $e^V_{\lambda, S}f$. This deeper analysis offers new perspectives on its theoretical framework and behavior. The application of these results to nonconvex variational inequalities will be addressed in a series of forthcoming papers.

2.   Main results on the $V$-Moreau envelope $e^V_{\lambda, S}f$

In the following proposition, we prove the local Lipschitz behavior of the $V$-Moreau envelope $e^V_{\lambda, S}f$.

Proposition 2.1. Let $X$ be a reflexive Banach space. Assume that $f$ is bounded below on $S$ by $\beta\in \mathbb R$. Then, for any $x^*\in X^*$, the function $e^V_{\lambda, S}f$ is Lipschitz on every neighborhood of $x^*$, that is, for any $x^*\in X^*$ and for any $\delta > 0$, there exists $K_{x^*, \delta} > 0$ such that

Proof. Let $x^*\in X^*$ and fix some $\delta > 0$. Let $\epsilon\in (0, \delta)$ and $r: = e^V_{\lambda, S}f(x^*)\ge \beta$. Fix now any $y^*, z^*\in x^*+\delta \mathbb B_*$. By definition of the infimum in the expression of $e^V_{\lambda, S}f$, there exists $s_\epsilon \in S$ such that

Then,

We need to find an upper bound of $\|s_\epsilon\|$. To do that, we use once again the definition of the infimum in $e^V_{\lambda, S}f(x^*)$ to get an element $x_\epsilon\in S$ such that

From the definition of $V$, we have $V(y^*, s_\epsilon) = \|y^*\|^2-2\langle y^*, s_\epsilon\rangle +\|s_\epsilon\|^2$. This gives

Hence,

Then, we obtain:

Observe that $f(x_\epsilon) < e^V_{\lambda, S}f(x^*)+\epsilon < r+\epsilon$ and $f(y_\epsilon)\ge \beta$, and so we get

Therefore,

By taking $M_{\delta, x^*}: = \sqrt{\left(2\|x^*\|+ \sqrt{2\lambda(r+\delta)}+\delta\right)^2+\delta}+\|x^*\|+\delta$, we get an upper bound of $\|s_\epsilon\|$ in terms of $r$, $\delta$, and $x^*$. Thus, we can write

where $K_{\delta, x^*}: = \frac{1}{\lambda} \left(\|x^*\|+M_{\delta, x^*}+\delta \right)$. Taking $\epsilon \to 0$ and interchanging the roles of $z^*$ and $y^*$, we get

This completes the proof.□

We recall from ^[5] the following result needed in the proof of the next theorem.

Proposition 2.2. Assume that $X$ is a reflexive Banach space with smooth dual norm, and let $S$ be any closed nonempty set of $X$ and $f:S\to \mathbb R\cup\{\infty\}$ be any l.s.c. function. Then for any $x^*\in dom \, \pi_S^{f, \lambda}$, any $\bar x\in \pi_S^{f, \lambda}(x^*)$, and any $t\in [0, 1)$, we have $\pi_S^{f, \lambda}(J(\bar x)+t(x^*-J(\bar x))) = \{\bar x\}$.

We prove the following result ensuring the existence and uniqueness of the generalized $(f, \lambda)$-projection on closed nonempty sets under natural assumptions on the Fréchet subdifferentiability of the $V$-Moreau envelope.

Theorem 2.1. Assume that $X$ is a reflexive Banach space with smooth dual norm, and let $S$ be any closed nonempty set of $X$ and $f:S\to \mathbb R\cup\{\infty\}$ be any l.s.c. function. Then the following assertions hold.

$1)$ If $\partial^F e^V_{\lambda, S}f(x^*)\ne \emptyset$, then the generalized $(f, \lambda)$-projection of $x^*$ on $S$ exists and is unique and moreover $\partial^F e^V_{\lambda, S}f(x^*) = \{\frac{1}{\lambda}\left[J^*x^*-\pi^{f, \lambda}_S(x^*)\right]\}$;

$2)$ If $\pi^{f, \lambda}_S(x^*)\ne \emptyset$, then $\partial^F e^V_{\lambda, S}f(x^*)\subset\{\frac{1}{\lambda}\left[J^*x^*-\pi^{f, \lambda}_S(x^*)\right]\}$;

$3) \; \partial^F e^V_{\lambda, S}f(x^*)\ne \emptyset$ if and only if $e^V_{\lambda, S}f$ is Fréchet differentiable at $x^*$.

Proof. (1) Assume that $\partial^F e^V_{\lambda, S}f(x^*)\ne \emptyset$ and let $y\in \partial^F e^V_{\lambda, S}f(x^*)$ and let $\epsilon > 0$. By the definition of $\partial^F e^V_{\lambda, S}f(x^*)$, there exists $\delta > 0$ such that for any $t\in (0, \delta)$ and any $v^*\in \mathbb{B}$, we have

By the definition of $e^V_{\lambda, S}f(x^*)$, for any $n\ge 1$, the exists some $y_n\in S$ such that

Therefore,

Thus,

Since the norm of the dual space is smooth, we can take the limit $t\to 0^+$ to get

and hence,

This ensures that $\lim\limits_{n\to \infty}\| y+\frac{1}{\lambda}[y_n-J^*x^*] \| = 0$, that is, $y_n \to J^*x^*-\lambda y$ as $n\to \infty$. Set $\tilde x: = J^*x^*-\lambda y$, and take the limit as $n\to \infty$ in the inequality (2.1), we obtain:

which means that $\tilde y\in \pi^{f, \lambda}_S(x^*)$. The uniqueness can be shown easily and so the first assertion is proved.

(2) This assertion follows directly from (1). Indeed, if $\partial^F e^V_{\lambda, S}f(x^*) = \emptyset$, then we are done. Otherwise, we assume that $\partial^F e^V_{\lambda, S}f(x^*)\ne \emptyset$, and so the assertion (1) ensures that $\partial^F e^V_{\lambda, S}f(x^*) = \{\frac{1}{\lambda}\left[J^*x^*-\pi^{f, \lambda}_S(x^*)\right]\}.$ Consequently, for both cases, we have $\partial^F e^V_{\lambda, S}f(x^*)\subset \{\frac{1}{\lambda}\left[J^*x^*-\pi^{f, \lambda}_S(x^*)\right]\}$, and so the proof of (2) is complete.

(3) Obviously, the Fréchet differentiability of $e^V_{\lambda, S}f$ ensures that $\partial^F e^V_{\lambda, S}f(x^*)\ne \emptyset$. So, we have to prove the reverse implication. We assume that $\partial^F e^V_{\lambda, S}f(x^*)\ne \emptyset$, and we are going to prove that $e^V_{\lambda, S}f$ is Fréchet differentiable at $x^*$. Using the assertion (1), we get a generalized $(f, \lambda)$-projection $\bar y\in S$, such that

Thus, we have $e^V_{\lambda, S}f(x^*) = f(\bar y)+\frac{1}{2\lambda} V(x^*; \bar y)$. By the definition of the Fréchet subdifferential, there exists $\delta > 0$ such that for any $t\in (0, \delta)$ and any $v^*\in \mathbb{B}_*$, we have

Hence,

and hence, for any $v^*\in \mathbb{B}_*$ and any $\epsilon > 0$,

On the other hand, we have, by the definition of $e^V_{\lambda, S}f$,

and so,

Combining this inequality (2.3) with (2.2), and taking $\epsilon \to 0^+$, we obtain the existence of the Fréchet derivative of $e^V_{\lambda, S}f$ at $x^*$ and $\nabla^F e^V_{\lambda, S}f(x^*) = \frac{1}{ \lambda} \left[ J^*x^*-\bar y \right]$.□

We prove in the next two theorems various characterizations of the continuous Fréchet differentiability of the $V$-Moreau envelope $e^V_{\lambda, S}f$ over open sets. We need to recall the Kadec property of the Banach space $X$, that is, for any sequence $(x_n)_n$ in $X$, we have that $(x_n)$ is strongly convergent to some limit $\bar x$ if and only if $(x_n)$ is weakly convergent to $\bar x$ and $\|x_n\| \to \|\bar x\|$.

Theorem 2.2. Assume that $X$ is a reflexive Banach space with Kadec property and with smooth dual norm. Let $U$ be an open subset in $X^*$. Consider the following assertions:

$1) \; e^V_{\lambda, S}f$ is $C^1$ on $U$;

$2) \; e^V_{\lambda, S}f$ is Fréchet differentiable on $U$;

$3) \; e^V_{\lambda, S}f$ is Fréchet subdifferentiable on $U$, that is, $\partial^F e^V_{\lambda, S}f(x^*)\ne \emptyset, \forall x^*\in U$;

$4) \; \pi^{f, \lambda}_S$ is single-valued and norm-to-weak continuous on $U$;

$5) \; \pi^{f, \lambda}_S$ is single-valued and norm-to-norm continuous on $U$.

Then, the following implications and equivalences are true.

Proof. The implications $(1)\Rightarrow (2)$ and $(5)\Rightarrow (4)$ follow directly from the definitions. The equivalence $(2)\Leftrightarrow(3)$ follows from part (3) in Theorem 2.1. We have to prove the implication $(2)\Rightarrow (4)$. We assume that $e^V_{\lambda, S}f$ is Fréchet differentiable on $U$, and let $x_n^*$ be a sequence in $U$ converging to some point $x^*\in X^*$. First, we prove that $\nabla^F e^V_{\lambda, S}f(x^*_n)$ weakly converges to $\nabla^F e^V_{\lambda, S}f(x^*)$. Observe that

with $h(x^*): = \sup\limits_{y\in X}\left\{\frac{1}{\lambda}\langle x^*; y\rangle -f(y)-\frac{\|y\|^2}{2\lambda}-\psi_S(y)\right\}$. The function $h$ is clearly convex Fréchet differentiable on $U$ and so its derivative $\nabla^F h$ is norm-to-weak continuous on $U$, and so $\nabla^F h(x^*_n)$ weakly converges to $\nabla^F h(x^*)$. Since the norm of the dual space $X^*$ is smooth, we have $\nabla^F \frac{\|x^*_n \|^2}{2\lambda} \to \nabla^F \frac{\|x^* \|^2}{2\lambda}$ and consequently, we get that $\nabla^F e^V_{\lambda, S}f(x^*_n) = \nabla^F \frac{\|x^*_n \|^2}{2\lambda}-\nabla^F h(x^*_n)$ weakly converges to $\nabla^F \frac{\|x^* \|^2}{2\lambda}-\nabla^F h(x^*) = \nabla^F e^V_{\lambda, S}f(x^*)$. We use Theorem 2.1 to write $\nabla^F e^V_{\lambda, S}f(x^*_n) = \frac{1}{\lambda}\left[J^*x^*_n-\pi^{f, \lambda}_S(x^*_n)\right]$ and $\nabla^F e^V_{\lambda, S}f(x^*) = \frac{1}{\lambda}\left[J^*x^*-\pi^{f, \lambda}_S(x^*)\right]$. Therefore, we obtain the weak convergence of $\pi^{f, \lambda}_S(x^*_n)$ to $\pi^{f, \lambda}_S(x^*)$, thereby satisfying the assertion (4).□

This result extends Theorem 2.10 in ^[6] from the case $f\equiv 0$ to $f\not \equiv 0$.

In order to get the equivalence between all the assertions in Theorem 2.2, we need an additional assumption on the function $f$, which is the weak lower semicontinuity of $f$.

Theorem 2.3. Assume that $X$ is a reflexive Banach space with Kadec property and with smooth dual norm. Assume further that $f$ is weak lower semicontinuous on $U$. Then, all the assertions in Theorem 2.2 are equivalent, that is,

Proof. We have to prove the implications $(2)\Rightarrow (1)$ and $(4)\Rightarrow (5)$. We start with $(4)\Rightarrow (5)$. Assume that $\pi^{f, \lambda}_S$ is single-valued and norm-to-weak continuous on $U$. Let $x_n^*$ be a sequence in $U$ converging to some point $x^*\in X^*$. We have to prove that $x_n: = \pi^{f, \lambda}_S(x^*_n)$ converges to $\bar x: = \pi^{f, \lambda}_S(x^*)$. By assumption (4), we have $(x_n)$ weakly converges to $\bar x$. Using the local Lipschitz continuity of $e^V_{\lambda, S}f$ proved in Proposition 2.1, we can write

Observe that

Taking the limit superior as $n\to +\infty$ in this equality and using the weak l.s.c. of $f$, we get

On the other hand, we always have $\| \bar x\| \le \liminf\limits_{n \to +\infty} \| x_n\|$. Thus, we obtain $\lim\limits_{n \to +\infty} \| x_n\| = \| \bar x\|$. Finally, we use the fact that $x_n$ weakly converges to $\bar x$ and $\| x_n\|$ converges to $\| \bar x\|$, and the Kadec property of the space to deduce the convergence of $x_n$ to $\bar x$, and so the proof of (5) is complete. We turn to prove the implication $(2)\Rightarrow (1)$. We assume that $e^V_{\lambda, S}f$ is Fréchet differentiable on $U$. We have to prove that $\nabla^F e^V_{\lambda, S}f$ is continuous on $U$. Let $x_n^*$ be a sequence in $U$ converging to some point $x^*\in X^*$, and we have to prove that $\nabla^F e^V_{\lambda, S}f(x^*_n)\to \nabla^F e^V_{\lambda, S}f(x^*)$. Using Theorem 2.1, we have the existence and uniqueness of $\pi^{f, \lambda}_S(x^*_n)$ and

Similarly, we have

Using the implications $(2)\Rightarrow (4)$ and $(4)\Rightarrow (5)$, we get the convergence of $\pi^{f, \lambda}_S(x^*_n)$ to $\pi^{f, \lambda}_S(x^*)$. Consequently, we use the continuity of $J^*$ to deduce the following:

and so the proof of the theorem is complete.

3.   Main results on the generalized $(f, \lambda)$-projection $\pi_{S}^{f, \lambda}$

In this section, we need more regularity assumptions on the function $f$ and the set $S$ to establish our main results on the generalized $(f, \lambda)$-projection. First, we start with the generalized uniform $V$-prox-regularity concept introduced and studied in ^[6].

Definition 3.1. A nonempty closed subset $S$, in a reflexive smooth Banach space $X$, is called $V$-uniformly generalized prox-regular if and only if there exists $r > 0$ such that $\forall x\in S, \forall x^*\in N^\pi(S; x)$ (with $x^*\ne0$), we have $x\in \pi_S\left(J(x)+r\frac{x^*}{\|x^*\|}\right)$.

Example 3.1.

$1)$ Any closed convex set is generalized uniformly $V$-prox-regular with any positive number $r > 0$.

$2)$ We state from ^[6] the following nonconvex example of generalized uniformly $V$-prox-regular sets. Let $x_0\in X$ with $\|x_0\| > 3$. The set $S: = \mathbb{B}\cup (x_0+\mathbb{B})$ is nonconvex but it is generalized uniformly $V$-prox-regular for some $r > 0$ (for its proof, we refer to Example 4.1 in ^[6]).

Remark 3.1.

$1)$ It has been proved in Theorem 3.2 in ^[7] that for generalized uniformly $V$-prox-regular sets $S$, we have $N^\pi(S; x) = N^{L\pi}(S; x)$, $\forall x\in S$.

$2)$ From Theorem 3.3 in ^[7], we deduce that for bounded generalized uniformly $V$-prox-regular sets $S$, there exists some $r > 0$ such that for all $x\in S$ and any $x^*\in N^\pi(S; x)$ with $\|x^*\| < 1$, we have

Now, we state the concept of $V$-prox-regular functions uniformly over sets.

Definition 3.2. Let $f:X \to \mathbb R\cup \{\infty\}$ be a l.s.c. function, and let $S\subset dom\, f$ be a nonempty set. We say that $f$ is $V$-prox-regular uniformly over $S$ provided that there exists some $r > 0$ such that for any $x\in S$ and any $x^*\in \partial^{L\pi} f(x)$:

Example 3.2.

$1)$ Any l.s.c. convex function $f$ is $V$-prox-regular with any positive number $r > 0$ uniformly over any closed subset $S\subset \text{dom} \, f$.

$2)$ The distance function $d_S$ associated with generalized uniformly $V$-prox-regular set $S$ (in the sense of Definition 3.1) is $V$-prox-regular uniformly over $S$ with the same positive number $r > 0$. Indeed, for any $x\in S$ and any $x^*\in \partial^{L\pi} d_S(x)$, we have $x^*\in N^{L\pi}(S; x) = N^{\pi}(S; x)$ and $\|x^*\|\le 1$. We set $y^*: = \frac{x^*}{\|x^*\|+\epsilon}$ for $\epsilon > 0$. We have $y^*\in N^{\pi}(S; x)$ with $\|y^*\| < 1$. Then, by (3.1), we have

Thus,

Taking $\epsilon \to 0^+$ gives

This ensures by definition that $d_S$ is a $V$-prox-regular function uniformly over $S$ with the same constant $r > 0$.

We start by proving the following important result for this class of $V$-prox-regular functions uniformly over sets. It proves the $r$-hypomonotony of $\partial^{L\pi}f$ uniformly over sets for $V$-prox-regular functions $f$ uniformly over closed sets.

Proposition 3.1. Assume that $f$ is $V$-prox-regular uniformly over $S \subset dom\, f$ with constant $r > 0$. Then for any $x_1, x_2 \in S$ and any $y^*_1\in \partial^{L\pi} f(x_1)$ and $y^*_2\in \partial^{L\pi} f(x_2)$, we have

Proof. Let $x_1, x_2 \in S \subset dom\, f$ and $y^*_1\in \partial^{L\pi} f(x_1)$ and $y^*_2\in \partial^{L\pi} f(x_2)$. Then, we have, by the $V$-prox-regularity of $f$ uniformly over $S$,

and

Adding these two inequalities yields

Notice that we always have

Thus,

This completes the proof.□

Lemma 3.1. Let $S$ be any closed nonempty set in a reflexive Banach space $X$, and let $f:S\to \mathbb R\cup\{\infty\}$ be any l.s.c. function. Then for any $(x^*, x)$ in the graph of $\pi^{f, \lambda}_S$, we have

Proof. Let $x^*\in X^*$ and $x\in \pi^{f, \lambda}_S(x^*)$. Then, by the definition of $\pi^{f, \lambda}_S$, we have

Hence,

Observe that

Hence,

and so the inequality (3.6) becomes

Thus,

and so,

This ensures, by the definition of $\partial^\pi$, that

and hence, $x^* \in J(x)+\lambda \partial^{L\pi} f (x)+N^{L\pi}(S; x)$. This completes the proof.□

We recall from ^[4] the following density theorem for the generalized $(f, \lambda)$-projection on closed nonempty sets.

Theorem 3.2. Assume that $X$ is a reflexive Banach space with smooth dual norm and let $S$ be any closed nonempty set of $X$, and let $f:S\to \mathbb R\cup\{\infty\}$ be any l.s.c. function. Then, the set of points in $X^*$ admitting unique generalized $(f, \lambda)$-projection on $S$ is dense in $X^*$, that is, for any $x^*\in X^*$, there exists $x^*_n \to x^*$ with $\pi^{f, \lambda}_S(x^*_n)\not = \emptyset, \forall n$.

Now, we are ready to prove one of the main results in this paper. We define the argmin of a function $f$ over a given set $S$ as the set of elements in $S$ that achieve the global minimum of $f$ in $S$, that is,

Also, we define the set:

Notice that for any $x\in {\rm arg}\min\limits_{S}(f)$, we have $e^V_{\lambda, S}f(J(x)) = f(x)$. Indeed, for any $x\in {\rm arg}\min\limits_{S}(f)$, we have $f(x)\le f(y), \forall y\in S$, and so $\forall \lambda \ge 0$

This ensures that $f(x)\le e^V_{\lambda, S}f(J(x))$. Since the reverse inequality is always valid, we obtain the desired equality. We state and prove the Hölder continuity of the generalized $(f, \lambda)$-projection $\pi^{f, \lambda}_S$.

Theorem 3.3. Let $X$ be a $q$-uniformly convex and $p$-uniformly smooth Banach space. Assume that the following assumptions hold:

$1) \; S$ is generalized uniformly $V$-prox-regular with constant $r_2 > 0$;

$2) \; f$ is $V$-prox-regular uniformly over $S$ with constant $r_1 > 0$;

$3) \; f$ is $L$-locally Lipschitz over $S$, that is, for any $\bar x\in S$, there exists $\delta > 0$ such that

$4) \; f$ is bounded from below by some real number $\beta\in \mathbb{R}$;

$5) \; {\rm arg}\min\limits_{S}(f)\ne \emptyset$;

$6) \; \lambda \in \left(0, \min\left\{\frac{r_2}{8L}, \frac{r_1}{2}\right\}\right)$.

Then, there exist $\alpha_0\ge 0$ and $r_0\ge 0$ such that for any $\alpha > \max \{ \alpha_0, \sqrt{ 2\lambda (r_2^2-\beta)} \}$, $\beta \le \frac{16c \alpha^2 }{\lambda}\left(\frac{r_2 }{64 \alpha }\right)^{\frac{p}{p-1}}$, and any $r'\in \left(r_0, \min\left\{r_2, \sqrt{ \frac{16c \alpha^2 }{\lambda}\left(\frac{r_2 }{64 \alpha }\right)^{\frac{p}{p-1}}+\beta }\right\}\right)$, we have that the generalized $(f, \lambda)$-projection $\pi^{f, \lambda}_S$ is single-valued and Hölder continuous with coefficient $\frac{1}{q-1}$ on $U^{V, \lambda}_{S, f}(r')\cap \alpha \, \text{int} (\mathbb B_*)$, i.e., for some $\gamma > 0$, we have

Proof. First, we choose some $\alpha_0\ge 0$ and some $r_0\ge 0$ so that

Indeed, by assumption, we have ${\rm arg}\min\limits_{S}(f)\ne \emptyset$, that is, there exists $z_0\in {\rm arg}\min\limits_{S}(f)$. Set $\alpha_0: = \|z_0\|$ and $r_0: = \sqrt{f(z_0)}, \, \text{ if } f(z_0) > 0$, and $r_0: = 0, \text{ if } f(z_0)\le0$. Clearly, $J(z_0)\in U^{V, \lambda}_{S, f}(r')\cap \alpha\, \text{int} (\mathbb B_*)$, and so $U^{V, \lambda}_{S, f}(r')\cap \alpha\, \text{int} (\mathbb B_*)\ne \emptyset, \; \forall \alpha > \alpha_0, \forall r\ge r_0.$

Fix now any $\alpha > \max \{ \alpha_0, \sqrt{ 2\lambda (r_2^2+\beta)} \}$, $\beta \le \frac{16c \alpha^2 }{\lambda}\left(\frac{r_2 }{64 \alpha }\right)^{\frac{p}{p-1}}$, and any $r'\in \left(r_0, \min\left\{r_2, \sqrt{ \frac{16c \alpha^2 }{\lambda}\left(\frac{r_2 }{64 \alpha }\right)^{\frac{p}{p-1}}-\beta }\right\}\right)$. Then, $U^{V, \lambda}_{S, f}(r')\cap \alpha \, \text{int} (\mathbb B_*)\not = \emptyset$. We divide our proof into two steps.

Step 1. In the first step, we prove the conclusion of the theorem for any $x_1^*, x_2^* \in U^{V, \lambda}_{S, f}(r')\cap \alpha \, \text{int} (\mathbb B_*)$ with $\pi^{f, \lambda}_S(x^*_i)\not = \emptyset$, $i = 1, 2$, that is, $x_1^*, x_2^* \in U^{V, \lambda}_{S, f}(r')\cap dom\, \pi^{f, \lambda}_S \cap \alpha\, \text{int} (\mathbb B_*)$. Fix any two points $x_1^*, x^*_2 \in U^{V, \lambda}_{S, f}(r')\cap dom\, \pi^{f, \lambda}_S \cap \alpha\, \text{int} (\mathbb B_*)$. Then, there exist $x_i\in \pi^{f, \lambda}_S(x^*_i)$, $i = 1, 2$. Without loss of generality, we assume that $x_1\not = x_2$. We have to prove that for some $\gamma > 0$,

By Lemma 3.1, there exist $y_i^* \in \partial^{L\pi}f(x_i)$ ($i = 1, 2$) such that $z_i^*: = x_i^*-J(x_i)-\lambda y_i^*\in N^{L\pi}(S; x_i) = N^{\pi}(S; x_i)$, $i = 1, 2$ (by Part (1) in Remark 3.1). So, by the generalized uniform $V$-prox-regularity of $S$ with ratio $r_2$, we have

Then by the definition of $\pi_S$, we have

and so,

with $w^*_i: = Jx_i+r_2\frac{z_i^*}{\|z_i^*\|}$, $i = 1, 2$. Since the function $V(w_i^*; \cdot)$ is convex differentiable on $X$ and its derivative is given by $\nabla^FV(w^*_i; \cdot)(z) = 2\left[Jz-w^*_i\right]$, then, we can write

Taking $y = x_i$ and $z\in S$ in the previous inequality yields

Hence,

and hence,

Thus, by taking $z = x_2$ and $z = x_1$, respectively, we obtain:

and

Now, we turn to the bound of $z^*_i, \, \text{ for } i = 1, 2$. First, observe that $\|x_i^*\| < \alpha$. Since $f$ is locally Lipschitz over $S$ with constant $L$, we have $\|y_i^*\|\le L$. Also, we have for $i = 1, 2$,

Let $M: = 2\alpha$. Since $X$ is $p$-uniformly smooth, the dual space $X^*$ is $p'$-uniformly convex with $p' = \frac{p}{p-1}$. Thus, by Part (3) in Proposition 1.1, there exists some $c > 0$ depending on the dual space $X^*$ such that

where $C: = \sqrt{\frac{\|x^*\|^2+\|y^*\|^2}{2}}$. Set $\bar c: = \frac{4^{p'-1}M ^{p'-2}}{c}$. Since $\text{ for } i = 1, 2$, $\|J(x_i)\| = \|x_i\| \le M$ and $\|x_i^*\| \le M$, we have $C = \sqrt{\frac{\|x_i^*\|^2+\|J(x_i)\|^2}{2}}\le M.$ Thus,

Thus, $\text{ for } i = 1, 2$,

where the last inequality follows from our assumptions on $\lambda$ and $r'$. Thus, the two inequalities (3.9)–(3.10) become

Adding these two inequalities gives

Hence,

On the other hand, we have by the $r_1$-hypomonotony of $f$ uniformly over $S$ proved in Proposition 3.1, we have

Therefore,

which ensures that

Using the assumption that $X$ is $q$-uniformly convex, Part (1) in Proposition 1.1, and the fact that $\|x_i\| \le M$, we have for some positive constant $K_{M} > 0$

and so,

Thus,

$\text{with } \gamma: = \left(\frac{2r_1}{K_{M} (r_1-2 \lambda)} \right)^{\frac{1}{q-1}} > 0.$ This completes the proof of the first step.

Step 2. We are going to prove that $U^{V, \lambda}_{S, f}(r')\cap \alpha\, \text{int} (\mathbb B_*) \subset \text{ dom } \pi^{f, \lambda}_S$ with $\alpha$ and $r'$ as in Step 1. Let $z^*\in U^{V, \lambda}_{S, f}(r')\cap \alpha\, \text{int} (\mathbb B_*)$, and choose $\delta > 0$ so that $z^*+\delta \mathbb B_*\subset U^{V, \lambda}_{S, f}(r')\cap \alpha\, \text{int} (\mathbb B_*)$. Let $\eta\in (0, \frac{\delta}{2})$ and fix any $x^*\in z^*+\eta \mathbb B_*$ and any $k\ge 1$. By the density theorem stated in Theorem 3.2, we can choose, for any $n\ge k$, some $x^*_n \in x^*+\frac{1}{n} \mathbb B_*$ and $y_n\in\pi^{f, \lambda}_S(x^*_n)$. For $n$ sufficiently large, we have $\frac{1}{n} < \frac{\delta}{2}$, and hence, we obtain $x_n^*\in z^*+(\eta+\frac{1}{n}) \mathbb B_*\subset z^*+\delta \mathbb B_* \subset U^{V, \lambda}_{S, f}(r')\cap \alpha\, \text{int} (\mathbb B_*)$. Clearly, $(x_n^*)_n$ is a sequence in $U^{V, \lambda}_{S, f}(r')\cap\alpha\, \text{int} (\mathbb B_*)\cap dom\, \pi^{f, \lambda}_S$. Then by Step 1, we can write for any $n, m \ge k$

Since the sequence $(x^*_n)_n$ is convergent to $x^*$, then the sequence $(y_n)_n$ is a Cauchy sequence in $X$, and hence, it converges to some limit $\bar y\in S$. By construction, we have $y_n\in \pi^{f, \lambda}_S(x^*_n)$, that is,

Using the continuity of $V$ and the Lipschitz continuity of $f$ over $S$, and the convergence of $x^*_n$ to $x^*$ and $y_n$ to $\bar y$, we obtain:

which means by definition that $\bar y\in \pi^{f, \lambda}_S(x^*)$, that is, $x^*\in dom\, \pi^{f, \lambda}_S$, and hence, $z^*+\eta \mathbb B_*\subset dom\, \pi^{f, \lambda}_S$. This ensures that $U^{V, \lambda}_{S, f}(r')\cap \alpha\, \text{int} (\mathbb B_*) \subset dom \pi^{f, \lambda}_S$, that is, $U^{V, \lambda}_{S, f}(r')\cap \alpha\, \text{int} (\mathbb B_*) \cap dom\, \pi^{f, \lambda}_S = U^{V, \lambda}_{S, f}(r')\cap \alpha\, \text{int} (\mathbb B_*)$. This equality with Step 1 completes the proof of the Hölder continuity of $\pi^{f, \lambda}_S$ over $U^{V, \lambda}_{S, f}(r')\cap \alpha\, \text{int} (\mathbb B_*)$.

We end the proof of the theorem by proving the single-valuedness of $\pi^{f, \lambda}_S$ on $U^{V, \lambda}_{S, f}(r')\cap \alpha\, \text{int} (\mathbb B_*)$. Let $x^*\in U^{V, \lambda}_{S, f}(r')\cap \alpha\, \text{int} (\mathbb B_*)$ with two generalized $(f, \lambda)$-projections $x_1, x_2\in \pi^{f, \lambda}_{S}(x^*)$. Then the inequality (3.8) gives $\|x_1-x_2\| \le \gamma \|x^*-x^*\|^{\frac{1}{q-1}} = 0$, and hence $x_1 = x_2$. This completes the proof.□

First, we derive straight-forwardly the following particular case when $f = 0$ proved in Theorem 4.4 in ^[6]. In this case, we have $\beta = 0$, $L = 0$, $r_1 = +\infty$, $\text{arg}\min\limits_S(f) = S$, and we take $\lambda = \frac{1}{2}$.

Theorem 3.4. Let $X$ be a $q$-uniformly convex and $p$-uniformly smooth Banach space. Assume that $S$ is generalized uniformly $V$-prox-regular with constant $r_2 > 0$. Then, there exists $\alpha_0\ge 0$ such that for any $\alpha > \max \{ \alpha_0, r_2 \}$ and any $r'\in \left(0, \min\left\{r_2, 4\alpha \sqrt{ 2c \left(\frac{r_2 }{64 \alpha }\right)^{\frac{p}{p-1}} }\right\}\right)$, we have that the generalized projection $\pi_S$ is single-valued and Hölder continuous with coefficient $\frac{1}{q-1}$ on $U^{V}_{S}(r')\cap \alpha \, \mathit{\text{int}} (\mathbb B_*)$, i.e., for some $\gamma > 0$, we have

The convex case when $f$ is a convex function and $S$ is a closed convex set in $\text{dom }f$ is deduced from Theorem 3.3 as follows: First, we notice that any convex function is $V$-prox-regular with $r_1 = +\infty$ uniformly over any closed subset in its domain and any closed convex set is generalized uniformly $V$-prox-regular with $r_2 = +\infty$.

Theorem 3.5. Let $X$ be a $q$-uniformly convex and $p$-uniformly smooth Banach space and let $\lambda > 0$. Assume that $S$ is a closed convex set and that $f$ is convex $L$-Lipschitz over $S$. Assume that $f$ is bounded from below by $\beta\in \mathbb{R}$. Then for any $\alpha \ge 0$, the generalized $(f, \lambda)$-projection $\pi^{f, \lambda}_S$ is single-valued and Hölder continuous with coefficient $\frac{1}{q-1}$ on $\alpha \, \mathit{\text{int}} (\mathbb B_*)$, i.e., for some $\gamma > 0$, we have

We have to mention that even in the convex case, the best result obtained on the $(f, \lambda)$-projection has been proved in Theorem 3.4 in ^[17] in which the authors proved the continuity (not the Hölder continuity) under the positive homogenous assumption on the function $f$ and the compactness assumption on $S$. These two very strong assumptions are not needed in our proof.

Now, we are going to study the local property of the generalized $(f, \lambda)$-projection, that is, for a given $\epsilon > 0,$ a closed subset $S$, and a given point $\bar x\in \text{arg}\min_{S}f$, we are interested in the localization of the generalized $(f, \lambda)$-projection of the elements $x^*$ in the $\epsilon$-neighborhood of $J(\bar x)$. First, we prove the following technical result.

Proposition 3.2. Let $M > 0$ such that ${\rm arg }\min\limits_S f\cap M\mathbb{B}\ne \emptyset$. Then for any $x^*\in M\mathbb{B}_*$, we have

Proof. Let $x_0\in {\rm arg }\min\limits_S f\cap M\mathbb{B}\ne \emptyset$. Then, $x_0\in S\cap M\mathbb{B}$ with

Hence,

Fix now any $y\in S$ with $\|y\| > 3M$. Then, we have

Taking the infimum over all $y\in S\setminus 3M\mathbb{B}$ and using the inequality (3.13), we obtain:

On the other hand, we have

Combining this inequality with (3.14), we get

Therefore,

This completes the proof.□

We deduce the following proposition.

Proposition 3.3. Assume that $X$ is a $q$-uniformly convex Banach space. Let $S$ be a closed nonempty set in $X$ with $\bar x\in {\text{arg}}\min\limits_{S}f$ and let $\epsilon > 0$. Let $M: = \|\bar x\|+\epsilon$, $\epsilon_1: = \frac{c\epsilon^q }{8^{q-1}M^{q-2}}$, and

Then, for any $x^* \in {\mathcal N}_{\epsilon_1, \frac{\epsilon}{2}}(J\bar x)$, we have

Proof. Fix $\epsilon > 0$ and $\bar x\in \text{arg}\min_{S}f$ and let $M: = \|\bar x\|+\epsilon$ and $\epsilon_1: = \frac{c\epsilon^q }{8^{q-1}M^{q-2}}$, where $c > 0$ is the constant given in the definition of the $q$-uniform convexity of $X$.

Set

Then,

Using Part (3) in Proposition 1.1, we have for any $x^*\in {\mathcal N}_{\epsilon_1, \frac{\epsilon}{2}}(J\bar x)$ and any $y\in S\cap M\mathbb{B}$

Observe that $(\bar x+\epsilon \mathbb{B})\cap S \cap 3M\mathbb{B} = (\bar x+\epsilon \mathbb{B})\cap S$. Take any $x^* \in {\mathcal N}_{\epsilon_1, \frac{\epsilon}{2}}(J\bar x)$ and any $y\in [S\cap 3M\mathbb{B}]\setminus (\bar x+\epsilon \mathbb{B})$. Then, we have

Taking the infimum over all $y\in [S\cap 3M\mathbb{B}]\setminus (\bar x+\epsilon \mathbb{B})$, we obtain:

Hence,

On the other side, since $\bar x\in {\rm arg }\min\limits_S f$ and $\bar x\| \le M$, we have ${\rm arg }\min\limits_S f\cap M\mathbb{B}\ne \emptyset$. Consequently, we obtain by Proposition 3.2, $e^V_{\lambda, S \cap 3M \mathbb{B}} f(x^*) = e^V_{\lambda, S} f(x^*)$, which ensures the equality

Thus, the proof is achieved.□

Observe that ${\mathcal N}_{\epsilon_1, \frac{\epsilon}{2}}(J\bar x)$ is an open neighborhood of $J(\bar x)$ in $X^*$. So, for any $\epsilon > 0$, we can find some constant $\delta > 0$ such that $J(\bar x)+\delta \mathbb{B} \subset {\mathcal N}_{\epsilon_1, \frac{\epsilon}{2}}(J\bar x)$. Therefore, we can state the following localization theorem.

Theorem 3.6. Assume that $X$ is a $q$-uniformly convex Banach space. Let $S$ be a closed nonempty set in $X$ with $\bar x\in {\rm arg }\min\limits_S f\cap M\mathbb{B}$. Then for any $\epsilon > 0$, we can find some constant $\delta > 0$ such that for any $x^* \in J(\bar x)+\delta \mathbb{B}$, we have

4.   Conclusions

In this paper, we introduced and explored an appropriate extension of the well-known Moreau envelope. Taking into account the nice and favorable properties of the functional $V$ in uniformly smooth and uniformly convex Banach spaces, we defined the $V$-Moreau envelope based on $V$. Within the framework of reflexive Banach spaces, we established several important properties of the $V$-Moreau envelope. Furthermore, under the additional assumptions of uniform smoothness and convexity of the space, we demonstrated the Hölder continuity of the generalized $(f, \lambda)$-projection. Several key properties of both the $V$-Moreau envelope and the generalized $(f, \lambda)$-projection were also proven.

The convex case in Theorem 3.5 presents a novel result. It is noteworthy that, even in the convex case, the best result regarding the $(f, \lambda)$-projection was shown in Theorem 3.4 of ^[17], where the authors established continuity under two strong conditions: the positive homogeneity of the function $f$ and the compactness of $S$. In contrast, our proof avoids these restrictive assumptions.

For future research, we are focusing on applying our results on the $V$-Moreau envelope and the generalized $(f, \lambda)$-projection to problems such as nonconvex variational inequalities and nonconvex complementarity problems in Banach spaces. Another potential research direction is extending our results to nonreflexive Banach spaces.

Acknowledgments

The author extends his appreciation to Researchers Supporting Project number (RSPD2024R1001), King Saud University, Riyadh, Saudi Arabia.

Conflict of interest

The author declares that he has no conflicts of interest.