Least energy sign-changing solutions for a class of fractional $(p, q)$-Laplacian problems with critical growth in $\mathbb{R}^N$

1.   Introduction and main results

In this paper, we investigate the existence of the least energy sign-changing solution for the following fractional $(p, q)$-Laplacian problem:

where $s\in (0, 1)$, $2 < p < q < \frac{N}{s}$, $\lambda > 0$. The potential $V \in C\left(\mathbb{R}^{N}, \mathbb{R}\right)$ and the operator $(-\Delta)^{s}_{t}$ with $t\in \left\{ p, q\right\}$ is the fractional Laplacian which, up to a normalizing constant, may be defined for any $u: \mathbb{R}^N \to \mathbb{R}$ smooth enough by setting

along functions $u \in C_0^{\infty} (\mathbb{R}^N)$, where $B_{\varepsilon}(x)$ denotes the ball of $\mathbb{R}^N$ centered at $x \in \mathbb{R}^N$ and radius $\varepsilon > 0.$

When $s = 1$, problem (1.1) boils down to a $(p, q)$-Laplacian problem of the following type:

As can be seen in ^[5] and ^[30], the applications in plasma physics, chemical process design, and biology have generated the majority of interest in this broad class of problems. In the last decade, many authors investigated problem (1.2), for example, Barile and Figueiredo ^[5] showed that (1.2) has a least energy sign-changing solution by using the deformation lemma and the Brouwer degree theory. For more interesting results involving $(p, q)$-Laplacian problems, we also mention ^{[9,22,24,30,32,39]} and references therein.

When $s\in (0, 1)$ and $p = q = 2$, problem (1.1) appears in the study of standing wave solutions, i.e., solutions of the form $\psi(x, t) = u(x) e^{-i \omega t}$, to the following fractional Schrödinger equation:

where $\hbar$ is the Planck constant, $W:\mathbb{R}^{N} \rightarrow \mathbb{R}$ is an external potential and $f$ is a suitable nonlinearity. Laskin ^[28,29] first introduced the fractional Schrödinger equation due to its fundamental importance in the study of particles on stochastic fields modeled by Lévy processes. After that, fractional Schrödinger equations received a lot of attention, and a lot of interesting results were obtained. We direct the curious reader to ^[33] for a basic overview of this topic for more information. For the existence, multiplicity, and behavior of standing wave solutions to Eq (1.3), we refer to ^{[10,11,14,16,21,23,36,37]} and the references therein.

When $p = q \neq 2$, problem (1.1) boils down to the following fractional Laplacian problem:

Problem (1.4) piques the interest of researchers because of its nonlocal character and the operator's nonlinearity. In ^[15], the authors obtained infinitely many sign-changing solutions of (1.4) by using descent flow with invariant sets. By applying the deformation Lemma and the Brouwer degree, they also proved that (1.4) has a least energy sign-changing solution. It is noteworthy that Wang and Zhou ^[37] used a similar method to obtain the least energy sign-changing of (1.4) with $p = 2$. In addition, for Eq (1.4), we refer to ^{[2,3,18,19,34,35]} for existence and multiplicity results, to ^[13,25] for regularity results.

However, only a few papers considered fractional $(p, q)$-Laplacian problems. For instance, the authors of ^[17] investigated the existence, nonexistence and multiplicity of solutions for a fractional $(p, q)$-Laplacian problem with subcritical growth. Alves et al ^[1] studied the following problem:

where the potential $V(x)$ satisfies the Rabinowitz conditions. By virtue of the Ljusternik-Schnirelmann theory and minimax theorems, they explored the existence, multiplicity, and concentration of nontrivial solutions provided that $\varepsilon$ is sufficiently small. Ambrosio and R$\check{a}$dulescu ^[4] considered the existence and concentration of positive solutions for (1.5) with the del Pino-Felmer type potential conditions. For the other work on (1.1) or similar problems, we refer the reader to ^{[4,20,26,40,41,42,43,44]} and the references therein.

Motivated by the above results, it is natural to ask, whether the problem (1.1) had sign-changing solutions when the nonlinear term $f$ has critical growth. To our knowledge, this question is open. In ^[23], the authors considered the following problem:

where $\Omega\subset \mathbb{R}^N$ is a bounded domain, $2^{*}_{s} = \frac{2N}{N-2s}$ and $f$ satisfies some suitable conditions. By using the constrained variational methods, they proved the least energy sign-changing solution of (1.6) when $\lambda$ sufficiently large. However, since (1.1) contains the nonlocal and nonlinear term $(-\Delta)^{s}_{p}+(-\Delta)^{s}_{q}$, the decomposition of functional $I_\lambda$ (see the definition in (1.10)) is more complicated than that in ^[23]. Therefore, some difficulties arise in studying the existence of a least energy sign-changing solution for problem (1.1), and this makes the study interesting.

To study problem (1.1), we consider the following assumptions on $V$ and $f$:

$(V_1) \; V(x) \in C\left(\mathbb{R}^{N}\right)$ and there exists $V_{0} > 0$ such that $V(x) \geq V_{0}$ in $\mathbb{R}^{N}$. Moreover, $\lim _{|x| \rightarrow \infty} V(x) = +\infty.$

$(f_1) \; \lim\limits_{|t|\rightarrow 0^+}\frac{f(t)}{|t|^{p-1}} = 0.$

$(f_2) \; f$ has a "quasicritical growth" at infinity, namely,

We suppose that the function $f$ satisfies the Ambrosetti-Rabinowitz condition:

$(f_3)$ There exists $\theta \in\left(q, q_{s}^{*}\right)$ such that

furthermore, we assume that:

$(f_4)$ The map $f$ and its derivative $f'$ satisfy

Clearly, $\left(f_{4}\right)$ implies that the map $t \mapsto \frac{f(t)}{|t|^{q-1}} \text { is strictly increasing for all }|t| > 0.$

Before starting our results, we recall some useful notations. Let $1 \leq \zeta \leq \infty$, we denote by $|u|_{\zeta}$ the $L^{\zeta}$-norm of $u: \mathbb{R}^{N} \rightarrow \mathbb{R}$ belonging to $L^{\zeta}\left(\mathbb{R}^{N}\right)$. For $0 < s < 1$, let us define $\mathcal{D}^{s, \zeta}\left(\mathbb{R}^{N}\right) = \overline{\mathcal{C}_{c}^{\infty}\left(\mathbb{R}^{N}\right)}^{[\cdot]{s, \zeta}}$, where

Let us denote by $W^{s, \zeta}\left(\mathbb{R}^{N}\right)$ the set of functions $u \in L^{\zeta}\left(\mathbb{R}^{N}\right)$ such that $[u]_{s, \zeta} < \infty$, endowed with the natural norm

According to ^[33], let $s \in(0, 1)$ and $N > s q$, there exists a sharp constant $S_{q} > 0$ such that for any $u \in \mathcal{D}^{s, q}\left(\mathbb{R}^{N}\right)$

where $q_{s}^{*} = \frac{Nq}{N-qs}$ is the Sobolev critical exponent. Moreover, $W^{s, q}\left(\mathbb{R}^{N}\right)$ is continuously embedded in $L^{\gamma}\left(\mathbb{R}^{N}\right)$ for any $\gamma \in\left[q, q_{s}^{*}\right]$ and compactly in $L^{\gamma}\left(B_{R}(0)\right)$, for all $R > 0$ and for any $\gamma \in\left[1, q_{s}^{*}\right)$.

To ensure that problem (1.1) has a variational structure, we consider the following Sobolev space:

endowed with the norm

Notice that $W^{s, r}\left(\mathbb{R}^{N}\right)$ is a separable reflexive Banach space for all $r \in(1, +\infty)$, then $X$ is also a separable reflexive Banach space. We also introduce the following Banach space

endowed with the norm

where $\|u\|_{V, t}^{t}: = [u]_{s, t}^{t}+\int_{\mathbb{R}^{N}} V(x)|u|^{t} d x$ for $t\in \{p, q\}$. For the weak solution to (1.1), we mean a function $u\in X_V$ such that

for all $\varphi \in X_V$.

Define the energy functional $I_\lambda:X_V\rightarrow \mathbb{R}$ by

By the similar arguments as in ^[1], we can deduce that $I_\lambda(u)\in \mathcal{C}^{1}(X_V, \mathbb{R})$.

For convenience, we consider the operator $\mathcal{A}_p: X_V \rightarrow X_V^{*}$ and $\mathcal{A}_q: X_V \rightarrow X_V^{*}$ given by

and

where $X_V^{*}$ is the dual space of $X_V$. In this sequel, for simplicity, we denote $\langle \cdot, \cdot \rangle_{X_V^{*}, X_V}$ by $\langle \cdot, \cdot \rangle$. Moreover, we denote the Nehari set $\mathcal{N_\lambda}$ by

Clearly, $\mathcal{N_\lambda}$ contains all the nontrivial solutions of (1.1). Denote $u^{+}(x): = \max \left\{ u(x), 0 \right\}$ and $u^{-}(x): = \min \left\{ u(x), 0 \right\}$. Then, the sign-changing solutions of (1.1) stay on the following set:

Set

and

The main results of this paper are stated in the following theorem.

Theorem 1.1. Suppose that $\left(f_{1}\right)-\left(f_{4}\right)$ are satisfied. Then there exists $\Lambda > 0$ such that for all $\lambda \geq \Lambda$, the problem (1.1) possesses a least energy sign-changing solution $u_{\lambda}$. Moreover, $c_\lambda > 2c$.

The proof of Theorem 1.1 is based on the arguments presented in ^[8]. First, we make sure that the minimum of functional $I_\lambda$ restricted on set $\mathcal{M}_\lambda$ can be achieved. Then, we demonstrate that it is a critical point of $I_\lambda$ by applying a suitable variant of the quantitative deformation Lemma. However, one cannot obtain a corresponding equivalent definition of $(-\Delta)^{s}_{t}$ by the harmonic extension approach because of the two fractional $t$-Laplacian operators $(-\Delta)^{s}_{t}$ with $s\in (0, 1)$ and $t\in \{p, q\}$ (see ^[11]). Thus, we don't get the decomposition

which is very useful to get sign-changing solutions of (1.1), see for instance ^[5,6,7,8,12]. Furthermore, we could not adapt similar methods like in ^[23,37] to conclude the set $\mathcal{M_\lambda}$ is non empty. This is because for the linear operator $(-\Delta)^{s}$, one can easily deduce that

which is important to prove that $\mathcal{M_\lambda}$ is non-empty. But, for the nonlinear operators $(-\Delta)^{s}_{p}$ and $(-\Delta)^{s}_{q}$, the above decomposition seems invalid. Fortunately, we find a new way to overcome those difficulties. We use another decomposition estimation by dividing $\mathbb{R}^{2N}$ into several regions (see Lemma 2.2) as following:

where $(\mathbb{R}^{N})^{+} = \{x\in \mathbb{R}^{N}: u(x)\geq 0 \}$ and $(\mathbb{R}^{N})^{-} = \{x\in \mathbb{R}^{N}: u(x) < 0 \}$. As we can see that it will also play an important role in proving $\deg(\Psi_1, D, 0) = 1$ (see Section 4), and then we can get the minimizer $u_\lambda$ of $c_\lambda$ (that is, $I_\lambda (u_\lambda) = c_\lambda$) is exactly a sign-changing solution of Problem (1.1). Besides, due to the critical growth of the nonlinear term, another difficulty arises in verifying the compactness of the minimizing sequence in $X_V$. Fortunately, thanks to the sharp constant $S_q$, we overcome this difficulty by choosing $\lambda$ appropriately large to ensure the compactness of the minimizing sequence. Therefore, to obtain the least energy sign-changing solutions of (1.1), a more accurate investigation and meticulous calculations are needed in our setting.

The paper is organized as follows: Section 2 contains some compactness results and the decomposition characteristics of $I_\lambda$, which will be crucial to proving the main results. In Section 3, we provide several technical lemmas. The main results are proved in Section 4 by combining the reduced arguments with a variation of the Deformation Lemma and Brouwer degree theory.

Throughout this paper, we will use the following notations: $L^{\lambda}(\mathbb{R}^N)$ denotes the usual Lebesgue space with norm $|\cdot|_{\lambda}$; $C, C_1, C_2, \cdots$ will denote different positive constants whose exact values are not essential to the exposition of arguments.

2.   Preliminaries

We provide the variational framework for the problem (1.1) in this section and provide some preliminary Lemmas. To begin with, we obtain the following compactness results by recalling the notion of fractional Sobolev space $X_V$ in (1.9).

Lemma 2.1. Suppose that $\left(V_{1}\right)$ holds, then for all $\gamma \in\left[ p, q^*_s \right]$, the embedding $X_V \hookrightarrow L^{\gamma}\left(\mathbb{R}^{N}\right)$ is continuous. For all $\gamma \in\left[ p, q^*_s \right)$, the embedding $X_V \hookrightarrow L^{\gamma}\left(\mathbb{R}^{N}\right)$ is compact.

Proof. Denote $Y = L^{\gamma}\left(\mathbb{R}^{N}\right)$ and $B_{R} = \left\{x \in \mathbb{R}^{N}:|x| < R\right\}, B_{R}^{c} = \mathbb{R}^{N} \backslash \overline{B_{R}}$. Denote $X_p: = \left\{u \in W^{s, p}\left(\mathbb{R}^{N}\right): \int_{\mathbb{R}^{N}} V(x)|u|^{p} d x < +\infty\right\}$.

For any $p \leq \gamma\leq q_{s}^{*}$, the space $X_p$ is continuously embedded in $Y$, the space $X_V$ is continuously embedded in $X_p$, so $X_V\hookrightarrow Y$ is continuous.

For any $p \leq \gamma < q_{s}^{*}$, Let $X_p(\Omega)$ and $Y(\Omega)$ be the spaces of functions $u \in X_p, u \in Y$ restricted onto $\Omega \subset \mathbb{R}^{N}$ respectively. Then, it follows from theorems $6.9, 6.10$ and $7.1$ in ^[33] that $X_p\left(B_{R}\right) \hookrightarrow Y\left(B_{R}\right)$ is compact for any $R > 0$. Denote $V_{R} = \inf _{x \in B_{R}^{c}} V(x)$. By $\left(V_{1}\right)$, we deduce that $V_{R} \rightarrow \infty$ as $R \rightarrow \infty$. Therefore, we have

which implies

By virtue of Theorem $7.9$ in ^[27], we can see that $X_p \hookrightarrow Y$ is compact, moreover, $X_V \hookrightarrow X_p$ is compact, therefore, by interpolation inequality, the embedding $X_{V} \hookrightarrow Y$ is compact for any $p \leq \gamma < q_{s}^{*}$.

Remark 2.1. It follows from Lemma 2.1 and $(f_1)$, $(f_2)$ that $I_\lambda$ is well-defined on $X_V$. Moreover, $I_\lambda \in \; C^{1}\left(X_V, \mathbb{R}^{N}\right)$ and

for all $v \in X_V$. Consequently, the critical point of $I_\lambda$ is the weak solution of the problem (1.1).

Our goal is to find the sign-changing solution to the problem (1.1). As we saw in section 1, one of the challenges is the fact that the functional $I_\lambda$ does not possess a decomposition like (1.15). Inspired by ^[15,37], we have the following:

Lemma 2.2. Let $u \in X_V$ with $u^{\pm} \neq 0$. Then,

(i) $I_\lambda(u) > I_\lambda\left(u^{+}\right)+I_\lambda\left(u^{-}\right)$,

(ii) $\langle I_\lambda^{\prime}(u), u^{\pm}\rangle > \langle I_\lambda^{\prime}\left(u^{\pm}\right), u^{\pm}\rangle$.

Proof. Observe that

By density (see Theorem $2.4$ in ^[33]), we can assume that $u$ is continuous. Defining

Then for $u \in X_V$ with $u^{\pm} \neq 0$, by a straightforward computation, one can see that

and

Similarly, we also have

Taking into account (2.3)–(2.5), we deduce that $I_\lambda(u) > I_\lambda(u^+)+I_\lambda(u^-)$. Analogously, one can prove $(ii)$.

The following Brézis-Lieb type Lemma will be very useful in this work, its proof is similar to Lemma 2.8 in ^[1] and we omit it here.

Lemma 2.3. Let $\left\{u_{n}\right\} \subset X_V$ be a sequence such that $u_{n} \rightharpoonup u$ in $X_V$. Set $v_{n} = u_{n}-u$, then we have:

$(i)\ \left[v_{n}\right]_{s, p}^{p}+\left[v_{n}\right]_{s, q}^{q} = \left(\left[u_{n}\right]_{s, p}^{p}+\left[u_{n}\right]_{s, q}^{q}\right)-\left([u]_{s, p}^{p}+[u]_{s, q}^{q}\right)+o_{n}(1),$

$(ii)\ \int_{\mathbb{R}^{N}} V(x)\left(\left|v_{n}\right|^{p}+\left|v_{n}\right|^{q}\right) d x = \int_{\mathbb{R}^{N}} V(x)\left(\left|u_{n}\right|^{p}+\left|u_{n}\right|^{q}\right) d x-\int_{\mathbb{R}^{N}} V(x)\left(|u|^{p}+|u|^{q}\right) d x+o_{n}(1),$

$(iii)\ \int_{\mathbb{R}^{N}}\left(F\left(v_{n}\right)-F\left(u_{n}\right)+F(u)\right) d x = o_{n}(1),$

$(iv)\ \sup\limits_{\|w\| \leq 1} \int_{\mathbb{R}^{N}}\left|\left(f\left(v_{n}\right)-f\left(u_{n}\right)+f(u)\right)w\right| d x = o_{n}(1).$

3.   Some technical lemmas

The purpose of this section is to prove some technical lemmas related to the existence of a least energy sign-changing solution. Firstly, we collect some preliminary lemmas which will be fundamental to prove our main results.

Now, fixed $u \in X_V$ with $u^{\pm} \neq 0$, we define function $\psi_{u}:[0, \infty) \times[0, \infty) \rightarrow \mathbb{R}$ and mapping $T_{u}:[0, \infty) \times[0, \infty) \rightarrow \mathbb{R}^{2}$ by

and

Lemma 3.1. For any $u\in X_V$ with $u^{\pm}\neq0,$ there exists a unique maximum point pair $(\tau_u, \sigma_u)$ of the function $\psi_u$ such that $\tau_u u^++\sigma_u u^-\in {\mathcal M}_\lambda$.

Proof. Our proof will be divided into three steps.

Step 1: For any $u\in X_V$ with $u^{\pm}\neq0,$ in the following, we will prove the existence of $\sigma_u$ and $\tau_u$. Form $(f_1)$, $(f_2)$ and Lemma 2.2 we deduce that

Similarly, we have that

Choose $\varepsilon > 0$ such that $\left(1-\lambda C\varepsilon \right) > 0$. Since $p < q < q^*_s$, there exists $r > 0$ small enough such that

and

On the other hand, by $(f_3)$, there exist $D_1, D_2 > 0$ such that

Then we have

where $A^{+} \subset \operatorname{supp}\left(u^{+}\right)$ is measurable set with finite and positive measure $\left|A^{+}\right|$. Due to the fact $\theta > p$, for $R$ sufficiently large, we get

Similarly, we get

Hence, by virtue of Miranda's Theorem ^[31], and taking (3.3), (3.4), (3.6) and (3.7) into account, we can see that there exists $\left(\sigma_{u}, \tau_{u}\right) \in[r, R] \times[r, R]$ such that $T_{u}(\sigma, \tau) = (0, 0)$, i.e., $\sigma_{u} u^{+}+\tau_{u} u^{-} \in \mathcal{M}_\lambda$.

Step 2: Now we prove the uniqueness of the pair $\left(\sigma_{u}, \tau_{u}\right)$.

Case 1: $u \in \mathcal{M}_\lambda$.

If $u \in \mathcal{M}_\lambda$, we have that

and

We will show that $\left(\sigma_{u}, \tau_{u}\right) = (1, 1)$ is the unique pair of numbers such that $\sigma_{u} u^{+}+\tau_{u} u^{-} \in \mathcal{M}_\lambda$. Let $\left(\sigma_{u}, \tau_{u}\right)$ be a pair of numbers such that $\sigma_{u} u^{+}+\tau_{u} u^{-} \in \mathcal{M}_\lambda$ with $0 < \sigma_{u} \leq \tau_{u}$, then one can see

and

Since $0 < \sigma_{u} \leq \tau_{u}$, it follows from (3.11) that

If ${\tau}_u > 1$, by (3.9) and (3.12), we get

The left side of the above inequality is negative, which is absurd because the right side is positive. Therefore, we conclude that $0 < \sigma_{u} \leq \tau_{u} \leq 1$.

Similarly, by (3.10) and $0 < \sigma_{u} \leq \tau_{u}$, we have that

This fact implies that $\sigma_{u} \geq 1$. Consequently, $\sigma_{u} = \tau_{u} = 1$.

Case 2: $u\notin {\mathcal M}_\lambda.$

Suppose that there exist $(\widetilde {\sigma}_1, \widetilde{\tau}_1), \; (\widetilde{\sigma}_2, \widetilde{\tau}_2)$ such that

Hence,

Since $u_1\in {\mathcal M}_\lambda$, we deduce from case 1 that

which implies $\widetilde{\sigma}_1 = \widetilde{\sigma}_2, \; \widetilde{\tau}_1 = \widetilde{\tau}_2$.

Step 3: We assert that $\left(\sigma_{u}, \tau_{u}\right)$ is the unique maximum point of $\psi_{u}$ on $[0, +\infty) \times[0, +\infty).$ In fact, by $(f_3)$ we can see that

which implies that $\lim _{|\sigma, \tau| \rightarrow \infty} \phi_{u}(\sigma, \tau) = -\infty$ due to $q_{s}^{*} > q$. Noticing that $\sigma_{u}u^{+} + \tau_{u}u^{-} \in \mathcal{M_{\lambda}}$, we conclude that $\left(\sigma_{u}, \tau_{u}\right)$ is the unique critical point of $\psi_{u}$ in $(0, +\infty) \times(0, +\infty)$. Hence, it is sufficient to check that a maximum point cannot be achieved on the boundary of $[0, +\infty) \times[0, +\infty)$. By contradiction, we assume that $\left(0, \tau_{1}\right)$ is a maximum point of $\psi_{u}$ with $\tau_{1} \geq 0$. Then, arguing as Lemma 2.2, we have

On the other hand, by the growth condition $(f_1)$ and $(f_2)$, one can easily check that $\psi_u\left(\sigma, 0\right) > 0$ for $\sigma$ sufficiently small. Combining this with (3.13), we see that

if $\sigma$ is small enough, which yields a contradiction. Similarly, $\psi_u$ can not achieve its global maximum point at $\left(\sigma_{1}, 0\right)$, where $\sigma_1\ge 0$. As a result, we complete the proof of Lemma 3.1.

Lemma 3.2. For any $u\in X_V$ with $u^{\pm}\neq0,$ such that $\langle I_\lambda^{\prime}(u), u^{\pm}\rangle\leq0$, the unique maximum point of $\psi_u$ in $[0, +\infty)\times[0, +\infty)$ satisfies $0 < \sigma_u, \tau_u\leq1$.

Proof. If $\sigma_u = 0$ or $\tau_u = 0$, according Lemma 3.1, $\psi_u$ can not achieve maximum. Without loss of generality, we assume ${\sigma_u}\geq \tau_u > 0$. Since $\; \sigma_u u^+ +\tau_u u^- \in {\mathcal M}_\lambda$, there holds

On the other hand, by $\langle I_\lambda^{\prime}(u), u^{+}\rangle\leq0$, we have

Then it follows (3.14) and (3.15) that

In view of $\left(f_{4}\right)$, we conclude that $\sigma_{u} \leq 1$. Thus, we have that $0 < \sigma_{u}, \tau_{u} \leq 1$.

Lemma 3.3. There exists $\rho > 0$ such that $\left\|u^{\pm}\right\| \geq \rho$ for all $u \in \mathcal{M}_{\lambda}$.

Proof. For any $u\in {\mathcal M}_\lambda$, by $(f_1), (f_2)$ and the Sobolev inequalities, we have that

Thus we get

where $C_{0}^{\prime} = (1-\lambda\varepsilon C_1)$, $\widetilde{C}_{2} = (C_3+\lambda C_2 C_\varepsilon)$ with $C$ is a Sobolev embedding constant. If $0 < \|u\| < 1$, then $\|u\|_{V, p}, \|u\|_{V, q} < 1$ and by order relations between $p$ and $q$ and by (3.17) we have

where $C^{\prime} = \min \left\{C_{0}^{\prime}, 1\right\}$ and $C^{\prime \prime} = \frac{C^{\prime}}{2^{q-1}}$. Hence, there exists a positive radius $\rho_{1} > 0$ such that $\|u\| \geq \rho_{1}$ with $\rho_{1} = \left(\frac{C^{\prime \prime}}{\widetilde{C}_{\varepsilon}}\right)^{\frac{1}{q^{*}-q}}$. Clearly we can reason analogously if $\|u\| \geq 1$ so that for some $\rho > 0$ and for every $u \in \mathcal{M}_\lambda$, we get $\rho \leq\|u\|$.

Lemma 3.4. Let $c_\lambda = \inf_{u\in {\mathcal M}_\lambda}I_\lambda(u),$ then we have that $\lim_{\lambda\rightarrow \infty}c_\lambda = 0$.

Proof. Since $u \in \mathcal{M}_\lambda$, we have $\langle I_\lambda^{\prime}(u), u\rangle = 0$ and then

thus $I_\lambda$ is bounded below on $\mathcal{M}_\lambda$, which implies $c_\lambda$ is well-defined.

For any $u \in X_V$ with $u^{\pm} \neq 0$, by Lemma 3.1, for each $\lambda > 0$, there exists $\sigma_{\lambda}, \tau_{\lambda}$ such that $\sigma_{\lambda} u^{+}+\tau_{\lambda} u^{-} \in \mathcal{M}_\lambda$, we have

Next, we will prove that $\sigma_{\lambda} \rightarrow 0$ and $\tau_{\lambda} \rightarrow 0$ as $\lambda \rightarrow \infty$.

Let $Q_{u} = \left\{\left(\sigma_{\lambda}, \tau_{\lambda}\right) \in[0, +\infty) \times[0, +\infty): T_{u}\left(\sigma_{\lambda}, \tau_{\lambda}\right) = (0, 0), \lambda > 0\right\}$. Due to $\sigma_{\lambda} u^{+}+\tau_{\lambda} u^{-} \in \mathcal{M}_\lambda$, there holds

Therefore, $Q_{u}$ is bounded in $\mathbb{R}^{2}$. Let $\left\{\lambda_{n}\right\} \subset(0, \infty)$ be such that $\lambda_{n} \rightarrow \infty$ as $n \rightarrow \infty$. Then there exist $\sigma_{0}$ and $\tau_{0}$ such that $\left(\sigma_{\lambda_{n}}, \tau_{\lambda_{n}}\right) \rightarrow \left(\sigma_{0}, \tau_{0}\right)$ as $n \rightarrow \infty$.

Now, we claim $\sigma_{0} = \tau_{0} = 0$. By contradiction, suppose that $\sigma_{0} > 0$ or $\tau_{0} > 0$ by $\sigma_{\lambda_{n}} u^{+}+\tau_{\lambda_{n}} u^{-} \in \mathcal{M}_{\lambda_{n}}$, then for any $n \in \mathbb{N}$, there holds

Thanks to $\sigma_{\lambda_{n}} u^{+} \rightarrow \sigma_{0} u^{+}$ and $\tau_{\lambda_{n}} u^{-} \rightarrow \tau_{0} u^{-}$ in $X_V, (f_1), (f_2)$ and the Lebesgue dominated convergence theorem, we deduce that

as $n \rightarrow \infty$. It follows from $\lambda_{n} \rightarrow \infty$ and (3.20) that the right hand side of (3.19) tends to infty, which contradict with the boundness of $\left\{\sigma_{\lambda_{n}} u^{+}+\tau_{\lambda_{n}} u^{-}\right\}$ in $X_V$. Hence, $\sigma_{0} = \tau_{0} = 0$. As a result, we conclude that $\lim _{\lambda \rightarrow \infty} c_{\lambda} = 0$.

Lemma 3.5. There exists $\lambda^{*} > 0$ such that for all $\lambda \geq \lambda^{*}$, the infimum $c_{\lambda}$ is achieved.

Proof. By the definition of $c_{\lambda} = \inf _{u \in \mathcal{M}_{\lambda}} I_\lambda(u)$, there exists a sequence $\left\{u_{n}\right\} \subset \mathcal{M}_{\lambda}$ such that

Obviously, $\left\{u_{n}\right\}$ is bounded in $X_V$. Up to a subsequence, still denoted by $\left\{u_{n}\right\}$, there exists $u \in X_V$ such that $u_{n}\rightharpoonup u$ weakly in $X_V$. Since the embedding $X_V \hookrightarrow L^{r}(\mathbb{R}^{N})$ is compact for all $r \in [p, q_{s}^{*})$, we have $u_{n}^{\pm} \rightarrow u^{\pm}$ in $L^{r}\left(\mathbb{R}^{N}\right)$ for all $r \in [p, q_{s}^{*})$, $u_{n}^{\pm}(x) \rightarrow u^{\pm}(x)$ a.e. $x \in \mathbb{R}^{N}$.

Denote $\delta: = \frac{s}{N} S_q^{\frac{N}{sq}}$, according to Lemma 3.4, there is $\lambda^{\star} > 0$ such that $c_{\lambda} < \delta$ for all $\lambda \geq \lambda^{\star}$. Fix $\lambda \geq \lambda^{\star}$, it follows from Lemma 3.1 that $I_{\lambda}\left(\sigma u_{n}^{+}+\tau u_{n}^{-}\right) \leq I_{\lambda}\left(u_{n}\right)$ for all $\sigma, \tau \geq 0$. Then by using Brézis-Lieb type Lemma 2.3 and the Fatou's Lemma, it follows that

where

Hence, we can see that for all $\sigma\geq 0$ and $\tau\geq 0$, there holds

Now we divide the proof into three steps.

Step 1: We prove that $u^\pm\neq0$. Here we only prove $u^+\neq 0$ since $u^- = 0$ is similar, by contradiction, we suppose $u^+ = 0$. Then we have the following two cases.

Case 1: $B_1 = 0.$ If $A_1 = A_3 = 0$, that is, $u^+_n\rightarrow {u}^+$ in $X_V$. According to Lemma 3.3, we obtain $\|u^+\| > 0$, which contradicts $u^+ = 0$. If $A_1$ or $A_3 > 0$, By (3.21) we get $\frac{1}{p}\sigma^p A_1+\frac{\sigma^q}{q}A_3 < c_\lambda$ for all $\sigma\geq0$, which is a contradiction.

Case 2: $B_1 > 0$. According to definition of $S_q$, we have that $\delta: = \frac{s}{N}S_q^{\frac{N}{sq}}\leq \frac{s}{N}(\frac{A_3}{(B_1)^{\frac{q}{q^*_s}}})^{\frac{N}{sq}}$, by direct calculation, we have that

Since $c_\lambda \rightarrow 0$ as $\lambda\rightarrow \infty,$ there exists $\lambda^* > 0$ such that for all $\lambda > \lambda^*, c_\lambda\leq \delta$. Then, without loss of generality, we can assume $c_\lambda < \delta$. Choosing $\tau = 0,$ by (3.21) it follows that

which is impossible. From the above discussion, we have that $u^+\neq0$. Similarly, we obtain $u^-\neq0$.

Step 2: we prove that $B_1 = 0, \; B_2 = 0$. We just prove $B_1 = 0$ (the proof of $B_2 = 0$ is analogous). By contradiction, we suppose that $B_1 > 0$.

Case 1: $B_2 > 0$, Let $\widehat{\sigma}_1$ and ${\widehat{\tau}_1}$ satisfy

and

According to $\left[0, \widehat{\sigma}_{1}\right] \times\left[0, \widehat{\tau}_{1}\right]$ is compact, there exists $\left(\sigma_{u}, \tau_{u}\right) \in\left[0, \widehat{\sigma}_{1}\right] \times\left[0, \widehat{\tau}_{1}\right]$ such that $\psi_{u}\left(\sigma_{u}, \tau_{u}\right) = \max _{(\sigma, \tau) \in\left[0, \widehat{\sigma}_{1}\right] \times\left[0, \widehat{\tau}_{1}\right]} \psi_{u}(\sigma, \tau)$.

In the following, we prove that $\left(\sigma_{u}, \tau_{u}\right) \in\left(0, \widehat{\sigma}_{1}\right) \times\left(0, \widehat{\tau_{1}}\right)$. Obviously, if $\tau$ is small enough, we have

Hence, there exists $\tau_0$ such that $\psi_{u}(\sigma, 0)\leq \psi_{u}(\sigma, \tau_0)$, for all $\sigma \in\left[0, \widehat{\sigma}_{1}\right]$. That is, $\left(\sigma_{u}, \tau_{u}\right) \notin [0, \widehat{\sigma}_{1}]\times\left\{0\right\}$. Similarly, one can prove that $\left(\sigma_{u}, \tau_{u}\right) \notin \left\{ 0\right\}\times [0, \widehat{\tau}_{1}]$.

On the other hand, we can easily deduce that

and

Then, for all $\sigma\in (0, {\widehat{\sigma}_1}]$ and $\tau\in (0, {\widehat{\tau}_1}]$, we get

Together with (3.21), we obtain $\psi_u(\sigma, {\widehat{\tau}_1})\leq 0, \; \psi_u({\widehat{\sigma}_1}, \tau)\leq 0$, for all $\sigma\in [0, {\widehat{\sigma}_1}]$ and $\tau\in [0, {\widehat{\tau}_1}]$, which is absurd. Therefore, $(\sigma_u, \tau_u)\notin[0, {\widehat{\sigma}_1}]\times\{{\widehat{\tau}_1}\}$ and $(\sigma_u, \tau_u)\notin\{0, {\widehat{\sigma}_1}\}\times[0, {\widehat{\tau}_1}]$.

In conclusion, we get $(\sigma_u, \tau_u)\in (0, {\widehat{\sigma}_1})\times(0, {\widehat{\tau}_1})$. Hence, $\sigma_u u^++\tau_u u^-\in {\mathcal M}_\lambda$. So, combining (3.21), (3.22) with (3.23), we have that

Therefore, we have a contradiction.

Case 2: $B_2 = 0$. In this case, we can maximize in $[0, {\widehat{\sigma}_1}]\times[0, \infty)$. Indeed, it is possible to show that there exists ${\widehat{\tau}_0}\in [0, \infty]$ such that $I_\lambda(\sigma u^++\tau u^-) < 0$ for all $(\sigma, \tau)\in [0, {\widehat{\sigma}_1}]\times [{\widehat{\tau}_0}, \infty)$. Hence, there exists $(\sigma_u, \tau_u)\in [0, {\widehat{\sigma}_1}]\times [0, \infty)$ that satisfies $\psi_u(\sigma_u, \tau_u) = \max\limits_{\sigma\in[0, {\widehat{\sigma}_1}]\times[0, \infty)}\psi_u(\sigma, \tau).$

Following, we prove that $(\sigma_u, \tau_u)\in (0, {\widehat{\sigma}_1})\times (0, \infty)$.

Indeed, since $\psi_u(\sigma, 0)\leq\psi_u(\sigma, \tau)$ for $\sigma\in [0, {\widehat{\sigma}_1}]$ and $\tau$ is small enough, we have $(\sigma_u, \tau_u)\notin[0, \widehat{\sigma}_1]\times\{0\}$. Analogously, we have $(\sigma_u, \tau_u)\notin \{0\} \times [0, \infty)$. On the other hand, for all $\tau\in [0, \infty)$, it is obvious that

Hence, we have that $\psi_u({\widehat{\sigma}_1}, \tau)\leq0$ for all $\tau\in [0, \infty)$, Thus, $(\sigma_u, \tau_u)\notin\{{\widehat{\sigma}_1}\}\times[0, \infty)$. In summary, we have $(\sigma_u, \tau_u)\in (0, {\widehat{\sigma}_1})\times (0, \infty)$, namely, $\sigma_u u^++\tau_u u^-\in {\mathcal M}_\lambda$. Therefore, according to (3.22), we have that

which is a contradiction.

Therefore, from the above discussion, we deduce that $B_1 = B_2 = 0$.

Step 3: we prove that $c_\lambda$ is achieved. Since $u^\pm\neq0,$ by Lemma 3.1, there exist $\sigma_u, \tau_u > 0$ such that

Furthermore, $B_1 = B_2 = 0$ and Fatou's Lemma implies $\langle I_\lambda^{\prime}(u), u^\pm\rangle \leq 0.$ By Lemma 3.2, we obtain $\sigma_u, \tau_u\leq 1$. Since $u_n\in {\mathcal M}_\lambda$, then according to Lemma 3.1 there holds

Due to $\sigma_u, \tau_u\leq 1$, arguing as Lemma 2.2, one has $\|\sigma_u u^+ +\tau_u u^-\|^{p}_{V, p}\le \|u\|^{p}_{V, p}$. Then by $(f_4)$, Fatou's Lemma and a straightforward calculation, we deduce that

Therefore, $\sigma_u = \tau_u = 1$, and $c_\lambda$ is achieved by $u_\lambda: = u^++u^-\in {\mathcal M}_\lambda$. This ends the proof of Lemma 3.5.

4.   Proof of Theorem 1.1

Proof of Theorem 1.1. Since ${u_\lambda}\in {\mathcal M}_\lambda$, we have $\langle I_\lambda^{\prime}(u_\lambda), u^+_\lambda\rangle = \langle I_\lambda^{\prime}(u_\lambda), u^-_\lambda\rangle = 0$. By Lemma 3.5, for $(\sigma, \tau)\in (\mathbb{R}^+ \times \mathbb{R}^+)\setminus(1, 1),$ we have

Now we prove $u_\lambda$ is a solution of (1.1). Arguing by contradiction, we assume that $I_\lambda^{\prime}(u_\lambda)\neq 0$, then there exists $\delta > 0$ and $\kappa > 0$ such that

Define $D: = \left[1-\delta_1, 1+\delta_1\right] \times\left[1-\delta_1, 1+\delta_1\right]$ and a map $g: D \rightarrow X_V$ by

where $\delta_1\in (0, \frac12)$ small enough such that $\left\|g (\sigma, \tau)-w \right\|\leq 3\delta$ for all $(\sigma, \tau)\in \bar{D}.$ Thus, by virtue of Lemma 3.5, we can see that

Therefore,

By using [38, Theorem 2.3] with

and $c: = c_\lambda$. Then, choosing $\varepsilon: = \min \left\{\frac{c_\lambda-\beta}{4}, \frac{\kappa \delta}{8}\right\}$, we deduce that there exists a deformation $\eta \in C([0, 1] \times X_V, X_V)$ such that:

(i) $\eta(t, v) = v$ if $v \notin I^{-1}\left(\left[c_\lambda-2 \varepsilon, c_\lambda+2 \varepsilon\right]\right)$;

(ii) $I_\lambda(\eta(1, v)) \leq c_\lambda-\varepsilon$ for each $v \in X_V$ with $\|v-u\| \leq \delta$ and $I_\lambda(v) \leq c_\lambda+\varepsilon$;

(iii) $I_\lambda(\eta(1, v)) \leq I_\lambda(v)$ for all $u \in X_V$.

By $(ii)$ and $(iii)$ we conclude that

Therefore, to complete the proof of this Lemma, it suffices to prove that

Indeed, if (4.3) holds true, then by the definition of $c_\lambda$ and (4.2), we get a contradiction.

In the following, we will prove (4.3). To this end, for $(\sigma, \tau)\in \overline{D}$, let $\gamma(\sigma, \tau): = \eta(1, g(\sigma, \tau))$ and

and

Firstly, let us denote

Clearly, $C_p = D_p > 0$, $C_q = D_q > 0$, $A_p, A_q, B_p, B_q > 0$ and notice that $u_\lambda\in\mathcal{M}_\lambda$, we can see that

Moreover, $(f_4)$ guarantees

Then by direct computation, we have

and

Let

So we have

Since $\Psi_0(\alpha, \beta)$ is a $C^1$ function and (1, 1) is the unique isolated zero point of $\Psi_0$, by using the degree theory, we deduce that ${\rm deg}(\Psi_0, D, 0) = 1$. Furthermore, combining (4.2) and $(a)$, we get

Consequently, we deduce that $\deg(\Psi_1, D, 0) = 1$. Therefore, $\Psi_1(\sigma_0, \tau_0) = 0$ for some $(\sigma_0, \tau_0)\in D$ so that

which is contradicted to (4.2). From the above discussions, we deduce that $u_\lambda$ is a sign-changing solution for the problem (1.1).

Next, we prove that the energy of $u_b$ is strictly larger than two times the ground state energy.

Similar to proof of Lemma 3.1, there exists $\lambda^*_1 > 0$ such that for all $\lambda\geq\lambda^*_1 > 0$, there exists $v\in\mathcal{N_\lambda}$ such that $I_\lambda(v) = c^* > 0.$ By standard arguments, the critical points of the functional $I_\lambda$ on $\mathcal{N_\lambda}$ are critical points of $I_\lambda$ in $X_V,$ we obtain $\langle I_\lambda^{\prime}(v), v\rangle = 0$, that is, $v$ is a ground state solution of (1.1).

According to Theorem 1.1, we know that the problem (1.1) has the least energy sign-changing solution $u_b$ when $\lambda\geq\lambda^*$. Denote $\Lambda: = \max\{\lambda^*, \lambda^*_1\}.$ As Proof of Lemma 3.5, there exist $\sigma_{u^+_\lambda} > 0$ and $\tau_{u^-_\lambda} > 0$ such that

Furthermore, Lemma 3.2 implies that $\sigma_{u^+_\lambda}, \tau_{u^-_\lambda}\in (0, 1)$.

Therefore, in view of Lemma 3.1, we have that

The proof is complete.

5.   Conclusions

This paper considers the least energy sign-changing solution for a class of fractional $(p, q)$-Laplacian problems with critical growth in $\mathbb{R}^N$. We use constrained variational methods, quantitative deformation lemma and Brouwer degree theory to prove that the above problem has a least energy sign-changing solution $u_{\lambda}$ if $\lambda$ is large enough. Moreover, we show that the energy of $u_{\lambda}$ is strictly larger than two times the ground state energy.

Acknowledgments

K. Cheng was supported by the Natural Science Foundation program of Jiangxi Provincial (20202BABL211005), L. Wang was supported by the National Natural Science Foundation of China (No. 12161038) and the Science and Technology project of Jiangxi provincial Department of Education (Grant No. GJJ212204), Y. Zhan was the Science and Technology project of Jiangxi provincial Department of Education (Grant No. GJJ211346).

Conflict of interest

The authors declare that there is no conflict of interest.