Existence of solutions to a generalized quasilinear Schrödinger equation with concave-convex nonlinearities and potentials vanishing at infinity

1.   Introduction and main results

This article is concerned with a class of generalized quasilinear Schrödinger equations

where $N\geq3$, $\lambda > 0$, $f, h$: $\mathbb{R}\rightarrow \mathbb{R}$ and $V, K, W$: $\mathbb{R}^{N}\rightarrow \mathbb{R}$ are nonnegative continuous and $g(s)\in C^1(\mathbb{R}, \mathbb{R}^+)$, which is nondecreasing with respect to $|s|$.

These equations are related to the existence of solitary waves for the Schrödinger equation

where $z$: $\mathbb{R}\times\mathbb{R}^N\rightarrow \mathbb{C}$, $V$: $\mathbb{R}^N\rightarrow \mathbb{R}$ is a given potential, $l$: $\mathbb{R}\rightarrow \mathbb{R}$ and $k$: $\mathbb{R}^N\times\mathbb{C}\rightarrow \mathbb{R}$ are fixed functions. Quasilinear equations of the form (1.2) appear naturally in mathematical physics and have been derived as models of several physical phenomena corresponding to various types of $l$. For instance, the case $l(s) = s$ appears in the superfluid film equation in plasma physics ^[18]. If $l(s) = \sqrt{1+s}$, the equation models the propagation of a high-irradiance laser in a plasma, as well as the self-channeling of a high-power ultrashort laser in matter^[19]. For more physical motivations and more references dealing with various applications, we refer to ^{[5,16,17,26,28]}.

If we set $z(t, x) = e^{-iEt}u(x)$ in (1.2), we obtain the corresponding equation of elliptic type

Notice that if we let

we have the following equation

One of the most interesting cases is that $g(s) = \sqrt{1+2s^2}$, and then (1.4) changes to

The Schrödinger equation is quasilinear as the term $[\Delta(u)^2]u$ is linear about the second derivatives. Over the past decades, many interesting results about the existence of solutions to (1.5) have been established. It is difficult to give a complete reference, so we only refer to some early works ^[23,24] for special $k(x, u)u$ and some papers ^{[1,6,9,13,22,35]} closely related to our paper. Particularly, Wang and Yao ^[36] studied the existence of nontrivial solutions to (1.5) with concave-convex nonlinearities $\mu |u|^{\hat{p}-2}u+|u|^{\hat{q}-2}u$, $2 < \hat{p} < 4$, $4 < \hat{q} < 22^*$, and the potential $V(x)$ satisfied the following conditions:

$(V'_1)$ $V\in C(\mathbb{R}^N, \mathbb{R})$ and $0 < V_0\leq\inf\limits_{x\in\mathbb{R}^N}V(x)$;

$(V'_2)$ There exists $V_1 > 0$ such that $V(x) = V(|x|)\leq V_1$ for all $x\in\mathbb{R}^N$;

$(V'_3)$ $\nabla V(x)x\leq 0$ for all $x\in\mathbb{R}^N$.

In this paper we investigate the more general Eq (1.4) where the nonlinearity is like $\mu W(x)|u|^{\hat{p}-2}u+K(x)|u|^{\hat{q}-2}u$, $1 < \hat{p} < 2$, $4 < \hat{q} < 22^*$ and $V, K, W$: $\mathbb{R}^{N}\rightarrow \mathbb{R}$ satisfy some conditions listed below. There are also many works on the equation in the recent years, but we only mention those closely related to our paper, ^{[7,8,10,29,30]} and the references therein. Particularly, Furtado et al. ^[14] investigated solutions to (1.4) with a huge class of functions $g$ satisfying the following condition $(g_0)$.

$(g_0)$ $g\in C^1(\mathbb{R}, (0, +\infty))$ is even, non-decreasing in $[0, +\infty)$, $g(0) = 1$ and satisfies

and

When $g$ satisfies $(g_{0})$, the existence of solutions to (1.4) has been investigated by several authors over the past years ^[15,27] and the references therein. In particular, in ^[25] the authors considered the positive solutions to it when the nonlinearity is like $\mu |u|^{\hat{p}-2}u+|u|^{\hat{q}-2}u$, $1 < \hat{p} < 2$, $4 < \hat{q} < 22^*$ where the potential $V(x)$ satisfied $(V'_{1})$ and the following condition:

$(V'_4)$ $[V(x)]^{-1}\in L^1(\mathbb{R}^N)$.

An important class of problems associated to (1.1) is the case when $V(x)$ vanishes at infinity

which has been extensively investigated for the corresponding second order nonlinear Schrödinger equations after the researches of e.g., ^[2,3]. See also ^{[11,21,32,33,34]} for some work about $V(x)$ vanishing at infinity. However, there are only few works in this case for the more general Eq (1.1). Motivated by the above articles, we investigate the existence of solutions to (1.1) when the potential $V$ vanishes at infinity for a huge class of $g$ (satisfying $(g_{0})$).

In this paper, we consider the generalized quasilinear Schrödinger Eq (1.1) with vanishing potentials and concave-convex nonlinearity $K(x)f(u)+\lambda W(x)h(u)$. Since the problem is set on the whole space $\mathbb{R}^N$, we have to deal with the loss of compactness. In this respect we use the class of functions $V, K$ introduced in ^[2] for second order Schrödinger equations, which is more general than those in ^[3].

As in ^[2], it is said that $(V, K)\in\mathcal{K}$ if the following conditions hold:

(I) $K(x), \ V(x) > 0, \ \forall x\in\mathbb{R}^N$ and $K\in L^\infty(\mathbb{R}^N)$.

(II) If $\{A_n\}\subset\mathbb{R}^N$ is a sequence of Borel sets, such that $|A_n|\leq R$ for some $R > 0$ and for all $n\in\mathbb{N}$, then

(III) One of the below conditions satisfies:

or there is $\sigma\in(2, 2^*)$ such that

We also use the following conditions on $V$ and $W$:

$(V_1)$ $V(x)\in L^\infty(\mathbb{R}^N)$;

$(W_0)$ $W(x) > 0$ for all $x\in\mathbb{R}^N$;

$(W_1)$ $W(x)\in L^{1}(\mathbb{R}^{N})\cap L^{\infty}(\mathbb{R}^{N})$;

$(W_2)$ $\frac{W(x)}{V(x)}\in L^{\infty}(\mathbb{R}^{N})$.

We impose the following conditions on $h$ and $f$:

$(H_0)$ $h\in C(\mathbb{R}, \mathbb{R}^+)$ and $h(t) = 0$ for all $t\leq 0$;

$(H_1)$ There exists $b_1, \ b_2 > 0$ such that $h(t)\leq b_1|t|^{\tau_1-1}+b_2|t|^{\tau_2-1}$, $\tau_1, \tau_2\in (1, 2)$ for any $t\in\mathbb{R}$;

$(F_0)$ $f\in C(\mathbb{R}, \mathbb{R}^+)$ and $f(t) = 0$ for all $t\leq 0$;

$(F_1)$ $\lim\limits_{|t|\rightarrow +\infty}\frac{f(t)}{|t|^{22^*-1}} = 0$;

$(F_2)$ $\lim\limits_{|t|\rightarrow 0}\frac{f(t)}{|t|} = 0$ if ($K_2$) holds or $\lim\limits_{|t|\rightarrow 0}\frac{f(t)}{|t|^{\sigma-1}} = 0$ if ($K_3$) holds;

$(F_3)$ $\frac{F(t)}{t^{4}}\rightarrow +\infty$, as $t\rightarrow +\infty$;

$(F_4)$ There exists $\mu > 2+2\beta$ such that $\frac{1}{\mu}f(t)t\geq F(t)$, where $\beta$ is in (1.7).

Observe that there are many natural functions $f(t), h(t)$ satisfying the above conditions. For example, $f(t) = |t|^{2^*+1}$ and $h(t) = |t|^{\frac{1}{2}}$ may serve as examples satisfying $(F_{1})$–$(F_{4})$ and $(H_{1})$, respectively.

Our main theorem is stated as follows.

Theorem 1.1. Assume that $(V, K)\in\mathcal{K},$ $(g_{0}),$ $(V_{1}),$ $(W_{0})$–$(W_{2}),$ $(F_{0})$–$(F_{4}),$ $(H_{0})$ and $(H_{1})$ hold. Then, there exists $\lambda_0 > 0$ such that (1.1) possesses a positive solution for any $\lambda\in(0, \lambda_0)$.

Furthermore, for the case where ($K_2$) holds, we can prove that (1.1) possesses a ground state solution. To this end, we assume the following conditions on $h$ and $f$:

$(H_0')$ $h\in C(\mathbb{R}, \mathbb{R})$, $h(t)$ is odd and $h(t)\geq0$ for all $t\geq 0$.

$(H_1')$ There exists $b_3 > 0$ and $\tau_3\in(1, 2)$ such that $h(t)\leq b_3|t|^{\tau_3-1}$.

$(H_2')$ There exists a constant $\tilde{C} > 0$ such that $\lim\limits_{t\rightarrow0}\frac{H(t)}{|t|^{\tau_3}} = \tilde{C}$.

$(F_0')$ $f\in C(\mathbb{R}, \mathbb{R})$, $f(t)$ is odd and $f(t)\geq0$ for all $t\geq 0$.

Proposition 1.2. Assume that $(V, K)\in\mathcal{K}$ where ($K_2$) holds and $(g_{0}),$ $(V_{1}),$ $(W_{0})$–$(W_{2}),$ $(F'_{0}),$ $(F_{1}),$ $(F_{2}),$ $(F_{4}),$ $(H'_{0})$–$(H'_{2})$ hold. Then, there exists $\lambda_1 > 0$ such that (1.1) possesses a ground state solution for any $\lambda\in(0, \lambda_1)$.

We emphasize that the main result in this paper is essentially different from the aforementioned works. Indeed, in ^[25,36] the authors considered two kinds of quasilinear Schrödinger equations with concave-convex nonlinearities, but required that the potential $V(x)$ have a positive lower bound. In ^[11,21] the authors showed the existence of nontrivial solutions for different problems with vanishing potentials. In this paper, we investigate a different class of generalized quasilinear Schrödinger equations with vanishing potentials and concave-convex nonlinearities. As far as we know, few works in this case seem to have appeared in the literature.

The paper is organized as follows. In Section 2, some preliminary results are presented. In Section 3, we verify that the functional associated to the problem satisfies the geometric conditions of the mountain pass theorem, and the boundedness of the Cerami sequences associated with the corresponding minimax level is proved. Lastly, in Section 4, the existence of a positive solution and a ground state solution for (1.1) is established.

2.   Preliminaries

As usual, we use the Sobolev space

endowed with the norm

The weighted Lebesgue space is defined as follows

endowed with the norm

The space $L_W^p(\mathbb{R}^N)$ with the norm $\|u\|_{W, p}$ is similarly defined.

The following proposition is proved in ^[2].

Proposition 2.1. ^[2] Assume that $(V, K)\in\mathcal{K}$. Then, $X$ is compactly embedded in $L_K^q(\mathbb{R}^N)$ for all $q\in(2, 2^*)$ if ($K_2$) holds. If ($K_3$) holds, $X$ is compactly embedded in $L_K^\sigma(\mathbb{R}^N)$.

To resolve (1.1), due to the appearance of the nonlocal term $\int_{\mathbb{R}^N}g^2(u)|\nabla u|^2dx$, the right working space seems to be

However, generally $X_0$ is not a linear space and the functional

may be not well defined on $X_0$, where

To avoid these drawbacks, following ^[20,26,30], we make a change of variables

Then, it follows from the properties of $g$, $G$ and $G^{-1}$, which will be listed in Lemma 2.4 that if $v\in X$, then $u = G^{-1}(v)\in X$ and

After the change of variables, (1.1) changes to

One can easily derive that if $v\in X$ is a classical solution to (2.4), then $u = G^{-1}(v)\in X$ is a classical solution to (1.1). Thus, we only need to seek weak solutions to (2.4). The associated function to (2.4) is

By the conditions on $g$, $f$ and $h$, it is easy to prove that $J_\lambda$ is well defined and belongs to $C^1$ on $X$. Hence, $X$ is a proper working space for the problem. Here, we say that $v\in X$ is a weak solution to (2.4) if

for all $\varphi\in X$.

Before proving the main theorem, we show some technical embedding results for possiblely $p\leq 2$, which can be used to deal with sublinear problems comparing with Proposition 2.1.

Lemma 2.2. Assume that $(W_{0})$–$(W_{2})$ hold. Then, $X$ is continuously embedded in $L_W^p(\mathbb{R}^N)$ for all $p\in(1, 2^*/2)$.

Proof. As mentioned in ^[2], $W(x)$ satisfies ($K_1$) and ($K_2$) since it satisfies $(W_{1})$ and $(W_{2})$. It is clearly $2p\in(2, 2^*)$ for $p\in(1, 2^*/2)$. Therefore, Proposition 2.1 shows that $X$ is compactly embedded in $L_W^{2p}(\mathbb{R}^N)$ for every $p\in(1, 2^*/2)$, and, thus, there exists $\nu_{W, 2p} > 0$ such that

for every $p\in(1, 2^*/2)$. Moreover, since $W(x)\in L^1(\mathbb{R}^N)$, by Hölder's inequality and $(W_{0})$–$(W_{2})$, we deduce for any $u\in X$

for all $p\in(1, 2^*/2)$, implying that $X$ is continuously embedded in $L_W^p(\mathbb{R}^N)$. □

Lemma 2.3. Assume that $(W_{0})$–$(W_{2})$ hold. Then, $X$ is compactly embedded in $L_W^p(\mathbb{R}^N)$ for all $p\in(1, 2)$, $N\geq3$.

Proof. Lemma 2.2 shows that $X$ is continuously embedded in $L_W^p(\mathbb{R}^N)$ for every $p\in(1, 2)$, and $N\leq4$ since $2\leq2^*/2$ in this case. For every $p\in(1, 2)$, fix $p_0\in(1, p)$ and $q_0\in(2, 2^*)$. Then, it follows by Hölder's inequality that

which implies by Lemma 2.2 and Proposition 2.1 that $X$ is compactly embedded in $L_W^p(\mathbb{R}^N)$ for all $p\in(1, 2)$ and $N\leq4$. Moreover, in the case $N\geq5$, for every $p\in[2^*/2, 2)$, we fix $p_1\in(1, 2^*/2)$ and $q_1\in (2, 2^*)$. By a similar inequality, we obtain that $X$ is compactly embedded in $L_W^p(\mathbb{R}^N)$ for all $p\in(1, 2)$, $N\geq5$.

In conclusion, $X$ is compactly embedded in $L_W^p(\mathbb{R}^N)$ for all $p\in(1, 2)$. □

Now we list the main properties of the function $G^{-1}$ ^[14,29].

Lemma 2.4. Suppose that $g$ satisfies $(g_{0})$. Then, the function $G^{-1}\in C^2(\mathbb{R}, \mathbb{R})$ satisfies the following properties:

$(g_1)$ $G^{-1}$ is increasing and $G$, $G^{-1}$ are odd functions;

$(g_2)$ $0 < \frac{d}{dt}\big(G^{-1}(t)\big) = \frac{1}{g(G^{-1}(t))}\leq\frac{1}{g(0)}$ for all $t\in\mathbb{R}$;

$(g_3)$ $|G^{-1}(t)|\leq\frac{|t|}{g(0)}$ for all $t\in\mathbb{R}$;

$(g_4)$ $\lim\limits_{t\rightarrow 0}\frac{G^{-1}(t)}{t} = \frac{1}{g(0)}$;

$(g_5)$ $1\leq \frac{tg(t)}{G(t)}\leq 2$ and $1\leq\frac{G^{-1}(t)g(G^{-1}(t))}{t}\leq 2$ for all $t\neq 0$;

$(g_6)$ $\frac{G^{-1}(t)}{\sqrt{t}}$ is non-decreasing in $(0, +\infty)$ and $|G^{-1}(t)|\leq (2/g_{\infty})^{1/2}\sqrt{|t|}$ for all $t\in\mathbb{R}$;

$(g_7)$ The following inequalities hold

$(g_8)$ $\frac{t}{g(t)}$ is increasing and $\big{|}\frac{t}{g(t)}\big{|}\leq\frac{1}{g_{\infty}}$ for all $t\in\mathbb{R}$;

$(g_{9})$ $[G^{-1}(s-t)]^2\leq 4([G^{-1}(s)]^2+[G^{-1}(t)]^2)$ for all $s, t\in\mathbb{R}$;

$(g_{10})$ $\lim\limits_{t\rightarrow +\infty}\frac{G^{-1}(t)}{\sqrt{t}} = (\frac{2}{g_{\infty}})^{1/2}$.

Remark 2.1. Define the function $\Psi$: $X\rightarrow \mathbb{R}$ by

It is easy to verify that it is a $C^1$ function on $X$ by the conditions on $g$. Moreover, by $(g_{3})$ and $V(x) > 0\ {for\ all}\ x\in\mathbb{R}^N$, we have

and as stated in ^[1], by $(g_{3})$, $(g_{7})$ and $(V_{1})$, there is a constant $\xi > 0$ such that

Throughout this paper, $C$ denotes the various positive constant. $\nu_{K, q} > 0$ denotes the Sobolev embedding constant for $X\hookrightarrow L_K^q(\mathbb{R}^N)$, that is $\|u\|_{K, q}\leq \nu_{K, q}\|u\|$ for any $u\in X$, and the definition of Sobolev embedding constant for $X\hookrightarrow L_W^p(\mathbb{R}^N)$ is similar. Besides, it is well known that the embedding $D^{1, 2}({\mathbb{R}^{N}})\hookrightarrow L^{2^*}(\mathbb{R}^{N})$ is continuous, i.e., there exists $\nu_1 > 0$ such that $\|u\|_{2^*}\leq \nu_1\|u\|_{D^{1, 2}({\mathbb{R}^{N}})}$ for any $u\in D^{1, 2}({\mathbb{R}^{N}})$.

3.   The mountain pass geometry and the boundedness of the Cerami sequences

In this section, we first state a version of the mountain pass theorem due to Ambrosetti and Rabinowitz ^[4], which is an essential tool in this paper, then we show that the function associated to (2.4) possesses a Cerami sequence at the corresponding mountain pass level. Afterward, the boundedness of the Cerami sequence is established.

We recall the definition of Cerami sequence. Let $X$ be a real Banach space and $J_\lambda$: $X\rightarrow \mathbb{R}$ a functional of class $C^1$. We say that $\{v_n\}\subset X$ is a Cerami sequence at $c$ ($(Ce)_c$ for short) for $J_\lambda$ if $\{v_n\}$ satisfies

and

as $n\rightarrow \infty$. $J_\lambda$ is said to satisfy the Cerami condition at $c$, if any Cerami sequence at $c$ possesses a convergent subsequence.

Theorem 3.1. ^[31] Let $X$ be a real Banach space and $J\in C^1(X, \mathbb{R})$. Let $\Sigma$ be a closed subset of $X$, which disconnects (arcwise) $X$ into distinct connected $X_1$ and $X_2$. Suppose further that $J(0) = 0$ and

$(J_1)$ $0\in X_1$, and there is $\alpha > 0$ such that $J|_{\Sigma}\geq\alpha > 0$,

$(J_2)$ there is $e\in X_2$ such that $J(e) < 0$.

Then, $J$ possesses a $(Ce)_c$ sequence with $c\geq\alpha > 0$ given by

where

Lemma 3.2. Assume that $(V, K)\in\mathcal{K}$. $(g_{0}),$ $(F_{0})$–$(F_{3}),$ $(W_{0})$–$(W_{2}),$ $(H_{0})$ and $(H_{1})$ hold. Then, there exists $\lambda_0, \alpha_0 > 0$ such that for any $\lambda\in(0, \lambda_0),$ $J_\lambda$ possesses a Cerami sequence at

where

Proof. It is enough to prove that the function satisfies the mountain pass geometry. We only consider the case where $(K_2)$ holds and the proof is similar if $(K_3)$ holds.

First note that $J_\lambda(0) = 0$ for any $\lambda > 0$. For every $\rho > 0$, define

Since the function $\int_{\mathbb{R}^N}\big(|\nabla v|^2+V(x)[G^{-1}(v)]^2\big)dx$ is continuous on $X$, ${\Sigma}_\rho$ is a closed subset in $X$ which disconnects the space $X$.

$(1)$ There exists $\lambda_0, \rho_0, \alpha_0 > 0$ such that $J_\lambda(v)\geq\alpha_0 > 0$ for any $\lambda\in(0, \lambda_0), \ v\in{\Sigma}_{\rho_0}$. Indeed, for every $\rho > 0$, by $(K_2)$, we have

for any $v\in {\Sigma}_{\rho}$. Moreover, by $K(x)\in L^{\infty}(\mathbb{R}^N)$, $(g_{6})$ and Sobolev embedding, we conclude that

for any $v\in {\Sigma}_{\rho}$. Thus, by $(F_{0})$–$(F_{2})$, (3.3) and (3.4), we obtain for any $\varepsilon > 0$, there exists $C_\varepsilon > 0$ such that

for any $v\in {\Sigma}_{\rho}$.

In addition, according to Lemma 2.3, $(g_{2})$ and $(g_{3})$, we deduce that

and

for any $v\in {\Sigma}_{\rho}$.

Thus, by $(H_{0})$, $(H_{1})$, (3.6) and (3.7), it follows that

for any $v\in {\Sigma}_{\rho}$.

Choose $\varepsilon_0 > 0$ such that $\text{ess}\sup_{x\in\mathbb{R}^N}\big|\frac{K(x)}{V(x)}\big|\varepsilon_0 < \frac{1}{2}$. By (3.5) and (3.8), we conclude that

for any $\lambda > 0$, $\rho > 0$, $v\in {\Sigma}_{\rho}$.

Choose $\rho_0 > 0$ such that

and set

Then,

for any $\lambda\in(0, \lambda_0)$, $v\in {\Sigma}_{\rho_0}$.

$(2)$ For any $\lambda\in(0, \lambda_0)$, there exists $e\in X$ such that

and $J_\lambda(e) < 0$. To this end, for any $\lambda\in(0, \lambda_0)$, fixed $v\in X$ is a nonnegative smooth function with $m(\text{supp}v) > 0$, where

is the support of $v$. We prove $J_\lambda(tv) < 0$ if $t > 0$ and $\int_{\mathbb{R}^N}\big(|\nabla(tv)|^2+V(x)|G^{-1}(tv)|^2\big)dx$ is large enough. Suppose by contradiction that there exists a sequence $\{t_n\}\subset\mathbb{R}^+$ such that

and $J_\lambda(t_nv)\geq0$ for all $n\in\mathbb{N}$. By $(g_{3})$, we know

which means that $t_n\rightarrow +\infty$. Set $\varpi = \frac{v}{\|v\|}$. Noticing that $K(x), \ W(x) > 0$, $\forall x\in\mathbb{R}^N$, by $(H_{0})$, $(F_{0})$ and $(g_{3})$ we get

Since $t_nv(x)\rightarrow +\infty$ as $n\rightarrow +\infty$, for $x\in\text{supp}v$, it follows from $(g_{10})$, $K(x) > 0$, $(F_{0})$, $(F_{3})$ and Fatou's lemma that

as $n\rightarrow +\infty$, which is a contradiction by inequality (3.9).

The proof is ended. □

We now show the boundedness of the Cerami sequence.

Lemma 3.3. Assume that $(g_{0}),$ $(V_{1}),$ $(W_{0})$–$(W_{2}),$ $(H_{0}),$ $(H_{1})$ and $(F_{4})$ hold, then any $(Ce)_{c_\lambda}$ sequence of $J_\lambda$ is bounded in $X$ for any $\lambda\in(0, \lambda_0)$.

Proof. Let $\{v_n\}$ be the corresponding $(Ce)_{c_\lambda}$ sequence for $J_\lambda$. Denote $\omega_n = G^{-1}(v_n)g(G^{-1}(v_n))$. Then, it follows from (1.7) that

By (1.7) and $(g_{5})$, we get

Hence, $\omega_n\in X$ and $\|\omega_n\|\leq4\|v_n\|$, which gives

Therefore, taking into account $(H_{0})$, $(H_{1})$, $(W_{0})$, $(F_{4})$, (3.10) and (3.11), we conclude that

Hence, combining with $(W_{0})$–$(W_{2})$, Lemma 2.3, (3.12) and $(g_{2})$, we deduce that for any $\lambda > 0$,

Since $\tau_1, \ \tau_2\in(1, 2)$, $\{\Psi(v_n)\}$ is bounded in $X$, by Remark 2.1 we obtain that $\{v_n\}$ is bounded in $X$. □

4.   Proofs of main theorems

Under the hypotheses of Lemmas 3.2 and 3.3, for any fixed $\lambda\in(0, \lambda_0)$, let $\{v_n\}$ be the $(Ce)_{c_\lambda}$ sequence for $J_\lambda$. Then, by Lemma 3.3 we know that $\{v_n\}$ is bounded in $X$. Thus, there exists a subsequence still denoted by $\{v_n\}$, and $v\in X$ such that

and there is $L > 0$ such that

We conclude this section showing that the weak limit $v$ is a positive solution to (1.1).

Lemma 4.1. Assume that $(g_{0}),$ $(W_{0})$–$(W_{2}),$ $(H_{0}),$ $(H_{1})$ hold and $\{v_n\}$ is a $(Ce)_{c_\lambda}$ sequence for $J_\lambda$ given by Lemmas 3.2 and 3.3. Then, the following statements hold:

Proof. First, we give the proof of (4.3). Since $\tau_1, \ \tau_2\in(1, 2)$, from $(W_{0})$–$(W_{2})$ and Lemma 2.3, we have

Then, given $\varepsilon > 0$, there is $r > 0$ such that

where $B_r^c: = \{x\in\mathbb{R}^N:|x| > r\}$, which together with $(H_{0})$, $(H_{1})$ and $(g_{3})$ yields that

for any $n\in\mathbb{N}$.

Moreover, for each fixed $r > 0$, it is easy to verify that

where $B_r(0) = \{x\in\mathbb{R}^N:|x|\leq r\}$. This completes the proof of (4.3). □

Proof. Now we are going to prove (4.4). Noticing (4.7), given $\varepsilon > 0$, there is $r > 0$ such that

By $(H_{0})$, $(H_{1})$, $(W_{0})$, $(g_{2})$, $(g_{3})$ and Hölder's inequality, we obtain that

for any $n\in\mathbb{N}$, $\varphi\in X$. Since $\tau_1, \tau_2\in(1, 2)$, Lemma 2.3 implies that $\int_{B_r^c}W(x)|\varphi|^{\tau_1}dx < \infty$ and $\int_{B_r^c}W(x)|\varphi|^{\tau_2}dx < \infty$. Thus, combining with (4.9) and (4.10), we conclude that

for any $\varphi\in X$, where $C_1 = b_1\|\varphi\|_{W, \tau_1}+b_2\|\varphi\|_{W, \tau_2}$.

Moreover, for each fixed $r > 0$, it is easy to verify that

This completes the proof of (4.4). □

Repeating the similar arguments used in the proofs of (4.3) and (4.4), we can obtain that (4.5) and (4.6) hold.

Lemma 4.2. Assume that $(V, K)\in\mathcal{K},$ $(g_{0}),$ $(F_{0})$–$(F_{2})$ hold and $\{v_n\}$ is a $(Ce)_{c_\lambda}$ sequence for $J_\lambda$ given by Lemmas 3.2 and 3.3. Then, the following statements hold:

Proof. (1) We begin the proof of (4.11) by assuming that $(K_2)$ holds. By $(F_{0})$–$(F_{2})$, we obtain that there exists $C_2 > 0$ such that

which together with $(F_{0})$–$(F_{2})$, $(g_{3})$ and $(g_{6})$ yields that, for any fixed $q\in(2, 2^*)$, given $\varepsilon > 0$ there exists $0 < s_0 < s_1$ such that

for all $s\in\mathbb{R}$, where $\theta = G^{-1}(s)$.

In addition, by $K(x), \ V(x) > 0$ for all $x\in\mathbb{R}^N$, $(K_2)$ and $K(x)\in L^{\infty}(\mathbb{R}^N)$, we obtain that there exists $C_3 > 0$ such that

for any $s\in\mathbb{R}^N$.

Furthermore, noticing $q\in(2, 2^*)$, then from Proposition 2.1 we have

which gives that there is $r > 0$ such that

Therefore, combining with (4.2), (4.15), (4.16) and (4.18), we conclude that

for all $n\in\mathbb{N}$.

Now, if $(K_3)$ holds, by $(F_{0})$–$(F_{2})$, we obtain there exists $C_4 > 0$ such that

which together with $(F_{0})$–$(F_{2})$, $(g_{3})$ and $(g_{6})$ yields that, given $\varepsilon\in(0, 1)$, there exists $0 < s_0 < s_1$ such that

for any $s\in\mathbb{R}$.

Furthermore, noticing $\sigma\in(2, 2^*)$, by Proposition 2.1 we have

which gives that there is $r > 0$ such that

Therefore, by (4.2), (4.19), (4.21), $K(x) > 0$ for all $x\in\mathbb{R}^N$ and $K\in L^\infty(\mathbb{R}^N)$, we obtain that

for any $n\in\mathbb{N}$, where

Furthermore, for each fixed $r > 0$, it is easy to verify that

This completes the proof of (4.11).

(2) We begin the proof of (4.12) if $(K_2)$ holds. By $(F_{0})$–$(F_{2})$, we have that there is $C_6 > 0$ such that

which together with $(F_{0})$–$(F_{2})$, $(g_{2})$, $(g_{3})$, $(g_{6})$ and $(g_{8})$ yields that, for any fixed $q\in(2, 2^*)$, given $\varepsilon\in(0, 1)$ there exists $0 < s_0 < s_1$ such that

for any $s\in\mathbb{R}$.

On the other hand, noticing $q\in(2, 2^*)$, by (4.17) we have there is $r > 0$ such that

Thus, taking into account $(K_2)$, $K(x), V(x) > 0$ for all $x\in\mathbb{R}^N$, $K(x)\in L^{\infty}(\mathbb{R}^N)$, (4.22) and Hölder's inequality, we obtain that, for any $\varphi\in X$,

for any $n\in\mathbb{N}$. Moreover, for any $\varphi\in X$, by Proposition 2.1, $(K_2)$ and Sobolev embedding, we have

and

respectively. Thus, it follows from (4.2), (4.23) and (4.24) that

where

Now, if $(K_3)$ holds, by $(F_{0})$–$(F_{2})$ we obtain there exists $C_8 > 0$ such that

which together with $(F_{0})$–$(F_{2})$, $(g_{3})$, $(g_{6})$ and $(g_{8})$ yields that, given $\varepsilon\in(0, 1)$ there exists $0 < s_0 < s_1$ such that

for any $s\in\mathbb{R}$. Furthermore, from $(V, K)\in\mathcal{K}$ and Proposition 2.1, we infer that there is $r > 0$ such that

Combining with Hölder's inequality, (4.2), (4.25), (4.26), $K(x), V(x) > 0$ for all $x\in\mathbb{R}^N$, $K(x)\in L^{\infty}(\mathbb{R}^N)$ and Proposition 2.1, it follows that

where

for all $\varphi\in X$, $n\in\mathbb{N}$.

Moreover, for each fixed $r > 0$, it is easy to verify that

This completes the proof of (4.12)

Repeating the similar arguments used in the proof of (4.11) and (4.12), we obtain that (4.13) and (4.14) hold. □

Lemma 4.3. Assume that $(V, K)\in\mathcal{K},$ $(g_{0}),$ $(V_{1}),$ $(W_{0})$–$(W_{2}),$ $(F_{0})$–$(F_{2}),$ $(H_{0}),$ $(H_{1})$ hold and $\{v_n\}$ is a $(Ce)_{c_\lambda}$ sequence for $J_\lambda$ given by Lemmas 3.2 and 3.3. Then, the following statements hold:\\ (i) For each $\varepsilon > 0$ there exists $r_0 > 1$, such that for any $r > r_0$

and

(ii) The weak limit $v$ of $\{v_n\}$ is a critical point for the function $J_\lambda$ on $X$.

(iii) The weak limit $v$ is a nontrivial critical point of $J_\lambda$ and $J_\lambda(v) = c_\lambda$. Moreover, the function $J_\lambda$ satisfies the Cerami condition on $X$.

Proof. (i) For $r > 1$, we choose a cut-off function $\eta = \eta_r\in C_0^\infty(B_r^c)$ such that

and

As $\{v_n\}$ is bounded in $X$, the sequence $\{\eta\omega_n\}$ where $\omega_n = G^{-1}(v_n)g(G^{-1}(v_n))$ is also bounded in $X$. Hence, from (3.11) we have

that is

Then, by $(g_{0})$, $(F_{0})$, $(H_{0})$, (4.31) and (4.33) we infer that

for any $r > 1$.

By (4.2), (4.32), $(g_{5})$ and Hölder's inequality, we obtain

for any $r > 1$, $n\in\mathbb{N}$. Noticing that $v_n\rightarrow v$ in $L^2(B_{2r}\setminus B_r)$ and $|B_{2r}\setminus B_r|\leq|B_{2r}| = \omega_N(2r)^N$ for any fixed $r > 1$, then (4.35) follows that

for any $r > 1$. Furthermore, for any $\varepsilon > 0$ there exists $r_1 > 1$, and for any $r > r_1$

Therefore, combining with (4.36) and (4.37), we have that for any $r > r_1$,

In addition, according to the (4.5) and (4.13), we infer that there is $r_2 > 1$ such that

and

for any $r > r_2$.

Set $r_0 = \max\{r_1, r_2\}$. Then, taking into account (4.34), (4.38), (4.39) and (4.40), we know that (4.27) is valid.

Noticing $(g_{5})$, we know that $\big|\frac{v_n}{g(G^{-1}(v_n))}\big| < \big|G^{-1}(v_n)\big|$ for any $n\in\mathbb{N}$. Then, (4.27) implies that (4.28) holds.

Moreover, the limit (4.27) gives that

for any $r > r_0$ and consequently,

for any $r > r_0$. Since $v_n\rightarrow v$ in $L^2(B_{2r}(0))$ for any fixed $r\in(0, +\infty)$, then by $(g_{3})$ and the continuity of $V(x)$, using the Lebesgue dominated convergence theorem we know that

Then, (4.41)–(4.43) yield that

and, hence, (4.29) holds. Similarly, it follows from (4.28) that (4.30) holds.

(ii) It is clear that

as $n\rightarrow +\infty$. Noting that $\big\{\sqrt{V(x)}\frac{G^{-1}(v_n(x))}{g(G^{-1}(v_n(x)))}\big\}$ is bounded in $L^2(\mathbb{R}^N)$ and $\sqrt{V}\varphi\in L^2(\mathbb{R}^N)$ for any $\varphi\in X$, we have that

in $L^2(\mathbb{R}^N)$, as $n\rightarrow +\infty$ and, hence, the following equality holds

Furthermore, since $v_n\rightharpoonup v$ in $D^{1, 2}(\mathbb{R}^N)$ and $\varphi\in D^{1, 2}(\mathbb{R}^N)$, we have

Thus, by (4.4), (4.12), (4.44) and (4.45), we deduce that

Thus, $J_\lambda'(v) = 0$, which implies that (ii) holds.

(iii) We have proved that $J_\lambda'(v) = 0$. Now, we show that $v\neq 0$. Suppose that $v\equiv 0$; because $\{v_n\}$ is a $(Ce)_{c_\lambda}$ sequence, according to (3.2), we know

Moreover, by (4.5), (4.13) and (4.30), we get

and

Then, it follows from (4.46) that

In addition, from (4.3), (4.11) and (4.29), we have

and

Hence,

which is a contradiction to $J_\lambda(v_n)\rightarrow c_\lambda > \alpha_0 > 0$. Therefore, $v\neq 0$.

Now, we show that $J_\lambda(v) = c_\lambda$. By $\langle J_\lambda'(v_n), v_n\rangle = o_n(1)$, passing to the limit in the following expression

and using (4.6), (4.14) and (4.30) together with $(J_\lambda'(v), v) = 0$, we obtain that

By (4.3), (4.11), (4.29) and (4.47), we conclude that

which results that $J_\lambda(v) = c_\lambda$.

To show that the function $J_\lambda$ satisfies the Cerami condition, we verify that $\|v_n-v\|\rightarrow 0$. By Remark 2.1, we have

where

Combining with $(g_{9})$, (4.41) and (4.42), we obtain that given $\varepsilon > 0$

for any $r > r_0$. Furthermore, noticing $v_n\rightarrow v$ in $L^2(B_{2r})$ for any fixed $r > 0$, by $(g_{3})$ we infer that

implying that

Thus, (4.48) and (4.49) lead to

which together with (4.47) yields that $\Psi(v_n-v)\rightarrow0$. Then, $\|v_n-v\|\rightarrow 0$ holds, implying that $J_\lambda$ satisfies the Cerami condition. □

Proof of Theorem 1.1. Combining all the results above, we get that for every $\lambda\in(0, \lambda_0)$, (2.4) possesses a nontrivial solution $v$. Furthermore, letting $v^{-} = \max\{-v, 0\}$, by $J_\lambda'(v) = 0$, $(F_{0})$ and $(W_{0})$, we have that

Since $G^{-1}(v)v^{-}\geq 0$, $V(x) > 0$ and $g(G^{-1}(v)) > 0$, we have that

Thus, $v^{-} = 0$ a.e., $x\in\mathbb{R}^N$. Therefore, $v$ is a positive solution to (2.4); that is, $u = G^{-1}(v)$ is a positive solution to (1.1).

Now, we prove Proposition 1.2. At first we show the following lemma.

Set $S_{r} = \big\{v\in X:\|v\| = r\big\}$ and $\mathcal{B}_{r} = \big\{v\in X:\|v\| < r\big\}$.

Lemma 4.4. Assume that $(V, K)\in\mathcal{K}$ where $(K_2)$ holds and $(g_{0}),$ $(V_{1}),$ $(W_{0})$–$(W_{2}),$ $(F'_{0}),$ $(F_{1}),$ $(F_{2}),$ $(F_{4}),$ $(H'_{0})$–$(H'_{2})$ hold. Then, there exists $r_1, \alpha_1, \lambda_1 > 0$ such that $J_{\lambda}|_{S_{r_1}}\geq\alpha_1$ and $\inf\limits_{v\in \mathcal{B}_{r_1}}J_{\lambda}(v) < 0$ for any $\lambda\in(0, \lambda_1)$.

Proof. Noticing Remark 2.1, we have

Choose $\varepsilon_1 > 0$ such that

By $(F'_{0})$, $(F_{1})$, $(F_{2})$, $(H'_{0})$, $(H'_{1})$, $(W_{0})$–$(W_{2})$, (K_2), $K(x)\in L^{\infty}(\mathbb{R}^N)$, $K(x), V(x) > 0$ for all $x\in\mathbb{R}^N$, $(g_{6})$, (4.50) and Lemma 2.3, we obtain there exists $C_{\varepsilon_1} > 0$ such that

for any $v\in X$, $\lambda > 0$.

Consider

for $t\geq0$. Obviously, there exists $r_1 > 0$ such that $\max\limits_{t\geq0}l_1(t) = l_1(r_1)\triangleq \Lambda_1 > 0$. Then, it follows from (4.51) that

Set

and $\alpha_1 = \frac{\Lambda_1}{2}$, and it follows from (4.52) that

On the other hand, (4.51) implies that $J_{\lambda}(v)$ is bounded blow in $\mathcal{B}_{r_1}$ for any $\lambda > 0$. Taking $\varphi\in X$ and $\varphi\neq 0$, by $(g_{4})$, $(H'_{2})$ and $(W_{0})$ and combining the Lebesgue dominated convergence theorem and Sobolev embedding theorem we have

Then, there exists $\delta > 0$ for any $0 < t < \delta$,

Then, by $(F'_{0})$, $K(x) > 0$ for all $x\in\mathbb{R}^N$, $(g_{3})$ and (4.53), we obtain that

for any $0 < t < \delta$, $\lambda > 0$. Since $\tau_3\in(1, 2)$, there exists small $t > 0$ such that $t\varphi\in \mathcal{B}_{r_1}$ and $J_{\lambda}(t\varphi) < 0$ for any $\lambda > 0$. Therefore, we complete the proof of this lemma. □

Proof of Proposition 1.2. By Lemma 4.4 and Ekeland's variational principle ^[12], we infer that, for any $\lambda\in(0, \lambda_1)$, there is a minimizing sequence $\{v_n\}\subset\bar{\mathcal{B}}_{r_1}$ of the infimum $c_0 = \inf\limits_{v\in\bar{\mathcal{B}}_{r_1}}J_{\lambda}(v) < 0$, such that

and

First, we claim that $||v_n|| < r_1$ for large $n\in\mathbb{N}$. Otherwise, we may assume that $||v_n|| = r_1$. Up to a subsequence, by Lemma 4.4 we get $J_{\lambda}(v_n)\geq\alpha_1 > 0$, which and (4.55) imply that $0 > c_0\geq\alpha_1 > 0$, which is a contradiction. In general, we suppose that $||v_n|| < r_1$ for all $n\in\mathbb{N}$. Next, we will show that $J'_{\lambda}(v_n)\rightarrow 0$ in $X^*$. For any $n\in\mathbb{N}$ and $\varphi\in X$ with $||\varphi|| = 1$, we choose sufficiently small $\delta_n > 0$ such that $||v_n+t\varphi|| < r_1$ for all $0 < t < \delta_n$. It follows from (4.56) that

Letting $t\rightarrow 0^+$, we get

for any $n\in\mathbb{N}$. Similarly, replacing $\varphi$ with $-\varphi$ in the above arguments, we have

for any $n\in\mathbb{N}$. Therefore, we conclude that, for all $\varphi\in X$ with

Thus, we obtain that $J_{\lambda}(v_n)\rightarrow c_0$ and $J'_{\lambda}(v_n)\rightarrow 0$ as $n\rightarrow \infty$. Noticing that $||v_n|| < r_1$, we get that $\{v_n\}$ is a $(Ce)_{c_0}$ sequence for $J_{\lambda}(v)$ in $X$, and there exists $v_*\in\bar{\mathcal{B}}_{r_1}$ such that $v_n\rightharpoonup v_*$ in $X$. Using the same type of arguments in Lemmas 4.1–4.3, we obtain that $v_*$ is a critical point for $J_{\lambda}(v)$ in $X$ satisfying $v_*\neq0$ and $J_{\lambda}(v_*) = c_0 < 0$.

Next, we investigate the existence of ground state solutions for (1.1). For any $\lambda\in(0, \lambda_1)$, define

Clearly, $\mathcal{S}$ is nonempty and $\mathcal{M}_0 < 0$. For all $v\in\mathcal{S}$, set

Then, we deduce from (1.7), $K(x) > 0$ for all $x\in\mathbb{R}^N$, $(F_{4})$, $(H'_{0})$, $(H'_{1})$, $(g_{2})$, $(W_{0})$–$(W_{2})$ and Lemma 2.3 that for any $\lambda\in(0, \lambda_1)$,

Consider the function

for $t\geq 0$. Since $\tau_3\in (1, 2)$, for any fixed $\lambda\in(0, \lambda_1)$ there exists $t_2 > 0$ such that

Then, it follows from (4.58) that $J_{\lambda}(v)\geq l_2(t_2) > -\infty$ for any $v\in\mathcal{S}$, which implies $\mathcal{M}_0 > -\infty$. Letting $\{v_n\}\subset \mathcal{S}$ be a minimizing sequence of $\mathcal{M}_0$ such that $J_{\lambda}(v_n)\rightarrow \mathcal{M}_0$, set

Since $v_n\in\mathcal{S}$ for any $n\in\mathbb{N}$, then $\langle J'_{\lambda}(v_n), \varpi_n\rangle = 0$ for any $n\in\mathbb{N}$. Repeating the ideas explored in the proof of Lemma 3.3, we have that $\{v_n\}$ is bounded in $X$. Thus, $\{v_n\}$ is a $(Ce)_{\mathcal{M}_0}$ sequence and there exists $v^*\in X$ such that $v_n\rightharpoonup v^*$ in $X$. Using the same type of arguments in Lemmas 4.1–4.3, we obtain that $v^*$ is a critical point for $J_{\lambda}(v)$ satisfying $v^*\neq 0$ and $J_{\lambda}(v^*) = \mathcal{M}_0 < 0$. Thus, $v^*$ is a ground state solution of (2.4). This ends the proof of Proposition 1.2.

5.   Conclusions

By using the variational method, this paper studies a kind of generalized quasilinear Schrodinger equation with concave-convex nonlinearities and potentials vanishing at infinity. We use the mountain pass theorem to prove that this problem has a positive solution. In addition, the existence of a ground state solution is also proved by Ekeland's variational principle. To the best of our knowledge, few works in this case seem to have appeared in the literature.

Use of AI tools declaration

The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

Acknowledgments

We are grateful to the referees for their important suggestions, particularly for the suggestion in the ground state solution.

Conflict of interest

All authors declare no conflicts of interest in this paper.