A stochastic linear-quadratic optimal control problem with jumps in an infinite horizon

1.   Introduction

LQ optimal control is an important branch of control theory. The state equation of the control system is a linear equation, the performance index is a quadratic index and the optimal control can be given in the form of linear feedback. Getting the feedback form of optimal control is the most basic issue of LQ optimal control problems. The research on LQ optimal control problems has a long history. It can be traced back to the works of Bellman-Glicksberg-Gross ^[1] in 1958, and they first attempted to solve deterministic LQ optimal control problem. In 1960, Kalman ^[10] solved the deterministic LQ optimal control problem in the form of linear state feedback, and introduced Riccati equation into the control theory. The above-mentioned works were concerned with deterministic cases, i.e., the state equation is a linear ordinary differential equation (ODE, for short), and all the involved functions are deterministic. Compared with LQ optimal control problems, the control system of stochastic LQ optimal control probelms is stochastic. In 1962, Kushner ^[11] first study a stochastic LQ optimal control probelm driven by Brownian motion with stochastic differential equation of Itô-type by dynamic programming. In 1968, Wonham ^[21,22] first extended the deterministic LQ optimal control problem to a stochastic LQ optimal control problem that contained Riccati differential equation, followed by several researchers (see, for example, Davis ^[27] and Bensoussan ^[28]). Kohlmann and Zhou ^[29] discussed the relationship between a stochastic control problem and a backward stochastic differential equation (BSDE, for short). Based on ^[29], Lim and Zhou ^[12] firstly solved a general LQ optimal control problem of BSDE and gave an explicit form of optimal control. Li, Sun and Yong ^[17] studied the open-loop and closed-loop solvability of stochastic LQ optimal control.

In classical optimal control problems, the termination time is a real number. But among many dynamic optimization problems in economics, the termination time of the optimal control problem might be infinite. We call the mentioned case the optimal control problems with infinite horizon. In 1974, Halkin ^[6] introduced the necessary conditions for optimal control problems with infinite horizon. In 2000, Rami, Zhou, and Moore ^[15] discussed well-posedness and attainability of indefinite stochastic linear-quadratic control problems over an infinite time horizon. In 2003, Wu and Li ^[23] studied an infinite horizon LQ problem with unbounded controls in a Hilbert space. In 2005, Guatteri and Tessitore ^[3] studied the backward stochastic Riccati equation in infinite dimensions, and in 2008, Guatteri and Tessitore ^[4] studied LQ optimal control problems with stochastic coefficients over an infinite horizon. In 2009, Guatteri and Masiero ^[5] discussed ergodic optimal quadratic control problems for an affine equation with stochastic coefficients over an infinite horizon. Then, Hu ^[7] studied the optimal quadratic control for an affine equation driven by Lévy processes over an infinite horizon in 2013. In 2015, Huang, Li and Yong ^[9] discussed a LQ optimal control problem for mean-field stochastic differential equations over an infinite horizon. In 2016, Sun-Li-Yong ^[17] put forward the concepts of open-loop and closed-loop solvabilities, and it was shown that the closed-loop solvability is equivalent to the existence of a regular solution of the Riccati equation. Different from finite-horizon, Sun-Yong ^[18] found that for infinite-horizon LQ optimal control problems, both the open-loop and closed-loop solvabilities are equivalent to the existence of a $static\ stabilizing\ solution$ to the associated generalized ARE. Also, every open-loop optimal control admits a closed-loop representation.

In fact, many random phenomena presented discontinuous motion characteristics with jumps. Therefore, the Poisson jump process which describes such discontinuous random phenomena came into. The following are the research status of the system with jumps. In ^[2], Boel Varaiya et al. discussed the optimal control problem of processes with jumps for the first time. In 1994, Tang and Li ^[20] first proved the necessary condition of stochastic optimal control with jumps and first discussed the BSDE with a Poisson process. In 1997, Situ ^[16] made a further research on the solutions of BSDE with jumps. In 2003, Wu and Wang ^[24] studied the stochastic LQ optimal control problem with state equation driven by Brownian motion and Poisson jump process, and obtained the existence and uniqueness result of solutions of the deterministic Riccati equation. In 2008, Hu and Øksendal ^[8] discussed the partial information linear quadratic control for jump diffusions. Then in 2009, Oksendal and Sulem ^[14] studied the maximum principles for optimal control of forward-backward stochastic differential equations with jumps. In 2014, Meng ^[13] discussed the existence and uniqueness of solutions to the backward stochastic Riccati equation with jumps. In 2018, Li-Wu-Yu ^[30] studied the stochastic LQ optimal control problem with Poisson processes under the indefinite case.

Different from the research on the optimal control problem of the jump diffusion system mentioned above, in this paper we will generalize the results of the infinite horizon stochastic LQ problem of Sun-Yong ^[18] to the jump diffusion system and our goal is to establish the corresponding stability theory and the optimal state feedback representing of the optimal control for jump diffusion system. Firstly, we introduce the concept of $L^2$-stabilizability of a jump-diffusion system over an infinite horizon, and find that the existence of the admissible control set is equivalent to the $L^2$-stabilizability of a jump-diffusion system over an infinite horizon. Secondly, we introduce the definition of the ARE of a jump-diffusion system over an infinite horizon and prove the equivalent among the existence of the admissible control set, the $L^2$-stabilizability of the system and the positive solvalibity of the ARE. Thirdly, we give the concepts of the open-loop and closed-loop solvability of the stochastic LQ optimal control problem with jumps over an infinite horizon. We then introduce the concept of the stabilizing solution of the associated generalized ARE and find that both the open-loop and closed-loop solvability of the problem are equivalent to the existence of a stabilizing solution of the associated generalized ARE, which is different from the finite horizon case. Finally, we find that any open-loop optimal control for the initial state $x$ admits a closed-loop representation. In addition, Our results are generalizations of a recently published paper, Sun and Yong ^[18], on the similar topic to jump diffusion systems. The state equation in our case is a stochastic differential equation driven by a one-dimensional standard Brownian motion and a Poisson stochastic martingale measure, which is more general. It is well-known that processes arising by Poisson random measures play an increasing role in modeling stochastic dynamical systems. And it helps us deal with unexpected situations in financial problems, so it is necessary to use a jump system to characterize.

The paper is organized as follows. In Section 2, we introduce some basic notions and lemmas used throughout this paper. In Section 3, we state the problem. Section 4 aims to give the definition of $L^2$-stable and $L^2$-stabilizable, further we describe the structure of admissible control set and prove the equivalence of the non-emptiness of the admissible control set, the $L^2$-stabilizability of the control system and the existence of a positive solution to an algebraic Riccati equation. In Section 5, we introduce the notions of open-loop and closed-loop solvabilities as well as the algebraic Riccati equation, and finally we state the main result of the paper.

2.   Preliminaries

Throughout this paper, we let $(\Omega, \mathcal {F}, \{\mathcal {F}_{t}\}_{t\geq0}, \mathbb P)$ be a complete probability space on which a one-dimensional standard Brownian motion $W = \{W(t); t\geq0\}$ is defined. Denote by $\mathcal P$ the $\mathcal F_t$-predictable $\sigma$-field on $[0, \infty)\times\Omega$ and by $\mathcal B(\Lambda)$ the Borel $\sigma$-algebra of any topological space $\Lambda$. Let $(Z, \mathcal {B}(Z), \nu)$ be a measurable space with $\nu(Z) < \infty$ and $\eta:\Omega\times D_{\eta}\rightarrow Z$ be an $\mathcal F_t$-adapted stationary Poisson point process with characteristic measure $\nu$, where $D_{\eta}$ is a countable subset of $(0, \infty)$. Then the counting measure induced by $\eta$ is

And $\tilde{\mu}(dt, d\theta) = \mu(dt, d\theta)-dt\nu(d\theta)$ is a compensated Poisson random martingle measure which is assumed to be independent of the Brownian motion $W$. In the following, we introduce the basic notations used throughout this paper.

2.1.   Notations

● $\mathbb H$: The Hilbert space with norm $\|\cdot\|_{\mathbb H}$.

● $\mathbb{R}^{n}$: The n-dimensional Euclidean space.

● $\mathbb{R}^{n\times m}$: The space of all $(n\times m)$ matrices.

● $\langle \alpha, \beta \rangle$: The inner product in $\mathbb{R}^{n}, \ \forall \alpha, \ \beta \in \mathbb{R}^{n}$.

● $|\alpha| = \sqrt{\langle \alpha, \alpha \rangle}$: The norm of $\alpha, \ \forall \alpha \in \mathbb{R}^{n}$.

● $M^\top$: The transpose of matrix $M$.

● $M^{\dagger}$: The Moore-Penrose pseudoinverse of a matrix $M$.

● $\langle M, N \rangle = tr(M^\top N)$: The inner product in $\mathbb{R}^{n\times m}, \ \forall M, \ N \in \mathbb{R}^{n\times m}$.

● $|M| = \sqrt{tr(M^\top M)}$: The norm of $M, \ \forall M \in \mathbb{R}^{n\times m}$.

● $\mathscr R(M)$: The range of a matrix or an operator $M$.

● $\mathbb{S}^{n}\in \mathbb{R}^{n\times n}$: The set of all $(n\times n)$ symmetric matrices.

● $\mathbb{S}_{+}^{n}\in \mathbb{S}^{n}$: The set of all $(n\times n)$ non-negative definite symmetric matrices.

● $\mathbb{F} = \{\mathcal {F}_{t}\}_{ t \geq 0}$: The natural filtration. In other words, the flow of information generated by all market noises.

● $(\Omega, \mathcal {F}, P)$: A complete probability space.

● $(\Omega, \mathcal {F}, \mathbb{F}, P)$: The complete filtered probability space.

● $C([0, \infty); \mathbb{R})$: The space of all continuous functions $\varphi: [0, \infty) \rightarrow \mathbb{R}$.

● $L_{\mathbb{F}}^2(\mathbb H)$: The space of all $\mathbb H$-valued and $\mathbb F$-progressively measurable processes $g(\cdot)$ satisfying $g: [0, \infty)\times \Omega \rightarrow \mathbb H$ and

● $\mathcal X_{loc}[0, \infty)$: The space of all $\mathcal {F}_{t}$-adapted and càdlàg processes $g(\cdot)$ satisfying $g: [0, \infty)\times \Omega \rightarrow \mathbb H$ and

for every $T > 0.$

● $\mathcal X[0, \infty)$: The space of all $\mathcal {F}_{t}$-adapted and càdlàg processes $g(\cdot)$ satisfying $g\in\mathcal X_{loc}[0, \infty)$ and

● $L^{\nu, 2}(Z; \mathbb H)$: The space of all $\mathbb H$-valued measurable function $r = \{r(\theta), \ \theta \in Z\}$ defined on the measurable space $(Z, \mathcal {B}(E); \nu)$ satisfying

● $L^{\nu, 2}_{\mathbb{F}}([0, \infty)\times Z; \mathbb H)$: The space of all $L^{\nu, 2}(\mathbb H)$-valued and $\mathcal {F}_{t}$ - predictable processes $r = \{r(t, \omega, \theta), \ (t, \omega, \theta) \in [0, \infty)\times\Omega\times Z\}$ satisfying

2.2.   Pseudoinverse

We recall some properties of the reference ^[19].

Lemma 2.1. For any M $\in\mathbb R^{m\times n}$, there exists a unique matrix $M^{\dagger}\in\mathbb R^{n\times m}$ such that

In addition, if $M\in\mathbb S^n$, then $M^{\dagger}\in\mathbb S^n, \; MM^{\dagger} = M^{\dagger}M$, and $M\geq0$ if and only if $M^{\dagger}\geq0$.

Lemma 2.2. Let L $\in\mathbb R^{n\times k}$ and $N\in\mathbb R^{n\times m}$. The matrix equation NX = L has a solution if and only if

in which case the general solution is given by

while $Y\in\mathbb R^{m\times k}$ is arbitrary.

The matrix $M^{\dagger}$ above is called the $Moore$-$Penrose\ pseudoinverse$ of $M$.

Remark 2.1. (ⅰ) Clearly, condition (2.1) is equivalent to $NN^{\dagger}L = L$.

(ⅱ) By Lemma 2.2, if $N\in\mathbb S^n$, and $NX = L$, then $X^\top NX = L^\top N^{\dagger}L$.

Lemma 2.3. (Extended Schur's lemma) Let $L\in\mathbb R^{n\times m}, \ M\in\mathbb S^n$, and $N\in\mathbb S^m$. The following conditions are equivalent:

(i) $M-LN^{\dagger}L^\top\geq0, \ N\geq0$, and $\mathscr R(L^\top)\subseteq\mathscr R(N);$

(ii) $\left(\begin{array}{cc}M&L\\L^\top&N\end{array}\right)\geq0.$

3.   Formulation of problem

Now we consider a controlled linear stochastic system on the infinite horizon $[0, \infty)$, the state equation is as follows:

and the quadratic cost functional is given by:

where $A, C\in\mathbb R^{n\times n}; \; E\in L^{\nu, 2}(Z; \mathbb R^{n\times n}); \; B, D, \in\mathbb R^{n\times m}; \; F\in L^{\nu, 2}(Z; \mathbb R^{n\times m}); \; Q\in\mathbb S^n; \; S\in\mathbb R^{m\times n}; \; R\in\mathbb S^m$ are given constant matrices, and $b(\cdot), \sigma(\cdot), q(\cdot)\in L^2_{\mathbb F}({\mathbb R}^n); h(\cdot, \cdot)\in L^{\nu, 2}_{\mathbb F}([0, \infty)\times Z; \mathbb R^n); \rho(\cdot)\in L^2_{\mathbb F}({\mathbb R}^m)$ are given vector-valued $\mathcal F_t$-measurable processes. In the above, $u(\cdot)\in L^2_{\mathbb F}({\mathbb R}^m)$ is called the $control\ process, \ x$, which belongs to $\mathbb R^n$ is called the $initial\ state$, and the $X(\cdot)\equiv X(\cdot; x, u(\cdot))\in\mathbb R^n$, which is the solution of SDE (3.1) is called the $state\ process$ corresponding to the initial state $x$ and the control $u(\cdot)$.

Different from the finite case, the solution $X(\cdot)\equiv X(\cdot; x, u(\cdot))$ of (3.1) might not always be square-integrable for $(x, u(\cdot))\in \mathbb R^n \times L^2_{\mathbb F}({\mathbb R}^m)$ over an infinite time horizon. To make sure the cost functional $J(x; u)$ is well defined, we introduce:

The element $u\in\mathscr U$ is called an $admissible \ control$ associated with $x$, and the linear quadratic optimal control problem over an infinite time horizon now can be stated as follows.

Problem 3.1. For any given initial state $x\in \mathbb R^n$, find an admissible control $u^*\in \mathscr U$ such that

$u^*\in\mathscr U$ is called an open-loop optimal control of Problem (SLQ)$_{\infty}$ for the initial state $x$ if it satisfies (3.3), and the corresponding state process $X^*(\cdot)\equiv X(\cdot; x, u^*)$ is called an $optimal \ state \ process.$ The function $V(\cdot)$ is called the $value \ function$ of Problem (SLQ)$_{\infty}$. A special case when $b, \sigma, h, q, \rho = 0$, we use (SLQ)$_{\infty}^0$, $J^0(x; u)$, and $V^0(x)$ to denote the Problem, the cost functional, and the value function corresponding to Problem 3.1.

The following assumptions on the coefficients will be in force throughout this paper.

Assumption 3.1. The coefficients of the state equation satisfy the following: $A, C\in\mathbb R^{n\times n}; \; B, D\in\mathbb R^{n\times m}$ are given constant matrices; $\; E(\cdot)\in L^{\nu, 2}(Z; \mathbb R^{n\times n}); \; \; F(\cdot)\in L^{\nu, 2}(Z; \mathbb R^{n\times m})$ are given deterministic matrix-valued function, $b(\cdot), \sigma(\cdot)\in L^2_{\mathbb F}({\mathbb R}^n); h(\cdot, \cdot)\in L^{\nu, 2}_{\mathbb F}([0, \infty)\times Z; \mathbb R^n)$ are given vector-valued $\mathcal F_t$-measurable processes.

Assumption 3.2. The coefficients of the cost functional satisfy the following: $Q\in\mathbb S^n; \; S\in\mathbb R^{m\times n}; \; R\in\mathbb S^m$ are given constant matrices, $q(\cdot)\in L^2_{\mathbb F}({\mathbb R}^n), \rho(\cdot)\in L^2_{\mathbb F}({\mathbb R}^m)$ are given vector-valued $\mathcal F_t$-measurable processes.

Assumption 3.3. For all $\theta\in Z$, there exists a constant $\lambda > 0$ such that

As we mentioned before, admissible controls might not exist. To solve this question, the concept of $stability$ is introduced as follows. Let us consider the following uncontrolled linear system:

and we use [$A, C, E$] to denote this system.

Definition 3.1. System [$A, C, E$] is said to be $L^2$-stable if for any initial state $x\in \mathbb R^n$, its solution $X(\cdot; x)$ satisfies

i.e., $X(\cdot; x)\in\mathcal{X}[0, \infty).$

Consider the following BSDE over an infinite horizon [0, $\infty$):

where Assumption 3.1 holds, and {$\varphi(t); 0\leq t < \infty$} is a given $\mathbb F$-progressively measurable, $\mathbb R^n$-valued process. We call the solution ($Y(\cdot), Z(\cdot), r(\cdot, \cdot)$) the adjoint processes corresponding to $X(\cdot)$.

Definition 3.2. If $(Y(\cdot), Z(\cdot), r(\cdot, \cdot))\in\mathcal{X}[0, \infty)\times L^{2}_{\mathbb{F}}(\mathbb R^n)\times L^{\nu, 2}_{\mathbb{F}}([0, \infty)\times Z; \mathbb R^n)$ satisfies the integral version of (3.4):

then we call it an $L^2$-stable adapted solution of (3.4).

Lemma 3.2. Let Assumptions 3.1–3.3 be satisfied. Suppose that [A, C, E] is $L^2$-stable, then for any $\varphi\in L^2_{\mathbb F}(\mathbb R^n)$, Eq (3.4) admits a unique $L^2$-stable adapted solution (Y, Z, r).

The proof is similar to proof of Theorem A.2.2. of ^[19].

Lemma 3.3. Under Assumptions 3.1–3.3, the strong solution $X(\cdot)$ of the SDE (3.1) with $u(\cdot) = 0$ has the explicit expression:

where $\Lambda(\cdot)$ is the unique solution of the following matrix-valued SDE:

and $\Lambda(t)^{-1}$ exists, satisfying

The proof is similar to the proof of Theorem 6.14 of ^[26].

4.   Admissible control sets and stabilizability

4.1.   Stability

Regarding $L^2$-stability of the system [$A, C, E$] the following result holds:

Theorem 4.1. Let Assumptions 3.1–3.3 be satisfied. The system [A, C, E] is $L^2$-stable if and only if there exists a $K \in \mathbb{S}_+^n$ such that

In this case, the Lyapunov equation

admits a unique solution $K\in \mathbb S^n$ for any $\Psi\in \mathbb S^n$, which is given by

where $\Lambda$ is the solution of (3.7).

Proof. Necessity. Consider the following linear ODE on [0, $\infty$) for any fixed $\Psi\in\mathbb S^n$:

It is clear that (4.2) is uniquely solvable on [0, $\infty$). And to the solution $\Phi(t)$, we define a function

which is a solution of the following equation

on the interval [0, $\tau$] for any fixed $\tau > 0$. Clearly, we have $X(s) = \Lambda(s)x$, and $X(\cdot)$ is the solution of [$A, C, E$] with initial state $x$. Applying Itô's formula to $s\mapsto \langle \Phi_{\tau}(s)X(s), X(s)\rangle$, we have

Thus,

Since the system [$A, C, E$] is $L^2$-stable, it is easy to check that

Because $\Phi(t)$ is the solution to (4.2), we have for any $t > 0$,

Letting $t\rightarrow \infty$, we have

What's more, (4.1) holds when we take $\Psi = I_n$, and the corresponding $K\in\mathbb S_+^{n}$.

Sufficiency. Now we suppose that $X(\cdot) \equiv X(\cdot; x)$ is a solution of [$A, C, E$] with initial state $x$, and suppose $K\in\mathbb S_+^n$ satisfies (4.1). Applying Itô's formula to $s\mapsto \langle KX(s), X(s)\rangle$, we have for any $t > 0$,

Let $\lambda > 0$ be the smallest eigenvalue of $-\big(KA+A^\top K+C^\top KC+\int_ZE(\theta)^\top KE(\theta)\nu(d\theta)\big).$ We get that

which implies the $L^2$-stability of [$A, C, E$].

With regard to the nonhomogeneous system:

we have the following result.

Proposition 4.2. Let Assumptions 3.1–3.3 hold. If [$A, C, E$] is $L^2$-stable, then the solution $X(\cdot)\equiv X(\cdot; x, \varphi, \rho, \omega)$ of (4.3) is in $\mathcal X[0, \infty)$ for any $\varphi, \rho\in L^2_{\mathbb F}(\mathbb R^n), \omega\in L^{\nu, 2}_{\mathbb F}([0, \infty)\times Z; \mathbb R^n)$, and any initial state $x\in\mathbb R^n$. Moreover, there exists a positive constant $M$, which is independent of $x, \varphi, \rho\ and \ \omega$, such that

Proof. Since [$A, C, E$] is $L^2$-stable, there exists a $K\in \mathbb{S}_+^n$ such that

because of Theorem 4.1. Applying Itô's formula to $s\mapsto\langle KX(s), X(s)\rangle$, we have

for all $t > 0$. We define

According to Cauchy-Schwarz inequality, we obtain

where the $\lambda > 0$ is the smallest eigenvalue of $K$. Finally, by Gronwall's inequality, we get that

In summary, the conclusion of the integrability of $\mathbb E|X(t)|^2$ over $[0, \infty)$ can be obtained together with Young's inequality.

4.2.   Stabilizability

According to Proposition 4.2, the admissible control set $\mathscr U$ is nonempty (actually it equals to $L_{\mathbb F}^2(\mathbb R^n)$) for all $x\in\mathbb R^n$ if the system [$A, C, E$] is $L^2$-stable. Now we introduce the concept of $stabilizability$ to characterize the admissible control set. Denote by [$A, C, E; B, D, F$] the following controlled linear system:

Definition 4.1. [$A, C, E; B, D, F$] is said to be $L^2$-stabilizability if there exists a matrix $\Phi\in\mathbb R^{m\times n}$ such that [$A+B\Phi, C+D\Phi, E+F\Phi$] is $L^2$-stable, and we call $\Phi$ a $stabilizer$ of [$A, C, E; B, D, F$]. $\mathscr{T}\equiv\mathscr{T}$[$A, C, E; B, D, F$] is the set of all stabilizers of [$A, C, E; B, D, F$].

Proposition 4.3. Let the Assumptions 3.1–3.3 be satisfied. Suppose that $\Phi\in\mathscr T$[$A, C, E; B, D, F$]. Then for any $x\in\mathbb R^n$,

where $X_{\Phi}(\cdot; x, v)$ is the solution to the SDE

Proof. Since $\Phi\in\mathscr T$[$A, C, E; B, D, F$], the system [$A+B\Phi, C+D\Phi, E+F\Phi$] is $L^2$-stable. Let $X_{\Phi}(\cdot)\equiv X_{\Phi}(\cdot; x, v)$ be the corresponding solution to (4.4) and let $v\in L_{\mathbb F}^2(\mathbb R^m)$. By Proposition 4.2, $X_{\Phi}\in\mathcal X[0, \infty).$ We set

and let $X\in\mathcal X_{loc}[0, \infty)$ be the solution of

Then we have $X = X_{\Phi}\in\mathcal X[0, \infty)$. For the proof the reader is referred to Theorem 1.19 in ^[31]. And hence $u\in\mathscr U.$

On the flip side, let $X\in\mathcal X[0, \infty)$ be the corresponding solution to (4.5) and suppose that $u\in\mathscr U$. Then we denote the control $v$ by

Again by the uniqueness of solutions, the solution $X_{\Phi}$ of (4.4) coincides with $X$, and we get that $u$ admits a representation of the form $\Phi X_{\Phi}(\cdot; x, v)+v$.

The above result shows that the $L^2$-stabilizability is sufficient for the existence of an admissible control and gives an explicit description of the admissible control set. The following we will show a further result that the $L^2$-stabilizability is not only sufficient, but also necessary, for the non-emptiness of $\mathscr U$ for all $x\in\mathbb R^n$. We define

for $K\in\mathbb S^n$. We first present the following lemma.

Lemma 4.4. Let Assumptions 3.1–3.3 hold. Suppose that for each $T > 0$, the following differential Riccati equation

admits a solution $K_T\in C([0, T]; \mathbb S^n)$, which satisfies

Then the following algebraic Riccati equation:

has a solution $K$ if $K_T(0)$ converges to $K$ as $T\rightarrow \infty$ and $\mathcal N(K)$ is invertible.

Proof. We denote

for any fixed $T_1$ and $T_2$, which satisfy $0 < T_1 < T_2 < \infty$. Then we define

It is easy to conclude that both $K_1$ and $K_2$ are solutions of the following equatiion:

on the interval [0, $T_1$]. Thus, for any $s\in[0, T_1]$, it follows that

because of the uniqueness of solutions to ODEs, and the difference $\Delta = K_1-K_2$ satisfies $\Delta(0) = 0$. To interpret it, we get that

By assumption, $\Phi_1$ and $\Phi_2$ are continuous and hence bounded. Thus, for some positive constant $M$ which is independent of $\Delta$, we have

Then we get $\Delta(s) = 0$ for all $s\in[0, T_1]$ by Gronwall's inequality. Thus, the function $\Sigma:[0, \infty)\rightarrow \mathbb S^n$ can be defined by

Assume $\mathcal N(K)$ is invertible, since we define

it is easy to get that $\lim_{s\rightarrow \infty}\Theta(s) = \Theta_{\infty}$ if $\Sigma(T) = K_T(0)$ converges to $K$ as $T\rightarrow \infty$. On the other side, we get that

on the whole interval [0, $\infty$) since $\Sigma$ satisfies (4.9). Thus, we have

Letting $T\rightarrow \infty$, the desired result can be proved.

Theorem 4.5. Under Assumptions 3.1–3.3, the following statements are equivalent:

(i) $\mathscr U\neq\varnothing$ for all $x\in\mathbb R^n$;

(ii)$\mathscr T$[$A, C, E; B, D, F$] $\neq\varnothing$;

(iii)The following algebraic Riccati equation (ARE, for short) admits a positive solution $K\in\mathbb S_+^n$:

We define

If the above are satisfied and $K$ is a positive solution of (4.10), then we have $\Pi\in\mathscr T[A, C, E; B, D, F]$.

Proof. The implication (ⅱ) $\Rightarrow$ (ⅰ) has been proved in Proposition 4.3. Then we show the implication (ⅲ) $\Rightarrow$ (ⅱ). We assume $K\in\mathbb S_+^n$ is a solution of (4.10) and $\Pi$ has the expression (4.11), then we have

Thus, $\Pi$ is a stabilizer of [$A, C, E; B, D, F$] according to Theorem 4.1 and Definition 4.1.

We next show that (ⅰ) $\Rightarrow$ (ⅲ). Yet the general, we may assume $b = \sigma = 0$. Now take a series of $u_i\in\mathscr U(e_i), \; i = 1, ..., n$, and define $U = (u_1, ..., u_n)$ while $e_1, ..., e_n$ is the standard basis for $\mathbb R^n$. Then, we have $Ux\in\mathscr U$ for all $x\in\mathbb R^n$ because of the linearity of the state equation. Consider the following cost functional

while $\mathbb X\in L_{\mathbb F}^2(\mathbb R^{n\times n})$ solves the following matrix SDE:

We have for any $x\in\mathbb R^n$,

Now for a fixed but arbitrary $T > 0$, let us consider the optimal control problem in the finite time horizon [0, $T$] with state equation

and cost functional

According to Proposition 4.2 of the reference ^[25], the differential Riccati equation:

admits a unique solution $K_T\in C([0, T]; \mathbb S_+^n)$ such that

Because of the restriction $u|_{[0, T]}$ of $u$ to [0, $T$] belongs to $\mathscr U[0, T]\equiv L^2_{\mathbb F}(0, T; \mathbb R^m)$ for any $u\in\mathscr U,$ we have

And along with (4.12), it implies that

where $\Psi\triangleq\int_0^{\infty}\big[\mathbb X(t)^\top\mathbb X(t)+U(t)^\top U(t)\big]dt$. On the flip side, the restriction $u|_{[0, T]}$ of $u\in L^2_{\mathbb F}(0, T; \mathbb R^m)$ also belongs to $L^2_{\mathbb F}(0, T'; \mathbb R^m)$ for any fixed $T' > T > 0$. Thus, we have

which in turn gives

Combining (4.13) and (4.14), since $K_T\in C([0, T]; \mathbb S_+^n),$ it is easy to get that

Thus, it follows that $K_T(0)$ converges increasingly to some $K\in\mathbb S_+^n$ as $T\nearrow\infty.$ By Lemma 4.4, the limit matrix $K$ is a solution of the ARE (4.10).

5.   Solvability and the algebraic Riccati equation

According to Theorem 4.5, we have proved that $\mathscr U\neq\varnothing$ for all $x\in\mathbb R^n$ is equivalent to $\mathscr T$ [$A, C, E; B, D, F$] $\neq\varnothing$. Thus, it is reasonable to give the following assumption:

(A) System [$A, C, E; B, D, F$] is $L^2$-stabilizable, i.e., $\mathscr T[A, C, E; B, D, F]\neq\varnothing.$

Definition 5.1. For an initial state $x\in\mathbb R^n$, an element $u^*\in\mathscr U$ is called an open-loop optimal control of (SLQ)$_{\infty}$ if

Problem (SLQ)$_{\infty}$ is said to be (uniquely) open-loop solvable at x if an open-loop optimal control (uniquely) exists for $x$. If it is (uniquely) open-loop solvable at all $x\in\mathbb R^n$, Problem (SLQ)$_{\infty}$ is said to be (uniquely) open-loop solvable.

Definition 5.2. A pair $(\Phi^*, v^*)\in\mathscr T[A, C, E; B, D, F]\times L^2_{\mathbb F}(\mathbb R^m)$ is called a closed-loop optimal strategy of Problem (SLQ)$_{\infty}$ if

for all $(x, \Phi, v)\in\mathbb R^n\times\mathscr T[A, C, E; B, D, F]\times L^2_{\mathbb F}(\mathbb R^m)$, where $X^*$ and $X$ are the closed-loop state processes corresponding to $(x, \Phi^*, v^*)$ and $(x, \Phi, v)$, respectively. Problem (SLQ)$_{\infty}$ is said to be ($uniquely$) closed-loop $solvable$ if a closed-loop optimal strategy (uniquely) exists. The outcome

of a closed-loop strategy ($\Phi, v$) is called a closed-loop control for the initial state $x$, where $X$ is the closed-loop state process corresponding to :

Remark 5.1. From Proposition 4.3, $\mathscr U$ is composed of closed-loop controls for all $x$ when (A) holds. In Definition 5.2, the outcome $u^*\equiv\Phi^*X^*+v^*$ of a closed-loop optimal strategy $\left(\Phi^*, v^*\right)$ is an open-loop optimal control for the initial state $X^*(0)$, which implies that closed-loop solvability is sufficient to open-loop solvability.

According to the definition of the open-loop and closed-loop solvabilities in ^[17], it is easy to get that for finite horizon cases, closed-loop solvability implies open-loop solvability, whereas open-loop solvability does not necessarily imply closed-loop solvability. However, when it comes to infinite horizon, the open-loop and closed-loop solvabilities are equivalent, and both are equivalent to the existence of a $stabilizing \ solution$ to a $generalized \ algebraic \ Riccati \ equation$ which we will prove later.

Definition 5.3. Let Assumptions 3.1–3.3 be satisfied. The following constrained a generalized algebraic Riccati equation (ARE)

If there exists a $\Theta\in\mathbb R^{m\times n}$ such that the matrix

is a stabilizer of [$A, C, E; B, D, F$], then the solution $K\in\mathbb S^n$ of (5.1) is said to be $stabilizing$.

Remark 5.2. Because of the properties of the Moore-Penrose pseudoinverse (see Lemma 2.2 and Remark 2.1), one has

if $K$ is a solution (not necessarily stabilizing) to the ARE (5.1) and $\Phi$ is defined by (5.2).

5.1.   A study of Problem (SLQ)$_{\infty}^0$

In order to solve Problem (SLQ)$_{\infty}$, we first discuss Problem (SLQ)$_{\infty}^0$ in this section. Assume that the nonhomogeneous terms $b, \sigma, h, q$, and $\rho$ are all zero, and we assume that the system [$A, C, E$] is $L^2$-stable (i.e., 0 $\in\mathscr T[A, C, E; B, D, F]$) for the sake of simplicify.

Proposition 5.1. Let Assumptions 3.1–3.3 hold. Suppose that [$A, C, E$] is $L^2$-stable. Then there exist a bounded self-adjoint linear operator $H_2: L^2_{\mathbb F}(\mathbb R^m)\rightarrow L^2_{\mathbb F}(\mathbb R^m)$, a bounded linear operator $H_1: \mathbb R^n\rightarrow L^2_{\mathbb F}(\mathbb R^m)$, a matrix $H_0\in\mathbb S^n$, and $\tilde{u}\in L^2_{\mathbb F}(\mathbb R^m), \ \tilde{x}\in\mathbb R^n, \ g\in\mathbb R$ such that for any $(x, u)\in\mathbb R^n\times L^2_{\mathbb F}(\mathbb R^m)$,

In particular, in the case of Problem $({\rm{SLQ}}) _{\infty}^0 (i.e., b, \sigma, h, q, \rho = 0),$

where

and

Proposition 5.2. Under Assumptions 3.1–3.3, if [$A, C, E$] is $L^2$-stable, then we have the following conclusion:

(i) Problem (SLQ)$_{\infty}$ is open-loop solvable at x if and only if $H_2\geq0$ (i.e., $H_2$ is a positive operator) and $H_1x+\tilde{u}\in\mathscr R(H_2).$ In this instance, $u^*$ is an open-loop optimal control for the initial state x if and only if

(ii) If Problem (SLQ)$_{\infty}$ is open-loop solvable, then so is Problem (SLQ)$_{\infty}^0$.

(iii) If Problem (SLQ)$_{\infty}^0$ is open-loop solvable, then there exists a $U^*\in L^2_{\mathbb F}(\mathbb R^{m\times n})$ such that for any $x\in\mathbb R^n, \ U^*x$ is an open-loop optimal control of Problem (SLQ)$_{\infty}^0$ for the initial state x.

Proof. (ⅰ) By definition, a process $u^*\in L^2_{\mathbb F}(\mathbb R^m)$ is an open-loop optimal control for the initial state $x$ if and only if

According to Proposition 5.1, we have

Thus, (5.3) is equivalent to

which in turn is equivalent to

So (ⅰ) is proved.

For conclusion (ⅱ), since Problem (SLQ)$_{\infty}$ is open-loop solvable, it is easy to get that $H_2\geq0$ and $H_1x+\tilde{u}\in\mathscr R(H_2)$ for all $x\in\mathbb R^n$ according to (ⅰ). Then we take $x = 0$, it is tempting to conclude that $\tilde{u}\in\mathscr R(H_2)$, and hence $H_1x\in\mathscr R(H_2)$ for all $x\in\mathbb R^n$. Finally, we obtain the open-loop solvability of Problem (SLQ)$_{\infty}^0$ by using (ⅰ) again.

For conclusion (ⅲ), assume $e_1, ..., e_n$ is the standard basis for $\mathbb R^n$. It is easy to get that $U^*\triangleq(u_1^*, ..., u_n^*)$ has the desired properties since $u_i^*$ ia an open-loop optimal control of Problem (SLQ)$_{\infty}^0$ for the initial state $e_i$.

5.1.1.   A finite horizon approach

Let $\Lambda$ be the solution of (3.7), and the matrix

is well-defined if [A, C, E] is $L^2$-stable. Thus, we can consider the following LQ problem over the finite time horizon [$0, T$] for any $T > 0$.

Problem (SLQ)$_T^0$. On the finite time horizon [$0, T$], we assume $X_T(t)$ is a solution of the state equation:

then for any given $x\in\mathbb R^n$, find a $u^*\in\mathscr U[0, T]$ such that the cost functional

is minimized over $\mathscr U [0, T]$.

Proposition 5.3. Let Assumptions 3.1–3.3 be satisfied. If [A, C, E] is $L^2$-stable, since we use $V_T^0(x)$ to denote the value function of Problem (SLQ)$_T^0$, and use $V^0(x)$ to denote the value function of Problem (SLQ)$^0_{\infty}$, it is easy to get the following conclusions:

(i) For any $x\in\mathbb R^n$ and $u\in\mathscr U[0, T]$,

where $u_e\in L^2_{\mathbb F}(\mathbb R^m)$ is the zero-extension of u:

(ii) If there exists a $\varepsilon > 0$ such that

then

(iii) $\lim_{T\rightarrow \infty}V_T^0(x) = V^0(x)$ for all $x\in\mathbb R^n$.

Proof. (ⅰ) Let $X_T$ be the solution of (5.4) and $X$ be the solution of

for a fixed $x\in\mathbb R^n$ and an arbitrary $u\in\mathscr U[0, T].$ Thus, we have that

Noting that for $t\geq T, \ \Lambda(t)\Lambda(T)^{-1}$ and $\Lambda(t-T)$ have the same distribution and are independent of $\mathcal F_T$, we have

It follows that

(ⅱ) Taking $x = 0$ in (5.7), we obtain

which proves (ⅱ).

Finally let us prove (ⅲ). According to (5.7), it is easy to obtain

Let $V_T^0(x)$ be the infimum of $J_T^0(x; u)$ over $u\in\mathscr U[0, T]$, thus

On the flip side, when $V^0(x) > -\infty,$ for any given $\varsigma > 0$, we can find a $u^{\varsigma}\in L_{\mathbb F}^2(\mathbb R^m)$ such that

where $X^{\varsigma}$ is the solution of

Since by Proposition 4.2 $X^{\varsigma}\in\mathcal X[0, \infty)$, it is easy to get that for large $T > 0$,

Now we use $u_T^{\varsigma}$ to be the restriction of $u^{\varsigma}$ to $[0, T]$, thus we have

Since we combine (5.9) and (5.10), it is not hard to get that for large $T > 0$,

To summarize, we have that $V_T^0(x)\rightarrow V^0(x)$ as $T\rightarrow \infty.$ The case when $V^0(x) = -\infty$ can also be proved by a similar method.

Proposition 5.2(ⅰ) implies that if [A, C, E] is $L^2$-stable and the operator $H_2$ is uniformly positive, then Problem $({\rm SLQ})^0_{\infty}$ is uniquely open-loop solvable and the unique optimal control is given by

And if we substitute the optimal control $u_x^*$ into the cost functional, it is easy to get

Notice that $H_0-H_1^*H_2^{-1}H_1$ is a matrix.

Theorem 5.4. Let Assumptions 3.1–3.3 hold. If [A, C, E] is $L^2$-stable and (5.5) holds for some $\varepsilon > 0$, then we have the following conclusions

(i) the matrix $K\triangleq H_0-H_1^*H_2^{-1}H_1$ solves the ARE

(ii) the matrix $\Phi\triangleq-\mathcal N(K)^{-1}\mathcal M(K)$ is a stabilizer of [A, C, E; B, D, F];

(iii) the unique open-loop optimal control of Problem (SLQ)$_{\infty}^0$ for the initial state $x$ is given by

where $X_{\Phi}(\cdot; x)$ is the solution to the following equation:

Proof. By Proposition 5.3(ⅱ), (5.6) holds. According to Proposition 4.2 of ^[25], it is easy to get that for any $T > 0$, the differential Riccati equation

admits a unique solution $K_T\in C([0, T]; \mathbb S^n)$ such that

From Proposition 5.3(ⅲ), we see that

Thus the conclusion (ⅰ) can be proved by Lemma 4.4.

Then we will prove the conclusions (ⅱ) and (ⅲ). Firstly, we fix an $x\in\mathbb R^n$ and let $(X_x^*, u_x^*)$ be the corresponding optimal pair of Problem (SLQ)$^0_{\infty}.$ Applying Itô's formula to $t\rightarrow \langle KX_x^*(t), X^*_x(t)\rangle$ and noting that $\lim_{t\rightarrow \infty}\langle KX_x^*(t), X^*_x(t)\rangle = 0,$ we have

On the flip side, we have

Adding the last two equations yields

Since $\mathcal N(K) = R+D^\top KD+\int_ZF(\theta)^\top KF(\theta)\nu(d\theta) > 0,$ it is easy to get that

and hence $X_x^*$ is the solution of

Since $X_x^*\in\mathcal X[0, \infty)$ and $x$ is arbitrary, it is not hard to get that $\Phi$ is a stabilizer of [$A, C, E; B, D, F$], and the rest of the proof is clear.

5.1.2.   Open-loop and closed-loop solvability

As we have proved in Proposition 5.2, the condition $H_2\geq0$ is merely necessary for the existence of an open-loop optimal control, and according to Theorem 5.4, the uniform positivity condition (5.5) is only sufficient. In order to find the connection between the above, let us consider the following cost functional for $\varsigma > 0$:

Now we denote by Problem (SLQ)$_{\infty}^{0, \varsigma}$ the problem of minimizing $J_{\varsigma}^0(x; u)$ subject to the state equation

and by $V_{\varsigma}^0(x)$ the corresponding value function. Since we assume $H_2\geq0$, and then the operator $H_2+\varsigma I$ is uniformly positive for all $\varsigma > 0$, it is easy to apply Theorem 5.4 to Problem (SLQ)$_{\infty}^{0, \varsigma}$. Thus we will obtain a characterization for the value function $V^0(x)$ of Problem (SLQ)$^0_{\infty}$ while $\varsigma\rightarrow0.$

Theorem 5.5. Let Assumptions 3.1–3.3 be satisfied. If [A, C, E] is $L^2$-stable, and Problem (SLQ)$^0_{\infty}$ is open-loop solvable, then the generalized ARE (5.1) admits a stabilizing solution $K\in\mathbb S^n.$ Moreover, $V^0(x) = \langle Kx, x\rangle$ for all $x\in\mathbb R^n.$

Proof. According to Proposition 5.2, the open-loop solvability of Problem (SLQ)$^0_{\infty}$ implies $H_2\geq0.$ Thus, for any $\varsigma > 0$, the ARE:

admits a unique solution $K_{\varsigma}\in\mathbb S^n$ such that $V^0_{\varsigma}(x) = \langle K_{\varsigma}x, x\rangle$ for all $x\in\mathbb R^n$ according to Theorem 5.4. Then we define a stabilizer of [A, C, E; B, D, F]:

Thus, it follows that the unique open-loop optimal control $u_{\varsigma}^*(\cdot; x)$ of Problem (SLQ)$_{\infty}^{0, \varsigma}$ for the initial state $x$ is given by

since $\Pi_{\varsigma}$ is a solution of the matrix SDE:

Then we denote $U^*\in L^2_{\mathbb F}(\mathbb R^{m\times n})$ to be a process which is defined in Proposition 5.2(ⅲ). According to the definition of the value function, it is easy to get that for any $x\in\mathbb R^n$ and $\varsigma > 0,$

Equation (5.13) implies that for any $x\in\mathbb R^n$ and $\varsigma > 0,$

From (5.14)) we see that $K\equiv\lim_{\varsigma\rightarrow 0}K_{\varsigma}$ exists and $V^0(x) = \langle Kx, x\rangle$ for all $x\in\mathbb R^n$.

From (5.15) we see that the family of positive semi-definite matrices:

is bounded, and the system [$A+B\Phi_{\varsigma}, C+D\Phi_{\varsigma}, E+F\Phi_{\varsigma}$] is $L^2$-stable for $\Phi_{\varsigma}$ is a stabilizer of [$A, C, E; B, D, F$]. According to Theorem 4.1, we have

It follows that

The above, together with the boundedness of $\left\lbrace \Theta_{\varsigma}\right\rbrace _{\varsigma > 0}$, shows that

for some constant $J > 0.$ Noting that (5.16) implies the boundedness of $\left\lbrace \Phi_{\varsigma}\right\rbrace _{\varsigma > 0}$, we may choose a sequence $\left\lbrace \varsigma_k\right\rbrace _{k = 1}^{\infty}\subseteq(0, \infty)$ with $\lim_{k\rightarrow \infty}\varsigma_k = 0$ such that $\Phi\equiv\lim_{k\rightarrow \infty}\Phi_{\varsigma_k}$ exists. Observe that

Thus, we have by Lemma 2.2 that

for some $\Theta\in\mathbb R^{m\times n}.$ Notice that by (5.12), $\mathcal M(K_{\varsigma})^\top = -\Phi_{\varsigma}^\top[\mathcal N(K_{\varsigma})+\varsigma I].$ Thus (5.11) can be written as

Now let $k\rightarrow \infty$ along $\left\lbrace \varsigma_k\right\rbrace _{k = 1}^{\infty}$, then $\varsigma_k\rightarrow0$ in the above. Thus we have

which, together with (5.17) and (5.18), implies that $K$ solves the generalized ARE (5.1). Following we will show that $K$ is a stabilizing solution, and we need only to show $\Phi\in\mathscr T$[$A, C, E; B, D, F$]. To prove it, we define $\Pi$ as a solution of the matrix SDE:

Since $\Phi_{\varsigma_k}\rightarrow \Phi$ as $k\rightarrow \infty$, we get $\Pi_{\varsigma_k}(t)\rightarrow \Pi(t)$, a.s.for all $t\geq 0$. By Fatou's lemma and (5.15), we get that

This implies $\Phi\Pi\in L_{\mathbb F}^2(\mathbb R^{m\times n}).$ Thus, by Proposition 4.2, it is easy to get that $\Pi\in L_{\mathbb F}^2(\mathbb R^{m\times n}).$ Consequently, $\Phi\in\mathscr T[A, C, E; B, D, F].$

According to Theorem 5.5, if [$A, C, E$] is $L^2$-stable, then the existence of a stabilizing solution to the generalized ARE is necessary for the open-loop solvability of Problem (SLQ)$_{\infty}^0.$ Later we will show that the converse is also true.

Proposition 5.6. Let Assumptions 3.1–3.3 hold. Suppose that the generalized ARE (5.1) admits a stabilizing solution $K\in\mathbb S^n$. Then Problem ${\rm(SLQ)} _{\infty}^0$ is closed-loop solvable.

Proof. Let $X(\cdot)\equiv X(\cdot; x, u)$ be the solution of the following equation:

for arbitrary fixed initial state $x$ and admissible control $u\in\mathscr U$. Applying Itô's formula to $t\mapsto\langle KX(t), X(t)\rangle$, it is easy to get that

Thus, we have

By the extended Schur's lemma (Lemma 2.3), we obtain

Thus,

On the flip side, we can choose a $\Theta\in\mathbb R^{m\times n}$ such that the matrix

is a stabilizer of [$A, C, E; B, D, F$] since $K$ is stabilizing. By Remark 5.2,

Thus, for any $x^*\in\mathbb R^n$,

Following we will show that $(\Phi^*, 0)$ is a closed-loop optimal strategy of Problem (SLQ)$_\infty^0$. Firstly, let $X^*$ be the closed-loop state process corresponding to $(x, \Phi^*, 0)$:

Then, applying Itô's formula to $t\mapsto\langle KX^*(t), X^*(t)\rangle$ and using (5.20), we obtain

Combining the last equation with (5.19), the conclusion is proved.

Remark 5.3. According to the proof of Proposition 5.6, it is clear that if $K$ is a stabilizing solution to the generalized ARE (5.1), then $V^0(x) = \langle Kx, x\rangle$ for all $x\in\mathbb R^n$.

Combining Remark 5.1, Theorem 5.5, and Proposition 5.6, we have the following result.

Theorem 5.7. Let Assumptions 3.1–3.3 be satisfied. Suppose that [$A.C, E$] is $L^2$-stable. Then the following statements are equivalent:

(i) Problem (SLQ)$_\infty^0$ is open-loop solvable;

(ii) Problem (SLQ)$_\infty^0$ is closed-loop solvable;

(iii) The generalized ARE (5.1) admits a unique stabilizing solution.

5.1.3.   Nonhomogeneous problems

Following we will focus our attention to Problem (SLQ)$_\infty$, and show a result that is similar to Theorem 5.7.

Denote by $\Omega$ a stabilizer of [$A, C, E; B, D, F$], and define

Then we use $\hat{X}(\cdot; x, v)$ to denote the solution of the following state equation

with regard to $x$ and $v$. And we have the cost functional

It is easy to find that [$\hat{A}, \hat{C}, \hat{E}$] is $L^2$-stable. And we denote by (SLQ)$_{\infty}^{'}$ the problem of minimizing (5.23) subject to (5.22). According to Proposition 4.3, we have the following obvious conclusions about Problem (SLQ)$_{\infty}^{'}$.

Proposition 5.8. Under Assumptions 3.1–3.3, let $\Omega$ be a stabilizer of [A, C, E; B, D, F]. Then

(i) Problem (SLQ)$_{\infty}^{'}$ is open-loop solvable at $x\in\mathbb R^n$ if and only if Problem (SLQ)$_{\infty}$ is so. In this case, $v^*$ is an open-loop optimal control of Problem (SLQ)$_{\infty}^{'}$ if and only if $u^*\triangleq v^*+\Omega\hat{X}(\cdot; x, v^*)$ is an open -loop optimal control of Problem (SLQ)$_{\infty}$;

(ii) Problem (SLQ)$_{\infty}^{'}$ is closed-loop solvable if and only if Problem (SLQ)$_{\infty}$ is so. In this case, $(\Omega^*, v^*)$ is a closed-loop optimal strategy of Problem (SLQ)$_{\infty}^{'}$ if and only if $(\Omega^*+\Omega, v^*)$ is a closed-loop optimal strategy of Problem (SLQ)$_{\infty}$.

Following we will show the main result of this section.

Theorem 5.9. Under Assumptions 3.1–3.3 and (A), we have the following equivalent statements:

(i) Problem (SLQ)$_\infty$ is open-loop solvable;

(ii) Problem (SLQ)$_\infty$ is closed-loop solvable;

(iii) The generalized ARE (5.1) admits a stabilizing solution $K\in\mathbb S^n$, and the BSDE

admits an $L^2$-stable adapted solution ($\eta, \zeta_1, \zeta_2$) such that

In the above case, all closed-loop optimal strategies ($\Phi^*, v^*$) are given by

where $\Theta\in\mathbb R^{m\times n}$ and $w\in L^2_{\mathbb F}(\mathbb R^m)$ are arbitrary. And it turns out that $\Phi^*\in\mathscr T[A, C, E; B, D, F]$. Then every open-loop optimal control $u^*$ for the initial state $x$ admits a closed-loop representation:

where ($\Phi^*, v^*$) is a closed-loop optimal strategy of Problem (SLQ)$_{\infty}$ and $X^*$ is the corresponding closed-loop state process. Moreover,

Proof. Since Remark 5.1 holds, it is obvious that the implication (ⅱ) $\Rightarrow$ (ⅰ) follows.

To prove the implication (ⅰ) $\Rightarrow$ (ⅲ), we will consider Problem (SLQ)$_{\infty}^{'}$ firstly. According to Proposition 5.8(ⅰ), it is open-loop solvable. Since the system [$\hat{A}, \hat{C}, \hat{E}$] is $L^2$-stable, it is easy to get that by Proposition 5.2 and Theorem 5.5, the ARE

admits a (unique) stabilizing solution $K\in\mathbb S^n$. Choose a $\Gamma\in\mathbb R^{m\times n}$ such that

is a stabilizer of [$\hat{A}, \hat{C}, \hat{E}; B, D, F$]. According to Remark 5.2 and (5.21),

It follows that $\mathscr R(\mathcal M(K))\subseteq\mathscr R(\mathcal N(K))$. Substituting (5.21) into the first equation of (5.28) gives

Therefore, $K$ solves the ARE (5.1). Since $\Omega^*+\Omega$ is a stabilizer of [$A, C, E; B, D, F$], it is easy to get that $K$ is stabilizing according to (5.29) and Lemma 2.2.

Now choose $\Theta\in\mathbb R^{m\times n}$ such that the matrix

is a stabilizer of [$A, C, E; B, D, F$], and think over the following BSDE on [0, $\infty$):

Since [$A+B\Phi, C+D\Phi, E+F\Phi$] is $L^2$-stable, it is not hard to get the conclusion that (5.30) admits a unique $L^2$-stable adapted solution ($\eta, \zeta_1, \zeta_2$) by Lemma 3.2. Now we let $X(\cdot)\equiv X(\cdot; x, u)$ be the corresponding state process for fixed but arbitrary $x$ and $u\in\mathscr U$. Applying Itô's formula to $t\mapsto\langle KX(t), X(t)\rangle$, then it follows that

Applying Itô's formula to $t\mapsto\langle \eta(t), X(t)\rangle$ yields

Denote $\varpi(t) = B^\top\eta(t)+D^\top[\zeta_1(t)+K\sigma(t)]+\int_Z F(t, \theta)^\top[\zeta_2(t)+Kh(t, \theta)]\nu(d\theta)+\rho(t)$, then we have

Now we assume that $u^*$ is an open-loop optimal control of Problem (SLQ)$_{\infty}$ for the initial state $x$, and $X_{\Phi}(\cdot; x, v)$ is a solution to the following SDE:

By Proposition 4.3, we have the conclusion that any admissible control with respect to the initial state $x$ is of the form

Thus $u^* = \Phi X_{\Phi}(\cdot; x, v^*)+v^*$ for some $v^*\in L^2_{\mathbb F}(\mathbb R^m)$. Then we have

Since we take $u = \Phi X_{\Phi}(\cdot; x, v)+v$, it is easy to get that

thus, according to (5.31) and (5.32), it follows that for any $v\in L^2_{\mathbb F}(\mathbb R^m)$,

The above inequality implies that $v^*$ is a minimizer of the functional

Thus, it follows that

By Lemma 2.2, we have

Observing that

we obtain

Thus, it follows that ($\eta, \zeta_1, \zeta_2$) is an $L^2$-stable adapted solution of the BSDE (5.24). And from Remark 2.1, we have

Therefore, since we replace $u$ by $u^* = \Phi X_{\Phi}(\cdot; x, v^*)+v^*$ in (5.31), it is not hard to get that

Finally, to prove the implication (ⅲ)$\Rightarrow$(ⅱ), we firstly take an arbitrary ($x, u$)$\in\mathbb R^n\times\mathscr U$ and let $X(\cdot)\equiv X(\cdot; x, u)$ be the corresponding state process. According to (5.31), we have

Let ($\Phi^*, v^*$) be defined by (5.26). Then by Lemma 2.2 and Remark 2.1, we have

Then we substitute the above into (5.33) and complete the square, it follows that

Thus, owing to $\mathcal N(K)\geq0$ and $\Phi^*$ is a stabilizer of [$A, C, E; B, D, F$], it is not hard to get that

which shows that ($\Phi^*, v^*$) is a closed-loop optimal strategy of Problem (SLQ)$_{\infty}$.

Finally, assume that ($\check{\Phi}, \check{v}$) is an another closed-loop optimal strategy, and $\check{X}$ is the solution of the following closed-loop system

Since we denote by $\check{u} = \check{\Phi}\check{X}+\check{v}$ the outcome of $(\check{\Phi}, \check{v})$, it is obvious that

Now (5.34) and (5.35) imply that

from which we have that

Multiplying the above by $\mathcal N(K)^{\frac1{2}}$, we obtain

Since $\check{\Phi}, \Phi^*, \check{v},$ and $v^*$ are independent of $x$, and (5.36) holds for all $x\in\mathbb R^n$, it is easy to get that for any $x\in\mathbb R^n$, the solution $X_0$ of

satisfies $\mathcal N(K)(\check{\Phi}-\Phi^*)X_0 = 0$. It follows that $\mathcal N(K)(\check{\Phi}-\Phi^*) = 0$ and hence $\mathcal N(K)(\check{v}-v^*) = 0$. Now we have

According to Lemma 2.2, $(\check{\Phi}, \check{v})$ must be of the form (5.26). Since we denote $\bar{X}$ the corresponding optimal state process, if $\bar{u}$ is an open-loop optimal control for the initial state $x$, it is not hard to get that

i.e.,

According to Lemma 2.2, there exists a $w\in L^2_{\mathbb F}(\mathbb R^m)$ such that

where $\Theta\in\mathbb R^{m\times n}$ satisfies

which shows that $\bar{u}$ has the closed-loop representation (5.27).

6.   Conclusions

This paper mainly studies a kind of stochastic LQ optimal control problem with jumps in an infinite horizon. Firstly, we prove that the $L^2$-stabilizability of our control system is equivalent to the non-emptiness of the admissible control set for all initial state and is also equivalent to the existence of a positive solution to some integral ARE. Then we conclude the equivalence of the open-loop and closed-loop solvability and the existence of a stabilizing solution of the associated generalized ARE. Finally, we explore that any open-loop optimal control for the initial state admits a closed-loop representation, and the representation is obtained.

Acknowledgments

The authors would like to thank anonymous referees for helpful comments and suggestions which improved the original version of the paper. Q. Meng was supported by the Key Projects of Natural Science Foundation of Zhejiang Province (No. Z22A013952) and the National Natural Science Foundation of China (No. 12271158 and No. 11871121). Maoning Tang was supported by the Natural Science Foundation of Zhejiang Province (No. LY21A010001).

Conflict of interest

The authors declare that they have no competing interests.