On the equilibrium strategy of linear-quadratic time-inconsistent control problems

1.   Introduction

The history of the time-inconsistent problem can be traced back to the research of Hume ^[1] and Smith ^[2]. However, it was not until 1955 that Strotz ^[3] first established a mathematical formulation for the hyperbolic discounting Ramsey problem. Since then, extensive studies on time-inconsistent problems have emerged due to their importance and wide-ranging applications. Bj$\ddot{\text{o}}$rk et al. ^[4,5,6] explored time-inconsistent stochastic control and its application in finance and economics in both discrete and continuous time; they derived the Hamilton-Jacobi-Bellman (HJB) equations using systematic methods and obtained verification theorems. Ekeland et al. ^[7,8] researched feedback equilibrium control for time-inconsistent problems as well as time-consistent portfolio management. Yong and collaborators ^{[9,10,11,12,13]} studied time-inconsistent optimal control problems within the framework of game theory, obtaining results related to time-consistent equilibrium control by discretizing time intervals. L$\ddot{\text{u}}$ et al. ^[14,15] examined stochastic linear-quadratic time-inconsistent control problems with both definite and indefinite cost functional. Hu et al. ^[16,17] investigated the existence and uniqueness of open-loop equilibrium control for stochastic linear-quadratic time-inconsistent control problems by employing a flow of forward and backward stochastic differential equations. Ni et al. ^[18] researched mixed equilibrium strategies for linear-quadratic time-inconsistent control problems.

Recently, Peng and collaborators ^[19,20] explored the equivalence between the equilibrium control of deterministic linear-quadratic time-inconsistent problems, the solvability of the two-point boundary value problems, and the Riccati-type equations in a closed-loop framework. They established the existence and uniqueness of time-consistent equilibrium control for deterministic linear-quadratic time-inconsistent models by discussing the solvability of Riccati-type equations. Additionally, related studies ^[21,22] have also been conducted in the context of open-loop.

It is well known that the Pontryagin-type maximum principle serves as one of the most important tools for solving classical control problems (time-consistent control problems). Furthermore, suppose the linear-quadratic control problem admits both closed-loop and open-loop optimal control, and that the open-loop optimal control has a closed-loop representation. This implies that the representation must stem from the closed-loop optimal control ^{[23,24,25,26]}. Consequently, it is natural to pose the following questions: (1) Does an equilibrium maximum principle, similar to the Pontryagin maximum principle, exist for time-inconsistent control problems? (2) If the linear-quadratic time-inconsistent control problem allows for both open-loop and closed-loop equilibrium strategies, and the open-loop equilibrium strategy has a closed-loop representation, does this mean it arises from the closed-loop equilibrium strategy? (3) What is the connection between the open-loop equilibrium strategy, the closed-loop equilibrium strategy for time-inconsistent problems, and the optimal solution for the control problem?

In this paper, we primarily focus on the equilibrium maximum principle for the open-loop equilibrium strategy in linear-quadratic time-inconsistent control problems. We explore the equivalence among the open-loop equilibrium strategy, two-point boundary value problems, and equilibrium Riccati equations. Additionally, we provide examples to highlight the essential difference between the open-loop and closed-loop equilibrium strategies in time-inconsistent control problems and the optimal control in classical control problems. Our approach is inspired by recent developments in linear-quadratic time-inconsistent differential games and control problems ^[19,21,22].

The remainder of this paper is organized as follows: In Section 2, we formulate the mathematical model for the linear-quadratic time-inconsistent control problem and present some necessary assumptions and notations that will be frequently used throughout the paper. Section 3.1 is dedicated to deriving the equilibrium maximum principle for the open-loop equilibrium strategy. In Section 3.2, we characterize the relationships between the open-loop equilibrium strategy, two-point boundary value problems, and the equilibrium Riccati equation. Section 4 addresses the relationships among the open-loop equilibrium, the closed-loop equilibrium for time-inconsistent control problems, and optimal control in classical control problems.

2.   Problem setting

Let $L_0 > 0$. The following function spaces and notations are to be used throughout this article:

For any $(\nu, {z})\in[0, L_0]\times \mathbb R^n$, we research the following linear control system:

with an LQ cost functional

Here, $\mathcal{A}(\cdot)$, $\mathcal{B}(\cdot)$, $\mathcal{Q}(\cdot, \cdot)$, $\mathcal{M}(\cdot, \cdot)$, $\mathcal{G}(\cdot)$ are appropriate matrix-valued functions. We are making the following hypotheses.

(S1) $\mathcal{A}\in L^1\left([0, L_0];\mathbb{R}^{n\times n}\right)$, $\mathcal{B}\in L^2\left([0, L_0];\mathbb{R}^{n\times m}\right)$.

(S2) $\mathcal{M}\in C\left([0, L_0]\times[0, L_0];\mathbb{R}^{m\times m}\right)$ is a positive definite matrix-valued functions.

(S3) $\mathcal{Q}\in C^1\left([0, L_0]\times[0, L_0];\mathbb{R}^{n\times n}\right)$ and $\mathcal{G}\in C^1\left([0, L_0];\mathbb{R}^{n\times n}\right)$ are symmetric matrix-valued functions.

(S4) For $0\leq \nu\leq \mu \leq L_0$, $\mathcal{G}(\nu)$, $\mathcal{Q}(\nu, \mu)$, $\mathcal{M}_\nu(\nu, \mu)$, $\dot{\mathcal{G}}(\nu)$, and $\mathcal{Q}_\nu(\nu, \mu)$ are positive semi-definite. Here, $\mathcal{M}_\nu(\nu, \mu) = \frac{\partial\mathcal{M}}{\partial \nu}(\nu, \mu), \; \dot{\mathcal{G}}(\nu) = \frac{d\mathcal{G}}{d \nu}(\nu), \; \mathcal{Q}_\nu(\nu, \mu) = \frac{\partial\mathcal{Q}}{\partial \nu}(\nu, \mu).$

It is obvious that for any $(\nu, {z})\in[0, L_0]\times \mathbb R^n$ and $u(\cdot)\in L^2\left([0, L_0];\mathbb{R}^m\right)$, the linear control system (2.1) admits a unique solution under (S1). Specifically, we define $\phi_\mathcal{A}(\nu, \mu)$ as a matrix solution of the differential equation $\frac{d }{d\mu}\mathcal{Z(\mu)} = \mathcal{A}(\mu)\mathcal{Z(\mu)}$ with $\phi_\mathcal{A}(\nu, \nu) = I_{n\times n}$ in this paper. Further, if (S2) and (S3) are also assumed, then the LQ cost functional (2.2) is well defined for any ${z}\in \mathbb R^n$, $\mu, \nu\in[0, L_0]$ and $u(\cdot)\in L^2\left([0, L_0];\mathbb{R}^m\right)$.

$\mathbf{Problem}\; (\mathbf{I})$. For any initial pair $(\nu, z)$, we want to find an $\bar u(\cdot)\in \ L^2\left([0, L_0]; \mathbb{R}^m\right)$ such that the cost functional $\mathbb{F}(\nu, z; u(\cdot))$ is minimized.

Note that the coefficients $\mathcal{Q}$, $\mathcal{M}$, and $\mathcal{G}$ in the objective functional (2.2) explicitly depend on the initial time $\nu$. This implies that the cost functional will change over time, leading to time-inconsistency, which makes it quite different from classical optimal control problems. We refer to Problem (Ⅰ) as linear-quadratic time-inconsistent control problem.

Let $\bar u(\cdot)\in L^2\left([0, L_0];\mathbb{R}^m\right)$ be a given control, and $\bar {\mathcal{Z}}(\cdot)$ be the state trajectory corresponding to the control $\bar u$ and fixed initial pair $(0, z_0)$, i.e., $\bar {\mathcal{Z}}(\cdot)\equiv \mathcal{Z}_{0, z_0}^{\bar u}(\cdot)$. We then present the following definition.

Definition 2.1. ^[16] The control $\bar u(\cdot)\in L^2\left([0, L_0];\mathbb{R}^m\right)$ is called an open-loop equilibrium strategy if

where

and $c\in \mathbb{R}^m$ is a constant vector. The corresponding control trajectory $\bar {\mathcal{Z}}(\cdot)$ and $\left(\bar {\mathcal{Z}}(\cdot), \bar u(\cdot)\right)$ are called an equilibrium strategy trajectory and equilibrium strategy pair, respectively.

3.   Characterization of open-loop equilibrium strategy

3.1.   Equilibrium maximum principle

We first state the following equilibrium maximum principle, which gives a set of necessary conditions for equilibrium strategy in the sense of open-loop.

Theorem 3.1. Suppose (S1)–(S3) hold. Let $\left(\bar {\mathcal{Z}}(\cdot), \bar u(\cdot)\right)$ be an open-loop equilibrium strategy pair of Problem (I). Then, there exists a $\omega(\cdot):[0, L_0]\rightarrow \mathbb R^n$ satisfying the following integral equation:

and such that

Proof. Suppose that the Problem (Ⅰ) has an open-loop equilibrium strategy pair $\left(\bar {\mathcal{Z}}(\cdot), \bar u(\cdot)\right)$, we then define

For any fixed $\nu\in [0, L_0)$ and any $\varepsilon\in(0, 1)$ with $\nu+\varepsilon\leq L_0$, it follows from the definition of the perturbation control (2.4) that the control system (2.1) with $u^{\varepsilon, \nu, c}$ have a unique solution $\mathcal{Z}^{u^{\varepsilon, \nu, c}}\in C\left([0, L_0];\mathbb{R}^n\right)$ given by

for all $\mu\in [0, L_0]$. We then have

It follows from (2.2), (3.3)–(3.5) that

which yields that

where $\nu\in[0, L_0]$.

We can have

Using $\left(\bar {\mathcal{Z}}(\cdot), \bar u(\cdot)\right)$ as an open-loop equilibrium strategy pair, we then have

Combining (3.8) and (2.1), we then have

Thus, the differential equation (3.9) admits a unique solution $\bar {\mathcal{Z}}(\cdot)$ given by

Substituting (3.10) into (3.7), we can have (3.1). This implies that $\omega(\cdot)$ is a solution of (3.1); we thus complete the proof. □

3.2.   The equivalent relationships of open-loop equilibrium strategy

In this subsection, we investigate the equivalence among the open-loop equilibrium strategy, two-point boundary value problems, and equilibrium Riccati equations. We first introduce the following two-point boundary value problems:

and the equilibrium Riccati equations

Theorem 3.2. Suppose (S1)–(S3) hold. Then, the Problem (I) admits an open-loop equilibrium strategy, if and only if, the two-point boundary value problems (3.11) admits a solution in $C\left([0, L_0];\mathbb{R}^n\right)\times C\left([0, L_0];\mathbb{R}^n\right)$ for all $\nu\in[0, L_0]$.

Proof. We assert that the necessary condition is satisfied according to Theorem 3.1. Next, we will establish sufficiency. Let the two-point boundary problem (3.11) have a solution $(\bar {\mathcal{Z}}, \omega)\in C\left([0, L_0];\mathbb{R}^n\right)\times C\left([0, L_0];\mathbb{R}^n\right)$. By introducing the control function $\bar u$ as defined in (3.2), we claim that $\bar{u}$ serves as an equilibrium strategy. Following a similar derivation process as in (3.6), it follows (3.7) that

It follows from the assumption (S2) and the above equation that

Consequently, $\bar{u}$ is an open-loop equilibrium strategy of the linear-quadratic time-inconsistent control problems. □

Theorem 3.3. Let (S1)–(S4) hold. Then, $\mathcal{P}(\cdot)\in C \left([0, L_0]; \mathbb{R}^{n\times n}\right)$ is a solution of the equilibrium Riccati equations (3.12), if and only if, the two-point boundary value problems (3.11) admits a solution $\left(\bar {\mathcal{Z}}(\cdot), \omega(\cdot) \right) \in C \left([0, L_0]; \mathbb{R}^n\right) \times C \left([0, L_0]; \mathbb{R}^n\right)$ be given by

Proof. Let $\mathcal{P}$ be a solution of the equilibrium Riccati equations (3.12); then we can define

It is clear that $\tilde {\mathcal{Z}}$ and $\tilde\omega$ are continuous and differentiable. Taking the first order derivative on $\tilde {\mathcal{Z}}$ and $\tilde\omega$, we then have

and

Observer that

Result in

Because

Invoking this into (3.16), we obtain that

By combining (3.14) and (3.17), we demonstrate that $\left(\tilde {\mathcal{Z}}, \tilde\omega \right)$ is a solution to the two-point boundary value problem (3.11). This concludes the proof of necessity.

On the other hand, if $\left(\bar {\mathcal{Z}}, \omega \right)$ is a solution of the two-point boundary value problems (3.11) and $\left(\bar {\mathcal{Z}}, \omega \right)$ is given by (3.13). We can have

Since

It follows (3.18) and (3.19) that

This implies that $\mathcal{P}$ is continuous and differential. Therefore, taking the first-order derivative on both sides of $\omega(\nu) = \mathcal{P}(\nu)\bar {\mathcal{Z}}(\nu)$, we can obtain that

Therefore, it is clear that $\mathcal{P}$ satisfies the equilibrium Riccati equations (3.12) based on (3.11) and (3.21). Thus, we complete the proof. □

4.   Relationships between open-loop equilibrium strategy, closed-loop equilibrium strategy and optimal control

4.1.   Relationship between open-loop equilibrium strategy and closed-loop equilibrium strategy

The closed-loop representation of an open-loop optimal control can be derived from a closed-loop control for classical linear-quadratic control problems. We will now establish a similar result for linear-quadratic time-inconsistent control problems. For convenience, we introduce the following notations:

where

We now present the following lemma.

Lemma 4.1. ^[19] Let (S1)–(S4) hold. Then, the following equilibrium Riccati equations

admits a unique symmetric positive semi-definite solution $\mathcal{P}\in C \left([0, L_0]; \mathbb{R}^{n\times n}\right)$.

Proposition 4.1. Let (S1)–(S4) hold. Then, the closed-loop representation of an open-loop equilibrium strategy must be the outcome from a closed-loop equilibrium strategy if and only if

Proof. Since (S1)–(S4) hold, we know that there is a solution $\mathcal{P}$ to the equilibrium Riccati equation (4.3) as shown in ^[19]. Therefore, we can define the following function:

Clearly, $\bar u(\cdot, \cdot)$ of (4.4) is a unique closed-loop equilibrium control of linear-quadratic time-inconsistent control problems by ^[19]. Let $(0, z_0)$ be fixed; we make the notations as below:

It follows from (4.5) that $\bar {\mathcal{Z}}\in C\left([0, L_0];\mathbb{R}^n\right)$ given by (3.3).

Let

with ${\mathcal{Z}}_1^\varepsilon(\cdot)$ satisfy the equations

This implies that

Next, let

with $\mathcal{Z}_2^\varepsilon(\mu)$ solving the following equations:

Thus,

Let

Result in

Here,

This yields that

Let

This deduces

Thus,

Next, we have

where

Consequently,

In summary, we have

This implies that the $\bar u(\cdot)$ is an open-loop equilibrium strategy if and only if $b(\nu) = 0$. We thus complete the proof that $\bar u(\cdot)$ is an open-loop equilibrium strategy if and only if $\Omega(\nu) = 0$ almost every $\nu\in[0, L_0]$ by (S1). □

Remark 4.1. Note that above (4.2), $\Omega(\nu) = 0$ is not true for almost every $\nu\in[0, L_0]$ in general, which implies that the closed-loop structure associated with an open-loop equilibrium strategy fails to be valid in almost all cases.

We now provide an example to illustrate that, in general, a closed-loop representation of an open-loop equilibrium strategy does not emerge from a closed-loop equilibrium strategy. This highlights the fundamental difference between time-inconsistent control problems and classical control problems.

Example 4.1. In Problem (Ⅰ), let $\mathcal{A} = \mathcal{B} = 1$, $\mathcal{Q} = 0$, $\mathcal{M}(\nu, \mu) = \mu^2+\nu+\frac{1}{2}$, $\mathcal{G}(\nu) = \frac{1}{3}\nu^3+2\nu^2+\nu+\frac{5}{24}$ and $L_0 = \frac{1}{2}$. Then,

Let $\mathcal{P}(\nu) = \mathcal{M}(\nu, \nu)$; then the equilibrium Riccati equation (4.3) yields that

Solving the above ordinary differential equation, we then have

for all $\nu\in[0, L_0].$

Next, plug $\mathcal{A}, \mathcal{B}, \mathcal{Q}, \mathcal{M}$, and $\mathcal{G}$ into (3.12), we then have

Obviously,

for all $\nu\in[0, \frac{1}{2}]$.

4.2.   Relationship between open-loop equilibrium strategy and optimal control

Before presenting the relationship between the open-loop equilibrium strategy of time-inconsistent control problems and the optimal control of classical control problems, we will first introduce a lemma related to optimal control problems.

In cost functional (2.2), let $\nu\in[0, L_0]$ be fixed or

then Problem (Ⅰ) is reduced to a classical control problem. We introduce the following result for the optimal control problem.

Lemma 4.2. ^[26] Suppose (S1)–(S4) hold and $\nu\in[0, L_0]$ is fixed. Then, the classical optimal control problem is uniquely solvable at each $\mu\in[\nu, L_0]$, if and only if, the following Riccati equation

is uniquely solvable on $[\nu, L_0].$

Example 4.2. In Problem (Ⅰ), let $\mathcal{A} = \mathcal{B} = 1$, $\mathcal{Q} = 0$, $\mathcal{G}(\nu) = L_0^2+\nu$, and $\mathcal{M}(\nu, \mu) = \frac{\left(\mu^2+\nu\right)^2}{2\left(\mu^2+\mu+\nu\right)}$ for any $0\leq \nu < \mu \leq L_0$. Then,

On the one hand, if the initial pair $(\nu, z)\in[0, L_0]\times R^n$ is fixed, then Problem (Ⅰ) is an optimal control problem; we thus have the following Riccati equation:

Plug $\mathcal{A}$, $\mathcal{B}$, $\mathcal{Q}$, $\mathcal{G}$, $\mathcal{Q}_\nu$, $\dot {\mathcal{G}}$, and $\mathcal{M}$ into (4.25), we have

Solving the above Eq (4.26), we then have

for any $\mu\in[\nu, L_0)$.

On the other hand, for any $\nu\in[0, L_0]$, substitute $\mathcal{A}$, $\mathcal{B}$, $\mathcal{Q}$, $\mathcal{G}$, $\mathcal{Q}_\nu$, $\dot {\mathcal{G}}$ and $\mathcal{M}$ into the equilibrium Riccati equation (3.12), then we can obtain that

In (4.27), let $\mu = \nu$, we have

Plug $\mathcal{P}(\nu)$ into the main equation of (4.28); we have the left-hand of the main equation of (4.28) is equivalent to

which implies that $1-\frac{\left(1+\nu\right)^2}{\left(1+L_0\right)^2}\neq 0 \; \text{for any} \; \nu\in[0, L_0)$. Thus, $\mathcal{P}(\nu) = \nu^2+\nu$ is not a solution of Eq (4.28).

The example above demonstrates that there is no essential connection between the open-loop equilibrium strategy and optimal control. This suggests that the existence of an equilibrium strategy for time-inconsistent control problems does not derive the existence of optimal control for classical control problems, and vice versa.

5.   Conclusions

Under the open-loop framework, we derive the equilibrium maximum principle for time-inconsistent linear-quadratic control problems. By appropriately introducing the two-point boundary value problems and the equilibrium HJB equations associated for time-inconsistent linear-quadratic control problems, we establish the equivalence among the existence of open-loop equilibrium controls, the solvability of the two-point boundary value problems, and the solvability of the equilibrium HJB equations. Finally, two examples demonstrate the essential differences between the open-loop equilibrium of time-inconsistent problems and both the closed-loop equilibrium and the optimal control in classical settings: (1) The closed-loop representation of the open-loop equilibrium in time-inconsistent problems cannot yield a closed-loop equilibrium, which fundamentally diverges from classical results. (2) The existence of an open-loop equilibrium does not guarantee the existence of an optimal control in classical frameworks.

Use of Generative-AI tools declaration

The author declares they have not used Artificial Intelligence (AI) tools in the creation of this article.

Acknowledgments

The author was supported by the Doctoral Research Start-Up Foundation of Guiyang University (No. GYU-KY-[2024]) and the Natural Science Research Foundation of Education Department of Guizhou Province (No. OJJ[2024]190).

Conflict of interest

The author declares there is no competing interest.