The Pontryagin type maximum principle for Caputo fractional optimal control problems with terminal and running state constraints

1.   Introduction

Fractional calculus of derivatives and integrals can be viewed as generalizations of their classocal ones to any real arbitrary order ^[1,2,3]. One important application of fractional calculus is a class of fractional differential equations, involving fractional derivatives (or integrals), which enables us to describe more general and extraordinary phenomena observed in real world. Indeed, various types of fractional differential equations and their applications have been studied in applied mathematics, science, engineering, and economics; see ^[1,2,3,4,5] and the references therein.

Along with fractional differential equations, fractional optimal control problems, i.e., optimal control for fractional differential equations in the sense of Riemann-Liouville (RL) and/or Caputo, have been studied extensively in the literature under different formulations; see ^{[6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22]} and the references therein. In particular, classical results on fractional optimal control problems were obtained in ^[6,7,8]. The time optimal control problem for fractional differential equations was studied in ^[9,10]. The existence of optimal solutions for fractional optimal control problems was studied in ^[11,20]. The fractional dynamic programming approach with the Hamilton-Jacobi-Bellman PDE was studied in ^[12,13]. The linear optimal control problem for fractional differential equations was considered in ^[14,23]. Applications and numerical approaches of various fractional optimal control problems were studied in ^{[15,16,17,18,19,24]}.

The optimal control problems and their Pontryagin maximum principles for the fractional optimal control problems were studied in several different directions; see ^{[20,25,26,27,28,29,30,31,32,33,34,35,36]} and the references therein. Specifically, the researches in ^{[25,26,27,28]} considered the optimal control problem for RL and Caputo fractional differential equations without state constraints, which use the Euler-Lagrangian approach similar to ^[6,7]. In ^[34], the Lagrangian multiplier method was applied to obtain the necessary condition for optimality. However, it requires that the corresponding Hamiltonian is smooth, and does not know the nontriviality of the corresponding Lagrangian multipliers. Moreover, the terminal state constraint was not considered in ^[34]. In ^[29], a simple convex variation was applied to obtain the maximum principle for the RL fractional optimal control problem. The Caputo fractional optimal control problem without state constraints was studied in ^[30], where the sufficient condition was also obtained under the convexity assumption. In ^[31,32], the Caputo and Cucker-Smale multi-agent fractional control problems without state constraints were studied, where the corresponding control space was assumed to be compact. The authors in ^[36] studied the controllability and existence of optimal control for the unconstrained case in the infinite-dimensional case. In ^[35], the pseudospectral method for fractional optimal control problem was studied without the state constraints. In addition, ^[33] studied the singular fractional optimal control problem. Recently, the Caputo fractional optimal control problem with the terminal state constraint only was considered in ^[20]. However, the research in ^[20] did not consider the running state constraint. We also mention that the research in ^[22] studied the equivalent relationship between control problems for fractional differential equations and Volterra integral equations, and its infinite-dimensional case was treated in ^[21]. Notice that the research in ^[21,22] did not consider the state constraints in the corresponding maximum principles. {We note that ^[37,38] studied the nonfractional optimal control problems with state constraints. Recently, ^[39] studied the general numerical methods for the linear-quadratic fractional optimal control problem, where the Bolza-Riemann-Liouville objective functional was considered. Moreover, ^[39] provided several practical examples of linear-quadratic fractional optimal control problems such as the optimal drug scheduling of cancer chemotherapy problem and the minimum fuel optimal control problem. We also mention that the research in ^[40] considered the linear-quadratic fractional optimal control problem without state constraints, where instead of the open-loop optimal solutions, the closed-loop optimal solution was obtained in terms of the Riccati-type equation.

In this paper, we consider the fractional optimal control problem with terminal and running state constraints. The precise problem statement is given in Section 3. In our problem setup, the dynamic constraint is captured by the fractional differential equation in the sense of Caputo with order $\alpha \in (0, 1)$, and the objective functional is formulated by the left RL fractional integral. In addition, there are terminal and running state constraints; while the former is described by initial and final states within a convex set, the latter is expressed by an explicit instantaneous inequality state constraint. Note that the general fractional optimal control problem with terminal and running state constraints has not been sufficiently studied in the existing literature, which we address in this paper. Indeed, the earlier results for fractional optimal control problems discussed above did not consider the case of both terminal and running state constraints. We also mention that the research in ^[39] studied the numerical approaches for solving the linear-quadratic fractional optimal control problem. Finally, in the existing literatures, some standing assumptions in the fractional optimal control problem (e.g., , the assumption of the control space being closed or convex) are different from those in our paper.

We prove the wellposedness (equivalently, the existence and uniqueness of the solution) of the Caputo fractional differential equation with order $\alpha \in (0, 1)$ (see Theorem 3.1). Although similar versions of Theorem 3.1 are reported in the existing literature (e.g., ^[2,41]), we provide its different version and different proof. In particular, to prove Theorem 3.1, we first need to show the equivalent RL fractional integral representation of the Caputo fractional differential equation. Then, we use the Bielecki norm under an appropriate function space to apply the contraction mapping theorem. We also obtain several important estimates of the Caputo fractional differential equation using the fractional Gronwall's inequality.

We obtain the explicit representation results of left Caputo linear fractional differential equations with initial conditions and right RL fractional differential equations with terminal conditions, where the latter is also dependent on the variable of the Lebesgue-Stieltjes measure (see Section 4). In both cases, to obtain the desired representation formulas, a detailed analysis of the fundamental solution of the left and right RL fractional state-transition matrices has to be carried out, and the careful use of Fubini's formula is needed. While the representation result of left Caputo linear fractional differential equations was presented in (^[42], Theorem 5.1), we provide its different version of the representation result in Lemma 5 using the definition of (left and right) RL fractional state-transition matrices. Furthermore, the representation result of right RL fractional differential equations with terminal conditions in Lemma 6 has not been reported in the existing literature. Note that the results in Section 4 are independent from our main result.

The main result of this paper is the Pontryagin maximum principle (see Theorem 5.1). Since the control space $U$ is only a separable metric space and does not have any algebraic structure (see Assumption 1), we have to employ the spike perturbation technique. Due to the inherent complex nature of the fractional control problem and the presence of the terminal and running state constraints, our maximum principle and its detailed proof must be different from those of the existing literature (e.g., ^[20,29,31]). Specifically, in the proof (see Section 7), we need to formulate the penalized unconstrained fractional control problem, for which the Ekeland variational principle has to be applied. Then, to develop the variational and duality analysis via the spike variation, we have to use the fractional calculus and the intrinsic properties of distance functions (with functional analysis techniques), together with the fractional Gronwall's inequality, the representation results of (left and right) Caputo and RL linear fractional differential equations (see Theorem 3.1 and Section 4^*), and the technical lemma (see Lemma C.1 in Appendix C^†). Moreover, the proofs for the complementary slackness and the transversality condition are essentially required due to the presence of the terminal and running state constraints. We mention that our proof technique of the maximum principle for the fractional optimal control problem has not been presented in the existing literature.

^*In Section 4, we prove the representation formulas of the solution to the left Caputo and right RL fractional differential equations, where the latter includes the Lebesgue-Stieltjes integral, in terms of the (left and right) RL fractional state-transition matrices. Note that the results in Section 4 have not been reported in the existing literature.

^†We mention that the proof of Lemma C.1 in Appendix C requires a new technique compared with its nonfractional version in (^[43], Corollary 3.8, Chapter 4).

As an application, we consider the linear-quadratic fractional optimal control problem with terminal and running state constraints (see Section 6 and Proposition 6.1). We obtain the optimal solution to the linear-quadratic problem using the maximum principle of this paper (see Theorem 5.1), and then discuss the computational method to obtain the unknown variables in the maximum principle (see Proposition 6.1 and Remark 6.1). Indeed, we mention that various numerical approaches for solving linear-quadratic fractional optimal control problems can be found in ^[39,44,45] and the references therein (see Remark 6.1).

Our maximum principle in Theorem 5.1 can be viewed as an extension of ^[20,29,31] in several different directions (see Remarks 2 and 4). In particular, the researches in ^[20,29,31] did not consider both terminal and running state constraints. In addition, the control spaces in ^[20,29,31] need to be compact or closed, whereas we assume only separability without convexity (see Assumption 3.1). Hence, the problem formulation of this paper is different from that of ^[20,29,31], and the new approaches and techniques have to be developed in this paper (particularly the proof of the maximum principle in Section 7 and the representation results in Section 4). Indeed, the maximum principle of this paper is new in the optimal control problem context and its proof requires to develop a new technique, both of which are not reported in the existing literature. Note also that recently, ^[46] obtained the maximum principle for optimal control of infinite-dimensional Caputo-type evolution equations under the end-point terminal state constraint only. Hence, the approach of ^[46] cannot be used to solve the problem with both terminal and running state constraints considered in our paper. We also mention that (^[46], Theorem 3.1 and Remark 3.2) requires additional assumptions in the corresponding maximum principle due to the infinite-dimensional nature of the problem formulation. Finally, the example studied in this paper is different from that in ^[46]. Indeed, this paper considers the linear-quadratic problem with both terminal and running state constraints, whereas ^[46] studied the fractional heat equation with the terminal state constraint only.

The paper is organized as follows. Preliminaries on fractional calculus are given in Section 2. We formulate the problem and then provide the wellposedness result in Section 3. In Section 4, we present the representation results of (left and right) Caputo and RL linear fractional differential equations. The maximum principle of this paper is stated in Section 5. The application of the maximum principle to the state-constrained fractional linear-quadratic problem is studied in Section 6. The proof for the maximum principle is provided in Section 7. We conclude our paper in Section 8. In Appendix A, we provide the proof of the results in Section 3. In Appendix B, we give the variational analysis, which is needed in Section 7. In Appendix C, we provide the technical lemma used in Section 7.

2.   Preliminaries on fractional calculus

In this section, we provide some preliminary results on fractional calculus. More detailed results on fractional calculus can be found in ^[1,2] and the references therein.

Let $\mathbb{R}^n$ be the $n$-dimensional Euclidean space, where $\langle x, y \rangle : = x^\top y$ is the inner product and $|x| : = \langle x, x \rangle^{1/2}$ is the norm for $x, y \in \mathbb{R}^n$. For $A \in \mathbb{R}^{m \times n}$, $A^\top$ denotes the transpose of $A$. Let $I_n$ be an $n \times n$ identity matrix. Define $\mathbb{1}_{S}(\cdot)$ by the indicator function of any set $S$. Let $\Gamma$ be the Gamma function.

For any differentiable map $f:\mathbb{R}^n \rightarrow \mathbb{R}^l$, let $\partial_x f: \mathbb{R}^n \rightarrow \mathbb{R}^{l \times n}$ be the partial derivative of $f$ with respect to $x \in \mathbb{R}^n$. Note that $\partial_x f = \begin{bmatrix} \partial_x f_{1}^\top & \cdots & \partial_x f_{l}^\top \end{bmatrix}^\top$ with $\partial_x f_{j} \in \mathbb{R}^{1 \times n}$, and when $l = 1$, $\partial_x f \in \mathbb{R}^{1 \times n}$. For any differentiable map $f : \mathbb{R}^n \times \mathbb{R}^l \rightarrow \mathbb{R}^l$, $\partial_x f : \mathbb{R}^n \times \mathbb{R}^l \rightarrow \mathbb{R}^{l \times n}$ for $x \in \mathbb{R}^n$, and $\partial_y f: \mathbb{R}^n \times \mathbb{R}^l \rightarrow \mathbb{R}^{l \times l}$ for $y \in \mathbb{R}^l$.

Let $t_0, t_f \in [0, T]$ with $t_0 < t_f$ be fixed. Define the following spaces:

● $L^p([t_0, t_f]; \mathbb{R}^n)$, $p \geq 1$: the space of functions $\psi:[t_0, t_f] \rightarrow \mathbb{R}^n$ such that $\psi$ is measurable and $\|\psi(\cdot)\|_{L^{p, n}} : = (\int_{t_0}^{t_f} |\psi(t)|^p_{\mathbb{R}^n} {\rm{d}} t)^{\frac{1}{p}} < \infty$ ($\|\psi(\cdot)\|_{L^{\infty, n}} : = {\rm{ess}}\;{\rm{sup}}_{t \in [t_0, t_f]} |\psi(t)| < \infty$ when $p = \infty$);

● $C([t_0, t_f]; \mathbb{R}^n)$: the space of functions $\psi:[t_0, t_f] \rightarrow \mathbb{R}^n$ such that $\psi$ is continuous and $\|\psi(\cdot)\|_{\infty, n} : = \sup_{t \in [t_0, t_f]} |\psi(t)| < \infty$;

● $\text{AC}([t_0, t_f]; \mathbb{R}^n)$: the space of functions $\psi:[t_0, t_f] \rightarrow \mathbb{R}^n$ such that $\psi$ is absolutely continuous;

● $\text{BV}([t_0, t_f]; \mathbb{R}^n)$: the space of functions $\psi:[t_0, t_f] \rightarrow \mathbb{R}^n$ such that $\psi$ is of bounded variation on $[t_0, t_f]$.

The norm on $\text{BV}([t_0, t_f]; \mathbb{R}^n)$ is defined by $\|\psi(\cdot)\|_{\text{BV}^n} : = \psi(t_0) + \text{TV}(\psi)$, where $\text{TV}(\psi) : = \sup_{(t_k)_k} \bigl \{ \sum_{k} |\psi(t_{k+1}) - \psi(t_k)| \bigr \} < \infty$ with the supremum being taken by all partitions of $[t_0, t_f]$. Let $\text{NBV}([t_0, t_f]; \mathbb{R}^n)$ be the space of functions $\psi(\cdot) \in \text{BV}([t_0, t_f]; \mathbb{R}^n)$ such that $\psi(\cdot) \in \text{BV}([t_0, t_f]; \mathbb{R}^n)$ is normalized, i.e., $\psi(t_0) = 0$ and $\psi$ is left continuous. The norm on $\text{NBV}([t_0, t_f]; \mathbb{R}^n)$ is defined by $\|\psi(\cdot)\|_{\text{NBV}^n} : = \text{TV}(\psi)$. When $\psi(\cdot) \in \text{NBV}([t_0, t_f]; \mathbb{R})$ is monotonically nondecreasing, we have $\|\psi(\cdot)\|_{\text{NBV}} : = \|\psi(\cdot)\|_{\text{NBV}^1} = \psi(t_f)$. Recall that both $(\text{BV}([t_0, t_f]; \mathbb{R}^n), \|\cdot\|_{\text{BV}^n})$ and $(\text{NBV}([t_0, t_f]; \mathbb{R}^n), \|\cdot\|_{\text{NBV}^n})$ are Banach spaces.

Definition 2.1 (i) For $f(\cdot) \in L^1([t_0, t_f]; \mathbb{R}^n)$ and $t \in [t_0, t_f]$, the left Riemann-Liouville (RL) fractional integral $I_{t_0+}^{\alpha}[f]$ of order $\alpha > 0$ is defined by $I_{t_0+}^{\alpha}[f](t) : = \int_{t_0}^t \frac{(t-s)^{\alpha-1}}{\Gamma(\alpha)} f(s) {\rm{d}} s$.

(ii) For $f (\cdot) \in L^1([t_0, t_f]; \mathbb{R}^n)$ and $t \in [t_0, t_f]$, the right RL fractional integral $I_{t_f-}^{\alpha}[f]$ of order $\alpha > 0$ is defined by $I_{t_f-}^{\alpha}[f](t) : = \int_{t}^{t_f} \frac{(s-t)^{\alpha-1}}{\Gamma(\alpha)} f(s) {\rm{d}} s$.

Lemma 2.1. (^[1], Lemma 2.1) For any $f(\cdot) \in L^1([t_0, t_f]; \mathbb{R}^n)$, $\| I_{t_0+}^{\alpha}[f](\cdot)\|_{L^{1, n}} \leq K \|f(\cdot)\|_{L^{1, n}}$ and $\| I_{t_f-}^{\alpha}[f](\cdot)\|_{L^{1, n}} \leq K \|f(\cdot)\|_{L^{1, n}}$, which implies $I_{t_0+}^{\alpha}[f](\cdot), I_{t_f-}^{\alpha}[f](\cdot) \in L^1([t_0, t_f]; \mathbb{R}^n)$.

Lemma 2.2. (^[1], Lemma 2.3) For any $f(\cdot) \in L^1([t_0, t_f]; \mathbb{R}^n)$ and $\alpha, \beta > 0$, the following semigroup property holds: $I_{t_0+}^{\alpha} [I_{t_0+}^{\beta}[f]](\cdot) = I_{t_0+}^{\alpha+\beta}[f](\cdot) = I_{t_0+}^{\beta + \alpha}[f](\cdot) = I_{t_0+}^{\beta} [I_{t_0+}^{\alpha}[f]](\cdot)$ and $I_{t_f-}^{\alpha} [I_{t_f-}^{\beta}[f]](\cdot) = I_{t_f-}^{\alpha+\beta}[f](\cdot) = I_{t_f-}^{\beta + \alpha}[f](\cdot) = I_{t_f-}^{\beta} [I_{t_f-}^{\alpha}[f]](\cdot)$.

Lemma 2.3. (^[2], Theorem 2.6) For $f(\cdot) \in L^{\infty}([t_0, t_f]; \mathbb{R}^n)$, $I_{t_0+}^{\alpha}[f] (\cdot), I_{t_f-}^{\alpha}[f] (\cdot) \in C([t_0, t_f]; \mathbb{R}^n)$.

Definition 2.2. (i) For $f(\cdot) \in L^1([t_0, t_f]; \mathbb{R}^n)$, the left RL fractional derivative $D_{t_0+}^{\alpha}[f]$ of order $\alpha \in (0, 1)$ is defined by $D_{t_0+}^{\alpha}[f](t) : = \frac{{\rm{d}} }{ {\rm{d}} t} \Bigl [I_{t_0+}^{1-\alpha}[f] \Bigr] (t)$, provided that $I_{t_0+}^{1-\alpha}[f] (\cdot)\in \text{AC}([t_0, t_f]; \mathbb{R}^n)$. In this case, we have $D_{t_0+}^{\alpha}[f] (\cdot) \in L^1([t_0, t_f]; \mathbb{R}^n)$. Let $^{ {RL}}_n{\mathcal{D}}{_{t_0+}^{\alpha}}$ be the set of functions $f(\cdot) \in L^1([t_0, t_f]; \mathbb{R}^n)$ such that $f$ admits the left RL fractional derivative $D_{t_0+}^{\alpha}[f]$ of order $\alpha \in (0, 1)$.

(ii) For $f (\cdot) \in L^1([t_0, t_f]; \mathbb{R}^n)$, the right RL fractional derivative $D_{t_f-}^{\alpha}[f]$ of order $\alpha \in (0, 1)$ is defined by $D_{t_f-}^{\alpha}[f](t) : = - \frac{{\rm{d}}}{{\rm{d}} t} \Bigl [I_{t_f-}^{1-\alpha}[f] \Bigr] (t)$, provided that $I_{t_f-}^{1-\alpha}[f] (\cdot) \in \text{AC}([t_0, t_f]; \mathbb{R}^n)$. In this case, we have $D_{t_f-}^{\alpha}[f] (\cdot) \in L^1([t_0, t_f]; \mathbb{R}^n)$. Let $^{ {RL}}_n{\mathcal{D}}{_{t_f-}^{\alpha}}$ be the set of functions $f (\cdot) \in L^1([t_0, t_f]; \mathbb{R}^n)$ such that $f$ admits the right RL fractional derivative $D_{t_f-}^{\alpha}[f]$ of order $\alpha \in (0, 1)$.

Remark 2.1. Note that when $\alpha = 1$, the notions of fractional integral and derivatives are reduced to those of usual ones ^[1,20] (see ^[1], Chapter 2).

Let $D_{t_0+}^{\alpha}[f(\cdot)](\cdot) : = D_{t_0+}^{\alpha}[f](\cdot)$ and $D_{t_f-}^{\alpha}[f(\cdot)](\cdot) : = D_{t_f-}^{\alpha}[f](\cdot)$, where $(\cdot)$ in the square bracket is used to emphasize the integration of $f$ with respect to the other variable.

Definition 2.3. (i) For $f(\cdot) \in C([t_0, t_f]; \mathbb{R}^n)$, the left Caputo fractional derivative $^{{C}}{ D}{_{t_0+}^{\alpha}}[f]$ of order $\alpha \in (0, 1)$ is defined by $^{{C}}{ D}{_{t_0+}^{\alpha}}[f](t) : = D_{t_0+}^{\alpha}[f(\cdot) -f(t_0)](t)$, where $f(\cdot) - f(t_0) \in ^{ {RL}}_n{\mathcal{D}}{_{t_0+}^{\alpha}}$. In this case, we have $^{{C}}{ D}{_{t_0+}^{\alpha}}[f] (\cdot) \in L^1([t_0, t_f]; \mathbb{R}^n)$. Let $^{ {C}}_n{\mathcal{D}}{_{t_0+}^{\alpha}}$ be the set of functions $f(\cdot) \in C([t_0, t_f]; \mathbb{R}^n)$ such that $f$ admits the left Caputo fractional derivative $^{{C}}{ D}{_{t_0+}^{\alpha}}[f]$ of order $\alpha \in (0, 1)$.

(ii) For $f(\cdot) \in C([t_0, t_f]; \mathbb{R}^n)$, the right Caputo fractional derivative $^{{C}}{ D}{_{t_f-}^{\alpha}}[f]$ of order $\alpha \in (0, 1)$ is defined by $^{{C}}{ D}{_{t_f-}^{\alpha}}[f](t) : = D_{t_f-}^{\alpha}[f(\cdot) -f(t_f)](t)$, where $f(\cdot) - f(t_f) \in ^{ {RL}}_n{\mathcal{D}}{_{t_f-}^{\alpha}}$. In this case, we have $^{{C}}{ D}{_{t_f-}^{\alpha}}[f] (\cdot) \in L^1([t_0, t_f]; \mathbb{R}^n)$. Let $^{ {C}}_n{\mathcal{D}}{_{t_f-}^{\alpha}}$ be the set of functions $f(\cdot) \in C([t_0, t_f]; \mathbb{R}^n)$ such that $f$ admits the right Caputo fractional derivative $^{{C}}{ D}{_{t_f-}^{\alpha}}[f]$ of order $\alpha \in (0, 1)$.

3.   State-constrained fractional optimal control problem

Consider the $\mathbb{R}^{n}$-valued left Caputo fractional differential equation with order $\alpha \in (0, 1)$:

where $X(\cdot) \in \mathbb{R}^n$ is the state with the initial condition $X_0 \in \mathbb{R}^n$, $u:[t_0, t_f] \rightarrow U$ is the control input with $U$ being the control set, and $f :[t_0, t_f] \times \mathbb{R}^{n} \times U \rightarrow \mathbb{R}^{n}$ is the driver of the Caputo fractional differential equation. Let $\mathcal{U} : = \{ u:[t_0, t_f] \rightarrow U\; |\; u\; \text{is measurable} \}$ be the space of admissible controls for (3.1). We sometimes use the notation $X(\cdot; X_0, u) : = X(\cdot)$ to stress the dependency of (3.1) on the pair $(X_0, u(\cdot)) \in \mathbb{R}^n \times \mathcal{U}$.

Assumption 3.1. (i) $(U, \rho)$ is a separable metric space, where $\rho$ is the corresponding metric.

(ii) $f(\cdot, X, u) \in L^{\infty}([t_0, t_f]; \mathbb{R}^n)$, and it holds that $\sup_{u \in U} |f(t, x, u)| < \infty$ for $(t, x) \in [t_0, t_f] \times \mathbb{R}^n$. $(X, u) \mapsto f(t, X, u)$ is Lipschitz continuous, i.e., there is a constant $L \geq 0$ such that for any $t \in [t_0, t_f]$ and $(X, u), (X^\prime, u^\prime) \in \mathbb{R}^{n} \times U$, $|f(t, X, u) - f(t, X^\prime, u^\prime) | \leq L (|X - X^\prime| + \rho(u, u^\prime))$, and it holds that $|f(t, 0, u)| \leq L (1+ |X|)$.

(iii) $X \mapsto f(t, X, u)$ is continuously differentiable, where $(t, X, u) \mapsto \partial_X f(t, X, u)$ is continuous, and $(X, u) \mapsto \partial_X f(t, X, u)$ is Lipschitz continuous with the constant $L \geq 0$.

The following theorem shows the wellposedness of (3.1). Its proof can be found in Appendix A.

Theorem 3.1. Let Assumption 3.1 hold. Then, the following statements hold:

(i) The solution of (3.1), $X(\cdot) \in ^{ {C}}_{n}{\mathcal{D}}{_{t_0+}^{\alpha}}$, can be expressed as the left RL fractional integral form given by $X(t) = X_0 + I_{t_0+}^{\alpha}[f(\cdot, X(\cdot), u(\cdot))] (t)$;

(ii) For any $(X_0, u(\cdot)) \in \mathbb{R}^{n} \times \mathcal{U}$, (3.1) has a unique solution of $X(\cdot) \in ^{ {C}}_{n}{\mathcal{D}}{_{t_0+}^{\alpha}}$;

(iii) For any $(X_0, u(\cdot)), (X_0^\prime, u^\prime(\cdot)) \in \mathbb{R}^{n} \times \mathcal{U}$, we have

where $b(t) : = |X_0 - X_0^\prime| + L \int_{t_0}^{t} \frac{(t-s)^{\alpha-1}}{\Gamma(\alpha)} \rho(u(s), u^\prime(s)) {\rm{d}} s$ and $b^\prime(t) : = |X_0| + L \int_{t_0}^{t} \frac{(t-s)^{\alpha-1}}{\Gamma(\alpha)} {\rm{d}} s$.

The objective functional is given by the following left RL fractional integral with order $\beta \geq 1$:

where by Definition 2.1,

In (3.4), $l:[t_0, t_f] \times \mathbb{R}^{n} \times U \rightarrow \mathbb{R}$ is the running cost and $m : \mathbb{R}^{n} \times \mathbb{R}^{n} \rightarrow \mathbb{R}$ is the terminal cost. The fractional optimal control problem of this paper is formulated as follows:

subject to the terminal and running state constraints given by

Assumption 3.2. (i) $l(\cdot, X, u) \in L^{\infty}([t_0, t_f]; \mathbb{R})$, and $(X, u) \mapsto l(t, X, u)$ is continuous. $(X, X^\prime) \mapsto m(X, X^\prime)$ is continuous.

(ii) $X \mapsto l(t, X, u)$ is continuously differentiable, where $(t, X, u) \mapsto \partial_X l(t, X, u)$ is continuous, and $(X, u) \mapsto \partial_X l(t, X, u)$ is Lipschitz continuous with the constant $L \geq 0$. $(X, X^\prime) \mapsto m(X, X^\prime)$ is continuously differentiable, where $(X, X^\prime) \mapsto \partial_X m(X, X^\prime)$ and $(X, X^\prime) \mapsto \partial_{X^\prime} m(X, X^\prime)$ are Lipschitz continuous with the constant $L \geq 0$.

(iii) $F$ is a nonempty closed convex subset of $\mathbb{R}^{2n}$. $(t, X) \mapsto G(t, X) : = \begin{bmatrix} G_1(t, X) & \cdots & G_q(t, X) \end{bmatrix}^\top$ with $G_i:[t_0, t_f] \times \mathbb{R}^n \rightarrow \mathbb{R}$, $i = 1, \ldots, q$, is continuous, where $X \mapsto G(t, X)$ is continuously differentiable with $(t, X) \mapsto \partial_X G(t, x)$ being bounded.

Remark 3.1. Assumptions similar to Assumptions 3.1 and 3.2 have been used in various optimal control problems and their maximum principles; see ^{[21,22,43,47,48,49,50,51]} and the references therein. Also, we extend ^[20,29,31]. In fact, the researches in ^[29,31] did not consider any state constraints, and ^[20] considered only the terminal state constraint. Furthermore, the control spaces in ^[20,29,31] need to be compact or closed, whereas we assume only separability without convexity (see Assumption 3.1). The fractional control problem with both terminal and running state constraints formulated in (P) has not been studied in the existing literature.

4.   Representation results on linear fractional differential equations

In this section, we provide the representation results of (RL and Caputo) linear fractional differential equations via RL fractional state-transition matrices. Note that the results of this section are independent from the main result of this paper in Theorem 5.1.

Definition 4.1. Let $F(\cdot) \in L^{\infty}([t_0, t_f]; \mathbb{R}^{n \times n})$. The left RL fractional state-transition matrix $\Pi(\cdot, \tau) \in ^{ {RL}}_{n \times n}{\mathcal{D}}{_{\tau+}^{\alpha}}$ with $t_0 \leq \tau\leq t \leq t_f$ is defined by

which is the solution to the following left RL fractional differential equation:

Lemma 4.1. The left RL fractional state-transition matrix given in (4.1) can be written as the following backward fractional integration form:

Proof. By the right-hand-side of (4.2), let

Then, we need to show the equivalence of (4.1) and (4.3). Hence, we have to prove the following result:

Clearly, (4.4) holds when $t = \tau$. Then, (4.4) follows by Fubini's formula (^[52], Theorem 3.3, Chapter 6), since we can show that

This completes the proof. □

Indeed, we can see that (4.2) in Lemma 4.1 comes from the following definition:

Definition 4.2. Let $F(\cdot) \in L^{\infty}([t_0, t_f]; \mathbb{R}^{n \times n})$. The right RL fractional state-transition matrix $\Pi (t, \cdot) \in ^{ {RL}}_{n \times n}{\mathcal{D}}{_{t-}^{\alpha}}$ with $t_0 \leq \tau\leq t \leq t_f$ is defined by

which is the solution to the following right RL fractional differential equation:

We state the representation result of left linear Caputo fractional differential equations with initial conditions.

Lemma 4.2. Let $F(\cdot) \in L^{\infty}([t_0, t_f]; \mathbb{R}^{n \times n})$ and $G(\cdot) \in L^{\infty}([t_0, t_f]; \mathbb{R}^{n})$. Consider the following left Caputo fractional differential equation:

Then, (4.6) admits a unique solution $Q(\cdot) \in ^{ {C}}_{n}{\mathcal{D}}{_{t_0+}^{\alpha}}$. Furthermore, the solution of (4.6) is given by

where $\Pi$ is the left RL fractional state-transition matrix associated with $F$ in Definition 4.1.

Proof. As (4.6) is a linear Caputo fractional differential equation, the existence and uniqueness of the solution of $Q(\cdot) \in ^{ {C}}_{n}{\mathcal{D}}{_{t_0+}^{\alpha}}$ follows from Theorem 3.1. We now prove (4.7). By Theorem 3.1 and Definition 2.1, (4.6) is equivalent to

To show (4.7), we have to prove that (4.7) satisfies the integral representation in (4.8). Specifically, by comparing (4.7) and (4.8), we need to verify the following result:

Obviously, (4.9) holds when $t = t_0$. For $t \in (t_0, t_f]$, using (4.7), it follows that

By Fubini's formula (see ^[52], Theorem 3.3, Chapter 6),

Then, by Definition 4.1 and (4.10), it follows that

Hence, by (4.11), (4.9) is verified. This completes the proof. □

The following result shows the representation result of right linear RL fractional differential equations with terminal conditions. Here, the corresponding RL fractional differential equation includes the variable of the Lebesgue-Stieltjes measure on $[t_0, t_f]$. Hence, its solution has to be written in terms of the integral of the Lebesgue-Stieltjes measure.

Lemma 4.3. Let $F(\cdot) \in L^{\infty}([t_0, t_f]; \mathbb{R}^{n \times n})$, $G_1(\cdot), G_2(\cdot) \in L^{\infty}([t_0, t_f]; \mathbb{R}^n)$, and $\eta(\cdot) \in \text{NBV}([t_0, t_f]; \mathbb{R})$. Consider the following right RL fractional differential equation:

where ${\rm{d}} \eta$ denotes the Lebesgue-Stieltjes measure of $\eta$ on $([t_0, t_f], \mathcal{B}([t_0, t_f]))$.^‡ Then, (4.13) admits a unique solution $Q(\cdot) \in \text{BV}([t_0, t_f]; \mathbb{R}^n)$. Furthermore, the solution of (4.13) is given by

^‡$\mathcal{B}([t_0, t_f])$ denotes the Borel $\sigma$-algebra generated by subintervals of $[t_0, t_f]$.

where $\Pi$ is the right RL state-transition matrix associated with $F$ defined in (4.5) of Definition 4.2.

Proof. The proof for the existence and uniqueness is similar to that of Theorem 3.1 and Lemma 4.2. To prove (4.14), by using the definition of $\Pi$ in (4.5),

The multiple integrals in the above equation have to be analyzed. By Fubini's formula (see ^[52], Theorem 3.3, Chapter 6),

Hence, it follows that

By Definitions 2.1 and 2.2, this is equivalent to

As $F(\cdot) \in L^{\infty}([t_0, t_f]; \mathbb{R}^{n \times n})$ and $G_1(\cdot), G_2(\cdot) \in L^{\infty}([t_0, t_f]; \mathbb{R}^n)$, from Lemmas 2.1 and 2.3 and Definitions 2.1 and 2.2 with (^[2], Theorem 2.2) and Fubini's formula,

Indeed, to show (4.15), let $B$ the beta function. By Fubini's formula and change of integral variables with $s = t - r(\tau-t)$ (which implies ${\rm{d}} s = - (\tau-t) {\rm{d}} r$), we can show that

Hence, (4.15) holds. Note that $I_{t_f-}^{1-\alpha}[Q](t_f) = Q_f$. As $I_{t_f-}^{1-\alpha}[Q] (\cdot) \in \rm{AC}([t_0, t_f]; \mathbb{R}^n)$, by Definitions 2.1 and 2.2, the differential form of (4.15) can be written by (4.13). We can easily observe that $Q(\cdot) \in \text{BV}([t_0, t_f]; \mathbb{R}^n)$. This completes the proof. □

5.   Main result: Pontryagin maximum principle of (P)

Assume that $(\overline{u}(\cdot), \overline{X}(\cdot)) \in \mathcal{U} \times ^{ {C}}_{n}{\mathcal{D}}{_{t_0+}^{\alpha}}$ is the optimal solution of (P), where $\overline{u}(\cdot) \in \mathcal{U}$ is the optimal solution of (P) and $\overline{X}(\cdot) : = \overline{X}(\cdot; \overline{X}_0, \overline{u}) \in ^{ {C}}_{n}{\mathcal{D}}{_{t_0+}^{\alpha}}$ is the corresponding optimal state trajectory of (3.1) controlled by $\overline{u}(\cdot) \in \mathcal{U}$. Note that $\overline{X}(\cdot) \in ^{ {C}}_{n}{\mathcal{D}}{_{t_0+}^{\alpha}}$ satisfies the terminal and running state constraints in (3.6). For notational convenience, let

We state the Pontryagin maximum principle of (P). The proof is provided in Section 7.

Theorem 5.1. Suppose that Assumptions 3.1 and 3.2 hold. Assume that the pair $(\overline{u}(\cdot), \overline{X}(\cdot)) \in \mathcal{U} \times ^{ {C}}_{n}{\mathcal{D}}{_{t_0+}^{\alpha}}$ is the optimal solution of (P). Then, there exist a tuple $(\lambda, \xi, \theta) \in \mathbb{R} \times \mathbb{R}^{2n} \times \text{NBV}([t_0, t_f]; \mathbb{R}^{q})$ and the adjoint vector $p(\cdot) \in \text{BV}([t_0, t_f]; \mathbb{R}^n)$, where $\xi = (\xi_1, \xi_2)$ with $\xi_1, \xi_2 \in \mathbb{R}^n$, and $\theta(\cdot) = (\theta_1(\cdot), \ldots, \theta_q(\cdot))$ with $\theta_i$ satisfying $\theta_i(\cdot) \in \text{NBV}([t_0, t_f]; \mathbb{R})$, $i = 1, \ldots, q$, such that the following conditions are satisfied:

(i) Nontriviality condition: $(\lambda, \xi, \theta) \neq 0$ with $\lambda \geq 0$, $\xi = (\xi_1, \xi_2) \in N_F(\overline{X}_0, \overline{X}(t_f))$ and $\|\theta_i(\cdot) \|_{\text{NBV}} = \theta_i(t_f) \geq 0$ for $i = 1, \ldots, q$, where $N_F(\overline{X}_0, \overline{X}(t_f))$ is the normal cone to the convex set $F$ at $(\overline{X}_0, \overline{X}(t_f)) \in F$ defined in (7.1), and $\theta_i(\cdot) \in \text{NBV}([t_0, t_f]; \mathbb{R})$, $i = 1, \ldots, q$, is finite, nonnegative and monotonically nondecreasing on $[t_0, t_f]$;

(ii) Adjoint equation: The adjoint vector $p(\cdot) \in \text{BV}([t_0, t_f]; \mathbb{R}^n)$ is the unique solution of the following right RL fractional differential equation: a.e. $t \in [t_0, t_f]$,

which is equivalent to the following right RL fractional integral form:

(iii) Transversality condition:

(iv) Complementary slackness condition:

(v) Hamiltonian minimization condition:

where $\mathcal{H} : [t_0, t_f] \times \mathbb{R}^n \times \mathbb{R}^n \times U \rightarrow \mathbb{R}$ is the Hamiltonian defined by

Remark 5.1. The adjoint equation $p(\cdot) \in \text{BV}([t_0, t_f]; \mathbb{R}^n)$ in (5.1) is expressed as the right RL fractional differential (or integral) form that includes the Lebesgue-Stieltjes integral, where ${\rm{d}} \theta_i$, $i = 1, \ldots, q$, is the corresponding Lebesgue-Stieltjes measure on $([t_0, t_f], \mathcal{B}([t_0, t_f]))$ with ${\rm{d}} \theta_i(t) \geq 0$ for $t \in [t_0, t_f]$ by (i) of Theorem 5.1. When $\theta_i(\cdot) \in \rm{AC}([t_0, t_f]; \mathbb{R})$, $i = 1, \ldots, q$, $\theta_i$ is differentiable a.e. $t \in [t_0, t_f]$. In this case, we have $p(\cdot) \in ^{ {RL}}_n{\mathcal{D}}{_{t_f-}^{\alpha}}$, which can be expressed as the following right RL fractional differential equation:

Here, $\frac{{\rm{d}} \theta_i(t)}{{\rm{d}} t}$ is the derivative of $\theta_i(\cdot) \in \rm{AC}([t_0, t_f]; \mathbb{R})$ for $i = 1, \ldots, q$ (^[52], Theorem 3.11, Chapter 3).

Remark 5.2. (i) Without (3.5), Theorem 5.1 becomes similar to (^[31], Theorem 3.1), where the control space of (^[31], Theorem 3.1) is assumed to be compact. In this case, only (ii) and (v) of Theorem 5.1 are needed with $\lambda = 1$, $\xi = 0$, and $\theta = 0$, where the adjoint equation $p(\cdot) \in ^{ {RL}}_n{\mathcal{D}}{_{t_f-}^{\alpha}}$ becomes

Note also that without the running state constraint, Theorem 5.1 is specialized to (^[20], Theorem 3.12) with $\theta = 0$. However, our proof technique of Theorem 5.1 is different from that of (^[20], Theorem 3.12), since we adopt the general measurable admissible controls.

(ii) Note that by Remark 2.1, when $\alpha = \beta = 1$ in Theorem 5.1, Theorem 5.1 is reduced to the maximum principle for the nonfractional optimal control problem with terminal and running state constraints (see ^[37,48,53] and the references therein for different versions of maximum principles for nonfractional optimal control problems with state constraints).

Remark 5.3. The maximum principle in Theorem 5.1 are of interest only when the running state constraint is nondegenerate in the sense that $\partial_X \overline{G}_i(t)^\top \neq 0$ whenever $\overline{G}_i(t) = 0$ for all $t \in [t_0, t_f]$ and $i = 1, \ldots, q$. A similar remark is given in (^[54], page 330, Remarks (b)) for the classical state-constrained optimal control problem for ordinary state equations.

6.   Example: Fractional linear quadratic control with state constraints

In this section, we study an application of Theorem 5.1 to the state-constrained fractional linear-quadratic control problem. The purpose of this section is to demonstrate the general applicability of Theorem 5.1 by providing a computation method of unknown variables $(\lambda, \xi, \theta)$ in Theorem 5.1.

Consider the following $\mathbb{R}^2$-valued left Caputo fractional differential equation with order $\alpha \in (0, 1)$ and $X(\cdot) = \begin{bmatrix} X_1(\cdot) & X_2(\cdot) \end{bmatrix}^\top$:

Let $U = [-u^\prime, u^\prime] \subset \mathbb{R}$ be the control space. The quadratic objective functional to be minimized is given by

subject to the terminal and running state constrains:

We observe that (6.1)–(6.3) hold Assumptions 3.1 and 3.2. Therefore, we can apply Theorem 5.1 to obtain the optimal solution of (LQ-P).^§

^§We assume the existence of the optimal solutions of (LQ-P).

By Theorem 5.1, the adjoint equation $p(\cdot) = \begin{bmatrix} p_1(\cdot) & p_2(\cdot) \end{bmatrix}^\top \in \mathbb{R}^2$ is given by

The transversality condition (recall $p(\cdot) = \begin{bmatrix} p_1(\cdot) & p_2(\cdot) \end{bmatrix}^\top \in \mathbb{R}^2$) is as follows:

Due to the terminal state constraints in (6.1) and the definition of the normal cone in (7.1), $\xi_1$ can be arbitrary, whereas $\xi_2$ has to be determined based on the terminal condition of (6.4).

In view of Lemma 4.3 and (^[1], Corollary 2.1 and Property 2.1, Chapter 2), the solution of the adjoint equation in (6.4) can be written as

The complementary slackness condition with $\theta(\cdot) \in \rm{NBV}([t_0, t_f]; \mathbb{R})$ is as follows (see also (7.22)):

The Hamiltonian minimization condition is as follows:

where $\mathcal{H}(t, X, p;u) : = p_1(t) X_2(t) + p_2(t) u(t) + \lambda \frac{(t_f - t)^{\beta-1}}{\Gamma(\beta)} \frac{u(t)^2}{2}$. By (6.5) and the first-order optimality condition, the (candidate) optimal solution can be obtained by (note that $\lambda \neq 0$ by the nontriviablity condition)

where

By Theorem 3.1 (and Lemma 2.2) and (6.7), the optimal state trajectory is as follows:

When $t = t_f$, the state trajectory must hold the terminal state constraint in (6.3), i.e.,

We study the computation method to obtain unknown variables $(\lambda, \xi_{21}, \xi_{22}, \theta)$. By (6.7), it should be $\lambda \neq 0$. Hence, by the nontriviality condition, we take $\lambda = 1$. Recall from Theorem 5.1 that $\theta(\cdot) \in \rm{NBV}([t_0, t_f]; \mathbb{R})$ is finite, nonnegative, and monotonically nondecreasing on $[t_0, t_f]$. Hence, $\theta$ has a countable number of jumps on $[t_0, t_f]$ (see ^[52], Lemma 3.12), which implies that we may use the approximation of ${\rm{d}} \theta([t_0, t_f]) = \sum_{k = 0}^{K} \theta_k$, where $\theta_i \geq 0$, $i = 0, \ldots, K$, and $K$ defines the $K+1$ disjoint intervals in $[t_0, t_f]$ with $t_0 \leq t_1 \leq t_2 \leq \cdots \leq t_{K-1} \leq t_K \leq t_{K+1} = t_f$. Here, you can choose $K$ such that the algebraic equation given below (see (6.14)) admits a solution. Also, as $\theta(0) = 0$, we have $\theta_0 = 0$.

When $t \in [t_{k}, t_{k+1})$, $k \in \{0, \ldots, K \}$, it follows that

For $t \in [t_{k}, t_{k+1})$, $k \in \{0, \ldots, K \}$, let

Then from (6.10), for $t \in [t_{k}, t_{k+1})$, $k \in \{0, \ldots, K \}$, it holds that

The complementary slackness condition in (6.6) becomes

where by (6.8)–(6.10) and the definition of $\widehat{\zeta}$ and $\widetilde{\zeta}$ (note that $t_{K+1} = t_f$),

Then from (6.9), (6.11) and (6.12), we arrive the following (linear and nonlinear) algebraic equations to compute $\xi_{21}$, $\xi_{22}$, and $\theta_k$, $k = 1, \ldots, K$:

and (note that $\theta_0 = 0$)

Based on the above analysis, we get the following conclusion of this section:

Proposition 6.1. The pair $(\overline{u}(\cdot), \overline{X}(\cdot)) \in \mathcal{U} \times ^{ {C}}_{n}{\mathcal{D}}{_{t_0+}^{\alpha}}$ given in (6.7) and (6.8) is the optimal solution of (LQ-P), where $(\xi_2, \theta)$ can be obtained by solving the algebraic equations in (6.13) and (6.14).

Remark 6.1. (i) In order to solve the algebraic equations in (6.13) and (6.14) and compute the optimal state trajectory $\overline{X}$ (see (6.8)) as well as the adjoint equation in (6.5), we have to rely on numerical approaches. Note that to solve (6.13) and (6.14), we need to compute RL fractional integrals, for which ^[55,56,57] provide several related numerical schemes. Various numerical methods for solving fractional differential equations to obtain the optimal state trajectory $\overline{X}$ (see (6.8)) as well as the adjoint equation in (6.5) can be found in ^[55,58,59] and the references therein.

(ii) We mention that ^[39] studied general numerical approaches for solving the linear-quadratic fractional control problem with state constraints. We believe that the approaches of ^[39] can be applied to solve the linear-quadratic problem in this section. Note that the researches in ^[44,45] also provided different numerical methods for solving fractional optimal control problems.

7.   Proof of Theorem 2

This section is devoted to prove Theorem 5.1.

7.1.   Preliminaries on distance functions

Let $d_{F} : \mathbb{R}^{2n} \rightarrow \mathbb{R}_{+}$ be the standard Euclidean distance function to $F$ defined by $d_{F}(x) : = \inf_{y \in F} |x-y|$ for $x \in \mathbb{R}^{2n}$. Note that $d_{F}(x) = 0$ when $x \in F$. Then the terminal state constraint in (3.5) can be written as $(X_0, X(t_f)) \in F \Leftrightarrow\; d_F(X_0, X(t_f)) = 0$. By the projection theorem (^[60], Theorem 2.10), there is a unique $P_{F}(x) \in F$ with $P_{F}(x) : \mathbb{R}^{2n} \rightarrow F \subset \mathbb{R}^{2n}$, the projection of $x \in \mathbb{R}^{2n}$ onto $F$, such that $d_{F}(x) = \inf_{y \in F} |x-y| = |x - P_{F}(x)|$. By (^[60], Lemma 2.11), $P_{F}(x) \in F$ is the corresponding projection if and only if $\langle x - P_{F}(x), y- P_{F}(x) \rangle \leq 0$ for all $y \in F$. In view of (^[60], Definition 2.37), we have $x - P_{F}(x) \in N_{F}(P_{F}(x))$ for $x \in \mathbb{R}^{2n}$, where $N_{F}(x)$ is the normal cone to the convex set $F$ at a point $x \in F$ defined by

Lemma 7.1. (^[61], page 167) and (^[62], Proposition 2.5.4) $d_F(x)^2$ is Fréchet differentiable with the Fréchet differentiation of $d_F(x)^2$ at $x \in \mathbb{R}^{2n}$ given by $D d_F(x)^2(h) = 2 \langle x-P_F(x), h \rangle$ for $h \in \mathbb{R}^{2n}$.

We define $\psi(X(\cdot)) : = G(\cdot, X(\cdot)) = \begin{bmatrix} G_1(\cdot, X(\cdot)) & \cdots & G_q(\cdot, X(\cdot)) \end{bmatrix}^\top$, where $\psi: C([t_0, t_f]; \mathbb{R}^n) \rightarrow C([t_0, t_f]; \mathbb{R}^q)$. Let $S : = C([t_0, t_f]; \mathbb{R}^q_{-}) \subset C([t_0, t_f]; \mathbb{R}^q)$, where $\mathbb{R}^q_{-} : = \mathbb{R}_{-} \times \cdots \times \mathbb{R}_{-}$. Note that $S$ is a nonempty closed convex cone of $C([t_0, t_f]; \mathbb{R}^q)$ with a nonempty interior (^[51], Appendix B) (see also (^[48], Section 2.2)). Then, the running state constraint in (3.5) is equivalent to

As $G$ is continuously differentiable, $\psi$ is Fréchet differentiable with its Fréchet differentiation at $X(\cdot) \in C([t_0, t_f]; \mathbb{R}^n)$ given by $D \psi(X(\cdot))(w) = \partial_X G(\cdot, X(\cdot)) w$ for $w \in C([t_0, t_f]; \mathbb{R}^n)$ (^[61], page 167). The normal cone to $S$ at $x \in S$ is defined by

where $\langle \cdot, \cdot \rangle_{C_q^* \times C_q} : = \langle \cdot, \cdot \rangle_{C([t_0, t_f]; \mathbb{R}^q)^* \times C([t_0, t_f]; \mathbb{R}^q)}$ stands for the duality paring between $C([t_0, t_f]; \mathbb{R}^q)$ and $C([t_0, t_f]; \mathbb{R}^q)^*$ with $C([t_0, t_f]; \mathbb{R}^q)^*$ being the dual space of $C([t_0, t_f]; \mathbb{R}^q)$.^¶

^¶Let $(X, \|\cdot\|_{X})$ be a Banach space. We denote $(X^*, \|\cdot\|_{X^*})$ by the dual space of $(X, \|\cdot\|_{X})$, where $X^*$ is the space of bounded linear functionals on $X$ with the norm given by $\|\psi\|_{X^*} : = \sup_{x \in X, \; \|x\|_{X} \leq 1} \langle \psi, x \rangle_{X^* \times X}$. Here, $\langle \cdot, \cdot \rangle_{X^* \times X}$ denotes the usual duality paring between $X$ and $X^*$, i.e., $\langle \psi, x \rangle_{X^* \times X} : = \psi(x)$. Recall that $(X^*, \|\cdot\|_{X^*})$ is also a Banach space.

Remark 7.1. Note that $(C([t_0, t_f]; \mathbb{R}^q), \|\cdot\|_{\infty, q})$ is a separable Banach space (^[63], Theorem 6.6, page 140). Then, by (^[43], Theorem 2.18, page 42), there exists a norm $\|\cdot \|_{C([t_0, t_f]; \mathbb{R}^q)}$ on $C([t_0, t_f]; \mathbb{R}^q)$, equivalent to $\|\cdot\|_{\infty, q}$ (^[43], Definition 2.17, page 42), such that $(C([t_0, t_f]; \mathbb{R}^q)^*, \|\cdot \|_{C([t_0, t_f]; \mathbb{R}^q)^*})$ is strictly convex, i.e., $\|x \|_{C([t_0, t_f]; \mathbb{R}^q)^*} = \|y \|_{C([t_0, t_f]; \mathbb{R}^q)^*} = 1$ and $\|x+y \|_{C([t_0, t_f]; \mathbb{R}^q)^*} = 2$ imply that $x = y$ for $x, y \in C([t_0, t_f]; \mathbb{R}^q)^*$ (^[43], Definition 2.12, page 41).

Let $d_{S}: C([t_0, t_f]; \mathbb{R}^q) \rightarrow \mathbb{R}_{+}$ be the distance function to $S$ defined by $d_S(x) : = \inf_{y \in S} \|x-y\|_{C([t_0, t_f]; \mathbb{R}^q)}$ for $x \in C([t_0, t_f]; \mathbb{R}^q)$. Then, we note that (7.2) is equivalent to $d_S (\psi(X(\cdot)) = 0 \; \Leftrightarrow \; \psi (X(\cdot)) \in S$.

Lemma 7.2. (^[62], Proposition 2.4.1 and page 53) $d_S$ is nonexpansive, continuous, and convex.

We define the subdifferential of $d_S$ at $x \in C([t_f, t_f]; \mathbb{R}^q)$ by (^[62], pages 10, 27)

By (^[62], page 27), since $d_S$ is continuous, $\partial d_S(x)$ is a nonempty ($\partial d_S(x) \neq \emptyset$), convex, and weak–$^*$ compact subset of $C([t_0, t_f]; \mathbb{R}^q)^*$. From (^[62], Proposition 2.1.2) and Lemma 7.2, $\|y^\prime\|_{C([t_0, t_f]; \mathbb{R}^q)^*} \leq 1$ for all $y^\prime \in \partial d_S(x)$. Then, by (^[43], Proposition 3.11, Chapter 4) (see also (^[51], Proposition A.2) and (^[48], Proposition 5, Appendix B.2)) and the fact that $S \subset C([t_0, t_f]; \mathbb{R}^q)$ is a nonempty closed convex subset, we can conclude that $\|y^\prime\|_{C([t_0, t_f]; \mathbb{R}^q)^*} = 1$ for all $y^\prime \in \partial d_S(x)$ with $x \notin S$.

Remark 7.2. We claim that $\partial d_S(x) = \{y^\prime\}$, i.e., $\partial d_S(x)$ is a singleton, for $x \in C([t_0, t_f]; \mathbb{R}^q) \setminus S$. To show this, let us assume that $\partial d_S(x)$ is not a singleton. We take $y^\prime, y^{\prime \prime} \in \partial d_S(x)$ with $y^\prime \neq y^{\prime \prime}$. As $\partial d_S(x)$ is convex, we have $\eta y^\prime + (1-\eta) y^{\prime \prime} \in \partial d_S(x)$ for $\eta \in [0, 1]$. Note that by the preceding discussion, $\|\eta y^\prime + (1-\eta) y^{\prime \prime} \|_{C([t_0, t_f]; \mathbb{R}^q)^*} = 1$ for $\eta \in [0, 1]$. Then $\|y^\prime\|_{C([t_0, t_f]; \mathbb{R}^q)^*} = 1$ and $\|y^{\prime \prime} \|_{C([t_0, t_f]; \mathbb{R}^q)^*} = 1$ when $\eta = 1$ and $\eta = 0$, respectively. In addition, when $\eta = \frac{1}{2}$, we have $\|y^\prime + y^{\prime \prime} \|_{C([t_0, t_f]; \mathbb{R}^q)^*} = 2$. This, together with the fact that $(C([t_0, t_f]; \mathbb{R}^q)^*, \|\cdot \|_{C([t_0, t_f]; \mathbb{R}^q)^*})$ is strictly convex by Remark 7.1, implies that $y^\prime = y^{\prime \prime}$, which contradicts our assumption $y^\prime \neq y^{\prime \prime}$. Hence, $y^\prime = y^{\prime \prime} \in \partial d_S(x)$ and $\|y^\prime\|_{C([t_0, t_f]; \mathbb{R}^q)^*} = 1$. Then we repeat this process for all elements of $\partial d_S(x)$, which implies $\partial d_S(x) = \{y^\prime\}$, i.e., $\partial d_S(x)$ is a singleton, and $\|y^\prime\|_{C([t_0, t_f]; \mathbb{R}^q)^*} = 1$.

Lemma 7.3. $d_S$ is strictly Hadamard differentiable on $C([t_0, t_f]; \mathbb{R}^q) \setminus S$ with the Hadamard differential $D d_S$ satisfying $\|D d_S(x)\|_{C([t_0, t_f]; \mathbb{R}^q)^*} = \|y^\prime\|_{C([t_0, t_f]; \mathbb{R}^q)^*} = 1$ for all $x \in C([t_0, t_f]; \mathbb{R}^q) \setminus S$. Also, $d_S(x)^2$ is strictly Hadamard differentiable on $C([t_0, t_f]; \mathbb{R}^q) \setminus S$ with the Hadamard differential given by $D d_S(x)^2 = 2 d_S(x) D d_S(x)$ for $x \in C([t_0, t_f]; \mathbb{R}^q) \setminus S$. Finally, $d_S(x)^2$ is Fréchet differentiable on $S$ with the Fréchet differential being $D d_S(x)^2 = 0 \in C([t_0, t_f]; \mathbb{R}^q)^*$ for all $x \in S$.

Proof. Recall $d_S(x) = 0$ when $x \in S$, and $d_S(x) > 0$ for $x \in C([t_0, t_f]; \mathbb{R}^q) \setminus S$. Consider $d_S$ when $x \in C([t_0, t_f]; \mathbb{R}^q) \setminus S$. The nonexpansive property of $d_S$ from Lemma 7.2 implies that $d_S$ is Lipschitz continuous. In addition, by Remark 7.2, the subdifferential of $d_S$ is a singleton. Then we are able to invoke (^[64], Theorem 3.54) to conclude that $d_S(x)$ is strictly Hadamard differentiable on $C([t_0, t_f]; \mathbb{R}^q) \setminus S$, where by Remark 7.2, $\|D d_S(x)\|_{C([t_0, t_f]; \mathbb{R}^q)^*} = 1$. Consequently, $d_S(x)^2$ is strictly Hadamard differentiable on $C([t_0, t_f]; \mathbb{R}^q) \setminus S$ with $D d_S(x)^2 = 2 d_S(x) D d_S(x)$ for $x \in C([t_0, t_f]; \mathbb{R}^q) \setminus S$. When $d_S$ is on $S$, we have $D d_S(x)^2 = 0 \in C([t_0, t_f]; \mathbb{R}^q)^*$ for $x \in S$ due to the fact that $d_S(x) = 0$ for $x \in S$ and $d_S$ is nonexpansive shown in Lemma 7.2. □

7.2.   Ekeland variational principle

Recall that $(\overline{u}(\cdot), \overline{X}(\cdot)) \in \mathcal{U} \times ^{ {{\rm{C}}}}_{n}{\mathcal{D}}{_{t_0+}^{\alpha}}$ is the optimal solution of (P). For $\epsilon > 0$, define the penalized objective functional

where $J$ is given in (3.4). Define the Ekeland metric by

where for $u(\cdot), \tilde{u}(\cdot) \in \mathcal{U}$, $\tilde{d}(u(\cdot), \tilde{u}(\cdot)) : = | \{ t \in [t_0, t_f]\; |\; u(t) \neq \tilde{u}(t) \} |$. Note that $(\mathbb{R}^n \times \mathcal{U}, \widehat{d})$ is a complete metric space (^[43], Proposition 3.10, Chapter 4), and $J_\epsilon$ is a continuous functional on $(\mathbb{R}^n \times \mathcal{U}, \widehat{d})$ in view of Lemmas 7.1 and 7.2, Theorem 3.1.^||

^||Note that Theorem 3.1 with Assumption 3.1 (particularly $\sup_{u \in U} |f(t, x, u)| < \infty$ for $(t, x) \in [t_0, t_f] \times \mathbb{R}^n$) and the Arzeá–Ascoli theorem imply that all state trajectories $X$ of the control system (3.1) is equicontinuous and bounded, and therefore relatively compact in the uniform topology. Then there is a subsequence of the state trajectories converging uniformly, which together with Assumption 3.2 and Lemmas 7.1 and 7.2 implies the continuity of $J_{\epsilon}$ on $(\mathbb{R}^n \times \mathcal{U}, \widehat{d})$.

We observe that

Then, by the Ekeland variational principle ^[65] (see also (^[43], Corollary 2.2, Chapter 4)), there exists a pair $(X_0^{\epsilon}, u^{\epsilon}(\cdot)) \in \mathbb{R}^n \times \mathcal{U}$ such that

and

From (7.8), we observe that the pair $(X_0^\epsilon, u^\epsilon(\cdot)) \in \mathbb{R}^n \times \mathcal{U}$ is the optimal solution to the unconstrained problem given by $J_{\epsilon}(X_0;u(\cdot)) + \sqrt{\epsilon} \widehat{d}((X_0^\epsilon, u^\epsilon(\cdot)), (X_0;u(\cdot)))$.

7.3.   Spike variation and notation

For $\delta \in (0, 1)$, define (with $|E|$ being the Lebesgue measure of $E$)

For any $E_{\delta} \in \mathcal{E}_{\delta}$ and $u(\cdot) \in \mathcal{U}$, we introduce the spike variation:

Clearly $u^{\epsilon, \delta} (\cdot) \in \mathcal{U}$ and $\tilde{d}(u^{\epsilon, \delta}(\cdot), u^\epsilon(\cdot)) \leq |E_{\delta}| = \delta (t_f-t_0)$. Consider the variations $X^\epsilon(\cdot) : = X(\cdot; X_0^{\epsilon}, u^\epsilon)$ and $X^{\epsilon, \delta}(\cdot) : = X(\cdot; X_0^\epsilon+\delta a, u^{\epsilon, \delta})$, where $a \in \mathbb{R}^n$. Note that $X^\epsilon(\cdot)$ is the state trajectory of (3.1) under $(X_0^\epsilon, u^\epsilon(\cdot)) \in \mathbb{R}^n \times \mathcal{U}$, and $X^{\epsilon, \delta}$ is the variational equation under (7.10) and the perturbed initial condition $a$.

We introduce (with $E^{\epsilon, \delta}(\cdot) : = X^{\epsilon, \delta}(\cdot) - X^{\epsilon}(\cdot)$)

7.4.   Variational analysis I

By (7.5) and (7.8), together with (7.6) and (7.10), we have

By Lemmas B.1 and B.2 in Appendix B, we introduce

From Lemmas B.1 and B.2, $Z^\epsilon$ and $\widehat{Z}^\epsilon$ are the variational equations related to the pair $(X_0^\epsilon, u^\epsilon(\cdot)) \in \mathbb{R}^n \times \mathcal{U}$ in (7.8). In fact, by Theorem 3.1, $Z^\epsilon$ is equivalent to (B.2) in Lemma B.1.

We now consider the limit of $\delta \downarrow 0$ in (7.11). By continuity of $J_\epsilon$ on $\mathbb{R}^n \times \mathcal{U}$, it follows that $\lim_{\delta \downarrow 0} J_\epsilon (X_0^\epsilon + \delta a; u^{\epsilon, \delta}(\cdot)) = J_\epsilon(X_0^\epsilon; u^{\epsilon}(\cdot))$. Then, the following facts hold:

(F.1) By Lemmas B.1 and B.2, we have

where since $J_\epsilon (X_0, u(\cdot)) > 0$ for $(X_0, u(\cdot)) \in \mathbb{R}^n \times \mathcal{U}$,

(F.2) From Lemmas 7.1 and B.1,

where with $\xi_1^\epsilon, \xi_2^\epsilon \in \mathbb{R}^n$,

(F.3) By Theorem 3.1, and Lemmas 7.3 and B.1,

where as $D d_S(\psi(X^{\epsilon}(\cdot)))$ is the subdifferential of $d_S$ at $\psi(X^{\epsilon}(\cdot)) \in C([t_0, t_f]; \mathbb{R}^q)$ (see Lemma 7.3), by (7.3) and (7.4), $\mu^{\epsilon} \in N_S (\psi(X^{\epsilon}(\cdot)))$, and

Therefore, by (F.1)–(F.3), as $\delta \downarrow 0$, (7.11) becomes for $(a, u(\cdot)) \in \mathbb{R}^n \times \mathcal{U}$,

where from Lemma 7.3 and the property of $d_F$ in Section 7.1,

7.5.   Variational analysis II

We consider the limit of $\epsilon \downarrow 0$. We introduce

where by Lemma B.3, $Z$ and $\widehat{Z}$ are variational equations related to the optimal solution $\overline{u}(\cdot) \in \mathcal{U}$. By Theorem 3.1, $Z$ in (7.14) is equivalent to (B.4) in Lemma B.3.

Instead of taking the limit of $\epsilon \downarrow 0$, let $\{\epsilon_k\}$ be a sequence of $\epsilon$ such that $\epsilon_k \geq 0$ and $\epsilon_k \downarrow 0$ as $k \rightarrow \infty$. We replace $\epsilon$ by $\epsilon_k$. By (7.13), the sequences $(\{\lambda^{\epsilon_k} \}, \{\xi^{\epsilon_k} \}, \{\mu^{\epsilon_k}\})$ are bounded for $k \geq 0$. Note also from (7.13) that the ball generated by $\|\mu^{\epsilon_k}\|^2_{C([t_0, t_f]; \mathbb{R}^q)^*} \leq 1$ is a closed unit ball in $C([t_0, t_f]; \mathbb{R}^q)^*$, which is weak–$^*$ compact by the Banach-Alaoglu theorem (^[63], page 130). Then, by the standard compactness argument, we may extract a subsequence of $\{\epsilon_k\}$, still denoted by $\{\epsilon_k\}$, such that as $k \rightarrow \infty$,

where the convergence of $\{\mu^{\epsilon_k}\} \rightarrow \mu$ (as $k \rightarrow \infty$) is understood in the weak–$^*$ sense ^[63].

We claim that the tuple $(\lambda, \xi, \mu)$ holds

Indeed, $\lambda \geq 0$ holds due to (F.1). $\xi \in N_F (P_F (\overline{X}_0, \overline{X}(t_f)))$ follows from (F.2) and the property of limiting normal cones (^[54], page 43). To prove $\mu \in N_S (\psi(\overline{X}(\cdot)))$, $\mu^{\epsilon_k} \in N_S (\psi(X^{\epsilon}(\cdot)))$ in (F.3) means $\langle \mu^{\epsilon_k}, z - \psi(X^{\epsilon_k}(\cdot)) \rangle_{C_q* \times C_q} \leq 0$ for $z \in S$ (see (7.3)). Then it follows that

where the convergence follows from (7.16) and Lemma B.3, together with the boundedness of $\{\mu^{\epsilon_k}\}$ by (7.13). This convergence is understood in the weak–$^*$ sense. This and (7.3) imply $\mu \in N_S (\psi(\overline{X}(\cdot)))$.

By (7.13) and (7.16), together with Lemma B.3, it follows that

and similarly, together with the definition of the weak–$^*$ convergence,

Therefore, as $k \rightarrow \infty$, (7.12) becomes for $(a, u(\cdot)) \in \mathbb{R}^n \times \mathcal{U}$,

Note that (7.18) is the crucial inequality obtained from the Ekeland variational principle as well as the estimates of the variational equations in Lemmas B.1 and B.3.

7.6.   Proof of Theorem 5.1 (iv): Complementary slackness

Let $\mu = (\mu_1, \ldots, \mu_q) \in C([t_0, t_f]; \mathbb{R}^q)^*$. For any $z = (z_1, \ldots, z_q) \in C([t_0, t_f]; \mathbb{R}^q)$, it holds that $\langle \mu, z \rangle_{C_q^* \times C_q} = \sum_{i = 1}^q \langle \mu_i, z_i \rangle_{C_1^* \times C_1}$. Recall that $\mu \in N_S (\psi(\overline{X}(\cdot)))$ from (7.17), where $\psi(\overline{X}(\cdot)) = \begin{bmatrix} G_1(\cdot, \overline{X}(\cdot)) & \cdots & G_q(\cdot, \overline{X}(\cdot)) \end{bmatrix}^\top = (\psi_1(\overline{X}(\cdot)), \ldots, \psi_q(\overline{X}(\cdot))) \in S$. Based on (7.3), this implies that

Take $z$ in (7.19) as in the following cases (note that $\overline{G}_i(\cdot) = G_i(\cdot, \overline{X}(\cdot))$):

Then, (7.19) is equivalent to

For (7.21), by the Riesz representation theorem (see (^[63], page 75 and page 382)), there is a unique $\theta(\cdot) = (\theta_1(\cdot), \ldots, \theta_q(\cdot))$ with $\theta_i(\cdot) \in \text{NBV}([t_0, t_f]; \mathbb{R})$, i.e., $\theta_i$, $i = 1, \ldots, q$, are normalized functions of bounded variation on $[t_0, t_f]$, such that we have $\langle \mu_i, z_i \rangle_{C_1^* \times C_1} = \int_{t_0}^{t_f} z_i(s) {\rm{d}} \theta_i(s) \geq 0$ for $i = 1, \ldots, q$, where every $\theta_i$ is finite, nonnegative, and monotonically nondecreasing on $[t_0, t_f]$ with $\theta_i(0) = 0$. Moreover, for (7.20), the Riesz representation theorem leads to

where ${\rm{d}} \theta_i$, $i = 1, \ldots, q$, denotes the Lebesgue-Stieltjes measure on $([t_0, t_f], \mathcal{B}([t_0, t_f]))$ with ${\rm{d}} \theta_i(t) \geq 0$ for $t \in [t_0, t_f]$ due to the fact that $\theta_i(\cdot) \in \text{NBV}([t_0, t_f]; \mathbb{R})$ is nondecreasing and finite. Hence, the relation in (7.22) proves the complementary slackness condition in Theorem 5.1.

7.7.   Proof of Theorem 5.1 (i): Nontriviality condition

Recall (7.19), i.e., for any $z \in S$, $\langle \mu, z - \psi(\overline{X}(\cdot)) \rangle_{C_q^* \times C_q} = \sum_{i = 1}^q \langle \mu_i, z_i - \psi_i (\overline{X}(\cdot)) \rangle_{C_1^* \times C_1} \leq 0$. By the Riesz representation theorem (see (^[63], page 75 and page 382)) and the fact that $\theta_i(\cdot) \in \rm{NBV}([t_0, t_g]; \mathbb{R})$, $i = 1, \ldots, q$, is finite, nonnegative, and monotonically nondecreasing on $[t_0, t_f]$ with $\theta_i(0) = 0$, it follows that $\|\mu_i \|_{C([t_0, t_f]; \mathbb{R})^*} = \|\theta_i(\cdot)\|_{\text{NBV}} = \theta_i(t_f) \geq 0$ for $i = 1, \ldots, q$.

By (7.17) and the fact that $(\overline{X}_0, \overline{X}(t_f)) \in F$ implies $P_F (\overline{X}_0, \overline{X}(t_f)) = (\overline{X}_0, \overline{X}(t_f))$ (see Section 7.1), we have $\xi = (\xi_1, \xi_2) \in N_F (\overline{X}_0, \overline{X}(t_f))$.

From the fact that $S = C([t_0, t_f]; \mathbb{R}_{-}^q)$ has an nonempty interior, there are $z^\prime \in S$ and $\sigma > 0$ such that $z^\prime + \sigma z \in S$ for all $z \in \overline{B}_{(C([t_0, t_f]; \mathbb{R}^q), \|\cdot\|_{C([t_0, t_f]; \mathbb{R}^q)})}(0, 1)$ (the closure of the unit ball in $C([t_0, t_f]; \mathbb{R}^q)$). Then similar to (7.19) and the fact that $\mu^{\epsilon} \in N_S (\psi(X^{\epsilon}(\cdot)))$, it follows that $\sigma \langle \mu^{\epsilon}, z \rangle_{C_q^* \times C_q} \leq \langle \mu^{\epsilon}, \psi(X^{\epsilon}(\cdot)) - z^\prime \rangle_{C_q^* \times C_q}$ for all $z \in \overline{B}_{(C([t_0, t_0];\mathbb{R}^q), \|\cdot\|_{C([t_0, t_f]; \mathbb{R}^q)})}(0, 1)$. From (7.13) and the definition of the norm of the dual space (the norm of linear functionals on $C([t_0, t_f]; \mathbb{R}^q)$ (see Section 7.1)), by taking the limit as $\epsilon \downarrow 0$, we get

Note that $\sigma > 0$. When $\mu = 0 \in C([t_0, t_f]; \mathbb{R}^q)^*$ and $\xi = 0$, we have $\lambda = 1$. When $\lambda = 0$ and $\mu = 0 \in C([t_0, t_f]; \mathbb{R}^q)^*$, we have $|\xi| = 1$. When $\lambda = 0$ and $\xi = 0$, we have $\mu \neq 0 \in C([t_0, t_f]; \mathbb{R}^q)^*$. This implies $(\lambda, \xi, \theta) \neq 0$, i.e., they cannot vanish simultaneously.

Based on the above discussion, it holds that $(\lambda, \xi, \theta) \neq 0$, where $\lambda \geq 0$ (see (7.17)), $\xi \in N_F (\overline{X}_0, \overline{X}(t_f))$, $\theta_i(\cdot) \in \rm{NBV}([t_0, t_g]; \mathbb{R})$ is nondecreasing with $\|\mu_i \|_{ C([t_0, t_f]; \mathbb{R})^*} = \|\theta_i(\cdot)\|_{\text{NBV}} = \theta_i(t_f) \geq 0$ for $i = 1, \ldots, q$. This shows the nontriviality condition in Theorem 5.1.

7.8.   Proof of Theorem 5.1 (ii): Adjoint equation and duality analysis

From Lemma 4.3 and the transversality condition, the solution of the adjoint equation in (5.1) can be written as

where $\Pi$ is the right RL fractional state-transition matrix associated with $\partial_X \overline{f}$ (see Definition 4.2 and Lemma 4.1 in Section 4). Moreover, from Lemma 4.2 and Theorem 3.1, $Z$ in (7.14) can be written as

where $\Pi$ is the left RL state-transition matrix associated with $\partial_X \overline{f}$ (see Definition 4.1).

By using (7.24), (7.15) and the Riesz representation theorem, (7.18) becomes

In (7.25), by Fubini's formula (see (^[52], Theorem 3.3, Chapter 6)),

Then it follows from Lemma 4.1 in Section 4 that

We may apply the same technique to the terms in (7.25) related to the initial variation $a$. Hence, by applying the above result and using (7.23) (with Lemma 4.1 in Section 4), (7.25) becomes

Below, we use (7.26) to prove the transversality condition, the nontriviality of the adjoint equation, and the Hamiltonian-like maximum condition of Theorem 5.1.

7.9.   Proof of Theorem 5.1 (iii): Transversality Condition

When $u = \overline{u}$, (7.26) becomes

Then, by the adjoint equation in (5.1), (7.27) can be written as

Recall from (7.23) and Lemma 4.3 that

Note that from Definition 2.2 and (5.1),

Then (7.28) becomes $0 \leq \langle \xi_1 + \lambda \partial_{X_0} \overline{m}^\top + I_{t_f-}^{1-\alpha}[p](t_0), a \rangle$. As this holds for $a, -a \in \mathbb{R}^n$,

Hence, (7.29) and (7.30) show the transversality condition.

7.10.   Proof of Theorem 5.1 (v): Hamiltonian minimization condition

When $a = 0$, (7.26) becomes $0 \leq \int_{t_0}^{t_f} \langle p(s), \widehat{f}(s) \rangle {\rm{d}} s + \lambda \int_{t_0}^{t_f} \frac{(t_f - s)^{\beta-1}}{\Gamma(\beta)} \widehat{l}(s) {\rm{d}} s$. Then, by (5.2),

Since $U$ is separable by Assumption 3.1, there exists a countable dense set $U_0 = \{u_i, \; i \geq 1\} \subset U$. Moreover, there exists a measurable set $S_i \subset [t_0, t_f]$ such that $|S_i| = t_f-t_0$ and any $t \in S_i$ is the Lebesgue point of $\mathcal{H}(t, \overline{X}(t), p(t); u(t))$, i.e., $\lim_{\tau \downarrow 0} \frac{1}{2\tau} \int_{t-\tau}^{t+\tau} \mathcal{H}(s, \overline{X}(s), p(s); u(s)) {\rm{d}} s = \mathcal{H}(t, \overline{X}(t), p(t); u(t))$ (^[66], Theorem 5.6.2). We fix $u_i \in U_0$. For any $t \in S_i$, define

It then follows that

Note that (7.31) implies that

Since $\cap_{i \geq 1} S_i = [t_0, t_f]$ by the fact that $U_0$ is countable, $\mathcal{H}$ is continuous in $u \in U$, and $U$ is separable, (7.32) implies the Hamiltonian minimization condition. This is the end of the proof for Theorem 5.1.

8.   Conclusions

In this paper, we obtained the Pontryagin maximum principle for the fractional optimal control problem with terminal and running state constraints (see Theorem 5.1). Indeed, our main result indicates the maximum principle leads to the necessary optimality conditions for the problem, which include the nontriviality condition, the transversality condition, the complement slackness condition, and the Hamiltonian minimization condition. Regarding the framework of general theory of control systems, the maximum principle of this paper provides a new approach for solving fractional optimal control problems with state constraints. Indeed, we need to develop the new proof of the maximum principle due to the inherent complex nature of the fractional control problem, the presence of the terminal and running state constraints, and the generalized standing assumptions.

There are several potential future research directions arising from this paper: (i) Generalization to other fractional differential equations such as the Caputo-Katugampola differential equation and systems driven by fractional Brownian motion ^[67]; (ii) studying the dynamic programming principle for the fractional control problem with state constraints; and (iii) the fractional control problem with delay and state constraints ^[36]. Furthermore, in engineering problems, instead of the open-loop optimal solution, the closed-loop optimal solution is more appropriate. Hence, one important future research problem is the characterization of the closed-loop optimal solution, particularly for the linear-quadratic problem. Note that we may have to extend techniques of ^[40] to the case of state constraints.

We should mention that similar to ^[20], our study has limitations in applications to real practical problems, including those for high-order systems. Recently, for the linear-quadratic case, various numerical methods for several practical fractional control applications were studied in ^[39] (e.g., the optimal drug scheduling of cancer chemotherapy problem and the minimum fuel optimal control problem; see [39,Section 6 and Example 9]). In addition, the tumor growth problem in fractional optimal control was considered in ^[45]. It is important to extend the major ideas of ^[39] to general nonlinear fractional optimal control problems to study practical applications.

Use of AI tools declaration

The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

Acknowledgments

This research was supported in part by the Korea Institute of Energy Technology Evaluation and Planning (KETEP) and the Ministry of Trade, Industry & Energy (MOTIE) of the Republic of Korea (RS-2024-00422103) and in part by Institute of Information & communications Technology Planning and Evaluation (IITP) grant funded by the Korea government (Ministry of Science and ICT, MSIT) (RS-2020-II201373).

The author would like to thank two anonymous reviewers for their careful reading and suggestions, which have helped to improve the paper.

Conflict of interest

All authors declare no conflicts of interest in this paper.

Appendix

A.   Proof of Theorem 1

We prove Theorem 3.1.

A.1.   Proof of (i)

We first show (3.1) implies the integral expression. As $X(\cdot) \in ^{ {{\rm{C}}}}_{n}{\mathcal{D}}{_{t_0+}^{\alpha}}$, ${\rm{I}}_{t_0+}^{1-\alpha}[X(\cdot) -a](\cdot)\in \text{AC}([t_0, t_f]; \mathbb{R}^{n})$. By Definition 2.3,

As ${\rm{I}}_{t_0+}^{1-\alpha}[X(\cdot) -X_0](t_0) = 0$ and $f(\cdot, X, u) \in L^{\infty}([t_0, t_f]; \mathbb{R}^{n})$, by Lemmas 2.1–2.3, we have

This implies ${\rm{I}}_{t_0+}^{1}[X(\cdot) -X_0](t) = {\rm{I}}_{t_0+}^{1} \bigl [{\rm{I}}_{t_0+}^{\alpha} [f(\cdot, X(\cdot), u(\cdot))](\cdot) \bigr] (t)$. Differentiating the above equality shows the desired expression.

We prove the integral expression implies (3.1). Note that by Lemma 2.3, we have ${\rm{I}}_{t_0+}^{\alpha}[f(\cdot, X(\cdot), u(\cdot)] (\cdot) \in C([t_0, t_f]; \mathbb{R}^n)$, which implies $X(t_0) = X_0$. Then, by Lemmas 2.1–2.3, it follows that ${\rm{I}}_{t_0+}^{1-\alpha} [X(\cdot) -X_0](t) = {\rm{I}}_{t_0+}^{1-\alpha} [{\rm{I}}_{t_0+}^{\alpha}[f(\cdot, X(\cdot), u(\cdot)] (\cdot)](t) = {\rm{I}}_{t_0+}^{1}[f(\cdot, X(\cdot), u(\cdot)] (t)$. Since $f(\cdot, X, u) \in L^{\infty}([t_0, t_f]; \mathbb{R}^{n})$, it is clear that $X(\cdot) \in \text{AC}([t_0, t_f]; \mathbb{R}^n)$, which, together with Definition 2.3, implies that $^{{C}}{ D}{_{t_0+}^{\alpha}}[X](t) = \frac{{\rm{d}}}{{\rm{d}} t} \Bigl [{\rm{I}}_{t_0+}^{1-\alpha} [X(\cdot) -X_0] \Bigr] (t) = f(t, X(t), u(t))$.

A.2.   Proof of (ii)

The proof for the existence and uniqueness is modified from (^[41], Theorem 3). For $t \in [t_0, t_f]$, define $\mathcal{F}[X(\cdot)](t) : = X_0 + {\rm{I}}_{t_0+}^{\alpha}[f(\cdot, X(\cdot), u(\cdot)] (t)$. Below, we show that $\mathcal{F}$ admits a unique fixed point on $C([t_0, t_f]; \mathbb{R}^{n})$. Then the existence and uniqueness of the solution in $^{ {{\rm{C}}}}_{n}{\mathcal{D}}{_{t_0+}^{\alpha}}$ follows from (i) of Theorem 3.1.

Recall that $\|X(\cdot)\|_{\infty} : = \|X(\cdot)\|_{\infty, n} : = \sup_{s \in [t_0, t_f]} | X(s)|$ for $X(\cdot) \in C([t_0, t_f]; \mathbb{R}^{n})$. By a slight abuse of notation, for $k \in \mathbb{N}$ and $X(\cdot) \in C([t_0, t_f]; \mathbb{R}^{n})$, define the Bielecki norm: $\|X(\cdot)\|_{\infty, k} : = \max_{s \in [t_0, t_f]} |e^{-k(s-t_0)} X(s)|$. Note that $\|X(\cdot)\|_{\infty, k} \leq \|X(\cdot)\|_{\infty} \leq e^{k(t_f-t_0)} \|X(\cdot)\|_{\infty, k}$.^**

^**This means that $\|\cdot\|_{\infty, k}$ is equivalent to $\|\cdot\|_{\infty}$, and the space $(C([t_0, t_f]; \mathbb{R}^{n}), \|\cdot\|_{\infty, k})$ is complete.

As $f(\cdot, X, u) \in L^{\infty}([t_0, t_f]; \mathbb{R}^{n})$, we have $\mathcal{F}:C([t_0, t_f]; \mathbb{R}^{n}) \rightarrow C([t_0, t_f]; \mathbb{R}^{n})$. By Assumption 3.1, for any $X (\cdot), X^\prime (\cdot) \in C([t_0, t_f]; \mathbb{R}^{n})$, it follows that

Note that $| X(s) - X^\prime(s)| \leq e^{k(s - t_0)} \|X(\cdot) - X^\prime(\cdot)\|_{\infty, k}$ for $s \in [t_0, t_f]$. Hence,

By definition of the Gamma function $\Gamma$, $\int_{t_0}^{t} (t-s)^{\alpha - 1} e^{-k(t-s)} {\rm{d}} s \leq \int_0^{\infty} s^{\alpha - 1} e^{-ks} {\rm{d}} s = \frac{\Gamma(\alpha)}{k^{\alpha}}$. This leads to $\|\mathcal{F}[X(\cdot)](\cdot) - \mathcal{F}[X^\prime(\cdot)](\cdot) \|_{\infty, k} \leq \|X(\cdot) - X^\prime(\cdot)\|_{\infty, k} \frac{L}{k^{\alpha}}$. Then, by choosing $k \in \mathbb{N}$ such that $\frac{L}{k^{\alpha}} < 1$, $\mathcal{F}:C([t_0, t_f]; \mathbb{R}^{n}) \rightarrow C([t_0, t_f]; \mathbb{R}^{n})$ is contraction. Note that this holds for any $t \in [t_0, t_f]$, where $\frac{L}{k^{\alpha}} < 1$ can be chosen independently of $t$. Therefore, by (i) of Theorem 3.1 and the contraction mapping theorem, (3.1) admits a unique solution of $X(\cdot) \in ^{ {{\rm{C}}}}_{n}{\mathcal{D}}{_{t_0+}^{\alpha}}$.

A.3.   Proof of (iii)

Let $X(\cdot) : = X(\cdot; X_0, u)$ and $X^\prime(\cdot) : = X(\cdot; X_0^\prime, u^\prime)$. By Assumption 3.1, we can show that $|X(t) - X^\prime(t)| \leq |X_0 - X_0^\prime| + L \int_{t_0}^{t} \frac{(t-s)^{\alpha-1}}{\Gamma(\alpha)} [|X(s)- X^\prime(s)| + \rho(u(s), u^\prime(s))] {\rm{d}} s$. By Gronwall's inequality (see [68,Theorem 1]) with $b(t) = |X_0 - X_0^\prime| + L \int_{t_0}^{t} \frac{(t-s)^{\alpha-1}}{\Gamma(\alpha)} \rho(u(s), u^\prime(s)) {\rm{d}} s$ and $z(t) = |X(t) - X^\prime(t)|$, it follows that $|X(t) - X^\prime(t)| \leq b(t) + \int_{t_0}^{t} \sum_{k = 1}^{\infty} \frac{(L \Gamma(\alpha))^k}{\Gamma(k \alpha)} (t-s)^{k \alpha -1} b(s) {\rm{d}} s$. Hence, (3.3) follows. We can show (3.4) using a similar technique; thus completing the proof.

B.   Variational analysis

Lemma B.1. The following result holds:

where $Z^\epsilon$ is the variational equation related to the pair $(X_0^\epsilon, u^\epsilon(\cdot)) \in \mathbb{R}^n \times \mathcal{U}$ expressed by the left Caputo fractional differential equation (with $\partial_X f^\epsilon$ and $\widehat{f}^\epsilon$ defined in Section 7.3):

Proof. Let $Z^{\epsilon, \delta}(t) : = \frac{X^{\epsilon, \delta}(t) - X^{\epsilon}(t)}{\delta}$, where by the Taylor series expansion, $Z^{\epsilon, \delta}$ satisfies (with $f_X^{\epsilon, \delta}$ defined in Section 7.3)

Then (B.1) is equivalent to showing that $\lim_{\delta \downarrow 0} \sup_{t \in [t_0, t_f]} |Z^{\epsilon, \delta}(t) - Z^\epsilon(t)| = 0$ for $t \in [t_0, t_f]$. Hereafter, $K$ represents a generic constant, which may differ at different places.

By Theorem 3.1, it follows that $\sup_{t \in [t_0, t_f]} |X^{\epsilon, \delta}(t)| \leq K$ and $\sup_{t \in [t_0, t_f]} |X^{\epsilon}(t)| \leq K$. Then, by Theorem 3.1 and (7.10) (with some minor modification of the proof for Theorem 3.1),

By Theorem 3.1, (B.2) is equivalent to

As (B.2) is a linear Caputo fractional differential equation, by Theorem 3.1, (B.2) admits a unique solution. In addition, by Theorem 3.1, $\sup_{t \in [t_0, t_f]} |Z^{\epsilon}(t)| \leq K$.

We have

By Assumption 3.1, there is a constant $K \geq 0$ such that

where

For $b_1$, based on (7.10),

In view of Assumption 3.1, and the fact that $\tilde{d}(u^{\epsilon, \delta}(\cdot), u^\epsilon(\cdot)) \leq |E_{\delta}| = \delta (t_f-t_0)$ and $\sup_{t \in [t_0, t_f]} |Z^{\epsilon}(t)| \leq K$ by Theorem 3.1, the first term can be estimated by the dominated convergence theorem:

Similarly, by (7.10) and (B.3), and the fact that $\tilde{d}(u^{\epsilon, \delta}(\cdot), u^\epsilon(\cdot)) \leq |E_{\delta}| = \delta (t_f-t_0)$ and $\sup_{t \in [t_0, t_f]} |Z^{\epsilon}(t)| \leq K$ by Theorem 3.1, the second term can be estimated by the dominated convergence theorem:

These two estimates imply that for any $t \in [t_0, t_f]$,

Then, it follows that $\lim_{\delta \downarrow 0} \sup_{t \in [t_0, t_f]} b_1(t) = 0$. In addition, from Assumption 3.1 and Lemma C.1, it follows that $\sup_{t \in [t_0, t_f]} b_2(t) \leq \delta = o(1)$.

By the preceding analysis, we have

In view of Fractional Gronwall's inequality (see ^[68], Theorem 1 and Corollary 2), it follows that $\sup_{t \in [t_0, t_f]} |Z^{\epsilon, \delta}(t) - Z^{\epsilon}(t)| \leq o(1) E_{\alpha} [K \Gamma(\alpha) (t_f-t_0)^{\alpha}]$. This implies that as $\delta \downarrow 0$, $\sup_{t \in [t_0, t_f]} |Z^{\epsilon, \delta}(t) - Z^{\epsilon}(t)| \rightarrow 0$, i.e., $\lim_{\delta \downarrow 0} \sup_{t \in [t_0, t_f]} |Z^{\epsilon, \delta}(t) - Z^\epsilon(t)| = 0$, which shows (B.1). We complete the proof. □

Lemma B.2. The following result holds: as $\delta \downarrow 0$,

where

Proof. Recall $Z^{\epsilon, \delta}(t) = \frac{X^{\epsilon, \delta}(t) - X^{\epsilon}(t)}{\delta}$. By the Taylor series expansion,

Note that $\lim_{\delta \downarrow 0} |Z^{\epsilon, \delta}(s) - Z^{\epsilon}(s)| = 0$ for $s \in [t_0, t_f]$ by Lemma B.1. From Assumption 3.2 and Lemma C.1, it follows that $|\int_{t_0}^{t_f} \frac{(t_f - s)^{\beta-1}}{\Gamma(\beta)} (\frac{ \mathbb{1}_{E_{\delta}}(s)}{\delta} - 1) \widehat{l}^{\epsilon}(s) {\rm{d}} s | \leq \delta$. Then using a similar proof technique of Lemma B.1, we can show the desired result. This completes the proof. □

Lemma B.3. The following results hold:

(i) $\lim_{\epsilon \downarrow 0} \widehat{d}((\overline{X}_0;\overline{u}(\cdot)), (X_0^\epsilon, u^\epsilon(\cdot))) = 0$;

(ii) $\sup_{t \in [t_0, t_f]} | Z^{\epsilon}(t) - Z(t) | = o(1)$ and $| \widehat{Z}^{\epsilon}(t_f) - \widehat{Z}(t_f) | = o(1)$ as $\epsilon \downarrow 0$, where $Z$ is the left Caputo fractional differential equation given by

and $\widehat{Z}$ is given by

Proof. Part (i) follows from (7.7). The proof for part (ii) is similar to that for Lemmas B.1 and B.2. We complete the proof. □

C.   Technical lemma

Lemma C.1. For any $\alpha \in (0, 1)$, $\delta \in (0, 1)$ and $g \in L^1([t_0, t_f]; \mathbb{R})$, there is an $E_{\delta} \in \mathcal{E}_{\delta}$, where $\mathcal{E}_{\delta}$ is defined in (7.9), such that

Proof. Note that as $g \in L^1([t_0, t_f]; \mathbb{R})$, there is a simple function $g^{(n)}$, which is an approximation of $g$ in $L^1$ such that $g^{(n)} \rightarrow g$ in $L^1$ as $n \rightarrow \infty$ (^[66], Lemma 4.2.1). For any $\epsilon \geq 0$, consider

Note that $\frac{(t-s)^{\alpha-1}}{\Gamma(\alpha) + \epsilon} g^{(n)} (s)$ is continuous in $t$ with $s < t \leq t_f$ and is an element of $L^1$ in both variables $(t, s)$ with $t_0 \leq s < t \leq t_f$. Then, by (^[43], Corollary 3.8, Chapter 4), there is an $E_{\delta} \in \mathcal{E}_{\delta}$ such that

Also, from (^[66], Lemma 4.2.1), as $g^{(n)}$ is an approximation of $g$, for any $t \in [t_0, t_f]$,

By the dominated convergence theorem, $\sup_{t \in [t_0, t_f]} H^{(2)} = o(1)$ as $\epsilon \downarrow 0$. Note that these estimates are independent of $t \in [t_0, t_f]$. Hence, by combining the preceding estimates, as $\epsilon \downarrow 0$, the result follows. This completes the proof. □