Uncertain random problem for multistage switched systems

1.   Introduction

Switched systems consist of several subsystems, where under some presumed conditions, only one of the subsystems is active at each switching time. Networked control systems with possible communication interruptions, automobile transmission systems and on/off heating systems are all typical examples of switched systems.

A simple multistage switched system can be described as

where $v(t)$ is the discrete switching control rule that determines the corresponding active subsystem at time $t\in Z^{+}$, $\boldsymbol{u}(t)$ is the control input, and $\boldsymbol{x}(t)$ is the state.

Unlike customary non-switched system optimal control issues, the challenge for optimal control problems of switching systems is to choose the switching control $v(t)$ determines the switching sequence of the subsystems. In recent years, the problem has extensively studied ^{[1,2,3,4,5,6,7]}. For example, Zhu and Antsaklis ^[2] focusd on optimal control techniques for piecewise affine according to the type of switching, for the IFS problem. For the EFS problem, two-stage optimization, embedding transformation and switching LQR design methods are studied. Wu et al. ^[6] studied the optimal control problem of a switching system with input and state constraints. Yang et al. ^[7] concentrated on the sliding mode control problem for a set of discrete-time switched systems.

So far, the research on optimal control problems for deterministic switched systems is the focus of most scholars. However, the complexity of the real world leads us to face various forms of indeterminacy. In many cases, two kinds of indeterminacy exist in the control system. One is characterized as randomness and implemented as stochastic noise^{[8,9,10,11,12]}. For example, Liu et al. ^[11,12] solves the optimal control problem for a class of time-invariant stochastic switching systems with multiple switching times. The other is subjective which is modeled as uncertain noise^{[13,14,15,16]}, Jia et al. ^[15] studied finite-time synchronization (FTS) of uncertain fractional-order memristive neural networks (FMNNs) with leakage and discrete and Suriguga^[16] studies the mean square exponential stability of higher order Markovian jump reaction-diffusion HNNs (RHNNs) with uncertain transition rates (GUTR) and time-varying delays. However, in practice, the system may be affected not only by randomness but also by uncertainty. To handle this system, Liu^[17] introduced chance theory to model complicated including both randomness and uncertainty. After that, Liu^[18] studied uncertain stochastic programming and proposed its mathematical properties, the concept of uncertain stochastic graphs and uncertain stochastic networks. Numerous academics have recently conducted substantial research in this field. In the chance space, Yu et al. ^[19] looked into uncertain random variables based on the possibility that uncertainty and randomness phenomena may occur simultaneously in some complex systems. Li et al.^[20] applied uncertain random variables to security returns in financial markets, in the an uncertain stochastic portfolio selection issue with simultaneous stochastic and uncertain returns is presented. Based on the study of the significance of components in uncertain random dependability systems, Gao and Yao ^[21] have developed the notion of an importance index that is connected to this. Chen et al. ^[22] proposed the optimal equation to solve the dynamic optimization problems under uncertain stochastic continuous-time systems.

Currently, the study of uncertainty optimal control models for switching systems is mostly separate from the study of random optimal control. However, uncertainty and randomness can exist simultaneously in reality. Uncertain random systems may be described by differential equations or difference equations are concerned with the dynamic uncertain random phenomena. The uncertain random phenomena occur in a variety of aspects, such as uncertain random programming ^[23,24] and uncertain random portfolio selection ^[25,26]. However, the optimal control of uncertain random switched systems has not been studied yet. Different from the separate indeterministic environment (stochastic or uncertain situation), this paper considers an optimal control problem for uncertain random switched systems. The dynamic systems are described by both the stochastic differential equation and uncertain differential equation. For the uncertain stochastic optimal control of switched systems, the goal of realizing the optimal control is to make the system realize the dynamic evolution in different stages according to the optimal logical relations under the constraints of uncertain random variables. For multistage uncertain random switched systems, two kinds of linear quadratic (LQ) optimal control problems are examined in this study. The first model is a LQ model where the disturbance is described by the sum of an uncertain variable and a random variable. The second model is also a LQ model but the disturbance is described by the product of an uncertain variable and a random variable. We derive the analytical expressions of these two models. The analytical expressions are derived characterizing by a sequence of sets with ordered pairs of matrices. The size of these sets grows exponentially. To overcome difficulty, we apply genetic algorithm to achieve the optimal control. Our paper contributes to the literature in the following way. First, an optimal control problem for uncertain random switched systems is introduced. Second, the recurrence equation for switched systems in uncertain random environments is given, and then the equation for the proposed model is obtained. Third, the analytical expressions of the optimal control and optimal values of two types of optimal control problems are obtained. At last, optimal control is implemented by genetic algorithm instead of enumerating all the elements of a series of sets whose size grows exponentially.

The following describes how this paper is organized. In section 2, some fundamental concepts and theorems in chance theory are reviewed. In section 3, the expectation criteria is used to build an uncertain stochastic optimal control problem of multistage switched systems. In section 4, two kinds of LQ models are introduced and the analytical expressions of the optimal result are derived. In section 5, genetic algorithm and its implementation are presented. In section 6, three illustrative numerical examples are given. Finally, section 7 summarizes this paper.

2.   Preliminaries

Liu ^[17] proposed chance theory to deal with the uncertain random phenomena.

The triple $(\Gamma_{v}, \mathcal{L}_{v}, \mathcal{M}_{v}) \times (\Omega_{r}, \mathcal{A}_{r}, P_{r})$ = $(\Gamma_{v} \times \Omega_{r}, \mathcal{L}_{v} \times \mathcal{A}_{r}, \mathcal{M}_{v} \times P_{r})$ is known as a chance space^[17,18], where $(\Gamma_{v}, \mathcal{L}_{v}, \mathcal{M}_{v})$ is an uncertainty space and $(\Omega_{r}, \mathcal{A}_{r}, P_{r})$ is a probability space. $\Gamma_{v} \times \Omega_{r}$, $\mathcal{L}_{v} \times \mathcal{A}_{r}$, and $\mathcal{M}_{v} \times P_{r}$ are the universal set, $\sigma$-algebra and product measure, respectively. Define an uncertain random variable $\xi^{vr}$ in the chance space $(\Gamma_{v}, \mathcal{L}_{v}, \mathcal{M}_{v}) \times (\Omega_{r}, \mathcal{A}_{r}, P_{r})$ = $(\Gamma_{v} \times \Omega_{r}, \mathcal{L}_{v} \times \mathcal{A}_{r}, \mathcal{M}_{v} \times P_{r})$ with chance distribution

The expected value of $\xi^{vr}$ is defined by

provided that one of $\int_{0}^{+ \infty }Ch\{ \xi^{vr} \ge z\}dz$ and $\int_{- \infty}^{0}Ch\{\xi^{vr} \le z\}dz$ is finite. The chance measure of $\xi^{vr}$ is defined as

Theorem 2.1. (Liu^[17]) For numbers $A$ and $B$, we have

if $\xi^{vr}$ is an uncertain random variable and its expected value exists.

Theorem 2.2. (Liu^[18]) $\zeta _{1}, \zeta _{2}, \cdots, \zeta _{m}$ are independent random variables whose probability distributions are $\Psi _{1}, \Psi _{2}, \cdots, \Psi _{m}$, $\eta _{1}, \eta _{2}, \cdots, \eta _{m}$ are uncertain variables, and $f$ is a measurable function. Then

has an expected value

where $E_{ch}[f(z _{1}, z _{2}, \cdots, z _{m}, \eta _{1}, \eta _{2}, \cdots, \eta _{m})]$ is the expected value of the uncertain variable $f(z _{1}, z _{2}, \cdots, z _{m}, \eta _{1}, \eta _{2}, \cdots, \eta _{m})$ for any real numbers $z _{1}, z _{2}, \cdots, z _{m}$.

Lemma 2.1. (Chen ^[27]) If the independent uncertain random variables $\xi^{vr}_{1}, \xi^{vr}_{2}, \cdots, \xi^{vr}_{m}$ whose expected values exist, it follows

3.   Problem statement

Chance theory is effective at resolving problems when switched systems are perturbed by indeterminate factors which include both uncertainty and randomness. In this section, we will introduce uncertain random problems for multistage switched systems. The general form of the model can be written as follows:

(i) where $\boldsymbol{x}(t)\in R^{l}$, $\boldsymbol{u}(t)\in R^{r}$ are respectively the state vector and the control vector at stage $t$, for $t\in \{0, 1, \cdots, \tau \}$; (ii) $v(t)\in {N}\triangleq \{1, \cdots, n\}$ denotes the switching control of the subsystem at stage $t$; (iii) for each $i \; \in$ N, $\boldsymbol{\psi}_{i}, f_{i}: R^{l} \times R^{r} \times [0, \infty) \rightarrow R^{l}$ are vector-value functions, associated with the subsystem $i$, the system's first stage vector is denoted by $\boldsymbol{x}(0)$; (iv) for each $t \in \{0, 1, \cdots, \tau \}$, $\boldsymbol{\sigma}_{t}\neq 0$, ${h} (\xi_{t}, \eta _{t}, t)$ is the disturbance, $\xi_{1}, \xi_{2}, \cdots, \xi_{\tau }$ are random variables, $\eta_{1}, \eta_{2}, \cdots, \eta_{\tau }$ are uncertain variables, they are independent of each other.

It is worth noting that the switching signal $v(\cdot)$ is viewed as an external input. $Problem \ (3.1)$ of interest is to design jointly a continuous input $\boldsymbol{u}(\cdot)$ and a discrete switching sequence $v(\cdot)$ so as to optimize the performance.

For $0\leq j \leq \tau$, let $J(\boldsymbol{x}_{j}, j)$ be the optimal reward obtained in $[j, \tau]$ provided that at stage $j$, we are in state $\boldsymbol{x}(j) = \boldsymbol{x}_{j}$.

Based on the dynamic approach, the recurrence formula can be found as follows:

Theorem 3.1. For $Problem \ (3.1)$, we have

Reference ^[28] provides a reference for the proof of Theorem 3.1. According to Theorem 3.1, we should solve Eqs (3.2) and (3.3) sequentially from the final stage to the first stage or in reverse order in order to solve ${Problem \ (3.1)}$.

4.   Analytical solution for linear quadratic model

The following uncertain stochastic optimal control problem, which has a linear quadratic objective function and is part of an uncertain random linear system, can be analytically solved by applying recurrence Eqs (3.2) and (3.3).

where for any $i\in N$, $\boldsymbol{Q}_{i}\geq 0$, $\boldsymbol{R}_{i} > 0$, $(\boldsymbol{G}_{i}, \boldsymbol{F}_{i})$ and $(\boldsymbol{Q}_{i}, \boldsymbol{R}_{i})$ constitute the state constraints matrix pair and the cost matrix pair of the $i$-th subsystem, respectively. $\boldsymbol{Q}_{f} > 0$ is the terminal penalty matrix.

We will discuss two cases of uncertain random variables ${h} (\xi_{t}, \eta _{t}, t)$ : the sum $\xi_{t}$ + $\eta _{t}$ and the product $\xi_{t} \cdot \eta_{t}$.

4.1.   LQ model with uncertain random variable ${h} (\xi_{t}, \eta _{t}, t)=\xi _{t}\times \eta _{t}$

Now we consider $Problem \ (4.1)$ when uncertain random variable ${h} (\xi_{t}, \eta _{t}, t) = \xi _{t}\times \eta _{t}$. As in ^[29], define the following Riccati operator $\rho _{i}(\boldsymbol{A}): S_{l}^{+} \rightarrow S_{l}^{+}$ for given $i\in N$ and $\boldsymbol{A}\in S_{l}^{+}$,

Let $\{H_{i}\}_{i = 0}^{\tau }$ stand for the set of recursively defined ordered pairs of vectors:

with

for $j = 0, 1, \cdots, \tau -1$.

Assume that the following condition is true for each $i\in N$, $j = 0, 1, \cdots, \tau -1$ and $A\geqslant 0$,

this means that at each stage $j$, the interference on each subsystem is relatively small.

Theorem 4.1. Suppose ${h} (\xi_{t}, \eta _{t}, t) = \xi _{t}\times \eta _{t}$, where $\xi _{t}\sim \mathcal{U}(-1, 1)$ is a uniform random variable and $\eta _{t}\sim \mathcal{L}(-1, 1)$ is a common linear uncertain variable, for $t$ = 0, 1, 2, $\cdots, \; \tau -1$. Then at stage $j$, for given $\boldsymbol{x}_{j}$, the optimal switching control of ${Problem \ (4.1)}$ is

and the optimal continuous control

where

The optimal value of ${Problem \ (4.1)}$ is

Proof. It can be proved by induction.

We will derive the analytical solution of ${Problem \ (4.1)}$. First, we have

For $j = \tau -1$, the following equation holds by Theorem 3.1,

Denote $\boldsymbol{a}_{\tau -1} = 2\left (\boldsymbol{G}_{v\left (\tau -1 \right)} \boldsymbol{x}(\tau -1)+\boldsymbol{F}_{v(\tau -1)}\boldsymbol{u}(\tau -1)\right)^{T}\boldsymbol{Q}_{f}\boldsymbol{\sigma} _{\tau -1}$, $\boldsymbol{b}_{\tau -1} = \boldsymbol{\sigma} _{\tau -1}^{T}\boldsymbol{Q}_{f}\boldsymbol{\sigma} _{\tau -1}$. Suppose ${h} (\xi_{t}, \eta _{t}, t) = \xi _{t}\times \eta _{t}$, where $\xi _{t}\sim \mathcal U(-1, 1)$ is a uniform random variable and $\eta _{t}\sim \mathcal L(-1, 1)$ is a common linear uncertain variable, for $t = 0, 1, \cdots, \tau -1,$ we have

Because $\frac{a_{\tau -1}}{b_{\tau -1}z_{\tau -1}}\eta _{\tau -1} +\eta _{\tau -1}^{2}$ is an uncertain variable, according to ^[30], we have

For $z_{(\tau -1)}\in (-1, 1)$, with condition (4.3), we know that $\left | \frac{a_{\tau -1}}{b_{\tau -1}z_{\tau -1}}\right |\geqslant 4$. According to Example 2 in ^[31], we can get $E_{ch} \left [\frac{a_{\tau -1}}{b_{\tau -1}z_{\tau -1}}\eta _{\tau -1} +\eta _{\tau -1}^{2} \right] = \frac{1}{3}$. Thus

Substituting (4.5) into (4.4) yields

The optimal control $u^{*}(\tau -1)$ satisfies

Since

we have

Substituting (4.8) into (4.6), we get

According to the definition of $\rho _{i}(\boldsymbol{A})$ and $H_{j}$, Eq.(4.9) can be written as

Moreover, we have

For $j = \tau -2$, we have

From a calculation similar to (4.5), it can be seen that

By a similar process as above, we can obtain

and

By induction, Theorem 4.1 can be proved. □

In this section, LQ model with uncertain random variable ${h} (\xi_{t}, \eta _{t}, t) = \xi _{t}\times\eta _{t}$ is derived in details. The optimal switching control and continuous control of $Problem \ (4.1)$ in the ${h} (\xi_{t}, \eta _{t}, t) = \xi _{t}+ \eta _{t}$ case still needs to be derived, we will argue in the next section.

4.2.   LQ model with uncertain random variable ${h} (\xi_{t}, \eta _{t}, t)=\xi _{t}+ \eta _{t}$

Theorem 4.2. Suppose ${h} (\xi_{t}, \eta _{t}, t) = \xi _{t}+ \eta _{t}$, where $\xi _{t}\sim \mathcal{U}(-1, 1)$ is a uniform random variable and $\eta _{t}\sim \mathcal{L}(-1, 1)$ is a common linear uncertain variable, for $t = 0, 1, 2, \cdots, \tau -1$. Then at stage $j$, for given $x_{j}$, the optimal switching control of ${Problem \ (4.1)}$ is

and the optimal continuous control

where

The optimal value of ${Problem \ (4.1)}$ is

Proof. If ${h} (\xi_{t}, \eta _{t}, t) = \xi _{t}+ \eta _{t}$, for $t = 0, 1, 2, \cdots, \tau -1$. The conclusions may be not the same as Theorem 4.1.

At stage $j$, denote $\boldsymbol{a}_{j} = 2\left (\boldsymbol{G}_{v\left (j \right)} \boldsymbol{x}_{j}+\boldsymbol{F}_{v(j)}\boldsymbol{u}(j)\right)^{T}\boldsymbol{A}_{j+1}\boldsymbol{\sigma} _{j}$, $\boldsymbol{b}_{j} = \boldsymbol{\sigma} _{j}^{T}\boldsymbol{A}_{j+1}\boldsymbol{\sigma} _{j}$. We have

By using Theorem 2.2, we have

For $z_{(j)}\in (-1, 1),$ we know that $\left | \frac{a_{j}}{b_{j}}+2z_{j}\right |\geqslant 2$. According to Example $2$ in ^[31], we can get $E_{ch} \left [(\frac{a_{j}}{b_{j}}+2z_{j})\eta _{j} +\eta _{j}^{2} \right] = \frac{1}{3}$. Thus

□

Then by the similar process to the proof of Theorem 4.1, the theorem can be proved.

According to Theorem 4.1 and Theorem 4.2, we can obtain the analytical expressions of optimal control input and the optimal switching law by finding all the elements in the $H_{j} \; (j = \tau, \tau -1, \cdots, 0)$.

The cardinality of $H_{j}$ is $n^{j}$ which grows exponentially with respect to $j$. When the number of subsystems $n$ and stages $\tau$ of the uncertain system is small, we can adapt the enumeration algorithm. However, when the number of subsystems and stages becomes big, the enumeration method is no longer applicable. Genetic algorithm is an effective method that can be used to find optimal switching control and optimal continuous control and may be a good choice to implement efficiently.

5.   Genetic algorithm and its implementation

Genetic algorithm ^[32] is one of the artificial intelligence exhaustive searching techniques that is inspired by the natural selection. The basic idea of the genetic algorithm approach is to encode the solutions of the problem as chromosomes or individuals and each chromosome is estimated by calculating its fitness. For model (4.1), we hope to find the optimal switching sequences $v(j)|_{j = 0}^{\tau -1} = (v(0), \cdots, v(\tau -1))$, so they are presented as chromosomes. The optimal value $J(\boldsymbol{x}_{0}, 0)$ for model (4.1) is chosen as the fitness of the chromosome. Chromosomes with high fitness have a higher probability to be selected. The chromosomes for the next generation are reproduced and formal through "crossover" and "mutation" processes. Single crossover strategy is used in this paper, the genes of two parents solutions are swapped before and after a single point. Cells are chosen to be "mutated" randomly through the mutation process. If selected, the value of this gene is displaced by another random subsystem which does not appear in this chromosome. The steps of our proposed genetic algorithm can now be described as follows.

Parameters for genetic algorithm: $pop\_size$, $c$, $m$, stopping criterion.

(1) Generate initial population including $pop\_size$ random chromosome, each individual is presented as switching law $v(j)|_{j = 0}^{\tau -1}$.

(2) Calculate the fitness $J(\boldsymbol{x}_{0}, 0)$ of each individual.

(3) Use the roulette wheel method to create the mating pool from the current individual.

(4) Selection of two parents for crossover based on crossover rate $c$.

(5) Application of the single crossover approach to the random mating of these chosen individuals.

(6) The external mutation approach is used to choose individuals from the mating pool for breakthrough depending on mutation rate $m$.

(7) After repeated execution steps 2–6, once the termination condition is reached, the best individual is selected as the result of the optimal control $v^{*}(j)|_{j = 0}^{\tau -1}$.

6.   Illustrate examples

In this section, given the initial $\boldsymbol{G}_{i}, \boldsymbol{F}_{i}, \boldsymbol{Q}_{i}, \boldsymbol{R}_{i}$, genetic algorithm is applied to solve some problems, which proves that the content of our previous argument can be realized.

Example 6.1. Consider the uncertain discrete-time optimal control ${Problem \ (4.1)}$ with $\tau = 5$, $n = 3$, $\boldsymbol{x}(0) = (2, 1)^{T}$, $\xi _{t}\sim\mathcal{U}(-1, 1)$, $\eta_{t}\sim\mathcal{L}(-1, 1)$, ${h} (\xi_{t}, \eta _{t}, t) = \xi _{t}\times \eta _{t}$ and

Assume that $\boldsymbol{x}(0) = (2, 1)^{T}$ is the initial state. Then the optimal switching control $v^{*}(j)$ and optimal objective values $J(\boldsymbol{x}_{j}, j)$ of ${Problem \ (4.1)}$ are got by enumeration method and genetic algorithm respectively. The optimal results using the enumeration method are reflected in Table 1, and the results we get using the genetic algorithm are shown in Table 2. By comparison, we find that the difference between the final optimal switched controls and the optimal values is tiny in numerical terms, the genetic algorithm can therefore be used to implement the optimal control for ${Problem \ (4.1)}$. The more complex examples will be challenged.

Example 6.2. In this example, we consider the uncertain discrete-time optimal control ${Problem \ (4.1)}$ comprised of five subsystems, ten stages.

Subsystem 1:

Subsystem 2:

Subsystem 3:

Subsystem 4:

Subsystem 5:

In this example, we let $\boldsymbol{x}(0) = (2, 1)^{T}$, $\tau = 10$, ${h} (\xi_{t}, \eta _{t}, t) = \xi _{t}\times \eta _{t}$, $\xi _{t}\sim\mathcal{U}(0, 1)$, $\eta_{t}\sim\mathcal{L}(0, 1)$, $R_{t} = 1$, $\boldsymbol{Q}_{t} = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}$, $(t = 1, 2, 3, 4, 5)$, $\boldsymbol{Q}_{f} = \begin{bmatrix} 8 & 3 \\ 3 & 2 \end{bmatrix}$, $\boldsymbol{\sigma} = \begin{pmatrix} 0.01\\ 0.01 \end{pmatrix}$.

Then, the optimal switching control and the optimal continuous control of ${Problem \ (4.1)}$ are depicted in this example. Figure 1 shows the trajectory of state variables. Applying genetic algorithm stated in section 5 yields the results of optimal switching sequence $v^{*}(\cdot) = (2, 4, 2, 2, 2, 2, 2, 2, 2, 1)$ which is displayed in Figure 2. Figure 3 presents the optimal continuous control. The state space trajectories is shown in Figure 4. We can respectively find the optimal matrix $A_{j}$ in $H_{j}$ for each stage, which is shown in Table 3. The optimal value is $J(\boldsymbol{x}_{0}, 0)$ = 13.3290.

Example 6.3. In this example, we consider the uncertain discrete-time optimal control ${Problem \ (4.1)}$ comprised of five subsystems, which is similar to the Example 6.2, but the difference is that uncertain random variable ${h} (\xi_{t}, \eta _{t}, t) = \xi _{t} + \eta _{t}$.

Subsystem 1:

Subsystem 2:

Subsystem 3:

Subsystem 4:

Subsystem 5:

The optimal switching control and the optimal continuous control of ${Problem \ (4.1)}$ are depicted in this example. The trajectory of state variables is shown in the Figure 5. Figure 6 shows the optimal switching sequence $v^{*}(\cdot) = (2, 4, 5, 2, 2, 2, 2, 2, 2, 1)$. Figure 7 presents the optimal continuous control. The state space trajectories is shown in Figure 8. We can respectively find the optimal matrix $A_{j}$ in $H_{j}$ for each stage, which is shown in Table 4. The optimal value is $J(\boldsymbol{x}_{0}, 0)$ = 13.7860.

7.   Conclusions

In this paper, we study the optimal control problem of uncertain random multistage switching systems and propose methods for designing the switching laws and continuous control strategies. The analytical solution of the optimal strategy and the optimal objective function can be described exactly by $H_{j}$, whose magnitude varies with the length of the control time domain. The genetic algorithm is applied to obtain the optimal control and optimal values. The effectiveness of the method is verified with examples.

Use of AI tools declaration

The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

Conflict of interest

The authors declare that there is no conflict of interest.