Efficient state estimation strategies for stochastic optimal control of financial risk problems

1.   Introduction

Chaotic economics is a physical economics theory that can reveal ordered structures in random economic phenomena (Zhang et al., 2013). Recently, chaotic economics has gained extensive attention from the control communities and has been raised as an alternative scientific approach to understanding the complex dynamics of the real financial market (Sukono et al., 2020). Chaos is a special complex dynamic phenomenon that supports an endogenous explanation of the complexity observed in an economic time series. The existence of a chaotic phenomenon in any financial investment leads to the suffering of certain financial risks, as the primary function of financial markets is to transfer risk (Gao et al., 2018). Hence, understanding and handling the chaotic economics are crucial for the financial investment, as well as financial risk control.

In fact, financial risk is the possibility of suffering losses caused by uncertain changes in endogenous factors in financial or investment activities that have unpredictable fluctuations (Sukono et al., 2020). Thus, the high frequency of the up-and-down trends increases the loss incidence in the stock market. On the other hand, the financial crisis, which has the expression of chaos characteristics, can be caused by the occurrence of financial risk (Pfaff, 2016; Gao et al., 2018; Li et al., 2021). Since the rise in the occurrence of financial crises, the modeling and measurement of financial market risk have become necessary to devise and employ techniques that are better able to cope with the empirically observed extreme fluctuations in the financial markets.

From the review of literature, Burlando (1994) introduced the structure and terminology of chaos theory to risk management. The ultimate consequence of emerging chaos and risk is indeterminable, so a new perspective on how chaos and risk can persist in disrupting each other shall be further examined. Guillen et al. (2005) manipulated the financial risk associated with a given supply chain configuration under demand uncertainty. Zhang et al. (2013) investigated the stability and chaos of the improved financial risk system and described the change in financial market risk by using the complex dynamics of the system. In addition, the positive feedback gain matrix method has been applied to construct and prove the stability of the control system (Gao et al., 2018). This method shows that financial market risk can be controlled effectively under certain conditions, such as optimality and stability conditions, although chaotic systems are difficult to control.

Therefore, it is important to design and improve the dynamic chaotic control system because chaos disappearance is able to guard against financial system risks (Zhang et al., 2013). The rapid expansion of technology has provided the possibility for in-depth research on risk control, and many control systems have been developed to control the chaotic behavior of financial risk systems. Although financial risk cannot be eliminated totally, it can be managed (Kim et al., 2012). Hence, computational approaches such as extended Kalman filter (EKF) and unscented Kalman filter (UKF) techniques could be applied to solve the chaotic dynamics that are exhibited in the financial risk system. Some studies of using a Kalman filter in financial risk systems include financial crisis dynamics (Fatma and Sami, 2011), corporate financial distress (Zhuang and Chen, 2014), derivative portfolios (Haugh and Lacedelli, 2020), nonperforming loans (Ahmadi et al., 2022) and intellectual property pledge financing (Yin et al., 2022).

The main contribution of this paper is to demonstrate the effectiveness of the UKF technique in estimating the state while achieving optimal control of the financial risk system, and to compare the results of the UKF technique with the results from the EKF technique. In our work, the financial risk system, which consists of occurrence risk, analysis value risk and control value risk, is assumed to be disturbed by Gaussian random noise. Thus, the use of these filtering techniques is more appropriate than using the particle filter for non-Gaussian random noise. Unlike the EKF, the UKF does not require any Jacobian matrix calculations because it does not approximate the nonlinear functions of the process and the observation. By applying the unscented transformation, the UKF uses nonlinear models to approximate the distribution of the state variables and the observed variables with a normal distribution. For estimation accuracy, the sum of squared errors (SSE) and mean squared error (MSE) are calculated for both the EKF and UKF techniques. With these state estimates, the linear optimal control law is designed such that the performance index of the system, which measures the efficiency of the controller design, is minimized.

The rest of this paper is organized as follows. In Section 2, the stochastic optimal control problem of the financial risk model is described. In Section 3, the EKF and UKF computational approaches for solving the problem are discussed. Accordingly, the calculation procedures are summarized into two algorithms, which are the EKF for state control (EKFSC) algorithm and the UKF for state control (UKFSC) algorithm. In Section 4, an illustrative example that shows chaotic behavior is presented. Then, significant results on state estimation and feedback control of the system are presented and discussed. Finally, some concluding remarks are made.

2.   Problem description

Consider a mathematical model of the financial risk system (Sukono et al., 2020) that is described as

where ${x}_{1}\left(t\right)$ is the occurrence value risk, ${x}_{2}\left(t\right)$ is the analysis value risk and ${x}_{3}\left(t\right)$ is the control value risk in the current market. These variables (${x}_{1}, {x}_{2}, {x}_{3}$) are state variables and must be positive because risk in financial markets always exists in the market. In addition, a is the analysis risk efficiency, b is the transmission rate of the previous risk and c is the distortion coefficient for risk control. These scalars, i.e., a, b and c, are positive constant parameters with a, b, c ≥ 0.

By imposing the admissible control input $u\left(t\right)\in {\mathfrak{R}}^{3}$, k = 0, 1, …, N–1, to the dynamic model (1), the state equation becomes

and its equivalent discrete-time model is given by

where $x =$ is the state vector, the function $f:{\mathfrak{R}}^{3}\to {\mathfrak{R}}^{3}$ is the plant dynamic given by

with the sampling time τ for k = 0, 1, …, N–1, and B is a 3 × 3 control coefficient matrix. Note that, in the presence of the random disturbances $\omega \left(k\right)\in {\mathfrak{R}}^{3}$, k = 0, 1, …, N–1, which is the process noise, and $\eta \left(k\right)\in \mathfrak{R}$, k = 0, 1, …, N–1, which is the measurement noise, the state equation (3) is written as

and the output measurement is denoted by

where G is a 3 × 3 noise coefficient matrix and $h:{\mathfrak{R}}^{3}\to \mathfrak{R}$ is the output channel function defined by

Here, the random disturbances $\omega \left(k\right)$ and $\eta \left(k\right)$ are Gaussian white noise sequences with a zero mean; their covariance matrices are ${Q}_{\omega }$ and ${R}_{\eta }$, respectively. The initial state $x\left(0\right) = {x}_{0}$ is a random vector with an expected value and error covariance that are respectively given by

where ${M}_{0}\in {R}^{3\times 3}$ is a positive definite matrix and $E\left[\cdot \right]$ is the expectation operator. It is assumed that the initial state, process noise and measurement noise are statistically independent.

Hence, the aim of the study was to find a set of the optimal control sequences $u\left(k\right)$, k = 0, 1, …, N–1, such that the cost function

is minimized over the dynamical system defined by (5) and (6). Here, $\phi :{\mathfrak{R}}^{3}\to \mathfrak{R}$ is the terminal cost function and $L:{\mathfrak{R}}^{3}\times {\mathfrak{R}}^{3}\to \mathfrak{R}$ is the operating cost function. Therefore, this problem is referred to as the discrete time nonlinear stochastic optimal control problem for financial risk system, and it is regarded as Problem (P).

3.   State estimation strategies

In this section, the state estimation using the EKF and UKF approaches is discussed and the optimal control law is designed for solving Problem (P).

3.1.   Extended kalman filtering

Consider the state mean propagation for the state dynamics described by (5), i.e.,

where $\overline{x}\left(k\right)$ represents the state mean sequences, and we define the weighted least-squares error as

By taking the necessary condition $\nabla {J_{lse}}(x) = 0$, the optimal state estimate is obtained from

where $\hat x(k)$ is the filtered state estimate, $\bar x(k)$ is the predicted state estimate and $\bar y(k)$ is the output estimate (Lewis et al., 2012).

Here, the Kalman filter gain is

whereas the state error covariance matrices are

and the output error covariance matrix is

with the initial condition ${M_x}(0) = {M_0}$. The filtered state error covariance $P(k)$, the predicted state error covariance ${M_x}(k)$ and the output error covariance ${M_y}(k)$ are positive definite matrices. The linearization of the dynamical system described by (11)–(12) will be done for the following Jacobian matrices:

Thus, it is noticed that (11) is the measurement update and (12) is the time update. These two equations are known as the Kalman filtering equations. This method is commonly known as the EKF approach (Bryson and Ho, 1975; Lewis et al., 2012).

3.2.   Unscented kalman filtering

Assume that the n-dimensional random state vector x has a mean $\bar x$ and covariance ${P_{xx}}$. On this basis, a set of sigma points (Julier and Uhlmann, 1997) is denoted by

with the components

for $i = 1, \cdots, n,$ and the weights

for $i = 1, \cdots, 2n.$ Here, ${\left(\sqrt{(n+\lambda){P}_{xx}}\right)}_{i}$ is the ith column of the matrix square root $(n + \lambda){P_{xx}}$, and ${W_i}$ is the weight value, which satisfies the conditions

Here,

is a scaling factor, where $\alpha$ determines the spread of the sigma points around $\bar x$ and it is assigned a small positive value (10^–3) in the range of $0 < \alpha \leqslant 1$, and $\kappa$ is a secondary scaling factor, which is in the range of $0 \leqslant \kappa \leqslant 3$ and is usually set to 0. Moreover, $\beta$ is used to incorporate prior knowledge of the distribution of the state x with $\beta \geqslant 0$ and $\beta = 2$ is the optimal value for Gaussian distributions (Julier and Uhlmann, 2004).

3.2.1.   Unscented transformation

Referencing (18), the sigma points are propagated through the nonlinear function h given in (13) to generate the transformed sigma points from

for $i = 1, \cdots, 2n.$ By using a weighted sample mean and covariance of the transformed sigma points, the mean and covariance for the output variable y are approximated from

where ${R}_{\eta }$ is the output noise covariance.

Notice that the unscented transformation described by (28) and (29) are more accurate than the linearization method for propagating means and covariances of the nonlinear functions (Wan and van der Merwe, 2000; Julier and Uhlmann, 2004).

3.2.2.   State estimation

Consider the initial value of the predicted mean and covariance of the state given by

the sigma points using the a priori mean and covariance of the state are provided by

where $\gamma = \sqrt {n + \lambda }$, as stated in (21) for $k = 0, 1, \cdots, N - 1.$ Therefore, in the time-update procedure, the state of the transformed sigma points is predicted from

with the estimated state mean for

and the estimated state error covariance

where ${Q_\omega }$ is the process noise covariance (Wan and van der Merwe, 2000).

On the other hand, in the measurement update procedure, the output of the transformed sigma points is measured by

with the estimated observation

and the observation error covariance

The predicted state mean is updated by

with the updated state error covariance

where

Here, ${R_\eta }$ is the observation noise covariance, ${K_f}(k)$ is the Kalman filter gain and ${P_{xy}}(k)$ is the cross-correlation matrix. This method is known as the UKF approach (Julier and Uhlmann, 1997; Wan and van der Merwe, 2000; Julier and Uhlmann, 2004).

3.3.   Optimality conditions

For measurement purposes, the cost function given by (8) is written in its expectation form:

We define the Hamiltonian function as follows (Bryson and Ho, 1975; Kirk, 2004):

where $p(k)$ is a 3  1 co-state vector to be determined later. The augmented cost function is written as

According to the Lagrange multiplier theory, at a constrained minimum, the increment $d{J}^{\text{'}}$ should be zero (Lewis et al., 2012). Hence, the following necessary conditions are derived:

Stationary condition

(a) State equation

(b) Co-state equation

(c) Boundary conditions

3.4.   Optimal control law

Assume that the cost function (43) can be approximated into its quadratic criterion, that is,

where $S(N),$ Q and R are the weighting matrices. Hence, the optimality conditions (46) and (48) are simplified (Lewis et al., 2012; Teo et al., 2021) as follows:

the Jacobian matrix is $A \approx {\nabla _x}f$ and the boundary value of the co-state is

Suppose the co-state equation has the following solution:

and consider this solution with the optimality conditions (52) and (53). After doing some algebraic manipulations, the linear feedback control law is designed as follows:

with

where $S\left(N\right) = {S}_{N}$ given. Here, $K(k)$ is the Kalman feedback gain and $S(k)$ is the solution of the Riccati equation (Bryson and Ho, 1975; Kirk, 2004; Lewis et al., 2012; Teo et al., 2021).

3.5.   Computational algorithms

From the discussion above, the calculation procedure for estimating the state dynamics and designing the optimal control law is summarized as the computational algorithm given below. The first algorithm is named as the EKFSC algorithm and the second algorithm is known as the UKFSC algorithm.

3.5.1.   EKFSC algorithm

Data Given $f,$ $h,$ $\varphi,$ $L,$ $A,$ $B,$ $C,$ $G,$ $N,$ $Q,$ $R,$ $S(N),$ ${M_0},$ ${Q_\omega },$ ${R_\eta },$ ${\bar x_0},$ $y$.

Step  1   Calculate the state and output error covariance matrices $P(k),$${M_x}(k)$ and ${M_y}(k)$ using (15), (16) and (17), respectively.

Step  2   Calculate the filter gain ${K_f}(k),$feedback gain $K(k)$ and Riccati solution $S(k)$ using (14), (57) and (58), respectively.

Step  3   Compute the state and output estimates $\hat x(k),$$\bar x(k)$ and $\bar y(k)$ using (11), (12) and (13), respectively.

Step  4   Evaluate the weighted least square error ${J_{lse}}$ using (10).

Step  5   Compute the feedback control law $u(k)$ using (56).

Step  6   Update the state equation forward in time by using (47) to obtain the state solution $\bar x(k)$; solve the co-state equation backward in time by using (53) to obtain the co-state solution $p(k).$

Step  7   Evaluate the cost function J using (43).

Remarks

(a)    In Step 1 and Step 2, the off-line calculations are performed to store the values of matrices.

(b)    In Step 3, the state estimation is performed using the EKF approach.

(c)    In Step 5, the linear feedback control law is designed.

(d)    In Step 6, a two-point boundary-value problem is solved to give the state and co-state solutions.

3.5.2.   The UKFSC algorithm

Data Given $f,$ $h,$ $\varphi,$ $L,$ $A,$ $B,$ $C,$ $G,$ $N,$ $Q,$ $R,$ $S(N),$ ${P_0},$ ${Q_\omega },$ ${R_\eta },$ ${\bar x_0},$ $y$.

Step  1    Calculate the feedback gain $K(k)$ and the Riccati solution $S(k)$ using (57) and (58), respectively.

Step  2    Prepare the sigma points $\chi (k)$ defined in (32) by using the a priori state mean $\bar x(k)$ and state error covariance $P(k)$. After that, calculate the predicted state of the transformed sigma points $\chi (k + 1)$ using (33), as well as its mean $\hat x_{}^ - (k)$ and covariance $P_{}^ - (k)$ using (34) and (35), respectively.

Step  3    Compute the output measurement $\Upsilon (k)$ using (36), as well as its mean observation $\hat y_{}^ - (k)$ and the observation error covariance ${P_{yy}}(k)$ using (37) and (38), respectively. After that, calculate the cross-correlation matrix ${P_{xy}}(k)$ using (42) and the Kalman filter gain ${K_f}(k)$ using (41).

Step  4   Update the state estimate $\hat x(k)$ using (39) and the state error covariance $P(k)$ using (40).

Step  5   Update the state mean $\bar x(k)$ forward in time using (47); solve the co-state equation backward in time by using (53) to obtain the co-state solution $p(k)$.

Step  6   Compute the feedback control law $u(k)$ using (56).

Step  7   Evaluate the cost function J using (43).

Remarks

(a)    Step 1 is called the off-line calculation step, where the feedback gain $K(k)$ and the Riccati solution $S(k)$ are stored to design the feedback control law.

(b)    In Steps 2 and 3, the unscented transform is performed to generate a set of sigma points $\chi \left(k\right)$ and $\Upsilon \left(k\right)$. In Step 4, the correction step is performed, where the output estimate is measured and the state estimate is updated. These steps comprise the state estimation procedure.

(c)    From Steps 5 to 7, the two-point boundary-value problem is solved in order to obtain the solution of the state mean and the co-state, and the feedback control law is designed. These steps comprise the system optimization procedure.

4.   Illustrative example

Consider the following parameters for the dynamic model (1) to be chaotic (Sukono et al., 2020): $a = 10, b = 28\text{and}c = 8/3,$ where the initial conditions are ${x}_{1}\left(0\right) = 2.5, {x}_{2}\left(0\right) = 0.5\text{and}{x}_{3}\left(0\right) = 4.$ For the simulation work, we define the following coefficient matrices:

and the sampling time is set to $\tau = 0.01$ seconds. Also, the weighting matrices in the cost function are ${S_N} = 100{I_{3 \times 3}},$$Q = diag\left(\mathrm{0.01, 1.00, 1.00}\right)$ and $R = diag(0.01, 0.001, 0.01)$ for $k = 0, 1, \cdots, 60,$ where the notation diag represents the diagonal matrix. Assume the Gaussian white noise sequences have the respective covariance given by ${Q}_{\omega } = 0.001{I}_{3\times 3}$ and ${R}_{\eta } = 0.01,$ while the initial state error covariance is ${M}_{0} = 0.2{I}_{3\times 3}.$ So, the quadratic criterion cost function and the SSE were employed to measure the performances of the system and the algorithm proposed, respectively.

Table 1 shows the simulation results for Problem (P) that were obtained by using the EKFSC and UKFSC algorithms. The optimal cost of 81.5 units, which was provided by using the UKFSC algorithm, was 0.88% less than the optimal cost given by the EKFSC algorithm. This optimal cost indicates the quadratic criterion performance of the financial risk system under the conditions of optimal trajectories of states given by the occurrence value risk, analysis value risk and control value risk in the current market. It also shows that the controller in the UKFSC algorithm is efficient. On the other hand, the estimation performance of the UKFSC algorithm, which was revealed by the SSE, was 99% more accurate than the EKFSC algorithm when estimating the real output response.

Figures 1 and 2 show the output trajectories for the respective algorithms. The red line represents the output estimate trajectory, and the blue line with asterisk symbols (_*) denotes the real output trajectory. The output reduces to about zero from 2.5 in one unit of time; then, it fluctuates around zero for a period of one to six units of time. The output trajectories given by the algorithms proposed were well matched to the real output trajectory. Notice that the output trajectory from the UKFSC algorithm was more accurate than the output trajectory from the EKFSC algorithm. Thus, the occurrence value risk can be estimated when a financial investment decision is needed.

Figures 3 and 4 show the state trajectories derived from the state estimation process by using the algorithms proposed. The red line represents the state estimate trajectories, while the blue line with asterisk symbols (_*) represents the real state trajectories. Both algorithms provided similar state estimate trajectories, which matched the real state trajectories closely. The occurrence value risk dropped the value from 2.5 and toward zero after one unit of time, while the analysis value risk reduced dramatically to −0.4 and then started to increase slowly to zero. The control value risk in the current market also decreased gradually from four units to zero over the period of six units of time. Hence, the occurrence value risk, analysis value risk and control value risk were mitigated and under control.

By applying the algorithms proposed, the control trajectories, which regulate the state trajectories efficiently, were obtained as shown in Figures 5 and 6. The control law for the occurrence value risk fluctuated slightly and the control effort increased from a negative value to zero, while the control effort for the analysis value risk was extremely important to stabilize the risk. In addition, the control value risk was smoothly regulated, and the control effort was the lowest in value compared to the control efforts for the first and second state variables. This characteristic shows that the controller in the algorithms proposed is practical.

Obviously, the financial risk system must be under control to give the optimal solution. This optimal solution satisfied the stationary conditions, as shown in Figures 7 and 8, which were respectively obtained by using the EKFSC and UKFSC algorithms. The curve of the stationary conditions fluctuated since random noises were introduced to the system. Nonetheless, the gradient satisfied the stopping rule within a small tolerance during the computational process.

5.   Conclusions

The application of the EKF and UKF techniques to solve the stochastic optimal control problem of the financial risk system has been discussed in this paper. In the presence of random disturbances, the financial risk system, which exhibits chaotic behavior, becomes very difficult to control. The system was initially linearized by using the EKFSC algorithm, while, through the unscented transform, the distributions of state variables and observed variables were approximated with a normal distribution by using the UKFSC algorithm. Then, state estimation was carried out by using the EKFSC and UKFSC algorithms. With these state estimates, the state feedback control law was designed to determine the optimal solution for the system. Given the parameter values that cause the financial risk system to be chaotic, the simulation results showed that the system was under control within the allotted time interval, where the risks were mitigated. The performance of the UKFSC algorithm was proven to be better than the EKFSC algorithm since the SSE of the UKFSC was smaller than the SSE of the EKFSC algorithm. In conclusion, the efficiency of both filtering algorithms used to solve the nonlinear stochastic optimal control problem of the financial risk system has been validated. Regarding the future research direction, efficient computational methods should be applied to handle financial risk problems, either by minimizing the financial risk or predicting the financial risk, when making a financial investment decision.

Acknowledgments

This research was supported by the Ministry of Higher Education (MHOE) Malaysia through the Fundamental Research Grant Scheme (FRGS/1/2018/STG06/UTHM/02/5) and Universiti Tun Hussein Onn Malaysia (UTHM) through Tier 1 (vot Q121).

Conflict of interest

All authors declare no conflict of interest regarding this study.