Zeroing neural network model for solving a generalized linear time-varying matrix equation

1.   Introduction

The matrix equation are often encountered in the control and system fields ^[1,2]. For example, one should solve the Sylvester matrix equation in the fields of the analysis and synthesis dynamic control systems ^[3,4]. Various strategies have been suggested to solve these linear matrix equations. In the engineering and control fields, the time-variant matrix equations are often encountered ^[5,6]. For example, the state equations of the linear time-variant system are time-varying. To control the robot manipulators, we need to solve the time-varying equations in the robot kinematics ^[7,8,9]. In this manuscript, we focus our energy on how to solve the following generalized linear time-varying matrix equation:

where ${\mathit{\boldsymbol{A}}}(t), {\mathit{\boldsymbol{B}}}(t), {\mathit{\boldsymbol{C}}}(t), {\mathit{\boldsymbol{D}}}(t), {\mathit{\boldsymbol{F}}}(t)\in {\mathbb{R}}^{n\times n}$ are the time-varying coefficient matrices and the ${\mathit{\boldsymbol{X}}}(t)\in {\mathbb{R}}^{n\times n}$ is the unknown time-varying matrix to be determined.

The zeroing neural network (shorted for ZNN) method is a very active topic in the fields of control and engineering ^[10,11] and automatic control field ^[12-14]. This method can achieve the global and exponential convergence. Due to this important convergence property of the ZNN method, based on the Lyapunov stable theory, the ordinary different equation theory and the Laplace transform theory, three different convergence approaches were established ^[15-18]. The ZNN method or the improved ZNN method can be used to solve a variety of matrix inversions or matrix equations. For example, the complex matrix Drazin inverse was discussed ^[19], the time-varying Sylvester matrix equations were solved ^[20-22] and the time-varying matrix square root was determined ^[23]. Using the ZNN method, the QR decomposition of the complex value matrix can be done and this decomposition can be used to track a robotic motion ^[7,24,25].

Due to the exponential convergence and robustness in dealing with the time-varying equation, the ZNN method has acquired some new developments. Based on the ZNN method, the varying-parameter recurrent neural-network method for solving online time-varying matrix equations has been established ^[26,27] and the super-exponential convergence was proved. This varying-parameter recurrent zeroing neural network has been succeed in dealing with the complex Sylvester matrix equation ^[28] and quadratic programming problems ^[29]. In this paper, we use the ZNN method to solve Eq (1.1). First, we construct a ZNN model to solve Eq (1.1) and the convergence analysis is offered. Second, with an activation function, the predefined-time convergence property and noise suppression model is suggested and the convergence proof is presented. Third, to illustrate the efficacy of the suggested methods, two numerical experiments are offered. The main contributions of this paper can be emphasized as follows:

● Zeroing neural network model for solving linear time-varying matrix equation (1.1) is constructed and the convergence proof is offered.

● The predefined-time convergence property is discussed and the noise suppression model is suggested.

● Two numerical experiments are given to verify the effectiveness of the suggested models.

The remainder of this paper is organized as follows: Two preliminary lemmas and the analytical time-varying solution of Eq (1.1) are presented in Section 2. Section 3 offers the ZNN model for solving this time-varying linear matrix equation (1.1). Section 4 discusses the role of the activation function and the predefined-time convergence property of the ZNN model. Two numerical experiments are offered in Section 5 to illustrate the efficacy of the suggested models. Section 6 ends this note with some concluding remarks.

The following notation is used in the paper. Symbols $\dot{\phi}(t)$ and $\frac{{\rm{d}} \phi(t)}{{\rm{d}} t}$ denote the derivative of $\phi(t)$ in the argument $t$. ${\mathit{\boldsymbol{I}}}_n$ represents an identity matrix of size $n\times n$. ${\mathit{\boldsymbol{O}}}$ denotes a zero matrix with proper size. For a real square matrix ${\rm{tr}}({\mathit{\boldsymbol{A}}})$ represents the trace function, the Fobenious norm $\|{\mathit{\boldsymbol{A}}}\|_{{\mathit{\boldsymbol{F}}}}$ is defined by formula $\|{\mathit{\boldsymbol{A}}}\|^2_{{\mathit{\boldsymbol{F}}}} = {\rm{tr}}({\mathit{\boldsymbol{A}}}^{{\rm{T}}}{\mathit{\boldsymbol{A}}})$. ${\mathit{\boldsymbol{A}}}\otimes {\mathit{\boldsymbol{B}}}$ stands for the Kronecker product of matrices ${\mathit{\boldsymbol{A}}} = (a_{ij})$ and ${\mathit{\boldsymbol{B}}}$ by formula

${\rm{col}}[{\mathit{\boldsymbol{A}}}]$ denotes the vectorization operator of matrix ${\mathit{\boldsymbol{A}}}$ by defining ${\rm{col}}[{\mathit{\boldsymbol{A}}}] = [{\mathit{\boldsymbol{\alpha}}}_1^{{\rm{T}}}, {\mathit{\boldsymbol{\alpha}}}_2^{{\rm{T}}}, \cdots, {\mathit{\boldsymbol{\alpha}}}_n^{{\rm{T}}}]^{{\rm{T}}}$, where ${\mathit{\boldsymbol{A}}} = [{\mathit{\boldsymbol{\alpha}}}_1, {\mathit{\boldsymbol{\alpha}}}_2, \cdots, {\mathit{\boldsymbol{\alpha}}}_n]$.

2.   Preliminary work and the analytical solution

In this section, we give two preliminary lemmas ^[30] and the analytical solution of Eq (1.1).

Lemma 1. Let ${\mathit{\boldsymbol{A}}}(t)\in {\mathbb{R}}^{m\times n}$, ${\mathit{\boldsymbol{B}}}(t)\in{\mathbb{R}}^{n\times p}$ and ${\mathit{\boldsymbol{C}}}(t)\in{\mathbb{R}}^{p\times q}$. Then

Let ${\mathit{\boldsymbol{e}}}_{in}$ denote an $n$-dimensional unit column vector which has 1 in the $i$th position and 0's elsewhere, i.e.,

Define the vec-permutation matrix

based on the definition of the vec-permutation matrix, we obtain the conclusion ${\mathit{\boldsymbol{P}}}_{mn}^{{\rm{T}}} = {\mathit{\boldsymbol{P}}}_{nm}$ and the following conclusion.

Lemma 2. For any matrix ${\mathit{\boldsymbol{A}}}\in{\mathbb{R}}^{m\times n}$, it can be proved that

For example, let ${\mathit{\boldsymbol{A}}}_{2\times 3} = \left[\begin{array}{rrr} a_{11}&a_{12}&a_{13}\\ a_{21}&a_{22}&a_{23}\\ \end{array}\right]$ and ${\mathit{\boldsymbol{P}}}_{32} = \left[ {\begin{array}{*{20}{c}} {{{\boldsymbol{I}}_3} \otimes {\boldsymbol{e}}_{12}^{\rm{T}}}\\ {{{\boldsymbol{I}}_3} \otimes {\boldsymbol{e}}_{22}^{\rm{T}}} \end{array}} \right] = \left[\begin{array}{rrrrrr} 1 & 0 & 0 & 0 & 0 & 0\\ 0 & 0 & 1 & 0 & 0 & 0\\ 0 & 0 & 0 & 0 & 1 & 0\\ 0 & 1 & 0 & 0 & 0 & 0\\ 0 & 0 & 0 & 1 & 0 & 0\\ 0 & 0 & 0 & 0 & 0 & 1 \end{array}\right],$

it can be verified that ${\rm{col}}{\mathit{\boldsymbol{A}}}^{{\rm{T}}}_{23} = {\mathit{\boldsymbol{P}}}_{32}{\rm{col}}{\mathit{\boldsymbol{A}}}_{23}$ is right.

Using Lemmas 1 and 2, Eq (1.1) can be rewritten as

If the coefficient matrix ${\mathit{\boldsymbol{B}}}^{{\rm{T}}}(t)\otimes {\mathit{\boldsymbol{A}}}(t)+[{\mathit{\boldsymbol{D}}}^{{\rm{T}}}(t)\otimes {\mathit{\boldsymbol{C}}}(t)]{\mathit{\boldsymbol{P}}}_{nn}$ is invertible, then the exact solution of Eq (1.1) is

This conclusion will be used in the numerical experiments section.

3.   ZNN model for the time-varying linear matrix equation

In this section, we use the ZNN method to design a model to solve Eq (1.1). We use ${\mathit{\boldsymbol{X}}}(t)$ to denote the neural state solution and introduce the time-varying error function matrix ${\mathit{\boldsymbol{\Phi}}} (t)$ by defining

The ZNN method for solving Eq (1.1) is

where $\gamma > 0$ is a design parameter. To reduce the convergence time and improve the convergence performance, an activation function version ZNN model is

where ${\mathcal{F}}({\mathit{\boldsymbol{\Phi}}}(t))$ represents an activation function array with $f(\phi_{ij}(t))$ as its entries. Ordinary speaking, the activation function $f(*)$ is monotonous increasing odd function. Next, we establish ZNN model for solving Eq (1.1).

Taking the derivative of ${\mathit{\boldsymbol{\Phi}}}(t)$ gives

Substituting Eqs (3.1) and (3.4) into Eq (3.2) gives

With a proper manipulation gives

Setting

and using the formula

Eq (3.6) can be rewritten as

Referring to Lemma 2, Eq (3.8) can be rewritten as

The activation function version ZNN model (3.9) is

The convergence proof of models (3.9) and (3.10) is given by the following theorem.

Theorem 1. Suppose that the design parameter $\gamma$ is a positive number, ZNN model (3.10) is global asymptotic convergent.

Proof. Obviously, ZNN model equation (3.9) is a special case of Eq (3.10) when the activation function is the linear function.

We use the Lyapunov stable theory to prove the global asymptotic convergence of model (3.10). By introducing the Lyapunov energy function ${\mathit{\boldsymbol{\epsilon}}} (t): = \frac{1}{2}\|{\mathit{\boldsymbol{\Phi}}}(t)\|^2_{{\mathit{\boldsymbol{F}}}}$. Due to the definitions of the Frobenius norm and the matrix trace, we have

To calculate the derivative of the energy function ${\mathit{\boldsymbol{\epsilon}}}(t)$, we get

Using ZNN model $\frac{{\rm{d}} {\mathit{\boldsymbol{\Phi}}}(t)}{{\rm{d}} t} = -\gamma {\mathcal{F}}({\mathit{\boldsymbol{\Phi}}}(t))$ in Eq (3.11) gives

Because the entries $f[\phi_{ij}(t)]$, $i, j = 1, 2, \cdots, n$ are the continuous monotonous increasing odd functions, we get

Due to the Lyapunov stability theory ^[31-33], if $\|{\mathit{\boldsymbol{\Phi}}}(t)\|_{{\mathit{\boldsymbol{F}}}} = 0$ then the neural state matrix ${\mathit{\boldsymbol{X}}}(t)$ is the exact solution of Eq (3.10). If $\|{\mathit{\boldsymbol{\Phi}}}(t)\|_{{\mathit{\boldsymbol{F}}}} > 0$ then $\dot{{\mathit{\boldsymbol{\epsilon}}}} (t) < 0$ and this denotes that $\|{\mathit{\boldsymbol{\Phi}}}(t)\|_{{\mathit{\boldsymbol{F}}}}$ monotonically decreasing and converges to the global asymptotic stable point. That is, we have $\|{\mathit{\boldsymbol{\Phi}}}(t)\|^2_{{\mathit{\boldsymbol{F}}}} = 0$ or the neural state matrix ${\mathit{\boldsymbol{X}}}(t)$ will converge to the exact solution of Eq (3.10). It means that ZNN model (3.10) is global asymptotic convergent. Note that model (3.9) is the special case of model (3.10). We complete the proof.

Finding the solutions of matrix equations and coupled matrix equations are very related to deriving the identification algorithms of dynamical systems for their parameter estimation ^[34-36]. For some identification problems, minimizing a criterion function about the differences between the system outputs and the model outputs leads to a matrix equation. Based on the optimization techniques and the obtained matrix equations about the parameter estimation, we can derive the gradient-based identification algorithms, the least squares identification algorithms and the Newton identification algorithms. In the numerical section, we will illustrate the effectiveness of model (3.9) by numerical experiments.

4.   Activation function and predefined-time convergence

In the practical control work, the noise should be suppressed ^[37,38]. To suppress the noise, some activation functions were introduced ^[39,40] and the related theoretical analysis were presented. In this section, we use an activation function to obtain the predefined-time time convergence property ^[8,9] and to reduce the influence of the noise. We mainly refer to ^[39,40] in reasoning the following conclusion, for more detailed discussion and more effective conclusion, one can refer to ^[41-44]. Referring to ^[39], we have the following result.

Lemma 3. Let $\alpha > 0, \beta > 0$, $0 < p < 1$ and $q > 1$ be some constant parameters, the following dynamic nonlinear inequality system

is globally predefined-time stable and the setting time is bounded by

The following universal activation function was first presented in ^[39].

Here, ${\rm{sgn}}(x)$ denotes the signal function. Using this activation function, the predefined-time ZNN model can be obtained.

In the practical work, some different noises can be emerged in the system, a noise version of Eq (3.10) is

Here, the matrix ${\mathit{\boldsymbol{Y}}}(t)$ represents the noise matrix. Referring to ^[39] and using the universal activation function (4.1), one can prove that ZNN model (4.2) is effective and the predefined-time is bounded by

Theorem 2. Suppose that the exact solution matrix ${\mathit{\boldsymbol{X}}}^*(t)$ is derivable and the noise matrix ${\mathit{\boldsymbol{Y}}}(t) = [y_{ij}(t)]\in{\mathbb{R}}^{n\times n}$ satisfies $|y_{ij}(t)|\leqslant \delta$, where $\delta\in (0, +\infty)$. If the universal activation function is used with $\gamma k_4\geqslant \delta$ then for any initial neural state solution matrix ${\mathit{\boldsymbol{X}}}(0)$ the neural state stable matrix ${\mathit{\boldsymbol{X}}}(t)$ of model (4.2) will converge to the exact solution ${\mathit{\boldsymbol{X}}}^*(t)$ of Eq (1.1) in predefined-time $t_c$. Here, $t_c$ satisfies

Proof. Using error function (3.1) and activation function (4.1), the noise-perturbed ZNN model

entry-wisely consists $n^2$ sub-autonomous systems

where $\dot{\phi}_{ij}(t)$, $\phi_{ij}(t)$ and $y_{ij}(t)$ are the $ij$th entry of matrix $\dot{\bf{\mathit{\boldsymbol{\Phi}}}}(t)$, ${\mathit{\boldsymbol{\Phi}}}(t)$ and ${\mathit{\boldsymbol{Y}}}(t)$ respectively.

Setting the Lyapunov energy function as $u(t) = |\phi_{ij}(t)|^2$ for the sub-autonomous system (4.2) and computing the time derivative of $u(t)$ gives

Substituting Eq (4.1) into Eq (4.3) and with proper operations gives

Using Lemma 3, we get the convergence time of the $ij$th sub-system (4.2) is

Because the time $t_{ij}$ depends on the parameter $\gamma$ of the ZNN model and the parameters $k_1, k_2, p, q$ of the activation function and does not depend on any initial value ${\mathit{\boldsymbol{X}}}(0)$, the convergence time of model (4.2) is

Thus, we complete the proof.

Referring to ^[39], a similar work can prove that ZNN model (3.10) has the same convergence time upper bound, we omit it here. Theorem 4.1 shows that the predefined-time is independent of the initial values for a class of systems. We will verify this fact in the numerical section.

5.   Numerical examples

In this section, we verify the effectiveness of the suggested ZNN models by giving two numerical examples. The strategy of choosing the coefficient matrices is inspired by ^[20,26].

Example 1. We use ZNN model (3.9) to solve the time-varying matrix equation (1.1). We take

as the coefficient matrices.

In this numerical experiment, we take the design parameter $\gamma = 10$ and the initial neural state matrix ${\mathit{\boldsymbol{X}}}(0) = i\times {\mathit{\boldsymbol{I}}}_2$, $i = 1, 2, 3$, respectively, and use model (3.9) to solve time-varying nonlinear matrix equation (1.1).

To illustrate the convergence trajectory, we use

to denote the neural state solution, and the convergence trajectories of $x_{ij}(t)$ are shown in Figure 1. The red curves denote trajectories of the true solution matrix ${\mathit{\boldsymbol{X}}}^*(t)$. We use

to denote the Frobenius norm of the absolute error matrix, the trajectory of $\ln\delta_1(t)$ is shown in Figure 2. These two figures show that ZNN model (3.9) is efficacious for solving nonlinear matrix equation (1.1).

From Figure 1, we find that the trajectories of $x_{ij}(t)$ converge fast and neural state matrix ${\mathit{\boldsymbol{X}}}(t)$ converges to the time-varying solution matrix ${\mathit{\boldsymbol{X}}}^*(t)$ exactly. Figure 2 shows that $\ln\delta_1(t)$ fluctuates between $-6$ and $-8$ as the time $t$ greater than $1$ second. In the numerical experiment, if we set $\gamma = 100$, then convergence precision will be improved greatly. This shows that the convergence behavior of the absolute error norm $\delta_1(t)$ will be dominated by some random factors when it converges to a certain degree. How to explain the perturbation behavior of this stage requires further study.

To illustrate the effectiveness of the noise-perturbed ZNN model (4.2), we set ${\mathit{\boldsymbol{Y}}}(t) = \sin(t){\mathit{\boldsymbol{I}}}_2$ as the noise matrix and use model (4.2) to solve Eq (1.1). Here, we set the related parameters as

Using these parameters, the upper bound of the predefined-time convergence time is

The convergence index $\delta_1(t)$ is shown in Figure 3. Figure 3 shows that the theoretical predefined convergence time conforms to the practical convergence time. Although the predefined convergence time is $0.3$ second, on the computer testing, we find that there much more time are consumed due to the computation of the activation function.

Example 2. Set ${\mathit{\boldsymbol{F}}}(t) = \left[ {\begin{array}{*{20}{c}} {\sin(t)+9}&{0}\\ {0}&{\sin(t)+10} \end{array}} \right]$. The other initial conditions are the same as Example 1, we use ZNN model (3.9) to solve Eq (1.1). The convergence performance of the neural state matrix ${\mathit{\boldsymbol{X}}}(t)$ is shown in Figure 4. We use $\delta_2(t)$ to denote the absolute error Frobenius matrix norm of this example and $\delta_2(t)$ shares the same definition of $\delta_1(t)$, the natural logarithm of the absolute error Frobenius matrix norm $\ln\delta_2(t)$ versus the time $t$ is shown in Figure 5.

From Figure 4, we find that every entries of the neural state matrix ${\mathit{\boldsymbol{X}}}(t)$ converge to some fixed time-varying functions fast. This means the neural state matrix ${\mathit{\boldsymbol{X}}}(t)$ converge to the solution matrix ${\mathit{\boldsymbol{X}}}^*(t)$. From Figure 5, we find that the absolute error logarithm goes to $-8$ fast about $2$ seconds. These shows that the ZNN model (3.9) is effective in solving the time-varying linear matrix equation (1.1).

6.   Conclusions

To solve the linear time-varying matrix equation ${\mathit{\boldsymbol{A}}}(t){\mathit{\boldsymbol{X}}}(t){\mathit{\boldsymbol{B}}}(t)+{\mathit{\boldsymbol{C}}}(t){\mathit{\boldsymbol{X}}}^{{\rm{T}}}(t){\mathit{\boldsymbol{D}}}(t) = {\mathit{\boldsymbol{F}}}(t)$, the zeroing neural network model is constructed. The convergence proof is given and the predefined-time convergence property and noise suppression technique are discussed. Two numerical examples are offered to illustrate the efficacy of the suggested models. Although we have proven that the Frobenius norm of the absolute error matrix is monotonic decreasing, the numerical experiments show that the convergence behavior will be dominated by some random factors when the absolute error norm decreasing to certain degree. We will study this fluctuation in the future. The proposed algorithms in this paper can combine other estimation methods to explore new iterative solution algorithms of some linear and nonlinear matrix equations and of the parameter estimation for dynamical systems and can be applied to other literatures such as signal processing and system modeling.

Acknowledgments

This work was supported by Natural Science Foundation of Anhui Provincial Education Department (KJ2020A0063) and Talent Introduction Projection of Anhui Science and Technology University (XWYJ201809).

Conflict of interest

The authors declare there is no conflict of interest.