
Citation: Yuyang Zhou, Randa Herzallah. Probabilistic message passing control and FPD based decentralised control for stochastic complex systems[J]. AIMS Electronics and Electrical Engineering, 2020, 4(2): 216-233. doi: 10.3934/ElectrEng.2020.2.216
[1] | Lawrence O. Aghenta, M. Tariq Iqbal . Design and implementation of a low-cost, open source IoT-based SCADA system using ESP32 with OLED, ThingsBoard and MQTT protocol. AIMS Electronics and Electrical Engineering, 2020, 4(1): 57-86. doi: 10.3934/ElectrEng.2020.1.57 |
[2] | Cherechi Ndukwe, M. Tariq Iqbal, Xiaodong Liang, Jahangir Khan, Lawrence Aghenta . LoRa-based communication system for data transfer in microgrids. AIMS Electronics and Electrical Engineering, 2020, 4(3): 303-325. doi: 10.3934/ElectrEng.2020.3.303 |
[3] | Jun Yoneyama . H∞ disturbance attenuation of nonlinear networked control systems via Takagi-Sugeno fuzzy model. AIMS Electronics and Electrical Engineering, 2019, 3(3): 257-273. doi: 10.3934/ElectrEng.2019.3.257 |
[4] | José M. Campos-Salazar, Pablo Lecaros, Rodrigo Sandoval-García . Dynamic analysis and comparison of the performance of linear and nonlinear controllers applied to a nonlinear non-interactive and interactive process. AIMS Electronics and Electrical Engineering, 2024, 8(4): 441-465. doi: 10.3934/electreng.2024021 |
[5] | Farhan Khan, Sing Kiong Nguang . Location-based reverse data delivery between infrastructure and vehicles. AIMS Electronics and Electrical Engineering, 2021, 5(2): 158-175. doi: 10.3934/electreng.2021009 |
[6] | J. Rajeshwari, M. Sughasiny . Modified PNN classifier for diagnosing skin cancer severity condition using SMO optimization technique. AIMS Electronics and Electrical Engineering, 2023, 7(1): 75-99. doi: 10.3934/electreng.2023005 |
[7] | Thanh Tung Pham, Chi-Ngon Nguyen . Adaptive PID sliding mode control based on new Quasi-sliding mode and radial basis function neural network for Omni-directional mobile robot. AIMS Electronics and Electrical Engineering, 2023, 7(2): 121-134. doi: 10.3934/electreng.2023007 |
[8] | Said Oucheriah, Abul Azad . Current-sensorless robust sliding mode control for the DC-DC boost converter. AIMS Electronics and Electrical Engineering, 2025, 9(1): 46-59. doi: 10.3934/electreng.2025003 |
[9] | Yipeng Yang . Stability analysis of networked control systems with bounded random delay and state compensation: How large is the actual system scale?. AIMS Electronics and Electrical Engineering, 2019, 3(1): 16-32. doi: 10.3934/ElectrEng.2019.1.16 |
[10] | Tarun Naruka, Debasis Tripathy, Prangya Mohanty . Power quality enhancement by a solar photovoltaic-based distribution static compensator. AIMS Electronics and Electrical Engineering, 2025, 9(2): 192-214. doi: 10.3934/electreng.2025010 |
Complex dynamical systems, which contain large ensembles of elements interacting with each other and are affected by noises and uncertainties, are ubiquitous in nature, industry, and human societies. The characterstics of complex dynamical systems, for instance, the high dimensionality of the systems, their complex structure, and high uncertainty bring difficulties in the systems analysis, estimation, and especially control. A considerable amount of literature about complex systems has been published to try to address the above issues. Examples include pinning control [1], multi-agent control [2], mean field optimal control [3], distributed control [4], exponential stabilization problem for stochastic complex networks [5,6], and probabilistic control [7,8,9,10,11,12,13]. However, for such a complicated system, its too challenging to control it in a centralised manner. The decentralised control is more suitable, considering that the network has no need to be controllable with a technically feasible amount of centrally controlled nodes. The basic idea of the decentralised control is to decompose the large-scale systems into several subsystems, and the local controllers only need to focus on individual nodes and achieve the local objectives. Decentralised control is proven as an efficient way to cope with complex systems, but most of the existing techniques lack communication between subsystems and all the decisions are based only on disconnected knowledge, which leads to unfulfilment of the global objective.
To address the above mentioned issues, this paper will introduce a computational and communication approach called message passing that allows the individual subsystems to harmonize their acting by sharing information between subsystems. The information in this approach is retrieved and disseminated in a consistent probabilistic manner, which can offer a precise expression of how information is shared between subsystems for stochastic systems. As one of the new techniques for decentralised methods [14], this approach has combined many subjects such as artificial intelligence [15], communications theory [16] and statistical physics [17]. However, the potential generalisations of the message passing approach have not been widely known by the control community field, which can be considered as one of the contributions of this paper.
Under the proposed control framework, this paper also considers the regulation problem for a class of complex stochastic systems with noises. For the propose of better coping with the stochastic nature and inherent randomness associated with the control of complex practical networks [13,18,19,20,21,22,23], the FPD [24,25,26,27,28] is implemented to each subsystem to accomplish the control goal. The main feature of the FPD is that it provides a specific form of the randomised optimal controller, which is the minimizer of the Kullback-Leibler divergence (KLD) [29] of the probability density describing the closed-loop dynamics to its ideal one. Furthermore, with the fact that the system parameters are usually unknown in real complex systems, linear optimization is applied in this paper to estimate the system parameters, implying that the whole system control strategy is implemented adaptively.
The contribution of this paper can be summarized as follows,
(1) This paper offers a clear and novel decentralised control framework for a class of large scale network complex systems, which can overcomes the difficulties in analysing and controlling such systems by controlling the subsystems in a decentralised manner;
(2) Since decisions in this framework are based only on disconnected knowledge, message passing is developed to allow coordination between the local subsystems. With message passing, not only all the individual local control objectives can be guaranteed, but also the global objective;
(3) Both the message passing approach and the FPD use its unified probabilistic language, which does not force the knowledge sharing subsystems to increase their complexity;
(4) Unlike the other existing distributed method, the concept of external state is introduced in this work, which can do the estimation job and guarantee the subsystem work normally once the connection with other subsystems is lost.
(5) This paper provides a basic understanding of the message-passing between the subsystems in the decentralised control process. For that purpose, the proposed strategy, including the information transmission and reception and how the local controller harmonizes the subsystems states based on the neighbours new received information, is demonstrated on a simple linear quadratic example for better understanding.
The remainder of this paper is organised as follows. Section Ⅱ introduces the structure of the subsystems and the estimation of the parameters. In Section Ⅲ, the FPD is presented and the stages of the proposed strategy are provided. Section Ⅳ gives details about the message passing approach. In Section Ⅴ, the proposed algorithm is applied to a numerical example to show its effectiveness. Finally, the conclusion is summarised in Section Ⅵ.
In this section, the basic decentralised probabilistic framework of message passing will be introduced. Within the proposed decentralised probabilistic framework, the considered complex system is decomposed into K subsystems based on the systems' conditions such as 1) Some states can be physically controlled together; 2) the global system is composed of many independent agents, like a drone, ext. Each subsystem is estimated as a probabilistic model and controlled independently by a probabilistic controller.
Unlike other decentralised frameworks, in this paper, the neighbouring subsystems states will also be treated as the considered subsystems states. More specifically, the full state of a subsystem is formed by two parts, internal states, and external states. The subsystem's own states are defined as internal states, which are controlled by their own local controllers. The states received from the neighbour subsystems are defined as external states, therefore the local controller has no control power over it. Note that the external state in each subsystem is also estimated as a linear probabilistic model, which is only related to itself. Furthermore, the local control decision will be made based on both internal and external states.
To better explain the proposed framework, we will demonstrate the subsystem structure and the message passing process using two subsystems: subsystem α and subsystem β as an example.
The considered subsystem α (which is also called node α) can be shown as follows,
xk;α=ˉA1xk−1;α+ˉA2yk−1;α+ˉBαuk−1;α+vk−1;α, | (2.1) |
yk;α=A3yk−1;α+wk−1;α, | (2.2) |
where the xk;α∈ℜn1 is the internal state of subsystem α and yk;α∈ℜn2 is the external state of the subsystem α which is related to the neighbour subsystem β. Note that the full state of subsystem α is formed as zk;α=[xTk;α,yTk;α]T. uk−1;α∈ℜn3 stands for the control input of subsystem α which need to be designed, ˉA1, ˉA2, A3, and ˉBα are the system parameters with appropriate dimensions. In addition, vk−1;α and wk−1;α are Gaussian noises with zero means and covariance Qα and Rα, respectively.
vk;α∼N(0,Qα),wk;α∼N(0,Rα). | (2.3) |
Similar to Eq (2.1) and Eq (2.2), the state equation of subsystem β is given by,
xk;β=ˉC1xk−1;β+ˉC2yk−1;β+ˉBβuk−1;β+vk−1;β, | (2.4) |
yk;β=C3yk−1;β+wk−1;β. | (2.5) |
where the xk;β∈ℜn2 represents the internal state of subsystem β and yk;β∈ℜn1 is the external state of the subsystem β which is passed from the neighbour subsystem α. Similarly, the full state of subsystem β is defined as zk;β=[xTk;β,yTk;β]T, uk−1;β∈ℜn4 stands for the control input of subsystem β which need to be designed, ˉC1, ˉC2, C3, and ˉBβ are the system parameters with appropriate dimensions. In addition, vk−1;β and wk−1;β are Gaussian noises with zero means and covariance Qβ and Rβ, respectively.
vk;β∼N(0,Qβ), | (2.6) |
wk;β∼N(0,Rβ). | (2.7) |
In each subsystem, Eq (2.1) and Eq (2.4) represent the internal state equation while Eq (2.2) and Eq (2.5) are the external state equations. Based on Eq (2.1) and Eq (2.4), we can see that the internal state xk depends on the previous internal state xk−1 and the previous external state yk−1 which is passed by the neighbour subsystem and the designed controller. This means that the internal states of each subsystem will be controlled by their own controller and will be affected by the external states received from neighbour subsystems. In addition, from Eq (2.2) and Eq (2.5), it can be seen that the external state yk is only affected by the previous external state yk−1. Besides, the designed local controller of each subsystem can only control their internal states. Another thing need to be noted is that the local subsystem control decision is made based on both their own states (internal states) and the neighbour systems states (external states). In this way, the global system can cooperatively achieve its goal. Without message passing between subsystems, each subsystem is controlled by its own control strategy independently and not affected by the neighbours' information, which leads to the consequence that the global system goal will never be reached.
In industrial processes, the precise system models are normally unavailable. Therefore, in this paper, all the system parameters will be estimated first. To achieve this, linear optimisation method is applied and details will be introduced in the following text using subsystem α as an example.
The system Eq (2.1) can be rewritten in the following form,
xk;α=θαφTk−1;α, | (2.8) |
where φk−1;α stands for the input matrix which is constructed as,
φk−1;α=[xTk−1;α,yTk−1;α,uTk−1;α]T, | (2.9) |
and θα is the weight matrix which is formed as follows,
θα=[ˉA1,ˉA2,ˉBα]. | (2.10) |
Denote the estimated weight matrix ˆθα as,
ˆθα=[A1,A2,Bα], | (2.11) |
where A1, A2 and B are the estimated parameters for ˉA1, ˉA2 and ˉBα. Note that at each step, all the observed data up to time k will be applied to estimate the system parameters to ensure accurate estimation. Thus using observed data up to time k, Ψ=[φ0;α,...,φk−1;α] and X=[xT1;α,...,xTk;α]T, one can get the following equation,
X=θαΨ. | (2.12) |
Based on Eq (2.12), the estimated weight matrix ˆθα can then be obtained as follows,
ˆθα=XΨ†, | (2.13) |
where Ψ† represents the pseudo inverse of Ψ which is given by,
Ψ†=ΨT(ΨΨT)−1. | (2.14) |
Similarly, using the same approach, the parameter A3 in Eq (2.2) can be estimated via the following equation,
A3=Yֆ, | (2.15) |
where Y is the output up to time k, Y=[yT1;α,...,yTk;α]T, Φ is the input up to time k, Φ=[yT0;α,...,yTk−1;α]T and Φ† represents the pseudo inverse of Φ.
Therefore, the conditional distribution of the system dynamic can be estimated as follows
s1(xk;α|uk−1;α,zk−1;α)∼N(μk;α,Qα), | (2.16) |
s2(yk;α|yk−1;α)∼N(δk;α,Rα), | (2.17) |
where
μk;α=A1xk−1;α+A2yk−1;α+Bαuk−1;α,δk;α=A3yk−1;α. | (2.18) |
Following the same estimation approach for subsystem α, the system parameters of subsystem β can be estimated and the system distribution is given as follows using the estimated parameters,
s1(xk;β|uk−1;β,zk−1;β)∼N(μk;β,Qβ),s2(yk;β|yk−1;β)∼N(δk;β,Rβ), | (2.19) |
where,
μk;β=C1xk−1;β+C2yk−1;β+Bβuk−1;β, | (2.20) |
δk;β=C3yk−1;β. | (2.21) |
where C1, C2, and Bβ are the estimated parameters for ˉC1, ˉC2, and ˉBβ, respectively.
Therefore, using the estimated parameters, the full state system equation of subsystem α can be specified in the following form,
zk;α=˜Azk−1;α+˜Bαuk−1;α+˜vk−1;α, | (2.22) |
where ˜A=[A1A20A3], ˜Bα=[Bα0], ˜vk−1;α=[vk−1;αwk−1;α]. Thus, the conditional distribution of the system dynamic of the full state of subsystem α can be expressed as
s(zk;α|uk−1;α,zk−1;α)∼N(˜μk;α,˜Qα), | (2.23) |
where
˜μk;α=˜Azk−1;α+˜Bαuk−1;α,˜Qα=[Qα00Rα]. | (2.24) |
Similarly, for subsystem β, the state equation of subsystem β is given by,
zk;β=˜Czk−1;β+˜Bβuk−1;β+˜vk−1;β, | (2.25) |
where ˜C=[C1C20C3], ˜Bβ=[Bβ0], ˜vk−1;β=[vk−1;βwk−1;β]. The conditional distribution of the system dynamic of the full state of subsystem β can be expressed as
s(zk;β|uk−1;β,zk−1;β)∼N(˜μk;β,˜Qβ), | (2.26) |
where
˜μk;β=˜Czk−1;β+˜Bβuk−1;β,˜Qβ=[Qβ00Rβ]. | (2.27) |
The control strategy presented in this paper is to design the subsystem controller for each subsystem and achieve the objective of each subsystem and then consequently achieve the objectives of the overall complex system. The objective of each subsystem considered in this paper is to design a randomised control input c(uk−1|zk−1) to solve the regulation problem and bring all the internal states back to zero. Considering the stochastic nature of the complex systems, the FPD will be employed to each subsystem to achieve that.
The performance index is formed by the Kullback-Leibler divergence (KLD) which is applied to describe the distance between the pdf of the joint distribution of the closed loop control system and the desired joint pdf. The KLD between the actual joint pdf f(F) of the observed data F=(x(H),u(H)) and the ideal joint pdf fI(F) on a set of possible F is defined as follows,
F(f‖fI)=∫f(F)ln(f(F)fI(F))dF, | (3.1) |
where H is the control horizon. According to the chain rule for pdfs [30], the joint distribution of the probabilistic closed-loop description of the system dynamics can be evaluated as follows:
f(F)=H∏k=1s(zk|uk−1,zk−1)c(uk−1|zk−1), | (3.2) |
where c(uk−1|zk−1) is the actual conditional pdf of system controller uk−1. Similarly, the ideal closed-loop pdf can be expressed in the same form as Eq (3.2) with ideal system model pdf sI(zk|uk−1,zk−1) and ideal controller pdf cI(uk−1|zk−1),
fI(F)=H∏k=1sI(zk|uk−1,zk−1)cI(uk−1|zk−1). | (3.3) |
With the KL-distance (3.1), the closed loop joint pdf (3.2) and the desired closed loop joint pdf (3.3), the performance index can be formalised to be given by the following expression:
−ln(γ(zk−1))=minc(uk−1|zk−1)∫s(zk|uk−1,zk−1)c(uk−1|zk−1)[ln(s(zk|uk−1,zk−1)c(uk−1|zk−1)sI(zk|uk−1,zk−1)cI(uk−1|zk−1))−ln(γ(zk))]d(zk,uk−1), | (3.4) |
where the first term in parenthesis in Eq (3.4) stands for the partial cost while the second term is the expected minimum cost-to-go function. The recursive formulation of performance index (3.4) is similar to Dynamic programming. Full derivation of Eq (3.4) can be found in [27].
Based on the Fully Probabilistic Design (FPD) [23,27,31], the control law c∗(uk−1|zk−1) for the subsystem which minimises the performance index (3.4) is given by,
c∗(uk−1|zk−1)=cI(uk−1|zk−1)exp[−β1(uk−1,zk−1)−β2(uk−1,zk−1)]γ(zk−1), | (3.5) |
where,
γ(zk−1)=∫cI(uk−1|zk−1)exp[−β1(uk−1,zk−1)−β2(uk−1,zk−1)]duk−1,β1(uk−1,zk−1)=∫s(zk|uk−1,zk−1)[lns(zk|uk−1,zk−1)sI(zk|uk−1,zk−1)],β2(uk−1,zk−1)=−∫s(zk|uk−1,zk−1)ln(γ(zk))dzk. | (3.6) |
Full derivation of Eqs (3.4)–(3.6) can be found in [7].
Following the FPD algorithm described by Eq (3.6), the generalised fully probabilistic control solution of the regulation problem using subsystem α as an example will be derived in this section. As mentioned in the last section, the objective of the controller is to return the system states back to zero from their initial values. Therefore, the ideal distribution of the system is specified as,
sI(zk;α|uk−1;α,zk−1;α)∼N(0,Σ2), | (3.7) |
where Σ2 means the ideal covariance of the state.
The ideal distribution of the controller can also be defined as follows,
cI(uk−1;α|zk−1;α)∼N(0,Γ), | (3.8) |
where Γ is the ideal covariance of the subsystem control input. Note that the covariance Γ indicates the allowed range of optimal control input.
Based on Eq (3.5), Eq (3.6), Eq (3.7) and Eq (3.8), the optimal controller form can be given by the following theorem.
Theorem 1. By submitting the ideal distribution of the system dynamics (3.7), the ideal distribution of the controller (3.8), and the real distribution of the system dynamics (2.23) and (2.24) into Eq (3.6), the optimal controller for system (2.1) that minimizes the performance index (3.4) is given by
uk;α=−Lk;αzk;α, | (3.9) |
where,
Lk;α=(Γ−1+˜BTαMk;α˜Bα)−1˜BTαMTk;α˜A,Mk;α=Σ−12+Sk;α,Sk−1;α=−˜AT(Σ−12+Sk;α)˜Bα[˜BTα(Σ−12+Sk;α)˜Bα+Γ−1]−1˜BTα(Σ−12+Sk;α)T˜A+˜AT(Σ−12+Sk;α)˜A. | (3.10) |
Note that this part is not the main contribution of this paper, therefore, FPD is taken as a ready methodology here. The detailed proof can be found in [7,24]. In addition, same method will be applied to the subsystem β, which will be omitted here.
We have stated the FPD control methodology for each subsystem in Section 3. As we mentioned earlier, without communication between neighbouring subsystems, each subsystem is controlled by its own control strategy individually, therefore might fail in achieving the global system goal. Besides, in this work the subsystem dynamics are described by probabilistic state space models considering the stochastic nature of the complex systems, which implies that the communication between subsystems should also be formed in a probabilistic fashion. Thus, in this section, a novel probabilistic framework for message passing between two subsystems in a decentralised and synchronous manner will be introduced using the progress of message passing from subsystem α to subsystem β as an example.
In general, the message passing approach can be divided into two stages: Passing and Receiving. More specific, once the internal states of subsystem α are updated, it will be passed in a probabilistic way to the neighbour subsystem β. After the subsystem β receive the probabilistic information that subsystem α passed, the next step is for the neighbour subsystem β to fuse the received information with its own prior external states distribution (2.19) to obtain the posterior external state distribution. The process can be shown as xk−1;α→yk−1;β. The detailed processes will be presented in two steps as follows.
The first step is for the subsystem α to pass its updated internal state distribution to subsystem β, which is given by the following theorem.
Theorem 2. Denote the distribution of the controller uk−1;α as follows,
c(uk−1;α|zk−1;α)∼N(mk−1;α,Σk−1;α), | (4.1) |
where mk−1;α is the mean of the input uk−1;α and Σk−1;α is the variance. The probabilistic model that subsystem α passes to subsystem β can be described as follows
Mβ←α∝Nxk;α(μk;β←α,Σk;β←α), | (4.2) |
where
μk;β←α=A1xk−1;α+A2yk−1;α+Bαmk−1;α,Σk;β←α=Qα+BαΣk−1;αBTα. | (4.3) |
Proof. Based on the chain rule, the conditional joint distribution of the behaviour of the closed loop system is given by
Lα(xk;α,yk;α,uk−1;α|zk−1;α)=s1(xk;α|uk−1;α,zk−1;α)s2(yk;α|yk−1;α)c(uk−1;α|zk−1;α). | (4.4) |
By substituting Eq (2.18) and Eq (4.1) into Eq (4.4), the conditional joint distribution of the behaviour of the closed loop system can be further expressed as follows,
Lα(xk;α,yk;α,uk−1;α|zk−1;α)∝exp{−0.5[xk;α−(A1xk−1;α+A2yk−1;α+Bαuk−1;α)]TQ−1α[xk;α−(A1xk−1;α+A2yk−1;α+Bαuk−1;α)]−0.5[yk;α−A3yk−1;α]TR−1α[yk;α−A3yk−1;α]−0.5[uk−1;α−mk−1;α]TΣ−1k−1;α[uk−1;α−mk−1;α]}. | (4.5) |
To update the knowledge that the subsystem β maintains about its external variables, all state variables of the closed loop probability density description of subsystem α need to be integrated except the internal states.
Mβ←α(xk;α|zk−1;α)=∫∫Lα(xk;α,yk;α,uk−1;α|zk−1;α)dyk;αduk−1;α. | (4.6) |
Substituting Eq (4.5) into Eq (4.6), we have
Mβ←α(xk;α|zk−1;α)=∫∫exp{−0.5[xk;α−(A1xk−1;α+A2yk−1;α+Bαuk−1;α)]TQ−1α[xk;α−(A1xk−1;α+A2yk−1;α+Bαuk−1;α)]−0.5[yk;α−A3yk−1;α]TR−1α[yk;α−A3yk−1;α]−0.5[uk−1;α−mk−1;α]TΣ−1k−1;α[uk−1;α−mk−1;α]}dyk;αduk−1;α∝exp{−0.5[xTk;αQ−1αxk;α+(A1xk−1;α+A2yk−1;α)TQ−1α(A1xk−1;α+A2yk−1;α)−2xTk;αQ−1α(A1xk−1;α+A2yk−1;α)+mTk−1;αΣ−1k−1;αmk−1;α−(Σ−1k−1;αmk−1;α+BTαQ−1αxk;α−BTαQ−1α(A1xk−1;α+A2yk−1;α))T(BTαQ−1αBα+Σ−1k−1;α)−1(Σ−1k−1;αmk−1;α+BTαQ−1αxk;α−BTαQ−1α(A1xk−1;α+A2yk−1;α))]}. | (4.7) |
The above Eq (4.7) can then be presented as,
Mβ←α(xk;α|zk−1;α)∝exp{−0.5[xTk;α(Q−1α−Q−TαBα(BTαQ−1αBα+Σ−1k−1;α)−1BαQ−1α)xk;α]+2xTk;αbk−1+ck−1]}, | (4.8) |
where,
bk−1=−Q−1α(A1xk−1;α+A2yk−1;α)+Q−TαBα(BTαQ−1αBα+Σ−1k−1;α)−1(BTαQ−1α(A1xk−1;α+A2yk−1;α)−Σ−1k−1;αmk−1;α), | (4.9) |
ck−1=(A1xk−1;α+A2yk−1;α)TQ−1α(A1xk−1;α+A2yk−1;α)+mTk−1;αΣ−1k−1;αmk−1;α−(Σ−1k−1;αmk−1;α−BTαQ−1α(A1xk−1;α+A2yk−1;α))T(BTαQ−1αBα+Σ−1k−1;α)−1(Σ−1k−1;αmk−1;α−BTαQ−1α(A1xk−1;α+A2yk−1;α)). | (4.10) |
Based on the Woodbury Identity, the term Q−1α−Q−TαBα(BTαQ−1αBα+Σ−1k−1;α)−1BαQ−1α in Eq (4.8) can be rewritten as following form,
Q−1α−Q−TαBα(BTαQ−1αBα+Σ−1k−1;α)−1BαQ−1α=(Qα+BαΣk−1;αBTα)−1. | (4.11) |
Then, Eq (4.8) can be shown as follows,
Mβ←α(xk;α|zk−1;α)∝exp{−0.5[(xk;α+(Qα+BαΣk−1;αBTα)bTk−1)T(Qα+BαΣk−1;αBTα)−1(xk;α+(Qα+BαΣk−1;αBTα)bTk−1)−bTk−1(Qα+BαΣk−1;αBTα)bk−1+ck−1]}. | (4.12) |
Using the push-through identity as,
Q−TαBα(BTαQ−1αBα+Σ−1k−1;α)−1=(Qα+BαΣk−1;αBTα)−1BαΣk−1;α, | (4.13) |
bk−1 can be simplified as,
bk−1=−(Qα+BαΣk−1;αBTα)−1(A1xk−1;α+A2yk−1;α+Bαmk−1;α). | (4.14) |
Similarly, ck−1 can be solved following the push-through identity and the Woodbury Identity as follows
ck−1=(A1xk−1;α+A2yk−1;α)T(Q−1α−Q−1αBα(BTαQ−1αBα+Σ−1k−1;α)−1BTαQ−1α)(A1xk−1;α+A2yk−1;α)+mTk−1;α×(Σ−1k−1;α−Σ−1k−1;α(BTαQ−1αBα+Σ−1k−1;α)−1Σ−1k−1;α)mk−1;α+2mTk−1;αΣ−1k−1;α(BTαQ−1αBα+Σ−1k−1;α)−1×BTαQ−1α(A1xk−1;α+A2yk−1;α). | (4.15) |
As
Σ−1k−1;α−Σ−1k−1;α(BTαQ−1αBα+Σ−1k−1;α)−1Σ−1k−1;α=Σ−1k−1;α−Σ−1k−1;α(Σk−1;α−Σk−1;αBTα×(Qα+BαΣk−1;αBTα)−1BαΣk−1;α)Σ−1k−1;α=BTα(Qα+BαΣk−1;αBTα)−1Bα, | (4.16) |
then Eq (4.15) is given by
ck−1=(A1xk−1;α+A2yk−1;α+Bαmk−1;α)T(Qα+BαΣk−1;αBTα)−1(A1xk−1;α+A2yk−1;α+Bαmk−1;α). | (4.17) |
Substituting bk−1 and ck−1 into Eq (4.12), we can get
Mβ←α(xk;α|zk−1;α)∝exp{−0.5[(xk;α−(A1xk−1;α+A2yk−1;α+Bαmk−1;α))T(Qα+BαΣk−1;αBTα)−1×(xk;α−(A1xk−1;α+A2yk−1;α+Bαmk−1;α))−(A1xk−1;α+A2yk−1;α+Bαmk−1;α)T(Qα+BαΣk−1;αBTα)−1(A1xk−1;α+A2yk−1;α+Bαmk−1;α)+(A1xk−1;α+A2yk−1;α+Bαmk−1;α)T×(Qα+BαΣk−1;αBTα)−1(A1xk−1;α+A2yk−1;α+Bαmk−1;α)]}=exp{−0.5[(xk;α−(A1xk−1;α+A2yk−1;α+Bαmk−1;α))T(Qα+BαΣk−1;αBTα)−1(xk;α−(A1xk−1;α+A2yk−1;α+Bαmk−1;α))]}. | (4.18) |
The proof is completed.
Once the subsystem β received the distribution (4.2) passed by the subsystem α, the subsystem β will update the prior external state distribution by merging with the newly obtained distribution. The theorem is given as follows.
Theorem 3. The posterior external state probability distribution yfused,β;k after fusing with newly obtained Mβ←α(xk;α|zk−1;α) is given as follows,
yfused,β;k∝N(μk;f;β,Σk;f;β), | (4.19) |
where,
μk;f;β=δk;β+Kk;β(μk;β←α−δk;β). | (4.20) |
Σk;f;β=Rβ−Kk;βRβ, | (4.21) |
Kk;β=Rβ(Rβ+Σk;β←α)−1. | (4.22) |
Proof. Recall the prior external distribution of subsystem β as follows
s2(yk;β|yk−1;β)∼N(δk;β,Rβ). | (4.23) |
The probabilistic model (4.2) that subsystem α passed to subsystem β and the distribution (4.23) that subsystem β has priori about its external state can be fused using Bayes' rule by multiplying the two together. Thus, the new probability description of the fusion of the information is given by
yfused,β;k∝exp{−(yk;β−μk;β←α)TΣ−1k;β←α(yk;β−μk;β←α)−(yk;β−δk;β)TR−1β(yk;β−δk;β)}=exp{−yTk;β(Σ−1k;β←α+R−1β)yk;β+2yTk;β(Σ−1k;β←αμk;β←α+R−1βδk;β)−μTk;β←αΣ−1k;β←αμk;β←α−δTk;βR−1βδk;β}=exp{−(yk;β−μk;f;β)TΣ−1k;f;β(yk;β−μk;f;β)+(Σ−1k;β←αμk;β←α+R−1βδk;β)T(Σ−1k;β←α+R−1β)−1×(Σ−1k;β←αμk;β←α+R−1βδk;β)−μTk;β←αΣ−1k;β←αμk;β←α−δTk;βR−1βδk;β}, | (4.24) |
where
μk;f;β=δk;β+Rβ(Rβ+Σk;β←α)−1(μk;β←α−δk;β), | (4.25) |
Σk;f;β=(Σ−1k;β←α+R−1β)−1. | (4.26) |
By defining
Kk;β=Rβ(Rβ+Σk;β←α)−1, | (4.27) |
Equations (4.25) and (4.26) can be expressed as follows,
μk;f;β=δk;β+Kk;β(μk;β←α−δk;β), | (4.28) |
Σk;f;β=Rβ−Kk;βRβ, | (4.29) |
which competes the proof.
Similarly, based on Theorem 2 and 3, the probabilistic distribution passed from subsystem β to subsystem α is taking the same form, which is given by,
Mα←β∝Nxk;β(μk;α←β,Σk;α←β), | (4.30) |
where
μk;α←β=C1xk−1;β+C2yk−1;β+Bβmk−1;β,Σk;α←β=Qβ+BβΣk−1;βBTβ. | (4.31) |
And the mean and covariance of the distribution that subsystem α merges the information passed by subsystem β are given by,
μk;f;α=δk;α+Kk;α(μk;α←β−δk;α), | (4.32) |
Σk;f;α=Rα−Kk;αRα, | (4.33) |
where,
Kk;α=Rα(Rα+Σk;α←β)−1. | (4.34) |
Remark 1. The message receiving form (4.28) and (4.32) take the same form as the updating equation from Kalman Filter as well as the gain (4.27) and (4.34).
Remark 2. Compared with the other existing distributed approaches, we have brought up the concept of the external state in this work. On the one hand, the external state equation can be treated as part of the full state, which makes the controller design part easier. On the other hand, the external state equation can briefly estimate the next time steps of external values, which can make sure each subsystem works normally if the connection with the neighbour subsystem is lost, or when the future external value is required for some systems. Connection lost and requirement of the prediction of the future state values will be demonstrated in our future work. Besides, our proposed framework follows a fully probabilistic approach, which is more efficient than most existing decentralised methods that usually do not take the stochastic property into the consideration.
The procedure of the proposed probabilistic message passing framework can then be summarized as follows:
1) Initialize all the subsystem states and parameters, including the system parameters A3 and C3 and the FPD Raccatti matrix S0;
2) Form the matrices ˜A and ˜Bα as specified in Eq (2.22). Apply the same for ˜C, and ˜Bβ as specified in Eq (2.25);
3) For each subsystem, calculate the control input of each subsystem, uk following Eqs (3.9)-(3.10) and update the system as give in Eqs (2.1)-(2.2) and Eqs (2.4)-(2.5). Then obtain the internal state xk;α, xk;β and the prior external state yk;α, yk;β;
4) Pack the new calculated internal state distributions and pass the states to neighbour subsystem as specified by Eqs (4.2)-(4.3) and Eqs (4.30)-(4.31);
5) At the same time as step 4, for each system, receive the external state probabilistic model passed by the neighbour subsystem and then update the external states following Eq (4.20) to Eq (4.22) and Eq (4.32) to Eq (4.34);
6) For the subsystem α and β, use new updated states to estimate the parameters A1, A2, A3, Bα, C1, C2, C3 and Bβ following Eqs (2.8)-(2.15);
7) Move to the next sampling instant k=k+1 and update the system using step 2.
To demonstrate the effectiveness of the proposed control strategy, the numerical example used in [32] will be employed in this paper to test the proposed framework.
The system is given by the following discrete time dynamical equation [32],
xk+1=[0.9429−0.02798−0.26110.022240.9798−0.021350.26160.014520.943]xk+[0.0093840.005471−0.00072−0.0015630.00931−0.00055−0.002088−0.001470.005401]uk+vk. | (5.1) |
where xk is the global system state, uk represents the system controller and vk is Gaussian noise with zero mean and variance 0.1.
vk∼N(0,0.1) |
Unlike [32] splitting the original system into three decoupled subsystems, in this paper the original system is divided into two subsystems for showing the presented control strategy clearer. The first subsystem is taking the first two states as the internal states while the second subsystem is taking the last state as the internal state. To control the whole system, there are two control inputs that will be formed for each subsystem, respectively. Thus, the first subsystem is given by:
xk;1=A(1)xk−1;1+B(1)uk−1;1+vk−1. | (5.2) |
where A(1)=[0.9429−0.02798−0.26110.022240.9798−0.0213500a33], B(1)=[0.0093840.005471−0.00072−0.0015630.00931−0.00055000] Similarly, the second subsystem is presented as,
xk;2=A(2)xk−1;1+B(2)uk−1;2+vk−1. | (5.3) |
where A(2)=[a11a120a21a2200.26160.014520.943], B(2)=[000000−0.002088−0.001470.005401]
Among these equation, the parameters a11,a12,a21,a22, and a33 are related to the external observable states which are updated and communicated between the two subsystems. They are initialised randomly and will be updated each instant.
The initial values of the states are x0=[2.5,−2.3,0.4]T. The objective for each subsystem is to use the FPD algorithm to design local controllers so that all the internal states of the individual subsystems are returned to the origin. Then the subsystems exchange their internal states information with each other via message passing method. In the end, all the system states should be all back to zero.
The simulation results are given in Figures 1–5. The system states in both subsystems are shown in Figures 1–3, which can be seen that all the states in both subsystems are back to the origin within 20 steps. That means the whole system is successfully decentrally controlled by individually controlling the two subsystems. In addition, from Figures 1–3, we can see that all the states in both subsystems match with each other's trajectories, which means that the message passing approach is successfully implemented. Figure 4 and Figure 5 show the FPD gain for subsystem 1 and subsystem 2, respectively. From Figure 4 and Figure 5, we can see that the gain converged to the optimal values within 20 steps, meaning that the FPD algorithm works in both subsystems. In conclusion, the control objective is successfully reached using our proposed control strategy.
This paper concentrated on the communication problems between the subsystems of complex dynamical stochastic systems. A detailed discussion about how the two subsystems exchange their new updated internal states has been offered. Also, the process detailing how the external states affect the subsystems and then reach the global control goal has been explained. In the meantime, a regulation problem has been considered for a complex system with a large number of subsystems and a decentralised control strategy using the proposed message passing approach has been developed adaptively. The conventional FPD has been applied to the subsystems as a randomised controller to reach the local control goals. Finally, the associated simulation results have been produced to verify the proposed control algorithm and the desired results have been obtained.
This work was supported by the Leverhulme Trust under Grant RPG-2017-337.
All authors declare no conflicts of interest in this paper.
[1] |
Porfiri M and Di Bernardo M (2008) Criteria for global pinning-controllability of complex networks. Automatica 44: 3100-3106. doi: 10.1016/j.automatica.2008.05.006
![]() |
[2] |
Van Den Broek B, Wiegerinck W, Kappen B (2008) Graphical model inference in optimal control of stochastic multi-agent systems. J Artif Intell Res 32: 95-122. doi: 10.1613/jair.2473
![]() |
[3] |
Fornasier M and Solombrino F (2014) Mean-field optimal control. ESAIM: Control, Optimisation and Calculus of Variations 20: 1123-1152. doi: 10.1051/cocv/2014009
![]() |
[4] |
Wang Z, Lu R, Shen B (2014) Distributed estimation and control for general systems. Int J Gen Syst 43: 247-253. doi: 10.1080/03081079.2014.883710
![]() |
[5] |
Li S, Yao X, Li W (2020) Almost sure exponential stabilization of hybrid stochastic coupled systems via intermittent noises: A higher-order nonlinear growth condition. J Math Anal Appl 489: 124150. doi: 10.1016/j.jmaa.2020.124150
![]() |
[6] | Xu Y, Shen R, Li W (2019) Finite-time synchronization for coupled systems with time delay and stochastic disturbance under feedback control. J Appl Anal Comput 10: 1-24. |
[7] |
Herzallah R and Kárnỳ M (2017) Towards probabilistic synchronisation of local controllers. Int J Syst Sci 48: 604-615. doi: 10.1080/00207721.2016.1197979
![]() |
[8] |
Herzallah R (2011) Enhancing the performance of intelligent control systems in the face of higher levels of complexity and uncertainty. International Journal of Modelling, Identification and Control 12: 311-327. doi: 10.1504/IJMIC.2011.040076
![]() |
[9] |
Herzallah R and Lowe D (2003) Robust control of nonlinear stochastic systems by modelling conditional distributions of control signals. Neural Comput Appl 12: 98-108. doi: 10.1007/s00521-003-0375-y
![]() |
[10] |
Herzallah R and Lowe D (2007) Distribution modeling of nonlinear inverse controllers under a bayesian framework. IEEE T neural networks 18: 107-114. doi: 10.1109/TNN.2006.883721
![]() |
[11] |
Zhang QC, Hu L, Gow J (2020) Output feedback stabilization for mimo semi-linear stochastic systems with transient optimisation. International Journal of Automation and Computing 17: 83- 95. doi: 10.1007/s11633-019-1193-8
![]() |
[12] |
Zhang Q and Wang A (2016) Decoupling control in statistical sense: minimised mutual information algorithm. International Journal of Advanced Mechatronic Systems 7: 61-70. doi: 10.1504/IJAMECHS.2016.082625
![]() |
[13] | Ren M, Zhang Q, Zhang J (2019) An introductory survey of probability density function control. Systems Science & Control Engineering 7: 158-170. |
[14] |
Aji SM and McEliece RJ (2000) The generalized distributive law. IEEE T Inform Theory 46: 325- 343. doi: 10.1109/18.825794
![]() |
[15] | Pearl J (2014) Probabilistic reasoning in intelligent systems: networks of plausible inference. Elsevier. |
[16] |
Gallager R (1962) Low-density parity-check codes. IRE Transactions on information theory 8: 21-28. doi: 10.1109/TIT.1962.1057683
![]() |
[17] | Mézard M, Parisi G, Virasoro M (1987) Spin glass theory and beyond: An Introduction to the Replica Method and Its Applications. World Scientific Publishing Company. |
[18] |
Zhou Y, Wang A, Zhou P, et al. (2020) Dynamic performance enhancement for nonlinear stochastic systems using rbf driven nonlinear compensation with extended kalman filter. Automatica 112: 108693. doi: 10.1016/j.automatica.2019.108693
![]() |
[19] | Zhou Y, Zhang Q, Wang H, et al. (2017) Ekf-based enhanced performance controller design for nonlinear stochastic systems. IEEE T Automat Contr 63: 1155-1162. |
[20] | Zhang Q, Zhou J, Wang H, et al. (2015) Minimized coupling in probability sense for a class of multivariate dynamic stochastic control systems. 2015 54th IEEE Conference on Decision and Control (CDC), 1846-1851. |
[21] | Herzallah R and Lowe D (2002) Improved robust control of nonlinear stochastic systems using uncertain models. |
[22] | Herzallah R and Lowe D (2006) Bayesian adaptive control of nonlinear systems with functional uncertainty. Proceedings of the 7th Portuguese Conference on Automatic Control. |
[23] |
Herzallah R (2018) Generalised probabilistic control design for uncertain stochastic control systems. Asian J Control 20: 2065-2074. doi: 10.1002/asjc.1717
![]() |
[24] | Zhou Y, Herzallah R, Zafar A (2019) Fully probabilistic design for stochastic discrete system with multiplicative noise. 2019 IEEE 15th International Conference on Control and Automation (ICCA), 940-945. |
[25] |
Kárnỳ M (1996) Towards fully probabilistic control design. Automatica 32: 1719-1722. doi: 10.1016/S0005-1098(96)80009-4
![]() |
[26] |
Kárnỳ M and Guy TV (2006) Fully probabilistic control design. Syst Control Lett 55: 259-265. doi: 10.1016/j.sysconle.2005.08.001
![]() |
[27] |
Herzallah R and Kárnỳ M (2011) Fully probabilistic control design in an adaptive critic framework. Neural networks 24: 1128-1135. doi: 10.1016/j.neunet.2011.06.006
![]() |
[28] |
Zhou Y and Herzallah R (2020) Dobc based fully probability design for stochastic system with the multiplicative noise. IEEE Access 8: 34225-34235. doi: 10.1109/ACCESS.2020.2974279
![]() |
[29] |
Kullback S and Leibler RA (1951) On information and sufficiency. The annals of mathematical statistics 22: 79-86. doi: 10.1214/aoms/1177729694
![]() |
[30] |
Peterka V (1981) Bayesian system identification. Automatica 17: 41-53. doi: 10.1016/0005-1098(81)90083-2
![]() |
[31] |
Herzallah R (2013) Probabilistic dhp adaptive critic for nonlinear stochastic control systems. Neural Networks 42: 74-82. doi: 10.1016/j.neunet.2013.01.014
![]() |
[32] | Alessio A and Bemporad A (2007) Decentralized model predictive control of constrained linear systems. 2007 European Control Conference (ECC), 2813-2818. |
1. | Qichun Zhang, Xuewu Dai, Special Issue “Uncertainties in large-scale networked control systems”, 2020, 4, 2578-1588, 345, 10.3934/ElectrEng.2020.4.345 | |
2. | Qichun Zhang, Yuyang Zhou, Recent Advances in Non-Gaussian Stochastic Systems Control Theory and Its Applications, 2022, 2653-6226, 111, 10.53941/ijndi0101010 | |
3. | Yuyang Zhou, Randa Herzallah, Probabilistic message passing control for complex stochastic switching systems, 2021, 358, 00160032, 5451, 10.1016/j.jfranklin.2021.04.040 | |
4. | Abdessamad Belfakir, Randa Herzallah, 2022, Estimation of the probabilistic description of quantum physical systems, 978-1-6654-9113-6, 154, 10.1109/QCE53715.2022.00034 | |
5. | Randa Herzallah, Abdessamad Belfakir, From certainty to chance: probabilistic insights into quantum control, 2024, 78, 1434-6060, 10.1140/epjd/s10053-024-00848-0 | |
6. | Randa Herzallah, Abdessamad Belfakir, Towards optimal control: the probabilistic route in quantum systems, 2024, 0020-7179, 1, 10.1080/00207179.2024.2342939 | |
7. | Yuyang Zhou, Randa Herzallah, Qichun Zhang, Decentralised fully probabilistic design for stochastic networks with multiplicative noise, 2023, 54, 0020-7721, 1841, 10.1080/00207721.2023.2210568 |