
Hydraulic servo actuators (HSAs) are often used in the industry in tasks that request great power, high accuracy and dynamic motion. It is well known that an HSA is a highly complex nonlinear system, and that the system parameters cannot be accurately determined due to various uncertainties, an inability to measure some parameters and disturbances. This paper considers an event-triggered learning control problem of the HSA with unknown dynamics based on adaptive dynamic programming (ADP) via output feedback. Due to increasing practical application of the control algorithm, a linear discrete model of HSA is considered and an online learning data driven controller is used, which is based on measured input and output data instead of unmeasurable states and unknown system parameters. Hence, the ADP-based data driven controller in this paper requires neither the knowledge of the HSA dynamics nor exosystem dynamics. Then, an event-based feedback strategy is introduced to the closed-loop system to save the communication resources and reduce the number of control updates. The convergence of the ADP-based control algorithm is also theoretically shown. Simulation results verify the feasibility and effectiveness of the proposed approach in solving the optimal control problem of HSAs.
Citation: Vladimir Djordjevic, Hongfeng Tao, Xiaona Song, Shuping He, Weinan Gao, Vladimir Stojanovic. Data-driven control of hydraulic servo actuator: An event-triggered adaptive dynamic programming approach[J]. Mathematical Biosciences and Engineering, 2023, 20(5): 8561-8582. doi: 10.3934/mbe.2023376
[1] | Na Zhang, Jianwei Xia, Tianjiao Liu, Chengyuan Yan, Xiao Wang . Dynamic event-triggered adaptive finite-time consensus control for multi-agent systems with time-varying actuator faults. Mathematical Biosciences and Engineering, 2023, 20(5): 7761-7783. doi: 10.3934/mbe.2023335 |
[2] | Xiangfei Meng, Guichen Zhang, Qiang Zhang . Robust adaptive neural network integrated fault-tolerant control for underactuated surface vessels with finite-time convergence and event-triggered inputs. Mathematical Biosciences and Engineering, 2023, 20(2): 2131-2156. doi: 10.3934/mbe.2023099 |
[3] | Xiaohan Yang, Yinghao Cui, Zhanhang Yuan, Jie Hang . RISE-based adaptive control of electro-hydraulic servo system with uncertain compensation. Mathematical Biosciences and Engineering, 2023, 20(5): 9288-9304. doi: 10.3934/mbe.2023407 |
[4] | Siyu Li, Shu Li, Lei Liu . Fuzzy adaptive event-triggered distributed control for a class of nonlinear multi-agent systems. Mathematical Biosciences and Engineering, 2024, 21(1): 474-493. doi: 10.3934/mbe.2024021 |
[5] | Qiushi Wang, Hongwei Ren, Zhiping Peng, Junlin Huang . Dynamic event-triggered consensus control for nonlinear multi-agent systems under DoS attacks. Mathematical Biosciences and Engineering, 2024, 21(2): 3304-3318. doi: 10.3934/mbe.2024146 |
[6] | Mingxia Gu, Zhiyong Yu, Haijun Jiang, Da Huang . Distributed consensus of discrete time-varying linear multi-agent systems with event-triggered intermittent control. Mathematical Biosciences and Engineering, 2024, 21(1): 415-443. doi: 10.3934/mbe.2024019 |
[7] | Chaoyue Wang, Zhiyao Ma, Shaocheng Tong . Adaptive fuzzy output-feedback event-triggered control for fractional-order nonlinear system. Mathematical Biosciences and Engineering, 2022, 19(12): 12334-12352. doi: 10.3934/mbe.2022575 |
[8] | Songyu Hu, Ruifeng Hu, Liping Tang, Weiwei Jiang, Banglin Deng . Quantitative generation of microfluidic flow by using optically driven microspheres. Mathematical Biosciences and Engineering, 2019, 16(6): 6696-6707. doi: 10.3934/mbe.2019334 |
[9] | Tongyu Wang, Yadong Chen . Event-triggered control of flexible manipulator constraint system modeled by PDE. Mathematical Biosciences and Engineering, 2023, 20(6): 10043-10062. doi: 10.3934/mbe.2023441 |
[10] | Duoduo Zhao, Fang Gao, Jinde Cao, Xiaoxin Li, Xiaoqin Ma . Mean-square consensus of a semi-Markov jump multi-agent system based on event-triggered stochastic sampling. Mathematical Biosciences and Engineering, 2023, 20(8): 14241-14259. doi: 10.3934/mbe.2023637 |
Hydraulic servo actuators (HSAs) are often used in the industry in tasks that request great power, high accuracy and dynamic motion. It is well known that an HSA is a highly complex nonlinear system, and that the system parameters cannot be accurately determined due to various uncertainties, an inability to measure some parameters and disturbances. This paper considers an event-triggered learning control problem of the HSA with unknown dynamics based on adaptive dynamic programming (ADP) via output feedback. Due to increasing practical application of the control algorithm, a linear discrete model of HSA is considered and an online learning data driven controller is used, which is based on measured input and output data instead of unmeasurable states and unknown system parameters. Hence, the ADP-based data driven controller in this paper requires neither the knowledge of the HSA dynamics nor exosystem dynamics. Then, an event-based feedback strategy is introduced to the closed-loop system to save the communication resources and reduce the number of control updates. The convergence of the ADP-based control algorithm is also theoretically shown. Simulation results verify the feasibility and effectiveness of the proposed approach in solving the optimal control problem of HSAs.
Important properties of the HSA, such as fast and accurate responses, a high force/mass ratio and relatively good stiffness, have attracted great interest in the HSA and its applications. In the last two decades, high-performance controller design of the HSA has attracted increasing attention due to the expanded performance requirements of technical systems in the industry [1,2,3,4].
A large number of machines driven by HSAs often work with high payloads in harsh and mostly external environments. As a result of variables of their environment, such as temperature, dust, humidity, wear, variable loads and disturbances, the HSA is usually subject to large uncertainties during operation. Hence, high-precision control of the HSA has always challenged researchers due to its unmodeled dynamics, large nonlinearities, parametric uncertainties, unmeasurable states in practice, etc. It is well known that it is impossible to determine most of the physical parameters of HSA components. While some HSA parameters are available only with certain accuracy, the other parameters are completely unknown. Dominant nonlinear sources existing in HSAs are impossibility of accurate determining parameters, which are very difficult to handle with high accuracy. These unknown parameters are caused by protection of proprietary data of individual manufacturers or indirect measuring and calculating, pressure losses, transient and turbulent flow conditions, friction, leakage characteristics, and generation discontinuous control signals to HSAs due to effects of saturation and changing the direction of servo valve. Furthermore, variable working conditions during operation, such as oil temperature, the bulk modulus, fluctuating supply pressure and pipe volume will lead to parameter changes, which worsen the existing control performances. These facts make it difficult to realize high-quality control of the HSA, which cannot be achieved without knowing the accurate HSA model [5,6,7,8].
Further, direct measurement of the whole HSA state vector is not feasible for practical implementation and in addition would require very expensive measurement equipment. It is more convenient to use control algorithms which apply methods based on state reconstruction rather than to perform direct measurements of the states [9].
In modern control theory, optimal control of the HSA plays a vital role in the controller design. Namely, the main challenge is to design optimal control algorithms that will affect the minimum energy consumption [10,11]. The optimal control design is an offline control technique that usually depends on perfect knowledge of the HSA model, which is not possible to obtain in most practical situations. Even if an approximated model of the HSA can be developed, the dynamic uncertainty, produced by the mismatch between the approximated model and the true HSA model, will degrade the control performance of the traditionally designed optimal controller [9,12]. Therefore, further research on the design of optimal controllers of HSAs is very important and our primary aim for this study.
The practical applicability of control algorithms is enhanced by the fact that nonlinear systems can be very precisely represented by linear models with online estimated dynamics [13,14]. Many modern engineering applications such as intelligent vehicles [15,16], modernized microgrids [17], microphone sensing [18], strain prediction for fatigue [19], maintaining the security of cyber–physical systems [20], robotic manipulation tasks [21], 2-degree-of-freedom helicopter [22] and requests for online controller design which rely on linear systems.
Adaptive dynamic programming (ADP) ensures an effective way to achieve high performance of the optimal controller which relies on adaptive control, optimal control and reinforcement learning [9,23,24,25,26]. ADP represents a kind of data-based control technique which can guarantee the stability of the feedback control system [9]. Recently, the field of ADP application has also been expanded to various research areas, including robotic systems, aerospace systems, guided missiles, spacecraft, etc. [27,28,29]. In circumstances of unknown system dynamics and unmeasurable states, of great interest is to use ADP techniques based on measured input/output data from linear systems, which are commonly called output feedback. A main benefit of the output feedback techniques is that knowledge of the HSA dynamics is not needed for their application. For an unknown HSA model, this indirect technique generates a sequence of suboptimal controllers which converge to the optimal control policy with an increasing number of iterations.
The implementation of ADP algorithms is usually based on periodic selection [30]. In order to save limited communication and computational resources, event-triggered strategies have recently started to be applied in control algorithms based on ADP [31,32,33,34]. Moreover, the number of updates of the control inputs in this way will be smaller compared to the periodic update of the controller, because it is updated only when necessary (e.g., when the performance of the system deteriorates). The implementation of event-triggered algorithms is based on aperiodic sampling. Several event-based controllers have been proposed in the literature, most of which are state-feedback controllers [35,36,37,38,39]. In contrast, this paper will consider the event-triggered ADP-based control problem of HSAs in the case where only output feedback is available.
This paper considers an online learning technique, where during operation, from measured input/output data, the controller learns to compensate unknown HSA dynamics, various disturbances and modeling errors, ensuring desired performances of the control system. The optimal control law is accomplished iteratively based on output feedback, state reconstruction and ADP. The unknown HSA model is first identified after which the algebraic Riccati equation (ARE) is iteratively solved. To ensure consistency of approximations and obtain unique solutions in each iteration step, some exploration noise must be added to control input to meet the requirements of the persistent excitation condition [40,41,42]. For exploration noise, some persistent excitation is usually applied such as white noise or pseudo random binary signals (PRBS). The selection of exploration noise is a non-trivial task for the most learning problems, as it can affect the accuracy of solutions, especially for large systems [43]. By applying the theory of experimental design, we will use the sum of sinusoidal signals as an exploration noise that will enable the output of the system to carry maximum information about the system, which will shorten the learning time, i.e., speed up the controller design process. Thus, the obtained input and output signals are used to reconstruct the state vector of the model, which is of great practical importance in relation to control techniques with direct state measurement that rely on a large number of sensors.
Due to implementation of the ADP-based control techniques, it is easier to realize data acquisition for the discrete-time HSA model in relation to its continuous-time model. ADP-based control methodology for discrete-time systems is proposed in [44].
We chose to use the measured input and output data to reconstruct the state vector of the discretized HSA model, after which ADP-based control can be implemented. The control law is learned iteratively and very efficiently provides solutions for optimal control of HSAs based only on measurements in real time. The main advantage of the proposed control methodology is avoiding the knowledge of system dynamics, which is very important under practical conditions.
By applying an event-based control strategy, the number of control input updates will be reduced relative to periodic update of the controller, because the controller is only updated when certain conditions are met. In this way, energy, computing and communication resources will be significantly preserved.
The rest of the paper is organized as follows. The problem of modelling an HSA with unknown dynamics is presented in Section 2. Event-triggered control based on ADP is shown in Section 3. Simulation results show the validity and effectiveness of the event-triggered ADP-based controller for HSAs in the presence of completely model uncertainty in Section 4. Finally, Section 5 gives the concluding remarks.
The HSA under study is shown in Figure 1, and it consists of a servo valve and a hydraulic cylinder. The analysis of the properties of the HSA comes out from the dynamics of its components, which involves the piston motion dynamics, pressure dynamics at the cylinder and servo valve dynamics. Hence, the model of the HSA is derived from complex nonlinear equations that depend on many parameters which cannot be accurately obtained [7,8].
See Table 1 for the description of the HSA parameters. Using the notation in Figure 1, and defining the area ratio of the piston α=Ab/Aa as well as Va=Va0+yAa, Vb=Vb0+(L−y)αAa and qLi=cLi(pa−pb), where cLi is the internal leakage flow coefficient; cvi>0 denote discharge coefficients, the sign function sg(x)={xx≥00x<0 and assuming an external leakage negligible, the considered model can be described by the following equations:
mt¨y=Aapa−Abpb−Ff(˙y)−Key–−Fext, | (2.1) |
˙pa=βeVa(y)(qa−Aa˙y−qLi−qLea), | (2.2) |
˙pb=βeVb(y)(qb+αAa˙y+qLi−qLeb), | (2.3) |
qa=qsv1−qsv2=cv1sg(xv)sign(ps−pa)√|ps−pa|−cv2sg(−xv)sign(pa−p0)√|pa−p0|, | (2.4) |
qb=qsv3−qsv4=cv3sg(−xv)sign(ps−pb)√|ps−pb|−cv4sg(xv)sign(pb−p0)√|pb−p0|. | (2.5) |
Notations | Descriptions |
xv | The spool valve displacement |
pa, pb | Forward and return pressure |
qa, qb | Forward and return flows |
y | Piston displacement |
L | Piston stroke |
Ke | Load spring gradient |
pS, p0 | Supply and tank pressure |
mt, mp, m | total mass, piston mass, payload mass |
Ff | Friction force |
Fext | Disturbance force |
Aa, Ab | Effective areas of the head and rod piston side |
Va, Vb, Va0, Vb0 | Fluid volumes of the head and rod piston side and corresponding initial volumes |
qLi, qLe | Internal and external leakage flow |
βe | Bulk modulus of the fluid |
According to Eqs (2.1)–(2.5), and by defining the state and input variables as
x(t)=[x1(t)x2(t)x3(t)x4(t)]T≜[y(t)˙y(t)pa(t)pb(t)]T, | (2.6) |
u(t)=xv(t), | (2.7) |
the governing nonlinear continuous-time dynamics of the HSA can be expressed in a state-space form as follows:
˙x(t)=f(x(t))+g(x(t),u(t))+h(t),y(t)=η(x(t)), | (2.8) |
where f(x(t)) and g(x(t),u(t)) are the state dynamics and the input function, respectively:
f(x(t))=[x21mt(Aax3−αAax4−Ff(x2)−Kex1)−βeAax1+Va0(Aax2+cLi(x3−x4))βeαAa(L−x1)+Vb0(αAax2+cLi(x3−x4))], |
g(x(t),u(t))=[00βeAax1+Va0(cv1sg(u)sign(ps−x3)√|ps−x3|−−cv4sg(−u)sign(x3−p0)√|x3−p0|)βeαAa(L−x1)+Vb0(cv3sg(−u)sign(ps−x4)√|ps−x4|−−cv2sg(u)sign(x4−p0)√|x4−p0|)], |
where output function η(x(t))=x1(t) and disturbance function h(t)=[h1(t)−Fext/mt+h2(t)h3(t)h4(t)] include loads, unmodelled dynamics and parameter uncertainties.
One of the main nonlinearities of the cylinder model is the nonlinear friction force Ff, which consists of static friction, Coulomb friction and Stribeck effect of velocity. An extensive study related to acting friction forces upon the HSA can be found in [7]. Further, we consider the linearized model of the HSA, whose parameters are experimentally identified for different working points of the HSA (i.e., different positions and external load conditions) [8]. Now, the model equations are expressed in a more suitable way in terms of the load pressure:
pL=pa−αpb, | (2.9) |
which leads to simplified dynamic equations. At last, using the new state vector [x1(t)x2(t)x3(t)]T≜[y(t)˙y(t)pL(t)]T allows us to express the HSA in a more compact form. Taking an operating point x0≜[y0˙y0pL0]T, and assuming dominance of the first order term from the Taylor series expansion, the linearized continuous-time description of the reduced order is stated as follows
˙x(t)=Ax(t)+B(t)u(t), | (2.10) |
y(t)=Cx(t), | (2.11) |
where A=[0100−BCmtAamt0−KdKp], B=[00Kx]T and C=[100]. The sensibility constants can be found as follows:
Kd=Aa(βeVA+α2βeVB)−1,Kp=βe(KpA−CLi(1+α2))VA(1+α2)−αβe(KpBα2+CLi(1+α2))VB(1+α2),Kx=βeVAKxA−αβeVBKxB, |
where the flow sensibility constants regarding the pressure at the cylinder chambers are stated as:
KpA={−cvxv0√ps−pA0forxv>0−cvxv0√pA0−p0forxv<0, | (2.12) |
KpB={−cvxv0√pB0−p0forxv>0−cvxv0√ps−pB0forxv<0, | (2.13) |
and the flow sensibility constants regarding the spool position are stated as
KxA={cv√ps−pA0forxv>0−cv√pA0−p0forxv<0, | (2.14) |
KxB={−cv√pB0−p0forxv>0cv√ps−pB0forxv<0. | (2.15) |
The previously mentioned valve sensibility constants are very significant in defining system stability and other dynamic characteristics [8]. Namely, the flow gain Kx has a direct impact on the stability of the HSA, because it directly affects the gain constant in the open loop of the HSA. Further, direct impact on the damping ratio of the HSA has the flow-pressure constant Kp. Hence, the pressure sensibility Kpx=Kx/Kp is quite high, which explains the ability of the HSA to transfer large friction loads with a small error.
Let us consider a linear continuous-time model of the HSA with unknown dynamics, as follows:
˙x(t)=Ax(t)+Bu(t), | (3.1) |
y(t)=Cx(t), | (3.2) |
where x(t)∈Rn, u(t)∈Rm and y(t)∈Rr are the system state vector, the control input vector, and the output vector, respectively. A∈Rn×n, B∈Rn×m and C∈Rr×n are unknown system matrices, assuming that (A,B) is controllable and (A,C) is observable.
For the HSA described by (3.1) and (3.2), the performance index is stated as
J(x0)=∞∫0[yT(τ)Qy(τ)+uT(τ)Ru(τ)]dτ, | (3.3) |
where x0∈Rn is an initial state, Q=QT≥0 and R=RT>0, with (A,Q1/2C) being observable.
A control law is also called a policy. The design objective is to find a linear optimal control policy in the form of
u=−K∗x, | (3.4) |
which minimizes the performance given by index (3.3). The optimal feedback gain matrix K∗can be determined as
K∗=R−1BTP∗, | (3.5) |
where P∗=(P∗)T>0 is a unique symmetric positive definite solution of the well-known ARE
ATP∗+P∗A+CTQC−P∗BR−1BTP∗=0, | (3.6) |
under conditions that the system matrices are accurately known, as well as conditions that the pair (A,B) is controllable and the pair (A,Q1/2C) is observable [12]. It should be noted that this optimal control design is mainly applicable to low order simple linear systems. In fact, for high-order large scale systems, it is usually difficult to directly solve P∗ from (3.6), which is nonlinear in P.
Also, for practical implementation of the control system, it is easier to realize the data acquisition for discrete-time systems than for continuous-time systems. Consequently, we transform the continuous-time HSA into the following discrete-time HSA:
xk+1=Adxk+Bduk, | (3.7) |
yk=Cxk, | (3.8) |
where Ad=eAh and Bd=h∫0(eAτdτ)B, where h>0 is a specific sampling period, assuming ωh=2π/h is the nonpathological sampling frequency whose existence is well known [45]. In other words, the controllability and observability of the original continuous-time HSA system is kept after discretization. Namely, if the state, input and output vectors at the sampled instant kh are xk, uk and yk, respectively, then (Ad,C) and (Ad,Q1/2C) are observable while (Ad,Bd) is controllable.
As depicted in Figure 2, the ADP-based controller for the discretized HSA system consists of three parts: the state reconstruction, critic, and actor. The state reconstruction provides the relationship between the input/output data and the HSA states, which allows one to solve the optimal control problem of an HSA with unknown dynamics. Based on the input/output data, the critic part of the controller is designed to evaluate the performance of the control policy. The controller learns online in order to maximize its performance. Finally, the actor applies the improved control policy. The updates of the control actions are governed by an event-triggering mechanism to reduce the amount of data transmission from the controller to the HSA system.
The event-triggered design is based on a periodic sampling with a nonpathological h>0. We use ˆuk to represent the sampled value of uk, that is
ˆuk=ukj,k∈[kj,kj+1), | (3.9) |
where {kj}∞0 is a monotonically increasing sequence of the sampling time instants, and the control input is only updated at the discrete-time instants: k0,k1,k2,… For the convenience of discussions, define the sampling error of the input data as
Δk=ˆuk−uk. | (3.10) |
Hence, the discrete-time system described by (3.7) and (3.8) can be rewritten as
xk+1=Adxk+(Bduk+Δk), | (3.11) |
yk=Cxk. | (3.12) |
Further, the performance index for the discretized system described by (3.7) and (3.8) is
Jd(x0)=∞∑j=0yTjQdyj+uTjRduj, | (3.13) |
where Qd=Qh and Rd=Rh. The optimal control low minimizing (3.13) is
uk=−K∗dxk, | (3.14) |
where the discrete optimal feedback gain matrix is K∗d=(R+BTdP∗dBd)−1BTdP∗dAd, where P∗d is the unique symmetric positive definite solution to
ATdP∗dAd−P∗d+CTQC−ATdP∗dBdK∗d=0. | (3.15) |
Since (3.15) is nonlinear in P∗d, it is difficult to directly solve P∗d for high-order large-scale systems. Nevertheless, many efficient algorithms have been developed to numerically approximate the solution of (3.15). One of such algorithms was developed by Hewer [46], and it is introduced in the form of Lemma 3.1.
Lemma 3.1. Let K0∈Rm×n be any stability feedback gain matrix and Pj be the symmetric positive definite solution of the Lyapunov equation
(Ad−BdKj)TPj(Ad−BdKj)+CTQdC+KTjRdKj=0, | (3.16) |
where Kj, j=1,2,… can be updated as follows:
Kj=(R+BTdPj−1Bd)−1BTdPj−1Ad. | (3.17) |
Then, it holds that
1) Ad−BdKj is a stability matrix
2) P∗d≤Pj+1≤Pj
3) limj→∞Kj=K∗d, limj→∞Pj=P∗d.
By iteratively solving the Lyapunov equations given by (3.16), which is linear in Pj, and recursively updating the control policy Kj by (3.17), the solution to the nonlinear equation given by (3.15) is numerically approximated [46]. It has been concluded that the sequences {Pj}∞j=0 and {Kj}∞j=0, computed from this algorithm, converge to P∗d and K∗d, respectively. Moreover, for j=0,1,…, Ad−BdKj is a Schur matrix. It should be noted that the method by Hewers involves a model-based policy iteration (PI) algorithm, which cannot be directly applied to the problem studied in this paper since it is an offline algorithm which depends on the system parameters. To apply this algorithm online for the discretized HSA described by (3.7) and (3.8), we will develop the control algorithm based on ADP via output feedback, which does not depend on the knowledge of HSA matrices.
Motivated by [44,47], the discrete-time HSA described by (3.7) and (3.8) can be extended by using input/output sequences on the time horizon [k−N,k−1] as follows:
xk=ANdxk−N+V(N)ˉuk−1,k−N,ˉyk−1,k−N=U(N)xk−N+T(N)ˉuk−1,k−N, | (3.18) |
where
ˉΔk=[ΔTk−1ΔTk−2…ΔTk−N]T,ˉuk−1,k−N=[ˆuTk−1ˆuTk−2…ˆuTk−N]T,ˉyk−1,k−N=[yTk−1yTk−2…yTk−N]T,V(N)=[BdAdBd…AN−1dBd]],U(N)=[(CAN−1d)T(CAd)T…CT]T,T(N)=[0CBdCAdBd⋯CAN−2dBd00CBd⋯CAN−3dBd⋮⋮⋱⋱⋮00⋯0CBd00⋯00], |
and the observability index is N=max(ρu,ρv) where ρu is the minimum integer which ensures that U(ρu) has full column rank and ρv is the minimum integer which ensures that V(ρv) has full row rank [44]. Therefore, there exists a left inverse of U(N), stated as U+(N)=[UT(N)U(N)]−1UT(N). With the state reconstruction in (3.18), the idea of an ADP-based controller with output feedback can be applied to solve the optimal control problem of HSAs with unknown dynamics. A uniqueness of state reconstruction is stated in the form of Lemma 3.2 as follows [48].
Lemma 3.2. If the conditions of observability and controllability of the system described by (3.7) and (3.8) are fulfilled, then the states of the HSA are uniquely received in terms of measured inputs and outputs signals as follows:
xk=Θzk, | (3.19) |
where Θ=[MuMy] has full row rank, Mu=V(N)−MyT(N), My=ANdU+(N) and zk=[ˉuTk−1,k−NˉyTk−1,k−N]T∈Rq, where q=N[dim(u)+dim(y)].
Now, based on (3.16) and (3.17), an online learning strategy using output feedback can be introduced in the form of u∗k=−¯Kdzk, providing suboptimal property of the closed-loop system. The discrete-time model (3.11) can be stated as follows:
xk+1=Ajxk+Bd(Kjxk+ˆuk), | (3.20) |
where Aj=Ad−BdKj. Setting ˉKj=KjΘ and ˉPj=ΘTPjΘ, from (3.16) and (3.20), it can be obtained
zTk+1ˉPjzk+1−zTkˉPjzk=(ˉKjzk+ˆuk)T[BTdˉPjBdBTdˉPjAd][−ˉKjzk+ˆuk2zk]−(yTkQyk+zTkˉKTjRˉKjzk)=[ˆuTk⊗ˆuTk−(zTk⊗zTk)(ˉKTj⊗ˉKTj)]vec(ˉH1j)+2[(zTk⊗zTk)(Iq⊗ˉKTj)+(zTk⊗uTk)]vec(ˉH2j)−(yTkQyk+zTkˉKTjRˉKjzk)∧=ϕ1vec(ˉH1j)+ϕ2vec(ˉH2j)−(yTkQyk+zTkˉKTjRˉKjzk), | (3.21) |
where ˉH1j=BTdˉPjBd, ˉH2j=BTdˉPjAdΘ, ϕ1=ˆuTk⊗ˆuTk−(zTk⊗zTk)(ˉKTj⊗ˉKTj) and ϕ2=2[(zTk⊗zTk)(Iq⊗ˉKTj)+(zTk⊗ˆuTk)].
The symbol ⊗ is used to denote a Kronecker product operator. The vector function vec(V)=[vT1vT2…vTm]T is stated as an mn-vector formed by stacking the columns of V∈Rn×m on top of one another, where vi∈Rn denotes the columns of matrix V. For an arbitrary symmetric matrix M∈Rn×n, vecs(M)=[m11,2m12,…,2m1n,m22,2m23,…,2mn−1,n,mnn]T∈Rn(n+1)/2 and for an arbitrary column vector v∈Rn, ˜v=[v21,v1v2,…,v1vn,v22,v2v3,…,vn−1vn,v2n]T∈Rn(n+1)/2.
The convergence of the online learning control using output feedback is guaranteed under the rank condition stated in the form of Lemma 3.3 [47]. Lemma 3.3 is about the condition of persistent excitation in adaptive control theory [49,50].
Lemma 3.3. Let us suppose that for a sufficiently large s∈Z+, it holds that
rank(Γ)=(dim(u)+dim(z))(dim(u)+dim(z)+1)/2, | (3.22) |
where
Γ=[ηk0⊗ηk0,ηk1⊗ηk1,⋯,ηks⊗ηks],wherek0<k1<⋯<ks∈Z+andηkj=[ˆuTkj,zTkj]T,j=¯0,s; | (3.23) |
then (ˉPj,ˉH1j,ˉH2j) can be uniquely solved based on ˉKj and measurable online data during the period k∈[k0,ks]. Further, ˉKj+1 is obtained as follows:
ˉKj+1=(R+ˉH1j)−1ˉH2j. | (3.24) |
Some exploration noise ek, which satisfies the persistent excitation condition, must be added into the input signal during the online learning phase due to the satisfaction of the rank condition given by (3.22), without affecting the convergence of the learning phase [43,51,52]. Note that (3.21) is called the policy evaluation, which is used to uniquely solve ˉPj, and (3.24) is the policy improvement, which is used to update the control gain ˉKj+1. Finally, we present the ADP-based online learning control algorithm in Figure 3.
It should be noted that solving (3.21) instead of (3.16), completely eliminates the original request on the accurate knowledge of the HSA dynamics. Now, we only need to measure uk and yk. Namely, having in mind the expression for zk, we can see that the control policy ˆuk=−¯K∗kzk+Δk contains only the previously measured input-output data. With the event-triggered control law ˆuk, the system given by (3.20) is globally asymptotically stable (GAS) at the origin if
‖Δk‖2≤αγ‖yk‖2+λmin(Rd)‖ˆuk‖2η, | (3.25) |
where α∈(0,1) and η is a positive constant satisfying η≥λmax(Rd+BTdˉPdBd).
The convergence of the ADP-based control algorithm is presented in the form of Theorem 3.4. For Hurwitz feedback matrix A−BK, K∈Rm×n is called stabilizing feedback gain matrix for a linear system ˙x=Ax+Bu.
Theorem 3.4. If the condition of Lemma 3.3 is fulfilled, with some initial stabilizing feedback gain matrix ¯K0, then the sequences {¯Pj}∞j=0 and {¯Kj}∞j=0 received from this algorithm, converge to their optimal values ¯P∗and ¯K∗, respectively [46,47].
Proof. If Pj=PTj represents the solution of (3.16), under the stability feedback gain matrix ¯Kj, then Kj+1 is uniquely obtained from (3.17). It can be easily shown that ¯Pj and ¯Kj+1 fulfill (3.21) and (3.24). Now, setting ¯P and ¯K as solutions of (3.21) and (3.24), Lemma 3.3 provides that ¯Pj=¯P and ¯Kj+1=¯K are uniquely stated. Furthermore, from Lemma 3.1, we have that limj→∞¯Kj=¯K∗d and limj→∞¯Pj=¯P∗d. The proof of convergence is proved.
The hybrid nature of the controller is shown in Figure 4. It is shown there that the feedback gain or policy is updated at discrete times by using (3.24) after the solution to (3.21) has been determined. On the other hand, the control input is a discrete time signal depending on the state z(k) at each time k. From Figure 4, it can be seen that the control gains are updated at discrete times, but the control signal is piecewise continuous.
In this section, we apply the proposed event-triggered ADP-based control design to the HSA. In the case of unknown dynamics and unmeasurable states of the HSA, it is meaningful to use the ADP-based method. Consequently, we conduct simulations on the HSA given by the linearized continuous-time description of (2.10) and (2.11) to show the effectiveness of the ADP-based control algorithm. A basic condition for energy savings in many hydraulically driven industrial systems is a high-quality design of event-triggered ADP control for the HSA.
For this purpose, the HSA is discretized by applying the periodic sampling period h=0.1 s and the zero-order holder. The approximated optimal feedback gain and performance index for the discretized model of the HSA are iteratively obtained.
The effectiveness of the ADP-based control algorithm will be considered for the HSA model described by (2.10) and (2.11) with the following parameters: the viscous friction BC=200 N s m−1, the supply pressure pS=45 bar, the tank pressure p0=1.6 bar, the bulk modulus of the fluid βe=2×108 Pa, the total mass m=25 kg, the initial chamber volumes Va0=Vb0=8.2×10−6 m3, the load spring gradient Ke=10−1, the effective area of the head side of the piston Aa=4.91×10−4 m2, the effective area of the rod side of the piston Ab=2.43×10−4 m2, the internal leakage coefficient cLi=5×10−14, the piston stroke L=1 m and discharge coefficients of valve orifices cvi=1.15, i=¯1,4.
For the purpose of demonstrating the event-triggered ADP method with the HSA, the weight matrices, Q and R, are chosen to be identity matrices, the observability index is N=3, initial state vector is x0=[5−5−10] and the convergence threshold ε is selected as 10−1.
It should be noted that our event-driven ADP control design does not require exact knowledge of the HSA matrices. But, only for numerical verification via simulation, it is assumed that the system matrices in (2.10) and (2.11) are known.
To verify the benefits of the ADP based online learning controller, Figure 5 depicts the errors between ˉPj and ˉP∗d and ˉKj and ˉK∗d, which indicate the convergence of ˉPj and ˉKj.
The evolution of the maximum cost for HSA is shown in Figure 6(a), where V1 is the maximum cost by using the initial control policy, and V7 is the maximum cost by using the control policy after seven iterations. It can be seen that the approximated cost function V7 has been remarkably reduced relative to the initial cost V1. Figure 6(b) shows the 3D plot of the approximation error of the cost function. This error is close to zero which confirms that good approximation of the optimal cost function is achieved during the learning process.
The improved control policy and the initial control policy are compared in Figure 7(a). Further, Figure 7(b) shows the 3D plot of the difference between the approximated control obtained by using the online ADP-based control algorithm and the optimal control. This error is close to zero, which confirms that good approximation of the optimal input is also achieved during the learning process.
Figure 8 shows the control input and the states of the HSA system desribed by (2.10) and (2.11) by using the ADP-based controller with periodic sampling.
To illustrate the benefits of the event-triggered ADP method, the control input and the states of the original HSA system described by (2.10) and (2.11), as obtained by using the event-triggered ADP-based controller is shown in Figure 9.
The comparison of sampling numbers by using the event-triggered ADP controller versus the ADP controller with periodic sampling is shown in Figure 10.
It can be observed that similar control effects have been achieved by the two methods, however, for the event-triggered ADP method, the control input is updated only when the squared norm of the triggering error reaches the threshold, and it is kept constant otherwise. It is also shown that about 54% communication between the controller and the HSA is reduced by using the event-triggered ADP method instead of the ADP method. The sequence of steps of event-triggered sampling is depicted in Figure 11.
This paper has considered the event-triggered data-driven optimal controller of the HSA with completely unknown dynamics as based on an ADP framework. A basic advantage of the presented control methodology is its ability to avoid the knowledge of entire system dynamics, which is very important in real conditions. By using the output feedback and the state reconstruction method an applied ADP-based control technique has been shown to be a useful tool for digital implementation in a real HSA. For that purpose, a discrete-time control policy was iteratively learned based on the discretized HSA model. The learned control policy very efficiently ensures online solutions to data-driven optimal control problems for the HSA. The presented online control policy only uses measured input/output data to learn the optimal control gain. Then, to reduce the communication between the controller and the HSA, an output feedback event-triggered ADP controller has been designed. The simulation results have shown the validity and effectiveness of the applied control approach for the HSA.
This research was supported in part by the Serbian Ministry of Education, Science and Technological Development under grant 451-03-47/2023-01/200108, the National Natural Science Foundation of China under grants 61773181, 61976081, 62073001, 62103293 and 62203153,111 Project under grant B23008, the Fundamental Research Funds for the Central Universities under grant JUSRP51733B, the Anhui Provincial Key Research and Development Project under grant 2022i01020013 and the Natural Science Fund for Excellent Young Scholars of Henan Province under grant 202300410127.
The authors declare that there is no conflict of interest.
[1] | J. Vyas, B. Gopalsamy, H. Joshi, Electro-Hydraulic Actuation Systems: Design, Testing, Identification and Validation, 1st edition, Springer, Singapore, 2019. https://doi.org/10.1007/978-981-13-2547-2 |
[2] | A. Vacca, G. Franzoni, Hydraulic Fluid Power: Fundamentals, Applications, and Circuit Design, 1st edition, John Wiley & Sons, Hoboken, New Jersey, 2021. |
[3] | N. Manring, Fluid Power Pumps and Motors: Analysis, Design, and Control, 1st edition, McGraw-Hill Education, New York, 2013. |
[4] |
N. Nedic, V. Stojanovic, V. Djordjevic, Optimal control of hydraulically driven parallel robot platform based on firefly algorithm, Nonlinear Dyn., 82 (2015), 1457–1473. https://doi.org/10.1007/s11071-015-2252-5 doi: 10.1007/s11071-015-2252-5
![]() |
[5] |
V. Stojanovic, N. Nedic, D. Prsic, Lj. Dubonjic, V. Djordjevic, Application of cuckoo search algorithm to constrained control problem of a parallel robot platform, Int. J. Adv. Manuf. Technol., 87 (2016), 2497–2507. https://doi.org/10.1007/s00170-016-8627-z doi: 10.1007/s00170-016-8627-z
![]() |
[6] |
V. Filipovic, N. Nedic, V. Stojanovic, Robust identification of pneumatic servo actuators in the real situations, Forsch. Ingenieurwes., 75 (2011), 183–196. https://doi.org/10.1007/s10010-011-0144-5 doi: 10.1007/s10010-011-0144-5
![]() |
[7] | J. F. Blackburn, G. Reethof, J. L. Shearer, Fluid Power Control, MIT Press, Cambridge, Massachusetts, 1960. |
[8] | M. Jelali, A. Kroll, Hydraulic Servo-Systems: Modelling, Identification and Control, 1st edition, Springer, London, 2003. https://doi.org/10.1007/978-1-4471-0099-7 |
[9] | F. L. Lewis, D. Liu, Reinforcement Learning and Approximate Dynamic Programming for Feedback Control, 1st edition, John Wiley & Sons, Hoboken, New Jersey, 2013. |
[10] | D. Bertsekas, Reinforcement Learning and Optimal Control, 1st edition, Athena Scientific, Belmont, Massachusetts, 2019. |
[11] | D. Bertsekas, Dynamic Programming and Optimal Control: Volume I, 4th edition, Athena Scientific, Belmont, Massachusetts, 2012. |
[12] | F. L. Lewis, D. Vrabie, V. L. Syrmos, Optimal Control, 3rd edition, John Wiley & Sons, Hoboken, New Jersey, 2012. |
[13] | M. Tomás-Rodríguez, S. P. Banks, Linear, Time-Varying Approximations to Nonlinear Dynamical Systems: With Applications in Control and Optimization, 1st edition, Springer, London, 2010. https://doi.org/10.1007/978-1-84996-101-1 |
[14] |
V. Stojanovic, D. Prsic, Robust identification for fault detection in the presence of non-Gaussian noises: application to hydraulic servo drives, Nonlinear Dyn., 100 (2020), 2299–2313. https://doi.org/10.1007/s11071-020-05616-4 doi: 10.1007/s11071-020-05616-4
![]() |
[15] |
M. Mynuddin, W. Gao, Distributed predictive cruise control based on reinforcement learning and validation on microscopic traffic simulation, IET Intel. Transport Syst., 14 (2020), 270–277. https://doi.org/10.1049/iet-its.2019.0404 doi: 10.1049/iet-its.2019.0404
![]() |
[16] | M. Mynuddin, W. Gao, Z. P. Jiang, Reinforcement learning for multi-agent systems with an application to distributed predictive cruise control, in American Control Conference (ACC), (2020), 315–320. https://doi.org/10.23919/ACC45564.2020.9147968 |
[17] |
M. Davari, W. Gao, Z.P. Jiang, F. L. Lewis, An optimal primary frequency control based on adaptive dynamic programming for islanded modernized microgrids, IEEE Trans. Autom. Sci. Eng., 18 (2020), 1109–1121. https://doi.org/10.1109/TASE.2020.2996160 doi: 10.1109/TASE.2020.2996160
![]() |
[18] |
A. van de Walle, F. Naets, W. Desmet, Virtual microphone sensing through vibro-acoustic modelling and Kalman filtering, Mech. Syst. Signal Process., 104 (2018), 120–133. https://doi.org/10.1016/j.ymssp.2017.08.032 doi: 10.1016/j.ymssp.2017.08.032
![]() |
[19] |
K. Maes, A. Iliopoulos, W. Weijtjens, C. Devriendt, G. Lombaert, Dynamic strain estimation for fatigue assessment of an offshore monopile wind turbine using filtering and modal expansion algorithms, Mech. Syst. Signal Process., 76 (2016), 592–611. https://doi.org/10.1016/j.ymssp.2016.01.004 doi: 10.1016/j.ymssp.2016.01.004
![]() |
[20] |
Y. H. Chang, Q. Hu, C. J. Tomlin, Secure estimation based Kalman filter for cyber-physical systems against sensor attacks, Automatica, 95 (2018), 399–412. https://doi.org/10.1016/j.automatica.2018.06.010 doi: 10.1016/j.automatica.2018.06.010
![]() |
[21] |
A. Cavallo, G. De Maria, C. Natale, S. Pirozzi, Slipping detection and avoidance based on Kalman filter, Mechatronics, 24 (2014), 489–499. https://doi.org/10.1016/j.mechatronics.2014.05.006 doi: 10.1016/j.mechatronics.2014.05.006
![]() |
[22] |
W. Gao, M. Huang, Z.P. Jiang, T. Chai, Sampled-data-based adaptive optimal output-feedback control of a 2-degree-of-freedom helicopter, IET Control Theory Appl., 10 (2016), 1440–1447. https://doi.org/10.1049/iet-cta.2015.0977 doi: 10.1049/iet-cta.2015.0977
![]() |
[23] |
J. J. Murray, C. J. Cox, G. G. Lendaris, R. Saeks, Adaptive dynamic programming, IEEE Trans. Syst. Man Cybern. Part C Appl. Rev., 32 (2002), 140–153. https://doi.org/10.1109/TSMCC.2002.801727 doi: 10.1109/TSMCC.2002.801727
![]() |
[24] | P. J. Werbos, Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences, Ph.D thesis, Harvard University, 1974. |
[25] |
W. Gao, Z. P. Jiang, Learning-based adaptive optimal tracking control of strict-feedback nonlinear systems, IEEE Trans. Neural Networks Learn. Syst., 29 (2017), 2614–2624. https://doi.org/10.1109/TNNLS.2017.2761718 doi: 10.1109/TNNLS.2017.2761718
![]() |
[26] |
T. Bian, Z.P. Jiang, Value iteration and adaptive dynamic programming for data-driven adaptive optimal control design, Automatica, 71 (2016), 348–360. https://doi.org/10.1016/j.automatica.2016.05.003 doi: 10.1016/j.automatica.2016.05.003
![]() |
[27] |
M. Roozegar, M. J. Mahjoob, M. Jahromi, Optimal motion planning and control of a nonholonomic spherical robot using dynamic programming approach: simulation and experimental results, Mechatronics, 39 (2016), 174–184. https://doi.org/10.1016/j.mechatronics.2016.05.002 doi: 10.1016/j.mechatronics.2016.05.002
![]() |
[28] | J. L. Sun, C. S. Liu, An overview on the adaptive dynamic programming based missile guidance law, Acta Autom. Sin., 43 (2017), 1101–1113. |
[29] |
Q. Hu, Robust adaptive sliding mode attitude maneuvering and vibration damping of three-axis-stabilized flexible spacecraft with actuator saturation limits, Nonlinear Dyn., 55 (2009), 301–321. https://doi.org/10.1007/s11071-008-9363-1 doi: 10.1007/s11071-008-9363-1
![]() |
[30] |
W. Gao, Y. Jiang, Z. P. Jiang, T. Chai, Output-feedback adaptive optimal control of interconnected systems based on robust adaptive dynamic programming, Automatica, 72 (2016), 37–45. https://doi.org/10.1016/j.automatica.2016.05.008 doi: 10.1016/j.automatica.2016.05.008
![]() |
[31] | K. J. Åström, Event based control, in Analysis and Design of Nonlinear Control Systems (eds. A. Astolfi, L. Marconi), Springer, Berlin, Heidelberg, (2007), 127–147. https://doi.org/10.1007/978-3-540-74358-3 |
[32] |
W. P. M. H. Heemels, M. C. F. Donkers, A. R. Teel, Periodic event-triggered control for linear systems, IEEE Trans. Autom. Control, 58 (2012), 847–861. https://doi.org/10.1109/TAC.2012.2220443 doi: 10.1109/TAC.2012.2220443
![]() |
[33] |
B. Jiang, H. R. Karimi, Y. Kao, C. Gao, Takagi-Sugeno model based event-triggered fuzzy sliding-mode control of networked control systems with semi-Markovian switchings, IEEE Trans. Fuzzy Syst., 28 (2019), 673–683. https://doi.org/10.1109/TFUZZ.2019.2914005 doi: 10.1109/TFUZZ.2019.2914005
![]() |
[34] |
T. Liu, Z. P. Jiang, A small-gain approach to robust event-triggered control of nonlinear systems, IEEE Trans. Autom. Control, 60 (2015), 2072–2085. https://doi.org/10.1109/TAC.2015.2396645 doi: 10.1109/TAC.2015.2396645
![]() |
[35] |
Y. S. Ma, W. W Che, C. Deng, Z. G. Wu, Observer-based event-triggered containment control for MASs under DoS attacks, IEEE Trans. Cybern., 52 (2021), 13156–13167. https://doi.org/10.1109/TCYB.2021.3104178 doi: 10.1109/TCYB.2021.3104178
![]() |
[36] |
X. Wang, H. R. Karimi, M. Shen, D. Liu, L. W. Li, J. Shi, Neural network-based event-triggered data-driven control of disturbed nonlinear systems with quantized input, Neural Networks, 156 (2022), 152–159. https://doi.org/10.1016/j.neunet.2022.09.021 doi: 10.1016/j.neunet.2022.09.021
![]() |
[37] |
M. Shen, Y. Gu, J. H. Park, Y. Yi, W. W. Che, Composite control of linear systems with event-triggered inputs and outputs, IEEE Trans. Circuits Syst. II Express Briefs, 69 (2021), 1154–1158. https://doi.org/10.1109/TCSII.2021.3098820 doi: 10.1109/TCSII.2021.3098820
![]() |
[38] | X. Wang, M. Shen, J. H. Park, Event-triggered control of uncertain nonlinear discrete-time systems with extended state observer, preprint. https://doi.org/10.21203/rs.3.rs-644060/v1 |
[39] |
A. Sahoo, H. Xu, S. Jagannathan, Neural network-based event-triggered state feedback control of nonlinear continuous-time systems, IEEE Trans. Neural Networks Learn. Syst., 27 (2015), 497–509. https://doi.org/10.1109/TNNLS.2015.2416259 doi: 10.1109/TNNLS.2015.2416259
![]() |
[40] | L. Ljung, System Identification: Theory for the User, 2nd edition, Prentice-Hall, Upper Saddle River, New Jersey, 1999. |
[41] | R. Pintelon, J. Schoukens, System Identification: A Frequency Domain Approach, 1st edition, John Wiley & Sons, Hoboken, New Jersey, 2012. |
[42] |
C. R. Rojas, J. C. Aguero, J. S. Welsh, G. C. Goodwin, A. Feuer, Robustness in experiment design, IEEE Trans. Autom. Control, 57 (2011), 860–874. https://doi.org/10.1109/TAC.2011.2166294 doi: 10.1109/TAC.2011.2166294
![]() |
[43] |
A. Al-Tamimi, F. L. Lewis, M. Abu-Khalaf, Model-free Q-learning designs for linear discrete-time zero-sum games with application to H-infinity control, Automatica, 43 (2007), 473–481. https://doi.org/10.1016/j.automatica.2006.09.019 doi: 10.1016/j.automatica.2006.09.019
![]() |
[44] |
F. L. Lewis, K. G. Vamvoudakis, Reinforcement learning for partially observable dynamic processes: Adaptive dynamic programming using measured output data, IEEE Trans. Syst. Man Cybern. Part B Cybern., 41 (2010), 14–25. https://doi.org/10.1109/TSMCB.2010.2043839 doi: 10.1109/TSMCB.2010.2043839
![]() |
[45] | T. Chen, B. A. Francis, Optimal Sampled-Data Control Systems, 1st edition, Springer, London, 1995. https://doi.org/10.1007/978-1-4471-3037-6 |
[46] |
G. Hewer, An iterative technique for the computation of the steady state gains for the discrete optimal regulator, IEEE Trans. Autom. Control, 16 (1971), 382–384. https://doi.org/10.1109/TAC.1971.1099755 doi: 10.1109/TAC.1971.1099755
![]() |
[47] | W. Gao, Y. Jiang, Z. P. Jiang, T. Chai, Adaptive and optimal output feedback control of linear systems: An adaptive dynamic programming approach, in Proceeding of the 11th World Congress on Intelligent Control and Automation, (2014), 2085–2090. https://doi.org/10.1109/WCICA.2014.7053043 |
[48] | W. Aangenent, D. Kostic, B. de Jager, R. van de Molengraft, M. Steinbuch, Data-based optimal control, in Proceedings of the 2005, American Control Conference, (2005), 1460–1465. https://doi.org/10.1109/ACC.2005.1470171 |
[49] | K. J. Åström, B. Wittenmark, Adaptive Control, 2nd edition, Dover Publication Inc, Mineola, New York, 2008. |
[50] | P. A. Ioannou, J. Sun, Robust Adaptive Control, 2nd edition, Dover Publication Inc, Mineola, New York, 2012. |
[51] |
K. G. Vamvoudakis, F. L. Lewis, Multi-player non-zero-sum games: Online adaptive learning solution of coupled Hamilton–Jacobi equations, Automatica, 47 (2011), 1556–1569. https://doi.org/10.1016/j.automatica.2011.03.005 doi: 10.1016/j.automatica.2011.03.005
![]() |
[52] |
H. Xu, S. Jagannathan, F. L. Lewis, Stochastic optimal control of unknown linear networked control system in the presence of random delays and packet losses, Automatica, 48 (2012), 1017–1030. https://doi.org/10.1016/j.automatica.2012.03.007 doi: 10.1016/j.automatica.2012.03.007
![]() |
1. | Ibrahim Yousif, Liam Burns, Fadi El Kalach, Ramy Harik, Leveraging computer vision towards high-efficiency autonomous industrial facilities, 2024, 0956-5515, 10.1007/s10845-024-02396-1 | |
2. | Xin Cai, Bingpeng Gao, Xinyuan Nan, A collective neurodynamic approach to distributed resource allocation with event-triggered communication, 2024, 10, 2199-4536, 5071, 10.1007/s40747-024-01436-w | |
3. | Lu Chen, Fei Hao, Optimal tracking control for unknown nonlinear systems with uncertain input saturation: A dynamic event-triggered ADP algorithm, 2024, 564, 09252312, 126964, 10.1016/j.neucom.2023.126964 | |
4. | Farhad Pourkamali-Anaraki, Jamal F. Husseini, Evan J. Pineda, Brett A. Bednarcyk, Scott E. Stapleton, Two-stage surrogate modeling for data-driven design optimization with application to composite microstructure generation, 2024, 138, 09521976, 109436, 10.1016/j.engappai.2024.109436 | |
5. | Jinan Yang, Jialei Deng, Jiahou Zhao, Sujuan Jiao, Xinhua Long, A novel parallel multi-harmonic global multi-channel control algorithm for helicopter active vibration control, 2024, 142, 09670661, 105772, 10.1016/j.conengprac.2023.105772 | |
6. | Yazhou Wang, Gang Wang, Huike Xu, Jianhui Liu, Zhen Wang, A prediction model of gear radial composite deviation based on digital twin mesh, 2025, 240, 02632241, 115619, 10.1016/j.measurement.2024.115619 | |
7. | Qingyu Shi, Xia Huang, Bo Meng, Zhen Wang, Neural network-based iterative learning control for trajectory tracking of unknown SISO nonlinear systems, 2023, 232, 09574174, 120863, 10.1016/j.eswa.2023.120863 | |
8. | Xiaogang Deng, Jiayan Li, Deep one-class classification model assisted by radius constraint for anomaly detection of industrial control systems, 2024, 138, 09521976, 109357, 10.1016/j.engappai.2024.109357 | |
9. | Mustafa Demetgul, Qi Zheng, Ibrahim Nur Tansel, Jürgen Fleischer, Monitoring the misalignment of machine tools with autoencoders after they are trained with transfer learning data, 2023, 128, 0268-3768, 3357, 10.1007/s00170-023-12060-2 | |
10. | Heng Zhao, Huanqing Wang, Xiaoheng Chang, Adil M. Ahmad, Xudong Zhao, Neural network-based adaptive critic control for saturated nonlinear systems with full state constraints via a novel event-triggered mechanism, 2024, 675, 00200255, 120756, 10.1016/j.ins.2024.120756 | |
11. | Xinyi He, Chang Liu, Xiaodi Li, A practical leader–follower hybrid control scheme for wheeled mobile robots, 2024, 184, 09600779, 114954, 10.1016/j.chaos.2024.114954 | |
12. | Shunyi Zhao, Zheng Zhou, Chengxi Zhang, Jin Wu, Fei Liu, Guangyi Shi, Localization of underground pipe jacking machinery: A reliable, real-time and robust INS/OD solution, 2023, 141, 09670661, 105711, 10.1016/j.conengprac.2023.105711 | |
13. | Zishuo Dong, Xu Li, Feng Luan, Lingming Meng, Jingguo Ding, Dianhua Zhang, Fusion of theory and data-driven model in hot plate rolling: A case study of rolling force prediction, 2024, 245, 09574174, 123047, 10.1016/j.eswa.2023.123047 | |
14. | Weizhen Wang, Xin Chen, Jiangbo Jia, Kaili Wu, Mingyang Xie, Optimal formation tracking control based on reinforcement learning for multi-UAV systems, 2023, 141, 09670661, 105735, 10.1016/j.conengprac.2023.105735 | |
15. | Maria Letizia Corradini, Resilience enhancement to loss of actuator effectiveness in a Model-Free Adaptive framework, 2024, 361, 00160032, 106957, 10.1016/j.jfranklin.2024.106957 | |
16. | Xinggui Zhao, Bo Meng, Zhen Wang, Event-triggered integral sliding mode control for uncertain networked linear control systems with quantization, 2023, 20, 1551-0018, 16705, 10.3934/mbe.2023744 | |
17. | Yu Wan, Xuehui Gao, Extended-state-observer-based output feedback control for hydraulic systems with performance constraint, 2024, 112, 0924-090X, 18333, 10.1007/s11071-024-09957-2 | |
18. | Yongwei Zhang, Shunchao Zhang, Integral sliding mode-based event-triggered optimal fault tolerant tracking control of continuous-time nonlinear systems, 2024, 79, 09473580, 101021, 10.1016/j.ejcon.2024.101021 | |
19. | Yawu Wang, Zhichao Xu, Jundong Wu, Yue Zhang, Chun-Yi Su, Modelling and model-based tracking control of soft twisted and coiled actuators, 2023, 141, 09670661, 105722, 10.1016/j.conengprac.2023.105722 | |
20. | Hao Shen, Ziwei Li, Jing Wang, Jinde Cao, Nonzero-sum games using actor-critic neural networks: A dynamic event-triggered adaptive dynamic programming, 2024, 662, 00200255, 120236, 10.1016/j.ins.2024.120236 | |
21. | Youness Boutyour, Abdellah Idrissi, Dynamic confidence-based constraint adjustment in distributional constrained policy optimization: enhancing supply chain management through adaptive reinforcement learning, 2024, 0956-5515, 10.1007/s10845-024-02492-2 | |
22. | Bo Huang, Sirui Zheng, Hamido Fujita, Jin Liu, A multi-task learning model for recommendation based on fusion of dynamic and static neighbors, 2024, 133, 09521976, 108190, 10.1016/j.engappai.2024.108190 | |
23. | Haoming Zou, Guoshan Zhang, Zhiguo Yan, Wanquan Liu, Dynamic event-triggered finite-horizon robust suboptimal control of multi-player systems with input disturbances, 2025, 611, 09252312, 128665, 10.1016/j.neucom.2024.128665 | |
24. | Héctor Escobar-Cuevas, Erik Cuevas, Jorge Gálvez, Karla Avila, A novel hybrid search strategy for evolutionary fuzzy optimization approach, 2024, 36, 0941-0643, 2633, 10.1007/s00521-023-09161-0 | |
25. | Zongsheng Huang, Xiaoyang Gao, Tieshan Li, Yue Long, Hanqing Yang, Prescribed performance event-triggered fuzzy optimal tracking control for strict-feedback nonlinear systems, 2024, 658, 00200255, 120014, 10.1016/j.ins.2023.120014 | |
26. | Huanqing Wang, Muxuan Li, Haikuo Shen, Dynamic event‐triggered adaptive neural nonsingular fixed‐time attitude control for multi‐UAVs systems, 2024, 38, 0890-6327, 3102, 10.1002/acs.3863 | |
27. | Haoran Zhang, Chunhui Zhao, Jinliang Ding, Constrained Reinforcement Learning-Based Closed-Loop Reference Model for Optimal Tracking Control of Unknown Continuous-Time Systems, 2024, 21, 1545-5955, 7312, 10.1109/TASE.2023.3340726 | |
28. | Naseem Ahmad, Shoulin Hao, Tao Liu, Yihui Gong, Qing-Guo Wang, Data-driven set-point learning control with ESO and RBFNN for nonlinear batch processes subject to nonrepetitive uncertainties, 2024, 146, 00190578, 308, 10.1016/j.isatra.2023.12.044 | |
29. | Wei Xie, Gan Yu, David Cabecinhas, Carlos Silvestre, Weidong Zhang, Wei He, Robust collision-free formation control of quadrotor fleets: Trajectory generation and tracking with experimental validation, 2024, 145, 09670661, 105842, 10.1016/j.conengprac.2024.105842 | |
30. | Paolo Righettini, Roberto Strada, Monica Tiboni, Filippo Cortinovis, Jasmine Santinelli, A systematic management and control methodology for high energy saving in applications equipped with hydraulic servo-axes, 2024, 145, 09670661, 105847, 10.1016/j.conengprac.2024.105847 | |
31. | Ting Shi, Peng Shi, Jonathon Chambers, Dynamic Event-Triggered Model Predictive Control Under Channel Fading and Denial-of-Service Attacks, 2024, 21, 1545-5955, 6448, 10.1109/TASE.2023.3325534 | |
32. | Chunping Xiong, Qian Ma, Guopeng Zhou, ADP‐based robust consensus for multi‐agent systems with unknown dynamics and random uncertain channels, 2024, 34, 1049-8923, 4051, 10.1002/rnc.7177 | |
33. | Yanzhe Wang, Qian Yang, Weiwei Qu, A collision-free transition path planning method for placement robots in complex environments, 2024, 10, 2199-4536, 8481, 10.1007/s40747-024-01585-y | |
34. | Jinggao Sun, Huiyong Lu, Huaicheng Yan, Zhicheng Kou, Dynamic event‐triggered model‐free adaptive control for networked control systems with random packet loss, 2024, 1049-8923, 10.1002/rnc.7647 | |
35. | Ziwen Gu, Yatao Shen, Zijian Wang, Jiayi Qiu, Wenmei Li, Chun Huang, Yaqun Jiang, Peng Li, Load forecasting model considering dynamic coupling relationships using structured dynamic-inner latent variables and broad learning system, 2024, 133, 09521976, 108180, 10.1016/j.engappai.2024.108180 | |
36. | Jue Wang, Huihui Pan, Weichao Sun, Event-Triggered Adaptive Output Constraint Tracking Control of Uncertain MIMO Nonlinear Systems With Sensor and Actuator Faults, 2024, 21, 1545-5955, 6774, 10.1109/TASE.2023.3330966 | |
37. | Lokesh Soni, Neeta Pandey, A Reliable and high performance Radiation Hardened Schmitt Trigger 12T SRAM cell for space applications, 2024, 176, 14348411, 155161, 10.1016/j.aeue.2024.155161 | |
38. | Yuzhu Huang, Zhaoyan Zhang, Xiong Yang, Backstepping based neural H∞ optimal tracking control for nonlinear state constrained systems with input delay and disturbances, 2024, 595, 09252312, 127869, 10.1016/j.neucom.2024.127869 | |
39. | Seyed Adel Alizadeh Kolagar, Alireza Taheri, Ali F. Meghdari, NAO Robot Learns to Interact with Humans through Imitation Learning from Video Observation, 2023, 109, 0921-0296, 10.1007/s10846-023-01938-8 | |
40. | Deshuai Zheng, Jin Yan, Tao Xue, Yong Liu, A knowledge-based task planning approach for robot multi-task manipulation, 2023, 2199-4536, 10.1007/s40747-023-01155-8 | |
41. | Nan Jiang, Qingping Xiang, Hongzhi Wang, Bo Zheng, Time series compression based on reinforcement learning, 2023, 648, 00200255, 119490, 10.1016/j.ins.2023.119490 | |
42. | Chenye Hu, Jingyao Wu, Chuang Sun, Xuefeng Chen, Ruqiang Yan, Intelligent temporal detection network for boundary-sensitive flight regime recognition, 2023, 126, 09521976, 106949, 10.1016/j.engappai.2023.106949 | |
43. | Ding Wang, Hongyu Ma, Jin Ren, Ning Gao, Junfei Qiao, Adaptive critic design with weight allocation for intelligent learning control of wastewater treatment plants, 2024, 133, 09521976, 108284, 10.1016/j.engappai.2024.108284 | |
44. | Haixiu Xie, Yuanwei Jing, Yantong Liu, Jiqing Chen, Event-based adaptive fuzzy tracking control for nonlinear systems with guaranteed performance, 2023, 643, 00200255, 119267, 10.1016/j.ins.2023.119267 | |
45. | Yuxue Li, Xiaoyuan Zhu, Guodong Yin, Robust actuator fault detection for quadrotor UAV with guaranteed sensitivity, 2023, 138, 09670661, 105588, 10.1016/j.conengprac.2023.105588 | |
46. | Shuofeng Weng, Chaochun Yuan, Youguo He, Jie Shen, Long Chen, Lizhang Xu, Zhihao Zhu, Qiuye Yu, Zeyu Sun, Neural network energy management strategy for plug-in hybrid electric combine harvesters based on quasi-periodic samples, 2024, 136, 09521976, 109051, 10.1016/j.engappai.2024.109051 | |
47. | Chunyang Sheng, Qinghui Wang, Tao Su, Haixia Wang, Induction motor torque closed-loop vector control system based on flux observation and harmonic current suppression, 2024, 142, 09670661, 105755, 10.1016/j.conengprac.2023.105755 | |
48. | Zitao Chen, Kairui Chen, Ruizhi Tang, Optimal synchronization with L2-gain performance: An adaptive dynamic programming approach, 2024, 179, 08936080, 106566, 10.1016/j.neunet.2024.106566 | |
49. | Haipeng Wang, Application of new features based on artificial intelligent robot technology in medium-scale urban design pedigree and intelligent management and control, 2024, 22, 26673053, 200379, 10.1016/j.iswa.2024.200379 | |
50. | Qingyong Yang, Shu-Chuan Chu, Jeng-Shyang Pan, Jyh-Horng Chou, Junzo Watada, Dynamic multi-strategy integrated differential evolution algorithm based on reinforcement learning for optimization problems, 2024, 10, 2199-4536, 1845, 10.1007/s40747-023-01243-9 | |
51. | Julio Yuzo Yassuda, Cristiano Marcos Agulhari, Emerson Ravazzi Pires da Silva, Sampled-data robust control of a 2-DoF helicopter modeled using a quasi-LPV framework, 2024, 145, 09670661, 105870, 10.1016/j.conengprac.2024.105870 | |
52. | Zhenguo Zhang, Tianhao Ma, Yadan Zhao, Shuai Yu, Fan Zhou, Adaptive dynamic programming-based multi-fault tolerant control of reconfigurable manipulator with input constraint, 2024, 10, 2199-4536, 8341, 10.1007/s40747-024-01550-9 | |
53. | Qing-xin Meng, Jian-wei Liu, Nonstationary online convex optimization with multiple predictions, 2024, 654, 00200255, 119862, 10.1016/j.ins.2023.119862 | |
54. | Adam Zielonka, Andrzej Sikora, Marcin Woźniak, Fuzzy rules intelligent car real-time diagnostic system, 2024, 135, 09521976, 108648, 10.1016/j.engappai.2024.108648 | |
55. | HaiTao Wang, XiangShuai Zhai, Tao Wen, ZiDu Yin, Yang Yang, Data-driven hierarchical learning approach for multi-point servo control of Pan–Tilt–Zoom cameras, 2024, 136, 09521976, 108987, 10.1016/j.engappai.2024.108987 | |
56. | Antai Li, Datong Qin, Zheng Guo, Yu Xia, Chang Lv, Wet clutch pressure hysteresis compensation control under variable oil temperatures for electro-hydraulic actuators, 2023, 141, 09670661, 105723, 10.1016/j.conengprac.2023.105723 | |
57. | Yuan Wang, Zhenbin Du, Yanming Wu, Pseudo-partial-derivative information-driven adaptive fault-tolerant tracking control for discrete-time systems, 2024, 10, 2199-4536, 2531, 10.1007/s40747-023-01280-4 | |
58. | M. Tanhaeean, S.F. Ghaderi, M. Sheikhalishahi, A decision-making framework for optimal maintenance management: An integrated simulation-mathematical programming-expert system approach, 2023, 185, 03608352, 109671, 10.1016/j.cie.2023.109671 | |
59. | Yue Wang, Yonghui Yang, Libing Wu, Adaptive fault-tolerant consensus control of multi-agent systems with event-triggered inputs, 2023, 650, 00200255, 119594, 10.1016/j.ins.2023.119594 | |
60. | Guozeng Cui, Hui Xu, Jinpeng Yu, Qian Ma, Muwei Jian, Event-Triggered Fixed-Time Adaptive Fuzzy Control for Nontriangular Nonlinear Systems With Unknown Control Directions, 2024, 5, 2691-4581, 2397, 10.1109/TAI.2023.3318895 | |
61. | Xiaolei Ji, Fei Hao, Distributed asynchronous event-triggered cooperative control for virtually coupled train set subject to gradient terrain and input saturation, 2023, 360, 00160032, 11809, 10.1016/j.jfranklin.2023.09.029 | |
62. | Ding Wang, Hongyu Ma, Junfei Qiao, Multilayer adaptive critic design with digital twin for data-driven optimal tracking control and industrial applications, 2024, 133, 09521976, 108228, 10.1016/j.engappai.2024.108228 | |
63. | Yuejie Yao, Yiping Luo, Jinde Cao, Finite-time guarantee-cost H∞ consensus control of second-order multi-agent systems based on sampled-data event-triggered mechanisms, 2024, 174, 08936080, 106261, 10.1016/j.neunet.2024.106261 | |
64. | Yu Wan, Wenlong Yue, Xuehui Gao, Qiang Chen, Ruiyin Xu, Adaptive finite-time prescribed performance tracking control for hydraulic servo systems with friction compensation, 2024, 564, 09252312, 126967, 10.1016/j.neucom.2023.126967 | |
65. | Bo Dong, Zhendong Ding, Tianjiao An, Yiming Cui, Xinye Zhu, Integral reinforcement learning-based event-triggered optimal tracking control for modular robot manipulators via non-zero-sum game, 2024, 35, 0957-0233, 096205, 10.1088/1361-6501/ad50f8 | |
66. | Wenyan Ye, Ping Zhang, Haohsuan Chang, A data-driven optimal time-delayed control approach and its application to aerial manipulators, 2024, 142, 09670661, 105754, 10.1016/j.conengprac.2023.105754 | |
67. | Onuchukwu Godwin Chike, Norhayati Ahmad, Wan Fahmin Faiz Wan Ali, Neural network prediction of thermal field spatiotemporal evolution during additive manufacturing: an overview, 2024, 134, 0268-3768, 2107, 10.1007/s00170-024-14256-6 | |
68. | Quangui He, Wei Liu, Formation control for linear multi-agent systems with asynchronously sampled outputs, 2024, 658, 00200255, 119992, 10.1016/j.ins.2023.119992 | |
69. | Amir Veisi, Hadi Delavari, Deep reinforcement learning optimizer based novel Caputo fractional order sliding mode data driven controller, 2025, 140, 09521976, 109725, 10.1016/j.engappai.2024.109725 | |
70. | Shanke Li, Kun Peng, Fei Hui, Ziqi Li, Cheng Wei, Wenbo Wang, A Decision-Making Approach for Complex Unsignalized Intersection by Deep Reinforcement Learning, 2024, 73, 0018-9545, 16134, 10.1109/TVT.2024.3408917 | |
71. | Chenhang Yan, Liping Yan, Yuezu Lv, Yuanqing Xia, Dynamic Event-Triggered Byzantine-Resilient Output Regulation in Continuous-Time High-Order Multiagent Systems With Static/Dynamic Leader, 2024, 71, 1549-8328, 5532, 10.1109/TCSI.2024.3381191 | |
72. | Jianying Li, Hailong Yang, Hui Ji, Characterization of Two-Cylinder Parallel Electro-hydraulic Force/Position Synchronization Based on RBF Fuzzy Neural Network Control, 2024, 1562-2479, 10.1007/s40815-024-01846-5 | |
73. | Xiong Yang, Qinglai Wei, Adaptive Dynamic Programming for Robust Event-Driven Tracking Control of Nonlinear Systems With Asymmetric Input Constraints, 2024, 54, 2168-2267, 6333, 10.1109/TCYB.2024.3418904 | |
74. | Chengli Fan, Chunyi Xiao, Tao Xu, Dengxiu Yu, C.L. Philip Chen, Decentralized event-trigger-based predefined-time adaptive control of large-scale nonlinear systems, 2024, 00160032, 107446, 10.1016/j.jfranklin.2024.107446 | |
75. | Weiqi Liu, Shuai Sui, C.L. Philip Chen, Event-triggered predefined-time output feedback fuzzy adaptive control of permanent magnet synchronous motor systems, 2025, 142, 09521976, 109882, 10.1016/j.engappai.2024.109882 | |
76. | A. Aziz Khater, Mohamed Fekry, Mohammad El-Bardini, Ahmad M. El-Nagar, Deep reinforcement learning-based adaptive fuzzy control for electro-hydraulic servo system, 2025, 0941-0643, 10.1007/s00521-024-10741-x | |
77. | Wenhui Dou, Shihong Ding, Ju H. Park, Keqi Mei, An Adaptive Generalized Super-Twisting Algorithm via Event-Triggered Control, 2025, 22, 1545-5955, 393, 10.1109/TASE.2024.3351122 | |
78. | Bo Dong, Yuhang Gao, Tianjiao An, Hucheng Jiang, Bing Ma, Nonzero-sum game-based decentralized approximate optimal control of modular robot manipulators with coordinate operation tasks using value iteration, 2025, 36, 0957-0233, 026209, 10.1088/1361-6501/ad880d | |
79. | Feng Qin, Azlan Mohd Zain, Kai-Qing Zhou, Norfadzlan Bin Yusup, Didik Dwi Prasetya, Rozita Abdul Jalil, Zaheera Zainal Abidin, Mahadi Bahari, Yusri Kamin, Mazlina Abdul Majid, Hybrid Harmony Search Algorithm Integrating Differential Evolution and Lévy Flight for Engineering Optimization, 2025, 13, 2169-3536, 13534, 10.1109/ACCESS.2025.3529714 | |
80. | Suhuan Zhang, Fanglai Zhu, Xufeng Ling, Event-triggered UIO-based security control for discrete-time systems under deception attacks, 2025, 00200255, 121902, 10.1016/j.ins.2025.121902 | |
81. | Yuebing Wen, Shuhua Teng, Qiang Li, Jianping Tan, Yuwei Song, Shiyuan Sun, Investigating the Symmetric Control of a Hydraulic System Based on Status Feedback, 2025, 17, 2073-8994, 246, 10.3390/sym17020246 | |
82. | Guhui Li, Zidong Wang, Xingzhen Bai, Zhongyi Zhao, Hongli Dong, Event-Triggered Set-Membership Filtering for Active Power Distribution Systems Under Fading Channels: A Zonotope-Based Approach, 2025, 22, 1545-5955, 1139, 10.1109/TASE.2024.3360600 | |
83. | Cong Guan, Tao Jiang, Yi-Chen Li, Zongzhang Zhang, Lei Yuan, Yang Yu, Constraining an Unconstrained Multi-agent Policy with offline data, 2025, 08936080, 107253, 10.1016/j.neunet.2025.107253 | |
84. | Zhikai Yao, Xianglong Liang, Shuping Wang, Jianyong Yao, Model-Data Hybrid Driven Control of Hydraulic Euler–Lagrange Systems, 2025, 30, 1083-4435, 131, 10.1109/TMECH.2024.3390129 | |
85. | Giseung Park, Whiyoung Jung, Seungyul Han, Sungho Choi, Youngchul Sung, Adaptive multi-model fusion learning for sparse-reward reinforcement learning, 2025, 09252312, 129748, 10.1016/j.neucom.2025.129748 |
Notations | Descriptions |
xv | The spool valve displacement |
pa, pb | Forward and return pressure |
qa, qb | Forward and return flows |
y | Piston displacement |
L | Piston stroke |
Ke | Load spring gradient |
pS, p0 | Supply and tank pressure |
mt, mp, m | total mass, piston mass, payload mass |
Ff | Friction force |
Fext | Disturbance force |
Aa, Ab | Effective areas of the head and rod piston side |
Va, Vb, Va0, Vb0 | Fluid volumes of the head and rod piston side and corresponding initial volumes |
qLi, qLe | Internal and external leakage flow |
βe | Bulk modulus of the fluid |
Notations | Descriptions |
xv | The spool valve displacement |
pa, pb | Forward and return pressure |
qa, qb | Forward and return flows |
y | Piston displacement |
L | Piston stroke |
Ke | Load spring gradient |
pS, p0 | Supply and tank pressure |
mt, mp, m | total mass, piston mass, payload mass |
Ff | Friction force |
Fext | Disturbance force |
Aa, Ab | Effective areas of the head and rod piston side |
Va, Vb, Va0, Vb0 | Fluid volumes of the head and rod piston side and corresponding initial volumes |
qLi, qLe | Internal and external leakage flow |
βe | Bulk modulus of the fluid |