Fault-tolerant control of a hydraulic servo actuator via adaptive dynamic programming

Vladimir Stojanovic; Vladimir Stojanovic

doi:10.3934/mmc.2023016

Mathematical Modelling and Control

2023, Volume 3, Issue 3: 181-191. doi: 10.3934/mmc.2023016

Previous Article Next Article

Research article

Fault-tolerant control of a hydraulic servo actuator via adaptive dynamic programming

Vladimir Stojanovic ^,

Faculty of Mechanical and Civil Engineering, University of Kragujevac, Kraljevo, 36000, Serbia

Received: 25 July 2023 Revised: 25 July 2023 Accepted: 09 August 2023 Published: 01 September 2023

The fault-tolerant control problem of a hydraulic servo actuator in the presence of actuator faults is studied utilizing adaptive dynamic programming. This task is challenging because of unknown system dynamics, uncertain disturbances or unmeasurable system states of such highly nonlinear systems in real applications. The aim is to achieve asymptotic tracking and actuator faults compensation by minimizing some predefined performance index. The discrete-time algebraic Riccati equation is iteratively solved by the adaptive dynamic programming approach. For practical reasons, adaptive dynamic programming techniques and fault compensation are integrated to iteratively compute an approximated optimal fault-tolerant control using real-time input/output data without any a priori knowledge of the system dynamics and unmeasurable states. As a result, a fault-tolerant control of hydraulic servo actuator is then designed based on adaptive dynamic programming via output feedback. Also, the convergence analysis of a data-driven fault-tolerant control is theoretically shown as well. Finally, intensive simulation results are given to prove the validity and merits of the developed data-driven fault-tolerant control strategy.
- adaptive dynamic programming,
- fault-tolerant control,
- hydraulic servo actuator,
- unknown dynamics,
- actuator faults
Citation: Vladimir Stojanovic. Fault-tolerant control of a hydraulic servo actuator via adaptive dynamic programming[J]. Mathematical Modelling and Control, 2023, 3(3): 181-191. doi: 10.3934/mmc.2023016

Related Papers:

Abstract

The fault-tolerant control problem of a hydraulic servo actuator in the presence of actuator faults is studied utilizing adaptive dynamic programming. This task is challenging because of unknown system dynamics, uncertain disturbances or unmeasurable system states of such highly nonlinear systems in real applications. The aim is to achieve asymptotic tracking and actuator faults compensation by minimizing some predefined performance index. The discrete-time algebraic Riccati equation is iteratively solved by the adaptive dynamic programming approach. For practical reasons, adaptive dynamic programming techniques and fault compensation are integrated to iteratively compute an approximated optimal fault-tolerant control using real-time input/output data without any a priori knowledge of the system dynamics and unmeasurable states. As a result, a fault-tolerant control of hydraulic servo actuator is then designed based on adaptive dynamic programming via output feedback. Also, the convergence analysis of a data-driven fault-tolerant control is theoretically shown as well. Finally, intensive simulation results are given to prove the validity and merits of the developed data-driven fault-tolerant control strategy.

References

[1]	P. Werbos, Handbook of intelligent control: Neural, fuzzy, and adaptive approaches, In: White DA, Sofge DA, editors. Approximate dynamic programming for real-time control and neural modeling, New York: Van Nostrand Reinhold, 1992,493–525.
[2]	Q. Wei, D. Liu, H. Lin, Value iteration adaptive dynamic programming for optimal control of discrete-time nonlinear systems, IEEE Trans. Cybern., 46 (2016), 840–853. https://doi.org/10.1109/TCYB.2015.2492242 doi: 10.1109/TCYB.2015.2492242
[3]	X. Yang, D. Liu, Q. Wei, D. Wang, Guaranteed cost neural tracking control for a class of uncertain nonlinear systems using adaptive dynamic programming, Neurocomputing, 198 (2016), 80–90. https://doi.org/10.1016/j.neucom.2015.08.119 doi: 10.1016/j.neucom.2015.08.119
[4]	L. Dong, X. Zhong, C. Sun, H. He, Adaptive event-triggered control based on heuristic dynamic programming for nonlinear discrete-time systems, IEEE Trans. Neural Networks Learn. Syst., 28 (2016), 1594–1605. https://doi.org/10.1109/TNNLS.2016.2541020 doi: 10.1109/TNNLS.2016.2541020
[5]	Q. Sun, C. Zhang, N. Jiang, J. Yu, L. Xu, Data-driven nonlinear near-optimal regulation based on multi-dimensional Taylor network dynamic programming, IEEE Access, 8 (2020), 36476–36484. https://doi.org/10.1109/ACCESS.2020.2975391 doi: 10.1109/ACCESS.2020.2975391
[6]	Y. Jiang, Z. Jiang, Robust adaptive dynamic programming for large-scale systems with an application to multimachine power systems, IEEE T. CIRCUITS-II, 59 (2012), 693–697. https://doi.org/10.1109/TCSII.2012.2213353 doi: 10.1109/TCSII.2012.2213353
[7]	B. Zhao, D. Liu, C. Luo, Reinforcement learning-based optimal stabilization for unknown nonlinear systems subject to inputs with uncertain constraints, IEEE Trans. Neural Networks Learn. Syst., 31 (2020), 4330–4340. https://doi.org/10.1109/TNNLS.2019.2954983 doi: 10.1109/TNNLS.2019.2954983
[8]	D. Wang, D. Liu, H. Li, B. Luo, H. Ma, An approximate optimal control approach for robust stabilization of a class of discrete-time nonlinear systems with uncertainties, IEEE Trans. Syst. Man Cybern. Syst., 46 (2016), 713–717. https://doi.org/10.1109/TSMC.2015.2466191 doi: 10.1109/TSMC.2015.2466191
[9]	X. Yang, D. Liu, D. Wang, Reinforcement learning for adaptive optimal control of unknown continuous-time nonlinear systems with input constraints, Int. J. Control, 87 (2014), 553–566. https://doi.org/10.1080/00207179.2013.848292 doi: 10.1080/00207179.2013.848292
[10]	H. Xia, B. Zhao, F. Zhou, B. Dong, G. Liu, Y. Li, Fault tolerant control for reconfigurable manipulators based on adaptive dynamic programming with an improved performance index function, In: 2016 12th World Congress on Intelligent Control and Automation (WCICA), Guilin, China. New Jersey: IEEE; (2016), 2273–2278. https://doi.org/10.1109/WCICA.2016.7578344
[11]	X. Xiao, X. Li, Adaptive dynamic programming method-based synchronisation control of a class of complex dynamical networks with unknown dynamics and actuator faults, IET Control Theory Appl., 12 (2018), 291–298. https://doi.org/ 10.1049/iet-cta.2017.0845 doi: 10.1049/iet-cta.2017.0845
[12]	C. Chen, K. Xie, F.L. Lewis, S. Xie, A. Davoudi, Fully distributed resilience for adaptive exponential synchronization of heterogeneous multiagent systems against actuator faults, IEEE Trans. Automat. Contr., 64 (2019), 3347–3354. https://doi.org/10.1109/TAC.2018.2881148 doi: 10.1109/TAC.2018.2881148
[13]	C. Xie, G. Yang, Approximate guaranteed cost fault-tolerant control of unknown nonlinear systems with time-varying actuator faults, Nonlinear Dyn., 83 (2016), 269–282. https://doi.org/10.1007/s11071-015-2324-6 doi: 10.1007/s11071-015-2324-6
[14]	G. Dong, H. Li, H. Ma, R. Lu, Finite-time consensus tracking neural network FTC of multi-agent systems, IEEE Trans. Neural Networks Learn. Syst., 32 (2021), 653–662. https://doi.org/10.1109/TNNLS.2020.2978898 doi: 10.1109/TNNLS.2020.2978898
[15]	M. Benosman, K.Y. Lum, Passive actuators' fault-tolerant control for affine nonlinear systems, IEEE Trans. Control Syst. Technol., 18 (2010), 152–163. https://doi.org/10.1109/TCST.2008.2009641 doi: 10.1109/TCST.2008.2009641
[16]	V. Djordjevic, V. Stojanovic, D. Prsic, L. J. Dubonjic, M. M. Morato, Observer-based fault estimation in steer-by-wire vehicle, Engineering Today, 1 (2022), 7–17. https://doi.org/10.5937/engtoday2201007D doi: 10.5937/engtoday2201007D
[17]	W. Guo, A. Qiu, C. Wen, Active actuator fault tolerant control based on generalized internal model control and performance compensation, IEEE Access, 7 (2019), 175514–175521. https://doi.org/10.1109/ACCESS.2019.2957824 doi: 10.1109/ACCESS.2019.2957824
[18]	Y. Fu, T. Chai, Nonlinear Adaptive Decoupling Control Based on Neural Networks and Multiple Models, International Journal of Innovative Computing, Information and Control, 8 (2012), 1867–1878.
[19]	N. Meskin, E. Naderi, K. Khorasani, A multiple model-based approach for fault diagnosis of jet engines, IEEE Trans. Control Syst. Technol., 21 (2011), 254–262. https://doi.org/10.1109/TCST.2011.2177981 doi: 10.1109/TCST.2011.2177981
[20]	Z. Abbasfard, A. Baniamerian, K. Khorasani, Fault diagnosis of gas turbine engines: A symbolic multiple model approach, In: European Control Conference 2014, Strasbourg, France. New Jersey: IEEE; 2014,944–951. https://doi.org/10.1109/ECC.2014.6862358
[21]	D. Liu, D. Wang, D. Zhao, Adaptive dynamic programming for optimal control of unknown nonlinear discrete-time systems, In: IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning 2011, Paris, France. New Jersey: IEEE; 2011,242–249. https://doi.org/10.1109/ADPRL.2011.5967357
[22]	J. J. Murray, C. J. Cox, G. G. Lendaris, R. Saeks, Adaptive dynamic programming, IEEE Trans. Syst. Man Cybern. Syst., 32 (2002), 140–153. https://doi.org/10.1109/TSMCC.2002.801727 doi: 10.1109/TSMCC.2002.801727
[23]	F. Wang, H. Zhang, D. Liu, Adaptive dynamic programming: An introduction, IEEE Comput. Intell. Mag., 4 (2009), 39–47. https://doi.org/10.1109/MCI.2009.932261 doi: 10.1109/MCI.2009.932261
[24]	Z. Jiang, Y. Jiang, Robust adaptive dynamic programming: recent results and applications, In: Chinese Control Conference 2013: Proceedings of the 32nd Chinese Control Conference, Xi'an, China. New Jersey: IEEE; 2013,968–973.
[25]	Q. Fan, G. Yang, Adaptive fault-tolerant control for affine non-linear systems based on approximate dynamic programming, IET Control Theory Appl., 10 (2016), 655–663. https://doi.org/10.1049/iet-cta.2015.1081 doi: 10.1049/iet-cta.2015.1081
[26]	D. Ye, T. Song, Decentralized reliable guaranteed cost control for large-scale nonlinear systems using actor-critic network, Neurocomputing, 320 (2018), 121–128. https://doi.org/10.1016/j.neucom.2018.09.029 doi: 10.1016/j.neucom.2018.09.029
[27]	B. Zhao, D. Liu, Y. Li, Observer based adaptive dynamic programming for fault tolerant control of a class of nonlinear systems, INFORM. SCIENCES, 384 (2017), 21–33. https://doi.org/10.1016/j.ins.2016.12.016 doi: 10.1016/j.ins.2016.12.016
[28]	Y. Li, K. Sun, S. Tong, Observer-based adaptive fuzzy fault-tolerant optimal control for SISO nonlinear systems, IEEE Trans. Cybern., 49 (2019), 649–661. https://doi.org/ 10.1109/TCYB.2017.2785801 doi: 10.1109/TCYB.2017.2785801
[29]	V. Stojanovic, N. Nedic, D. Prsic, L. J. Dubonjic, V. Djordjevic, Application of cuckoo search algorithm to constrained control problem of a parallel robot platform, Int. J. Adv. Manuf. Technol., 87 (2016), 2497–2507. https://doi.org/10.1007/s00170-016-8627-z doi: 10.1007/s00170-016-8627-z
[30]	M. Jelali, A. Kroll, Hydraulic servo-systems: Modelling, identification and control, 1st ed., London: Springer, 2002. https://doi.org/10.1007/978-1-4471-0099-7
[31]	F. L. Lewis, D. Liu, Reinforcement learning and approximate dynamic programming for feedback control, 1st ed., New Jersey: John Wiley & Sons, 2013.
[32]	P. J. Werbos, Beyond regression: New tools for prediction and analysis in the behavioral sciences, PhD thesis: Harvard University; 1974.
[33]	W. Gao, Z. Jiang, Learning-based adaptive optimal tracking control of strict-feedback nonlinear systems, IEEE Trans. Neural Networks Learn. Syst., 29 (2018), 2614–2624. https://doi.org/10.1109/TNNLS.2017.2761718 doi: 10.1109/TNNLS.2017.2761718
[34]	T. Bian, Z. Jiang, Value iteration and adaptive dynamic programming for data-driven adaptive optimal control design, AUTOMATICA, 71 (2016), 348–360. https://doi.org/10.1016/j.automatica.2016.05.003 doi: 10.1016/j.automatica.2016.05.003
[35]	M. Tomás-Rodríguez, S. P. Banks, Linear, time-varying approximations to nonlinear dynamical systems: with applications in control and optimization, 1st ed., London: Springer, 2010. https://doi.org/10.1007/978-1-84996-101-1
[36]	V. Stojanovic, D. Prsic, Robust identification for fault detection in the presence of non-Gaussian noises: Application to hydraulic servo drives, Nonlinear Dyn., 100 (2020), 2299–2313. https://doi.org/10.1007/s11071-020-05616-4 doi: 10.1007/s11071-020-05616-4
[37]	C. R. Rojas, J. C. Aguero, J. S. Welsh, G. C. Goodwin, A. Feuer, Robustness in experiment design, IEEE Trans. Automat. Contr., 57 (2011), 860–874. https://doi.org/10.1109/TAC.2011.2166294 doi: 10.1109/TAC.2011.2166294
[38]	F. L. Lewis, D. Vrabie, V. L. Syrmos, Optimal control, 3rd ed. New Jersey: John Wiley & Sons; 2012.
[39]	T. Chen, B. A. Francis, Optimal sampled-data control systems, 1st ed. London: Springer; 2012.
[40]	G. Hewer, An iterative technique for the computation of the steady state gains for the discrete optimal regulator, IEEE Trans. Automat. Contr., 16 (1971), 382–384. https://doi.org/10.1109/TAC.1971.1099755 doi: 10.1109/TAC.1971.1099755
[41]	F. L. Lewis, K. G. Vamvoudakis, Reinforcement learning for partially observable dynamic processes: Adaptive dynamic programming using measured output data, IEEE Trans. Syst. Man Cybern. Syst., 41 (2011), 14–25. https://doi.org/10.1109/TSMCB.2010.2043839 doi: 10.1109/TSMCB.2010.2043839
[42]	W. Gao, Y. Jiang, Z. Jiang, T. Chai, Adaptive and optimal output feedback control of linear systems: An adaptive dynamic programming approach, In: World Congress on Intelligent Control and Automation 2014: Proceeding of the 11th World Congress on Intelligent Control and Automation; 2014 Jun 29- Jul 4; Shenyang, China. New Jersey: IEEE; 2014. pp. 2085–2090. https://doi.org/10.1109/WCICA.2014.7053043
[43]	W. Aangenent, D. Kostic, B. de Jager, R. VandeMolengraft, M. Steinbuch, Data-based optimal control, In: Foster JA, Lutton E, Miller J, Ryan C, Tettamanzi AG, editors. American Control Conference 2005: Proceedings of the 2005, American Control Conference, 2005; 2005 Apr 3-5; City, Country. Berlin: Publisher; 2005. pp. 1460–1465. https://doi.org/10.1109/ACC.2005.1470171
[44]	K. J. Åström, B. Wittenmark, Adaptive control, 2nd ed. New York: Dover Publications, 2008.
[45]	K. G. Vamvoudakis, F. L. Lewis, Multi-player non-zero-sum games: Online adaptive learning solution of coupled Hamilton–Jacobi equations, AUTOMATICA, 47 (2011), 1556–1569. https://doi.org/10.1016/j.automatica.2011.03.005 doi: 10.1016/j.automatica.2011.03.005
[46]	H. Xu, S. Jagannathan, F. L. Lewis, Stochastic optimal control of unknown linear networked control system in the presence of random delays and packet losses, AUTOMATICA, 48 (2012), 1017–1030. https://doi.org/10.1016/j.automatica.2012.03.007 doi: 10.1016/j.automatica.2012.03.007

Reader Comments

Your name:*

Email:*
© 2023 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)