In this paper, optimal control of an inverted pendulum on a cart system is studied. Since the nonlinear structure of the system is complex, and in the presence of random disturbances, optimization and control of the motion of the system become more challenging. For handling this system, a discrete-time stochastic optimal control problem for the system is described, where the external force is considered as the control input. By defining a loss function, namely, the mean squared errors to be minimized, the stochastic approximation (SA) approach is applied to estimate the state dynamics. In addition, the Hamiltonian function is defined, and the first-order necessary conditions are derived. The gradient of the cost function is determined so that the SA approach is employed to update the control sequences. For illustration, considering the values of the related parameters in the system, the discrete-time stochastic optimal control problem is solved iteratively by using the SA algorithm. The simulation results show that the state estimation and the optimal control law design are well performed with the SA algorithm, and the motion of the inverted pendulum cart is addressed satisfactorily. In conclusion, the efficiency of the SA approach for solving the inverted pendulum on a cart system is verified.
Citation: Xian Wen Sim, Sie Long Kek, Sy Yi Sim. State estimation and optimal control of an inverted pendulum on a cart system with stochastic approximation approach[J]. Applied Computing and Intelligence, 2023, 3(1): 79-92. doi: 10.3934/aci.2023005
In this paper, optimal control of an inverted pendulum on a cart system is studied. Since the nonlinear structure of the system is complex, and in the presence of random disturbances, optimization and control of the motion of the system become more challenging. For handling this system, a discrete-time stochastic optimal control problem for the system is described, where the external force is considered as the control input. By defining a loss function, namely, the mean squared errors to be minimized, the stochastic approximation (SA) approach is applied to estimate the state dynamics. In addition, the Hamiltonian function is defined, and the first-order necessary conditions are derived. The gradient of the cost function is determined so that the SA approach is employed to update the control sequences. For illustration, considering the values of the related parameters in the system, the discrete-time stochastic optimal control problem is solved iteratively by using the SA algorithm. The simulation results show that the state estimation and the optimal control law design are well performed with the SA algorithm, and the motion of the inverted pendulum cart is addressed satisfactorily. In conclusion, the efficiency of the SA approach for solving the inverted pendulum on a cart system is verified.
| [1] |
A. Ghanbari, M. Farrokhi, Decentralized neuro-fuzzy controller design using decoupled sliding-mode structure for two-dimensional inverted pendulum, 2006 IEEE International Conference on Engineering of Intelligent Systems, (2006), 1‒6. https://doi.org/10.1109/ICEIS.2006.1703155 doi: 10.1109/ICEIS.2006.1703155
|
| [2] | P. Kumar, K. Chakraborty, R. R. Mukherjee, S. Mukherjee, Modelling and controller design of inverted pendulum, International Journal on Advanced Research in Computer Engineering & Technology (IJARCET), 2 (2013), 200‒206. |
| [3] |
O. Boubaker, The inverted pendulum: a fundamental benchmark in control theory and robotics, Journal of Electronic Systems, 2 (2012), 154‒164. https://doi.org/10.1109/ICEELI.2012.6360606 doi: 10.1109/ICEELI.2012.6360606
|
| [4] | S. M. Azimi, H. Miar-Naimi, Designing an analog CMOS fuzzy logic controller for the inverted pendulum with a novel triangular membership function, Scientia Iranica D, 26 (2019), 1736‒1748. |
| [5] |
S. Hasnain, K. D. Kallu, M. H. Nawaz, N. Abbas, C. I. Pruncu, Dynamic response of an inverted pendulum system in water under parametric excitations for energy harvesting: a conceptual approach, Energies, 13 (2020), 5215. https://doi.org/10.3390/en13195215 doi: 10.3390/en13195215
|
| [6] |
S. J. Mohammadi Doulabi Fard, S. Jafari, Fuzzy controller structures investigation for future gas turbine aero-engines, International Journal of Turbomachinery Propulsion and Power, 6 (2021), 1‒23. https://doi.org/10.3390/ijtpp6010002 doi: 10.3390/ijtpp6010002
|
| [7] |
R. Eide, P. M. Egelid, A. Stamsø, H. R. Karimi, LQG control design for balancing an inverted pendulum mobile robot, Intelligent Control and Automation, 2 (2011), 160‒166. https://doi.org/10.4236/ica.2011.22019 doi: 10.4236/ica.2011.22019
|
| [8] | A. N. K. Nasir, M. A. Ahmad, M. F. A. Rahmat, Performance comparison between LQR and PID controllers for an inverted pendulum system, AIP Conference Proceedings, American Institute of Physics, 1052 (2008), 124‒128. https://doi.org/10.1063/1.3008655 |
| [9] | S. Dhobaley, P. Bhopale, A. Pandey, LMI based control for balancing an inverted pendulum mobile robot, International Journal of Engineering Research, 3 (2014), 2023‒2026. |
| [10] | J. C. Spall, Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control, Hoboken: John Wiley & Sons, 2003. https://doi.org/10.1002/0471722138 |
| [11] |
L. B. Prasad, B. Tyagi, H. O. Gupta, Optimal control of nonlinear inverted pendulum system using PID controller and LQR: performance analysis without and with disturbance input, Int. J. Autom. Comput., 11 (2014), 661‒670. https://doi.org/10.1007/s11633-014-0818-1 doi: 10.1007/s11633-014-0818-1
|
| [12] |
K. György, The LQG control algorithms for nonlinear dynamic systems, Procedia Manufacturing, 32 (2019), 553‒563. https://doi.org/10.1016/j.promfg.2019.02.252 doi: 10.1016/j.promfg.2019.02.252
|
| [13] | A. E. Bryson, Y. C. Ho, Applied Optimal Control. Hemisphere, Washington, DC, USA, 1975. |
| [14] | E. K. P. Chong, S. H. Zak, An Introduction to Optimization. 4th Ed. John Wiley & Sons, Inc., Hoboken, New Jersey, 2013. |
| [15] | D. E. Kirk, Optimal Control Theory: An Introduction. Mineola, NY: Dover Publications, New York, 2004. |
| [16] | F. L. Lewis, V. Vrabie, V. L. Symos, Optimal Control. 3rd Ed. John Wiley & Sons, Inc., New York, 2012. |
| [17] | K. L. Teo, B. Li, C. Yu, V. Rehbock, Applied and Computational Optimal Control: A Control Parametrization Approach, Springer, 2021. |
Figures(5) / Tables(3)
Xian Wen Sim, Sie Long Kek, Sy Yi Sim. State estimation and optimal control of an inverted pendulum on a cart system with stochastic approximation approach[J]. Applied Computing and Intelligence, 2023, 3(1): 79-92. doi: 10.3934/aci.2023005
DownLoad: