A feedback design for numerical solution to optimal control problems based on Hamilton-Jacobi-Bellman equation

Zhen-Zhen Tao; Bing Sun; Zhen-Zhen Tao; Bing Sun

doi:10.3934/era.2021046

Electronic Research Archive

2021, Volume 29, Issue 5: 3429-3447. doi: 10.3934/era.2021046

Previous Article Next Article

Special Issues

A feedback design for numerical solution to optimal control problems based on Hamilton-Jacobi-Bellman equation

Zhen-Zhen Tao ^{1
,},
Bing Sun ^{1,2
,
,}

1.
School of Mathematics and Statistics, Beijing Institute of Technology, Beijing 100081, China
2.
Beijing Key Laboratory on MCAACI, Beijing Institute of Technology, Beijing 100081, China

Received: 01 November 2020 Revised: 01 May 2021 Published: 24 June 2021
49L25, 49M30, 49N35, 65D05

In this paper, we present a feedback design for numerical solution to optimal control problems, which is based on solving the corresponding Hamilton-Jacobi-Bellman (HJB) equation. An upwind finite-difference scheme is adopted to solve the HJB equation under the framework of the dynamic programming viscosity solution (DPVS) approach. Different from the usual existing algorithms, the numerical control function is interpolated in turn to gain the approximation of optimal feedback control-trajectory pair. Five simulations are executed and both of them, without exception, output the accurate numerical results. The design can avoid solving the HJB equation repeatedly, thus efficaciously promote the computation efficiency and save memory.
- Dynamic programming,
- viscosity solution,
- interpolation,
- optimal feedback control,
- upwind finite difference scheme,
- Hamilton-Jacobi-Bellman equation
Citation: Zhen-Zhen Tao, Bing Sun. A feedback design for numerical solution to optimal control problems based on Hamilton-Jacobi-Bellman equation[J]. Electronic Research Archive, 2021, 29(5): 3429-3447. doi: 10.3934/era.2021046

Related Papers:

Abstract

In this paper, we present a feedback design for numerical solution to optimal control problems, which is based on solving the corresponding Hamilton-Jacobi-Bellman (HJB) equation. An upwind finite-difference scheme is adopted to solve the HJB equation under the framework of the dynamic programming viscosity solution (DPVS) approach. Different from the usual existing algorithms, the numerical control function is interpolated in turn to gain the approximation of optimal feedback control-trajectory pair. Five simulations are executed and both of them, without exception, output the accurate numerical results. The design can avoid solving the HJB equation repeatedly, thus efficaciously promote the computation efficiency and save memory.

References

[1]	A. Alla, Model Reduction for A Dynamic Programming Approach to Optimal Control Problems with PDE Constraints, Ph.D thesis, University of Rome in Sapienza, Italy, 2014.
[2]	A. Alla, B. Haasdonk and A. Schmidt, Feedback control of parametrized PDEs via model order reduction and dynamic programming principle, Adv. Comput. Math., 46 (2020), Paper No. 9, 28 pp. doi: 10.1007/s10444-020-09744-8
[3]	M. Bardi and I. Capuzzo-Dolcetta, Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations, Systems & Control: Foundations & Applications, with appendices by Maurizio Falcone and Pierpaolo Soravia, Birkhäuser, Boston, 1997., doi: 10.1007/978-0-8176-4755-1
[4]	Control of Kalman-like filters using impulse and continuous feedback design. Discrete Contin. Dyn. Syst. Ser. B (2002) 2: 169-184.
[5]	Feedback stabilization of the three-dimensional Navier-Stokes equations using generalized Lyapunov equations. Discrete Contin. Dyn. Syst. (2020) 40: 4197-4229.
[6]	On a parabolic Hamilton-Jacobi-Bellman equation degenerating at the boundary. Commun. Pure Appl. Anal. (2016) 15: 1251-1263.
[7]	M. G. Crandall, Viscosity solutions: A primer, in Lecture Notes in Mathematics (eds. I. Capuzzo-Dolcetta and P. L. Lions), Springer-Verlag, Berlin, (1997), 1–43.
[8]	Some properties of viscosity solutions of Hamilton-Jacobi equations. Trans. Amer. Math. Soc. (1984) 282: 487-502.
[9]	Viscosity solutions of Hamilton-Jacobi equations. Trans. Amer. Math. Soc. (1983) 277: 1-42.
[10]	G. Fabrini, M. Falcone and S. Volkwein, Coupling MPC and HJB for the computation of POD-based Feedback Laws, Numerical Mathematics and Advanced Applications—ENUMATH, (2017), 941–949, Lect. Notes Comput. Sci. Eng., 126, Springer, Cham, 2019.
[11]	M. Falcone and R. Ferretti, Semi-Lagrangian Approximation Schemes for Linear and Hamilton-Jacobi Equations, SIAM, Philadelphia, 2014.
[12]	W. H. Fleming and H. M. Soner, Controlled Markov Processes and Viscosity Solutions, Stochastic Modelling and Applied Probability, Springer-Verlag, New York, 2006.
[13]	S. Gombao, Approximation of optimal controls for semilinear parabolic PDE by solving Hamilton-Jacobi-Bellman equations, in Electronic Proceedings of Fifteenth International Symposium on Mathematical Theory of Networks and Systems (eds. D. S. Gilliam and J. Rosenthal), South Bend, USA, (2002), 1–15.
[14]	Dynamic programming approach to the numerical solution of optimal control with paradigm by a mathematical model for drug therapies of HIV/AIDS. Optim. Eng. (2004) 15: 119-136.
[15]	Numerical solution to the optimal feedback control of continuous casting process. J. Global Optim. (2007) 39: 171-195.
[16]	On centralized optimal control. IEEE Trans. Automat. Control (2005) 50: 537-538.
[17]	Polynomial approximation of high-dimensional Hamilton-Jacobi-Bellman equations and applications to feedback control of semilinear parabolic PDEs. SIAM J. Sci. Comput. (2018) 40: 629-652.
[18]	HJB-POD-based feedback design for the optimal control of evolution problems. SIAM J. Appl. Dyn. Syst. (2004) 3: 701-722.
[19]	POD-based feedback control of the Burgers equation to solving the evolutionary HJB equation. Comput. Math. Appl. (2005) 49: 1113-1126.
[20]	K. Kunisch and L. Xie, Suboptimal feedback control of flow separation by POD model reduction, in Real-Time PDE-Constrained Optimization, (eds. L. T. Biegler, O. Ghattas, M. Heinkenschloss, D. Keyes and B. van Bloemen Waanders), Computational Science & Engineering, SIAM, (2007), 233–250. doi: 10.1137/1.9780898718935.ch12
[21]	H. J. Kushner and P. Dupuis, Numerical Methods for Stochastic Control Problems in Continuous Time, Stochastic Modelling and Applied Probability, Springer-Verlag, New York, 2001. doi: 10.1007/978-1-4613-0007-6
[22]	Convergence rate for a curse-of-dimensionality-free method for Hamilton-Jacobi-Bellman PDEs represented as maxima of quadratic forms. SIAM J. Control Optim. (2009) 48: 2651-2685.
[23]	L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkrelidze and E. F. Mishchenko, The Mathematical Theory of Optimal Processes, Interscience Publishers John Wiley & Sons, Inc., New York, 1962.
[24]	(1986) Control and Optimization: The Linear Treatment of Nonlinear Problems (Nonlinear Science: Theory and Applications). Manchester: Manchester University Press.
[25]	T. Sauer, Numerical Analysis, 2$^nd$ edition, Pearson Education, Essex, 2012.
[26]	A TB-HIV/AIDS coinfection model and optimal control treatment. Discrete Contin. Dyn. Syst. (2015) 35: 4639-4663.
[27]	J. Stoer and R. Bulirsch, Introduction to Numerical Analysis, Texts in Applied Mathematics, Vol. 12, Springer-Verlag, New York, 2002. doi: 10.1007/978-0-387-21738-3
[28]	Convergence of an upwind finite-difference scheme for Hamilton-Jacobi-Bellman equation in optimal control. IEEE Trans. Automat. Control (2015) 60: 3012-3017.
[29]	An upwind finite-difference method for the approximation of viscosity solutions to Hamilton-Jacobi-Bellman equations. IMA J. Math. Control Inform. (2000) 17: 167-178.
[30]	(2006) A Concise Lecture Note on Optimal Control Theory. Beijing: Higher Education Press.
[31]	Verification theorems within the framework of viscosity solutions. J. Math. Anal. Appl. (1993) 177: 208-225.

Reader Comments

Your name:*

Email:*
© 2021 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)