An optimal control problem without control costs

Mario Lefebvre; Mario Lefebvre

doi:10.3934/mbe.2023239

Mathematical Biosciences and Engineering

2023, Volume 20, Issue 3: 5159-5168. doi: 10.3934/mbe.2023239

Previous Article Next Article

Research article Special Issues

An optimal control problem without control costs

Mario Lefebvre ^,

Department of Mathematics and Industrial Engineering, Polytechnique Montréal, C.P. 6079, Succursale Centre-ville, Montréal, H3C 3A7, Canada

Academic Editor: Jesús Martín Vaquero

Received: 22 November 2022 Revised: 28 December 2022 Accepted: 31 December 2022 Published: 09 January 2023

A two-dimensional diffusion process is controlled until it enters a given subset of $ \mathbb{R}^2 $. The aim is to find the control that minimizes the expected value of a cost function in which there are no control costs. The optimal control can be expressed in terms of the value function, which gives the smallest value that the expected cost can take. To obtain the value function, one can make use of dynamic programming to find the differential equation it satisfies. This differential equation is a non-linear second-order partial differential equation. We find explicit solutions to this non-linear equation, subject to the appropriate boundary conditions, in important particular cases. The method of similarity solutions is used.
- stochastic optimal control,
- diffusion processes,
- first-passage time,
- dynamic programming,
- partial differential equation
Citation: Mario Lefebvre. An optimal control problem without control costs[J]. Mathematical Biosciences and Engineering, 2023, 20(3): 5159-5168. doi: 10.3934/mbe.2023239

Related Papers:

Abstract

A two-dimensional diffusion process is controlled until it enters a given subset of $ \mathbb{R}^2 $. The aim is to find the control that minimizes the expected value of a cost function in which there are no control costs. The optimal control can be expressed in terms of the value function, which gives the smallest value that the expected cost can take. To obtain the value function, one can make use of dynamic programming to find the differential equation it satisfies. This differential equation is a non-linear second-order partial differential equation. We find explicit solutions to this non-linear equation, subject to the appropriate boundary conditions, in important particular cases. The method of similarity solutions is used.

References

[1]	P. Whittle, Optimization Over Time, Vol. I, Wiley, Chichester, 1982.
[2]	P. Whittle, Risk-Sensitive Optimal Control, Wiley, Chichester, 1990.
[3]	M. Kounta, N. J. Dawson, Linear quadratic Gaussian homing for Markov processes with regime switching and applications to controlled population growth/decay, Methodol. Comput. Appl. Probab., 23 (2021), 1155–1172. https://doi.org/10.1007/s11009-020-09800-2 doi: 10.1007/s11009-020-09800-2
[4]	C. Makasu, Homing problems with control in the diffusion coefficient, IEEE Trans. Autom. Control, 67 (2022), 3770–3772. https://doi.org/10.1109/TAC.2022.3157077 doi: 10.1109/TAC.2022.3157077
[5]	M. Lefebvre, Minimizing or maximizing the first-passage time to a time-dependent boundary, Optimization, 71 (2022), 387–401. https://doi.org/10.1080/02331934.2021.1914039 doi: 10.1080/02331934.2021.1914039
[6]	M. Lefebvre, M. Kounta, Discrete homing problems, Arch. Control Sci., 23 (2013), 5–18. https://doi.org/10.2478/v10170-011-0039-6 doi: 10.2478/v10170-011-0039-6
[7]	M. Lefebvre, A. Moutassim, Exact solutions to the homing problem for a Wiener process with jumps, Optimization, 70 (2021), 307–319. https://doi.org/10.1080/02331934.2019.1711084 doi: 10.1080/02331934.2019.1711084
[8]	M. Lefebvre, The homing problem for autoregressive processes, IMA J. Math. Control Inf., 39 (2022), 322–344. https://doi.org/10.1093/imamci/dnab047 doi: 10.1093/imamci/dnab047
[9]	Z. Yang, H. K. Koo, Optimal consumption and portfolio selection with early retirement option, Math. Oper. Res., 43 (2018), 1378–1404. https://doi.org/10.1287/moor.2017.0909 doi: 10.1287/moor.2017.0909
[10]	N. Rodosthenous, H. Zhang, Beating the omega clock: an optimal stopping problem with random time-horizon under spectrally negative Lévy models, Ann. Appl. Probab., 28 (2018), 2105–2140. https://doi.org/10.1214/17-AAP1322 doi: 10.1214/17-AAP1322
[11]	W. Y. Yun, C. H. Choi, Optimum replacement intervals with random time horizon, J. Qual. Maint. Eng., 6 (2000), 269–274. https://doi.org/10.1108/13552510010346798 doi: 10.1108/13552510010346798
[12]	A. Khatab, N. Rezg, D. Ait-Kadi, Optimum block replacement policy over a random time horizon, J. Intell. Manuf., 22 (2011), 885–889. https://doi.org/10.1007/s10845-009-0364-9 doi: 10.1007/s10845-009-0364-9
[13]	Z. Yu, Continuous-time mean-variance portfolio selection with random horizon, Appl. Math. Optim., 68 (2013). https://doi.org/10.1007/S00245-013-9209-1
[14]	M. Lefebvre, A stochastic model for computer virus propagation, J. Dyn. Games, 7 (2020), 163–174. https://doi.org/10.3934/jdg.2020010 doi: 10.3934/jdg.2020010
[15]	J. Marín-Solano, E. V. Shevkoplyas, Non-constant discounting and differential games with random time horizon, Automatica, 47 (2011), 2626–2638. https://doi.org/10.1016/j.automatica.2011.09.010 doi: 10.1016/j.automatica.2011.09.010
[16]	A. Zaremba, E. Gromova, A. Tur, A differential game with random time horizon and discontinuous distribution, Mathematics, 8 (2020). https://doi.org/10.3390/math8122185

Reader Comments

Your name:*

Email:*
© 2023 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)