Filtering of hidden Markov renewal processes by continuous and counting observations

Andrey Borisov; Andrey Borisov

doi:10.3934/math.20241453

AIMS Mathematics

2024, Volume 9, Issue 11: 30073-30099. doi: 10.3934/math.20241453

Previous Article Next Article

Research article Special Issues

Filtering of hidden Markov renewal processes by continuous and counting observations

Andrey Borisov ^,

Federal Research Center "Computer Science and Control" of the Russian Academy of Sciences, 44/2 Vavilova Str., 119333 Moscow, Russia

Received: 27 August 2024 Revised: 08 October 2024 Accepted: 14 October 2024 Published: 23 October 2024
MSC : 60G35, 62M05, 94A40

This paper introduces a subclass of Markov renewal processes (MRPs) and presents a solution to the optimal filtering problem in a stochastic observation system, where the state is modeled by an MRP and observed indirectly through noisy measurements. The MRPs considered here can be interpreted as continuous-time Markov chains (CTMCs) with a finite set of abstract states representing distributions of random vectors. The paper outlines the probabilistic properties of MRPs, emphasizing the ability to express any arbitrary function of the MRP as the solution to a linear stochastic differential system (SDS) with a martingale on the right-hand side (RHS). Using these properties, an optimal filtering problem is formulated in stochastic observation systems, where the hidden state belongs to the class of MRPs, and the observations consist of both diffusion and counting components. The drift terms in all observations depend on the system state. An optimal filtering estimate for a scalar function of the MRP is provided through the solution of an SDS with innovation processes on the RHS. Additionally, the paper presents a version of the Kushner-Stratonovich equation, describing the evolution of the conditional probability density function (PDF). To demonstrate the practical application of the estimation method, the paper presents a communications-related example, focusing on monitoring the qualitative state and numerical characteristics of a network channel using noisy observations of round-trip time (RTT) and packet loss flow. The paper also highlights the robustness of the filtering algorithm in scenarios where the MRP distribution is uncertain.
- hidden Markov model,
- Markov renewal process,
- martingale representation,
- optimal filtering problem,
- Kushner-Stratonovich equation
Citation: Andrey Borisov. Filtering of hidden Markov renewal processes by continuous and counting observations[J]. AIMS Mathematics, 2024, 9(11): 30073-30099. doi: 10.3934/math.20241453

Related Papers:

Abstract

This paper introduces a subclass of Markov renewal processes (MRPs) and presents a solution to the optimal filtering problem in a stochastic observation system, where the state is modeled by an MRP and observed indirectly through noisy measurements. The MRPs considered here can be interpreted as continuous-time Markov chains (CTMCs) with a finite set of abstract states representing distributions of random vectors. The paper outlines the probabilistic properties of MRPs, emphasizing the ability to express any arbitrary function of the MRP as the solution to a linear stochastic differential system (SDS) with a martingale on the right-hand side (RHS). Using these properties, an optimal filtering problem is formulated in stochastic observation systems, where the hidden state belongs to the class of MRPs, and the observations consist of both diffusion and counting components. The drift terms in all observations depend on the system state. An optimal filtering estimate for a scalar function of the MRP is provided through the solution of an SDS with innovation processes on the RHS. Additionally, the paper presents a version of the Kushner-Stratonovich equation, describing the evolution of the conditional probability density function (PDF). To demonstrate the practical application of the estimation method, the paper presents a communications-related example, focusing on monitoring the qualitative state and numerical characteristics of a network channel using noisy observations of round-trip time (RTT) and packet loss flow. The paper also highlights the robustness of the filtering algorithm in scenarios where the MRP distribution is uncertain.

References

[1]	E. B. Dynkin, Markov processes, Heidelberg: Springer, 1 (2012). https://doi.org/10.1007/978-3-662-00031-1
[2]	S. Ethier, T. Kurtz, Markov processes: Characterization and convergence, John Wiley & Sons, 2009.
[3]	D. White, A survey of applications of Markov decision processes, J. Oper. Res. Soc., 44 (2014), 1073–1096. https://doi.org/10.1057/jors.1993.181 doi: 10.1057/jors.1993.181
[4]	J. Jacod, Multivariate point processes: Predictable projection, Radon-Nikodym derivatives, representation of martingales, Z. Wahrscheinlichkeitstheorie Verw. Gebiete, 31 (1975), 235–253. https://doi.org/10.1007/BF00536010 doi: 10.1007/BF00536010
[5]	N. Limnios, G. Oprişan, Semi-Markov processes and reliability, Boston: Birkhäuser, 2001. https://doi.org/10.1007/978-1-4612-0161-8
[6]	C. Cocozza-Thivent, Markov renewal and piecewise deterministic processes, Springer Cham, 2021. https://doi.org/10.1007/978-3-030-70447-6
[7]	Y. Ishikawa, H. Kunita, Malliavin calculus on the Wiener-Poisson space and its application to canonical SDE with jumps, Stoch. Process. Appl., 116 (2006), 1743–1769. https://doi.org/10.1016/j.spa.2006.04.013 doi: 10.1016/j.spa.2006.04.013
[8]	A. Borisov, Analysis and estimation of the states of special Markov jump processes. I. Martingale Representation, Autom. Remote Control, 65 (2004), 44–57. https://doi.org/10.1023/B:AURC.0000011689.11915.24 doi: 10.1023/B:AURC.0000011689.11915.24
[9]	Y. Bar-Shalom, X. R. Li, T. Kirubarajan, Estimation with applications to tracking and navigation: Theory algorithms and software, John Wiley & Sons, 2002. https://doi.org/10.1002/0471221279
[10]	X. R. Li, V. P. Jilkov, Survey of maneuvering target tracking. Part V. Multiple-model methods, IEEE Trans. Aerosp. Electron. Syst., 41 (2005), 1255–1321. https://doi.org/10.1109/TAES.2005.1561886 doi: 10.1109/TAES.2005.1561886
[11]	D. Delahaye, S. Puechmorel, P. Tsiotras, E. Feron, Mathematical models for aircraft trajectory design: A survey, In: Air traffic management and systems. Lecture notes in electrical engineering, Tokyo: Springer, 290 (2014), 205–247. https://doi.org/10.1007/978-4-431-54475-3_12
[12]	J. Lan, X. R. Li, V. P. Jilkov, C. Mu, Second-order Markov chain based multiple-model algorithm for maneuvering target tracking, IEEE Trans. Aerosp. Electron. Syst., 49 (2013), 3–19. https://doi.org/10.1109/TAES.2013.6404088 doi: 10.1109/TAES.2013.6404088
[13]	Y. Shen, T. K. Siu, Asset allocation under stochastic interest rate with regime switching, Econom. Model., 29 (2012), 1126–1136. https://doi.org/10.1016/j.econmod.2012.03.024 doi: 10.1016/j.econmod.2012.03.024
[14]	S. Goutte, Pricing and hedging in stochastic volatility regime switching models, J. Math. Finance 3 (2013), 70–80. http://dx.doi.org/10.4236/jmf.2013.31006 doi: 10.4236/jmf.2013.31006
[15]	J. D. Hamilton, Macroeconomic regimes and regime shifts, In: Handbook of macroeconomics, Elsevier, 2 (2016), 163–201. https://doi.org/10.1016/bs.hesmac.2016.03.004
[16]	R. Sueppel, Classifying market regimes, 2021. Available from: https://research.macrosynergy.com/classifying-market-regimes/#classifying-market-regimes.
[17]	X. Zhang, R. J. Elliott, T. K. Siu, J. Guo, Markovian regime-switching market completion using additional Markov jump assets, IMA J. Manag. Math., 23 (2012), 283–305. https://doi.org/10.1093/imaman/dpr018 doi: 10.1093/imaman/dpr018
[18]	M. A. Kouritzin, Sampling and filtering with Markov chains, Signal Process., 225 (2024), 109613. https://doi.org/10.1016/j.sigpro.2024.109613 doi: 10.1016/j.sigpro.2024.109613
[19]	R. Sh. Liptser, A. N. Shiryayev, Theory of martingales, Dordrecht: Springer, 1989. https://doi.org/10.1007/978-94-009-2438-3
[20]	R. W. Brockett, Nonlinear systems and nonlinear estimation theory, In: Stochastic systems: The mathematics of filtering and identification and applications, 78 (1981), 441–477. https://doi.org/10.1007/978-94-009-8546-9_23
[21]	V. E. Beneš, Exact finite-dimensional filters for certain diffusions with nonlinear drift, Stochastics, 5 (1981), 65–92. https://doi.org/10.1080/17442508108833174 doi: 10.1080/17442508108833174
[22]	M. Hazewinkel, S. I. Marcus, H. J. Sussmann, Nonexistence of finite-dimensional filters for conditional statistics of the cubic sensor problem, Syst. Control Lett., 3 (1983), 331–340. https://doi.org/10.1016/0167-6911(83)90074-9 doi: 10.1016/0167-6911(83)90074-9
[23]	T. Björk, Finite optimal filters for a class of nonlinear diffusions with jumping parameters, Stochastics, 6 (1982), 121–138. https://doi.org/10.1080/17442508208833198 doi: 10.1080/17442508208833198
[24]	S. Tang, Brockett's problem of classification of finite-dimensional estimation algebras for nonlinear filtering systems, SIAM J. Control Optim., 39 (2000), 900–916. https://doi.org/10.1137/S036301299833464X doi: 10.1137/S036301299833464X
[25]	W. Dong, J. Shi, A survey of estimation algebras in application of nonlinear filtering problems, Commun. Inf. Syst., 19 (2019), 193–217. https://dx.doi.org/10.4310/CIS.2019.v19.n2.a4 doi: 10.4310/CIS.2019.v19.n2.a4
[26]	H. J. Kushner, On the differential equations satisfied by conditional probability densities of Markov processes, with applications, J. Soc. Indust. Appl. Math. Ser. A, 2 (1962), 106–119. https://doi.org/10.1137/0302009 doi: 10.1137/0302009
[27]	A. V. Borisov, Numerical schemes of Markov jump process filtering given discretized observations I: Accuracy characteristics, Inform. Appl., 13 (2019), 68–75. https://doi.org/10.14357/19922264190411 doi: 10.14357/19922264190411
[28]	A. V. Borisov, Numerical schemes of Markov jump process filtering given discretized observations Ⅱ: Additive noises case, Inform. Appl., 14 (2020), 17–23. https://doi.org/10.14357/19922264200103 doi: 10.14357/19922264200103
[29]	E. Platen, N. Bruti-Liberati, Numerical solution of stochastic differential equations with jumps in finance, Berlin Heidelberg: Springer-Verlag, 2010. https://doi.org/10.1007/978-3-642-13694-8
[30]	A. V. Borisov, The Wonham filter under uncertainty: A game-theoretic approach, Automatica, 47 (2011), 1015–1019. https://doi.org/10.1016/j.automatica.2011.01.056 doi: 10.1016/j.automatica.2011.01.056
[31]	A. Borisov, I. Sokolov, Optimal filtering of Markov jump processes given observations with state-dependent noises: Exact solution and stable numerical schemes, Mathematics, 8 (2020), 506. https://doi.org/10.3390/math8040506 doi: 10.3390/math8040506
[32]	E. Wong, B. Hajek, Stochastic processes in engineering systems, New York: Springer, 1985. https://doi.org/10.1007/978-1-4612-5060-9
[33]	A. Borisov, A. Gorshenin, Identification of continuous-discrete hidden Markov models with multiplicative observation noise, Mathematics, 10 (2022), 3062. https://doi.org/10.3390/math10173062 doi: 10.3390/math10173062
[34]	P. Cheng, S. He, V, Stojanovic, X. Luan, F. Liu, Fuzzy fault detection for Markov jump systems with partly accessible hidden information: An event-triggered approach, IEEE Trans. Cybernet., 52 (2022), 7352–7361. https://doi.org/10.1109/TCYB.2021.3050209 doi: 10.1109/TCYB.2021.3050209
[35]	X. Zhang, H. Wang, V. Stojanovic, P. Cheng, S. He, X. Luan, Asynchronous fault detection for interval type-2 fuzzy nonhomogeneous higher level Markov jump systems with uncertain transition probabilities, IEEE Trans. Fuzzy Syst., 30 (2022), 2487–2499. https://doi.org/10.1109/TFUZZ.2021.3086224 doi: 10.1109/TFUZZ.2021.3086224
[36]	R. S. Mamon, R. J. Elliott, Hidden Markov models in finance, New York: Springer, 2014. https://doi.org/10.1007/978-1-4899-7442-6
[37]	D. J. Wilkinson, Stochastic modelling for systems biology, 2nd Eds., Boca Raton: CRC Press, 2011. https://doi.org/10.1201/b11812
[38]	A. Bureau, S. Shiboski, J. P. Hughes, Applications of continuous time hidden Markov models to the study of misclassified disease outcomes, Stat. Med., 22 (2003), 441–462. https://doi.org/10.1002/sim.1270 doi: 10.1002/sim.1270
[39]	S. N. Cohen, R. J. Elliott, Stochastic calculus and applications, New York: Birkhäuser, 2015. https://doi.org/10.1007/978-1-4939-2867-5
[40]	F. Klebaner, R. Liptser, When a stochastic exponential is a true Martingale. Extension of the Beneš method, Theory Probab. Appl., 58 (2014), 38–62. https://doi.org/10.1137/S0040585X97986382 doi: 10.1137/S0040585X97986382
[41]	R. S. Liptser, A. N. Shiryayev, Statistics of random processes: I. General theory, Heidelberg: Springer-Verlag Berlin, 2001. https://doi.org/10.1007/978-3-662-13043-8

Reader Comments

Your name:*

Email:*
© 2024 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)