Value functions in a regime switching jump diffusion with delay market model

1.   Introduction

The classical Merton portfolio optimization problem, in its basic form, involves an investor who is limited to investing to two types of assets-risky and non-risky. An investor's goal is to determine the optimal allocation strategy such that the wealth-dependent performance criterion is a maximum. Under certain conditions, Merton found out that the optimal strategy is to keep a constant fraction of the wealth in the risky asset ^[1].

Merton's portfolio optimization problem belongs to a wider set of stochastic optimal control problems which are known to be generally hard to solve and explicit solutions are rare. Despite this, several authors have improved on Merton's work by relaxing particular assumptions of the original paper. For instance, Davis and Norman ^[8] incorporated proportional transaction costs and successfully obtained optimal solutions from their formulation. In addition, they found out that the solution space can be divided into three regions-no transaction, sell, and buy. Framstad, Oksendal and Sulem ^[2] studied the case when the risky asset is a jump-diffusion in a portfolio problem with transaction costs. Pagliarani and Vargiolu ^[12] examined the case when risky assets are defaultable Levy processes.

Meanwhile, the series of works involving Pang (^[6,7,14]) considered portfolio problems that incorporate the path history of the risky asset. These type of problems are called stochastic systems with delay or memory. The rationale for considering delay stems from the tendency of market participants to look at the past performance of assets and decide based on these information. While the concept of stochastic systems with delay or memory seems to run counter to the Markovian nature of dynamic programming, Larssen ^[4] provided the settings where dynamic programming still applies.

Bauerle and Rieder ^[11] considered a market model and portfolio problem where the drift and volatility of the price process are driven by a continuous time Markov chain. The work of Valdez and Vargiolu ^[13] provided a framework for dealing with a portfolio problem involving multidimensional risky assets which are diffusions with switching coefficients. Azevedo et al. ^[9] considered Markov switching jump-diffusions, thereby expanding the framework into discontinuous settings. The main motivation for incorporating regime switching in portfolio optimization problems is to take into account the effect of market regimes or states of the economy in the dynamics of asset prices.

Several tweaks and modifications have been made to the original Merton portfolio probem in order to capture market realities. However, to the best of our knowledge, there has been no study, in the context of the classical Merton portfolio optimization problem, that considered systems with delay and regime switching in discontinuous models. This is due to the fact that the memory or delay variable, as a time integral, becomes undefined in the face of jumps or discontinuities. Hence, this paper aims to supplement this gap and provides a way to overcome the problem of the delay variable breaking down in discontinuous settings.

2.   Problem formulation

Let $T > 0$ be finite. Let $(\Omega, \mathbb{F}, \mathbb{P})$ be a complete filtered probability space where the filtration $\mathbb{F} = \{\mathcal{F}_{t}:t\in[0, T]\}$ satisfies all the usual conditions.

To model market regimes, we set $\{\alpha_{t}:t\in[0, T]\}$ to be a continuous time Markov chain defined on a finite state space $\mathcal{M} = \{a_{1}, a_{2}, \cdots, a_{n}\}$ with generator $Q = (q_{ij})_{i, j\in\mathcal{M}}$. Let

be a counting process that counts the number of jumps of the Markov chain $\alpha_{t}$ from state $i$ to state $j$ up to time $t$. Corresponding to this counting process is the intensity process

such that the purely discontinuous, square-integrable process

is a martingale.

As in Larssen and Risebro ^[5] or in the case of Pang and Hussain ^[7], we define the delay variable, which is a form of sliding average of past values, to be

where $\rho\in\mathbb{R}$ and $\chi_{t}$ is the continuous approximation of the path of the risky asset until time $t$. With this definition (2.1) is defined even when the risky asset has discontinuities. The motivation for considering (2.1) in our market model is to account for the tendency of market participants to look at past performances of stocks before making investment decisions which in turn affect stock prices ^[7].

The investment opportunities in our model are the non-risky asset $P_{t}$ that follows

and the risky asset $X_{t}$ which is assumed to be a Levy process given by

where $r:[0, T]\times\mathbb{R}\times\mathcal{M}\to\mathbb{R}$, $\mu:[0, T]\times\mathbb{R}\times\mathcal{M}\to\mathbb{R}$, $\sigma:[0, T]\times\mathbb{R}\times\mathcal{M}\to\mathbb{R}$ and $\Gamma:[0, T]\times\mathbb{R}\times\mathcal{M}\times\mathbb{R}\to\mathbb{R}$ are uniformly continuous functions representing the risk-free rate, drift, volatility, and jump coefficient, respectively, where $z = \Delta X_{t} = X_{t}-X_{t-}$ is the jump size of the risky asset. $N(t, A)$ is the compensated Poisson random measure given by

where $\overline{N}(t, A)$ is the Poisson random measure that counts the number of jumps of the risky asset up to time $t$ and $\nu(A)$ is the Levy measure for each $A\in\mathcal{B}_{0}$, where $\mathcal{B}_{0}$ is a Borel $\sigma$-field generated by the open subsets $\mathcal{O}$ of $\mathbb{R}_{0} = \mathbb{R}\setminus\{0\}$ whose closure does not contain the zero element.

We assume that the Brownian motion $B_{t}$, the Markov chain $\alpha_{t}$ and the compensated Poisson random measure $N(t, A)$ are all independent and adapted to the filtration $\mathbb{F}$.

We also assume that

for every $(y, a)\in \mathbb{R}\times\mathcal{M}$.

Let $\pi_{t}$ be $\mathbb{F}$-progressively measurable and that for a fixed $T > 0$,

This $\pi_{t}$ represents the proportion of wealth $W_{t}$ invested by an agent in the risky asset while the balance $1-\pi_{t}$ is allocated to the non-risky asset. A self-financing portfolio resulting from these investments evolves according to

where

The corresponding state variable is the wealth process

We define the performance criterion to be

and the value function

where $U(\cdot)$ is a utility function, $\mathbb{E}_{s, w, y, a}[\cdot]$ is the conditional expectation conditioned on the initial data $(s, w, y, a)$, and $\mathcal{A}[s, T]$ is the set of admissible controls such that (2.6) has a unique strong solution $W_{t}^{\pi; s, w, y, a}$ for every $t\in[s, T]$.

Lemma 2.1. Let $Y_{t}$ be defined as in (2.1). Then

Proof: The proof follows ^[7]. We have

Thus, the delay satisfies

The next result is the equivalent of Ito's Lemma for the particular portfolio optimization problem.

Lemma 2.2. Let $V(s, w, y, a)$ such that $V(\cdot, \cdot, \cdot, a)\in\mathcal{C}^{1, 2, 1}\big([0, T]\times\mathbb{R}^{+}\times\mathbb{R}\big)$ for every $a\in\mathcal{M}$. Then, we have

where

and

Proof: Applying the change of variable rules (see ^[18,9]) and using Lemma (2.1) for the delay, we get

Combining all terms with $ds$, we get

and setting the rest as

and

Thus,

Integrating over $[0, T]$, we obtain

3.   Dynamic programming

We employ dynamic programming to solve the particular portfolio optimization problem.

Theorem 3.1 (Dynamic programming principle). Assuming that the value function as given by (2.8) is continuous over the space of controls $\mathcal{A}$ with the state variable (2.6), then for any $(s, w, y, a)\in [0, T]\times\mathbb{R}^{+}\times\mathbb{R}\times\mathcal{M}$, we have that

for all $s+h\in[s, T]$.

Proof: For any $(s, w, y, a)\in [0, T)\times\mathbb{R}^{+}\times\mathbb{R}\times\mathcal{M}$ and any arbitrary admissible control $\pi$, we have

Taking supremum over admissible controls, we obtain

For the other direction, we follow Cartea et al. ^[23] by considering an $\epsilon$-optimal control. Let $\epsilon > 0$ and take an admissible control $\pi^{\epsilon}\in\mathcal{A}[s, T]$ such that

This is guaranteed because the value function is continuous over $\mathcal{A}[s, T]$. Next, we consider the modification of the $\epsilon$-optimal control

Then we have

Taking limits as $\epsilon \to 0$,

Taking supremum over admissible controls,

Since both inequalities are true for any arbitrary admissible control $\pi$, we conclude that

for all $s+h\in[s, T]$.

Theorem 3.2 (Hamilton-Jacobi-Bellman equation). Assume that $V(\cdot, \cdot, \cdot, \alpha)\in\mathcal{C}^{1, 2, 1}\big([0, T]\times\mathbb{R}^{+}\times\mathbb{R}\big)$ for every $\alpha\in\mathcal{M}$. Then for each $\alpha\in\mathcal{M}$, the value function $V(\cdot, \cdot, \cdot, \alpha)$ defined on $[0, T]\times\mathbb{R}^{+}\times\mathbb{R}$ is the solution to the Hamilton-Jacobi-Bellman (HJB) equation

where the Hamiltonian function is defined to be

Proof: From the dynamic progamming principle (Theorem 3.1), we have that for $s+h\in[s, T]$,

Using Lemma 2.2, we have for $t\in[s, s+h]$

From the assumption on $M^{\alpha j}_{t}$ and from Davis ^[24] and Fleming and Soner ^[25] for $B_{t}$ and $N(t, z)$, it follows that

We get,

Dividing by $h$, taking limits as $h\to 0$, and by Mean Value Theorem,

We conclude that

Taking supremum over all admissible controls, we get

Theorem 3.3 (Verification Theorem). Let $V(\cdot, \cdot, \cdot, \alpha)\in\mathcal{C}^{1, 2, 1}\big([0, T]\times\mathbb{R}^{+}\times\mathbb{R}\big)$ for every $\alpha\in\mathcal{M}$. If $V(s, w, y, a)$ is the solution to the HJB Eq (3.2), then

holds for every $\pi \in \mathcal{A}[s, T]$ and $(s, w, y, a)\in [0, T]\times\mathbb{R}^{+}\times\mathbb{R}\times\mathcal{M}$. Moreover, $\pi^{*}\in\mathcal{A}[s, T]$ is optimal if and only if

for a.e. $t\in [s, T]$.

Proof: For any $\pi \in \mathcal{A}[s, T]$ and by Lemma 2.2, we have that

By Theorem 3.2, the integral of the last line is at most zero for any $\pi \in \mathcal{A}[s, T]$. It follows that

For the second part, we assume that $\pi^{*}\in\mathcal{A}[s, T]$ is optimal. Then from the last equality of (3.4),

Again, using the fact that the integral is at most zero by Theorem 3.2, it follows that

4.   Value functions

Before taking on a particular utility function, we consider at time $s$ a portion of the Hamiltonian $\mathcal{H}$ that involves wealth $w$ and control $\pi_{s}$,

4.1.   Power utility

Theorem 4.1. If the utility function is $U(w) = \frac{w^\gamma_{a}}{\gamma_{a}}$, $0 < \gamma_{a} < 1$, then the value function is

where $\zeta_{a}(s, y)$ is the solution to the boundary-value problem

with

Proof: With this value function (4.1) becomes

The HJB Eq (3.2) now becomes

Since $w^{\gamma_{a}} > 0$, Eq (4.5) further becomes

By applying the first order condition on $F(\pi_{s}; s, y, a)$, we have

Solving for $\pi_{s}$, we get

Since for every $(s, y, a)\in[0, T]\times\mathbb{R}\times\mathcal{M}$,

it follows that (4.7) is a maximum and $F(\pi^{*}; s, y, a)$ is optimal. Thus, (4.6) becomes

Note that (4.8) is a first order linear partial differential equation. Hence, it is solvable and solutions exist. To obtain a unique solution for (4.8) we impose the boundary condition

and also to be consistent with (3.2).

4.2.   Logarithmic utility

Theorem 4.2. If the utility function is $U(w) = \ln(w)$, then the value function is

where

and $\zeta_{a}(s)$ is the solution to the coupled ordinary differential equation terminal-value problem:

with

Proof: With this value function (4.1) becomes

And the HJB Eq (3.2) now reads

We split (4.14) into

and

For $\ln w\neq 0$, (4.15) becomes

Since the delay is a function of time, we integrate (4.17) with respect to $t$ to obtain

We impose the terminal condition $\xi_{a}(T) = \frac{1}{2} = \varrho(Y_{T})$ so that (4.10) remains consistent with (3.2). Hence (4.18) becomes

which can be expressed as

and which can be substituted into (4.16).

Similarly, by applying the first order condition on $F(\pi_{s}; s, y, a)$, we have

Solving for $\pi_{s}$, we get

Now, for every $(s, y, a)\in[0, T]\times\mathbb{R}\times\mathcal{M}$,

It follows that (4.21) is a maximum and $F(\pi^{*}; s, y, a)$ is optimal. Thus (4.16) becomes

Since both the Markov chain $\alpha_{t}$ and delay $Y_{t}$ are dependent on the time variable, (4.22) is a first order linear ordinary differential equation. Thus, solutions exist for (4.22). We impose the terminal condition

in order for (4.22) to have a unique solution and to be consistent with (3.2).

5.   Conclusions

We considered a portfolio optimization problem where the riskless asset and the coefficients of the risky asset, represented by a Levy price process, depends on time $t$, delay $Y_{t}$ and Markov chain $\alpha_{t}$. We formulated a finite time horizon Merton-type optimization problem and came up with a version of the stochastic chain rule for the system we are working. This chain rule serves as the main machinery in solving the optimization problem which generates a Hamilton-Jacobi-Bellman (HJB) equation bearing the aforementioned variables. In the world of dynamic programming involving portfolio optimization, the HJB equation we came up with is novel in the sense that it incorporated both the delay and regime switching, a result deemed unlikely at first given the discontinuous nature of Levy processes. This obstacle was overcame by fixing the delay variable in order for it to be defined even in stochastic systems with jumps.

The main results of this paper are the optimal portfolio $\pi^{*}$, which we obtained for logarithmic and power utility functions, and the value functions which represent the solution to the portfolio optimization problem. The optimal portfolio was found by invoking the first order condition on the HJB equation. We found that for a logarithmic utility function, the solution consists of four functions, three of which are interrelated via a coupled differential equation. The value function is simpler for the case of a power utility in the sense that the solution is represented by the product of wealth and a function which solves a first order linear partial differential equation.

Conflict of interest

The authors declare that they have no competing interests.