Evolutionary game analysis on the recycling strategy of household medical device enterprises under government dynamic rewards and punishments

1.   Introduction

The susceptible-infective-removed-susceptible (SIRS) model is one of the most popular epidemic models, in which total host population is divide into three classes called $susceptible\; (S)$, $infective\; (I)$ and $removed\; (R)$. Since the Kermack and McKendrick pioneering work ^[1], a large number of studies have been carried out on the model of continuous infectious diseases attempting to gain a better understanding of diseases transmission, especially for the control policies and dynamics ^[2,3,4,5]. Recent years, many researchers have studied the epidemic with SIRS models such as COVID-19(2019), hand-foot-and-mouth diseases(HFMD) and other infectious diseases recently. Although scholars have flocked to study many properties of the SIRS models, there are still many aspects that deserve further study, such as the threshold dynamics of SIRS model with vaccination and optimal control using dynamic programming method.

In studying the dynamics of the disease, one of the most important concepts is the basic reproduction number $R_0$. It is epidemiologically defined as the expected number of secondary cases produced by a typical infectious individual during its entire infectious period in a completely susceptible host population. To investigate the global behavior of the prevalence of infectious diseases, the stability analysis of equilibrium for epidemic models have been carried out (see ^[6,7] and the references therein). For ordinary differential SIRS models, Mena-Lorca and Hethcote considered several kinds of SIRS epidemic models and a threshold parameters were also found in ^[6] to determine whether the disease dies out or approaches to an endemic equilibrium, we also note that the global stability of SIRS epidemic model have already been investigated and the threshold theorems are well obtained. In the subsequent studies, it was generally believed that the introduction of age-structure into epidemic models is necessary and reasonable. Since the fact that the age structure of a population affects the dynamics of disease transmission was recognized, various age-structured epidemic models have been investigated by many authors. Some theoretics on the transmission dynamics of age-structured epidemic models have been developed in ^[8,9,10,11]. He et al. ^[12] studied the optimal birth control of age-dependent competitive species. Yang et al. ^[13] proposed epidemic model with age-since-infection and diffusion, the next generation operator $R$ is also given as the basic regeneration numbers. In paper ^[14], Yang et al. also studied the threshold dynamics by the method of Lyapunov functionals for an age-of-infection epidemiological model with nonlinear incidence rate. Chekroun et al. ^[15] showed the stability of the equilibrium by using Lyapunov functions. Lan et al. ^[16] just studied the impact of hospital resources and environmental perturbation to the dynamics of SIRS model. But the effect of vaccination on threshold dynamics was not considered in these articles.

In fact, we notice that vaccination has a significant impact on the thresholds dynamics of many infectious diseases, such as COVID-19, HPV, etc. Obviously, vaccination of susceptible individuals can greatly reduce the probability that susceptible individuals become infected. Thus, decision makers hope to achieve the optimal proportion of vaccines with the lowest cost. Optimal control is a promising strategy to control disease outbreaks. The main purpose of using the control in some diseases is to search for the most effective one that reduces the infection to a minimum level while minimizing the cost of applying control measures. In papers ^[7,9,17], the strategies such as health promotion campaigns or lockdown policies can contribute to reducing the disease transmission. In paper ^[18], Mu and Zhang investigated the near-optimal control of SIRS multi-strain epidemic model with age-structure using Pontryagin maximum principle. Bolzoni et al. ^[19] investigated optimal control of epidemic size and duration with limited resources. Zhou et al. ^[20] researched optimal isolation strategies of emerging infectious diseases with limited resources. In papers ^[21,22,23], some measures correspond to screening and quarantining of infected were implemented to achieve the purpose of controlling the disease.

However, we note that the most of these articles by using maximum principle to show optimal controls and only present necessary conditions. Defectively, the method of maximum principle only addresses the optimal control problem with initial value case of $t_0 = 0$. In fact, vaccination can be administered at any time $t = t_0$ in the control of the disease. Surpassingly, the dynamic programming method can give the necessary and sufficient conditions for the existence of optimal control via the HJB equation at any initial times $t = t_0$ and initial states. In other words, the dynamic programming method is an extension of the maximum principle, namely, the dynamic programming method is equivalent to maximum principle method when $t_0 = 0$.

Overall, the threshold dynamics results about age-structured SIRS epidemic models with the proportion of the vaccine injections are comparatively scarce, as well as few articles show that dynamic programming approach to solve the optimal control problem. Thus, in this paper, taking into account for the effect of the proportion of the vaccine injections, we incorporate the vaccine injection rate in solving the threshold of age-structured SIRS epidemic models. And vaccination optimal control is also obtained through dynamic programming approach.

The layout of this paper is as follows: in section 2, an age-structure SIRS epidemic model is reproduced and new age-structure SIRS epidemic model with vaccination is given; in section 3, we give the basic regeneration number and discuss the threshold dynamics; in section 4, we formulate and solve the corresponding optimal control problem. The existence of optimal control is proved by HJB equation and the expression of optimal control is present. in section 5, some numerical simulations are performed to demonstrate the theoretical results. Brief conclusions are given in section 6.

2.   Model and preliminaries

We reproduce the age-structured SIRS epidemic model based on paper ^[24] as follows

where $S(a, 0) = S_0(a), I(a, 0) = I_0(a), R(a, 0) = R_0(a)$ are initial conditions, with the following conditions:

where $k_1, \; k_2\in[0, 1]$ present the weight of the infected and recovered, respectively. All the parameters are related to age $a$, positive and bounded, These parameters and their meaning are listed in the Table 1, where $\eta (a) = \sigma (a)+\alpha(a)$, and $\delta(a) = \mu(a)+\phi(a)+ \sigma (a)+\alpha(a)$.

As we all know that the age density distribution function of the total population reaches a stable state, i.e

We focus on threshold dynamics on an age-structured SIRS epidemic model with vaccination. The spread of many epidemics are age-related, such as Human Papilloma Virus (HPV) spread across the population after ten years olds usually. And the World Health Organization believes that the most appropriate age for HPV vaccination was 11-12 years old. Therefore, it makes sense to study the vaccination is only given for a certain age to age-structured SIRS epidemic model.

The introduced vaccination rate $u(A_0)$ satisfies $0\leq u(A_0)\leq 1$ and is only given for a certain age and susceptible individuals of age $A_0$ is inoculated at some point. The disease does not spread among people younger than age $A_0$, that is, there are no patients before the vaccination age $A_0$. Therefore, we can get that the system satisfied by the susceptible, infective, and recovering individuals based on model (2.1) as following

we call infectivity

where $S(A_0^-, t)$ and $I(A_0^-, t)$ are the densities of susceptible and recovering individuals in the left neighborhood of age $A_0$, $k(a)$ is a positive continuous function. The vaccination rate $u(A_0)$ is a constant and $k(a)$ is a nonnegative continuous function defined on $0\leq a \leq A$, where $A$ is the maximum age.

In this paper, unless otherwise specified. The $\mathbb{H}$ represents the Hilbert space, with inner product

and we consider the state equations on space $L^1(0, A; \mathbb{H}^3)$, when $\varphi = (\varphi_1, \varphi_2, \varphi_3)$ and $\phi = (\phi_1, \phi_2, \phi_3)$, $\varphi_i, \; \phi_i \in \mathbb{H}$, the definition of inner product on $L^1(0, A; \mathbb{H}^3)$ as

3.   Threshold dynamics

The existence of time-independent equilibrium solutions for the model (2.4) is discussed. Obviously, the model (2.4) has disease-free equilibrium

In order to obtain the endemic disease equilibrium solution, $S(a, t) = S(a), \; I(a, t) = I(a)$ and $R(a, t) = R(a)$ are substituted into the model (2.4) obtained

And we know when $R_0(a) = u(A_0)P(a)$, the $R(a, t) = u(A_0)P(a)$, and when the population reaches a dynamic equilibrium $P(a)$, Combining the equations (2.3) and (3.2), and substitute into (2.4) for the equation that satisfies $I(a, t)$. From (2.3), it shows that

where $D (t) = \int _{A_0}^A I(\hat{a}, t)d\hat{a}$. Let $\zeta (a, D(t)) = \delta(a)+ k(a)D(t)$, in order to solve the (3.3), when $A_0 < a < A$, the expression of $I(a, t)$ is

we can substitute (3.4) into the $D(t)$ yields when $A_0 < a < A$

Since

obviously, we have $D(t) = D_\infty$ if $I(a, t) = I_{\infty}$ as $t\rightarrow \infty$ is equilibrium of model, then

then

Then it expresses $I_{\infty} = 0$ or

It is quite obvious the right-hand side of the equation (3.8) is the minus function about $D_{\infty}$, thus, we can define

When $R_0 < 1$, (3.7) has only a unique zero solution and no positive root, when $R_0 > 1$, the (3.7) formula has zero root and the only positive root $I^* > 0$ satisfies (3.8) formula, thus there are only positive root $E^* = (S^*, I^*, R^*)$ of (2.4). For ease of analysis, we converse $i(a, t) = \frac{I(a, t)}{(1-u(A_0))P}$, equation (2.4) about $I(a, t)$ can be transformed into

From the transformation, we can see that the existence and stability of the positive equilibrium of model (2.4) is equivalent to positive equilibrium of (3.10) model. $R_0$ can be used as a threshold for the existence or absence of endemic diseases. It is also a threshold for the disappearance of diseases. Let $\hbar = 1-\frac{1}{R_0}$ and we call $\hbar$ vaccination elimination point.

Similar to the ^[25] method, the following conclusion can be obtained. Before discussing the global stability of equilibriums, we first propose the lemma 3.1, and the similar proof process have been presented in ^[26].

Lemma 3.1. Let $i_1(a, t)$ and $i_2(a, t)$ are the solutions of the $(3.10)$ equations, satisfying $i_1(a, 0) = i_{10}(a) \leq i_2(a, 0) = i_{20}(a)$, where $i_{10}(a), \; i_{20}(a)\in K$, then there are following properties

(1) $i_j(a, t)\in K, \; j = 1, \; 2;$

(2) $\; i_1(a, t) \leq i_2(a, t), \; t \geq 0$;

(3) When $0 < \xi < 1$, the solution $i(a, t)$ of $(3.10)$ that the initial value satisfies the $i(a, 0) = i_{10}(a)\xi$, then it must satisfy the $i_1(a, t)\xi \leq i(a, t)$,

where the $K$ is denoted

Theorem 3.2. When $R_0 < 1$, the model $(3.10)$ has only a disease-free equilibrium, which is globally asymptotically stable, while when $R_0 > 1$, the model $(3.10)$ also has a unique endemic equilibrium $E^*$, which is globally asymptotically stable and the disease-free equilibrium is unstable.

Proof. We only consider the asymptotic behavior of solutions where the initial values of the model (3.10) satisfy $0\leq i(a, 0) \leq 1$ in $(A_0, A)$. The theorem of the global stability is given for solutions.

It is not difficult to see if the model (3.10) has a positive equilibrium equivalent to the (3.5) has a positive root. Since the right of the (3.5) is reduced about $D(t)$ and tends to zero when the $D(t)$ tends to positive infinity. There is no positive real root to the (3.5) when $R_0 < 1$, and there is a unique positive real root when $R_0 > 1$, i.e. the model (2.4) has only a unique positive equilibrium when $R_0 > 1$ and, instead, no positive equilibrium when $R_0 < 1$.

When $R_0 < 1$, in order to study the global stability of the disease-free equilibrium, expression of the solution of (3.10) about $I(a, t)$ is obtained by using the characteristic line method

When $t > A$, we obtain

where $\chi(t) = \int_{A_0}^A P(\hat{a}) i(\hat{a}, t)d\hat{a}$, when $t < A$, the $\chi(t)$ satisfy is

For natural numbers $n > 1$, let $\chi_n = \max \limits_{nA\leq t \leq (n+1)A}\chi(t), \; L_n = \max \{\chi_{n-1}, \chi_n \}$. According to the (3.14) when $R_0 < 1$, we get

So we obtain $\chi(t) \leq R_0 \max \{ \chi_{n-1}, \; \chi_n \}$ when $nA \leq t \leq (n+1)A$. Because of $R_0 < 1$, we also obtain $\chi_n < L_n = \chi_{n-1}$, i.e. $\chi_n < \chi_{n-1}$, $\chi_n$ is monotonous and reduced and $\chi_n < \chi_{n-1}R_0$, from this delivery $\lim\limits_{n\rightarrow +\infty} \chi_n = 0$. Considering above conditions and (3.13) can also derive $\lim\limits_{t\rightarrow +\infty} i(a, t) = 0$. Finally, it is concluded from (3.13) that the disease-free equilibrium of the model (3.10) is stable $R_0 < 1$.

When $R_0 > 1$, substituting $i(a, t) = i(a)$ into the (3.10), the only endemic equilibrium of (3.10) is given by

When $A_0\leq a < A$, the $\forall\; i(a, t)$ satisfy $i(a, 0) > 0$, and $\chi(t) > 0$ can be derived from $(3.11)$ and $(3.14)$ when $0 \leq t\leq A$, and we also know that $i(a, A) > 0\; (A_0 < a < A)$ by (3.13). Repeat this process then the $\chi(t) > 0$ is obtained when $0 \leq t\leq 3A$, let $\chi^0 = \min\limits_{A \leq t\leq 2A} \chi(t)$, $\chi^* = \max\limits_{A \leq t\leq 2A}\chi(t)$, the following is got as

Comparing the forms of $(3.16)$ and $(3.17)$, we can find a positive number $0 < \xi < 1$ such that $i(a, 2A) \geq \xi i(a)$, namely, the following formula holds for the solution $i(a, t)$ of $(2.4)$

The Lemma 3.1 shows that the solutions $i_{\xi}(a, t)$ and $i_1(a, t)$ with initial values of $\xi \cdot i(a, 0)$ and $1$ respectively when $t = 0$, must have

Based on the fact that $i(a)$ is the equilibrium of $(3.10)$, then the $\xi i(a) \leq i_{\xi}(a, t)$, i.e. $i_{\xi}(a, t)$ is a monotone increasing function about $t$. Since $i_1(a, t)$ is a monotone subtraction function about $t$, and the corresponding $\chi_{\xi}(t)$ and $\chi_1(t)$ are monotone increasing and monotone decreasing, respectively, and there are $\lim\limits_ {t\rightarrow \infty} \chi_{\xi}(t) = \chi_{\xi}^0 \leq \lim\limits_ {t\rightarrow \infty} \chi_{1}(t) = \chi_{1}^0$. From (3.14), the equation has only unique normal number solutions when $R_0 > 1$, i.e. $\chi_{\xi}^0 = \chi_{1}^0 = \xi^{*}$, the $\chi (t)$ corresponding to the $i(a, t)$ also obtain $\lim\limits_ {t\rightarrow \infty} \chi (t) = \xi^{*}$. Reuse (3.13) when $t > A_0$, can be obtained $\lim\limits_ {t\rightarrow \infty} i(a, t) = i(a)$, i.e. the local disease equilibrium of (3.10) is globally attractive. The stability of the local disease equilibrium can be obtained by linearization. Therefore, the local disease equilibrium is globally asymptotically stable.

Remark 3.3. 3.2 shows that the epidemic disease will die out if $R_0 < 1$, while the disease will be prevalent if $R_0 > 1$. This implies that $R_0$ is the threshold of model $(2.4)$.

4.   Optimal control strategies

4.1.   Formulation of the optimal control problem

In section 3, we consider the effect of a age determined vaccine immunization of the system on threshold dynamics, which is one of the cases in which vaccines are added to all susceptible persons in the system. In this section, we focus on the optimal control of (4.1) on the basis of age-structured SIRS by implementing proportion of vaccination $u(a, t)$ as control strategy at all ages of system, and vaccination rate $u(a, t)$ satisfies $0 \leq u(a, t) \leq 1$. We aim to minimize the infected population, while keeping the cost of applying such control strategy at the minimum level.

Let $((a, t_0), x(a, t_0))\in [0, T)\times(\mathbb{H}\times\mathbb{H}\times\mathbb{H})$, and consider the following control system over on $(a, t)\in (0, A)\times[t_0, T]$:

where $S(a, t_0) = S_{t_0}(a), I(a, t_0) = I_{t_0}(a), R(a, t_0) = R_{t_0}(a)$ is varying initial time value and denote the initial state $x(a, t_0)$ as $x_{t_0}(a)$. Here the control $\{u(\cdot, \cdot):(0, A)\times[t_0, T]\rightarrow \mathcal {U}\}$, and $u(\cdot, \cdot)$ is measurable. We obtain the schematic diagram of model (4.1) with introducing vaccination control (see Figure 1).

The object is to design the optimal controller $u^*(a, t), \; (a, t)\in (0, A)\times[t_0, T]$ for the system (4.1) which minimizes the following performance index during the finite time interval $[t_0, T]$ ^{[27,28,29,30]}. Our goal is to minimize the total number of the infected and susceptible individuals by using minimal control efforts. The cost function as follows

where $I(a, t): (a, t)\in [0, A]\times[t_0, T]$ and we denote $x = (S, I, R), x_{t_0} \in (\mathbb{H}\times\mathbb{H}\times\mathbb{H})$, the terminal value is $V(x, T) = \int_0^Ah(I(a, T))da$. Let $L(t, x, u): = \tau_1I(a, t)+\tau_2u(a, t)+\frac{1}{2}\tau_3u^2 (a, t)$, where $\tau_1$, $\tau_2$ and $\tau_3$ are the positive weighting factors, representing the cost per unit time of the components $I(a, t)$, $u(a, t)$ and $u^2(a, t)$, respectively. In particular, $\int_{t_0}^T\int_0^A\tau_1I(a, t)dadt$ is the cost that infected individual creates for the society due to standard medical care, not including the vaccination treatment $u(a, t)$. $\int_{t_0}^T \int_0^A(\tau_2u(a, t)+\frac{1}{2}\tau_3u^2 (a, t))dadt$ is the cost of treating infected individuals. $h:\mathbb{H}\times\mathbb{H}\times\mathbb{H} \rightarrow \mathbb{H}$ are measurable and $h(I(a, T))$ denotes the total number disease in population depend on the age at the time $T$. The value function as

The function $V(\cdot, \cdot)$ will play an important role in obtaining optimal control, thus, we would like to study $V(\cdot, \cdot)$ in great detail. Let $(x^*, u^*)$ is optimal pair, where $x^* = (S^*, I^*, R^*)$. Noting that the initial time $(t = 0)$ and the initial state $(x(a, 0) = x_0)$ are fixed in the model ^[18]. The $h(I(a, T))$ satisfies

where $C$ is a positive constant, $\|\cdot\|$ represents norm on space $\mathbb H$ and $I(a, T)$, $\tilde{I}(a, T)\in \mathbb H$. The following hypothesis are put forward and need to be satisfied.

D1 $(\mathcal{U}, d)$ is a separable metric space and $T > 0$.

D2 The control set $\mathcal{U}$ is bounded convex

D3 Different controls correspond to same terminal value.

Then we study dynamic programming method, to give optimal strategy of model (4.1). The basic idea of this approach applied to optimal controls is to consider a family of optimal control problems with different initial times $(t = t_0\geq 0)$ and initial states $(x(a, t_0) = x_{t_0}(a))$ [see ^[31,32,33]], to establish relationships among these problems through the HJB equation. Firstly, we present the Bellman's principle of optimality as follows.

Theorem 4.1. Let (D1)-(D3) hold. Then for any $(t_0, x_{t_0}(a))\in [0, T)\times \mathbb{H}\times\mathbb{H}\times\mathbb{H}$,

where $L(t, x(t; t_0, x_{t_0}(a), u(\cdot, \cdot)), u(a, t)) = \tau_1 I_{t_0}(a)+\tau_2u(a, t)+\frac{1}{2}\tau_3u^2 (a, t)$.

Proof. Let us denote the right-hand side of (4.5) by $\overline{V}(t_0, x_{t_0}(a))$. By (4.3), we have

Thus, taking the infimum over $u(\cdot, \cdot) \in \mathcal {U}$ we get

Conversely, for any $\varepsilon > 0$, there exists a $u_{\varepsilon}(\cdot, \cdot) \in \mathcal {U}$ such that

Combing (4.6) and (4.7), we obtain (4.5).

The equation (4.5) is dynamic programming equation, which gives a relationship among globally and locally optimal via value function.

We would like to further study the (4.5), trying to obtain an equation for $V(t_0, x_{t_0}(a))$ simpler form. The following results give a partial differential equation that a continuously differentiable value function ought to satisfy. We know the $V(t_0, x_{t_0}(a))$ is a continuously differentiable value function via 4.1 and basic theorem of calculus.

Define the Lyapunov equation (LE)

That is

where $V_{x} = (\frac{\partial V}{\partial S}, \; \frac{\partial V}{\partial I}, \; \frac{\partial V}{\partial R})^{\top}$, $V_{t} = \frac{\partial V}{\partial t}$. The equation LE is the HJB equation when $u(a, t)$ is optimal control of (4.1).

In this subpart, we write the HJB equation of model (4.1) as well as its viscosity solution is discussed as following.

Theorem 4.2. Let (D1)-(D3) and $V(t, x)\in C^{1}([0, T]\times \mathbb{H}\times\mathbb{H}\times\mathbb{H})$. Then $V(t, x)$ is a solution of the following terminal value problem of HJB equation:

which can be written as $(4.9)$.

Proof. Fix $(t_0, x_{t_0}(a))\in [0, T)\times \mathbb{H}\times\mathbb{H}\times\mathbb{H}$ and $u\in \mathcal{U}$. Let $x(\cdot, \cdot)$ be the state trajectory corresponding to the control $u(\cdot, \cdot)\in \mathcal{U}^{\omega}[s, T]$ with $u(a, t)\equiv u$. On the one hand, we can write according to (4.5)

Rearranging the equation (4.11) and dividing the both side of the equation by $\hat{t}_0-t_0$ yield

By (4.5) with $\hat{t}_0\downarrow t_0$, we obtain

This results in

On the other hand, for any $\varepsilon > 0$, $\forall0\leq t_0\leq \hat{t_0} < T$ with $\hat{t}_0-t_0 > 0$ small enough, there exists a $u(\cdot, \cdot)\equiv u_{\varepsilon, \hat{t}_0}(\cdot, \cdot)\in \mathcal{U}$ such that

Thus, it follows that as (noting $V\in C^1([0, T]\times \mathbb{H}\times\mathbb{H}\times\mathbb{H})$)

Taking limit on (4.16) with respect to $\hat{t}_0\downarrow t_0$, we have

Combining (4.14) and (4.17), we obtain our results.

Definition 4.3. ^[34] A function $v \in C([0, T]\times \mathbb{H}\times\mathbb{H}\times\mathbb{H})$ is called a $viscosity \; solution$ of $(4.9)$ if

and for any $\psi \in C^1([0, T]\times \mathbb{R})$, whenever $v-\psi$ attains a local maximum at $(t, x)\in [0, T)\times \mathbb{H}\times\mathbb{H}\times\mathbb{H}$, we have

The function $v \in C([0, T]\times \mathbb{H}\times\mathbb{H}\times\mathbb{H})$ is called a $viscosity\; solution$ of $(4.9)$ if in $(4.18)-(4.19)$ the inequalities $\leq$ are changed to $\geq$ and $local\; maximum$ is changed to $local\; minimum$. In the case that $v$ is both a viscosity subsolution and supersolution of $(4.9)$, it is called a viscosity solution of $(4.9)$.

Theorem 4.4. Let (D1)-(D3) hold. Then the value function of $V(t, x)$ satisfies

for some $K > 0$. Moreover, $V(t, x(a, t))$ is the only solution of $(4.9)$ in the class $C([0, T]\times\mathbb{H}\times\mathbb{H}\times\mathbb{H})$.

Proof. Through 4.2 and $t_0\leq \hat{t}_0$ we obtain

Thus, $V(t, x(a, t))$ satisfies (4.21). We can know that $V(t, x(a, t))$ is unique viscosity of the HJB equation via the same method as [^[34], Theorem 2.5].

4.2.   Optimal control

In this subsection, we study the existence of the optimal control pair in finite time for the system (4.1) and construct the Hamiltonian of the optimal control problem.

Theorem 4.5. There exists an optimal control $u^* \in \mathcal{U}$ and a corresponding optimal state $S^*, \; I^*, \; R^*$ such that

subject to the control system (4.1).

Proof. To show the existence of the optimal control for the problem under consideration, we use the result in ^[35,36]. We notice that the state and control variables are non-negative, and the control set $\mathcal{U}$, by definition, is closed and bounded. The optimal system is bounded, which ensures the compactness needed for the existence of an optimal control. Further, the integrand $\int_{t_0}^T \int_0^A(\tau_1I(a, t)+\tau_2u(a, t)+\frac{1}{2}\tau_3u^2(a, t))dadt$ is convex on the control set $\mathcal{U}$ due to the biquadratic and quadratic nature of control variable $u(a, t)$, which represents vaccination proportion. In addition, there exists a constant $\nu > 1$ and positive numbers $\kappa_1$, $\kappa_2$ such that

We complete the existence of the optimal control of state variables.

Theorem 4.6. Let $u^*(a, t)$ be optimal control variable, $S^*(a, t)$, $I^*(a, t)$ and $R^*(a, t)$ be corresponding optimal state variable of model $(4.1)$. The corresponding optimal control is given as follows:

where $\frac{\tau_2}{\tau_3} = \tau$.

Proof. We define the Hamiltonian function $H(t, x, u(a, t), V_x), \; t\in[0, T], \; u(a, t)\in \mathcal{U}_{ad}, \; x, \; V_x(x, t) \in \mathbb{H}\times\mathbb{H}\times\mathbb{H}$ and by constructing the Hamiltonian function and HJB equation to characterize this optimal control $u(a, t)$ as follows:

Let $u^*(a, t)$ be given optimal control and $S^*(a, t)$, $I^*(a, t)$ and $R^*(a, t)$ be corresponding optimal state variable of model (4.1). By the Hamiltonian function (4.22), optimal control $u(a, t) = u^*(a, t)$ is obtained

From the properties of the control set with the findings, then there is exists optimal control

Remark 4.7. If we could obtain the value function $V(t, x(a, t))$ by solving the HJB equation, then we could, at least formally, construct an optimal pair for each of optimal control problem for different initial values. The principle involves the following steps to solve optimal control problem

$Step.1.$ Solve the HJB equation $(4.10)$ to find the $V(T, x(a, t))$;

$Step.2.$ Find $u^*(a, t)$ through Hamiltonian function $(4.22)$;

$Step.3.$ Combine the $u^*(a, t)$ to solve model $(4.1)$ to get the optimal pair $(x^*(a, t), u^*(a, t))$.

5.   Numerical simulation

This section aims to illustrate the effectiveness of our theoretical results obtained in previous sections.

5.1.   Numerical simulation of the stability of equilibria

In the numerical simulation, the parameters are given in Table 2. We assume that $(a, t) \in [0, 80]\times [0,500], \; \Delta t = 0.1, \; \Delta a = 1$, take $p_{\infty} = 1.0, \; \varrho = 0.2, \; \delta = 0.1827, \; k = 0.2$, then $R_0\approx 2.69 > 1$. The parameters that we partially need to assume are reasonable according to the study of Yang et al. ^[14] has shown that the range of the basic reproduction number of the pandemic influenza in $1.0-4.0$. We see from 3.2 the $E^*$ is globally asymptotically stable. Actually, as showing in the following Figure 2 (a), the density of the infected individual $I(a, t)$ tends to be a positive constant over time.

Take $\varrho = 0.1, \; \delta = 0.1827, \; k = 0.1$, then $R_0\approx 0.78 < 1$, thus we can know the $E^0$ is globally asymptotically stable, as showing in the following Figure 2 (b), the density of the infected individual $I(a, t)$ tends to zero over time. These are the same conclusions as 3.2.

We choose $\delta = 0.1827$, $0.2027$ and $0.2227$ to study the effect on the infected population and all the other parameters are fixed as in Table 2. We calculate the corresponding basic reproduction number $R_0 \approx 0.9427$, $0.8979$ and $0.7802$ when $R_0 < 1$, respectively. In Figure 3 (a), we can see that the extinction time of decreases with the increase of infected removal rate $\delta$. In order to simulate the cases of $R_0 > 1$, we calculate the corresponding basic reproduction number $R_0 \approx 1.3468$, $2.2441$ and $2.6930$, respectively. The simulation result is presented in Figure 3 (b). Synthetically, it is obvious to see from Figure 3 that the peak of $I(a, t)$ decreases significantly as infected removal rate $\delta$ of age a goes up.

5.2.   Numerical simulation for optimal control

In this section, we show the result of optimal control using a numerical example. The parameters are given in Table 2. $(a, t) \in [0, 80]\times [0,500], \; \Delta t = 0.1, \; \Delta a = 1$ by performing some numerical simulations. We take $\tau_1 = 11.7$, $\tau_2 = 3.4$ and $\tau_3 = 12$. The corresponding paths of control and infective populations $I(a, t)$ are plotted in Figure 4, Figure 5 (a), Figure 6 and Figure 7 (a) show the section view at age $a = 20$ and $a = 30$ of the image variation tendency of $I(a, t)$ with and without control and the tendency of corresponding optimal control $u(a, t)$. We observe that the comprehensive use of control $u(a, t)$ works better than no control applied during the course of disease. The effect of applying control reduces the number of infective populations $I(a, t)$ dramatically. In short, control variable $u(a, t)$ play an important role in the control of the disease. At the same time, we can also observe a reduced epidemic time of population infection.

We note that a significant decline in the number of infected persons $I(a, t)$ during vaccine control of susceptible at age $a = 20$ and $a = 30$ from these following figures.

When $R_0 > 1$, $I(a, t)$ trends is regard to the optimal controls, where the upper-level image is the evolution of $I(a, t)$ without control and the lower-level image is the evolution of $I(a, t)$ with control on in Figure 4. In Figure 5 (a) shows the optimal control pathes at ages $a = 20$ and $a = 30$, repectively. Eventually the disease tends to extinction with control. Compared with the path of non-vaccinated $I(a, t)$, the epidemic time of $I(a, t)$ with vaccination is significantly shortened, as well as the density of infected population verge to zero eventually.

When $R_0 < 1$, $I(a, t)$ trends is regard to the optimal controls, where the upper-level image is the evolution of $I(a, t)$ without control and the lower-level image is the evolution of $I(a, t)$ with control in Figure 6. In Figure 7 (a) shows the optimal control pathes at ages $a = 20$ and $a = 30$, repectively. Eventually the disease tends to extinction with control. Compared with Compared with the path of non-vaccinated $I(a, t)$, the epidemic time of $I(a, t)$ with vaccination is significantly shortened. Combined with Figure 4 and the corresponding conclusions, it is clear that vaccination have a certain degree of hindrance to the transmission of epidemics. We also observe the Figure 5 (b) and Figure 7 (b) that vaccine control varies for different ages, mainly adolescents and middle age, and this control strategy is consistent with the actual control of many epidemics such as pneumonia, influenza and so on. The pneumococcal vaccine, COVID-19 vaccine and influenza vaccine are not suitable for the elderly. Overall, population age has a significant effect on the implementation of vaccine control strategies for many common diseases.

6.   Concluding remarks

In this paper, we recurred an age-structured SIRS epidemic model (2.1) with vaccination control, and obtained the condition for disease extinction and persistence. The research results showed that when $R_0 < 0$, the disease extincted; when $R_0 > 0$, the disease persisted. The conditions of disease extinction and permanence were given in 3.2. And we verified the resluts by numerical simulation. However, since the bifurcation phenomenon will occur when $R_0 = 0$, this part will also be an important point for our further study of the age-structured SIRS epidemic system dynamics. Besides, we also studied the control problem of SIRS epidemic model with age structure. By proving the existence of viscosity solution, we obtain the existence of optimal control. Then, we obtained the implicit expression for optimal control and HJB equation. Finally, by using Milstein method, the optimal proportion of vaccination is obtained. In addition, we plan that in the next step, traveling wave solutions of an age-structured SIRS epidemic model will become one of our main research contents. Wu et al. ^[37] studied the existence and non-existence of traveling wave solutions for a diffusive age-structured SIR epidemic model. Their work provides some insights on how to deal with the high dimensional age-structured epidemic models.

Acknowledgments

The authors thank the editor and referees for their careful reading and valuable comments. The research was supported in part by the Ningxia Natural Science Foundation (No. 2020AAC03067).

Conflict of interest

The authors have no conflict of interest.