Model-free based control of a HIV/AIDS prevention model

1.   Introduction

The SICA (Susceptible-Infected-Chronic-AIDS) compartmental model for HIV/AIDS transmission dynamics was proposed by Silva and Torres in their seminal paper ^[1]. Since then, the model has been extended to stochastic systems of differential equations ^[2,3], to fractional-order ^[4,5] as well as discrete-time dynamics ^[6], and applied with success to describe very different HIV/AIDS epidemics, like the ones in Cape Verde ^[7,8] or Morocco ^[9]. For a survey see ^[10].

In this work, we consider a SICA epidemic problem of controlling the transmission of the human immunodeficiency viruses (HIV), by considering not only the medical treatment with multiple antiretroviral (ART) drugs, but also the pre-exposure prophylaxis (PrEP), which are medicines taken to prevent getting HIV infection. According to the Centers for Disease Control and Prevention, PrEP is highly effective for preventing HIV, when taken as prescribed, and reduces the risk of getting HIV from sex and from injection drug use by about 99 and 74%, respectively ^[11].

The main contribution of this work is to propose a model-free based control algorithm that closes the loop between the infected individuals with HIV and PreP medication, in such manner that the medication is driven in 'real-time', according to the number of infected individuals that has to be asymptotically reduced. We highlight that model-free control offers the advantages of a simple Proportional-Integral-Derivative (PID) controller in the framework of model-free design, that is, one whose parameters can be easily tuned without a precise knowledge of the controlled epidemiological model.

The model-free control methodology, originally proposed by Fliess and Join in ^[12], has been designed to control a priori any "unknown" dynamical system in a "robust" manner, and is referred to as "a self-tuning regulator" in ^[13]. This control law can be considered as an alternative to PI and PID controllers ^[14] and the performances are really satisfactory taking into account that the control is calculated based only on the information provided by the controlled input and the measured output signal of the controlled systems. This control law has been extensively and successfully applied to control many nonlinear processes: see, e.g., ^[12,15,16] and the references therein. In particular, some applications have been dedicated to the control of chemistry and biological processes ^{[15,17,18,19,20]}, including the development of an artificial pancreas ^[21]. A derivative-free-based version of this control algorithm has been proposed by the first author in ^[22], for which some interesting capabilities of online optimization have been highlighted. To the best of our knowledge, the application of model-free control to SICA modeling has never been discussed before.

Here, we compare the solutions obtained by the model-free control method with the corresponding solutions of an optimal control problem for HIV/AIDS transmission from ^[8], which has a mixed state-control constraint. In ^[8], the control system is based on a SICAE (Susceptible, HIV-Infected, Chronic HIV-infected under ART, AIDS-symptomatic individuals, E-under PrEP medication) model for the transmission of HIV in a homogeneously mixing population. The control $u$ represents the fraction of susceptible individuals under PrEP, with $0 \leq u(t) \leq 1$, that is, when $u(t) = 0$, no susceptible individual takes PrEP at time $t$, and when $u(t) = 1$ all susceptible individuals are taking PrEP at time $t$. The mixed state-control constraint refers to the fact that only people who are HIV-negative and at a very high risk of HIV infection should take PrEP, and also to the high costs of PrEP medication. Therefore, the number of susceptible individuals that takes PrEP, at each day, must be bounded by a positive constant. Moreover, the cost functional, which is aimed to be minimized, represents a balance between the number of HIV infected individuals and the costs associated with PrEP implementation.

The paper is structured as follows. In Section 2, we propose a model-free control method and the procedure to minimize the HIV infected cases is described. In Section 3, we present some numerical simulations and provide a comparison of the results obtained using the model-free based approach with the ones in ^[8] from the Pontryagin maximum principle. Section 4 discusses and compares the results. Finally, some concluding remarks and possible directions for future work are given in Section 5.

2.   Materials and methods

In this section, we propose our model-free based control method and apply it to an epidemiological problem of minimizing HIV-infected individuals.

2.1.   Principle of the model-free based control

Model-free control was introduced in 2008 and 2009 by Fliess and Join in ^[23,24]. It is an alternative technique to control complex systems based on elementary continuously updated local modeling via unique knowledge of the input-output behavior. The key feature of this approach lies in the fact that the control system, which might be highly nonlinear and/or time-varying, is taken into account without any modeling procedure ^[25]. The model-free based control approach has been successfully implemented in concrete applications to diverse fields, ranging from intelligent transportation systems to energy management, etc., see ^[12] and references cited therein. To the best of our knowledge, no one as yet used this approach in the context of epidemiology.

Consider a nonlinear dynamical system $f : u \mapsto y$ to control:

where $f$ is the function describing the behavior of a nonlinear system and $\boldsymbol{x} \in {\mathbb{R}}^n$ is the state vector. The proposed control is an application $\mathcal{C}_{\pi} : (y, y^*) \mapsto u$, whose purpose is to control the output $y$ of system (2.1) following an output reference $y^*$. In simulation, the system (2.1) is controlled in its "original formulation", without any modification or linearization.

For any discrete moment $t_k, \, k \in \mathbb{N}^*$, one defines the discrete controller $\mathcal{C}_{\pi} : (y, y^*) \mapsto u$ as an integrator associated to a numerical series $(\Psi_k)_{k \in \mathbb{N}^*}$, symbolically represented by

with the recursive term

where $y^\ast$ is the output (or tracking) reference trajectory; $K_p$ and $K_i$ are real positive tuning gains; $\varepsilon_{k-1} = y^\ast_{k} - y_{k-1}$ is the tracking error; and $k_\alpha e^{-k_\beta k}$ is an initialization function, where $k_\alpha$ and $k_\beta$ are real positive constants. In practice, the integral part is discretized using, for example, Riemann sums.

The set of the $\mathcal{C}_{\pi}$-parameters of the controller, is defined as the set of the tuning coefficients $\{K_p, K_i, k_\alpha, k_\beta\}$.

2.2.   Methodology

Consider the problem of minimizing the number of infected individuals with HIV, given by the state trajectory $I$, through the control measure $u$ associated to PrEP medication, and satisfying a mixed state-control constraint $S u \leq \gamma_{max}$, where $\gamma_{max}$ is a positive constant.

The control sequence is divided into two steps, in order to manage, separately, the increasing $u$ transient and the associated decreasing $u$ transient that must decrease afterwards both the infected cases and the control input $u$ to lower values on $u$ and $I$. Concerning the control input $u$, it is expected that $u(t > t_f) = 0$, where $t_f$ is the time from which the medication is stopped.

● A first sequence, associated to the setting up of the medication, aims to progressively increase the medication (and thus start decreasing the infected states) until a certain threshold of infected cases is reached, above which the number of infected cases could be considered as stabilizable around a low value. This control sequence can be managed by a simple $L$-linear or $Q$-quadratic slope, such as

driven in open-loop, i.e., independently from any feedback of the infected state, that increases gradually the medication while satisfying the constraint on $S u \leq \eta$ due to the remaining high level of the susceptible cases. It appears crucial to accelerate the medication at the beginning, in order to reach rapidly $\gamma_{max}$ and allow a strong decreasing of the infected cases. This point will be discussed later in Section 3.

● Denote $u_{max} = \max u(t)$, the maximum value of $u$. The second sequence is associated to the decrease of the medication until the infected state is stabilized around a low value. This sequence is managed by our proposed model-free based control that interacts, in real-time, with the number of infected cases and, consequently, calculates the "optimal" medication $u$ in order to decrease and stabilize the infected state. According to Eq (2.2), the control reads:

where $I^\ast$ denotes the infected cases reference that practically can be chosen as $I^\ast = \min I(t)$, where the minimum value of $I$ can be reached online, updating the tracking reference and, therefore, ensuring that the control law is "synchronized" on the lowest value that can be reachable. It is worth to note that, depending on the behavior of the closed-loop, a saturation is added to bound the controlled $u$: $0 \leq u \leq u_{max} \leq 1$.

The implementation of the control scheme is depicted in Figure 1, where the control sequence starts from the transient slope or the quadratic function and then, once $u_{max}$ is reached, switches to the proposed model-free based controller. The parameters of the control sequence to be adjusted, comprise the $\mathcal{C}_{\pi}$-parameters set of the model-free control algorithm and the $(u_0, L, Q)$ parameters of the first sequence, depending if a linear or a quadratic slope is involved.

Additional constraints. The slope can be adjusted according to a state-control constraint that determines the maximum number of susceptible individuals that take PrEP medication, at each instant of time. This constraint reads as $S(t) \cdot u(t) < \gamma_{max}$, for all $t$, where $\gamma_{max}$ is the corresponding upper bound.

To properly tune each sequence, in order to satisfy both the state-control constraint as well as to minimize the cost criteria, a derivative-free based optimization procedure can be applied ^[26].

2.3.   Classical optimal control problem applied to the HIV/AIDS SICAE model with a mixed state-control constraint

Consider the mathematical model for HIV/AIDS transmission in a homogeneously mixing population proposed in ^[8] and based in ^[1,7].

The model subdivides human population into five mutually-exclusive compartments: susceptible individuals ($S$); HIV-infected individuals with no clinical symptoms of AIDS (the virus is living or developing in the individuals but without producing symptoms or only mild ones) but able to transmit HIV to other individuals ($I$); HIV-infected individuals under ART treatment (the so called chronic stage) with a viral load remaining low ($C$); HIV-infected individuals with AIDS clinical symptoms ($A$); individuals that are under PrEP medication ($E$). The total population at time $t$, denoted by $N(t)$, is given by $N(t) = S(t) + I(t) + C(t) + A(t) + E(t)$. The model assumptions are the following ^[1,8]. Effective contact with people infected with HIV is at a rate $\lambda$, given by

where $\beta$ is the effective contact rate for HIV transmission. The modification parameter $\eta_A \geq 1$ accounts for the relative infectiousness of individuals with AIDS symptoms, in comparison to those infected with HIV with no AIDS symptoms. Individuals with AIDS symptoms are more infectious than HIV-infected individuals (pre-AIDS) because they have a higher viral load and there is a positive correlation between viral load and infectiousness ^[27]. On the other hand, $\eta_C \leq 1$ translates the partial restoration of immune function of individuals with HIV infection that use ART correctly ^[28]. All individuals suffer from natural death, at a constant rate $\mu$. HIV-infected individuals, with and without AIDS symptoms, have access to ART treatment. HIV-infected individuals with no AIDS symptoms $I$ progress to the class of individuals with HIV infection under ART treatment $C$ at a rate $\phi$, and HIV-infected individuals with AIDS symptoms are treated for HIV, at rate $\alpha$. HIV-infected individuals with AIDS symptoms $A$, that start treatment, move to the class of HIV-infected individuals $I$, moving to the chronic class $C$ only if the treatment is maintained. HIV-infected individuals with no AIDS symptoms $I$ that do not take ART treatment progress to the AIDS class $A$, at rate $\rho$. Individuals in the class $C$ that stop ART medication are transferred to the class $I$, at a rate $\omega$. Only HIV-infected individuals with AIDS symptoms $A$ suffer from an AIDS induced death, at a rate $d$. The proportion of susceptible individuals that takes PrEP is denoted by $\psi$. It is assume that PrEP is effective, so that all susceptible individuals under PrEP treatment are transferred to class $E$. The individuals that stop PrEP become susceptible individuals again, at a rate $\theta$. Susceptible individuals are increased by the recruitment rate $\Lambda$.

Such model is given by the following system of ordinary differential equations:

Recall that PrEP medication should only be administrated to people who are HIV-negative and at very high risk for HIV infection. Moreover, PrEP is highly expensive and it is still not approved in many countries. Therefore, the number of individuals that should take PrEP should be limited at each instant of time for a fixed interval of time $[0, t_f]$ ^[8]. The optimal control problem proposed in ^[8], and considered in this paper for comparison of results, takes into account this health public problem.

The main goal of the optimal control problem is to determine the PrEP strategy $\psi$ that minimizes the number of individuals with pre-AIDS HIV-infection $I$ as well as the costs associated with PrEP. Let the fraction of individuals that takes PrEP, at each instant of time, be a control function, that is, $\psi \equiv u(t)$ with $t \in [0, t_f]$, and assume that the total population $N$ is constant: the recruitment rate is proportional to the natural death rate, $\Lambda = \mu N$, and there are no AIDS-induced deaths ($d = 0$). The controlled model is given by

Remark 1. All the parameters of the SICAE model $(2.5)$ are fixed with the exception of the control function $u(t)$. This system is deterministic and there is no uncertainty. However, the model-free based control proposed in this paper does not use these equations. They are only needed in the classical optimal control approach that is used here for comparison. The sensitivity analysis of the parameters of the SICA model, which is in the basis of the SICAE model $(2.4)$, was studied before in ^[7].

The classical optimal control problem proposed in ^[8], and that is considered here in comparison with the model-free control method, considers the cost functional

where the constants $w_1$ and $w_2$ represent the weights associated with the number of HIV infected individuals $I$ and on the cost associated with the PrEP prevention treatment, respectively. It is assumed that the control function $u$ takes values between 0 and 1. When $u(t) = 0$, no susceptible individual takes PrEP at time $t$; if $u(t) = 1$, then all susceptible individuals are taking PrEP at time $t$. Let $\gamma_{max}$ denote the total number of susceptible individuals under PrEP for a fixed time interval $[0, t_f]$. This constraint is represented by

which should be satisfied at almost every instant of time during the whole PrEP program.

Let

The classical optimal control problem proposed in ^[8] consists to find the optimal trajectory $\tilde{x}$, associated with the control $\tilde{u}$, satisfying the control system (2.5), the initial conditions

the constraint inequality (Eq 2.7), and where the control $\tilde{u} \in \Omega$,

minimizes the objective functional Eq (2.6).

2.4.   Comparative study and cost criteria definitions

We evaluate the accuracy of our model-free proposed approach compared to the classical optimal control one when applied to the SICAE model described in Section 2.3.

Let $T_e$ denote the final time of the $u$ treatment such as $u(t \geq T_e) = 0$ and $I(T_e)$ denotes the final value of the state "Infected" at the time $T_e$. Regarding the "energy" of the control input $u$ with respect to the behavior of the infected cases and the period of time $T_e$ for which the medication $u$ is in effect, i.e., $u(t \geq T_e) = 0$, let us consider the following cost criteria:

● Cost criterion for performances over the final time $T_e$:

● Time-pondered cost criterion:

which takes into account the effective period needed to stabilize the infected case, i.e., the period for which $u > 0$.

3.   Numerical simulations and results

To perform the numerical simulations, we consider the following parameter values, borrowed from ^[8]: $N = 10200$, $\mu = 1/69.54$, $\beta = 0.582$, $\eta_C = 0.04$, $\eta_A = 1.35$, $\theta = 0.001$, $\omega = 0.09$, $\rho = 0.1$, $\phi = 1$ and $\alpha = 0.33$. The weight constants take the values $w_1 = w_2 = 1$.

The initial conditions are given by

and the mixed state-control constraint is

In Table 1, we evaluate several cases with the cost criteria $J_{u+I}$, $J_{I}$ and $J_{u+I}^{Te}$, according to the final time of medication $T_e$. We compare the constrained and unconstrained classical optimal control problems and the unconstrained and constrained model-free problems with two types of configurations: slope and quadratic initial transient. The classical optimal control problem corresponds to the one performed in ^[8]. Figures 2–6 illustrate several scenarios: the constrained case is not satisfied under slope, see Figure 2; the constraint inequality (Eq 3.1) is satisfied under slope, see Figures 3 and 4; the constrained cases are satisfied under quadratic function, see Figures 5 and 6.

From Figure 2(a), we see that, although the model-free based control and the classical optimal control approaches propose completely different control functions, the associated number of HIV infected individuals $I(t)$, $t \in [0, 25]$, are very similar. Interestingly, the control solution of the model-free based control is active in a much smaller interval of time that the one obtained with the classical optimal control approach: $T_e = 11.3$ versus $T_e = 25$ (see first line of Table 1). Analogous conclusions are taken from Figures 3(a)–6(a).

It should be noted, however, that in Figure 2(b), the control obtained from the model-free based approach does not satisfy the mixed state-control constraint $S(t) u(t) \leq 2000$ for all $t$. In order to satisfy this constraint, one must increase the time interval where the model-free control is active, see Figures 3(b)–6(b). This depends on the configuration of the initial transient of the control (slope or quadratic), see Figures 3(a)–6(a) and Table 1.

4.   Discussion

The proposed control sequence can be considered as "quasi-optimal" in the sense that it does not obey to the Pontryagin maximum principle, so it is not an optimal control by definition, but it offers similar properties in terms of cost criteria minimization and reduction of the duration of treatment that is assured by our procedure.

The sequence is fully parametrized thanks to the initial transient coefficients $(L$ or $Q)$ associated to $u_{max}$, including the $\mathcal{C}_{\pi}$-parameters for the decreasing transient, that must be adjusted according to the evolution of the infected state. In particular, the maximum value on the product $S u$ depends on the "speed" of the increasing transient as well as the final value $I(T_e)$, which depends on the "speed" of the increasing transient, the initial value $u_0$, and the feedback control that stabilizes $I$ through the decreasing transient. The transient slope plays a key role in the "accuracy" of the initial decrease of the infected state since a sufficient dose of the medication $u$ must be injected to the population in order to maintain the infected state to a lower level. Therefore, the maximum value of $u$ is a trade-off between the constraint $S u \leq \gamma_{max}$ to be satisfied and the duration of the treatment. Figure 2 illustrates a rather quick treatment, involving thus a fast initial transient but the constraint $S u \leq 2000$ is not satisfied; slower medical treatment for which the injection of the medication $u$ takes more time due to the constraint $S u \leq 2000$, can reduce the final asymptotic value $I(T_e)$ despite not necessarily fully reducing the cost criteria. The model-free based control aims to relax the treatment until $u = 0$ is reached. A first tuning has been made according to the gained experience and a more precise tuning can be performed using ^[26]. The numerical evaluation of the cost criteria shows that our approach is globally better in terms of energy minimization. Moreover, the time-pondered criteria shows that the proposed control procedure is favorable to our approach taking into account the reduction of the duration of the treatment. These results illustrate afterwards that tuning the parameters of the proposed control sequence is a trade-off between considering minimizing the cost criteria, or minimizing the final value $I(T_e)$, depending of additional constraints.

We remark that the model-free based control could have been also used to drive the initial transient instead of the slope or the quadratic function, but the current algorithm offers slower performances at the very beginning to initiate the increase of the medication that prevents it to deal with, for example, the constraint $S u \leq 2000$.

5.   Conclusion and future work

We have considered a SICAE epidemiological model for HIV/AIDS transmission, proposing, for the first time in the literature, a model-free based approach to minimize the number of infected individuals. This approach consists in initializing PrEP medication, using a basic linear or quadratic function, and, after that, creating a direct feedback to control the decrease of infected individuals with respect to the considered measure of infected cases. Globally, the advantages of the proposed approach, when compared with the classical optimal control based on the Pontryagin Maximum Principle, is that it does not need any a priori knowledge of the model and a simple tuning of the proposed control sequence values allows good performances in terms of "energy" minimization as well as minimization of the medical treatment duration. We concluded that our control strategy highlights interesting performances compared with the classical optimal control approach used in ^[8].

From a biological point of view, our application of the model-free based control approach allows to propose new solutions for the implementation of PrEP in the prevention of HIV transmission, considering the constraints associated to the limitations on the availability of medicines for HIV and on number of individuals that the health systems have capacity to follow up during their treatment. To the best of our knowledge, we were the first to apply the model-free based control approach in the context of epidemiology.

Future work may include replacing the slope or the quadratic initial transient by an optimal control; improvement of the proposed model-free based control; implementation and comparison with the original Fliess-Join version of the model-free control, as it has been done, for example, for the glycemia control ^[14]. Stability issues regarding the closed loop are very important and a promising LMI framework dedicated to study the stability of optimization algorithms is also of interest ^[29,30].

Acknowledgments

This work was partially supported by Portuguese funds through CIDMA, The Center for Research and Development in Mathematics and Applications of University of Aveiro, and the Portuguese Foundation for Science and Technology (FCT–Fundação para a Ciência e a Tecnologia), within project UIDB/04106/2020. Silva is also supported by the FCT Researcher Program CEEC Individual 2018 with reference CEECIND/00564/2018.

The authors are grateful to Reviewers for several constructive suggestions and remarks.

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.