Asymptotic optimality of a joint scheduling–control policy for parallel server queues with multiclass jobs in heavy traffic

1.   Introduction

We consider a stochastic service system consisting of n servers and m classes of impatient customers and introduce a novel joint scheduling–control policy with three parts: customer admission control, service scheduling control, and service rate control (see Figure 1 and Section 2 for a detailed illustration).

Let $k = 1, 2, \cdots, m$; in this case, $U_{k}^{n}(\cdot)$ represents the customer admission control process in Figure 1, and $\xi_{k}^{n}(\cdot)$ represents the service scheduling control process. Let $\xi ^{n} = (\xi_{1}^{n}, \xi_{2}^{n}\cdots\xi_{m}^{n})$; thus, $H_{k}^{n}\big(\xi^{n}(\cdot)\big)$ represents the service rate control process. The arrivals of each customer class in the system follow a renewal process, represented by $\lambda_{k}^{n}$ in Figure 1. Any server can offer service to any customer, and the corresponding service rate depends on the overall state of the system, not only on the individual customer or the individual server, which is a unique feature of our proposed policy. Note that the vector $\xi^{n}$ is used in the rate control process $H_{k}^{n}(\cdot)$ for each type of service, not some component $\xi_{k}^{n}$.

Our joint scheduling–control policy has three parts, which are dynamically adjusted to each other and combined with a cost function to optimize our system. For example, if too many customers of a certain type arrive, we can either stop letting them in (customer admission control), schedule a few more service stations to work for such customers (service scheduling control), or increase the service rate for such customers (service rate control). These three approaches can be used simultaneously, or one or two of them can be chosen in combination, which has to be chosen in conjunction with the cost function to optimize the system.

Our service discipline is not first in–first out (FIFO) but a priority model and is inclusive of both non-preemptive and preemptive resume policies (i.e., policies that either prevent or allow the service to be interrupted and the customer to return to the queue without completing the service). For example, if the cost of losing or waiting for a certain type of customer is particularly large, the service in progress can be interrupted so that this type of customer can be served quickly and the waiting time can be reduced.

Incoming customers are impatient, with random waiting times that they tolerate, shown as $\theta_{k}^{n}$ in Figure 1. These waiting times are independent and identically distributed (i.i.d.) and are independent of the arrival and service processes. Customers face two situations in the system: being served or being in suspension. A customer departs if his or her service is completed, and abandons the queue if he or she has spent too much time waiting.

Of course, these added control components increase the cost, and the ultimate goal of this paper is to obtain an asymptotically optimal joint scheduling–control policy to show that the added control cost is worthwhile and that the total cost decreases. Our costs are linear or nonlinear expected cumulative discount functions of appropriately normalized performance metrics. Our costs include fixed components, such as the depreciation of equipment and basic salaries of personnel, as well as some dynamic components, such as the cost of idle servers, the cost of customer abandonment, the cost of customer queuing, the cost of customer access control, the cost of server scheduling control, and the cost of server rate control. Of course, in practice, they can be added or subtracted accordingly.

In contrast to the control policies proposed in existing studies (see, e.g., Bell and Williams ^[1], Harrison ^[2], Krichagina and Taksar ^[3], Kumar ^[4], Kushner and Chen ^[5], Martins et al. ^[6], Plambeck et al. ^[7], Van Mieghem ^[8], Weerasinghe ^[9] and Atar et al. ^[10]), our joint scheduling–control policy is designed more intelligently via a non-negative vector function of queue states and their corresponding ratios. This problem is new to the literature concerning the design of joint scheduling and control policies.

Given the inherent complexity of the system$^\prime$s equations and the difficulty of solving them directly, we apply diffusion approximation theory and consider the Halfin–Whitt heavy traffic regime (see, e.g., Halfin and Whitt ^[11] and Jagerman ^[12]). This approach yields a formally weak limit of the joint scheduling control problem. This limit problem, which we call the diffusion control problem (DCP), is a stochastic differential equation (SDE). Next, we present the corresponding Hamilton–Jacobi–Bellman (HJB) equation and prove the existence and uniqueness of the solution to this equation. This solution is the optimal Markov policy for the diffusion control problem (cf. ^[13]), and we use this solution to devise a policy for the original joint scheduling control problem and prove its asymptotic optimality.

The effectiveness of the designed joint scheduling–control policy is illustrated by numerical examples, as shown in Figure 2. Compared with other policies, our designed joint scheduling–control policy has the lowest cost, as shown by the leftmost blue rectangle in Figure 2. The results show that our designed joint scheduling–control policy has significant advantages in reducing the system$^\prime$s cost and improving service efficiency. The results of this numerical simulation not only confirm the rationality and practicality of the previous theoretical analysis but also provide a valuable reference for practical applications. A more detailed description of these numerical simulations and comparisons is presented in Section 3.

Therefore, determining how to establish the relationship between the physically optimal model and the limit diffusion-based optimal control system is the key to proving the asymptotic optimality of our designed policy. In doing so, we extend several techniques developed in Weerasinghe ^[9] to broaden our goal from a single customer class to multiple customer classes and from rate control alone to integrated server scheduling and rate control. This study also extends the standalone problem of scheduling (see, e.g., Arapostathis et al. ^[14,15] and Atar et al. ^[10]) to the joint scheduling and control problem developed in this paper. Additionally, instead of employing joint server scheduling and rate control methods, we consider related studies on joint admission and rate control, which can be found in Kocağa ^[16].

Dynamic scheduling and control for parallel server queues in the Halfin–Whitt regime have attracted interest in both academic studies and real-world applications (see, e.g., Sze ^[17], Arapostathis et al. ^[14,15], Armony et al. ^[18], Atar et al. ^[10], Dai ^[19], Garnett et al. ^[20]), Kocağa ^[16], and Weerasinghe ^[9,21]). Practical applications include modern telephone call centers, high-performance computer systems, cloud computing, and even quantum computing-based future communication systems and internet applications (see, e.g., Dai ^[22,23,24]).

More precisely, the aim of designing the policy is to choose a suitable number of servers while balancing the system$^\prime$s inputs and outputs by considering a sequence of parallel server queues, which is diffusively scaled by the number $n$ of servers. Furthermore, the traffic intensity associated with each $n$ tends toward unity as $n$ increases. Scheduling involves allocating servers to customers, and the associated rate control is conducted by introducing a vector of state-dependent feedback control functions involving rate perturbations. When the server is interpreted as a quantum qubit, this server number is the number of qubits required to build a quantum computer (see the recent breakthrough reported in Dai ^[19]). Our research is widely applied and up-to-date and can provide valuable references for practical operation.

This paper is organized as follows. In Subsections 2.1 and 2.2, we establish the mathematical equations of the stochastic model, specify the necessary assumptions, and perform the relevant scaling of the equations. In Subsection 2.3, we present the cost function of the original queueing system. To facilitate numerical simulation, a linear version is presented, and we prove that any cost function satisfying Assumption 3 can be used. The diffusion control problem is described in Subsection 2.4, and the corresponding HJB equation is described in Subsection 4.1. We present the main theorem (Theorem 2) of this paper in Subsection 2.5, which shows the asymptotic optimality of the joint scheduling control policy that we design. In Section 3, we use several experiments to compare the system$^\prime$s performance and value functions under different control policies. Section 4 presents the proof of the theorem, Subsection 4.1 presents the proof of the existence and uniqueness of the solutions of the HJB equation, Subsection 4.2 presents the proof of asymptotic boundedness. Finally, we present the proof of Theorem 2 at the end of this paper in Subsection 4.3.

2.   System model and main theorem

The study in this section is based on the model of parallel server queues that is presented in Figure 1 and explained in the introduction of this paper. First, define ${\cal M} = \{1, 2, \cdots, m\}$ for an integer $m\in\mathbb{N} = \{1, 2, \cdots\}$, and let

where $\mathbb{R}^{m}_{+} = [0, \infty)^{m}$. Furthermore, all related $m$-dimensional vector stochastic processes are assumed to be in the Skorohod topological space ${\cal D}(\mathbb{R}^{m})$ of right-continuous functions with left limits and to have the Skorohod topology (see, e.g., Ethier and Kurtz ^[25]). In addition, we use "$\Rightarrow$" to represent the weak convergence of the processes in this space.

2.1.   The queueing system under a joint scheduling–control policy

In this subsection, we specify mathematical expressions for the queuing state, the service state, the arrival process, the service process, and the abandonment process. In this way, we outline a basic form of the system. Importantly, our design of the service rate, which depends on the overall state of the system and not on a single customer or a single server, is a unique feature of our policy.

For each $n\in\mathbb{N}$, sequences of stochastic processes in the QED (both Quality and Efficiency driven) regime, all the related processes indexed by $n$ are assumed to be defined in a complete probability space $(\Omega, \mathfrak{F}, P)$ that may depend on $n$, where $n$ denotes the number of servers. We also need to assume that the expectation of $P$ is denoted by $E$ and that $t\geq0$ (appearing below) represents the time. At time $t\geq0$, for each $k\in {\cal M}$, the number of customers of class $k$ queuing in buffer $k$ is denoted by $\Phi^{n}_{k}(t)$, which has the corresponding vector-form expression

The prime symbol used in (2.2) denotes the transpose of a vector. Similarly, $\Psi^{n}_{k}(t)$ represents the number of customers of class $k$ being served at time $t$, which has the corresponding vector-form expression

Clearly, the processes in (2.2)-(2.3) are integer-valued vector processes, and

Let $\Phi^{n}(0)$ and $\Psi^{n}(0)$ represent the corresponding initial values. Here, we assume that they are predetermined. Let $X_{k}^{n}(t)$ represent the total number of customers of class $k$ in the system at time $t$, and let $X^{n}(t)$ denote the corresponding vector form. Thus, we have

The service times of class $k$ customers for each $k\in{\cal M}$ are assumed to be exponentially distributed with the rate function $\mu^{n}_{k}(\cdot, \cdot)$, and each server is assumed to obey a work-conserving rule (i.e., a no-idling rule). Furthermore, the rate $\mu^{n}_{k}(\cdot, \cdot)$ is designed to be more realistic and efficient than in previous works, as follows. We first use the queue state to design the server scheduling policy, i.e., to determine the number of different classes of customers to be served simultaneously by $n$ servers, based on the proportion of customers in different classes of queues at time $t$. This step can be expressed as

where $\xi^{n}_{k}(\cdot):\mathbb{R}^{m}_{+}\longrightarrow\mathbb{R}_{+}$ is the proportion of customers in class $k$ to all customers in the system. Since $\Psi^{n}_{k}(t)$ must be an integer, we specify (2.6) as follows:

In what follows, we use $\xi^n(X^{n}_t) = \big(\xi^n_1(X^{n}_t), \cdots, \xi^n_m(X^{n}_t) \big)^\prime\in\mathbb{S}^m$ to denote the corresponding $m$-dimensional ratio vector. Thus, the non-negative rate function $\mu^{n}_{k}(\cdot, \cdot)$ can be given as $\mu^{n}_{k}(\xi^{n}(X^{n}_t), X^{n}_t)$ for time $t$. Hence, if we use $S^{n}_{k}(\cdot)$ to denote the standard Poisson process, at time $t$, the total number of class $k$ customers completing the service can be expressed as

The server rate control and server scheduling control in our joint scheduling–control policy are contained in $\mu^{n}_{k}(\xi^{n}(X^{n}_t), X^{n}_t)$. In addition, we denote customer admission control as $L^{n}_{k}(t), k\in {\cal M}$. Their exact form will be given in Assumption 2 in Section 2.2.

The arrival processes $\{A^{n}_{k}, k\in {\cal M}\}$ are assumed to be mutually independent renewal processes. This assumption is common in the literature (see, e.g., Atar et al. ^[10]) and is realistic. It can be described as follows. For each $k\in {\cal M}$, let $\{\hat{\tau}_{k}(j), j\in \mathbb{N}\}$ be a strictly positive i.i.d. sequence of random variables with the mean $E[\hat{\tau}_{k}(1)] = 1$ and a squared coefficient of variation

Furthermore, we assume that there is a constant $N\geq 2$ such that $E(\hat{\tau}_{k}(1))^{N} < \infty$. We can then define

where $\lambda^{n}_{k} > 0$ is the arrival rate of customers of class $k$. The notation in (2.9) is interpreted as follows: $\tilde{\tau}^{n}_{k}(1)$ represents the first arrival time of class $k$ customers, and $\tilde{\tau}^{n}_{k}(a)$ for each $a\in\{2, 3, ...\}$ represents the amount of time between the $(a-1)$th and the $a$th arrival of the $k$th customer. With the convention that $\sum^{0}_{1} = 0$, we define the number of arrivals of class $k$ customers up to time t as follows:

In addition, we assume that the rate at which class-$k$ customers abandon their queue is a constant denoted by $\theta_{k}^{n}\in[0, \infty)$. Let $\{R_{k}^{n}, k\in {\cal M}\}$ be standard Poisson processes independent of each other and of the process $\{A_k^n, \; S_k^n, \; k\in {\cal M}\}$. Then, the number of abandonments of queue $k$ up to time $t$ is given by

To cover both non-preemptive and preemptive resume policies (i.e., policies that either prevent or allow the service to be interrupted and the customer to return to the queue without completing the service), we introduce the corresponding process $D_k^n$ for each $k\in {\cal M}$. This process has three properties: $(i)$ Initially, it takes the value of zero, i.e., $D_k^n(0) = 0$; $(ii)$ it increases by 1 when a customer of class $k$ is allocated to a server; and $(iii)$ it decreases by 1 when a customer of class $k$ returns to the queue. We can then model the associated dynamic queueing system as

where

Furthermore, an induced filtration ${\cal F}^{n} = \{{\cal F}_{t}^{n}, t\geq 0\}$ with the corresponding sigma algebra ${\cal F}_{t}^{n}$ is given by

It is obvious that $X^n_k(t)$ is adapted to filtration ${\cal F}^{n}$ and that ${\cal F}_{t}^{n}$ is the information available at time $t$. In addition, let $\tau_{k}^{n}(t)$ be the first arrival time in queue $k$ that is not earlier than time $t$, i.e.,

We can then introduce the corresponding information field as follows:

On the basis of this notation, we introduce the concept of an admissible policy.

Definition 1. For each $k$ and $t$, a joint scheduling–control policy is admissible if the following three properties hold:

(1) $\mathcal{F}_t^n$ is independent of $\mathcal{G}_t^n$;

(2) $S^{n}_{k}\big(E^{n}_{k}(t)+a\big)-S^{n}_{k}\big(E^{n}_{k}(t)\big)$ is equal in law to $S^{n}_{k}(a)$;

(3) $R_{k}^{n}\big(\tilde{E}^{n}_{k}(t)+a\big)-R_{k}^{n}\big(\tilde{E}^{n}_{k}(t)\big)$ is equal in law to $R_{k}^{n}(a)$.

2.2.   Scaling under the QED regime

We first make the following heavy- traffic assumption (cf. ^{[11,20,26,27]}) for our subsequent study.

Assumption 1. There are constants $\lambda_k^{0}\in(0, \infty)$, $\mu_k^{0}\in(0, \infty)$, and $\bar{\lambda}_{k}\in(0, \infty)$ for each $k \in \cal{M}$ such that

and, as $n \rightarrow \infty$,

Second, for all $t\geq0$, we introduce the related scaling processes as follows. Let $\hat{A}_k^n(\cdot)$ be the process obtained by diffusion scaling and centering of $A_k^n(\cdot)$, which takes the form

Furthermore, let $\bar{\Phi}_k^n$ and $\bar{\Psi}_k^n$ be fluid-scaled processes defined by

Now, let

We can then introduce the corresponding diffusion-scaled processes as

In addition, there are diffusion-scaled processes as follows:

We then define

and

Next, we define

where $\hat{\xi}^{n}_{k}(\cdot):\mathbb{R}^{m}\longrightarrow\mathbb{R}_{+}$ can be approximated as the ratio of $k$ class customers among all customers in the system of $\hat{X}^{n}(t)$. Let $\hat{\xi}^n(\hat{X}^{n}_t) = \big(\hat{\xi}^n_1(\hat{X}^{n}_t), \cdots, \hat{\xi}^n_m(\hat{X}^{n}_t)\big)^{\prime}$. From (2.25), we can obtain

We can then denote $\hat{\Psi}^{n}_{k}(t)$ for each $k\in{\cal M}$ as

Remark 1. Note that $X^{n}(t)\in\mathbb{R}^{m}_{+}$, but $\hat{X}^{n}(t)\in\mathbb{R}^{m}$. The reason is that, in some cases, for a fixed $\tilde{k} \in \cal{M}$, $\Phi_{\tilde{k}}^n(t) = 0$, and $\Psi_{\tilde{k}}^{n}(t) < n\gamma_{\tilde{k}}^{0}$, we have $\hat{\Phi}_{\tilde{k}}^n(t) = 0$ and $\hat{\Psi}_{\tilde{k}}^n(t) < 0$. Consequently, $\hat{X}^{n}_{\tilde{k}}(t) < 0$, and so $\hat{X}^{n}(t)\in\mathbb{R}^{m}$.

With the notation above, (2.12) can be rewritten as follows:

where

Assumption 2. For each $k \in \cal{M}$, there are non-negative bounded functions $U_{k}^{n}(\cdot):\mathbb{R}^{m}\longrightarrow\mathbb{R}_{+}$ that satisfy $U_{k}^{n}(\cdot) = 0$ when $\xi_{k}^{n}(X_{t}^{n}) = 0$ such that

and there are non-negative bounded functions $H_{k}^{n}(\cdot):\mathbb{S}^{m}\longrightarrow\mathbb{R}_{+}$ that satisfy $H_{k}^{n}(\cdot) = 0$ when $\xi_{k}^{n}(X_{t}^{n}) = 0$ such that

where $\mu^{0}_{k} > 0$ for all $k\in{\cal M}$ are constants called the basic service rates defined in Assumption 1 (cf. Weerasinghe ^[9]); $\hat{\xi}_{k}^{n}(\cdot)$, defined in (2.25), represents the server scheduling control; $U_{k}^{n}(\cdot)$ represents customer admission control; and $H_{k}^{n}(\cdot)$ represents the server rate control.

Remark 2. A typical example of the rate control law for $\mu_{k}^{n}(\cdot, \cdot)$ in (2.8) can be designed by generalizing the discussion in Weerasinghe ^[9] for the single-class case to our multiclass case. The function $\mu_{k}^{n}(\cdot):\mathbb{R}^{m}\longrightarrow\mathbb{R}_{+}$ can take the following basic form:

Here $\mu^{0}_{k}$ for all $k\in{\cal M}$ is called the basic service rates.

Remark 3. Based on Assumption 2, (2.6), and (2.26), $\frac{1}{\sqrt{n}}E^{n}_{k}(t)$ in (2.28) can be written as

According to the notation introduced in Assumptions 1 and 2, from (2.28), we can obtain

where

Remark 4. Now, we show that (2.34) includes the case of $\xi_{k}^{n}(X_{t}^{n}) = 0$.

First, when $\xi_{k}^{n}(X_{t}^{n}) = 0$, we have $\Psi^{n}_{k}(t) = 0$, $\hat{\Psi}_k^n(t) = -\sqrt{n}\gamma_{k}^{0}$, $\hat{\xi}^{n}_{k}\big(\hat{X}^{n}_t\big) = 0$, $E^{n}_{k}(t) = 0$, and $\hat{S}^{n}_{k}\left(\frac{1}{n}E^{n}_{k}(t)\right) = 0$. Substituting these values into (2.28), we obtain

Second, when $\xi_{k}^{n}(X_{t}^{n}) = 0$, (2.34) can be written as

The two equations above have the same form, verifying that (2.34) includes the case of $\xi_{k}^{n}(X_{t}^{n}) = 0$.

The corresponding vector form of (2.34) can be written as

where $x^n = (x_{1}^{n}, \cdots, x_{m}^{n})^\prime$, $\sigma = \mbox{diag}(\sigma_{k}, \; k\in \cal{M})$ (here, "diag" means "diagonal matrix"), $\hat{W}^n(t) = (\hat{W}^n_1(t), ..., \hat{W}^n_{m}(t))^\prime$, and

Furthermore, the terms used in (2.39) have the following expressions:

At this point, our system$^{\prime}$s equations are fully developed. In this section, we scale the system equations, present some necessary heavy traffic assumptions, and, most importantly, present the specific form of the control function $\mu_{k}^{n}\big(\xi^{n}(X^{n}_t), X^{n}_t\big)$. Next, we provide the cost and value functions so that our original physical model is fully described.

2.3.   Cost and value functions

Let $\big(x^{n}, \hat{X}^n, U^n, \hat{\xi}^{n}, H^n\big)_{n\geq1}$ be as described in Section 2.2; the corresponding cost function can be defined as

where $C:\mathbb{R}^m\times\mathbb{R}^{m}\times\mathbb{R}^m\times\mathbb{S}^{m}\mapsto\mathbb{R}_{+}$ is a non-negative continuous function as follows:

where $c^{n}, p_{k}^{n}$, $s_{k}^{n}$, $q_{k}^{n}$, and $h_{k}^{n}$ are constants. This equation can be interpreted as follows. At time $t$, the number of idle servers can be represented by $n- \sum\limits_{k = 1}^{m}\Psi_{k}^{n}(t)$; subsequently, the idle server cost can be represented by $- \sum\limits_{k = 1}^{m}c_{k}^{1}\hat{\Psi}_{k}^{n}(t)$, where $c_{k}^{1}$ is a constant. The abandonment cost can be represented by $\sum\limits_{k = 1}^{m}c_{k}^{2}\theta_{k}^{n}\hat{\Phi}_{k}^{n}(t)$, where $c_{k}^{2}$ is a constant and $\theta_{k}^{n}$ is the rate at which class $k$ customers abandon their queue, as defined in Section 2.1. The delay cost can be represented by $\sum\limits_{k = 1}^{m}c_{k}^{3}\hat{\Phi}_{k}^{n}(t)$, where $c_{k}^{3}$ is a constant. These three costs can be unified in the form of $\sum\limits_{k = 1}^{m}p_{k}^{n}\hat{X}_{k}^{n}(t)$, which we call the queuing cost. $\sum\limits_{k = 1}^{m}s_{k}^{n}\hat{\xi}_{k}^{n}(t)$ represents the corresponding server scheduling costs, $\sum\limits_{k = 1}^{m}q_{k}^{n}U_{k}^{n}(t)$ represents the costs of customer admission control, and $\sum\limits_{k = 1}^{m}h_{k}^{n}H_{k}^{n}(t)$ represents the costs of server rate control. Fixed costs, such as equipment depreciation and the basic salary of personnel, can also be considered; here, they are denoted by $c^{n}$, which is a constant.

Many cost factors need to be considered in practical applications. We provide a few typical examples here. The cost factors to be considered can be increased or decreased according to the specific situation. We will show later that any cost function can be used as long as it satisfies the following assumption.

Assumption 3. We assume that function $C$ satisfies the following:

(1) There is $\delta \in (0, 1)$, and for any compact $G\subset\mathbb{R}^m$, there is $c_{0}$ depending only on $G$ such that

holds for $(U, \hat{\xi}, H) \in\mathbb{R}^{m}\times\mathbb{R}^m\times\mathbb{S}^{m}$ and $x, y \in G$.

(2) Constants $c_{1} > 0$ and $k_{C}\geq 0$ exist such that

where $\|x\| = \sum_{k = 1}^{m}|x_{k}|$, $(U, \hat{\xi}, H) \in\mathbb{R}^{m}\times\mathbb{R}^m\times\mathbb{S}^{m}$, and $x \in \mathbb{R}^m$.

Definition 2. Let $\mathfrak{D^{n}}$ be the collection of all such admissible $(\hat{X}^n, U^n, \hat{\xi}^{n}, H^{n})$; the value function $V^{n}$ is defined by

where the infimum is taken over all available processes $(\hat{X}^n, U^n, \hat{\xi}^{n}, H^{n})$ in $\mathfrak{D^{n}}$.

2.4.   The diffusion control problem

In this section, we present a diffusion control problem that can be regarded as a limiting form of the stochastic model described in Section 2.2. The use of the Brownian system as a heavy traffic approximation of a queuing system has been a long-standing idea, and the reader is referred to ^[3] and ^[28] for a list of comprehensive references.

Let us define a complete probability space, $\big(\Omega, F, P\big)$, where the expectation with respect to $P$ is denoted by $E$; in this space, all the stochastic processes below are defined. First, we make two assumptions to facilitate obtaining the corresponding diffusion control problem from the previously established joint scheduling–control problem.

Assumption 4. As $n\rightarrow \infty$, for each $k\in \cal{M}$, there are constants $\theta_k^{0}\in[0, \infty)$, $c_k\in[0, \infty)$ and $\lambda_k\in\mathbb R$ such that

Assumption 5. As $n\rightarrow \infty$, for each $k\in \cal{M}$, $t\geq0$, there are constants $\phi_k\in[0, \infty)$, $\psi_k\in\mathbb{R}$ such that

where

and $\sum_{\cal M}\psi_k\leq0$.

Next, let $x_k = \phi_k+\psi_k$; thus, we consider a controlled state process $X_k(t)$, which is a weak solution to the following equation:

where $U_{k}(\cdot):\mathbb{R}^{m}\longrightarrow\mathbb{R}_{+}$, $H_{k}(\cdot):\mathbb{S}^{m}\longrightarrow\mathbb{R}_{+}$, $\xi_{k}(\cdot):\mathbb{R}^{m}\longrightarrow\mathbb{R}_{+}$, $\sum_{k = 1}^{m}\xi_{k}(\cdot) = 1$ and

Now, we can obtain

where $x = (x_{1}, \cdots, x_{m})^\prime$, $\sigma = diag(\sigma_{k}, \; k\in \cal{M})$, $W(t) = (W_{1}(t), \cdots, W_{m}(t))^\prime$, and

Furthermore, the terms used in (2.46) have the following expressions:

Assumption 6. To ensure that $X_{k}(\cdot)$ in (2.44) does not explode, we assume that there exists a constant $M\geq 0$ such that

where $k\in \cal{M}$.

Now, we introduce the following cost function for the state Eq (2.45):

where $C:\mathbb{R}^m\times\mathbb{R}^{m}\times\mathbb{R}^m\times\mathbb{S}^{m}\mapsto\mathbb{R}_{+}$ is a non-negative continuous function as follows:

where $c, p_{k}$, $s_{k}$, $q_{k}$, and $h_{k}$ are constants.

In Section 3, we present several numerical examples in which the cost functions refer to (2.50). The costs presented here are not used in every example. We provide this form to facilitate some of the numerical simulations given in Section 3. Practical applications can be modified to suit a given situation, and we will show later that any cost function that satisfies Assumption 3 is usable.

Definition 3. Given an initial state $x$, we call

an admissible control system if

(1) $\big(\Omega, F, (F_t), P\big)$ is a complete filtered probability space;

(2) $(U, \xi, H)\in\mathbb{R}^{m}\times\mathbb{R}^m\times\mathbb{S}^m$ is an F-measurable, $(F_t)$-progressively measurable process;

(3) W is standard m-dimensional $(F_t)$-Brownian motion.

Definition 4. Let $\mathfrak{D}$ be the set of all such admissible $(X, U, \xi, H)$. The value function for the control problem is defined as follows:

where the infimum is taken over all available processes $(X, U, \xi, H)$ in $\mathfrak{D}$.

Theorem 1. Let the system equation be (2.45) and the cost function be (2.49); then, there is a Markov control policy that is optimal for all $x\in \mathbb{R}^{m}$.

Theorem 1 is an integral part of Theorem 3, which states that the value function $V$ is the unique solution of the corresponding HJB equation. For this reason, Theorem 1 holds without proof as long as Theorem 3 is proved later. We present Theorem 1 here to facilitate the construction of an asymptotically optimal policy for our original joint scheduling–control problem in the next subsection.

2.5.   An asymptotically optimal ioint scheduling–control policy

In this section, we present an asymptotically optimal control policy for the original problem, which is defined by the optimal Markov control policy described in Theorem 1.

To state this result, we need to introduce a Markov control policy, namely $\widetilde{\eta} = \big(\widetilde{U}^n, \widetilde{\xi}^{n}, \widetilde{H}^n\big)$, where $\widetilde{U}^n = U^{*}, \widetilde{\xi}^{n} = T(\xi^{*}), \widetilde{H}^n = H^{*}$, $T$ is a measurable map that guarantees that the components of $\Psi^{n}(t)$ are integer-valued, and $\eta^{*} = (U^{*}, \xi^{*}, H^{*})$ is the optimal Markov control policy described in Theorem 1. Let $\Psi^{n, *} = n\xi^{*}$ and $\widetilde{\Psi}^{n} = n\widetilde{\xi}^{n} = nT(\xi^{*})$, and define

Clearly, $\sum\limits_{k = 1}^{m}\Psi_{i}^{n, *} = \sum\limits_{k = 1}^{m}\widetilde{\Psi}_{k}^{n} = n$ and $\widetilde{\Psi}_{k}^{n}\in \mathbb{Z}_{+}$. Moreover, $\parallel\widetilde{\Psi}^{n}-\Psi^{n, *}\parallel\leq 2m$.

Now, for a given initial condition $x^{n}$ and a stochastic sequence system, we write

where $J^{n}$ is defined in (2.39) and represents the cost function for the joint scheduling–control problem.

Theorem 2. Let $\big(x^{n}, \hat{X}^n, U^n, \hat{\xi}^{n}, H^n\big)_{n\geq1}$ be any sequence of the stochastic system described in Section 2.2, let $\big(J^n(x^{n}, \hat{X}^n, U^n, \hat{\xi}^{n}, H^n)\big)_{n\geq1}$ be the associated cost function defined in (2.39) that satisfies Assumption 3, and let $X_{k}^{n}\geq\widetilde{\Psi}_{k}^{n}(k\in \cal{M})$. We thus obtain

Theorem 2 is the main theorem of this paper, which states the asymptotic optimality of the joint scheduling–control policy $\widetilde{\eta} = \big(\widetilde{U}^n, \widetilde{\xi}^{n}, \widetilde{H}^n\big)$ by using the optimal Markov policy $\eta^{*} = (U^{*}, \xi^{*}, H^{*})$. Since the proof of this theorem is complicated and requires the assistance of lemmas, it is presented in Section 4.

3.   Numerical simulations for a two-class model

In this section, we present several comparative experiments to compare the system$^{\prime}$s performance and value functions under different control policies. The results show that our designed joint scheduling–control policy has significant advantages in reducing the system$^{\prime}$s cost and improving service efficiency. This numerical simulation not only confirms the rationality and practicality of the previous theoretical analysis but also provides a valuable reference for practical applications.

The diffusion control problem is a stochastic differential equation, which can be regarded as the limiting case of our original problem. $U$, $\xi$, and $H$ in (2.46) can be regarded as the customer admission control function, the server scheduling control function, and the server rate control function, respectively, in the diffusion control problem. Therefore, in this section, we provide a specific example of the diffusion control problem and some corresponding examples to illustrate the superiority of our model.

Example 1 (Two-class model with our joint scheduling–control policy). For a numerical example, we refer to (2.44) and (2.45) in Section 2.4, with $m = 2$, and consider a system of our joint scheduling–control policy with two customer classes as follows:

as well as the following parameter values:

where $\lambda_{k}, k = 1, 2$ can be regarded as the arrival rate of customers of class $k$, $\theta_{k}, k = 1, 2$ can be regarded as the abandon rate of customers of class $k$, $c_{k}, k = 1, 2$ can be regarded as the server scheduling control rate of customers of class $k$, $\mu^{0}_{k}, k = 1, 2$ can be regarded as the basic service rates defined in Assumption 1, and $\sigma_{k}, k = 1, 2$ can be regarded as the diffusion coefficient defined in (2.34).

Now, we refer to Eq (2.50) to define the relevant cost function of this model as follows:

with the parameter values

The constant $1$ in (3.2) represents the fixed costs, such as equipment depreciation and the basic salary of personnel; $p_{k}x_{k}, \; k = 1, 2$ represents the queuing costs; $s_{k}\xi_{k}, \; k = 1, 2$ represents the corresponding server scheduling costs; $q_{k}U_{k}, \; k = 1, 2$ represents the costs of customer admission control; and $h_{k}H_{k}, \; k = 1, 2$ represents the costs of server rate control.

To facilitate calculation, we refer to Eq (2.51) in Section 2.4 and define a deformation of the value function as follows:

Through the HJB equation, we obtain an optimal policy, i.e., the server scheduling control function, the customer admission control function, and the server rate control function, as follows:

Given that $x_{1}(0) = 2$ and $x_{2}(0) = 3$, we use the Euler–Maruyama (EM) method, which is a standard numerical tool for solving SDEs. With MATLAB (cf. ^[29], ^[30], ^[31]), we obtain $V_{12} = 4.0850$. The plot of this two-class model (Figure 3) shows that the sizes of the two customer classes are more balanced because of customer admission control, server scheduling control, and server rate control. Cases in which the number of one type of customer is large and the number of the other type is small are relatively rare. Additionally, the numbers of both classes of customers decline in the later period. The plot of the cost function $C_{12} = C_{12} \big(X_{t}, U_{t}, \xi_{t}, H_{t} \big)$ (Figure 4) shows that the cost function of this two-class model increases rapidly in the initial stage but then continues to decline. These two figures tentatively illustrate that the joint scheduling–control policy we have devised is effective.

Below, we present five other models (i.e., without control policy, with customer admission control only, with server scheduling control only, with customer admission control and server scheduling control, and with server scheduling control and server rate control) to illustrate the superiority of our joint scheduling–control policy.

Example 2 (Two-class model without control policy). We present a system without any control as follows:

We use the following parameter values:

Given $x_{3}(0) = 2$ and $x_{4}(0) = 3$, we can obtain an operational image of this system (Figure 5), where the total trend of the number of customers in both categories is increasing. This trend shows that without the control function, the system is very inefficient, and customers will increasingly accumulate.

Next, we define the corresponding cost function of this example as follows:

where $p_{1} = 0.50$ and $p_{2} = 1.00$; the plot of the cost function $C_{34} = C_{34} \big(X_{t}\big)$ (Figure 6) shows that the cost of this model continues to increase, which is not desirable.

The value function is

A comparison of Figures 3 and 5 reveals that the sizes of the two classes of customers in the model of our joint scheduling–control policy are more balanced because of customer admission control, server scheduling control, and server rate control. Cases in which the number of one type of customer is large and the number of the other type is small are relatively rare, as shown in Figure 3. In Figure 3, the numbers of both classes of customers decline in the later period, while in Figure 5, the decline is not obvious. A comparison of Figures 4 and 6 reveals that the cost function of our model increases rapidly in the initial stage but then continues to decline. However, the cost function in Figure 6 does not decline significantly after increasing to a certain extent. The size of the value function is more convincing: $V_{12} = 4.0850$, $V_{34} = 4.9069$. We compare the value functions of the five examples in Figure 2.

Example 3 (Two-class model with customer admission control only). We present a system with customer admission control only as follows:

with the parameter values

We define the customer admission control functions $U_{1}$ and $U_{2}$ in the same way as before; we assume that $x_{5}(0) = 2$ and $x_{6}(0) = 3$ are given.

We define the relevant cost function of this example as follows:

where $p_{1} = 0.50$, $p_{2} = 1.00$, $q_{1} = 0.25$, and $q_{2} = 1.00$.

The value function is

Moreover, as seen from the system's operating diagram (Figure 7) and the image of the cost function (Figure 8), this model is clearly not as effective as the one that uses our joint scheduling–control policy.

Example 4 (Two-class model with server scheduling control only). We present a system with server scheduling control only as follows:

with the parameter values

We define the server scheduling control functions $\xi_{1}$ and $\xi_{2}$ as previously described and assume that $x_{7}(0) = 2$ and $x_{8}(0) = 3$ are given.

Now, we define the relevant cost function of this model as follows:

with $p_{1} = 0.50$, $p_{2} = 1.00$, $s_{1} = 0.25$, and $s_{2} = 1.00$.

The value function is

Moreover, as seen from the system's operating diagram (Figure 9) and the image of the cost function (Figure 10), this model is clearly not as effective as the one that uses our joint scheduling–control policy.

Example 5 (Two-class model with customer admission and server scheduling control). We present a system with customer admission control and server scheduling control as follows:

with the parameter values

We define the server allocation functions $\xi_{1}$, $\xi_{2}$, $U_{1}$, and $U_{2}$ as previously described and take $x_{9}(0) = 2$ and $x_{10}(0) = 3$ as given.

Now, we define the relevant cost function of this model as follows:

with $p_{1} = 0.50$, $p_{2} = 1.00$, $s_{1} = 0.25$, $s_{2} = 1.00$, $q_{1} = 0.25$, and $q_{2} = 1.00$.

The value function is

Moreover, as seen from the system's operating diagram (Figure 11) and the image of the cost function (Figure 12), this model is clearly not as effective as the one that uses our joint scheduling–control policy.

Example 6 (Two-class model with server scheduling and server rate control). We present a system with server scheduling control and server rate control as follows:

with the parameter values

We define the server allocation functions $\xi_{1}$, $\xi_{2}$, $H_{1}$, and $H_{2}$ as previously described and take $x_{11}(0) = 2$ and $x_{12}(0) = 3$ as given.

We define the relevant cost function of this model as follows:

with $p_{1} = 0.50$, $p_{2} = 1.00$, $s_{1} = 0.25$, $s_{2} = 1.00$, $h_{1} = 0.25$, and $h_{2} = 1.00$.

The value function is

Moreover, as seen from the system's operating diagram (Figure 13) and the image of the cost function (Figure 14), this model is clearly not as effective as the one that uses our joint scheduling–control policy.

Compared with other policies, our designed joint scheduling–control policy has the lowest cost, as shown by the leftmost blue rectangle in Figure 2. The results show that our designed joint scheduling–control policy has significant advantages in reducing the system$^{\prime}$s costs and improving service efficiency. This numerical simulation not only confirms the rationality and practicality of the previous theoretical analysis but also provides a valuable reference for practical applications.

4.   Proof of the main theorem

In this section, we analyze in depth the existence and uniqueness of the solution of the HJB equation for the diffusion control problem presented in Section 2.4, which provides a solid theoretical foundation for designing an asymptotically optimal control policy for the original problem. Then, the asymptotic optimality of the joint scheduling control policy for multiclass parallel queuing models is proved in detail through the proof of several lemmas.

4.1.   The HJB equation

We prove the existence of a unique solution to the corresponding Hamilton–Jacobi–Bellman (HJB) equation, which is the optimal Markov control policy for the diffusion control problem. Theorem 3, which is proven in this section, is a complete version of Theorem 1 in Section 2.4. Thus, proving Theorem 3 effectively proves Theorem 1.

First, we introduce $\mathcal{H}:\mathbb{R}^m\times\mathbb{R}^m\mapsto\mathbb{R}$ as follows:

where $b(x, U, \xi, H)$, defined in (2.46), is the corresponding vector form of $b_{k}(x, U, \xi, H)$ and $C(x, U, \xi, H)$ is defined in (2.49). The corresponding HJB equation (cf. ^[13]) can generally be written in the following form:

where $\sigma_k$ is defined in (2.44) and $\alpha$ is defined in (2.49).

Remark 5. The system equation is

The cost function is

We define

The HJB equation for $V(x) = \inf\limits_{(U, \xi, H)\in\mathbb{R}^{m}\times\mathbb{R}^m\times\mathbb{S}^m}J\big(x, X, U, \xi, H\big)$ is

Theorem 3. There exists a classical solution $f\in C^{2}(\mathbb{R}^{m})$ to the HJB equation (4.2), and we have the following:

(i) Constants $c > 0$ and $k > 0$ exist such that

(ii) The solution $f$ is unique, and

i.e., there is an optimal Markov control policy for all $x\in \mathbb{R}^{m}$.

Before this theorem is proven, we need to introduce several lemmas.

Lemma 1. The Hamiltonian $\mathcal{H}(x, h)$, which is defined in (4.1), is Hölder continuous with the exponent $\delta$, where $\delta$ is defined in Assumption 3.

Proof. For $\varepsilon > 0$, $\forall y\in \mathbb{R}^m$, $\forall h_{1}\in \mathbb{R}^m$, by the infimum definition of $\mathcal{H}(x, h)$, $(U_{0}, \xi_{0}, H_{0})\in\mathbb{R}^{m}\times\mathbb{R}^m\times\mathbb{S}^m$ exists such that

Additionally, with the infimum definition of $\mathcal{H}(x, h)$, for $\forall x\in \mathbb{R}^m$, $\forall h_{2}\in \mathbb{R}^m$, and for the same $(U_{0}, \xi_{0}, H_{0})\in\mathbb{R}^{m}\times\mathbb{R}^m\times\mathbb{S}^m$ in (4.5), we can obtain

According to the above two inequalities, we can derive the following:

where the last inequalities use the Lipschitz property of $b$ and the Hölder property of $C$. Since $\varepsilon > 0$ is arbitrary, $\mathcal{H}(x, h)$ is Hölder continuous with the exponent $\delta$. □

Lemma 2. For any $x\in\mathcal{R}^{m}$ and an admissible system $\pi$, with $X$ as the controlled process associated with $x$ and $\pi$, we have

where $N\in\mathbb{N}$ and the constants $c_{N}$ are independent of $\pi, x$, and $t$.

Proof. Equation (2.44) can be written as follows:

Next, for all $t\geq 0$, by Theorem 5.2.5 of ^[32], we define a process $\widetilde{X}_{k}$ as the unique solution to the equation

Letting $\widetilde{X} = \big(\widetilde{X}_1, \cdots, \widetilde{X}_m \big)\prime$, we have

where

and $M \geq 0$ is as given in Assumption 6. Additionally,

In this case, a constant $K\geq 0$ exists such that

for every $0\leq t < \infty$ and $x\in\mathbb{R}^m, y\in\mathbb{R}^m$. That is, the coefficients $\widetilde{b}, \; \sigma$ satisfy the global Lipschitz and linear growth conditions.

For every $T > 0$, according to Theorem 5.2.9 of ^[32], a positive constant $c_{K, T}$ depending only on $K$ and $T$ exists such that

Since $\widetilde{X}_{k}$ is Gaussian, we have

Next, we aim to show that $|{X}_{k}(t)|\leq2|\widetilde{X}_{k}(t)|$, so we introduce $Y_k(t) = X_k(t)-\widetilde{X}_{k}(t)$; we then have

The process $\widetilde{U}_{k}^{1}$ is given by

The process $\widetilde{U}_{k}^{2}$ is given by

The process $\widetilde{U}_{k}^{3}$ is given by

where $M$ is defined in Assumption 6. Next, we introduce a non-negative, differentiable function F on $\mathbb{R}$ such that

where $K = |\widetilde{X}_{k}(t)|$.

We then have

If $Y_{k}(t) < -K$, then $X_{k}(t) < \widetilde{X}_{k}(t)-|\widetilde{X}_{k}(t)|\leq0.$ Using(4.13)–(4.15), we have $[\theta_k^{0}Y_k(t)+c_{k}\widetilde{U}_{k}^{1}(s)+\mu_{k}^{0}\widetilde{U}_{k}^{2}(s)+\mu_{k}^{0}\widetilde{U}_{k}^{3}(s)]\leq0.$ Consequently,

If $Y_{k}(t) > K$, then $X_{k}(t) > \widetilde{X}_{k}(t)+|\widetilde{X}_{k}(t)|\geq0.$ Using (4.13)–(4.15), we have $[\theta_k^{0}Y_k(t)+c_{k}\widetilde{U}_{k}^{1}(s)+\mu_{k}^{0}\widetilde{U}_{k}^{2}(s)+\mu_{k}^{0}\widetilde{U}_{k}^{3}(s)]\geq0.$ Consequently,

If $|Y_{k}(t)|\leq K$, we have $F'\big(Y_{k}(t)\big) = 0$.

By combining these facts, we obtain $F\big(Y_{k}(t)\big)\leq0$, so $F(x) = 0$ only on $[-K, K]$, which yields $|Y_{k}(t)|\leq K$. Consequently,

Using (4.11), it follows that $E_{x}^{\pi}\parallel X(t)\parallel^{N}\leq c_{N}(1+\parallel x \parallel^{N}).$ □

Proof of Theorem 3. We first use a standard truncation idea (cf. ^[33]), which enables us to study a sequence of quasilinear Partial Differential Equations (PDEs) with a Dirichlet boundary condition.

Fix $i\in \mathbb{N}$, and let $B(0, i) = \{x:\; \parallel x \parallel\; \leq i\}$. Then, a policy $\pi$ and an initial condition $X(0) = x\in B(0, i)$ are fixed. In addition, set $T_{i}^{\pi} = \inf\{t\geq0:X(t)\in \partial B(0, i)\}$. Next, we consider the diffusion process $X$, which satisfies (2.45) and terminates at the boundary of $B(0, i)$.

Let

Let $\mathfrak{D_{i}}$ be the set of all such admissible $(X, U, \xi, H)$; the value function $\mathcal{V}_{i}$ is defined as

where the infimum is taken over all the available processes $(X, U, \xi, H)$ in $\mathfrak{D}_{i}$.

Assuming that $\overline{f}(x) = 0, x\in\partial B(0, i)$ and using Theorem 8.3 of ^[34] and Lemma 1 (for details, see ^[26]), we find that $\mathcal{V}_{i}(x)$ is the unique bounded solution in $C^{2}(\mathcal{R}^{m})$ of

where $\mathcal{H}(x, h)$ is defined in (4.1).

First, we prove that the function $V(x)$ satisfies the growth condition. In fact, from the polynomial condition on $C(X_{t}, U_{t}, \xi_{t}, H_{t})$ in Assumption 3, using Fubini's theorem, we can obtain

where $c$ is a constant that is independent of $i$, and $k_{C}$ is defined in Assumption 3. Now, we appeal to Lemma 2; thus, we can obtain $V(x)\leq c(1+\|x\|^{k_{C}})$ for all $x\in \mathbb{R}^{m}$. Note that $\mathcal{V}_{i}\leq V(x)$, which implies a uniform bound on $\mathcal{V}_{i}$.

We then fix $j > 0$ and set $B(0, j) = \{x:\; \parallel x \parallel\; \leq j\}$; let $\Delta_{\sigma}$ denote the second-order operator in the HJB equation with the weights $\sigma_{i}^{2}$. Similar to Step 2 in the proof of Theorem 1 in ^[26], we can see that the families $\{\mathcal{V}_{i}\}$, $\{\nabla \mathcal{V}_{i}\}$, and $\{\Delta_{\sigma}\mathcal{V}_{i}\}$ are equicontinuous and bounded. Therefore, according to the Arzela–Ascoli Theorem (cf. ^[35]), convergent subsequences of $\{\mathcal{V}_{i}\}$, $\{\nabla \mathcal{V}_{i}\}$, and $\{\Delta_{\sigma}\mathcal{V}_{i}\}$ exist. For brevity, we also denote these convergent subsequences with the subscript $i$. Then, there is a $V(x)$, which satisfies $\mathcal{V}_{i}\rightarrow V$, $\nabla \mathcal{V}_{i}\rightarrow \nabla V$, and $\Delta_{\sigma}\mathcal{V}_{i}\rightarrow \Delta_{\sigma}V$ uniformly on $B(0, j)$.

The improved smoothness of $V(x)$ is then obtained with the standard PDE arguments (cf. ^[34]). $\mathcal{V}_{i}$ adheres to the Hamilton–Jacobi–Bellman (HJB) equation, complete with its specified boundary conditions, and converges uniformly to $V$ within the ball $B(0, j)$. Utilizing Lemma 1, it is feasible to transfer these limits through the modified version of the HJB equation (4.19), thereby confirming that $V(x)$ is in accordance with the original HJB equation $V(x)$ on $B(0, j)$. Given that the choice of $j$ is unrestricted, it follows that it is compliant with the original HJB equation across the entire space $\mathbb{R}^{m}$. Furthermore, the definitions of $\mathcal{V}_{i}$ and $V(x)$ indicate that the principle of monotonic convergence dictates

Thus, the proposed limit $V(x)$ is the value function of the diffusion control problem, implying that the value function $V(x)$ is a classical solution to the HJB equation (4.2).

The uniqueness of $V(x)$ and the existence of optimal Markov control policies for the problem remain to be shown. Let $\widetilde{f}(x)\in C^{2}(\mathbb{R}^{m})$ be any solution to the HJB equation (4.2), and fix a policy $\pi\in\Pi$, assuming that $\widetilde{f}(x)$ satisfies the growth condition. Now, by applying the Itô formula to $e^{-\alpha t}\widetilde{f}(X_{t})$, we can obtain the following (for details, see Theorem 5.1 of ^[13]):

Consequently, when $\widetilde{f}(x)\leq c(1+\|x\|^k)$, the last term on the right-hand side of the inequality above converges to zero. Thus, $\widetilde{f}(x)\leq J(x, X, U, \xi, H)$, and because $\pi$ is arbitrary, we have $\widetilde{f}(x)\leq V(x)$.

Let

This result is obviously a solution to Eq (4.19). By repeating our treatment of $\mathcal{V}_{i}$, we can obtain $\widetilde{f}_{i}(x)\rightarrow V(x)$. Hence, $\widetilde{f}(x) = V(x)$ on $\mathbb{R}^{m}$; i.e., $V(x)$ is the unique solution of the HJB equation (4.2), and there is a Markov control policy that is optimal for all $x\in \mathbb{R}^{m}$. □

4.2.   Asymptotic bound

In this section, we show that the value function $V$ of the diffusion control problem is an asymptotic lower bound for the value function ${V}^{n}$ of the original problem.

Theorem 4. Let $\big(x^{n}, \hat{X}^n, U^n, \hat{\xi}^{n}, H^n\big)_{n\geq1}$ be any sequence of the stochastic system, and let the associated cost function be $\left(J^n\big(x^{n}, \hat{X}^n, U^n, \hat{\xi}^{n}, H^n\big)\right)_{n\geq1}$, as described in Section 2.3, and let $\underline{V}^{n}$ be as defined in (2.53) in Section 2.5. Assuming that $x^{n}\rightarrow x$, $V$ is the value function of the diffusion control problem given in Section 2.4.

We then have

Before proving this theorem, we need to introduce several lemmas.

Lemma 3. Assuming that for $x^{n}\rightarrow x$, the processes $\hat{X}^{n}$ described in Theorem 4 are applied, and we have

where $N\in\mathbb{N}$ and the constants $c_{N}$ are independent of $n, x$, and $t$.

Proof. Analogous to the proof of Lemma 2, we can obtain

where

$\tilde{I} = \hat{\lambda}_{k}^{n}-\sqrt{n}(\mu_{k}^{0}-\theta_{k}^{n})I_{[\hat{X}_{k}^{n}(s) < 0]}-2\mu_{k}^{0}MI_{[\hat{X}_{k}^{n}(s) < 0]}$, and $M\geq0$ is the bound of $U_{k}^{n}(\cdot)$ and $H_{k}^{n}(\cdot)$ on $\mathbb{R}$. Analogous to proving Proposition 4 in ^[10], we obtain

where $Z^{n}(t) = (Z^{n}_{1}, Z^{n}_{1}, \cdots, Z^{n}_{m})'$ and $c$ is independent of $n$ and $t$. Solving for $Z_{k}^{n}(t)$ in (4.23), we obtain

Therefore,

By

and letting $a^{n}(s) = \max_{k}[n^{-1}\theta_{k}^{n}(X_{k}^{0, n}+A_{k}^{n}(s))]$, we can obtain

Given Lemma 2 of ^[10], under the assumption in the definition of $\{A^{n}_{k}, k\in {\cal M}\}$ that there is a constant $N\geq 2$ such that $E(\hat{\tau}_{k}(1))^{N} < \infty$, then by the definition of $\hat{A}^{n}(t)$, we can obtain

where $c$ is independent of $n$ and $t$.

For the (disjunctive) martingale $\hat{S}^{n}$, we apply Burkholder's inequality (see ^[36], p. 175). Letting $\alpha_{k}^{n}(t)$ be a Poisson random variable with the parameter $2n\mu_{k}^{0}t$, we can obtain

where $c$ is independent of $n$ and $t$, and $[\hat{S}_{k}^{n}]$ denotes the quadratic variation processes associated with $\hat{S}_{k}^{n}$.

Similarly, we can obtain $E \sup_{s\leq t}|\hat{R}_{k}^{n}(t)|^{N}\leq ct^{N/2}$. By the independence of $A^{n}$ and $R^{n}$, we can obtain

where $c$ is independent of $n$ and $t$.

By applying Minkowski's inequality to (4.26), we can obtain

where $N$ and $c$ do not depend on $n$ or $t$.

According to (4.25), this lemma follows. □

Lemma 4. As $n\rightarrow \infty$, the following results hold for each $k\in \cal{M}$:

where $A_{k}$ is a Brownian motion with zero drift, the variance matrices $\lambda_{k}C^2_{\tau, k}$, $S_{k}, R_{k}$ are standard Brownian motions; and $A_{k}, S_{k}, R_{k}$ are independent of each other.

Proof. Applying Theorem 17.3 in ^[37] to $\hat{A}^n_k$, it directly follows that $\hat{A}^n_k \Longrightarrow A_k$, where $A_{k}$ is a Brownian motion with zero drift and the variance matrices $\lambda_{k}C^2_{\tau, k}$. Additionally, applying this theorem to $\hat{S}^n_k$ and $\hat{R}^n_k$, we can find that $\hat{S}^n_k \Longrightarrow S_k$ and $\hat{R}^n_k \Longrightarrow R_k$, where $S_{k}, R_{k},$ are standard Brownian motions. According to Lemma 3, we obtain

Because

the previous results, we can obtain $\frac{\hat{W}^{n}}{\sqrt{n}}\Longrightarrow 0$. It is easy to show that $\frac{x^{n}}{\sqrt{n}}\Longrightarrow 0$. By Gronwall's lemma, for any t, we can obtain $\frac{ \sup\limits_{s\leq t}\parallel Z^{n}(t)\parallel}{\sqrt{n}}\rightarrow 0$ in the distribution. Therefore, $\frac{Z^{n}(t)}{\sqrt{n}}\Longrightarrow 0$. Consequently, $\frac{\hat{X}^{n}}{\sqrt{n}} = \bar{X}^{n}- \gamma^{0}\Longrightarrow 0$, where $\bar{X}^{n} = \frac{X^{n}}{n} = \bar{\Phi}^n+\bar{\Psi}^n$. Note that $\sum\limits_{1}^{m}\bar{\Phi}^n_k = \big(\sum\limits_{1}^{m}\bar{X}^n_k-1\big)^{+}$. Then, by Assumption 1, we have $\sum_{k = 1}^{m}\lambda_k^{0}\setminus{\mu_k}^{0} = 1$. Because $\gamma_k^{0} = \lambda_k^{0}/\mu_k^{0}$, we can obtain $\sum\limits_{1}^{m}\bar{\Phi}^n_k\Longrightarrow 0$. Since $\bar{\Phi}^n_k\geq 0$, $\bar{\Phi}^n_k \Longrightarrow 0$. □

Lemma 5. Write

Then, the sequence $(\hat{X}^n, \hat{W}^n, \hat{B}^{n}, \hat{C}^{n})$ is tight in the space $\left(\mathbb{D}(\mathbb{R}^{m})\right)^{4}$.

Proof. By Lemma 4 and a time change lemma (cf. ^[37]), we can see that

Because

it follows that

where $A = (A_{1}, \cdots, A_{m})$ and $A_{k}, k\in\cal{M}$ are as described in Lemma 4. They are tight because $\hat{W}^n$ is relatively compact, and from Theorem 16.10 of ^[37], we can obtain the following:

(i) For each $m$,

(ii) For $1\leq i\leq v$, let $[t_{i-1}, t_{i})$ be decompositions of $[0, m)$ such that $t_{i} - t_{i-1} > \delta$ and let $\omega_{m}^{'}(x, \delta) = \inf\max_{1\leq i\leq v}\omega(x, [t_{i-1}, t_{i}))$; the infimum extends over all decompositions $[t_{i-1}, t_{i})$. For each $m$ and $\varepsilon$,

Since

noting that the Lipschitz property holds uniformly for the functions $\hat{X}^n_{s}\rightarrow b^{n}\big(\hat{X}^n_{s}, U^n_{s}, \hat{\xi}^{n}_{s}, H^n_{s}\big)$ in $\hat{X}^n, U^n, \hat{\xi}^{n}, H^{n}$ and $n$, we can obtain

Analogous to the proof of Lemma 4 in ^[10], we have the tightness of $\hat{X}^n$. Now, by Theorem 16.10 of ^[37], it is easy to obtain (cf.^[10]) the tightness of $\hat{B}^{n}$ and $\hat{C}^{n}$. □

Lemma 6. Denote $(X, W, B, C)$ as a limit point of $(\hat{X}^n, \hat{W}^n, \hat{B}^{n}, \hat{C}^{n})$ along a subsequence. In this case, $X, B$ and $C$ have continuous sample paths, and $B$ has the sample paths of bounded variation over finite time intervals. Denote $(F_{t})$ as the filtration generated by $(X, W, B)$; in that case, $W$ is an $(F_{t})$-standard Brownian motion.

Proof. Analogous to the proof of Lemma 6 in ^[10], this lemma is easy to obtain. Since $\hat{B}^{n}$ and $\hat{C}^{n}$ have continuous sample paths, by ^[25], we can see that processes $B$ and $C$ have continuous sample paths. By Lemma 5, we have $\hat{W}^n(t)\Rightarrow W$, so $X = x + \sigma W + B$ since $\hat{X}^n(t) = x^{n} + \sigma\hat{W}^n(t)+\hat{B}^{n}$. We write $\hat{B}^{n} = \hat{B}^{n, +} - \hat{B}^{ n, -}$, where $\hat{B}_{k}^{n, +}(t) = \int_{0}^{t}\big(\hat{B}_{k}^{n}(s)\big)^{+}ds$ and $\hat{B}_{k}^{n, -}(t) = \int_{0}^{t}\big(\hat{B}_{k}^{n}(s)\big)^{-}ds$. By the definition of $b^{n}$ in Section 2.2, we have

where $c$ is independent of $t, n$. Thus, the tightness of $\big(\hat{B}^{n, +}, \hat{B}^{ n, -}\big)$ follows from the tightness of $\hat{X}^n$. Since $\hat{B}^{n, +}$ and $\hat{B}^{n, -}$ have continuous sample paths, let $\big(B^{+}, B^{-}\big)$ denote any subsequential limit point in $\left(\mathbb{D}(\mathbb{R}^{m})\right)^{2}$. We can see that $B^{+}$ and $B^{-}$ have continuous sample paths; therefore, $B = B^{+} - B^{-}$. Since $B^{+}$ and $B^{-}$ have nondecreasing sample paths and $t$ is arbitrary, $B$ has sample paths of bounded variation over finite time intervals.

By Lemma 5, we have $\hat{W}^n(t)\Rightarrow W$, where $W$ is a standard Brownian motion. By the definitions of $W$ and $(F_{t})$, $W$ is adapted to $(F_{t})$. Next, we show that $F_{t}$ is independent of $\sigma\{W_{t+a}-W_{t} : a > 0\}$ for each $t$. In fact, we fix a $t \geq 0$ and let

where $a \geq 0$, $E^{n}_{k}(t)$ and $\tilde{E}^{n}_{k}(t)$ are as defined in Section 2.1. In addition, we have

We then, let

where $\tau_{k}^{n}(t) = \inf\{a\geq t:A_{k}^{n}(a)-A_{k}^{n}(a-) > 0\}$. We then have

Let $Y^{n} = (\hat{X}^n_{s}, \hat{W}^n_{s}, \hat{B}^{n}_{s})$, $Y = (X_{s}, W_{s}, B_{s})$, $0 \leq s \leq t$, $F:(\mathbb{R}^{m})^{3}\rightarrow \mathbb{R}^{m}$, and $G:\mathbb{R}^{m}\rightarrow \mathbb{R}^{m}$. By the definitions of $\mathcal{F}_t^n$ and $\mathcal{G}_t^n$, we can conclude that $Y^{n}$ and $\alpha^{n}$ are measurable on $\mathcal{F}_t^n$ and $\mathcal{G}_t^n$, respectively. By Definition (1), we can obtain

According to the random change of the time lemma (cf.^[37]), $\tau_{k}^{n}(t)$ converges in distribution to 0, and thus, $\hat{A}_{k}^{n}$ converges in distribution to a continuous process. Thus, we can see that $\beta_{k}^{n}$ converges in distribution to 0. Therefore, $\alpha_{k}^{n}$ converges in distribution to $W_{t+a}-W_{t}$. According to the continuous mapping theorem, $Y^{n} = (\hat{X}^n, \hat{W}^n, \hat{B}^{n})\Rightarrow Y = (X, W, B)$, we can obtain

Since $a, s$ are arbitrary, according to the Dynkin class theorem, $F_{t}$ is independent of $\sigma\{W_{t+a}-W_{t} : a > 0\}$. Thus, W is a $(F_{t})$- Brownian motion because $t$ is arbitrary. □

Proof of Theorem 4. Let $f$ be the solution of the HJB equation (4.2) described in Theorem 3. From (4.1), we can obtain

Moreover, we have

By the definitions of $b(X, U, \xi, H)$ and $b^{n}(\hat{X}^{n}, U^{n}, \hat{\xi}^{n}, H^{n})$ and Assumption 6, we have

where $\varepsilon_{n}\rightarrow0$. Therefore, by Lemma 3 and the continuous mapping theorem, we have

We set $(X, W, B, C)$ as a weak limit point of $(\hat{X}^n, \hat{W}^n, \hat{B}^{n}, \hat{C}^{n})$, and set $(F_{t})$ as the filtration generated by $(X, W, B)$. Since $X_{t} = x + \sigma W_{t} + B_{t}$, by Lemma 6, $W$ is an $(F_{t})$-standard Brownian motion and $B$ has the sample paths of bounded variation over finite time intervals. Therefore, for $n\rightarrow \infty$, we apply Lemma 5 of ^[10] with $U^{n} = e^{-\alpha s}\nabla f(\hat{X}^{n}_{s})$, $V^{n} = \hat{B}^{n}$, and decompose $\hat{B}^{n} = M^{n}+N^{n}$ into $M^{n} = 0$ and $N^{n} = \hat{B}^{n}$. We thus obtain

Using the continuity of $X_{s}\mapsto \mathcal{H}\big(X_{s}, \bigtriangledown f(X_{s})\big)$, we can obtain

Applying It$\hat{o}$'s formula to $f$, we can obtain

Combining (4.36) and (4.37), we have

For $t\geq0$ and $N\in\mathbb{N}$, by Theorem 3(i) and Lemma 3, we can obtain

Because for each $t$, $f(\hat{X}^{n}_{t})$ converges in distribution to $f(X_{t})$, and $f(X_{t})$ is uniformly integrable, we can obtain

We then have

where $\varepsilon(t)\rightarrow 0$ as $t\rightarrow \infty$. By Lemma 3 and Assumption 3, for a fixed $\delta > 0$, there is a $T$ large enough to satisfy

Hence,

When $T\rightarrow \infty$ and $\delta\rightarrow 0$, we have

□

4.3.   Proof of Theorem 2

First, for a given initial condition $x^{n}$ and a sequence stochastic system, write

Before proving Theorem 2, we need to introduce a lemma.

Lemma 7. Let $\big(x^{n}, \hat{X}^n, U^n, \hat{\xi}^{n}, H^n\big)_{n\geq1}$ be any sequence of the stochastic system; the associated cost functions $\big(J^n(x^{n}, \hat{X}^n, U^n, \hat{\xi}^{n}, H^n)\big)_{n\geq1}$ are described in Section 2.3. Assume that $x^{n}\rightarrow x$, $V$ is the value function given in Section 2.4, and $\int_{0}^{t}e^{-\alpha s}\Gamma_{s}^{n}ds\Rightarrow 0$ holds, where $\Gamma_{t}^{n}$ is described as in (4.32). We then have

Proof. First, as in the proof of Theorem 4, (4.32)–(4.35), and (4.37) hold for this case. By $\int_{0}^{t}e^{-\alpha s}\Gamma_{s}^{n}ds\Rightarrow 0$, we can obtain the following equation instead of (4.36):

Combining (4.37) and (4.39), we have

We then have

By Assumption 3 and Lemma 3, for all $n$ and $\forall \delta > 0$, there is a $T$ such that

Since $\hat{C}^{n}\Rightarrow C$ and $C$ have continuous sample paths, $\hat{C}_{T}^{n}$ converges in distribution to $C_{T}$. We thus obtain the following equation on the basis of Jensen's inequality, Assumption 3 and Lemma 3.

where $c$ is independent of $n$, and $k_{C}$ is defined in Assumption 3. For $n\in\mathbb{N}$, $\hat{C}_{T}^{n}$ is uniformly integrable, and as $n\rightarrow \infty$, we have $E\hat{C}_{T}^{n}\rightarrow EC_{T}$. By (4.41) and (4.42), we have

As $\delta\rightarrow 0$, we have

□

Proof of Theorem 2. First, $\big(x^{n}, \hat{X}^n, \widetilde{U}^n, \widetilde{\xi}^{n}, \widetilde{H}^n\big)_{n\geq1}$ satisfy the conditions in Theorem 4; i.e.,

Next, we show that $\big(x^{n}, \hat{X}^n, \widetilde{U}^n, \widetilde{\xi}^{n}, \widetilde{H}^n\big)_{n\geq1}$ satisfy the conditions of Lemma 7. Let $\Omega^{n}$ denote the event in which $X_{k}^{n}\geq\widetilde{\Psi}_{k}^{n}$ holds. According to Assumption 3 of $C(X, U, \xi, H)$ in Section 2.3, $C(X, U, \xi, H)$ is uniformly continuous on compact spaces. Let $g^{M}(\epsilon)$ be satisfied with the limit

and let $\mid C\big(\hat{X}^{n}_{t}, \widetilde{U}^n_{t}, \widetilde{\xi}^{n}_{t}, \widetilde{H}^n_{t}\big)-C\big(\hat{X}^{n}_{t}, U^{*}_{t}, \xi^{*}_{t}, H^{*}_{t}\big)\mid\leq g^{M}(\epsilon)$ when $\parallel\widetilde{U}^n_{t}\parallel, \parallel U^{*}_{t}\parallel, \parallel\widetilde{\xi}^{n}_{t}\parallel, \parallel\xi^{*}_{t}\parallel, \parallel\widetilde{H}^n_{t}\parallel, \parallel H^{*}_{t}\parallel\leq M$, and $\parallel\widetilde{\xi}^{n}_{t}-\xi^{*}_{t}\parallel\leq\epsilon$. Following holds for the event $\Omega^{n, M} = \Omega^{n}\cap\{\parallel\hat{X}^{n}_{t}\parallel_{T}^{*}+\parallel\hat{\Phi}^{n}_{t}\parallel_{T}^{*}+\parallel\hat{\Psi}^{n}_{t}\parallel_{T}^{*}\leq M\}$, where $\parallel X_{t}\parallel_{T}^{*} = \sup_{0\leq t\leq T}\parallel X_{t}\parallel$:

By $\parallel\widetilde{\Psi}^{n}-\Psi^{n, *}\parallel\leq 2m$, we can obtain $\parallel\widetilde{\xi}^{n}_{t}-\xi^{*}_{t}\parallel\leq 2\frac{m}{n}$. Therefore, for $\Omega^{n, M}$ and for some $\delta_{n}\rightarrow 0$, we have $\mid\Gamma_{t}^{n}\mid_{T}^{*}\leq \delta_{n}$. $\lim_{n \to \infty} P(\Omega^{n}) = 1$, which follows from the convergence $\frac{\hat{X}^{n}}{\sqrt{n}} = \bar{X}^{n}- \gamma^{0}\Longrightarrow 0$ in Lemma 3. By the tightness of $\hat{X}^{n}$ and $\hat{\Phi}_{t}^{n}\in\mathbb{R}^{m}_{+}$, we have

Therefore, $\mid\Gamma_{t}^{n}\mid_{T}^{*}$ converges to zero in the distribution. Since $T$ is arbitrary, $\Gamma_{t}^{n}\Rightarrow 0$, so $\big(x^{n}, \hat{X}^n, \widetilde{U}^n, \widetilde{\xi}^{n}, \widetilde{H}^n\big)_{n\geq1}$ satisfies the conditions of Lemma 7. We then

Combining (4.44) and (4.46), we have

□

5.   Conclusions

The present study introduces and evaluates a novel joint scheduling–control policy for managing multiclass customer flows in parallel server queues under heavy traffic conditions. Our approach, which is based on in the principles of customer admission control, service scheduling control, and service rate control, provides a comprehensive strategy to optimize queuing systems where the traffic intensity parameter is close to unity.

Our research makes the following contributions to the literature:

$(1)$ Innovative policy formulation: This paper presents a unique joint scheduling–control policy that considers the overall system state rather than isolated customer or server conditions, leading to more holistic and efficient queuing management.

$(2)$ Theoretical validation: This paper demonstrates the asymptotic optimality of the proposed policy within the Halfin–Whitt heavy traffic regime, ensuring that the additional control components are not only justified but also reduce the total system cost.

$(3)$ Numerical simulations: This paper provides empirical evidence from comparative experiments that illustrate the superior performance of our policy over other control strategies in terms of reducing system$^{\prime}$s costs and improving service efficiency.

The results of our study have practical implications for various real-world applications, including (but not limited to) modern call centers, high-performance computing systems, cloud computing services, and emerging quantum computing platforms. The newly developed policy can serve as a valuable tool for operators and managers aiming to enhance service quality while maintaining high server utilization rates.

Our work also opens avenues for further research. The joint scheduling–control policy could be extended to more complex systems with additional constraints or different customer behavior patterns. Moreover, the integration of machine learning techniques to dynamically adjust control parameters in real time could be an exciting direction for future exploration.

In conclusion, our study presents a significant advancement in the field of queuing theory and control, offering a robust policy that promises to improve operational efficiency in multiclass server environments. We are confident that our findings will not only benefit academic researchers but also inform the decision-making of practitioners in related industries.

Use of Generative-AI tools declaration

The author declares that he has not used Artificial Intelligence (AI) tools in the creation of this article.

Acknowledgments

The authors would like to thank the editor and reviewers for their help and suggestions for revisions to this paper. Thanks are also due to those who have provided help and advice in the writing, revision, and publication of this paper.

Conflict of interest

The author has no relevant financial or non-financial interests to disclose.