A mixture deep neural network GARCH model for volatility forecasting

1.   Introduction

Volatility forecasting is essential in asset pricing, portfolio allocation and risk management research. Early volatility forecasting was based on economic models. The most famous economic models are the ARCH model ^[1] and the GARCH model ^[2], which can capture volatility clustering and heavy-tail features. However, they fail to capture asymmetry, such as leverage effects. The leverage effect is due to the fact that negative returns have a more significant impact on future volatility than positive returns. To overcome this drawback, the exponential GARCH (EGARCH) model ^[3] and GJR model ^[4] were proposed. In the following years, new volatility models based on the GARCH model emerged, such as the stochastic volatility model ^[5] proposed by Hull and White and the realized volatility model ^[6] offered by Blair et al. They formed a class of GARCH-type volatility models for financial markets.

The traditional GARCH model has strict constraints and requires the financial time series to satisfy the stationarity condition. It usually assumes conditional variances have a linear relationship with previous errors and previous variances. However, many financial time series show certain nonstationary and nonlinear characteristics in practice. Consequently, some extended model from GARCH is necessary to study the volatility of these time series.

With the development of computer and big data technologies, machine learning brings new ideas to volatility forecasting ^{[7,8,9,10,11,12,13,14,15,16,17,18,19,20,21]}. Especially artificial neural networks(ANN) have shown an outstanding performance. It derives its computational ideas from biological neurons and is now widely used in various fields.

In financial risk analysis, researchers have utilized neural networks to study the volatility of financial markets. Hamid and Iqbal ^[22] apply ANN to predict the S&P500 index implied volatility, finding that ANN's forecasting performance surpasses that of the American option pricing model. Livieris proposes an artificial neural network prediction model for forecasting gold prices and trends ^[23]. Additionally, Dunis and Huang ^[24] explore neural network regression (NNR), recurrent neural networks, and their collaborative NNR-RNN models for predicting and trading the volatility of daily exchange rates of GBP/USD and USD/JPY, with results indicating that RNNs have the best volatility forecasting performance. Beyond the direct application of neural networks, researchers have investigated a series of mixture models ^{[25,26,27,28,29,30,31]} that combine ANNs and GARCH models. Liu et al. ^[32] introduce a volatility forecasting model based on the recurrent neural network (RNN) and the GARCH model. Experiments reveal that such mixture models enhance the predictive capabilities of traditional GARCH models, capturing normality, skewness and kurtosis of financial volatility more accurately.

This study employs a mixture model (DeepAR-GMM-GARCH) that combines the deep autoregressive network, the Gaussian mixture model and the GARCH model for probabilistic volatility forecasting. First, we discuss the design of the mixture model; Second, the article presents the model's inference and the training algorithm of the model; Third, we conduct a simulation experiment using artificial data and compare the outcomes with traditional GARCH models, finding that our model yields smaller RMSE and MAE; Last, we investigate the correlation between the square of extreme values and the square of returns for the CSI300 index. The empirical data is partitioned into training and test sets. After training and testing, we analyze the prediction results and observe that our proposed model outperforms other models in both in-sample and out-of-sample analyses.

The key offerings presented in this article can be summarized as follows: Initially, this article introduces a novel conditional volatility probability prediction model, which addresses the leptokurtic and heavy-tail traits of conventional financial volatility. This model is built upon a deep autoregressive network combined with a Gaussian mixture distribution. Subsequently, we incorporate extreme values into the mixture model via the neural network. It is discovered that the inclusion of extreme values enhances the accuracy of volatility predictions.

The structure of this paper is as follows: Section 2 outlines the GARCH model and the deep autoregressive network. Section 3 delves into the mixture model, elaborating on inference, prediction and the relevant algorithm. Section 4 encompasses the simulation studies that we propose. Lastly, Section 5 focuses on the empirical analysis of our proposed model.

2.   The GARCH model and the deep autoregressive network

2.1.   The GARCH model

Scholars usually believe that stock price or stock index returns are nonlinear, asymmetric, heavy-tailed, returns are generally uncorrelated. Aggregation characterises volatility, which was first found by Engle (1982) and Bollerslev (1986) in ARCH and GARCH models. The GARCH model is defined as follows:

where $h_{t}$ is the conditional heteroscedastic variance of return series $r_{t}$.

Although there are many criteria of GARCH($p$, $q$) models to find $p$ and $q$, it is sufficient to apply the GARCH($1$, $1$) model to characterize the conditional volatilities.

2.2.   The DeepAR model

The DeepAR model ^[33], illustrated in Figure 1, is a time series forecasting model that employs a deep autoregressive recurrent network architecture. Distinct from other time series forecasting models, DeepAR generates probabilistic predictions.

Figure 1.  The network structure of the DeepAR model with $p$ inputs and $q$ outputs.

DownLoad: Full-Size Img PowerPoint

Consider a time series $[x_{1}, \dots, x_{t_{0}}, x_{t_{0}+1}, \dots, x_{T}] : = x_{1:T}$. Given its past time series $[x_{1}, \dots, x_{t_{0}-2}, x_{t_{0}-1}] : = x_{1:t_{0}-1}$, our objective is to predict the future time series $[x_{t_0}, \dots, x_{t_{0}+(T-1)}, x_{t_{0}+T}] : = x_{t_{0}:t_ {0}+T}$. The DeepAR model constructs the conditional distribution $P_{\Theta}(x_{t_{0}: T} |x_{1:t_{0}-1})$ using a latent factor $z$, which is implemented by a deep recurrent network architecture. This conditional distribution, $P_{\Theta}(x_{t_{0}: T} |x_{1:t_{0}-1})$, comprises a product of likelihood factors ($z$)

The likelihood $p(x_{t}|\theta(z_{t}))$ is a fixed distribution with parameters determined by a function $\theta(z_{t}, \Theta)$ of the network output $z_{t}$. As suggested by the model's authors, Gaussian likelihood is appropriate for real-valued data.

3.   The mixture model

Forecasting the probability of volatility in finance and economics is an important problem. There are mainly two methods to tackle this. First, statistical models such as ARCH and the GARCH models are usually adopted. These models are specifically designed to capture the dynamic nature of volatility over time and help to predict future levels of volatility based on past patterns.

Another strategy involves using machine learning models, such as neural networks, which can analyze vast amounts of data and uncover patterns that may not be readily apparent to human analysts. A case in point is the DeepAR model is a series-to-series probabilistic forecasting model. The advantages of the DeepAR model are: it makes probabilistic forecasting and allows to introduce additional covariates. Due to these advantages, it can be used to predict financial volatility($h_{t}$) based on the series $r_{t}^{2}$. However, the DeepAR model usually assumes that $p(x_{t}|\theta(z_{t}))$ (given in (2.2)) follows a Gaussian distribution, which may be unreasonable due to the non-negative, leptokurtic and heavy-tail characteristics of traditional financial volatility. To avoid this problem, people use the gaussian mixture distribution to describe the density of $p(ln(x_{t})|\theta(z_{t}))$, see references ^[34]. Motivated by the above results, this paper propose an improved mixture model: DeepAR-GMM-GARCH.

The conditional distribution of $ln(h_{t})$ can be expressed as:

where $h_{t}$ represents the future volatility at time $t$, $[r_{1}, ..., r_{t-2}, r_{t-1}]: = r_{1:t-1}$ denotes the past return series during the $[1:t-1]$ period, and ${\bf x}_{1:t-1}$ refers to the covariate, which is observable at all times. The past time horizon is represented by $[1:t-1]$.

The proposed hybrid model assumes that the conditional density for logarithm of the volatility is given by $p(ln(h_{t})|r_{1:t-1}^{2}, {\bf x}_{1:t-1})$, which includes a set of latent factors, denoted as $z_{t}$. A recurrent neural network with hyperparameters($\Theta_{1}$), specifically an LSTM, encodes the squared returns $r_{t}^{2}$, the input features ${\bf x}_{t}$ and the previous latent factors $z_{t-1}$, generating the updated latent factors $z_{t}$. The likelihood $p(ln(h_{t})|\theta (z_{t}))$ follows a Gaussian mixture distribution with parameters determined by a function $\theta(z_{t}, \Theta_{2})$ of the network output $z_{t}$. The network architecture of the DeepAR-GMM-GARCH model is depicted in Figure 2.

Figure 2.  The network structure of the DeepAR-GMM-GARCH model with $m$ inputs and one output.

DownLoad: Full-Size Img PowerPoint

3.1.   The DeepAR-GMM-GARCH model

Due to the complex interplay between volatility and the factors that influence it, this paper's central model component declares that the volatility $h_{t}$ of a time series at time $t$ is derived from the latent variable $z_{t-1}$ at time $t-1$, the square of return $r_{t-1}^{2}$ and the covariates ${\bf x}_{t-1}$. $p(\ln(h_{t})|\theta(z_{t}))$ follows a Gaussian mixture distribution composed of K components. In the empirical analysis, ${\bf x}_{t-1}$ will be substituted with a vector of extreme values. A nonlinear mapping function $g$ is used to establish this relationship. The DeepAR-GMM-GARCH model proposed in this paper is as follows.

The model can be viewed as a structure for nonlinear volatility prediction models since the conditional distribution of the perturbation $\varepsilon_{t}$ in the model can be selected as $N(0, 1)$ and $T(0, 1, v)$. Consequently, this gives rise to two distinct models, referred to as DeepAR-GMM-GARCH and DeepAR-GMM-GARCH-t.

Assuming that the distribution of $p(ln(h_{t})|\theta(z_{t}))$ follows a Gaussian distribution, model (3.2) will be reduced to a more simple version:

For similarity, we call the above as DeepAR-GARCH model.

3.2.   Inference and learning

3.2.1.   Algorithm

For a given time series, our goal is to estimate the parameters $\Theta_{1}$ of the LSTM cells and the parameters $\Theta_{2}$ in function $\theta$ which applies an affine transformation followed by a softplus activation. We employ a quasi-maximum likelihood estimation method with the likelihood function: $\Theta = argmax\sum_{i}logp(\widetilde{h_{i}}|\Theta_{1}, \Theta_{2})$. Inferring from this likelihood function necessitates taking into account the latent variable $z_{t}$.

The flowchart for the training algorithm of the model is shown below. First, we utilize the BIC criterion to identify the number of classifications, $K$, for all samples. Each data point is assigned a label from 1 to $K$, and each cluster $k$ has its mean vector and covariance matrix. Based on these findings, we establish the initial $\pi_{k}$ as the proportion of data points labelled as $k$ and set the initial mean vector $\mu_{k}$ and covariance matrix $\sum_{k}$ to the mean vector and covariance matrix within cluster $k$. As a result, we obtain the parameter values $(\theta = {\widetilde \pi_{k, 0}, \widetilde \mu_{k, 0}, \widetilde \sigma_{k, 0}^{2}})$ from the initial cluster and use them to pre-train the DeepAR-GMM-GARCH model. This approach allows our model to converge quickly. Next, we partition the training sample data into multiple batches, select one sample from a batch and use the sample $(r_{t0-m}^{2}, \dots, r_{t0-1}^{2})$ as the input for the DeepAR-GMM-GARCH model. The model calculates a set of $\widetilde \pi_{k, t}, \widetilde \mu_{k, t}, \widetilde \sigma_{k, t}^{2}$, after which we sample from this Gaussian mixture model, compute the loss, and update the parameters through gradient descent. Since direct differentiation of the sampling is infeasible, we apply the reparameterization trick to adjust the model's parameters. We continue this training process until the end of the training cycle. Last, we input the training set sample into our trained model for prediction evaluation. The model sequentially calculates the parameters for both the latent variable and the mixed Gaussian model and then proceeds with sampling. Ultimately, we provide the prediction results derived from the sampling outcomes.

The training algorithm is shown in the Algorithm $1$.

3.2.2.   Loss function

During the training process, the definition of the loss function determines the prediction quality of the model. We use the the average negative loglikelihood function as the loss function $Loss = L_{h}$. In the GARCH model, we usually assume that $\varepsilon_{t}$ obeys a Gaussian distribution or a Student distribution, two loss functions are as follows:

(1) When $\varepsilon_{t} \sim N(0, 1)$, the loss function is:

(2) When $\varepsilon_{t} \sim t(0, 1, v)$, the loss function is:

To calculate the above loss functions, we need to get samples for $\widetilde{h_{t}}$ based on algorithm1 given in Section 3.2.1. In practice, if $\varepsilon_{t}$ follows other distributions, based on the idea of QMLE, we still can use the loss function given in (3.4) see Liu and So, 2020.

4.   Simulation

Experiments are carried out on volatility inference using simulated time series. These series exhibit flexibility, with both volatility and mixing coefficients changing over time, as detailed below:

where $F_{t-1}$ denotes the information set through time $t-1$ and $\phi$ is the Gaussian density function. $\eta_{1, t}$ and $\eta_{2, t}$ are mixing coefficients of two gaussian distribution and satisfy: $\eta_{2, t} = (1-\eta_{1, t})$. When generating the simulation data, we set: $\alpha_{01} = 0. 01$, $\alpha_{11} = 0. 1$, $\beta_{1} = 0. 15$, $\alpha_{02} = 0. 04$, $\alpha_{12} = 0. 15$, $\beta_{2} = 0. 82$, $c_{0} = 0. 02, c_{1} = 0. 90, a_{0} = 0. 02, a_{1} = 0. 6$. The time series has initial values:$r_{0} = 0. 1, \sigma_{0}^{2} = 0$, $h_{0} = 0$. The sample sizes of T = 500, 1000 and 1500 are considered, and the replication time is 1000.

For the series simulated from (4.1), we apply three models to forecast their volatility, namely, the GARCH model with $\epsilon_{t} \sim N(0, 1)$(GARCH-n), the GARCH model with$\epsilon_{t} \sim T(0, 1, v)$(GARCH-t) and the DeepAR-GMM-GARCH model. Using MCMC sampling method, the degree of freedom for the Student-t distribution is determined to be 6. For the DeepAR-GMM-GARCH model, we set a recurrent neural network with three LSTM layers and 24 hidden nodes. For the input nodes m in Figure 2, we use the Grid Search Algorithm to find their optimal values. We use BIC rule to choose K based on the Mclust package ^[35]. Our model's hyperparameters are trained by the software Optuna, a commonly applied automatic hyperparameters optimization software.

Table 1 informs the three volatility forecasting models' in-sample errors (RMSE and MAE). The GARCH-n and GARCH-t show similar performance, and our DeepAR-GMM-GARCH model is outstanding; All the models get decreased RMSE as sample size increase. These results imply the proposed estimation can be asymptotically convergent. Table 2 reports the average out-of-sample errors of the three volatility forecasting models. Similar to the in-sample results, our DeepAR-GMM-GARCH model is superior to the GARCH-n and GARCH-t models.

Table 1.  The average in-sample errors of the GARCH-n, the GARCH-t and the DeepAR-GMM-GARCH models.

sample size	Modle	RMSE	MAE
$T=500$	$\text{GARCH-n}$	$0. 1364$	$0. 0628$
	$\text{GARCH-t}$	$0. 1211$	$0. 0591$
	$\text{DeepAR-GMM-GARCH}$	$\underline{0. 1068}$	$\underline{0. 0548}$
$T=1000$	$\text{GARCH-n}$	$0. 0621$	$0. 0437$
	$\text{GARCH-t}$	$0. 0578$	$0. 0419$
	$\text{DeepAR-GMM-GARCH}$	$\underline{0. 0398}$	$\underline{0. 0331}$
$T=1500$	$\text{GARCH-n}$	$0. 0604$	$0. 0428$
	$\text{GARCH-t}$	$0. 0652$	$0. 0401$
	$\text{DeepAR-GMM-GARCH}$	$\underline{0. 0300}$	$\underline{0. 0325}$
Note: Number of replications = 1000.

---

 | Show Table

DownLoad: CSV

Table 2.  The average out-of-sample errors of the GARCH-n, the GARCH-t and the DeepAR-GMM-GARCH models.

sample size	Modle	RMSE	MAE
$T=500$	$\text{GARCH-n}$	$0. 3564$	$0. 3028$
	$\text{GARCH-t}$	$0. 3271$	$0. 3091$
	$\text{DeepAR-GMM-GARCH}$	$\underline{0. 2761}$	$\underline{0. 2311}$
$T=1000$	$\text{GARCH-n}$	$0. 2619$	$0. 2117$
	$\text{GARCH-t}$	$0. 2318$	$0. 2033$
	$\text{DeepAR-GMM-GARCH}$	$\underline{0. 2091}$	$\underline{0. 1834}$
$T=1500$	$\text{GARCH-n}$	$0. 2241$	$0. 2179$
	$\text{GARCH-t}$	$0. 2213$	$0. 1971$
	$\text{DeepAR-GMM-GARCH}$	$\underline{0. 1911}$	$\underline{0. 1722}$
Note: Number of replications = 1000.

---

 | Show Table

DownLoad: CSV

5.   Empirical anaysis

Comprehensive stock index represents the average of the economic performance of the whole financial market. In this section, we study the China Shanghai Shenzhen index(CSI 300 index) daily OHLC data. The OHLC data contains daily high, low, open and close prices. Scholars have pointed out that combining the open, high, low and close prices can obtain more effective volatility estimates. Hence, We also intruduce OHLC data to our mixture model.

5.1.   CSI 300 data set

The data of the CSI 300 index studied in this paper are from January 4, 2010, to December 30, 2021, with a total of 2916 trading days. Let $r_{t}$ be the returns of the corresponding series, which are calculated using the closing price $C_{t}$ series of the CSI300 index.

The time series of $r_{t}$ and $r_{t}^{2}$ are ploted in Figure 3. We can find that $r_{t}^{2}$ displays a significant volatility clustering characteristic, and the amplitude of volatility is gradually decreasing.

Figure 3.  The time series of $r_{t}$ and $r_{t}^{2}$ for the CSI 300 index, as shown in (5.1), spans from January 4, 2010, to December 30, 2021.

DownLoad: Full-Size Img PowerPoint

The collected data is divided into training data and test data. Table 3 describes the statistical characteristics of training and test data, respectively. The mean of $r_{t}$ is relatively small, only 0.007465. The standard deviation is 2.251006, which displays that the degree of variation is large. The skewness is less than 0, and the kurtosis is greater than 3, indicating that the sequence is of left deviation and heavy tail. The test data has the same characteristics as the training data, such as data dispersion, large volatility, left deviation and higher protrusion than the normal distribution. This may imply that the normal distribution may not be suitable for our data, and other heavy-tailed distributions, such as t distribution or mixed normal distribution, could be more suitable.

Table 3.  Descriptive statistics for the training and test sets of CSI 300 returns.

Data Set	Period	Mean	Std.	Skew.	Kurt.
$\text{training data}$	$04 / 01 / 2010$ to $29 / 12 / 2017$	$0.007465$	$2.251006$	$-0.755580$	$5.136707$
$\text{test data}$	$02 / 01 / 2018$ to $30 / 12 / 2021$	$0.019125$	$1.717302$	$-0.427819$	$3.248347$

---

 | Show Table

DownLoad: CSV

Besides the close price($C_{t}$), we also introduce the high price($H_{t}$), the open price($O_{t}$) and the low price($L_{t}$). Define $u_{t} = (H_{t}-O_{t})^{2}$, $d_{t} = (L_{t}-O_{t})^{2}$, $c_{t} = (C_{t}-O_{t})^{2}$.

The correlation matrix below (5.2) shows the correlation coefficients between $u_{t}$, $d_{t}$, $c_{t}$ and $r_{t}^{2}$. It can be found that there is large correlation coefficient between the pair ($u_{t}$, $r_{t}^{2}$), ($d_{t}$, $r_{t}^{2}$) and ($c_{t}$, $r_{t}^{2}$). From a common sense, large values for $u_{t}$, $d_{t}$ and $c_{t}$, usually means large volatility($r_{t}^{2}$). However, classical volatility models do not take such extreme values into account. Consequently, it is reasonable to use neural network together with $u_{t}$, $d_{t}$ and $c_{t}$ to forecast volatility, because neural network can introduce additional covariates and capture the complex relation between different covariates.

5.2.   Error measures

This paper uses four evaluation indicators to measure the predictive performance of the model, they are: NMAE, HR, linear correlation coefficient and rank correlation coefficient, which is defined as follows:

where N represents the number of predicted samples. Both NMAE and HR values range between 0 and 1. The smaller the values of these two indicators, the better the model's performance.

Scholars usually use high-frequency data volatility estimates as a proxy for actual volatility to evaluate forecasting models. We also use realized volatility($\sigma_{RV, t}^{2}$) as a proxy for actual volatility, calculated by summing up the squares of intra-day returns every 5 minutes.

We focus on the out-of-sample predictive performance of the models, the correlation between realized volatilities $\sigma_{RV, t+1}^{2}$ and predicted volatilities $\widetilde{h}_{t+1}$ is measured only on the test set. We calculated Pearson's coefficient

where $\sigma_{RV}^{2}$ and $\widetilde{h}$ denote the respective mean values, and Spearman's rank order correlation coefficient $r_{s}$. $r_{s}$ is also calculated using Eq (5.6). However, the actual volatilities are replaced by their ranks. Spearman's rank order correlation coefficient is considered more robust than Pearson's coefficient. $r$ and $r_{s}$ are both between $-1$ and $1$. A value of $r$($r_{s}$) around $0$ means that the realized volatilities and predicted volatilities are uncorrelated.

5.3.   In-sample results and out-sample results

5.3.1.   In-sample results

Simulation experiments demonstrate that our proposed model exhibits greater prediction accuracy than the GARCH model. In this section, to highlight the advantages of our model, we compare it to the classic GARCH model, ANN-GARCH (an existing neural network GARCH model) and the DeepAR-GARCH model using empirical data.

Among the four models, the GARCH and ANN-GARCH models predict conditional volatility, whereas the DeepAR-GARCH and DeepAR-GMM-GARCH models provide probabilistic forecasts for conditional volatility. To facilitate comparison, we will calculate the mean and quantiles of the probability density function for conditional volatility derived from the DeepAR-GARCH and DeepAR-GMM-GARCH models.

The estimated parameters of the GARCH models(GARCH-n and GARCH-t) are summarized in Tables 4 and 5. For the GARCH-t model, the degrees of freedom parameter $v$ is estimated at around $6$. The GARCH-n and GARCH-t models are all nearly estimated with higher values of $\beta_{1}$ and lower values of $\alpha_{1}$. The sum of $\alpha_{1}$ and $\beta_{1}$ is almost 1, Which implies the sequence may be non-stationary. Therefore, our model without the stationary constraint is more suitable. The ANN-GARCH model employs a three-layer ANN structure, featuring two input nodes, 24 nodes for the hidden layer and a single output node. Likewise, the DeepAR-GARCH and DeepAR-GMM-GARCH models also have a three-layer design, consisting of 14 input nodes, 24 nodes for the hidden layer and an output layer with two output nodes and five output nodes.

Table 4.  Parameter estimation of GARCH-n model.

Data Set	$a_{0}$	$\alpha_{1}$	$\beta_{1}$
$CSI300$	$1.8365e-04$	$0.1000$	$0.8800$

---

 | Show Table

DownLoad: CSV

Table 5.  Parameter estimation of GARCH-t model.

Data Set	$a_{0}$	$\alpha_{1}$	$\beta_{1}$	$\nu$
$CSI300$	$1.8103e-04$	$0.1000$	$0.8400$	$6.4625$

---

 | Show Table

DownLoad: CSV

Table 6 lists the performance of five volatility prediction models. In the in-sample study, the DeepAR-GMM-GARCH model has the smallest HR and loss, and the DeepAR-GARCH model has the smallest NMAE. The volatility prediction performance of the DeepAR-GMM-GARCH model is better than the traditional GARCH and DeepAR models.

Table 6.  In-sample forecasting results for the GARCH-n, GARCH-t, DeepAR-GARCH and DeepAR-GMM-GARCH models.

Data Set	Model	Loss	NMAE	HR
In-sample	$\text{GARCH-n}$	$1. 401$	$0. 763$	$0. 704$
	$\text{GARCH-t}$	$1. 331$	$0. 761$	$0. 637$
	$\text{ANN-GARCH}$	$1. 603$	$0. 827$	$0. 690$
	$\text{DeepAR-GARCH}$	$1. 541$	${\bf 0. 748}$	$0. 717$
	$\text{DeepAR-GMM-GARCH}$	${\bf 1. 311}$	$0. 751$	${\bf 0.630}$

---

 | Show Table

DownLoad: CSV

In Figure 4, we display a portion of the forecasting results from various models and compare them with $r_{t}^{2}$. As shown in (a), the forecasting results from the GARCH models differ from $r_{t}^{2}$. The GARCH models fail to capture significant changes in $r_{t}^{2}$. From (b), (c), it is evident that the neural network models capture the trend of $r_{t}^{2}$ and more accurately predict large fluctuations, with the DeepAR-GMM-GARCH model demonstrating the best performance. In (d), we observe that the estimated 90% quantiles from the DeepAR-GMM-GARCH model appear to be closer to the observations ($r_{t}^{2}$).

Figure 4.  Subplots $(a), (b)$, $(c)$ and $(d)$ are the plots of the in-sample volatility forecasting results of the classic GARCH, ANN-GARCH, DeepAR-GARCH and DeepAR-GMM-GARCH models. (a) is the comparison of volatility forecasting results from the GARCH models. (b) is the comparison of volatility forecasting results between the ANN-GARCH model and $r_{t}^{2}$. (c) is the comparison of probabilistic forecasting models between the DeepAR-GARCH model and DeepAR-GMM-GARCH model. (d) is the comparison of 90% quantile forecasting results between the DeepAR-GARCH model and DeepAR-GMM-GARCH model.

DownLoad: Full-Size Img PowerPoint

To sum up, from the estimation results of Table 6 and the plots in Figure 4, it is shown that introducing the extreme values($u_{t}$, $d_{t}$, $c_{t}$) can help to improve the forecasting accuracy of the mixture volatility models. Hence the proposed approach is of particular practical value.

5.3.2.   Out-sample results

In the out-of-sample analysis, as discussed in Section 5.1, the test set comprises the time series of 972 trading days subsequent to the respective training set.

Table 7 presents the performance of the models using common error measures (loss function, NMAE and HR). The DeepAR-GARCH model attains a lower loss function value compared to the neural network models on the test set. The neural network models display lower NMAE and HR values than the GARCH model on the training set. The DeepAR-GMM-GARCH models exhibit the lowest NMAE and HR values on the test data set.

Table 7.  Out-of-sample forecasting results for the GARCH-n, GARCH-t, ANN-GARCH, DeepAR-GARCH and DeepAR-GMM-GARCH models.

Data Set	Model	Loss	NMAE	HR
Out-sample	$\text{GARCH-n}$	$2. 320$	$0. 917$	$0. 868$
	$\text{GARCH-t}$	$2. 008$	$0. 915$	$0. 859$
	$\text{ANN-GARCH}$	$2. 517$	$0. 903$	$0. 783$
	$\text{DeepAR-GARCH}$	${\bf 1. 916}$	$0. 929$	$0. 801$
	$\text{DeepAR-GMM-GARCH}$	$2. 100$	${\bf 0.790}$	${\bf 0.722}$

---

 | Show Table

DownLoad: CSV

In Figure 5, we plot part of the forecasting results from the five models mentioned with out-sample data set and compare it with $r_{t}^{2}$. It can be seen from (a) that the GARCH models do not capture significant changes of $r_{t}^{2}$, the same as the in-sample results. For (b), (c), The neural network models capture most of the fluctuations of $r_{t}^{2}$ well, the DeepAR-GMM-GARCH model performing the best.

Figure 5.  Subplots $(a), (b)$, $(c)$ and $(d)$ are the plots of the out-sample volatility forecasting results of the classic GARCH, ANN-GARCH, DeepAR-GARCH and DeepAR-GMM-GARCH models. (a) is the comparison of volatility forecasting results from the GARCH models. (b) is the comparison of volatility forecasting results between the ANN-GARCH model and realized volatility($r_{t}^{2}$). (c) is the comparison of probabilistic forecasting models between the DeepAR-GARCH model and DeepAR-GMM-GARCH model. (d) is the comparison of 90% quantile forecasting results between the DeepAR-GARCH model and DeepAR-GMM-GARCH model.

DownLoad: Full-Size Img PowerPoint

From (d), we could find that The estimated 90% quantiles from the DeepAR-GMM-GARCH model seem to be more closely aligned with the observations ($r_{t}^{2}$).

Section 5.2 mentions that the linear correlation $r$ and the rank correlation $r_{s}$ are two measures for comparing realized volatilities. The linear correlation r and the rank correlation $r_{s}$ between predicted and realized volatilities of the test set are reported in Table 8. On average, the DeepAR-GMM-GARCH model shows the best performance of all models. It obtains the highest rank correlation on the test set. Rank correlation is more robust than linear correlation since it detects correlations nonparametrically.

Table 8.  A comparison of linear correlation (r) and rank correlation ($r_s$) between the realized volatility and volatility predicted by GARCH-n, GARCH-t, DeepAR-GARCH, ANN-GARCH and DeepAR-GMM-GARCH models for test data set on the CSI300 index. The best model (the highest correlation) is underlined.

Out-sample	GARCH-n	GARCH-t	ANN-GARCH	DeepAR-GARCH	DeepAR-GMM-GARCH
$r$	$0. 381$	$0.420$	$0.490$	$0. 473$	$\underline{0. 504}$
$r_s$	$0. 477$	$0.502$	$0.500$	$0. 516$	$\underline{0. 527}$

---

 | Show Table

DownLoad: CSV

6.   Conclusions

This paper studies a mixture volatility forecasting model based on the autoregressive neural work and the GARCH model to obtain more precise forecasting for the conditional volatility model. The inference, loss functions and training algorithm of the mixture model are given. The simulation results show that our model performs better with less error than the classic GARCH models. The empirical study based on the CSI300 index shows that our model can significantly improve the forecasting accuracy with extreme values compared to the usual models.

Our research findings can offer valuable insights into the prediction of volatility uncertainty. In future studies, our model can be employed for various high-frequency volatility analysis, where it is anticipated to exhibit enhanced performance.

Acknowledgments

This work is partially supported by Guangdong Basic and Applied Basic Research Foundation (2022A1515010046) and Funding by Science and Technology Projects in Guangzhou (SL2022A03J00654).

Conflict of interest

The authors declare no conflict of interest.