An optimized fractional grey model based on weighted least squares and its application

1.   Introduction

Grey system theory was pioneered in 1982 ^[1,2], and one of the important study contents is how to abstract and establish a model from an unclear system with insufficient overall information to make a fuzzy long-term description of the law of development of things. This model can make the unclear factors of the grey system clear, and provide the research basis. GM(1, 1) is a grey prediction model proposed earlier, which has been applied in many fields, such as energy, economy, and education ^[1,2]. For example, Kun ^[3] applied the grey predicting model to the Chinese automobile industry, Feng et al. ^[4] forecasted the measles incidence in China with GM(1, 1), and similarly, Wang et al. ^[5] predicted the prevalence of infectious diseases with GM(1, 1) model.

Nevertheless, GM(1, 1) model has very small samples, and the high overall simulation accuracy does not necessarily guarantee high prediction accuracy. In order to further improve the accuracy and scope of the application of GM(1, 1), many researchers proposed improved GM(1, 1) models. For instance, Li et al. ^[6] put forward the weighted least square method to estimate parameters in GM(1, 1) model. Hsu ^[7] improved the GM(1, 1) model by combining the artificial network with residual correction and predicted the electricity demand in Taiwan. Ma et al. ^[8] put forward an improved GM(1, 1) model by reconstructing the background value with the Simpson formula. Mao et al. ^[9] proposed a grey multivariate time-delay model which solved the modelling problem between oil price and exchange rate. Wu et al. ^[10] presented the Grey Riccati model by using the trapezoidal integral formula. Wang et al. ^[11] proposed a new grey model which is used to predict the annual electricity consumption in China. Wu ^[12] put forward an improved nonlinear grey Bernoulli model to forecast China's GDP. Wu et al. ^[13] put forward a new grey Bernoulli model to make a short-term forecast of natural gas. Ma et al. ^[14] predicted primary energy consumption using a novel NDGM(1, 1, K, C) model, where the background value was constructed by Simpson numerical integration formula. Zhou et al. ^[15] proposed a grey seasonal least square support vector regression model.

However, when describing the evolution of the complex system with constant order differential equations, it often fails to accurately describe some characteristics of the system. Accordingly, many scholars have studied the accumulation operator and sought new solutions. In 2013, Wu et al. ^[16] proposed a fractional-order grey model. Compared with the traditional GM(1, 1), the grey prediction model with fractional order accumulation has a better performance. Based on the fractional accumulation operator, many scholars have proposed some new grey prediction models ^{[17,18,19,20]}.

Chen et al. ^[21] studied a new fractional-order cumulative grey prediction model with time delay polynomials. Duan et al. ^[22] predicted the crude oil consumption in China with a new fractional-order grey model. Meng et al. ^[23] used a genetic algorithm to optimize the order of fractional order grey model and predicted the emission of sulfur dioxide in China. In order to further improve the prediction performance of the fractional grey model, Xie et al. ^[24] proposed a new grey model with conformable fractional derivative and applied it to predict China's consumption of energy and coal. Xie et al. ^[25] also used the quantum evolutionary algorithm to optimize the order of the grey model and predicted China's annual electricity consumption. Mao et al. ^[26] employed a new nonlinear fractional order grey model to predict urban traffic flow. Wu et al. ^[27] proposed a novel fractional order non-homogeneous grey model and obtained the closed solution of this model. Wu et al. ^[28] also presented a fractional order cumulative nonlinear Bernoulli grey mode based on the fractional cumulative generation matrix and Bernoulli equation and used a particle swarm optimization algorithm to seek the optimal parameters. In order to deal with the system containing both linear and nonlinear trends, Gao et al. ^[29] used the least square perturbation theory to determine the perturbation order of the model and proposed a fractional order grey model with a time-varying parameter. Wu et al. ^[30] put forward a seasonal grey prediction model with fractional order accumulation to describe the characteristics of seasonal fluctuations and improve the prediction performance ^[31]. Xie et al. ^[32] proposed an optimized non-equidistant model with time-varying characteristics. Xie et al. employed an optimized nonlinear grey Model to predict investment funds and enrollment in higher education in China ^[33]. Liu et al. ^[34] proposed a new grey model with fractional order accumulation and Markov chain for short-term power load prediction. In order to make the parameters adjusted according to the characteristics of the actual data, Kang et al. showed a multi-variable grey prediction model with a fractional derivative for non-stationary sequence by optimizing the cumulative order of the differential equation using the particle ant colony algorithm to predict the municipal solid waste ^[35]. Based on the new principle of information priority and considering the influence of non-homogeneous terms, Wu et al. ^[36] proposed a single variable grey prediction model based on a fractional accumulation generating operator and trapezoidal approximation formula of integral using particle swarm optimization algorithm to search parameters of the model. Then, they presented a fractional order multi-variable grey prediction model by using a genetic algorithm to determine the optimal order ^[37]. Liu and Wu ^[38] proposed an adjacent non-homogeneous grey model based on the principle of adjacent accumulation considering the weight between historical data and the latest value. Yuan et al. ^[39] put forward a nonlinear grey system model with fractional accumulation to predict the consumption of industrial water in Wuhan. Kang et al. ^[40] proposed a fractional viscoelastic traffic flow model in combination with the modelling principle of the Bass model and application of fractional calculus in a viscoelastic fluid and then established a fractional grey viscoelastic traffic flow model that can reflect time-varying characteristics by introducing the conformable fractional derivative and the fractional grey model. Mao et al. ^[41] presented a fractional derivative grey model with time delay by introducing fractional derivative and delay factor into the GLV model. Kang et al. ^[42] put forward a variable order fractional derivative grey model based on a definition of variable order fractional accumulation generation sequence. Yan et al. ^[43] proposed the grey model with the fractional Hausdorff derivative is put forward to enhance the forecasting accuracy of the traditional grey model. The proposed model will not be effect by the initial value. The relationship between the error and the order is proved.

The effective estimation of parameters in the grey model is an important index that affects the prediction accuracy of the grey model. Although many improved grey models have been proposed, the research on the estimation of parameters in the grey model is still insufficient. For example, although Li et al. ^[6] proposed the least square method to solve the parameters, it is easy to fall into the local minimum, resulting in overfitting and low prediction accuracy in the data training stage. In order to further improve the prediction accuracy of the grey model, in this paper, we propose an optimized fractional grey model, which uses weighted least squares to estimate the parameters of the grey model and different values have different weights. As we all know, the traditional least squares method assigns the same weights to all the parameters, and then the noisy data will be amplified and affect the prediction accuracy. However, the weighted least squares algorithm can produce different weights, noisy data will be assigned smaller weights and the valid data will have bigger weights, which will reduce the affection of noises and improve the accuracy. The main contributions are:

(1) We introduced the weighted least square method into the parameter estimation of the grey prediction model, which improves the accuracy of the parameter estimation.

(2) We proposed an optimized fractional-order grey prediction model based on weighted least squares estimation, and used the optimization algorithm to solve the fractional order automatically, which improves the prediction accuracy of the model.

(3) We applied the new model to the fields of water consumption, power consumption and education investment. Compared with the popular grey models and machine learning algorithms, we obtained better fitting and prediction results, which verified the effectiveness of the model.

The rest of this paper is organized as follows: Section 2 shows the description of the FGM(1, 1) model and depicts the detailed modelling procedure of the proposed model. Section 3 provides real cases of water consumption, power consumption and education investment to further verify the fitting and predicting accuracy of the model. Section 4 lists the main conclusions and the proofs are given in the final section. The primary nomenclatures are listed in the appendix.

2.   Methodology

This section introduces the definition of the fractional-order grey model and then introduces the novel fractional grey model based on weighted least squares, abbreviated as WFGM(1, 1).

2.1.   Presentation of fractional grey model

As reported in Ref. ^[16], the fractional grey model (denoted as FGM(1, 1) for short) have been received extensive attention from method to practice. More importantly, combining fractional accumulation with the grey model enables the model to successfully capture data patterns behind time-series sequence. By reference to ^[16], the construction of the fractional grey model includes three steps covering fractional-order accumulation, time response series solution, and fractional-order accumulation restoration. Specifically, these steps can be outlined as follows:

(1) Fractional-order accumulation

Let the observed univariate data be $X^{(0)} = \left\{ {{x^{(0)}}(1), {x^{(0)}}(2), \cdots, {x^{(0)}}(n)} \right\}$ under the action of $r$-order accumulation operator, the $r$-order accumulation sequence generated is ${X^{(r)}} = \left\{ {{x^{(r)}}(1), {x^{(r)}}(2), \cdots, {x^{(r)}}(n)}\right\}$, where

(2) Parameter solution

Based on the above definition, we can define the following differential equation to describe the change process of time series:

Integrate both sides of Eq (2.3) over the spacing $[k, k+1]$, we have

The above formula is solved using the numerical integration method

where

With the least square algorithm, the model parameters can be estimated as

where

(3) Time response series solution

Let ${\hat x^{(r)}}(1) = {x^{(r)}}(1)$, Solve Eq (2.3) to get:

where ${\hat x^{(r)}}(k)$ represents the output value of the model, $n$ represents the number of original sequences, and $N$ is the step size of backward prediction.

(4) Fractional-order accumulation restore

We carry out fractional-order cumulative subtraction according to the following formula

2.2.   The fractional grey model based on weighted least squares

In Literature ^[6], weighted least square was introduced into the grey prediction model to estimate the parameters of the GM(1, 1) model, and a good effect is obtained. In addition, literature ^[6] analyzed the problems of the common least square method in the grey model: (1) It may cause serious model morbidity in the solution process; (2) All the sum of squares of errors had the same weight.

Inspired by the reference ^[6], we introduce the weighted least squares algorithm in FGM(1, 1) to solve the model parameters and improve the accuracy of the model by making different error sums of squares with different weights. The minimization objective function is designed as follows:

Let the derivative of ${\chi ^2}$ with respect to model parameter $\hat{a}$ and $\hat{b}$ be 0, we have

The above expression can be written in matrix form as

Using the weighted least squares algorithm, we modify the basic form of FGM(1, 1) as

where

Remark 1. In the actual programming of parameter estimation, we can use a simple matrix form to get the parameter estimation of the model, which helps us to write the program. A specific deduction is shown in the following analysis.

where

2.3.   Optimization prediction

The flow chart of optimization prediction is exhibited in Figure 1.

The optimal order $r$ of WFGM(1, 1) is estimated with optimizers ALO (Ant Lion Optimizer), PSO (Particle Swarm Optimization), WOA (Whale Optimization Algorithm) and GWO (Grey Wolf Optimizer), respectively, which are the most widely used random-based optimization techniques and are often used to solve optimization problems. PSO is a population-based stochastic optimization technique proposed by Eberhart and Kennedy in 1995. It is easy to implement with high precision and fast convergence. ALO is a meta-heuristic swarm intelligence algorithm proposed by Mirjalili in 2015. it is a kind of search technology with a diverse population, strong optimization performance, few adjustment parameters and easy implementation because of the introduction of random walk, roulette strategy and elite strategy. WOA is a swarm intelligence optimization algorithm proposed by Mirjalili et al. in 2016. It is simple and easy to implement, and it has loose requirements on objective function conditions and less parameter control. GWO is a population intelligence optimization algorithm proposed by Mirjalili et al. in 2014 inspired by the predation behaviour of grey wolves. It has strong convergence performance, a simple structure, few parameters that need to be adjusted, an adaptive convergence factor and information feedback mechanism, and can achieve a balance between local optimization and global search, so it has good performance in problem-solving accuracy and convergence speed. In the four optimizers, PSO needs hyper-parameters and the configuration of PSO is listed in Table 1.

3.   Application

In order to verify the validity of the proposed model, we fit and predict the comprehensive production capacity of water supply (CPCWS), power consumption (PC) and educational expenditure (EE). The data gathered from the National Bureau of Statistics of China, which can be downloaded at http://www.stats.gov.cn/english/. In the experiments, the corresponding data set was divided into two sets, one for fitting and the other for prediction. The fitting stage is to train the model, and the prediction stage is to test the validity of the model. The experimental environment is Matlab 2019b in a windows 10 system, the processor is AMD Ryzen 7 5800 H with Radeon Graphics 3.20 GHz.

To assess the prediction accuracy of the novel model, the absolute percentage error (APE) and mean absolute percentage error (MAPE) are taken as the evaluation indexes, which are defined as

respectively, where $x^{(0)}(k)$ and $\widehat{x}^{(0)}(k)$ are the actual value and corresponding fitted value at time $k$.

3.1.   Fitting and prediction of CPCWS

Water supply is the most basic public utility in a city, which belongs to the category of public facilities. Most of them take the government as the main body of supply to ensure the public resources and fairness that serve society to the greatest extent. The importance of water supply as public goods lies not only in its contribution to economic growth but also in maintaining the sustainable development of economic and social equity. Since the 21st century, China's water supply industry has developed rapidly. For example, the comprehensive production capacity of urban water supply reached 312 million $m^3$/ day in 2018.

In this section, the proposed model is applied to predict the comprehensive production capacity of water supply (CPCWS) of Hebei and Liaoning Provinces of China to examine the further applicable ability of WFGM(1, 1) compared with other competitors.

3.1.1.   CPCWS of Hebei province

GM(1, 1), DGM(1, 1), FGM(1, 1), ANN and WFGM(1, 1) are employed to fit and predict the CPCWS (10,000 $m^3$/day) of Hebei province, where the data from 2004 to 2015 are used for the fitting and the data from 2016 to 2019 are for predicting.

The track of seeking the optimum parameters with the four optimizers, ALO, PSO, WOA and GWO, is given in Figure 2. The figure indicated that the optimal orders $r$ are all about 0.18. In order to verify the optimizer's stability, we repeated each of these optimizers 100 times and the results are shown in Figure 3. It can be seen that all the algorithms are relatively stable. Nevertheless, compared with the other three optimizers (ALO, PSO and WOA), GWO is optimal on stability and the exact optimal $r$ is 0.185.

The corresponding MAPEs of five models are listed in Table 2. Figure 4 visualizes the model errors of Table 2, where APE is the prediction error of the model at each point, and the MAPE is the corresponding relative error. From Table 2 and Figure 4, we can see that the MAPEs of five models are 4.83%, 4.86%, 4.25%, 4.41%, and 4.23% in the fitting period, and the MAPEs of these competitors are 6.05%, 5.97%, 5.55%, 8.63%, and 5.05% in predicting period, respectively. Compared to the grey models of GM(1, 1), DGM(1, 1), FGM(1, 1), and ANN, WFGM(1, 1) achieved more precise fitting result, 4.23%, followed by FGM(1, 1), 4.25%. At the same time, our WFGM(1, 1) model also gained the best result in predicting the CPCWS with MAPE of 5.05, and ANN has the worst MAPE, 8.63%. Accordingly, by comparison, our method is superior to other models in fitting and predicting. At the same time, the left image in Figure 4 depicts the APE of each year from 2004 to 2019, and it can be observed that the APEs are lower than other models in most of the years.

3.1.2.   CPCWS of Liaoning province

As in the first case, our model is compared with GM(1, 1), DGM(1, 1), FGM(1, 1), and ANN in fitting and predicting the CPCWS of Liaoning province. The data are empirically divided into two groups, the data from 2004 to 2015 are used to verify the fitting accuracy of the grey models and the data from 2016 to 2019 is applied to test the predicting accuracy. In this case, the optimal order $r$ is estimated with optimizers ALO, PSO, WOA and GWO, respectively, and the track of seeking the optimum parameters is shown in Figure 5. The $r$ is obtained after the optimizer is repeated 100 times shown in Figure 6. It can be seen that all the algorithms are stable, and the optimal $r$ is 0.196.

The corresponding MAPEs are exhibited in Table 3. Figure 7 visualizes the model errors of Table 3, where APE is the prediction error of the model at each point, and the MAPE is the corresponding relative error. From Table 3 and Figure 7, it can be seen that the MAPE values of five models are 1.59%, 1.59%, 0.97%, 2.0%, and 0.96% in fitting the CPCWS of Liaoning province, and the MAPE values of these competitors are 9.00%, 8.99%, 6.05%, 7.55%, and 6.02% for predicting the CPCWS, respectively. It can be observed that our model achieves the best MAPEs in both fitting and predicting CPCWS of Liaoning Province.

Overall, the proposed model, WFGM (1, 1), has higher accuracy than other models in forecasting CPCWS of Hebei and Liaoning provinces, and it can provide relatively reliable support for electric systems so as to assist decision-makers in understanding the trend of electricity consumption, and further formulate corresponding strategies in advance.

3.2.   Fitting and prediction of PC

Energy is the lifeblood of industry and the foundation of economic development. The accurate prediction of the city's medium and long-term power consumption is related to the development of a city and related to the power supply enterprises on the transmission, scheduling and other issues. We employ the proposed WFGM (1, 1) to fit the power consumption from 2011 o 2019 in 30 regions in China and the results are depicted in Table 4. Figure 8 depicts the track of seeking the optimal parameter of WFGM(1, 1) with GWO in fitting the power consumptions of 30 cities. Then, we use WFGM(1, 1) to forecast the power consumption of 30 regions from 2020 to 2029 and the result is exhibited in Table 5. The results contribute to decision-making. For example, whether China has enough energy to support its energy-intensive development and how much energy it will need to meet its economic needs in the coming decades.

3.3.   Fitting and prediction of EE

Educational expenditure (EE) is an essential financial condition for running a school. In China, educational expenditure mainly refers to the state's expenditure for the development of educational undertakings at all levels, which is the material basis and guarantee for the realization of educational tasks, and it plays a fundamental role in the development of education. We fit and predict the EE of the Inner Mongolia Autonomous Region with the proposed WFGM(1, 1) model. The data from 2002 to 2009 are used for fitting and 2010 and 2011 are used for prediction.

In order to further verify the validity of the proposed model, we compare WFGM(1, 1) with the latest grey model and popular approaches including support vector regression (SVM), linear regression (LR), Artificial Neural Network (ANN), Bayesian linear regression (BLR), and conformable fractional grey model (CFGM)^[20]. The best order is obtained with PSO and the order $r$ is 0.28626. The results are shown in Table 6. It's noted that the machine learning model (such as SVM, LR, ANN and BLR) is predicted by a rolling mode, and here it is one roll for every two data, so the first two values are not output in the fitting stage.

It can be observed that the MAPE values of six models are 1.59%, 1.59%, 0.97%, 2.0%, and 0.96% in fitting the EE of Inner Mongolia Autonomous Region, and the MAPE values of these competitors are 9.3402%, 5.2099%, 5.8702%, 5.3506%, 4.8392%, and 3.6413% for predicting the EE, respectively. It shows that our model achieves the best MAPEs in fitting, followed by the CFGM model. In the prediction stage, the MAPE values are 8.6512%, 3.4826%, 3.2954%, 2.9570%, 4.8392%, and 2.4530%, respectively. It exhibits that our model achieves the best MAPEs in prediction, followed by the BLR model. Accordingly, the WFGM (1, 1) model achieved the best result both in fitting and prediction.

4.   Conclusions and discussion

This paper aims to further enhance the prediction performance of the classic fractional grey model (FGM), and develops an optimized fractional grey model by incorporating the weighted least square method, as a consequence, an extended fractional grey model (abbreviated as WFGM(1, 1)) is presented. The primary conclusions can be summarized as follows.

(1) We introduce the traditional fractional grey model and on this basis, we develop an optimised factional grey model by combining the grey modelling technique with the weighted least square method. By doing so, an enhanced grey model is presented to further improve the prediction accuracy of the model already in place. After that, we estimate the model parameters, determine weights in the coefficient matrix and derive the time response function.

(2) In particular, for convenience purposes, we introduce four advanced algorithms to search for the optimal fractional accumulation order and weights, which are the grey wolf optimization (GWO), whale optimization algorithm (WOA), particle swarm optimization (PSO) and ant lion algorithm (ALO). By observing error-value metrics by these four algorithms, we conclude the GWO algorithm has a better performance in numerical experiments.

(3) To demonstrate the superiority of the presented model, we apply it to three real-world data sets. It is clear that the proposed model has a higher prediction accuracy in these data sets, indicating the presented model can achieve successful prediction in these experiments, also verify that the proposed model can increase the prediction performance of the classic fractional grey model.

The least squares algorithm can find the best function match of data by minimizing the sum of squares of error. It can be used to obtain the unknown data easily and minimize the sum of squares between the obtained data and the actual data. When both the independent variable and the dependent variable have random errors with zero mean and the same variance, this method can give the best parameter-fitting results in the statistical sense. Using the weighted least squares algorithm in the estimation of grey model parameters can prevent local amplification, increase the stability of its stability estimation parameters, and effectively prevent overfitting.

Nevertheless, in reality, much of the data is a time series generated by a nonlinear system. Time series can be regarded as the mapping of a nonlinear system in one-dimensional space, so if a linear model is used to describe the dynamic changes of a nonlinear system, it will produce an inaccurate estimation. In this paper, the proposed model is linear, therefore, in future work, we will propose a nonlinear grey prediction model based on the least squares algorithm to broaden the application of the grey models.

Acknowledgments

The work in this paper was supported by National Natural Science Foundation of China (Grant No. 62007028), and Doctoral Research Foundation of Jiangsu Normal University(NO. 21XSRX005).

Conflict of interest

No potential conflicts of interest was reported by the authors.

Nomenclature

$r$    The order of accumulation generation

$x^{(r)}(k)$    Fractional order cumulative sequence, $k = 1, 2, 3\cdots, n$

$X^{0}$    Original time series

$(a, b)$    Parameter to be estimated

$z^{(r)}(k)$    Background value coefficient

$W$    The weighted matrix

ANN    Artificial Neural Network

MAPE    The mean absolute percentage error

GM(1, 1)    Univariate grey prediction model

DGM(1, 1)    Univariate discrete grey prediction model

FGM(1, 1)    Fractional order prediction model

WFGM(1.1)    Grey prediction model based on weighted least squares