An evaluation of quantitative easing effectiveness based on out-of-sample forecasts

1.   Introduction

The extensive and rather persistent use of unconventional monetary policies in the aftermath of the great financial crisis of 2008 has now offered adequate data to help us assess the efficacy of new tools that have been employed in a liquidity trap environment with nominal policy rates close to the zero lower bound. The main unconventional tool used in this setting has been the purchase of both long-term public sector assets and corporate bonds, also known as quantitative easing (QE), and the relevant purchases are reflected in increases in central bank assets and the monetary base.

The case for QE is based on a portfolio balance effect and a signaling effect (Bernanke, 2020). The portfolio balance effect works through the elimination of the duration risk of long-term sovereign bonds, thus inducing market participants to bid up the prices of similar assets, while the signaling effect works through its impact on expected short-term interest rates. The latter results from the commitment to low policy rates inferred from central bank announcements regarding the extent and the time length of asset purchases. Then, an important policy question is whether these effects are lasting or transitory, since both cases are theoretically possible. For example, in the preferred-habitat model of Vayanos and Vila (2009) asset demand shocks generate lasting portfolio effects when investors exhibit preferences for assets with specific maturities. However, in the intertemporal approach of Eggertsson and Woodford (2003) expansions of the monetary base through QE purchases have only a transitory effect on asset yields and aggregate expenditure. In recent research, Cebula and Rossi (2022) showed that QE influences macroeconomic stability conditions and has a considerable impact on the size of discretionary policy multipliers in the context of an IS-LM model. Also, the theoretical importance of QE is verified in Bhattarai and Neely (2022), who suggest that unconventional policies seem appropriate only in exceptional circumstances like liquidity trap conditions.

On the empirical side, one line of arguments in favor of QE was based on event studies which explored the reaction of asset prices and yields in short periods around QE announcements by central banks. For example, Gagnon et al. (2011) found considerable effects on 10-year T-bonds from the first round of asset purchases by the Fed (QE1), over the period 2008–2010, while the limited reaction of bond yields to subsequent rounds of QE has been attributed to anticipation of new asset purchases which appears to have been taken into account in advance of the respective official QE announcements (e.g., Gagnon, 2018). Also, Cahill et al. (2013) presented evidence that portfolio balance effects operate when there is a discrepancy between the actual and the expected allocation of QE purchases among different assets, leading to substantial changes in relative asset prices, that is, lower prices and higher yields for those assets whose purchase shares turn out to be lower than expected.

Nevertheless, other studies (e.g., Greenlaw et al., 2018; Wright, 2012) have reported evidence that the effects of QE policies on long-term yields are rather short-lived and smaller than what they initially appeared to be, thus making such interventions less likely to have a substantial impact on aggregate demand. A counterargument to these findings was developed around the claim that if price and yield changes are short-lived then market participants should be able to form profitable strategies by betting on such reversals. In that vein, Neely (2022) found that a vector autoregression does not forecast asset prices out of sample as well as a naive model and, therefore, inferred that there are no exploitable profit opportunities since changes in asset prices and yields after unconventional monetary interventions appear to be persistent.

Following this line of research, in this work we compare the out-of-sample forecasting performance of a model that accounts for price reversals to that of a naive random walk process. In particular, we use a Markov switching regimes stochastic representation with unobserved states to produce one- to six-month out-of-sample forecasts of long-term (10-year) sovereign bond yields for European areas and countries which have pursued QE policies over the last decade, namely the Eurozone, Sweden, and the UK. We estimate the model on monthly data which cover the period from January 2008 to September 2021, extracted from the OECD statistical database (stats.oecd.org), and the generated forecasts are compared to those of a random walk with drift by means of their root mean squared error (RMSE). We find that the Markov process gives equally good forecasts to those of a naive model, in all cases, and, given the additional information conveyed by the Markov representation regarding the probability of regime shifts, we can infer that there are instances in which agents perceive policy regimes as temporary, thus reversing the impact of QE purchases and limiting the effectiveness of unconventional monetary interventions.

The choice of a Markov switching regimes process with unobserved states to describe the dynamics of long-term bond yields, under QE interventions, has the advantage that the dates of the switch are not prespecified, but we allow the data to tell us if and when such switches occur. Indeed, the state variable, associated with policy regimes, is considered to be unobserved and governed by a Markov chain which makes it possible to derive the probability that the process is in a particular state at any point in time, using information either up to that date or from the full sample of observations.¹

¹The probability of a particular regime, based on information up to a particular date, is known as filter probability, while the probability based on information drawn from the full sample is referred to as smoothed probability.

Thus, these probabilities reflect the perceptions of market participants regarding the likelihood of policy changes and this information is crucial in forming market stances. In other words, if out-of-sample Markov forecasts are at least as good as naive predictions and the probabilistic inferences about the state of the process imply frequent regime shifts, then agents can use the additional information conveyed by the switching model to try to beat the market, thus offsetting the intended changes in asset prices and yields. Also, another reason for choosing the Markov representation for long-term yields is that it appears to capture the dynamics implied by the observed changes in the monetary base (the instrument of unconventional monetary policy) in countries which actively pursued QE after 2008 (see Kirikos, 2020, 2021).

The generation of out-of-sample forecasts is discussed in the next section and then, in the third section, we report the RMSEs of forecasts as well as the additional information extracted from the Markov process in terms of probabilistic inferences regarding the state of the process at any time. The final section contains concluding remarks.

2.   Naive and Markov Forecasts

Naive forecasts will be based on a random walk with drift. Specifically, under a random walk with drift, the yield of an asset, say ${y}_{t}$, will have the representation:

where d is the drift and ${\varepsilon }_{t}$ is a white noise error term. Then, the time-t forecast of ${y}_{t+k}$ will be:

where $\widehat{y}$ denotes forecast of y and $\bar {d} = \frac{1}{t-1}\sum _{i = 1}^{t-1}({y}_{i+1}-{y}_{i})$.

An alternative stochastic representation that allows for yield (and asset price) reversals is the following:

where ${\omega }_{t}$ is a normally distributed error term, ${z}_{t}$ is a state variable that cannot be observed and takes on the values 1 or 2, that is, the change in y is expected to be either μ₁ or μ₂ depending on the state of the process, and the variance differs across regimes. Since the state variable is unobserved, the regime must be inferred probabilistically on the basis of information conveyed by the data, and this requires the specification of a law of motion for ${z}_{t}$. The latter will be taken to be an irreducible Markov chain with transition probabilities ${\pi }_{ij} = Pr({z}_{t} = j|{z}_{t-1} = i)$, $i = \mathrm{1, 2}$, $j = \mathrm{1, 2}$, and stationary probability matrix:

Estimation of the parameters (${\mu }_{1}$, ${\mu }_{2}$, ${\pi }_{11}$, ${\pi }_{22}$, ${\sigma }_{1}^{2}$, ${\sigma }_{2}^{2}$) of the Markov switching regimes model through the EM algorithm (see Hamilton, 1990, 1993) requires that the series be stationary, which, as we shall see in a following section, is true for the first differences of sovereign bond yields. Therefore, the process in Equation (3) can be rewritten as:

and the period-t forecast of $\Delta {y}_{t+k}$ is:

where the information set I_t includes all data up to time t, ${p}^{{'}} = [\begin{array}{cc}Pr({z}_{t} = 1|{I}_{t})& Pr({z}_{t} = 2|{I}_{t}\left)\right]\end{array}$ is the vector of filter inferences about the state of the process at period t, Π is the transition probability matrix, k is the forecast horizon, and ${\mu }_{z}^{{'}} = \left[\begin{array}{cc}{\mu }_{1}& {\mu }_{2}\end{array}\right]$ is the vector of state means.

Forecasts given by Equation (6) are non-linear since the filter probabilities in the vector $p{'}$ are non-linear functions of the data and so are the Markov model parameters. Also, k-period-ahead forecasts of the level of the series y, as of time t, are taken as:

For all models, out-of-sample forecasts are computed through rolling estimation of the stochastic processes. That is, we start estimation with a sub-sample of size m and compute the k-period-ahead out-of-sample forecast, and then the models are re-estimated using the sub-sample $m+1$ by adding the next available observation of the series and take a new k-period-ahead forecast. This recursive estimation goes on until the size of the sub-sample becomes $T-k$, where T is the full sample size, and this procedure generates $T-m-k+1$ out-of-sample forecasts, where m is the initial sub-sample and $k$ is the forecast horizon. Thus, the root mean squared error (RMSE)² of these forecasts is:

²It should be noted that the relative forecasting performance of the models, reported in the next section, does not change when the Mean Absolute Error (MAE) of forecasts is used instead of the RMSE.

Apparently, if the Markov model predicts asset yields out of sample as well as a naive model, that is, the RMSE of its forecasts is not higher than that of simple forecasts, then agents could use the inference on regime switches to bet on asset price reversals, thus rendering the effects of QE short-lived. In other words, a good forecasting performance of the switching regimes process suggests that QE effects are perceived as transitory and, therefore, the effectiveness of unconventional monetary policy is rather limited.

3.   Data and results

Monthly data on the yields of 10-year sovereign bonds of Eurozone, Sweden, and UK are drawn from the Main Economic Indicators dataset of the OECD database (stats.oecd.org) and cover the period from January 2008 until September 2021 (165 observations). These countries have been selected because their monetary authorities pursued QE policies in the aftermath of the great financial crisis of 2008, mainly through the purchase of long-term government bonds, and the summary statistics of the yield series (measured in percentages) are given in Table 1 both for the levels and for their first differences (change).

Since estimation of the Markov process must be based on stationary series, we conducted some preliminary tests regarding the presence of unit roots in long-term government bond yields. The results of the KPSS test (see Kwiatkowski et al., 1992) for testing the null hypothesis that a series is stationary are reported in Table 2 and reveal that the stationarity is rejected for the levels of the series but not for their first differences. Therefore, the following Markov forecasts are based on estimates derived from the differenced series.

Initial estimates of the models are based on data up to December 2018 and over periods in which the corresponding central banks pursued QE asset purchases, that is, from January 2015 for the Eurozone (48 observations),³ January 2012 for Sweden (84 observations), and March 2009 for the UK (118 observations). Out-of-sample forecasts from 1 to 6 months ahead are computed through rolling estimation over the post-sample period so that the final forecast, for all forecast horizons, corresponds to September 2021.⁴ The RMSE of forecasts for the post-sample period January 2019 to September 2021 are given in Table 3.

³In fact, the European Central Bank began QE operations in 2014, but a formal decision was announced in early 2015.

⁴Thus, when the post-sample period is January 2019 to September 2021, there are 33 one-month, 32 two-month, 31 three-month, 30 four-month, 29 five-month, and 28 six-month out-of-sample forecasts.

To compare the models more easily, in terms of their forecasting performance, we depict the RMSEs of different processes in the following graphs (Figure 1a–1c) for the post-sample period from January 2019 until September 2021. Apparently, the graphs reveal that, in all cases considered, naive forecasts are not better than Markov forecasts, and this makes the Markov switching process more useful since it provides additional information regarding the likelihood of a regime change. That is, if there are regime shifts and agents can predict them, then they can form short-run profitable strategies by betting on yield and price reversals that offset the effects of central bank intervention through quantitative easing. These actions, whenever possible, reduce the effectiveness of QE policies making it only temporary.

However, before we conclude on the advantages of the switching regimes process in forecasting bond yields out of sample, we must test the presence of Markovian dynamics, and this can be carried out by means of Wald tests of the presence of different regimes and different means across them. More specifically, the existence of different states can be tested through the null hypothesis H_o: ${\pi }_{11} = 1-{\pi }_{22}$, which implies that going to a particular state does not depend on the previous state, that is, the Markov property does not hold. In addition, if the Markov property is verified, a similar test of the null hypothesis H_o: ${\mu }_{1} = {\mu }_{2}$ can tell us whether the expected change in yields is different across regimes. Thus, if these null hypotheses are not rejected, then there is no superiority in the Markov process relative to a naive model and, therefore, QE could have long-term effects on the grounds of yield unpredictability. To conduct the tests, we estimated the Markov process via the EM algorithm, over the full period of QE interventions of each central bank, and the parameter estimates are reported in Table 4.

The Wald test statistics of the null hypotheses H_o: ${\pi }_{11} = 1-{\pi }_{22}$ and H_o: ${\mu }_{1} = {\mu }_{2}$ are respectively:

where a ^ over a parameter denotes the relevant estimate through the EM algorithm. The values of these statistics and their significance levels are reported in Table 5, for the periods over which each central bank has implemented QE purchases. Obviously, the Markov property is not rejected for the Eurozone, Sweden, and the UK which also appear to have statistically significant differences in the mean yield change across regimes. Thus, for these European areas and countries the predictability of regime shifts and of the ensuing long-term sovereign bond yield changes suggest that the effects of QE interventions are rather transitory.

An additional advantage of the Markov switching process is that it provides information on agents' perceptions regarding the state of policy regime. This information is derived from the full sample of observations through the so-called smoothed probabilities that the process is in a particular state at time t, that is, the conditional probabilities $Pr({z}_{t} = i|{y}_{1}, {y}_{2}, \dots, {y}_{r})$, where $i = \mathrm{1, 2}$ and $T$ is the full sample. These probabilities are easily derived as a by-product of the estimation through the EM algorithm. Thus, assuming that agents perceive a particular regime of yield changes, say regime $i$, whenever $Pr({z}_{t} = i|{y}_{1}, {y}_{2}, \dots, {y}_{r}) > 1/2$, then our estimates show that agents have inferred multiple regime shifts using the largest available set of information.

Indeed, in the case of the Eurozone, estimation of the Markov process for the period after January 2014 and up to September 2021, over which the European Central Bank pursued expansionary monetary policy and actively engaged in QE policies, the smoothed probability of the state of decreasing bond yields, depicted in Figure 2a along with the yield of euro sovereign bonds, shows that the process captures very well the periods of falling bond yields (i.e. when the smoothed probability is greater than ½) which coincide with periods of QE interventions. At the same time, the estimated smoothed probabilities reveal that agents expected changes in policy as the process appears to have switched between states 10 times, over the period of QE purchases 2014–2021.

Similarly, estimation of the switching process for Sweden and the UK, using the QE periods January 2012 to September 2021 and March 2009 to September 2021, respectively, produced the smoothed inferences depicted in Figure 2b and 2c. These conditional probabilities show that the model identifies the turning points of Swedish and British long-term bond yield series remarkably well and suggests that agents have not perceived the monetary expansion through QE as permanent since the processes have often switched between increasing and decreasing yield states. Indeed, starting in 2012 the Riksbank has acquired Swedish government bonds totaling SEK 373 billion in September 2021, while the Bank of England has purchased UK government bonds amounting to £875 billion between March 2009 and November 2020. However, these programs have not eliminated interim periods of falling asset prices and rising yields which reverse the effects on long-term interest rates sought by QE policies.

4.   Conclusions

The claim that QE policies have persistent effects on long-term interest rates, because models that account for asset price reversals do not outperform naive out-of-sample forecasts of bond yields, is not empirically supported for European central banks which have carried out extensive QE interventions. Specifically, using data on 10-year sovereign bond yields, we have obtained evidence that a naive random walk model does not beat a Markov switching regimes process in out-of-sample forecasting and that the Markov representation⁵ captures accurately the dynamics of the series for the Eurozone, Sweden. and the UK. This implies that interim halts in asset purchases generate regime switches which reverse the effects of QE interventions, thus reducing the effectiveness of unconventional monetary policy. Besides, these results corroborate those of Kirikos(2020, 2021) that QE policies do not have sizable effects on inflation, investment, and broad monetary aggregates in the decade after the 2008 financial crisis.

⁵Of course, other specifications, like a state-space representation (see Dimitriou et al., 2021), can be useful in checking the robustness of our results. In any case, similar out-of-sample forecasts under an augmented information set which includes the instrument of QE policy, namely the monetary base, reinforce our current findings (see Kirikos, 2022).

Of course, our findings should cautiously be interpreted as evidence of limited QE effectiveness and not as an indication of complete policy failure since long-term interest rates have overall declined throughout the period of QE pursuit. However, taking also into account the fall in inflationary expectations over the same period, this evidence has the important implication that QE policy appears to have transitory effects on long-term rates, and this limits its usefulness as a tool of monetary policy outside periods of financial turmoil.

Conflict of Interest

The author declares no conflicts of interest in this paper.