A comparison of methodologies in the stress testing of credit risk – alternative scenario and dependency constructs

Michael Jacobs Jr.; Frank J. Sensenbrenner; Michael Jacobs Jr.; Frank J. Sensenbrenner

doi:10.3934/QFE.2018.2.294

Quantitative Finance and Economics

2018, Volume 2, Issue 2: 294-324. doi: 10.3934/QFE.2018.2.294

Previous Article Next Article

Research article

A comparison of methodologies in the stress testing of credit risk – alternative scenario and dependency constructs

Michael Jacobs Jr. ^{1
,
,},
Frank J. Sensenbrenner ²

1.
Ph.D., CFA, Principal Director, Accenture Consulting, Finance and Risk Services Advisory/Models, Methodologies & Analytics, 1345 Avenue of the Americas, New York, USA
2.
Non-Visiting Fellow, US Commodity Futures Trading Commission, 525 West Monroe Street Chicago, IL, 60611, USA

Received: 10 December 2017 Accepted: 13 March 2018 Published: 27 April 2018
JEL Codes: C31, C53, E27, E47, E58, G01, G17, C54, G21, G28, G38

In the aftermath of the financial crisis of the last decade, banking supervisors have sought the solution to the problem of determining the optimal capital levels that an institution should hold, in order to support their risk taking activities. The experience of this financial downturn has given rise to the conclusion that traditional approaches, such as regulatory or economic capital are inadequate to this end, leading to the prevalence of supervisory stress testing as a primary tool of prudential supervision. A critical input into this process is the set of macroeconomic scenarios, either provided by the prudential supervisors, or developed by financial institutions. Prevalent among approaches in the industry is the combination of expert opinion and an econometric methodology, for example the Vector Autoregression ("VAR") model that captures the dependency structure among and between macroeconomic explanatory variables and banking loss / income target variables. Despite the prevalence of this approach, we know from the previous finance literature that Gaussian VAR models are unable to cope with the empirical fact of deviation from normality. In this paper we investigate the alternative Markov Switching VAR ("MS-VAR") model, featured more commonly in the academic realm as opposed to being applied in practice. We conduct an empirical experiment using data from regulatory filings and Federal Reserve macroeconomic data released by the regulators for mandated stress testing exercises. Our finding is that the MS-VAR model performs better than the VAR model, both in terms of producing severe scenarios conservative than the VAR model, as well as showing superior predictive accuracy. Furthermore, we find that the multiple equation VAR model outperforms the single equation autoregressive ("AR") models according to various metrics across all modeling segments.
- stress testing,
- CCAR; DFAST,
- credit risk,
- financial crisis,
- model risk,
- vector autoregression,
- Markov switching model,
- scenario generation
Citation: Michael Jacobs Jr., Frank J. Sensenbrenner. A comparison of methodologies in the stress testing of credit risk – alternative scenario and dependency constructs[J]. Quantitative Finance and Economics, 2018, 2(2): 294-324. doi: 10.3934/QFE.2018.2.294

Related Papers:

Abstract

In the aftermath of the financial crisis of the last decade, banking supervisors have sought the solution to the problem of determining the optimal capital levels that an institution should hold, in order to support their risk taking activities. The experience of this financial downturn has given rise to the conclusion that traditional approaches, such as regulatory or economic capital are inadequate to this end, leading to the prevalence of supervisory stress testing as a primary tool of prudential supervision. A critical input into this process is the set of macroeconomic scenarios, either provided by the prudential supervisors, or developed by financial institutions. Prevalent among approaches in the industry is the combination of expert opinion and an econometric methodology, for example the Vector Autoregression ("VAR") model that captures the dependency structure among and between macroeconomic explanatory variables and banking loss / income target variables. Despite the prevalence of this approach, we know from the previous finance literature that Gaussian VAR models are unable to cope with the empirical fact of deviation from normality. In this paper we investigate the alternative Markov Switching VAR ("MS-VAR") model, featured more commonly in the academic realm as opposed to being applied in practice. We conduct an empirical experiment using data from regulatory filings and Federal Reserve macroeconomic data released by the regulators for mandated stress testing exercises. Our finding is that the MS-VAR model performs better than the VAR model, both in terms of producing severe scenarios conservative than the VAR model, as well as showing superior predictive accuracy. Furthermore, we find that the multiple equation VAR model outperforms the single equation autoregressive ("AR") models according to various metrics across all modeling segments.

References

[1]	Acharya VV, Schnabl P, Richardson M (2010) How Banks Played the Leverage Game. John Wiley 18: 144–145.
[2]	Bangia A, Diebold FX, Kronimus A, et al. (2002) Ratings migration and the business cycle, with application to credit portfolio stress testing. J Bank Financ 26: 445–474. doi: 10.1016/S0378-4266(01)00229-1
[3]	Bank for International Settlements, Basel Committee on Banking Supervision (BCBS) (1988) Internal convergence of capital measurement and capital standards: A revised framework.
[4]	Basel Committee on Banking Supervision (BCBS) (1996) Amendment to the capital accord to incorporate market risks.
[5]	Basel Committee on Banking Supervision (BCBS) (2006) International convergence of capital measurement and capital standards: A revised framework.
[6]	Basel Committee on Banking Supervision (BCBS) (2009a) Principles for sound stress testing practices and supervision.
[7]	Basel Committee on Banking Supervision (BCBS) (2009b) Guidelines for computing capital for incremental risk in the trading book.
[8]	Basel Committee on Banking Supervision (BCBS) (2009c) Revisions to the Basel II market risk framework.
[9]	Basel Committee on Banking Supervision (BCBS) (2009d) Analysis of the trading book quantitative impact study (October).
[10]	Basel Committee on Banking Supervision (BCBS) (2010a) Strengthening the resilience of the banking sector-consultative document (December).
[11]	Basel Committee on Banking Supervision (BCBS) (2010b) An assessment of the long-term economic impact of stronger capital and liquidity requirements (August).
[12]	Basel Committee on Banking Supervision (BCBS) (2010c) Basel III: A global regulatory framework for more resilient banks and banking systems (December).
[13]	Baum LE, Petrie T (1966) Statistical inference for probabilistic functions of finite state Markov chains. Ann Math Stat 37: 1554–1563. doi: 10.1214/aoms/1177699147
[14]	Baum LE, Petrie T, Soules G, et al. (1970) A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. Ann Math Stat 41: 164–171. doi: 10.1214/aoms/1177697196
[15]	Berkowitz J (1999) A coherent framework for stress-testing. FEDS Work Pap.
[16]	Bidder R, McKenna A (2015) Robust stress testing. Work Pap.
[17]	Blackwell D, Koopmans L (1959) On the identifiable problem for functions of finite Markov chains. Ann Math Stat 28: 1011–1015.
[18]	Board of Governors of the Federal Reserve System (1996) Joint Policy Statement on Interest Rate Risk, Supervisory Letter 96-13, May 23^rd.
[19]	Board of Governors of the Federal Reserve System (1999) Supervisory Guidance Regarding Counterparty Credit Risk Management, Supervisory Letter 99–03, February 1^st.
[20]	Board of Governors of the Federal Reserve System (2002) Interagency Guidance on Country Risk Management, Supervisory Letter 02–05, March 8^th.
[21]	Board of Governors of the Federal Reserve System (2011) Supervisory Guidance on Model Risk Management, Supervisory Letter 11–7, April 4^th.
[22]	Box G, Jenkins G (1976) Time Series Analysis: Forecasting and Control, New York, Wiley, 2^nd Edition.
[23]	Brockwell PJ, Davis RA (1991) Time series: theory and methods. Technometrics 31: 121.
[24]	Commandeur JJF, Koopman SJ (2007) Introduction to state space time Series analysis. RePEc 36: 3–25.
[25]	Dacco R, Satchell S (1999) Why do regime-switching models forecast so badly? J Forecasting 18: 1–16. doi: 10.1002/(SICI)1099-131X(199901)18:1<1::AID-FOR685>3.0.CO;2-B
[26]	Demirguc-Kunt A, Detragiache E, Merrouche O (2013) Bank capital: lessons from the financial crisis. J Money Credit Bank 45: 1147–1164. doi: 10.1111/jmcb.12047
[27]	Dickey DA, Fuller WA (1981) Likelihood ratio statistics for autoregressive time series with a unit root. Econometrica 49: 1057–1072. doi: 10.2307/1912517
[28]	Diebold FX, Mariano RS (2002) Comparing predictive accuracy. J Bus Econ Stat 20: 134–144. doi: 10.1198/073500102753410444
[29]	Embrechts P, Straumann EMD (1999) Correlation: pitfalls and alternatives. Risk 12: 69–71.
[30]	Embrechts P, McNeil AJ, Straumann D (2002) Correlation and dependence in risk management: properties and pitfalls, In: Dempster, M.A.H., ed., Risk Management: Value at Risk and Beyond, Cambridge University Press, Cambridge, UK, 176–223.
[31]	Embrechts P, Lindskog F, McNeil AJ (2003) Modeling dependence with copulas and applications to risk management, In: Rachev, S., ed., Handbook of Heavy Tailed Distributions in Finance, Elsevier, Rotterdam, 329–384.
[32]	Engel C (1994) Can the Markov switching model forecast exchange rates? J Int Econ 36: 151–165. doi: 10.1016/0022-1996(94)90062-0
[33]	Frame RS, Fuster A, Tracy J, et al. (2015) The rescue of Fannie Mae and Freddie Mac. J Econ Perspect 29: 25–52. doi: 10.1257/jep.29.2.25
[34]	Foglia A (2009) Stress testing credit risk: a survey of authorities' approaches. Soc Sci Electron Publ 5: 9–45.
[35]	Frey R, McNeil AJ (2001) Modeling dependent defaults, Work Pap.
[36]	Goldfeld SM, Quandt RE (1973) A Markov model for switching regressions. Journal Econ 1: 3–15. doi: 10.1016/0304-4076(73)90002-X
[37]	Hamilton JD (1988) Rational expectations econometric analysis of changes in regime: an investigation of the term structure of interest rates. J Econ Dyn Control 12: 385–423. doi: 10.1016/0165-1889(88)90047-4
[38]	Hamilton JD (1989) A new approach to the economic analysis of nonstationary time series and the business cycle. Econometrica 57: 357–384. doi: 10.2307/1912559
[39]	Hanan EJ (1971) The identification problem for multiple equation systems with moving average errors. Econometrica 39: 751–766. doi: 10.2307/1909577
[40]	Hanan EJ, Deistler M (1988) The statistical theory of linear systems. J Wiley 32: 292–293.
[41]	Heller A (1965) On stochastic processes derived from Markov chains. Ann Math Stats 36: 1286–1291. doi: 10.1214/aoms/1177700000
[42]	Inanoglu H, Jacobs, Jr. M (2009) Models for risk aggregation and sensitivity analysis: an application to bank economic capital. J Risk Financ Manage 2: 118–189. doi: 10.3390/jrfm2010118
[43]	J.P. Morgan (1994) RiskMetricsTM, Second Edition, New York.
[44]	J.P. Morgan (1997) CreditMetricsTM, First Edition, New York
[45]	Jacobs, Jr. M (2010) Validation of economic capital models: state of the Practice, supervisory expectations and results from a bank study. J Risk Manage Finance Inst 3: 334–365.
[46]	Jacobs, Jr. M (2013) Stress Testing Credit Risk Portfolios. J Financ Transform 37: 53–75.
[47]	Jacobs, Jr. M, Karagozoglu A, Sensenbrenner F (2015) Stress testing and model validation: application of the Bayesian approach to a credit risk portfolio. J Risk Model Validation 9: 1–70.
[48]	Jacobs, Jr. M (2015) The impact of asset price bubbles on credit risk measures. J Financ Risk Manage 4: 251–266. doi: 10.4236/jfrm.2015.44019
[49]	Jacobs, Jr. M (2016) Stress testing and a comparison of alternative methodologies for scenario generation. J Appl Finan Bank6: 123–156.
[50]	Jacobs, Jr. M (2017a) A mixture of distributions model for the term structure of interest rates with an application to risk management. Am Res J Bus Manage 3: 1–17.
[51]	Jacobs, Jr. M (2017b) Asset price bubbles and the quantification of credit risk capital with sensitivity analysis, empirical implementation and an application to stress testing. J Risk Model Validation 11: 1–35.
[52]	Jacobs, Jr. M (2018) The validation of machine learning models for the stress testing of credit risk. Int J Econ Manage Sci 7: 1–21.
[53]	Jorion P (1996) Risk2: measuring the Value at Risk. Financ Analyst J 52: 47–56. doi: 10.2469/faj.v52.n6.2039
[54]	Jorion P (1997) Value at Risk: The New Benchmark for Controlling Market Risk, volume 2.
[55]	Jorion P (2006) Value at Risk: The Benchmark for Managing Financial Risk, Third Ed.
[56]	Kwiatkowski D, Phillips PCB, Schmidt P, et al. (1992) Testing the null hypothesis of stationarity against the alternative of a unit root: how sure are we that economic time series have a unit root? J Econometrics 54: 159–178. doi: 10.1016/0304-4076(92)90104-Y
[57]	Kohn R (1979) Asymptotic results for ARMAX structures. Econometrica 47: 1295–1304. doi: 10.2307/1911964
[58]	Koyluoglu H, Hickman A (1998) Reconcilable differences. Risk 56–62.
[59]	Kupiec PH (2009) Risk capital and VaR. J Deriv 7: 41–52.
[60]	Li DX (1999) On default correlation: A copula function approach. J Fix Inc 9: 43–54.
[61]	Loregian A, Meucci A (2013) Neither "Normal" nor "Lognormal": modeling interest rates across all regimes. Financ Analyst J 72: 68–82.
[62]	Merton R (1974) On the pricing of corporate debt: the risk structure of interest rates. J Financ 29: 449–470.
[63]	Mosser PC, Fender I, Gibson MS (2001) An international survey of stress tests. Curr Issues Econ Finan 7: 1–6.
[64]	Pearson K (1895) Contribution to the mathematical theory of evolution. Philos Trans Royal Soc 185: 71–110.
[65]	Poon SH, Rockinger M, Tawn J (2004) Extreme value dependence in financial markets: diagnostics, models and financial implications. Rev Financ Stud 17: 581–610. doi: 10.1093/rfs/hhg058
[66]	R Development Core Team (2017) R: A Language and Environment for Statistical Computing.
[67]	Rebonato R (2010) Coherent Stress Testing: A Bayesian Approach. Wiley, New York.
[68]	Schuermann T (2014) Stress testing banks. Int J Forecasting 30: 717–728. doi: 10.1016/j.ijforecast.2013.10.003
[69]	Sims CA (1980) Macroeconomics and reality. Econometrica 48: 1–48. doi: 10.2307/1912017
[70]	Stock JH, Watson MW (2001) Vector autoregressions. J Econ Perspect 15: 101–115. doi: 10.1257/jep.15.4.101
[71]	Tjøstheim D (1986) Estimation in nonlinear time series models. Stoch Process their Appl 21: 251–273. doi: 10.1016/0304-4149(86)90099-2
[72]	Wilde T (1997) CreditRisk+ A Credit Risk Management Framework. Credit Suisse First Boston.

Reader Comments

Your name:*

Email:*
© 2018 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)