Proving prediction prudence

Dirk Tasche; Dirk Tasche

doi:10.3934/DSFE.2022017

Data Science in Finance and Economics

2022, Volume 2, Issue 4: 335-355. doi: 10.3934/DSFE.2022017

Previous Article Next Article

Research article Special Issues

Proving prediction prudence

Dirk Tasche ^,

Independent researcher, Switzerland

Received: 15 August 2022 Revised: 02 September 2022 Accepted: 11 September 2022 Published: 23 September 2022
JEL Codes: G21, C12

We study how to perform tests on samples of pairs of observations and predictions in order to assess whether or not the predictions are prudent. Prudence requires that the mean of the difference of the observation-prediction pairs can be shown to be significantly negative. For safe conclusions,we suggest testing both unweighted (or equally weighted) and weighted means and explicitly taking into account the randomness of individual pairs. The test methods presented are mainly specified as bootstrap and normal approximation algorithms. The tests are general but can be applied in particular in the area of credit risk,both for regulatory and accounting purposes.
- paired difference test,
- weighted mean,
- credit risk,
- probability of default (PD),
- loss given default (LGD),
- exposure at default (EAD),
- credit conversion factor (CCF)
Citation: Dirk Tasche. Proving prediction prudence[J]. Data Science in Finance and Economics, 2022, 2(4): 335-355. doi: 10.3934/DSFE.2022017

Related Papers:

Abstract

We study how to perform tests on samples of pairs of observations and predictions in order to assess whether or not the predictions are prudent. Prudence requires that the mean of the difference of the observation-prediction pairs can be shown to be significantly negative. For safe conclusions,we suggest testing both unweighted (or equally weighted) and weighted means and explicitly taking into account the randomness of individual pairs. The test methods presented are mainly specified as bootstrap and normal approximation algorithms. The tests are general but can be applied in particular in the area of credit risk,both for regulatory and accounting purposes.

References

[1]	BCBS. International Convergence of Capital Measurement and Capital Standards. A Revised Framework, Comprehensive Version, 2006.
[2]	Bellini T(2019) IFRS 9 and CECL Credit Risk Modelling and Validation: A Practical Guide with Examples Worked in R and SAS. Academic Press.
[3]	Blümke O(2019) Out-of-Time Validation of Default Probabilities within the Basel Accord: A comparative study. SSRN Electron J. http://dx.doi.org/10.2139/ssrn.2945931
[4]	Casella G, Berger RL (2002) Statistical Inference. Duxbury Press, second edition.
[5]	Davison AC, Hinkley DV (1997) Bootstrap Methods and their Application. Cambridge University Press.
[6]	ECB. Instructions for reporting the validation results of internal models: IRB Pillar I models for credit risk. European Central Bank – Banking Supervision, 2019.
[7]	Fischer M, Pfeuffer M (2014) A statistical repertoire for quantitative loss given default validation: overview, illustration, pitfalls and extensions. J Risk Model Validation 8: 3–29. http://dx.doi.org/10.21314/JRMV.2014.115 doi: 10.21314/JRMV.2014.115
[8]	Gürtler M, Hibbeln MT, Usselmann P (2018) Exposure at default modeling – A theoretical and empirical assessment of estimation approaches and parameter choice. J Bank Financ 91: 176–188. http://dx.doi.org/10.1016/j.jbankfin.2017.03.004 doi: 10.1016/j.jbankfin.2017.03.004
[9]	Hand DJ (1997) Construction and Assessment of Classification Rules. John Wiley & Sons, Chichester.
[10]	Kazianka H (2016) Objective Bayesian estimation of the probability of default. J R Stat Soc Ser C 65: 1–27, 2016. http://dx.doi.org/10.1111/rssc.12107 doi: 10.1111/rssc.12107
[11]	Li D, Bhariok R, Keenan S, et al. (2009) Validation techniques and performance metrics for loss given default models. J Risk Model Validation 3: 3–26. http://dx.doi.org/10.21314/JRMV.2009.045 doi: 10.21314/JRMV.2009.045
[12]	Loterman G, Debruyne M, Vanden BK, et al. (2014) A proposed framework for backtesting loss given default models. J Risk Model Validation 8: 69–90. http://dx.doi.org/10.21314/JRMV.2014.117 doi: 10.21314/JRMV.2014.117
[13]	Mendenhall W, Beaver RJ, Beaver BM (2008) Introduction to probability and statistics. Cengage Learning, 13th edition.
[14]	Rolski T, Schmidli H, Schmidt V, et al. (1999) Stochastic Processes for Insurance and Finance. Wiley Series in Probability and Statistics. John Wiley Sons.
[15]	Scandizzo S (2016) The validation of risk models: A handbook for practitioners. Springer. https://doi.org/10.1057/9781137436962
[16]	Tasche D (2013a) The art of probability-of-default curve calibration. J Credit Risk, 9: 63–103. https://doi.org/10.21314/JCR.2013.169 doi: 10.21314/JCR.2013.169
[17]	Tasche D (2013b) The Law of Total Odds. Working paper. https://doi.org/10.48550/arXiv.1312.0365.
[18]	Venables WN, Ripley BD (2002) Modern Applied Statistics with S. Springer, fourth edition.

DSFE-02-04-017-s001.pdf

Reader Comments

Your name:*

Email:*
© 2022 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)