Skip to main content
SpringerLink
Log in

Deviation-Based Model Risk Measures

  • Published:
Computational Economics Aims and scope Submit manuscript

Abstract

In practice, risk forecasts are obtained by risk measures based on a given probability measure on a measurable space. In our study, we consider the probability measures as alternative scenarios, which refer to, for instance, different distribution assumptions, models, or economic situations. Using an improper probability measure can affect risk forecasting and lead to wrong financial decisions. In this context, we propose a Deviation-based approach for quantifying model risk associated with choosing an inappropriate probability measure for risk forecasting. This measuring approach provides us with information about how far our risk measurement process could be affected by model risk. We provide examples of Deviation-based model risk measures defined in the literature. Moreover, we are proposing new alternatives to quantify model risk, for example, Gini and Extended Gini-type model risk measures. We provide a practical example using Value-at-risk (VaR) and Expected Shortfall forecasting to illustrate our approach. Our results indicate that using an inadequate probability measure (distribution assumptions) can largely affect risk forecasting. We verify that model risk estimates present skewness and heavy tail, have significant auto-correlation and do increase in periods that coincide with the highest variability of returns.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. The literature focuses mainly on the comparison of candidate probability measures when S refers to candidate models. We refer to Kuester et al. (2006), Righi and Ceretta (2015), Müller and Righi (2018) and Bianchi and Sorrentino (2020) and the references therein as examples of studies that perform the comparison of alternative strategies to forecast risk measures. For elicitable risk measures the performance comparisons from alternative models is possible (see Gneiting (2011) and Ziegel (2016)). However, classic measures in the literature, such as Expected Shortfall, are not elicitable. Besides that, models are genarally attempts for simplification of reality and it must be recognized that there is no perfect model (see Daníelsson (2008) and Reserve (2011)).

  2. It is also possible to apply risk measures directly in a loss or error function, from some estimation or forecasting procedure. Details are in Bignozzi and Tsanakas (2016) and Detering and Packham (2016). Müller and Righi (2019) name this approach as loss function. The measures from this approach assess the precision of individual models to forecast a particular functional. For this reason, loss function measures can be applied as a complementary criterion for forecast model selection.

  3. Regulatory arbitrage, in the sense used, refers to two institutions with the same portfolio that use different internal models, approved by the regulator, and therefore quantify different amounts of capital requirement. As they keep the same portfolio, they are required to hold the same or at least almost the same amount of regulatory capital. See Kellner and Rösch (2016).

  4. \(\varOmega \) is a no-empty set for the possible states of the world, while \(\mathcal {F}\) is a family form subsets of \(\varOmega \) closed under operations of countable unions and complements, by representing the possible events.

  5. In probability and statistics, a random variable is a random quantity whose values depend on outcomes of a random phenomenon. In the context of the present paper, the value of a random variable depends on the outcome of the event \(\omega \) within the underlying space \(\varOmega \).

  6. The expected value is the arithmetic mean of a large number of independent realizations of a random variable.

  7. The cumulative distribution function gives the probability that a random variable is less than or equal to a given value.

  8. Practically, the relative entropy is a statistical measure that describes how one given probability distribution differs from a second.

  9. The covariance is a well-known measure, in probability theory and statistics, which quantifies the joint variability of two random variables.

  10. We say a function f is measurable with respect to a sigma-algebra \(\mathcal {S}\) if its information is contained on such a set, i.e., \(f^{-1}(B)\in \mathcal {S}\) for any Borel set B.

  11. We use these distribution-based risk measures because they are the most commonly used in the financial industry.

  12. We interchangeably use the term returns and log-returns.

  13. In the risk forecasting literature, AR(p)-GARCH(q,l) is a frequently employed model for forecasting risk measures, and, besides this, there is evidence that points to the superior performance of the GARCH model against other models. See Zhou and Anderson (2012), Degiannakis et al. (2013), and Diaz et al. (2017). We consider different specifications of the probability function F for \(z_t\) because, in a practical sense, risk measures are estimated from data distributions. We want to check that the risk forecasts are sensitive to distributional assumptions and whether these different assumptions generate model risk. Furthermore, our intention here is to illustrate deviation-based model risk measures and not comparing different models for predicting risk measures. However, we emphasize that other models could be considered for forecasting risk measures, such as quantile regression, stochastic volatility, and multifractal models, for instance [for references, see Koenker and Bassett (1978), Calvet and Fisher (2004), Segnon et al. (2017), and Lux and Segnon (2012). We are grateful to an anonymous reviewer for the latter mentioned remark and references.] Although our empirical analysis is univariate, extensions to the multivariate case are operable. We refer to studies such as Weiß (2013) and Müller and Righi (2018) for a comparison of alternative models in a multivariate context, regarding financial risk forecasting. We chose the univariate estimation for simplicity and since it is typically used in illustrative examples in the framework of model risk measurement. See Danielsson et al. (2016) and Kellner and Rösch (2016).

  14. This rolling window estimation is regular in risk forecasting studies. Further, some studies show this number of observations is a good sample size for daily data. See Righi and Ceretta (2016).

  15. Consider a financial return distribution that have a fully parametric location-scale specification based on the expectation, dispersion and random component, represented by \(X_t = \mu _t + \sigma _tz_t\), where \(\mu _t\) and \(\sigma _t\) are mensurable considering the information available up to \( t-1 \), and \(z_t\) are i.i.d. innovations, which can assume different probability distribution functions. Assume that \(\rho \) refers to risk measure, such as VaR and ES, being \(\rho ^*(X) = -\rho (X)\). Under this specification, risk measures become \(\rho ^*(X_t) = \rho ^*(\mu _t + \sigma _tz_t) = -\mu _t + \sigma _t\rho ^*(z_t)\). Thus, using a conditional model, we first filter \(\mu _t\) and \(\sigma _t\), and, then we get \(\rho ^*\), for a given distribution. For more details, see McNeil et al. (2015).

  16. We do not use Extended Gini-type model risk measure (expression (23)) in our illustration because it is a general form of Gini-type model risk measure for \(r=2\). We suggest that future work should be conducted in this sense to investigate how to determine the degree of model risk-aversion according to each profile. See Berkhouch et al. (2018) for an illustration of the use of Extended Gini in the framework of market risk measurement.

  17. Ljung-Box test considers as null hypothesis (\(\mathcal {H}_0\)) that auto-correlations up to lag k are zero, and as alternative hypothesis (\(\mathcal {H}_1\)) that auto-correlations up to lag k differ from zero.

References

  • Acerbi, C. (2002). Spectral measures of risk: A coherent representation of subjective risk aversion. Journal of Banking and Finance, 26(7), 1505–1518.

    Article  Google Scholar 

  • Adam, A., Houkari, M., & Laurent, J.-P. (2008). Spectral risk measures and portfolio selection. Journal of Banking and Finance, 32(9), 1870–1882.

    Article  Google Scholar 

  • Artzner, P., Delbaen, F., Eber, J.-M., & Heath, D. (1999). Coherent measures of risk. Mathematical Finance, 9(3), 203–228.

    Article  Google Scholar 

  • Bannör, K. F., & Scherer, M. (2013). Capturing parameter risk with convex risk measures. European Actuarial Journal, 3(1), 97–132.

    Article  Google Scholar 

  • Barrieu, P., & Scandolo, G. (2015). Assessing financial model risk. European Journal of Operational Research, 242(2), 546–556.

    Article  Google Scholar 

  • Basel Committee on Banking Supervision (2013). Fundamental review of the trading book: A revised market risk framework. Consultative Document, October.

  • Berkhouch, M., Lakhnati, G., & Righi, M. B. (2018). Extended gini-type measures of risk and variability. Applied Mathematical Finance, 25(3), 295–314.

    Article  Google Scholar 

  • Bianchi, M. L., & Sorrentino, A. M. (2020). Measuring covar: An empirical comparison. Computational Economics, 55(2), 511–528.

    Article  Google Scholar 

  • Bignozzi, V., & Tsanakas, A. (2016). Parameter uncertainty and residual estimation risk. Journal of Risk and Insurance, 83(4), 949–978.

    Article  Google Scholar 

  • Breuer, T., & Csiszár, I. (2016). Measuring distribution model risk. Mathematical Finance, 26(2), 395–411.

    Article  Google Scholar 

  • Calvet, L. E., & Fisher, A. J. (2004). How to forecast long-run volatility: Regime switching and the estimation of multifractal processes. Journal of Financial Econometrics, 2(1), 49–83.

    Article  Google Scholar 

  • Ceriani, L., & Verme, P. (2012). The origins of the Gini index: Extracts from variabilità e mutabilità (1912) by Corrado Gini. The Journal of Economic Inequality, 10(3), 421–443.

    Article  Google Scholar 

  • Cont, R. (2006). Model uncertainty and its impact on the pricing of derivative instruments. Mathematical Finance, 16(3), 519–547.

    Article  Google Scholar 

  • Daníelsson, J. (2008). Blame the models. Journal of Financial Stability, 4(4), 321–328.

    Article  Google Scholar 

  • Danielsson, J., James, K. R., Valenzuela, M., & Zer, I. (2016). Model risk of risk models. Journal of Financial Stability, 23, 79–91.

    Article  Google Scholar 

  • Danielsson, J., & Shin, H. S. (2003). Endogenous risk. Modern risk management: A history (pp. 297–316).

  • Degiannakis, S., Floros, C., & Dent, P. (2013). Forecasting value-at-risk and expected shortfall using fractionally integrated models of conditional volatility: International evidence. International Review of Financial Analysis, 27, 21–33.

    Article  Google Scholar 

  • Delbaen, F. (2012). Monetary Utility Functions. Osaka University CSFI lecture notes series.

  • Detering, N., & Packham, N. (2016). Model risk of contingent claims. Quantitative Finance, 16(9), 1357–1374.

    Article  Google Scholar 

  • Diaz, A., Garcia-Donato, G., & Mora-Valencia, A. (2017). Risk quantification in turmoil markets. Risk Management, 19(3), 202–224.

    Article  Google Scholar 

  • Dotson, D., McTaggart, R., Daroczi, G., & Leung, C. (2019). Package Quandl. Comprehensive R Archive Network.

  • Dowd, K. (2007). Measuring market risk. Hoboken: Wiley.

    Google Scholar 

  • Emmer, S., Kratz, M., & Tasche, D. (2015). What is the best risk measure in practice? A comparison of standard measures. Journal of Risk, 18(2).

  • Reserve, Federal. (2011). Supervisory guidance on model risk management. Office of the Comptroller of the Currency: Board of governors of the federal reserve system.

  • Föllmer, H., & Schied, A. (2002). Convex measures of risk and trading constraints. Finance and Stochastics, 6(4), 429–447.

    Article  Google Scholar 

  • Föllmer, H., & Schied, A. (2016). Stochastic Finance: An Introduction in discrete time. De Gruyter Textbook: De Gruyter.

  • Furman, E., Wang, R., & Zitikis, R. (2017). Gini-type measures of risk and variability: Gini shortfall, capital allocations, and heavy-tailed risks. Journal of Banking and Finance, 83, 70–84.

    Article  Google Scholar 

  • Gini, C. (1921). Measurement of inequality of incomes. The Economic Journal, 31(121), 124–126.

    Article  Google Scholar 

  • Giorgi, G. M. (1993). A fresh look at the topical interest. Metron: International Journal of Statistics, 51(1–2), 83–98.

    Google Scholar 

  • Giorgi, G. M. (2005). Bibliographic portrait of the Gini concentration ratio. Technical report, University Library of Munich, Germany.

  • Glasserman, P., & Xu, X. (2014). Robust risk measurement and model risk. Quantitative Finance, 14(1), 29–58.

    Article  Google Scholar 

  • Gneiting, T. (2011). Making and evaluating point forecasts. Journal of the American Statistical Association, 106(494), 746–762.

    Article  Google Scholar 

  • Henryk, G., & Silvia, M. (2006). On a relationship between distorted and spectral risk measures. Working Paper.

  • Jokhadze, V., Schmidt, W. M., et al. (2020). Measuring model risk in financial risk management and pricing. International Journal of Theoretical and Applied Finance (IJTAF), 23(02), 1–37.

    Google Scholar 

  • Kellner, R., & Rösch, D. (2016). Quantifying market risk with value-at-risk or expected shortfall? Consequences for capital requirements and model risk. Journal of Economic Dynamics and Control, 68, 45–63.

    Article  Google Scholar 

  • Kerkhof, J., Melenberg, B., & Schumacher, H. (2010). Model risk and capital reserves. Journal of Banking and Finance, 34(1), 267–279.

    Article  Google Scholar 

  • Koenker, R., & Bassett Jr, G. (1978). Regression quantiles. Econometrica: Journal of the Econometric Society. 46: 33–50.

  • Krajcovicova, Z., Perez-Velasco, P. P., & Vázquez Cendón, C. (2019). A new approach to the quantification of model risk for practitioners. Journal of Computational Finance, 23(2).

  • Kuester, K., Mittnik, S., & Paolella, M. S. (2006). Value-at-risk prediction: A comparison of alternative strategies. Journal of Financial Econometrics, 4(1), 53–89.

    Article  Google Scholar 

  • Lux, T., & Segnon, M. (2018). Multifractal models in finance: Their origin, properties and applications. The Oxford Handbook of Computational Economics and Finance.

  • Mao, T., & Wang, R. (2018). Risk aversion in regulatory capital principles. Working Paper.

  • Markowitz, H. (1952). Portfolio selection. The Journal of Finance, 7(1), 77–91.

    Google Scholar 

  • McNeil, A. J., Frey, R., & Embrechts, P. (2015). Quantitative risk management: concepts, techniques and tools-revised edition. New Jersey: Princeton University Press.

    Google Scholar 

  • Müller, F. M., & Righi, M. B. (2018). Numerical comparison of multivariate models to forecasting risk measures. Risk Management, 20(1), 29–50.

    Article  Google Scholar 

  • Müller, F. M., & Righi, M. B. (2019). Model risk measures: A review and new proposals on risk forecasting. Working Paper.

  • Pflug, G., & Römisch, W. (2007). Modeling. Measuring and managing risk: World Scientific.

  • Righi, M., & Ceretta, P. (2016a). Shortfall deviation risk: An alternative to risk measurement. Journal of Risk, 19(2), 81–116.

    Article  Google Scholar 

  • Righi, M. B. (2018). A composition between risk and deviation measures. Annals of Operations Research, 1–15.

  • Righi, M. B. (2018). A theory for combinations of risk measure. Available at: arXiv:1807.01977.

  • Righi, M. B., & Ceretta, P. S. (2015). A comparison of expected shortfall estimation models. Journal of Economics and Business, 78, 14–47.

    Article  Google Scholar 

  • Righi, M. B., & Ceretta, P. S. (2016b). On the existence of an optimal estimation window for risk measures. Economics Bulletin, 36(1), 1–9.

    Google Scholar 

  • Rockafellar, R. T., Uryasev, S., & Zabarankin, M. (2006). Generalized deviations in risk analysis. Finance and Stochastics, 10(1), 51–74.

    Article  Google Scholar 

  • Segnon, M., Lux, T., & Gupta, R. (2017). Modeling and forecasting the volatility of carbon dioxide emission allowance prices: A review and comparison of modern volatility models. Renewable and Sustainable Energy Reviews, 69, 692–704.

    Article  Google Scholar 

  • Shalit, H., & Yitzhaki, S. (1984). Mean-gini, portfolio theory, and the pricing of risky assets. The Journal of Finance, 39(5), 1449–1468.

    Article  Google Scholar 

  • Taylor, J. W. (2019). Forecast combinations for value at risk and expected shortfall. International Journal of Forecasting, In Press.

  • Wang, R., Wei, Y., & Willmot, G. (2018). Characterization, robustness and aggregation of signed Choquet integrals. Working Paper.

  • Weiß, G. N. F. (2013). Copula-GARCH versus dynamic conditional correlation: An empirical study on VaR and ES forecasting accuracy. Review of Quantitative Finance and Accounting, 41(2), 179–202.

    Article  Google Scholar 

  • Yitzhaki, S. (1983). On an extension of the Gini inequality index. International Economic Review, 24(3), 617–628.

    Article  Google Scholar 

  • Yitzhaki, S. (1998). More than a dozen alternative ways of spelling Gini. In Research in economic inequality.

  • Yitzhaki, S., & Schechtman, E. (2005). The properties of the extended gini measures of variability and inequality. Available at SSRN 815564.

  • Yitzhaki, S., & Schechtman, E. (2012). The gini methodology: A primer on a statistical methodology. Berlin, Newyork: Springer. in Statistics.

    Google Scholar 

  • Zhou, J., & Anderson, R. I. (2012). Extreme risk measures for international reit markets. The Journal of Real Estate Finance and Economics, 45(1), 152–170.

    Article  Google Scholar 

  • Ziegel, J. F. (2016). Coherence and elicitability. Mathematical Finance, 26(4), 901–918.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. LISAD, ENSA, Ibn Zohr University, Agadir, Morocco

    Mohammed Berkhouch & Ghizlane Lakhnati

  2. Business School, Federal University of Rio Grande do Sul, Porto Alegre, Brazil

    Fernanda Maria Müller & Marcelo Brutti Righi

Authors
  1. Mohammed Berkhouch
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fernanda Maria Müller
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ghizlane Lakhnati
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marcelo Brutti Righi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammed Berkhouch.

Ethics declarations

Conflict of interest

The authors report no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Berkhouch, M., Müller, F.M., Lakhnati, G. et al. Deviation-Based Model Risk Measures. Comput Econ 59, 527–547 (2022). https://doi.org/10.1007/s10614-021-10093-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10614-021-10093-x

Keywords

Search

Navigation