The limited usefulness of macroeconomic Bayesian VARs when estimating the probability of a US recession
Highlights
► We estimate the probability of a US recession using Bayesian VAR models. ► In contrast to judgemental probabilities, the BVARs assign very low probabilities. ► Also when survey data are included in the models, the estimated probabilities are low. ► Bayesian VAR models appear to be of limited usefulness for predicting recessions.
Introduction
As a part of the decision making process, economic agents have to assign probabilities to certain developments in the economy. These probabilities are rarely communicated but a number of central banks – such as the Bank of England, Norges Bank and Sveriges Riksbank – regularly publish fan charts for several macroeconomic variables. Such fan charts can be used to answer a number of probability questions, where focus historically has been on the evolution of future inflation.
Recently, it has been argued that predictive densities from VAR models can be used to make probability statements; see, for example, Garratt et al., 2003, Leeper and Zha, 2003, Österholm, 2009. While this clearly is a convenient approach, a highly relevant question to ask before adopting it is whether the estimated probabilities are reasonable and thereby useful in, for example, policy analysis. The purpose of this paper is hence to scrutinise the probability statements from standard macroeconomic VAR models when it comes to an issue of general interest to forecasters and decision makers, namely the US economy being in a recession.
Relying on VAR models, the NBER’s classification of recessions is not particularly useful and we will instead use the commonly employed definition of two quarters of negative GDP growth in a row.1 We will focus on two quarters of the most recent US recession, namely 2008Q3–2008Q4, both of which are associated with fairly large negative values of GDP growth. It is difficult to evaluate the estimated recession probabilities from the model using a formal criterion since ex post, the probability of the recession taking place is (close to) one.2 The approach taken here will hence be to relate the estimated probabilities to judgemental forecasts made around the relevant time point. For example, former chairman of the Federal Reserve, Alan Greenspan said in May 2008 in Financial Times that the probability of a recession was larger than 50% and in August 2008, InTrade recession bet put the probability of a recession close to 30%. In a similar spirit, it was mentioned at the budget briefing on the 24th of October 2008 that “according to Moody’s Economy.com, the probability of a national recession increased to 43 percent in August compared with 35 percent in July”. If the models generate recession probabilities that are substantially lower than these numbers, it is a reason to be concerned regarding the properties of the model.
Several models are estimated and evaluated. The two benchmark models are (i) a univariate Bayesian AR model for GDP growth and (ii) a trivariate Bayesian VAR model with GDP growth, CPI inflation and the three month treasury bill rate. Both of these specifications are commonly used in empirical work and particularly the popularity of small models – including a measure of real activity, inflation and a nominal interest rate – cannot be overstated in macroeconomic analysis and forecasting; see, for example, Clarida et al., 1999, Rudebusch and Svensson, 1999, Cogley and Sargent, 2001, Primiceri, 2005, Ribba, 2006, Rubaszek and Skrypczynski, 2008, Berger and Österholm, 2009.
Results show that the benchmark models assign very low probabilities to a recession at the investigated time point and the analysis indicates that this is due to the dynamic properties of the model. In an attempt to see whether the usage of leading indicators can improve the estimated probabilities, we augment the benchmark models with a number of variables. Three variables based on survey data are used: the University of Michigan Consumer Sentiment Index, the Institute for Supply Management PMI Composite Index and a measure of bank lending tightening from the Board of Governors of the Federal Reserve’s Senior Loan Officer Survey on Bank Lending Practices. Finally, we augment the models with oil price changes. However, despite the popularity of these variables, only some of them show evidence of marginally being able to improve the models’ recession-prediction ability. None of the estimated models generate recession probabilities that are even close to the judgemental probabilities.
The rest of this paper is organised as follows. Section 2 briefly presents the Bayesian VAR model used for the estimation. In Section 3, data are described and the forecasting performance of the benchmark and augmented models is investigated. We also conduct some sensitivity analysis. Finally, Section 4 concludes.
Section snippets
Methodology
The forecasting tool used for the analysis in this paper is a state-of-the-art Bayesian VAR (BVAR) model developed by Villani (2009).
Data
Quarterly real-time data ranging from 1982Q1 to 2008Q2 are used in the main analysis.5 Wanting to make predictions for 2008Q3 and 2008Q4, we are standing partway through Q3 in order for Q2
Conclusions
The Bayesian VAR model provides a convenient tool for generating predictive densities and making probability statements regarding the future development of economic variables. However, convenience is obviously not the only criterion that one should take into account when deciding how to estimate probabilities concerning economic events. It is also of great importance that the model in question can generate probabilities of the event that we are interested in that are reasonable.
In this paper we
Acknowledgement
I am grateful to two anonymous referees for valuable comments on this paper. The views in this paper are solely the responsibility of the author and should not be interpreted as reflecting the views of the Executive Board of Sveriges Riksbank.
References (39)
- et al.
Improving macro-economic forecasts – the role of consumer confidence
International Journal of Forecasting
(1998) - et al.
Forecasting inflation in an inflation targeting regime: a role for informative steady-state priors
International Journal of Forecasting
(2010) - et al.
Does money still matter for US output?
Economics Letters
(2009) - et al.
A real-time data set for macroeconomists
Journal of Econometrics
(2001) - et al.
Why did forecasters fail to predict the 1990 recession
International Journal of Forecasting
(1999) What is an oil shock?
Journal of Econometrics
(2003)Calling recessions in real time
International Journal of Forecasting
(2011)- et al.
Modest policy interventions
Journal of Monetary Economics
(2003) - et al.
On the forecasting performance of a small-scale DSGE model
International Journal of Forecasting
(2008) - et al.
Forecasting with a real-time data set for macroeconomists
Journal of Macroeconomics
(2002)