Unpredictability in economic analysis, econometric modeling and forecasting

Abstract

Unpredictability arises from intrinsic stochastic variation, unexpected instances of outliers, and unanticipated extrinsic shifts of distributions. We analyze their properties, relationships, and different effects on the three arenas in the title, which suggests considering three associated information sets. The implications of unanticipated shifts for forecasting, economic analyses of efficient markets, conditional expectations, and inter-temporal derivations are described. The potential success of general-to-specific model selection in tackling location shifts by impulse-indicator saturation is contrasted with the major difficulties confronting forecasting.

Introduction

Unpredictability has been formalized as intrinsic stochastic variation in a known distribution, where conditioning on available information does not alter the outcome from the unconditional distribution, as in the well-known prediction decomposition, or sequential factorization, of a density: see Doob (1953). Such variation can be attributed (inter alia) to chance distribution sampling, ‘random errors’, incomplete information, or in economics, many small changes in the choices by individual agents. A variable that is intrinsically unpredictable cannot be modeled or forecast better than its unconditional distribution.

However, the converse does not hold: a variable that is not intrinsically unpredictable may still be essentially unpredictable because of two additional aspects of unpredictability. The first concerns independent draws from fat-tailed or heavy-tailed distributions, which leads to a notion we call ‘instance unpredictability’. Here the distribution of a variable that is not intrinsically unpredictable is known, as are all conditional and unconditional probabilities, but there is a non-negligible probability of a very discrepant outcome. While that probability is known, it is not known on which draw the discrepant outcome will occur, nor its magnitude, leading to a ‘Black Swan’ as in Taleb (2007), with potentially large costs when that occurs: see Barro (2009). The third aspect we call ‘extrinsic unpredictability’, which derives from unanticipated shifts of the distribution itself at unanticipated times, of which location shifts (changes in the means of distributions) are usually the most pernicious. Intrinsic and instance unpredictability are close to ‘known unknowns’ in that the probabilities of various outcomes can be correctly pre-calculated, as in rolling dice, whereas extrinsic unpredictability is more like ‘unknown unknowns’ in that the conditional and unconditional probabilities of outcomes cannot be accurately calculated in advance as in the first quote of Clements and Hendry (1998). The recent financial crisis and ensuing deep recession have brought both instance and extrinsic unpredictability into more salient focus: see Taleb (2009), and Soros, 2008, Soros, 2010.

These three aspects of unpredictability suggest that different information sets might explain at least a part of their otherwise unaccounted variation. This is well established both theoretically and empirically for intrinsic unpredictability, where ‘regular’ explanatory variables are sought. Empirically, population distributions are never known, so even to calculate the probabilities for instance unpredictability, it will always be necessary to estimate the distributional form from available evidence, albeit few ‘tail draws’ will occur from which to do so. New aspects of distributions have to be estimated when extrinsic unpredictability occurs. Consequently, each type of unpredictability has substantively different implications for economic analyses, econometric modeling, and economic forecasting. Specifically, inter-temporal economic theory, forecasting, and policy analyses could go awry facing extrinsic unpredictability, yet ex post, the outcomes that eventuated are susceptible to being modeled. We briefly discuss the possible role of impulse-indicator saturation for detecting and removing in-sample location shifts. The availability of such tools highlights the contrast between the possibilities of modeling extrinsic unpredictability ex post against the difficulties confronting successful ex ante forecasting, where one must forecast outliers or shifts, which are demanding tasks. However, transformations of structural models that make them robust after shifts, mitigating systematic forecast failure, are feasible.

The structure of the paper is as follows. Section  2 considers intrinsic unpredictability in Section  2.1; instance unpredictability in Section  2.2; and extrinsic unpredictability in Section  2.3. Theoretical implications are drawn in Section  3, with the relationships between intrinsic, instance and extrinsic unpredictability in Section  3.1, and the impact of reduced information in Section  3.2. The possibility of three distinct information sets associated respectively with ‘normal causality’, the timing of outliers, and the occurrence of distributional shifts is discussed in Section  3.3. The difficulties both economists and economic agents confront facing unanticipated breaks are analyzed in Section  3.4. Section  4 investigates some consequences for empirical applications. The fundamental separation between modeling and forecasting from instance and extrinsic unpredictability–but not intrinsic unpredictability–is discussed in Section  4.1. Then Section  4.2 considers the relationships between the three aspects of unpredictability for model selection in processes with unanticipated breaks, leading to a reconsideration of the role of congruent modeling for forecasting in Section  4.3. These analyses are illustrated in Section  4.4 by an empirical application of robust forecasting. Section  5 concludes.