A new technique for postsample model selection and validation

doi:10.1016/S0165-1889(97)00085-7

Journal of Economic Dynamics and Control

Volume 22, Issue 5, May 1998, Pages 647-665

https://doi.org/10.1016/S0165-1889(97)00085-7 Get rights and content

Abstract

The model selection and Granger-causality literatures have focused on insample rather than postsample hypothesis testing. This is because postsample model validation periods are usually short, whereas the correlations in typical postsample forecast errors require large-sample methods. A resampling-based postsample inference procedure is described which explicitly estimates the uncertainty which its large-sample approximation induces in the inference significance levels it produces. The procedure is applied to postsample forecasting errors from the Ashley et al. (1980) study examining Granger-causation between US consumption and advertising expenditures. Postsample model validation is apparently feasible, but requires longer postsample periods than past studies have allocated.

References (13)

R. Ashley et al.
Advertising and aggregate consumption: an analysis of causality
Econometrica
(1980)
R. Ashley
Inflation and the distribution of price changes across markets: a causal analysis
Economic Inquiry
(1981)
R. Ashley
A statistical inference engine for small dependent samples with applications to ostsample model validation, model selection, and Granger-causality analysis
V.P.I, and S.U. Economics Department Working Paper # E92-24
(1992)
R. Beran
Simulated power functions
Annals of statistics
(1986)
R. Beran
Prepivoting to reduce level error of confidence sets
Biometrika
(1987)
P.J. Bickel et al.
Some asymptotic theory for the bootstrap
Annals of Statistics
(1981)

There are more references available in the full text version of this article.

Cited by (38)

Persistence and cycles in the us federal funds rate
2017, International Review of Financial Analysis
This paper uses long-range dependence techniques to analyse two important features of the US Federal Funds effective rate, namely its persistence and its cyclical behaviour. It examines annual, monthly, bi-weekly and weekly data, from 1954 until 2017. Two models are considered. One is based on an I(d) specification with AR(2) disturbances and the other one on two fractional differencing structures, one at the zero and the other at a cyclical frequency. Thus, the two approaches differ in the way the cyclical component of the process is modelled. In both cases we obtain evidence of long memory and fractional integration with cycles repeating approximately every 8 years. The in-sample goodness-of-fit analysis supports the second specification in the majority of cases. An out-of-sample forecasting experiment also suggests that the long-memory model with two fractional differencing parameters is the most adequate one, especially over long horizons.
Forecasting in a Mixed Up World: Nowcasting Hawaii Tourism
2017, Annals of Tourism Research
We evaluate the short term forecasting performance of methods that systematically incorporate high frequency information via covariates. Our study provides a thorough introduction of these methods to the tourism literature. We highlight the distinguishing features and limitations of each tool and evaluate their forecasting performance in two tourism-specific applications. The first uses monthly indicators to predict quarterly tourist arrivals to Hawaii; the second predicts quarterly labor income in the accommodations and food services sector. Our results indicate that compared to the exclusive use of low frequency aggregates, including timely intra-period data in the forecasting process results in significant gains in predictive accuracy. Anticipating growing popularity of these techniques among empirical analysts, we present practical implementation guidelines to facilitate their adoption.
Estimating the long rate and its volatility
2015, Economics Letters
Forecasting welfare caseloads: The case of the Japanese public assistance program
2014, Socio-Economic Planning Sciences
Forecasting welfare caseloads has grown in importance in Japan because of their recent rapid increase. Given that the forecasting literature on welfare caseloads only focuses on US cases and utilizes limited classes of forecasting models, this study employs multiple alternative methods in order to forecast Japanese welfare caseloads and compare forecasting performances. In pseudo real-time forecasting, VAR and forecast combinations tend to outperform the other methods investigated. In real-time forecasting, however, a simple version of forecast combinations seems to perform better than the remaining models, predicting that welfare caseloads in Japan will surpass 1.7 million by the beginning of 2016, an approximately 20% increase in five years from the beginning of 2011.
Asymptotics for out of sample tests of Granger causality
2007, Journal of Econometrics
This paper presents analytical, Monte Carlo and empirical evidence concerning out-of-sample tests of Granger causality. The environment is one in which the relative predictive ability of two nested parametric regression models is of interest. Results are provided for three statistics: a regression-based statistic suggested by Granger and Newbold [1977. Forecasting Economic Time Series. Academic Press Inc., London], a t-type statistic comparable to those suggested by Diebold and Mariano [1995, Comparing Predictive Accuracy. Journal of Business and Economic Statistics, 13, 253–263] and West [1996. Asymptotic Inference About Predictive Ability, Econometrica, 64, 1067–1084], and an F-type statistic akin to Theil's U. Since the asymptotic distributions under the null are nonstandard, tables of asymptotically valid critical values are provided. Monte Carlo evidence supports the theoretical results. An empirical example evaluates the predictive content of the Chicago Fed National Activity Index for growth in Industrial Production and core PCE-based inflation.
A TEST of the GARCH(1, 1) SPECIFICATION for DAILY STOCK RETURNS
2010, Macroeconomic Dynamics

View all citing articles on Scopus

^∗: The author gratefully acknowledges the support of NSF grant # SES-8922394.

View full text

A new technique for postsample model selection and validation

Abstract

Advertising and aggregate consumption: an analysis of causality

Econometrica

Inflation and the distribution of price changes across markets: a causal analysis

Economic Inquiry

A statistical inference engine for small dependent samples with applications to ostsample model validation, model selection, and Granger-causality analysis

V.P.I, and S.U. Economics Department Working Paper # E92-24

Simulated power functions

Annals of statistics

Prepivoting to reduce level error of confidence sets

Biometrika

Some asymptotic theory for the bootstrap

Annals of Statistics