Fig. 1. Representative screeplot of aggregate prediction error.

Table 1. Average prediction error from 100 bootstrap samples as a function of the number of variables in the model. The column headings are the true model

Table 2. Results for 1000 simulations for model selection from [Shao 1996] using least-squares parameter estimates. Observations 8, 15, 28, and 39 are made residual outliers. The true model [2,0,0,4,8] is shaded. The top number in each cell is the fraction of time the model was selected using the minimization of prediction error metric and the bottom uses the change in purity metric with constant 0.025

Table 3. Results for 100 simulations for model selection from [Shao 1996] using the Simpson and Montgomery compound estimator. Observations 8, 15, 28, and 39 are made residual outliers. The true model [2,0,0,4,8] is shaded. The top number in each cell is the fraction of time the model was selected using the minimization of prediction error metric and the bottom number uses the change in prediction error criterion with constant 0.025

Table 4. 2_V⁵⁻¹ design and results for bootstrap methods using compound estimators. The columns represent models with p=2,3,4, and 5 parameters. The last four columns are results using weighted residuals to compute prediction error. The shaded columns represent the correct model. The top values in each cell are the proportion of times that the model is selected out of 250 replications using the minimum prediction error criterion. The bottom values are the proportion selected using the change in prediction error criterion

Table 5. 2⁴ design and results for cross-validation methods using compound estimators. The columns represent models with p=2,3,4, and 5 parameters. The last four columns are results using weighted residuals to compute prediction error. The shaded columns represent the correct model. The top values in each cell are the proportion of times that the model is selected out of 250 replications using the minimum prediction error criterion. The bottom values are the proportion selected using the change in prediction error criterion