Skip to main content
SpringerLink
Log in

Evaluating Estimates

  • Chapter
  • First Online:
Subnational Population Estimates

Part of the book series: The Springer Series on Demographic Methods and Population Analysis ((PSDE,volume 31))

  • 1195 Accesses

Abstract

Population estimates are used for a wide variety of purposes. Businesses use them to develop customer profiles, identify market clusters, and determine optimal site locations. Researchers use them to study development patterns, environmental conditions, and social trends. State and local governments use them to monitor growth trends, the impact of public policies and to estimate the need for schools, roads, parks, public transportation, fire protection, and other goods and services. Producers of estimates use this information to evaluate and improve estimation methodologies. Given these widespread uses of population estimates it is essential to evaluate their error. This chapter provides such an evaluation. We start with a discussion of various statistics that can be used to measure estimate error. We illustrate these measures using 2010 estimates for counties in Washington State and then provide an overview of the empirical evidence, focusing on the effects of differences in estimation methodology, population size, and population growth rate. We conclude the first section of this chapter with a discussion on ways to account for the uncertainty in population estimates.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.00
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • D’Agostino, R. B., Belanger, A., & D’Agostino, R., B., Jr. (1990). A suggestion for using powerful and informative tests of normality. The American Statistician, 44(3), 316–321.

    Google Scholar 

  • Ahlburg, D. A. (1995). Simple versus complex models: Evaluation, accuracy, and combining. Mathematical Population Studies, 5, 281–290.

    Article  Google Scholar 

  • Alho, J., & Spencer, B. D. (1997). The practical specification of the expected error of population forecasts. Journal of Official Statistics, 13, 203–225.

    Google Scholar 

  • Armstrong, J. S., & Collopy, F. (1992). Error measures for generalizing about forecasting methods: Empirical comparisons. International Journal of Forecasting, 8, 69–80.

    Article  Google Scholar 

  • Belsley, D. A., Kuh, E., & Welsch, R. E. (1980). Regression diagnostics: Identifying influential data and sources of collinearity. New York: John Wiley & Sons.

    Google Scholar 

  • Box, G. P., & Cox, D. (1964). An analysis of transformations. Journal of the Royal Statistical Society, Series B, 26, 211–252.

    Google Scholar 

  • Box, G. P., & Jenkins, G. (1976). Time series analysis: Forecasting and control. San Francisco: Holden-Day.

    Google Scholar 

  • Brockwell, P. J., & Davis, R. A. (2002). Introduction to time series and forecasting, Second Edition. Dordrecht, Heidelberg, London, and New York: Springer.

    Google Scholar 

  • Bryan, T. (1999). Small area population estimation technique using administrative records and evaluation of results with loss functions and optimization criteria. Paper presented at the Federal Committee on Statistical Methodology Research Conference, Washington, D.C

    Google Scholar 

  • Cochran, W. G. (1977). Sampling techniques, Third Edition. New York: John Wiley & Sons.

    Google Scholar 

  • Cohen, J. E. (1986). Population forecasts and confidence intervals for Sweden: A comparison of model-based and empirical approaches. Demography, 23, 105–126.

    Article  Google Scholar 

  • Committee on National Statistics. (1980). Estimating population and income for small areas. Washington, DC: National Academy Press.

    Google Scholar 

  • D'Allesandro, F., & Tayman, J. (1980). Ridge regression for population estimation: Some insights and clarification Staff Document No. 56. Olympia, WA: Office of Financial Management, State of Washington.

    Google Scholar 

  • Davis, S. T. (1994). Evaluation of post-censal county estimates for the 1980s Working Paper No. 5. Washington, DC: Population Division, US Bureau of the Census.

    Google Scholar 

  • Draper, N., & Smith, H. (1981). Applied regression analysis, Second Edition. New York: John Wiley & Sons.

    Google Scholar 

  • Duncan, O. D., Cuzzort, R., & Duncan, B. (1961). Statistical geography: Problems in analyzing areal data. Glencoe: Free Press.

    Google Scholar 

  • Emerson, J. D., & Stoto, M. (1983). Transforming data. In D. C. Hoaglin, F. Mosteller & J. W. Tukey (Eds.), Understanding Robust and Exploratory Data Analysis (pp. 97–128). New York: John Wiley & Sons.

    Google Scholar 

  • Emerson, J. D., & Strenio, J. (1983). Boxplots and batch comparisons. In D. C. Hoaglin, F. Mosteller & J. W. Tukey (Eds.), Understanding Robust and Exploratory Data Analysis (pp. 58–96). New York: John Wiley & Sons.

    Google Scholar 

  • Ericksen, E. P. (1973). A method for combining sample survey data and symptomatic indicators to obtain population estimates for local areas. Demography, 10(2), 137–160.

    Article  Google Scholar 

  • Ericksen, E. P. (1979). Defining criteria for evaluation local estimates Research Monograph Series, Synthetic Estimates for Small Areas (Vol. No. 24). Washington, DC: US Department of Health, Education and Welfare.

    Google Scholar 

  • Espenshade, T. J., & Tayman, J. (1982). Confidence intervals for post-censal state population estimates. Demography, 19(2), 191–210.

    Article  Google Scholar 

  • Fonseca, L., & Tayman, J. (1989). Post-censal estimates of household income distributions. Demography, 26(1), 149–160.

    Article  Google Scholar 

  • Goodall, C. (1983). M-estimators of location: An outline of the theory. In D. C. Hoaglin, F. Mosteller & J. W. Tukey (Eds.), Understanding Robust and Exploratory Data Analysis (pp. 339–403). New York: John Wiley & Sons.

    Google Scholar 

  • Hampel, F. R., Ronchetti, E. M., Rousseeuw, P. J., & Stahel, W. A. (1986). Robust statistics: The approach based on influence functions. New York: John Wiley & Sons.

    Google Scholar 

  • Harper, G., Coleman, C., & Devine, J. (2003). Evaluation of 2000 subcounty population estimates Working Paper Series No. 70. Washington, DC: Population Division, US Census Bureau.

    Google Scholar 

  • Hedayat, A. S., & Sinha, B. K. (1991). Design and inference in finite population sampling. New York: John Wiley & Sons.

    Google Scholar 

  • Hodges, K., & Healy, M. K. (1984). A micro application of a modified housing unit method for tract level population estimates. Paper presented at the annual meeting of the Population Association of America, Minneapolis, MN.

    Google Scholar 

  • Hodges, K., Wilcox, F., & Poveromo, A. (2002). An evaluation of small area estimates produced by the private sector. Paper presented at the annual meeting of the Population Association of America, Atlanta, Georgia.

    Google Scholar 

  • Hoque, N. (2010) An Evaluation of Small Area Population Estimates Produced by Component Method II, Ratio-correlation and Housing Unit Methods for 1990. The Open Demography Journal, 3, 18–30.

    Google Scholar 

  • Judson, D. H., Popoff, C. L., & Batutis, M. J. (2002). An evaluation of the accuracy of US Census Bureau county population estimates. Statistics in Transition, 5(2), 205–235.

    Google Scholar 

  • Keilman, N., Pham, P. Q., & Hetland, A. (2002). Why population forecasts should be probabilistic-Illustrated by the case of Norway. Demographic Research, 6, 409–453.

    Article  Google Scholar 

  • Keilman, N. W. (1990). Uncertainty in national population forecasting. Amsterdam: Swets and Zeitlinger.

    Google Scholar 

  • Keyfitz, N. (1972). On future population. Journal of the American Statistical Association, 67, 347–363.

    Article  Google Scholar 

  • Keyfitz, N. (1981). The limits to population forecasting. Population and Development Review, 7, 579–593.

    Article  Google Scholar 

  • Kish, L. (1965). Survey Sampling. New York: John Wiley & Sons.

    Google Scholar 

  • Kmenta, J. (1971). Elements of Econometrics. New York: Macmillan Publishing Co.

    Google Scholar 

  • Long, J. F. (1993). Post-censal population estimates: States, counties, and places Working Paper No. 3. Washington, DC: Population Division, US Bureau of the Census.

    Google Scholar 

  • Long, J. F. (1995). Complexity, accuracy, and utility of official population projections. Mathematical Population Studies, 5, 203–216.

    Article  Google Scholar 

  • Lowe, T. J., Myers, W. R., & Weisser, L. M. (1984). A special consideration in improving housing unit estimates: The interaction effect. Paper presented at the annual meeting of Population Association of America, Minneapolis, MN.

    Google Scholar 

  • Mahmoud, E. (1987). The evaluation of forecasts. In S. G. Makridakis & S. C. Wheelwright (Eds.), The Handbook of Forecasting (pp. 504–522). New York: John Wiley & Sons.

    Google Scholar 

  • Makridakis, S. G. (1993). Accuracy measures: Theoretical and practical concerns. International Journal of Forecasting, 9, 527–529.

    Article  Google Scholar 

  • Makridakis, S. G., Hibon, M., Lusk, E., & Belhadjali, M. (1987). Confidence intervals: An empirical investigation of the series in the M-competition. International Journal of Forecasting, 3, 489–508.

    Article  Google Scholar 

  • Makridakis, S. G., & Hibon M. (1995). Forecasting accuracy (or error) measures INSEAD Working Paper Series 95/18/TM. Fontainebleau, France: INSEAD.

    Google Scholar 

  • Makridakis, S. G., Wheelwright, S. C., & Hyndman, R. J. (1998). Forecasting methods and applications, Third Edition. New York: John Wiley & Sons.

    Google Scholar 

  • Murdock, S. H., & Hoque, M. N. (1995). The effect of undercount on the accuracy of small-area population estimates: Implications for the use of administrative data for improving population enumeration. Population Research and Policy Review, 14, 251–271.

    Article  Google Scholar 

  • Murdock, S. H., Kelley, C., Jordan, J., Pecotte, B., & Luedke, A. (2006). Demographics: A guide to methods and sources of data for demographic analysis in the media, business, and government. Boulder: Paradigm Publishers.

    Google Scholar 

  • Pflaumer, P. (1992). Forecasting US population totals with the Box-Jenkins approach. International Journal of Forecasting, 8, 329–338.

    Article  Google Scholar 

  • Poole, R. W., Tarver, J. D., White, D., & Gurley, W. R. (1966). An evaluation of alternative techniques for estimating county population in a six-state area Economic Research Series 3. Stillwater, OK: College of Business, Oklahoma State University.

    Google Scholar 

  • Rainford, P., & Masser, I. (1987). Population forecasting and urban planning practice. Environmental and Planning A, 19, 1463–1475.

    Article  Google Scholar 

  • Rayer, S. (2007). Population forecast accuracy: does the choice of summary measure of error matter? Population Research and Policy Review, 26, 163–184.

    Article  Google Scholar 

  • Rosenberger, J. L., & Gasko, M. (1983). Comparing location estimators: Trimmed means, medians and trimean. In D. C. Hoaglin, F. Mosteller & J. W. Tukey (Eds.), Understanding Robust and Exploratory Data Analysis (pp. 297–337). New York: John Wiley & Sons.

    Google Scholar 

  • Rynerson, C., & Tayman, J. (1998). An Evaluation of Address-Level Administrative Records Used to Prepare Small Area Population Estimates. Paper presented at the annual meeting of the Population Association of America, Chicago, IL.

    Google Scholar 

  • Siegel, J. S. (2002). Applied demography: Applications in business, government, law, and public policy. San Diego: Academic Press.

    Google Scholar 

  • Siegel, S. (1956). Nonparametric statistics for the behavioral sciences. New York: McGraw-Hill.

    Google Scholar 

  • Smith, S. K., & Cody, S. (1994). Evaluating the housing unit method: A case study of 1990 population estimates in Florida. Journal of the American Planning Association, 60, 209–221.

    Article  Google Scholar 

  • Smith, S. K., & Cody, S. (2004). An evaluation of population estimates in Florida: April 1, 2000. Population Research and Policy Review, 23, 1–24.

    Article  Google Scholar 

  • Smith, S. K., & Cody, S. (2011). An evaluation of population estimates in Florida, 2011 Special Population Reports No. 8. Gainesville, FL: Bureau of Economic and Business Research.

    Google Scholar 

  • Smith, S. K., & Mandell, M. (1984). A comparison of population estimation methods: Housing unit versus component II, ratio correlation and administrative records. Journal of the American Statistical Association, 79(386), 282–289.

    Article  Google Scholar 

  • Smith, S. K., & Sincich, T. (1992). Evaluating the forecast accuracy and bias of alternative population projections for states. International Journal of Forecasting, 8, 495–508.

    Article  Google Scholar 

  • Smith, S. K., & Tayman, J. (2003). An evaluation of population projections by age. Demography, 40(4), 741–757.

    Article  Google Scholar 

  • Smith, S. K., Tayman, J., & Swanson, D. A. (2001). State and local population projections: Methodology and analysis. New York: Kluwer Academic/Plenum Publishers.

    Google Scholar 

  • Starsinic, D. E., & Zitter, M. (1968). Accuracy of the housing unit method in preparing population estimates for cities. Demography, 5, 474–484.

    Google Scholar 

  • Stock, J. H., & Watson, M. W. (2003). Introduction of Econometrics. Boston: Addison Wesley.

    Google Scholar 

  • Stoto, M. (1983). The accuracy of population projections. Journal of the American Statistical Association, 78, 13–20.

    Article  Google Scholar 

  • Swanson, D. A. (1981). Allocation accuracy in population estimates: An overlooked criterion with fiscal implications Small Area Population Estimates and Their Accuracy, Series GE-41, No. 7 (pp. 13-21). Washington, DC: US Bureau of the Census.

    Google Scholar 

  • Swanson, D. A. (2008). Measuring Uncertainty in population data generated by the cohort component method. In S. H. Murdock & D. A. Swanson (Eds.), Applied Demography in the 21st Century. Dordrecht, Heidelberg, London, and New York: Springer.

    Google Scholar 

  • Swanson, D. A., & Beck, D. M. (1994). A new short-term county population projection method. Journal of Economic and Social Measurement, 20, 25–50.

    Google Scholar 

  • Swanson, D. A., & Coleman, C. (2007). On the MAPE-R as a measure of cross-sectional estimation and forecast accuracy. Journal of Economic and Social Measurement, 32(4), 219–233.

    Google Scholar 

  • Swanson, D. A., & Tayman, J. (1995). Between a rock and a hard place: The evaluation of demographic forecasts. Population Research and Policy Review, 14(2), 233–249.

    Article  Google Scholar 

  • Swanson, D. A., & Tayman, J. (1999). On the validity of the MAPE as a measure of population forecast accuracy. Population Research and Policy Review, 18(4), 299–322.

    Article  Google Scholar 

  • Swanson, D. A., Roe, L., & Carlson, J. (1992). A variation of the housing unit method for estimating the population of small, rural Areas: A case study of the local expert procedure. Survey Methodology, 18(1), 155–163.

    Google Scholar 

  • Swanson, D. A., Carlson, J., Roe, L. & Williams, C. (1995). Estimating the population of rural communities by age and gender: A case study of the local expert procedure. Small Town May-June, 14–21.

    Google Scholar 

  • Swanson, D. A., Tayman, J., & Barr, C. F. (2000). A note on the measurement of accuracy for subnational demographic estimates. Demography, 37(2), 193–202.

    Article  Google Scholar 

  • Swanson, D. A., Tayman, J., & Bryan, T. (2011). MAPE-R: A rescaled measure or accuracy for cross-sectional, sub-national forecasts. Journal of Population Research, 28, 225–243.

    Article  Google Scholar 

  • Swanson, D. A., & Tedrow, L. M. (1984). Improving the measurement of temporal change in regression models used for county population estimates. Demography, 21(3), 373–381.

    Article  Google Scholar 

  • Tayman, J. (1996). The accuracy of small area population forecasts based on a spatial interaction modeling system. Journal of the American Planning Association, 62, 85–98.

    Article  Google Scholar 

  • Tayman, J., & Schafer, E. (1985). The impact of coefficient drift and measurement error in the accuracy of ratio correlation population estimates. The Review of Regional Studies, 15(2), 3–11.

    Google Scholar 

  • Tayman, J., Smith, S. K., & Lin, J. (2007). Precision, bias and uncertainty for state population forecasts: An exploratory analysis of time series methods. Population Research and Policy Review, 26, 347–369.

    Article  Google Scholar 

  • Tayman, J., Swanson, D. A., & Barr, C. F. (1999). In search of the idea measure of accuracy for subnational demographic forecasts. Population Research and Policy Review, 18(5), 387–409.

    Article  Google Scholar 

  • Theil, H. (1966). Applied economic forecasting. Amsterdam: North Holland Publishing Co.

    Google Scholar 

  • United Nations. (1971). Methods of estimating total population for current dates ST/SOA/Series A/No. 10. New York: Population Division, United Nations Department of Social Affairs.

    Google Scholar 

  • Voss, P. R., Palit, C. D., Kale, B. D., & Krebs, H. C. (1981). Forecasting state population using ARIMA time series models. Madison, WI: Applied Population Laboratory, University of Wisconsin.

    Google Scholar 

  • Zitter, M., & Shyrock, H. S. (1964). Accuracy of methods for preparing post-censal estimates for states and local areas. Demography, 1(1), 227–241.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of California Riverside, Riverside, CA, 92521, USA

    David A. Swanson

  2. Department of Economics, University of California San Diego, Gilman Drive 9500-0508, La Jolla, CA, 92093, USA

    Jeff Tayman

Authors
  1. David A. Swanson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jeff Tayman
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer Science+Business Media B.V.

About this chapter

Cite this chapter

Swanson, D.A., Tayman, J. (2012). Evaluating Estimates. In: Subnational Population Estimates. The Springer Series on Demographic Methods and Population Analysis, vol 31. Springer, Dordrecht. https://doi.org/10.1007/978-90-481-8954-0_14

Download citation

Publish with us

Policies and ethics

Search

Navigation