Abstract
Income is regarded as one of the clearest indicators of socioeconomic status and wellbeing in the developed world and is highly correlated with a wide range of outcomes. Despite its importance, there remains an issue as to the best way to collect income as part of surveys. This paper examines differences in how income is collected in a nationally representative UK birth cohort, the Millennium Cohort Study, looking at variations by questions asked and by respondent characteristics before then examining the implications different methods of collecting and reporting income may have for measuring poverty. Results show that less than a third of respondents give consistent information on income between measurement tools. Using multiple questions is associated with a substantially lower response rate but this method generally results in a higher estimate of family income than using a single question. This is particularly true for certain groups of the population—those on means tested benefits, in self-employment and in part-time employment. Not surprisingly then in our analysis of poverty, using a single question produces an inflated proportion of families who could be classified as living in poverty and is less associated with other measures of financial deprivation than the more conservative poverty measure based on multiple questions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
In the developing world, expenditure is regarded as a clearer indicator of socioeconomic status (for example Van de Poel et al. 2008).
Although Britain varies somewhat in the high prevalence of ‘current’ measures of income (the amount of income last received, reported in a variety of units) as opposed to annual (Boheim and Jenkins 2006).
Although this depends on the number and width of bands, and the density of income within bands.
Elements excluded from this measure of income were irregular payments such as lottery winnings, inheritance or retirement and redundancy pay outs.
Although according to Hurd et al. (2003) there is very little literature on these effects.
This only included sweeps 1 and 2 which did not include questions on the amount of state benefits received.
Although the focus of this paper is on data collected at age 7, Hansen and Kneale (2011) examine trends between the third sweep in 2006 (MCS3) collected at age 5 and the age 7 sweep collected in 2008 (MCS4). They show consistency in the reporting of income over the different sweeps. The majority of respondents who reported higher incomes using the multiple questions at MCS4 also did so at MCS3.
Although this varies from sweep to sweep [for more detailed information see Hansen and Kneale (2011)].
Our measure is family rather than household income as it refers only to the parental unit of the child and any dependent children in the household who are siblings of the cohort member (biological, adopted, step or foster), excluding other adults. MCS collects employment information for all household members over the age of 15. As such, it may be questionable to classify 16–18-year-olds as dependent if they are employed. However, as the employment status theoretically treats any paid employment, from a paper round upwards, as being ‘in employment’, then we treat any 16–18-year-old who is a sibling of the cohort member as dependent member of the household because of this ambiguity.
Although in the case of investments, dividends and studentships, these may not necessarily be paid on a monthly basis to respondents.
A different set of income bands was given to main respondents in couples and those who were lone parents (Centre for Longitudinal Studies 2009).
Once logical inconsistencies and missing data were excluded from the data.
For the top brackets (which are not closed, accounting for 0.4% of lone parents and 2.0% of couples), we selected the mean value from the continuous income for those whose selected the top bracket and whose income also fell into the top bracket. For the lowest bracket (0.8% lone parents and 0.4% of couples), we selected the mid-point.
Based on the ethnic group of the child.
Note, income bands varied in width ranging from the smallest for lone parents of £1,000 at the bottom of the income distribution, to the open bands for families at the top of the income distribution.
We initially tested a multinomial logistic regression model but were unable to satisfy the Independence of Irrelevant Alternatives test using the Hausman test.
The negative coefficient on the workless family variable being in the opposite direction to the coefficient on families claiming means-tested benefits is an unexpected result. For this reason we also ran the same regression as Model A but this time included an interaction term in an attempt to illuminate this issue but the interaction term was not significant. This was further explored by examining the predicted probabilities for each variation of worklessness and means tested benefits. The results (not shown here) indicate that those families in receipt of means tested benefits, but where at least one partner worked, are particularly likely to record a higher income using the multiple questions (52%), but those who were workless and do not claim means tested benefit are among the least likely to (27%) and to report a lower band using the multiple questions (56%). Although this analysis compares the cleaned (banded) family income variable from the multiple questions with the single question variable, the latter result for workless families not claiming means tested benefits could suggest a residual effect of underreporting of benefits income. Nevertheless, clearly these results demonstrate that the consistency in reporting income between measurement instruments is very much dependent on socioeconomic characteristics.
This will be possible if MCS data are matched to administrative records on income but this has not been done to date. We did carry out analysis which compare MCS income data to income data recorded in the FRS. We also considered the validity of our income estimates using other sources. We identified a number of possible comparisons (Table 1) although were unable to find a suitable match. We identified the closest match as the Family Resources Survey (FRS) collected by the Department of Work and Pensions, and selected only those families with a child aged 6–8 years. However, even after weighting, the FRS average household income estimate was much higher at £40,863 than any estimate for the MCS (there were similar discrepancies for the median and quartile values). Likely reasons for this discrepancy include the estimation of housing benefit in FRS incomes and the more detailed collection of incomes from ‘other’ sources. However, this does not necessarily affect the generalizability of our results to other surveys, as our focus is on within survey differences in income by measurement instruments.
This innovation allows for income from housing benefits not being included in our family income measure.
We estimate regressions using the log of income to reflect the shape of the within-band distribution within the critical band.
As we are only interested in the predicted values and not in the effect of the covariates, we do not present the full output.
All our estimates of poverty are also lower compared to other estimates in the literature because of our treatment of those in receipt of housing benefits, our calculation of the OECD equivalisation factor, and our choice of income predictors (Ketende and Joshi 2008). In addition, we make no correction for non-response on our income variables here, which may bias the sample composition. As is the case elsewhere in the literature, the results for income measured through a single question using either the band mid-point or predicted income from interval regression are almost identical (Ketende and Joshi 2008).
Families are eligible for free school meals and housing benefit if they are in receipt of unemployment benefits or low income benefits. Free school meals are a lunchtime meal provided to children on school days. Housing benefit is given to families to assist with accommodation rental costs; both are used in the UK as indicators of poverty.
References
Adams, N., Barton, A., Bray, S., Johnson, G., & Matejic, P. (2010). Households below average income: An analysis of the income distribution 1994/95–2008/09. London: Department for Work and Pensions.
Atkinson, A. B., & Micklewright, J. (1983). On the reliability of income data in the Family Expenditure Survey 1970–1977. Journal of the Royal Statistical Society Series A, 146(1), 33–61.
Becker, I., Frick, J. R., Grabka, M. M., Hauser, R., Krause, P., & Wagner, G. G. (2003). A comparison of the main household income surveys for Germany: EVS and SOEP. In R. Hauser & I. Becker (Eds.), Reporting on the income distribution and poverty: Perspectives from a German and European point of view. Berlin: Springer.
Biancotti, C., D’Alessio, G., & Neri, A. (2008). Measurement error in the Bank of Italy’s Survey of household income and wealth. Review of Income and Wealth, 54(3), 466–493.
Boheim, R., & Jenkins, S. P. (2006). A comparison of current and annual measures of income in the British Household Panel Survey. Journal of Official Statistics, 22(4), 733–758.
Bound, J., Brown, C., & Mathiowetz, N. (2000). Measurement error in survey data: Report No. 00-450. Ann Arbor, MI: Population Studies Center, University of Michigan.
Burchell, B., & Marsh, C. (1992). The effect of questionnaire length on survey response. Quality & Quantity, 26, 233–244.
Centre for Longitudinal Studies (2009). Millennium cohort study sweep 4: Questionnaire documentation. http://www.cls.ioe.ac.uk/core/documents/download.asp?id=1235andlog_stat=1.
Clarke, P. M., Fiebig, D. G., & Gerdtham, U.-G. (2008). Optimal recall length in survey design. Journal of Health Economics, 27(5), 1275–1284.
Collins, D., & White, A. (1996). In search of an income question for the 2001 census. Survey Methodology Bulletin, 39(7), 2–10.
Davern, M., Rodin, H., Beebe, T., & Thiede Call, K. (2005). The effect of income question design in health surveys on family income, poverty and eligibility estimates. HSR: Health Services Research, 40(5), 1534–1552.
Drewinka, S. (2010). Cross-sectional variation in individuals’ earnings instability. Review of Income and Wealth, 56(2), 291–326.
Easterlin, R. A. (2001). Income and happiness: Towards a unified theory. The Economic Journal, 111(473), 465–484.
European Commission. (2010). Income poverty and material deprivation in European countries. Luxembourg: Publications Office of the European Union.
Francesconi, M., Sutherland, H., & Zantomio, F. (2009). A comparison of earnings measures from longitudinal and cross-sectional surveys: Evidence from the UK. Colchester: Institute for Economic and Social Research.
Gordon, D. (1998). Measuring income in the Breadline Britain 1998 survey. In J. Bradshaw, D. Gordon, R. Levitas, S. Middleton, C. Pantazis, S. Payne, & P. Townsend (Eds.), Perceptions of poverty and social exclusion 1998. Bristol: Townsend Centre for International Poverty Research, University of Bristol.
Hansen, K., & Kneale, D. (2011). Measuring income in the millennium cohort study: Bands, units and groups—does it make a difference? CLS Working Paper. London: Centre for Longitudinal Studies, Institute of Education.
Hansen, K., & Machin, S. (2002). Spatial crime patterns and the introduction of the UK Minimum Wage. Oxford Bulletin of Economics and Statistics, 64(Supplement), 677–697.
Hawkes, D., & Plewis, I. (2008). Missing income data in the millennium cohort study. London: Centre for Longitudinal Studies, Institute of Education.
Hicks, J. R. (1939). Value and capital: An inquiry into some fundamental principles of economic theory. Oxford: Clarendon Press.
Hurd, M. D., Juster, F. T., & Smith, J. P. (2003). Enhancing the quality of data on income. The Journal of Human Resources, 38(3), 758–772.
Ketende, S. C., & Joshi, H. (2008). Income and poverty. In K. Hansen & H. Joshi (Eds.), Millennium cohort study third survey: A user’s guide to initial findings. London: Centre for Longitudinal Studies, Institute of Education.
Lynch, J. W., Smith, G. D., & Kaplan, G. A. (2000). Income inequality and mortality: Importance to health of individual income, psychosocial environment, or material conditions. British Medical Journal, 320, 1200–1204.
Mayer, S. E. (1997). What money can’t buy: Family income and children’s life chances. Cambridge, Massachusetts: Harvard University Press.
Micklewright, J., & Schnepf, S. (2010). How reliable are income data collected with a single question? Journal of the Royal Statistical Society Series A, 173(2), 409–429.
Moore, J. C., Stinson, L. L., & Welniack, E. J. J. (2000). Income measurement error in surveys: A review. Journal of Official Statistics, 16(4), 331–362.
Saunders, P., & Adelman, L. (2006). Income poverty, deprivation and exclusion: A comparative study of Australia and Britain. Journal of Social Policy, 35(4), 559–584.
Schrapler, J.-P. (2006). Explaining income non-response—a case study by means of the British Household Panel Study (BHPS). Quantity and Quality, 40, 1013–1036.
The Canberra Group. (2001). Final report and recommendations. Ottawa: The Canberra Group, Expert Group on Household Income Statistics.
Van de Poel, E., Hosseinpoor, A. R., Speybroek, N., Van Ourti, T., & Vega, J. (2008). Socioeconomic inequality in malnutrition in developing countries. Bulletin of the World Health Organisation, 86(4), 241–320.
Whelan, C. T., Layte, R., & Maître, B. (2004). Understanding the mismatch between income poverty and deprivation: A dynamic comparative analysis. European Sociological Review, 20(4), 287–302.
Wilkinson, R. G. (1992). Income distribution and life expectancy. British Medical Journal, 304, 165–168.
Acknowledgments
The authors would like to thank John Micklewright for early discussions about the issues covered in this paper and to he and Heather Joshi for comments on an earlier draft of this paper. Thanks too to the anonymous referees and the Editor of Social Indicators Research for their useful comments and suggestions.
Author information
Authors and Affiliations
Corresponding author
Additional information
Kirstine Hansen and Dylan Kneale are joint first authors.
Rights and permissions
About this article
Cite this article
Hansen, K., Kneale, D. Does How You Measure Income Make a Difference to Measuring Poverty? Evidence from the UK. Soc Indic Res 110, 1119–1140 (2013). https://doi.org/10.1007/s11205-011-9976-5
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11205-011-9976-5