Skip to main content
SpringerLink
Log in

Comparing acquiescent and extreme response styles in face-to-face and web surveys

  • Published:
Quality & Quantity Aims and scope Submit manuscript

Abstract

Likert scales are popular for measuring attitudes, but response style, a source of measurement error associated with this type of question, can result in measurement bias. This study investigates the effect of data collection mode on both types of response styles using data from the 2012 American National Election Studies (ANES). 2012 was the 1 year in which ANES conducted two parallel surveys, one through face-to-face interviews and another through Web, using two independent national probability samples and an identical questionnaire. We used three sets of balanced Likert scales from the survey to measure the acquiescent and extreme response styles. Using the latent class analysis modeling approach, we find that: (1) both acquiescent and extreme response styles exist in both face-to-face and Web survey modes; (2) face-to-face respondents demonstrate more acquiescent and extreme response styles than Web respondents; (3) the mode effect is similar for white, black and Hispanic respondents.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1

Similar content being viewed by others

Notes

  1. Content factor 1 class sizes: 0.15, 0.52, 0.23, 0.10; content factor 2 class sizes: 0.14, 0.46, 0.25, 0.15; Content factor 3 class sizes: 0.13, 0.60, 0.19, 0.08).

  2. The results for Model 6, which has the smallest BIC, are presented in the appendix. The results remain the same as Model 6b, meaning that the findings reported in this paper are not an artifact of the selected model.

References

  • Aday, L.A., Chiu, G.Y., Andersen, R.: Methodological issues in health care surveys of the Spanish heritage population. Am. J. Public Health 70(4), 367–374 (1980)

    Article  Google Scholar 

  • Bachman, J.G., O’Malley, P.M.: Black-white differences in self-esteem: are they affected by response styles? Am. J. Sociol. 90(3), 624–639 (1984a)

    Article  Google Scholar 

  • Bachman, J.G., O’Malley, P.M.: Yea-saying, nay-saying, and going to extremes: black-white differences in response styles. Public Opin. Q. 48(2), 491–509 (1984b)

    Article  Google Scholar 

  • Bachman, J. G., O’Malley, P. M.: Response styles revisited: racial/ethnic and gender differences in extreme responding. Retrieved from http://monitoringthefuture.org/pubs/occpapers/occ72.pdf (2010)

  • Baumgartner, H., Steenkamp, J.-B.E.M.: Response styles in marketing research: a cross-national investigation. J. Mark. Res. 38(2), 143–156 (2001)

    Article  Google Scholar 

  • Billiet, J.B., Davidov, E.: Testing the stability of an acquiescence style factor behind two interrelated substantive variables in a panel design. Sociol. Methods Res. 36(4), 542–562 (2008). doi:10.1177/0049124107313901

    Article  Google Scholar 

  • Billiet, J.B., McClendon, M.J.: Modeling acquiescence in measurement models for two balanced sets of items. Struct. Equ. Model. 7(4), 608–628 (2000). doi:10.1207/S15328007SEM0704_5

    Article  Google Scholar 

  • Bolt, D.M., Newton, J.R.: Multiscale measurement of extreme response style. Educ. Psychol. Meas. 71(5), 814–833 (2011). doi:10.1177/0013164410388411

    Article  Google Scholar 

  • Carr, L.G.: The srole items and acquiescence. Am. Sociol. Rev. 36(2), 287–293 (1971)

    Article  Google Scholar 

  • Clarke III, I.: Extreme response style in cross-cultural research. Int. Mark. Rev. 18(3), 301–324 (2001)

    Article  Google Scholar 

  • Couch, A., Keniston, K.: Yeasayers and naysayers: agreeing response set as a personality variable. J. Abnorm. Soc. Psychol. 60(2), 151–174 (1960)

    Article  Google Scholar 

  • Cronbach, L.J.: Studies of acquiescence as a factor in the true-false test. J. Educ. Psychol. 33(6), 401–415 (1942)

    Article  Google Scholar 

  • Davis, R.E., Resnicow, K., Couper, M.P.: survey response styles, acculturation, and culture among a sample of mexican american adults. J. Cross-Cult. Psychol. 42(7), 1219–1236 (2010). doi:10.1177/0022022110383317

    Article  Google Scholar 

  • De Leeuw, E.D.: Data quality in mail, telephone and face to face surveys. ERIC. Retrieved from http://eric.ed.gov/?id=ED374136 (1992)

  • De Leeuw, E.D.: To mix or not to mix data collection modes in surveys. J. Off. Stat. 21(5), 233–255 (2005)

    Google Scholar 

  • de Spinoza, B., Shirley, S., Feldman, S.: The Ethics and Selected Letters. Hackett Pub. Co, Indianapolis (1982)

    Google Scholar 

  • Gibbons, J.L., Zellner, J.A., Rudek, D.J.: Effects of language and meaningfulness on the use of extreme response style by Spanish–English bilinguals. Cross-Cult. Res. 33(4), 369–381 (1999)

    Article  Google Scholar 

  • Gilbert, D.T.: How mental systems believe. Am. Psychol. 46(2), 107–119 (1991)

    Article  Google Scholar 

  • Heerwegh, D., Loosveldt, G.: Face-to-face versus web surveying in a high-internet-coverage population differences in response quality. Public Opin. Q. 72(5), 836–846 (2008). doi:10.1093/poq/nfn045

    Article  Google Scholar 

  • Heerwegh, D., Loosveldt, G.: Assessing mode effects in a national crime victimization survey using structural equation models: social desirability bias and acquiescence. J. Off. Stat. 27(1), 49 (2011)

    Google Scholar 

  • Hui, C.H., Triandis, H.C.: Effects of culture and response format on extreme response style. J. Cross-Cult. Psychol. 20(3), 296–309 (1989). doi:10.1177/0022022189203004

    Article  Google Scholar 

  • Jordan, L.A., Marcus, A.C., Reeder, L.G.: Response styles in telephone and household interviewing: a field experiment. Public Opin. Q. 44(2), 210–222 (1980)

    Article  Google Scholar 

  • Kankaraš, M., Moors, G.: Measurement equivalence in solidarity attitudes in Europe insights from a multiple-group latent-class factor approach. Int. Sociol. 24(4), 557–579 (2009). doi:10.1177/0268580909334502

    Article  Google Scholar 

  • Kieruj, N.D., Moors, G.: variations in response style behavior by response scale format in attitude research. Int. J. Public Opin. Res. 22(3), 320–342 (2010). doi:10.1093/ijpor/edq001

    Article  Google Scholar 

  • Kieruj, N.D., Moors, G.: Response style behavior: question format dependent or personal style? Qual. Quant. 47(1), 193–211 (2013). doi:10.1007/s11135-011-9511-4

    Article  Google Scholar 

  • Kiesler, S., Sproull, L.S.: Response effects in the electronic survey. Public Opin. Q. 50(3), 402–413 (1986). doi:10.1086/268992

    Article  Google Scholar 

  • Knowles, E.S., Condon, C.A.: Why people say yes: a dual-process theory of acquiescence. J. Pers. Soc. Psychol. 77(2), 379–386 (1999)

    Article  Google Scholar 

  • Krosnick, J.A.: Response strategies for coping with the cognitive demands of attitude measures in surveys. Appl. Cogn. Psychol. 5(3), 213–236 (1991)

    Article  Google Scholar 

  • Leary, M.R., Kowalski, R.M.: Impression management: a literature review and two-component model. Psychol. Bull. 107(1), 34–47 (1990)

    Article  Google Scholar 

  • Madsen, W.: Mexican-Americans of South Texas. Retrieved from http://www.eric.ed.gov/ERICWebPortal/recordDetail?accno=ED021661 (1964)

  • Marin, G., Gamba, R.J., Marin, B.V.: extreme response style and acquiescence among hispanics the role of acculturation and education. J. Cross-Cult. Psychol. 23(4), 498–509 (1992). doi:10.1177/0022022192234006

    Article  Google Scholar 

  • Marin, G., Marin, B.V.: Research with Hispanic Populations. 2455 Teller Road, Newbury Park California 91320 United States of America: SAGE Publications, Inc. Retrieved from http://srmo.sagepub.com/view/research-with-hispanic-populations/SAGE.xml (1991)

  • Moors, G.: Diagnosing response style behavior by means of a latent-class factor approach. Socio-demographic correlates of gender role attitudes and perceptions of ethnic discrimination reexamined. Qual. Quant. 37(3), 277–302 (2003). doi:10.1023/A:1024472110002

    Article  Google Scholar 

  • Moors, G.: Exploring the effect of a middle response category on response style in attitude measurement. Qual. Quant. 42(6), 779–794 (2008). doi:10.1007/s11135-006-9067-x

    Article  Google Scholar 

  • Moors, G.: ranking the ratings: a latent-class regression model to control for overall agreement in opinion research. Int. J. Public Opin. Res. 22(1), 93–119 (2010). doi:10.1093/ijpor/edp036

    Article  Google Scholar 

  • Moors, G., Kieruj, N.D., Vermunt, J.K.: The effect of labeling and numbering of response scales on the likelihood of response bias. Sociol. Methodol. (2014). doi:10.1177/0081175013516114

    Google Scholar 

  • Morren, M., Gelissen, J.P.T.M., Vermunt, J.K.: dealing with extreme response style in cross-cultural research: a restricted latent class factor analysis approach: extreme response style in cross-cultural research. Sociol. Methodol. 41(1), 13–47 (2011). doi:10.1111/j.1467-9531.2011.01238.x

    Article  Google Scholar 

  • Naemi, B.D., Beal, D.J., Payne, S.C.: Personality predictors of extreme response style. J. Personal. 77(1), 261–286 (2009). doi:10.1111/j.1467-6494.2008.00545.x

    Article  Google Scholar 

  • Paulhus, D. L.: Measurement and control of response bias. Measures of Personality and Social Psychological Attitudes. Measures of Social Psychological Attitudes, Vol. 1. Retrieved from http://doi.apa.org/psycinfo/1991-97206-001 (1991)

  • Ross, C.E., Mirowsky, J.: Socially-desirable response and acquiescence in a cross-cultural survey of mental health. J. Health Soc. Behav. 25(2), 189–197 (1984)

    Article  Google Scholar 

  • Saris, W.E., Revilla, M., Krosnick, J.A., Shaeffer, E.M.: Comparing questions with agree/disagree response options to questions with item-specific response options. Surv. Res. Methods 4(1), 61–79 (2010)

    Google Scholar 

  • Schlenker, B. R.: Impression Management: the Self-Concept, Social Identity, and Interpersonal Relations. Brooks/Cole Publishing Company, Monterey. Retrieved from http://www.getcited.org/pub/102061888 (1980)

  • Thomas, T.D., Abts, K., Weyden, P.V.: Response styles and the rural-urban divide. Educ. Psychol. Meas. 74(1), 97–115 (2014). doi:10.1177/0013164413502487

    Article  Google Scholar 

  • Vannieuwenhuyze, J.T.A., Revilla, M.: Relative mode effects on data quality in mixed-mode surveys by an instrumental variable. Surv. Res. Methods 7(3), 157–168 (2013). doi:10.18148/srm/2013.v7i3.5137

    Google Scholar 

  • Van Vaerenbergh, Y., Thomas, T.D.: Response styles in survey research: a literature review of antecedents, consequences, and remedies. Int. J. Public Opin. Res. 25(2), 195–217 (2013). doi:10.1093/ijpor/eds021

    Article  Google Scholar 

  • Warnecke, R.B., Johnson, T.P., Chávez, N., Sudman, S., O’rourke, D.P., Lacey, L., Horm, J.: Improving question wording in surveys of culturally diverse populations. Ann. Epidemiol. 7(5), 334–342 (1997)

  • Weijters, B., Geuens, M., Schillewaert, N.: The individual consistency of acquiescence and extreme response style in self-report questionnaires. Appl. Psychol. Meas. 34(2), 105–121 (2010). doi:10.1177/0146621609338593

    Article  Google Scholar 

  • Weijters, B., Schillewaert, N., Geuens, M.: Assessing response styles across modes of data collection. J. Acad. Mark. Sci. 36(3), 409–422 (2008). doi:10.1007/s11747-007-0077-6

    Article  Google Scholar 

  • Welkenhuysen-Gybels, J., Billiet, J., Cambré, B.: Adjustment for acquiescence in the assessment of the construct equivalence of likert-type score items. J. Cross-Cult. Psychol. 34(6), 702–722 (2003). doi:10.1177/0022022103257070

    Article  Google Scholar 

  • Yang, Y., Harkness, J.A., Chin, T.-Y., Villar, A.: Response styles and culture. In: Survey Methods in Multinational, Multiregional, and Multicultural Contexts, 203–223 (2010)

Download references

Author information

Authors and Affiliations

  1. SurveyMonkey, Palo Alto, CA, USA

    Mingnan Liu

  2. Program in Survey Methodology, University of Michigan, Ann Arbor, MI, USA

    Frederick G. Conrad

  3. Joint Program in Survey Methodology, University of Maryland, College Park, MD, USA

    Frederick G. Conrad

  4. Survey Research Center, University of Michigan, Ann Arbor, MI, USA

    Sunghee Lee

Authors
  1. Mingnan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Frederick G. Conrad
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sunghee Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mingnan Liu.

Appendix

Appendix

Table 6.

Table 6 Estimated regression coefficients (log odds) and standard errors of mode, race, interaction, and control variables on extreme response style (ERS) and acquiescent response style (ARS), 2012 American National Election Studies, based on model 6 in Table 3
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, M., Conrad, F.G. & Lee, S. Comparing acquiescent and extreme response styles in face-to-face and web surveys. Qual Quant 51, 941–958 (2017). https://doi.org/10.1007/s11135-016-0320-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11135-016-0320-7

Keywords

Search

Navigation