Skip to main content
SpringerLink
Log in

Tracing Cognition with Assessment Center Simulations: Using Technology to See in the Dark

  • Chapter
  • First Online:
Simulations for Personnel Selection

Abstract

Shifts in the global economy have placed more pressure on the decisions of employees, managers, and CEOs than ever before. At the same time, the proliferation of technology in the workplace has become ubiquitous. Fortunately, the assessment center method is evolving alongside these other trends. With just a few modifications to existing assessment center simulations, we can use newer technology to capture previously difficult to observe behaviors that tap directly into candidate’s decision making processes. By combining passive data logging of key strokes and mouse clicks, eye tracking, and physiological responses such as skin conductivity, we are able to capture behaviors in real time that can be used to supplement traditional assessor ratings. Doing so allows assessors to compile a more accurate and holistic summary of a candidate’s performance. This wealth of new behavioral information has implications for high stakes hiring decisions as well as targeted training and development.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    We are also implying that over time, or by virtue of incumbent validity studies, a normative sample could be collected with which to provide feedback on decisional strategies relative to the average employee, average expert, and the like. We also recommend benchmarking such data against industry, role, etc.

  2. 2.

    The authors wish to thank George C. Thornton, III for his input on this matter.

References

  • Abelson, R. P., & Black, J. B. (1986). Introduction. In J. A. Galambos, R. P. Abelson & J. B. Black (Eds.), Knowledge structures (pp. 1–18). Hillsdale: Lawrence Erlbaum Associates.

    Google Scholar 

  • Bettman, J. R., & Park, C. W. (1980). Effects of prior knowledge and experience and phase of the choice process on consumer decision processes: A protocol analysis. Journal of Consumer Research, 7(3), 234–248.

    Article  Google Scholar 

  • Bruine de Bruin, W., Parker, A. M., & Fischoff, B. (2007). Individual differences in adult decision-making competence. Journal of Personality and Social Psychology, 92, 938–956.

    Article  PubMed  Google Scholar 

  • Chase, W. G., & Simon, H. A. (1973). Perception in chess. Cognitive Psychology, 4, 55–81.

    Article  Google Scholar 

  • Cook, G. J., & Swain, M. R. (1993). A computerized approach to decision process tracing for decision support systems design. Decision Science, 24, 683–697.

    Article  Google Scholar 

  • Corporate Leadership Council (2005). Trends in developmental assessment centers. http://www.scribd.com/doc/57017184/Trends-in-Developmental-Assessment-Centers. Accessed 1 Dec 2011.

  • Dean, M. A., Roth, P. L., & Bobko, P. (2008). Ethnic and gender subgroup differences in assessment center ratings: A meta-analysis. Journal of Applied Psychology, 93(3), 685.

    Google Scholar 

  • Dilchert, S., & Ones, D. S. (2009). Assessment center dimensions: Individual differences correlates and meta-analytic incremental validity. International Journal of Selection and Assessment, 17(3), 254–270.

    Article  Google Scholar 

  • Driskell, J. E., Salas, E., & Hall, J. K. (1994). The effect of vigilant and hypervigilant decision training on performance. In Annual Meeting of the Society of Industrial and Organizational Psychology, Nashville.

    Google Scholar 

  • Duchowski, A. (2007). Eye tracking methodology: Theory and practice (2nd Edition). London: Springer.

    Google Scholar 

  • Edwards, W., & Fasolo, B. (2001). Decision technology. Annual Review of Psychology, 52(1), 581–606.

    Article  PubMed  Google Scholar 

  • Einhorn, H. J., & Hogarth, R. M. (1981). Behavioral decision theory: Processes of judgment and choice. Annual Review of Psychology, 32, 53–88.

    Article  Google Scholar 

  • Evans, M. (2012). The key lessons from RIM’s struggles. Forbes. From http://www.forbes.com/sites/markevans/2012/07/04/key-lessons-from-rim/. Accessed 1 Mar 2013.

  • Farr, J. L. (1973). Response requirements and primacy-recency effects in a simulated selection interview. Journal of Applied Psychology, 57(3), 228–232.

    Article  Google Scholar 

  • Ferreira, M. B., Garcia-Marques, L., Sherman, S. J., & Sherman, J. W. (2006).Automatic and controlled components of judgment and decision making. Journal of Personality and Social Psychology, 91(5), 797.

    Google Scholar 

  • Figner, B., & Murphy, R. O. (2011). Using skin conductance in judgment and decision making research. In M. Schulte-Mecklenbeck, A. Kuehberger & R. Ranyard (Eds.), A handbook of process tracing methods for decision research: A critical review and user’s guide (pp. 163–184). New York: Psychology Press.

    Google Scholar 

  • Ford, J. K., Schmitt, N., Schechtman, S. L., Hults, B. M., & Doherty, M. L. (1989). Process tracing methods: Contributions, problems, and neglected research questions. Organizational Behavior and Human Decision Processes, 43, 75–117.

    Article  Google Scholar 

  • Gibbons, A. M., Hughes, D., Riley, P., Thornton, G. C., & Sanchez, D. (2013). Is the future here? Current uses and perceived benefits of technology in operational assessment centers Unpublished Manuscript.

    Google Scholar 

  • Hammond, K. R. (1996). Human judgment and social policy: Irreducible uncertainty, inevitable error, unavoidable injustice. USA: Oxford University Press.

    Google Scholar 

  • Hammond, K. R., Hamm, R. M., Grassia, J., & Pearson, T. (1997). Direct comparison of the efficacy of intuitive and analytical cognition in expert judgment. In W. M. Goldstein & R. M. Hogarth (Eds.), Research on judgment and decision making: Currents, connections, and controversies (pp. 144–180). Cambridge series on judgment and decision making. Cambridge, UK: Cambridge University Press.

    Google Scholar 

  • Hayes-Roth, B. (1977).Evolution of cognitive structures and processes. Psychological Review, 84(3), 260.

    Google Scholar 

  • Highhouse, S. (1997). Understanding and improving job-finalist choice: The relevance of behavioral decision research. Human Resource Management Review, 7(4), 449–470.

    Article  Google Scholar 

  • Huber, J., Payne, J. W., & Puto, C. (1982). Adding asymmetrically dominated alternatives: Violations of regularity and the similarity hypothesis. Journal of Consumer Research, 9, 90–98.

    Article  Google Scholar 

  • Hughes, D., Riley, P., Shalfrooshan, A., Gibbons, A., Thornton, G. (2012). A global survey of assessment center practices. A research report by a & dc and Colorado State University.

    Google Scholar 

  • International Task Force on Assessment Center Guidelines (2009). Guidelines and ethical considerations for assessment center operations. International Journal of Selection and Assessment, 17, 243–253.

    Article  Google Scholar 

  • Jacoby, J., Mazursky, D., Troutman, T., Kuss, A. (1984). When feedback is ignored: Disutility of outcome feedback. Journal of Applied Psychology, 69(3), 531–545.

    Article  Google Scholar 

  • Jasper, J. D., & Shapiro, J. (2002). MouseTrace: A better mousetrap for catching decision processes. Behavior Research Methods, Instruments, & Computers, 34, 364–374.

    Article  Google Scholar 

  • Johnston, J. H., Driskell, J. E., & Salas, E. (1997). Vigilant and hypervigilant decision making. Journal of Applied Psychology, 82(4), 614–622.

    Article  PubMed  Google Scholar 

  • Kirschenbaum, S. S. (1992). Influence of experience on information-gathering strategies. Journal of Applied Psychology, 77(3), 343–352.

    Article  Google Scholar 

  • Klein, J. D. (1997). ETR & D-Development: An analysis of content and survey of future direction. Educational Technology Research and Development, 45(3), 57–62.

    Article  Google Scholar 

  • Koop, G. J., & Johnson, J. G. (2011). Response dynamics: A new window in the decision process. Judgment and Decision Making, 6(8), 750–758.

    Google Scholar 

  • Lievens, F. (2008). What does exercise-based assessment really mean? Industrial and Organizational Psychology: Perspectives on Science and Practice, 1(1), 112–115.

    Article  Google Scholar 

  • Lievens, F., & Schollaert, E. (2011). Reshaping exercise design in assessment centers: Theory, practice, and research. In N. Povah & G. C. Thornton III (Eds.), Assessment centres and global talent management (pp. 47–60). Surrey, UK: Gower.

    Google Scholar 

  • Lievens, F., Tett, R. P., & Schleicher, D. J. (2009). Assessment centers at the crossroads: Toward a reconceptualization of assessment center exercises. Research in Personnel and Human Resources Management, 28, 99–152.

    Article  Google Scholar 

  • Mann, L., Harmoni, R., & Power, C. (1991). The GOFER course in decision making. In J. Baron & R. V. Brown (Eds.), Teaching decision making to adolescents (pp. 61–78). Hillsdale: Lawrence Erlbaum Associates, Inc.

    Google Scholar 

  • McGregor, J. (2012). In cruise ship sinking, leadership failures from captain to Carnival CEO. The Washington Post Online. http://www.washingtonpost.com/blogs/post-leadership/post/in-cruise-ship-sinking-leadership-failures-from-captain-to-carnival-ceo/2011/04/01/gIQACNtkAQ_blog.html. Accessed 1 Feb 2012.

  • Miller, C. C., & Kopytoff, V. G. (2011). Once a leader, Yahoo now struggles to find its way. The New York Times Online. http://www.nytimes.com/2011/09/08/technology/once-a-leader-yahoo-now-struggles-to-find-its-way.html?pagewanted=all&_r=0. Accessed 1 Mar 2012.

  • Miller, J. W., & Rowe, P. M. (1967). Influence of favorable and unfavorable information upon assessment decisions. Journal of Applied Psychology, 51(5), 432–435.

    Article  PubMed  Google Scholar 

  • National Bureau of Economic Research. (2008). Determination of the December 2007 peak in economic activity. National Bureau of Economic Research. From http://www.nber.org/cycles/dec2008.pdf. Accessed 1 Nov 2011.

  • Nisbett, R. E., & Wilson, T. D. (1977).Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84(3), 231.

    Google Scholar 

  • Nisbett, R. E., Zukier, H., & Lemley, R. E. (1981). The dilution effect: Nondiagnostic information weakens the implications of diagnostic information. Cognitive Psychology, 13(2), 248–277.

    Article  Google Scholar 

  • Onken, J., Hastie, R., & Revelle, W. (1985). Individual differences in the use of simplification strategies in a complex decision-making task. Journal of Experimental Psychology: Human Perception and Performance, 11(1), 14–27.

    Article  PubMed  Google Scholar 

  • Patrick, J., & James, N. (2004). Process tracing of complex cognitive work tasks. Journal of Occupational and Organizational Psychology, 77, 259–280.

    Article  Google Scholar 

  • Payne, J. W. (1976). Task complexity and contingent processing in decision making: An informational search and protocol analysis. Organizational Behavior and Human Performance, 16, 366–387.

    Article  Google Scholar 

  • Payne, J. W. (1982). Contingent decision behavior. Psychological Bulletin, 92(2), 382–402.

    Article  Google Scholar 

  • Payne, J. W., Bettman, J. R., & Johnson, E. J. (1993). The adaptive decision maker. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  • Payne, J. W., Braunstein, M. L., & Carroll, J. S. (1978). Exploring predecisional behavior: An alternative approach to decision research. Organizational Behavior and Human Performance, 22(1), 17–44.

    Article  Google Scholar 

  • Perrin, B. M., Barnett, B. J., Walrath, L., & Grossman, J. D. (2001). Information order and outcome framing: An assessment of judgment bias in a naturalistic decision-making context. Human Factors: The Journal of the Human Factors and Ergonomics Society, 43(2), 227–238.

    Article  Google Scholar 

  • Povah, N., & Thornton, G. C. III. (2011). Assessment centers and global talent management. Surrey, UK: Gower Publishing Limited.

    Google Scholar 

  • Reynolds, D. H., & Dickter, D. N. (2010). Technology and employee selection. In J. L. Farr & N. T. Tippins (Eds.), Handbook of employee selection. New York: Psychology Press.

    Google Scholar 

  • Reynolds, D. H., & Rupp, D. E. (2010). Advances in technology-facilitated assessment. In J. C. Scott & D. H. Reynolds (Eds.), Handbook of workplace assessment: Evidence-based practices for selecting and developing organizational talent (pp. 609–641). San Francisco: Jossey-Bass.

    Google Scholar 

  • Riedl, R., Brandstätter, E., & Roithmayr, R. (2008). Identifying decision strategies: A process- and outcome-based classification method. Behavioral Research Methods, 40(3), 795–807.

    Article  Google Scholar 

  • Rigney, J. W., & DeBow, C. H. (1967). Multidimensional scaling analysis of decision strategies in threat evaluation. Journal of Applied Psychology, 51(4), 305–310.

    Article  PubMed  Google Scholar 

  • Rowe, P. M. (1967). Order effects in assessment decisions. Journal of Applied Psychology, 51(2), 170–173.

    Article  PubMed  Google Scholar 

  • Rupp, D. E., Gibbons, A. M., & Snyder, L. A. (2008). The role of technology in enabling third-generation training and development. Industrial and Organizational Psychology, 1(4), 496–500.

    Article  Google Scholar 

  • Russo, J. E. (2011). Eye fixation as a process trace. In M. Schulte-Mecklenbeck, A. Kühberger & R. Ranyard (Eds.), A handbook of process tracing methods for decision research: A critical review and user’s guide (pp. 43–64). New York: Psychology Press.

    Google Scholar 

  • Schleicher, D. J., Day, D. V., Mayes, B. T., & Riggio, R. E. (2002). A new frame for Frame-of-Reference Training: Enhancing the construct validity of assessment centers. Journal of Applied Psychology, 87(4), 735–746.

    Article  PubMed  Google Scholar 

  • Sherer, P. D., Schwab, D. P., & Heneman, H. G. III (1987). Managerial salary-raise decisions: A policy-capturing approach. Personnel Psychology, 40, 27–38.

    Article  Google Scholar 

  • Simon, Z. (2012). Malev airline stops services after 66 years as Hungary moves to cut losses. Bloomberg Online. From http://www.bloomberg.com/news/2012–02-03/malev-stops-flying-after-66-years-as-hungary-cuts-its-losses.html. Accessed 1 Mar 2012.

  • Simonson, I. (1989). Choice based on reasons: The case of attraction and compromise effects. Journal of Consumer Research, 16, 158–174.

    Article  Google Scholar 

  • Sims, H. P., & Gioia, D. A. (1986). The thinking organization. San Francisco: Jossey-Bass.

    Google Scholar 

  • Soll, J. B., & Larrick, R. P. (2009). Strategies for revising judgment: How (and how well) people use others’ opinions. Journal of Experimental Psychology: Learning, Memory,and Cognition, 35(3), 780.

    Google Scholar 

  • Spector, M., & Trachtenberg, J. A. (2011). Borders forced to liquidate, close all stores. The Wall Street Journal Online. http://online.wsj.com/article/SB10001424052702303661904576454353768550280.html. Accessed 1 Jan 2013.

  • Starbuck, W. H., & Milliken, F. (1988). Executives’ perceptual filters: What they notice and how they make sense. In D. Hambrick (Ed.), The executive effect: Concepts and methods of studying top managers (pp. 35–65). Greenwich, CT: JAI Press.

    Google Scholar 

  • Svenson, O. (1979). Process descriptions of decision making. Organizational Behavior and Human Performance, 23, 86–112.

    Article  Google Scholar 

  • Tabatabai, M. (1998). Investigation of decision making process: A hypermedia approach. Interacting With Computers, 9, 385–396.

    Article  Google Scholar 

  • Tetlock, P. E., & Boettger, R. (1989). Accountability: A social magnifier of the dilution effect. Journal of Personality and Social Psychology, 57(3), 388.

    Google Scholar 

  • Thorngate, W. (1980). Efficient decision heuristics. Behavioral Science, 25(3), 219–225.

    Article  Google Scholar 

  • Thornton, G. C. III, & Mueller-Hanson, R. (2004). Developing organizational simulations: A guide for practitioners and students. Series in applied psychology. Mahwah: Lawrence Erlbaum Associates.

    Google Scholar 

  • Thornton, G. C. III, & Rupp, D. E. (2006). Assessment centers in human resource management: Strategies for prediction, diagnosis, and development. Mahawah: Lawrence Erlbaum Associates, Inc.

    Google Scholar 

  • Walsh, J. P. (1995). Managerial and organizational cognition: Notes from a trip down memory lane. Organizational Science, 6(3), 280–321.

    Article  Google Scholar 

  • Wang, J. T. Y. (2011). Pupil dilation and eye tracking. In M. Schulte-Mecklenbeck, A. Kühberger & R. Ranyard (Eds.), A handbook of process tracing methods for decision research: A critical review and user’s guide (pp. 185–204). New York: Psychology Press.

    Google Scholar 

  • Worthen, B. (2012). H-P, Dell struggle as buyers shun PCs. The Wall Street Journal Online. http://online.wsj.com/article/SB10000872396390444812704577605703329715394.html. Accessed 1 Mar 2012.

  • Zakay, D., & Wooler, S. (1984). Time pressure, training and decision effectiveness. Ergonomics, 27(3), 273–284.

    Article  PubMed  Google Scholar 

  • Zukier, H. (1982).The dilution effect: The role of the correlation and the dispersion of predictor variables in the use of nondiagnostic information. Journal of Personality and Social Psychology, 43(6), 1163.

    Google Scholar 

Download references

Acknowledgments

This research was supported by the SIOP Foundation’s Douglas W. Bray and Ann Howard Research Grant. The authors would like to thank Carolyn Jagacinski and James LeBreton for helpful feedback.

Author information

Authors and Affiliations

  1. Purdue University, West Lafayette, IN, USA

    Brett W. Guidry & Deborah E. Rupp

  2. Global Assessor Pool, Denver, CO, USA

    Martin Lanik

Authors
  1. Brett W. Guidry
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Deborah E. Rupp
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Martin Lanik
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Brett W. Guidry .

Editor information

Editors and Affiliations

  1. SHL-CEB, Alpharetta, Georgia, USA

    Michael Fetzer

  2. SHL Group Ltd., Arlington, Virginia, USA

    Kathy Tuzinski

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer Science+Business Media New York

About this chapter

Cite this chapter

Guidry, B., Rupp, D., Lanik, M. (2013). Tracing Cognition with Assessment Center Simulations: Using Technology to See in the Dark. In: Fetzer, M., Tuzinski, K. (eds) Simulations for Personnel Selection. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-7681-8_11

Download citation

Publish with us

Policies and ethics

Search

Navigation