Abstract
Game-based assessments and learning environments create unique opportunities to provide learners with the ability to demonstrate their proficiency with cognitive skills and behaviors in increasingly authentic environments. Effective task designs, and the effective alignment of tasks with constructs, are also improving our ability to provide learners with insights about their proficiency with these skills. Sharing these insights within the industry with those working toward the same goal contributes to the rising tide that lifts all boats. In this paper we present insights from our work to develop and measure collaborative problem solving skills using a game-based assessment “Circuit Runner.” Our innovative educational game design allows us to incorporate item response data, telemetry data, and stealth- telemetry data to provide a more authentic measure of collaborative problem solving skills. Our study design included 379 study participants on Amazon Mechanical Turk (MTurk), who completed the “Circuit Runner” CPS assessment. The paper provides details on the design of educational games and scoring techniques and discusses findings from the pilot study.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
von Davier, A.A., Zhu, M., Kyllonen, P.C. (eds.): Innovative Assessment of Collaboration. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-33261-1
OECD: PISA 2015 Results: Collaborative Problem Solving. PISA, OECD Publishing, Paris (2017)
Griffin, P., Care, E., McGaw, B.: The changing role of education and schools. In: Griffin, P., McGaw, B., Care, E. (eds.) Assessment and Teaching 21st Century Skills, pp. 1–15. Springer, Heidelberg (2012). https://doi.org/10.1007/978-94-007-2324-5_1
Rosen, Y.: Computer-based assessment of collaborative problem solving: exploring the feasibility of human-to-agent approach. Int. J. Artif. Intell. Educ. 25(3), 380–406 (2015)
Camara, W., O’Connor, R., Mattern, K., Hanson, M.A.: Beyond Academics: A Holistic Framework for Enhancing Education and Workplace Success. ACT Research Report Series. ACT, Inc. (2015)
Briggs, D.C., Alonzo, A.C., Schwab, C., Wilson, M.: Diagnostic assessment with ordered multiple-choice items. Educ. Assess. 11(1), 33–63 (2006)
Rosen, Y.: Chatbot Authoring Tool for Innovative Learning and Assessment Tasks: Microsoft-Harvard-TalkCoaches Collaboration. Microsoft Symposium on Advancements in Assessments, Stanford (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Stoeffler, K., Rosen, Y., Bolsinova, M., von Davier, A.A. (2018). Gamified Assessment of Collaborative Skills with Chatbots. In: Penstein Rosé, C., et al. Artificial Intelligence in Education. AIED 2018. Lecture Notes in Computer Science(), vol 10948. Springer, Cham. https://doi.org/10.1007/978-3-319-93846-2_64
Download citation
DOI: https://doi.org/10.1007/978-3-319-93846-2_64
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-93845-5
Online ISBN: 978-3-319-93846-2
eBook Packages: Computer ScienceComputer Science (R0)