Hostname: page-component-55f67697df-twqc4 Total loading time: 0 Render date: 2025-05-10T12:51:19.004Z Has data issue: false hasContentIssue false
  • English
  • Français

Development of adaptive human–computer interaction games to evaluate attention

Published online by Cambridge University Press:  05 May 2021

Hasan Kandemir  and
Hatice Kose Open the ORCID record for Hatice Kose [Opens in a new window]
Hasan Kandemir
Affiliation:
Faculty of Computer and Informatics Engineering, Istanbul Technical University, Istanbul, Turkey
Hatice Kose*
Affiliation:
Faculty of Computer and Informatics Engineering, Istanbul Technical University, Istanbul, Turkey
*
*Corresponding author. Email: hatice.kose@itu.edu.tr
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

In this paper, several physical activity-based human–computer interaction (HCI) games which are developed and implemented for the improvement of attention, emotion, and sensory–motor coordination will be presented. The interface and the difficulty levels of these games are specially designed for the use of people with different age groups and disabilities. The games involve physical activities for the fulfillment of some basic HCI tasks which require hand and arm motion for control, such as fruit picking and air hockey, with adaptive difficulty levels based on varying parameters of the games and human performance. In the fruit picking game, several fruit images are moving from top to the bottom of the screen. Objective is to collect apples while avoiding the pears. The player’s hand will control the basket that collects the fruits. In the air hockey game, the player will try to score goals against computer-controlled opponent. The player’s hand will control the paddle to hit the puck to score or to defend his/her goal area. The player’s hand is recognized by Kinect RGB-D sensors in both games. Aim of the adaptive difficulty-based system is keeping the players engaged in the games. The games are tested with a group of deaf children (3.5–5 years) as a part of an ongoing project,1 to decrease the stress of the children and increase their positive emotions, attention, and sensory–motor coordination before the audiology tests. The game performances and the evaluation of the therapists show that the games have a positive impact on the children. The games are also tested with a group of adults as a control group, where a mobile EEG device is employed to detect the attention levels. For this purpose, the adults also attended a third game featuring a maze and controlled with Myo sensors.

Keywords

Brain–computer interfaceLearningCognitive skills developmentHCI gamesKinectMyo sensor
Type
Article
Information
Robotica , Volume 40 , Issue 1 , January 2022 , pp. 56 - 76
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.
Copyright
© The Author(s), 2021. Published by Cambridge University Press

References

Altanis, G., Boloudakis, M., Retalis, S. and Nikou, N., “Children with motor impairments play a kinect learning game: First findings from a pilot case in an authentic classroom environment,” J. Interact. Des. Architect. 19(1), 91–104 (2014).Google Scholar
Bartoli, L., Corradi, C., Garzotto, F. and Valoriani, M., “Exploring Motion-based Touchless Games for Autistic Children’s Learning,” Proceedings of the 12th International Conference on Interaction Design and Children (ACM, 2013) pp. 102–111.CrossRefGoogle Scholar
Burguillo, J. C., “Using game theory and competition-based learning to stimulate student motivation and performance,” Comput. Edu. 55(2), 566–575 (2010).CrossRefGoogle Scholar
Cameron, B. and Dwyer, F., “The effect of online gaming, cognition and feedback type in facilitating delayed achievement of different learning objectives,” J. Interact. Learn. Res. 16(3), 243–258 (2005).Google Scholar
Çankaya, S. and Karamete, A., “The effects of educational computer games on students’ attitudes towards mathematics course and educational computer games,” Procedia-Soc. Behav. Sci. 1(1), 145–149 (2009).CrossRefGoogle Scholar
Charlton, B., Williams, R. L. and McLaughlin, T. F., “Educational games: A technique to accelerate the acquisition of reading skills of children with learning disabilities,” Int. J. Spec. Edu. 20(2), 66–72 (2005).Google Scholar
Chen, C.-L. D., Yeh, T.-K. and Chang, C.-Y., “The effects of game-based learning and anticipation of a test on the learning outcomes of 10th grade geology students,” Eurasia J. Math. Sci. Technol. Edu. 12(5), 1379–1388 (2016).Google Scholar
Cohen, A., Ivry, R. I and Keele, S. W., “Attention and structure in sequence learning,” J. Exp. Psychol. Learn. Mem. Cognit. 16(1), 17 (1990).CrossRefGoogle Scholar
Divjak, B. and Tomić, D., “The impact of game-based learning on the achievement of learning goals and motivation for learning mathematics-literature review,” J. Inf. Organizat. Sci. 35(1), 15–30 (2011).Google Scholar
Gurpinar, C., Uluer, P., Akalin, N. and Köse, H., “Sign recognition system for an assistive robot sign tutor for children,” Int. J. Soc. Rob. 12(2), 355–369 (2020).CrossRefGoogle Scholar
Ho, S. S., Lwin, M. O., Sng, J. R. H. and Yee, A. Z. H., “Escaping through exergames: Presence, enjoyment, and mood experience in predicting children’s attitude toward exergames,” Comput. Hum. Behav. 72, 381–389 (2017).CrossRefGoogle Scholar
Huang, W.-H., Huang, W.-Y. and Tschopp, J., “Sustaining iterative game playing processes in DGBL: The relationship between motivational processing and outcome processing,” Comput. Edu. 55(2), 789–797 (2010).CrossRefGoogle Scholar
de Jesús Luis, J., Ibánez, G. and Wang, A. I., “Learning recycling from playing a kinect game,” Int. J. Game-Based Learn. (IJGBL) 5(3), 25–44 (2015).Google Scholar
Jones, M., Defever, E., Letsinger, A., Steele, J. and Mackintosh, K. A., “A mixed-studies systematic review and meta-analysis of school-based interventions to promote physical activity and/or reduce sedentary time in children,” J. Sport Health Sci. 9(1), 3–17 (2020).Google ScholarPubMed
Joronen, K., Aikasalo, A. and Suvitie, A., “Nonphysical effects of exergames on child and adolescent well-being: A comprehensive systematic review,” Scand. J. Caring Sci. 31(3), 449–461 (2017).Google ScholarPubMed
Kamijo, K., Pontifex, M. B., O’Leary, K. C., Scudder, M. R., Wu, C.-T., Castelli, D. M. and Hillman, C. H., “The effects of an afterschool physical activity program on working memory in preadolescent children,” Dev. Sci. 14(5), 1046–1058 (2011).CrossRefGoogle ScholarPubMed
Kebritchi, M., Kebritchi, M. and Hirumi, A., “Examining the pedagogical foundations of modern educational computer games,” Comput. Edu. 51(4), 1729–1743 (2008).CrossRefGoogle Scholar
Kim, B., Park, H. and Baek, Y., “Not just fun, but serious strategies: Using meta-cognitive strategies in game-based learning,” Comput. Edu. 52(4), 800–810 (2009).CrossRefGoogle Scholar
Kourakli, M., Altanis, I., Retalis, S., Boloudakis, M., Zbainos, D. and Antonopoulou, K., “Towards the improvement of the cognitive, motoric and academic skills of students with special educational needs using kinect learning games,” Int. J. Child-Comput. Interact. 11, 28–39 (2017).CrossRefGoogle Scholar
Laffey, J. M., Espinosa, L., Moore, J. and Lodree, A., “Supporting learning and behavior of at-risk young children: Computers in urban education,” J. Res. Technol. Edu. 35(4), 423–440 (2003).CrossRefGoogle Scholar
Lopez-Morteo, G. and López, G., “Computer support for learning mathematics: A learning environment based on recreational learning objects,” Comput. Edu. 48(4), 618–641 (2007).CrossRefGoogle Scholar
Ma, Y., Veldhuis, A., Bekker, T., Hu, J. and Vos, S., “A Review of Design Interventions for Promoting Adolescents’ Physical Activity,” Proceedings of the 18th ACM International Conference on Interaction Design and Children, IDC’19, New York, NY, USA (Association for Computing Machinery, 2019) pp. 161–172.CrossRefGoogle Scholar
Manero, B., Torrente, J., Serrano, Á., Martnez-Ortiz, I. and Fernández-Manjón, B., “Can educational video games increase high school students’ interest in theatre?,” Comput. Edu. 87, 182–191 (2015).CrossRefGoogle Scholar
Matallaoui, A., Koivisto, J., Hamari, J. and Zarnekow, R., “How Effective is “Exergamification”? a Systematic Review on the Effectiveness of Gamification Features in Exergames,” Proceedings of the 50th Hawaii International Conference on System Sciences (2017).CrossRefGoogle Scholar
McFarlane, A., Sparrowhawk, A. and Heald, Y., Report on the educational use of games. Teachers evaluating educational multimedia (2002).Google Scholar
Monti, J. M., Hillman, C. H. and Cohen, N. J., “Aerobic fitness enhances relational memory in preadolescent children: The fitkids randomized control trial,” Hippocampus 22(9), 1876–1882 (2012).CrossRefGoogle ScholarPubMed
Müller, L., Bernin, A., Ghose, S., Gozdzielewski, W., Wang, Q., Grecos, C., von Luck, K. and Vogt, F., Physiological Data Analysis for an Emotional Provoking Exergame,” 2016 IEEE Symposium Series on Computational Intelligence (SSCI) (IEEE, 2016) pp. 1–8.Google Scholar
Papastergiou, M., “Digital game-based learning in high school computer science education: Impact on educational effectiveness and student motivation,” Comput. Edu. 52(1), 1–12 (2009).CrossRefGoogle Scholar
Papastergiou, M., “Exploring the potential of computer and video games for health and physical education: A literature review,” Comput. Edu. 53(3), 603–622 (2009).CrossRefGoogle Scholar
Rauf, M. F. B. A., Ismail, S., Marjudi, S., Amran, M. F. M., Majid, N. A. A., Adnan, Z. and Yusop, N. M. M., “Data capture of exergames using kinect sensor for gameplay analysis,” Int. J. Eng. Technol. 7(4.29), 124–127 (2018).Google Scholar
Retalis, S., Korpa, T., Skaloumpakas, C., Boloudakis, M., Kourakli, M., Altanis, I., Siameri, F., Papadopoulou, P., Lytra, F. and Pervanidou, P., “Empowering Children with Adhd Learning Disabilities with the Kinems Kinect Learning Games,” European Conference on Games Based Learning, vol. 2 (Academic Conferences International Limited, 2014) p. 469.Google Scholar
Rock, I. and Gutman, D., “The effect of inattention on form perception,” J. Exp. Psychol. Hum. Percept. Perform. 7(2), 275 (1981).CrossRefGoogle ScholarPubMed
Roelfsema, P. R., van Ooyen, A. and Watanabe, T., “Perceptual learning rules based on reinforcers and attention,” Trends Cognit. Sci. 14(2), 64–71 (2010).CrossRefGoogle ScholarPubMed
Sarmiento, D., Daz, Y. and Ferro, R., “Using Games to Improve Learning Skills in Students with Cognitive Disabilities Through Kinect Technology,” International Workshop on Learning Technology for Education in Cloud (Springer, 2016) pp. 51–60.Google Scholar
Schmidt, R. A. and Lee, T. D., Motor Control and Learning: A Behavioral Emphasis (Human Kinetics, 1999).Google Scholar
Stadler, M. A., “Role of attention in implicit learning,” J. Exp. Psychol. Learn. Mem. Cognit. 21(3), 674 (1995).CrossRefGoogle Scholar
Torres, A. C. S., “Cognitive effects of video games on old people,” Int. J. Disability Hum. Dev. 10(1), 55–58 (2011).CrossRefGoogle Scholar
Tsai, C.-H., Kuo, Y.-H., Chu, K.-C. and Yen, J.-C., “Development and evaluation of game-based learning system using the microsoft kinect sensor,” Int. J. Distrib. Sens. Networks 11(7), 498560 (2015).CrossRefGoogle Scholar
Tüzün, H., Yılmaz-Soylu, M., Karakuş, T., İnal, Y. and Kzlkaya, G., “The effects of computer games on primary school students’ achievement and motivation in geography learning,” Comput. Edu. 52(1), 68–77 (2009).CrossRefGoogle Scholar
Uluer, P., Kose, H., Oz, B. K., Can Aydinalev, T. and Barkana, D. E., “Towards an Affective Robot Companion for Audiology Rehabilitation: How Does Pepper Feel Today?,” 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN) (2020) pp. 567–572.Google Scholar
Virvou, M., Katsionis, G. and Manos, K., “Combining software games with education: Evaluation of its educational effectiveness,” J. Edu. Technol. Soc. 8(2) (2005).Google Scholar
Yang, Y.-T. C., “Building virtual cities, inspiring intelligent citizens: Digital games for developing students’ problem solving and learning motivation,” Comput. Edu. 59(2), 365–377 (2012).CrossRefGoogle Scholar
Zelinski, E. M. and Reyes, R., “Cognitive benefits of computer games for older adults,” Gerontechnol. Int. J. Fund. Aspects Technol. Serve Ageing Soc. 8(4), 220 (2009).Google ScholarPubMed
Zeng, N., Gao, X., Liu, Y., Lee, J. and Gao, Z., “Reliability of using motion sensors to measure children’s physical activity levels in exergaming,” J. Clin. Med. 7(5), 100 (2018).CrossRefGoogle ScholarPubMed