Skip to main content
SpringerLink
Log in

Entertainment capture through heart rate activity in physical interactive playgrounds

  • Original Paper
  • Published:
User Modeling and User-Adapted Interaction Aims and scope Submit manuscript

Abstract

An approach for capturing and modeling individual entertainment (“fun”) preferences is applied to users of the innovative Playware playground, an interactive physical playground inspired by computer games, in this study. The goal is to construct, using representative statistics computed from children’s physiological signals, an estimator of the degree to which games provided by the playground engage the players. For this purpose children’s heart rate (HR) signals, and their expressed preferences of how much “fun” particular game variants are, are obtained from experiments using games implemented on the Playware playground. A comprehensive statistical analysis shows that children’s reported entertainment preferences correlate well with specific features of the HR signal. Neuro-evolution techniques combined with feature set selection methods permit the construction of user models that predict reported entertainment preferences given HR features. These models are expressed as artificial neural networks and are demonstrated and evaluated on two Playware games and two control tasks requiring physical activity. The best network is able to correctly match expressed preferences in 64% of cases on previously unseen data (p−value 6 · 10−5). The generality of the methodology, its limitations, its usability as a real-time feedback mechanism for entertainment augmentation and as a validation tool are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Andrade, G., Ramalho, G., Santana, H., Corruble, V.: Extending reinforcement learning to provide dynamic game balancing. In: Proceedings of the Workshop on Reasoning, Representation, and Learning in Computer Games, 19th International Joint Conference on Artificial Intelligence (IJCAI), pp. 7–12 (2005)

  • Aristotle: On the Soul, Book III. Kessinger Publishing (2004)

  • Bobick, A., Intille, S., Davis, J., Baird, F., Pinhanez, C., Campbell, L., Ivanov, Y., Schutte, A., Wilson, A.: The kidsroom: A perceptually-based interactive and immersive story environment. Technical Report 398, MIT Media Laboratory (1996)

  • Champandard, A.J.: AI Game Development. New Riders Publishing (2004)

  • Conati C. (2002). Probabilistic assessment of user’s emotions in educational games. J. Appl. Artif. Intell. special issue on “Merging Cognition and Affect in HCI” 16: 555–575

    Google Scholar 

  • Conati, C., Chabbal, R., Maclaren, H.: A study on using biometric sensors for detecting user emotions in educational games. In: Proceedings of the Workshop “Assessing and Adapting to User Attitude and Affects: Why, When and How?”. In conjunction with UM’03, 9th International Conference on User Modeling, Pittsburgh, PA, U.S.A., pp. 60–65 (2003)

  • Critchley H.D., Rotshtein P., Nagal Y., O’Doherty J., Mathias C.J. and Dolan R.J. (2005). Activity in the human brain predicting differential heart rate responses to emotional facial expressions. Neuroimage 24: 751–762

    Article  Google Scholar 

  • Csikszentmihalyi M. (1990). Flow: The Psychology of Optimal Experience. Harper & Row, New York

    Google Scholar 

  • Devijver P. and Kittler J. (1982). Pattern Recognition—A Statistical Approach. Prentice-Hall, Engelwood Cliffs, NJ

    MATH  Google Scholar 

  • D’Mello, S., Craig, S.D., Whitherspoon, A., McDaniel, B., Graesser, A.: Automatic detection of learner’s affect from conversational cues. User Model. User-Adap. Inter., Special Issue: User Model. Affect. Comput. (2008) (to appear)

  • Funge, J.D.: Artificial Intelligence for Computer Games. A. K. Peters Ltd, Wellesley, Massachusetts, USA

  • Haapalainen, E., Laurinen, P., Junno, P., Tuovinen, H., Roening, J.: Methods for classifying spot welding process: A comparative study of performance. In: IEA/AIE, pp. 412–421 (2005)

  • Hazlett, R.L.: Measuring emotional valence during interactive experiences: boys at video game play. In: CHI ’06: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1023–1026. ACM Press, New York, NY, USA (2006)

  • Holland J.H. (1975). Adaptation in Natural and Artificial Systems. University of Michigan Press, Ann Arbor, MI

    Google Scholar 

  • Hughes M. and Hutt C. (1979). Heart-rate correlates of childhood activities: play, exploration, problem-solving and day-dreaming. Am. Psychol. 8: 253–263

    Google Scholar 

  • Hunicke, R., Chapman, V.: AI for dynamic difficulty adjustment in games. In: Proceedings of the Challenges in Game AI Workshop, 19th Nineteenth National Conference on Artificial Intelligence (AAAI’04), AAAI Press (2004)

  • Hutt, C. In: Exploration and Play, in Play and Learning, pp. 175–194. Gardner Press, New York (1979)

  • Iida, H., Takeshita, N., Yoshimura, J.: A metric for entertainment of boardgames: its implication for evolution of chess variants. In: Nakatsu R., Hoshino J. (eds.) IWEC2002 Proceedings, pp. 65–72. Kluwer (2003)

  • Ishii, H., Wisneski, C., Orbanes, J., Chun, B., Paradiso, J.: PingPongPlus: design of an athletic-tangible interface for computer-supported cooperative play. In: CHI ’99: Proceedings of the SIGCHI Conference on Human factors in Computing Systems, pp. 394–401. ACM Press, New York, NY, USA (1999)

  • James, W. In: William James: Wrtitings 1878–1899. The Library of America, pp. 350–365 (1992). Originally published in 1890

  • Kapoor, A., Mota, S., Picard, R.: Towards a learning companion that recognizes affect. In: Proceedings of Emotional and Intelligent II: The Tangled Knot of Social Cognition, AAAI Fall Symposium, AAAI Press (2001)

  • Kline, S.: Out of the Garden: Toys and Children’s Culture in the Age of Marketing. Verso (1993)

  • Konami Digital Entertainment B.V.: (DDR and Kick & Kick) http://www.konami.com

  • Koster, R.: A Theory of Fun for Game Design. Paraglyph Press (2005)

  • Laird, J.E., van Lent, M.: Human-level AI’s killer application: Interactive computer games. In: Proceedings of the Seventh National Conference on Artificial Intelligence (AAAI), pp. 1171–1178, AAAI Press (2000)

  • Lazzaro, N.: Why we play games: Four keys to more emotion without story. Technical Report, XEO Design Inc. (2004)

  • Liljedahl, M., Lindberg, S.: DigiWall—an audio mostly game. In: International Conference on Auditory Display, London (2006)

  • Lindstrom, M., Seybold, P.: BRANDchild: Insights into the Minds of Today’s Global Kids: Understanding Their Relationship with Brands. Kogan Page (1994)

  • Lund, H.H., Jessen, C.: Playware—intelligent technology for children’s play. Technical Report TR-2005-1, Maersk Institute For Production Technology, University of Southern Denmark (2005)

  • Lund H.H., Klitbo T. and Jessen C. (2005). Playware technology for physically activating play. Artif. Life Robot. J. 9: 165–174

    Article  Google Scholar 

  • Magerkurth, C., Stenzel, R., Prante, T.: STARS – a ubiquitous computing platform for computer augmented tabletop games. In: Extended Abstracts of UbiComp ’03, pp. 267–268. Springer (2003)

  • Malone T.W. (1981). What makes computer games fun?. Byte 6: 258–277

    Google Scholar 

  • Mandryk, R.L., Atkins, M.S., Inkpen, K.M.: A continuous and objective evaluation of emotional experience with interactive play environments. In: Proceedings of the Conference on Human Factors in Computing Systems (CHI 2006), pp. 1027–1036, ACM Press (2006a)

  • Mandryk, R.L., Inkpen, K.M., Calvert, T.W.: Using psychophysiological techniques to measure user experience with entertainment technologies. Behav. Inform. Technol. (Special Issue on User Experience) 25, 141–158 (2006b)

  • McQuiggan, S., Lee, S., Lester, J.: Predicting user physiological response for interactive environments: an inductive approach. In: Proceedings of the 2nd Artificial Intelligence for Interactive Digital Entertainment Conference, pp. 60–65, AAAI Press (2006)

  • Metaxas, G., Metin, B., Schneider, J., Shapiro, G., Zhou, W., Markopoulos, P.: SCORPIODROME: An Exploration in Mixed Reality Social Gaming for Children. In: Proceedings of ACM SIGCHI International Conference on Advances in Computer Entertainment Technology, ACM Press (2005)

  • Mixed Reality Lab: (Singapore) http://www.mixedreality.nus.edu.sg/

  • Montana, D.J., Davis, L.D.: Training feedforward neural networks using genetic algorithms. In: Proceedings of the Eleventh International Joint Conference on Artificial Intelligence (IJCAI-89), pp. 762–767, Morgan Kauffman, San Mateo, CA (1989)

  • Mueller, F., Agamanolis, S., Picard, R.: Exertion interfaces: sports over a distance for social bonding and fun. In: CHI ’03: Proceedings of the SIGCHI conference on Human Factors in Computing Systems, pp. 561–568. ACM Press, NY, USA (2003)

  • Nareyek, A.: Intelligent agents for computer games. In: Marsland, T., Frank, I. (eds.) Computers and Games, Second International Conference, CG 2002, pp. 414–422, LNCS, Springer (2002)

  • Orr, R.J., Abowd, G.D.: The smart floor: a mechanism for natural user identification and tracking. In: CHI ’00: CHI ’00 Extended Abstracts on Human Factors in Computing Systems, pp. 275—276. ACM Press, NY, USA (2000)

  • Picard R.W., Vyzas E. and Healey J. (2001). Toward machine emotional intelligence: analysis of affective physiological state. IEEE Trans. Pattern Anal. Mach. Intell. 23: 1175–1191

    Article  Google Scholar 

  • Pincus S.M. and Goldberger A.L. (1994). Physiological time-series analysis: what does regularity quantify?. Am. J. Physiol. 266: H1643–56

    Google Scholar 

  • Pincus S.M. (1991). Approximate entropy as a measure of system complexity. Proc. Natl. Acad. Sci. 88: 2297–2301

    Article  MATH  MathSciNet  Google Scholar 

  • Porayska-Pomsta, K., Mavrikis, M., Pain, H.: Towards predictive modelling of tutorial interactions: student affect from tutor perspective. User Model. User-Adap. Inter., Special Issue: User Model. Affect. Comput. (2008) (to appear)

  • Postman N. (1983). The Disappearance of Childhood. Allen, London

    Google Scholar 

  • Rani, P., Sarkar, N., Liu, C.: Maintaining optimal challenge in computer games through real-time physiological feedback. In: Proceedings of the 11th International Conference on Human Computer Interaction, Lawrence Erlbaum (2005)

  • Ravaja N., Saari T., Turpeinen M., Laarni J., Salminen M. and Kivikangas M. (2006). Spatial presence and emotions during video game playing: does it matter with whom you play?. Presence Teleoperators & Virtual Environments 15: 381–392

    Article  Google Scholar 

  • Read, J., MacFarlane, S., Cassey, C.: Endurability, engagement and expectations. In: Proceedings of International Conference for Interaction Design and Children (2002)

  • Richardson, B., Leydon, K., Fernström, M., Paradiso, J.A.: Z-Tiles: Building Blocks for Modular, Pressure-Sensing Floorspaces. In: Proceedings of CHI 2004, pp. 1529–1532. ACM Press, NY, USA (2004)

  • Rosenthal R. (2003). Covert communication in laboratories, classrooms and the truly real world. Curr. Direct. Psychol. Sci. 12: 151–154

    Article  Google Scholar 

  • Spronck, P., Sprinkhuizen-Kuyper, I., Postma, E.: Difficulty Scaling of Game AI. In: Proceedings of the 5th International Conference on Intelligent Games and Simulation (GAME-ON 2004), pp. 33–37 (2004)

  • Sweetser P., Wyeth P.: GameFlow: a model for evaluating player enjoyment in games. ACM Comput. Entertain. 3, (2005)

  • Taatgen, N.A., van Oploo, M., Braaksma, J., Niemantsverdriet, J.: How to construct a believable opponent using cognitive modeling in the game of set. In: Proceedings of the Fifth International Conference on Cognitive Modeling, pp. 201–206 (2003)

  • Turkle S. (1984). The Second Self: Computers and the Human Spirit. Simon and Schuster, New York

    Google Scholar 

  • Verma, M.A., McOwan, P.W.: An adaptive methodology for synthesising Mobile Phone Games using Genetic Algorithms. In: Congress on Evolutionary Computation (CEC-05), Edinburgh, UK, pp. 528–535, IEEE (2005)

  • Wiberg, C.: A Measure of Fun. Extending the Scope of Web Usability. PhD Thesis, Department of Informatics. Umea University (2003)

  • Wiberg, C., Jegers, K.: Satisfaction and learnability in edutainment: a usability study of the knowledge game ‘Laser Challenge’ at the Nobel e-museum. In: Proceedings of HCI International—10th International Conference on Human Computer Interaction, Crete, Greece (2003)

  • Yannakakis, G.N.: AI in Computer Games: Generating Interesting Interactive Opponents by the use of Evolutionary Computation. Ph.D. Thesis, University of Edinburgh (2005)

  • Yannakakis G.N. and Hallam J. (2004). Evolving opponents for interesting interactive computer games. In: Schaal, S., Ijspeert, A., Billard, A., Vijayakumar, S., Hallam, J. and Meyer, J.A. (eds) From Animals to Animats 8: Proceedings of the 8th International Conference on Simulation of Adaptive Behavior (SAB-04), pp 499–508. The MIT Press, Santa Monica, LA, CA

    Google Scholar 

  • Yannakakis, G.N., Hallam, J.: A scheme for creating digital entertainment with substance. In: Proceedings of the Workshop on Reasoning, Representation, and Learning in Computer Games, 19th International Joint Conference on Artificial Intelligence (IJCAI), pp. 119–124 (2005)

  • Yannakakis, G.N., Hallam, J.: Towards capturing and enhancing entertainment in computer games. In: Proceedings of the 4th Hellenic Conference on Artificial Intelligence, Lecture Notes in Artificial Intelligence, vol. 3955, pp. 432–442. Springer-Verlag, Heraklion, Greece (2006)

  • Yannakakis, G.N., Hallam, J.: Towards optimizing entertainment in computer games. Appl. Artif. Intell. to appear (2007a)

  • Yannakakis, G.N., Hallam, J.: Preliminary studies for capturing entertainment through physiology in physical play. Technical Report TR-2007-5, Maersk Institue, University of Southern Denmark (2007b)

  • Yannakakis, G.N., Lund, H.H., Hallam, J.: Modeling children’s entertainment in the playware playground. In: Proceedings of the IEEE Symposium on Computational Intelligence and Games, Reno, USA, pp. 134–141. IEEE (2006a)

  • Yannakakis, G.N., Hallam, J., Lund, H.H.: Capturing entertainment through heart-rate dynamics in the playware playground. In: Proceedings of the 5th International Conference on Entertainment Computing, Lecture Notes in Computer Science, vol. 4161, pp. 314–317. Springer-Verlag, Cambridge, UK (2006b)

  • Yannakakis, G.N., Hallam, J., Lund, H.H.: Comparative Fun Analysis in the Innovative Playware Game Platform. In: Proceedings of the 1st World Conference for Fun’n Games, pp. 64–70 (2006c)

  • Yao, X.: Evolving artificial neural networks. In: Proceedings of the IEEE, vol. 87, pp. 1423–1447, IEEE (1999)

  • Zuckerman M.: Sensation seeking in entertainment. In: Phychology of Entertainment, pp. 367–387. Lawrence Erlbaum Associates Publishers (2006)

Download references

Author information

Authors and Affiliations

  1. Maersk Mc-Kinney Moller Institute, University of Southern Denmark, Campusvej 55, Odense M, 5230, Denmark

    Georgios N. Yannakakis, John Hallam & Henrik Hautop Lund

Authors
  1. Georgios N. Yannakakis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. John Hallam
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Henrik Hautop Lund
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Georgios N. Yannakakis.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yannakakis, G.N., Hallam, J. & Lund, H.H. Entertainment capture through heart rate activity in physical interactive playgrounds. User Model User-Adap Inter 18, 207–243 (2008). https://doi.org/10.1007/s11257-007-9036-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11257-007-9036-7

Keywords

Search

Navigation