Susana Sanchez
Tilman Dingler
Kai Kunze
ABSTRACT
Reading fiction is a silent activity, where readers come to know imaginary worlds and characters from the book's pages. However, we perceive the natural world with more than our eyes, and literature should be no different. Thus, an embodied reading experience is proposed, adding sound effects and haptic feedback to allow readers to listen and feel the narrative text. This paper presents a preliminary prototype for multisensory narratives and an experimental methodology to measure embodiment in literature. Results for the subjective assessment of immersion and user experience from 15 participants in three modalities: haptic, sound, both combined are discussed.
- Oliver Amft, Florian Wahl, Shoya Ishimaru, and Kai Kunze. 2015. Making regular eyeglasses smart. Pervasive Computing, IEEE 14, 3 (2015), 32-43.Google Scholar
Digital Library
- Maribeth Back, Jonathan Cohen, Rich Gold, Steve Harrison, and Scott Minneman. 2001. Listen Reader: An Electronically Augmented Paper-based Book. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '01). ACM, New York, NY, USA, 23-29. DOI:http://dx.doi.org/10. 1145/365024.365031 Google ScholarDigital Library
- P Bertelson and B de Gelder. 2003. The psychology of multimodal perception. Crossmodal space and crossmodal attention (2003).Google Scholar
- Ralf Biedert, Georg Buscher, and Andreas Dengel. 2010a. The eyebook-using eye tracking to enhance the reading experience. Informatik-Spektrum 33, 3 (2010), 272-281.Google ScholarCross Ref
- Ralf Biedert, Georg Buscher, Sven Schwarz, Manuel Moller, Andreas Dengel, and Thomas Lottermann. 2010b. The text 2.0 framework: writing web-based gaze-controlled realtime applications quickly and easily. In Proceedings of the 2010 workshop on Eye gaze in intelligent human machine interaction. ACM, 114- 117. Google ScholarDigital Library
- Mark Billinghurst, Hirkazu Kato, and Ivan Poupyrev. 2001. The MagicBook--Moving Seamlessly Between Reality and Virtuality. IEEE Comput. Graph. Appl. 21, 3 (May 2001), 6-8. DOI: http://dx.doi. org/10.1109/38.920621 Google ScholarDigital Library
- Emily Brown and Paul Cairns. 2004. A grounded investigation of game immersion. In CHI'04 extended abstracts on Human factors in computing systems. ACM, 1297-1300. Google ScholarDigital Library
- Nitasha Buldeo. 2015. Interoception: A Measure of Embodiment or Attention? International Body Psychotherapy Journal 14, 1 (2015).Google Scholar
- Oliver G Cameron. 2001. Interoception: the inside story--a model for psychosomatic processes. Psychosomatic medicine 63, 5 (2001), 697-710.Google Scholar
- Andy Clark. 2008. Supersizing the mind: Embodiment, action, and cognitive extension: Embodiment, action, and cognitive extension. Oxford University Press.Google Scholar
- Arthur D Craig. 2009. How do you feelnow? The anterior insula and human awareness. (2009).Google Scholar
- Hugo D Critchley, Stefan Wiens, Pia Rotshtein, Arne Ohman, and Raymond J Dolan. 2004. Neural systems supporting interoceptive awareness. Nature neuroscience 7, 2 (2004), 189-195.Google Scholar
- Wes Folkerth. 2014. The sound of Shakespeare. routledge.Google Scholar
- Lyn Frazier and Keith Rayner. 1982. Making and correcting errors during sentence comprehension: Eye movements in the analysis of structurally ambiguous sentences. Cognitive psychology 14, 2 (1982), 178--210.Google Scholar
- Kunze Inami Heng, Sanchez. 2015. An Augmented e-Reader for Multimodal Literacy. Demo for the 13th Annual International Conference on Mobile Systems, Applications, and Services (2015). Google ScholarDigital Library
- Beate M Herbert and Olga Pollatos. 2012. The body in the mind: on the relationship between interoception and embodiment. Topics in cognitive science 4, 4 (2012), 692-704.Google Scholar
- Shoya Ishimaru, Kai Kunze, Katsuma Tanaka, Yuji Uema, Koichi Kise, and Masahiko Inami. 2015. Smart Eyewear for Interaction and Activity Recognition. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 307-310. Google ScholarDigital Library
- Shoya Ishimaru, Yuji Uema, Kai Kunze, Koichi Kise, Katsuma Tanaka, and Masahiko Inami. 2014. Smarter eyewear: using commercial EOG glasses for activity recognition. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication. ACM, 239-242. Google ScholarDigital Library
- Charlene Jennett, Anna L Cox, Paul Cairns, Samira Dhoparee, Andrew Epps, Tim Tijs, and Alison Walton. 2008. Measuring and defining the experience of immersion in games. International journal of human-computer studies 66, 9 (2008), 641-661. Google ScholarDigital Library
- Mariana Julieta Lopez and Sandra Pauletto. 2009. The design of an audio film for the visually impaired. (2009).Google Scholar
- Mika Koivisto, Pasi Kainulainen, and Antti Revonsuo. 2009. The relationship between awareness and attention: evidence from ERP responses. Neuropsychologia 47, 13 (2009), 2891-2899.Google ScholarCross Ref
- Kai Kunze, Masai Katsutoshi, Yuji Uema, and Masahiko Inami. 2015a. How much do you read?: counting the number of words a user reads using electrooculography. In Proceedings of the 6th Augmented Human International Conference. ACM, 125-128. Google ScholarDigital Library
- Kai Kunze, Katsutoshi Masai, Masahiko Inami, Omer Sacakli, Marcus Liwicki, Andreas Dengel, Shoya Ishimaru, and Koichi Kise. 2015b. Quantifying reading habits: counting how many words you read. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 87-96. Google ScholarDigital Library
- Kai Kunze, Susana Sanchez, Tilman Dingler, Olivier Augereau, Koichi Kise, Masahiko Inami, and Terada Tsutomu. 2015c. The augmented narrative: toward estimating reader engagement. In Proceedings of the 6th Augmented Human International Conference. ACM, 163-164. Google ScholarDigital Library
- Kai Kunze, Katsuma Tanaka, Shoya Ishimaru, Yuji Uema, Koichi Kise, and Masahiko Inami. 2015d. MEME: eye wear computing to explore human behavior. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers. ACM, 361-363. Google ScholarDigital Library
- Peter H Lindsay and Donald A Norman. 2013. Human information processing: An introduction to psychology. Academic Press.Google Scholar
- Claire F Michaels and Claudia Carello. 1981. Direct perception. Prentice-Hall Englewood Cliffs, NJ.Google Scholar
- Kouta Minamizawa, Yasuaki Kakehi, Masashi Nakatani, Soichiro Mihara, and Susumu Tachi. 2012. TECHTILE toolkit: a prototyping tool for design and education of haptic media. In Proceedings of the 2012 Virtual Reality International Conference. ACM, 26. Google ScholarDigital Library
- Keith Rayner, Kathryn H Chace, Timothy J Slattery, and Jane Ashby. 2006. Eye movements as reflections of comprehension processes in reading. Scientific Studies of Reading 10, 3 (2006), 241-255.Google ScholarCross Ref
- Alfred Schutz. 1951. Making music together: A study in social relationship. Social research (1951), 76-97.Google Scholar
- Evelyn Tribble, John Sutton, and others. 2011. Cognitive ecology as a framework for Shakespearean studies. Shakespeare Studies 39, 94 (2011), 10.Google Scholar
- Hiroki Yamada. 2010. SequenceBook: Interactive Paper Book Capable of Changing the Storylines by Shufffing Pages. In CHI '10 Extended Abstracts on Human Factors in Computing Systems (CHI EA '10). ACM, New York, NY, USA, 4375-4380. DOI: http://dx.doi.org/10.1145/1753846.1754156 Google ScholarDigital Library
- Nesra Yannier, Ali Israr, Jill Fain Lehman, and Roberta L. Klatzky. 2015. FeelSleeve: Haptic Feedback to Enhance Early Reading. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 1015-1024. DOI: http://dx.doi.org/10.1145/ 2702123.2702396 Google ScholarDigital Library
Index Terms
- Embodied Reading: A Multisensory Experience.
Recommendations
Miniature Haptics: Experiencing Haptic Feedback through Hand-based and Embodied Avatars
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing SystemsWe present Miniature Haptics, a new approach to providing realistic haptic experiences by applying miniaturized haptic feedback to hand-based, embodied avatars. By shrinking haptics to a much smaller scale, Miniature Haptics enables the exploration of ...
How Parents Engage Children in Tablet-Based Reading Experiences: An Exploration of Haptic Feedback
CSCW '17: Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social ComputingWhile recent work examines parents and children reading together on tablet computers, the ways in which interactive elements within e-books affect parent-child interaction are not well understood. We examine haptic feedback as a new form of e-book ...
Embodied media: expanding human capacity via virtual reality and telexistence (keynote)
ICMI '16: Proceedings of the 18th ACM International Conference on Multimodal InteractionThe information we acquire in real life gives us a holistic experience that fully incorporates a variety of sensations and bodily motions such as seeing, hearing, speaking, touching, smelling, tasting, and moving. However, the sensory modalities that ...
Comments