Daniel Gaspar-Figueiredo
Emilio Insfran
Jean Vanderdonckt
ABSTRACT
Background: Adaptive user interfaces have the advantage of being able to dynamically change their aspect and/or behaviour depending on the characteristics of the context of use, i.e. to improve user experience. User experience is an important quality factor that has been primarily evaluated with classical measures (e.g. effectiveness, efficiency, satisfaction), but to a lesser extent with physiological measures, such as emotion recognition, skin response, or brain activity. Aim: In a previous exploratory experiment involving users with different profiles and a wide range of ages, we analysed user experience in terms of cognitive load, engagement, attraction and memorisation when employing twenty graphical adaptive menus through the use of an Electroencephalogram (EEG) device. The results indicated that there were statistically significant differences for these four variables. However, we considered that it was necessary to confirm or reject these findings using a more homogeneous group of users. Method: We conducted an operational internal replication study with 40 participants. We also investigated the potential correlation between EEG signals and the participants’ user experience ratings, such as their preferences. Results: The results of this experiment confirm that there are statistically significant differences between the EEG variables when the participants interact with the different adaptive menus. Moreover, there is a high correlation among the participants’ user experience ratings and the EEG signals, and a trend regarding performance has emerged from our analysis. Conclusions: These findings suggest that EEG signals could be used to evaluate user experience. With regard to the menus studied, our results suggest that graphical menus with different structures and font types produce more differences in users’ brain responses, while menus which use colours produce more similarities in users’ brain responses. Several insights with which to improve users’ experience of graphical adaptive menus are outlined.
- John J.B. Allen, James Arthur Coan, and Maria Nazarian. 2004. Issues and assumptions on the road from raw signals to metrics of frontal EEG asymmetry in emotion. Biological Psychology 67, 1 (2004), 183–218.Google Scholar
Cross Ref
- Pavlo Antonenko, Fred Paas, Roland Grabner, and Tamara Van Gog. 2010. Using electroencephalography to measure cognitive load. Educational psychology review 22, 4 (2010), 425–438. https://doi.org/10.1007/s10648-010-9130-yGoogle Scholar
- Ainhoa Apraiz Iriarte, Ganix Lasa, and Maitane Mazmela. 2021. Evaluating User Experience with physiological monitoring: A Systematic Literature Review. Dyna (Bilbao) 8, 21. https://doi.org/10.6036/NT10072Google Scholar
- Jennifer Romano Bergstrom, Sabrina Duda, David Hawkins, and Mike McGill. 2014. Physiological response measurements. In Eye tracking in user experience design. Elsevier, 81–108.Google Scholar
- Jacques Bertin. 1967. Sémiologie graphique, Paris, Mouton/Gauthier-Villard. Réédition (2005) EHESS (1967).Google Scholar
- Bitbrain. 2020. SennsMetrics: Analysis Software of Biometrics. https://www.bitbrain.com/neurotechnology-products/software/sennsmetrics. accessed: 2020-04.Google Scholar
- N. Cliff. 1993. Dominance statistics: ordinal analyses to answer ordinal questions. Psychological Bulletin 144 (1993), 494–509. https://doi.org/10.1037/0033-2909.114.3.494Google ScholarCross Ref
- Electrode Position Nomenclature Committee. 1994. Guideline thirteen: guidelines for standard electrode position nomenclature. J. Clin. Neurophysiol. 11, 111–113.Google ScholarCross Ref
- Sarah Fakhoury, Yuzhan Ma, Venera Arnaoudova, and Olusola Adesope. 2018. The effect of poor source code lexicon and readability on developers’ cognitive load. In Proc. Int. Conf. Program Comprehension (ICPC). IEEE, 286–28610.Google ScholarDigital Library
- Robert Feldt, Richard Torkar, Lefteris Angelis, and Maria Samuelsson. 2008. Towards individualized software engineering: empirical studies should collect psychometrics. In Proceedings of the 2008 international workshop on Cooperative and human aspects of software engineering. 49–52.Google ScholarDigital Library
- Leah Findlater and Krzysztof Z Gajos. 2009. Design space and evaluation challenges of adaptive graphical user interfaces. AI Magazine 30, 4 (2009), 68–68.Google ScholarDigital Library
- International Organization for Standardization. 2010. Ergonomics of Human-system Interaction: Part 210: Human-centred Design for Interactive Systems. ISO.Google Scholar
- Krzysztof Z Gajos, Mary Czerwinski, Desney S Tan, and Daniel S Weld. 2006. Exploring the design space for adaptive graphical user interfaces. In Proceedings of the working conference on Advanced visual interfaces. 201–208.Google ScholarDigital Library
- Daniel Gaspar-Figueiredo, Jean Vanderdonckt, Silvia Abrahão, and Emilio Insfran. 2023. User Experience with Adaptive User Interfaces: Comparing User Performance and Preferences. ACM Trans. Softw. Eng. Methodol. (submitted on April 2023).Google Scholar
- Omar S. Gómez, Natalia Juristo, and Sira Vegas. 2014. Understanding replication of experiments in software engineering: A classification. Information and Software Technology 56, 8, 1033–1048. https://doi.org/10.1016/j.infsof.2014.04.004Google ScholarCross Ref
- Eddie Harmon-Jones, Philip A. Gable, and Carly K. Peterson. 2010. The role of asymmetric frontal cortical activity in emotion-related phenomena: A review and update. Biological Psychology 84, 3 (2010), 451–462.Google ScholarCross Ref
- Xiyuan Hou, Fitri Trapsilawati, Yisi Liu, Olga Sourina, Chun-Hsien Chen, Wolfgang Mueller-Wittig, and Wei Tech Ang. 2017. EEG-based human factors evaluation of conflict resolution aid and tactile user interface in future air traffic control systems. In Advances in Human Aspects of Transportation. Springer, 885–897.Google Scholar
- Maxwell K.D.2002. Applied Statistics for Software Managers. Applied Statistics for Software Managers (2002).Google Scholar
- Barbara Kitchenham, Lech Madeyski, David Budgen, Jacky Keung, Pearl Brereton, Stuart Charters, Shirley Gibbs, and Amnart Pohthong. 2017. Robust statistical methods for empirical software engineering. Empir. Software Eng. 22, 2, 579–630.Google ScholarDigital Library
- Wolfgang Klimesch. 1999. EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis. Brain Research Reviews 29, 2 (1999), 169–195. https://doi.org/10.1016/S0165-0173(98)00056-3Google ScholarCross Ref
- Sari Kujala, Virpi Roto, Kaisa Väänänen-Vainio-Mattila, Evangelos Karapanos, and Arto Sinnelä. 2011. UX Curve: A method for evaluating long-term user experience. Interacting with computers 23, 5 (2011), 473–483.Google ScholarDigital Library
- Haeinn Lee, Jungtae Lee, and Ssanghee Seo. 2009. Brain Response to Good and Bad Design. In Human-Computer Interaction. New Trends, Julie A. Jacko (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 111–120.Google Scholar
- Nicole M. Long, John F. Burke, and Michael J. Kahana. 2014. Subsequent memory effect in intracranial and scalp EEG. NeuroImage 84 (2014), 488–494.Google ScholarCross Ref
- Peter J. Mikulka, Mark W. Scerbo, and Frederick G. Freeman. 2002. Effects of a Biocybernetic System on Vigilance Performance. Human Factors 44, 4, 654–664.Google ScholarCross Ref
- Jefferson Seide Molléri, Indira Nurdiani, Farnaz Fotrousi, and Kai Petersen. 2019. Experiences of studying Attention through EEG in the Context of Review Tasks. In Proceedings of the Evaluation and Assessment on Software Engineering. 313–318.Google ScholarDigital Library
- Meinard Müller. 2007. Dynamic Time Warping. In Information Retrieval for Music and Motion. Springer Berlin Heidelberg, 69–84.Google Scholar
- Jakob Nielsen and Jonathan Levy. 1994. Measuring Usability: Preference vs. Performance. Commun. ACM 37, 4 (apr 1994), 66–75.Google ScholarDigital Library
- Norman Peitek, Janet Siegmund, Chris Parnin, Sven Apel, Johannes C. Hofmeister, and André Brechmann. 2018. Simultaneous Measurement of Program Comprehension with FMRI and Eye Tracking: A Case Study. In Proc. Int. Symp.Empirical Softw. Eng. Meas.Association for Computing Machinery, Article 24.Google ScholarDigital Library
- Peter Schmutz, Silvia Heinz, Yolanda Métrailler, and Klaus Opwis. 2009. Cognitive load in eCommerce applications—measurement and effects on user satisfaction. Advances in Human-Computer Interaction (2009).Google Scholar
- Janet Siegmund, Christian Kästner, Sven Apel, Chris Parnin, Anja Bethmann, Thomas Leich, Gunter Saake, and André Brechmann. 2014. Understanding understanding source code with functional magnetic resonance imaging. In Proceedings of the 36th international conference on software engineering. 378–389.Google ScholarDigital Library
- Alexandre N. Tuch, Paul Van Schaik, and Kasper Hornbæk. 2016. Leisure and Work, Good and Bad: The Role of Activity Domain and Valence in Modeling User Experience. ACM Trans. Comput.-Hum. Interact. 23, 6, Article 35 (dec 2016).Google ScholarDigital Library
- Jean Vanderdonckt, Sara Bouzit, Gaëlle Calvary, and Denis Chêne. 2019. Exploring a design space of graphical adaptive menus: normal vs. small screens. ACM Transactions on Interactive Intelligent Systems (TiiS) 10, 1 (2019), 1–40.Google Scholar
- Barbara Weber, Thomas Fischer, and René Riedl. 2021. Brain and autonomic nervous system activity measurement in software engineering: A systematic literature review. Journal of Systems and Software 178 (2021), 110946.Google ScholarCross Ref
- Tarannum Zaki and Muhammad Nazrul Islam. 2021. Neurological and physiological measures to evaluate the usability and user-experience (UX) of information systems: A systematic literature review. Computer Science Review 40, 100–375.Google ScholarDigital Library
Index Terms
- Measuring User Experience of Adaptive User Interfaces using EEG: A Replication Study
Recommendations
User experience evaluation through the brain's electrical activity
NordiCHI '14: Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, FoundationalA novel system for measuring the user experience of any user interface by measuring the feedback directly from the brain through Electroencephalography (EEG) is described. We developed an application that records data for different emotions of the user ...
Understanding the Formation of User's First Impression on an Interface Design from a Neurophysiological Perspective - EEG Pilot Study
HCIK '16: Proceedings of HCI KoreaAlthough traditional UX evaluation methods have focused on the experience users undergo when they actually use a service or product, the first impression that users obtain before actually using it is also an important factor to take into consideration. ...
From usability tasks to usable user interfaces
TAMODIA '05: Proceedings of the 4th international workshop on Task models and diagramsIn this paper we describe how the identification of usability tasks in the task model as an early consideration of usability in the process can directly influence the design of usable User Interfaces (UI). We intend to make system analysts and UI ...
Comments