Abstract
Human-Computer Interaction (HCI) and Artificial Intelligence (AI) are two disciplines that followed parallel trajectories for about four decades. They also both complement each other and overlap in various problem-rich domains. This chapter is far from being exhaustive, but provides a representative story of how HCI and AI cross-fertilise each other since their inception. It reviews the following domains: intelligent user interfaces and more specifically conversational animated affective agents; capitalisation, formulation and use of ergonomic knowledge for the design and evaluation of interactive systems; synergy between visualisation and data mining.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The term “Human-Computer Interaction”, still very much used, is shifting toward “Human-Systems Interaction” since we are developing “systems” that include both computing and physical things.
- 2.
- 3.
See also Grudin (2009).
- 4.
ACM (Association for Computing Machinery)-SIGCHI (Special Interest Group on Computer-Human Interaction).
- 5.
The reader can refer to several papers in Volume 1 for an overview of the methods in Artificial Intelligence for the representation of knowledge and the reasoning.
- 6.
An example of a BDI formalisation of emotions is proposed in chapter “Formalization of Cognitive-Agent Systems, Trust and Emotions” of Volume 1.
- 7.
Machine learning methods are presented in details in chapter “Designing Algorithms for Machine Learning and Data Mining” of Volume 2.
- 8.
The results of such algorithms are difficult to explain and to interpret, for instance System Vector Machine (SVM) or Neural Networks.
References
Abed M, Anli A, Kolski C, Grislin-Le Strugeon E (2011) A generic method for personalizing interactive systems: application to traveler information. In: Kolski C (ed) Human-computer interactions in transport. ISTE Ltd and Wiley, London, pp 51–91
Abowd GD, Dey AK, Brown PJ, Davies N, Smith M, Steggles P (1999) Towards a better understanding of context and context-awareness. In: Proceedings of the 1st international symposium on Handheld and Ubiquitous Computing, HUC ’99, pp 304–307
Akiki PA, Bandara AK, Yu Y (2014) Adaptive model-driven user interface development systems. ACM Comput Surv 47(1):9:1–9:33
Amar R, Eagan J, Stasko J (2005) Low-level components of analytic activity in information visualization. In: IEEE symposium on information visualization. IEEE Computer Society, pp 111–117
Andrienko N, Andrienko G (2006) Exploratory analysis of spatial and temporal data—a systematic approach. Springer, Berlin
Asimov I (2008) I robot. Random House Publishing Group
Baccino T, Bellino C, Colombi T (2005) Mesure de l’utilisabilité des interfaces. TIC et Sciences Cognitive, Hermes
Bastien JMC, Scapin DL (1993) Ergonomic criteria for the evaluation of human-computer interfaces. Technical report 156, INRIA, Rocquencourt
Bastien JMC, Scapin DL (2001) Evaluation des systèmes d’information et critères ergonomiques. In: Kolski C (ed) Environnement évolués et évaluation de l’IHM, Interaction Homme-Machine pour les SI, vol 2. Hermes, Paris, pp 53–80
Bauer C, Dey AK (2016) Considering context in the design of intelligent systems: current practices and suggestions for improvement. J Syst Softw 112:26–47
Bauer JS, Newman MW, Kientz JA (2014) What designers talk about when they talk about context. Hum Comput Interact 29(5–6):420–450
Beale R, Creed C (2009) Affective interaction: how emotional agents affect users. Int J Hum Comput Stud 67(9):755–776
Beaudouin-Lafon M, Mackay W (2003) Prototyping tools and techniques. In: Jacko JA, Sears A (eds) The human computer handbook: fundamentals, evolving technologies, and emerging applications. L. Erlbaum Associates Inc, Hillsdale, pp 1006–1031
Beka Be Nguema M, Kolski C, Malvache N, Waroux D (2000) Design of a human-error-tolerant interface using fuzzy logic. Eng Appl Artif Intell 13(3):279–292
Bellot P (2011) Recherche d’information contextuelle, assistée et personnalisée. Hermes Science Publications/Lavoisier, Collection recherche d’information et web
Berners-Lee T, Hendler J, Lassila O (2001) The semantic web. Sci Am Mag 284:34–43
Bertin J (1998) Sémiologie Graphique: Les Diagrammes, Les Réseaux, Les Cartes. Les ré-impressions, Ecole des Hautes Etudes en Sciences Sociales
Boy G (1991a) Intelligent assistant systems. Academic Press, New York
Boy GA (1991b) Indexing hypertext documents in context. In: Proceedings of hypertext ’91. ACM Press, New York, pp 51–61
Boy GA (1992) Semantic correlation in context: application in document comparison and group knowledge design. In: Boose JH, Clancey W, Gaines B, Rappaport A (eds) AAAI spring symposium on cognitive aspects of knowledge acquisition. Stanford University, CA, USA
Boy GA (1998) Cognitive function analysis. Ablex/Greenwood, Westport
Boy GA (2013) Orchestrating human-centered design. Springer, U.K
Brehmer M, Munzner T (2013) A multi-level typology of abstract visualization tasks. IEEE Trans Vis Comput Graph 19(12):2376–2385
Brossard A, Abed M, Kolski C (2011) Taking context into account in conceptual models using a model driven engineering approach. Inf Softw Technol 53(12):1349–1369
Brusilovsky P, Kobsa A, Nejdl W (eds) (2007) The adaptive web: methods and strategies of web personalization. Springer, Heidelberg
Calvary G, Coutaz J, Thevenin D, Limbourg Q, Bouillon L, Vanderdonckt J (2003) A unifying reference framework for multi-target user interfaces. Interact Comput 15(3):289–308
Calvary G, Coutaz J, Dâassi O, Balme L, Demeure A (2004) Towards a new generation of widgets for supporting software plasticity: The “comet”. In: Engineering human computer interaction and interactive systems, joint working conferences EHCI-DSVIS 2004, Hamburg, Germany, 11–13 July 2004, Revised selected papers, pp 306–324
Calvary G, Pribeanu C, Santucci G, Vanderdonckt J (eds) (2008) Computer-aided design of user interfaces V—sixth international conference of computer-aided design of user interfaces 2006. Springer, Bucharest
Canamero L (2003) Designing emotions for activity selection in autonomous agents. In: Petta P, Payr S, Trappl R (eds) Emotions in humans and artifacts. MIT Press, Cambridge, pp 115–148
Caplier A (2011) Visual emotion recognition: status and key issues. In: Pelachaud C (ed) Emotion-oriented systems. Wiley, New York
Card S, Mackinlay J, Shneiderman B (1999) Readings in information visualization. Morgan Kaufmann Publishers, San Francisco
Card SK, Moran TP, Newell A (1983) The psychology of human-computer interaction. Lawrence Erlbaum Associates, Hillsdale
Cassell J (2000) More than just another pretty face: embodied conversational interface agents. Commun ACM 43:70–78
Chi EH (2000) A taxonomy of visualization techniques using the data state reference model. In: IEEE symposium on information vizualization. IEEE Computer Society, pp 69–75
Chignell MH, Hancock PA (1988) Intelligent interface design. In: Helander M (ed) Handbook of human-computer interaction. Elsevier, Amsterdam, pp 969–995
Chollet M, Ochs M, Pelachaud C (2017) A methodology for the automatic extraction and generation of non-verbal signals sequences conveying interpersonal attitudes. IEEE Trans Affect Comput
Clavel C, Richard G (2011) Recognition of acoustic emotion. In: Pelachaud C (ed) Emotion-oriented systems. Wiley, Hoboken
Coutaz J, Calvary G (2012) HCI and software engineering for user interface plasticity. CRC Press, Taylor and Francis Group, Boca Raton, pp 1195–1220
Coutaz J, Rey G (2002) Foundations for a theory of contextors. In: Kolski C, Vanderdonckt J (eds) Computer-aided design of user interfaces III In: Fourth international conference of computer-aided design of user interfaces, vol 3. Kluwer Academic, Dordrecht, pp 13–34
Daassi C, Nigay L, Fauvet MC (2006) A taxonomy of temporal data visualization techniques. Rev Inf Interact Intell 5(2):41–63
deRosis F, Pelachaud C, Poggi I, Carofiglio V, Carolis BD (2003) From Greta’s mind to her face: modelling the dynamics of affective states in a conversational embodied agent. Int J Hum-Comput Stud 59(1–2):81–118
Dey AK (2001) Understanding and using context. Pers Ubiquitous Comput 5(1):4–7
dos Santos S, Brodlie K (2004) Gaining understanding of multivariate and multidimensional data through visualization. Comput Graph 28(3):311–325
Edmonds EA (1981) Adaptive man-computer interfaces. In: Coombs MJ, Alty JL (eds) Computing skills and the user interface. Academic, London, pp 389–426
Ezzedine H, Trabelsi A, Tran C, Kolski C (2012) Criteria and methods for interactive system evaluation: application to a regulation post in the transport domain. In: Hammadi S, Ksouri M (eds) Advanced mobility and transport engineering. ISTE Wiley, London, pp 183–230
Findlater L, Gajos KZ (2009) Design space and evaluation challenges of adaptive graphical user interfaces. AI Mag 30(4):68–73
Ford KM, Glymour C, Hayes PJ (1997) Cognitive prostheses. AI Mag 18(3):104–105
Gallud JA, Tesoriero R, Penichet VMR (eds) (2011) Distributed user interfaces - designing interfaces for the distributed ecosystem. Springer, London
Gallud JA, Lozano MD, Vanderdonckt J (2014) Distributed user interfaces: usability and collaboration. Int J Hum Comput Stud 72(1):44. https://doi.org/10.1016/j.ijhcs.2013.10.006
Germanakos P, Belk M (2016) Human-centred web adaptation and personalization - from theory to practice. Human-computer interaction series. Springer, Berlin
Gonzalez Calleros J, Meixner G, Paterno F, Pullmann J, Raggett DS, Vanderdonckt J (2010) Model-based ui xg final report. In: Cantera Fonseca J (ed) W3C incubator group report
Grammenos D, Akoumianakis D, Stephanidis C (2000) Integrated support for working with guidelines: the sherlock guideline management system. Interact Comput 12(3):281–311
Gratch J, Marsella S (2004) A domain-independent framework for modeling emotion. J Cogn Syst Res 5(4):269–306
Grudin J (1994) Computer-supported cooperative work: history and focus. Computer 27(5):19–26
Grudin J (2009) AI and HCI: two fields divided by a common focus. AI Mag 30(4):48–57
Hancock PA, Chignell MH (eds) (1989) Intelligent interfaces: theory, research and design. North-Holland, Amsterdam
Herman I, Marshall MS, Melançon G (2000) Graph visualisation and navigation in information visualisation: a survey. IEEE Trans Vis Comput Graph 6(1):24–43
Hewett T, Baecker R, Card S, Carey T, Gasen J, Mantei M, Perlman G, Strong G, Verplank W (1992) ACM SIGCHI curricula for human-computer interaction. ACM, New York
Hofer G, Richmond K (2010) Comparison of HMM and TMD methods for lip synchronisation. In: Kobayashi T, Hirose K, Nakamura S (eds) INTERSPEECH 2010, 11th annual conference of the international speech communication association, Makuhari, Chiba, Japan, 26–30 September 2010. ISCA, pp 454–457
Hoffman PE, Grinstein GG (2002) A survey of visualizations for high-dimensional data mining. In: Fayyad U, Grinstein G, Wierse A (eds) Information visualization in data mining and knowledge discovery. Morgan Kaufmann Publishers, San Francisco, pp 47–82
Hoffmann L, Krämer NC, Lam-Chi A, Kopp S (2009) Media equation revisited: do users show polite reactions towards an embodied agent? International workshop on intelligent virtual agents. Springer, Berlin, pp 159–165
Hook K (2000) Steps to take before intelligent user interfaces become real. Interact Comput 12:409–426
Huart J, Kolski C, Bastien C (2008) L’évaluation de documents multimédias, état de l’art. In: Merviel S (ed) Objectiver l’humain? Qualification, quantification, vol 1. Hermes, Paris, pp 211–250
Hussmann H, Meixner G, Zuehlke D (eds) (2011) Model-driven development of advanced user interfaces. Studies in computational intelligence, vol 340. Springer, Berlin
Hutchins E (1995) Cognition in the wild. MIT Press, Cambridge
Isenberg T, Isenberg P, Chen J, Sedlmair M, Möller T (2013) A systematic review on the practice of evaluating visualization. IEEE Trans Vis Comput Graph 19(12):2818–2827
Ivory MY, Hearst MA (2001) The state of the art in automating usability evaluation of user interfaces. ACM Comput Surv 33(4):470–516
Jacko JA (ed) (2012) The human computer handbook: fundamentals, evolving technologies, and emerging applications, 3rd edn. CRC Press, Boca Raton
Jacob RJK, Limbourg Q, Vanderdonckt J (eds) (2004) Computer-Aided Design of User Interfaces IV - Fifth International Conference of Computer-Aided Design of User Interfaces. Kluwer, Funchal
Jambon F, Brun P, Ait-Ameur Y (2001) Spécification des systèmes interactifs. In: Kolski C (ed) Analyse et Conception de l’IHM. Interaction Homme-Machine pour les SI, vol 1. Editions Hermes, Paris, pp 175–206
Jameson A (2007) Adaptive interfaces and agents. In: Sears A, Jacko JA (eds) The human computer handbook: fundamentals, evolving technologies, and emerging applications. CRC Press, Boca Raton, pp 433–458
Jameson A, Gajos KZ (2012) Systems that adapt to their users. In: Jacko JA (ed) The human computer handbook: fundamentals, evolving technologies, and emerging applications. CRC Press, Boca Raton, pp 431–456
Jaques PA, Viccari RM (2004) A BDI approach to infer student’s emotions. In: The proceedings of the Ibero-American conference on artificial intelligence (IBERAMIA). Springer, Puebla, pp 901–911
Kolski C, Le Strugeon E (1998) A review of intelligent human-machine interfaces in the light of the arch model. Int J Hum-Comput Interact 10(3):193–231
Kolski C, Vanderdonckt J (eds) (2002) Computer-aided design of user interfaces III - fourth international conference of computer-aided design of user interfaces. Kluwer, Valenciennes
Kolski C, Tendjaoui M, Millot P (1992) A process method for the design of “intelligent” man-machine interfaces: case study: “the decisional module of imagery”. Int J Hum Factors Manuf 2(2):155–175
Kubicki S, Lebrun Y, Lepreux S, Adam E, Kolski C, Mandiau R (2013) Simulation in contexts involving an interactive table and tangible objects. Simul Model Pract Theory 31:116–131
Lazarus RS (1991) Emotion and adaptation. Oxford University Press, Oxford
Lebrun Y, Adam E, Mandiau R, Kolski C (2015) A model for managing interactions between tangible and virtual agents on an RFID interactive tabletop: case study in traffic simulation. J Comput Syst Sci 81(3):585–598
Lee B, Plaisant C, Parr CS, Fekete J, Henry N (2006) Task taxonomy for graph visualization. In: AVI workshop on beyond time and errors: novel evaluation methods for information visualization BELIV ’06. ACM, New York, pp 1–5
Lieberman H (1995) Letizia: an agent that assists web browsing. In: Fourteenth international joint conference on artificial intelligence, IJCAI 95, pp 924–929
Limbourg Q, Vanderdonckt J, Michotte B, Bouillon L, Florins M, Trevisan D (2004) Usixml: a user interface description language for context-sensitive user interfaces. In: ACM AVI’2004 workshop “developing user interfaces with xml: advances on user interface description languages”, pp 55–62
Lozano MD, Mashat AS, Fardoun HM, Gallud JA, Penichet VMR, Tesoriero R, Vanderdonckt J (eds) (2013) Distributed user interfaces: models, methods and tools, DUI 2013. In: Conjunction with ACM EICS 2013 conference, London, UK, 24 June 2013
Mandiau R, Kolski C, Millot P, Chaib-Draa B (1991) A new approach for the cooperation between human(s) and assistance system(s): a system based on intentional states. In: Liebowitz J (ed) Expert systems world congress proceedings, vol 3. Pergamon Press, New-York, pp 1672–1679
Maybury MT (1999) Intelligent user interfaces: an introduction (tutorial notes)
de Melo CM, Carnevale P, Read S, Antos D, Gratch J (2012) Bayesian model of the social effects of emotion in decision-making in multiagent systems. In: Proceedings of the 11th international conference on autonomous agents and multiagent systems - volume 1, international foundation for autonomous agents and multiagent systems, Richland, SC, AAMAS ’12, pp 55–62
Meyer M, Sedlmair M, Munzner T (2012) The four-level nested model revisited: Blocks and guidelines. In: Workshop on beyond time and errors: novel evaluation methods for information visualization (BELIV)
Moussa F, Ismail I, Jarraya M (2015) Towards a runtime evolutionary model of user-adapted interaction in a ubiquitous environment: the RADEM formal model. Cogn Technol Work 17(3):391–415
Munzner T (2009) A nested process model for visualization design and validation. IEEE Trans Vis Comput Graph 15:921–928
Nielsen J (1993) Usability engineering. Academic, New York
Niewiadomski R, Pelachaud C (2007) Fuzzy similarity of facial expressions of embodied agents. In: Proceedings of the 7th international conference on intelligent virtual agents. Springer, Berlin, pp 86–98
Norman DA (1986) Cognitive engineering. In: Norman DA, Draper SW (eds) User centered system design: new perspectives on human-computer interaction. Lawrence Erlbaum Associates, Hillsdale, pp 31–61
Ochs M, Niewiadomski R, Brunet P, Pelachaud C (2012a) Smiling virtual agent in social context. Cogn Process 13(2):519–532
Ochs M, Sadek D, Pelachaud C (2012b) A formal model of emotions for an empathic rational dialog agent. Auton Agents Multi-Agent Syst 24(3):410–440
de Oliveira KM, Bacha F, Houda M, Abed M (2013) Transportation ontology definition and application for the content personalization of user interfaces. Expert Syst Appl 40(8):3145–3159
Paterno F (1999) Model-based design and evaluation of interactive applications, 1st edn. Springer, London
Peintner B, Viappiani P, Yorke-Smith N (2008) Preferences in interactive systems: technical challenges and case studies. AI Mag 29(4):13–24
Pelachaud C (2009) Modeling multimodal expression of emotion in a virtual agent. Philos Trans R Soc B Biol Sci 364:3539–3548
Picard R (1997) Affective computing. MIT Press, Cambridge
Porhet C, Ochs M, Saubesty J, Montcheuil G, Bertrand R (2017) Mining a multimodal corpus of doctor’s training for virtual patient’s feedbacks. In: Proceedings of 19th ACM international conference on multimodal interaction (ICMI), Glasgow, UK
Provost F, Fawcett T (2013) Data science and its relationship to big data and data-driven decision making. Big Data 1(1):51–59
Purchase HC (2012) Experimental human-computer interaction: a practical guide with visual examples. Cambridge University Press, Cambridge
Rouse WB, Morris NM (1987) Conceptual design of a human error tolerant interface for complex engineering systems. Automatica 23(2):231–235
Salovey P, Bedell B, Detweiler J, Mayer J (2000) Current directions in emotional intelligence research. In: Lewis M, Haviland-Jones J (eds) Handbook of emotions. Guilford Press, New York, pp 504–520
Scapin DL (1986) Guide ergonomique de conception des interfaces homme-ordinateur. Technical report 77, INRIA, Le Chesnay
Scherer K (2000) Emotion. In: Hewstone M, Stroebe W (eds) Introduction to social psychology: a European perspective. Oxford Blackwell Publishers, Oxford, pp 151–191
Schneider-Hufschmidt M, Khme T, Malinkowski U (eds) (1993) Adaptive user interfaces. North Holland, Amsterdam
Sedlmair M, Meyer M, Munzner T (2012) Design study methodology: reflections from the trenches and the stacks. IEEE Trans Vis Comput Graph (Proc InfoVis 2012) 18(12):2431–2440
Seffah A, Vanderdonckt J, Desmarais M (eds) (2009) Human-centered software engineering, software engineering models, patterns and architectures for HCI, vol 2. Springer, London
Shadbolt N, Hall W, Berners-Lee T (2006) The semantic web revisited. IEEE Intell Syst 21(3):96–101
Shneiderman B (1996) The eyes have it: a task by data type taxonomy for information visualization. In: IEEE conference on visual languages (VL’96). IEEE CS Press, pp 336–343
Shneiderman B, Maes P (1997) Direct manipulation versus interface agents. Interactions 4(6):42–61
Silva SF, Catarci T (2000) Visualization of linear time-oriented data: a survey. In: Proceedings of the first international conference on web information systems engineering, vol 1, pp 310–319
Smith SL, Mosier JN (1986) Guidelines for designing user interface software. Technical report MTR-10090, ESD-TR-86-278, MITRE Corp., Bedford, MA
Sottet J, Calvary G, Favre J, Coutaz J (2009) Megamodeling and metamodel-driven engineering for plastic user interfaces: MEGA-UI. In: Human-centered software engineering - software engineering models, patterns and architectures for HCI, pp 173–200
Stephanidis C (2009) The universal access handbook. CRC Press, Boca Raton
Stewart T, Travis D (2002) Guidelines, standards, and style guides. In: Jacko JA, Sears A (eds) The human computer handbook: fundamentals, evolving technologies, and emerging applications. L. Erlbaum Associates Inc, Hillsdale, pp 991–1005
Su X, Khoshgoftaar TM (2009) A survey of collaborative filtering techniques. Adv Artif Intell 2009:1–19. Article ID 421,425
Szekely P (1996) Retrospective and challenges for model-based interface development. Design, specification and verification of interactive systems, vol 96. Springer, Berlin, pp 1–27
Thevenin D, Coutaz J (1999) Plasticity of user interfaces: framework and research agenda. In: Sasse A, Johnson C (eds) IFIP TC.13—Human-computer interaction, INTERACT 99. IOS Press
Thomas JJ, Cook KA (2006) Illuminating the path: the research and development agenda for visual analytics. IEEE Computer Society
Tran CD, Ezzedine H, Kolski C (2013) Eiseval, a generic reconfigurable environment for evaluating agent-based interactive systems. Int J Hum Comput Stud 71(6):725–761
van den Bergh J (2006) High-level user interface models for model-driven design of context-sensitive user interfaces. PhD thesis, Hasselt University, Belgium
Vanderdonckt J (1994) Guide ergonomique des interfaces homme-machine. Presses Universitaires de Namur, Namur, Facultés universitaires Notre-Dame de la Paix, Namur, Belgium
Vanderdonckt J (1999) Development milestones towards a tool for working with guidelines. Interact Comput 12(2):81–118
Vanderdonckt J, Coyette A (2007) Modèles, méthodes et outils de support au prototypage multi-fidélité des interfaces graphiques. Revue d’Interaction Homme-Machine 8(1):91–123
Vanderdonckt J, Puerta AR (eds) (1999) Computer-aided design of user interfaces II - third international conference of computer-aided design of user interfaces. Kluwer, Louvain-la-Neuve
van Wijk J (2005) The value of visualization. In: Silva C, Groeller E, Rushmeier H (eds) IEEE visualization. IEEE Computer Society, pp 79–86
von Landesberger T, Kuijper A, Schreck T, Kohlhammer J, van Wijk JJ, Fekete JD, Fellner DW (2011) Visual analysis of large graphs: state-of-the-art and future research challenges. Comput Graph Forum 30(6):1719–1749
Ware C (2000) Information visualization: perception for design. Morgan Kaufmann Publishers, San Francisco
Winograd T (2001) Architectures for context. Hum-Comput Interact 16(2):401–419
Woodson W, Conover D (1964) Human engineering guide for equipment designers. University of California Press, Berkeley
Zhang L, Stoffel A, Behrisch M, Mittelstadt S, Schreck T, Pompl R, Weber S, Last H, Keim D (2012) Visual analytics for the big data era?a comparative review of state-of-the-art commercial systems. In: 2012 IEEE conference on visual analytics science and technology (VAST). IEEE, pp 173–182
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Kolski, C., Boy, G.A., Melançon, G., Ochs, M., Vanderdonckt, J. (2020). Cross-Fertilisation Between Human-Computer Interaction and Artificial Intelligence. In: Marquis, P., Papini, O., Prade, H. (eds) A Guided Tour of Artificial Intelligence Research. Springer, Cham. https://doi.org/10.1007/978-3-030-06170-8_11
Download citation
DOI: https://doi.org/10.1007/978-3-030-06170-8_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-06169-2
Online ISBN: 978-3-030-06170-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)