Skip to main content
SpringerLink
Log in

Ontology-based context modeling for emotion recognition in an intelligent web

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

We describe an ontological model for representation and integration of electroencephalographic (EEG) data and apply it to detect human emotional states. The model (BIO_EMOTION) is an ontology-based context model for emotion recognition and acts as a basis for: (1) the modeling of users’ contexts, including user profiles, EEG data, the situation and environment factors, and (2) supporting reasoning on the users’ emotional states. Because certain ontological concepts in the EEG domain are ill-defined, we formally represent and store these concepts, their taxonomies and high-level representation (i.e., rules) in the model. To evaluate the effectiveness for inferring emotional states, DEAP dataset is used for model reasoning. Result shows that our model reaches an average recognition ratio of 75.19 % on Valence and 81.74 % on Arousal for eight participants. As mentioned above, the BIO-EMOTION model acts like a bridge between users’ emotional states and low-level bio-signal features. It can be integrated in user modeling techniques, and be used to model web users’ emotional states in human-centric web aiming to provide active, transparent, safe and reliable services to users. This work aims at, in other words, creating an ontology-based context model for emotion recognition using EEG. Particularly, this model completely implements the loop body of the W2T data cycle once: from low-level EEG feature acquisition to emotion recognition. A long-term goal for the study is to complete this model to implement the whole W2T data cycle.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Astrova, I.: Reverse Engineering of Relational Database to Ontologies. The Semantic Web: Research and Applications First European Semantic Web Symposium, pp. 327–341 (2004)

  2. Astrova, I., Stantic, B.: Reverse Engineering of Relational Databases to Ontologies: An Approach Based on an Analysis of HTML Forms. Proc. the Workshop on Knowledge Discovery and Ontologies at ECML/PKDD, pp. 73–78 (2004)

  3. Benta, K. L., Rarău, A., Cremene, M.: Ontology based affective context representation. In: Proceedings of the 2007 Euro American conference on Telematicsand information systems (EATIS’07). Faro, Portugal (2007)

  4. Busso, C., Deng, Z., et al.: Analysis of emotion recognition using facial expressions, speech and multimodal information. Proceedings of the 6th International Conference on Multimodal interfaces. State College, PA, USA, ACM: 205–211 (2004)

  5. Caroglio, V., de Rosis, F. d.: Combining logical with emotional reasoning in natural argumentation. In: Conati, C., Hudlika, E., Lisetti, C. (eds.) The UM’03 Workshop on Affect, Pittsburgh (2003)

  6. Cearreta, I., López, J.M., Garay, N.: Modelling multimodal context-aware affective interaction. In: Proceedings of the Doctoral Consortium of the Second international conference on Affective Computing and Intelligent Interaction (ACII 2007), pp. 57–64. Lisbon, Portugal (2007)

  7. Chaouchi, H.: The Internet of Things-Connecting Objects to the Web. ISTE Ltd.Wiley, New York (2010)

    Google Scholar 

  8. Chi, Y.-L., Peng, S.-Y., Yang, C.-C.: Creating Kansei engineering-based ontology for annotating and archiving photos database. In: Jacko, J. (ed.) Human-Computer Interaction, Part I, HCII 2007, LNCS vol. 4550, pp. 701–710. Springer (2007)

  9. Conati, C.: Probabilistic assessment of user’s emotions in educational games. J. Appl. Artif. Intell. 16(7–8), (2010)

  10. Dillon, T., Talevski, A., Potdar, V., Chang, E.: Web of things as a framework for ubiquitous intelligence and computing. In: Proc the 6th International Conference on Ubiquitous Intelligence and Computing, pp 1–10 (2009)

  11. Dou, D., Frishkoff, G., Rong, J., Frank, R., Malony, A. and Tucker, D.: Development of Neuroelectromagnetic Ontologies (Nemo): a framework for mining brainwave ontologies. In Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining, 270–279. San Jose, California, USA: ACM (2007)

  12. Fisher, D.H.: Knowledge acquisition via incremental conceptual clustering. Mach. Learn. 2(2), 139–172 (1987)

    Google Scholar 

  13. Francisco, V., Gervás, P., Peinado, F.: Ontological reasoning to configure emotional voice synthesis. In: Proceedings of the First International Conference of Web Reasoning and Rule Systems (Innsbruck,Austria, 2007), pp. 88–102. Springer, Berlin/Heidelberg (2007)

  14. Frantzidis, C.A., Bratsas, C., Klados, M.A., Konstantinidis, E., Lithari, C.D., Vivas, A.B., Papadelis, C.L., Kaldoudi, E., Pappas, C., Bamidis, P.D.: On the classification of emotional biosignals evoked while viewing affective pictures: an integrated data-mining-based approach for healthcare applications. IEEE Trans. Inf. Technol. Biomed. 14(2), 309–318 (2010)

    Article  Google Scholar 

  15. Galunov, V.I., Lobanov, B.M., Zagoruiko, N.G.: Ontology of the subject domain. In: Speech Signals Recognition and Synthesis SPECOM’2004, 9th Conference Speechand Computer Saint-Petersburg, Russia (2004)

  16. González, G., López, B., Rosa, J.L.D.L.: Managing emotions in smart user models for recommender systems. In: Proceedings of 6th International Conference on Enterprise Information Systems ICEIS 2004, vol. 5, pp. 187–194 (2004)

  17. Gruber, T.: A translation approach to portable ontology specifications. In: Knowledge Acquisition, vol. 5, pp. 199–220 (1993)

  18. Hudlicka, E., McNeese, M.D.: Assessment of user affective and belief states for interface adaptation: application to an air force pilot task. User Model. User-Adap. Inter. 12, 1–47 (2002)

    Article  MATH  Google Scholar 

  19. Jena: A semantic web Framework for java. http://jena.sourceforge.net/

  20. Klein, J., Moon, Y., Picard, R.W.: This computer responds to user frustration: theory, design, and results. Interact. Comput. 14, 119–140 (2002)

    Article  Google Scholar 

  21. Koelstra, S., Muhl, C., et al.: DEAP: A database for emotion analysis using physiological signals. IEEE Trans. Affect. Comput., 99: 1–1 (2011)

    Google Scholar 

  22. Kunii, T.L., Ma, J.H., Huang, R.H.: Hyper world modeling. In: Proc the International Conference on Visual Information Systems (VIS’96), pp. 1–8 (1996)

  23. López, J.M., Gil, R., García, R., Cearreta, I., Garay, N.: Towards an ontology for describing emotions. In: Proceedings of the 1st World Summiton the Knowledge Society (Athens, Greece, 2008), pp. 96–104. Springer, Berlin/Heidelberg (2008)

  24. López, J.M., Gil, R., et al.: Towards an Ontology for Describing Emotions. Emerging Technologies and Information Systems for the Knowledge Society, vol. 5288, pp. 96–104. Springer, Berlin (2008)

    Book  Google Scholar 

  25. Ma, J.H., Huang, R.H.: Improving human interaction with a Hyper world. In: Proc the Pacific Workshop on Distributed Multimedia Systems (DMS’96), pp. 46–50 (1996)

  26. Ma, J.H.: Smart u-Things-challenging real world complexity. In: IPSJ Symposium Series, vol 19, pp. 146–150 (2005)

  27. Ma, J.H., Yang, L.T., Apduhan, B.O., Huang, R.H., Barolli, L., Takizawa, M.: Towards a smart world and ubiquitous intelligence: a walkthrough from smart things to smart hyperspaces and UbicKids. Int. J. Pervasive Comput. Commun 1(1), 53–68 (2005)

    Article  MATH  Google Scholar 

  28. Maedche, A., Staab, S.: Ontology learning for the semantic web. IEEE Intell. Syst. 16(2), 72–79 (2001)

    Article  Google Scholar 

  29. Maedche, A., Staab, S.: Discovering Conceptual Relations from Text. Proc. ECAI 2000, pp. 32l–325 (2000)

  30. Mandryk, R.L., Atkins, M.S.: A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies. Int. J. Hum. Comput. Stud. 65(4), 329–347 (2007)

    Article  Google Scholar 

  31. Mathieu, Y.: Annotation of emotions and feelings in texts. In: Proceedings of the First International Conference ASCII 2005 (Beijing, China, 2005), pp. 350–357. Springer, Berlin/Heidelberg (2005)

  32. McGuinness, D.L., van Harmelen, F.: OWL Web Ontology Language Overview. W3C Recommendation, 10 February 2004. Available at: http://www.w3.org/TR/2004/REC-owl-features-20040210/

  33. Moore, P., Jackson, M., et al.: Fuzzy ECA rules for pervasive decision-centric personalised mobile learning computational intelligence for technology enhanced learning. In: Xhafa, F., Caballé, S., Abraham, A., Daradoumis, T., Juan Perez, A. (eds.). Springer Berlin, Heidelberg, 273: 25–58 (2010)

  34. Mudie, P., Cottam, A., et al.: An exploratory study of consumption emotion in services. Serv. Ind. J. 23(5), 84–106 (2003)

    Article  Google Scholar 

  35. Nakaya, N., Kurematsu, M., Yamaguchi, T.: A domain ontology development environment using a MRD and text corpus. Proc. the Fifth Joint Conference on Knowledge-based Software Engineering Frontiers in Artificial Intelligence and Applications, pp. 242–251 (2002)

  36. Nicholas, G., Stephen, H., Nigel, S.: Agent-based Semantic Web Services. Web Semant. Sci. Serv. Agents World Wide Web 1(2), 141–154 (2004)

    Article  Google Scholar 

  37. Peng, H., Hu, B., Liu, Q.Y., Dong, Q.X., Zhao, Q.L., Moore, P.: User-Centered Depression Prevention: An EEG Approach to Pervasive Healthcare, MindCare workshop in Pervasive Health 2011, Dublin, Ireland. pp. 325–330

  38. Peng, H., Hu, B., et al.: An improved EEG de-noising approach in electroencephalogram (EEG) for home care. Pervasive Computing Technologies for Healthcare (Pervasive Health), 2011 5th International Conference on (2011)

  39. Petrantonakis, P. C., Hadjileontiadis, L.J.: Adaptive extraction of emotion-related EEG segments using multidimensional directed information in time-frequency domain. Engineering in Medicine and Biology Society (EMBC), 2010 Annual International Conference of the IEEE (2010)

  40. Quilan, R.J.: C4.5: Programs for Machine Learning. Morgan Kauffman, San Mateo (1993)

    Google Scholar 

  41. RDF Vocabulary Description Language 1.0: RDF Schema. http://www.w3.org/TR/rdf-schema/

  42. Resource Description Framework (RDF): Concepts and Abstract Syntax. http://www.w3.org/TR/rdf-concepts/

  43. Russell, J.A.: A circumplex model of affect. J. Personal. Soc. Psychol. 39(6), 1161–1178 (1980)

    Article  Google Scholar 

  44. Russell, S., Norvig, P.: Artificial intelligence: a modern approach. Prentice Hall, Englewood Cliffs (1995)

    MATH  Google Scholar 

  45. Sabeti, M., Boostani, R., et al.: Selection of relevant features for EEG signal classification of schizophrenic patients. Biomed. Signal Process. Control 2(2), 122–134 (2007)

    Article  Google Scholar 

  46. Shamsfard, M., Barforoush, A.A.: Learning ontologies from natural language texts. Int. J. Hum. Comput. Stud. 60(1), l7–l63 (2004)

    Article  Google Scholar 

  47. Stickel, C., Ebner, M., Steinbach-Nordmann, S., Searle, G., Holzinger, A.: Emotion detection: application of the valence arousal space for rapid biological usability testing to enhance universal access. In: Proceedings of the 5th International Conference on Universal Access in Human-Computer Interaction, pp. 615–624. (2009).

  48. Stirbu, V.: Towards a RESTful plug and play experience in the web of things. In: Proc the 2008 IEEE International Conference on Semantic Computing, pp. 512–517 (2008)

  49. Villon, O. Lisetti, C.: A user-modeling approach to build user’s psycho-physiological maps of emotions using bio-sensors. Proc. IEEE Int. Workshop Robot Human Interactive Commun., p. 269 (2006)

  50. Wehrle, T., Scherer, K.R.: Towards Computational Modeling of Appraisal Theories. Appraisal Processes in Emotion: Theory, Methods, Research, pp. 350–365. Oxford University Press, New York (2001)

  51. Welbourne, E., Battle, L., Cole, G., Gould, K., Rector, K., Raymer, S., Balazinska, M., Borriello, G.: Building the Internet of things using RFID. IEEE Internet Comput. 33(3), 48–55 (2009)

    Article  Google Scholar 

  52. Xu, Z.M. Cao, X., Dong, Y.S. and Su, W.P.: Formal Approach and Automated Tool for Translating ER Schemata into OWL Ontologies. Proc. PAKDD 2004, pp. 464–476, (2004)

  53. Zhong, N.: Impending brain informatics research from Web Intelligence perspective. Int. J. Inf. Technol. Decis. Mak. 5(4), 713–727 (2006)

    Article  Google Scholar 

  54. Zhong, N., Liu, J., Yao, Y.Y., Ohsuga, S.: Web Intelligence (WI). In Proceedings of the 24th IEEE Computer Society International Computer Software and Applications Conference (COMPSAC 2000), pages 469-470, IEEE Computer Society Press, Taipei, Taiwan, October 25–28, 2000

  55. Zhong, N., Liu, J.M., Yao, Y.Y., Wu, J.L., Lu, S.F., Qin, Y.L., Li, K.C. and Wah, B.: Web Intelligence meets Brain Informatics. Proc. The First WICI International Workshop on Web Intelligence Meets Brain Informatics (WImBI 2006), pp. 1–31, (2006)

  56. Zhong, N., Ma, J.H., Huang, R.H., Liu, J.M., Yao, Y.Y., Zhang, Y.X., Chen, J.H.: Research Challenges and Perspectives on Wisdom Web of Things (W2T). Journal of Supercomputing, Springer (2010)

Download references

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Lanzhou University, Lanzhou, 730000, China

    Xiaowei Zhang, Bin Hu & Jing Chen

  2. School of Computing, Telecommunications and Networks, Birmingham City University, Birmingham, B42 2SU, UK

    Philip Moore

Authors
  1. Xiaowei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bin Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jing Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Philip Moore
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bin Hu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zhang, X., Hu, B., Chen, J. et al. Ontology-based context modeling for emotion recognition in an intelligent web. World Wide Web 16, 497–513 (2013). https://doi.org/10.1007/s11280-012-0181-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11280-012-0181-5

Keywords

Search

Navigation