Skip to main content

Advertisement

SpringerLink
Log in

Real-time EEG-based emotion monitoring using stable features

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

In human–computer interaction (HCI), electroencephalogram (EEG) signals can be added as an additional input to computer. An integration of real-time EEG-based human emotion recognition algorithms in human–computer interfaces can make the users experience more complete, more engaging, less emotionally stressful or more stressful depending on the target of the applications. Currently, the most accurate EEG-based emotion recognition algorithms are subject-dependent, and a training session is needed for the user each time right before running the application. In this paper, we propose a novel real-time subject-dependent algorithm with the most stable features that gives a better accuracy than other available algorithms when it is crucial to have only one training session for the user and no re-training is allowed subsequently. The proposed algorithm is tested on an affective EEG database that contains five subjects. For each subject, four emotions (pleasant, happy, frightened and angry) are induced, and the affective EEG is recorded for two sessions per day in eight consecutive days. Testing results show that the novel algorithm can be used in real-time emotion recognition applications without re-training with the adequate accuracy. The proposed algorithm is integrated with real-time applications “Emotional Avatar” and “Twin Girls” to monitor the users emotions in real time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Allen, J.J., Urry, H.L., Hitt, S.K., Coan, J.A.: The stability of resting frontal electroencephalographic asymmetry in depression. Psychophysiology 41(2), 269–280 (2004)

    Article  Google Scholar 

  2. Bouton, M.E.: Learning and Behavior: A Contemporary Synthesis. Sinauer Associates, Sunderland (2007)

    Google Scholar 

  3. Bradley, M.M., Lang, P.J.: The international affective digitized sounds (; iads-2): affective ratings of sounds and instruction manual. Technical report B-3, University of Florida, Gainesville (2007)

  4. Brown, L., Grundlehner, B., Penders, J.: Towards wireless emotional valence detection from EEG. In: Engineering in Medicine and Biology Society, EMBC, 2011 Annual International Conference of the IEEE, pp. 2188–2191. IEEE (2011)

  5. Chanel, G., Kierkels, J.J., Soleymani, M., Pun, T.: Short-term emotion assessment in a recall paradigm. Int. J. Hum.–Comput. Stud. 67(8), 607–627 (2009)

    Article  Google Scholar 

  6. Chang, C.C., Lin, C.J.: Libsvm: a library for support vector machines. ACM Trans. Intell. Syst. Technol. (TIST) 2(3), 27 (2011)

    Google Scholar 

  7. Emotiv. http://www.emotiv.com. Accessed 26 Feb 2015

  8. Frantzidis, C., Bratsas, C., Papadelis, C.L., Konstantinidis, E., Pappas, C., Bamidis, P.D., et al.: Toward emotion aware computing: an integrated approach using multichannel neurophysiological recordings and affective visual stimuli. Inf. Technol. Biomed. IEEE Trans. 14(3), 589–597 (2010)

    Article  Google Scholar 

  9. Gasser, T., Bächer, P., Steinberg, H.: Test-retest reliability of spectral parameters of the EEG. Electroencephalogr. Clin. Neurophysiol. 60(4), 312–319 (1985)

    Article  Google Scholar 

  10. Gasser, T., Jennen-Steinmetz, C., Verleger, R.: EEG coherence at rest and during a visual task in two groups of children. Electroencephalogr. Clin. Neurophysiol. 67(2), 151–158 (1987)

    Article  Google Scholar 

  11. Gudmundsson, S., Runarsson, T.P., Sigurdsson, S., Eiriksdottir, G., Johnsen, K.: Reliability of quantitative EEG features. Clin. Neurophysiol. 118(10), 2162–2171 (2007)

    Article  Google Scholar 

  12. Haptek. http://www.haptek.com. Accessed 28 Feb 2015

  13. Higuchi, T.: Approach to an irregular time series on the basis of the fractal theory. Phys. D: Nonlinear Phenom. 31(2), 277–283 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  14. Hosseini, S.A., Khalilzadeh, M.A., Naghibi-Sistani, M.B., Niazmand, V.: Higher order spectra analysis of EEG signals in emotional stress states. In: 2010 Second International Conference on Information Technology and Computer Science (ITCS), pp. 60–63. IEEE (2010)

  15. Ishino, K., Hagiwara, M.: A feeling estimation system using a simple electroencephalograph. In: IEEE International Conference on Systems, Man and Cybernetics, 2003. vol. 5, pp. 4204–4209. IEEE (2003)

  16. Kedem, B., Yakowitz, S.: Time Series Analysis by Higher Order Crossings. IEEE Press, Piscataway (1994)

    MATH  Google Scholar 

  17. Koelstra, S., Mühl, C., Soleymani, M., Lee, J.S., Yazdani, A., Ebrahimi, T., Pun, T., Nijholt, A., Patras, I.: Deap: a database for emotion analysis; using physiological signals. Affect. Comput. IEEE Trans. 3(1), 18–31 (2012)

    Article  Google Scholar 

  18. Kondacs, A., Szabó, M.: Long-term intra-individual variability of the background EEG in normals. Clin. Neurophysiol. 110(10), 1708–1716 (1999)

    Article  Google Scholar 

  19. Lan, Z., Sourina, O., Wang, L., Liu, Y.: Stability of features in real-time EEG-based emotion recognition algorithm. In: 2014 International Conference on Cyberworlds (CW), pp. 137–144. IEEE (2014)

  20. Lang, P.J., Bradley, M.M., Cuthbert, B.N.: International affective picture system (iaps): affective ratings of pictures and instruction manual. Technical report A-8 (2008)

  21. Li, M., Lu, B.L.: Emotion classification based on gamma-band EEG. In: Engineering in Medicine and Biology Society, 2009. EMBC 2009. Annual International Conference of the IEEE, pp. 1223–1226. IEEE (2009)

  22. Lin, Y.P., Wang, C.H., Jung, T.P., Wu, T.L., Jeng, S.K., Duann, J.R., Chen, J.H.: EEG-based emotion recognition in music listening. Biomed. Eng. IEEE Trans. 57(7), 1798–1806 (2010)

    Article  Google Scholar 

  23. Lin, Y.P., Wang, C.H., Wu, T.L., Jeng, S.K., Chen, J.H.: EEG-based emotion recognition in music listening: a comparison of schemes for multiclass support vector machine. In: IEEE International Conference on Acoustics, Speech and Signal Processing, 2009. ICASSP 2009, pp. 489–492. IEEE (2009)

  24. Liu, Y., Sourina, O.: EEG databases for emotion recognition. In: 2013 International Conference on Cyberworlds (CW), pp. 302–309. IEEE (2013)

  25. Liu, Y., Sourina, O.: Real-time fractal-based valence level recognition from EEG. In: Transactions on Computational Science XVIII, pp. 101–120. Springer (2013)

  26. Liu, Y., Sourina, O.: Real-time subject-dependent EEG-based emotion recognition algorithm. In: Transactions on Computational Science XXIII, pp. 199–223. Springer (2014)

  27. Liu, Y., Sourina, O., Nguyen, M.K.: Real-time EEG-based emotion recognition and its applications. In: Transactions on computational science XII, pp. 256–277. Springer (2011)

  28. McGraw, K.O., Wong, S.P.: Forming inferences about some intraclass correlation coefficients. Psychol. Methods 1(1), 30 (1996)

    Article  Google Scholar 

  29. Murugappan, M., Nagarajan, R., Yaacob, S.: Combining spatial filtering and wavelet transform for classifying human emotions using EEG signals. J. Med. Biol. Eng. 31(1), 45–51 (2011)

    Article  Google Scholar 

  30. Petrantonakis, P.C., Hadjileontiadis, L.J.: Emotion recognition from EEG using higher order crossings. Inf. Technol. Biomed. IEEE Trans. 14(2), 186–197 (2010)

    Article  Google Scholar 

  31. Petrantonakis, P.C., Hadjileontiadis, L.J.: A novel emotion elicitation index using frontal brain asymmetry for enhanced EEG-based emotion recognition. Inf. Technol. Biomed. IEEE Trans. 15(5), 737–746 (2011)

    Article  Google Scholar 

  32. Petrantonakis, P.C., Hadjileontiadis, L.J.: Adaptive emotional information retrieval from EEG signals in the time-frequency domain. Signal Process. IEEE Trans. 60(5), 2604–2616 (2012)

    Article  MathSciNet  Google Scholar 

  33. Picard, R.W., Vyzas, E., Healey, J.: Toward machine emotional intelligence: analysis of affective physiological state. Pattern Anal. Mach. Intell. IEEE Trans. 23(10), 1175–1191 (2001)

    Article  Google Scholar 

  34. Salinsky, M., Oken, B., Morehead, L.: Test-retest reliability in EEG frequency analysis. Electroencephalogr. Clin. Neurophysiol. 79(5), 382–392 (1991)

    Article  Google Scholar 

  35. Sanei, S., Chambers, J.A.: EEG Signal Processing. Wiley, New York (2013)

    Google Scholar 

  36. Schaaff, K.: EEG-Based Emotion Recognition. Universitat Karlsruhe (TH), Karlsruhe (2008)

    Google Scholar 

  37. Schaaff, K., Schultz, T.: Towards an EEG-based emotion recognizer for humanoid robots. In: The 18th IEEE International Symposium on Robot and Human Interactive Communication, 2009. RO-MAN 2009, pp. 792–796. IEEE (2009)

  38. Sohaib, A.T., Qureshi, S., Hagelbäck, J., Hilborn, O., Jerčić, P.: Evaluating classifiers for emotion recognition using EEG. In: Foundations of Augmented Cognition, pp. 492–501. Springer (2013)

  39. Sourina, O., Liu, Y., Nguyen, M.K.: Real-time EEG-based emotion recognition for music therapy. J. Multimodal User Interf. 5(1–2), 27–35 (2012)

    Article  Google Scholar 

  40. Tomarken, A.J., Davidson, R.J., Wheeler, R.E., Kinney, L.: Psychometric properties of resting anterior EEG asymmetry: temporal stability and internal consistency. Psychophysiology 29(5), 576–592 (1992)

    Article  Google Scholar 

  41. Wang, X.W., Nie, D., Lu, B.L.: EEG-based emotion recognition using frequency domain features and support vector machines. In: Neural Information Processing, pp. 734–743. Springer (2011)

  42. Williams, C.E., Stevens, K.N.: Emotions and speech: some acoustical correlates. J. Acoust. Soc. Am. 52(4B), 1238–1250 (1972)

    Article  Google Scholar 

Download references

Acknowledgments

The work is supported by Fraunhofer IDM@NTU, which is funded by the National Research Foundation (NRF) and managed through the multi-agency Interactive and Digital Media Programme Office (IDMPO).

Author information

Authors and Affiliations

  1. Fraunhofer IDM@NTU, Nanyang Technological University, 50 Nanyang Avenue, Singapore, 639798, Singapore

    Zirui Lan, Olga Sourina & Yisi Liu

  2. School of Electrical and Electronic Engineering, Nanyang Technological University, 50 Nanyang Avenue, Singapore, 639798, Singapore

    Lipo Wang

Authors
  1. Zirui Lan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Olga Sourina
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lipo Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yisi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zirui Lan.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lan, Z., Sourina, O., Wang, L. et al. Real-time EEG-based emotion monitoring using stable features. Vis Comput 32, 347–358 (2016). https://doi.org/10.1007/s00371-015-1183-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-015-1183-y

Keywords

Search

Navigation