Skip to main content
SpringerLink
Log in

Can recurrent neural network enhanced EEGNet improve the accuracy of ERP classification task? An exploration and a discussion

  • Original Paper
  • Published:
Health and Technology Aims and scope Submit manuscript

Abstract

Autism Spectrum Disorder (ASD) is one of the most common developmental conditions with one in 160 children worldwide being diagnosed. Both Virtual Reality (VR) and Brain-Computer Interfaces (BCI) are believed to be beneficial to enhance communication for people with ASD. However, BCI solutions for ASD are not yet commercially available. This is partly due to the current challenge with long and fatiguing calibration sessions with conventional gel based BCI. BCI using active electrodes hold the potentials to resolve part of this issue but might increase the challenges to classification of tasks due to reduced signal quality. The dataset considered in this paper, available from the IFMBE Scientific Challenge (IFMBE SC) of 15 participants with ASD, contained data captured using electroencephalogram (EEG) headsets, from g.Nautilus system, with active electrodes in a VR environment. Known approaches, such as the Support Vector Machine (SVM), Linear Discriminant Analysis (LDA), Convolutional Neural Networks (CNN) and Long-Short Term Memory (LSTM) have demonstrated potential solutions to enhancing current algorithms. Nevertheless, in this paper, a novel Recurrent Neural Network (RNN) solution with several pre-processing methods was introduced. Our results show that our novel RNN solution achieved 92.59% accuracy, an improvement with 0.61 percentage point from the previously best reported algorithm during the IFMBE SC. Furthermore, with a standard 80%-20% initial separation strategy, our solution also generated a compatible accuracy at 89.92%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Ifmbe scientific challenge (2019). https://www.medicon2019.org/scientific-challenge/.

  2. Amaral C, Mouga S, Simões M, Pereira HC, Bernardino I, Quental H, Playle R, McNamara R, Oliveira G, Castelo-Branco M. A feasibility clinical trial to improve social attention in autistic spectrum disorder (asd) using a brain computer interface. Frontiers in Neuroscience 2018;12:477. https://doi.org/10.3389/fnins.2018.00477. https://www.frontiersin.org/article/10.3389/fnins.2018.00477.

    Article  Google Scholar 

  3. Amaral CP, Simões MA, Mouga S, Andrade J, Castelo-Branco M. A novel brain computer interface for classification of social joint attention in autism and comparison of 3 experimental setups: a feasibility study. Journal of Neuroscience Methods 2017;290:105–115. https://doi.org/10.1016/j.jneumeth.2017.07.029. http://www.sciencedirect.com/science/article/pii/S0165027017302728.

    Article  Google Scholar 

  4. Asif U, Roy S, Tang J, Harrer S. 2019. Seizurenet: a deep convolutional neural network for accurate seizure type classification and seizure detection. arXiv:1903.03232.

  5. Blankertz B, Muller KR, Krusienski D, Schalk G, Wolpaw J, Schlogl A, Pfurtscheller G, Millan J, Schroder M, Birbaumer N. The bci competition iii: validating alternative approaches to actual bci problems. IEEE Transactions on Neural Systems and Rehabilitation Engineering June 2006;14(2):153,159.

    Google Scholar 

  6. Bleichner MG, Debener S. Concealed, unobtrusive ear-centered eeg acquisition: ceegrids for transparent eeg. Frontiers in Human Neuroscience 2017;11:163. https://doi.org/10.3389/fnhum.2017.00163. https://www.frontiersin.org/article/10.3389/fnhum.2017.00163.

    Article  Google Scholar 

  7. Borra D, Fantozzi S, Magosso E. Convolutional neural network for a p300 brain-computer interface to improve social attention in autistic spectrum disorder. Mediterranean conference on medical and biological engineering and computing. Springer; 2019. p. 1837–1843.

  8. Cataltepe Z, Abu-Mostafa YS, Magdon-Ismail M. No free lunch for early stopping. Neural Computation 1999;11(4):995–1009.

    Article  Google Scholar 

  9. Cho J, Lee K, Shin E, Choy G, Do S. 2015. How much data is needed to train a medical image deep learning system to achieve necessary high accuracy? arXiv:1511.06348.

  10. Chollet F, et al. 2015. Keras. https://keras.io.

  11. Chung J, Gulcehre C, Cho K, Bengio Y. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv:1412.3555.

  12. Didoné DD, Garcia MV, Oppitz SJ, Silva TFFD, Santos SND, Bruno RS, Filha VAVDS, Cóser PL. Auditory evoked potential p300 in adults: reference values. Einstein (São Paulo) 2016;14(2):208–212.

    Article  Google Scholar 

  13. Disease G, Incidence I, Collaborators P. Global, regional, and national incidence, prevalence, and years lived with disability for 310 diseases and injuries, 1990-2015: a systematic analysis for the global burden of disease study 2015. Lancet (London, England) 2016;388(10053):1545–1602. https://doi.org/10.1016/S0140-6736(16)31678-6. https://www.ncbi.nlm.nih.gov/pubmed/27733282.

    Article  Google Scholar 

  14. Dougherty ER, Hua J, Sima C. Performance of feature selection methods. Current Genomics 2009;10(6):365–374 . http://ezproxy.library.usyd.edu.au/login? https://search-proquest-com.ezproxy1.library.usyd.edu.au/docview/733659832?accountid=14757. Date completed - 2011-07-14; Date created - 2010-03-02; Date revised - 2019-05-07; SuppNotes - Cited By: BMC Bioinformatics. 2005;6:97 15826317] Cancer Res. 2003 Jan 1;63(1):60-6 12517778] Bioinformatics. 2005 Aug 1;21(15):3301-7 15905277] Bioinformatics. 2007 Oct 1;23(19):2507-17 17720704] Bioinformatics. 2006 Apr 1;22(7):837-42 16428263] Nature. 2002 Jan 31;415(6871):530-6 11823860] BMC Bioinformatics. 2006;7:359 16872483] Bioinformatics. 2005 Apr 15;21(8):1509-15 15572470] Mol Cancer Ther. 2002 Nov;1(13):1229-36 12479704] Int J Oncol. 2003 Sep;23(3):617-25 12888896] Bioinformatics. 2004 Oct 12;20(15):2429-37 15087314] EURASIP J Bioinform Syst Biol. 2007;:16354 17713587] Brief Bioinform. 2008 Sep;9(5):392-403 18562478] Cancer Inform. 2006;2:189-96 19458767] Stat Appl Genet Mol Biol. 2006;5:Article6 16646870] N Engl J Med. 2002 Dec 19;347(25):1999-2009 12490681] Curr Genomics. 2008 Apr;9(2):69-79 19440447] Bioinformatics. 2006 Oct 1;22(19):2430-6 16870934] Bioinformatics. 2004 Feb 12;20(3):374-80 14960464; Last updated - 2019-05-22.

    Article  Google Scholar 

  15. Friedrich EVC, Suttie N, Sivanathan A, Lim T, Louchart S, Pineda JA. Brain–computer interface game applications for combined neurofeedback and biofeedback treatment for children on the autism spectrum. Frontiers in Neuroengineering 2014;7:21. https://doi.org/10.3389/fneng.2014.00021. https://www.frontiersin.org/article/10.3389/fneng.2014.00021.

    Article  Google Scholar 

  16. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y. Generative adversarial nets. Advances in neural information processing systems; 2014. p. 2672–2680.

  17. Gramfort A, Luessi M, Larson E, Engemann DA, Strohmeier D, Brodbeck C, Goj R, Jas M, Brooks T, Parkkonen L, et al. Meg and eeg data analysis with mne-python. Frontiers in Neuroscience 2013;7:267.

    Article  Google Scholar 

  18. Itti L, Koch C, Niebur E. 1998. A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis & Machine Intelligence (11):1254–1259.

  19. Kingma DP, Ba J. 2014. Adam: a method for stochastic optimization. arXiv:1412.6980.

  20. Kolev V, Demiralp T, Yordanova J, Ademoglu A, Isoglu-Alkaç U. Time-frequency analysis reveals multiple functional components during oddball p300. Neuroreport 1997;8(8):2061,2065. http://search.proquest.com/docview/79129813/.

    Article  Google Scholar 

  21. Lawhern VJ, Solon AJ, Waytowich NR, Gordon SM, Hung CP, Lance BJ. Eegnet: a compact convolutional neural network for eeg-based brain–computer interfaces. Journal of Neural Engineering 2018;15(5):056013.

    Article  Google Scholar 

  22. Lin CT, Liao LD, Liu YH, Wang IJ, Lin BS, Chang JY. Novel dry polymer foam electrodes for long-term eeg measurement. IEEE Trans Biomed Eng 2010;58(5):1200–1207.

    Google Scholar 

  23. Lopez-Gordo MA, Sanchez-Morillo D, Valle FP. Dry eeg electrodes. Sensors 2014;14(7):12847–12870.

    Article  Google Scholar 

  24. Lotte F, Bougrain L, Cichocki A, Clerc M, Congedo M, Rakotomamonjy A, Yger F. A review of classification algorithms for eeg-based brain–computer interfaces: a 10 year update. J Neural Eng 2018;15(3):031005.

    Article  Google Scholar 

  25. Lotte F, Congedo M, Lécuyer A, Lamarche F, Arnaldi B. A review of classification algorithms for eeg-based brain–computer interfaces. J Neural Eng 2007;4(2):R1.

    Article  Google Scholar 

  26. Löw F, Michel U, Dech S, Conrad C. Impact of feature selection on the accuracy and spatial uncertainty of per-field crop classification using support vector machines. ISPRS Journal of Photogrammetry and Remote Sensing 2013;85:102,119.

    Article  Google Scholar 

  27. Maddula R, Stivers J, Mousavi M, Ravindran S, de Sa V. Deep recurrent convolutional neural networks for classifying p300 bci signals. GBCIC; 2017.

  28. Mendonça EBS, Muniz LF, de Carvalho Leal M, da Silva Diniz A. Applicability of the p300 frequency pattern test to assess auditory processing. Brazilian Journal of Otorhinolaryngology 2013;79(4):512–521.

    Article  Google Scholar 

  29. Milne E, Dunn SA, Freeth M, Rosas-Martinez L. Visual search performance is predicted by the degree to which selective attention to features modulates the erp between 350 and 600 ms. Neuropsychologia 2013; 51(6):1109–1118.

    Article  Google Scholar 

  30. Nassif AB, Shahin I, Attili I, Azzeh M, Shaalan K. Speech recognition using deep neural networks: a systematic review. IEEE Access 2019;7:19143–19165.

    Article  Google Scholar 

  31. Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, Lin Z, Desmaison A, Antiga L, Lerer A. Automatic differentiation in pytorch. NIPS-W; 2017.

  32. Patricia Evans Sailaja Golla MAM. Autism spectrum disorders: clinical considerations, 5th ed. London: Academic Press; 2015.

    Google Scholar 

  33. Peng J, Huang Z, Cheng J. A deep recurrent network for web server performance prediction. 2017 IEEE second international conference on data science in cyberspace (DSC). IEEE; 2017. p. 500–504.

  34. Prechelt L. Early stopping-but when? Neural networks: tricks of the trade. Springer; 1998. p. 55–69.

  35. Rakotomamonjy A, Guigue V. Bci competition iii: dataset ii- ensemble of svms for bci p300 speller. IEEE Trans Biomed Eng 2008;55(3):1147,1154.

    Article  Google Scholar 

  36. San Martín R. Event-related potential studies of outcome processing and feedback-guided learning. Frontiers in Human Neuroscience 2012;6:304.

    Article  Google Scholar 

  37. Santamaría-Vázquez E, Martínez-Cagigal V, Gomez-Pilar J, Hornero R. Deep learning architecture based on the combination of convolutional and recurrent layers for erp-based brain-computer interfaces. XV mediterranean conference on medical and biological engineering and computing – MEDICON 2019. In: Henriques J, Neves N, and de Carvalho P, editors. Cham: Springer International Publishing; 2020. p. 1844–1852.

  38. Schwartz S, Kessler R, Gaughan T, Buckley AW. Electroencephalogram coherence patterns in autism: an updated review. Pediatric Neurology 2017;67:7–22. https://doi.org/10.1016/j.pediatrneurol.2016.10.018. https://www.ncbi.nlm.nih.gov/pubmed/28065825.

    Article  Google Scholar 

  39. Shih JJ, Krusienski DJ, Wolpaw JR. Brain-computer interfaces in medicine. Mayo Clin Proc 2012;87(3):268–279. https://doi.org/10.1016/j.mayocp.2011.12.008.

    Article  Google Scholar 

  40. Sowndhararajan K, Kim M, Deepa P, Park SJ, Kim S. Application of the p300 event-related potential in the diagnosis of epilepsy disorder: a review. Scientia pharmaceutica 2018;86(2):10.

    Article  Google Scholar 

  41. Still K, Rehfeldt RA, Whelan R, May R, Dymond S. Facilitating requesting skills using high-tech augmentative and alternative communication devices with individuals with autism spectrum disorders: a systematic review. Research in Autism Spectrum Disorders 2014;8(9):1184–1199.

    Article  Google Scholar 

  42. Sur S, Sinha V. Event-related potential: an overview. Industrial Psychiatry Journal 2009;18(1):70.

    Article  Google Scholar 

  43. Tal O, Friedman D. 2019. Recurrent neural networks for p300-based bci. arXiv:1901.10798.

  44. Wang J, Barstein J, Ethridge LE, Mosconi MW, Takarae Y, Sweeney JA. Resting state eeg abnormalities in autism spectrum disorders. Journal of Neurodevelopmental Disorders 2013;5(1):24–24. https://doi.org/10.1186/1866-1955-5-24. https://www.ncbi.nlm.nih.gov/pubmed/24040879.

    Article  Google Scholar 

  45. Wing L. 1996. Autistic spectrum disorders.

  46. Woodman GF. A brief introduction to the use of event-related potentials in studies of perception and attention. Attention, Perception & Psychophysics 2010;72(8):2031–2046. https://doi.org/10.3758/APP.72.8.2031. https://www.ncbi.nlm.nih.gov/pubmed/21097848.

    Article  Google Scholar 

  47. Yamashita R, Nishio M, Do RKG, Togashi K. Convolutional neural networks: an overview and application in radiology. Insights into Imaging 2018;9(4):611–629.

    Article  Google Scholar 

  48. Yin W, Kann K, Yu M, Schütze H. 2017. Comparative study of cnn and rnn for natural language processing. arXiv:1702.01923.

  49. Zhao H, Yu S, Prinable J, McEwan A, Karlsson P. A feasible classification algorithm for event-related potential (erp) based brain-computer-interface (bci) from ifmbe scientific challenge dataset. Mediterranean conference on medical and biological engineering and computing. Springer; 2019. p. 1861–1868.

Download references

Funding

Neurodisability Assist Trust

Cerebral Palsy Alliance

Author information

Authors and Affiliations

  1. University of Sydney, Camperdown, NSW, 2006, Australia

    Haifeng Zhao, Yikai Yang, Petra Karlsson & Alistair McEwan

  2. Cerebral Palsy Alliance, Sydney, NSW, Australia

    Petra Karlsson

Authors
  1. Haifeng Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yikai Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Petra Karlsson
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alistair McEwan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Haifeng Zhao or Alistair McEwan.

Ethics declarations

Conflict of interests

The authors declare that they have no conflict of interest.

Additional information

Availability of data and material

The dataset was from MEDICON 2019. It is available online (https://www.medicon2019.org/scientific-challenge/). The true labels will be published soon according to the email from MEDICON 2019 Scientific Challenge Committee.

Code availability

The code can be accessed by requesting from corresponding authors.

Main contributions

The main contributions of this paper are a novel solution for event-related potentials classification and a discussion of the processing methods’ impacts, including filters and deep learning algorithms.

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

Table 4 Single session results - EEGNet vs. EEGNet+RNN
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, H., Yang, Y., Karlsson, P. et al. Can recurrent neural network enhanced EEGNet improve the accuracy of ERP classification task? An exploration and a discussion. Health Technol. 10, 979–995 (2020). https://doi.org/10.1007/s12553-020-00458-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12553-020-00458-x

Keywords

Search

Navigation