Automated detection of driver fatigue from electroencephalography through wavelet-based connectivity
Introduction
In recent years, driver fatigue is becoming a severe problem which increases the number of accidents. This is mainly appeared due to the long driving routes and loss of mental and physical performances, which leads to the impairment of recognition and perception abilities as well as the loss of vehicle control [1]. Therefore, it is essential to detect driver fatigue with a warning system to reduce accidents in our modern societies.
Recently, many subjective and objective techniques are developed to detect fatigue. Objective methods could be considered in three categories of vehicle driving parameters such as speed, steering wheel rotation [2], analysis and acquisition of driver physiological parameters such as Electrocardiogram (ECG) [3], Electrooculogram (EOG) [4], Electromyogram (EMG) [5], and EEG [1,[6], [7], [8]] and also driver behavior characteristic such as eye blink rate using video imaging techniques [9,10]. In another study, Mulhall et al. [11] proved that in shift workers, pre‐drive ocular parameters show the promise of predicting driving impairment in a naturalistic environment. The proposed method in this article cannot be implemented online while driving, and the samples examined cannot be generalized to all drivers.
The most common and effective method among all mentioned objective ones is based on EEG signals to identify the fatigue status and operation of the brain. There are many pieces of researches discussed fatigue and alert states using these signals. Ma et al. [12] designed a PCANet model to detect driving fatigue. The network model integrates PCA and deep learning to extract features. SVM and KNN classification is then used. In this paper, no analysis has been performed on the reduction of EEG channels, and 32 EEG channels have performed the fatigue detection process. Fujiwara et al. [13] proposed a driver drowsiness detection algorithm based on heart rate variability (HRV) analysis using the ECG. This study was validated in comparison with EEG-based sleep scoring.
Fatigue detection based on physiological signals is discussed in many kinds of research that are mainly focused on feature extraction and classification methods. The most effective methods to extract features based on EEG signals are as follows:
waveform and statistics from time series [14], power spectral density [15], phase synchronization [1], wavelet coefficients [16], functional connectivity [17], and visually evoked potentials (VEP) [18]. The absolute power and relative power features have often been used to investigate driver fatigue detection using EEG signals [19]. In a study by Lee et al. [19], from 16 channels of EEG data, 12 types of energy parameters were calculated. Then the kernel principal component analysis (KPCA) method was used to select the effective number of electrodes. In another study by Luo et al., an adaptive multi-scale entropy feature extraction algorithm was used for fatigue driving detection [20]. There are also some types of researches based on image processing, such as eye movement and blink tracking to monitor fatigue hopefully [21].
In most important and complex tasks, including many brain regions, network theory is used effectively to study the brain functional organization [22]. Functional connectivity networks are representing the existing patterns of different brain states as well as the interactions between different brain states using a high temporal solution. Many scientists study the alterations between alert and fatigue states by applying different network methods. As an illustration, Lee et al. [23] used the phase lag index (PLI) to measure the functional connectivity strength in the Alpha band. They were concluded that the main nodes in fatigue detection are placed in the frontal cortex. Liu et al. [24], also used the direct transfer function (DTF) to have more research in the strength and directionality of the frontal cortex. Classification based on network connectivity is a successful technique being studied in a few papers [25,26].
Various techniques in time and frequency domains are implemented to extract the features and also to diagnose changes in signal spectral, which fast Fourier transform (FFT) is the most popular one [27,28]. This technique is an invaluable method in analyzing stationary signals; however, it is not a suitable method for local feature extraction with time and frequency analysis at the same time. Physiological signals such as EEG are non-stationary, and the methods based on this technique are not useful to detect fatigue; therefore, other methods such as wavelet transform are applied to EEG signals. Wavelet transform is a common way to convert signals into frequency bands and has been used many times for fatigue detection systems [1,17,29] and EEG-based BCI systems [[30], [31], [32], [33], [34]].
With the aim of finding an efficient way to achieve excellent driving fatigue detection, the spatial reorganization of the fatigue-related-time-frequency was investigated by calculating the wavelet-based connectivity between alert and fatigue states in our study. Through brain connectivity and network analysis methods in multi-channel EEG recordings, the potential for intra-brain interaction has successfully presented [1,[35], [36], [37], [38]]. Comparing connectivity properties under alert and fatigue states could be one of the valuable methods in driving fatigue detection. Human neural activity during driver fatigue is a challenging topic, and the complex nature of the fatigue-related neural mechanism needs further research. This paper presents a new expert automatic system for detecting fatigue while driving based on EEG signals. The proposed method is based on the connectivity between the EEG electrodes in the inside and between each wavelet coefficients. The GCMI feature is proposed for the calculation of connectivity, which, based on our knowledge, is a new feature in driving fatigue detection systems. The optimum features in the train data are selected by the t-test, and the classification of related trails of the fatigue and alert was done through SVM. In this paper, a new method for optimizing the number of channels is introduced to increase processing speed and reduce computational load.
Section snippets
Materials and methods
In this paper, a new method to detect driving fatigue using the EEG signal is introduced. A schematic summary of the proposed method is presented in Fig. 1. The continuous signals in both fatigue and alert conditions are sectioned to 1-s trials, and all fatigue detection process is evaluated on these trials. The data is filtered into 1–40 Hz frequency band, and the signal is then pre-processed automatically. The signal is decomposed to frequency bands using stationary wavelet transform (SWT),
Results
In this study, GCMI values were used to calculate connectivity between EEG channels. The EEG signal is divided into one-second trials in two groups of alert and fatigue. Trials were decomposed to the wavelet coefficient using SWT and the GCMI values between EEG channels, which are calculated in the inside, and between each wavelet coefficient. After selecting the features, the classification was done through SVM classifier with linear kernel. A summary of the proposed method is given in Fig. 1.
Discussion
The three leading causes of road accidents described in recent studies [50] are distraction, fatigue, and aggressive driver behavior. These factors contribute to more than 90% of all road accidents. The term “fatigue” means a decrease in mental or physical performance and a feeling of drowsiness. For drivers, mental and central nervous exhaustion is the most dangerous type of fatigue, which eventually leads to drowsiness. Fatigue detection is not only applicable to driving but also in other
Conclusion
This paper aims to design an expert and automatic system to detect fatigue during driving using EEG signal analysis. Wavelet transform is an effective method to analyze non-stationary signals such as EEG. The brain connectivity is calculated via GCMI features in and between wavelet coefficients. The optimal features are then selected automatically, and classification is done using SVM. The results presented the efficiency of the wavelet-based GCMI method in fatigue detection; the average
Authorship statement
All contributors who meet authorship criteria are listed as authors in this manuscript, and all authors certify that they have participated sufficiently in work to take public responsibility for explaining the content, including participation in the concept, design, analysis, writing, and revision of the manuscript.
Competing interests
The authors declare that they have no competing interests.
References (79)
- et al.
Driver fatigue transition prediction in highly automated driving using physiological features
Expert Syst Appl
(2020) - et al.
Constructing multi-scale entropy based on the empirical mode decomposition(EMD) and its application in recognizing driving fatigue
J Neurosci Methods
(2020) - et al.
Driver behavior detection and classification using deep convolutional neural networks
Expert Syst Appl
(2020) - et al.
A pre-drive ocular assessment predicts alertness and driving impairment: a naturalistic driving study in shift workers
Accid Anal Prev
(2020) - et al.
Assessment of driver drowsiness using electroencephalogram signals based on multiple functional brain networks
Int J Psychophysiol
(2018) - et al.
Research on fatigue driving detection using forehead EEG based on adaptive multi-scale entropy
Biomed Signal Process Control
(2019) - et al.
Mid-task break improves global integration of functional connectivity in lower alpha band
Front Hum Neurosci
(2016) - et al.
Estimation of the cortical functional connectivity by directed transfer function during mental fatigue
Appl Ergon
(2010) - et al.
Classification of pilots’ mental states using a multimodal deep learning network
Biocybern Biomed Eng
(2020) - et al.
MMSFL-OWFB: a novel class of orthogonal wavelet filters for epileptic seizure detection
Knowl Based Syst
(2018)
Upper limb complex movements decoding from pre-movement EEG signals using wavelet common spatial patterns
Comput Methods Programs Biomed
Granger causality analysis in combination with directed network measures for classification of MS patients and healthy controls using task-related fMRI
Comput Biol Med
Automatic epilepsy detection using wavelet-based nonlinear analysis and optimized SVM
Biocybern Biomed Eng
Schizophrenia detection using MultivariateEmpirical Mode Decomposition and entropy measures from multichannel EEG signal
Biocybern Biomed Eng
The ADHD effect on the actions obtained from the EEG signals
Biocybern Biomed Eng
A passenger reduces sleepy driver’s activation in the right prefrontal cortex: a laboratory study using near-infrared spectroscopy
Accid Anal Prev
Electroencephalography based fatigue detection using a novel feature fusion and extreme learning machine
Cogn Syst Res
Application of multiresolution analysis for automated detection of brain abnormality using MR images: a comparative study
Future Gener Comput Syst
Automated sleep stage identification system based on time-frequency analysis of a single EEG channel and random forest classifier
Comput Methods Programs Biomed
Electroencephalographic study of drowsiness in simulated driving with sleep deprivation
Int J Ind Ergon
EEG-based estimation of mental fatigue by using KPCA-HMM and complexity parameters
Biomed Signal Process Control
Monitoring driver fatigue using a single-channel electroencephalographic device: a validation study by gaze-based, driving performance, and subjective data
Accid Anal Prev
Electroencephalogram and electrocardiograph assessment of mental fatigue in a driving simulator
Accid Anal Prev
Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness
Neurosci Biobehav Rev
Assessment of driving fatigue based on intra/inter-region phase synchronization
Neurocomputing
Feature subset selection using differential evolution and a statistical repair mechanism
Expert Syst Appl
Automatic detection of drowsiness in EEG records based on multimodal analysis
Med Eng Phys
Evaluation of driver fatigue on two channels of EEG data
Neurosci Lett
Driving fatigue recognition with functional connectivity based on phase synchronization
IEEE Trans Cogn Dev Syst
A fuzzy recurrent neural network for driver fatigue detection based on steering-wheel angle sensor data
Int J Distrib Sens Netw
Detection and analysis: driver state with electrocardiogram (ECG)
Phys Eng Sci Med
Driver sleepiness detection from EEG and EOG signals using GAN and LSTM networks
Neurocomputing
Multi-channel EEG recordings during a sustained-attention driving task
Sci Data
A dynamic center and multi threshold point based stable feature extraction network for driver fatigue detection utilizing EEG signals
Cogn Neurodyn
HCF: a hybrid CNN framework for behavior detection of distracted drivers
IEEE Access
Driving fatigue detection from EEG using a modified PCANet method
Comput Intell Neurosci
Heart rate variability-based driver drowsiness detection and its validation with EEG
IEEE Trans Biomed Eng
Driver drowsiness detection based on time series analysis of steering wheel angular velocity
Regression based continuous driving fatigue estimation: towards practical implementation
IEEE Trans Cogn Dev Syst
Cited by (25)
Electromyography and dynamometry in the prediction of risk of falls in the elderly using machine learning tools
2024, Biomedical Signal Processing and ControlA novel approach for cross-subject unfavorable driving state detection by cooperation between phase-based functional connectivity and convolutional neural network
2023, Biomedical Signal Processing and ControlHeartbeat detection from single-lead ECG contaminated with simulated EMG at different intensity levels: A comparative study
2023, Biomedical Signal Processing and ControlReal-time fatigue driving detection system based on multi-module fusion
2022, Computers and Graphics (Pergamon)Citation Excerpt :The first method is to judge the fatigue state based on real-time analysis of the driver’s physiological information. This method mainly analyzes the physiological characteristic parameters of the driver, monitors the body function related data such as electroencephalogram (EEG) [2–4], electrocardiogram (ECG) [5,6], electromyogram (EMG) [7], and electrooculogram (EOG) [8], extracts and analyzes the corresponding body characteristic parameter signals, compares the obtained data with the data of non-fatigue state, and judges whether the driver is tired [9]. The second method is to judge the fatigue state according to the operating conditions of the vehicle [10].
Ensemble feature selection for stable biomarker identification and cancer classification from microarray expression data
2022, Computers in Biology and MedicineCitation Excerpt :Second, the proposed framework is a general one, and we can take as the building blocks other base feature selection algorithms. However, this would raise new issues related to hyperparameter optimization such as how many features to keep (i.e., the thresholding problem) and how to combine the ranked features (i.e., the aggregation problem) [33,41]. This deserves a systematic and comprehensive study for practical guidelines.
Deep Multidilation Temporal and Spatial Dependence Modeling in Stereoscopic 3-D EEG for Visual Discomfort Assessment
2024, IEEE Transactions on Systems, Man, and Cybernetics: Systems