Recent advances in heart sound analysis

We provided a benchmark classifier that relied on relatively obvious parameters extracted from the heart sound segmentation code. For the detailed description of this benchmark classifier, challengers can refer to Liu et al (2016) and Clifford et al (2016). Here we briefly describe how the benchmark classifier is constructed and how it works. First, a balanced database from training set was selected. Then, Springer's segmentation code (Springer et al 2016b) was used to segment heart sound recording. Twenty features were extracted according to the position and waveform amplitude information of the segmented signals. A forward likelihood ratio selection was used to train the binary logistic regression (BLR) model. Finally, seven features were identified as the predictable features, and a derived BLR prediction formula was constructed for normal/abnormal heart sound recordings classification. In a 10 fold cross validation, the constructed BLR model provided a sensitivity of 0.66, a specificity of 0.77 and a Challenge score of 0.71 on the training data. It should be noted that this was not intended to be a good classifier, or properly trained, but merely an example set of code to enable a researcher to understand the mechanics of the submission process, and to provide a simple baseline for Challenge entrants to beat in the early stages of the Challenge.

We also implemented a voting approach to combine together varying numbers of the submitted algorithms (Clifford et al 2016). A simple unweighted voting of using the N best performing final entries from the Challenge, ranked by their score on the training data (to prevent over-fitting on the test scores), was implemented. N was varied from 1 to 48 with tied, absent or no vote was treated as 'normal' type.

A modified accuracy (MAcc) with the combination of sensitivity (Se) and specificity (Sp) for scoring as:

$$\begin{align*} \displaystyle {\mathit{MAcc}} = \frac{\mathit{Se}+{\mathit{Sp}}} {2}. \nonumber \end{align*}$$

The score on the complete test set determines the ranking of the entries. For details on the scoring mechanism please see Liu et al (2016) and Clifford et al (2016).

The algorithm proposed by Abdollahpur et al (2017) used a novel cycle quality assessment (CQA) method for assessing the signal quality of the segmented cardiac cycle. Features were extracted only on the cycles which higher signal quality and superior segmentation. The method achieved a MAcc of 0.8263 in the last phase of the challenge (Abdollahpur et al 2016).

The authors note that the recordings were down sampled to 1 kHz and filtered by the fourth order Butterworth high pass (25 Hz) and low pass (600 Hz) filters. Spikes were removed using the algorithm proposed by Schmidt et al (2010). Then, after the heart sound segmentation with Springer's HSMM model (Springer et al 2016b), correctly segmented heart cycles without excessive noise or spikes were selected for further feature extraction process using a novel CQA method detailed in Abdollahpur et al (2016). Frequency and amplitude criteria were applied for detecting correctly segmented heart sound cycles. A total of 90 features were calculated from the time domain, time-frequency, perceptual and mel-frequency cepstral coefficient (MFCC) analysis. Before starting the main classification process, the derived 90 dimensional feature vector was mapped to a new feature space by applying a Fishers discriminant analysis. The main classification procedure was then performed using three feed-forward NNs and a voting system among classifiers. A final MAcc score of 0.826 was achieved on the hidden test data.

The algorithm proposed by Homsi and Warrick (2017) used an ensemble based classification with a special consideration for outliers and achieved a MAcc score of 0.801 for the hidden test data in the Challenge.

In this paper, a total of 131 features in time, frequency, wavelet and statistical domains were extracted from the heart sound signals. Outlier signals were detected and separated from those with a standard range using an interquartile range threshold. Then, feature extreme values were given special consideration, and finally features were reduced to the most significant ones using a feature reduction technique. In the classification stage, the selected features either for standard or outlier signals were fed separately into an ensemble of 20 two-step classifiers. The first step of the classifier included a nested set of ensemble algorithms which was cross validated on the training data, while the second step used a voting rule of the class label. The results showed that the proposed method achieved an overall score of 0.9630 for standard signals and 0.9018 for outlier signals on a cross-validated experiment using the training data. This method achieved an overall score of 0.801 on the hidden test set (0.796 sensitivity and 0.806 specificity).

Kay and Agarwal (2017) proposed an algorithm that employed DropConnected neural networks trained on time-frequency and inter-beat features for heart sound classification. This algorithm achieved a MAcc of 0.8520 on the test data, and ranked third in the Challenge (Kay and Agarwa 2016). This paper provides an extensive analysis concerning the profile differences of the open training data, including the recording numbers, recording sensors, unbalanced data and the specific pathology of the recordings.

In this paper, first the heart sounds were segmented using Springer's open-source segmentation algorithm based on a hidden semi-Markov model (HSMM) (Springer et al 2016b). Then, a total of 675 features were extracted from the analysis of continuous wavelet transform (220), MFCC (400), inter-beat behaviour (20 and complexity measures (35). Then, the extracted features were normalized and the dimensionality was reduced to 50 using principal component analysis (PCA). Subsequently, the features were used as the input to a fully-connected, two-hidden-layer neural network, trained by error backpropagation, and regularized with DropConnect. When the algorithm was submitted to be evaluated on the test data, a number of different networks were trained with a range of hyper-parameters and different training sets. The networks are then ensembled based on their scores. The best result obtained by the ensemble of networks, on the test data, was 0.8520, which is the third best performance in the Challenge. The authors also updated their algorithm by excluding the training-e set for training since the recording sensor type for training-e set is different from others. However, a significantly worse score of 0.580 was obtained because 69% of recordings in the test set are from dataset-e indicating that the algorithm is sensitive to the recording type and struggles to generalize from one dataset to another.

Most algorithms for automated analysis of heart sound require segmentation of the signal into the characteristic heart sounds. Langley and Murray (2017) aimed to assess the feasibility for accurate classification of heart sounds on short, unsegmented recordings.

At the first step, initially the 5 second segment (seg 1) at the start of each heart sound recording was analyzed. For some recordings with considerable noise at the start of the recordings, so a repeated 5 s segments (seg 2) with lowest noise was extracted for each recording. Segments were zero-mean but otherwise had no prepossessing or segmentation. Then normalized spectral amplitude was determined by FFT and wavelet entropy was calculated by wavelet analysis ('Gaus4' mother wavelet). For each of these a simple single feature threshold based classifier was implemented and the frequency/scale and thresholds for optimum classification accuracy determined. The analysis was then repeated using relatively noise free 5 s segments (seg 2) of each recording by applying a wavelet entropy measure for signal noise assessment. Spectral amplitude and wavelet entropy features were then combined in a classification tree (Langley and Murray 2016). Detailed results were reported as follows. There were significant differences between normal and abnormal recordings for both wavelet entropy and spectral amplitude across scales and frequency. In the wavelet domain the differences between groups were greatest at highest frequencies whereas in the frequency domain the differences were greatest at low frequencies (12 Hz). Abnormal recordings had significantly reduced high frequency wavelet entropy, suggesting the presence of discrete high frequency components in these recordings. Abnormal recordings exhibited significantly greater low frequency (12 Hz) spectral. Classification accuracy was greatest for wavelet entropy and was further improved by selecting the lowest noise segment (seg 2). Classification tree with the combined features gave an accuracy (not MAcc) of 0.79 ($Sp = 0.80$, $Se = 0.77$). The study demonstrated the feasibility of accurate classification without segmentation of the characteristic heart sounds.

Maknickas and Maknickas (2017) describe the use of mel-frequency spectral coefficients (MFSC) fed to a CNN, and which achieved a MAcc of 0.8415 in the last phase of the Challenge, ranked sixth overall with an unofficial entry. There are existing studies that leverage MFCC analysis for heart sound classification (Chauhan et al 2008). However, the authors claimed that MFSC analysis could outperform MFCC since during the calculation of the MFCC the discrete cosine transform (DCT) projects the spectral energies into a new basis that may not maintain locality.

In this paper the authors describe a process which first splits the training heart sound files into equal numbers of normal and abnormal data files. Then MFSC (i.e. MFCC with no DCT) was calculated for each file, and was cut into frames with width and height of both 128 samples. The difference and second-order difference of the MFSC were also calculated as second and third dimensions of the frame. All frames were normalised. Then CNN was trained to predict the normal/abnormal label for each frame in the file, and used the average of all predicted frame labels as the final label of the file. Finally, the model with best performance was selected during the training phase. Testing on the separate validation set achieved the highest score when using 256 hidden layers for the deep CNN, although the score slightly improved on the selected training data when increasing the number of hidden layers from 128 to 2048. Therefore, the Challenge results were achieved by weights and topology of 256 hidden layers and the final score was 0.842, just 0.018 below the highest score of 0.860. This impressive result indicates the potential of CNNs for future use, but also illustrates how enormous volumes of data are likely to be required to outperform well chosen features and standard classification approaches.

Plesinger et al (2017) proposed an algorithm based on fuzzy logic which they termed 'probability assessment' for normal/abnormal heart sound classification, which achieved a MAcc of 0.8411 in the last phase of the challenge, and was ranked $7{\rm th}$ highest (Plesinger et al 2016). The presented solution produced different results in specific databases. For database-c, it gave 100% sensitivity and specificity in both training and testing. Database-e also provided an extremely high score. However, the method failed to accurately classify database-g and database-i (not present in the training set), where it reported nearly all records as normal. This poor performance with these completely hidden databases indicates the method also struggles to generalize to unseen data.

In their methods, they first derived amplitude envelopes in five frequency bands low frequency (LF, 15–90 Hz), middle frequency (MF, 15–90 Hz), high frequency (HF, 100–250 Hz), super frequency (SF, 200–450 Hz) and ultra frequency (UF, 400–800 Hz) were computed using an FFT band-pass filter and Hilbert transformation. Then invalid time segments were checked for each 1 s window. Then heart sounds S1 and S2 were detected using amplitude envelopes in the LF band. The averaged shapes of the S1/S2 pair were computed from amplitude envelopes in all five bands (15–90 Hz; 55–150 Hz; 100–250 Hz; 200–450 Hz; 400–800 Hz). A total of 228 features were extracted from the statistical properties and the symmetry of the averaged shapes, and the independent of S1 and S2 detection. Then the features are processed using logical rules and probability assessment based on histograms, and a fuzzy logic-like approach, which they termed 'PROBAfind'. This software contains a function suggesting a feature with the best impact on the sum of final sensitivity and specificity, and can be used as a semi-automatic feature selection method. The authors found 53 features were selected as the normal/abnormal/unsure classification. A final score MAcc of 0.8411 achieved on the hidden test data ($7{\rm th}$ place in the Challenge), indicating that the performance of probability assessment is comparable to other machine-learning approaches. However, it the human oversight required and long training time required for this approach is a significant limitation and may have led to the lack of generalization.

Whitaker et al (2017) proposed an algorithm combining sparse coding and time domain features for normal/abnormal heart sound classification, which achieved a MAcc of 0.807 in the Challenge (Whitaker and Anderson 2016). This study introduced sparse coding as a tool for unsupervised feature extraction in heart sound classification, and was also the first to use matrix norm sparse coding in a practical classification setting for heart sounds. Previous work by Poian et al (2017) has demonstrated the utility of this technique, using compressed sensing for atrial fibrillation detection in the ECG. As the first step, Whitaker et al used Springer's HSMM segmentation code (Springer et al 2016b) to separate each audio file into five arrays of smaller audio segments. The first four arrays contained a list of all S1, systole, S2 and diastole sounds respectively. The fifth array contained copies of the full heart cycles, starting at the start of the S1 state and ending at the last sample in diastole. Each state or sound segment was converted to the frequency domain with an N-point FFT and sparse coding was applied on the aforementioned five data matrices as a form of unsupervised feature extraction. In sparse coding, frequency-domain data is decomposed into a dictionary matrix and a sparse coefficient matrix. The dictionary matrix represents statistically important features of the audio segments and becomes fixed after training. In effect it represents the basis functions. The sparse coefficient matrix is a mapping that represents which features are useful in each segment. Working in the sparse domain, the authors trained SVMs for each audio segment, as well as the full cardiac cycle. Then a sixth SVM was trained to combine the results from the preliminary SVMs into a single binary label for the entire heart sound recording. Compared with the CinC paper in Whitaker and Anderson (2016), this paper presented two novel modifications. The first modification involved a matrix norm in the dictionary update step of sparse coding to encourage the dictionary to learn discriminating features from the abnormal heart recordings. The second combined the sparse coding features with twenty time domain features described in Liu et al (2016) in the final SVM classification stage. The authors demonstrated an improved cross-validated MAcc of 0.893 ($Se = 0.901$ and $Sp = 0.885$). However, improved version did not generate a higher score on the hidden test data than their challenge's score. A new score MAcc of 0.803 (0.801 sensitivity and 0.806 specificity) in this follow-up phase was achieved.

This study showed that sparse coding is an effective way to define spectral features of the cardiac cycle and its sub-cycles for the purpose of classification. In addition, it demonstrated that sparse coding can be combined with additional feature extraction methods to improve classification accuracy. Further work may incorporate additional features to improve the classification accuracy or robustness to novel data and noise.

A hidden Markov model (HMM)-based approach has received increased interest for heart sound segmentation due to its robustness on processing noisy recordings, particularly when incorporating physiological models. The focus of this article was on evaluating the performance of the recently published logistic regression-based HSMM heart sound segmentation method (Springer et al 2016b), which was open sourced for the Challenge. By using a wider variety of heart sound data in the PhysioNet/CinC Challenge 2016. The HSMM-based model was trained on the training-a dataset only (per the original work) and was tested on all other separate test datasets, which comprised 102 306 heart sounds. The results confirm the high accuracy of the HSMM-based algorithm with an average F₁ score of 98.5% for segmenting S1 and systole intervals and 97.2% for segmenting S2 and diastole intervals. The described evaluation framework, combined with the largest collection of open access heart sound data, provides essential resources for researchers who need to test their algorithms with realistic data and share reproducible results.