Design of Abnormal Heart Sound Recognition System Based on HSMM and Deep Neural Network

Introduction

The heart sound signal is generated by the vibration of the whole heart structure caused by the opening and closing of the heart valves.¹ It is an important physiological signal in the human body and has an important role in the prevention and diagnosis of cardiovascular diseases.² At present, domestic and international hospitals still use manual auscultation in the diagnosis of heart sound, which is susceptible to both the interference of the external environment and personal experience. In addition, with the development of smart wear, a new requirement for remote real-time monitoring of heart sound signal has been put forward, and the traditional way of heart sound auscultation is unable to meet this requirement.

At present, the main research on heart sounds is extracting heart sound feature, and heart sound classification models mostly use convolutional neural network (CNN) algorithms. For example, Chaowen Tan et al^3,4 proposed data acquisition of more than 1000 heart sound fragments in the time domain using Meier coefficients, combining the collected signal data into two-dimensional feature samples, and finally using convolutional neural networks to identify these heart sound signals; Noman et al⁵ combined Markov switching autoregressive (MSAR) model with switched linear dynamic system (SLDS) to build a heart sound segmentation model, which can accurately segment heart sounds and is convenient for subsequent heart sound recognition applications; Xingzhi Wang et al⁶ used stacked subband signal envelopes to construct time-frequency features and then used CNNs as classification models; Zeenat Tariq et al⁷ proposed a feature-based fusion network to transform audio data into image feature vectors to construct three optimal CNNfusions for heart sound classification; Meng Li-Nan et al⁸ used sliding window and Meier coefficients for feature extraction of heart sound signals to form a heart sound feature map, and then used CNNas a classification model; Liu Wei-Wei et al⁹ made improvements on the classification model, but also used the combination between CNNand long short-term memory (LSTM) artificial neural network.

Although the heart tone feature extraction algorithms used in the above studies are different, they all convert the heart tone feature vectors into two-dimensional vectors and then put into the convolutional neural network. This is because CNN is originally used mainly for two-dimensional image recognition, thus, when processing one-dimensional data, it is often necessary to expand the dimension of the data, which leads to an increase in the amount of data input, thus increasing the resource consumption of the processing computer CPU and reducing the speed of model abnormal heart sound recognition, and this algorithm is not suitable for deployment on mobile smart devices with small computing resources. Therefore, it has good practical value to find a heart sound signal processing method suitable on mobile smart devices.

To solve the above problems, a dataset from the official “challenge heart sound” database was used.¹⁰ A minimal amount of heart sound feature input was performed using a deep neural network structure. An analytical model is obtained for fast training to identify abnormal heart sound signals.

Exemption from ethics was obtained from the Medical Ethics Committee of Hubei University of Science and Technology.

Overall System Design

As Figure 1 shows the structure of the above analysis model, the heart sound recognition system in this paper is mainly divided into two modules: data processing and neural network model. The first module is the data processing module, whose main functions are divided into data preprocessing, HSMM heart sound signal segmentation and heart sound signal feature value extraction. Among them, the data preprocessing is mainly to de-noise the input heart sound signal. The HSMM is responsible for the accurate segmentation of the heart sound signal into four segments: first heart sound, diastolic, second heart sound, and systolic. The feature extraction is to extract the feature data of the heart sound signal in terms of time and amplitude of the segmented four heart sound segments. Then, these feature data are sent to the trained neural network model. Once the neural network model receives the feature data, it will analyze the data to identify the abnormal heart sound signal and output the analysis results.

Figure 1 Overall system structure.

By using python language and with the help of TensorFlow 2.0 developed by Google in 2019, this paper can quickly build deep neural networks and train and test deep neural network models. Then, the data processing module uses MATLAB (2016 version) to process the data and packages the processing program as a dynamic link library that can be called directly in python.

Data Processing

Heart Sound Signal Dataset

The dataset used in this paper comes from the official heart sound database provided by the 2016 CHALLENGE HEART SOUND challenge. The database contains a total of 3240 heart sound clips from volunteers around the world. The quality of the heart sound clips varies due to the different health conditions of the volunteers and the different environments in which the heart sound was collected. Some of these heart sound clips are not even identifiable, and these unidentifiable heart sound clips are also classified as abnormal heart sound. These heart sound clips are usually between 5 and 120 seconds in length, and each heart sound clip has been labeled by experts, so these heart sound data are authoritative. The details of this dataset are shown in Table 1.

Table 1 Data Set

Heart Sound Signal Preprocessing

The quality of heart sound in this dataset varies. Therefore, the heart sound signal needs to be de-noised before the heart sound segmentation. At present, the signal de-noise processing methods commonly used are low-pass filter, high-pass filter, band-pass filter, Fourier transform, short-time Fourier transform and other methods. These methods often filter out the useful signal when removing noise, while wavelet transform can greatly retain the detailed information of the signal while filtering out the noise, so it is widely used for signal de-noising processing. In this paper, according to the related research, db6 wavelet is used as wavelet base to perform 6-layer decomposition of heart sound signal, and soft thresholding is used to de-noise the decomposed signal.^11,12

Heart Sound Signal Segmentation

The heart sound signal contains important physiological information of human heart. The heart sound signal consists of one heartbeat cycle, and each heartbeat cycle is about 600ms – 1s. A heart cycle contains four parts: the first heart sound (S₁), the second heart sound (S₂), the third heart sound (S₃) and the fourth heart sound (S₄). Among these four parts, S₁ and S₂ are the most critical heart sound, also known as the basic heart sound. S₃ and S₄, on the other hand, exist in specific populations and are not very relevant to the study. Within a cardiac cycle, the interval between S₁ and S₂ is the systolic period, and the interval between S₂ and the next cardiac cycle S₁ is the diastolic period.^13,14

Heart sound signal segmentation is a very important step in the heart sound abnormality recognition system. The effect of heart sound signal segmentation will directly affect the later accuracy of heart sound classification. The current heart sound segmentation methods include feature extraction analysis-based segmentation, machine learning-based segmentation, envelope-based segmentation and Hidden Markov model (HMM)-based segmentation.¹⁵ Among them, the heart sound signal segmentation method based on Hidden Markov model (HMM) requires very little computational effort in training and recognition, because of its ability to describe the smoothness and variability of short-time smooth signals well, and its dynamic modeling capability. Therefore, HMMs are very effective in dealing with speech-like signal analysis.¹⁶

HMM is a statistical model with implicit unknown parameters on time series. The HMM describes the relationship between the implied sequence and the corresponding observable sequence, a class of double discrete random sequences.¹⁷ As shown in Figure 2, h₁, h₂ … h_n-1, hn are the implied sequences of the HMM, which are unknowable. v₁, v₂ … v_n-1, v_n are the observable sequences of HMM, which are knowable, and the rules between the implied sequences and the corresponding observed sequences are knowable, so we can then derive the implied sequences from the observed sequences.

Figure 2 HMM process.

The HMM is mainly composed of three parameters, namely, the initial state, the state transfer probability matrix and the observation probability matrix, and its model equation is

(1)

In the above equation, A is the probability transfer matrix, π is the initial state, and φ is the observation probability matrix.

In the HMM, the state of the implied sequence transferred to the next moment is only related to the state of the implied sequence at this moment, but not to the residence time of the implied sequence with the observable sequence. This means that the probability of transitioning to the next state is the same no matter how long the model is in the same state, so there are some limitations of HMM in modeling.

In order to solve the limitations of HMM, hidden semi-Markov models (HSMM) have been proposed. This model makes up for the shortcomings of HMM by adding state residence time to the model. When judging the state transfer, the HSMM will only perform the state transfer when the joint probability distribution of the state jump time satisfies certain conditions, otherwise it still maintains the current state.¹⁸ Compared with HMM, HSMM adds a time parameter, which improves the accuracy and analysis ability of the model. The equation of the HSMM is

(2)

In the above equation, A is the probability transfer matrix, π is the initial state, φ is the observation probability matrix, and D is the state residence time probability distribution.

A method based on HSMM for heart sound segmentation proposed by Springer David et al¹⁹ is used in this paper. As shown in Figure 3, the ECG signal R-wave position corresponds to the heart sound signal S1 position and the ECG signal T-wave end position corresponds to the heart sound signal S2 position in the ECG signal and the heart sound signal acquired simultaneously is used to segment the heart sound signal accurately. After that, the accurately segmented heart sound signal is used as the training set to train the HSMM. Finally, the trained HSMM is then used to segment the heart sound signal accurately.

Figure 3 ECG signal and PCG relationship diagram.

Feature Extraction

After segmenting the heart sound signal, this paper obtains the feature values from the time of the heart sound signal divided into four segments and amplitude. In terms of time, the mean and standard deviation of each heart sound cardiac cycle, the mean and standard deviation of S₁ interval, the mean and standard deviation of S₂ interval, the mean and standard deviation of systole, the mean and standard deviation of diastole, the mean and standard deviation of systole to cardiac cycle ratio, the mean and standard deviation of diastole to cardiac cycle ratio, and the mean and standard deviation of systole to diastole ratio are obtained for each segment. In terms of amplitude, the mean and standard deviation of the absolute amplitude ratio between S₁ and systole and the mean and standard deviation of the absolute amplitude ratio between S₂ and diastole are selected in this paper.

Deep Neural Network Model Construction

In this paper, a number of neural networks are quickly built using TensorFlow2.0. The model is trained using the training set train and validation set valid in Table 1, and then the trained model is tested using the test set test. In this paper, the performance strengths and weaknesses of each neural network are evaluated by loss function and accuracy. These evaluations are then used to finally select the neural network structure. The experimental procedure is as follows:

Step 1: Find the optimal number of hidden layers of the neural network. In this paper, we borrow the selection method of neural network in the construction of heart sound intelligent analysis platform by Yurno Yang et al,²⁰ and construct neural networks with 1–5 hidden layers, respectively, so that the number of neurons in each hidden layer is the same, and then find the optimal number of layers by comparing with the loss function and accuracy. Considering the possible influence of the number of neurons on the loss function and accuracy, the number of neurons in each layer is set to 50 and 100 for 1–5 hidden layers, respectively. Experimenting separately on neural networks containing the above two numbers of neurons two data tables are obtained. By comparing the two tables, we can determine the most suitable number of hidden layers. The data obtained from the tests are shown in Tables 2 and 3.

Table 2 Number of Neurons in Hidden Layer is 50 Test Data

Table 3 Test Data with 100 Hidden Neurons

From Tables 2 and 3, we can see that when the number of hidden layers is 1, the test accuracy is the highest, but the error is also the largest, so it is not suitable to be considered as the number of hidden layers. When the number of hidden layers is 3, the loss is the smallest and the test accuracy is also higher, so the neural network with three hidden layers is chosen as the focus of the next study.

Step 2: According to the conclusion of the first step, three neural network structures with hidden layers of 20–100-200, 100–200-20, and 200–100-20 are designed, and these three neural network structures are set as A_1, A₂, A₃. The same training set train_loss and validation set valid_loss are used to train the three neural networks, and the trained models are tested with the test set test. The training data and test data are shown in Figure 4A–C and Table 4. From Figure 4, it can be found that the result of A₂ shown in Figure 4B converges most quickly in the loss function and the loss is smallest, and the test accuracy is the highest compared with the results of A₁ and A₃ shown in Figure 4A and C. Therefore, A₂ is chosen to train the final anomalous heart sound recognition model in this paper.

Table 4 Test Data of Three Neural Network Structures

Figure 4 Loss function and test accuracy of different neural network structures. (A–C) denote the loss function and test accuracy of three neural network structures with designed hidden layers of 20–100-200, 100–200-20, and 200–100-20, respectively.

Since the model recognition speed is related to the amount of data processed by the computer CPU, the less data the CPU needs to process, the faster the model recognition speed will be. Therefore, in this paper, the difference in recognition speed between the model designed in this paper and the mainstream CNNclassification model is verified to determine the difference between the two recognition algorithms in terms of CPU computing resources consumption and anomaly recognition speed. This paper calls the model trained by A₂ and the model trained by CNN to conduct 10 test experiments on the test set. The input feature vectors of the training set and the test set used in CNN will be expanded before the neural network is input.

The test results are shown in Tables 5 and 6. The maximum and minimum values are removed from the two tables, respectively, and the average value of the remaining 8 test results is found to be about 5.32 s for the average recognition speed using CNN, and about 0.17 s for the model trained by the algorithm in this paper, and the algorithm in this paper is 31 times faster than the CNN algorithm in terms of recognition speed. Therefore, it can be concluded that the neural network model trained by the algorithm in this paper has faster recognition speed and occupies less CPU computing resources than the current mainstream method CNN abnormal heart sound recognition algorithm.

Table 5 Recognition Speed of the Designed Model on the PCG Test Set

Table 6 Recognition Speed of CNN on the PCG Test Set

Conclusion

This system uses the powerful data processing function of MATLAB to process the raw heart sound signal, and packages the signal processing program into a MATLAB dynamic link library. The MATLAB dynamic link library can then be called directly in python to the trained deep neural network model for abnormal heart sound signal recognition. In the data processing function, the HSMM is used to accurately segment the heart sound signal, and 20 feature values are extracted for the segmented heart sound signal in terms of time and amplitude. This feature extraction method reduces the amount of data input to the deep neural network model and facilitates faster recognition and processing by the deep neural network model. In the neural network model, this paper conducts several experiments to find an optimal neural network structure for training, and obtains the anomaly heart sound recognition model. The model is also compared with the current mainstream abnormal heart sound classification model CNN classification model for experiments, and the results show that this classification model greatly outperforms the CNN classification model in recognition speed. The whole system is relatively streamlined and is expected to be deployed in embedded mobile devices, which is meaningful for telemedicine monitoring research. In future research, the abnormal heart sound recognition model can also be improved to further enhance the generalization ability and recognition accuracy of the system.