Automatic Sleep Stage Classification of Children with Sleep-Disordered Breathing Using the Modularized Network

Introduction

Sleep-disordered breathing (SDB) represents a spectrum of breathing disorders ranging from primary snoring (PS) to obstructive sleep apnea (OSA) that disrupts nocturnal respiration and architecture of sleep, which is highly prevalent in children who are at the critical stage of growth and development.¹ It is meaningful for early diagnosis and intervention because SDB was verified to be associated with the functioning of various organs, including immune responses, cardiovascular function, and neurocognitive function.²

The mobility of SDB in children who suffer from snoring, mouth breathing, or apnea ranged from 7.9 to 13.4% as measured according to the Pediatric Sleep Questionnaire (PSQ).^3,4 Based on guidelines provided by the American Academy of Paediatrics in 2012, the morbidity due to OSA in children is 1 to 5%.⁵ Since overnight polysomnography (PSG) remains the gold standard for diagnosing the severity of SDB, diagnostic efficiency is largely dependent on the availability and accessibility of this technique.

Sleep stage classification is the first step for data analysis in PSG, according to the strict criteria proposed by the American Academy of Sleep Medicine (AASM).⁶ It takes nearly 1 to 2 hours for technicians to identify sleep stages manually. However, intra-rater reliability (IRR) of classification is also known to be the subject of considerable variability. The SIESTA database arising from an EU-funded project found that the overall agreement for the AASM standard was 82.0% (ĸ = 0.76) based on an epoch-by-epoch comparison.⁷ The accurate and user-friendly sleep staging system would assist sleep experts and provide critical clinical utility.

Because of time consuming nature and labor intensiveness of the manual method, several methods for automatic scoring of long-term sleep data have been researched in the past decades. The accuracy of the models mostly ranges from 78–90%.^8–12 Development and evaluation of software dedicated to automatic sleep staging (AS) face several issues, among which are: (1) EEG has a low signal-to-noise ratio (SNR), as the brain activity measured is often covered by multiple sources of environmental, physiological, and activity-specific noise called “artifacts”; (2) the generalization capabilities of models need to be further verified, for patients of different ages, pathophysiological, and treatment in the real world.^13,14

Behavioral and physiological characteristics of sleep in normal children vary significantly from sleep in adults.¹ There is a dynamic changing process in the frequency and amplitude of the characteristic waves in different ages. Considerable differences occur within and between individual children. How to classify the sleep stages in children of different ages correctly concerns the evaluation of sleep efficiency and management of pediatric sleep-related disorders.

An automated deep neural network which has achieved human-level annotation performance with an average accuracy of 81.81% was proposed by using a multi-model integration strategy with multiple signals in our laboratory.⁸ Based on unfiltered EEG in a large sample of children with sleep-disordered breathing, our study was devoted to developing an automatic sleep stage analysis model with good generalizability in the clinical setting.

Patients and Methods

Study Datasets

The study was conducted under the principles stated in the Declaration of Helsinki and approved by the Institutional Review Board (IRB) of the Beijing Tongren Hospital (TRECKY2017-032). Written informed consent was obtained from each parent of the children for inclusion in the study and the use of their medical records. According to the IRB’s decision, this study used public datasets of sleep-EDF for model verification without obtaining patients’ informed consent.

Clinical Data

We recruited children in the Beijing Tongren Hospital between January 2017 and June 2021. The inclusion criteria were as follows: (1) 2 to 18 years old; (2) snoring more than 3 days every week; (3) total sleep time more than 6 hours; (4) children’s parents voluntarily participated in the study and signed informed consent forms. The exclusion criteria were as follows: (1) cannot cooperate to complete overnight polysomnography; (2) PSG recording integration failed; (3) PSG recordings could not be analyzed by technicians because of a large number of artifacts.

Sleep-EDF Database

For validating the generalization of the model, we used expanded version of the sleep-EDF database in which PSG recordings lasting approximately 20 hours or 9 hours each were collected from healthy subjects and some who had mild difficulty falling asleep after temazepam intake but were otherwise healthy, including sleep-EDF-13 (61 PSG recordings) and sleep-EDF-18 (197 PSG recordings).¹⁵ We combined the N3 and N4 according to the R&K rule into N3 with the AASM guidelines. For comparison with other articles, we input the integral PSG data with long periods of wake and only included 30 minutes of the wake before and after sleep.

Overnight Polysomnography (PSG)

The PSG was performed using the different computerized data collection systems of Compumedics S series (Compumedics Inc, Australia) and Alice 6 (Phillips Inc, America), including EEG(C3/A2, C4/A1), EOG (ROC, LOC), EMG, electrocardiogram (ECG), nasal and oral cannula pressure, recording of respiratory (thoracic and abdominal) movements, and pulse oximetry for oxygen saturation (SpO2).

The highly trained, experienced (more than 10 years) PSG technician scored sleep stages and respiratory events in a 30s epoch following the AASM guidelines (2012).⁶ It is classified into one of the following five categories: (1) Wake (W), (2) Non-rapid eye movement stage 1 (N1), (3) Non-rapid eye movement stage 2 (N2), (4) Non-rapid eye movement stage 3 (N3) or (5) Rapid eye movement stage (REM). Sleep stages of N1 and N2 are also called shallow sleep, and N3 is called deep sleep. In our study, we additionally classified sleep stages into 4 stages (W vs shallow sleep vs deep sleep vs REM), 3 stages (W vs non-rapid eye movement stage vs REM), and 2 stages (W vs Sleep).

The obstructive apnea-hypopnea index (OAHI) was defined as the number of apnea and hypopnea events per hour of sleep and was used to indicate the severity of SDB: PS (OAHI<1); mild OSA (1≤OAHI<5); moderate OSA (5≤OAHI<10); severe OSA (OAHI≥10).^16,17

Data Processing

Signal Preprocessing

The frequency of the PSG data in the A6 dataset was 200 Hz while it is 256 Hz in the Compumedics dataset. To facilitate subsequent processing and retain necessary information of the signal at the same time, we filtered the signal at 50Hz and then downsampled it to 50 Hz. Based on this operation, we could eliminate high-frequency noise without spectral aliasing. Moreover, the amplitude of the signal was scaled to 100.

Label Smoothing

In the multi-classification task, the neural network would output the degree of confidence of each category. The probability vector could be obtained after the softmax processing. The cross-entropy loss of the network was computed as:

(1)

Where p is the ground truth probability vector and q is the network predicted probability vector, and K is the total number of classes. When the ground truth probability vector is in the form of one-hot (hard label), pi=1 if i equals to the ground truth class c, otherwise pi = 0. In this way, it may induce overfitting. To solve this problem, label smoothing converts hard label to soft label:

(2)

Where ɛ is constant and 0<ɛ<1. Label smoothing can improve the generalization of neural networks and prevent the network from becoming over-confident and overfitting.

One-to-Many Label

When doctors classify the sleep stage, it is not only based on the current epoch but also the adjacent ones. To make sure that the neural network can learn the features of the adjacent epoch, we used the one-to-many label, which means that the label of the current epoch corresponds to the signal of the current epoch and the adjacent ones. In our work, we took the signal of epoch before and after into consideration, which means that each label corresponded to a 90 seconds signal.

Neural Network

In this work, we used the modularized network architecture which consists of convolution block and multi-branch convolution block. Modularized network architecture has been widely used in neural networks nowadays such as VGG-nets¹⁸ and ResNets,¹⁹ of which the effectiveness has been proven by a variety of tasks.^20–23 The neural network can become deeper without the limitation of the growing number of hyper-parameters by stacking blocks of the same structure.

Convolution Block

The structure of the convolution block is shown in Figure 1. The convolution layers are used to perform channel number conversion and extract the feature’s potential mapping. The shortcut connection is used to concatenate the input and output of the second ReLU, which can make the network deeper without the problem of vanishing gradient and exploding gradient. The length of the feature is halved by the average pooling layer. Batch normalization (BN) and ReLU are used before and after the convolution layer respectively. BN normalizes the distribution of input and ensures the input of the convolution layer has the same distribution as far as possible, which can alleviate the problem of vanishing gradient in training and accelerate the training speech of the model.

Figure 1 An overview of convolution block. Note: Batch normalization (BN) and ReLU were used before and after the convolution layer respectively.

Multi-Branch Convolution Block

The structure of the multi-branch convolution block is shown in Figure 2. Multi-branch architecture is widely used by the family of Inception models^24–26 and ResNeXt.²⁷ The first convolution layer is used to perform channel number conversion. The channel of feature is split into g group and as input of the convolution layer with a kernel size of 3×1 respectively, then concatenate them together. The third convolution layer with a kernel size of 1×1 guarantees that the number of channels is the same as the input. We also used the shortcut and average pooling as in convolution block. Multi-branch convolution block can reduce the number of parameters of the network and prevent the model from overfitting.

Figure 2 An overview of multi-branch convolution block. Notes: The channel of feature was split into g group and as input of the convolution layer respectively, then we concatenated them together. The specific values of c and g are shown in Figure 3.

Overall Architecture and Training

The proposed overall network architecture is shown in Figure 3, which is mainly composed of convolution block and multi-branch convolution block. After the feature was extracted by the last multi-branch convolution block, we used the global average pooling layer to reduce the parameter amount of the model. Our model had great effectiveness even though it only had 0.16 million parameters, which is fewer parameters than most models. In other words, the model was easier to train and obtain higher training speed. The optimizer used in model training was Adabound²⁸ (learning rate=0.0001, beta1=0.9, beta2=0.999, gamma=0.001) and batch size was 32. Moreover, early stopping callback on the validation loss with the patience of 5 epochs. All experiments in this study were performed on an NVIDIA GeForce RTX 3090 GPU.

Figure 3 An overview of our overall network architecture based on convolution block and multi-branch convolution block.

Model Evaluation and Statistical Analysis

The performance of the neural network model was evaluated by the overall accuracy, precision, recall, weighted F1 score, and Cohen’s Kappa coefficient. Their calculation formula was as follows:

(3)

(4)

(5)

(6)

where TP, TN, FP, and FN are the true positive, the true negative, the false positive, and the false negative, respectively.

(7)

where, , , and

Statistical analysis was performed using SPSS 22 software (SPSS Inc., Chicago, IL). We used Shapiro–Wilk test for normal data distribution test. Data were presented as mean±standard deviation or median (P25, P75). Variables with normal distribution were analyzed by t-test. Variables with nonnormal distribution were analyzed by Wilcoxon rank-sum test. The limit of agreement between technicians and the models was also analyzed using Bland-Altman plots.

Results

Study Population

The numbers of raw data collected by Alice 6 and Compumedics S series were 201 and 143, respectively. All the epochs were randomly split into training, validation, and testing sets at a ratio of 7:1:2. There was no significant difference in the demographic and polysomnographic parameters among the different datasets (Table 1). The distribution of W, N1, N2, N3, and REM was imbalanced at the radio of 1.74:1:7.33:3.94.

Table 1 The Demographic and Polysomnographic Data of Children in Different Datasets

Model Performance

In Table 2, we showed the accuracy and Cohen’s kappa of different models using channel combinations. The best performance with an average accuracy of 83.36% (ĸ=0.7817) in 5-stages appeared in the model ensemble which included the C3/A2+C4/A1+LOC+ROC+EMG, C3/A2, C4/A1, LOC, and ROC. And this model could also classify the stage of wake and sleep successfully with an average accuracy of 96.76% (ĸ=0.8236). In the single-channel EEG model, the performance of C3 was better than LOC and EMG. The confusion matrix for displaying the five sleep stage classification between the network prediction and technician was pictured in Figure 4. Except for single-channel EMG, N1 had precision (45.24–53.48%) and sensitivity (14.93–29.82%), which were lower than others. Most of them were classified as N2. We also compared the sleep stage classification performance of various ages of children, different data collection systems, and severities of OSA. The results were listed in the supplemental material (Table S1–S3).

Table 2 Comparison of Testing Performances Using Different Input Channels

Figure 4 The confusion matrix displaying the 5-stages classification between the network prediction and technician (true sleep stage). Abbreviations: Pre, precision; Sen, sensitivity; F1, F1-score. Notes: (A) The performance of the model ensemble; (B) the performance of the model using the channels of EEG, EOG, and EMG; (C) the performance of the model using the channels of EEG and EOG; (D) the performance of the model using the channel of EEG(C3/A2); (E) the performance of the model using the channel of EOG(LOG); (F) the performance of the model using the channel of EMG.

Quantitative Sleep Parameters

There was no significant difference in total sleep time (TST), sleep efficiency (SE), sleep onset latency (SOL), time of wake, time of shallow sleep, and time of deep sleep between the network prediction using the model ensemble and technicians (P>0.05). The model would overestimate the sleep time in REM (P<0.001) by about 7.73min (Figure 5). Using the single-channel EEG model, we found that it underestimated SOL by about 1.63min (Supplemental Material, Figure S1).

Figure 5 Bland-Altman plots showing the agreement between technicians and model ensemble for TST, SOL, time in W, time in N1+N2 (shallow sleep), time in N3 (deep sleep), and time in REM. Abbreviations: TST, total sleep time; SE, sleep efficiency; SOL, sleep onset latency; W, wake stage; N1+N2, shallow sleep; N3, deep sleep; REM, rapid eye movement stage; OAHI, obstructive apnea-hypopnea index. Note: The solid horizontal lines indicate the upper and lower limits of agreement, and the dotted line indicates the mean bias for the model.

Analysis of Respiratory Parameters

Using the stage classification of the model ensemble, we reviewed the analysis of respiratory events in the testing dataset. In Table 3, there was good consistency between the model and technicians (P=0.303). And there was no mistake in the diagnosis of OSA severity between the two sleep stage analyses.

Table 3 Comparison of the Sleep Parameters and Respiratory Parameters Between Network Prediction and Analysis of Technicians

Comparative Experiment on the Public Dataset

We conducted 4-fold cross-validation to evaluate the performance of the model which used channels of Fpz-Cz, Pz-Oz, and EOG. For 5-stages, the testing achieved accuracy of 92.76%(ĸ=0.8778) and 91.94% (ĸ=0.8521) of sleep-EDF-13 and sleep-EDF-18 with integral data, respectively. When removing large amounts of stage of wake, model performance dropped with the accuracy of 85.75% (ĸ=0.8015) and 84.58% (ĸ=0.7862) of sleep-EDF-13 and sleep-EDF-18, respectively. The comparison of similar research was shown in Table 4.

Table 4 Performance Comparison of Various Research on the Public Dataset of Sleep-EDF

Discussion

This study was the first to use unfiltered raw data of PSG based on large clinical samples in children with SDB for sleep staging model training, and compared the quantitative sleep parameters under sleep stage classification by experts and the model, such as total sleep time, sleep efficiency, sleep onset latency, and the time of each sleep stage. We used the modularized network which had lower number of parameters to propose an automatic sleep stage analysis model for children. For the 5-stages, the accuracy was 83.36%, and the cohen’s Kappa coefficient was 0.7881. This model could accurately distinguish between wake and sleep stages and showed no significant difference in diagnosis of the severity of children with SDB. Compared with similar studies, using sleep-EDF for verification, this model performed well.

Previous studies have trained models based on public datasets (collected from healthy adults and insomnia patients without other diseases), which have achieved good accuracy (78–92%). However, the original PSG data may have electrodes falling off, unstable baseline, high impedance, etc., and the accuracy of the model is affected by factors such as sleep fragmentation and arousal.⁸ Before this, other studies established sleep staging models based on pediatric PSG data,^10,11 but its sample size, using more channels, and larger model load might limit its clinical application. Due to the different EEG patterns of children of different ages, we used the 2–18 years old to train children’s PSG, and the model had better generalization. The accuracy of sleep staging for children aged 2–6, 7–13, and 14–18 were 82.68%, 83.86%, and 84.71%, respectively. Similar to other research, the accuracy decreased as the severity of SDB increased.^8,32 The testing set of this study included 10 children, whose AHI ranged from 5.03 to 39.42. The accuracy of automatic sleep stage classification was 82.74% in 5-stages, which was a little less than the performance in children with primary snoring and mild OSA. However, there was less influence on the calculation of classification in the severity of SDB.

Considering the clinical application of the automatic sleep staging model, we analyzed the performance of different channel combinations and single-channel EEG. Except for the single-channel EMG, all models had an accuracy of more than 95% for 2-stages, and the performance of 5-stages was equivalent to that of experienced sleep technicians. The lower accuracy of single-channel EEG model might be related to the smaller differences in different sleep stages. Affected by sweating and electrode shedding, many artifacts in EMG and small differences in EMG signals in N1, N2, and N3 phases were the explanation for the poor effect of the single-channel EMG model. Similar to other studies, the specificity and sensitivity of N1 stage classification were at a low level due to the small proportion of N1 in this database. A large number of N1 was divided into N2 and REM, which may be related to N1 which is mostly in the transition of sleep stage and the characteristics are not obvious. Some studies have performed separate data amplification for the N1-³⁵-, which can increase the diagnostic efficiency of N1. We will use it in the optimization of the model in the future. The N3 stage has a higher proportion in children’s sleep, and the slow-wave has the characteristics of high amplitude. Compared with the adult EEG staging model,^8,36,37 the accuracy of the N3 in this study had been greatly improved.

Comparing the ability of sleep stage classification and quantitative sleep parameters illustrates the practicality of the neural network model. The difference between model ensemble and technician analysis was small. The model underestimated the total sleep time by 1.00min, time in slight sleep (N1+N2) by 5.21min, time in N3 by 3.52min, and overestimated sleep onset latency by 0.44min, time in W by 0.56min, and time in REM by 7.73min. This inconsistency is tolerable. Similarly, the single-channel EEG model obtained a similar performance. Others based on non-contact radar technology, wearable devices, electrocardiograms, and respiratory dynamics, etc., had lower accuracy in judging different sleep stages than the single-channel EEG model of this study.^38–40 Using the automated sleep stage, we re-analyzed the respiratory events, and the calculated OAHI was not different from the manual analysis. In the future, adding a single-channel EEG module to wearable devices and performing automatic sleep stage classification will be more efficient to screen children with sleep-disordered breathing.

As shown in Table 4, a variety of deep learning networks achieved good performance.^10,29–34 With combined channels of EEG and EOG, our results were at a leading level. Recently, researchers have begun to consider how to design algorithms to be small, efficient, and robust. Based on previous research, we have improved data preprocessing and algorithms: (1) we filtered the signal at 50Hz, which would preserve EEG characteristics, remove high-frequency noise while reducing the amount of calculation. (2) Label smoothing mainly uses soft label, which modifies the weight of the real sample label category when calculating the loss function, and finally has the effect of suppressing overfitting. (3) Compared with other classifiers,^30,41–43 our model achieved a huge performance improvement, meanwhile the number of parameters (number of trainable parameters in a single model was 0.15 million) was much smaller than in other similar studies (Table 5).

Table 5 Overall Accuracy in 5-Stages and Training Model Number of Trainable Parameters of Similar Classifiers

There are still several limitations in our study. Even though our study population involved children aged 2–18, there is still a lack of validation for children under 2 years of age. SDB patients have mainly primary snoring and mild OSA, and the proportion of N1 in the children’s clinical sample database was very small, which could affect its performance.

Conclusion

In our study, we created an automatic sleep stage analysis model with the modularized network with satisfying reliability and generalizability, which could be applied to calculating quantitative sleep parameters and evaluating the severity of SDB.