Measuring Sleep Stages and Screening for Obstructive Sleep Apnea with a Wearable Multi-Sensor System in Comparison to Polysomnography

Introduction

Sleep disorders are an important public health problem with significant adverse consequences to individual health and are a significant economic burden for society;^1,2 therefore, they have attracted widespread attention. Sleep disorders comprise a wide range of diseases, and the diagnosis of many of their subtypes requires the help of objective indicators.³ Currently, polysomnography (PSG) is an acknowledged tool for measurement of sleep.^4,5 This technique uses collection of many electrodes attached to the surface of the body, each of which measures physiological parameters of sleep, including of the brain, eye, muscle, heart, respiratory activity, and oxygen saturation, throughout the night.^5,6

However, with the increase in its use and in-depth research, the limitations of PSG have gradually been recognized. First, the procedure is complicated and time consuming. Second, professional sleep technicians are required to monitor and interpret the data. Third, the body is covered with many electrodes and sensors, which causes obvious discomfort to patients, and finally the discomfort in monitoring sleep in a strange environment, given the need to monitor patients in a specific sleep laboratory. These interfere with the normal sleep of patients and cannot reflect the sleep situation in real life.^6–8 Therefore, a convenient, comfortable, and interference-free sleep monitoring method is essential in this field.

In recent years, clinicians and researchers have been trying to measure sleep or waking states using wrist actigraphy.^9–13 This technique measures wrist movement to evaluate sleep or wakefulness, that is accomplished by the accelerometer in the worn device.¹⁴ Compared to PSG, this method supports sleep research on a larger scale and promotes cheap and uninterrupted sleep measurements without causing disturbance. This technique can also be conveniently used in a wide range of environments and locations.^15–18 The opportunities for extensive participation rates could enhance the universality of the results, and makes longitudinal and repeated measurement designs more possible. However, there is limited validation with actigraphy compared to the gold standard of PSG; for instance, most actigraphy devices lack respiratory parameters.^19,20 Compared with PSG, it is well known that actigraphy overestimates sleep and underestimates wake-up time,^13,21 and the evaluation of sleep stage is often inaccurate.¹⁴ As sleep research continues to receive increasing attention, researchers require a series of effective technologies beyond PSG.

A new wearable multi-sensor system (SensEcho; SensEcho-5A, Beijing SensEcho Sci & Tech Co., Ltd., Beijing, China) was designed to assess sleep stages and respiratory events during sleep. The objective of this research was to assess the correspondence between SensEcho and PSG in the discrimination of sleep stages and searching for obstructive sleep apnea (OSA). In our study, the accuracy, sensitivity, and specificity were analyzed and compared between SensEcho and PSG.

Materials and Methods

Study Population and Design

This is a prospective study. Subjects with suspicious OSA, who had never been diagnosed or treated for their sleep problems, were recruited at the Sleep Medicine Center from Beijing HuiLongGuan Hospital, Beijing, China. One hundred and three adults aged 18–65 years participated in this study from August 1, 2021 to February 28, 2022. After removing missing data from polysomnography (PSG) or SensEcho measurements, 91 subjects were included in the analysis. Individuals were excluded if they met one or more of the following conditions: (1) serious arrhythmia, wearing auxiliary electronic products such as cardiac pacemakers, (2) respiratory failure and other related diseases, (3) unstable medical condition or psychiatric disorder, (4) incomplete fingers rendering them unable to be monitored by percutaneous oxygen saturation (SpO₂), (5) inability to independently cooperate with the test operation, and (6) other situations that the researcher believes make an individual unsuitable to participate in this clinical trial.

Participants slept in a sleep ward with a wearable multi-sensor system (SensEcho; SensEcho-5A, Beijing SensEcho Sci & Tech Co., Ltd., Beijing, China) and PSG (SOMNOscreen^TM plus, Somnomedics GmbH, Randersacker, Germany) for one night. All devices were worn simultaneously during all sleep episodes. During the falling-asleep stage, the light is turned off in the room where the subjects are located (the specific time of lights-off is adjusted according to the subjects’ sleep habits) and the subjects were required to turn off their mobile phones during the entire sleep period to prevent it from affecting sleep and the monitoring quality. During the monitoring period, the sensitivity of devices was confirmed using the monitoring software to ensure the normal collection of data. The next day, after the subjects got up, they turned on the light, and turned off and removed the collection equipment. There was no outside interference while collecting data, and participants were not allowed to use medications for their sleep.

The research has been approved by the Ethics Committee of Beijing HuiLongGuan Hospital, and informed consent was signed by each participant. This study was conducted in accordance with the Declaration of Helsinki.

SensEcho Measures

We used a wearable vest multi-sensor system (SensEcho) to capture single-lead electrocardiogram (ECG) signals and thoracoabdominal movement signals. A 3-axis accelerometer and a finger pulse oximeter were integrated into the system to capture posture and blood oxygen levels. A bidirectional long short-term memory (BLSTM) network was used to classify the four sleep stages. The Sleep Heart Health Study (SHHS) was used for model training.^22,23

Two 16 units-BLSTM layers and a well-connected layer, which represent the transition of the four sleep-stage classes, constituted the network architecture. A total of 152 features were recorded from the single-lead ECG and respiratory signals and were further fed into the BLSTM. SensEcho synchronously recorded the participants’ data on single-lead ECG, respiration, posture, and SpO₂ during the monitoring period. Subsequently, 30-second epoch sleep stages (wake, shallow sleep, deep sleep, and rapid eye movement [REM] sleep), sleep-disordered breathing events, apnea-hypopnea index (AHI), etc. were obtained by analyzing the physiological data from the SensEcho software (Figure 1).

Figure 1 Example of wearable vest multi-sensor system and schematic diagram of sleep stages and respiratory events detection algorithm. Abbreviations: ECG, electrocardiogram; SPO2, percutaneous oxygen saturation; LSTM, long short-term memory; REM, rapid eye movement.

PSG Measures

PSG is a method used for detecting and recording physiological changes during sleep. According to the standards of the American Academy of Sleep Medicine (AASM), PSG records the following indicators: electroencephalogram (EEG), electrooculogram (EOG), chin electromyogram, leg electromyogram, ECG, pressure (flow and snore), flow thermistor, effort (thorax/abdomen), SpO₂, pulse rate, body position, and movement (sleep/wake determination). The sampling rate was 512 Hz with a 16-bit resolution. PSG recordings were evaluated by the same registered polysomnographic technologist (RPSGT) using software in accordance with the AASM scoring criteria.²⁴

Snoring Severity

The degree of snoring was evaluated according to the guidelines for the diagnosis and treatment of obstructive sleep apnea hypopnea syndrome (2011 revision) formulated by the Sleep Respiratory Disease Group of the Chinese Thoracic Society in 2011.²⁵ Never snoring: 0 points; mild snoring: the breathing sound is heavier than that of normal people, 1 point will be recorded; moderate snoring: the loud degree of snoring is greater than the voice of ordinary people, 2 points; severe snoring: snoring is so loud that people in the same room cannot sleep, 3 points will be recorded.

Daytime Sleepiness

General sleepiness during the day was assessed using the Epworth Sleepiness Scale (ESS). The ESS is a self-evaluated scale that asks about the likelihood of falling asleep in various situations.^26,27 The total ESS score ranges from 0–24. Higher scores indicate higher risks of falling asleep. If the ESS score was > 10 points, it was rated as excessive sleepiness.

Statistical Analyses

The Statistical Package for the Social Sciences (SPSS) 26.0 and GraphPad Prism 8.0 were used for statistical analyses. Normally distributed continuous variables are expressed as mean ± standard deviation (M ± SD), and categorical variables are expressed as counts and percentages. Pearson’s correlation analysis was used to analyze the correlation between PSG and SensEcho by monitoring AHI and sleep-related indicators. Paired t-tests and Bland-Altman plots were used to analyze the consistency between PSG and SensEcho while monitoring AHI and sleep-related indicators. Sensitivity, specificity, positive/negative predictive value of the SensEcho AHI for the diagnostic characteristics of OSA were calculated using PSG AHI thresholds of ≥ 5, ≥ 15, and ≥ 30 (events/h) as the cut-off value. Sensitivity and specificity were further analyzed using ROC curve analysis. The Youden index was calculated to determine the optimal cutoff value of SensEcho for the diagnosis of mild, moderate, and severe OSA. Sensitivity, specificity, positive/negative predictive value of SensEcho AHI were also calculated at the optimal cut-off value for mild, moderate, and severe OSA.

Results

Out of the 103 adults recruited for the study, 91 of whom (aged 39.02 ± 13.84 and with a mean body mass index of 27.28 ± 5.12 kg/m²) fulfilled the study inclusion criteria and completed the assessments were considered for analysis. Of these, 56 (61.54%) were male and 58 (63.74%) reported at least a moderate level of snoring. Subjects had an average ESS score of 7.30 ± 4.98. Demographic data of the subjects are presented in Table 1.

Table 1 The General Demographic Characteristics, Snoring, and Sleepiness of Participants

Time in bed, sleeping time, awake time, sleep efficiency, wake, shallow sleep (non-REM stage 1 [N1] + non-REM stage 2 [N2]), deep sleep (non-REM stage 3 [N3]), REM sleep stage, AHI, and SPO₂ measured using SensEcho were compared to those from PSG (Table 2 and Table S1). There was no difference between the SensEcho and PSG in the time spent in bed (8.15 ± 1.06 versus [vs] 8.15 ± 1.07; P = 0.859), total sleep time (6.34 ± 1.40 vs 6.42 ± 1.28; P = 0.198), total wake time (1.73 ± 1.29 vs 1.82 ± 1.53; P = 0.182), and sleep efficiency (79.38 ± 14.39 vs 78.63 ± 16.56; P = 0.315). Results revealed that the wake (20.62 ± 14.39 vs 21.34 ± 16.54; P = 0.336), shallow sleep (69.96 ± 11.87 vs 69.09 ± 12.60; P = 0.297), deep sleep (16.12 ± 12.66 vs 16.62 ± 13.10; P = 0.422), and REM sleep (14.06 ± 7.33 vs 14.28 ± 7.74; P = 0.570) proportions of the participants have no difference between the measurements by SensEcho and those by PSG. A significant difference between the SensEcho and PSG in AHI > 30 (33.70 ± 18.91 vs 47.21 ± 15.59; P = 0.000), but when 5 ≤ AHI ≤ 30, there was no difference between them. Whereas there were no significant differences in Mean SpO₂, Minimum SpO₂, and SpO₂ < 90% between SensEcho and PSG.

Table 2 Comparison of Sleep Stages and Respiratory Parameters Observed Using SensEcho and PSG

As shown in Figure 2, a high degree of correlation between the recordings from SensEcho and PSG was found for the following measures: sleep efficiency (r = 0.917; P < 0.001); wake (r = 0.917; P < 0.001), shallow sleep (r = 0.792; P < 0.001), deep sleep (r = 0.895; P < 0.001), and REM sleep (r = 0.883; P < 0.001) proportions; and AHI (r = 0.887; P < 0.001). Bland-Altman analysis showed a mean difference of –0.01 (95% confidence interval, –0.14 to 0.12; Figure 3A) for sleep efficiency, 0.01 (95% confidence interval, –0.12 to 0.13; Figure 3B) for wake, –0.01 (95% confidence interval, –0.016 to 0.15; Figure 3C) for shallow sleep, 0.01 (95% confidence interval, –0.11 to 0.12; Figure 3D) for deep sleep, 0.002 (95% confidence interval, –0.07 to 0.07; Figure 3E) for REM sleep, and 2.46 (95% confidence interval, –13.79 to 18.71; Figure 3F) for AHI, between recordings from SensEcho and PSG. Respectively, after the calculations, 95.6%, 95.6%, 95.6%, 96.7%, and 98.9% points were settled in the limits of coincidence and its 95% confidence interval, which indicated a high level of coincidence between SensEcho and PSG measures for overall sleep efficiency, wake, shallow sleep, deep sleep, and REM sleep, except for AHI (93.4%).

Figure 2 Compare the relevance between manually edited parameters on SensEcho recording and PSG. ((A) Scatterplot of sleep efficiency on SensEcho compared to PSG; (B) Scatterplot of wake on SensEcho compared to PSG; (C) Scatterplot of shallow sleep on SensEcho compared to PSG; (D) Scatterplot of deep sleep on SensEcho compared to PSG; (E) Scatterplot of REM sleep on SensEcho compared to PSG; (F) Scatterplot of AHI on SensEcho compared to PSG). Abbreviations: PSG, polysomnography; REM, rapid eye movement; AHI, apnea-hypopnea index.

Figure 3 Compare the coherence between manually edited parameters on SensEcho recording and PSG. ((A) Bland-Altman plot of sleep efficiency on SensEcho compared to PSG; (B) Bland-Altman plot of wake on SensEcho compared to PSG; (C) Bland-Altman plot of shallow sleep on SensEcho compared to PSG; (D) Bland-Altman plot of deep sleep on SensEcho compared to PSG; (E) Bland-Altman plot of REM sleep on SensEcho compared to PSG; (F) Bland-Altman plot of AHI on SensEcho compared to PSG). Abbreviations: PSG, polysomnography; AHI, apnea-hypopnea index; SD, standard deviation.

Table 3 shows the comparisons of the diagnostic characteristics of SensEcho AHI based on the cutoff value of PSG AHI with variable severity criteria at 5, 15, and 30 events/h. Figure 4 shows the corresponding ROC curves. With an AHI cutoff of ≥ 5 events/h, SensEcho recording had 82.69% sensitivity, 89.74% specificity, and an AUC of 0.922. Almost the same results were obtained at an AHI threshold of ≥ 15 events/h. Although the specificity increased to 94.67% and the AUC increased to 0.936, the specificity decreased to 43.75% at an AHI cut-off of ≥ 30 events/h. According to the PSG diagnostic criterion of AHI > 5 events/h, the appropriate diagnostic cutoff value of SensEcho was 4.8 events/h, whose sensitivity is 84.62% and specificity is 89.75%. For moderate to severe OSA (AHI > 15 events/h), the SensEcho cut-off was 11.2 events/h, sensitivity was 87.10%, and specificity was 88.34%, as shown in Table 4.

Table 3 Prevalence, Sensitivity, Specificity, PPV, and NPV for Different Cutoff of the SensEcho Recording vs PSG

Table 4 Best Cutoffs for Different OSA Severity

Figure 4 Receiver operating characteristic curves for the SensEcho estimated AHI vs PSG AHI. Abbreviations: AHI, apnea-hypopnea index; AUC, area under the curve.

Discussion

This study is a clinical validation study for measuring the sleep stages and presence of obstructive events using a wearable vest multi-sensor system, SensEcho, which compared the data to simultaneously obtained PSG information. In the present study, specific variations in the accuracy, sensitivity, and specificity values were characterized. Overall, SensEcho has good accuracy in distinguishing sleep or wakefulness and for monitoring sleep stages. The total sleeping time, total awake time, shallow sleep, deep sleep, REM sleep, and wakefulness were consistent with the manual mapping of the PSG. Although SensEcho had a good correlation with PSG in the diagnosis of AHI, the consistency was unsatisfactory. SensEcho has high specificity for OSA screening. When AHI was ≥ 5 and ≥ 15, the sensitivity was good. When AHI was ≥ 30, the sensitivity was poor. SensEcho has a relatively good negative predictive value (NPV) for OSA but a poor positive predictive value (PPV), especially when AHI ≥ 30. This may be due to the fact that the proportion of health testers in the training set is more than that of patients with severe OSA, so the accuracy for AHI scores greater than 30 decreases. According to this study, the AHI correction thresholds for screening OSA using SensEcho were mild OSA, ≥ 4.8; moderate OSA, ≥ 11.2; and severe OSA, ≥ 17.7.

The SensEcho was designed to detect participants’ ECG, respiratory movement, SpO₂, body motion, and posture, as well as to use a specific algorithm to obtain the sleep status and sleep-related respiratory indicators of the subjects. In terms of sleep staging, the SensEcho used in this study had better results than similar devices used in most previous studies. For example, the partial-PSG system used in previous studies included Zeo (Newton, MA, USA).^28,29 The Zeo system can provide convincing sleep scoring for REM sleep and light and deep sleep. However, the system could not correctly detect the wake periods. In another study conducted with non-contact radar technology, Somnofy (VitalThings AS, Norway) found the sensitivity was 0.97 and specificity was 0.72, respectively. Sleeping stage differentiation for Somnofy was 0.75 for N1/N2, 0.74 for N3, and 0.78 for REM.³⁰ Wrist actigraphy cannot distinguish sleep stages but can simply distinguish between sleep and wakefulness.⁴ Therefore, SensEcho has greater advantage over wrist actigraphy due to its ability to differentiate sleep stages. However, it should be noted that SensEcho training model has extracted features from ECG and respiratory signals, a large part of which was heart rate variability (HRV), and the HRV characteristic parameters of patients with severe arrhythmia were relatively different from normal people, and there were many statistical deviations. Our model training set is mainly composed of non-arrhythmia testers, so sleep staging is less effective for patients with severe arrhythmias. In terms of respiratory monitoring, SensEcho is slightly inferior to other portable devices that can monitor respiratory function, such as OrbSense (Megahealth Medical, Inc., Zhejiang, China). Central sleep apnea (CSA) mainly manifests as repeated interruptions or weakening of respiratory airflow and ventilation during sleep, often related to other medical problems, especially heart failure, stroke, and the use of opioid drugs. SensEcho uses chest and abdominal breathing signals to estimate respiratory airflow, but there is little confidence in the judgment of CSA. Therefore, SensEcho currently can not detect CSA precisely. This monitoring equipment also has good sensitivity and specificity when AHI is ≥ 30.³¹ SensEcho is superior to wrist actigraphy in terms of respiration; as wrist actigraphy has no blood oxygen and respiratory indicators, it is unable to screen for OSA.

One advantage of the SensEcho is its design as a sleeping monitoring vest, without head and leg electrodes and nasal airflow tubes connected to the patient. This factor greatly reduces the impact on patients’ sleep and may allow for relatively natural sleep than in the traditional setup. In addition, it needs fewer settings, so it is possible to easily test sleep-related breathing disorders at home, which helps in outpatient diagnosis and expands the diagnostic capabilities. Once again, SensEcho is unobtrusive and can be worn throughout the day for over a week. This makes SensEcho more conducive to the long-term monitoring of patterns in sleeping rhythm.^32,33

This study has a few limitations. First of all, it was performed in a laboratory rather than in a home environment. There are some chances for SensEcho to behave differently in the home environment. Second, the current research focuses on ethnically Han patients with a low body mass index; therefore, the current research results are limited with regards to other populations. Finally, the overall AHI of the participants was low, and the recognition rate of SensEcho in patients with AHI ≥ 30 was poor. Therefore, in the future, it will be helpful to employ machine learning techniques to the database to improve the recognition rate of AHI equipment.

Conclusions

This study demonstrated the application of SensEcho as a portable monitoring device in the laboratory to evaluate sleep staging and detect sleep apnea in patients with suspicious OSA. The system may help with evaluating sleep status and screen for OSA in different settings. However, it is necessary to further improve the accuracy of its assessment of severe OSA and test its effectiveness in community and home environments.