Validation of PiezoSleep Scoring Against EEG/EMG Sleep Scoring in Rats

Introduction

Sleep disorders affect up to 56% of the general population.¹ Knowledge on the physiology of sleep and pathophysiology of sleep disorders has been obtained both from human and animal studies. Although significant value of data obtained by human sleep research, animal studies have become a vital part in the understanding of mechanisms of sleep and sleep disorders.² Current methods of sleep studying in rodents involve invasive surgical procedures to implant skull electroencephalography (EEG) and nuchal electromyography (EMG) electrodes.³ Despite it is an established method of sleep recording in rodents, it poses limitations such as postoperative recovery time; infections or mortality from surgery; and issues with misplacement of electrodes or noisy signals.^3,4

Recently, a new noninvasive method of measuring sleep, PiezoSleep, has been validated against implanted electrodes in mice.^5–11 PiezoSleep uses a piezoelectric film transducer that detects the rodent’s movements and respiration with high sensitivity and uses an algorithm to automatically score sleep. Since validation, PiezoSleep has been employed in studies of sleep, Alzheimer’s,^12,13 traumatic brain injury,^14–17 narcolepsy,^5,11,18 inflammation,¹⁹ mood disorders,^20,21 aging,²² and genomics.^23,24 All studies except for one²⁵ have been completed in mice. We validate PiezoSleep scoring against EEG/neck EMG implanted electrodes sleep detection, using both corded and telemetric EEG/EMG recording systems in rats.

Methods

General Procedures

Rats were maintained on a 12-hr/12-hr light–dark cycle (lights on from 08:00 hours to 20:00 hours) and were singly housed at 25°C with unrestricted access to food and water throughout the protocol. All procedures conformed to the American Physiological Society’s Guiding Principles for the Care and Use of Vertebrate Animals and were approved by the University of Illinois Chicago Animal Care and Use Committee.

Surgery for Telemetry Recording

Adult male Wistar Kyoto (N = 3) or Wistar Kyoto SHR (N = 2) rats (300–350 g; Charles River, Kingston, NY) were studied (N = 5) with implantable telemetry transmitters as previously described.^26,27 Rats were anesthetized with isoflurane (2–3% inhaled) or ketamine:xylazine (10 mg/kg:5 mg/kg) to undergo surgical implantation of a telemetry transmitter for measuring EEG/EMG (Model F40-EET, Data Sciences International [DSI], Minneapolis, MN). To measure the cortical EEG, two stainless steel screw electrodes were implanted into the skull (1.0 mm rostral and 2.0 mm medial/lateral from the bregma). Bipolar electrodes were inserted into the nuchal muscles to record EMG. The telemetry battery was placed subcutaneously via a flank incision. Rats were allowed to recover for 7 days before acclimation to the recording apparatus.

Telemetry Sleep Recording

Rats were individually placed in PiezoSleep cages, which were positioned on PhysioTel receivers to detect transmitter signals (Model RPC-3, DSI) for simultaneous recordings of PiezoSleep and EEG/EMG signals. The receivers were connected to a data exchange matrix (DSI) to relay continuous and simultaneous recordings of EEG and EMG (500 samples per sec) to a computer equipped with Ponemah software (Version 5.2, DSI). Before collecting data, rats were acclimatized to the chambers and had baseline recordings performed to assess the quality of EEG and EMG signals.

Telemetric data were analyzed using NeuroScore software’s (v. 3.0.7703, DSI) Rodent Sleep Scoring Module 2 automated scoring algorithm, as previously described.²⁷ Each 4-sec epoch was classified as wakefulness, non-REM sleep, or REM sleep based on cortical EEG, EEG theta-band, neck EMG and activity inputs. Wakefulness was defined as high-frequency, low-amplitude EEG with high EMG tone and/or activity levels above 0.1 counts; non-REM sleep was defined as increased spindle and delta ratio >1, decreased EMG tone and activity levels below 0.1 counts; and REM sleep was defined as high-frequency, low-amplitude EEG with a theta-to-delta ratio of 3, low EMG tone and activity levels below 0.1 counts. Epochs not meeting the above criteria, or containing artifacts, were scored as wake and accounted for <5% of scored time. All scoring was visually confirmed by manual review for accuracy.

Surgery for Corded Recording

Implantation of skull EEG and neck EMG electrodes was performed as previously described.^28,29 Adult male Brown Norway rats (N = 4) were anesthetized (ketamine:xylazine 100:10 mg/kg), stereotaxically immobilized, and implanted with EEG screw electrodes bilaterally threaded into the frontal and parietal bones. EMG wire electrodes were implanted into the dorsal nuchal musculature and tunneled subcutaneously to the skull. EEG and EMG leads were soldered to a miniature plastic connector plug (headset) and affixed to the skull by acrylic dental cement. Scalp wounds were closed with Vetbond Tissue Adhesive (3M, St. Paul, MN). Rats were allowed to recover for 7 days before acclimation to the recording apparatus.

Corded Sleep Recording

Rats were individually placed in PiezoSleep cages. A flexible cable was affixed on the top of the cage and attached to the rat’s headset. Rats were acclimatized to the chambers before collecting data and had baseline recording during 2 days prior to data collection. Rats underwent continuous simultaneous recordings from both systems.

EEG and EMG signals were amplified, band-passed filtered (0.5 to 100 Hz and 10 to 100 Hz, respectively), and digitized (250 samples/s; Bio-logic Sleepscan Premier). All data were stored to hard drive. Visual scoring was conducted by an experienced sleep scorer. Sleep stages (wake, NREM, and REM) were scored for every 10-second epoch. Wakefulness was characterized by high-frequency and low-amplitude EEG with high EMG tone. NREM sleep was characterized by low-frequency and high-amplitude EEG (increased spindles and delta activity) and low EMG tone, while REM sleep was characterized by high-frequency and high-amplitude (theta waves) EEG and an absence of EMG tone.

PiezoSleep Recording

PiezoSleep (Signal Solutions, LLC, Lexington, KY, USA) hardware and software were used to simultaneously record sleep in rats. Rats (N = 9) instrumented with EEG/EMG electrodes as described above were placed in cages equipped with PiezoSleep 1.0 sensors. The sensors were placed at the bottom of the cages, with cage liners (Techniplast 1291H, West Chester, PA, USA) and corncob bedding on top. The electrical signals from the sensors were amplified via PiezoSleep in-line amplifiers and were collected via Calamari SAS (8-channel) data acquisition system, which was connected to a computer running PiezoSleep 2.08r software. Sleep analysis was conducted offline using SleepStats 2.28r. Signals obtained by PiezoSleep that are irregular and high amplitude are scored as wake, while signals that are regular and low amplitude are scored as sleep.²⁵

Sleep Data Analysis and Statistics

Data are presented as mean ± standard error of the mean and were analyzed using IBM SPSS software as previously described.²⁹ Side-by-side sleep stage comparisons were made for each individual rat for DSI/PiezoSleep (N = 5) and between Corded/PiezoSleep (N = 4) recordings and sleep scoring. Confusion matrices were produced from these side-by-side comparisons, listing sleep and wake percentages, sensitivity, specificity, accuracy, positive predictive value (PPV), and negative predictive value (NPV). A Matthews Correlation Coefficient (MCC), a standard performance metric for machine learning, was also calculated and included in the confusion matrices (a coefficient of 0 represents random sleep scoring classification, −1 represents perfect sleep scoring misclassification, and +1 represents perfect sleep scoring classification).^30,31 Sleep stage percentages, defined as total time spent in a specific sleep stage (awake, NREM, or REM for EEG/EMG systems; awake or sleep for PiezoSleep), total sleep time (TST), defined as total time spent in sleep (combined NREM and REM for EEG/EMG systems) expressed as a percentage from the total recording time, and representative hourly sleep percentages were also quantified. TST data were analyzed using a linear mixed model analysis with “system” (DSI vs Corded vs PiezoSleep) as a factor, followed by post hoc multiple comparison test with Sidak’s correction if there was a significant main effect. The representative hourly sleep data were analyzed using a linear mixed model analysis with “system” (DSI vs PiezoSleep or Corded vs PiezoSleep) and “hour” as factors, followed by post hoc multiple comparison test with Sidak’s correction if there were significant main and/or interaction effects. Repeated covariance structure was chosen according to the best-fit Schwarz’s Bayesian information criterion. A Spearman correlation analysis was performed on the representative hourly sleep data recorded by the EEG/EMG telemetry and PiezoSleep systems or the corded EEG/EMG and PiezoSleep systems for the light and dark periods. Statistical significance was set at p <0.05. Statistical trends were set at 0.05 ≤ p < 0.10.

Results

A total of 9 rats underwent dual recording using corded/telemetric EEG/EMG recording and PiezoSleep systems for a total of 379,205 scored epochs. Five rats were surgically implanted by EEG/EMG telemetry, and their sleep was automatically scored using NeuroScore software for a total of 349,475 four-second epochs (16.2 days of recording). In four rats sleep recording was performed by corded EEG/EMG; their sleep was visually scored for a total 29,730 ten-second epochs (3.4 days of recording).

TST, light period TST, and dark period TST are shown in Figure 1. There was a statistical trend (system factor: F_2,15 = 3.42, p = 0.06) for difference in TST between EMG/EMG telemetry, corded EEG/EMG, PiezoSleep systems (Figure 1A). Post hoc analysis revealed statistical trend (p = 0.06) difference between EEG/EMG telemetry (66.27 ± 4.18%, N = 5) and PiezoSleep (56.92 ± 1.72%, N = 9), but no difference compared to corded EMG/EEG (58.49 ± 2.49%, N = 4). There was a significant system effect (F_2,15 = 4.39, p = 0.03) observed in the light period TST (Figure 1B). However, post hoc analysis only revealed a statistical trend (p = 0.06); NeuroScore in the telemetry recordings scored more sleep (72.58 ± 4.35%, N = 5) compared to both corded EEG/EMG (55.84 ± 3.41%, N = 4) and PiezoSleep (58.93 ± 3.43%, N = 9) systems. There was no significant difference in the dark period TST between the systems (Figure 1C).

Figure 1 Total sleep time (TST, (A)), dark period TST (B), and light period TST (C) expressed as a percentage, was assessed by EEG/EMG telemetry and Neuroscore sleep scoring (N = 5, white bar), or by corded EEG/EMG and human visual sleep scoring (N = 4, shaded bar), while simultaneously assessed and scored by PiezoSleep (N = 9, black bar). There was a statistical trend in the system main effect for TST ((A): F2,15 = 3.42, p = 0.06) and a significant difference in the light period TST ((B): F2,15 = 4.39, p = 0.03). Post hoc analysis revealed a statistical trend difference (p = 0.06) between the telemetry and PiezoSleep TST, and between DSI and telemetry/corded light period TST. Data were analyzed using a linear mixed model analysis with “system” (DSI vs Corded vs PiezoSleep) as a factor, followed by post hoc multiple comparison test with Sidak’s correction.

Confusion matrices between EEG/EMG and PiezoSleep systems are shown in Tables 1 and 2. The sensitivity was 79.2% and the specificity was 83.8% with 80.8% accuracy, and PPV and NPV was 90.2% and 68.3%, respectively, between the EEG/EMG telemetry and PiezoSleep systems (Table 1), while between the corded EEG/EMG and PiezoSleep systems, the sensitivity was 87.2% and the specificity was 83.4% with 85.6% accuracy, and PPV and NPV was 88.0% and 82.3%, respectively (Table 2). Moreover, the MCC for EEG/EMG telemetry and PiezoSleep systems and for the corded EEG/EMG and PiezoSleep systems were 0.61 and 0.71, respectively, representing agreement between PiezoSleep and EEG/EMG systems. Tables 3 and 4 show the agreements and disagreements for wake/REM/NREM and wake/sleep scoring between EEG/EMG and PiezoSleep systems, respectively. The most disagreements occurred when NeuroScore from the EEG/EMG telemetry recording scored epochs as REM (3.4%) and NREM (10.2%) sleep, while PiezoSleep scored these epochs as wake (Table 3). The disagreements of REM/NREM and wake epochs with corded EEG/EMG and PiezoSleep were 0.8% and 6.7% respectively (Table 4). As such, recordings scored with NeuroScore had the most TST compared to PiezoSleep or corded EEG/EMG (Figure 1). Epochs that were scored as wake by EEG/EMG telemetry and corded EEG/EMG when PiezoSleep scored the epochs as sleep were similar (5.6% vs 6.9%, respectively; Tables 3 and 4).

Table 1 Confusion Matrix for Wake/Sleep Scoring with EEG/EMG Telemetry (Scored with NeuroScore) and the PiezoSleep System (Scored with PiezoSleep/SleepStats)

Table 2 Confusion Matrix for Wake/Sleep Scoring with Corded EEG/EMG (Visually Scored by Human) and the PiezoSleep System (Scored with PiezoSleep/SleepStats)

Table 3 Agreements and Disagreements for Wake/REM/NREM and Wake/Sleep Scoring Between EEG/EMG Telemetry (Scored with NeuroScore) and the PiezoSleep System (Scored with PiezoSleep/SleepStats), Respectively

Table 4 Agreements and Disagreements for Wake/REM/NREM and Wake/Sleep Scoring Between EEG/EMG (Visually Scored by Human) and the PiezoSleep System (Scored with PiezoSleep/SleepStats), Respectively

Figure 2 illustrates a daily hourly sleep percentage between the EEG/EMG telemetry and PiezoSleep systems (Figure 2A) and between the corded EEG/EMG and PiezoSleep systems (Figure 2B). There were significant main interactions (system factor: F_1,45.69 = 6.72, p = 0.01; hour factor: F_23,105.45 = 2.65, p < 0.001), but no significant interaction, observed between the EEG/EMG telemetry and PiezoSleep systems (Figure 2A). Post hoc analysis of system factor revealed that NeuroScore scored significantly more sleep (64.50 ± 2.37%, N = 120 from 5 rats) compared to PiezoSleep (54.38 ± 2.07, N = 120 from 5 rats); which corresponds to TST data. In contrast, there were no significant system main or interaction effects, but only significant hour interaction (F_21,69.60 = 3.39, p < 0.001), observed in the corded EEG/EMG and PiezoSleep systems (Figure 2B). Spearman correlation analysis revealed significant correlation (p < 0.001) for both the EEG/EMG telemetry and PiezoSleep and corded EEG/EMG and PiezoSleep systems (Figures 2C and D, respectively). The Spearman’s rho was the lowest for EEG/EMG telemetry/PiezoSleep during the light period (rho = 0.80), signifying more disagreements in sleep scoring.

Figure 2 Representative hourly sleep percentages between the EEG/EMG telemetry (A) or corded EEG/EMG (B) and PiezoSleep systems, respectively, and Spearman correlation analysis between EEG/EMG telemetry and PiezoSleep (C) and corded EEG/EMG and PiezoSleep (D) systems. Shaded area represents the dark period. (A) There were significant main interactions (system factor: F1,45.69 = 6.72, p = 0.01; hour factor: F23,105.45 = 2.65, p < 0.001), but no significant interaction, observed between the EEG/EMG telemetry (N = 5, solid square/line) and PiezoSleep systems (N = 5, open square/dotted line). Post hoc analysis of system factor revealed that Neuroscore scored significantly more sleep compared to PiezoSleep. (B) In contrast, there were no significant system main or interaction effects, but only significant hour interaction (F21,69.60 = 3.39, p < 0.001), observed in the corded EEG/EMG (N = 3–4, solid square/line) and PiezoSleep systems (N = 3–4, open square/dotted line). (C and D) Spearman correlations were significant for both corded EEG/EMG and EEG/EMG telemetry when compared to PiezoSleep systems. The Spearman’s rho was the lowest for EEG/EMG telemetry/PiezoSleep during the light period. Data were analyzed using a linear mixed model analysis with “system” (DSI vs PiezoSleep or Corded vs PiezoSleep) and “hour” as factors, followed by post hoc multiple comparison test with Sidak’s correction. A Spearman correlation analysis was performed on the representative hourly sleep data recorded by the EEG/EMG telemetry and PiezoSleep systems or the corded EEG/EMG and PiezoSleep systems for the light and dark periods.

Discussion

This study has confirmed the use of the PiezoSleep system as a reliable non-invasive method of sleep assessment in rats against the gold-standard of EEG/EMG sleep scoring that employs both telemetry and corded recording systems. Our work represents a comprehensive comparative analysis of sleep/wake detection by PiezoSleep versus both automatic and human scoring of EEG/EMG sleep recordings in rats.

We show that TST and hourly sleep percentages are similar in all three systems and that sensitivity and specificity of sleep assessment is high, with accuracies of 80.5% and 85.6% of the EEG/EMG telemetry and corded EEG/EMG, respectively; this compares to 90% seen in PiezoSleep system for mice.^7,8 Currently, it is unknown why PiezoSleep accuracy for mice is higher; differences in body weight, breathing patterns, and activity levels between rats and mice could potentially lead to accuracy discrepancies in the piezoelectric signal. Moreover, representative hourly sleep percentages are similar between PiezoSleep and EEG/EMG systems, with high correlations. We found that detected disagreements between the PiezoSleep based and EEG/EMG-based sleep/wake estimates occur both in scoring sleep epochs as wake and wake as sleep, respectively, which could occur, for instance, during transitional stages.^32,33 Further analysis is needed.

PiezoSleep system records movements and respiration and uses an algorithm to automatically score sleep. The system was originally validated in mice,^5–11 and has been used extensively since validation.^5,11–24 However, only one study employed the PiezoSleep system in rats.²⁵

Although Vanneau et al²⁵ validated the PiezoSleep system against EEG/EMG telemetry with NeuroScore system, they conducted only human visual scoring of sleep stages using a single scorer, which was a stated limitation of their study. They showed a similarity in TST percentages between recordings by PiezoSleep (56.3 ± 3.2%) and EEG/EMG telemetry (54.8 ± 1.7%) systems. However, in their work TST had lower values compared to our EEG/EMG telemetry scored with NeuroScore (66.27 ± 4.18%), while it was similar to our human visually scored TST (58.49 ± 2.49%) from the corded EEG/EMG recordings (Figure 1).

Noticeably, TST results scored by PiezoSleep were similar between our study (56.92 ± 1.72%) and the work by Vanneau et al.

Unfortunately, the authors did not report NREM and REM sleep percentages from their EEG/EMG telemetry recordings, and it is unknown which stage contributed to the smaller TST in their recordings.

Previous literature using the NeuroScore automated or visual scoring of 10-second epochs have reported comparable levels of NREM and REM sleep between them, both in the light and dark periods,^34–36 which was also seen in our study.

In our work, minor differences in NREM sleep between the two EEG/EMG systems were observed. However, there was almost twice more REM sleep scored in the NeuroScore (9.9%) compared to the human visual scoring (5.3%).

Thus, the increase in TST was at the account for NeuroScore scoring more REM sleep instead of wake. The discrepancy could also be attributed to the use of different strains of rats.³⁷ It could also occur because of the differences in EEG/EMG recording: corded EEG/EMG, which applies some force on the skull headset by an external cord, may cause more discomfort than EEG/EMG telemetry (wires tunneled under the skin do not exert force on the skull) and therefore change sleep patterns. Future clarification of this issue would entail using PiezoSleep to distinguish between REM and NREM sleep, which Vanneau et al have already started to introduce.²⁵

According to Vanneau et al, PiezoSleep underestimated wake (specificity was lower than sensitivity). We observed equivalent results in scoring of our corded EEG/EMG recordings, where there was an underestimation in wake. However, comparison of our EEG/EMG telemetry recordings automatically scored by NeuroScore with PiezoSleep data showed overestimation of wake by PiezoSleep (specificity higher than sensitivity). Similarly to our TST data, it is possible that the automated scoring of NeuroScore overestimated sleep, specifically REM sleep, at the expense of wake and, therefore, decreased the specificity below the sensitivity as compared to PiezoSleep. Our correlation analysis further proved that the discrepancy in scoring occurred more during the light period between the EEG/EMG telemetry recording and PiezoSleep. As Vanneau et al reasonably pointed out, consideration for the sensitivity and specificity should be made in PiezoSleep studies that use pharmacologic treatments or rat models of human pathology that alter breathing, movement, or muscle tone that would alter the way PiezoSleep scores sleep.²⁵ A further limitation of this study is the limited number of rats that were used between the systems (N = 9). Though more than 379,000 epochs were scored, increasing the number of rats can increase validity/reliability of the data between the EEG/EMG and PiezoSleep systems. Also, similar to Vanneau et al,²⁵ our corded EEG/EMG recording was manual scored by a single experienced sleep scorer, which is a limitation. And lastly, due to PiezoSleep’s current inability to differentiate between NREM and REM sleep, or to record EEG, PiezoSleep will not replace the important utility of EEG/EMG systems to assess brain activity during sleep.

In conclusion, our results confirm the use of PiezoSleep as an accurate non-invasive sleep/wake detecting system comparable to EEG/EMG recording systems. Although at the present time, PiezoSleep cannot differentiate between NREM and REM sleep, it can be used as a high throughput first step in elucidating wake/sleep patterns in experimental interventions.