A validation of wrist actigraphy against polysomnography in patients with schizophrenia or bipolar disorder

Introduction

Patients with schizophrenia or bipolar disorder frequently suffer from disturbed sleep, not only during periods of florid psychosis or during episodes with elevated or depressed mood but also as a trait-like phenomenon.^1,2 It has been reported that up to 80% of patients with schizophrenia suffer from sleep disturbances,¹ and that sleep is significantly impaired in euthymic bipolar patients, with escalation of this disturbance occurring before and during a bipolar episode.² It has been speculated that sleep disturbances play a part in the pathogenesis of schizophrenia; it is also possible that they are linked to the cognitive deficits observed across multiple cognitive domains in schizophrenia patients, as compared with the background population.^3,4 Polysomnography (PSG) is currently the gold standard for examining sleep patterns, but it is an expensive and obtrusive measure that many patients with severe mental illness might not tolerate. Instead, actigraphy represents a simple alternative for clinical and research use in this patient population when a differential diagnosis of a specific sleep disorder is merited or when an objective measure of the presence and intensity of subjective sleep complaints is considered necessary. Sleep complaints in mentally ill people are at risk of being discarded as an inevitable consequence of the mental illness itself. Consequently, appropriate clinical and laboratory evaluation might be substituted by increasing the number or dosage of sedative medications. In this context, actigraphy has importance as a simple and accessible tool to monitor the sleep–wake pattern over a more prolonged period than is possible with PSG. Furthermore, PSG is much less accessible and associated with much more inconvenience than actigraphy. Actigraphy has been validated against PSG, and it has generally been found to exhibit a high level of sensitivity when detecting sleep; the most problematic validity issue is its low specificity when detecting wakefulness.^5–7 Actigraphic measurements have been used in several research studies of patients with schizophrenia,⁸ but actigraphy has never been validated against PSG in this specific patient population. A recent validation study in patients with bipolar disorder found that total sleep time estimates were highly correlated and other sleep outcomes moderately correlated.⁹

The aim of this study was to examine the usefulness of actigraphy, as compared with PSG, for measuring routine sleep variables in a sample of chronic, medicated patients with schizophrenia or bipolar disorder.

Materials and methods

Materials and participants

The data used in this study were derived from a clinical trial examining the efficacy of melatonin in benzodiazepine withdrawal. The trial was approved by the Committee on Biomedical Research Ethics of The Capital Region in Denmark (H-1-2011-025) and all participants provided written, informed consent upon inclusion in the trial. The findings of this trial will be published elsewhere. In the current validation study, only the baseline data of simultaneously collected actigraphy and PSG recordings for one night were used. All sleep examinations were recorded in the participants’ homes using ambulatory equipment. A total of 42 patients underwent both sleep examinations and had usable data. Another four patients attempted to undergo the sleep examinations but data could not subsequently be used due to equipment malfunction or other technical issues. The inclusion criteria included those with an age of 18 years or older; an International Classification of Diseases, 10th Revision diagnosis of schizophrenia, schizoaffective disorder, or bipolar mood disorder (euthymic at inclusion); daily use of at least one antipsychotic medication and at least one benzodiazepine or benzodiazepine-like medication for a minimum of 3 months; no current violent or aggressive behavior; not diagnosed with mental retardation, pervasive developmental disorder, dementia, hepatic impairment, terminal illness, severe somatic comorbidity, or epilepsy; and the ability to understand Danish. Fertile women were only included if not pregnant or nursing.

Actigraphy measures

We collected actigraphy data using the Actiwatch Spectrum (Philips Respironics, Murrysville, PA, USA). This is a wrist-worn device that measures movement in 30-second epochs. The Actiwatch was worn on the nondominant arm for three consecutive days and nights; however, for this study, only data from the first night with concomitant PSG recordings were included in the analysis. The participants had a sleep diary, where they recorded the time of lights off and lights on. Data were exported and downloaded using the Actiware software (version 6.0.0; Respironics, Murrysville, PA, USA). The default setting to analyze the various sleep variables was used, but the settings for bedtime and wake-up time were adjusted manually to comply with the sleep diary in case of any major inconsistencies between the automatically set values and the values reported in the sleep log. It should be noticed that such manual correction was only relevant in a minor number of patients where the sleep detection algorithm had obviously failed. The default setting for the Actiware software included the following specifications: medium sensitivity, one major rest interval per 24 hours, and a minimum minor rest interval size of 40 minutes. The following sleep continuity parameters were extracted from the Actiware software: total sleep time, sleep efficiency, sleep latency, number of awakenings, and wake after sleep onset.

PSG measures

Each subject underwent a full-night, unattended, in-home PSG in accordance with the 2007 guidelines of the American Academy of Sleep Medicine.¹⁰ The impedance of each electrode was maintained <10 kΩ, and the electroencephalogram was sampled at a frequency of 256 Hz. PSG was performed using portable equipment (Trackit) (www.cephalon.dk/).

The PSG comprised electroencephalography (with frontal, central, and occipital derivations), vertical and horizontal electrooculography, surface electromyography of the submentalis and tibialis anterior muscles, nasal air flow, temperature of inhaled and exhaled air, pulse oximetry, respiratory inductance plethysmography (measuring the effort to breathe), and electrocardiography. Montage was generally performed on weekdays between 12 pm and 15 pm, and the equipment was removed the morning after. A trained neurophysiologist and a neurologist who specialized in sleep medicine visually scored the sleep recordings in 30-second epochs according to the American Academy of Sleep Medicine criteria.¹⁰

Statistical analysis

PSG measures of the various sleep continuity parameters were used as the gold standard against which the actigraphically derived values were compared. The mean values of the absolute difference between the PSG and actigraphic measures were calculated together with paired t-tests to assess the difference between the methods. For each sleep variable, a Bland–Altman plot and a dot plot with a double Y-axis were used to graphically illustrate the level of agreement between the two methods. We used the intraclass correlation coefficient (ICC) to evaluate the consistency between the PSG and actigraphic measurements. Supplementary Spearman’s correlation coefficients were calculated to check the validity of the ICC analysis in case of nonnormally distributed variables. A subgroup analysis of the sample without the five subjects with bipolar disorder was performed to investigate if this would change the ICC estimates. A subgroup analysis was also performed splitting the sample according to sex.

Results

The demographic and clinical characteristics of the sample are listed in Table 1. All participants (N=42) were medicated with a combination of antipsychotics and benzodiazepines, and the majority of the sample (37 patients; 88.1%) had been diagnosed with schizophrenia or schizoaffective disorder for many years (mean illness duration 21 years). A subgroup of the sample (five patients; 11.9%) was diagnosed with bipolar disorder. The means and standard deviations of the PSG measurements, together with the difference between these measures and the actigraphically derived measures, are presented in Table 2. On a group level, there seemed to be a reasonable agreement between PSG and actigraphy if using the criterion of no statistically significant differences when using a paired t-test. However, from a clinical point of view, there seemed to be a clinically relevant difference regarding wake after sleep onset, which actigraphy underestimated by 37 minutes, on average. Additionally, the dispersions of the disparities were large for total sleep time and wake after sleep onset indicating wide variability between subjects. For total sleep time and wake after sleep onset, the actigraphy measurements were less precise in men than women. In men, actigraphy underestimated total sleep time by 36 minutes, and in women, only by 5 minutes; likewise, actigraphy underestimated wake after sleep onset in men by 47 minutes and in women less so by 24 minutes (data not shown).

Table 1 Sample characteristics Notes: Mean (SD) for continuous variables; N (%) for categorical variables. Abbreviations: SD, standard deviation; BMI, body mass index; AHI, apnea–hypopnea index.

Table 2 Comparison of sleep variables between PSG and actigraphy Notes: aValues denoted as mean (SD). bNone of the reported differences were statistically significant (P>0.05) using a paired t-test. Abbreviations: PSG, polysomnography; TST, total sleep time; min, minutes; SE, sleep efficiency; SL, sleep latency; AW, number of awakenings; WASO, wake after sleep onset.

The ICCs are presented in Table 3. There was a high level of agreement between the two methods when measuring total sleep time (ICC =0.78), a low-to-moderate agreement for the number of awakenings (ICC =0.22), and no detectable agreement when evaluating sleep efficiency, sleep latency, and wake after sleep onset (ICCs at or close to zero). Not all variables were normally distributed as assessed by visual inspection of the shape of the variables’ distribution using histograms. When analyzing the data using Spearman’s correlation coefficient, the results were similar, indicating that the ICC estimates were valid despite not all variables being normally distributed (total sleep time: Spearman’s rho =0.67; P<0.0001, all other sleep variables not statistically significantly correlated).

Table 3 ICCs for investigated sleep variables by subgroup Note: aICC (95% CI). Abbreviations: ICC, intraclass correlation coefficient; PSG, polysomnography; TST, total sleep time; SE, sleep efficiency; SL, sleep latency; AW, number of awakenings; WASO, wake after sleep onset; min, minute; CI, confidence interval.

The high ICC for total sleep time was replicated in the subgroup analyses; the ICC for total sleep time was even higher among women (0.95) when compared with men (0.59), and it was also higher for bipolar patients (0.96) than for patients with schizophrenia or schizoaffective disorder (0.56) (data not shown). For sleep efficiency, sleep latency, number of awakenings, and wake after sleep onset, the ICC estimates did not change substantially when splitting the analysis according to sex and diagnosis.

A visual inspection of the dot plot presentation of wake after sleep onset resulted in the identification of six patients who had extensive values of wake after sleep onset duration (ie, >100 minutes), as measured by PSG, whereas the rest of the sample had values <70 minutes (Figure 1). To determine the robustness of the results, we re-calculated the ICC without these six subjects included. This led to a marked increase in ICC values for all sleep continuity variables (except for sleep latency) to a level of moderate agreement between PSG and actigraphy (Table 3). However, these subgroup ICC estimates were uncertain due to the smaller sample size as reflected in the wide 95% confidence intervals. The results of the subgroup analysis with bipolar patients excluded showed essentially the same results compared with analyzing the whole sample, the only change was a smaller reduction in the ICC for total sleep time (ICC =0.56 compared with ICC =0.78 for the total sample).

Figure 1 Dot plot of WASO as measured using PSG (in blue) and actigraphy (in green). Abbreviations: WASO, wake after sleep onset; PSG, polysomnography; min, minute.

The Bland–Altman plots illustrating the level of agreement between actigraphic and PSG-measured sleep variables are presented in Figures 2–6. The rather broad 95% confidence limits of agreement indicated considerable discrepancies between the methods in individual subjects. For sleep latency and number of awakenings, these plots further illustrated that the level of agreement between the methods seemed to decrease as the mean values for these sleep variables increased.

Figure 2 Bland–Altman plot of TST. Abbreviations: TST, total sleep time; PSG, polysomnography; SD, standard deviation.

Figure 3 Bland–Altman plot of SE. Abbreviations: SE, sleep efficiency; PSG, polysomnography; SD, standard deviation.

Figure 4 Bland–Altman plot of SL. Abbreviations: AW, number of awakenings; SL, sleep latency; PSG, polysomnography; SD, standard deviation.

Figure 5 Bland–Altman plot of AW. Abbreviations: AW, number of awakenings; PSG, polysomnography; SD, standard deviation.

Figure 6 Bland–Altman plot of WASO. Abbreviations: WASO, wake after sleep onset; PSG, polysomnography; SD, standard deviation.

Discussion

This study aimed to evaluate whether actigraphy provides a valuable measure of certain sleep continuity variables when compared with PSG, which is currently the gold standard. Overall, we found that the total sleep time was reliably measured, whereas measurements of other sleep parameters were not consistent with the PSG-derived measures. The low reliability of actigraphically defined sleep latency corroborates previous findings¹¹ and is logically inferred from the difficulties encountered when distinguishing between rest and sleep in the period preceding sleep onset. The participants in this study suffered from schizophrenia or bipolar disorder, they presented with abnormal sleep habits and disturbed sleep macro architecture, and many subjects also had prolonged periods of wake after sleep onset. After excluding subjects with an excessive duration of wake after sleep onset, the validity of the actigraphically derived sleep continuity measures markedly increased. This is an important result to consider when deciding whether actigraphy can be used as a reliable measure of sleep continuity parameters and circadian sleep–wake patterns in patients with severe mental illness. The Bland–Altman plots supplemented this picture by indicating that for sleep latency and wake after sleep onset, there seemed to be more disagreement with higher mean values.

These findings seem to point to a potentially serious limitation of the actigraphically derived sleep variables. If the validity of the data depends on the duration of wake after sleep onset, and for some variables on the mean value of the respective variable, then the judgment of whether actigraphy would be a valuable tool in the individual patient would depend on knowledge of the value of these same sleep variables as measured by PSG. Naturally, this would limit the clinical usability of the actigraph in this group of patients.

Actigraphy has been validated against PSG in other specific populations, and it has demonstrated overall findings that are in line with those of the current study – ie, the validity of actigraphy in special populations or in individuals with poor sleep is more questionable when compared with healthy individuals with relatively good sleep patterns.⁶ It has been argued that complete agreement between actigraphy and PSG is theoretically impossible due to the residual unreliability of PSG data,¹² but the results of the present study clearly indicate a quantitative difference in the validity between different sleep continuity variables when measured actigraphically. Consequently, the discussion of whether actigraphy is valid and reliable should be more nuanced, taking both the characteristics of the patient population and characteristics of their sleep patterns into consideration.

The intention of the actigraphy–PSG comparison in this study was mainly as an aid to clinicians regarding the validity of the specific measurement of sleep continuity variables that the actigraphy provides. Consequently, this study did not focus on an epoch-by-epoch comparison of the ability of the actigraphy to detect sleep and wakefulness. This has been done in other studies investigating various populations. For instance, Marino et al⁵ has published such an analysis of the sensitivity, specificity, and accuracy of the actigraph in different samples of subjects and found high sensitivity and accuracy but low specificity, irrespective of age, presence of insomnia, and work-related reversal of the day–night cycle. Another study by McCall and McCall,¹³ comparing actigraphy with PSG and sleep diaries in depressed patients with insomnia found a high degree of agreement between these measurements using t-tests, correlations, and a Bland–Altman plot for each sleep continuity parameter. This study resembled ours due to the focus on the analysis of sleep variables rather than an epoch-by-epoch approach. Consistent with our results, they found no differences when using a t-test to compare the actigraphy and PSG recordings, except for sleep latency, which was underestimated by the actigraph. In contrast to our findings, McCall and McCall found significant correlations between actigraphy and PSG on most sleep variables. We believe that the nature of the current study population is probably causing the discrepancy to these previous studies.

The possible limitations of the current study include the following: first, both patients with schizophrenia and patients with bipolar disorder were included, which might have confounded the results due to differences in sleep and circadian rhythm disturbances between these two diagnostic groups. However, we do not believe that this was a major flaw in the design of the study since the subgroup analysis pointed to similar results in both diagnostic groups. Second, it was a small sample, and especially, the results from the subgroup analyses are associated with considerable uncertainty. Third, the PSG-measured sleep variables might not be representative of the average sleep quality among patients with severe mental illness due to the first-night effect. This phenomenon refers to the fact that sleep patterns during the first night with PSG equipment may be more disturbed than on average because of the lack of familiarity with the equipment. This should not substantially influence, however, the current comparison of PSG and actigraphy since data from the first night were used for both methods. In addition, previously published data seem to indicate that the first-night effect might be of reduced significance in patients with chronic schizophrenia.¹⁴ Finally, this method of analysis does not evaluate the ability of the actigraph to distinguish between sleep and wakefulness on an epoch-by-epoch level, but it still provides a clinically useful measure of the validity of actigraphically derived measurements of sleep variables.

Conclusion

Actigraphy seems to be a reliable measure of total sleep time in patients with severe mental illness, but for other sleep continuity parameters, the validity depends on the duration of wakefulness after sleep onset.

Acknowledgments

The study was conducted as a part of a post doc position financed by the Mental Health Services of the Capital Region in Denmark. The authors thank Helle Leonthin for her help collecting the data.

Disclosure

The authors report no conflicts of interest in this work.

---

References

1.		Cohrs S. Sleep disturbances in patients with schizophrenia: impact and effect of antipsychotics. CNS Drugs. 2008;22:939–962.
2.		Harvey AG. Sleep and circadian rhythms in bipolar disorder: seeking synchrony, harmony, and regulation. Am J Psychiatry. 2008;165:820–829.
3.		Bromundt V, Köster M, Georgiev-Kill A, et al. Sleep-wake cycles and cognitive functioning in schizophrenia. Br J Psychiatry. 2011;198:269–276.
4.		Keshavan MS, Montrose DM, Miewald JM, Jindal RD. Sleep correlates of cognition in early course psychotic disorders. Schizophr Res. 2011;131:231–234.
5.		Marino M, Li Y, Rueschman MN, et al. Measuring sleep: accuracy, sensitivity, and specificity of wrist actigraphy compared to polysomnography. Sleep. 2013;36:1747–1755.
6.		Sadeh A. The role and validity of actigraphy in sleep medicine: an update. Sleep Med Rev. 2011;15:259–267.
7.		Sivertsen B, Omvik S, Havik OE, et al. A comparison of actigraphy and polysomnography in older adults treated for chronic primary insomnia. Sleep. 2006;29:1353–1358.
8.		Tahmasian M, Khazaie H, Golshani S, Avis KT. Clinical application of actigraphy in psychotic disorders: a systematic review. Curr Psychiatry Rep. 2013;15:359.
9.		Kaplan KA, Talbot LS, Gruber J, Harvey AG. Evaluating sleep in bipolar disorder: comparison between actigraphy, polysomnography, and sleep diary. Bipolar Disord. 2012;14:870–879.
10.		Iber C, Ancoli-Israel S, Chesson AL, Quan SF. The AASM Manual for Scoring of Sleep and Associated Events: Rules, Terminology and Technical Specifications. Westchester, IL: American Academy of Sleep Medicine; 2007.
11.		Martin JL, Hakim AD. Wrist actigraphy. Chest. 2011;139:1514–1527.
12.		Tryon WW. Issues of validity in actigraphic sleep assessment. Sleep. 2004;27:158–165.
13.		McCall C, McCall WV. Comparison of actigraphy with polysomnography and sleep logs in depressed insomniacs. J Sleep Res. 2012;21:122–127.
14.		Shamir E, Rotenberg VS, Laudon M, Zisapel N, Elizur A. First-night effect of melatonin treatment in patients with chronic schizophrenia. J Clin Psychopharmacol. 2000;20:691–694.