A Systematic Review of Instruments for the Assessment of Insomnia in Adults

Introduction

Worldwide reports suggest that around one-third of the adult population complains of insomnia symptoms.^1–2 Insomnia is characterized by persistent sleep difficulty despite adequate sleep opportunity and associated daytime dysfunction.³ Insomnia has been associated with increased rates of road accidents, lower productivity and work absenteeism.^4,5 Several factors are known to contribute to the development or worsening of insomnia symptoms. Current findings from the literature suggest that negative cognitive processes including worry, rumination and catastrophizing thoughts are associated with worse sleep quality and insomnia.⁶ These processes have been shown to fuel anxiety and arousal resulting in delayed sleep onset and shorter sleep duration.⁷ Similarly, poor sleep hygiene, which consists of a combination of behavioral practices and environmental conditions which improves sleep, is common among insomniacs.^8–11

Studies suggest that chronic insomnia is an independent risk factor for developing mental illnesses in otherwise healthy individuals.^12,13 Chronic insomnia has also been reported to increase the probability of developing chronic medical conditions, such as hypertension, cardiovascular disease and type 2 diabetes.^14,15 Therefore, early detection and management of insomnia is important to minimize these associated risks. Although polysomnography (PSG) is considered the gold standard method for evaluating insomnia, it is not routinely used as it requires a specialized setting and equipment, and it is often labour intensive. Wrist actigraphy is another objective tool for the assessment of insomnia, but is limited by its lack of specificity.^16,17 Consequently, self-reported sleep instruments remain the most practical methods for the assessment of insomnia in clinical practice.^16,18 There are several systematic reviews on a range of sleep instruments utilized for the assessment of sleep dysfunction in a variety of medical and neuropsychiatric disorders, or in specialized populations.^19–21 In these reviews, there was insufficient evaluation of the psychometric properties of the instruments that were identified, and none of them included instruments which assess sleep hygiene practices in the adult population. Therefore, the main objective of this study is to present an updated systematic review of the literature on validated self-reported instruments used for the assessment of different dimensions of sleep in the adult population. In addition, the review also aimed to identify sleep instruments available in Arabic language.

Methods

Search methods

Data Sources and Search Strategy

A comprehensive systematic search was conducted to identify studies reporting the development and/or validation of instruments for the assessment of self-reported sleep and sleep hygiene in the adult population. The five databases and search engines utilized included PubMed (1966 - April 2018), EMBASE (1980 - April 2018), ProQuest Central (1947–2018), SCOPUS (1966 - April 2018), and Google Scholar (till April 2018). Grey literature was also searched by reviewing conference proceedings and abstracts of the Canadian Sleep Society and the American Academy of Sleep Medicine published in the period from January 2014 to December 2017. Additionally, a hand search of the bibliographies of the articles identified through the electronic databases search was undertaken. Search terms used in the electronic databases search were classified into three categories related to: sleep dysfunction (Category A), the instrument for assessment (Category B), and validation and psychometric properties (Category C). Terms from Category C were not used in databases such as PubMed which offered “validation studies” as one of the limits or filters. Publication language was limited to English and no limits were imposed on the publication year. Examples of full search strategies for two of the electronic databases can be found under the supplementary material. In addition to the English search, two separate searches were conducted to identify sleep instruments developed in Arabic language or translated into Arabic. The first used the same search terms as those used in the original search, in addition to the word “Arabic”. The second was conducted in Arabic language in the following databases: PubMed, Google Scholar and Dar Al-mandumah using the same search terms as those used in the original search.

Inclusion Criteria

Articles selected for full review were those providing results of validation studies or reporting psychometric properties of instruments and questionnaires assessing characteristics of sleep (quality, quantity, nocturnal awakenings), daytime consequences of poor sleep, or sleep hygiene in the adult population. The inclusion criteria were limited to instruments completed by self-report and to instruments written in English or Arabic.

Exclusion Criteria

Studies were excluded from the review if they included evaluating instruments designed to measure sleep disorders other than insomnia (eg obstructive sleep apnea, restless leg syndrome, etc.), those using sleep items as subdomains of an instrument assessing a condition other than insomnia, studies which focused on pediatric/geriatric populations, studies describing instruments designed to be completed by clinician or caregiver and not by the patient, using instruments developed in languages other than English or Arabic, or describing instruments not psychometrically validated.

Data Collection and Analysis

Data Collection Process and Data Extraction

Duplicate citations were removed after obtaining the initial records of relevant articles from different databases. Titles and abstracts of the articles were then screened for relevance. Full-text of eligible articles were obtained and screened against the inclusion and exclusion criteria. Data were extracted from the eligible studies according to eight key attributes, as established by the Scientific Advisory Committee, Medical Outcomes Trust (SAC-MOT).²² The extraction tool used in this study included the following elements: instrument’s name, authors, conceptual framework (domains and purpose), psychometric properties, validation population, general description of the instrument (number of items, scale, scoring, response format, burden), and cultural and linguistic adaptation. The psychometric properties were extracted from studies describing the validation of the original (English or Arabic) version of the instruments. The psychometric properties extracted from the validation studies included validity, reliability, responsiveness, and interpretability. In addition, instruments’ attributes related to the validation sample were extracted. Data extraction was done by one reviewer (RA) and reviewed by a second author (MZ). The psychometric properties for instruments validated in specific populations were not extracted.

Quality Assessment

A modified version of the criteria developed by Terwee et al²³ was used to assess the quality of the sleep instruments. The Terwee at al. criteria evaluates 8 psychometric properties including validity (content, construct and criterion), reliability (internal consistency and reproducibility), responsiveness, interpretability and floor and ceiling effects.²³ For the purposes of this study, the criteria for the floor and ceiling effects and the agreement component of the reproducibility were not used, primarily because, for the most part, these measures were not reported in the validation studies included in this review. In addition, Pearson’s Correlation Coefficient was used for rating the reliability, responsiveness, criterion and construct validity.

Results

As illustrated in Figure 1, a total of 4453 citations were retrieved from the search. One hundred and seven articles were deemed suitable for inclusion in the review. These articles included 31 distinct sleep instruments. Of the 107 articles included, 47 discussed the validation process of the instruments in English, while the remaining 60 articles reviewed the translation and cultural adaptation of these instruments into a variety of languages and populations. The two additional searches to identify sleep instruments in the Arabic language did not identify any additional results than those derived from the original search. Table 1 summarizes the results of the validation studies (n=47) related to the 31 sleep instruments included in this review. The table also describes the psychometric properties of the instruments and the characteristics of the populations in which they were validated. Table 2 provides a detailed description of the characteristics of the 31 instruments.

Table 1 Studies Testing the Psychometric Properties of Extracted Sleep Instruments

Table 2 Characteristics of Sleep Instruments Extracted from Selected Studies (n=112)

Figure 1 Flow chart of the systematic review process.

As summarized in Tables 1 and 2, the majority of the instruments identified contain less than 20 items. The longest of these instruments is the Sleep Practices and Attitudes Questionnaire (SPAQ), which contains 151 questions divided into 16 different domains.⁷⁵ While the Karolinska Sleepiness Scale (KSS),²⁴ the Minimal Insomnia Symptoms Scale (MISS),^25,26 and the Jenkins Sleep Scale (JSS)^27–73 are the shortest instruments identified, consisting of one, three, and four questions, respectively. Some of the instruments have multiple versions, each consisting of a different number of questions [eg, the Functional Outcomes of Sleep Questionnaire (FOSQ)-30 and the FOSQ-10 have 30 and 10 questions, respectively].^74,28 The majority of the instruments use Likert-type scales as response options to generate scores. The time needed to complete an instrument was not reported in the majority of the studies reviewed. However, the response burden, wherever reported, did not exceed 10 minutes. The recall period for the majority of the instruments was one month, except for the Insomnia Severity Index (ISI)^29,30 and the the Sleep Functional Impact Scale (SFIS)³¹ which had a recall period of 2 weeks and 1 week, respectively. Only four of the identified instruments were available in Arabic, three of which (PSQI,³² ISI,³³ and ESS³⁴) were originally developed in English, but translated and validated in Arabic-speaking populations. The Arabic Scale of Insomnia (ASI)³⁵ is the only instrument which was originally developed in Arabic.

The instruments were classified into four categories based on the outcomes that were assessed in the 107 studies, as follows: (1) instruments screening for insomnia symptoms (n=14); (2) instruments assessing consequences of poor sleep (n=8); (3) instruments assessing the cognitive aspect of insomnia (n=5) and; (4) instruments evaluating sleep hygiene (n=4). A detailed description of the reported psychometric properties for the four categories of sleep instruments retrieved from the studies included in this review is provided in Table 3.

Table 3 Quality Assessment of Extracted Sleep Instruments* (n=31)

Instruments Screening for Insomnia Symptoms

From the 14 instruments screening for insomnia symptoms, two [the Women’s Health Initiative Insomnia Rating Scale (WHIIRS)^36,37 and the Restorative Sleep Questionnaire (RSQ)³⁸] were dimension-specific and focus on evaluating the sleep quality through assessing problems with initiation and maintenance of sleep. The remaining 12 instruments were multidimensional, assessing sleep quality and consequences of poor sleep. Some of the identified instruments also measure other specific dimensions such as satisfaction with sleep in the Bergen Insomnia Scale (BIS)³⁹ and sleep environment in the Insomnia Screening Scale (ISS).⁴⁰ While the Daily Cognitive-Communication and Sleep Profile (DCCASP)⁴¹ measures the daily fluctuation in sleep quality and evaluates the impact of these fluctuations on the individual’s cognitive functioning and communication, while the General Sleep Disturbance Scale (GSDS)^42,43 assesses sleeping patterns and the use of sleep aids during the past month.

Instruments Assessing Consequences of Poor Sleep

These instruments measure an individual’s functioning and daytime performance. Two of these instruments [the Epworth Sleepiness Scale (ESS^44–46 and the KSS)^24,47 focus on assessing sleepiness only, whereas the Hyperarousal Scale (H-Scale)^48,49 also assesses daytime alertness. Three instruments, the FOSQ,^74,28 Occupational Impact of Sleep Questionnaire (OISQ),⁵⁰ and the Sleep Functional Impact Scale (SFIS),³¹ evaluate the effects of insomnia on overall functioning. Conversely, the Sleep Disturbance Questionnaire (SDQ)⁵¹ evaluates factors which contribute to poor sleep, and the Maugeri Sleep Quality and Distress Inventory (MaSQuDI-17)⁵² measures the emotional burden of insomnia.

Instruments Assessing Cognitive Aspects of Insomnia

These were in line with the ICD-10 criteria which identifies preoccupation with sleepiness and excessive worry about the consequences of insomnia during the day as one of the clinical features for insomnia.⁵³ Four of these instruments [Insomnia Daytime Worry Scale (IDWS),⁵⁴ Anxiety and Preoccupation about Sleep Questionnaire (APSQ),^55,56 Sleep Preoccupation Scale (SPS)⁵⁷] evaluate the extent of worry about insomnia. The Dysfunctional Beliefs and Attitudes about Sleep Scale (DBAS) explores individuals’ perceptions and beliefs regarding insomnia.^51,58,59 While the Catastrophic Thoughts about Insomnia Scale (CTIS)⁶⁰ assess exaggerated thinking about insomnia and the severity of the consequences.

Instruments Assessing Sleep Hygiene

Out of the four instruments included under this category [Sleep Hygiene Index (SHI),⁶¹ Sleep Hygiene Awareness and Practice Scale (SHAPS),⁶² Sleep Hygiene Self-test (SHS)⁶³ and the SPAQ⁷⁵], only the SPAQ is a comprehensive instrument which evaluates sleep quality, quantity, consequences of poor sleep, and sleep hygiene. The SHS also evaluates the effects of interventions which are expected to improve sleep hygiene, and is the only sleep-hygiene instrument which has been validated for responsiveness. The SHAPS measures knowledge about sleep hygiene in addition to behaviors surrounding sleep.

Some of the sleep instruments which were validated in the 107 articles included in this review, such as the BIS,³⁹ Sleep Condition Indicator (SCI),⁶⁴ ISS, Athens Insomnia Scale (AIS)^65,66 and Sleep Hygiene Index (SHI)⁶¹ were developed in accordance with international diagnostic criteria for insomnia such as the International Classification of Sleep Disorders 2^nd edition (ICSD-II), International Classification of Diseases 10^th edition (ICD-10), and the Diagnostic and Statistical Manual of Mental Disorders (DSM). The majority of these instruments were validated among university students and in primary insomniacs.

Many of the identified sleep instruments have been translated and culturally adapted in variety of languages, such as the Pittsburgh Sleep Quality Index (PSQI),^67–69 ISI, and ESS which are available in 18, 10, and 6 languages, respectively.

Quality Assessment

The majority of the instruments used in the 107 studies included in this review did not meet all of the eight criteria set by the SAC-MOT.²² However, the conceptual and measurement models were specified for all instruments.Additionally, the reliability of sleep measures was usually reported in the validation studies, particularly the internal consistency reliability was reported for all instruments, except for the H-scale and the KSS. The reported internal consistency values for the sleep and sleep hygiene instruments across the studies included in this systematic review ranged between 0.53 and 0.97.

As outlined in Table 3, the test-retest reliability was only reported for 15 of the 31 instruments. Validity measures were reported for almost all of the instruments identified (29 out of 31), of which criterion validity was the most commonly reported and content validity the least reported validity measure. The generalizability of the psychometric properties reported was examined for three instruments (ISS, ISI and AIS) in community samples and primary insomniacs. Only six instruments (MaSQuDI-17, ESS, FOSQ, DBAS, APSQ, SPS) provided evidence of discriminating capacity between healthy individuals and poor sleepers. As summarized in Table 1, some of the sleep instruments were tested for other validity measures such as incremental validity, diagnostic validity, and external validity.^{40,64-67,70,76} Additionally, the SHS was the only instrument that was assessed for responsiveness to change in sleep hygiene after interventions.⁴⁶ With the exception of SHAPS, all of these instruments had low internal consistency (α < 0.7) and only the SHI and the SHAPS were evaluated for their reproducibility.^75,61–63

Discussion

This study identified 31 sleep instruments and described in detail their psychometric properties as described in 107 validation and cultural adaptation studies. In the validation studies included in this review, only the ISI, FOSQ and MISS were evaluated for all four psychometric properties such as responsiveness, interpretability, reliability and validity.^{25,73-28,69,76} Because most validation studies try measuring the same construct in different settings, it was expected that the sleep instruments in these studies were mostly tested for their reliability. For the most part, the sleep instruments identified in this systematic review had good internal consistency with Cronbach’s alpha values ≥0.7, which is in line with the recommended threshold for adequate reliability reported in the literature.⁷⁷ However, overall lower Cronbach’s alpha values were reported for instruments measuring sleep hygiene. It has been suggested that the low internal consistency identified for sleep hygiene measures could be due to the definition of sleep hygiene, which consists of different factors (not necessarily related) that have the potential for negatively affecting sleep.⁶¹ Thus, items in sleep hygiene instruments may appear to be poorly related in the internal consistency test. This suggests that when assessing the psychometric properties of sleep hygiene instruments, the use of other validity and reliability measures is recommended to avoid depending on Cronbach’s alpha values alone.

The validation studies included in this systematic review reported on the reproducibility of only 15 out of the 31 instruments identified. Mostly were assessed through the test-retest reliability method. However, the time frame between the two measurements was highly variable, ranging between 2 days and 6 months.^27,69 There is still no consensus in the literature on the best time interval for reproducibility tests. Findings from studies that compared between different time intervals suggests that test results from an interval of 2 days and 2 months are similar to those resulting from a 2-weeks’ interval.^78,79

Criterion validity was the most commonly reported validity measure in the studies included in this review. Concurrent validity (which compares the score of the instrument to that of a gold standard which assesses the same construct) was evaluated for 18 out of the 31 sleep instruments identified, whereas the predictive validity (which provides evidence of an instrument’s ability to forecast a particular outcome in the future) was assessed only for 5 instruments. Considering that most of the validation studies included in this review were done on newly developed instruments, not prioritizing on assessing their predictive validity is understandable. Some of the validation studies such as those done on ISI, RSQ and PSQI, compared the scores generated by the instruments to non-subjective assessment tools such as the PSG.^29,37,65 The quality criteria for questionnaires developed by Terwee et al recommends providing the reasons for selecting a measure to be a gold standard in a validation study.²³ However, in the studies included in this systematic review, the choice of the instrument to be used as a comparator or as a gold standard was rarely justified. In situations where a gold standard measure is not available, construct validity could be used to assess the instrument’s validity.⁸⁰ This is the case for some of the instruments included in this review, such as the RSQ, APSQ and SHAPS, which evaluate new sleep-related concepts for which gold standards are not available.

The diagnostic validity of an instrument is an important measure, particularly for clinicians, as it examines the extent to which the instrument can accurately differentiate between healthy individuals and insomniacs at a specific score.⁸¹ Diagnostic validity was reported for only five of the instruments identified in this review (specifically the PSQI, ISI, AIS, MISS and the SCI). Incremental validity (which describes the ability of an instrument to predict a variable of interest beyond what is possible by other existing instruments) was only tested for the Sleep Quality Questionnaire (SQQ).⁶² This measure is important as it facilitates the comparison between different instruments and provides evidence of which instrument is superior.

The selection of an insomnia instrument in practice is also affected by factors such as the length of the instrument, the time required for completion and the languages in which it is available. Around half of the instruments identified in this review consisted of ten questions or less, and those assessing the cognitive aspects of insomnia or sleep hygiene were usually longer. However, the duration required for completion was not commonly reported. Only six instruments were available in more than three languages, and only four in Arabic language.³⁴ The limited availability of instruments in Arabic language offers a valuable opportunity for researchers interested in sleep medicine in the Arab region to translate and validate these instruments.

Despite the comprehensiveness of this systematic review, the time elapsed between the initial database search and the publication of the results is an important limitation. To address this, a supplementary literature search using the same search strategy and databases was undertaken in April 2020, which identified additional validation studies and 7 new instruments including: the Insomnia Catastrophising Scale (ICS),^82,83 the Single-item sleep quality scale (SQS),⁸⁴ the Lebanese insomnia scale (LIS-18),⁸⁵ the Daytime Sleepiness Perception Scale-4 (DSPS-4),⁸⁶ the Indian Sleepiness Scale (InSS),⁸⁷ Athlete Sleep Behavior Questionnaire (ASBQ),⁸⁸ and the Non-restorative Sleep Scale (NRS)⁸⁹. These findings are indicative that the improvement in scientists’ understanding of insomnia pathophysiology has been associated with a parallel increase in the development of instruments which measure different aspects of sleep dysfunction and related risk factors. Researchers also appear to be acknowledging the cultural effects on sleep behaviors and insomnia perception, as two of the newly developed instruments (LIS-18 and InSS) were developed to describe sleep from a non-Western perspective. Comparing between the psychometric properties of insomnia instruments across different populations is interesting and could be the focus of a future investigation. In addition, because this systematic review was focused on identifying instruments that assess the symptoms of insomnia, daytime consequences and those related to sleep hygiene, instruments assessing arousal before sleep only such as the Pre-Sleep Arousal Scale (PSAP)^90,91 were excluded.

Conclusion

Different instruments are available for evaluating various aspects of sleep and sleep hygiene. The validity and reliability of most of these instruments have been tested and are well established. However, psychometric testing for several of the available sleep instruments remains incomplete, particularly responsiveness and interpretability. Our findings suggest that future studies should focus on reporting more psychometric measures to ensure the trustworthiness of the findings generated by these instruments. The number of sleep instruments available in languages other than English including Arabic are limited, indicating a need to translate and culturally adapt many of these instruments into various languages to be available for use in clinical practice and research in different populations.