Validity and Reliability of the Turkish Version of the Sleep Condition Indicator: A Clinical Screening Instrument Based on the DSM-5 Criteria for Insomnia

Introduction

Although insomnia is one of the most common mental disorders, physicians often assess insomnia as a symptom only.^1,2 Insomnia is not just a temporary or benign symptom but a diagnostic-level disorder. Once established, insomnia becomes chronic without treatment and constitutes a risk factor for developing further physical and mental illnesses such as hypertension, diabetes mellitus, obesity, depression, and suicide.^3–7 Chronic insomnia also leads to work disability, poor quality of life, and high societal costs.^8,9 Parallel to these negative outcomes, The Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) recommends eliminating the primary-secondary dichotomy of insomnia.¹⁰ It codes insomnia as a disorder whenever the diagnostic criteria are met, whether or not there is a co-existing physical, mental, or sleep disorder.¹⁰

The clinical interview is the gold standard for diagnosing insomnia disorder.^11,12 Actigraphy, polysomnography, and a prospective sleep diary are additional tools for assessing insomnia.¹³ In clinical practice, self-report questionnaires stand out for assessing insomnia disorder because of time scarcity, polysomnography is not indicated for the diagnosis of insomnia, and actigraphy is not readily available in most health practices.^14,15 Questionnaires can also be used to identify individuals who may have poor sleep [PSQI], insomnia symptoms [ISI] or who may meet criteria for insomnia disorder [SCI] and require further diagnostic evaluation by a sleep medicine professional. The Pittsburgh Sleep Quality Index (PSQI)¹⁶ and the Insomnia Severity Index (ISI)¹⁷ are widely used self-reported scales in insomnia.¹⁸ The PSQI and ISI have been validated in Turkish and are extensively utilized clinically and in sleep research conducted in Turkey. The PSQI determines sleep quality and has a cut-off score to distinguish between good and poor sleepers. But it is not specific to insomnia.¹⁶ On the other hand, the ISI based on DSM-IV is very robust psychometrically and it is more typically used for insomnia.^14–19 Still, it does not assess the duration of insomnia symptoms.¹⁷ Moreover, neither the PSQI nor ISI was designed to evaluate the insomnia disorder diagnosis based on the DSM-5.²⁰

The Sleep Condition Indicator (SCI) is an eight-item self-report scale developed by Colin Espie and colleagues based on the DSM-5 insomnia disorder diagnostic criteria.^10,21 The SCI assesses nighttime and daytime insomnia symptoms and focuses on the frequency and persistence of these symptoms. The SCI has been adapted to many languages,^{15,20,22–25} and validation studies have confirmed robust psychometric properties of the SCI. In addition, studies show that SCI has a cut-off value for insomnia disorder and can distinguish between patients with insomnia diagnosed according to the DSM-5 criteria and healthy controls.²¹ SCI has not been adapted to Turkish, and its psychometric properties have yet to be examined in Turkey.^26,27

Although using a scale to screen for or diagnose any disease will save time and resources, it’s possible that a scale measuring the construct of interest is not readily available or that the scale is not in the language of the intended respondents. Therefore, researchers may need to develop a new survey or translate the existing scale into the language of the targeted respondents—translating and adapting the scale into a different language provides an additional resource for future research. In the current study, we first aimed to adapt the SCI to Turkish and to examine its psychometric properties in the general population. For the first time, we examined the factor structure of the SCI with confirmatory factor analysis (CFA), multigroup CFA, and network analysis, going beyond the exploratory factor analysis (EFA). We examined the correlations between the PSQI, ISI, and SCI and measured the temporal stability of the SCI. We also analyzed the discriminant validity of SCI according to the ISI cut-off value for insomnia disorder.

Materials and Methods

Participants, Procedure, and Sample Size

Our study targeted the general population in Turkey. We used the Google online form system to create a link that included an informed consent form, a sociodemographic data form, and the Turkish version of the SCI, PSQI, and ISI. The research team verified the functionality of the online link. When participants clicked on the link, they first encountered the consent form, which provided information about the study’s purpose, dates, organizers, ethical approval, data confidentiality, and the retest analysis one month later. Participants who consented completed the sociodemographic data form, the Turkish version of the SCI, PSQI, and ISI. The study link was active from March 1, 2022, to April 1, 2022, and was shared on Twitter and Facebook platforms in Turkey. A total of 834 participants completed the study during this period. For the retest analysis, participants provided their email addresses. After one month, we sent the link, including only the Turkish version of the SCI, to each participant. No incentives were offered to participants for their participation.

In the literature, 5–10 participants per scale item are sufficient for a scale validity and reliability study.²⁸ With 90% power, RMSEA of 0.05, and degrees of freedom of 13 for the confirmatory factor analysis, the minimum required sample was 696. In this study, 834 participants were determined to provide adequate power. In addition, the number of participants in scale validity and reliability studies is recommended to be close to 1000 in the literature.²⁹ The fact that the number of our participants was close to 1000 suggests the sample size 834 was sufficient.²⁹ This study was approved by the ethics committee of Gaziantep Islamic Science and Technology University (approval date and decision number: 07.01.2022 /73). Our study was conducted in compliance with the Declaration of Helsinki.

Sociodemographic Data Form and Sleep Instruments

A sociodemographic data form was prepared by the researchers inquiring about the participants’ age, gender, marital status, employment status, smoking, having a chronic disease (hypertension, diabetes mellitus, asthma, etc.), history of suicide attempts and current suicidal ideation.

Espie et al developed the SCI based on DSM-5. We obtained permission from the original author of the scale.²¹ According to the literature, we developed the Turkish version of the SCI after translation, back-translation, and control process.^30,31 Five sleep medicine experts autonomously translated the SCI English form into Turkish. Subsequently, the translation team discussed every Turkish version and ultimately selected the final scale. The Turkish version of the SCI underwent translation back into English. We subsequently shared it with the original authors and Dr. Christopher B Miller, a co-author on this manuscript and member of the research team who had previously been involved in the validation studies of SCI. They verified that no problems had been found in the SCI’s back translation. The SCI items evaluate difficulty in initiating sleep (item 1), maintaining sleep (item 2), sleep quality (item 4), daytime sleep-related symptoms (item 5, item 6), duration of sleep problems (item 8), nights per week having a sleep disturbance (item 3), and extent troubled by poor sleep (item 7). It has a 5-point Likert structure. The total score is obtained by summing up all the items, and there is no reverse-scored item. The total score ranges from 0 to 32; lower scores indicate worse sleep. A total of 16 scores or less is defined as insomnia disorder.²¹

The PSQI is the most commonly used self-rated psychometric tool for assessing sleep quality. It evaluates sleep quality in the last month. It consists of 19 questions and seven sub-parameters related to sleep quality. The sum of the scores of these sub-parameters yields the total score, and the total score ranges from 0 to 21.¹⁶ PSQI sum score > 5 shows poor sleep quality.³² The PSQI is not explicitly designed for insomnia, but sleep quality is highly correlated with insomnia.¹⁶ The Turkish version of PSQI has already been validated.³³ In this study, the PSQI also demonstrated high internal consistency reliability (Cronbach α = 0.88).

The ISI consists of seven questions and assesses individuals’ insomnia in the last two weeks. It has a 5-point Likert structure, the total score ranges from 0 to 32, and a higher score indicates more severe insomnia. Clinical insomnia is defined by an ISI score of 15 or higher.¹⁷ The Turkish version of the ISI has already been validated and has good psychometric properties.³⁴ In this study, ISI also showed strong internal consistency reliability (Cronbach α = 0.91). It is the most common scale used in the literature to distinguish between those with and without clinical insomnia in sleep-related studies.^17,34 We divided all the participants into two groups according to the ISI, with and without clinical insomnia. We used the ISI to calculate the cut-off value of the SCI.

Statistical Analysis

Data were analyzed using RStudio, JASP 0.16.2.0, and SPSS 23.0 (IBM Corp., Armonk, NY, USA). We presented descriptive statistics as means, standard deviations, and frequencies. The skewness-kurtosis value was calculated to determine the normality of the data. The item-total correlation, standard error of measurement, Cronbach’s α, and McDonald’s ω were used for the SCI’s internal consistency (0.7 < α < 0.9 and 0.7 < ω < 0.9 was considered acceptable).³⁵ We run confirmatory factor analysis (CFA) to confirm the factor structure compared to the literature. In CFA, we accepted the following values for good fit indices: χ2/df (chi-square/degree of freedom) less than 3.0, root-mean-square-error of approximation (RMSEA) less than 0.06,³⁶ comparative fit index (CFI) greater than 0.95, goodness-of-fit statistic (GFI) greater than 0.95,^37,38 and a standardized root-mean-square residual (SRMR) close to or below 0.08.³⁹ Multigroup CFA was run to assess the measurement invariance across gender, having clinical insomnia or not (ISI ≥ 15),³⁴ and having poor sleep quality or not (PSQI > 5).³² In the present study, network analysis was also run to assess the psychometric properties of the Turkish SCI. In this analysis, we calculated edge weights (the association between items), the stability of edges (assessed using the non-parametric bootstrap method), centrality indices (betweenness, closeness, and strength), and the stability of the centrality indices (assessed using the case-drop bootstrap method).⁴⁰

To evaluate construct validity, we investigated the correlation between SCI, ISI, and PSQI. It was hypothesized that SCI scores should correlate strongly (r > 0.7) with ISI and PSQI scores. A receiver operating characteristic (ROC) curve was plotted to evaluate the discriminating abilities of the SCI. The area under the curve (AUC) provides a value of the ability to discriminate clinical insomnia from non-clinical insomnia. Diagnostic power according to AUC values; < 0.6 = fail, 0.6–0.7 = poor, 0.7–0.8 = fair, and 0.9–1 = very good.⁴¹ We calculated the sensitivity and specificity percentages to evaluate the diagnostic ability of Turkish SCI. We used the ISI as a proxy since we did not use a clinical interview for insomnia diagnosis. We considered those with an ISI score of 15 or higher as clinical insomnia, as is commonly accepted in the literature.^17,34 P-values less than 0.05 were considered statistically significant.

We requested all participants fill out the SCI again using the e-mail link one month later. We calculated the correlation between their first SCI score and their last SCI score. We performed a retest analysis to evaluate the temporal stability of SCI. The temporal stability was examined with the intraclass correlation coefficient and a paired t-test.

Results

Sample Characteristics

Table 1 shows the sociodemographic data and clinical variables of all participants. There were a total of 834 participants. More than half of the participants were women (63.2% n = 527), and the mean age was 36.15 ± 9.64 (min-max: 17–71). The mean BMI values of the participants were 25.17 ± 4.37 (min-max: 18.01–40.4). The mean education year was 11.50 ± 5.23. Less than half of all participants were married (42.30% n = 353), and most were working (78.10% n = 651). The mean SCI score of all participants was 22.23 ± 6.84, and according to the ISI, 7.3% (n = 61) had clinical insomnia.

Table 1 Sociodemographic and Clinical Variables of All Participants (n = 834)

Item-Level Properties of the Turkish Sleep Condition Indicator

Item-level properties of the Turkish SCI are presented in Table 2. Skewness (between −1.28 and −0.51) and kurtosis (between −1.39 and 0.84) values show that the normality assumption is met. Item analysis results found all items acceptable with a corrected item-total correlation (≥ 0.30)⁴² suggesting they can discriminate between higher and lower scorers in the Turkish SCI.

Table 2 Item Level Properties of the Turkish Sleep Condition Indicator

Scale-Level Psychometric Properties of the Turkish Sleep Condition Indicator

Scale-level psychometric properties of the Turkish SCI are presented in Table 3. Confirmatory factor analysis results show that the two-factor correlated model had a good model fit for the Turkish SCI (χ²= 4.933, df = 13, p = 0.977, CFI = 1.000, GFI = 0.999, RMSEA = 0.000, SRMR = 0.017) (Table 3). Factor loadings ranged from 0.331 (Item 4) to 0.817 (Item 3) for the Sleep Pattern subscale and between 0.890 (Item 5) and 0.924 (Item 7) for the Sleep-related impact subscale (Table 2, Figure 1). Although Item 3 had a lower factor loading, it was still significant. Both subscales were moderately correlated with statistical significance (r = 0.664, p < 0.001). Table 4 also shows that the Turkish SCI had good internal consistency reliability at the full-scale level. Both subscales also had acceptable internal consistency reliability. Multigroup CFA results (Supplementary Table 1) show that the Turkish SCI had scalar level invariance across gender, clinical insomnia, and poor sleep quality. This finding suggests the scale assesses similar constructs for both males and females, clinical insomnia or not, and poor vs good sleepers. As hypothesized, SCI had a high correlation with ISI total scores (r = −0.804, p < 0.001, 95% CI [−0.779, −0.827]) and PSQI total scores (r = - 0.799, p < 0.001, 95% CI [−0.774, −0.823]). These strong correlations suggest the convergent validity of the Turkish SCI.

Table 3 Scale Level Properties of the Turkish Sleep Condition Indicator

Table 4 Weight Matrix of Items of the Turkish Sleep Condition Indicator

Figure 1 Factor Structure of the Turkish Sleep Condition Indicator.

Network Analysis

Figure 2 shows the network plot of the Turkish SCI. Figure 2 shows that Items 1–4 and 8 are grouped, and 5–7 are grouped. These groups suggest a two-factor structure. In this figure, all the edges in blue show a positive association between items. The weight matrix between nodes (items of the Turkish SCI scale) is presented in Table 4. Five edges have moderate effect size (Item 1 and Item 3 [0.305], Item 3 and Item 8 [0.449], Item 5 and Item 6 [0.480], Item 5 and Item 7 [0.238], and Item 6 and Item 7 [0.594].

Figure 2 Network Plot of the Turkish Sleep Condition Indicator. Abbreviation: SCI, sleep condition indicator.

Further edges have a smaller effect size or no effect. Regarding the accuracy of the edge weights (see Supplementary Figure 1), 11 estimated edge weight include zero. The rest of the edge weights do not include zero, indicating good precision for these remaining edge weights: Table 5 and Supplementary Figure 2 present the centrality measures of items. Item 6 has the highest strength (0.979), and Item 4 has the lowest (−1.627). These results suggested that Item 6 has more relative importance than other Turkish SCI items. Supplementary Figure 3 shows the correlation stability coefficients for centrality measures across subsets of the overall sample ranging from 95% to 25%. This figure shows that the drop in centrality measures across subsamples is not lower than 0.75, demonstrating the stability of centrality indices.

Table 5 Centrality Indices of the Turkish Sleep Condition Indicator

Discriminate Analysis

Receiver operator characteristic (ROC) curve analysis shows that the Turkish SCI has good sensitivity (90.3%) and specificity (91.8%) using ≤15 as a cut-off. The area under the ROC curve (Figure 3) is 0.963 (95% CI [0.948, 0.979]).

Figure 3 A Receiver Operating Characteristic Curve (ROC) of the Turkish Sleep Condition Indicator for Identifying People Having Insomnia.

Test Re-Test Reliability

One hundred sixteen (response rate 13.90%) participants completed the Turkish SCI one month after the first administration. Table 6 shows descriptive and item-total statistics of the first and second administration SCI. Descriptive results for its first and second administrations were similar. The intra-class correlation coefficient computed between the first and second SCI total scores was significant (r = 0.80 with a 95% confidence interval from 0.78 to 0.87; P < 0.001), suggesting adequate temporal stability. A paired t-test revealed that the total score of the SCI did not differ significantly from the first (23.91 ± 7.21) to the second (24.33 ± 6.13) administration (t [116] = −0.65, P = 0.53). These results suggest that Turkish SCI has adequate temporal stability.

Table 6 Descriptive and Item-Total Statistics of the Turkish Sleep Condition Indicator

Discussion

The present study aimed to adapt the SCI to Turkish and examine its psychometric properties. To the best of our knowledge, this is the first study to address the psychometric properties of the Turkish SCI. In addition, our study is the first to reveal the factor structure of the Turkish SCI in a general population sample using CFA and network analysis. The present study showed that the Turkish version of SCI has a two-factor structure with good internal consistency, reliability, and stability over time. Turkish SCI assesses a similar construct for both males and females, having clinical insomnia or not, and poor and good sleepers. In addition, a total score of 15 and lower in the Turkish version of SCI can predict insomnia disorder with 90.3% sensitivity and 91.8% specificity.

We examined the factorial structure of SCI with network analysis and CFA. Previous studies utilized EFA.^15,20–24 CFA has advantages over EFA. First, CFA reduces measurement error and enables comparison of alternatives. In addition, the application of CFA to investigate the scale’s construct validity increases the statistical precision.^43,44 The network approach is a newly developed perspective for understanding psychopathology or associated syndromes. It is an exploratory approach that provides quantitative information and visuals about the association between symptoms and the strength of these associations. This approach helps to identify the central sign and unique interactions with other symptoms which could not be identifiable in a multiple regression model.^45,46 This is the very first study that utilized network analysis to explore the psychometric properties of the SCI. We obtained a two-factor structure with items 1–4 and 8 loading on the sleep pattern subscale and items 5.6 and 7 loading on the sleep-related impact subscale in both CFA and network analysis. The Turkish version of SCI had good fit indices in CFA. Only the fourth item (how would you rate your sleep quality?) had a lower factor loading (0.331) than the other items. Still, we did not consider removing this item from the scale because it was above the value accepted in the literature⁴³ and was significant. In the network structure (Figure 2), item 4 has a lower association with items 3, 6, 7 and 8. However, comparing the strength, item 4 has a stronger association with item 8, while the rest of the associations are very marginal. In the Chinese version study, the fourth item was included in both factors. The author argued that this may be due to cultural difference.¹⁵ These factor analyses align with evidence from the SCI’s Italian, French, and Chinese versions.^15,20,22 The two components reflect two major complaints about insomnia. The first can be defined as the pattern of insomnia, and the second is the impact of insomnia on function. This duality has also been demonstrated in Espie’s study (2014), which represents a large sample of open-access web-based adults (The Great British Sleep Survey, n = 30,941).²¹ Only the Swedish version of the SCI study found a single-factor structure.²⁴ We advocate a two-factor structure in line with the widespread evidence in the literature. Another major strength of the present study is the measurement invariance across gender, clinical insomnia, and poor sleep quality. This scale has scalar level measurement invariance that supports that this scale assesses the same construct and is usable to determine insomnia across males or females, having clinical insomnia symptoms or not, and poor sleeper or not. This study is also the first to assess the factor structure’s invariance across different sub-groups of the target population.

In the present study, we found good internal consistency reliability (Cronbach’s Alpha and McDonald’s Omega) of the Turkish SCI. In addition, the full scale’s test-retest reliability and two Turkish SCI sub-scales were good. Other SCI version studies have shown that the SCI has good internal consistency and test-retest reliability.^15,20–25 We found a strong correlation between the SCI, ISI, and PSQI. This finding demonstrates that the construct validity of the SCI was good. These strong correlations are in line with other studies.^15,20,21

The prevalence of insomnia varies considerably across different countries, ranging from 6 to 76.3% depending on diagnostic and screening methods used.⁴⁷ In an international survey in France, Italy, Japan, and the USA, the estimated prevalence of insomnia ranged between 6.6% and 37.2%.⁴⁸ In a study conducted in Turkey in 2015 based on DSM-IV diagnostic criteria, insomnia disorder was found to be 15.3%.⁴⁹ In another study conducted in Turkey in 2015, the prevalence of insomnia among 18–24 year olds was found to be 9.8%.⁴⁷ In our study, the insomnia rate seemed relatively low. Our study was an opportunistic internet-based sample and was not designed to be nationally representative or evaluate the prevalence rate of insomnia. Because of this, they are not similar to national data. However, similar rates are also available in the literature. A score above eight on the ISI is the threshold value for insomnia. In this study, we evaluated 15 or above for clinical insomnia. The low rate could result from people portraying themselves as better than they are, or it could be related to this cut-off value. In our study, we did not conduct clinical interviews about insomnia disorder. We classified the clinical insomnia group as those with an ISI total score of 15 or more. We determined the SCI cut-off value according to this separation. A total SCI score of 15 or less had 90.3% sensitivity and 91.8% specificity for insomnia disorder in our study. In a recent study, the Indonesian version of the SCI score of 23 points or less had a 94% sensitivity and 97% specificity discrimination power for insomnia disorder in post-stroke patients.²⁵ Indonesian version study conducted a clinical interview for insomnia disorder, and 31% of the participants were diagnosed with insomnia according to the DSM-5 criteria.²⁵ This is a significantly higher rate of insomnia than in the current study population (7.3%). However, the median age in this study was 58.5 years, an older age group compared to other studies, and it was the study with the least number of participants among the SCI version studies (n = 160).²⁵ Insomnia disorder is more common in female and in the older population.^50,51 Different insomnia rates have been found in SCI validation studies due to gender and age differences in the samples. All but one of these studies differentiated insomnia based on clinical interviews. As in our study, the cut-off value was calculated in the Swedish version study according to ISI. In the Swedish version study, an SCI score of 16 or less had 86% sensitivity and 90% specificity for insomnia disorder.²⁴ A low sensitivity considering screening was observed than our study. In our research, (sensitivity (90.3%) and specificity (91.8%) the distinction of 15 points and above for clinical insomnia according to the ISI suggests that SCI can distinguish insomnia disorder. It provides a close parallel with the ISI. We found similar results with the Swedish version study if we accept and analyze the SCI cut-off value as 16 and below.

Insomnia is a common health problem and causes many sequelae. Evaluating patients with insomnia should be a priority for treatment.^7,52,53 We do not claim that the Turkish SCI is the gold standard for screening insomnia disorder. However, our data showed that Turkish SCI is as helpful as the ISI (the ISI is frequently used in the literature and clinical trials to assess insomnia symptoms) and can differentiate insomnia disorder. Future studies should test whether the Turkish SCI can distinguish patients diagnosed with insomnia disorder by clinical interview from those without insomnia disorder. Researchers may add objective measurements such as polysomnography and actigraphy to these future studies, but this is not required to diagnose insomnia. Since the SCI does not have an item questioning early morning awakening, a related item may be added in future studies. In addition, Turkish SCI should be validated in older patients in future studies.

Limitations

Our study had the following limitations:

1. This research was conducted online, and this may have caused bias in the responses.
2. We did not conduct a clinical interview to diagnose clinical insomnia.
3. Most participants were female, and belonged to the young age groups, so we could not generalize the results, but they are likely representative of the insomnia population.
4. The ratio of insomnia patients was relatively low.
5. The study did not have objective sleep assessments such as polysomnography or actigraphy.
6. No general questionnaire, such as a quality of life questionnaire, was used.
7. As stated in other version studies of SCI, SCI does not question early awakening insomnia.
8. The response rate to the link we sent for re-test analysis was low.

However, despite all these limitations, performing the Turkish validation of SCI for the first time is valuable. We hope that Turkish SCI will participate in sleep medicine studies in Turkey.

Conclusion

The Turkish SCI is a two-factor, high internal consistency coefficient, time-consistent, and fast, practical scale for clinic insomnia. Even primary care professionals can easily apply and interpret it, except in specific clinical areas dealing with clinic insomnia, such as psychiatry, neurology, or thoracic diseases alone. SCI can also be used in extensive field surveys to detect and early intervention clinic insomnia.