Back to Journals » Nature and Science of Sleep » Volume 18
Content Validity and Psychometric Evaluation of the Idiopathic Hypersomnia Severity Scale (IHSS) in Adults with Idiopathic Hypersomnia
Authors Lin J 
, Lowe C , Banderas B, Chandler P, Arenson E, Shields AL, Morris S, Srivastava B, Dauvilliers Y
Received 23 October 2025
Accepted for publication 8 April 2026
Published 26 June 2026 Volume 2026:18 571289
DOI https://doi.org/10.2147/NSS.S571289
Checked for plagiarism Yes
Review by Single anonymous peer review
Peer reviewer comments 2
Editor who approved publication: Dr Sarah L Appleton
Junji Lin,1 Caitlyn Lowe,2 Benjamin Banderas,2 Patricia Chandler,1 Ethan Arenson,2 Alan L Shields,2 Susan Morris,1 Bhavini Srivastava,1 Yves Dauvilliers3,4
1Jazz Pharmaceuticals, Palo Alto, CA, USA; 2Adelphi Values, Boston, MA, USA; 3Sleep and Wake Disorders Centre, Department of Neurology, Gui de Chauliac Hospital, Montpellier, France; 4University of Montpellier, INSERM Institute Neuroscience Montpellier (INM), Montpellier, France
Correspondence: Junji Lin, Email [email protected]
Purpose: To evaluate content validity, psychometric properties, and score interpretability of the Idiopathic Hypersomnia Severity Scale (IHSS).
Patients and Methods: Adults with idiopathic hypersomnia recruited from US patient organizations participated in qualitative interviews; transcripts were coded and qualitatively analyzed. Using data from a phase 3 trial of low-sodium oxybate in adults with idiopathic hypersomnia (NCT03533114), IHSS scores were psychometrically evaluated at Baseline, Weeks 1 and 8, end of titration, end of Stable Dose Period, and end of Double-Blind Randomized Withdrawal Period. Reliability, validity, and meaningful within-patient change (MWPC) were assessed.
Results: In qualitative interviews, 20 participants (mean [SD] age, 35.0 [10.2] years; 70.0% female; 90.0% White) reported 18 idiopathic hypersomnia–related symptoms. Most frequently reported symptom/impact domains were excessive daytime sleepiness (100.0%) and emotional impact (100.0%). All participants understood and interpreted IHSS instructions, items, and response options as intended. Using the psychometric sample (N=115 participants; mean [SD] age, 40.9 [13.9] years; 71.3% female; 80.9% White), IHSS scores were found to be sufficiently reliable with respect to internal consistency (α≥ 0.80, average inter-item correlations 0.15– 0.50) and reproducibility (intraclass correlation coefficient range, 0.73– 0.98). Moderate to strong (0.30 < |r| ≤ 0.90) correlations were found between IHSS and other assessments (eg, Epworth Sleepiness Scale, Clinical and Patient Global Impression of Change). A large effect size (Cohen’s |d|=0.89) was found in change scores. Interpretation analyses demonstrated a 3-point minimum increase for detecting MWPC for IHSS total score.
Conclusion: The IHSS is a content-valid patient-reported outcome questionnaire that may produce valid, reliable, interpretable scores.
Clinical Trial Registration: NCT03533114.
Plain Language Summary: Idiopathic hypersomnia (IH) is a rare sleep disorder that may make people very sleepy during the day, even after sleeping all night. To understand how IH affects daily life, it is important to hear directly from people with the disorder, not just from their doctors. The Idiopathic Hypersomnia Severity Scale (IHSS) is a questionnaire that can be used to find out more about how IH affects people. In this study, 20 people with IH took the IHSS and were later interviewed to see if they understood the questions and if the topics matched their experiences. All of them understood the questions as intended and felt that the IHSS reflected their experiences with IH. Data from 115 people with IH in an earlier study were also reviewed to see if the IHSS worked as intended and to learn how to interpret changes in scores. This study found that the IHSS questions worked well together, the IHSS worked as intended every time it was used, and the scores lined up with those from other tests in the earlier study. This study also found that a 3-point increase in IHSS score shows a meaningful worsening of IH. Overall, the results show that the IHSS can be used in other studies to better understand the experiences of people living with IH.
Keywords: idiopathic hypersomnia, patient-reported outcomes, qualitative interviews, psychometric evaluation
Introduction
Idiopathic hypersomnia, which often manifests in adolescence or young adulthood and persists into adulthood, is a rare neurologic sleep disorder marked by symptoms of excessive daytime sleepiness (EDS) and often severe and prolonged sleep inertia, including prolonged, nonrestorative nighttime sleep; unrefreshing naps; automatic behaviors; and brain fog.1–3 Research suggests that these symptoms have wide-ranging negative effects on behavior, functioning, and health-related quality of life (HRQoL) and are associated with motor vehicle accidents, poor mental health (eg, anxiety, depression), and difficulties with, for example, keeping jobs, remaining in school, getting diplomas, maintaining marriages, and engaging with family and friends.1,2,4–8
Although measures of sleep disorder symptoms and impacts have been developed, challenges remain in selecting condition-specific measures to be used in clinical trials.9–11 Existing measures like the Epworth Sleepiness Scale (ESS)12 and the Sleep Inertia Questionnaire13 do not assess the range of idiopathic hypersomnia symptoms and impacts. To address this gap, the Idiopathic Hypersomnia Severity Scale (IHSS) was developed to assess both idiopathic hypersomnia symptoms and their impact on HRQoL among patients 16 years of age or older.14 Specifically, the IHSS uses a 7-day recall period and consists of 14 items that measure the following: nighttime sleep symptoms and related sleep inertia, as well as daytime sleep symptoms and impaired daytime functioning attributed to hypersomnolence.
The IHSS was developed in the French language by sleep medicine experts and later cognitively debriefed among patients from the France-Narcolepsy-Hypersomnia Patients Association to evaluate its readability and comprehensiveness.14 After content validity was confirmed, the IHSS was administered in two studies to evaluate its psychometric performance and to generate score interpretation guidelines. A noninterventional study among participants 16 years of age or older (with untreated or treated idiopathic hypersomnia, untreated narcolepsy type 1, and controls without sleepiness) supported a two-factor structure for Night/Sleep Inertia domain and Daytime Performance domain; internal consistency and reliability for these domains and the total score were demonstrated.14 Convergent validity was supported by correlational analysis, indicating that IHSS scores related to other variables as expected (ie, ESS, Beck Depression Inventory II, and EQ-5D visual analog scale scores).14 The IHSS was shown to discriminate between patients with different sleep conditions and healthy controls.14
In the second quantitative study, the French-language version of the IHSS was administered cross-sectionally to 226 patients with idiopathic hypersomnia.15 Results supported the reliability and validity of the IHSS in clinical and research settings. This study15 focused on participants with idiopathic hypersomnia only, and results suggested a three-factor structure to the IHSS, which included Night/Sleep Inertia, Daytime Performance, and Napping domains. This three-factor structure explained 58% of the cumulative variance between items, offering a more nuanced fit for the complex constructs assessed by the IHSS. In addition, distribution-based methods were used to estimate the minimal clinically important difference (MCID) on a severity scale that ranges from mild to very severe. Results from these analyses indicated that an observed 4-point difference in the change of IHSS total scores between groups would reflect an MCID.
Finally, when included as a prespecified secondary endpoint in a phase 3 clinical trial evaluating the safety and efficacy of low-sodium oxybate (calcium, magnesium, potassium, and sodium oxybates oral solution; LXB) among participants with idiopathic hypersomnia, the IHSS demonstrated strong sensitivity to treatment effects (estimated median difference in change between active-treatment and placebo groups, –12.0; 95% confidence interval [95% CI], –15.0 to –8.0; P<0.0001).16 This phase 3 multicenter clinical trial was conducted in several countries, and the IHSS was translated from French and administered in English, French, German, Czech, Italian, Polish, Dutch, Spanish, and Finnish. During the open-label extension period of this trial, mean IHSS scores were maintained and further trended toward a decrease during active treatment.17
Although evidence supports the use of IHSS for evaluating treatment outcomes, several gaps exist. First, the content of the English version of the assessment, developed originally in French, has not been fully validated in an English-speaking population. Second, the psychometric performance of the English-language version of the assessment has not yet been evaluated in a regulated clinical trial sample. Third, further evidence is needed to understand what constitutes meaningful within-patient change (MWPC) thresholds on the IHSS total score. To address these gaps, the aims of the present research are to (1) provide evidence of the content validity of the IHSS among an English-speaking patient cohort and (2) present psychometric performance results of IHSS scores as generated from a phase 3 clinical trial sample, including an empirically derived MWPC for a worsening IHSS total score.
Material and Methods
Stage 1: Evaluation of Content Validity
Trained interviewers used a semi-structured interview guide to conduct hybrid concept elicitation (CE) and cognitive debriefing (CD) qualitative interviews via telephone with 20 adult participants diagnosed with idiopathic hypersomnia. The goals of the interviews were to (1) use CE to explore and document the experience of people with idiopathic hypersomnia from the patients’ perspective, including signs, symptoms, and HRQoL impacts, and (2) use CD to evaluate and document the readability, comprehensibility, relevance, and comprehensiveness of the IHSS. The study protocol and associated study documents were approved by a centralized institutional review board on 29 October 2020 (Sterling Institutional Review Board ID 8443), and all participants provided written informed consent before participating.
Participants
Participants with a clinician-confirmed diagnosis of idiopathic hypersomnia were recruited from either a patient advocacy group or a patient registry within the United States. Study inclusion criteria included (1) signed informed consent and Health Insurance Probability and Accountability Act (HIPAA) authorization; (2) adults between 18 and 75 years of age; (3) clinician-confirmed diagnosis of idiopathic hypersomnia; (4) history of average nightly total sleep time of ≥7 hours; (5) fluent in English; and (6) willingness and ability to participate in an audio-recorded 90-minute telephone interview. Study exclusion criteria included (1) presence of hypersomnia or hypersomnolence attributed to (a) another disorder or condition (medical, behavioral, or psychiatric), (b) medication or drug use, or (c) another sleep disorder (eg, sleep apnea, parasomnia, etc); (2) evidence of untreated or inadequately treated sleep-disordered breathing; (3) participation in another idiopathic hypersomnia study; (4) confounding comorbid conditions or situation that interferes significantly with an individual’s involvement in the study or could confound study results; and (5) ineligible as judged by the investigator for another reason.
Conduct of Qualitative Interviews
During the CE interviews, participants were first asked a series of open-ended questions to elicit spontaneous descriptions of their overall experience with idiopathic hypersomnia, including any relevant signs and symptoms specific to their experience and how these experiences impacted their lives. Concepts of interest were further probed to provide rich qualitative insights. Participants were asked to provide ratings of severity (0–10) and bothersomeness (0–10) for each reported sign or symptom; and bothersomeness ratings for each reported impact.
During the CD portion of the interviews, participants were asked to complete the IHSS using a 7-day recall period while “thinking aloud”18,19 about the process they used to arrive at each response to identify words, terms, or concepts within the questionnaire that they did not understand or did not interpret as intended. After the “think aloud” portion, participants were asked additional, targeted questions designed to assess IHSS content, including its readability, relevance, and comprehensiveness. Interviews were audio-recorded, transcribed, and anonymized before coding and analysis.
Qualitative Data Coding and Analysis
Each interview transcript was coded in ATLAS.ti (Scientific Software Development GmbH, Berlin, Germany) to organize and catalog participants’ experience with idiopathic hypersomnia and their ability to understand and respond to the IHSS. Data analysis was performed with a mixed-method approach using grounded theory,20 quantitative, and qualitative data analytic methods to organize the study findings into key themes. Concept saturation was assessed to confirm that no new CE-based information would be gained from conducting additional interviews. A saturation grid was prepared with transcripts divided into approximately equivalent groups (ie, four groups of five transcripts) based on the order in which interviews were conducted. Coded data were then reviewed to determine when a concept was first mentioned during the interviews and documented in the saturation grid. The saturation grid was then reviewed to evaluate the extent to which new concepts emerged in each transcript group. Saturation was achieved if a downward trend in newly emerged concepts was observed across the groups of interviews. Concepts elicited from the CE interviews were mapped onto IHSS concepts to evaluate conceptual coverage.
Stage 2: Psychometric Evaluation of IHSS
The psychometric analysis was completed with data from a phase 3, multicenter, double-blind, placebo-controlled, randomized withdrawal trial that evaluated the efficacy and safety of LXB for the treatment of adult participants (18–75 years of age) with diagnosed idiopathic hypersomnia according to International Classification of Sleep Disorders-2 or International Classification of Sleep Disorders-3 criteria.16 The study is registered online with ClinicalTrials.gov (NCT03533114) and EU Clinical Trials (2018–001311-79). The study methodology and primary results of this clinical trial have been previously published in full;16,17 briefly, during the 10–14-week Open-Label Titration and Optimization Period, participants (n=154) initiated LXB on a once- or twice-nightly dosing regimen at the investigators’ discretion, which could be increased by no more than 1.5 g/night per week to optimize efficacy and tolerability up to a maximum total nightly dosage of 9 g. During the Stable Dose Period (SDP), participants (n=123) remained on their optimized LXB dose for 2 weeks, before entering the 2-week Double-Blind Randomized Withdrawal Period (DBRWP), when participants (n=116) were randomly assigned 1:1 to their optimized LXB regimen or placebo. On completing the DBRWP, participants (n=106) entered a 24-week Open-Label Extension Period, during which all were treated with LXB, and a subsequent 2-week Safety Follow-up Period (n=95). Data from the Open-Label Extension and Safety Follow-up periods were not used for the psychometric evaluation of the IHSS.
Analysis Populations
Table S1 provides a summary of the analysis population included in the present psychometric evaluation. The psychometric evaluation of the IHSS was conducted on data blinded to treatment.
Timepoints for Analysis
Participants completed assessments at various timepoints throughout the study, including baseline visit, clinic visits during Open Label Treatment Titration and Optimization Period (Weeks 1, 4, and 8 and extension titration 1 or 2 if applicable), end-of-titration visit, end of SDP (eSDP) visit, and end of DBRWP (eDBRWP) visit (noting eDBRWP contains participants who were randomized to placebo, thus discontinuing LXB treatment, and participants who were randomized to continue LXB treatment). IHSS change scores were evaluated between (1) baseline and Week 1, (2) baseline and Week 8, and (3) eSDP and eDBRWP, unless otherwise specified.
Assessments
The IHSS total score is defined as the sum of scores from all 14 items under the assumption that all IHSS items are of equal weight.15 The target measurement concepts are rated on 4-point (eg, 0 [never] to 3 [always]) or 5-point (eg, 0 [no problem] to 4 [very significant]) Likert-type scales that can be used to create a total score (range, 0–50), with higher scores indicating more severe and frequent symptoms.14
Exploratory analyses of the Night/Sleep Inertia domain (Items 1‒4 and 8) and Daytime Performance domain (Items 5‒7 and 9‒14) were conducted to better understand the factor structure of the IHSS. Acceptable model fit was demonstrated when the root-mean-square error of approximation (RMSEA) was less than 0.08.21 Night/Sleep Inertia scores were defined as the sum of the scores for Items 1–4 and 8, and Daytime Performance scores were defined as the sum of the scores for Items 5–7 and 9–14.15
In addition to IHSS scores, scores of other assessments administered in the study were used to support the present psychometric evaluation (see Table S2 in the supplemental material). These assessments include the ESS, Clinical Global Impression of Change (CGI-C), Clinical Global Impression of Severity (CGI-S), 10-item Functional Outcomes of Sleep Questionnaire (FOSQ-10), Patient Global Impression of Change (PGI-C), Work Productivity and Activity Impairment Questionnaire: Specific Health Problem, Version 2.0 (WPAI:SHP), and a visual analog scale for sleep inertia (VAS-SI).
Analyses
All analyses were conducted with patients included in the psychometric analysis population (PsAP; n=115), with the exception of analyses of test–retest reliability, which were conducted with a subset of the PsAP (Table S1). The psychometric evaluation was performed to evaluate the reliability and validity of the IHSS. Table 1 summarizes the analyses and criteria used for this psychometric evaluation of the IHSS. Descriptive statistics of IHSS item, domain, and total scores were calculated at the prespecified analysis timepoints. Response distributions were also evaluated at each timepoint.
 |
Table 1 Summary of Psychometric Evaluation for IHSS Scores |
Score Interpretation
Interpretational analyses were conducted on changes in the IHSS total score from patients in the PsAP for the interval between eSDP and eDBRWP. Assuming a non-normative distribution of scores in this clinical trial population, particularly at baseline, Kruskal–Wallis tests were used to detect statistically significant differences between severity groups among the IHSS total score medians and ESS and CGI-S scores at eDBRWP. These anchor-based methods were supplemented by distribution-based methods, empirical cumulative distribution function curves, and probability density function curves and were used to estimate thresholds for MWPC (ie, the point at which an observed within-patient change supports treatment benefit for that patient) and between-groups differences (sometimes referred to as the MCID).32
Results
Stage 1: Qualitative Research for Content Validity in Idiopathic Hypersomnia
Participant Demographics
Of the 20 participants interviewed, most were female (n=14, 70.0%) and White (n=18, 90.0%); participants’ mean age was 35.0 years (standard deviation [SD], 10.2 years). While all participants self-reported “fair” to “excellent” health, they also reported overall idiopathic hypersomnia severity as “mild” (n=1, 5.0%), “moderate” (n=7, 35.0%), “severe” (n=10, 50.0%), or “very severe” (n=2, 10.0%). See Table S3 in the supplemental material for complete participant demographic and health information.
Concept Elicitation
In total, 18 idiopathic hypersomnia–related symptoms were reported. The most frequently reported sign or symptom concept was EDS, which was reported by all participants (n=20, 100.0%). Other frequently reported sign or symptom concepts included sleep inertia (n=19, 95.0%), brain fog (n=16, 80.0%), and prolonged nighttime sleep (>10–11 hours;n=16, 80.0%) (Figure 1). Additionally, EDS was most frequently reported as the “most bothersome” (n=14, 70.0%) and “most important to improve” (n=15, 75.0%) concept relative to all other symptoms.
 |
Figure 1 Frequency of sign and symptom concepts of idiopathic hypersomnia. |
When asked about the impact of idiopathic hypersomnia on HRQoL and their daily life, participants reported a total of 76 idiopathic hypersomnia–related impact concepts across 19 domains. The most frequently reported impact domain was emotional impact (n=20/20, 100.0%). Additionally, the most frequently reported impact concepts were related to limitations to completing household chores (n=17, 85.0%) and limited participation in social activities (n=16, 80.0%). Among the impact domains reported by participants, the cognitive domain was most frequently reported by participants as most bothersome (n=6, 30.0%) and most important to improve (n=8, 40.0%).
Analysis of concept saturation showed a downward trend in the spontaneous elicitation of both new sign and symptom concepts and impact concepts from participants, thereby confirming the adequacy of the sample size.33,34
IHSS Concept Coverage
Of the 18 signs and symptoms spontaneously elicited from participants, 10 are covered either directly or indirectly by the IHSS: EDS (Item 9), sleep inertia (Items 3 and 4), brain fog (Items 4 and 11), prolonged nighttime sleeping 10–11 hours (Item 1), sleep drunkenness (Items 4 and 5), feeling unrested after sleeping (Items 2 and 8), fatigue (Item 10), excessive sleeping during the day (Items 6 and 7), memory loss (Item 11), and muscle fatigue (Item 10). Additionally, 12 of the 19 impact domains spontaneously elicited from participants as impacting their quality of life are covered by the IHSS: emotional (Item 12), cognitive (Items 4, 11, and 14), work (Items 9 and 13), household chores/responsibilities (Item 13), recreational/leisure activities (Item 13), physical function (Items 5 and 10), activities of daily living (Items 5 and 10), adaptive behavior (Items 3 and 14), sleep (Item 14), school (Item 13), caregiver responsibility (Item 13), and family activities (Item 13). While participants spontaneously reported several sign and symptom concepts (n=8 concepts) and impact domains (n=7 domains) that were not assessed by the IHSS (eg, hallucinations, financial impacts), many of these concepts and domains were reported by only a single participant. These reports were considered idiosyncratic concepts and deemed not relevant to the idiopathic hypersomnia experience.
Supplemental materials present the signs, symptoms, and impact concepts most frequently reported during the CE portion of the hybrid interviews, including concepts covered by the IHSS (Tables S4 and S5).
Cognitive Debriefing
During the CD portion of the interviews, participants completed the IHSS. Across all items, >70.0% of participants reported experiencing assessed concepts within the recall period. All participants interpreted the IHSS instructions, recall period, and items as intended by the instrument’s developers, and found the IHSS easy to complete. Additionally, most participants (n=17, 85.0%) interpreted the response options of the IHSS as intended. Minor interpretation issues were reported for the response option scales for Items 4 (length of time after waking up to feel fully functional) and 7 (ideal length of naps [hours]); one participant (5.0%) reported that “less than 1 hour” was not different from “0-no naps” and therefore did not interpret “1-less than 1 hour” or “0-no naps” as intended, and one participant (5.0%) interpreted that “0-no naps” meant “no ideal length” of naps. For Items 10, 11, and 14, one participant (5.0%) suggested clarification of the “I do not drive” option (eg, to distinguish between someone who does not drive because they do not need to, someone who cannot drive because they do not have a license, and someone who cannot drive because they are too unwell). Any suggestions for changes to the IHSS, including those due to interpretation issues for the response options, were minimal, non-substantive, and inconsistent among participants; thus, no changes were made to the IHSS.
Stage 2: Psychometric Evaluation of IHSS
Study Sample
In total, 115 participants from the PsAP of the phase 3 trial were included in the psychometric analyses. The mean (SD) age of the sample was 40.9 (13.9) years, and most of the analysis population was female (n=82, 71.3%), non-Hispanic (n=91, 79.1%), and White (n=93, 80.9%).
Descriptive Statistics
All patients in the PsAP (n=115) completed the IHSS at baseline, end of titration, eSDP, and eDBRWP; during Weeks 1 and 8, 114 patients completed the IHSS. Patients endorsed the full range of responses for each IHSS item at each timepoint, with trends, as expected, suggesting poorer health status at baseline and earlier timepoints relative to later timepoints in the trial; for example, whereas 65 (56.5%) participants reported “always” feeling as if they have not had enough sleep (IHSS Item 2) at baseline, only nine (7.8%) and 30 (26.1%) participants gave this response at eSDP and eDBRWP (noting eDBRWP includes 50% of patients on treatment and 50% on placebo), respectively. Broadly, however, there were no highly unbalanced IHSS item responses (eg, ≥95% responses at the low or high end of the available response scale) that would predict poor performance or preclude use in correlational and/or more complex structural analyses.22
Reliability
Reliability estimates characterize the consistency and reproducibility of clinical outcome assessment scores; the context in which reliability is assessed and the assessments used to evaluate reliability may influence reliability estimates.35 IHSS total score reliability was evaluated in terms of internal consistency and using test–retest methods. Inter-item correlations (ICs) and the average IC (AIC) supported item consistency at baseline (n=115, AIC=0.27) and at eDBRWP (n=115, AIC=0.39); AIC values 0.15 to 0.50 support inter-item relatedness.22 The ranges of IC values at baseline and eDBRWP were −0.02 to 0.64 and 0.07 to 0.86, respectively; most observed ICs fell between 0.15 and 0.50 at baseline (76.9%) and at eDBRWP (87.9%). Cronbach alpha coefficients (α) for the IHSS total score supported internal consistency reliability at baseline (α=0.82) and at eDBRWP (α=0.90); α>0.70 supports use of scales in construct validation research.23 Although the results are not presented herein, the impact of removing items from the calculation of α was marginal. In terms of test–retest reliability, IHSS total scores demonstrated reproducibility among stable participants defined by the same ESS severity or reports of “no change” on the PGI-C from baseline to Week 1 (intraclass correlation coefficient [ICC]=0.90, 95% CI=0.81–0.95; and ICC=0.93, 95% CI=0.87–0.96), the beginning of the Stable Dose Period to eSDP (ICC=0.89, 95% CI=0.83–0.93; and ICC=0.98, 95% CI=0.57–0.99), and eSDP to eDBRWP (ICC=0.81, 95% CI=0.69–0.89; and ICC=0.81, 95% CI=0.62–0.91).
Structural Validity
Results in this section inform understanding of the IHSS structure. Table 2 presents the results from the multi-trait multi-item analysis at eDBRWP. Correlations between item scores and corrected total scores (ie, total scores with the item score removed in order to correct for item overlap bias) consistently exceeded ~0.30 (range, 0.284 to 0.787), indicating at least borderline weak to moderate associations. Additionally, moderate correlations were found between Items 1–4 and 8 and the Night/Sleep Inertia domain score, and moderate to strong correlations were found between Items 5–7 and 9–14 and the Daytime Performance domain score. While individual items were moderately correlated with both the Night/Sleep Inertia and Daytime Performance domains, the within-domain correlations were higher than the correlations of items with the other domain, aside from Item 7 related to ideal duration of naps. The higher correlation of Item 7 with the Night/Sleep Inertia domain may have been attributed to its assessment of naps, which may be more related to items that assessed nighttime sleep, sleep inertia, sleepiness, and consequences of hypersomnolence. Naps of ideal duration and frequency have a phenotype different from that of items in the Daytime Performance domain, which may have led to the lower correlation observed.
 |
Table 2 Multi-Trait Multi-Item Analysis: IHSS Item Descriptive Statistics and Spearman Item–Scale Correlations Corrected for Overlap at eDBRWP |
The scree plot (Figure 2) suggests a two- or three-domain factor structure across timepoints (see elbow in plot at second eigenvalue), noting that three eigenvalues were >1. The lower limit of the 95% CI for the RMSEA for the unidimensional model (not shown) was consistently >0.08 across timepoints, indicating poor model fit. The bifactor model demonstrated a lower limit for the RMSEA of ≤0.08 at baseline, Week 1, eSDP, and eDBRWP.27,28 With the exception of Item 14 (impact of hypersomnolence on driving a car) at baseline, the bifactor model resulted in moderate to strong factor loadings on the general domain across all timepoints. Overall, these results were inconclusive to support a two-domain structure but were supportive of a bifactor structure with two grouping factors. Table S6 in the supplemental material presents the standardized factor loadings from the IHSS item responses.
 |
Figure 2 Eigenvalues based on polychoric correlations of IHSS items. Abbreviations: eDBRWP, end of Double-Blind Randomized Withdrawal Period; eSDP, end of Stable Dose Period; IHSS, Idiopathic Hypersomnia Severity Scale. |
Construct-Related Validity
Table 3 presents correlations between the IHSS total score and other assessments. Moderate correlations were found between IHSS total and scores from the other measures (Table 3), with the strongest correlations observed at eDBRWP. As shown in Table 4, the Kruskal–Wallis test revealed statistically significant differences between severity groups among the IHSS total score medians and the ESS scores (normal, mildly excessive, moderately excessive, and severely excessive) at eDBRWP.36 Thus, the known-groups analyses showed that means and medians increased with increased group severity (ie, monotonicity). IHSS total scores exhibited a large positive effect (Cohen’s d=0.89), indicating that the IHSS effectively detected changes in idiopathic hypersomnia severity over time, from eSDP to eDBRWP. IHSS total scores showed moderate to strong correlations with changes in ESS total scores, with the strongest correlation observed at eDBRWP (r=0.72).
 |
Table 3 Spearman Correlation Coefficients Between IHSS Total Score and Concurrent Assessments |
 |
Table 4 Known-Groups Comparisons of Total Scores by CGI-S Groups and ESS Severity Groups at eDBRWP (PsAP; N=115) |
Score Interpretation
The results obtained from distribution-based methods indicate that an IHSS total score difference of ≤2 points may not be distinguished from measurement error between groups. To determine an MWPC in the IHSS total score that reflects a worsening condition, seven levels of change groups were established based on anchors, including the ESS, PGI-C, and CGI-C. The suggested minimum increases are a 3-point increase per ESS, a 3-point increase per PGI-C, and a 3-point increase per CGI-C. Additional support for these MWPC estimates was derived from empirical cumulative distribution function curves, as shown in Figure 3 (for the CGI-C anchor). The change score distributions for patients reporting no change were distinct from those of patients in the worsening anchor groups. Based on these anchor-based analyses, a 3-point increase in IHSS total score is recommended as the threshold for determining MWPC for a worsening condition. Table 5 shows the anchor-based estimates on the IHSS total score by ESS, CGI-C, and PGI-C level groups between eSDP and eDBRWP.
 |
Table 5 Anchor-Based Estimates on IHSS Total Score by Measure Level Groups Between eSDP and eDBRWP (PsAP; N=115) |
 |
Figure 3 Empirical cumulative distribution function curves for change in IHSS by CGI‑C from eSDP to eDBRWP. Abbreviations: CGI-C, Clinical Global Impression of Change; eDBRWP, end of Double-Blind Randomized Withdrawal Period; eSDP, end of Stable Dose Period; IHSS, Idiopathic Hypersomnia Severity Scale. |
Discussion
Patients with idiopathic hypersomnia experience a range of debilitating symptoms impacting their daily lives and HRQoL. While patient-reported outcome questionnaires like the ESS and Sleep Inertia Questionnaire assess some of these key symptoms, they may not capture the entire idiopathic hypersomnia experience. Conversely, the IHSS assesses a broader range of symptoms and HRQoL impacts of idiopathic hypersomnia among patients 16 years of age or older. This current study intended to replicate and extend previous documented evidence to evaluate the content validity of the IHSS and the instrument’s ability to produce reliable, valid, and interpretable change in scores.
Previous studies conducted by Dauvilliers et al (2019)14 and Rassu et al (2022)15 in French-speaking populations demonstrated some content validity and psychometric reliability and validity of the French version of the IHSS outside of the clinical trial context. While these studies provided a framework for a psychometrically sound patient-reported outcome instrument, gaps were identified in terms of language (developed in French), limited data in the clinical trial space (eg, validity, reliability, and interpretation), and estimated ranges regarding MWPC.
The findings from the hybrid CE and CD interviews conducted in English with 20 adults with idiopathic hypersomnia in the United States further confirmed both the profound burden of disease of idiopathic hypersomnia and the content validity of the IHSS. Patients articulated a constellation of symptoms and impacts and confirmed that the IHSS assesses these symptoms and impacts in ways that respondents can clearly understand. The results from the psychometric analysis of data collected from 115 participants in the phase 3 LXB trial supported the overall reliability and validity of the IHSS. Specifically, item scores were well distributed across the range of responses and were shown to be internally consistent and reproducible. The validity of the IHSS was supported by the observed relationships between the IHSS total scores and other variables, being as expected at single points in time (ie, convergent validity) and over time (ie, sensitivity to change). The IHSS was also able to distinguish among disease severity categories (ie, known-group validity). The reliability and construct-related validity of the total score produced by the IHSS demonstrated reproducible and sound scores. In the context of the phase 3 trial, score interpretation analyses, with the use of the ESS, PGI-C, and CGI-C as anchors, indicated that an MWPC of a 3-point increase in IHSS total scores was sufficient to indicate a worsening of idiopathic hypersomnia symptoms among participants.
This study is not without its limitations. First, the goal of this study was to provide evidence of content validity of the IHSS. Participants (n=20) in the hybrid CE and CD interviews were not asked about whether medication use was related to the reported symptoms and impacts. An additional limitation of the qualitative interviews lies in its skewed representation toward highly educated White women, potentially limiting the generalizability of findings. Future research should strive for a more diverse participant pool to ensure broader applicability of findings. While previous research14,15 suggests a two- or three-factor conceptual framework underlying the IHSS—that is, a two-factor structure for people with narcolepsy type 1, people with idiopathic hypersomnia, and controls without either condition, and a three-factor structure for people with idiopathic hypersomnia only—potentially due to a small sample size from a clinical study, the present study did not find evidence to support a two-factor structure, although a bifactor model with two grouping factors demonstrated acceptable model fit. Additional research is needed to determine whether a three-factor model supports the observed data. Finally, the clinical trial described herein was restricted to participants with a primary diagnosis of idiopathic hypersomnia, and ESS scores of 11 or higher were required at screening and baseline for all patients except those taking sodium oxybate at study entry, who were instead required to have had clinical improvement of EDS with sodium oxybate treatment per the investigator’s clinical judgment. Consequently, the results from this study may not reflect patients with idiopathic hypersomnia of diverse severity in a real-world setting. Additional real-world studies (eg, observational registry studies)37 in English-speaking populations may be warranted.
Conclusion
This study addressed the identified gaps associated with the IHSS with regard to content validity in English-speaking samples, psychometric performance in a clinical trial setting, and MWPC change thresholds. In conclusion, results support (1) the content validity of the IHSS; (2) the capability of the IHSS to produce scores that are reliable, construct-valid, and sensitive to change over time in a trial setting; and (3) the recommendation to use a 3-point increase in IHSS total scores as the threshold for determining MWPC indicative of a worsening condition among people with idiopathic hypersomnia. Together, results from the present study indicate that the IHSS may reflect the experience of patients with idiopathic hypersomnia in clinical trials and that observed patient change may be meaningfully interpreted to characterize treatment benefit.
Abbreviations
AIC, average inter-item correlation; CGI-C, Clinical Global Impression of Change; CGI-S, Clinical Global Impression of Severity; CD, cognitive debriefing; CE, concept elicitation; CI, confidence interval; eDBRWP, end of Double-Blind Randomized Withdrawal Period; EDS, excessive daytime sleepiness; eSDP, end of Stable Dose Period; ESS, Epworth Sleepiness Scale; FOSQ-10, 10-item Functional Outcomes of Sleep Questionnaire; HRQoL, health-related quality of life; IC, inter-item correlation; ICC, intraclass correlation coefficient; IHSS, Idiopathic Hypersomnia Severity Scale; LXB, low-sodium oxybate; MCID, minimal clinically important difference; mITT, modified intent-to-treat; MWPC, meaningful within-patient change; PGI-C, Patient Global Impression of Change; PsAP, psychometric analysis population; RMSEA, root-mean-square error of approximation; SD, standard deviation; TRT-AP, test–retest analysis population; VAS-SI, visual analog scale for sleep inertia; WPAI:SHP, Work Productivity and Activity Impairment Questionnaire: Specific Health Problem, Version 2.0.
Data Availability (Where Applicable)
All relevant data are provided within the article and supporting files. Jazz has established a process to review requests from qualified external researchers for data from Jazz-sponsored clinical trials in a responsible manner that includes protecting patient privacy, assurance of data security and integrity, and furthering scientific and medical innovation. Additional details on Jazz Pharmaceuticals’ data sharing criteria and process for requesting access can be found at: https://www.jazzpharma.com/science/clinical-trial-data-sharing. Further queries on the data from this article can be addressed to the corresponding author, Junji Lin: [email protected].
Ethics Approval and Informed Consent
All study documents for the cognitive debriefing/concept elicitation interviews were submitted to a centralized institutional review board (IRB): Sterling Institutional Review Board (https://sterlingirb.com). The study protocol and associated study documents were approved by Sterling IRB on 29 October 2020 (Sterling IRB ID 8443). The protocol for the phase 3, multicenter, double-blind, placebo-controlled, randomized withdrawal trial that evaluated the efficacy and safety of LXB for the treatment of adult participants (18–75 years of age) with idiopathic hypersomnia (NCT03533114) was approved by either an independent ethics committee or an institutional review board at each site. This clinical trial was conducted in accordance with the International Conference on Harmonisation Guidelines for Good Clinical Practice and the Declaration of Helsinki. All participants were required to provide written consent for inclusion in this study.
Consent For Publication
This publication does not contain any information that could potentially identify any individual.
Acknowledgments
Junji Lin was an employee of Jazz Pharmaceuticals at the time this research was conducted; her current role is at Bristol Myers Squibb (3401 Princeton Pike, Lawrence Township, NJ 08648). The authors thank Daniella Olonilua of Adelphi Values in Boston, MA, for providing medical writing support/editorial support; Kaitlin Chatterton, Naomi Suminski, Mike DeRosa, and Annette Yim for their collection and analysis of qualitative and quantitative data; and Luke Hickey, MSc, for his contributions to the study design and development of this article during his tenure at Jazz Pharmaceuticals. The Epworth Sleepiness Scale was used with permission and under license during the phase 3 clinical trial on which some of this research was based; we thank Noam Frey of Jazz Pharmaceuticals for his assistance in securing the licenses and permissions required to use this instrument. This research was funded by Jazz Pharmaceuticals in accordance with Good Publication Practice (GPP2022) (https://www.ismpp.org/gpp-2022).
Author Contributions
J Lin: Conceptualization, Project administration, Supervision, Writing—original draft, Writing—review and editing
C Lowe: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing—original draft, Writing—review and editing
B Banderas: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Visualization, Writing—original draft, Writing—review and editing
P Chandler: Conceptualization, Writing—original draft, Writing—review and editing
A Shields: Conceptualization, Data curation, Investigation, Methodology, Resources, Supervision, Validation, Writing—original draft, Writing—review and editing
E Arenson: Data curation, Formal analysis, Investigation, Methodology, Software, Validation, Visualization, Writing—original draft, Writing—review and editing
All authors gave final approval of the version to be published; have agreed on the journal to which the article has been submitted; and agree to be accountable for all aspects of the work.
Funding
This research was funded by Jazz Pharmaceuticals.
Disclosure
An abstract containing interim findings of this article was presented at the 2022 Annual Meeting of the American Academy of Neurology as a poster presentation. The poster’s abstract was published in the journal Neurology (Volume 98, Number 18 Supplement) which can be accessed here: https://doi.org/10.1212/WNL.98.18_supplement.1169.
J Lin, S Morris, and B Srivastava are former full-time employees of Jazz Pharmaceuticals who, in the course of this employment, received stock options exercisable for, and other stock awards of, ordinary shares of Jazz Pharmaceuticals, plc. C Lowe is a former employee of Adelphi Values, an outcomes consultancy that provides research services in the pharmaceutical and device development industry, including the research activities presented here. B Banderas, E Arenson, and A Shields are employees of Adelphi Values, an outcomes consultancy that provides research services in the pharmaceutical and device development industry, including the research activities presented here. P Chandler is a full-time employee of Jazz Pharmaceuticals who, in the course of this employment, has received stock options exercisable for, and other stock awards of, ordinary shares of Jazz Pharmaceuticals, plc. Y Dauvilliers is a consultant for and has participated in advisory boards for Alkermes, Avadel, Bioprojet, Centessa, Harmony Biosciences, Idorsia, Jazz Pharmaceuticals, Pharmanovia, and Takeda. All Authors confirm no other conflicts of interest.
References
1. Khan Z, Trotti LM. Central disorders of hypersomnolence: focus on the narcolepsies and idiopathic hypersomnia. Chest. 2015;148(1):262–16. doi:10.1378/chest.14-1304
2. Dauvilliers Y, Bogan RK, Arnulf I, Scammell TE, St Louis EK, Thorpy MJ. Clinical considerations for the diagnosis of idiopathic hypersomnia. Sleep Med Rev. 2022;66:101709. doi:10.1016/j.smrv.2022.101709
3. American Academy of Sleep Medicine. International Classification of Sleep Disorders—
4. Saini P, Rye DB. Hypersomnia: evaluation, treatment, and social and economic aspects. Sleep Med Clin. 2017;12(1):47–60. doi:10.1016/j.jsmc.2016.10.013
5. Stanford Health Care. Idiopathic hypersomnia. 2020. Available from: https://stanfordhealthcare.org/medical-conditions/sleep/idiopathic-hypersomnia.html.
6. Ozaki A, Inoue Y, Nakajima T, et al. Health-related quality of life among drug-naïve patients with narcolepsy with cataplexy, narcolepsy without cataplexy, and idiopathic hypersomnia without long sleep time. J Clin Sleep Med. 2008;4(6):572–578. doi:10.5664/jcsm.27352
7. Anderson KN, Pilsworth S, Sharples LD, Smith IE, Shneerson JM. Idiopathic hypersomnia: a study of 77 cases. Sleep. 2007;30(10):1274–1281. doi:10.1093/sleep/30.10.1274
8. Bassetti C, Aldrich MS, Quint D. Sleep-disordered breathing in patients with acute supra- and infratentorial strokes. A prospective study of 39 patients. Stroke. 1997;28(9):1765–1772. doi:10.1161/01.STR.28.9.1765
9. Churruca K, Pomare C, Ellis LA, et al. Patient-reported outcome measures (PROMs): a review of generic and condition-specific measures and a discussion of trends and issues. Health Expect. 2021;24(4):1015–1024. doi:10.1111/hex.13254
10. Medarov BI, Victorson DE, Judson MA. Patient-reported outcome measures for sleep disorders and related problems: clinical and research applications. Chest. 2013;143(6):1809–1818. doi:10.1378/chest.12-2489
11. Pevernagie D, Bauters FA, Hertegonne K. The role of patient-reported outcomes in sleep measurements. Sleep Med Clin. 2021;16(4):595–606. doi:10.1016/j.jsmc.2021.07.001
12. Johns MW. A new method for measuring daytime sleepiness: the Epworth Sleepiness Scale. Sleep. 1991;14(6):540–545. doi:10.1093/sleep/14.6.540
13. Kanady JC, Harvey AG. Development and validation of the Sleep Inertia Questionnaire (SIQ) and assessment of sleep inertia in analogue and clinical depression. Cognit Ther Res. 2015;39(5):601–612. doi:10.1007/s10608-015-9686-4
14. Dauvilliers Y, Evangelista E, Barateau L, et al. Measurement of symptoms in idiopathic hypersomnia: the Idiopathic Hypersomnia Severity Scale. Neurology. 2019;92(15):e1754–e1762. doi:10.1212/WNL.0000000000007264
15. Rassu AL, Evangelista E, Barateau L, et al. Idiopathic Hypersomnia Severity Scale to better quantify symptoms severity and their consequences in idiopathic hypersomnia. J Clin Sleep Med. 2022;18(2):617–629. doi:10.5664/jcsm.9682
16. Dauvilliers Y, Arnulf I, Foldvary-Schaefer N, et al. Safety and efficacy of lower-sodium oxybate in adults with idiopathic hypersomnia: a phase 3, placebo-controlled, double-blind, randomised withdrawal study. Lancet Neurol. 2022;21(1):53–65. doi:10.1016/S1474-4422(21)00368-9
17. Morse AM, Dauvilliers Y, Arnulf I, et al. Long-term efficacy and safety of low-sodium oxybate in an open-label extension period of a placebo-controlled, double-blind, randomized withdrawal study in adults with idiopathic hypersomnia. J Clin Sleep Med. 2023;19(10):1811–1822. doi:10.5664/jcsm.10698
18. Willis GB. Cognitive Interviewing: A Tool for Improving Questionnaire Design. Thousand Oaks, CA: SAGE Publications; 2005.
19. Ericsson KA, Simon HA. Protocol Analysis: Verbal Reports as Data. Cambridge, MA: MIT Press; 1993.
20. Strauss A, Corbin J. Grounded theory methodology: an overview. In: Denzin N, Lincoln Y, editors. Handbook of Qualitative Research. Thousand Oaks, CA: SAGE Publications; 1994:273–284.
21. Kline RB. Exploratory and confirmatory factor analysis. In: Petscher Y, Schatschneider C, Compton DL, editors. Applied Quantitative Analysis in Education and the Social Sciences. New York: Routledge; 2013:171–207.
22. Clark LA, Watson D. Constructing validity: new developments in creating objective measuring instruments. Psychol Assess. 2019;31(12):1412–1427. doi:10.1037/pas0000626
23. Nunnally JC. The assessment of reliability. In: Bernstein I, editor. Psychometric Theory. New York, NY: McGraw Hill; 1994:248–292.
24. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155–163. doi:10.1016/j.jcm.2016.02.012
25. McGraw KO, Wong SP. Forming inferences about some intraclass correlation coefficients. Psychol Methods. 1996;1(1):30–46. doi:10.1037/1082-989X.1.1.30
26. Campbell DT, Fiske DW. Convergent and discriminant validation by the multitrait-multimethod matrix. Psychol Bull. 1959;56(2):81–105. doi:10.1037/h0046016
27. Kline RB. Principles and Practice of Structural Equation Modeling.
28. Browne MW, Cudeck R. Alternative ways of assessing model fit. Sociol Methods Res. 1992;21(2):230–258. doi:10.1177/0049124192021002005
29. Hinkle DE, Jurs SG, Wiersma W. Applied Statistics for the Behavioral Sciences.
30. Cohen J. Statistical Power Analysis for the Behavioral Sciences.
31. Sawilowsky S. New effect size rules of thumb. J Mod Appl Stat Methods. 2009;8(2):597–599. doi:10.22237/jmasm/1257035100
32. Cappelleri JC, Zou KH, Bushmakin AG, Alvir JMJ, Alemayehu D, Symonds T. Patient-Reported Outcomes: Measurement, Implementation and Interpretation. Boca Raton, FL: CRC Press; 2013.
33. Patrick DL, Burke LB, Gwaltney CJ, et al. Content validity - establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: part 1-eliciting concepts for a new PRO instrument. Value Health. 2011;14(8):967–977. doi:10.1016/j.jval.2011.06.014
34. Turner-Bowker DM, Lamoureux RE, Stokes J, et al. Informing a priori sample size estimation in qualitative concept elicitation interview studies for clinical outcome assessment instrument development. Value Health. 2018;21(7):839–842. doi:10.1016/j.jval.2017.11.014
35. Thompson B, Vacha-Haase T. Psychometrics is datametrics: the test is not reliable. Educ Psychol Meas. 2000;60(2):174–195. doi:10.1177/0013164400602002
36. About the Epworth Sleepiness Scale. 2024. Available from: https://epworthsleepinessscale.com/about-the-ess/.
37. Dreyer NA, Garner S. Registries for robust evidence. JAMA. 2009;302(7):790–791. doi:10.1001/jama.2009.1092
© 2026 The Author(s). This work is published and licensed by Dove Medical Press Limited. The
full terms of this license are available at https://www.dovepress.com/terms
and incorporate the Creative Commons Attribution
- Non Commercial (unported, 4.0) License.
By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted
without any further permission from Dove Medical Press Limited, provided the work is properly
attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.
Recommended articles
Assessing the Validity of the Long-Term Conditions Questionnaire (LTCQ) in Women During Pregnancy and the First Year Following Birth
Kelly L, Fitzpatrick R, Kurinczuk JJ, Rivero-Arias O, Alderdice F
Patient Related Outcome Measures 2022, 13:221-228
Published Date: 19 October 2022
Experiences of Heart Failure and the Treatment Journey: A Mixed-Methods Study Among Patients with Heart Failure in Sweden
Liljeroos M, Agvall B, Eek D, Fu M
Patient Preference and Adherence 2023, 17:1935-1947
Published Date: 9 August 2023