Developing a Conceptual Disease Model of Patient Experiences and Identifying Patient-Reported Clinical Outcome Assessments for Use in Trials of Treatments for Focal Onset Seizures

Plain Language Summary

Epilepsy is a complex spectrum of seizure disorders affecting about 3.4 million people in the United States. The most common types of epileptic seizures in adults are called focal onset seizures (FOS). For patients to receive the greatest benefit from new treatments, clinical trials need to use suitable tools consistently and accurately to measure outcomes that are meaningful to patients. This study can serve as a guide for further research and for enabling clinicians to select the right tools for future trials.

We searched the literature to identify areas of patients’ lives most impacted by FOS. The findings were arranged by themes and key areas of interest. The identified impacts on patients’ lives include: limits on physical abilities and activities, impacts on cognition and social/emotional aspects of life, and types of social support and coping strategies used. We then conducted a review to identify tools to measure these impacts in clinical trials. The 10 tools we selected were assessed against guidance by the United States Food and Drug Administration related to the development, accuracy and reliability of each tool. The strength of evidence supporting the use of these tools in patients with FOS was widely variable; however, 2 measures of symptom severity and 1 measure of health-related quality of life were recommended.

Introduction

Epilepsy is a multifaceted, complex neurological syndrome that encompasses a spectrum of seizure disorders, which vary by cause, type, and severity.^1,2 It affects approximately 3.4 million adults and children in the United States (US), and 150,000 new patients are diagnosed each year.³ Furthermore, epilepsy is considered refractory and uncontrolled among approximately one-third of patients.^2,4 Epilepsy is more prevalent among people in the lowest socioeconomic class, those in poor health, and those who are socially deprived.⁵ In 2014, epilepsy and convulsion disorders resulted in 280,000 US hospital admissions, costing $2.5 billion.² Seizures are characterized by the presence of abnormal brain activity and are classified by their location: generalized onset seizures present as abnormal electrical activity in both hemispheres of the brain, while focal onset seizures (FOS) (previously called partial seizures) occur in a single hemisphere.^1,6

FOS are more common in adults than generalized onset seizures,^6,7 and approximately 60% of people with epilepsy have FOS.¹ FOS are categorized by whether patients retain awareness (previously called simple partial seizures) or display impaired awareness (previously called complex partial seizures).^6,8 During focal onset aware seizures, the patient maintains consciousness but may not be capable of interacting with others.⁸ During impaired awareness seizures, the patient has a change in level of awareness and/or exhibits involuntary movements including lip-smacking, chewing, or hand rubbing.⁶

Epilepsy treatment strategies aim to achieve seizure control for patients.² Currently, 24 antiepileptic drugs are available in the United States, most of which are used as monotherapy.¹ Options beyond medication, including surgery and implantable devices, exist for some patients.¹ Despite these new treatment options, the rate of positive outcomes for newly diagnosed patients (eg, remaining free of seizures for over a year) has not improved over the past ~30 years.⁹ In 2015, despite 90% of patients with epilepsy taking medication, only 44% had stayed seizure-free in the past year.³ For patients who have tried multiple treatments, the proportion who achieve seizure control decreases with each subsequent medication.² Seizures affect almost every facet of a patient’s life, including their ability to work, income level, ability to drive, independent living, social opportunities, and physical activity.² Psychiatric comorbidities are common among patients with epilepsy, with the most common being depression.⁴ Further research is needed to identify the mechanisms underlying seizures to design more effective therapies to improve patient outcomes.¹

Epilepsy is more prevalent among adult Medicaid recipients than in the general population, and uninsured patients or those covered by public insurance have reduced access to specialized epilepsy care.^10,11 Care from an epilepsy provider promotes medication adherence, which reduces seizure frequency.¹² Disabilities, both physical and intellectual, are common among adults with epilepsy.¹¹ For example, epilepsy is up to 20 times more common among patients with intellectual disabilities than in the general population, yet these individuals are less likely to report side effects, particularly those that impact cognition.¹³

Clinical outcome assessments (COAs) are used to assess the efficacy of treatment during clinical trials and include clinician-reported outcome (ClinRO) measures, outcomes reported by external observers, such as patient caregivers (ObsROs), performance outcome (PerfO) measures, and patient-reported outcome (PRO) measures.¹⁴ This current evaluation is focused on ClinRO and PRO measures, which constitute the most commonly used COAs in FOS clinical trials. ClinRO measures are objective assessments performed by trained clinician raters and are useful for identifying treatment benefits; however, many ClinRO measures are impractical for daily use and may not measure differences that are meaningful to patients, such as quality of life.¹⁵ PRO measures expand clinical definitions of efficacy to directly convey patients’ perspectives and priorities for understanding the impact of their condition without amendment or interpretation by others.^16,17 PRO measures can be used in clinical practice to screen for symptoms, monitor progress, and support decision-making by clinicians.¹⁸ They can also identify patients in need of in-person care and improve patients’ interactions with their providers.¹⁸ In a recent study of patients with epilepsy, completing PRO measures prior to a visit increased the scope of dialogue between patients and providers, encouraged patients to express emotion, and created more balanced feelings of power during the visit.¹⁸

PRO measures are widely used in interventional clinical trials to evaluate efficacy, side effects, and other impacts of antiseizure medications.¹⁹ A review of PRO measures identified 26 instruments assessing a range of outcomes in patients with epilepsy; however, PRO measures are rarely the basis of label claims for approved products for epilepsy.¹⁹ Seizure diaries are the most common PRO measures used by patients to track their seizure frequency, though patients may inadvertently underreport seizures when they are unaware of experiencing a seizure.²⁰ The addition of ObsRO seizure diaries can increase the accuracy of reported seizure frequency.²¹ While routinely collected in clinical trials, only 1 in 4 product labels approved by the US Food and Drug Administration (FDA) to treat patients with epilepsy contained PRO results.¹⁹ This phenomenon is complex: the lag between a medication’s clinical efficacy and a patient’s perception of its efficacy may delay improvements in PRO scores and thus may not warrant label inclusion. Additionally, to be included on label claims, PRO measures must meet stringent criteria,^22,23 which may be challenging to implement, as many were developed prior to publication of the regulatory guidances.^19,24

In 2011, Kerr et al published a conceptual disease model (CDM) covering a range of epilepsy types in which adult patients with epilepsy identified problematic relationships with partners and fulfilling family roles as concerns.¹⁷ Epilepsy was reported to impact patients’ lives directly via cognitive, physical, and seizure effects, and indirectly by affecting future hopes, caregiver burden, and self-esteem.¹⁷ Kerr’s CDM included a diverse range of seizure types with variable frequency, severity, and underlying pathology.

As FOS are the most common type among adults, in the current study we conducted a targeted literature review focused solely on this population to understand health-related quality of life (HRQoL) impacts on patients with FOS. Our literature review included all articles referenced by Kerr et al¹⁷ that met our inclusion criteria and new content. A novel CDM specific to FOS was constructed based on the following concepts identified as of high importance to patients: functional impacts and limitations (physical and cognitive) and impacts on social and emotional function, with the additional concept of social support and coping strategies to provide insight into patients’ lived experiences.

We performed a secondary literature review to identify COAs capable of measuring these core concepts of interest (COI), from our CDM. Content review of these COAs identified a subset of 10 with coverage of the COI. This subset was reviewed for evidence supporting the measure characteristics (eg, number of items and domains measured), development process, and psychometric measurement properties of each instrument in patients with FOS, based on FDA guidance documents detailing criteria for use of PROs in label claims and patient-focused drug development.^22,23

Methods

This research was completed between June 2019 and April 2020 and consisted of 2 phases of targeted literature reviews. Phase 1 was conducted to develop the CDM based on concepts important to patients, and Phase 2 identified COAs that map to these COI. A subset of COAs from phase 2 were found to have good coverage of COI from our CDM and were chosen for an in-depth gap analysis of their development and measurement properties.

Phase 1 Targeted Literature Review

Data for phase 1 were obtained by performing a targeted literature review on Embase/MEDLINE and PsychINFO for adults limited to English language literature (see Supplementary Tables 1 and 2 for search strategy). Briefly, qualitative studies published after 2010 that reported concepts important to patients with FOS (including both impaired awareness and aware FOS) were reviewed, as well as qualitative studies published before 2010 that were used in Kerr’s CDM and met the current study’s inclusion criteria.¹⁷ Articles and abstracts that meet our inclusion criteria are reported in Supplementary Table 3.

Abstracts from relevant professional meetings over the previous 2 years (or past 2 meetings for biennial conferences) were reviewed to identify emerging qualitative research conducted in patients with epilepsy. Clinicaltrials.gov and clinicaltrialsregister.eu were searched to identify ongoing or completed interventional Phase 3 or Phase 4 clinical trials conducted between March 1, 2009 and March 26, 2019 (see Supplementary Table 4 for search strategy). Labeling language was reviewed for 4 commonly used, approved products indicated for the treatment of FOS in adults (levetiracetam, lamotrigine, lacosamide, and oxcarbazepine) which have been available for some time to assess whether patient-relevant COI or PRO data were included. Additional details of the phase 1 literature review are provided in Supplementary Methods.

CDM Development

Findings from the qualitative literature search, including COI noted during qualitative interviews with patients and information from the Epilepsy Foundation on signs and symptoms experienced by patients, were used to develop our CDM focused specifically on patient-relevant outcomes for evaluation in adults with FOS.²⁵

Phase 2 Targeted Literature Review

The primary objectives of phase 2 were to identify available COAs capable of measuring the COI and to review the strength of evidence supporting the reliability and validity of a subset of COAs in a gap analysis.

From our CDM, 2 COI were chosen based on their suitability to measure relevant treatment outcomes in clinical trials: functional impacts and limitations as well as social and emotional impacts. Additionally, the concepts of social support and coping strategies were chosen to better understand patients’ lived experiences. To identify COAs used in clinical trials and observational studies capable of measuring these COI, a targeted literature search was conducted using published English language articles (beginning in 2011) and conference abstracts (beginning in 2016) through April 2020. Details of the search strategy are provided for MEDLINE/Embase and PsychINFO in Supplementary Tables 5 and 6, respectively. Additional details of the phase 2 COA literature reviews are provided in Supplementary Methods.

Gap Analysis

Ten COAs with content covering the COI were selected for review. The following data were extracted for the gap analyses: content (eg, recall period, number of items, domains within each COA), methods used for development (eg, literature review conducted, involvement of patients and/or clinicians), and measurement properties (eg, validity, reliability, responsiveness). The development population for each COA was reviewed for its similarity to the target patient population of adults experiencing FOS, and the development history was summarized. COAs were evaluated across the following measurement criteria: content validity, construct validity, internal-consistency reliability, test–retest reliability (including inter-rater reliability), the ability to detect change, and whether clinically meaningful change criteria were established. Further information and definitions for these measurement criteria are provided in Supplementary Methods.

Results

Phase 1: Qualitative Literature Review

A total of 136 abstracts from the qualitative literature and 30 abstracts from professional meetings were screened; after review, 13 articles and 1 professional meeting abstract were selected for inclusion in our CDM (Figure 1). Of the included literature, 4/13 articles (31%) and the meeting abstract reported signs and symptoms of FOS, while all articles (13/13 [100%]) and the meeting abstract reported impacts on patients (Supplementary Table 3). Symptoms reported in the literature included aura, blackouts, and dizziness or headaches (prior to seizure), while confusion during seizure was the only reported sign.

Figure 1 Flowchart of selected literature search findings. Abbreviation: n, number of search results.

A search was performed on clinicaltrials.gov and clinicaltrialsregister.eu to identify COAs used in phase 3 and 4 trials of patients with epilepsy. Of 81 identified clinical trials, 24 (30%) met the criteria for inclusion in the literature review (Figure 2). These trials measured seizure frequency, signs and symptoms, and burden, with a daily diary used most often (12/24 [50%] of trials) to measure seizure frequency, severity, and duration.

Figure 2 Flowchart of clinical trial identification. aChild or adolescent. bEarly terminated.

FDA and EMA labels for the products levetiracetam, lamotrigine, lacosamide, and oxcarbazepine, which are approved to treat patients with FOS, were reviewed. Eight labels were identified in the database review (Table 1). In instances where seizure frequency was used as a primary or secondary endpoint on a product label, a daily diary of seizure frequency, duration, and type was used. However, no additional PRO measures were identified in this focused label claims search.

Table 1 Results of Selected Products Review for Label Claims Approved to Treat Patients with FOS

Conceptual Disease Model

The CDM incorporates signs and symptoms of FOS from the literature review and those reported by the Epilepsy Foundation (Figure 3).^6,8 Signs and symptoms were classified to determine whether they occurred immediately prior to or during a seizure. Symptoms occurring immediately prior to a seizure included aura, unusual smells or tastes, dizziness or headaches, tingling or numbness, and an out-of-body feeling. Signs occurring immediately prior to a seizure included twitching or tremors, confusion, staring, and memory lapses; those occurring during an impaired awareness FOS included rubbing of hands, lip-smacking, chewing movements, and several frontal lobe effects, such as loss of awareness and inability to move or respond. Signs occurring during an aware FOS included being frozen, in which a patient may not be able to respond.

Figure 3 Signs and symptoms with related impacts by conceptual domain and their proximity to observable signs and symptoms.

The impacts of FOS on patients’ lives were grouped into categories: physical (eg, the ability to carry out daily tasks); cognitive (eg, impaired concentration and memory); emotional (eg, anxiety, worry, and stress); and social (eg, limitations on social activities and isolation). Patients experiencing FOS also reported impacts on caregivers, including less time for self-care and concern for patients experiencing seizures. These 5 impact categories were assembled into our CDM. COI of functional impacts and limitations, and social and emotional impacts were chosen as key measures for evaluating treatment outcomes for patients with FOS; social support and coping strategies were selected as an additional COI to understand overall patients’ lived experiences, to characterize the burden, rather than to evaluate the treatment benefit in the context of drug development (Figure 3). COAs suitable for measuring these COI were evaluated in phase 2.

Phase 2: Gap Analysis of COAs

A total of 445 abstracts were identified in the phase 2 targeted literature review. After reviewing 439 nonduplicate titles and abstracts, 137 were excluded due to the wrong population or study type, and 226 were excluded due to no mention of use of a COA, with 76 selected for review (Figure 4).

Figure 4 Flowchart of COA selection process. Abbreviations: ClinRO, clinician-reported outcome; COA, clinical outcome assessment; ObsRO, observer reported outcome; PerfO, performance outcome; PRO, patient reported outcome.

Fifty-one unique COAs were identified and categorized as shown in Table 2. COAs were categorized by the type of impacts on patients’ lives and reflect the concepts of the CDM, including: 20 COAs that measure seizure severity, functional impacts, and limitations; 29 COAs that measure social impacts (n = 2), emotional impacts (n = 10); other concepts such as HRQoL and medication side effects (n = 17); and 5 COAs for measuring coping strategies. Three COAs were classified into 2 different categories due to content coverage (2 symptom severity COAs and 1 HRQoL COA), resulting in a total of 54 unique COAs in 5 outcome categories.

Table 2 Phase 2 Unique COAs Based on Features Identified in the CDM

Five COAs included in our analysis were used as endpoints in phase 3 trials of adults with FOS. The Quality of Life in Epilepsy (QOLIE-31) assessment was used as a secondary endpoint in two trials, one evaluating eslicarbazepine acetate (https://ClinicalTrials.gov/show/NCT01162460) and one for remegal fixed dose (https://clinicaltrials.gov/ct2/show/NCT01749046). The Patients’ Global Impression of Change (PGI-C) assessment was used as a secondary endpoint in the same trial of remegal fixed dose as the QOLIE-31, while the Hospital Anxiety and Depression Scale-Anxiety (HADS-A), Hospital Anxiety and Depression Scale-Depression (HADS-D), and the Medical Outcomes Study Sleep Scale (MOS-SS) were used as secondary endpoints in a trial evaluating pregabalin controlled release (https://ClinicalTrials.gov/show/NCT01262677).

In-Depth COA Analysis

The content of the 10 COAs was determined to align closely with the selected COI from the CDM, based on the expected relevance for drug development programs. This subset was further analyzed for the development and measurement properties of each assessment. A brief overview of the findings is provided in Table 3.

Table 3 Phase 2 Gap Analysis Summary of Selected Measures

COAs to Measure Seizure Severity

Three COAs measuring seizure severity were evaluated.

Liverpool Seizure Severity Scale (LSSS) 2.0

The self-administered LSSS 2.0 uses a 12-item scale to assess patients’ perceptions of the most severe seizure they have experienced in the past 4 weeks (patients do not complete the full COA if they have not had a seizure in the last 4 weeks).²⁶ Patients are scored on a 4-point response scale, with 1 being least severe and 4 the most severe.^26,27 The LSSS 2.0 covers 3 domains (total, perception of seizure severity [percept], and ictal/postictal impacts) and results in a single unit-weighted score from 0 (no seizures) to 100 (most severe possible) after a linear transformation;^26,28 subscale scores are also available.²⁷ Both percept and ictal/postictal domains of the LSSS 2.0 assess the COI of functional impacts and limitations on patients, and social and emotional impacts.

Development Process

The LSSS 2.0 is the most recent version of the scale. The LSSS 1.0²⁹ was developed in 1991 to provide an assessment including seizure impact on patients. The tool assesses seizure severity based on perception of seizure control and severity of ictal and postictal phenomena in patients with simple partial, complex partial, secondary generalized tonic-clonic or primary generalized tonic-clonic seizures.²⁹ Four items were added to the LSSS 1.0 in 1998 to improve content validity, resulting in the Revised LSSS.²⁷ The Revised LSSS faced issues relative to the scale’s comprehensiveness, responsiveness, and ability to cope with individuals who have more than one type of seizure; therefore, in 2001, following feedback from patients and epileptologists, the Revised LSSS was modified to become the LSSS 2.0.²⁶

Measurement Properties

Two clinical trials of lamotrigine in patients with refractory epilepsy and FOS provided evidence in support of 4 out of 6 measurement criteria for the LSSS 2.0:²⁶ content validity, construct validity, internal-consistency reliability, and the ability to detect change (Table 3). Patients and epileptologists reported on factors that differentiated “minor seizures,” “major seizures,” and “most severe seizures,” and the LSSS 2.0 was scored on these seizure types. Internal-consistency reliability of ɑ = 0.66–0.87 was strong across all seizure types, and the highest internal reliability was found (ɑ = 0.73–0.87) for the most severe seizures. As most severe seizure scores from the LSSS 2.0 correlated closely to physician-rated seizure types, the study authors concluded these scores were the most accurate predictor of seizure severity. Known-groups validity (a form of content validity) was supported by this study. The review did not find evidence to support test–retest reliability of the LSSS 2.0, and a clinically meaningful change was not defined for this COA in patients with epilepsy.

National Hospital Seizure Severity Scale (NHS3)

The NHS3 is an 8-item ClinRO measuring seizure severity that spans 7 domains and is administered by a health professional in the presence of both the patient and a witness to the patient’s seizures.³⁰ Responses to questions about the patient’s seizure experiences since the last clinic visit are scored according to severity (2–5 options provided), resulting in a score from 1 (least severe) to 27 (most severe).³⁰ Domains that assess convulsion, fall, injury, incontinence, and warning correspond to the COI of functional impacts and limitations in our CDM. Domains of automatism and recovery map to COI of social and emotional impacts, and social support and coping strategies.

Development Process

The NHS3 is a revised version of the Chalfont Seizure Severity Scale (CSSS), which was originally developed from open interviews with patients with epilepsy and focused on the objective clinical events of a seizure.³¹ The CSSS was not responsive to change in a double-blind placebo-controlled trial.³² To create the NHS3, 4 items deemed to be redundant based on other items retained in the scale were removed from the CSSS, 1 item was revised, and the scoring system was simplified.³⁰ The redundant items that were removed included dropping objects (paralleled item on falls) and seizures occurring in sleep (change on this item was rarely observed); separate items on seizure duration and recovery phase were replaced with a single item on total time to complete recovery, and loss of consciousness was incorporated into the question on auras.³⁰

Measurement Properties

The NHS3 was first tested on a group of 87 patients with a variety of seizure types, including both impaired awareness and aware FOS, which provided evidence in support of 5 out of 6 measurement properties for the NHS3:³⁰ content validity, construct validity, internal-consistency reliability, test–retest reliability, and definition of a clinically meaningful change (Table 3). NHS3 scores were able to distinguish patients by seizure type, therefore supporting known-groups validity. Reliability of NHS3 scores was acceptable, including support for internal consistency, test–retest reliability when scores were assessed 1 to 8 weeks apart, and inter-rater reliability. While a minimally important change (MIC) threshold was not developed for this study, the authors suggested a 2–3-point change in NHS3 scores may indicate a clinically meaningful change for patients, based on findings from their validation experiments.³⁰ Evidence in support of the ability to detect change was not found for the NHS3.

Seizure Severity Questionnaire (SSQ)

The SSQ version 2.2 is a 24-item ClinRO measure assessing seizure activity experienced by the patient in the past 4 weeks administered by a health professional in the presence of the patient and a witness to the patient’s seizures.³³ The SSQ is scored on a 7-point scale and results in a score from 0/1 (very mild or no bother) to 7 (very severe or very bothersome), with assessments of change scored from 1 (very much improved) to 7 (very much worse).³⁴ Subscale scores can be generated for activity during seizures, overall recovery and severity scores, and the emotional, physical, and cognitive components of recovery.³⁴ Domains of the SSQ related to seizures (aura/warning, ictal/postictal events, seizure severity) and overall seizure severity relate to the COI of functional impacts and limitations in the CDM. Questions about seizure bothersomeness relate to the COI of social and emotional impacts.

The SSQ version 2.2 is the most recent version of this COA. The original SSQ was developed to account for changes in seizure expression with treatment, according to standards set by the FDA for PRO development²² and was based on a literature review of existing instruments.³⁵ The resulting draft was tested in patients with FOS and their observers. Review by an international panel of epileptologists resulted in the SSQ version 2.2, with questionnaires containing different items administered at baseline and follow-up.³³

Measurement Properties

Evidence in support of 5 out of 6 measurement criteria were identified for the SSQ (content validity, construct validity, test–retest reliability, the ability to detect change, and definition of a clinically meaningful change), though the specific version of the SSQ used in each study was not provided (Table 3). Construct validity was supported through a strong relationship with the LSSS 2.0 and a smaller correlation with the VA Seizure Frequency and Severity Rating COA. Evidence in support of the test-retest and inter-rater reliability of all summary scores for the 3 domains of the SSQ was identified.³⁵ SSQ total scores and scores on individual items differed between nonresponders and responders, defined as patients who experienced at least a 50% reduction in seizure frequency from baseline.³⁶ One study determined a MIC threshold of 0.48 points, suggesting that a change of this magnitude in the SSQ total score corresponds to a clinically meaningful change in seizure severity from the patients’ perspective.³⁴ Two later studies that used this threshold to assess changes in SSQ total scores between patients treated with an intervention versus placebo found that the treatment group exceeded the proposed MIC threshold.^37,38 Evidence supporting the internal-consistency reliability of the SSQ was not found.

COAs to Measure Epilepsy Impact (Functional, Social, and Emotional Impacts and Limitations)

Three COAs measuring the quality of life and functional impacts of epilepsy were evaluated.

Quality of Life in Epilepsy (QOLIE)

The QOLIE is self-administered by patients and measures HRQoL. Three versions of the QOLIE exist, each with a 4-week recall period. These versions of the QOLIE include 89 items spanning 17 subscales (QOLIE-89), 31 items spanning 7 subscales (QOLIE-31), and 10 items covering 3 subscales (QOLIE-10).^39–41 Individual items are scored on scales ranging from 1 to 2 points (for true/false items in the QOLIE-89) up to 6 points (for frequency-related items); overall scores range from 0 to 100, with higher scores indicating better HRQoL.^39,42 Scores for each of the subscales can be used to assess specific impacts. Subscales corresponding to the COI of functional impacts and limitations include medication effects, language, attention and concentration, memory, physical function, physical role limitation, energy and fatigue, and pain, plus 3 additional items covering changes in health, sexual relations, and overall health. Subscales measuring seizure worry, health discouragement, effects on work, driving or social functions, emotional role limitations, social isolation, overall quality of life, and health perceptions map to the COI of social and emotional impacts. Subscales measuring emotional well-being and social support relate to social support and coping strategies.

Development Process

The original QOLIE-89 was developed from generic and epilepsy-specific HRQoL measures based on input from patients with epilepsy, clinicians, and literature review.³⁹ The QOLIE-31 was derived from the QOLIE-89 as a tool for rapid evaluation of HRQoL and contains 31 items from 7 subscales considered most important to patients with epilepsy as determined by an expert panel.⁴¹ The QOLIE-10 was developed by scaling back the QOLIE-89 to 10 items chosen for their high correlation with patient outcomes across 7 domains.⁴⁰

Measurement Properties

Evidence in support of all 6 measurement criteria was identified for the QOLIE-89. Construct validity was supported by strong correlations between the QOLIE-89 and neuropsychological tests and with other PROs that measured emotional and cognitive function.³⁹ Known-groups validity of the QOLIE-89 was supported, as most factors and scales could distinguish between patients with high severity seizures and those who were seizure free for over 1 year.⁴² In patients with stable epilepsy, all subscales and the QOLIE-89 total score demonstrated acceptable intraclass correlation coefficients (ICCs) >0.7 for subscales, with mean QOLIE-89 total scores of 69.9 (standard deviation (SD): 15.2).⁴³ In a study of patients with medically refractory FOS, test–retest reliability was 0.87 (range: 0.77–0.93),⁴³ and the authors determined the minimum clinically important change (MCIC) to be a change of 10.1 points in the QOLIE-89 total score.⁴⁴

Evidence in support of all 6 measurement criteria was identified for the QOLIE-31. Internal-consistency reliability of overall ɑ = 0.93 and acceptable test–retest reliability (r > 0.7 for all scales, ICC = 0.89 for total QOLIE-31 score) was reported in a study of adults with epilepsy.⁴¹ A second study conducted in patients with stable epilepsy supported the test–retest reliability of the QOLIE-31 (test-retest: 0.86, range: 0.75–0.92).⁴³ In a follow-up study of patients with medically refractory FOS, the authors determined the MCIC to be a change of 11.8 points in the QOLIE-31 total score.⁴⁴ In a different study of patients with treatment-resistant epilepsy, an MIC threshold of 5.2 points in the QOLIE-31 total score was determined to represent a clinically meaningful change (either improvement or worsening of symptoms).⁴⁵

Evidence in support of 3 out of 6 measurement criteria was identified for the QOLIE-10: content validity, construct validity, and test–retest reliability. The content validity and construct validity of the QOLIE-10 were supported via an exploratory factor analysis of patients with epilepsy, though only limited evidence was found in support of test–retest reliability.⁴⁰ Unlike the other 2 QOLIE COAs, the internal consistency of the QOLIE-10 was low (ɑ < 0.6 for all scales), and no evidence for the ability to detect change or definition of a clinically meaningful change was identified.

Work Productivity and Activity Impairment (WPAI)

The WPAI-General Health (GH) and WPAI-Specific Health Problem (SHP) are self-administered measures used to assess the effect of general health and symptom severity on work productivity and regular activities for the previous 7 days.⁴⁶ In the GH version, patients are instructed to respond with reference to their general health status versus to a particular health problem, disease or condition in the SHP version. Both versions contain 6 items with 4 domains corresponding to time missed from work, time worked, days during which performing work was difficult, and the extent to which the individual was limited at work or felt their work was impaired. The WPAI is scored within domains, with each item assessed on a 6-point scale (lower scores indicating more severe symptoms), and an overall work productivity score is calculated assessing the percentage of work time and productivity at work.⁴⁷ All domains of the WPAI map to the COI of functional impacts and limitations in our CDM.

Development Process

The WPAI-GH and the WPAI-SHP were developed simultaneously from the work productivity literature and refined in response to comments from patients with allergic rhinitis.^46,47

Measurement Properties

No evidence was found to support any of the 6 criteria pertaining to measurement properties of either the WPAI-GH or the WPAI-SHP COAs in patients with epilepsy.

Personal Impact of Epilepsy Scale (PIES)

The PIES is used to evaluate the impacts of epilepsy and overall QoL and consists of a 25-item self-administered scale. Items represent 4 domains, including impact of seizures, medication side effects, impact of comorbidities, and overall QoL; each item has 5 answers scored from 0–4, with higher scores reflecting more severe impacts of epilepsy.⁴⁸ Overall and domain-specific scores are reported.⁴⁸ The recall period and administration information for the PIES were not available in the literature. All domains of the PIES assess functional impact and limitations on patients, with some overlap with the COI of social and emotional impacts in our CDM.

Development Process

The PIES was developed from open-ended input from patients with epilepsy, including their concerns, issues, questions, and priorities to provide a multidimensional PRO scale to quantify the overall impact of epilepsy on aspects of life important to patients.⁴⁸ Twenty-six items were chosen based on spontaneous participant feedback and correlations with the NHS3, QOLIE-31, Liverpool Adverse Event Profile, and Beck Depression Inventory measures, and one item was removed after validation, resulting in the final 25-item scale.

Measurement Properties

Evidence in support of 3 out of 6 measurement criteria was identified: content validity, internal-consistency reliability, and test–retest reliability. The test–retest reliability of PIES was assessed during development, when the same patients completed the COA at least 3 days apart.⁴⁸ Twenty-one items had acceptable ICCs > 0.7, indicating substantial or perfect internal consistency. Four items had ICCs < 0.67 (range: 0.56–0.67); one of those with the lowest ICC was rejected, and the other three were retained due to their perceived importance to the questionnaire’s internal structure. Internal-consistency reliability was supported for the PIES total score (α = 0.87). No evidence was found to support construct validity or the ability to detect change for the PIES, and a clinically meaningful change was not defined.

COAs to Measure Epilepsy Treatment Side Effects

Another tool of interest focusing exclusively on the side effects of treatment was evaluated.

Side Effects and Life Satisfaction (SEALS)

The SEALS is a self-administered 38-item assessment covering 5 subscales used to evaluate the side effects of epilepsy medications over the past week on a 4-point Likert-type scale.^49,50 Principal components of analysis-based regression weights are used to score each factor, and subscale scores are totaled for a total SEALS score. Two domains measuring temper and worry map to the COI of social and emotional impacts in our CDM, while the other 3 domains measuring cognition, dysphoria, and tiredness correspond to functional impacts and limitations on patients.

Development Process

The original version of the SEALS was developed to assess the interaction between antiseizure medication and psychosocial function by asking patients with epilepsy and their physicians about areas of functioning impacted by epilepsy medication, including cognitive difficulties, fatigue, interpersonal difficulties, and mood fluctuations.⁵¹ These interviews resulted in a 50-item version of the SEALS, which was revised to exclude 38 items based on the results of exploratory factor analysis.⁵²

Measurement Properties

Evidence supporting 4 out of 6 measurement criteria were identified for the SEALS: content validity, construct validity, internal-consistency reliability, and test–retest reliability. In an analysis of 3 clinical trials of lamotrigine, the internal consistency of the domains was found to be acceptable (ɑ > 0.7) for all domains except worry, with ɑ = 0.87 for total SEALS scores.⁵² The test–retest reliability for the total SEALS score was supported by r = 0.79 for the Spearman–Brown correlation.⁴⁹ Construct validity was supported through strong associations (correlation range: r = 0.53–0.84) between the SEALS and other related measures of mood, cognition, and depression.⁵⁰ No evidence in support of the ability to detect change or definition of a clinically meaningful change was found for the SEALS.

COAs to Measure Social Support and Coping Strategies

Three COAs measuring social support and coping strategies were evaluated.

Family APGAR

The Family APGAR is a 5-item PRO measure, self-administered by patients to evaluate adult and child satisfaction with social support from the family; no recall time frame is identified. Each item is scored from 0 (hardly ever) to 2 (almost always).⁵³ Total scores range from 0 to 10, with higher scores indicating better family functioning. This COA covered 5 domains corresponding to the COI of social support and coping strategies in our CDM, including adaptation, partnership, growth, affection, and resolve.

Development Process

The development of the original version of the Family APGAR is not documented.⁵³ The original 5 items were revised by 4 experts in school psychology to be understandable for children who could read at the second-grade level.⁵⁴

Measurement Properties

No evidence for any of the 6 measurement criteria for the Family APGAR was identified.

Adult Epilepsy Self-Management Instrument-65 (AESMMI-65)

The AESMMI-65 is a self-administered 65-item scale covering 11 domains and is used to evaluate the frequency of use of epilepsy self-management practices.⁵⁵ Each item is scored on a 5-point Likert-type scale for the patient’s behaviors in the past 3 months, with higher scores indicating more frequent self-management behaviors. Scores are reported for each subscale and overall. Four domains measuring seizure tracking, seizure response, safety, and medication adherence map to the COI of functional impacts and limitations within our CDM. Two domains measuring healthcare communication and treatment management assess social and emotional impacts. Five domains measuring coping, social support, wellness, stress management, and proactivity correspond to patients’ social support and coping strategies.

Development Process

The AESSMI was developed to provide a comprehensive epilepsy self-management assessment tool after a literature review of epilepsy self-management from 422 adults with epilepsy, followed by an expert panel review.⁵⁶ Twelve items on the AESSMI were adopted from original scales, 69 were rephrased from original scales, and 35 new items were created. Additional refinement and removal of items that do not contribute to content validity resulted in the current AESSMI-65-item tool.⁵⁵

Measurement Properties

Evidence in support of 3 out of 6 measurement criteria was identified: content validity, internal-consistency reliability, and construct validity. An expert analysis of the original version of the AESMMI, which contained 116 draft items, reported a content validity index of 89%.⁵⁶ In a study of patients with epilepsy, the total AESMMI-65 score demonstrated high internal-consistency reliability.⁵⁵ Six of 11 domains (55%) showed high internal consistency reliability with values >0.7, while the remaining 5 domains demonstrated low internal consistency. The AESMMI-65 demonstrated small correlations (range: r = –0.13 to –0.32) with other epilepsy-related measures, including health status, QoL, depression, seizure severity, and seizure problems/effects, which supports the convergent validity of this COA. No evidence was found to support the test–retest reliability or ability to detect change for the AESMMI-65, and a clinically meaningful change was not identified.

Self-Efficacy, Assertiveness, Social Support, Self-Awareness, and Helpful Thinking in People with Seizures (EASE)

The EASE is self-administered by patients and is a 40-item scale spanning 5 domains used to identify and clarify treatment targets at the start of psychotherapy. Each item has 2–5 possible responses, but scoring information was not publicly available for the EASE assessment, and a recall period was not defined.⁵⁷ Domains measuring general self-efficacy and epilepsy-specific self-efficacy correspond to the COI of social and emotional impacts in our CDM, while domains measuring mastery, resilience/vulnerability, and self-awareness or acceptance map to social support and coping strategies.

Development Process

The EASE was developed from interviews with patients with epilepsy participating in resource-oriented and mindfulness-based psychotherapy to complement the use of other COAs in the context of psychotherapeutic interventions for patients with epilepsy. Initial item development was followed by assessment of content validity by experts, a pilot study in patients with epilepsy and other conditions, and a second expert panel to assess validity.^57,58 Three items were eliminated, and 2 items were revised from the original version of the EASE, resulting in the current 40-item version.

Measurement Properties

Evidence in support of 3 out of 6 criteria was identified for the EASE: content validity, internal-consistency reliability, and construct validity. A pilot study in patients with epilepsy, which used the original version of the EASE, reported acceptable internal-consistency reliability, with ɑ = 0.92 for the total EASE score, and acceptable to good scores for subscales, except for the epilepsy self-efficacy subscale (ɑ = 0.51).⁵⁷ Following the pilot study, a panel of experts reported the content validity for individual items to range from 0.38 to 1, with a content validity index for the overall scale of 0.92.⁵⁷ This COA was updated following these studies; however, no evidence in support of reliability or validity was identified for the most recent version. No evidence was found to support the test–retest reliability or the ability to detect change, and a clinically meaningful change was not defined for the EASE.

Discussion

Interpretation of epilepsy clinical trial data can be challenging,^59,60 particularly as primary study outcomes may not closely align with the outcomes that are most important to patients.⁶⁰ For example, a 50% reduction in seizures is a common endpoint in clinical trials for patients with epilepsy.⁶¹ While relevant, this endpoint does not reflect patients’ experience of freedom from seizures, and, therefore, the impact of the condition on patients persists.⁶¹ An abundance of COAs exists to evaluate outcomes in epilepsy, but these vary widely in their suitability for use in the context of a clinical trial. In our research, the objectives were two-fold: in phase 1, we reviewed the literature to identify concepts important to patients with epilepsy, resulting in the creation of our CDM aimed at understanding the impact of FOS on patients’ lives and selection of the COI to evaluate outcomes in clinical trials. In phase 2, we identified available tools to measure those COI and evaluated their content. A subset of COAs that covered the COI were investigated for evidence supporting their development process and measurement properties.

Signs and symptoms of FOS were mentioned less frequently than impacts in the qualitative literature (5 out of 14, 36%), including aura, blackouts, and dizziness or headaches as signs and symptoms occurring prior to seizure, with confusion occurring during seizure. The CDM was built on the findings of this qualitative literature search and revealed that FOS impact patients’ lives in a complex, far-reaching grid. These impacts range from proximal physical impacts, such as the ability to drive a car, to more distal social and emotional impacts, such as fear of talking with others. The literature indicated that both patients and caregivers expressed concern about the impact of seizures on caregivers. The CDM was then used to inform the search strategy for COAs for measuring the patient experience related to treatment outcomes, functional, social, and emotional impacts, as well as concepts relating to social support and coping strategies.

The phase 2 literature search reviewed published articles and abstracts to identify COAs that measured COI in the CDM. While 51 unique COAs were identified, a content review highlighted a subset of 10 COAs with good coverage of the COI and potential utility in clinical research: LSSS, NHS3, SSQ, QOLIE, WPAI, PIES, SEALS, Family APGAR, AESMMI-65, and EASE.

The LSSS, NHS3, and SSQ, which were designed to evaluate seizure severity, involved patients during the development process. While the SSQ and LSSS 2.0 were developed specifically in patients with FOS, the NHS3 was evaluated in a study that included patients experiencing both impaired awareness and aware FOS, tonic clonic seizures, typical absence, or myoclonic epilepsy. Considerable overlap exists between the domains mapped by these COAs and COI in the CDM. Functional impacts such as falling, injury, and incontinence can also have social and emotional impacts on patients’ lives. The overall evidence supporting the measurement properties for all 3 COAs assessing seizure severity was strong, but each had at least one analysis criterion for which evidence could not be identified. Among these, the SSQ, a ClinRO measure, most closely aligns with development and validation standards suggested by the FDA,²² as it was developed for use in a clinical trial setting to overcome some of the weaknesses of the LSSS 2.0 and NHS3.

Our gap analysis identified evidence in support of 5 out of 6 criteria (83%) for the SSQ, with an MIC threshold developed in multiple studies. Likewise, we identified evidence in support of 5 out of 6 criteria for the NHS3; evidence in support of the ability to detect change was not identified for this COA. However, while O’Donoghue et al hypothesized what might constitute a meaningful change in NHS3 scores for patients, evidence from a clinical trial setting, in which a clinically meaningful change in NHS3 scores was defined, was not identified.³⁰ No evidence was found to support either test–retest reliability or the definition of a clinically meaningful change for the LSSS 2.0. These findings suggest that the evidence supporting the use of the SSQ assessment as a measure of symptom severity is compatible with most FDA guideline recommendations but would require additional work to generate the evidence for meaningful change. The NHS3 could potentially meet FDA criteria for assessing symptom severity, but additional work validating the tool is needed.

For COAs measuring functional, social, and emotional impacts, the QOLIE-89, QOLIE-31, QOLIE-10, WPAI, and PIES were investigated. Both the QOLIE (all versions) and the PIES were developed specifically for use in patients with epilepsy.

The QOLIE-10 did not contain domains that mapped to the concept of social support and coping strategies, while the other two QOLIE COAs had at least one domain that mapped to this concept (emotional well-being for the QOLIE-31, emotional well-being and social support for the QOLIE-89). The QOLIE-10 demonstrated low internal-consistency reliability, with clinically meaningful change criteria yet to be defined. Evidence supporting all 6 analysis criteria was identified for the QOLIE-89 and QOLIE-31, including the proposed MCIC thresholds for the QOLIE-31 and the QOLIE-89 in patients with medically refractory FOS. Of note, among the 10 COAs evaluated in our gap analysis, only the QOLIE-31 was actively used in epilepsy clinical research, in two trials as a secondary endpoint (NCT01749046, NCT01162460).

No evidence to support any of the 6 analysis criteria in our gap analysis was found for the WPAI in patients with FOS. For the PIES, support for only 3 out of 6 criteria was identified, with no evidence found to support construct validity, the ability to detect change, or the definition of a clinically meaningful change. These findings suggest that the evidence supporting the use of the QOLIE-31 and QOLIE-89 assessments as measures of HRQoL is compatible with guidelines proposed by the FDA for the use of PRO measures in clinical trials, though the QOLIE-31 may be more compatible for routine clinical use.

The Family APGAR, AESMMI-65, and EASE were investigated as measures of social support, self-management, and self-efficacy, all COI in the CDM reflecting social support and coping strategies to evaluate the burden of the condition. While use of the Family APGAR in pediatric populations is well established (eg,⁶²), no evidence was found in support of any of its 6 analysis criteria for adults with FOS. For both the AESMMI-65 and the EASE, evidence in support of only 3 out of 6 measurement criteria was found; no evidence supported test–retest reliability, the ability to detect change, or the definition of a clinically meaningful change for either assessment. For the AESMMI-65, overall internal-consistency reliability was acceptable, but results were mixed for the individual domains. Neither the Family APGAR nor the EASE appeared to evaluate functional impacts or limitations on patients to assess outcomes in the context of drug development.

The SEALS, developed using feedback from patients with epilepsy and their physicians, was the only tool reviewed that was specifically developed to measure medication side effects. Measuring the functional impact of medication side effects, including potentially new adverse effects from novel drug classes and side effects specific to patients with FOS is important for PROs to support drug development. Of note, 24 pharmaceutical compounds are currently available to treat patients with FOS, nearly a 3-fold increase from the 9 approved prior to 1993.^1,63 However, the SEALS, developed in 1982, remains the only assessment explicitly used to measure medication side effects in adults. The PIES, developed in 2015, contains 7 out of 25 items that address side effects but is not exclusive to these outcomes. This shortfall highlights the inherent problem with most COAs: epilepsy is more complex than simply the number of seizures experienced by a patient.

The most frequently used assessment in phase 3 or 4 clinical trials was a daily diary of seizures; likewise, patient diaries of seizure frequency, severity, and duration were the only assessment that appeared in the focused label claims search. Five of the COAs included in this analysis were secondary endpoints in two phase 3 trials of adults with FOS. With the need for new COAs reflecting patient-relevant impacts, tool development would also benefit from the consideration of vulnerable populations of patients with epilepsy. To be most useful in clinical research, COAs need to be developed to account for disparities across a wide range of domains, including age, race, ethnicity, and patients’ comorbidities, including disabilities.¹⁴ Likewise, investigators should choose COAs thoughtfully relative to the evidence supporting their clinical validity and reliability in the chosen target population.

Limitation

The current study was limited by evaluating only published literature; qualitative research with the target sample may reveal additional COI, in the context of patients’ experiences, including with new treatments and devices. Some information related to scoring or development of some COAs is not publicly available and was not included. Future studies may benefit from contacting the developers to pursue additional information. Likewise, 10 COAs were selected for the gap analysis based on their coverage of COI from our CDM and their potential utility within clinical trials; however, real-world practice differs from clinical trials, and other COAs may be more appropriate for routine clinical use. Future efforts to understand the patient’s perspective in FOS would benefit from evaluating trial results of more recent epilepsy interventions, as these would be expected to include COAs within the endpoint hierarchy.

Conclusion

The CDM, developed based on a review of the literature, identified COI important to adults experiencing FOS. Most of the COAs reviewed for evaluating these COI contain domains for assessing functional impacts and limitations experienced by patients; the NHS3 and SSQ were supported by evidence compatible with FDA PRO measure guidelines. Most of the COAs reviewed also contained domains for measuring social and emotional impacts. Fewer COAs assessed the COI of social support and coping strategies, and most were not backed by evidence supporting their measurement properties. Indeed, only the QOLIE-31 and QOLIE-89 mapped directly to this COI selected to evaluate treatment outcomes and had strong evidence in support of their validity and reliability. This analysis may enable clinical trial investigators and staff to explore these COAs as potential endpoints in trials of new epilepsy treatments, while more readily identifying gaps in their applicability. However, care is needed to identify COAs suitable for a given research purpose, as some measures of impacts on patients, particularly as related to social support or coping strategies, may not yet exist.