Disutility of Cognitive Processing Speed (CPS) Impairment in the Context of Multiple Sclerosis: A Time Trade-Off (TTO) Elicitation Study

Introduction

Multiple sclerosis (MS) is a chronic, inflammatory, and degenerative autoimmune disease of the central nervous system characterized by neuronal and axonal demyelinating injury that leads to both physical and cognitive disability.¹ Clinical features of MS may include weakness, sensory impairment, fatigue, impaired mobility, pain, depression, sexual dysfunction, bowel and bladder dysfunction, vision problems, and cognitive impairment.² Cognitive impairment is a major cause of disability affecting 34–65% of patients with MS.³ Cognitive impairment, together with related symptoms such as pain, fatigue, and depression, detrimentally affect many aspects of daily life, including family life, the ability to maintain employment, and to participate fully in society.⁴

Disability in MS is commonly quantified using the clinician-measured Expanded Disability Status Scale (EDSS).⁵ The EDSS quantifies disability using a 0–10 scale, with higher values indicating greater degrees of physical disability. It incorporates functional systems pertinent to MS: pyramidal, cerebellar, brainstem, sensory, bowel and bladder, visual, and cerebral.⁵ Progression to EDSS scores of >5.0 to 9.5 is mainly based upon ambulation impairment levels.⁵ Thus, progression on EDSS is commonly regarded as evidence of physical (rather than cognitive) disability.

Increasing physical disability as assessed by EDSS is associated with a significant decrease in health-related quality of life (HRQoL), as measured by utility (a measure of individual or population preference for a given health state, typically expressed on a scale between 0.0, representing death, and 1.0, representing perfect health). These health state utility values are frequently used in research that seeks to understand the economic and humanistic burden of MS or the impact of new therapies.

However, the EDSS has been criticized for its over-reliance on deficits in ambulation while neglecting the salient factor of cognitive impairment.⁶ One of the most widely accepted metrics of cognitive function in MS is the Symbol Digit Modalities Test (SDMT).⁷ The SDMT is a performance measure that discriminates individuals with MS versus healthy controls.⁸ Most studies show only a weak correlation between SDMT and the EDSS,⁹ and recently the sensitivity of the EDSS to relapses was enhanced by augmenting the EDSS score with SDMT results.¹⁰

Cost-effectiveness models for new treatment options typically incorporate estimates of patient HRQoL in the form of health state utility values. While generic (not disease-specific) instruments for generation of utility values, such as the EuroQoL five-dimensions questionnaire (EQ-5D) or Health Utilities Index (HUI), are generally preferred by health technology assessment (HTA) bodies, these instruments do not fully capture cognitive impairment.⁶ Health state utility values derived from these instruments may not provide a true reflection of patients’ HRQoL without performance-based measures.

Therefore, direct elicitation of the disutility brought about by cognitive impairment may be appropriate, to allow this symptom and its associated functional impairment to be adequately represented in economic modeling.

The primary objective of this study was to generate health state utility values for the varied levels of SDMT-measured cognitive impairment associated with MS, using a direct utility elicitation approach: the time trade-off (TTO) methodology.¹¹ A representative sample of the general population in the United Kingdom (UK) was sought to generate values. These outputs should allow future modeling in MS and other conditions where CPS impairment is seen, in order to provide a more comprehensive profile of the relevant aspects of disease affecting patients’ HRQoL.

Materials and Methods

This direct utility elicitation study was conducted between June and December 2022 and consisted of two phases: a qualitative phase comprising interviews with healthcare practitioners (HCPs), patients, and caregivers in the UK and United States (US) (to allow definition, development, and validation of written health state descriptions; total n=35), and a quantitative stage where utility values were elicited for each health state description during interviews with members of the general public in the UK (total n=120; see Figure 1 for full overview).

Figure 1 Overview flow chart of qualitative and quantitative study activities. Abbreviations: CPS, cognitive processing speed; HCP, healthcare professional; MS, multiple sclerosis; UK, United Kingdom; US, United States.

Table 1 presents advantages and disadvantages of two potential approaches for generation of health state utility values. In short, although development and valuation of health state description “vignettes” is not the conventionally preferred approach, this option was undertaken here due to its potential improved sensitivity in capturing changes in HRQoL relating to cognition, and its improved ability to isolate the HRQoL effect of impaired CPS.

Table 1 Advantages and Disadvantages of Potential Approaches Considered for Generation of Health State Utility Values

Detailed qualitative findings are presented in Results of definition, development, and validation of health state descriptions via qualitative interviews. Detailed quantitative results are presented in Results of health state utility elicitation via quantitative interviews.

As the SDMT is rarely used in routine neurology practice in the UK, initial definition and development of health state descriptions were undertaken via qualitative interviews in the US, where use of the SDMT is more established (SDMT is widely used by neuropsychologists in both countries).

Ethics exemption for this non-interventional study was granted by the UK National Health Service Research Ethics Committee, and local institutional research boards committees of participating neurology centers in the US (Providence Portland Medical Center: study ID #2021000756; University of Buffalo Institutional Review Board study ID #00006382). All study participants provided verbal informed consent based on standard form prior to their enrollment.

Methods for Definition, Development, and Validation of Health State Descriptions via Qualitative Interviews

To allow valuation of health state descriptions representing varying levels of SDMT performance, appropriate health states were first defined, and then described. A set of qualitative interviews for this purpose, each lasting approximately one hour, were conducted between June and November 2022. All interviews were conducted on a one-to-one basis by telephone and were facilitated through use of structured interview guides developed for each of the three respondent types.

Definition and initial development of written health state descriptions were undertaken based on insights gathered in the US, while validation of these descriptions was undertaken based on insights from UK-based healthcare professionals (HCPs).

Detailed qualitative findings are presented in Results (see Results of definition, development, and validation of health state descriptions via qualitative interviews).

Methods for Health State Utility Elicitation via Quantitative Interviews

A set of quantitative interviews with members of the general public in the UK, each lasting one hour, were conducted in order to value health states according to the TTO methodology outlined below, in November and December 2022.

Utility Elicitation Process

A TTO process based on the established “Measurement and Valuation of Health” (MVH) protocol was used to elicit utilities.²¹ The TTO approach is a mathematical exercise where respondents are asked to hypothetically consider giving up additional years of life in a given health state, in order to instead experience perfect health. As per the standard MVH approach, a timescale of 10 years was used; therefore, respondents answered an iterative series of questions indicating their preference for either x years in full health or 10 years in the health state, until a point of indecision was reached.²¹

Initially, all respondents were asked to consider 10 years in full health versus 10 years in the health state. If full health was selected, this was iterated to 0 years in full health versus 10 years in the health state. If the health state was selected, this was again iterated to 5 years in full health versus 10 years in the health state. Dependent on further responses, this could be further iterated up or down in 1-year, 0.5-year, and 0.25-year increments. The final result of each TTO exercise was a point of indecision where x years in full health was considered equivalent to 10 years in the health state, thereby allowing calculation of the perceived utility associated with that health state. For example, if a respondent reached a point of indecision at 7.5 years of full health versus 10 years in the health state, this indicated that the perceived utility of the health state was 0.75 of full health.

Due to the ongoing COVID-19 pandemic at the time of fieldwork and to improve feasibility and comfort for interviewees, all interviews were web-enabled and conducted 1:1 by a trained TTO moderator, who used visual aids to improve respondents’ comprehension of the valuation tasks. This approach is standard within the MVH and related valuation protocols.²¹ To avoid bias in participants’ responses, health state descriptions were not labeled in any manner that suggested differences in severity or importance and were presented to respondents in a random order.

Prior to TTO valuation, visual analog scale (VAS) scoring of the same health state descriptions was used as a “warm-up” exercise to allow participants to become comfortable with examining and valuing health state descriptions, and to collect additional data points for comparison and validation of TTO results. However, all health state utility values were derived by the TTO method. Prior to VAS valuation of each health state description, each participant was asked to value their own health on that day, using the same VAS scale.

Utility Elicitation Sample

A representative sample of the UK general population was recruited for interview using random sampling, according to predefined quotas based on UK census data.^22–25

In addition, the proportion of participants who reported having cognitive impairment, or who reported being caregivers for individual(s) with cognitive impairment, was capped at 15.0%, to ensure that the sample was representative of the UK general population as a whole (in terms of experience with the condition), to avoid bias at the valuation step.¹¹ This 15% figure was selected to represent the approximate proportion of the UK general population who would be expected to have cognitive performance of 1.5 standard deviations below the mean.

Participants were recruited using random sampling by a third-party agency and were remunerated for their time according to fair market values.

Pilot interviews (n=10) were conducted to validate respondents’ understanding of the health state descriptions, followed by the main phase (n=110). As no changes were made to the health state descriptions following the pilot interviews, data were included in the final calculation of health state utility values (therefore giving a total sample size of n=120).

Utility Data Analysis

As per the standard MVH approach, results provided by members of the general public in TTO exercises were values between 0 and 10 years (representing the point of indecision where each respondent reported being unable to select between x years in full health and 10 years in the health state).²¹ Results were subsequently converted to health state utility values between 0.0 and 1.0 through dividing by 10.²¹ For example, if a respondent reached a point of indecision at 5.5 years of full health versus 10 years in the health state, health state utility was calculated as 5.5 ÷ 10 = 0.55.

Initially, the distribution of individual TTO utility values within each health state was examined by Shapiro–Wilk test, to determine whether data were normally distributed. As the distribution of values within each health state was significantly skewed (p<0.05), the significance of differences in TTO utility values between health states was tested using non-parametric tests for skewed, non-normal data, namely the Kruskal–Wallis test (examining whether significant differences exist within the set of health states as a whole) and pairwise Wilcoxon tests (examining whether significant differences exist between pairs of health states). A threshold of p<0.05 was again applied.

Results

Results of Definition, Development, and Validation of Health State Descriptions via Qualitative Interviews

In total, 35 interviews for health state definition and development were conducted in the US (comprising 10 HCPs, 13 patients, and 12 caregivers), and 5 interviews for health state validation were conducted with UK-based HCPs.

Interviewed patients with MS and care recipients of interviewed caregivers, performed at least 1 standard deviation below an age-adjusted mean on SDMT, derived from normative data for this measure.²⁶ Interviewed HCPs were involved in the diagnosis and/or management of individuals with cognitive impairment, all having become qualified in their medical specialty at least three years prior, and having contact with ≥30 patients with MS per month (neurologists and MS nurses) or ≥5 patients with MS per month (neuropsychologists). The US-based HCPs interviewed as part of development of health state descriptions were also required to report being at least “somewhat” familiar with the SDMT; for UK-based HCPs interviewed as part of the validation of health state descriptions, familiarity with the SDMT was preferred, but not essential.

Definition of Health States by SDMT Categories, via Qualitative Interviews

Health states of interest were initially defined based on qualitative insights gathered from 10 HCPs based at two neurology sites specializing in treatment of MS in the US (Providence Multiple Sclerosis Center, Portland, Oregon and Jacobs MS Center, Department of Neurology at Buffalo, New York). Interviewed HCPs comprised 7 neurologists, 2 neuropsychologists involved in diagnosis of CPS impairment, and 1 nurse practitioner specializing in management of MS. A summary of key findings relating to definition of CPS impairment health states by HCPs is provided in Table S1.

These findings were used to define initial health states of interest, in relation to the expected cognitive capacity of individuals in a given age group. Subsequently, normative SDMT data and observational data from MS patients in the UK were applied to generate specific SDMT categories to define each hypothetical health state (see Table 2).^26,27

Table 2 Health State Definitions (as per Qualitative HCP Insights, Normative SDMT Data, and UK Observational Data in MS)

Development of Health State Descriptions via Qualitative Interviews

Following definition of health states of interest, detailed health state descriptions (and associated introductory text) were developed, through analysis of findings from the aforementioned qualitative interviews.

Patient and caregiver interviewees were recruited across a range of CPS impairment categories. Patients with MS (n=13) comprised 3 with raw SDMT of 40–49, 8 with raw SDMT of 30–39, and 2 with raw SDMT of 20–29. With application of age-adjusted thresholds, patients with MS fell into the following categories: 2 with “mild” impairment, 7 with “moderate” impairment, and 4 with “severe” impairment. Caregivers (n=12 in total) comprised 3 with care recipient with raw SDMT of 40–49, 6 with care recipient with raw SDMT of 30–39, and 3 with care recipient with raw SDMT of 20–29. With application of the same age-adjusted thresholds, caregivers fell into the following categories: 2 with care recipient with “mild” impairment, 5 with care recipient with “moderate” impairment, and 5 with care recipient with “severe” impairment.

Interviewees provided sufficiently detailed information on symptoms, functional impairment, and impact on emotion, which was used to inform the drafting of the health state descriptions and associated introductory text. A summary of key findings used in development of health state descriptions is provided in Table S2.

Validation of Health State Descriptions via Qualitative Interviews

After development, the drafted health state descriptions and associated introductory text were tested in validation interviews, where n=5 UK-based HCPs with experience of managing MS-related CPS impairment were able to comment on the accuracy and comprehensibility of each description. This sample comprised n=3 neurologists, and n=2 nurse practitioners specialized in the management of MS. These interviewees provided their assessment on the validity of the health state descriptions, and were able to put forward concepts or wording for inclusion or exclusion from the descriptions. These insights were subsequently used to develop the finalized validated health state descriptions. A summary of key findings used in validation of health state descriptions is provided in Table S3.

Final Health State Descriptions

Finalized validated health state descriptions and associated introductory text are presented in Tables S4 and S5.

Results of Health State Utility Elicitation via Quantitative Interviews

Utility Valuation Interview Respondent Characteristics

Characteristics of the 120 interviewed members of the public are presented in Table 3. The UK general population quotas used for recruitment are also presented in Table 3 for comparison. The interview sample was generally representative of the wider UK population when compared to UK general population quotas, in terms of age (<60 years: 70% versus 71%), gender (female: 51% versus 51%), marital status (married/civil partnered: 51% versus 51%), educational status (attainment of degree/A-level/higher/further education: 61% versus 60%), geographical region (South or East of England: 47% versus 45%) and employment status (working full- or part-time: 60% versus 60%).

Table 3 Demographic Characteristics of Participants in TTO Interviews (n=120) and UK Public

Participants’ “own health” VAS results were also collected. Mean “own health” VAS was 68.01, and median “own health” VAS was 75.00.

TTO Utility Results

Utility values derived via TTO for each health state are presented in Figure 2 and Table 4. Mean values revealed that the public perceives that increased CPS impairment is associated with decreased utility. Mean health state utility values (± SD) were 0.77 ± 0.24 in mild, 0.57 ± 0.26 in moderate, and 0.34 ± 0.28 in severe CPS impairment.

Table 4 TTO Utility Value per Health State

Figure 2 TTO utility value per health state (n=120 responses to each). HS1, “mild CPS impairment”: SDMT 43-40 (indicating deficit of ≥22% versus average for a population aged ~55 who are predominantly women; deficit of ≥26% versus average for a population aged ~40); HS2, “moderate CPS impairment”: SDMT 39-32 (indicating deficit of ≥29% versus average for a population aged ~55; who are predominantly women; deficit of ≥33% versus average for a population aged ~40); HS3, “severe CPS impairment”: SDMT ≤31 (indicating deficit of ≥44% versus average for a population aged ~55 who are predominantly women; deficit of ≥47% versus average for a population aged ~40). Abbreviations: CPS, cognitive processing speed; n, sample size; TTO, time trade-off.

Shapiro–Wilk testing demonstrated that the utility values in all health states were skewed (p<0.05). Therefore, non-parametric tests were employed to examine the significance of differences between health states. The Kruskal–Wallis test showed a significant difference in utility values within the set of health states as a whole (p<0.05), and pairwise Wilcoxon testing showed significant differences in utility values between all pairs of individual health states (p<0.05).

Sensitivity and Subgroup Analyses

With removal of extreme values (0.0 and 1.0), mean health state utility values continued to decrease with increasing level of impairment (Figure S1), and remained significantly different by both Kruskal–Wallis test and pairwise Wilcoxon testing (p<0.05).

With removal of values that were two or more standard deviations away from each health state mean, mean health state utility values again continued to decrease with increasing level of impairment (Figure S2) and remained significantly different by both Kruskal–Wallis test and pairwise Wilcoxon testing (p<0.05).

Subgroup analyses were conducted to determine any effect of demographic characteristics on valuation of health states. Differences between health states continued to be significant (p<0.05) in analyses of subgroups defined by age (high, medium, low age tertiles), gender (male versus female), and employment (full- or part-time employment versus other).

VAS Scores

VAS scores collected for each health state are presented in Figure 3 and Table 5. As seen in the TTO utility data, mean scores demonstrated that the public perceive that increased CPS impairment is associated with decreased scores in each case. Mean VAS scores varied between 55.17 in mild, and 25.61 in severe CPS impairment.

Table 5 VAS Score per Health State

Figure 3 VAS score per health state (n=120 responses to each). HS1, “mild CPS impairment”: SDMT 43-40 (indicating deficit of ≥22% versus average for a population aged ~55 who are predominantly women; deficit of ≥26% versus average for a population aged ~40); HS2, “moderate CPS impairment”: SDMT 39-32 (indicating deficit of ≥29% versus average for a population aged ~55; who are predominantly women; deficit of ≥33% versus average for a population aged ~40); HS3, “severe CPS impairment”: SDMT ≤31 (indicating deficit of ≥44% versus average for a population aged ~55 who are predominantly women; deficit of ≥47% versus average for a population aged ~40). Abbreviations: CPS, cognitive processing speed; n, sample size; VAS, visual analog scale.

Discussion

This study has elicited utility values for three health states, defined according to level of CPS impairment, incorporating information on the key symptoms and functional impairments associated with this condition, and according to the preferences of the UK general population. Mean health state utility values (± SD) were 0.77 ± 0.24 in mild, 0.57 ± 0.26 in moderate, and 0.34 ± 0.28 in severe CPS impairment, indicating that, in the perception of the UK public, individuals with CPS impairment face substantial impairment to their HRQoL.

Many HTA bodies express a preference for health state utility values derived through applying general public value sets to results of generic instruments completed by patients, such as the EQ-5D or HUI.^12–18 However, health state utility values collected by alternative methods may be accepted if sufficiently justified and documented. Taken alongside the relatively low sensitivity of the EQ-5D and EDSS measures across domains of cognitive impairment,^6,9 these results suggest that the values elicited in the current study, for states defined using the sensitive and validated SDMT measure of cognitive speed and efficiency, may form a valuable resource for future submissions.

The health state utility values elicited in this research may be compared to those that have previously been used in economic modeling. In a 2022 cost-effectiveness analysis on disease-modifying therapies in MS from the UK perspective, utility values for health states varying by physical disability were derived from clinical research and the published literature.²⁸ Siponimod, the disease-modifying therapy evaluated in this study, also benefitted cognition, as measured by the SDMT, in a Phase 3 clinical trial.²⁹ Health states with comparable utility to mild, moderate, and severe CPS impairment states from the current study (0.77, 0.57, and 0.34) were EDSS 1 (“no disability, minimal signs in one functional system”; utility 0.754), EDSS 5 (“disability severe enough to impair full daily activities … able to walk without aid or rest for 200m”; utility 0.595), and EDSS 7 (“unable to walk beyond approximately 5m even with aid … essentially restricted to wheelchair … up and about in wheelchair some 12 hours a day”; utility 0.370).^28,30 Therefore, the disutility of severe cognitive impairment is likely to be comparable to that of severe physical impairment, and should be included, with measures of physical impairment, in economic modeling of MS. In addition, these values may be applicable to similar economic modeling activities in other conditions where CPS impairment is seen.^26,31

The extensive qualitative findings gathered as part of this research provided a basis for the health state descriptions that were created and evaluated in this study. To the best of the investigators’ knowledge, this study provides the first estimate of public perceptions of the HRQoL experienced in health states of cognitive slowing, and therefore may form a useful resource for further research.

Certain limitations are inherent to the valuation method used here, such as the public’s inherently incomplete knowledge of specific health states.¹¹ However, limitations have also been identified when eliciting health state utility values from patients, whose adaptation, over time, to their own condition may lead to unrealistically high reported HRQoL (from the perspective of the general public).¹¹

Initial collection of qualitative findings in two different countries could be considered a limitation of this study. However, as the SDMT is not routinely used in the UK, this approach was necessary in order to ensure a sufficient understanding of CPS impairment and its measurement using SDMT among the development sample of HCPs and to ensure sufficient levels of CPS impairment by SDMT among the development sample of patients (and care recipients of caregivers). Developed health states were also subsequently validated for the UK context through further qualitative interviews.

The sample size of 120 members of the UK general population for utility elicitation interviews also follows existing guidance which suggests that a sample size of 100 is required to achieve a representative sample.³² In addition, valuation was conducted in one-to-one (web-based) interviews, to preserve response quality.

Several protocols are available for TTO valuation of health state descriptions and are associated with varied strengths and weaknesses.²¹ The MVH protocol was selected for use here due to its previous use in generating the EQ-5D value set commonly used in the UK.²¹ Additionally, as utility elicitation was conducted in the UK, these values are likely most useful for HTA in this region. However, the use of the internationally established MVH protocol for valuation may increase the likelihood of acceptability of these values in other regions.

Of note, as the health states created as part of this study did not include any mention of MS or associated physical disability, these health state utility values should not be taken in isolation to represent the true utility of individuals living with MS. However, this was by design, and these estimates should be considered alongside existing values for physical disability in MS and other conditions where CPS impairment is present (eg, Huntington’s disease, traumatic brain injury, dementia).^26,31 Future economic modeling activities in MS and other conditions where CPS impairment is seen should therefore carefully consider how the HRQoL impact of both cognitive and physical disability can be appropriately incorporated.

Conclusion

This study has elicited utility values for three health states of varying CPS impairment (as defined by the SDMT cognitive performance measure), according to the perceptions of the UK public, using a method that will allow these values to be used for economic modeling and HTA purposes alongside existing HRQoL estimates for physical disability. Mean health state utility values (± SD) were 0.77 ± 0.24 in mild, 0.57 ± 0.26 in moderate, and 0.34 ± 0.28 in severe CPS impairment. Of note, these estimates were generated using descriptions that are not specific to MS, and therefore may be applicable to studies of other conditions where CPS impairment is present. Results indicate that, in the perception of the UK public, CPS impairment is associated with a substantial reduction in health state utility, and therefore should be considered alongside physical disability when estimating the true impact of disease on patient functional status and HRQoL.