Elicitation of Health State Utility Values in Retinitis Pigmentosa by Time Trade-off in the United Kingdom

Introduction

Retinitis pigmentosa (RP) is an inherited retinal pathology associated with “night blindness” and progressive loss of peripheral vision in some cases leading to complete blindness, with consequent impact on patients’ health-related quality of life (HRQoL).¹ Key domains affected by RP include mobility, emotional wellbeing, social functioning, and daily activities relating to vision, especially in dark and/or crowded areas.² Few treatment options are currently available to prevent or reverse the progression of RP;³ however, targeted gene and cell-based therapies are being explored for the treatment of this condition.^4–6

While the visual and functional impairments associated with RP have been shown to lead to reduced HRQoL for patients,⁷ quantitative data on this aspect are difficult to collect by observational or clinical research due to the relatively low prevalence of this condition (with estimates ranging from 11.09 to 26.43 per 100,000 worldwide, dependent on region and study).⁸ Quantitative data in the form of health state utility values are required for activities such as modelling the burden of disease on those affected or the cost-effectiveness of new interventions, and therefore potentially contribute to new treatment options being made available to patients.⁹

As RP is a rare condition, and therefore HRQoL data are difficult to collect from patients, a study was conducted by Lloyd et al to quantify the utility of health states in RPE65-mediated retinal disease through measurement of health states by six retinal specialists using the EQ-5D-5L and HUI3 generic HRQoL measures.¹⁰ However, health technology assessment (HTA) bodies, such as the National Institute for Health and Care Excellence (NICE), typically prefer measurement of health by patients (rather than clinicians), followed by valuation by the general public.^11–17 Therefore, one potential approach within this preferred framework is the description of health state “vignettes” by patients, followed by valuation by the general public using a method such as time trade-off (TTO).^11–17

The current study therefore aimed to generate utility values for health states of varying levels of functional vision in RP, using TTO elicitation with members of the general public in the UK, in order to quantify the burden of this condition and support economic evaluations for novel treatment options in this area.

Materials and Methods

This direct utility elicitation study was conducted between October 2020 and August 2021, and consisted of two phases: a qualitative stage comprising development interviews with patients and confirmation of health state descriptions with healthcare practitioners (HCPs), and a quantitative stage where utility values were elicited with members of the general public in the UK. Ethics exemption for this non-interventional study was sought from Ethical and Independent Review Services (Missouri, United States), and was granted on 23^rd September 2020 (study identifier 20,155–01), as per Code of Federal Regulations §46.104(d)² addressing research comprising only educational tests, surveys, interviews and/or observation, where participants remain anonymous. All study participants provided informed consent prior to their involvement.

Health State Development and Confirmation

Health states were defined according to standard clinical measures of visual ability: visual acuity (VA) and visual field (VF). VA/VF categories were used to define states according to American Medical Association (AMA) guidelines on visual disability, and were consistent with the prior utility elicitation study by Lloyd et al in 2019,¹⁰ namely:

“Moderate impairment”, defined as VA better than 20/200 or VF radius of greater than 10°.

“Severe impairment”, defined as VA from 20/200 to 20/500 or VF radius of 6° to 10°.

“Profound impairment”, defined as VA from 20/500 to 20/1250 or VF radius of 2° to 6°.

“Counting fingers”, defined as VA from 20/1250 to 20/20,000 or VF radius of less than 2°.

“Hand motion” to ‘no light perception’, defined as VA worse than 20/20,000.

These VA/VF categories (based on AMA guidelines and research by Lloyd et al) were then used to create new written descriptions for each health state, incorporating new insights from qualitative interviews conducted as part of this research. Draft health state descriptions were developed following interviews with patients with RP in the UK (n=5), conducted between October 2020 and February 2021.

These interviews focused on the functional impairment patients with RP experience, in terms of domains such as social life, family life, education, or employment, both during the daytime and night-time. Patients with clinically confirmed RP were recruited and interviewed for each of the predefined VA/VF categories, and patients living with more-advanced RP were encouraged to discuss the progression of their condition through health states that they had experienced previously. Interviewed patients were adults (mean age 45.4 years, median 40 years), who had been diagnosed an average of 12.0 years prior (median 10 years). Discussion material for these patient interviews was developed using existing qualitative findings from previous interviews with patients with RP (which had been conducted prior to this research).⁷

Once draft health state descriptions were developed, further qualitative interviews were conducted with HCPs with specific experience of managing patients with RP in the UK (n=2), in April 2021. HCPs confirmed the overall relevance of each health state description to the experience of patients living with each level of visual impairment. Recruited HCPs included one ophthalmologist specializing in inherited retinal diseases and one low-vision rehabilitation specialist.

Confirmed health state descriptions (see Supplementary Material Table 1) were then put forward for use in pilot and then main-phase utility elicitation interviews. Relative to those used by Lloyd et al, the health state descriptions used in the current study had an increased focus on the functional impairment experienced by individuals with RP.

Health State Utility Elicitation

Quantitative interviews with members of the general public in the UK were conducted in order to value health states according to the TTO methodology outlined below, in July and August 2021.

A representative sample of the UK general population was recruited for interview using random sampling, according to predefined quotas based on UK census data. As the prevalence of low vision or blindness in the UK was estimated to be 3.0%,¹⁸ the proportion of participants who had low vision or blindness, or who were primary caregiver for a person with these conditions, was capped at 6.0%, to avoid possible bias at the valuation step due to prior personal experience of visual impairment.

Firstly, pilot interviews (n=10) were conducted to validate the methodology, followed by the main phase (n=100). As no substantial changes were made to the health state descriptions following the pilot interviews, data were included in the final calculation of utility values (therefore giving a total sample size of n=110).

A TTO process based on the established Measurement and Valuation of Health (MVH) protocol was used to elicit utilities.¹⁹ Due to the ongoing COVID-19 pandemic at the time of fieldwork, all interviews were web-enabled and conducted 1:1 by a trained TTO moderator, who used visual aids to improve respondents’ comprehension of the valuation tasks, as is standard within the MVH and related valuation protocols.¹⁹ To avoid biasing participants’ responses, health state descriptions were not labelled in any manner that suggested differences in severity or importance and were presented to respondents in a random order. All respondents received compensation according to fair market value.

Prior to TTO valuation, visual analogue scale (VAS) scoring of the same health state descriptions was used as a “warm-up” exercise to allow participants to become comfortable with examining and valuing health state descriptions, and to collect additional data points for comparison and validation of TTO results. However, all utility values were derived by the TTO method. Prior to VAS valuation of each health state description, each participant was asked to value their own health on that day, using the same VAS scale.

The TTO method was then used to elicit health state utility values.¹⁹ This method involves asking the participant to select between a fixed period of life in the health state (followed by death), and a variable shorter period of life in full health (followed by death).¹⁹ Each valuation task involved iteratively varying the period of life in full health that is being compared, until a point of indifference was reached: the period of life in full health that is considered equivalent to the fixed period of life in the health state (in this case, ten years).

The standard ten-year time horizon specified by the MVH protocol was applied, and the iteration procedure included a mixture of “ping-pong”, bisection, and titration iteration steps, in order to explore all possible points of indifference between zero and ten years.¹⁹ “Ping-pong” iteration involves moving from one maximum possible value on the scale to the other (eg from 10 years to 0 years in full health); bisection involves moving to the halfway point of the remaining possible values on the scale (eg from 0 years to 5 years in full health); titration involves moving incrementally up or down the scale (eg from 5 years to 4 years in full health).¹⁹

The second step in the iteration procedure of the standard MVH protocol (which determines whether a health state will be valued above or below 0.0) was not undertaken in the current study. Therefore, the final result of each TTO valuation exercise was a utility value between 0.0 (death) and 1.0 (full health).

Utility Data Analysis

Utility values were calculated from TTO results, giving values between 0.0 and 1.0. Quantitative results (including demographic characteristics, TTO utility values, and VAS scores) were analyzed descriptively, and are here presented as means, medians, standard deviations, and 95% confidence intervals (where appropriate).

Prior to further statistical analysis, the distribution of individual TTO utility values within health states was examined by Shapiro–Wilk test, to determine whether data were normally distributed. A significance threshold of p<0.05 was applied when determining the distribution, and the results were used to determine the type of test that should be used to test for significant differences between states.

Significance of differences in TTO utility values between health states were therefore tested using the Kruskal–Wallis and pairwise Wilcoxon tests (non-parametric tests for skewed, non-normal data). A threshold of p<0.05 was again applied when concluding significance of differences.

Results

Utility Valuation Interview Respondent Characteristics

Characteristics of the n=110 interviewed members of the public are presented in Table 1. The interview sample was generally representative of the wider UK population, in terms of age, gender, region, and other characteristics; the UK general population quotas used for recruitment are also presented in Table 1 for comparison.

Table 1 Demographic Characteristics of Participants in TTO Interviews (n=110) and UK public20–25

TTO Utility Results

Utility values by TTO for each health state are presented in Figure 1 and Table 2. Mean values followed the logical and expected order, with increasing visual impairment leading to decreased utility in each case. Mean values varied between 0.78 in “moderate impairment”, and 0.33 in “hand motion” to “no light perception”.

Table 2 TTO Utility Value per Health State

Figure 1 TTO utility value per health state (n=110 responses to each). “Moderate impairment” health state defined as VA better than 20/200 or VF radius of greater than 10°. “Severe impairment” health state defined as VA from 20/200 to 20/500 or VF radius of 6° to 10°. “Profound impairment” health state defined as VA from 20/500 to 20/1250 or VF radius of 2° to 6°. “Counting fingers” health state defined as VA from 20/1250 to 20/20,000 or VF radius of less than 2°. “Hand motion” to “no light perception” health state defined as VA worse than 20/20,000. Abbreviations: n, sample size; TTO, time trade-off; VA, visual acuity; VF, visual field.

The Shapiro–Wilk test was used to determine whether the distribution of utility values in each health state was skewed. At a threshold of p<0.05, the utility values in all health states were skewed.

As TTO values within health states were not normally distributed, non-parametric tests were employed to examine the significance of differences between health states. The Kruskal–Wallis test showed a significant difference in utility values between health states (p<0.05), and a pairwise Wilcoxon test was conducted to determine significant differences between individual states. All pairs of health states showed significant differences in utility values (p<0.05), with the exception of “profound impairment” and “counting fingers” (p=0.05).

Sensitivity and Subgroup Analyses of TTO Utility Results

Two sensitivity analyses were conducted on TTO utility data by removing all values that were two or more standard deviations away from each health state mean, and by removing all extreme values (0.0 and 1.0). These analyses were conducted to show the effect (or lack of effect) of extreme individual answers on mean average health state utility values. Additionally, subgroup analyses were conducted to determine any effect of demographic characteristics on valuation of health states.

With removal of values that were two or more standard deviations away from each health state mean, mean health state utility values continued to be logically ordered (see Supplementary Material Figure 1). The Kruskal–Wallis test showed a significant difference in utility values between health states in this subgroup analysis (p<0.05); the pairwise Wilcoxon test also showed significant differences in utility values between all pairs of health states (p<0.05).

With removal of extreme values (0.0 and 1.0), mean health state utility values were again logically ordered (see Supplementary Material Figure 2). The Kruskal–Wallis test showed a significant difference in utility values between health states in this sensitivity analysis (p<0.05). The pairwise Wilcoxon test also showed significant differences in utility values between all pairs of health states (p<0.05), with the exception of “profound impairment” and “counting fingers” (p=0.069), and “counting fingers” and “hand motion” to “no light perception” (p=0.074). Although described as removing extreme values, this sensitivity analysis will impact the health states with the highest and lowest utility values most, as they have average values closest to the extremes and are more likely to have 0.0 and 1.0 values; therefore, this analysis should be interpreted with caution.

When analyzing the possible effect of patient demographics on TTO valuation, no significant difference was observed within any health state, in terms of results given by subgroups defined by age (younger/middle/older tertiles), gender (male/female), or employment (characteristics as reported in Table 1) (all p>0.10; see Supplementary Material Table 2).

VAS Score Results

VAS scores collected for each health state are presented in Figure 2 and Table 3. As seen in the TTO utility data, mean scores followed the logical and expected order, with increasing visual impairment leading to decreased scores in each case. Mean values varied between 47.95 in “moderate impairment”, and 17.22 in “hand motion” to “no light perception”. Participants’ mean VAS rating for their own health at the time of interview was 73.28, suggesting that the participant sample was broadly healthy.

Table 3 VAS Score per Health State

Figure 2 VAS score per health state (n=110 responses to each). “Moderate impairment” health state defined as VA better than 20/200 or VF radius of greater than 10°. “Severe impairment” health state defined as VA from 20/200 to 20/500 or VF radius of 6° to 10°. “Profound impairment” health state defined as VA from 20/500 to 20/1,250 or VF radius of 2° to 6°. “Counting fingers” health state defined as VA from 20/1,250 to 20/20,000 or VF radius of less than 2°. “Hand motion” to ‘no light perception’ health state defined as VA worse than 20/20,000. Abbreviations: n, sample size; VAS, visual analogue scale.

Discussion

This study estimated utility values for five health states of varying visual impairment in RP, incorporating information on the key symptoms and functional impairments associated with this condition and using preference data gathered from the UK general population. Health states were defined according to AMA guidelines, and were consistent with previous research in this area.¹⁰ Mean utility values varied from 0.78 in “moderate impairment” to 0.33 in the “hand motion” to “no light perception” health state (the most severe), showing that the UK general population perceive that individuals living with RP face substantial impairment to their HRQoL as a result of this condition. This can be compared to the average utility value reported by the wider UK population for their own health (for whom a normative utility value of 0.86 has been measured, by EQ-5D rather than by direct TTO).²⁶ In the context of previous TTO-based research with the UK general population, the mean utility of the “hand motion” to “no light perception” health state can be compared to that of severe multiple sclerosis (0.27),²⁷ or acute myeloid leukemia treated with chemotherapy (0.36).²⁸

Many key HTA bodies, such as NICE, express a preference for health state utility values derived through generic questionnaires completed by patients, such as the EQ-5D or HUI.^11–16 However, utility values collected by alternative methods are typically accepted if reference-case values are not available, and if the alternative method is sufficiently justified and documented.¹⁷ Therefore, the values elicited here may form a valuable resource for future HTA submissions.

Utility values elicited in the current study may be compared to those from previous research in RPE65-mediated retinal disease (presented in Table 4 and discussed below).¹⁰ Other existing utility values in the area of visual impairment are much less comparable, due to major differences in methodology (such as direct valuation by patients rather than members of the public) or in the conditions that were examined (such as macular degeneration or diabetic retinopathy).^29–32

Table 4 Comparison of Health State Utility Values from the Current Study and Existing literature10

In the study by Lloyd et al in 2019, which involved EQ-5D and HUI valuation of vignettes by clinicians, mean utility values were distributed more widely than in the current study, and were substantially lower in the most severe health state: 0.709 to 0.152 (by EQ-5D), or 0.519 to −0.039 (versus 0.78 to 0.35 in the current study).¹⁰ Data from the current study (each above 0.0, and elicited using health state descriptions that focused on functional impairment)³³ may provide a useful alternative source of utility values for economic modelling.

Several methodological factors may have influenced the differences in findings between Lloyd et al and the current study. Notably, description of health states by clinicians and valuation of health states by the public (using the EQ-5D and HUI questionnaires and existing value sets), as opposed to description of health states by patients and prospective direct valuation by the public (using detailed written health states and TTO), likely contributed to differences in results.¹⁰

In particular, the current study did not allow values below 0.0 to be applied to any health state. While this may be considered beneficial if values below 0.0 are considered unrealistic,³³ this approach introduces the potential for a “floor effect” (where the “true” value is too low to be reliably estimated by the system of measurement being used). A small proportion of respondents in the current study (16%) gave a value of 0.0 to the health state describing the greatest level of impairment (“hand motion” to “no light perception”); therefore, it could be hypothesized that a proportion of these respondents may have answered with a value below 0.0, if this were possible – therefore, the mean utility value for this health state may be a conservative estimate.

Existing observational research into risks of stress and suicidality among individuals with RP may also support the adoption of lower utility values for more severe cases of RP, where individuals may have progressed to a state of total blindness with no visual ability.^34,35

Although valuation of health states by the general public is preferred by many HTA bodies,^11,12,36 certain limitations are inherent to this method, such as the public’s incomplete knowledge of specific health states and the ways in which patients may adapt to their condition over time (thereby improving their functioning and perceived HRQoL).⁹ However, limitations have also been identified when eliciting utility with patients, whose adaptation over time to their own condition may lead to unrealistically high utility (from the perspective of the general public).⁹

Notably, in both the current study and the study conducted by Lloyd et al, the definition of the health state with greatest visual impairment encompasses a relatively wide range of impairment levels (from ability to perceive a moving hand, to no ability to perceive any stimulus). For example, the results of previous economic modelling research in retinal pathology have been sensitive to changes in utility inputs at the lowest end of visual ability: scenarios were explored where utility was considered to decrease linearly or non-linearly towards total blindness, and this change was highly influential on the final estimate of the research.³⁷ Hence, the results of the current study should be interpreted with caution, especially in situations where multiple levels of extremely impaired vision are modelled.

The sample size of n=5 patient interviews in this study is consistent with that of previous utility elicitation research in retinal pathology by Lloyd et al, where five patient and caregiver testimonials were used as discussion material for initial development of health state descriptions.¹⁰ In addition, the extensive qualitative findings gathered in previous research provided a firm basis for the health state descriptions that were created and valued in the current study.

The sample size of n=110 members of the UK general population for utility elicitation interviews follows existing guidance, which suggests that a sample size of 100 is required to achieve a representative sample.³⁸ This sample size is similar to, or larger than, those of comparable primary research studies: similar to the n=108 members of the general public interviewed by Czoski-Murray et al in age-related macular degeneration,²⁹ and larger than the n=6 specialist clinicians interviewed by Lloyd et al in RPE65-mediated retinal pathology.¹⁰ In addition, “own health” VAS values are available for the current study population, suggesting that the participant sample was broadly healthy, and therefore that the current health of participants was unlikely to influence their valuation of the health states.³⁹

Conclusions

In conclusion, UK general population valuation of health state descriptions suggests that individuals living with RP face substantial impairment to their HRQoL as a result of this condition. Utility values for RP health states have been elicited here using a method and sample that may allow these values to be used for economic modelling and HTA purposes. Such activities may include estimating the value of novel therapy options in this area, which may reduce progression of visual impairment in RP.