Development and Content Validation of Novel Patient-Reported Outcome Measures to Assess Disease Severity and Change in Patients with Erythropoietic Protoporphyria: The EPP Impact Questionnaire (EPIQ)

Introduction

Erythropoietic protoporphyria (EPP) and X-linked protoporphyria (XLP) are rare, inherited inborn errors of heme metabolism that present with childhood‐onset, severe, painful phototoxicity.^1,2 Collectively these are both referred to as EPP. Additional complications can include anemia, cholelithiasis, liver dysfunction, and psychosocial issues.^1,3,4 The diagnosis of EPP is often delayed, with one study reporting 13 years as the mean time between initial symptoms and diagnosis.⁵

Symptoms after sun exposure typically include tingling, burning, and/or itching that may progress rapidly to severe pain, erythema, and swelling, usually on the face and dorsum of the hands, and any exposed areas.^1,2 In some cases, symptoms may also develop due to artificial light.⁶ Prevalence estimates based on individuals entering the health care system range from 1 in 75,000 to 1 in 200,000 individuals,⁷ however genetic data from the UK Biobank estimates pathogenic variants causing EPP are found in approximately 1 in 25,000 individuals, suggesting EPP is underdiagnosed.⁸ Although sun exposure tolerance and time to symptom resolution are highly variable, many individuals can only tolerate less than 30 minutes of sun exposure before onset of prodromal symptoms. Pain can be excruciating, and many individuals do not experience adequate pain relief from analgesics, including opioids.¹ In addition, severe phototoxic reactions can last for several days^1–4,7.

There is only one approved treatment for adults with EPP in the US, European Union, and Australia, afamelanotide, a subcutaneously administered α-melanocyte stimulating hormone analogue.⁹ Nevertheless, the mainstay of management continues to be sun protection and avoidance of sunlight for individuals in which afamelanotide is not an option (pediatric patients, access issues for adult patients, etc.).

Because of the need to avoid sunlight as much as possible for fear of severe phototoxic reactions, EPP can have a significant impact on daily life^4,10–13 and health-related quality of life (HRQoL). This has been demonstrated in several studies using generic and EPP-specific patient-reported outcome (PRO) measures.^4,13–16 However, the content of these currently available PRO measures is limited, and they do not capture all the relevant concepts important to individuals with EPP. For instance, generic PRO measures that have been used in studies of dermatologic conditions typically focus on significant visible symptoms, which may not be relevant to EPP. Although the Dermatology Life Quality Index (DLQI), a common PRO instrument in many dermatology studies, was used in several studies of EPP,^4,12,17,18 it was not sensitive to changes in response to treatment.⁹ Previous studies have also utilized the PROMIS-57, Hospital Anxiety and Depression Scale, and Illness Perception Questionnaire in EPP patients. Of these measures, the PROMIS-57 had the highest correlation with clinical features of EPP, specifically, the physical function, pain interference, and satisfaction with social roles domains.¹⁹ However, a recent trial of dersimelagon, an oral melanocortin 1 receptor agonist, showed it was not sensitive to changes in response to treatment, although scores in the physical function domain and pain intensity did appear to improve.²⁰

One EPP-specific measure referred to as the “EPP-QoL” was specifically developed for use in clinical trials with afamelanotide to assess long-term effects of treatment.^15,16,21,22 However, the “EPP-QoL” was developed without patient input as is recommended by current standards and regulatory guidance,^23,24 and it lacks a robust assessment of well-being.²⁵ Validation studies of the “EPP-QoL” showed that the well-being domain had poor psychometric properties and was removed from the final questionnaire, resulting in only a unidimensional “EPP Symptom” score.²⁵ It is also a proprietary tool, which limits its use. Furthermore, although the “EPP-QoL” generally showed EPP negatively impacts daily activities, it was not found to be very specific to clinical features of EPP,¹⁹ and does not specifically assess severe pain, avoidance of sunlight, or how these behaviors impact daily life. Therefore, new PRO measures developed with direct patient input are still required for this population.

Additional PROs have recently been proposed as useful endpoints in clinical trials, specifically measuring time-to-prodrome (TTP), and the amount of time a patient can be in direct sunlight before experiencing prodromal, or early warning symptoms.²⁶ This is useful in quantifying sun exposure and for not putting the patient at significant risk for a severe phototoxic reaction. However, these outcomes do not take into account symptom severity, and the concept of prodromal symptoms in EPP requires further study.

Having a content valid, reliable, and responsive measure of these PROs for use in this population is important, not only for accurately characterizing EPP patients’ well‐being, but also for assessing the efficacy of potential new therapies. Therefore, the goal of this current research was to conduct in-depth, qualitative interviews with adults and adolescents with EPP in order to develop a novel, comprehensive, and content valid PRO measure.

Materials and Methods

Data collection for this qualitative interview study was conducted from March 2022 to June 2022. Approval for the study was provided by an independent Institutional Review Board, WCG IRB.

Recruitment

Potential participants were referred from the United Porphyrias Association (https://www.porphyria.org). Once their interest was confirmed, potential participants were screened for eligibility. Participants aged 12 to 17 signed the Assent Form, while their parent signed the Parental Permission Form. Adult participants signed an Informed Consent Form. Participants’ assent or informed consent included publication of anonymized responses. The study was approved by an institutional review board (WCG IRB) and conducted in accordance with the Declaration of Helsinki. The target sample size was 20 participants to attain saturation.

Participants

Inclusion criteria included participants >12 years of age, having a written confirmed diagnosis of EPP (via a physician’s letter, genetic testing results, or chart note), and living in the US or Canada. If participants were currently receiving afamelanotide, they must have initiated treatment within the last three months to ensure they still had sufficient recall of their pre-treatment symptoms. Participants were excluded if they were currently enrolled in a clinical trial or had a medical or psychiatric condition (or treatment for condition) that resulted in a cognitive or other (visual, hearing) impairment that could potentially interfere with participating in this current study. Other methods of sun protection (ie, beta-carotene, sunscreens, etc.) were not considered exclusionary as they are not considered effective treatments for EPP. Participants completed a background questionnaire containing demographic and clinical information prior to their interview. After completing the questionnaire and interview, participants received a $150 Amazon gift card as compensation for their time.

Interviews

Interviews were conducted with individuals with EPP, as well as five clinical experts that treat EPP, who were asked to provide their opinions on concepts relevant to individuals with EPP. Input from clinical experts were primarily used in developing a Sunlight Exposure Diary described elsewhere²⁷ and used to inform the development of the interview guide in this current study. Interviews were done using semi-structured interview guides (one for individuals with EPP and one for clinical experts), developed specifically for this research. The first 5 patient interviews included open-ended concept elicitation questions only. Results from those interviews were used to inform the development of draft questions for the EPP Impact Questionnaire (EPIQ) that was debriefed during the subsequent interviews that used a combined concept elicitation portion (CE) and cognitive debriefing (CD) approach. Due to time constraints, not every question was asked of all participants. The purpose of the CD portion of the interview was to determine whether the PRO measure content, clarity, and relevance to patients was adequate. Due to the COVID-19 pandemic, to ensure the safety of participants, all interviews were conducted over Zoom by an experienced health service researcher. Zoom allowed the interviewer and the participant to see one another through the video camera. In addition, the interviewer was able to share her screen to display the items, so that participants could complete the questionnaires in real-time, while answering questions about each item. All interviews were recorded and transcribed, and reviewed for accuracy for analysis purposes.

Analysis

A coding dictionary was developed to organize and categorize concepts of interest from the interview transcripts and included descriptions and examples for each code to ensure consistency across coders. Each transcript was coded by one coder, and then reviewed, summarized, and analyzed by a second coder. Saturation tables were developed to categorize each symptom mentioned by each participant. Interviews were conducted in waves so that, as necessary, revisions could be made iteratively to the newly developed PRO measures based on results from the interviews.

Results

Participant Characteristics

Twenty-three participants were enrolled, including 17 adults and six adolescents. There were approximately equal numbers of males and females, and the majority of participants were White (94%). Most adults had a college degree or higher (71%) and worked full-time for pay (71%). The average age of the adults was 40 years. Most were not on treatment currently, although 2 adults (9% of all participants) were currently receiving afamelanotide. Six adults (26% of all participants) had received afamelanotide previously. Co-morbid conditions were fairly common, with liver disease (29%), anemia (24%), and gallstones (29%) most common. The average age of adolescents was 14 years of age (median: 13.5; range: 12–15 years of age). Most adolescents (67%) were currently taking beta-carotene. Additional details are provided in Table 1.

Table 1 Participant Characteristics

Concept Elicitation

Concept elicitation revealed the following results, and are also summarized in Table 2.

Table 2 Illustrative Quotes from Concept Elicitation Interviews

Living with EPP

Participants were asked what it is like to live with EPP. Several participants (n=6) said that it is something always on their mind, and they constantly need to be aware of their sun exposure. Some adult participants (n=4) referenced how difficult it was as a child (Table 2).

Time in Direct Sunlight

Participants were asked how much time they spend outside in direct sunlight each day. Estimates varied, and responses ranged from 1 minute to two to three hours. Forty-one percent of participants (n=7 of 17) said symptoms occur after less than 10 minutes of sun exposure. Forty-seven percent of participants (n=8) said symptoms occur within 10 and 60 minutes of sun exposure.

Symptoms

Participants were asked to describe what symptoms they have ever experienced as a result of EPP. The most common symptoms mentioned spontaneously included burning (91%), pain (74%), swelling (70%), tingling (65%), itching (57%), sensitivity to touch (43%), warmth/heat sensation (39%), blisters (35%), redness/discoloration (30%), and sensitivity to hot/cold (30%) (Table 3). While there was considerable overlap between burning and pain, not everyone that reported burning reported pain, and not everyone that reported pain reported burning. Specifically four participants that reported burning did not report pain, and one individual that reported pain did not report burning. Other concepts that were discussed included symptom causes, frequency, severity, locations, seasonality, time of day, what alleviates symptoms, most bothersome symptoms, and prodrome, which are summarized below and in Table 2.

Table 3 Saturation Grid of Spontaneously Reported Symptoms

Exposures Causing Symptoms

Sixty-three percent of participants (n=5 of 8) said they experience symptoms from both sunlight and indirect sunlight, (through windows and reflections), as well as artificial light (ie, from computer screens, indoor lighting), while 38% (n=3) only experience symptoms from sunlight.

Frequency and Severity of Symptoms

Participants were asked if their symptoms vary from day-to-day or whether they are fairly consistent. Sixty-three percent of participants (n=10 of 16) said their symptoms are fairly consistent, while 31% (n=5 of 16) said the symptoms vary. Nineteen percent (n=3 of 16) said that over time as they have gotten older, there was a more rapid onset of symptoms, which were also more intense. Specifically, they have less leeway when outdoors before the onset of symptoms. Two participants mentioned that they only have discoloration, blisters, or burst blood vessels when they are having a severe reaction, indicating that the frequency of these symptoms varies depending on the severity of the reaction. All participants who were asked (n=20) said the severity of their symptoms depends on duration in sunlight or exposure to indirect light.

What Helps with Symptoms

Participants were asked if there are things that help their symptoms or when their symptoms are not as bothersome. Forty-three percent of participants (n=6 of 14) said wearing sunscreen and protective clothing minimized symptoms and 43% (n=6 of 14) said using cold wash cloths/ice packs or taking a cold bath helped alleviate symptoms. The use of a fan, going indoors, and taking Advil or ibuprofen also was helpful (Supplementary Figure 1).

Most Bothersome Symptom

Participants were asked what they felt their most bothersome symptom was, regardless of whether it was a prodromal or full reaction symptom. Sixty-nine percent of participants (n=11 of 16) reported that burning was the most bothersome symptom, followed by itching (43%), pain (38%), tingling (19%), sensitivity to hot and cold (19%), and swelling (19%). Burning is often typically experienced as an early warning symptom. When asked about the worst symptom of a reaction, 80% (n=12 of 15) reported pain.

Early Warning Symptoms (Prodrome)

All participants asked (n=21) reported experiencing early warning symptoms. The most common early warning symptoms reported were tingling (75%), itching (25%), burning (25%), sensitivity to touch (20%), sensitivity to hot/cold (20%), and a warmth sensation (20%) (Table 3).

Participants were asked how long they could be in sunlight before they started experiencing early warning symptoms. Responses varied and ranged from 1 minute to two to three hours, with forty-one percent of participants (n=7 of 17) stating symptoms occur after less than 10 minutes of sun exposure. Participants were also asked how long they would avoid the sun after experiencing early warning symptoms. Responses varied and ranged from 20 minutes to several days or until their symptoms went away.

Symptom Impacts

Concepts discussed included daily activities, physical functioning, work impacts, social impacts, and emotional impacts. These are summarized below and in Table 2.

Daily Impacts

The most common impacts reported included not being able to participate in outdoor recreational activities (eg, go to the beach/pool) (n=6 of 14), being unable to sleep primarily due to pain (n=5), not being able to leave the house (n=4), and impacts related to driving (n=3). Several other daily impacts were mentioned by individual participants such as “hanging out with friends” and not being able to do housework. Of those asked, 80% (n=4 of 5) said their physical functioning is not impacted due to EPP. Of those asked, 80% (n=4 of 5) stated that EPP impacts their ability to work. One participant stated that she is not impacted because she is able to work from home and not leave her house.

Four participants who were asked said having EPP impacts their social life. For example, they may not be able to attend a social event because of the location (beach, picnic in the sun) or they might need to leave early to avoid the sun. Participants (n=11) were asked how, if at all, they are impacted emotionally due to EPP. Thirty-six percent of participants (n=4 of 11) reported feeling anxious, 18% (n=2) were depressed, and 18% (n=2) felt isolated and alone (Supplementary Figure 2).

Cognitive Debriefing of New PROs

The measures were debriefed in 9 waves, with revisions made after each wave of interviews. Sample PRO items are contained in Tables 4 and 5.

Table 4 Summary of Cognitive Debriefing of New EPP PRO Draft

Table 5 Meaningful Change on PGI-S and PGI-C

EPP Impact Questionnaire (EPIQ)

The questionnaire includes a total of 10 items of which 7 assess sun exposure, specifically overall time in the sun, TTP, time to a full reaction, time to improvement of symptoms, and time to full resolution of symptoms. Assessing sun exposure includes both direct and indirect sunlight at the suggestion of the participants, with indirect exposure capturing scenarios like sun coming through a window of a car or home, reflected off of surfaces, etc. Indirect sunlight does not include being outdoors in the shade. The EPIQ also includes 3 single items that assess the impact of EPP on ability to perform activities, the impact of EPP on overall health-related quality of life (HRQL), and the comparison of HRQL to those without EPP.

Most participants were able to correctly paraphrase the questions in the EPIQ measure (86–100% of items). Further, participants were able to distinguish between pain and burning. Participants found items to be clear (75–100% of items), with some exceptions. When asked “How much time were you able to be in sunlight before having early warning symptoms?”, a few participants suggested to not use the past 7 days, but instead ask about an average/typical day (this change was implemented). Table 4 provides sample items from the questionnaire and summarizes the revisions made based on results from the CD portion of the interview.

As part of the EPIQ, Patient Global Impression of Severity (PGI-S) items assess severity of full reactions, severity of prodromal (early warning) symptoms, and severity of EPP overall. Patient Global Impression of Change (PGI-C) items assessed change in severity of full reactions, change in severity of prodromal symptoms, change in EPP overall, change in TTP, and change in time to a full reaction.

Participants were asked how much change they would consider meaningful on the PGI-S, PGI-C, and TTP questions. More than half of participants would want a rating of “I did not have” or “mild” on the PGI-S items. On the PGI-C items, approximately half of respondents needed to see a change of “a little better” and half needed to see a change of “much better”. On the time in sunlight questions, most respondents would consider a meaningful change of “more time” or “much more time” in sunlight. Most participants found the questions easy to complete (78–100%). Table 5 contains a sample of these items and summarizes the responses.

Discussion

The goal of this qualitative research study was to better understand the symptoms and daily activity impacts experienced by individuals with EPP and to use this information to confirm the content validity, relevance, and clarity of newly developed PRO measures of severity and change in EPP. A total of 23 combined concept elicitation and cognitive debriefing interviews were conducted: 17 with adults with EPP and 6 with adolescents with EPP.

The results of these interviews revealed that individuals with EPP experience numerous symptoms with significant variability and range. The most common symptoms reported were burning, pain, swelling, tingling, itching, sensitivity to touch, warmth/heat sensation, blisters, redness/discoloration, and sensitivity to hot/cold, which is consistent with previous studies.^1,3 Though burning and pain were reported as distinct symptoms by some participants, consistent with previous studies and case reports,^5,28,29 others reported “burning pain”. It is likely patients interpret these concepts on a continuum when describing their symptoms. Most participants report that pain from a full reaction is preceded by tingling/itching/burning sensations of varying severity. Eighty percent of participants who were asked reported that pain was the worst symptom of a full reaction.

Participants also reported being negatively impacted in terms of their ability to do daily activities, social functioning, and emotional functioning, again consistent with results from previous studies.^4,10,19 However, more specific details were obtained in this current study, as well as the view of several adolescents which is novel. Many participants also reported impacts in terms of their ability to work for pay and be productive at their job. Understanding these impacts is critical to ensuring appropriate existing scales are used in this patient population, and that newly created PRO measures encompass domains that are important to patients and not included in existing scales.

Participants were asked to review and complete the newly developed PRO measures assessing full reactions and ability to do activities (EPP Impact Questionnaire). Finally, EPP severity and change in severity of early warning symptoms, full reactions, and EPP were developed as single-item measures (PGI-S and PGI-C). All measures were found to be comprehensive, clear, and relevant.

Our comprehensive concept elicitation resulted in a PRO measure that captures impacts that are important to patients. While some items of the EPIQ appear similar to the previously developed “EPP-QoL”, it is important to note that the phrasing of the EPIQ items were developed with direct patient input and carefully debriefed to be clear, which was not the case for the development of the “EPP-QoL”. As well, due to issues with the “EPP-QoL” well-being domain, only a “EPP Symptom” domain is recommended for use.²⁵ Therefore, the EPIQ items that capture effects on daily activities are particularly relevant. Finally, the EPIQ contains several concepts that are not included in the “EPP-QoL” such as changes in overall EPP severity and comparisons to individuals without EPP.

There were numerous strengths to this study; it included a fairly heterogeneous sample in terms of demographic characteristics such as gender, education, marital status, and household income. The demographics were also representative of the EPP population and consistent with characteristics reported in a larger observational study of individuals with EPP in the US.¹ Saturation of concepts was reached by the 11th interview for the total sample, by the 10th interview for the adult population, and by the 5th interview for the adolescent population (Table 3). The sample was also geographically diverse.

This study contained a number of limitations. The sample included both adults and adolescents; however ideally a larger sample of adolescents would have enrolled (although saturation was reached with the sample of 6 adolescents). In addition, the majority of the sample were White, non-Hispanic, and all from North America. Different results may have been obtained with a more diverse sample. It is possible that results would differ if more participants were on an effective treatment. However, our goal was to evaluate pre-treatment symptoms. The impact of other complications of EPP such as anxiety, depression, or liver disease are not specifically assessed in the EPIQ. As well, due to the number and length of the questionnaires, not all items could be cognitively debriefed. Finally, due to the length of the interview, not all interview questions could be asked of all participants.

Conclusion

EPIQ is a novel PRO measure assessing overall EPP severity and has the potential to be a well-defined and reliable endpoint measure of therapeutic response in future clinical trials. In addition, the EPIQ directly assesses patients’ prodromal and full phototoxic reaction symptoms which can be used in conjunction with a PRO measure of time-to-prodrome to capture more comprehensive information. The study demonstrates that EPIQ addresses the gaps in existing EPP measures as a result of having the patient voice drive its development ensuring the creation of a robust and comprehensive measure. Future studies may wish to assess its measurement properties, specifically internal consistency reliability, test-retest reliability, construct validity, known groups validity, and responsiveness. Important within-patient change should also be evaluated using anchor-based approaches.