Development and Content Validity Evaluation of the Condition-Agnostic Treatment Goal Evaluation Scale (TGES) for Use in Chronic Conditions

Introduction

Patient-reported outcome measures (PROMs) are tools and/or instruments used in both clinical trial and real-world settings to capture patients’ experiences with health conditions and the impact of treatments on how they feel or function. PROMs can capture patient-centric data on disease symptoms, treatment effectiveness, side effects, changes in functioning, and impact on health-related quality of life (HRQOL).¹ Patient-reported experience measures (PREMs) are used to capture patients’ experiences with healthcare systems to assess quality of care^2,3 and, collectively with PROMs, help to tailor patients’ healthcare and support shared decision-making.⁴ Multiple approaches are applied to map patient-reported outcome (PRO) responses or score changes to clinically significant outcomes, including defining thresholds for minimally important between-group or within-group differences and meaningful within-patient change at the individual level to inform responder analyses.^5–9 However, individuals with health conditions each have their own priorities and treatment goals, as well as their own perception of whether treatment-related changes in their condition are beneficial, which influences what they consider to be a meaningful change on a PROM.^10–12

Patient-focused drug development guidance from the United States (US) Food and Drug Administration (FDA) emphasizes evaluating the meaningfulness of treatment benefit alongside clinical outcome assessments used as trial endpoints.⁹ Inclusion of personalized endpoints can help to inform the extent to which an individual has experienced a benefit or change in health status that they would consider meaningful. Beyond the assessment of treatments in clinical trials, personalized goal-oriented care—a patient-centric approach that prioritizes an individual’s preferred outcomes and goals—identifies the symptoms or impacts of a condition that an individual feels affects them the most.^13,14 Methods to measure goal-associated outcomes, such as Goal Attainment Scaling (GAS), identify and measure goals that are most relevant to a particular individual and have proven useful for diseases with heterogeneous symptoms.^15,16 However, GAS maps goal attainment to prespecified levels, and existing goal-based scales are often disease-specific, lacking a universal tool for cross-therapeutic use. Therefore, there remains a need for a PROM designed to capture individual patients’ treatment goals and assess achievement of those goals with personalized evaluation of the extent to which a meaningful change has occurred.

The purpose of this study was to develop and evaluate an individualized, condition-agnostic PROM—the Treatment Goal Evaluation Scale (TGES)—that can be used in both clinical trials and routine care to identify a patient’s primary treatment goal, assess the patient’s perspective on the extent to which a change has occurred in this treatment goal, and determine whether this change was meaningful to them.

Materials and Methods

Study Design

This multicenter, qualitative study was conducted to develop and evaluate the content validity of the 2-part TGES—consisting of a single-item baseline (TGES – Baseline) and a 2-item follow-up (TGES – Follow-Up) questionnaire—through interviews conducted in a sample of patients with atopic dermatitis (AD) or migraine. As common chronic diseases that significantly impact multiple aspects of patients’ HRQOL, AD (a relapsing inflammatory skin disorder characteristic of dry skin and pruritus)¹⁷ and migraine (a type of headache disorder characterized by reoccurring headaches)¹⁸ were considered representative conditions for evaluating the TGES.^19,20 The TGES – Baseline and TGES – Follow-Up were drafted, assessed, and revised via 3 rounds of qualitative, one-on-one cognitive interviews with patients in the US with AD or migraine (Supplemental Figure S1). The TGES – Baseline was designed to elicit individualized content from each respondent at baseline to be evaluated during the course of treatment. Since each respondent generates their own personalized content when completing the baseline questionnaire, concept elicitation interviews and/or focus group discussions that are typically conducted to identify concepts for evaluation in PROMs were not necessary.

Patient Population and Recruitment

Participants were eligible for inclusion if they were aged ≥18 years, had been clinically diagnosed with either AD or migraine for at least 3 months, and had started or changed prescription therapy for their condition within the last 6 months. Additionally, participants were required to be able to read and understand English and consent to participate in a 60-minute web-based interview. A convenience sample of participants was recruited from 2 dermatology sites (Portland, Oregon; Louisville, Kentucky) and 2 neurology sites (Riverwoods, Illinois; Long Beach, California). Efforts were made to recruit participants across race, ethnicity, and gender identities to ensure the applicability of the TGES to the broader, diverse population. Patients who had recently started or changed prescription therapy for AD or migraine were identified through each clinic’s patient database or medical records. Clinic staff facilitated recruitment and reported AD and migraine diagnosis dates and overall disease severity information. Participants were then contacted by telephone to provide informed consent, including consent for the publication of anonymized responses and consent to participate in cognitive interviews. This study complies with the Declaration of Helsinki.

Development of the TGES Measure

Instructions, items, and response options were developed for the draft TGES – Baseline and TGES – Follow-Up (Figure 1). The TGES – Baseline is a single, open-ended item that identifies the primary treatment goal that is most important to a patient when changing treatment or starting a new treatment for an ongoing condition (What is the one most important benefit you would like to see from this treatment?). The TGES – Follow-Up is a 2-item questionnaire that assesses patients’ perspectives of (1) whether meaningful change has occurred in their self-identified primary treatment goal and (2) whether there has been a meaningful change in their overall condition. Item 1 assesses the degree to which the treatment goal was achieved using a numeric 4-point scale (1. I did not meet my treatment goal; 2. I partially met my treatment goal; 3. I mostly met my treatment goal; 4. I completely met my treatment goal), where participants who select 1 of options 2–4 are subsequently queried about the meaningfulness of this change (If this is your answer, was this change meaningful to you?). Item 2 assesses the respondent’s perspective on the change in their overall condition on a 5-point scale (1. My condition has worsened; 2. My condition has not changed; 3. I have noticed an improvement, but it is not enough to be meaningful to me; 4. I have noticed an improvement large enough to be meaningful, but not a complete improvement; and 5. I have noticed complete improvement). Three versions of both questionnaires were used in the cognitive interviews. The original content for both questionnaires was developed by measurement experts with experience in developing clinical outcome assessments and with goal attainment scaling.

Figure 1 TGES to assess within-patient meaningful change. Abbreviations: TGES, Treatment Goals Evaluation Scale.

Content Validation

The TGES measure was evaluated across 3 rounds of one-on-one cognitive interviews with a total of 28 participants (round 1, n=6; round 2, n=15; round 3, n=7), a sample size larger than that consistently reported to be sufficient for cognitive interviewing studies focusing on clarity and comprehension of PROMs.^21–24 Iterative revisions were made to versions 1 and 2 of the questionnaires after the first and second rounds, respectively, in alignment with participant feedback (Supplemental Figure S1). Trained, qualified interviewers conducted one-on-one cognitive interviews by telephone between 7 March and 11 August 2022. Interviews followed a semi-structured interview guide that included questions and prompts to assess the clarity, interpretation, and ease of completion of the TGES – Baseline and TGES – Follow-Up questionnaires. Interviews began with open-ended questions about the participant’s experience with treatment for AD or migraine, followed by the completion of the TGES – Baseline and the TGES – Follow-Up questionnaires and discussion of the TGES measure, including general questions (eg, Overall, what did you think of the questions asked?), instruction-specific questions (eg, What does this question mean to you?), and item-specific questions (eg, item instructions, question, and response options).

Qualitative and Statistical Analysis

All interviews were audio-recorded, transcribed, and anonymized. Transcripts were coded and qualitatively analyzed using ATLAS.ti version 9 to identify and categorize concepts and relate them to the TGES. The qualitative analysis included assessment of participants’ understanding of the instructions, items, and response options; the appropriateness of the response scales; and identification of any common treatment goal categories. Patient sociodemographic and clinical characteristics were summarized using descriptive statistics (eg, mean, median, standard deviation [SD], and range for continuous variables; frequency and percentages for categorical variables). Statistical analyses were performed using SAS 9.4 (SAS Institute Inc). Subgroup analyses were not conducted due to small sample sizes; no formal statistical comparisons were performed.

Results

Participant Characteristics

A total of 28 participants with AD (n=15) or migraine (n=13) were included in the study sample. The mean (SD) age of participants was 45.3 (15.7) years, and the majority identified as female (60.7% [n=17/28]), White (78.6% [n=22/28]), and not Hispanic or Latinx (92.9% [n=26/28]) (Table 1). Most participants were educated at college/university level or above (71.4% [n=20/28]) and married or cohabitating/living with a partner (67.9% [n=19/28]). Over half (60.7% [n=17/28]) of participants reported at least 1 health condition other than AD or migraine, with anxiety (39.3% [n=11/28]), depression (32.1% [n=9/28]), and hypertension (14.3% [n=4/28]) being the most frequently reported comorbidities in the overall participant sample (Table 1). Among participants with AD, the mean (SD) duration of AD diagnosis was 26.9 (18.6) years, and current AD severity level ranged from clear to severe, with the greatest proportion of participants having moderate severity (40.0% [n=6/15]). Among participants with migraine, the mean (SD) duration of migraine diagnosis was 18.4 (13.5) years, and the number of headache days per month ranged from 0–3 days to ≥15 days, with the greatest proportion of participants having ≥15 headache days per month (38.5% [n=5/13]).

Table 1 Participant Characteristics

Cognitive Interview Results

Treatment Experience

Participants were asked general questions about their experience with treatment for their condition, including their treatment history and goals (Supplemental Table S1). Two participants, both with AD, were currently on their first treatment; the remaining 26 participants (92.9%) had tried other treatments for their condition prior to their current treatment. These patients (n=26, including 13 with AD and 13 with migraine) were asked about their primary treatment goal when receiving previous treatment(s). The most common primary goal for previous treatment of AD was clearing up the skin (30.7% [n=4/13]), and the most common primary goal of previous treatment for migraine was eliminating migraines completely (46.2% [n=6/13]). Participants were then asked if their primary goal for previous treatment was different from their primary goal for their current treatment. Over half (57.7% [n=15/26]) reported their goal to be the same, but 42.3% (n=11/26) reported their goal had changed. For 4 participants (15.4% [n=4/26]), their initial goal was to manage their condition, while their current goal was to eliminate or prevent it altogether (“[I]nstead of trying to, to treat it every time it flares up […] I would rather just clear it up for good […]”). Two participants (7.7% [n=2/26]) reported giving up on their original goal and being willing to settle for a small improvement in HRQOL (“[M]y goal was to make all the headaches go away, I have kind of given up on that […] I would settle on quality of life […]”).

TGES – Baseline

Most respondents had no difficulty understanding or answering the TGES – Baseline item (Figure 2, Supplemental Table S2). After completing the questionnaire, participants were asked about their initial impressions and understanding of the instructions. Several respondents described the questionnaire as “good” (25.0% [n=7/28]) or “clear” (25.0% [n=7/28]), and most interpreted the instructions as intended without difficulty or confusion (82.1% [n=23/28]). All of the 6 participants who completed version 1 (100.0% [n=6/6]) reported having difficulty in selecting their most important treatment goal, and 7 participants across interview rounds 1 and 2 (33.3% [n=7/21]) reported that the question was broad or vague. Eleven participants (39.3% [n=11/28]) suggested revisions to the instructions across versions 1 through 3. Changes to version 1 included clarification of instructions to indicate that participant responses should be as specific as possible and include only 1 treatment goal; addition of examples illustrating the desired level of specificity for responses; removal of a label above the question 1 box (“Question 1: Identifying Your Primary Treatment Goal”) to reduce redundancy; and rewording of the item to make the question less vague and clarify that the response should include the “most important” treatment benefit (Item tracking matrix, Supplemental Table S3).

Figure 2 TGES – Baseline interpretation and comprehension. Abbreviations: TGES, Treatment Goals Evaluation Scale.

Three participants who completed version 2 (20.0% [n=3/15]) suggested using only the term “goal” or “benefit” to avoid confusion, with 2 participants reporting difficulty with distinguishing the terms “one treatment goal” and “one treatment benefit.” Version 2 was revised further by adding the label “Primary Treatment Goal” above the question to clarify the connection between “primary treatment goal” and “most important benefit.” No further revisions were suggested for version 3 (Supplemental Table S3).

Based on participants’ discussions of their interpretations of the questionnaire, their answers, and the reasoning they provided for their specific answers during the cognitive interviews, it appeared that nearly all respondents (96.4% [n=27/28]) interpreted the TGES – Baseline item as intended (Figure 2, Supplemental Table S2). Most participants (85.7% [n=24/28]) reported that it was easy to respond, and 85.2% (n=23/27) reported no confusion with the question. Although only 1 respondent misunderstood the instructions, over half of participants (60.7% [n=17/28]) were prompted to revise their initial response because it was vague (28.6% [n=8/28]), included multiple goals (25.0% [n=7/28]), or did not include a specific goal for how they wanted a symptom to change (3.6% [n=1/28]) (Supplemental Table S4). Examples of appropriate primary treatment goals provided by participants on the TGES – Baseline included, “I would like to see my skin clear with […] the treatment that I’m using” and “I want to be able to manage my migraine frequency without medication” (Table 2).

Table 2 Primary Treatment Goal Examples for Participants with AD or Migraine

Common concepts reported by participants with either AD or migraine enabled the derivation of condition-specific goal categories (Table 2). Three goal categories that were not mutually exclusive were identified for AD: improvement in physical symptoms (eg, flakiness, cracking, or peeling [n=9]), sensory symptoms (eg, itching and feeling uncomfortable [n=7]), and choice of clothing (eg, selecting outfits or wearing revealing clothes [n=2]). Four goal categories were identified for migraine: reduced frequency of migraines (n=5), impact on activities (n=3), severity of migraines (n=2), and symptoms associated with migraine other than headache, such as nausea and dizziness (n=2).

TGES – Follow-Up

The TGES – Follow-Up questionnaire was completed after the TGES – Baseline questionnaire. All participants confirmed that their responses from the TGES – Baseline were taken into consideration when answering the TGES – Follow-Up (100.0% [n=28/28]). As a follow-up question, some participants (n=26) were asked whether it was easy or difficult to consider their response to the TGES – Baseline questionnaire while completing the TGES – Follow-Up questionnaire; all of these participants stated it was easy to consider their responses to the previous questionnaire (100.0% [n=26/26]). All participants interpreted the instructions as intended (100.0% [n=28/28]), but 6 participants (21.4% [n=6/28]) suggested revisions, 2 of whom (33.3% [n=2/6]) suggested making the word “responses” singular, noting that there is only 1 response to the Baseline questionnaire (Figure 3, Supplementary Table S5). This change was implemented following feedback received on version 1, and there were no further revisions to the instructions (item tracking matrix, Supplemental Table S6).

Figure 3 TGES – Follow-up interpretation and comprehension. Abbreviations: TGES, Treatment Goals Evaluation Scale.

Nearly all participants understood TGES – Follow-Up item 1 (“Since you began treatment, to what extent have you met your treatment goal?” [version 1]) as intended (96.4% [n=27/28]) (Figure 3, Supplementary Table S5 and S6). Among 26 participants asked about confusion with item 1, most (84.6% [n=22/26]) reported there to be no confusion and most indicated that it was easy to respond to the question (96.2% [n=25/26]). A few participants in this group reported confusion (15.4% [n=4/26]), with 1 participant misunderstanding the phrase “since you began treatment.” Among the total study population, 4 participants proposed revisions to item 1 (14.3% [n=4/28]) to address ambiguity around the phrase “since you began treatment.” Following feedback received on version 2, this phrase was revised to “since you began your current treatment”; no additional changes were made. Many participants (82.1% [n=23/28]) reported no issues distinguishing between the item 1 response options and interpreted these as intended, and most (92.9% [n=26/28]) interpreted the meaningful change subquestion (“Was this change meaningful to you?”) as intended and could describe the smallest amount of change they would consider meaningful (“I have some spots, but they’re not nearly as noticeable […] like it’s still meaningful to me that it happened, even though it’s not completely gone […].”)

Although 6 participants (21.4% [n=6/28]) suggested changes to the subquestion, none were implemented as they would impact the purpose of the question (eg, changing “meaningful” to “important”).

Three-quarters of participants (75.0% [n=21/28]) understood the TGES – Follow-Up item 2 (“Please choose only one response below that best describes the change in your [insert name of condition] since you began treatment.” [version 1]) as intended, and most (82.1% [n=23/28]) indicated that it was easy to select a response (Figure 3, Supplementary Table S5 and S6). As similarly observed for item 1, some participants completing version 1 and 2 of the TGES – Follow-Up (23.8% [n=5/21]) misunderstood the phrase “since you began treatment.” Three of these participants recommended clarifying the item referring to the participants’ current treatment. As for question 1, following feedback received in version 2, the item 2 wording was revised to “since you began your current treatment.” Almost all participants (92.9% [n=26/28]) reported no issues distinguishing between the item 2 response options, with minimal difficulty choosing between response options (“Okay, so, since the beginning of the treatment, I’m kind of like stuck between 3 [‘I have noticed an improvement, but it is not enough to be meaningful to me’] and 4 [‘I have noticed an improvement large enough to be meaningful, but not a complete improvement’]. […] I’m kind of, like, in the middle of that. […].”)

Nine participants (32.1% [n=9/28]) suggested revisions to the response options on version 1 that were implemented in version 2 and retained in version 3. To address this feedback, response option 2 (“My condition has not become better or worse [version 1]”) was simplified (“My condition has not changed” [version 2 and 3]) and the wording of response option 5 (“I have noticed meaningful and complete improvement” [version 1]) was revised to remove redundancy (“I have noticed complete improvement” [versions 2 and 3]) (Supplemental Table S6).

Discussion

Whether a change in health status experienced during a course of treatment is considered “meaningful” is ultimately decided by the individual. The current study aimed to develop a tool for identifying and assessing meaningful treatment goals from an individual’s perspective and the extent to which they were achieved. The TGES guides individuals in setting personalized treatment goals that focus on the aspects of their condition most important to them (TGES – Baseline) and assesses their perspectives on the extent to which they met those goals (TGES – Follow-Up). The TGES – Follow-Up questionnaire also evaluates the meaningfulness of the changes in these self-identified goals. Ultimately, by prioritizing personalized endpoints, use of the TGES can enhance patient-centric approaches in both a clinical trial setting and real-world clinical practice.

This study was conducted with a sample of patients with AD and migraine, both of which are common chronic conditions that affect multiple aspects of patients’ HRQOL.^19,20 In support of the content validity of the TGES, common goal categories were identified from respondents’ baseline treatment goals (eg, “I would like to see my skin clear with using the treatment that I am using”; “I don’t want to cancel plans due to migraines”): 3 categories emerged for patients with AD (improvement in physical symptoms, sensory symptoms, and choice of clothing) and 4 for patients with migraine (frequency of migraines, impact on activities, severity of migraines, and symptoms associated with migraine other than headache). The identification of treatment goal categories supports broad applicability of the TGES to other chronic conditions, where it may facilitate identification of meaningful condition-specific goals and subsequent evaluation of treatment responses.

The TGES was designed to be applicable to both clinical trial and real-world settings. In clinical trials, the TGES could help contextualize treatment benefits by allowing comparisons of the proportions of participants who met their self-identified goals between given treatment groups. Importantly, many of the primary goals provided in this study (eg, “I would like to feel more confident and not be embarrassed by my skin”) may not be captured with non-personalized clinical trial endpoints (eg, investigator-reported Eczema Area and Severity Index [EASI] score),²⁵ highlighting the value of the TGES as an additional endpoint to align clinical trial outcomes with real-world priorities for patients. In clinical practice, capturing patient treatment priorities and the meaningfulness of changes in their condition is central to goal-oriented and patient-centered care.^26–28 Moreover, the TGES can facilitate dialogue between patients and providers to establish trackable, personalized goals that can be regularly evaluated over the course of treatment (Figure 1).

In addition, the TGES is not subject to a limitation of goal-setting PRO instruments that follow a collaborative approach (eg, GAS), where both patient and provider agree on a health-related goal.²⁹ This may not reflect the most meaningful goal to an individual and could result in decreased motivation to achieve the set goal. Furthermore, treatment priorities could be misaligned between patients and physicians or larger medical associations. For instance, in a study examining patient perspectives in the treatment of rheumatoid arthritis, priorities patients reported as being important—including psychosocial well-being (eg, life enjoyment) and specific aspects of function (eg, joint damage)—were not included in the professional core set of outcomes from the American College of Rheumatology, which assesses disease activity more globally.³⁰ We found most participants (≥82.1%) reported the TGES – Baseline and TGES – Follow-Up items were easy to answer, and most (92.9%) correctly interpreted the meaningful change concept in the TGES – Follow-Up as intended. This supports the independent identification of goals and meaningful changes by patients on the TGES without the need for shared decision-making or provider influence.

The TGES PROM has several strengths and potential applications. Designed to collect and track patients’ individual goals and progress, the ability of the TGES to capture changes in goals and internal anchors over time may reveal changes in patients’ understanding of their health condition, resulting in more realistic treatment goals. Through the inclusion of an open-ended baseline item, the TGES demonstrated the capacity to identify common goals within a disease area, as well as less common goals, which are not typically captured with fixed response options in other global PROM scales, such as the patient global impression of severity (PGIS) or change (PGIC). Further, treatment goal identification is rarely captured by PROMs specific for AD (eg, Atopic Dermatitis Control Tool)³¹ and migraine (eg, the 6-item Headache Impact Test),³² an important factor that facilitates treatment decisions that align with patients’ priorities. The use of common anchors enables follow-up responses regarding meaningful change to be used analytically and collectively across patients within a given treatment context. Therefore, the TGES has the potential to contextualize individual responses and indicate treatment benefits to other patients seeking improvements in similar goal categories. Moreover, the TGES was designed to distinguish between noticeable improvement and meaningful change, using verbal anchor options rather than numeric scales. As a result, both subject and central-tendency biases could be reduced in clinical trials and real-world settings.

Nonetheless, certain limitations, including those inherent in qualitative studies, should be considered when interpreting the findings of this study. The size of the sample was relatively small, as often observed in qualitative research.³³ Racial, ethnic, and demographic diversity was limited, and participants were limited to individuals under age 65, since older individuals are more likely to have multiple chronic conditions or advanced, terminal illnesses (eg, cancer).^34,35 Although the TGES may be broadly applicable across chronic conditions, its use may be limited for terminal conditions where increased survival is likely to be the primary treatment goal or for acute infections, such as the common cold³⁶ and acute bronchitis,³⁷ which have short treatment cycles. Taken together, the results presented here may not be fully representative of the broader population or those with different conditions. Of additional note, interviews were conducted virtually, which restricts the inclusion of participants to those with access to or familiarity with electronic devices and may have introduced a selection bias. However, a strength of this approach is that it allows for scheduling flexibility and increased access for certain participants, such as those with mobility challenges or those residing in remote geographic areas.³⁸ During this study, an acceptable level of specificity for the treatment goal on the TGES – Baseline questionnaire was determined subjectively by the interviewer; while interviewers received detailed training on defining “specific goals” using the semi-structured interview guide, bias and variability across interviewers or sites was still possible. While a strength of the TGES is that individual patients define what constitutes a meaningful change in their primary treatment goal and overall condition, this introduces the potential for subjectivity that must be considered in a clinical trial and regulatory review context. Of note, while the highly individualized, qualitative data from the TGES may not allow for formal statistical analyses in a clinical trial setting, when combined with data from other PROMs, the TGES may provide valuable context that allows for a more comprehensive understanding of the patient-specific impact of a novel intervention. Further, thematic analyses may be possible on the types of treatment goals that study participants have identified to better understand the outcomes that matter to patients. The design of the TGES allows for the construction of responder trial endpoints to contextualize the extent to which participants achieved their most salient treatment goal.

Future research qualitatively and quantitatively validating the TGES in a larger cohort of patients with differing demographics and health conditions to assess measurement properties and expand across therapeutic areas is warranted. As the TGES is designed to be used over time and content validity was assessed at a single time point in this study, qualitative studies assessing completion of the TGES – Baseline and Follow-Up over longer time periods are warranted to comprehensively evaluate content validity. This is particularly important to validate the ability of the TGES to measure patients’ long-term goals and how they may shift over the course of treatment. Finally, it is important to note that this study focused on content validation of the TGES via qualitative interviews, and quantitative evaluation of measurement and psychometric properties and implementation of additional bias mitigation measures are warranted to further support the validity of the TGES.

Conclusions

The TGES – Baseline and TGES – Follow-Up questionnaires were developed to guide identification of an individual patient’s primary treatment goal and to subsequently assess their perspective on the extent to which a meaningful change in this goal has been achieved. Cognitive interviews were used to evaluate content validity, and findings support the use of the TGES for patients with AD and migraine. As a patient-centric, condition-agnostic measure, the TGES has potential for application in assessing personalized endpoints and within-patient meaningful change in individualized goals in both clinical trial and real-world practice settings across a broad range of chronic health conditions. Future studies are warranted to assess the measurement properties and longitudinal validity of the TGES among larger, more culturally and demographically diverse groups of participants living with different conditions.