Goal Attainment Scaling for Depression: Validation of the Japanese GAS-D Tool in Patients with Major Depressive Disorder

Introduction

Recovery from depression is often a complex, personal journey and depends on several factors such as health beliefs, baseline clinical severity and functioning, as well as medication and treatment adequacy.^1,2 Currently, the concept of recovery is clinical, referring to recovery from psychiatric symptoms,^3,4 and the most commonly used assessment tools, such as the Montgomery–Åsberg Depression Rating Scale (MADRS), focus only on symptom severity.⁵ Even functional rating scales such as the Sheehan Disability Scale (SDS) do not address what is meaningful to patients. Although both patients’ and clinicians’ aim for “treatment success”, their definitions of remission and how remission may be measured may not be equivalent.⁶ Indeed, people with major depressive disorder (MDD) often have their own definitions of personal recovery, which may not always align with the physician perspective.^7,8 In one survey evaluating priorities for depression treatment, patients placed more importance on the restoration of positive affect, while the clinicians focused on alleviation of depressive symptoms.⁹

Given the variations in how treatment success may be defined, flexible measures that consider the patient’s unique perspective on their own condition are important. Goal attainment scaling (GAS) has been proposed as a person-centric, semi-quantitative measure that assimilates achievement of individually set goals into a single standardized “goal attainment score” that can be compared at the population level. Originally developed in the context of evaluating the effectiveness of community mental health programs,¹⁰ a version has been developed for use in MDD (GAS-D).^11–13 The GAS-D tool leverages the collaborative process of goal setting (as an integral part of shared decision-making) and allows for a measurement of overall treatment effectiveness that is meaningful to each individual patient. The GAS-D has been used as a primary outcome measure of the effectiveness of vortioxetine in two clinical studies. The first was a North American study that evaluated progress towards achievement of goals over 12 weeks and demonstrated correlation of GAS-D scores with traditional rating scales.¹⁴ The second was an observational, single-arm cohort study conducted in Japan (VGOAL-J), targeting employed patients with MDD treated with vortioxetine in an outpatient setting.¹⁵

We present here the first assessment of the psychometric properties of the Japanese version of the GAS-D. The primary results of the VGOAL-J study will be published separately.

Methods

Study Design and Patients

The VGOAL-J study was a 24-week, single-arm, observational study in employed patients with MDD initiating treatment with vortioxetine in outpatient settings in Japan.¹⁵ Eligible patients were employed Japanese outpatients (aged 20 to 65 years) with a diagnosis of MDD,¹⁶ and initiating treatment with vortioxetine according to the Japanese label. Key exclusion criteria included a diagnosis of schizophrenia or other psychotic disorder, bipolar disorder, dementia, or other neurodegenerative disease(s) significantly impacting cognitive functioning, a prescription of two or more antidepressants at baseline, and a significant risk of suicide or attempted suicide within the last six months. The study was conducted in accordance with the Declaration of Helsinki and International Society for Pharmacoepidemiology (ISPE) Guidelines for Good Pharmacoepidemiology Practices (GPP). The protocol was approved by each site’s local research committees (Ichigaya Himorogi Clinic, Japan Conference of Clinical Research, Yoyogi Mental Clinic, and Yokohama Minoru Clinic IRBs). All patients provided written informed consent before entering the study. The study was registered with the Japan Primary Registries Network (JRCT1031210200).

The primary outcome measures for the main study were the GAS-D (assessed at baseline and weeks 8, 12 and 24) and the Work Productivity and Activity Impairment questionnaire (WPAI).¹⁷ Traditional clinical rating scales, relevant to GAS-D validation, were assessed as secondary outcomes and included the MADRS (Total¹⁸ and Anhedonia factor¹⁹ scores), SDS,²⁰ Perceived Deficits Questionnaire-Depression 5-item (PDQ-D-5),²¹ EuroQol questionnaire,²² Patients and Clinicians Global Impression of change (PGI-C and CGI-C),²³ Oxford Depression Questionnaire (ODQ),²⁴ and digit symbol substitution test (DSST). In addition, at the end of the study (Week 24), patients and physicians were asked to rate how useful they found the GAS-D approach in establishing treatment goals, monitoring progress, and helping achieve successful treatment outcomes; each aspect of satisfaction was assessed on a 5-point scale (not useful at all to very useful).

Goal Setting and Goal Attainment Scaling

The original version of GAS-D¹¹ was translated into Japanese and modified to set two goals. Translation included back-translation to English and evaluation of the comparability of language and similarity of interpretability by a clinical expert. Cultural nuances in how the domains and subdomains categories are worded were identified and addressed during the adaptation process. All investigators received training and certification for using the GAS-D, with emphasis on setting SMART (specific, measurable, achievable, realistic, and timed) goals to be negotiated and agreed between the investigator and patient. Using the SMART framework, investigators had to consider the attainability of patient goals, use of observable objectives and benchmarks, equidistance of scaling, expected level of difficulty to achieving the goal, goal differentiation, and overall quality of goal statements.

Patients and clinicians met at baseline to determine two personal treatment goals (one patient-defined and one from predefined domain categories) and were followed at weeks 8, 12, and 24 to assess progress. The pre-specified domains included psychological, motivational, emotional, physical/functional, and cognitive categories, each with multiple subdomains. Each goal had its own written goal statement with levels of possible achievement set out on a predefined 5-point rating scale (–2 to 2), where a score of –2 indicated baseline status (0% goal achievement), 0 indicated that the expected goal was achieved as expected (100% goal achievement) and 2 indicated that the patient achieved much more than expected (200% goal achievement). Figure 1 shows an example goal statement from the study. Goals were also ranked for importance to the patient (1 = least important and 2 = most important) and achievability (1 = easy, 2 = average difficulty and 3 = difficult).²⁵

Figure 1 Examples of goal statements from the study.

At Week 24, a composite GAS-D T-score for each patient was derived from the product of their individual goal achievement scores multiplied by goal weighting, using the following standard formula, where w_i is the weight assigned to the i^th goal (if equal weights, w_i = 1) and x_i is the numerical value achieved (between –2 and +2):¹⁰

The GAS-D T score is standardized such that, assuming goals are set in an unbiased fashion, the mean GAS-D T score will be 50 with a standard deviation of 10.²⁵ As such, a score of 50 indicates goals were on average achieved as expected, less than 50 indicates goals were achieved less than expected, and over 50 indicating goals were achieved better than expected.

Data Analysis

Analyses were performed using the Full Analysis Set (FAS) which included all eligible patients who initiated vortioxetine treatment and completed the baseline visit and ≥1 follow-up visit. Change from baseline in GAS-D T scores was analyzed using a two-sided test and changes from baseline in the traditional clinical rating scales were analyzed using a restricted maximum likelihood (REML)-based mixed model for repeated measures (MMRM) with baseline age, sex, visit, baseline score, and baseline score-by-visit interaction included as fixed effects. Analyses were based on all available observations; in case of missing scores, total scores and/or subdomain scores were imputed, unless they exceeded 1 item for the SDS, 5 items for the ODQ, or 20% of MADRS and PDQ-D-5 items, in which case they were set to missing. Total scores or subdomain scores were calculated as the mean of non-missing items multiplied by the total expected number of items.

To assess convergent validity, Spearman-rank correlations between the absolute scores and change from baseline in GAS-D T scores and traditional clinical measures were assessed at Baseline and Weeks 8, 12, and 24. The relationship between GAS-D and baseline demographics, indices, and MADRS scores were also evaluated using correlations and multiple linear regressions. Since vortioxetine is an established treatment for MDD,^26,27 test-retest reliability was assessed for the Japanese version of the GAS-D by Intraclass Correlation Coefficient (ICC) using the data from Week 8 and 12 in those who did not report changes in PGI-C (ie, those with the same PGI-C scores at Week 8 and 12). Patient and clinician perceptions of the GAS-D approach were summarized using descriptive statistics.

Results

Patient Disposition and Baseline Characteristics

Overall, 124 working patients with MDD were enrolled from 19 sites across Japan, of which 116 were treated with vortioxetine according to the Japanese label and had ≥1 follow-up visit and were included in the FAS. Overall, 103 patients completed the study, reasons for premature discontinuation were withdrawal of consent (n=9), investigators decision (n=7), loss to follow-up (n=4) and other (n=1). Baseline characteristics for the FAS population are given in Table 1. The mean ± SD age was 38.4 ± 11.2 years, time since first MDD diagnosis was 5.8 ± 5.9 years, and duration of the current depressive episode was 402 ± 728 days; the majority of patients were educated to diploma or bachelor’s degree level.

Table 1 Baseline Demographics and Clinical Characteristics (Full Analysis Set)

Patient goal domains are shown in Table 2. The most common self-defined goal was related to motivation (41.4%), followed by physical/functional goals (20.7%) and psychological goals (18.1%). By contrast, the goals chosen from predefined lists were more evenly distributed, with 28.4% of patients setting goals related to motivation goals and the same proportion setting physical/functional goals. Approximately three-quarters (74.4%) of patients rated their self-defined goal as the most important of the two set goals.

Table 2 Goal Setting at Baseline

Changes in GAS-D T Scores Over 24 Weeks

At the population level, mean ± SD GAS-D T scores steadily increased over the study, from 41.9 ± 11.8 at Week 8 to 45.8 ± 13.8 at Week 12, and 51.9 ± 15.5 at Week 24 (Figure 2). Likewise, the respective mean ± SD changes from baseline of 15.8 ± 11.8, 19.7 ± 13.8, and 25.8 ± 15.5, indicated relevant improvement at each successive timepoint. Baseline characteristics, including baseline severity (MADRS scores) were not found to predict GAS-D T scores or change scores at any time point assessed (Table e1).

Figure 2 Change in GAS-D T scores over time.

Associations Between Goal Attainment and Traditional Outcome Measures

Directionally, the associations between GAS-D T and change in GAS-D T scores and other clinical measures demonstrated good convergent validity with measures of symptom severity and overall clinical impression.

At Week 24, GAS-D T scores showed a fair negative correlation with MADRS Total (correlation coefficient of −0.38), CGI-C (correlation coefficient of −0.35), and CGI-S (correlation coefficient of −0.33) scores (Figure 3a). While positive correlations were observed with overall quality of life (EQ-5D-5L and EQ-VAS) these were generally weaker (correlation coefficient <0.25). Analysis of the relationships between change from baseline in GAS-T scores and change from baseline in traditional outcome measures revealed a trend to increasing significance over time (Figure 3b). Whereas change in MADRS Total scores were the only traditional outcome to show a statistically significant correlation with the change from baseline in goal attainment at Week 8, other measures started to show significance starting at Week 12 and the strength of correlation also generally increased. Of note, the correlation between change from baseline of GAS-D and change from baseline of SDS total score reached significance at week 24. In addition, multiple regression analyses at Week 12 and at Week 24 confirmed significant associations between goal attainment (GAS-D and change in GAS-D) and symptom severity (MADRS Total and change in MADRS Total) and between goal attainment (GAS-D) and change in clinical impression (CGI-C) (Table 3).

Table 3 Multiple Linear Regressions of GAS-D with Traditional Rating Scales

Figure 3 Spearman correlation “heat map” (a) GAS-D T score correlations with traditional rating scales. (b) Change from baseline in GAS-D T score correlations with change from baseline in traditional rating scales. Data are derived using Spearman-rank correlations; *p<0.05, **p<0.01, ***p<0.001. Abbreviations: CGI-C, Clinical Global Impressions–Change; CGI-S, Clinical Global Impressions–Severity; DSST, Digit Symbol Substitution Test; EQ-5D-5L, EuroQol Five Dimensions Five Levels; EQ-VAS, EuroQol Visual Analog Scale; GAS-D, Goal Attainment Scale adapted for depression; MADRS, Montgomery-Åsberg Depression Rating Scale; PDQ-D-5, Perceived Deficits Questionnaire–Depression 5-item; PGI-C, Patient Global Impressions–Change; SDS, Sheehan Disability Scale; WPAI, Work Productivity and Activity Impairment questionnaire.

Test-Retest Reliability

An intraclass correlation coefficient value of 0.67 (95% CI 0.45–0.82) for GAS-D indicated moderate test-retest reliability (P<0.001) between Week 8 and Week 12 among patients who did not report changes in PGI-C.

GAS-D Satisfaction Survey

Most patients and clinicians reported the GAS approach to be “useful” or “very useful” for establishing treatment goals, monitoring progress toward treatment goals, and helping achieve successful treatment outcomes (Figure 4).

Figure 4 Patient and clinician satisfaction with the GAS-D approach.

Discussion

This study confirmed acceptable psychometric properties of the Japanese version of the GAS-D in patients with MDD, receiving open-label treatment with vortioxetine. Goal attainment was significantly associated with symptom severity as assessed by the MADRS scale, confirming convergent validity. In particular, GAS-D scores showed a fair negative correlation with MADRS total score at Weeks 12 and 24, indicating that improvements in overall symptom severity with vortioxetine treatment are likely to be reflected in the achievement of individualized treatment goals. Additionally, SDS scores improved in correlation to GAS-D after 24 weeks of treatment, highlighting the importance of long-term treatment to achieving functional recovery and personalized goals.

The GAS-D demonstrated good internal consistency and test-retest reliability, while proving independent of demographic characteristics, showing generalizability, and further strengthening its potential role in routine clinical practice in Japan. As with the original English version, goal attainment was fairly correlated with depressive symptoms (using the Patient Health Questionnaire-Depressive symptoms in the US study²⁸ and MADRS scores in the present study) and between goal attainment and global clinical impression.^28,29 The weaker correlations with measures of quality of life observed in the Japanese population (as assessed using the EuroQol) as compared to the strong correlations observed in the US population (as assessed by the Quality of Life Enjoyment and Satisfaction Questionnaire) might reflect the differences in scale used, cultural differences, other demographic differences in the population studied (eg our study focused on working patients).

Motivational deficits are common in MDD and are significantly linked to functional impairment, which in turn is linked to quality of life.³⁰ The fact that the most commonly set personal goal domain was related to motivation (41% of personal goals) as opposed to psychological or emotional (18.8% and 6.8%, respectively) goals emphasizes the importance of this domain to patients. By contrast, motivation is relatively poorly captured by traditional clinical rating scales such as the MADRS and the SDS,³¹ highlighting the potential utility of including the GAS-D as a complementary, patient-centered outcome in assessments of clinical efficacy. In this way, using the GAS-D in clinical studies may provide a more personalized picture of treatment response. Of note, the observation that goal attainment gradually developed over time till Week 24 suggests that goal achievement not only requires the continued development of symptom control but also time to practice the goal. Thus, clinical studies using the GAS-D as a key outcome measure may need to be longer than the current study to pick up the full efficacy of a treatment on goal attainment. In this respect, the time lag between symptom control and functional improvement is already well known,³² and the idea that goal achievement and functional improvement track together is supported by the gradual strengthening of the relationship between goal attainment and functional improvement on SDS over time.

Across all fields of medicine, there is wealth of evidence to support the positive impact of improving patient engagement on health outcomes in patients with long-term conditions.^33,34 Indeed, the goal setting approach is in itself a potentially treatment strategy for people with depression because it is considered a prominent behaviour change technique.³⁵ In this study, patient acceptance and the practical utility of the GAS-D approach were supported by the high ratings of satisfaction with the GAS-D as a means of establishing treatment goals, monitoring progress toward treatment goals, and helping achieve successful treatment outcomes. When goals are personally meaningful, patients report that they facilitate motivation and assist the recovery process,³⁶ and the process of goal setting using SMART techniques offers a structured framework for empowering patients to take ownership of their own care and personal recovery.

Limitations of the present analysis include the open-label study design and the study focus on employed patients who are only receiving one specific type of antidepressant (vortioxetine). It is conceivable that the broader MDD population will have different goals to those who are in active employment. GAS-D validation was a secondary objective of the study, and the main study included a relatively limited selection of measures against which the Japanese version of the GAS-D validated. For example, it would be useful to compare the GAS-D against the Hamilton Depression Rating Scale. Because most patients rated the self-determined goal as most important, we did not differentiate between the two types of goal (personal and predefined) set in the study. Previous studies have found stronger associations with personal goals (vs predefined goals) and functional improvement,²⁸ but our prespecified validation analyses did not include this categorization. In our experience, application of the GAS-D in a real-world setting requires significant training to successfully set goals and assess attainment, and the use of predefined goals has been suggested to make this process easier.²⁵

Conclusion

In summary, the results of this open-label study support the use of the Japanese version of the GAS-D tool as a valid and sensitive outcome measure in the assessment of treatment response in MDD that assesses treatment success in a way that is most meaningful to the patient.