Assessing Reliability and Validity of the Oxford Depression Questionnaire (ODQ) in a Japanese Clinical Population

Introduction

Major depressive disorder (MDD) is a leading cause of disability worldwide, with significant negative impacts on daily functioning and quality of life. While the clinical construct of MDD in Japan now aligns with international criteria, reported lifetime prevalence is typically lower than that reported in Western cultures.¹ A recent Japanese insurance claims database study estimated the prevalence of diagnosed MDD to be 2.0% (just prior to the Covid 19 pandemic);² although this is thought likely to be an underestimate.³ Other potential cultural differences include a tendency to report less psychological and more somatic symptoms of depression when compared to a western population.^1,4 To date, there has been scant culturally specific literature on the emotional aspects of MDD. Globally, the use of antidepressants is increasing, including in Japan,⁵ and the introduction of different classes of antidepressant makes it an imperative to understand how they impact specific symptoms. This necessitates the use of clinical scales validated for use in Japan.

Reduced emotional responsiveness or emotional blunting is a common symptom in patients with MDD,^6–9 but also frequently occurs during antidepressant treatment.^6–8,10 Whereas anhedonia, a core diagnostic criterion for MDD, is the inability to experience positive emotion, emotional blunting is a broader concept that includes a reduced ability to experience both positive and negative emotions. Patients report feeling “numb” or “flat”, unable to laugh, cry, and enjoy what they used to enjoy. They often cite feeling indifferent toward others and creative projects, as well as a loss of empathy.^11,12 Although often considered a consequence of medication, it has been recently argued that emotional blunting could be itself a symptom of depression^6,13 that has neurobiologic and phenotypic overlap with anhedonia and other depressive symptoms.^14–16 In a recent survey of 752 patients who reported emotional blunting, 56% of patients considered their emotional blunting to be caused by their depression, while 45% believed that their antidepressant medication was negatively affecting their emotions.¹² When present, emotional blunting negatively affects health-related quality of life and daily functioning,^8,11 is associated with a poorer quality of remission,⁶ and is a common reason for patients with MDD to stop antidepressant treatment.¹⁷

The Oxford Depression Questionnaire (ODQ) is a patient-reported scale specifically developed for assessing emotional blunting⁹ and is increasingly included in studies that want to assess the effect of antidepressant treatment on positive and negative emotions.^18–20 It has been validated in English⁹ (original version) and Chinese,^19,20 and has been shown to have high acceptability, construct validity, reliability and sensitivity to change.^9,19,20 The ODQ includes 26 items covering the four dimensions of emotional blunting: not caring, emotional detachment, positive reduction, and general reduction. The ODQ is divided into three sections where Section 1 evaluates the patient’s experience of emotional blunting during the past week and Section 2 compares the current experience of emotional blunting with the patient’s recollection of their normal emotional state before their depression.⁹ Section 3 is for respondents currently prescribed antidepressants for their depression and assesses the patient’s perception of a potential link between their current antidepressant and their experience of emotional blunting, and whether this has affected adherence to treatment. In the original validation study, Price et al found significant correlation of the positive reduction and not caring domains with the level of depressive symptoms as measured by the Beck depression inventory (with positive reduction closely related to cognition), supporting the concept that the ODQ total score is useful for measuring the emotional aspects of depression, rather than “just” an adverse effect of a treatment of depression.⁹ More recently, Christensen et al²¹ demonstrated the scale is sensitive to changes in the clinical state following 8-weeks treatment with vortioxetine.²¹

The original version of ODQ has been through a full translation linguistic validation and translated into Japanese by Oxford University Innovation Limited.²² We have recently reported the findings of a prospective, observational, multicenter, single-arm cohort study conducted in Japan showing that working patients with MDD treated with vortioxetine achieved their personalized treatment goals, with significant improvements in workplace productivity, functioning, disease severity, and health-related quality of life.²³ The study included the Japanese version of the ODQ and the objectives of this validation study were to:

1. Confirm that the good psychometric properties of original English ODQ scale are retained in the translated Japanese version of the ODQ scale as used in a Japanese working population.
2. Identify the Montgomery-Åsberg Depression Rating Scale (MADRS) items that predict emotional blunting as identified using the ODQ in order to develop a MADRS-emotional blunting subscale.

Methods

Study Design and Participants

This was a prospective, 24-week, observational, multicenter cohort study of employed Japanese outpatients (aged 20 to 65 years) with a diagnosis of MDD according to Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) criteria.²⁴ Patients were recruited from general practitioner and psychiatric outpatient practices and were to be initiating treatment with vortioxetine, according to the Japanese label.²⁵ Key exclusion criteria included a diagnosis of schizophrenia or other psychotic disorder, bipolar disorder, dementia, or other neurodegenerative disease(s) significantly impacting cognitive functioning, a prescription of two or more antidepressants at baseline, and a significant risk of suicide or attempted suicide within the last 6 months. The study was conducted in accordance with the Declaration of Helsinki and International Society for Pharmacoepidemiology Guidelines for Good Pharmacoepidemiology Practices. All procedures were performed in accordance with the ethical standards of the local and institutional research committees (Ichigaya Himorogi Clinic, Japan Conference of Clinical Research, Yoyogi Mental Clinic, and Yokohama Minoru Clinic IRBs) and with the Declaration of Helsinki. All patients provided written informed consent before entering the study. The study was registered with the Japan Primary Registries Network (JRCT1031210200).

The primary outcome measures for the main study were the Goal Attainment Scale for Depression (GAS-D)²⁶ (not relevant for this analysis) and the Work Productivity and Activity Impairment questionnaire (WPAI).²⁷ Secondary outcome measures, assessed at baseline and weeks 4, 8, 12, and 24, included the ODQ, Montgomery and Åsberg Depression Rating Scale (MADRS),²⁸ Sheehan Disability Scale (SDS),²⁹ Perceived Deficits Questionnaire-Depression 5-item (PDQ-D-5),³⁰ EuroQol questionnaire,³¹ Patients and Clinicians Global Impression of change (PGI-C and CGI-C),³² and digit symbol substitution test (DSST).³³

Data Analysis

Analyses were performed using the Full Analysis Set (FAS) which included all eligible patients who initiated vortioxetine treatment and completed the baseline visit and ≥1 follow-up visit. Changes from baseline in the ODQ total and domain scores as well as other clinical rating scales were summarized as continuous variables using descriptive statistics and analyzed using a restricted maximum likelihood (REML)-based mixed model for repeated measures (MMRM). The model includes the following fixed effects: baseline age, sex, visit, baseline score, and baseline score-by-visit interaction. An unstructured covariance structure was used to model the within-patient errors. The Kenward-Roger approximation is used to estimate denominator degrees of freedom. The analysis was based on all available observations (there were no missing data for ODQ, MADRS, and the PDQ-D-5 scales).

To evaluate the construct validity of ODQ, an exploratory factor analysis of the 26-item ODQ with data across all visits stacked was conducted using the principal axis method and orthogonal rotation to extract factors. Sampling adequacy was assessed using Kaiser–Meyer–Olkin (KMO); factors with an eigenvalue ≥ 1 were retained. Baseline data were used to assess internal consistency for each dimension and total scores with Cronbach’s reliability coefficient α. Since vortioxetine had a known positive impact on symptoms of emotional blunting,³⁴ test–retest reliability was assessed by Intraclass Correlation Coefficient (ICC) using data from a subgroup of 50 patients who did not report changes in PGI-C between Week 8 and Week 12. To assess concurrent validity, Spearman-rank correlations between the absolute scores and change from baseline in ODQ and other clinical measures were assessed at Baseline and Weeks 8, 12, and 24.

For the second objective, the relationships between total ODQ score and individual MADRS item scores were evaluated. Multiple linear regressions of ODQ total scores on MADRS 10 item scores with data stacked across all visits were conducted to get Variance Inflation Factor and tolerance for each item score to assess for multicollinearity. The relationship between total ODQ score and individual MADRS item scores was analyzed with a REML-based MMRM approach with an unstructured covariance matrix.

Results

A total of 124 patients were enrolled from 19 sites across Japan, of these 116 were included in the FAS and 103 patients completed the study. For FAS, 50.9% patients were female, and the mean (SD) age was 38.4 (11.2) years, time since first MDD diagnosis was 5.8 (5.9) years, and duration of the current depressive episode was 402 (728) days (Table e1). Overall, 49.1% of patients were antidepressant treatment naïve. Of those previously treated, 50.9% were previously treated with an SSRI.

Changes in ODQ Scores Over 24 Weeks

At baseline, the mean (SD) ODQ score was 78.3 (17.4). Statistically significant decreases were observed in ODQ total and all three subdomain scores (general reduction of emotional experience, absence of positive experiences, emotional detachment, and not caring) at each timepoint, showing continual improvement in emotional blunting over the duration of the study period (Figure 1, Table e2). By Week 24, ODQ total score decreased 13.00 points from baseline (P<0.001). Additionally, by Week 24 there was also a significant reduction in scores associating antidepressant treatment with the cause of emotional blunting. Multiple linear regression did not find any single baseline characteristic (including baseline MADRS score) to be a significant predictor of baseline ODQ scores or change from baseline in ODQ score (Table e3).

Figure 1 Change from baseline in ODQ scores over 24 weeks.

Construct Validity

The Kaiser–Meyer–Olkin measure value for sampling adequacy was 0.941. In the exploratory factor analysis, the eigenvalue was three factors, which accounted for 91.8% of the variance (Figure 2). Overall, the items related to ‘General reduction in emotions’, ‘Reduction in positive emotions’ and “Not Caring” tended to load onto Factor 1, while items related to “Antidepressant as cause” generally loaded onto Factor 2, and items related to “Emotional detachment from others” generally loaded onto Factor 3.

Figure 2 Factor analysis (a) scree plot (b) factor loadings.

Internal Consistency and Test-Retest Reliability

Overall, an excellent internal consistency (>0.9) was demonstrated for ODQ total score based on Cronbach’s α coefficient of 0.912 (Table 1). Most individual domain scores ranged from 0.756 to 0.900, indicating good (>0.8) and acceptable internal consistency (>0.7); one exception was the “not caring” domain, which showed a slightly lower Cronbach’s α coefficient due to the inclusion of item 16 “I do not care as much about my day-to-day responsibilities as I did before I developed my illness/problem.” Furthermore, the intraclass correlation coefficient for ODQ total score and ODQ domains ranged from 0.69 to 0.82, which indicated moderate to good reliability.

Table 1 Internal Consistency and Test–Retest Reliability of ODQ Total Score and Domain Scores at Baseline

Concurrent Validity

Directionally, the associations between ODQ scores and other clinical measures were as expected and demonstrated good concurrent validity. ODQ scores had positive correlations with WPAI, MADRS, CGI-S, CGI-C, SDS, PGI-C, and PDQ-D-5 and negative correlations with DSST, EQ-5D-5L, and EQ VAS (Figure 3a, Table e4). Strong correlations were observed between MADRS and ODQ total scores at Week 12 (correlation coefficient of 0.714) and Week 24 (correlation coefficient of 0.654). At Week 12, while ODQ scores were moderately correlated with work productivity, overall functioning, and health status/QoL, they were only weakly correlated with depressive symptom severity (CGI-C) and cognitive symptoms (DSST). However, the strength of the negative correlations between ODQ and DSST scores increased at Week 24.

Figure 3 Spearman Correlation “heat map” (a) ODQ score correlations with other measures (b) Change from baseline in ODQ correlations with change from baseline in other measures. Abbreviations: CGI-C, Clinical Global Impressions–Change; CGI-S, Clinical Global Impressions-Severity; DSST, Digit Symbol Substitution Test; EQ-5D-5L, EuroQol-5 Dimension-Five Levels; EQ-VAS, EuroQol-Visual Acuity Scale; MADRS, Montgomery-Åsberg Depression Rating Scale; PDQ-D5, Perceived Deficits Questionnaire-Depression 5-item PGI-C, Patient Global Impressions-Change; SDS, Sheehan Disability Scale; WPAI, Work Productivity and Activity Impairment questionnaire.

Change from baseline analyses of ODQ scores also showed positive correlations with changes from baseline in WPAI, MADRS, CGI-S, SDS, and PDQ-D-5 and negative correlations with changes from baseline in DSST, EQ-5D-5L, and EQ VAS (Figure 3b, Table e5). Of note, the strength of the negative correlations between change in ODQ scores and change in other clinical scales tended to increase over time.

ODQ Mapping with MADRS Item Scores

In a multiple regression analysis with data stacked across all visits, the individual MADRS items of [6] concentration, [8] inability to feel, and [4] reduced sleep were statistically significant predictors of ODQ total score. Item [7], lassitude was non-significant (P=0.073) in the regression model but reached statistical significance in the Spearman correlation with ODQ total score (0.526, P<0.001) (Table 2). The sum of these four individual MADRS “emotional blunting” items was significantly correlated with ODQ total score (ρ = 0.646, P<0.001) (Table e6).

Table 2 Multiple Linear Regression of ODQ Total Score on MADRS Item Scores

Discussion

This study confirmed acceptable psychometric properties of ODQ-Japan in patients with MDD, receiving open-label treatment with vortioxetine. ODQ scores correlated well with other validated measures that assess patients with MDD, confirming concurrent validity. A direct correlation of emotional blunting was observed with depression severity, patient functioning, and cognitive symptoms. In particular, changes from baselines in MADRS and ODQ scales showed strong correlation, indicating that improvement in emotional blunting is likely reflected in improvement of overall symptom severity. Multiple regression analysis identified the individual MADRS items of [6] concentration, [8] inability to feel, [4] reduced sleep, and [7] lassitude as correlated with ODQ scores, indicating that these four MADRS items could be considered the best approximation of emotional blunting in studies that do not include a specific scale such as the ODQ.

The Japanese version of the ODQ demonstrated good internal consistency and test–retest reliability, while proving independent of demographic characteristics, showing generalizability, and further strengthening its potential role in routine clinical practice in Japan. As with the original English version,⁹ the overall Cronbach’s α coefficient was ≥0.9 indicating that excellent internal consistency was maintained with translation and use in Japan. Of note, the Cronbach’s α coefficient for the “not caring” domain was slightly lower than for the other domains (0.65) due to the inclusion of item 16 “I do not care as much about my day-to-day responsibilities as I did before I developed my illness/problem.” This moderate level of consistency remains acceptable but contrasts with the internal consistency of original English version.⁹ An important difference between our study population and previous studies is that we specifically recruited working patients, and the significant self-stigma surrounding depression at work that persists in Japan³⁵ may have impacted responses on this specific item. The exploratory factor analyses identified three factors, whereas Price et al⁹ in the original development in 207 patients and Chen et al¹⁹ in their Chinese validation in 312 patients identified four factors. The three factors identified based on an eigenvalue above one explained approximately 90% of the variance in our study, whereas the four factors in Price et al and Chen et al explained approximately 68% and 58% of the variance, respectively.

As in the previous validation study reported by Christensen et al,²¹ we observed a strong direct correlation of emotional blunting on the ODQ with depression severity as assessed by the MADRS, indicating that improvement in emotional blunting with vortioxetine treatment is likely reflected in improvement of overall symptom severity rather than the absence of side effects. Also, like prior studies, we observed a positive correlation with SDS scores, indicating a consistent impact of improving emotional blunting on functioning. Notably, the strength of the correlations between emotional blunting and function improved over time – likely reflecting the time it takes for functional improvement to become apparent following treatment.³⁶ Similarly, we observed a direct negative correlation of emotional blunting on ODQ with cognitive performance as assessed by DSST. Again, the strength of DSST correlations increased over time (correlations in the change from baseline in scores only reached significance at Week 24).

A secondary aim of the analyses was to map the individual MADRS items that reliably correlate with ODQ total score. We found that four items ([6] concentration, [7] lassitude, [8] inability to feel, and [4] reduced sleep) reached statistical significance in the Spearman correlation with ODQ total scores, with the sum of these four individual MADRS “emotional blunting” scores showing a good correlation with ODQ total score (ρ = 0.646, P<0.001). Similar work in anhedonia has identified a MADRS anhedonia factor score,³⁷ which has proven useful in post-hoc analyses to explore the effects of treatment in studies that have included in MADRS as an indicator of overall depressive severity but have not included symptom-specific scales.^38,39 Just as there is a phenotypic overlap between emotional blunting and anhedonia, there is overlap between the 4 MADRS items identified as important for emotional blunting and those included in the MADRS anhedonia factor score.³⁷ Specifically, items for concentration difficulties, lassitude, and the inability to feel are present in both clinical constructs. Indeed, the inability to feel pleasure is often clustered with concentration difficulties,⁴⁰ as well as physical and mental “fatigue”.⁴¹ However, while multiple linear regression analysis also identified MADRS sleep item as relevant to emotional blunting, it did not pick up items of apparent and reported sadness that are included in the anhedonia factor score. This highlights the key difference between the two symptoms – while anhedonia conceptually refers to a reduction in positive emotion, emotional blunting is defined as a restricted range of both positive and negative emotions. The significant correlation with sleep likely reflects the complex interplay between sleep and emotional regulation,^42,43 and there is evidence that insomnia reduces the intensity of emotions.⁴⁴ Further work with this potential emotional blunting subscale is required, but our analyses already show that it is likely to be more sensitive to emotional blunting than the single “inability to feel” item, which has recently been used to claim lack of emotional blunting with bupropion and SSRIs.¹³

Limitations of this analysis include the focus on employed patients with MDD when previous studies in Japan have suggested that people with higher socioeconomic status and full-time work may be reluctant to consult professionals and receive medical treatment.⁴⁵ Although we found that three factors explained 90% of the variance, our sample size was smaller than used to develop the original English version.⁹ Nevertheless, the results of this study indicate that the Japanese version of the ODQ retains the good validity and reliability of the original version.

Conclusion

In summary, our findings indicate that the Japanese version of the ODQ retains the good validity and reliability of the original English scale and can be recommended for prospective studies wanting to evaluate the effects of an intervention on emotional blunting in patients with MDD treated in the Japanese setting.