Construct validity and reliability of the Tachikawa Resilience Scale in the Japan Ground Self-Defense Force personnel

Introduction

Experience of extremely stressful events can affect people’s mental health and cause depression, posttraumatic stress disorder, or other psychiatric disorders.¹ Resilience, which is defined as the ability to maintain a stable equilibrium, has been proposed as a recovery factor against stress.² The construct of resilience and its factors are complex and may be specific for a particular individual, family, organization, society, and culture.¹ The importance of resilience has been addressed in the domains of occupational health and military health – military personnel may experience severe events during their deployment, including combat and other life-threatening situations, as well as witnessing atrocities.³

The best way to evaluate resilience has yet to be determined. Resilience rating scales widely used around the world were typically created in Europe and the United States, such as the Resilience Scale (RS) and the Connor–Davidson Resilience Scale.^4,5 Although there is a Japanese version of the RS,⁶ it has not exhibited the same utility as the original because the validity of this Japanese version has been considered low compared to that of the original.⁶ One reason for this is a difference in the cultural sensitivity to resilience between Japan and Western countries. Accordingly, the Tachikawa Resilience Scale (TRS) was developed to measure the resilience of Japanese populations.⁷ However, to date, the subjects of TRS reports have been limited to company workers and patients with psychiatric disorders.⁷

In the context of military mental health, the Resilience Competence Scale (RCS) has been used to verify the effect of resilience training on US Army members.³ Although the validity of the RCS – Japanese Short Version (RCS-JS) has been established in the Japan Ground Self-Defense Forces (JGSDF),⁸ the original version was developed in the United States and does not reflect the Japanese cultural background.

The JGSDF has been engaged in a large number of United Nations peacekeeping operations and disaster relief activities. Through these JGSDF activities, the personnel are potentially exposed to traumatic events, and it is important to gather knowledge about the resilience of personnel. However, to the best of our knowledge, there are no established standard scales by which to measure resilience in the JGSDF.

The primary endpoint of this study was to examine the factor structure of the TRS with exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). The secondary endpoint was to examine the construct validity of the TRS and compare it to that of the RCS-JS and the Japanese version of the Kessler Screening Scale for Psychological Distress (K6)⁹ using a correlation analysis and a hierarchical multivariate regression analysis.

Materials and methods

Participants

Participants were all JGSDF personnel who were deployed to a peacekeeping mission – the United Nations Mission in the Republic of South Sudan – from November 2015 to May 2016 (n=353). Unlike other countries’ troops, a very large majority of the JGSDF personnel is Japanese. In addition, the country has not experienced traumatic events because of its constitutional constraints, and the JGSDF activities are limited to noncombat areas. The United Nations Mission in the Republic of South Sudan was tasked to consolidate peace and security and to help establish conditions for the development of South Sudan.

The exclusion criteria for this study consisted of refusal to receive analysis. Of the 353 JGSDF personnel, 281 individuals (79.6%) agreed to participate in this study. The self-reported questionnaire was administered in January 2016 – during their mission engagement.

We collected the following demographic information: age, sex, military rank (officer, enlisted/private), marital status, whether the individual had children, and previous deployment experience. In previous studies, older age, the existence of a spouse or child, and previous deployment experience corresponded to good mental health,^10,11 while being female or of a lower rank did not.^12–15

Measures

We used two psychological measures to evaluate resilience: the TRS and the RCS-JS. The TRS is a 10-item self-administered questionnaire used to evaluate resilience. Response are obtained via a 7-point scale ranging from 1 (strongly disagree) to 7 (strongly agree); the total score ranges from 10 to 70.⁷ Higher scores reflect higher resilience. The original text of the TRS is in the Japanese language, and several items reflect Japanese culture-bound cognitions. For example, items such as “I accept things as they are when there are no alternatives” and “I try not to worry about what is beyond my capabilities” can be regarded as culturally appropriate for Japanese individuals because these items reflect the idea of Morita therapy, which guides patients to accept anxiety as it is.⁷

The RCS-JS is a 12-item self-administered questionnaire used to evaluate resilience. Responses are obtained via a 5-point scale ranging from 1 (disagree) to 5 (agree), with a total score ranging from 12 to 60.⁸ The original RCS is a 20-item self-administered questionnaire with a total score ranging from 20 to 100.³ Higher scores reflect higher resilience. The validity of the RCS was reported in a study that examined the efficacy of the US Army’s resilience training program among US Army National Guard soldiers. The program addressed six core competencies of resilience: connection, optimism, mental agility, self-awareness, self-regulation, and character strength. The RCS was created to assess these six competencies.³ The study showed that the developed measure of Master Resilience Training (MRT) core competencies had adequate reliability and validity for purposes of examining the intended effects of the MRT training.³

We evaluated the participants’ degree of psychological distress using the Japanese version of the K6.⁹ The K6 is a 6-item self-administered questionnaire that is widely used to screen for mood and anxiety disorders according to the definition in the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV).^13,16 K6 responses are collected via a 5-point scale ranging from 0 (never) to 4 (always), with a total score ranging from 0 to 24. Higher scores reflect higher psychological distress.

Statistical analyses

A Mann–Whitney U-test was used to compare the averages of continuous data on demographic characteristics. EFA was used to determine the factor structure of the TRS. A principal components analysis, Kaiser’s criterion, and a scree test were used to determine the number of factors. We subsequently performed CFA to verify the factor structure of the TRS using structural equation modeling. The model fit of the CFA was evaluated using the following indicators: the ratio of χ² to the degrees of freedom (CMIN/df), the goodness of fit index (GFI), the comparative fit index (CFI), and the root mean square error of approximation (RMSEA). The acceptable threshold levels for these indicators were as follows¹⁷: CMIN/df <3,¹⁸ GFI and CFI >0.95,¹⁷ and RMSEA <0.07.¹⁹

Spearman correlations were calculated to evaluate the relationship between the TRS and other theoretically related measures, namely the RCS-JS and the K6. A Z-test was conducted to evaluate the difference between the TRS and the RCS-JS correlation coefficients. We subsequently conducted hierarchical multivariate regression analysis to evaluate the relationship between the TRS and the RCS-JS with the K6 (dependent variable). In Step 1, age, sex, military rank (officer, enlisted/private), marital status, having children, and previous deployment experience were entered as covariates. In Step 2A, the TRS was entered as a covariate, and in Step 2B the RCS-JS was entered likewise.

We assessed the reliability of the TRS by examining its internal consistency using Cronbach’s α coefficient. All statistical analyses were conducted using IBM SPSS Statistics 24 (IBM Corporation, Armonk, NY, USA) and IBM Amos 24 (IBM Corporation).

Results

Table 1 shows the demographic characteristics of the deployed personnel and scale scores of the TRS, RCS-JS, and K6. The total score of the TRS was significantly higher for males (median, IQR; 53.0, 46.5–59.5) than for females (46.5, 43.3–53.0) (P=0.048). The total score of the RCS-JS was significantly higher for personnel who had previous deployment experience (median, IQR; 47.0, 43.0–55.0) than for those who did not (46.0, 41.0–51.0) (P=0.035).

Table 1 Demographic characteristics of deployed personnel and scores of the TRS and other measures Abbreviations: IQR, interquartile range; K6, The Kessler Screening Scale for Psychological Distress; RCS-JS, The Resilience Competence Scale – Japanese Short Version; TRS, The Tachikawa Resilience Scale.

Construct validity

The EFA revealed only one factor with an eigenvalue (5.34) >1. This factor included 53.4% of the variance. The scree plot showed that one factor was suitable (Figure 1). The factor loadings were between 0.58 and 0.78 for the ten resilience items.

Figure 1 Scree plot of the TRS. Notes: EFA revealed only one factor with an eigenvalue (5.34) >1. This factor included 53.4% of the variance. This scree plot shows that one factor was suitable. Abbreviations: EFA, exploratory factor analysis; TRS, The Tachikawa Resilience Scale.

The CFA model showed good fit properties (CMIN =28.178, df=20, CMIN/df=1.409, P=0.105, GFI =0.981, CFI =0.994, RMSEA =0.038, and AIC =98.178). The individual item loadings for the factors are shown in Table 2.

Table 2 Means, standard deviations, correlations with the TRS total score, Cronbach’s α if item was deleted, and factor loading of each item of the TRS Note: ρ = Spearman’s rank correlation coefficient. Abbreviation: TRS, The Tachikawa Resilience Scale.

The total score of the TRS was significantly associated with that of the K6 (r=−0.499, P<0.001). The total score of the RCS-JS was also significantly associated with that of the K6 (r=−0.484, P<0.001). There was no significant difference between these two correlation coefficients (z=0.233, P=0.816). The total score of the TRS was significantly associated with that of the RCS-JS and its six domains (Table 3). The multivariate regression analysis revealed that the TRS and the RCS-JS were negatively related to the K6. The regression coefficient of the TRS was not lower than that of the RCS-JS (Table 4).

Table 3 Correlation between the TRS and the RCS-JS Note: ρ = Spearman’s rank correlation coefficient. Abbreviations: RCS-JS, The Resilience Competence Scale – Japanese Short Version; TRS, The Tachikawa Resilience Scale.

Table 4 Results of hierarchical multivariate regression analysis Notes: a0= male, 1= female. b0= officer, 1= enlisted/private. c0= unmarried, 1= married. d0= no, 1= yes. e0= no, 1= yes. Abbreviations: K6, The Kessler Screening Scale for Psychological Distress; RCS-JS, The Resilience Competence Scale – Japanese Short Version; TRS, The Tachikawa Resilience Scale.

Reliability

The Cronbach’s α coefficient of the 10-item TRS was 0.90, showing very good internal consistency. Elimination of any question item(s) did not significantly increase this value (Table 2).

Discussion

According to the present study, the construct validity and reliability of the TRS were acceptable in a sample of JGSDF personnel. The results supported a one-factor model of the TRS and showed good internal consistency. The TRS was inversely correlated with the K6. To the best of our knowledge, this is the first study to show this relationship within the personnel of the JGSDF or other services of the Japan Self-Defense Forces.

The TRS regression coefficients were comparable to those of the RCS-JS, even though the TRS questions differ slightly from those of the RCS-JS, which consists of six domains (connection, optimism, mental agility, self-awareness, self-regulation, and character strength). Some items show similarities between the scales; for example, we can regard two TRS items (“Even during hardships, I think I will be able to manage” and “Even during hardships, I can maintain a normal state of mind”) as similar to optimism and mental agility items, respectively, of the RCS-JS. In contrast, no TRS item corresponds to connection in the RCS-JS. The RCS defines connection as an “ability to develop and maintain strong interpersonal relationships through effective communication, empathy toward others, willingness to ask for help, and supporting others.”³ Therefore, this item represents social support, which is an important component of resilience.²⁰ We speculate that the link between the TRS and the Japanese cultural norms compensates for the lack of an item that corresponds to social support. However, future studies will be needed to verify this speculation.

Another characteristic of Japanese culture is a tendency to suppress the expression of positive affect.²¹ Therefore, Japanese individuals might tend to generate low scores on some RCS-JS items (eg, “Confident in handling stressful circumstances” or “Control my emotions”). In contrast, the TRS items that reflect the idea of Morita therapy⁷ might be answered naturally by Japanese individuals. TRS items nos 2 and 7 (eg, “I accept things as they are when there are no alternatives” and “I try not to worry about what is beyond my capabilities”) could be culturally sensitive, as mentioned above, and their factor loadings are 0.58 and 0.69 (Table 2). Contrary to expectations, these values were not very high, which could reflect the cultural implications of the scale. Future studies should assess these cross-cultural comparisons between the TRS and other resilience scales of Western origin. Convergent validity was shown by the significant correlation between the TRS and the RCS-JS (Table 3). The correlation coefficient between the TRS and connection in the RCS-JS was relatively low compared to the other five domains. This result was considered to be due to the abovementioned lack of the TRS item that corresponds to social support.

To date, the utility of the TRS has been reported with company workers and patients with psychiatric disorders.⁷ In this study, we showed the usefulness of the TRS for the JGSDF personnel. Therefore, we could use the knowledge about resilience obtained from populations other than the military by using the TRS. In addition, we could also apply the findings gained from the military to other populations.

Several limitations of this study should be acknowledged. First, the theoretical rationale for the TRS may not be strong, because it was developed on the basis of data obtained from unstructured interviews with motor vehicle accident survivors.⁷ Nonetheless, the TRS contains cultural aspects, which influence the way that resilience is constructed.^1,7 Second, there is no item concerning social support in the TRS. Third, there was selection bias in this study because the participants were limited to dispatched members of the JGSDF. Fourth, we conducted the EFA and the CFA using the same sample group because of the sample size, which could have led to similar results between the measures. Fifth, there was lack of comparison to measures that evaluate resilience other than the RCS-JS. Finally, the study design did not allow us to determine any causal associations between resilience and mental health status.

Conclusion

We established the construct validity and reliability of the TRS when applied to the JGSDF. Therefore, the TRS could be a useful resilience evaluation scale for the JGSDF. In the future, a more valid and reliable resilience evaluation scale could be created for JGSDF personnel by adding items related to social support to the TRS.

Ethics statement

This research followed the principles outlined in the Declaration of Helsinki. This study was conducted with the approval of the Ethics Committee of the National Defense Medical College (Approval No 2404). All participants provided written informed consent.

Acknowledgment

We thank all the participants and medical staff of the JGSDF, who supported our research. The views expressed in this article are those of the authors and do not reflect the position or policy of the National Defense Medical College, Japan Ministry of Defense, or the JGSDF. This work was partly supported by the Japanese Society for the Promotion of Science KAKENHI Grant Number 26461779 and 17K09181. We would like to thank Editage (www.editage.jp) for English language editing.

Author contributions

All authors conceived the study concept and design. TS, MN, JS, and MT compiled and synthesized the data. TS, MN, JS, and HT carried out statistical analyses. KS and AY supervised the study project. All authors participated in interpretation of the results and writing the report, approved the final version, and agreed to be accountable for all aspects of the work.

Disclosure

The authors report no conflicts of interest in this work.

---

References

1.		Southwick SM, Bonanno GA, Masten AS, Panter-Brick C, Yehuda R. Resilience definitions, theory, and challenges: interdisciplinary perspectives. Eur J Psychotraumatol. 2014;5:25338.
2.		Bonanno GA. Loss, trauma, and human resilience: have we underestimated the human capacity to thrive after extremely aversive events? Am Psychol. 2004;59(1):20–28.
3.		Griffith J, West C. Master resilience training and its relationship to individual well-being and stress buffering among army national guard soldiers. J Behav Health Serv Res. 2013;40(2):140–155.
4.		Wagnild GM, Young HM. Development and psychometric evaluation of the Resilience Scale. J Nurs Meas. 1993;1(2):165–178.
5.		Connor KM, Davidson JR. Development of a new resilience scale: the Connor-Davidson Resilience Scale (CD-RISC). Depress Anxiety. 2003;18(2):76–82.
6.		Nishi D, Uehara R, Kondo M, Matsuoka Y. Reliability and validity of the Japanese version of the Resilience Scale and its short version. BMC Res Notes. 2010;3:310.
7.		Nishi D, Uehara R, Yoshikawa E, Sato G, Ito M, Matsuoka Y. Culturally sensitive and universal measure of resilience for Japanese populations: Tachikawa Resilience Scale in comparison with Resilience Scale 14-item version. Psychiatry Clin Neurosci. 2013;67(3):174–181.
8.		Terada T, Kawano H, Nagamine M. Development of the Resilience Competency Scale Japanese version. Paper presented at: Japanese Society for Traumatic Stress Studies 15th Annual Meeting; May 20–21; 2016; Sendai.
9.		Furukawa TA, Kawakami N, Saitoh M, et al. The performance of the Japanese version of the K6 and K10 in the World Mental Health Survey Japan. Int J Methods Psychiatr Res. 2008;17(3):152–158.
10.		Klose M, Jacobi F. Can gender differences in the prevalence of mental disorders be explained by sociodemographic factors? Arch Womens Ment Health. 2004;7(2):133–148.
11.		Sudom KA, Lee JEC, Zamorski MA. A longitudinal pilot study of resilience in Canadian military personnel. Stress Health. 2014;30(5):377–385.
12.		Van Hooff M, Mcfarlane AC, Davies CE, et al. The Australian Defence Force Mental Health Prevalence and Wellbeing Study: design and methods. Eur J Psychotraumatol. 2014;5(1):23950.
13.		Kessler RC, Andrews G, Colpe LJ, et al. Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychol Med. 2002;32(6):959–976.
14.		Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, Walters EE. Lifetime Prevalence and Age-of-Onset Distributions of DSM-IV Disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005;62(6):593–602.
15.		Kessler RC, Sonnega A, Bromet E, Hughes M, Nelson CB. Posttraumatic Stress Disorder in the National Comorbidity Survey. Arch Gen Psychiatry. 1995;52(12):1048–1060.
16.		American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (DSM-IV). 4th ed. Washington, DC: American Psychiatric Association; 1994.
17.		Hooper D, Coughlan J, Mullen M. Structural equation modelling: Guidelines for determining model fit. Electron J Business Res Meth. 2008;6(1):53–60.
18.		Kline RB. Principles and Practice of Structural Equation Modeling. 4th ed. New York: Guilford Press; 2005.
19.		Steiger JH. Understanding the limitations of global fit assessment in structural equation modeling. Pers Individ Dif. 2007;42(5):893–898.
20.		Luthar SS, Cicchetti D, Becker B. The construct of resilience: a critical evaluation and guidelines for future work. Child Dev. 2000;71(3):543–562.
21.		Iwata N, Roberts CR, Kawakami N. Japan-U.S. comparison of responses to depression scale items among adult workers. Psychiatry Res. 1995;58(3):237–245.