Validation of the Connor-Davidson Resilience Scale-10 in South Africa: Item Response Theory and Classical Test Theory

Introduction

A substantial body of recent research for example,^1,2 has confirmed an increase in clinically significant symptoms of psychological distress since the onset of the COVID-19 pandemic, including heightened levels of depression, anxiety and hopelessness. Specific aspects of the pandemic that may contribute to adverse mental health outcomes include social isolation, economic downturn and related job insecurity, loss of loved ones due to infection as well as stressors associated with school closures and the home schooling of children.^1,3 Furthermore, certain sub-groups of the population have been more vulnerable to adverse mental health outcomes owing to the distinctive impact of the pandemic on their work roles and responsibilities. School teachers represent one such population group.

The COVID-19 pandemic has had a profound impact on the educational sector and led to worldwide school closures and a rapid transition to remote online learning. Teachers have been at the coalface of this transition and have had to contend with the dramatic changes in the nature of their work role and tasks as well as the personal impact of the pandemic on their lives. According to the Job-Demands Resources Model of wellbeing in the work environment,⁴ these changes in the roles and responsibilities associated with the teaching profession (eg, upskilling in digital literacy and adapting the curriculum to suit online delivery formats) are job demands and represent a significant risk factor for work-related stress and adverse mental health outcomes. High levels of burnout, depression and anxiety among teachers have been documented since the onset of the pandemic.^5,6 However, individual resources (eg, sense of coherence and occupational skills) as well as job-related resources (eg, administrative supportive and latitude in decision-making) can serve to buffer the association between job demands and adverse outcomes and thereby facilitate coping.⁴ Resilience has been consistently identified as a salient personal resource that can contribute to adaptive functioning in the context of the pandemic.⁷

Resilience has been defined as a personality quality that promotes positive adaptation and coping in the face of trauma or adversity.⁸ It functions both to shield an individual from severe psychological distress and to promote a state of thriving and wellbeing. Studies of resilience have identified and described isolated characteristics associated with resilience as well as focused on how individuals identify, access, and navigate their way to health-sustaining resources. Some of these characteristics include high self-esteem, a strong sense of coherence (SOC) and the ability to identify and access social support.⁹ Various studies for example,¹⁰ have confirmed that SOC enables people to effectively navigate daily stressors as well as more severe experiences of adversity associated with the pandemic.^8,11 Diverse approaches to measuring resilience have been developed, each varying depending on the foci of resilience (eg, attitudes, cognitive appraisals, social support resources, etc.) and the method of assessment.¹² A critical review of scales that are most frequently used for the assessment of resilience identified the Connor-Davidson Resilience Scale (CD-RISC)¹³ as the most useful and robust of these measures.¹¹

The CD-RISC comprises 25 items that are scored on a 5-point scale.¹³ It focuses on an individual approach to conceptualizing resilience and addresses the shortcomings of previous scales.¹¹ Over the years, substantial normative data on the instrument from various population cohorts have been accumulated.¹⁴ The reliability and validity of the instrument have been supported by a range of studies conducted in diverse cultural contexts and in various samples, for example: India;¹⁵ Iran;¹⁶ Sweden.¹⁷

In the initial study, exploratory factor analysis resulted in a five-factor solution for the CD-RISC.¹³ However, various studies noted a divergence from this and reported a two-factor,¹⁸ three-factor,¹⁹ four-factor,²⁰ or unidimensional factor structure.^21,22 The CD-RISC-10 has been confirmed as psychometrically sound and proven to be an efficacious measuring instrument among numerous subgroups, eg, student nurses,²³ women with breast cancer,²⁴ depressive patients,²⁵ and non-professional caregivers.²⁶ In addition, the reliability and validity of the CD-RISC-10 have been extensively researched, predominantly using either confirmatory or exploratory factor analysis.^24,25,27 However, sufficient evidence confirming the instrument’s item independence and unidimensional structure remains lacking.¹² For this reason,¹² Heritage et al used the Rasch model approach to analyze the properties of the CD-RISC-10 and proposed a seven-item version of the scale. The Rasch model has certain advantages over other approaches including its ability to precisely predict the mathematical association between an item and a latent trait to be measured.¹² Furthermore, the model provides scale and item-level insights that can assist in the adaptation of instruments to suit specific research populations and settings. This can contribute to the development of a more consistent knowledge base on the psychometric properties of instruments.²⁸

In the current study, we build on prior research by focusing on the reliability, validity and dimensionality of the CD-RISC-10 when used in South Africa among a sample of school teachers, using both item response theory (IRT) and classical test theory (CTT). With respect to IRT, we used both a parametric (Rasch analysis), and a nonparametric (Mokken analysis) approach. The study was undertaken during the third wave of the 2019 novel coronavirus (COVID-19) pandemic in South Africa from July 2021 to September 2021. It has been well established that resilience differs among people and contexts and according to the nature of the stressor experienced.²⁹ The current pandemic offers an unprecedented opportunity to investigate the performance of standardized instruments, such as the CD-RISC-10, that have been predominantly validated using normative data from populations that have faced isolated stressors, such as physical illness (e.g.,²⁴) or exposure to trauma (e.g.,³⁰).

In general, combining CTT, Mokken analysis and Rasch analysis permit a more comprehensive overview of an instrument.³¹ CTT provides for the computation of a total score for resilience and offers a summative view of the respondents’ level of resilience, while the diagnostic capabilities of Mokken and Rasch analyses facilitate the identification of items with an increased likelihood of being selected by respondents who have different levels of resilience.^31–33 This can assist with the development of more accurate profiles of respondents as well as criterion-referenced interpretations.³³ This kind of information can be considered important in identifying the levels of resilience in different population groups and aid in the design of more responsive intervention efforts.³⁴

Materials and Methods

Participants

The study sample comprised a convenience sample of school teachers in South Africa (N = 355). Reportedly there are about 444, 900 teachers in South Africa,³⁵ and thus, a sample of 355 represents a 5.09% margin of error, with a confidence interval of 95%. Most of the respondents were female (76.6%), lived in the Western Cape (82.3%) and in urban areas (61.7). Most of them taught grades one to seven (61.1%). The mean age of the sample was 41.9 years (±12.4) and participants reported being in the teaching profession for an average of 15.7 years (±11.8). The sample may also be considered somewhat representative as it aligns very closely with population data reported in the OECD Teaching and Learning International Survey.³⁶ This international survey found that 60% of teachers in South Africa are women (60%), on average 43 years old with an average of 15 years work experience.

Instruments

Participants completed a demographic survey, the CD-RISC-10,²² the Center for Epidemiological Studies Depression Scale (CES-D),³⁷ the Beck Hopelessness Scale (BHS),³⁸ the Sense of Coherence Scale (SOC-13),³⁹ and the State-Trait Anxiety Inventory-Trait Scale (STAI-T).⁴⁰ The CD-RISC comprises 10 items, each of which are scored on a 5-point scale ranging from 0 (not true at all) to 4 (true nearly all of the time). Examples of items include, “I am able to adapt when changes occur” and “I am not easily discouraged by failure.” Higher scores on the CD-RISC-10 indicate higher levels of resilience. The reliability of the CD-RISC-10, as reported in the literature, can be considered satisfactory.^22–24

The SOC-13 is comprised of 13 items that assesses the capacity to identify and utilize intrinsic and extrinsic resources to successfully negotiate stressors and maintain health. This scale is comprised of three subscales: meaningfulness (four items), comprehensibility (five items), and manageability (four items). In the current study, the total scale was used as a global measure of sense of coherence. Examples of items include, “Do you have the feeling that you are in an unfamiliar situation and don’t know what to do?” and “How often do you have the feeling that there’s little meaning in the things you do in your daily life?” The SOC-13 are scored on a 7-point Likert scale. Higher scores on the SOC-13 reflect a stronger sense of coherence. The internal consistency estimates for this scale have been reported to range from α = 0.70 to α = 0.92.⁴¹

There are 20 items in the CES-D and it is used to screen for depression. It consists of seven items that focus on somatic symptoms, seven items that focus on depressed affect, four items that focus on positive affect, and two items that focus on interpersonal problems. Examples of items include “I enjoyed life” and “I could not get going.” Responses range from Rarely or None of the time (0) to All of the time (3). Higher scores on the CES-D indicate higher levels of depression. The estimates of reliability for this scale have been reported to be satisfactory (α = 0.70 to 0.90).⁴² The CES-D has also demonstrated acceptable reliability with a student sample in a prior South African study.⁴³

The BHS comprises 20 items that assess degree of hopelessness where responses are dichotomized as true or false. Examples of items include “I don’t expect to get what I really want” and “My future seems dark to me.” Higher scores on the BHS indicate higher levels of hopelessness. For this instrument,²⁸ Medvedev et al reported a satisfactory reliability coefficient of 0.93. The measure was also validated against a clinical assessment of hopelessness, and a validity coefficient of 0.74 was reported.²⁸ In South Africa, the BHS was used with a student sample and a reliability coefficient of 0.86 was reported.²

The STAI-T comprises 20 items that assess trait anxiety. Sample items from the STAI-T include: “I worry too much over something that really doesn’t matter” and “I am content; I am a steady person.” The items of the STAI-T are scored on a 4-point scale ranging from 1 (Almost Never) to 4 (Almost Always). Higher scores on the STAI-T indicate higher levels of anxiety. In general, internal consistency coefficients for the STAI-T have been reported as satisfactory (eg, α = 0.83).⁴⁴ A South African study, with young adults, also reported a satisfactory estimate of reliability (α = 0.88).⁴⁵

Procedure

Google Forms were used to generate a web-based survey consisting of the various scales as well as a demographic survey. The link was disseminated to school teachers between April and July 2021, using social networking sites such as Facebook. Institutional networks were used to further disseminate the questionnaire to prospective participants.

Ethics

The study was conducted in accordance with the Declaration of Helsinki. The institutional review board of the University of the Western Cape (ethics reference number: HS21/3/8) granted ethical approval for the study. Participants provided informed consent.

Data Analysis

The sample size was considered adequate for the purposes of exploratory factor analysis (EFA) and confirmatory factor analysis (CFA),⁴⁶ as well as Mokken analysis,⁴⁷ and Rasch analysis.⁴⁸

The CTT analyses included the determination of reliability (coefficient α), composite reliability (CR), average variance extracted (AVE), item-total correlations, inter-item correlations, standard error of measurement (SEM), EFA (principal components) and CFA. All of the CTT analyses, with the exception of CFA, were conducted with IBM SPSS Statistics (version 26). CFA was performed using IBM SPSS Amos (version 26) in order to examine the concurrent validity and dimensionality of the CD-RISC-10. In general, a reliability coefficient greater than 0.70, an AVE greater than 0.50, a small SEM, item-total correlations greater than 0.50 and inter-item correlations greater than 0.30, were considered acceptable.^49,50 Several fit indices were used to examine the dimensionality of the CD-RISC-10. In addition, concurrent validity was investigated through the associations between the CD-RISC-10 and sense of coherence, depression, hopelessness, as well as anxiety. It was hypothesized that resilience would be positively associated with sense of coherence, which is also regarded as a protective resource in the literature, and negatively associated with the indices of psychological distress. As suggested by Kline,⁵¹ the selected fit indices for the CFA included the comparative fit index (CFI; should be ≥0.90), the goodness-of-fit index (GFI; should be ≥0.95), the root-mean-square error of approximation (RMSEA; should be ≤0.08), and the Tucker–Lewis index (TLI; should be ≥0.95).⁵²

Winsteps (version 5.1.4)⁵³ was used to conduct the Rasch analysis. The Rasch indices that were examined were the item and person separation index, item and person reliability and the infit and outfit mean square (MnSq)⁵⁴ indicates that mean square values should ideally be between 0.5 and 1.5 indicating a good fit. In addition, a principal component analysis (PCA) of the residuals was conducted to determine the eigenvalue of the unexplained variance of a potential second dimension. With regards to dimensionality, if the eigenvalue of a possible second dimension is less than 2, it confirms that the instrument is unidimensional.⁵⁴ To demonstrate that the items can distinguish between different respondents, the person separation index should be greater than 2 and have a reliability greater than 0.8. To confirm the item hierarchy of the instrument, the item separation index should be greater than 3, while item separation reliability should be greater than 0.8. If the item hierarchy is confirmed, it is regarded as indicative of construct validity. To examine measurement invariance across gender, differential item functioning (DIF) was used. In general, DIF values greater than 0.5 suggest substantial DIF.⁵⁴

Mokken analysis was conducted with the “Mokken” package^55,56 in R.⁵⁷ This includes the determination of dimensionality, reliability (MS_Rho), invariant item ordering (IIO), and monotonicity. With respect to dimensionality, Mokken analysis determines a scalability coefficient for every item (H_i) of the scale as well as for the total scale (H). When evaluating H-coefficients, it is suggested that an H-value greater than 0.50 indicates a strong scale, while between 0.40 and 0.50 indicates a medium scale, and an H less than 0.40 indicates a weak scale.⁵⁸ The Mokken package uses an automated algorithm (ie, automated item selection procedure [AISP]) to search for unidimensional scales, which indicates whether a scale is unidimensional or multidimensional.⁵⁶ Monotonicity implies that as the levels of the latent variable increase the likelihood of positively endorsing items should also increase. The Mokken software package provides a Crit value that indicates whether significant violations of monotonicity have occurred. When there are no violations, it would indicate that the items effectively distinguish respondents with high compared to those with low levels of resilience.⁵⁶ IIO indicates the extent to which the items are endorsed in the same order at all levels of the latent variable.⁵⁹ The Mokken package indicates the significance of violations of IIO for each item, which is also called the Crit value. If an item is identified as violating the assumption of IIO, it would suggest that participants that have the same level of the underlying latent variable (resilience) have responded to the particular item in different ways. It is suggested that a Crit value for both monotonicity and IIO > 80 indicate serious violations, while a Crit value of between 40 and 80 indicates minor violations that are acceptable.⁵⁹

Results

Table 1 reports the intercorrelations, reliability (Cronbach’s alpha), and descriptive statistics of all the variables. The reliability of all the scales can be regarded as satisfactory (α > 0.80). In terms of intercorrelations, resilience was positively associated with sense of coherence (r₃₅₃ = 0.45, p < 0.001), and negatively related to the indices of psychological distress: depression (r₃₅₃ = −0.48, p < 0.001), hopelessness (r₃₅₃ = −0.48, p < 0.001), and anxiety (r₃₅₃ = −0.53, p < 0.001).

Table 1 Descriptive Statistics, Reliabilities and Intercorrelations Between Study Variables

In terms of mean scores, Table 1 indicates that the mean resilience score of the current sample was 26.9 ± 8.0. This is not significantly different from the mean scores reported in the original study of the CD-RISC-10 (Mean = 27.2 ± 5.8, t₃₅₄ = −0.70, p = 0.484)²² or a study among German students (Mean = 27.6 ± 5.4, t₃₅₄ = −1.62, p = 0.107).⁶⁰ It is, however, significantly lower than the mean resilience scores reported for a sample of competitive sports participants (Mean = 31.14 ± 5.4, t₃₅₄ = −9.96, p < 0.001).⁶¹

The inter-item correlations are reported in Table 2, while the CTT and IRT indices at the item level are reported in Table 3. This includes item-total correlations, factor loadings, MnSq values, Crit values for the assumptions of monotonicity and IIO, and the H_i-coefficients for each of the items of the CD-RISC-10. Table 2 reflects that the inter-item correlations ranged between 0.55 and 0.78, and thus were above the suggested cutoff (0.30), which provides evidence of construct validity.

Table 2 Inter-Item Correlations for the CD-RISC-10

Table 3 Classical Test Theory, Rasch, and Mokken Indices for the CD-RISC-10 at the Item Level

The EFA extracted one factor that explained 68.5% of the variance. Table 3 indicates that the factor loadings of the extracted factor ranged between 0.78 and 0.88, while the item-total correlations ranged between 0.72 and 0.84. Both the factor loadings and the item-total correlations were significant, which indicates that all of the items make a contribution towards the measurement of the latent variable. The MnSq values (infit and outfit) fell within the suggested range (0.5–1.5) since they ranged from 0.70 to 1.3. All the H_i indices were above the proposed criterion of 0.30⁶² and ranged between 0.65 and 0.75. These H_i values further indicate that all of the items make a contribution to the overall assessment of resilience.

The AISP in the Mokken analysis identified one dimension. For each item, the scalability coefficients (H_i) ranged between 0.65 and 0.75 (SE = 0.02 to 0.03), and for item pairs, the scalability coefficients (H_ij) ranged between 0.56 and 0.85 (SE = 0.03 to 0.05). The use of all of the items of the CD-RISC-10 as a Mokken scale was supported by these H coefficients. Based on a numerical and visual assessment, there were no significant violations of monotonicity as only one item, namely “Tried to see humorous side of problems” had a Crit value of 2. This is well below the value of 80 which is considered to be indicative of a serious violation. With respect to item ordering, IIO indicated violations related to items 1, 5, 6, and 9. However, as reflected in Table 2, the Crit values associated with these four items are below the suggested value of 80. The instrument also demonstrated measurement invariance across gender, as DIF for all items were <0.5.

To examine a one-factor model of the CD-RISC-10, CFA was employed. Figure 1 demonstrates the hypothesized one-factor model of the CD-RISC-10, where the 10 items of the instrument are the observed measurements, and resilience is presented as a latent variable on which these 10 items load. As suggested by the modification indices in Amos, the error variables for items 9 and 10, as well as 5 and 6, were allowed to covary. To assess concurrent validity, sense of coherence, depression, hopelessness, and anxiety were included in the CFA model to establish an association between these variables and resilience.

Figure 1 One-dimensional model of the CD-RISC-10. Notes: All regression weights are standardized. The rectangles are measured variables, whereas the ellipse is a latent construct. All paths are significant p < 0.001.

The fit indices obtained with CFA are presented in Table 4. All of the fit indices exceeded the proposed cut-off points (CFI = 0.983, GFI = 0.950, RMSEA = 0.051, and TLI = 0.977), which points to a one-factor model fitting the data appropriately. Furthermore, the regression weights were all above 0.50 (between 0.76 and 0.85) and were significant. This supports the EFA factor loadings and item-total correlations, which indicates that all items make a contribution to the measurement of resilience. Resilience was also positively associated with sense of coherence (β = 0.46, p < 0.001) and negatively associated with depression (β = −0.49, p < 0.001), hopelessness (β = −0.49, p < 0.001), and anxiety (β = −0.54, p < 0.001).

Table 4 Fit Indices for One-Factor Model of the CD-RISC-10

The IRT and the CTT indices for the CD-RISC-10, at the scale level, are presented in Table 5. The indices of reliability are satisfactory (α = 0.948, MS_Rho = 0.951, CR = 0.956) and AVE exceeded 0.50 (AVE = 0.684), which is considered satisfactory. The calculated SEM can also be regarded as low (SEM = 1.79). The item and separation reliability values were acceptable (0.91 and 0.93), as were the item and person index values (3.25 and 3.75). The H coefficient was 0.64, which exceeds the threshold for a strong scale (ie, H > 0.50) as suggested in the literature.⁵⁹ A PCA of the residuals found that the Rasch dimension explained 65.1% (eigenvalue = 18.68) of the variance, while a possible second dimension accounted for only 6% of the variance (eigenvalue = 1.73).

Table 5 Classical Test Theory, Rasch, and Mokken Indices for the CD-RISC-10 at the Scale Level

Discussion

The current study focused on extending the existing knowledge base on the CD-RISC-10 through an examination of its psychometric properties. This entailed an analysis of the instrument’s validity, reliability, and dimensionality, through the use of Rasch and Mokken analyses as well as CTT. The use of such complementary methods of analyses has the potential to offer information that could address ambiguities regarding the instruments’ psychometric properties and provide opportunities for further enhancements.¹² Due to the respective strengths and limitations of each of these approaches, integrating them offers a holistic overview of the validity, reliability, and dimensionality of the CD-RISC-10.³¹

Overall, the reliability indices found in the study can be considered satisfactory. The CD-RISC-10 exhibited acceptable internal consistency in terms of coefficient α and MS_Rho. Rasch analysis demonstrated that there is a hierarchical ordering of items (ie, item separation reliability) and that the scale can differentiate different levels of “performers” (ie, person separation reliability). This type of information is particularly beneficial for targeted intervention strategies. Given the physical and mental toil exacted by the COVID-19 pandemic, identifying individuals who are less resilient is beneficial in facilitating better planning for resource allocation and allows for interventions to be directed toward more vulnerable populations.

In terms of the results of EFA and CFA, the CD-RISC-10 can be regarded as a single dimension. A PCA of residuals in the Rasch analysis indicated that the Rasch dimension accounted for a significant amount of the variance with a possible second dimension having an eigenvalue below 2. The AISP in the Mokken analysis provided evidence that all the items loaded on one latent factor. It also demonstrated homogeneity in that the scalability coefficients (H and H_i) were all above the suggested threshold. Overall, the results of CTT, Rasch and Mokken analyses provided sufficient support for the unidimensionality of the CD-RISC-10. Ghadi et al⁵⁰ proposes that CFA factor loadings, the CR, and the AVE assist with the determination of convergent validity. The factor loadings were all above 0.76 and all were significant. The AVE of 0.60 and the CR of 0.91 offer further support for convergent validity.

The current findings also support the construct validity of the CD-RISC-10. In this regard, item-total correlations of >0.50 and inter-item correlations of >0.30 supports construct validity.⁴⁹ Both the item-total correlations as well as the inter-item correlations exceeded these suggested guidelines. Further evidence for construct validity is provided by the item separation indices in the Rasch analysis. For instance, Linacre⁵⁴ suggested that when the item separation index is greater than 3 and the item separation reliability is greater than 0.9. It indicates the existence of an item difficulty hierarchy. The item separation indices obtained exceeded these thresholds. In addition, the MnSq values were also within an acceptable range that offers additional support for validity. The association between resilience and sense of coherence as well as indices of psychological distress provide evidence of concurrent validity. As expected, there was a positive association between resilience and sense of coherence, while resilience was negatively associated with indices of psychological distress.

In terms of implications, the current findings suggest that the CD-RISC-10 is a reliable and valid, unitary scale for the measurement of resilience that can be used in developing contexts for research purposes and assessment in clinical settings. Given that resilience is a central resource in managing adversity, it is important to adapt resilience-promoting strategies to suit the needs of people within their context. The current study assessed the psychometric properties of the CD-RISC-10 among South African school teachers. They represent a population group that has been distinctively impacted by the pandemic and are more vulnerable to burnout and adverse mental health outcomes.⁵ According to the Job-Demands Resources Model,⁴ the best way to mitigate negative outcomes associated with work stress is to decrease the demands of the job by implementing additional support structures (eg, increased administrative support) as well as identify and amplify existing internal resources such as resilience. Culturally responsive interventions that can enhance resilience during the COVID-19 pandemic have been documented and include cognitive-behavioral interventions, meditation to target depressive symptoms, acceptance-based coping, and strategies to enhance the use of social support networks.⁶³ Future research investigating the efficacy of these approaches in enhancing resilience in sub-Saharan African contexts would be useful for planning interventions.

Limitations

There were limitations to the study. First, the study used self-report measures which might lead to self-reporting bias.⁶⁴ Second, the findings from the sample surveyed may not be generalizable to school teachers in other settings. These results need to be replicated in broader and more diverse samples.

Conclusion

In developing countries like South Africa, there is a substantial body of research assessing resilience. However, very few of these studies examine the psychometric properties of instruments used to measure resilience. To produce valid cross-cultural comparisons, assessing the internal consistency, validity and dimensionality of instruments remains imperative. This study found the CD-RISC-10 to be a unidimensional scale with satisfactory reliability and validity when used in the South African context. These findings contribute to extending the literature base on cross-culturally validated measures of resilience.