Developing and Validating an Adjustment Scale: The Adaptation Status Assessment of Drug-Resistant Tuberculosis Patients

Introduction

Drug-resistant tuberculosis (DR-TB) is a major global public health challenge.¹ China has the second highest DR-TB burden worldwide, with 73,000 reported cases in 2017.¹ DR-TB patients experience a long treatment duration, severe adverse drug reactions, and a heavy economic burden.² For DR-TB patients, effective adaptation is crucial to prevent disease progression.³

The Roy adaptation model (RAM) was proposed by Callista Roy. According to this model, adaptation is defined as the ability of an individual to adjust to the environment and to maintain the optimal state of the individual’s body and mind. The RAM suggests that all humans have certain needs that they need to meet to maintain integrity. These needs are met through the four modes of adaptation, namely, the physiologic, self-concept, role function, and interdependence modes.^4,5 Maladaptation presents in several ways, including physical discomfort, psychological distress, individual role reversal and interpersonal breakdown.⁶ Maladaptation increases disease recurrence and mortality; conversely, successful adjustment accelerates recovery.⁷ Being diagnosed with DR-TB is a strong emergent stimulus and an extremely stressful experience for patients.⁸ DR-TB patients suffer from fear, stigma, interpersonal breakdown, severe adverse drug reactions and, consequently, maladaptation (e.g., a serious decline in treatment confidence, poor drug compliance and a heavy psychological burden),^2,3,9 thus leading to worse treatment outcomes and a lower quality of life.¹⁰ Providing biopsychosocial care for DR-TB patients in hospitals and communities is a crucial issue.¹¹ It is vital for health service providers to assess the performance of patients’ adaptation status and to evaluate the adjustment level, thus managing maladaptation and promoting patients’ well condition during their treatment journey.

Examining patients’ adaptation status is premised on effective and reasonable assessment instruments. Several adjustment assessment tools are available for adaptation status assessment, such as the scales for patients with stoma,¹¹ patients with chronic obstructive pulmonary disease (COPD),¹² antenatal patients¹³ and patients with cancer.¹⁴ However, these scales are related to specific disease categories, and to date, relevant research focused on a TB patient adjustment scale has not been conducted. It is necessary to develop an adaptation status scale for the DR-TB population.

The RAM concepts of stimuli, coping mechanisms, and modes of adaptation have been translated into several middle-range concepts and measured using new and existing instruments.¹⁵ Based on the RAM, we developed an adjustment scale for DR-TB patients (AS-DRTBP) based on the following steps: a literature review, expert consultation, and pretesting. The reliability and validity of the scale were evaluated by calculating the internal consistency and reliability coefficients and by conducting exploratory factor analysis (EFA) and confirmatory factor analysis (CFA).

Materials and Methods

Participants

A convenience sampling strategy was used to recruit participants with DR-TB from six institutions in Shaanxi Province (Shaanxi Province Tuberculosis Hospital, Xi’an Chest Hospital, Huashan Hospital, and the designated TB hospitals of the Baqiao District, Weiyang District and Lianhu District) between January 2018 and May 2019.

The inclusion criteria were (a) having a confirmed diagnosis of TB infection with DR-TB, including rifampicin-resistant TB (RR-TR) or multidrug-resistant TB (MDR-TB), (b) being 18 years of age or older, (c) being conscious and able to answer questions (with the researchers reading the scale to elderly or illiterate participants), and (d) being willing to participate in the study and signing the written informed consent form. Considering that patients diagnosed with extensively drug-resistant TB (XDR-TB) often suffer from more serious conditions and complications, this study excluded patients with XDR-TB and other patients designated for intensive care.

In factor analysis, five participants per item is the minimum recommended sample size. Since the expected number of items was 33, a minimum of 165 participants were required.¹⁶ It has been widely acknowledged that the EFA and CFA samples should be two independent samples, with sizes of at least 100 participants¹⁷ and 200 participants,¹⁸ respectively. Taken together, assuming a 20% non-response rate, a minimum sample size of 360 was required.

Scale Development

Theoretical Framework

The AS-DRTBP was developed based on the theory of the RAM. Roy first published the RAM in 1970 and since then has continuously updated it.¹⁹ The RAM strengthens the adaptive level and process of people as an adaptive system facing various stimuli in the environment from a holistic perspective.⁵ According to the RAM, humans are considered an adaptive system. After stimulation by external and internal environments, the human body responds through a physiological regulation system and a cognitive regulation system that can generate changes in physiological function, self-concept, role function and interdependence. Finally, the body generates an adaptive response or an ineffective response.⁶ In recent years, the RAM has been used to guide the development of prenatal assessment tools for pregnant women and the development of tools for studying stoma patients²⁰ and cancer patients,¹⁴ among others. Patients living with DR-TB suffer from changes related to physiology, psychology, role and society, and they need to adjust and adapt to these changes, which are consistent with the four adaptation modes of Roy’s adaptation theory. Therefore, this study used Roy’s adaptation theory to guide the development of the AS-DRTBP. Based on the four adaptation modes, the framework of the AS-DRTBP is divided into four sections: physiological function, self-concept, role function, and interdependence.

Development of the Item Pool

A literature review and individual in-depth interviews were conducted to generate the initial item pool of the AS-DRTBP. The literature review included studies on the adjustment of patients, changes in the physiology, psychology, role and interpersonal relationships of DR-TB patients and the development of specific scales, identifying fields involving DR-TB patient adjustment. For the interviews, we recruited a convenience sample of 23 DR-TB patients to explore the changes and coping methods they experienced during their disease journey. At the end of this step, a draft of the AS-DRTBP, consisting of 32 candidate items, was formed.

Delphi Method

The Delphi method was used to select the items of the AS-DRTBP. Twelve experts with expertise in TB control and care in China (four TB prevention and control researchers, four clinical diagnosis and treatment physicians, two TB educators, and two nurse consultants in TB) evaluated the item pool of the scale. Two rounds of Delphi method expert letter consultation by email were used to screen the items in the AS-DRTBP. In each round, the experts were required to rate the importance of each item on a five-point Likert scale ranging from 1=least relevant to 5=most relevant. Items with an assigned average item score > 3.50, a coefficient of variation (CV) < 0.25 and an item-level content validity index (I-CVI, defined as the proportion of experts who rated the item as 4 or 5) > 0.78 were retained. The accuracy and clarity of the items were also evaluated by the experts.²¹ After the two-round Delphi consultation, there were a total of 33 items, and these items formed the initial AS-DRTBP: 2 items for physiological function, 8 items for self-concept, 15 items for role function and 8 items for interdependence.

Pilot Study

A pilot study of the AS-DRTBP was conducted at Xi’an Chest Hospital, Shaanxi Province, China, in June 2018. A convenience sampling method was used to survey 59 DR-TB patients to complete the prefinal version of the AS-DRTBP for testing the clarity and intelligibility of the items, the layout of the scale and the time needed to complete the test. The respondents were 18–67 years of age, and the mean age was 35.95 years (SD = 12.45). All of the patients who participated in the pilot study provided general demographic data and completed the initial draft of the AS-DRTBP in 15–18 min. The respondents showed few problems with the intelligibility and clarity of the items, and they presented a good acceptance of the scale. After revising the expression of some items based on feedback, the test version of the AS-DRTBP, consisting of 33 items, was established.

Data Collection

The research team that administered the questionnaire was composed of three trained researchers. We distributed the questionnaires to the patients in a face-to-face manner and acquired written informed consent prior to the survey. The questionnaire was composed of three parts: demographic data, the AS-DRTBP and the adjustment scale. The AS-DRTBP consisted of four sections: physiological function, self-concept, role function and interdependence. The physiological function section included multiple choice questions whose scores were summed. In contrast, a five-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree) was used to quantify the results of the other three parts. The adjustment scale was a general scale measuring the patient’s adaptive level.¹⁵ It consisted of a single item and a ten-point rating scale ranging from 1 (not well) to 10 (very well), with a lower number indicating more compromised life processes and a higher number indicating the opposite. The adjustment scale has been used to measure adjustment to living with spinal cord injury among spinal cord-injured individuals.^22,23

Data Analysis

Data analyses were performed using IBM SPSS (version 22.0) and AMOS (version 20.0). Descriptive statistics were used to describe the demographic variables.

Item Analysis

A quantitative method was used to verify the fitness of the items, thus examining whether an item should be removed or retained in this study. An item was removed if it met one or more of the following criteria: (1) the mean of the item was extreme, or its variance was zero; (2) the critical ratio (CI) value of an item was found to be nonsignificant; or (3) the correlation coefficient between the item and the total score was not significant.²⁴

Validity Analysis

Content validity was assessed in the final round of the Delphi survey. The experts were required to verify the relevance and importance of each item using a five-point Likert scale. The proportions of items given a rating of 4 or 5 by all experts were computed to determine the I-CVI. The average content validity index of each item represented the content validity index at the scale level (S-CVI). A statistically acceptable I-CVI should be > 0.78, and a statistically acceptable S-CVI should be > 0.90.²¹

The construct validity of the scale was tested by the total samples, which were randomly divided into two parts (N1 = 206, N2 = 227). One group of patients (N1 = 206) was used to perform the EFA, while the other group (N2 = 227) was used to perform the CFA. The number and nature of the potential factors in the scale were determined by the EFA. The factors of the scale were extracted by principal component factor analysis with the varimax orthogonal rotation method. Prior to performing EFA, the Kaiser-Meyer-Olkin (KMO) test for sampling adequacy and Bartlett’s test of sphericity were conducted. If Bartlett’s test of sphericity obtained a result of p<0.05 and the KMO value was > 0.6, then the data were suitable for EFA.²⁵ The number of potential factors was determined according to the following criteria: (a) an eigenvalue > 1.0 and (b) a scree plot representing all factors above the elbow or a break in the plot.²⁶ Items were retained if they met the following two criteria: (a) a factor loading > 0.4 without a crossing factor and (b) a conceptual coherence of the items with their corresponding factors.^27,28 The factor structure of the scale identified in the EFA was verified by a CFA. As the sample data (N2 = 227) did not have a normal distribution, the parameters of this model were estimated by the generalized least squares (GLS) method.²⁷ Model fitness was evaluated by the minimum discrepancy divided by its degrees of freedom (CMIN/DF), the goodness of fit index (GFI), the adjusted goodness of fit index (AGFI), and the root mean square error of approximation (RMSEA). If the CMIN/DF < 2 (p>0.05), the GFI and AGFI > 0.90, and the RMSEA < 0.08, then the data indicate a good model fit.²⁹

Concurrent validity was assessed by the Spearman correlation coefficient between the AS-DRTBP scores and the adjustment scale scores as well as the factor scores.

Reliability Analysis

The Cronbach’s alpha coefficient, split-half reliability coefficient and test-retest reliability coefficient were calculated to evaluate the reliability of the AS-DRTBP. The split-half reliability method was used to divide the scale items of the AS-DRTBP into odd-even parts according to the number, and then, the correlation of the score between the two parts was computed. Test-retest reliability was assessed by randomly selecting 51 DR-TB patients from the total number of participants. These patients were assessed again two weeks later. The interclass correlation coefficient (ICC) of the scores was calculated to determine the test-retest reliability of the scale. A statistically acceptable reliability coefficient of the total scale should be > 0.70, and a statistically acceptable reliability coefficient of a subscale should be > 0.6.³⁰

Results

Participants’ Demographic Characteristics

A total of 456 participants were recruited for the field test. A total of 442 participants completed the AS-DRTBP; thus, the response rate was 96.93%. The researchers checked the questionnaires before entering the data. Nine invalid questionnaires with obvious response rules were excluded. The valid AS-DRTBP sample size was 433, which was more than the minimum required number of participants. Overall, the respondents were 18–78 years of age, and the mean age was 34.92 years (SD = 14.35). The mean age of the DR-TB patients was basically similar to that of patients in other provinces in China.^31,32 The mean ages of the respondents in the EFA and CFA groups were 35.56 years (SD = 15.22) and 34.34 years (SD = 13.52), respectively. The demographic characteristics of the EFA, CFA and total samples are shown in Table 1. In the EFA (N1 = 206) and CFA samples (N2 = 227), the differences in demographic characteristics such as gender, age, educational level, nationality, marriage, residence, patient type, economic income, and insurance type were not statistically significant.

Table 1 Demographic Characteristics of Participants

Item Analysis

The means of all items ranged from 1.24 to 3.62. There were no items with a variance of 0. The CI values of all items except for item C10 were significant; the item-total correlations of all items except for item C10 were significant. According to the criteria mentioned above, item C10 (After I recovered from my illness, I can continue to take on my original job and family responsibilities) was removed. Finally, 32 items were retained. Table 2 shows the results of the item analysis.

Table 2 Results of Item Analysis

Assessing the Psychometric Properties of the Scale

Content Validity

In the final round of Delphi consultation, the CVI ranged from 0.80 to 1.00 for each item, and the average CVI for all items of the scale was 0.92. The results showed that the majority of experts assessed the items on the scale with a rating of “quite important” or “most important”, indicating strong agreement among the specialists and, therefore, high content validity.

Construct Validity by EFA

The KMO result was 0.85, and the result of Bartlett’s test of sphericity was significant (χ²= 4341.006, p<0.001), demonstrating that the sample data were suitable for factor analysis. To obtain the best factor structure, the stepwise deletion method in the EFA program was used for the 32 items of the scale. Prior to the final factor analysis, six items were removed because they had cross-loadings or did not conceptually fit with the other items pertaining to the same factor. Ultimately, four factors with eigenvalues > 1 were generated, and 26 items accounting for 64.605% of the variance were retained. The results showed that the factor loading of each item was above 0.4 and without cross-loadings. Cattell’s scree plot (Figure 1) presented a clear break after the fourth component. As shown in Table 3, based on the factor loading results and the item contents, factor 1 (10 items) was named “role function”, factor 2 (7 items) was named “self-concept”, factor 3 (7 items) was named “interdependence”, and factor 4 (2 items) was named “physiological function”.

Table 3 Results of Exploratory Factor Analysis

Figure 1 Cattell’s scree plot.

Construct Validity by CFA

The CFA model of the AS-DRTBP consisted of four factors and 26 items. An acceptable model fit was indicated by CMIN/DF = 1.681, GFI = 0.832, AGFI = 0.799, and RMSEA = 0.055. Figure 2 shows the standardized path diagram of the CFA model.

Figure 2 The standardized path diagram of the confirmatory factor analysis. Note: A1 to D7 represents the items of AS-DRTBP, F1 to F4 represents the four factors of AS-DRTBP.

Concurrent Validity

Concurrent validity was assessed by Spearman’s rank correlation analysis due to the non-normally distributed scores of the AS-DRTBP and the adjustment scale (p<0.10). As shown in Table 4, the correlation coefficient between the total scores of the AS-DRTBP and the adjustment scale was 0.355, and the correlation coefficient between the scores of each subscale of the AS-DRTBP and the adjustment scale ranged from 0.165 to 0.292.

Table 4 Concurrent Validity (the Correlation Between the Scores of as-DRTBP and the Adjustment Scale)

Reliability Analysis

As shown in Table 5, the Cronbach’s alpha coefficient was 0.893 for the total scale and ranged between 0.606 and 0.940 for each of the dimensions. The split-half reliability coefficient of the AS-DRTBP was 0.954 and ranged between 0.610 and 0.953 for each of the dimensions. The test-retest reliability coefficient of the total scale over a two-week interval for the 51-patient subsample was 0.853 and ranged between 0.716 and 0.820 for each of the dimensions.

Table 5 Reliability of the AS-DRTBP

Discussion

To the best of our knowledge, little research has focused on the development of a questionnaire for the adaptation status with respect to DR-TB. In this study, we developed a novel instrument consisting of 26 items and evaluated its reliability and validity. The AS-DRTBP was proven to be both reliable and valid for measuring the performance of adaptation status in patients living with DR-TB.

The split-half reliability coefficient and Cronbach’s alpha coefficient for the total scale were 0.954 and 0.893, respectively, and for each dimension, these values ranged from 0.610 to 0.954 and from 0.606 to 0.940, respectively. The reliability validation results fulfilled the requirement of a reliability coefficient > 0.7 for the total scale and a reliability coefficient > 0.6 for each dimension of the scale,³³ indicating satisfactory internal consistency. The overall test-retest reliability coefficient was 0.853, and the correlation coefficient of each dimension was between 0.716 and 0.820, indicating that the scale had acceptable stability over time.

Content validity refers to the degree to which a tool has an appropriate sample of items for the construct being measured.²¹ In this study, the most commonly used and convincing approach, the Delphi method, was applied to assess the content validity of the AS-DRTBP.³⁴ For the conventional Delphi method, a sufficient number of experts and a heterogeneous sample were used to ensure that the entire spectrum of opinions was effective.³⁵ Previous research has shown that the Delphi panel size requirements are modest and that a group size of 10 to 18 members is a fairly safe choice. In our study, the number of experts, 12, was within the recommended range.^36,37 Moreover, the experts in this study included TB preventive and control experts, clinicians, educators and nurses, covering a wide range of occupations and ensuring sample heterogeneity. In this study, the S-CVI was 0.9, and the I-CVI ranged from 0.80 to 1.00. The results met the criteria of acceptability (S-CVI > 0.90, I-CVI >0.78), indicating that the AS-DRTBP can accurately measure the true content of the adaptation level of TB patients.

Construct validity was evaluated by EFA and CFA. The EFA generated a clear four-factor solution consistent with theoretical concepts; the factors were named “role function”, “self-concept”, “interdependence” and “physiological function”.²⁶ The four factors accounted for 64.61% of the total variance, which was higher than the criterion of 60%,²⁷ indicating that the common factors extracted were reliable. The factor loading for each item was greater than 0.4 and without cross-loadings, suggesting tight relationships between the items and the factors. In general, these four factors represent the overall structure of the scale.

A CFA was performed to confirm the structure of the scale for the other sample. The CFA results revealed that CMIN/DF was 1.681, which met the criterion of less than 2.³⁸ The RMSEA was 0.055, which met the criterion of less than 0.08, indicating a reasonable model fit. The GFI was 0.832, and the AGFI was 0.799; these results deviated from the ideal standard of 0.90 but fell within the acceptable range.³³ Overall, the CFA showed that the fitting effect of the model was acceptable and that the structure of the scale was consistent with the theoretical concept.

The concurrent validity results demonstrated moderate correlations between the total scores of the AS-DRTBP and the adjustment scale and low correlations between the AS-DRTBP subscales and the adjustment scale. The adjustment scale was a general scale and measured the overall adaptation level, while the AS-DRTBP showed the adaptation issues that DR-TB patients suffered and measured patients’ TB-specific adjustment condition. The result of moderate correlations may partly reflect the specificity of the AS-DRTBP.

Implications for Clinical Practice

Successful adaptation to a disease is helpful for maintaining physical and mental health, thus accelerating disease rehabilitation. Early detection of the maladaptation of DR-TB patients and timely intervention are beneficial for maintaining the balanced life of patients. This study provides a simple, practical tool, the AS-DRTBP, for evaluating the adaptation status in patients with DR-TB. For clinical application, health care providers can use it to assess baseline data and the intervention effect on the adaptation status. For scientific research, the AS-DRTBP enriches the application scope of the RAM and provides a reference for the development of adaptation scales for other specific diseases. We excluded XDR-TB patients from participation in this study. Most of these patients were recurrent patients or MDR-TB patients who experienced treatment failure or relapse. Additionally, some XDR-TB patients often suffer from more serious conditions and live with complications, and some of them may be unable to complete investigations.^39,40 Thus, the accuracy of the AS-DRTBP may be affected. The applicability of the AS-DRTBP in XDR-TB patients could be validated in further research to determine whether it is necessary to develop a special scale for XDR-TB patients.

Several limitations of this study must be noted. First, we carried out field testing only at tuberculosis hospitals in Shaanxi Province; thus, in further studies, researchers can expand the samples to other regions of China. Second, the correlation coefficient between the items of the physiological function dimension and the total score of the adjustment scale was 0.165, which may be because the physiological function dimension contains only objective indicators while the other dimensions contain subjective factors. A moderate correlation between the total scores of the AS-DRTBP and the adjustment scale showed that the concurrent validity was acceptable.

Conclusion

The AS-DRTBP was developed following scale development steps based on the theory of the RAM. The scale includes 26 items divided into 4 dimensions, including role function, self-concept, interdependence and physiological function. The present study showed that the AS-DRTBP has good validity and reliability. The AS-DRTBP can be used by researchers, educators and clinicians as a tool for baseline data collection and intervention evaluations. The AS-DRTBP can be applied to investigate the adaptation status of DR-TB patients but not XDR-TB patients. The applicability of the AS-DRTBP in XDR-TB patients could be validated in further research.