Reliability and Validity of the Japanese Version of the Barthel Index Dyspnea Among Patients with Respiratory Diseases

Introduction

Dyspnea has a serious impact on the prognosis and quality of life of patients suffering from chronic respiratory disease (CRD). Chronic and progressive dyspnea is the most characteristic symptom of chronic obstructive pulmonary disease (COPD),^1,2 and is also the most important factor influencing the ability to perform activities of daily living (ADL).³ ADL restriction in COPD not only impairs health-related quality of life but is also a predictor of mortality.^4,5 Moreover, dyspnea in interstitial lung disease (ILD) has been associated with ADL restriction.⁶ ADL restriction in ILD is associated with health-related quality of life and life prognosis.^7,8 Therefore, estimating dyspnea during ADL in patients with CRD is considered essential.

Pulmonary rehabilitation based on quantitative evaluation for patients with CRD is effective for improving dyspnea and the ability to perform ADL. A current guideline described that pulmonary rehabilitation for stable patients with COPD improves dyspnea, health status, and exercise tolerance (Evidence A).⁹ Pulmonary rehabilitation improves the physiologic response to and actual performance of ADL in patients with COPD.¹⁰ In addition, significant improvements of ADL in patients with ILD have been reported with pulmonary rehabilitation.¹¹ To clarify the degree of improvement in ADL after rehabilitation more clearly, a quantitative scale that reflects disease specificity is essential.¹²

Disease-specific assessment scales are needed for the evaluation of the ability to perform ADL in patients with CRD. The generic ADL scales, such as the Barthel Index (BI), mainly evaluate independence and do not include the element of dyspnea, which is an important symptom in patients with CRD.¹³ As a result, for example, the ability to perform ADL is sometimes overestimated when using these general ADL scales in patients with COPD.¹² Recently, CRD disease-specific ADL scales, which also include assessments about dyspnea, have been used globally.^14,15 The Barthel Index dyspnea (BI-d), which is a modification of the BI developed in Europe, is one of the CRD disease-specific ADL scales including the element of dyspnea.¹⁶ The BI-d is a questionnaire with established guidelines for its measurement. It is reported to be associated with the 6-minute walk distance (6MWD, an index of exercise tolerance) and the modified Medical Research Council scale (mMRC scale, an index of dyspnea).

In Japan, it is necessary to promote the use of disease-specific ADL scales for CRD. Patients with COPD in Japan have several characteristics that differ from those of patients in other countries, for example, being aged and leaner. Hence, it is clinically and academically valuable to make international comparisons in ADL abilities, as assessed by standardized scales. However, and as far as the authors know, few Japanese versions of CRD disease-specific ADL scales, such as the Nagasaki University Respiratory ADL questionnaire and the pulmonary emphysema-ADL, are used globally.¹⁷ Although the modified version of the Pulmonary Functional Status and Dyspnea Questionnaire has been determined to be reliable and valid for the Japanese population, there are few reports of its use, and its use is not widespread in Japan.¹⁷ Moreover, as BI is one of the most widely used ADL scales in Japanese clinical settings, its modified version, BI-d, is also expected to be acceptable in Japan. However, BI-d has not been translated into Japanese yet, and its reliability and validity have not been examined. Therefore, the purpose of this study was to develop the Japanese version of the BI-d (J-BI-d), and investigate its reliability and validity.

Materials and Methods

Barthel Index Dyspnea

BI-d is a scale whose reliability and validity have been established for patients with respiratory diseases.¹⁶ The BI-d was developed based on the BI items; it assesses dyspnea on the 10 ADL items of grooming, bathing, feeding, toilet use, using stairs, dressing, bowel movement, bladder discharge, mobility, wheelchair use, and transfers (bed to chair and back).¹⁶ The examiner judges each item on a five-point scale ranging from 0 to 4, where 0 indicates no sign of dyspnea during execution of ADL; 1 indicates slight dyspnea that does not prevent or slow down the execution of ADL; 2 indicates moderate dyspnea, which can slow down ADL; 3 indicates severe dyspnea, which can greatly slow down ADL; and 4 indicates an extremely severe level of dyspnea, which precludes or reduces ADL. The total BI-d score ranges from 0 (no dyspnea) to 100 (maximum level of dyspnea) according to the original BI grading score.

Translation Process

The translation procedure of the J-BI-d followed the basic guidelines for scale translation.¹⁸ First, we contacted the original author to obtain permission to translate the scale into Japanese. Second, two bilingual respiratory physicians and a physiotherapist translated the scale from English to Japanese in turn. Third, a group of experts in respiratory diseases was recruited to examine and integrate more appropriate wording in comparison to the forward-translated scale. Next, a bilingual respiratory physician and a physiotherapist performed the back-translation, which was different from the forward translation. The back-translated scale was reviewed by the original author to confirm its equivalence to the original version. After obtaining consent for using the scale’s back translation from the original author, cognitive debriefing was performed on 10 patients with CRD. Cognitive debriefing is a survey method used to check whether a patient understands concepts and items. The average time required for measurement was 196.7 seconds (SD = 88.0 seconds). Since no incomprehensible expressions or deficiencies were pointed out by the patients, we decided to finalize this tentative Japanese version of the scale.

Study Design and Participants

A cross-sectional study was performed to investigate the reliability and validity of the J-BI-d. The study participants were outpatients with CRD (confirmed diagnosis of COPD or ILD by a respiratory physician), who were recruited from outpatient care in two hospitals between January 2019 and February 2020. The inclusion criteria were: a stable condition and no infection or acute exacerbation within the past three months, no unstable or severe cardiac disease, and no disease that would interfere with activities of daily living (eg, stroke or orthopedic disease). The exclusion criteria were a diagnosis of asthma only, acute exacerbation during the study period, difficulty in understanding the questionnaire, and not providing consent for the study. Participants’ personal information used in this study was anonymized with a research-specific number and a correspondence table, and was stored on electronic media. The files were managed with a password so that only the researcher could view them. This study was approved by the ethics committee of Kobe University (N611-1); Kobe City Hospital Organization, Kobe City Medical Center, West Hospital (N17-003); and Nishioka Hospital (N1806). All participants provided written informed consent in accordance with the ethical standards set in the Declaration of Helsinki.

Measurements

Basic patient data were obtained from medical records. The J-BI-d was measured twice to verify intrarater reliability. The participants for the second measurement were those who had an appointment with a respiratory physician and whose interval from the first measurement was approximately one to four weeks. The anchor index, a self-administered questionnaire assessing health status on a 3-point scale, was used to analyze whether the condition of the participants in the second measurement had changed from the initial measurement.¹⁹ Participants whose status change was confirmed by the anchor index were excluded from the retest. The lung function test was performed according to the American Thoracic Society’s guidelines²⁰ and measured FVC, FEV_1. The mMRC scale was used as a measure of dyspnea. The 6-minute walk test was performed to assess functional exercise capacity according to standard procedures.²¹ The mMRC scale has been validated as a measure of dyspnea, and the 6MWD as a test of exercise tolerance.⁹ For divergent validity, we tested whether the BI-d measures different concepts than the BI. Specifically, we confirmed if there was low association between J-BI-D and BI.

Statistical Analysis

Data were presented as counts and percentages, means and standard deviations, or median and interquartile range, as appropriate. The Consensus-based Standards for the selection of health Measurement Instruments (COSMIN) taxonomy was applied for statistical analysis of validity and reliability.²² The floor effect and ceiling effect were visually confirmed by histograms. The statistical analyses were performed with R (ver. 3. 6. 1) on an EZR platform (Saitama Medical Center, Jichi Medical University, Saitama, Japan), and with R studio (ver. 1. 2. 5042). P values less than 0.05 were considered statistically significant.

Construct Validity

The criterion-related validity of the J-BI-d in comparison to the mMRC scale and 6MWD test was evaluated using Spearman’s rank correlation coefficient and Pearson’s correlation coefficient, respectively. The strength of the association was defined as very weak for < 0.2, weak for 0.2–0.35, moderate for 0.35–0.5, and strong for >0.5.²³ Comparisons between the J-BI-d and grades of the mMRC scale was performed using the Kruskal–Wallis test to confirm the significant main effect. Multiple comparison tests between the J-BI-d and mMRC scale were performed using Mann–Whitney U-test and adjusted for significance levels by the Bonferroni test. To verify the divergent validity of the BI-d, Pearson’s correlation analysis between the BI-d and BI was performed.

Reliability

Internal consistency, test–retest reliability, and smallest detectable change (SDC) were analyzed. The scale’s internal consistency was evaluated using Cronbach’s alpha. Test–retest reliability was for analyzing the J-BI-d’s reproducibility in the test (day 1) and the retest (day 2), we used the intraclass correlation coefficient (ICC (2,1)). The ICC and its 95% confidence intervals were calculated based on the absolute-agreement, two-way mixed-effects model. The ICC value is considered acceptable if it is >0.70.²⁴ SDC is based on standard error of measurement (SEM), which is the variability in measurements (SD) of the same individual, with a confidence of 95%, and is expressed in the unit of the measurement. It was estimated by computing the square root of the within-subject variance of the patients (SEMagreement = √σbetween measurement + σresidual). Variance components were obtained from a multilevel mixed effects model (restricted maximum-likelihood estimates). Since SDC is the smallest amount of change in individuals that can be detected beyond measurement error with a confidence of 95%, it is calculated as 1.96*√2*SEM.²⁵

Results

Sixty-four patients with outpatient CRD were convened, out of whom 57 were included in the analysis of this study (Figure 1). Fifty (87.7%) participants of the patient group in this study had COPD (involving an asthma-COPD overlap) and seven (12.3%) had ILD (Table 1). With regard to the severity level of the patients with COPD, eight (16%) were in GOLD Stage I, 26 (52%) were in Stage II, 11 (22%) were in Stage III, and five (10%) were in Stage IV. For the mMRC scale, 10 patients were in grade 0 (18%), 19 were in grade 1 (33%), 19 were in grade 2 (33%), nine were in grade 3 (16%), and none were in grade 4 (0%). The mean value of J-BI-d was 14±12.6, and the ADL of the present target population was relatively mild and varied. The histogram of the total J-BI-d scores showed a high percentage, biased toward lower scores (Figure 2). No ceiling effect was found; however, a floor effect was suspected.

Table 1 Baseline Characteristics of the Participants

Figure 1 Flow diagram of patients with chronic respiratory diseases enrolled in this study. Abbreviation: †CRD, chronic respiratory diseases.

Figure 2 Histogram of J-BI-d total score. Abbreviation: †J-BI-d, Japanese version of Barthel Index dyspnea.

Construct Validity

There was a moderate negative correlation between the BI-d and 6MWD test (r = −0.46, p < 0.01) (Figure 3). There was a strong positive correlation between the BI-d and mMRC scale (ρ = 0.76, p < 0.01). Significant main effects of the mMRC scale grades among J-BI-d scores were confirmed by the Kruskal–Wallis test. Multiple comparisons revealed that there were significant differences in the J-BI-d scores among all four grades of the mMRC scale (Figure 4). This result indicates that as the mMRC scale grade increases, the J-BI-d score increases. The J-BI-d and BI had a weak negative correlation (r = −0.29, p < 0.05), indicating high divergent validity.

Figure 3 Correlation between meters in the 6MWD test and J-BI-d scale. Abbreviations: †6MWD, 6-min walk distance; †J-BI-d, Japanese version of Barthel Index dyspnea.

Figure 4 Box plot of the distribution of J-BI-d scores according to mMRC scale dyspnea score grading measured at baseline (mMRC grade 4:n = 0). Abbreviations: †BI-d, Barthel Index dyspnea; †mMRC, modified Medical Research Council.

Reliability

The Cronbach’s alpha coefficient for the J-BI-d was 0.81, indicating high internal consistency. Cronbach’s alpha coefficient from deleting item by item is shown in Table 2.

Table 2 The Cronbach’s Alpha Coefficient by Deleting Item by Item

Test–retest reliability was analyzed in 42 patients, excluding 2 patients due to change in condition and 13 patients who did not have a medical appointment in the stipulated period (Figure 1). The average interval between the initial test and the retest was 8.1 days (SD = 3.0 days). The ICC (2,1) for the J-BI-d was 0.76 (95% CI: 0.62–0.85), indicating high intrarater reliability. SEM was 7.19, SDC was 19.92.

Discussion

The present study was the first to examine the reproducibility, internal consistency, and validity of the translated J-BI-d in patients with CRD. The results of the study showed that the J-BI-d had a high ICC value and Cronbach’s alpha coefficient, as well as a strong correlation with both the mMRC scale and 6MWD test. These results indicate high reproducibility, internal consistency, and validity of the J-BI-d.

The J-BI-d was developed by following the basic guidelines for scale translation.¹⁸ Both the forward and back translation of the BI-d involved independent bilinguals, and the original authors approved the questions of the J-BI-d as accurately reflecting the intent of the questions in the original version. When cognitive debriefing for the J-BI-d was conducted with 10 patients with CRD, none of the patients reported encountering incomprehensible words or expressions, and there were no missing values in the responses. The average time taken for cognitive debriefing in this study was about 3 minutes, which is less than the 9 minutes taken for administering the previous Japanese version of the Pulmonary Functional Status and Dyspnea Questionnaire—a disease-specific ADL scale for COPD.²⁶ For busy clinicians, a quick and easily understandable assessment scale is strongly required. Therefore, the J-BI-d is clinically useful as a questionnaire for patients with CRD.

The result of this study proved the J-BI-d’s high reliability in patients with CRD. Since this study excluded patients with symptomatic changes, as detected in the anchor index at the time of retesting, the effect on the J-BI-d scores is expected to be minimal. The ICC obtained from the measured data was 0.76, indicating excellent intrarater reliability.²⁷ The average interval period between the initial test and retest was 8.1 ± 3.0 days in the current study. Generally, it is suggested that the test–retest interval period be from one to two weeks. The high reliability of this study was demonstrated from the data obtained within this recommended appropriate interval. The test–retest ICC of the original English version of the scale was 0.99, which is higher than the ICC value obtained in the present study. The result may have been influenced by the difference in the test–retest interval period, since the interval period in the original English version of the scale was shorter by 48 hours than that in the current study. In terms of internal consistency, the Cronbach’s alpha coefficient of the J-BI-d was 0.81, which is as high as that of the original English version. Previous studies have suggested that high internal consistency is demonstrated by a Cronbach’s alpha coefficient of over 0.7.²⁸ Therefore, the J-BI-d can be used for repeated ADL assessments of patients with CRD, for instance, in the case of assessing patients before and after rehabilitation.

The current study demonstrated the J-BI-d’s sufficient criterion-related validity from multiple perspectives of analyses. First, the J-BI-d had a significant positive correlation with the mMRC scale and 6MWD test. This suggests that higher J-BI-d scores imply greater dyspnea and more severely impaired exercise tolerance. In addition, the J-BI-d total score was significantly different among the grades of the mMRC scale. These results suggest that ADL limitations caused by dyspnea may be stratified by the J-BI-d. A weak correlation between the J-BI-d and BI suggests that the J-BI-d evaluates a different aspect of ADL ability as compared to the BI. The correlation between BI-d and BI in the original English version was low (r = −0.38) as well, similar to the results of this study. Based on these results, it can be assumed that the J-BI-d provides an accurate measure of dyspnea during ADL in Japanese patients with CRD.

The J-BI-d developed in this study is expected to facilitate the study of limitations in performing ADL due to dyspnea in Japanese patients with CRD, which is an important step forward, as it has been difficult to compare the differences in performing ADL due to dyspnea internationally. Respiratory disease-specific ADL scales that have been developed and used in Japan, such as the Nagasaki University Respiratory ADL questionnaire and pulmonary emphysema-ADL, are hardly used outside of Japan. Thus, comparing the ability to perform ADL among patients with CRD in Japan to the ability of patients in other countries has been difficult. In addition, the reliability and validity of the London Chest Activity of Daily Living Scale, which is used internationally, have not been verified in the Japanese population because this scale does not capture the lifestyle of Japanese people and has not been translated into Japanese. In contrast, the Pulmonary Functional Status and Dyspnea Questionnaire is an instrument whose reliability and validity have been proven among the Japanese population, but there have been few reports of its use in Japan. The BI is one of the most widely used ADL scales in Japan; hence, the J-BI-d, which has been created and translated based on the BI, is expected to be accepted in actual clinical practice in Japan. We hope that the J-BI-d will be widely used both clinically and in research, and that the number of reports on the ADL of CRD patients in Japan will increase.

However, several limitations prevent the generalization of the results of this study. First, all of the patients in the study were outpatients with settled symptoms. Most of them were in mMRC scale grade 1–2 and there was no patient in mMRC scale grade 4. Thus, the results of this study may not apply to patients with the most severe dyspnea, such as those in mMRC scale grade 4. This observation is similar to that in the original English version of the scale, wherein patients with the most severe dyspnea were not included in the results.¹⁶ Thus, the applicability of the J-BI-d to cases of severe respiratory distress, including hospitalized patients, should be investigated in future studies. Second, the patients in this study were a mix of those having COPD and ILD. The pathogenesis of COPD and ILD is different, and most of the similar studies to date have confined their coverage to COPD.^14,15 However, COPD and ILD have in common the occurrence of chronic, progressive increases in dyspnea and reduced exercise tolerance.^1,2,6 Since patients with COPD and ILD were also included in the original English version of the scale, it can be assumed that the J-BI-d can be used for both sets of patients. Third, the sensitivity of changes before and after pulmonary rehabilitation could not be investigated using the J-BI-d. Further investigation of this issue is required. Fourth, the sample size was inadequate. COSMIN requires at least 80 participants;²² and it is undeniable that this led to a lack of testing power. Additionally, most of the subjects have COPD; therefore, it is not possible to generalize the results to all patients with CRD. In future, we would like to conduct a similar analysis by narrowing the target population to ILD patients.

Conclusion

This study demonstrated that the J-BI-d had a high reliability and sufficient validity as a tool to assess the ability to perform ADL in Japanese patients with CRD. The J-BI-d can be adopted as a new tool to adequately measure ADL in patients with CRD in Japan. Further studies should use the J-BI-d to evaluate the effects of the intervention for and prognosis of CRD.