The Reliability and Validity of the Brief ICF Core Set in Patients with Chronic Obstructive Pulmonary Disease

Introduction

Chronic obstructive pulmonary disease (COPD) has significant morbidity and mortality and poses a major health and economic burden to patients’ families and nationwide.¹ As stated by Global Initiative for Chronic Obstructive Lung Disease (GOLD) 2021, in an update on the disease burden of COPD, the incidence of this disease is expected to continue to rise over the next 40 years, resulting in more than 5.4 million deaths a year from COPD and related diseases by 2060 as smoking rates increase in developing countries and population aging accelerates in high-income countries.² GOLD has reported pulmonary rehabilitation is one of the content of the standard treatment for COPD.² However, whether the intervention is effective depends on considering the complexity and severity individualized to the unique needs of the patients.³ Therefore, it is especially important to assess the patient’s physical function and structure effectively to explain further the performance of daily activities and participation in life pursuits. The World Health Organization International Classification of Functioning, Disability and Health (ICF) provides the basis for such an assessment system that offers a useful framework for classifying the components of health and consequences of a disease. In addition, this classification aims to provide a unified and standard language for the description of health conditions in rehabilitation and a common framework for all health professions to promote exchanges between different disciplines and fields around the world.⁴ The ICF provides functional measurements for patients with COPD and combines the living environment to analyze the interaction and relationship between the state of health (diseases, disorders and injuries) and contextual factors. This framework specifically includes four aspects: body function, body structure, activity and participation and contextual factors. The ultimate goal of using ICF is to use it as a common classification and language system which can be used more widely in health care services. A preliminary study on the reliability and validity of the ICF-COPD Core Set was analyzed by Liu,⁵ and the results showed that the evaluation function for patients with COPD was reliable and effective, but its universality still needs to be further tested in more studies.

To promote more clinical practice and generalization of the ICF for chronic disease, WHO has developed the ICF core set based on various applied research projects in recent years. For instance, validates the ICF core set for obstructive pulmonary diseases from the perspective of adults with asthma;⁶ to validate the ICF core set for hearing loss (HL) of adults with HL internationally;⁷ and to ensure the ICF core set for cerebral palsy (CP) applicability by using the child and youth national dataset in Taiwan and the Functioning Scale of the Disability Evaluation System,⁸ which has good functionality and time efficiency. The ICF core set selects as few ICF items as possible related to function, disability and health under specific disease and environmental conditions. The brief core set, which extracts the most commonly suitable items from the core set, is the key to applying the ICF in clinical applications. The Brief ICF core set has been widely used in various fields to evaluate function, and a number of mature assessment sets have been developed. This core set plays an important role in guiding clinical nursing and rehabilitation work. Currently, the Brief ICF core set has received research certification, such as for chronic widespread pain,⁹ low back pain,¹⁰ osteoarthritis,¹¹ osteoporosis,¹² rheumatoid arthritis,¹³ chronic ischemic heart disease,¹⁴ depression,¹⁵ breast cancer,¹⁶ stroke,¹⁷ and spinal cord injury.¹⁸ The Brief ICF is also emphasized as a reference framework to evaluate and to compare the coverage of other models in clinical practice.¹⁹

By combining evidence and expert opinion, the first Brief ICF core set for obstructive pulmonary disease (OPD) patients was developed, consisting of 4 components and 14 categories, in 2004.²⁰ The applicability of the scale has been largely supported by physiotherapists and clinical experts,^21,22 and it was further verified by patients.²³ In addition, Marques and other scholars also pointed out, which has not been confirmed, the additional categories that were raised need further investigation in the Brief ICF core set.²⁴ Huang proposed through research that there is a high prevalence of problems in 8 categories that were not considered in the Brief ICF-OPD core set for patients with COPD.²⁵ Recent studies have shown that the Brief ICF core set has good reliability and validity on the part of “activity and participation” and can be used to test daily activities for patients with COPD.²⁶ Current studies on the Brief ICF core set for COPD lack an overall assessment of the four dimensions, which needs further confirmation.

By using the clinical data, Brief ICF core set was preliminarily studied to verify their reliability and validity and to provide an objective basis for their clinical application. According to the conclusion of the former study, suggestions for improvement of the Brief ICF core set for COPD were put forward. In addition, the Brief ICF core set will have different problems because of cultural and environmental discrepancies, thus the Brief ICF core set for COPD still needs further study. Therefore, the aim of this study is to test the reliability and validity of the Brief ICF Core Set for patients with COPD to provide an objective basis for its clinical application, which will lay the foundation for its further application in China.

Patients and Methods

Study Design

The study was designed as a cross-sectional survey. The research site was four tertiary hospitals, Tianjin Medical University General Hospital, The Second Hospital of Tianjin Medical University, The Tianjin First Central Hospital, and The Tianjin Chest Hospital. This study was conducted in accordance with the Declaration of Helsinki and approved by the Ethics Committee of Tianjin Medical University (TMUhMEC2020028).

Patients

The study subjects were patients with COPD who visited the four tertiary hospitals from October 2019 to October 2020. The inclusion criteria were as follows: (1) meeting the diagnostic criteria of the Global Initiative on Chronic Obstructive Pulmonary Disease (GOLD): the global strategy for diagnosis, treatment, and prevention of COPD (updated); and (2) signing a written informed consent form. The exclusion criteria were as follows: (1) having complications with severe primary diseases, such as cardiovascular, liver, kidney, and brain diseases; (2) being unable to respond to the instructions of the research.

According to the empirical method, the sample size is at least 5–10 times as large as the number of measured items, and there are 14 questions in total. At the same time, considering the possibility of invalid questionnaires, the sample size was increased by 10%. Finally, 100 patients with COPD were selected as research subjects.²⁷

Instruments

Social Demographic and Clinical Variables

The demographic variables included sociodemographic variables such as age, sex, height, weight, body mass index (BMI kg/m2), marital status, education level, employment status, income level, and smoking history, as well as clinical variables such as course of disease, stage of illness, family history, complications, use of drugs and home oxygen.

The Brief ICF-COPD Core Set

In this study, the Brief ICF-COPD Core Set was used to assess functioning and disability involving body functions, body structures, activities and participation as well as environmental factors in all patients. This Brief ICF Core Set comprises four categories in body functions, two categories in body structures, four categories in activities and participation, and four categories in environmental factors (Table 1). ICF qualifiers were applied to rate the degree of problems in each category of the body function and structures component and activity participation component with a generic five-point scale: 0, 1, 2, 3, and 4, indicating no problem, mild problem, moderate problem, severe problem, and complete problem, respectively. Environmental factors were graded with a nine-point scale: +4, +3, +2, +1, 0, 1, 2, 3, and 4, indicated to complete facilitator, substantial facilitator, moderate facilitator, mild facilitator, no barrier/facilitator, mild barrier, moderate barrier, severe barrier, and complete barrier. In addition, score numbers eight and nine also indicated unspecified and not applicable, respectively, in all categories.

Table 1 The Contents of the Brief ICF-COPD Core Set

The mMRC dyspnea score is a 5-point (0–4) scale based on the severity of dyspnea.²⁸ The MRC dyspnea scale rates breathlessness by the following responses: 0=“I am never breathless or short of breath;” 1=“I only get breathless with strenuous exercise;” 2=“I get short of breath when hurrying on the level or up a slight hill;” 3=“I walk slower than people of the same age on the level because of breathlessness or have to stop for breath when walking at my own pace on the level;” 4=“I stop for breath after walking 100 yards or after a few minutes on the level;” or 5=“I am too breathless to leave the house.”

The severity of airfow limitation in COPD is classified based on post-bronchodilator FEV1 value into four groups (GOLD1, GOLD2, GOLD3, and GOLD4, from 1 to 4 in score).² In patients with FEV1/FVC<70%, if FEV1>80% predicted, they are classified as mild (GOLD1), if FEV1 predicted<80% and ≥50% as moderate (GOLD2), if FEV1predicted<50% and ≥30% as severe (GOLD3) and if FEV1predicted<30% as very severe (GOLD4) airfow limitation.

Data Collection

Descriptive data were collected by interviewing the participants and by consulting their medical records. Data collection was performed by four raters in different hospitals with relevant research experience who had previously been trained in data collection skills. After confirming that the patients met the inclusion criteria, raters explained the purpose and content of the study to them and helped patients sign the informed consent form.

Data Analysis

Statistical analyses were performed using SPSS version 25.0 (IBM Corp, New York) and AMOS version 24.0 (IBM Corp, New York). The 0.05 level of significance was applied. Descriptive statistics were conducted to describe participant sociodemographic and clinical variables. Quantitative variables are presented as the mean ± standard deviation. Qualitative variables are presented as frequencies (percentages).

Internal Consistency Reliability

For the reliability test, the internal consistency was analyzed and expressed by Cronbach’s α coefficients. Data from 100 patients with COPD were collected to assess the internal reliability. A Cronbach’s α>0.7 is considered to be the acceptable value for reliability.²⁹

Content Validity

To identify whether the Brief ICF-COPD Core Set could cover the most common problems in patients with COPD, we used expert consultation to study the content validity. The content validity index (CVI) was determined by an expert panel group including 5 specialists (they all possessed Senior title of professional). Three levels of the CVI, item level (I-CVI), scale level (S-CVI) and S-CVI/AVE, were computed to evaluate the content validity. The I-CVI was evaluated with a 4-point rating scale and was calculated by the formula (the number of experts who marked 3 or 4/the total number of experts). The S-CVI is equal to the proportion of items rated 3 or 4 by all experts. The S-CVI/AVE was tested by taking the average of the item-level CVIs. The finding of an S-CVI greater than 0.80 and an I-CVI greater than 0.78 indicates acceptable content validity.³⁰

Construct Validity

To evaluate the construct validity of the Brief ICF-COPD Core Set in the Chinese context, factor analysis was conducted. Confirmatory factor analysis (CFA) was performed with the maximum likelihood method to confirm the structure of the Brief ICF-COPD Core Set. The criteria for goodness-of-fit indices included χ² statistics/degrees of freedom (χ²/df) of less than 3, a standardized root mean square residual of less than 0.08, and an incremental fit index closer to 1.^31,32

Convergent Validity

The GOLD proposed a new severity assessment system for emphasizing clinical system evaluation by mMRC dyspnea scores, and COPD clinical severity grade is also an important assessment system. GOLD reports the complete flow chart of COPD management and grouping criteria, in which mMRC is used to preliminarily evaluate the symptoms of patients.² To evaluate the convergent validity of the Brief ICF-COPD Core Set, we analyzed the relationship between the score of the Brief ICF-COPD Core Set, mMRC dyspnea scores and COPD clinical severity grade. The mMRC dyspnea score is a 5-point (0–4) scale based on the severity of dyspnea. The COPD clinical severity grade is a 4-point (I-IV) scale. To investigate the criterion validity of the Brief ICF-COPD Core Set, Pearson correlation coefficients between the scores of the core set components and the mMRC or COPD clinical severity grade were calculated. As recommended, correlation coefficients were interpreted as follows: weak correlation (0.10–0.39), moderate correlation (0.40–0.69), strong correlation (0.70–0.89), and very strong correlation (0.90–1.00).³³ We hypothesized that positive correlations were found between overall and component scores of the Brief ICF-COPD Core Set and mMRC dyspnea scores or COPD clinical severity grade, and higher scores in the Brief ICF-COPD Core Set corresponded to higher scores in the mMRC and COPD clinical severity grade.

Result

Social Demographic and Clinical Variables

A total of 100 patients with COPD were included in this study, consisting of 69 males and 31 females. The average age was 70.70±8.07. Approximately, 52% of the patients were referred for a smoking history, and 78% of patients were in the acute exacerbation stage. Each patient’s severity of disease and mMRC dyspnea score varied to some degree. Ninety-four percent of patients had a medication history, and less than half (42%) of patients received home oxygen therapy. The patient sociodemographic and clinical variables are presented in Table 2.

Table 2 Sociodemographic and Clinical Variables (N=100)

Internal Consistency Reliability

Cronbach’s α was used to examine the internal consistency of the items within each subscale. The internal consistency was 0.873 for the total scale, with values of 0.750, 0.640, and 0.843 for body functions, body structures, and activity and participation, respectively. Accordingly, Cronbach’s α for the total scale and two subscales was higher than 0.70, indicating sufficient internal consistency (Table 3).

Table 3 The Internal Consistency Reliability of the Total Scale and Three Subscales

Content Validity

Based on the opinions of the 5 specialists in clinical nursing and therapy, the results of the content validity analysis showed that the I-CVI ranged from 0.8–1.0, and the S-CVI/universal agreement and S-CVI/AVE were 0.929 and 0.986, respectively, indicating satisfactory content validity (Table 4).

Table 4 Experts’ Rating and CVI Calculation (n=5)

Confirmatory Factor Analysis

The construct validity was examined by means of CFA. The initial fit indices of the five-factor model (Figure 1) did not meet the criteria, and the structural model (Figure 2) was modified based on the results of the indices, resulting in a better model fit (Table 5). The results of the CFA were acceptable: χ²/df = 1.292, root mean square error of approximation = 0.054, Tacker-Lewis index=0.946, comparative fit index=0.961 and incremental fit index=0.962. The path diagram from CFA is depicted in Figures 1 and 2.

Table 5 Confirmatory Factor Analysis Results

Figure 1 The initial confirmatory factor analysis result of the Brief ICF-COPD Core Set (n=100).

Figure 2 The modified confirmatory factor analysis result of the Brief ICF-COPD Core Set (n=100).

Convergent Validity

For convergent validity, moderate correlations were found between the Brief ICF-COPD Core Set and mMRC dyspnea. As expected, the body function scores of the Brief ICF-COPD Core Set correlated positively with the mMRC dyspnea score (r=0.582, P<0.01). Similarly, 2 subscales of the Brief ICF-COPD Core Set had positive associations with the mMRC dyspnea score: body structures (r=0.458, P<0.01) and activity and participation (r=0.649, P<0.01). However, the environment of the Brief ICF-COPD Core Set had no correlation with the mMRC dyspnea score (P>0.05). There were relatively weak correlations between the first three subscales (body function, body structures, and activity and participation) of the Brief ICF-COPD Core Set and COPD clinical severity grade as correlation coefficients ranged from 0.177 to 0.363 (P<0.01). The total Brief ICF-COPD Core Set score had a moderate correlation with the mMRC dyspnea score (r=0.616, P<0.01) and a weak correlation with the COPD clinical severity grade (r=0.253, P<0.05) (Table 6).

Table 6 The Correlation of mMRC Dyspnea Score and COPD Clinical Severity Grade with the Respective Scales of the Brief ICF-COPD Core Set

Discussion

The purpose of our study was to verify the reliability and validity of the Brief ICF-COPD core set and to provide an objective basis for its clinical application. To our knowledge, the Brief ICF-COPD core set was developed by 17 experts with different professional backgrounds from 8 different countries.³⁴ This assessment tool has a high degree of authority and credibility. This study indicates that the Brief ICF-COPD core set was consistent with the original version and can be used to assess impairments in body function and body structure, limitations in activity and participation and to explore environmental influences in Chinese patients with COPD.

The results from the current study supported both the reliability and validity of the Brief ICF-COPD core set. The Cronbach’s α correlation coefficient of the Brief ICF-COPD core set total scale was 0.873 in this study, reflecting that the internal consistency of the total scale was acceptable Cronbach’s α for body function and activity and participation was also higher than 0.70. At the same time, the body structure was only 0.640, indicating that the internal consistency was not great. The possible reasons are as follows: (1) the number of items in this component is too small; (2) the content or characteristics of respiratory structure and cardiovascular structure are not identical. Moreover, some experts also referred to only the category structure of the respiratory system (s430), which was enough as part of the component body structures of the Brief ICF core set.²⁰

The Brief ICF-COPD Core Set revealed good content validity in this study. The results showed that the I-CVI of 14 items except b450 was 1.00 (the I-CVI of b450 was 0.80). An I-CVI of 0.78 was the cutoff for either removing or retaining an item. According to the criteria of content validity, b450 (additional respiratory functions) was retained. Meanwhile, the S-CVI was greater than 0.80 (S-CVI was 0.943), which indicated satisfactory content validity.

In summary, adequate construct validity has been evidenced using CFA and convergent validity statistical approaches. With respect to the CFA (model 1, Figure 1), not all fit indices in the initial model met the criteria. Only the fit index IFI was better than 0.9, so we adjusted the initial model. The model was modified by adding double arrows between the following factors according to the correction index. (1) Respiration functions (b440) and structure of respiration system (s430): The pathological process of COPD can lead to repeated circulation of airway wall injury and repair process. The repair process leads to the remodeling of airway wall structure, the increase of collagen content and the formation of scar tissue, resulting in airway stenosis and fixed airway obstruction, which leads to the restricted airflow in COPD and the change of respiratory rhythm, depth and frequency; namely, the change of respiratory function.³⁴ (2) respiration functions (b440) and carrying out daily routine (d230): Chronic inflammation of airway mucosa, the parenchyma and pulmonary vessels, as well as remodeling of lung ventilation and ventilation function are affected, resulting in changes in respiratory function such as insufficient alveolar ventilation, resulting in hypoxemia, resulting in insufficient oxygen supply to important organs, such as the heart, brain and kidney, resulting in functional and organic damage and affecting patients’ ability to carry out a daily routine.³⁵ (3) structure of respiration system (s430) and walking (d450): Patients with COPD are characterized by small airway lesions. When COPD progresses to a certain extent, extensive small airway dysfunction and partial complete destruction of small airways will appear, which are the clinical manifestations of structural changes in the respiratory system, leading to pulmonary hyperventilation and emphysema and the secondary manifestations of decreased diffuse function.^36,37 Zeng showed that with the deterioration of pulmonary ventilation function, the increase in airway resistance, the decrease in diffusion function and quality of life in patients with COPD, and the six-minute walk test decreased accordingly.³⁸ Therefore, there is a correlation between changes in the structure of the respiratory system and walking. (4) carrying out daily routine (d230) and doing housework (d640): Activity of daily living (ADL) includes basic activity of daily living (BADL) and instrumental activity of daily living (IADL). IADL refers to a number of necessary activities for a person to live independently, such as doing housework, cooking, making phone calls, taking medicine, shopping and managing money.³⁹ Therefore, there is a correlation between carrying out daily routines and doing housework. (5) Products of substances for personal consumption (e110) and air quality (e260): The 2020 GOLD guidelines state that COPD is usually caused by long-term exposure to harmful particles or gases and air pollution, which can worsen the illness.⁴⁰ Therefore, each medication regimen should be individualized according to the severity of the symptoms. In the case of air pollution, some patients complained that the effects of the drugs they took might not be better than before (item d230 includes food and drugs), so there is a correlation between products or substances for personal consumption and air quality. Compared with the original research, the modified 4-factor model (model 2, Figure 2) showed a better model fit.

The convergent validity of the Brief ICF-COPD Core Set was supported by significant correlations between the mMRC and COPD clinical severity grade. The correlation coefficients between mMRC scores and body functions, body structures, and activity and participation of the Brief ICF-COPD Core Set showed moderate correlations (0.458–0.690) in the present study, indicating acceptable convergent validity.³³ Nevertheless, the correlation between COPD clinical severity grade and all respective scales/total of the Brief ICF-COPD Core Set was weak (−0.250, 0.177–0.363). The most likely reason is that the clinical severity of COPD diagnosed solely on the basis of lung function is not consistent with the degree of daily functional and structural impairment following the onset of the disease. Marin et al showed that the pulmonary function test results of patients with COPD at rest can only reflect the respiratory symptoms of patients in daily life to a certain extent, while the dyspnea of patients with COPD is mainly manifested after exercise, so it is limited to evaluate the severity of COPD only by the pulmonary function test results at rest.⁴¹

Several limitations need to be addressed in this study: (1) The participants were considered to be representative of Chinese patients with COPD who live in Tianjin city, and these patients may not be generalizable to the overall population in China. Therefore, future studies should recruit more participants from different regions. (2) The majority of participants were inpatients, with fewer outpatients, who had more severe or acute disease. The origin of the patients may have influenced the results of the study. (3) In terms of reliability in the research, this study did not involve interrater reliability, which may lead to investigator bias in these results.

Conclusion

The Brief ICF-COPD core set is a reliable and valid convenient instrument for assessing comprehensive problems in the functioning of patients with COPD. Based on the outcomes of the Brief ICF-COPD, rehabilitation strategies can be set. Meanwhile, the instrument may be used to monitor the long-term effects of rehabilitation and self-care.