Efficacy of a Self-Designed Questionnaire for Community Screening of COPD

Introduction

Chronic obstructive pulmonary disease (COPD) is a common, preventable, and treatable disease characterized by restricted airflow and persistent respiratory symptoms, and it is an important public health challenge because of its increasing prevalence and related disability and mortality worldwide.¹ According to the Global Initiative for Chronic Obstructive Lung Disease (GOLD) Report 2022 and the China Pulmonary Health study, the overall prevalence of spirometry-defined COPD among the general Chinese population aged over 20, 40, and 60 years were 8.6%. 13.7%, and 27%, respectively.^1,2 The number of years of life lost (YLLs) due to premature death caused by COPD accounted for 81.2% of disability-adjusted life years (DALYs) in the world.³ Furthermore, data from the Chinese Centre for Disease Control and Prevention indicated that COPD ranked as the fourth leading cause of YLLs in China, with a case mortality is 68 deaths per 100,000 population.⁴ Early detection, diagnosis, and intervention need more attention in the prevention and treatment of COPD in China.⁵ However, there remains a deficit in screening for COPD. Indeed, patients with mild airflow limitation in the early disease stages usually have no symptoms, and those with sedentary lifestyles usually fail to appreciate the extent of their declining exercise tolerance.⁶ In addition, because community medical resources in China are relatively undeveloped, the cost of spirometry screening is high, and trained operators are limited, there is considerable underdiagnosis of COPD, especially for community-dwelling patients in the early stages of disease.^7,8

In order to optimize the application of medical resources and improve the efficiency of screening, a COPD screening questionnaire has been recommended by the “Chronic obstructive pulmonary disease diagnosis and treatment guidelines at the primary level, China, 2018” for primary health care settings, in which spirometry was not available.⁹ In China, certain screening questionnaires are used to identify high-risk groups and then followed by spirometry. Nonetheless, many of those commonly used questionnaires in China do not focus on respiratory symptoms but rely mainly on the assessment of risk factors such as age, smoking habits, and environmental exposure. As a result, non-smokers are usually fail to be identified with modest sensitivity and specificity.¹⁰ In addition, the influence of gender on the optimum diagnostic cut-off values of the screening questionnaires has not been considered. Specifically, some commonly used questionnaires in China including the self-scored COPD population screener (COPD-PS) and the COPD screening questionnaire (COPD-SQ), also have several limitations. The COPD-PS, developed in the United States, principally focuses on smoking habits and does not take into account the widespread use of biofuels in China. Furthermore, the COPD-SQ, developed in China, involves measuring height and weight at the time of screening to calculate body mass index (BMI), which leads to longer screening times.^11–14

We therefore develop a simple screening questionnaire based on the COPD-PS and COPD-SQ that includes only five questions, named the COPD-MH, and implement it in a community screening in Minhang, China. Moreover, considering the gender disparity in lifestyles, we also assess the relevant validity of the aforementioned screening questionnaires in both males and females. This study aims to assess and compare the validity of the COPD-PS, COPD-SQ, and COPD-MH, as well as to explore potential ways for improving the efficiency of community screening for COPD.

Materials and Methods

Screening Participants

From November 2019 to August 2020, a screening for COPD was conducted among residents over 40 years of age in three communities (Jiangchuan, Maqiao, and Wujing), Minhang District, Shanghai. The inclusion criteria were as follows: 1) resident in Jiangchuan Street, Maqiao Town, or Wujing Town of Minhang District for more than 1 year; 2) ≥40 years of age; 3) provided written informed consent. Individuals were excluded if they did not actively cooperate; could not understand the questionnaire; were diagnosed with a mental or behavioral disorder; had severe cardiovascular or cerebrovascular disease; or for whom spirometry tests were contraindicated. The study was approved by the Ethics Committee of Shanghai Fifth people’s Hospital (2018 Ethics Approval No. 130), and all the participants signed written informed consent.

Screening Questionnaires

A self-designed COPD-MH screening questionnaire was created based on the COPD-PS and COPD-SQ after a literature review and an assessment of the characteristics, risk factors, symptoms, and other environmental factors of the local population. The questionnaire was trialed in the Zhuanqiao Community in Minhang District, Shanghai, among residents over 40 years old and amended as appropriate. Considering that it is difficult to improve an asymptomatic individual and there is still no direct evidence that screening for COPD in asymptomatic patients has a positive effect on health-related quality of life, morbidity, or mortality,¹⁵ we focused on COPD-related symptoms. The revised questionnaire included five questions: three assessed COPD-related symptoms, one asked about age, and one asked about smoking habits.¹⁶ The maximum score for each question was 4, with a highest total score of 20. Conversely, the COPD-PS comprises five questions, including age, smoking, and three symptom items, with a highest total score of 10 points and a recommended cut-off score of 5 points.¹⁴ Additionally, the COPD-SQ comprises seven questions, with added items related to BMI, family history of respiratory disease, and exposure to biomass smoke; the highest total score is 38 points and the recommended cut-off score is 16 points.^8,17 The COPD-MH focused on symptoms, and each item was set the same answer for convenience. The differences between the questionnaires are presented in Table 1 and full questionnaires can be seen in the Supplementary Materials.

Table 1 Differences Between Items and Scores of the Three Questionnaires

Questionnaire Investigation

The questionnaire was administered by respiratory physicians working at Shanghai Fifth People’s Hospital and general practitioners at the Minhang Community Health Service Center, and all the interviewers were well-trained before the questionnaire administration. All the enrolled participants who satisfied the inclusion criteria completed the questionnaire investigation and underwent spirometry tests. In order to compare the time taken to complete these screening questionnaires (the COPD-PS, COPD-SQ, and COPD-MH), we conducted a pilot study with 80 residents. The time taken to complete each questionnaire for each participant was recorded, and the results of the questionnaires were collated.

Spirometry Tests

The study used a microQuark-PONY FX spirometer and 400 µg Salbutamol Sulphate Aerosol (Ventolin) as a short-acting bronchodilator. The spirometry tests were carried out according to the 2014 “Guidelines for Pulmonary function examination (part II)-spirometer examination”, and the same brand and standard equipment were used for all tests. According to the guidelines, body temperature, ambient barometric pressure, and saturated water vapor (BTPS) were adjusted as physiological conditions. Volume calibration verification was undertaken daily using a 3-L calibration cylinder at least three times. Linear calibration verification was undertaken once per week with low, medium, and high flow rates, at least three times for each rate.¹⁸ Every participant’s age, height, and weight were recorded.

A diagnosis of COPD and the existence of irreversible airflow limitation was confirmed when forced expiratory volume in the first second/forced vital capacity (FEV1/FVC) after inhalation of a short-acting bronchodilator was <70%, after the exclusion of other causative factors. The severity of airflow limitation (GOLD1-GOLD4) was rated according to the GOLD guidelines and based on the FEV1/Pred%. The flow of the study and screening results are presented in Figure 1.

Figure 1 The flow of the study and results of screening.Abbreviations: FEV1/FVC, forced expiratory volume in the first second/forced vital capacity; GOLD, Global Initiative for Chronic Obstructive Lung Disease.

Statistical Analysis

Data with normal distributions were presented as mean ± standard deviation, and Student’s t-tests were applied for comparisons. Data with non-normal distributions were presented as mean (interquartile range [IQR]), and non-parametric tests were used for the comparisons. The receiver operating characteristic (ROC) curve was plotted and the area under the curve (AUC) was calculated to assess the predictive effects of each screening questionnaire for COPD.¹⁹

Sensitivity and specificity were calculated to evaluate the validity of screening; the predictive value (PPV and NPV) and likelihood ratio (LR+, LR-) were used to evaluate the efficacy of screening. Youden’s index was calculated as “sensitivity + specificity −1”, and the score for which the Youden’s index was maximal was set as the optimal cut-off value for each questionnaire. The pairwise comparison of AUCs for different questionnaires was performed using the Delong test.²⁰ The differences in AUCs for the questionnaires were statistically significant (p < 0.05). SPSS, version 23.0 (International Business Machines Corporation, USA) and MedCalc, version 20.011 (MedCalc Software Ltd, USA) software were used for the statistical analyses.

Results

Characteristics of Participants

A total of 1197 participants were included, and those with incomplete data were excluded from the current analysis. Table 2 shows the demographic characteristics and spirometry results for the 1023 participants with complete questionnaire information and qualified spirometry results. Of the 1023 participants, 43.0% were males. The mean age was 67.45 (±9.74) years, ranging from 40 to 92 years. Of those 127 participants diagnosed with COPD, 78 (17.7%) were male and 49 were female (8.4%). Compared with females, males were more likely to have COPD, be smokers, have higher BMI, and have higher scores in COPD-PS, COPD-SQ, and COPD-MH screening questionnaires. There were no significant differences in age, daily cough, shortness of breath, biofuel exposure, family history of respiratory diseases, or severity of airflow limitation between males and females.

Table 2 Characteristics of Participants with Complete Questionnaire Data and Spirometry Results

Diagnostic Validity and Yield of Screening Questionnaire

The ROC curves for the three questionnaires are shown in Figure 2. The AUCs for COPD-PS, COPD-SQ, and COPD-MH were 0.646 (95% CI [0.616, 0.675]), 0.659 (95% CI [0.629, 0.688]), and 0.659 (95% CI [0.629, 0.688]), respectively. Results from the ROC curve analyses in terms of the identification of COPD by the three scales were not significant (AUC of COPD-SQ vs COPD-PS, p = 0.93; AUC of COPD-MH vs COPD-PS, p = 0.45; AUC of COPD-SQ vs COPD-MH, p = 0.42), suggesting comparable predictive validity of three questionnaires, while COPD-MH required the least information to be collected. To obtain a maximum Youden’s index for COPD diagnosis, the optimum cut-off values for COPD-PS, COPD-SQ, and COPD-MH were set as 4, 16, and 7, respectively. The relevant predictive values and likelihood ratios under certain cut-off values are presented in Table 3. All these questionnaires had PPVs around only 20% but relatively high NPVs of more than 90%.

Table 3 ROC Curve and Diagnostic Values of the Three COPD Screening Questionnaires

Figure 2 Comparison of ROC curves of the three COPD screening questionnaires. The blue curve is the ROC curve of COPD-PS, the green curve is ROC curve of COPD-SQ, and the Orange curve is of COPD-MH. The marker points are shown as empty circles representing the best cut-off values. Identification of COPD by the three questionnaires were similar. Abbreviation: ROC, receiver operator characteristic.

Gender Disparity on Different Screening Questionnaires

The ROC curves for gender-stratified analyses are shown in Figure 3. Gender had an effect on the results for the three screening questionnaires (Table 4). To achieve a maximum Youden’s Index, the cut-off values for the COPD-PS for males and females were set as 4 and 3, respectively, and the corresponding sensitivity for males (84.62%) was much higher than for females (44.9%). However, the Delong method showed no significant differences in AUCs between males and females. Nevertheless, the AUC of the COPD-SQ was lower in males than in females (0.606 vs 0.676), similar to the results for the COPD-MH. The optimum cut-off value for the COPD-MH for males was 7 (AUC 0.592, sensitivity 80.77%) and for females was 5 (AUC 0.622, sensitivity 77.5%). For COPD-SQ, sensitivity for females was higher than males. The self-designed COPD-MH had higher diagnostic efficiency and higher sensitivity with gender-specific cut-off values for both males and females.

Table 4 Effect of Gender Difference on Cut-off Value, AUC, and Validity of the Three Screening Questionnaires

Figure 3 Gender disparity on different screening questionnaires. A gender value of 1 represents male and 2 represents female. (A) shows the ROC curve of gender disparity for the COPD-PS and the sensitivity for females is higher than males; (B) is the ROC curve of gender disparity for the COPD-SQ; (C) is the ROC curve of gender disparity for the COPD-MH. The marker points are shown as empty circles representing the best cut-off values. The self-designed COPD-MH had higher diagnostic efficiency and higher sensitivity with gender-specific cut-off values for both males and females.

Impact of Severity of COPD on Questionnaire Score and Diagnostic Accuracy

The questionnaire scores of patients with COPD were higher than those of participants without COPD, but there were no significant differences between GOLD1 and GOLD4 when analyzed by Least Significance Difference (LSD). Moreover, the severity of COPD had no impact on diagnostic accuracy (Table 5).

Table 5 Impact of the Severity of COPD on Questionnaire Scores and Diagnostic Accuracy

Comparison of the Convenience of the Three Screening Questionnaires

The average time taken to complete the COPD-PS, COPD-SQ, and COPD-MH questionnaire in the 80 subjects was 2.25 ± 1.25 min, 3.5 ± 1.5 min, and 1.75 ± 0.75 min, respectively. Considering the readability and convenience of the questionnaires, among the 15 interviewers who completed the questionnaires, one preferred the COPD-PS (6.67%), two preferred the COPD-SQ (13.33%), and 12 preferred the COPD-MH (80%).

Discussion

Chronic obstructive pulmonary disease is a hidden and slowly progressing disease, especially in the elderly, and can be easily mistaken for age-related physical decline. As the disease progresses, patients experience frequent exacerbations and repeated hospitalizations. A 2015 multicenter study of 1634 COPD patients showed that the average financial burden of COPD was 20,197.57 yuan per patient per year, with only 2.5% of patients spending less than their daily salary over 30 days.²¹ The high cost of hospitalization and the heavy economic burden of disease highlights the importance of early diagnosis and treatment. Timely diagnosis and treatment before the disease develops into respiratory failure is vital for reducing the mortality and disease burden.

In a study of COPD awareness, only 10% of respondents knew about COPD, and fewer had ever undergone pulmonary function testing.² Furthermore, a survey of general practitioners in the Xuhui District of Shanghai found that the assessment of COPD by general practitioners was inaccurate, with only 12.1% being familiar with the symptoms associated with acute exacerbations and 30.5% with accurate pulmonary function grading.²² Finally, in a study of 131 community health service centers in Shanghai, only 21%were equipped with pulmonary function monitors.²³ Early intervention in COPD can effectively delay disease progression,²⁴ but low awareness of the disease among residents, poor diagnosis recognition and treatment by primary care physicians, and a shortage of primary medical care resources mean that too few cases of COPD are recognized. In addition, the criteria for diagnoses are not standardized in hospitals. The results of the first survey on the use of pulmonary function screening in China revealed that an imbalance in the use of pulmonary function screening combined with a lack of pulmonary function screening programs in primary health care systems have severely limited the screening of COPD in the community.²⁵ Concurrently, research on the early diagnosis and treatment of COPD has focused on the rapid identification of the high-risk group of COPD patients. In recent years, scholars in China and internationally have improved the efficiency of COPD screening with the use of screening questionnaires. Nonetheless, during their practical application in this epidemiological survey, we established that the optimum cut-off values for diagnosis differed in men and women for the same screening questionnaire.

At present, the most widely used COPD screening questionnaires in China and elsewhere include the COPD-PS, COPD-SQ, and COPD-CDQ. The COPD-PS is a screening questionnaire that was developed by the American COPD foundation and has been applied to the general population. It is recommended by the primary diagnosis and treatment of COPD guidelines in China. Nonetheless, in practice, direct English translations of the questionnaire are difficult to understand; additionally, the questionnaire may not be appropriate in China since the risk factors focus only on those related to cigarette smoking and the best cut-off value is controversial. Our results showed that the optimal truncation value of the COPD-PS questionnaire was 4 points, with a sensitivity of 61.42% and a specificity of 64.99%. The sensitivity increased to 84.62% when applied to males only. When American scholars developed the questionnaire, they determined the best cut-off value at 5–6 points, with a PPV of 56.8% and NPV of 86.4%.¹⁴ The sensitivity and specificity of the questionnaire were 67.1% and 72.9%, respectively.²⁶ In a Japanese study, the reported best cut-off value was the same as that identified in our study, but the sensitivity and specificity differed.

The COPD-SQ questionnaire is most commonly used in China and recommends a score of 16 as the best cut-off, with sensitivity and specificity of 60.6% and 85.2%, respectively. The area under the ROC curve was found to be 0.812, and the reliability was 0.991.⁸ In 2017, the research group used the questionnaire to conduct a cross-sectional survey of people over the age of 40 years, and the results showed that the sensitivity and specificity of the questionnaire were 76.8% and 75.7% respectively.²⁷ In this study, the sensitivity and specificity were 59.84% and 61.98%, respectively, with a median score of 16 points. Because biofuels are a common risk factor for COPD in Chinese women, the questions on the COPD-SQ include a history of exposure to cooking-related biological fumes. Nonetheless, a stratified analysis by gender revealed that, while the diagnostic efficiency and sensitivity were higher for female than male patients when choosing separate best cut-off values, they were lower than those of the COPD-PS and COPD-MH. Therefore, these findings suggest that the inclusion of a history of bio-fuel exposure in the screening questionnaire has no significant effect on screening efficacy. Although the questionnaire is the only one developed and tailored to the realities in China, there are some differences between the results of this study and previous ones for the COPD-SQ questionnaire. In addition, the need to calculate patient BMI based on accurate measurement of height and weight means that the screening process takes longer for this questionnaire. Finally, the severity of COPD did not affect the accuracy of diagnosis and did not support early diagnosis of the disease.

The COPD-MH questionnaire designed in this study was divided into five sections, each with a score of 0–4, and the questions were very short. The time taken to complete the questionnaire, the convenience for the investigator, and the understanding of the subjects were significantly better than the control questionnaires. When the cut-off value was 7, the sensitivity of the COPD-MH was better than that for the COPD-PS and COPD-SQ. The sensitivity was 80.77% for male patients with a cut-off value of 7 and 77.5% for females with a cut-off value of 5, which showed high sensitivity in both males and females.

The strengths of our study include the large sample size, the directly measured pulmonary function, which strictly complied with the 2014 guidelines by professional physicians, and the records on time consumption for each questionnaire. Our study also has several limitations. Firstly, the “Standardization of Spirometry 2019 Update” added some new contraindications and updated some criteria of spirometry²⁸ that should be addressed in further studies. Secondly, we used FEV1/FVC <0.70 as a diagnostic criterion, which may have resulted in an overestimation of COPD in elderly patients. In addition, we did not have repeated measures over time and were unable to directly address the potential issue of reliability. However, the questions in our self-designed questionnaire were from the COPD-SQ and COPD-PS, which have been evaluated and shown reliable results. Finally, the findings of our study conducted in rural Shanghai may not be applicable to other populations.

Conclusion

The best cut-off values for men and women differed between all three COPD screening questionnaires. COPD-MH is comparable to and less time-consuming than the existing screening methods for COPD.