Comparing the Performance of Two Screening Questionnaires for Chronic Obstructive Pulmonary Disease in the Chinese General Population

Introduction

Chronic obstructive pulmonary disease (COPD) is common worldwide and represents a significant economic and social burden. The Global Burden of Disease (GBD) study reported that the global prevalence of COPD reached 212.3 million in 2019, accounting for 3.3 million deaths and 74.4 million disability-adjusted life-years (DALYs).¹ The China Pulmonary Health (CPH) study reported a COPD prevalence of 8.6% among the Chinese general population, with 9.6% in rural areas and 7.4% in urban areas, accounting for 99.9 million people overall in China.² COPD was estimated to be the third leading cause of DALYs in China in 2017.³ However, most COPD cases are undiagnosed and few patients undergo spirometry,² which is the current diagnostic standard for COPD.⁴

Especially considering COPD’s heavy personal and societal burden, early diagnosis is key to optimal disease management. Although spirometry is required for diagnosis, broad screening for COPD in the general population with spirometry is not recommended due to the significant waste of healthcare resources that would result.⁵ Simple screening questionnaires were developed to help identify high-risk subjects for further spirometry tests, thereby improving the rate of diagnosis and optimizing the use of healthcare resources.

The COPD population screener (COPD-PS) and COPD screening questionnaire (COPD-SQ) are two commonly used screening questionnaires in primary care. The COPD-PS is a brief questionnaire developed in the United States,⁶ and has been validated in a health examination center at a tertiary hospital in China.⁷ It was recommended by the Chinese guidelines for primary care of COPD (2018)⁸ for screening individuals at a high risk of COPD. The COPD-SQ was developed by Chinese experts and validated in the Chinese population.⁹ This questionnaire assesses common risk factors for COPD in low- and middle-income countries (LMICs), such as biomass. The Chinese guidelines for the diagnosis and management of COPD (revised version 2021)¹⁰ suggested using the COPD-SQ when spirometry is not available at primary care centers. A recent case-control study demonstrated that these two questionnaires had similar diagnostic accuracy.¹¹ However, that study was performed using hospital patients rather than the general population. Considering the heavy societal burden of COPD and the urgent need to identify COPD patients as early as possible, it is important to determine which of the two guideline-recommended questionnaires is more suitable for screening for COPD among the general population.

Unbalanced socioeconomic development is a major problem in LMICs, with significantly different living conditions between their rural and urban areas. Smoking remains the leading cause of COPD in high income areas, while household air pollution may be more important in low-income settings.¹² Due to the significant differences that exist in the living conditions and household incomes of rural and urban families, the risk factors for COPD, which are an important part of COPD screening questionnaires, may be also different. However, the impact of urbanization on the performance of different COPD screening questionnaires is currently unknown.

The present study was designed to compare the screening performance of the COPD-PS and COPD-SQ for COPD diagnosed with spirometry alone or spirometry combined with respiratory symptoms in a general Chinese population, with further stratification based on urbanization.

Materials and Methods

Study Design and Population

Subjects who underwent a health checkup at three community health centers were consecutively recruited between June 2021 and October 2022 from an urban district (Gaobeidian, Chaoyang District) and two rural districts (Liyuan, Tongzhou District; Xinchengzi, Miyun District) of Beijing, China. Inclusion criteria were local residents aged 40 years or older. Exclusion criteria were subjects who were unable to perform spirometry (ie due to dementia or mental retardation) or who were contraindicated for spirometry (ie due to a thoracic, abdominal, or eye surgery, a respiratory infection, hemoptysis, severe angina, an arrhythmia in the last 4 weeks, a myocardial infarction or stroke in the last 3 months, uncontrolled high blood pressure, aortic aneurysm, retinal detachment, pregnancy, or breastfeeding). This study was approved by the Ethics Committee of Beijing Chao-Yang Hospital (no. 2021-6-22-2) in accordance with the Declaration of Helsinki. All subjects signed informed consent before participating in the study.

Questionnaires and Data Collection

Prior to spirometry measurement, trained coordinators administered a questionnaire to the participants that integrated the questions of the COPD-PS and COPD-SQ, including demographic characteristics, respiratory symptoms and risk factors. The COPD-PS questionnaire is comprised of 5 items that measure involving age, cigarette smoking, sputum/phlegm production, shortness of breath and functional impact due to breathing problems. The score for each item ranges from 0 to 2, permitting a total score of 0 to 10. We used the translated version of COPD-PS questionnaire from the Chinese guidelines for primary care of COPD (2018).⁸ The COPD-SQ consists of seven items: two on COPD-related symptoms (cough, dyspnea), two on demographic data (age, BMI), two on exposure to nicotine or biomass smoke and one on a family history of respiratory disease. The COPD-SQ that we used was derived from Zhou et al⁹ and the Chinese guidelines for the diagnosis and management of COPD (revised version 2021).¹⁰ Ever smokers were defined as current or former smokers who have smoked more than 100 cigarettes in their lifetime. A chest X-ray was performed to exclude serious alternative diagnoses (bronchiectasis, pulmonary fibrosis and pleural disease).

Spirometry

Spirometry measurements were performed by trained technicians using a portable spirometer (MasterScreen Pneumo, Vyaire Medical GmbH, Hoechberg, Germany) following the recommendations of the American Thoracic Society and the European Respiratory Society.¹³ Spirometer calibration was done with a 3 L syringe at the start of each day. We required participants to perform at least three forced expiratory maneuvers, of which the highest forced expiratory volume in 1 s (FEV₁) and forced vital capacity (FVC) values were recorded. Subjects with a pre-bronchodilator (BD) FEV₁/FVC of less than 70% underwent post-BD spirometry 20 min after receiving 400 μg of salbutamol aerosol via a metered-dose inhaler and spacer. All spirometric data were reviewed by an expert panel on the basis of the criteria and reference values of the American Thoracic Society and the European Respiratory Society.^13,14 Poor-quality data were excluded from our analysis.

Definition of COPD

COPD was diagnosed according to the following two criteria: (I) Spirometry-defined COPD: subjects with a post-bronchodilator FEV₁/FVC<70%; (II) Symptomatic COPD: subjects with a post-bronchodilator FEV₁/FVC<70% and concomitant respiratory symptoms. Respiratory symptoms were defined as a score of 1 or more on at least one of the following screening questions: (I) cough: “Do you usually have a cough without a cold?” (COPD-SQ); (II) dyspnea: “Which is the best description for your dyspnea?” (COPD-SQ); (III) phlegm: “Do you ever cough up any ‘stuff’, such as mucus or phlegm?” (COPD-PS); (IV) short of breath: “During the past 4 weeks, how much of the time did you feel short of breath?” (COPD-PS); and (V) activity limitation: “I do less than I used to because of my breathing problems in the past 12 months.” (COPD-PS).

Statistical Analysis

Quantitative variables were described as means and standard deviations (SDs) for normally distributed data and medians and interquartile ranges (IQRs) for non-normally distributed data. Qualitative variables were expressed as absolute and relative frequencies. The differences between patients with and without COPD were analyzed with Student’s t-test or the Mann–Whitney U-test for normally or non-normally distributed quantitative data. Chi-squared tests were used for qualitative data. The receiver operating characteristic (ROC) curve and the area under the ROC curve (AUC) were used to assess the discriminatory power of the COPD-PS and COPD-SQ for COPD graphically and quantitatively, and the binomial exact confidence interval (CI) for the AUC was calculated. An AUC of ≥0.9 was considered “excellent”, ≥0.8 “good”, ≥0.7 “fair”, and ≥0.6 “poor”.¹⁵ Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), correct classification ratio, positive likelihood ratio (+LR) and negative likelihood ratio (-LR) were established for the different questionnaires and cut-off values. The Youden index was used to identify the optimal cut-off score based on sensitivity and specificity. Comparisons between the AUCs of the two questionnaires in all subjects and subjects stratified by urbanization were preformed using the Delong test.¹⁶ Comparisons between the sensitivity and specificity of different questionnaires were analyzed using McNemar’s test. A p-value < 0.05 was considered statistically significant. All analyses were performed using SPSS version 23.0 (IBM, Armonk, NY, USA) and MedCalc version 20.0 (MedCalc Software, Ostend, Belgium).

Results

Participant Characteristics

The study enrolment flowchart is shown in Figure 1. We recruited 1629 subjects who underwent a health check-up at community health centers. Of these, 224 were ineligible for this study (175 because of an age under 40 years old, 4 subjects refused to participate and 45 subjects were unable to perform spirometry). The remaining 1405 subjects provided informed consent and completed both the questionnaires and the spirometry test. After checking the collected data, a further 55 subjects were excluded from our analysis (14 because of alternative diagnoses detected on X-ray and 41 because of unreliable spirometry results). Finally, 1350 subjects with complete and valid data were included in the analysis.

Figure 1 Subject selection flowchart. Abbreviation: COPD, chronic obstructive pulmonary disease.

Table 1 shows the demographic data, spirometry results and screening questionnaires scores of the enrolled subjects stratified by COPD diagnosis according to different criteria. The mean age of subjects was 59.0 (SD 10.1), 555 (41.1%) were men, and the proportion of urban residents was 40.6%. Four-hundred-eighty-two (35.7%) subjects were ever smokers, with a median smoking history of 0.0 (IQR: 0.0–15.3) pack-years. 129 (9.6%) subjects were classified as spirometry-defined COPD, while 92 (6.8%) subjects were classified as symptomatic COPD. Both spirometry-defined COPD and symptomatic COPD subjects were older, more likely to be male, more likely to be ever smokers, had a higher number of smoking pack-years and a lower BMI than non-COPD subjects. The scores of the two screening questionnaires were significantly higher in COPD groups compared with non-COPD groups when using either diagnostic criterion.

Table 1 Comparison of Subjects with and without COPD Using Different Diagnostic Criteria

Operating Characteristics of the Two Questionnaires at Various Cut-off Values

The operating characteristics of the COPD-PS are shown in Table 2. The optimum cut-off score for detecting spirometry-defined COPD was 4, with a sensitivity of 58.1% and a specificity of 76.2%. In contrast, the recommended cut-off of 5 had a relatively low sensitivity of 40.3%. The optimum cut-off score for detecting symptomatic COPD was 5, which matches the recommended cut-off. The sensitivity, specificity, PPV, NPV, correct classification ratio, + LR, and - LR were 56.5%, 90.2%, 29.7%, 96.6%, 87.9%, 5.78 and 0.48, respectively.

Table 2 Performance of the COPD-PS at Various Cut-off Values in Screening for Spirometry-Defined COPD or Symptomatic COPD

The optimum cut-off score of the COPD-SQ was 15 for both spirometry-defined COPD and symptomatic COPD (Table 3). The sensitivity, specificity, PPV, NPV, correct classification ratio, + LR, and - LR were respectively 62.8%, 77.6%, 22.9%, 95.2%, 76.2%, 2.81 and 0.48 for spirometry-defined COPD, and 78.3%, 77.6%, 20.3%, 98.0%, 77.6%, 3.49 and 0.28 for symptomatic COPD.

Table 3 Performance of the COPD-SQ at Various Cut-off Values in Screening for Spirometry-Defined COPD or Symptomatic COPD

Comparation of the Screening Performance of the Two Questionnaires

The AUC values of the COPD-PS and COPD-SQ for spirometry-defined COPD were 0.726 (95% CI [0.702, 0.750]) and 0.749 (95% CI [0.725, 0.772]), respectively, suggesting moderate diagnostic accuracy (Figure 2). The AUC values of the COPD-PS and COPD-SQ for symptomatic COPD were 0.799 (95% CI [0.776, 0.820]) and 0.823 (95% CI [0.801, 0.843]), respectively, suggesting moderate to good diagnostic accuracy (Figure 3).

Figure 2 Receiver operating characteristic curve of the COPD population screener (COPD-PS) and COPD screening questionnaire (COPD-SQ) for screening for spirometry-defined COPD.

Figure 3 Receiver operating characteristic curve of the COPD population screener (COPD-PS) and COPD screening questionnaire (COPD-SQ) for screening for symptomatic COPD.

When using the cut-off score defined by the maximum Youden index to screen for spirometry-defined COPD, COPD-PS ≥ 4 and COPD-SQ ≥ 15 had similar sensitivities (58.1% vs 62.8%, P = 0.377), specificities (76.2% vs 77.6%, P = 0.268) and AUC values (0.672 vs 0.702, P = 0.179) (Table 4). While using the recommended cut-off score, COPD-SQ showed a higher sensitivity (56.6% vs 40.3%, P < 0.001) but a lower specificity (82.2% vs 89.9%, P < 0.001) than COPD-PS. For symptomatic COPD, COPD-PS ≥ 5 and COPD-SQ ≥ 15 (cut-off score with the maximum Youden index) had similar AUC values (0.734 vs 0.779, P = 0.095). The sensitivity of the COPD-SQ was significantly higher than the COPD-PS (78.3% vs 56.5%, P < 0.001), and the specificity of the COPD-PS was significantly higher than the COPD-SQ (90.2% vs 77.6%, P < 0.001). Similar results were also achieved when using the recommended cut-off scores of COPD-PS ≥ 5 and COPD-SQ ≥ 16 for comparation.

Table 4 Comparation of the Screening Performances of the COPD-PS and COPD-SQ

Effects of Urbanization on Questionnaire Performance

The ROC curve analysis stratified by urbanization are shown in Table 5. The optimum cut-off score for both questionnaires was higher in rural districts than urban districts. In rural districts, the AUC value of COPD-SQ tended to be higher than that of COPD-PS for spirometry-defined COPD (0.700 vs 0.653, P = 0.093), while the AUC values of the two questionnaires for symptomatic COPD were comparable (0.725 vs 0.766, P = 0.209). In urban districts, the AUC values of the two questionnaires were similar for both spirometry-defined COPD (0.720 vs 0.734, P = 0.715) and symptomatic COPD (0.792 vs 0.808, P = 0.721).

Table 5 Effect of Urbanization on the Optimum Cut-off Scores, AUC and Validity of the Two Screening Questionnaires

Discussion

This study prospectively evaluated the performance of two brief COPD screening questionnaires in the general Chinese population. The optimum cut-off score for the COPD-PS was 4 for spirometry-defined COPD and 5 for symptomatic COPD. The optimum cut-off score for the COPD-SQ was 15 for both spirometry-defined COPD and symptomatic COPD. The two questionnaires showed moderate discriminatory power for spirometry-defined COPD and moderate to good discriminatory power for symptomatic COPD. The discriminatory power of COPD-SQ tended to be higher than that of the COPD-PS for spirometry-defined COPD in rural areas. To our knowledge, this is the first study to investigate the effects of urbanization on COPD screening questionnaire performance in the general population.

Underdiagnosis of COPD is a major health problem worldwide, especially in LMICs. The PLATINO study in Latin America reported that the prevalence of COPD ranged from 7.8% in Mexico City to 19.7% in Montevideo,¹⁷ and among patients with spirometry categorized of GOLD 2–4, only 19–20% had a previous medical diagnosis of COPD, emphysema or chronic bronchitis.¹⁸ The China Pulmonary Health (CPH) study reported that 99.9 million adults aged 20 years or older suffered from COPD in China, but only 2.6% of them were aware of their illness and only 12.0% of them had undergone pulmonary function test previously.² Though spirometry is the most important tool for diagnosing of COPD, it is not commonly integrated into clinical practice in primary hospitals and community health centers. Screening spirometry in asymptomatic subjects without a history of smoking or exposure to some other noxious stimulus may result in wasted healthcare resources.¹⁹ Various screening or case-finding questionnaires were therefore developed to identify individuals with undiagnosed COPD or at a high risk of COPD. Two brief questionnaires were recommended by the Chinese guidelines for identifying subjects at a high risk of COPD in the primary care setting: the COPD-PS and COPD-SQ. The present study found that the two questionnaires possessed similar discriminatory power for detecting both spirometry-defined COPD and symptomatic COPD in the general population.

The COPD-PS questionnaire was developed by Martinez et al⁶ using subjects from general practice and specialist sites in the United States. A cut-off score of ≥ 5 was recommended by the authors for identifying undiagnosed airflow obstruction, which was the same as the criteria for spirometry-defined COPD that was used in this study. However, we showed that the optimum cut-off score for screening for airflow obstruction in the Chinese general population was 4, which was in line with a previous study that validated the COPD-PS with subjects from a health examination center at a tertiary hospital in southwest China.⁷ Another study from Japan also established that COPD-PS ≥ 4 was the best cut-off for identifying persistent airflow obstruction in the general population, with a sensitivity of 67.1%, specificity of 72.9% and AUC of 0.70.²⁰ This discrepancy in the optimum cut-off score identified by different studies may be attributed to the fact that the above studies were performed on a different ethnic population or in a different target population, such as on hospital visitors or the general population.

The COPD-SQ was developed by Zhou et al⁹ to meet the characteristics of COPD patients in China. The development of this questionnaire was based on data from a large-scale Chinese epidemiological study,²¹ and the validation phase was performed using subjects from both urban and rural areas in Guangdong Province.⁹ In our study, the optimum cut-off score of COPD-SQ was achieved at 15, which is lower than the recommended cut-off. Our results showed that the AUCs of COPD-SQ ≥ 15 and COPD-SQ ≥ 16 for spirometry-defined COPD were 0.702 and 0.694, respectively. Another study from northern China reported that the optimal cut-off was 17 points with an AUC value of 0.653 in that population.¹¹ However, their results were not entirely comparable with our own because they were based on patients at a tertiary hospital. In the present study, the sensitivity and specificity of COPD-SQ ≥ 16 for spirometry-defined COPD were 56.6% and 82.2%, respectively. These results were similar to the findings by Pan et al²² which reported that the COPD-SQ had a sensitivity of 55.3% and specificity of 77.3% for detecting airflow obstruction in Chinese primary care population. But the gold standard they used was lower limit of normal rather than the fixed ratio.

The COPD-PS and COPD-SQ had moderate discriminatory power for spirometry-defined COPD in the present study, with comparable sensitivity and specificity. This is in concordance with the study from Yang et al²³ who reported similar AUC values for the COPD-PS and COPD-SQ to screen for spirometry-defined COPD among the general population of three rural communities in Shanghai, China. However, the PPV of the COPD-PS and COPD-SQ was not high, which means that among individuals who had a positive questionnaire screening result, the probability of having COPD was not high. This result is mainly due to the low prevalence of COPD in the general population.

According to the updated guidelines, the diagnosis of COPD was based on a comprehensive measurement of risk factors, chronic respiratory symptoms and confirmed by the presence of an incompletely reversible airflow limitation (FEV₁/FVC <70%).⁴ Spirometry is the gold standard for COPD and facilitates communication across the global medical community, but predicts poorly of symptoms.²⁴ Respiratory symptoms are sometimes under-recognized by individuals due to a blunted perception of dyspnoea,^25–27 so it is unclear whether so-called “asymptomatic” subjects really have no symptoms at all or just have poorly perceived symptoms. Asymptomatic, undiagnosed COPD also had an increased risk of exacerbation and pneumonia compared with smokers without COPD.²⁸ This study therefore defined COPD by spirometry-defined or combined with respiratory symptoms to identify both symptomatic and asymptomatic subjects for early intervention and treatment.

The COPD-PS and COPD-SQ both contain questions about risk factors for COPD. However, the main risk factors for COPD can differ between different environments and different socioeconomic levels. Our study first investigated the effects of urbanization on the results of different COPD screening questionnaires. We found that the AUC value of the COPD-SQ tended to be higher than that of the COPD-PS for detecting spirometry-defined COPD in subjects from a rural district. This may be due to the COPD-SQ containing items regarding biomass smoke, which is the leading risk factor for COPD in rural areas. Exposure to household air pollutants originating from biomass smoke can cause lung function decline and pronounced airway inflammation, resulting in a higher risk of COPD.^29,30 The global burden of COPD attributable to household air pollution was more pronounced in low socio-demographic index (SDI) regions.³¹ A study from Guangdong province found that exposure to biomass fuel for cooking was significantly associated with COPD among non-smoking women in rural South China.³² Our results indicate that since COPD-PS was developed in the United States, which was a high-income country, it was more generalizable to high-income settings whereas the COPD-SQ was more suitable for COPD screening in low-income settings. Considering the imbalance of socioeconomic development between different regions of one country, when using questionnaires to screen for COPD in different areas, a pilot study for validating and comparing the diagnostic accuracy of different questionnaires is required.

Strengths of the current study include the use of post-bronchodilator spirometry airflow limitation as the gold standard, multiple recruitment areas containing urban and rural areas, and questionnaire administration by trained coordinators. This study also has several limitations. First, this study included the general population of Beijing, and does not contain subjects from other areas with different cultures and ethnicities. Second, we used FEV₁/FVC < 0.70 as the diagnostic criterion rather than the lower limit of normal, which may overestimate COPD in elderly subjects. Third, our study included slightly more women (58.9%) than men, and the proportion of non-smokers (64.3%) was higher than ever smokers, which may amplify the effects of non-smoking risk factors on the presence of COPD.

Conclusions

In conclusion, the COPD-PS and COPD-SQ had comparable discriminatory power for identifying spirometry-defined COPD and symptomatic COPD. They are both useful for identifying subjects at a high risk of COPD, while the COPD-SQ is more suitable for screening for COPD in rural areas. When using questionnaires to screen for COPD in certain populations, a pilot study for validating and comparing the diagnostic accuracy of different questionnaires is required.