Accuracy of Six Chronic Obstructive Pulmonary Disease Screening Questionnaires in the Chinese Population

Introduction

Chronic obstructive pulmonary disease (COPD) is characterized by persistent airflow limitation.¹ The global burden of disease study showed that COPD affected 174 million patients globally in 2015.^1,2 However, if cases diagnosed by spirometry are added to these estimates, the number of COPD patients is likely to reach 384 million worldwide, creating a high socioeconomic burden.³ A previous study⁴ has shown that early diagnosis and treatment may delay the progression of lung function decline, reduce the number of acute exacerbations, and improve the quality of life in patients with early-stage COPD. However, the onset of COPD is insidious. Early disease symptoms are generic, including cough and sputum, increasing the risk of diagnosis and treatment delay.

The Global Initiative for Chronic Obstructive Lung Disease (GOLD) guidelines state that spirometry is the gold standard for detecting COPD.⁵ As a developing country with nearly 100 million COPD patients, it is difficult to perform pulmonary function tests for such a large number of patients, from a technological equipment perspective and economical.⁶ A large-scale epidemiological survey in China has shown that only 12.0% and 55.8% of patients diagnosed with COPD and self-reporting history of COPD had undergone spirometry assessments, respectively.⁶ The standard diagnosis and treatment of COPD in China continue to evolve. The low spirometry utilization rate at medical institutions leads to misdiagnosis and under diagnosis of COPD. The United States Preventive Services Task Force (USPSTF) recommended that the disease risk be assessed in primary care using a screening questionnaire; “positive” patients should then be transferred to higher-level medical institutions for spirometry assessment, followed by bronchodilator tests in those with abnormal pulmonary function.⁷ Simple self-administered questionnaires may help identify people who need spirometry testing.^8,9 Population screening with questionnaires may help increase patient awareness of symptoms, prompting them to seek medical help, as required. These questionnaires can also help primary care physicians efficiently estimate the patient risk of the disease, which is paramount in settings that involve large many patients, such as primary care.

The commonly used screening questionnaires for COPD include the COPD Diagnostic Questionnaire (CDQ), Revised COPD Diagnostic Questionnaire (Revised-CDQ), COPD Population Screener (COPD-PS), COPD Screening Questionnaire (COPD-SQ), COPD Assessment in Primary Care to Identify Undiagnosed Respiratory Disease and Exacerbation Risk (CAPTURE), and Lung Function Questionnaire (LFQ). CDQ was recommended by USPSTF as a screening tool and was applied in the whole world.¹⁰ COPD-PS was popularly used, too; a meta-analysis showed that it is a useful tool to improve the early diagnosis of COPD.¹¹ CAPTURE is a new questionnaire developed in recent years,¹² and LFQ has demonstrated its strength in inter-comparison with other questionnaires.¹³ Except for the COPD-SQ and Revised-CDQ, the other four questionnaires were developed based on epidemiological data acquired in Western countries. Nevertheless, there is limited evidence on the validity of screening in the Chinese population. Zhou¹⁴ developed the COPD-SQ based on epidemiological data acquired from 19,800 subjects aged ≥40 years in a 2002 multi-stage cluster sampling survey covering seven Chinese provinces and cities. The associated area under the curve (AUC) was 0.829. This questionnaire was subsequently validated in a dataset of 3231 subjects, yielding an AUC of 0.812, indicating high accuracy. When the cut-off value was 16 points, the sensitivity and specificity values of diagnosing COPD were 60.6% and 85.2%, respectively. The Revised CDQ¹⁵ is a modified version of the CDQ that accounts for the Chinese population characteristics. The development population included 3969 subjects aged ≥30 years and recruited from the Division of Respiratory Medicine or the Health Management Center. The corresponding AUC, sensitivity, and specificity values were 0.85, 82.45%, and 72.87%, respectively, at the cut-off score of 16 points. There may be differences in epidemiological characteristics of COPD between the Western and Chinese populations. No previous study has examined the effectiveness of these six questionnaires in screening for COPD in the Chinese population; the questionnaire most suitable for use with this population remains unclear. This study aimed to evaluate the screening accuracy of the six questionnaires in the Chinese population and to provide optimal cut-off values for diagnosis, ensuring high sensitivity. The present findings may help reduce the number of spirometry referrals, and provide a reference for the selection of screening tools for Chinese primary care settings.

Materials and Methods

Participants

It is a case-control study based on a hospital. Patients aged ≥35 years, presenting at the Department of Respiratory and Critical Care Medicine at the First Affiliated Hospital of China Medical University in 2021 were included in this study. The exclusion criteria were: (1) a previous formal diagnosis of COPD; (2) inability or unwillingness to complete spirometry evaluation; (3) inhalation of bronchodilators or corticosteroids on the day before the test; (4) pre-bronchodilator (Pred-BD) forced expiratory volume in 1 second/ forced expiratory capacity (FEV₁/FVC) values of <0.7 without a bronchodilator test. The questionnaires were completed by a combination of face-to-face interviews, self-reporting, and telephone interviews. This study was approved by the Institutional Review Board of China Medical University, and all subjects gave informed consent to this study. This study was conducted in accordance with the Declaration of Helsinki.

Questionnaires

CDQ

CDQ, also known as the International Primary Care Airway Group (IPAG) questionnaire, was proposed by Price¹⁶ in 2005. The questionnaire is comprised eight items: age, body mass index (BMI), smoking intensity, weather-affect coughing, phlegm without a cold, phlegm in the morning, wheeze frequency and allergic history, with a total score of 38 points. The IPAG guidelines recommend a cut-off value of 17 points for screening.¹⁷

Revised-CDQ

The Revised-CDQ¹⁵ was modified by Chinese scholars based on the differences in BMI distribution between Western countries and China. This questionnaire accounts for the frequency of exposure to second-hand smoking, cough without a cold, shortness of breath relative to that of peers, long-term exposure to dust or particles, and chronic respiratory disease history in childhood; phlegm without a cold and phlegm in the morning are not included due to their low discriminatory ability. The final questionnaire consists of 11 items; 17 points are recommended as the optimal cut-off value.

COPD-PS

The COPD-PS questionnaire was developed by Martinez¹⁸ in 2008. It consists of five items, including age, smoking history, shortness of breath frequency, cough and changes in exercise capacity, with a total score of 10 points. Among the original development population, the sensitivity and specificity of the questionnaire reached a good balance at 5 and 6 points, which correspond to the highest classification accuracy. The 2018 guidelines for primary care of COPD¹⁹ recommend 5 points as the best cut-off score.

COPD-SQ

The COPD-SQ¹⁴ contains six questions on age, smoking amount, BMI, cough, degree of dyspnea, family history of respiratory disease, and exposure to biomass smoke from cooking, with a total score of 38 points. In the development population, the AUC reached the maximum value of 0.753 when the cut-off value of the questionnaire was 16 points.

CAPTURE

The CAPTURE questionnaire was developed by Martinez¹² in 2016. It comprises five items, including exposure to polluted air, environment-affect breathing change, exercise-related breathing restriction, fatiguing easily, and acute respiratory illnesses, with a total score of 6 points. The questionnaire set a score of 0–1 points as representative of a low risk, indicating no need for further evaluation. Participants scoring in the range of 2–4 points should undergo peak expiratory flow measurement using an inexpensive mechanical device; those scoring in the range of 5–6 points may require further evaluation, including spirometry.

LFQ

Yawn²⁰ proposed the LFQ in 2010. The questionnaire consists of five variables, including age, cough, wheeze, dyspnea, and smoking. In 2010, Hanania²¹ revised this questionnaire and changed the scoring method from a binary variable to an ordinal categorical variable, with a total score of 25 points, and recommended a score of 18 points as the best cut-off value. For the LFQ, the lower the score, the higher the risk of airway obstruction.

Sample Size

The sample size was calculated based on the following parameters. The response distribution was defined as the lowest sensitivity (72.6%) and specificity (47.8%) of the six screening questionnaires at the recommended value in their development populations, the margin of error was 5%, and the confidence level was 95%. Therefore, according to the formula, we calculated that 302 people were needed for the case group and 377 people for the control group using Raosoft Sample Size Calculator version 2004 (http://www.raosoft.com/samplesize.html).

Spirometry

After completing the questionnaires, all participants underwent pre-bronchodilator (pre-BD) spirometry evaluation (Master Screen Pneumo, Jaeger, Germany). Participants were in the seated position and completed 3–8 forced expiratory capacity (FVC) tests to obtain at least three repeatable and acceptable trials. The highest FVC and forced expiratory volume in 1 second (FEV₁) values were used for analysis. The subject with pre-BD FEV₁/FVC value of <0.70 received 400 ug albuterol via a metered-dose inhaler with a spacer, followed by an extra post-bronchodilator spirometry evaluation 15 minutes later. All measurements were performed by professional technicians in accordance with the standards of the American Thoracic Society and European Respiratory Society task force.²² The double-blind principle was strictly implemented throughout the trial. Subjects with post-bronchodilator FEV₁/FVC values of <0.70 were defined as having COPD, according to the GOLD guidelines.⁵ The final diagnosis of COPD was confirmed by an experienced pulmonologist based on symptoms, risk factor exposure, spirometry and clinical examination findings, and exclusion of other diseases.⁵

Statistical Analysis

Descriptive analysis was used to evaluate the demographic characteristics, spirometry parameters, and questionnaire scores. Continuous parametric variables were reported as mean ± standard deviation. Differences between the COPD and non-COPD groups were assessed using the independent Student’s t-test for quantitative variables and the χ² test for qualitative variables. The optimal cut-off value of the screening questionnaire was defined as the one with the highest Youden index. Receiver operating characteristic curves (ROC) were drawn for each questionnaire. The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and the area under the curve (AUC) were calculated, respectively, when took the optimal cut-off value and previously recommended one. Screening accuracy of each questionnaire was evaluated by the AUC, with an AUC of 0.7–0.9, 0.9–1, and 1 representing moderate, high, and perfect accuracy, respectively.²³ The Delong method was used for comparing screening accuracy between questionnaires; curves with different directions were compared with the bootstrap method. The Bonferroni correction was applied for multiple comparisons. The previously recommended cut-off scores of the Revised CDQ, COPD-SQ, CAPTURE, and LFQ were defined as recommended by the development team. The previously recommended cut-off score of the CDQ was proposed by the IPAG guidelines.¹⁷ The previously recommended cut-off score of the COPD-PS was suggested by the 2018 guidelines for primary care of COPD.¹⁹ The Z-test was used to compare AUCs of questionnaire with different cut-off values. All statistical analyses were performed using SPSS 24.0 (Version 24.0, SPSS Inc., Chicago, IL, USA) and MedCalc (Version 14.8.1, MedCalc, Ostend, Belgium). The results were considered statistically significant at P-values of <0.05.

Results

A total of 1494 patients participated in this study. Among them, 1015 subjects completed both questionnaires and spirometry testing, and 909 patients (416 men) were included in the analysis. The reasons for exclusion are presented in Figure 1. A total of 330 (191 men) subjects were diagnosed with COPD. Demographic characteristics, and spirometry and screening questionnaire findings of the COPD and non-COPD groups are presented in Table 1.

Table 1 Comparison of Demographic Characteristics, Spirometric and Screening Questionnaire Scores Between Subjects with and without COPD

Figure 1 Flow chart of included data. Abbreviations: COPD, chronic obstructive pulmonary disease; FEV1, forced expiratory volume in one second; FVC, forced vital capacity.

Compared with the non-COPD group, the COPD group was older (60.71±9.26 vs 55.19±10.50 years, P<0.001) and taller (166.07±7.44 vs 163.99±7.87 cm, P<0.001), smoked more (49.87±29.84 vs 40.65±28.64 pack-year, P<0.003), and included more men (57.88% vs 38.86%, P<0.001) and active smokers (55.45% vs 29.36%, P<0.001). Spirometry parameter values were lower in the COPD group than in the non-COPD group. The LFQ scores were lower in the COPD group than in the non-COPD group. However, the remaining questionnaire scores were higher in the COPD group than the non-COPD group. The differences in questionnaire scores between the two groups were statistically significant, suggesting that all six screening questionnaires may have good diagnostic discrimination ability for COPD in this population (all P < 0.05).

Screening Accuracy of Six Questionnaires

The AUC values the LFQ, Revised CDQ, COPD-PS, CDQ, COPD-SQ, and CAPTURE were 0.785 (95% confidence interval: 0.752–0.812), 0.762 (0.727–0.795), 0.745 (0.707–0.778), 0.731 (0.695–0.762), 0.703 (0.671–0.734), and 0.667 (0.635–0.702), respectively. These findings suggest that all questionnaires could be used for COPD screening (P all <0.05). Except for CAPTURE, the other five screening questionnaires demonstrated moderate diagnostic accuracy (Figure 2).

Figure 2 ROC curves for six questionnaires analyzing the whole study sample. Abbreviations: CDQ, COPD Diagnostic Questionnaire; Revised-CDQ, the Revised COPD Diagnostic Questionnaire; COPD-PS, COPD Population Screener; COPD-SQ, COPD Screening Questionnaire; CAPTURE, COPD Assessment in Primary Care to Identify Undiagnosed Respiratory Disease and Exacerbation Risk; LFQ, the Lung Function Questionnaire.

Optimal Cut-off Values

In this study, the optimal cut-off value was defined as the one with the highest Youden index.

All six questionnaires had PPVs (31.8–38.1%), and NPVs (86.4–91.6%) at the optimal cut-off value. The optimal cut-off values for the CDQ questionnaire, Revised CDQ, COPD-PS, COPD-SQ, CAPTURE, and LFQ were: 19, 19, 5, 17, 3, and 16 points, respectively. The cut-off values of LFQ and CAPTURE had better screening accuracy than those previously recommended (Table 2).

Table 2 Characteristics of the Six COPD Screening Questionnaires

Screening Accuracy Between Six Questionnaires

Screening accuracy of the Revised CDQ was better than those of the CAPTURE or COPD-SQ. Screening accuracy of the LFQ was better than those of CDQ, COPD-SQ, and CAPTURE. Screening accuracy of the COPD-PS was better than CAPTURE (Table 3).

Table 3 Multiple Comparisons of the Discriminate Ability of Six Questionnaires

Discussion

In 2015, there were approximately 100 million COPD patients in China.⁶ A case-finding strategy involving the use of suitable screening tools may increase disease awareness among patients, promote the reduction of risk-related behaviors (eg, smoking), and help diagnose and treat COPD at an early stage. Spirometry remains the gold standard for COPD diagnosis. However, restricted access to specialist spirometry equipment and personnel increase the rates of COPD under diagnosis. The 2021 GOLD guidelines⁵ propose that screening questionnaires may help identify patients at risk in primary care settings. Sichletidis²⁴ investigated the validity of the IPAG questionnaire and Piko-6 for screening COPD in 1078 subjects aged ≥40 years and recruited from 25 general practice offices, showing that the combined application of the IPAG and PiKo-6 may improve PPV from 15% and 64% associated with using these tools separately, respectively, to 71% when using them in tandem. The CDQ, Revised CDQ, COPD-PS, COPD-SQ, CAPTURE, and LFQ were developed in different populations. However, the diagnostic accuracy of existing screening questionnaires differs significantly in the practical clinical appliance. Although some reports demonstrate the screening value of single questionnaires, only a few articles compare two to five kinds of questionnaires in the same population. To the best of our knowledge, there are no articles on comparative studies of these six questionnaires in the same population, especially Chinese. In this report, we aim to compare the screening accuracy of these six questionnaires in the same population and find a screening questionnaire with acceptable accuracy to improve early diagnosis and treatment of COPD for Chinese. This study has shown that all six questionnaires could be used for first-level COPD screening. In addition, the CAPTURE and LFQ scores of 3 and 16 points, respectively, have greater screening accuracy than do the previously reported scores. Finally, the LFQ showed greater screening accuracy than did the other questionnaires.

The USPSTF recommends the CDQ due to its high diagnostic accuracy.¹⁰ In the present study, the CDQ cut-off score for screening in symptomatic people was 19 points; the corresponding sensitivity, specificity, and Youden index values were 67.5%, 68.4%, and 0.359, respectively. In a study by Tao,²⁵ which involved participants aged ≥40 years and presenting with symptoms or a history of exposure to risk factors, sensitivity, specificity, and Youden index values were 98.5%, 22.7%, and 0.212, respectively, for a cut-off score of 17 points, and 78.5%, 67.0% and 0.455, respectively, for a cut-off score of 21 points. The Youden index was higher for the score of 21 points. In addition, Wang²⁶ recommended 19 points as a more appropriate cut-off score for smokers in China, while Kawayama²⁷ suggested 19.5 points as an alternative for screening smokers; both scores were higher than the 17 points proposed by the IPAG. However, a comparative analysis of screening accuracy between the optimal cut-off score and recommended one has shown that the cut-off score of 19 points was not better for screening than that of 17 points. Screening aims to identify as many potential patients as possible, so it is more beneficial for us to choose screening indicators with higher sensitivity under this circumstance.

The Revised CDQ is a modified version of the CDQ, adapted for the Chinese population. The AUC increased from 0.80 to 0.85 after this modification; the corresponding sensitivity, specificity, and Youden index values were 82.45%, 72.78%, and 0.552, respectively.¹⁴ In this study, the AUC of the Revised CDQ increased from 0.731 to 0.762. However, there was no significant difference in the screening accuracy before and after this modification. In addition, the AUC of the Revised CDQ was 0.762 in this study, which was lower than that in the development population. The optimal cut-off value was 19 points, which was higher than the recommended 17 points; the higher score has not shown better diagnostic accuracy than the recommended score. The Revised CDQ consists of 11 items; thus, its use is more time-consuming than that of the other five questionnaires. Screening accuracy was similar among the examined questionnaires, suggesting that the use of the COPD-PS or LFQ as the more convenient option.

The COPD-PS may be used as an auxiliary screening tool in primary care settings that lack spirometers in China.¹⁹ Patients with the scores of ≥ 5 points should be referred for further evaluation. The present cut-off values were comparable to those previously reported point. Kobayashi studied patients aged ≥40 years treated across 16 primary care settings and 4 hospitals, revealing the optimal cut-off value of 5 points, with the corresponding AUC, sensitivity, and specificity values of 0.71 (0.59–0.83), 63.0%, and 67.9%, respectively.²⁸ Gu conducted a meta-analysis to summarize the screening effect of COPD-PS, showing that the AUC was 0.79, and sensitivity of 74.52%, specificity of 70.24% at the cut-off valve of 4.¹¹

The optimal cut-off score of the COPD-SQ based on the Youden index was 16 points, and the corresponding AUC was 0.753.¹⁴ In this study, the optimal cut-off and AUC values were 17 points and 0.653, respectively. In a study of 3811 community subjects aged ≥40 years, Yao²⁹ reported sensitivity, specificity, PPV, NPV, and AUC values of 76.8%, 75.7%, 44.9%, and 44.9%, 90.1%, and 0.76, respectively, for a score of 16 points. A study conducted in primary care settings showed that the PPV was 34.7%; the corresponding value in the present study was 34.3%.³⁰ The present findings suggest that the cut-off values of 16 and 17 points have comparable screening accuracy.

The CAPTURE questionnaire had the lowest screening accuracy in the present study with an AUC of 0.667; the score of 3 points was the optimal cut-off value in this study, with the corresponding sensitivity and specificity values of 60.8% and 65.7%, respectively, compared with 95.7% and 67.8%, respectively, in the development population. The present AUC findings suggest that the score of 3 points is superior to the score of 2 points. We found that the development team classified the case group as COPD and ≥ 1 exacerbation past year or FEV₁ <60% predicted without exacerbation past year, which means their control group contains mild-to-moderate COPD, which leads to the recommended cut-off lower than ours. The CAPTURE contains five items; however, it requires medical staff input to complete; it also considers the impact of COPD on the quality of life, making it suitable for follow-up monitoring.

The LFQ had the highest screening accuracy in this study. Using the optimal cut-off score of 16 points, the sensitivity, specificity, and AUC values were 76.6%, 67.3%, and 0.719, respectively; the screening accuracy was greater than the one previously reported by Hanania.²¹ Although the study had the highest Youden index at 16 points, to identify potential COPD patients as much as possible, the cut-off value of 18 point was recommended by the author. The corresponding AUC, sensitivity, specificity values, and PPV and NPV were 82.6%, 47.8%, 26.5%, and 92.3%, respectively. Jarhyan applied it in community settings and showed 75.0% sensitivity and 55.3% specificity at a cut-off score of 18 points, which were not as good as the present study.³¹ A survey enrolled smokers ≧40 years old and found that the optimal cut-off value of LFQ based on the Youden method was 18 points, which is higher than ours. It may be that it was conducted in the smoking population, and the year of smoking is one of the indicators of LFQ.

Spyratios¹³ compared the CDQ, COPD-PS, and LFQ revealing NPV in the range of 94–96% and AUC values in the range of 0.794–0.809, with the corresponding cut-off score of 19 points for CDQ in COPD screening. Tsukuya³² has shown that the COPD-PS and CDQ were comparable in their ability to distinguish patients with or without airflow obstruction. The LFQ has the highest accuracy, followed by Revised CDQ, and COPD-PS in all six questionnaires, and there is no significance among them. However, considering the convenience, the Revised CDQ contains 11 items, and the LFQ and COPD-PS both have 5 items, which means the administration of the LFQ or COPD-PS tends not to be time-consuming, making it useful in clinical practice. Therefore, the present findings recommend the use of the LFQ and COPD-PS with cut-off scores of 16 and 5 points, respectively, in the Chinese population. Both LFQ and COPD-PS questionnaires account for similar factors, such as age, respiratory symptoms and smoking history. The other four questionnaires also care about risk factors of COPD such as family history, and risk factor exposure. Yet, the effect is not significantly superior to LFQ and COPD-PS. It may be related to the intensity and duration of the risk factors and the variability of individual susceptibility. These findings suggest that COPD screening should focus on respiratory symptoms, age, and smoking history.

Although we compared six questionnaires in the same population abiding by double-blind, there are still some limitations. Firstly, we use a fixed post-BD FEV₁/FVC<0.70 as diagnostic criteria, which may result in the overestimation of COPD in older patients.³³ In addition, every participant accomplished six questionnaires before the pulmonary function examination. Too many items of questionnaire and a tight schedule may lead to questionnaire fatigue or inattentive answers.

Conclusion

The present findings suggest that questionnaire-based screening for COPD may help in the early diagnosis of this disease, especially LFQ. The LFQ and COPD-PS are convenient to use and have high diagnostic accuracy. The optimum cut-off values in the Chinese population of the CAPTURE, and LFQ are 3 and 16 points, respectively; these values have greater diagnostic accuracy than those previously reported.