Comparative Study on Chronic Obstructive Pulmonary Disease Screening Tools in Primary Healthcare Institutions in Beijing, China

Introduction

Chronic Obstructive Pulmonary Disease (COPD), is a heterogeneous lung condition characterized by persistent, often progressive airflow obstruction resulting from abnormalities in the airways (bronchitis, bronchiolitis) and/or alveoli (emphysema).¹ In 2018, a study showed that the prevalence of COPD among people aged 40 and above in China was as high as 13.7%, with nearly 100 million patients.² It manifests through chronic respiratory symptoms such as dyspnea, cough, sputum production, and/or exacerbations.¹ But more than half were GOLD 1 patients with mild airflow obstruction but no obvious symptoms,³ of which had minor symptoms. Although mild COPD patients have minor symptoms, they account for a high proportion, and their lung function declines rapidly, which should be given high attention.⁴ Studies have shown that pathological changes have occurred in mild COPD patients, and the number of small airways has significantly decreased and the structure has changed.^5,6 Early intervention of COPD depends on early detection, but epidemiological surveys show that even patients with moderate or severe COPD have a knowledge rate of less than 1%, and the best treatment period has been missed by the time of diagnosis.^3,7 Therefore, early detection and treatment of COPD patients are crucial for disease prevention and control. Community health service centers are responsible for residents’ health management and basic medical services, and play an important role in COPD screening and prevention. Although there are various COPD screening tools, such as COPD screening questionnaires, peak flow meters, and pulmonary function tests, there is still a lack of screening tools suitable for use in primary healthcare institutions in China. This study aims to explore a simple, feasible, and highly accurate early COPD community screening tool.

Participants and Methods

Participants

Residents from Donghuashi Community in Dongcheng District, Beijing, China were recruited as study participants from September 1st, 2022 to March 1st, 2023. Inclusion criteria were age ≥ 35 years and voluntary participation in the study, either through referral by physicians or self-enrollment via posters/Wechat promotions. Exclusion criteria were inability to cooperate or having contraindications to pulmonary function tests (such as dementia, uncontrolled hypertension, pneumothorax, etc.), pregnant or breastfeeding women, previous adverse reactions to salbutamol, and deemed inappropriate for participation by the research personnel. The study aimed to recruit 1000 individuals, and 986 were ultimately enrolled and completed the study, including 166 who underwent further pulmonary function tests.

This study was conducted in accordance with the ethical principles outlined in the Declaration of Helsinki. This study was approved by the Ethics Committee of Beijing Puren Hospital (Ethical Approval No. prll-2022-2), and all study participants provided informed consent. Given that the Beijing Donghuashi Community Health Service Center does not have its own dedicated ethics committee, we sought ethical approvals for our study through the ethics committee of the Puren Hospital, which possesses the necessary qualifications for ethical review. To substantiate this, we have attached a sealed letter of authorization from the Beijing Donghuashi Community Health Service Center, officially granting the Beijing Puren Hospital the responsibility for conducting the ethical review.

Methods

This study adopted a cross-sectional design. The research subjects completed the CAPTURE questionnaire, COPD-SQ questionnaire, and PEF measurement.

The CAPTURE questionnaire was developed by American scholars specifically for primary healthcare institutions. It evaluates patients through a total of 5 questions that cover their environment, physical symptoms, and other factors. The responses to these questions are combined to calculate a score ranging from 0 (indicating a negative response to all 5 questions) to 6 (indicating a positive response to all questions and experiencing at least 2 respiratory events in the past year). A score of 2 or higher suggests the need for further diagnostic assessment for COPD. During its initial testing, a score of 2 or higher demonstrated a sensitivity (SN) of 95.7% and a specificity (SP) of 44.4% in distinguishing COPD cases (with FEV1% predicted < 60% or exacerbation risk) from controls (with no or mild/moderate risk of COPD) and a sensitivity of 95.7% and a specificity of 67.8% in distinguishing cases from individuals without COPD.⁸

The COPD-SQ questionnaire was developed by Chinese scholars using domestic population data. It consists of 7 questions that assess various factors such as age, physical symptoms, smoking status, and family medical history. The questionnaire includes items related to COPD symptoms (such as cough and dyspnea), demographic data (age and BMI), exposure to nicotine or biomass smoke, and family history of respiratory disease.⁹ The version of COPD-SQ used in the Chinese guidelines for the diagnosis and management of COPD (revised version 2021).¹⁰

PEF Measurement: PEF stands for Peak Expiratory Flow, which measures the speed at which a person forcefully exhales. In this study, PEF measurement was used as a screening tool to identify airflow limitation. Participants underwent PEF measurement using a portable peak flow meter. Each subject performed three maneuvers, and the highest recorded value (L/min) was used for analysis. Sex-specific thresholds were established to facilitate quick interpretation in clinical practice before referral for diagnostic spirometry: males <350 L/min and females <250 L/min.¹¹

A positive result was defined as a CAPTURE score of ≥2 points, a COPD-SQ score of ≥16 points, and a male PEF of <350 L/min or a female PEF of <250 L/min. If any of these criteria were positive, further diagnostic lung function tests would be conducted.

Lung function tests: According to the GOLD diagnostic criteria for COPD, the diagnostic criterion was an FEV1/FVC of less than 0.7 after inhaling a bronchodilator.¹² The bronchodilator challenge test was used to detect airway reversibility by observing the relaxation response of obstructed airways. It is used to show obstructive ventilation dysfunction or small airway dysfunction.¹³ The specific operational steps were as follows: 1. Preparation for the test: The operator evaluated the contraindications, understood the subjects’ underlying diseases and medication-related medical history. 2. Baseline pulmonary ventilation function testing: Complete the forced vital capacity curve. 3. Repeat the test after inhaling a bronchodilator: The bronchodilator selected for this test was inhaled salbutamol 400 μg, and the method of administration was to have the subject exhale the gas in their lungs to the residual volume, then inhale slowly and deeply, press the salbutamol sulfate aerosol 4 times (100 μg per press), and have the subject inhale the mist until the end of the deep inhalation, holding their breath for 5–10 seconds, then returning to normal breathing. After inhaling the medication for 15–30 minutes, repeat the pulmonary function test. The result indicator selected for the bronchodilator challenge test was the FEV1/FVC. To ensure the reliability and accuracy of FVC measurements, FVC should be performed at least three times. Acceptable repeatability is achieved when the difference between the largest and second-largest FVC values is less than 0.150 L and the difference between the largest and second-largest FEV1 values is less than 0.150 L.¹⁴ If these criteria are not met, additional measurements should be attempted, and volume-time or flow-volume curves from at least the best three FVC maneuvers should be retained. The study was conducted by researchers who trained healthcare providers and volunteers at primary healthcare facilities in the use of relevant questionnaires and peak flow meters. The researchers ensured that the CAPTURE questionnaire, COPD-SQ questionnaire, and PEF measurements were completed face-to-face and provided quality control for the questionnaires to ensure that all questions were answered completely, with no omissions, and logically, for them to be considered valid.

Statistical Methods

EpiData 3.1 software for data entry and SPSS 24.0 software for data analysis. Count data were expressed as frequency (percentage) and continuous data were presented as mean ± standard deviation. The CAPTURE questionnaire, COPD-SQ questionnaire, and PEF screening tools were compared individually and in combination with diagnostic pulmonary function testing, and sensitivity (Sensitivity refers to the ability of a test to correctly identify positive cases among all true positive cases. A higher sensitivity indicates that the test is effective in detecting positive samples and minimizing false negatives.), specificity (Specificity refers to the ability of a test to correctly exclude negative cases among all true negative cases. A higher specificity indicates that the test is effective in excluding negative samples and minimizing false positives.), positive predictive value (Positive predictive value is the probability that a positive test result truly indicates a positive case. It reflects the accuracy of the test in identifying true positive samples among all positive results.), negative predictive value (Negative predictive value is the probability that a negative test result truly indicates a negative case. It reflects the accuracy of the test in excluding true negative samples among all negative results.), and ROC curve analysis (ROC curve analysis is a method to evaluate the diagnostic accuracy of a test. The ROC curve is a graphical representation of sensitivity against 1 minus specificity. The area under the ROC curve (AUC) is a measure of the overall performance of the test, with a higher AUC indicating higher accuracy). Were calculated. McNemar’s test was used to compare the seven screening strategies using paired data. A p-value less than 0.05 was considered statistically significant.

Results

General Information of Participants

A total of 986 individuals completed the screening tests using CAPTURE questionnaire, COPD-SQ questionnaire, and PEF measurement. The mean age was 59.09±13.08 years, with 396 (40.16%) males and 590 (59.84%) females, and 254 (25.76%) reported a history of smoking. The screening results showed that 412 (41.78%) individuals had positive results on CAPTURE questionnaire, 287 (29.11%) on COPD-SQ questionnaire, and 513 (52.03%) on PEF measurement, with 203 (51.26%) males and 310 (52.54%) females. A total of 689 (69.88%) individuals had positive results on at least one of the three screening tools. Based on the willingness of the study participants and the evaluation of lung function screening by the researchers, 166 (24.09%) individuals underwent lung function diagnosis, with 64 (38.55%) males and 102 (61.45%) females, and 17 (10.24%) individuals had a prior diagnosis of COPD. Additionally, 37 (22.29%) individuals had FEV1/FVC < 0.7, of which 8 (21.62%) were previously diagnosed with COPD. Table 1 for details.

Table 1 Chronic Obstructive Pulmonary Disease Screening Tools and Gender Composition Ratio of Positive Lung Function Tests

Screening Results of Three Tools for COPD

The peak expiratory flow meter had the highest positive rate at 52.03%, followed by the CAPTURE questionnaire at 41.78%, and the COPD-SQ questionnaire had the lowest positive rate at 29.11%. McNemar’s test was used to compare the positive rates of any two screening tools, and the results showed that there were significant differences between the positive rates of any two tools (P < 0.001). Please refer to Table 2 for details.

Table 2 Comparison of Positive Rates of Three Screening Tools Used Alone

Comparison of the Accuracy of Three Screening Tools

When each of the three screening methods was used alone, the peak flow meter had the highest sensitivity (83.78%) and the lowest specificity (23.26%), while the COPD-SQ questionnaire had the highest specificity (59.69%) and the lowest sensitivity (64.86%). The COPD-SQ questionnaire had the highest positive predictive value (31.58%) and negative predictive value (58.56%). There were statistically significant differences (P<0.001) in the results of the three screening tools compared to lung function tests. Based on the correct index (Youden’s index), the accuracy of the COPD-SQ questionnaire (Youden’s index 0.246) was higher than that of the peak flow meter and the CAPTURE questionnaire when used alone. See Table 3 for details. ROC analysis results of the three screening tools compared to lung function tests as the gold standard: The results showed that the COPD-SQ questionnaire had the highest accuracy, with an area under the curve of 0.744 (95% CI 0.650, 0.837), followed by the CAPTURE questionnaire with an area under the curve of 0.589 (95% CI 0.492, 0.685). See Figure 1 for details.

Table 3 Accuracy of Screening Methods

Figure 1 Analysis of ROC curves for CAPTURE scale and COPD-SQ scale and PEF test screening.

Comparison of Accuracy of Different Screening Tool Combinations

Three screening methods were combined in pairs (COPD-SQ questionnaire + CAPTURE scale, PEF + CAPTURE scale, PEF + COPD-SQ questionnaire) and in a triple combination (PEF + COPD-SQ questionnaire + CAPTURE scale). Among the four screening combinations, the PEF + CAPTURE scale combination demonstrated the highest sensitivity (62.16%), followed by the PEF + COPD-SQ questionnaire combination (59.46%) and the COPD-SQ questionnaire + CAPTURE scale combination (51.35%). The lowest sensitivity was observed in the PEF + COPD-SQ questionnaire + CAPTURE scale combination. In terms of specificity, the triple combination of all three tools (79.07%) outperformed the double combinations, with the COPD-SQ questionnaire + CAPTURE scale combination exhibiting the highest specificity (74.42%) among the double combinations. The combination of PEF + COPD-SQ questionnaire + CAPTURE scale showed no statistically significant difference in detection results compared to lung function tests (P=0.382). Please refer to Table 3 for detailed information.

Based on the calculation of the Youden index and considering all the above data, the combination of peak expiratory flow meter screening and COPD-SQ questionnaire displayed the highest accuracy (Youden index of 0.277) among the different screening tool combinations. For further details, please refer to Figure 1. The results of the ROC analysis, using lung function tests as the gold standard, demonstrated that the combination of PEF + COPD-SQ questionnaire had the highest accuracy, with a Youden index of 0.277 and an area under the curve of 0.639 (95% CI 0.535, 0.742). The second-highest accuracy was observed in the combination of CAPTURE questionnaire + COPD-SQ questionnaire, with an area under the curve of 0.629 (95% CI 0.523, 0.735). Please refer to Figure 1 for detailed information.

Discussion

The prevalence of COPD in the population aged 40 years and above in China is as high as 13.7%.² In this study, only 21.62% of patients with FEV1/FVC<0.7, which is the criterion for diagnosing COPD by pulmonary function test, were previously diagnosed with COPD. Among 986 individuals screened, the positive rate of any of the three screening tools was as high as 69.88%, indicating that the diagnosis rate of COPD is still low. The primary care guidelines in China recommend screening for COPD in high-risk populations to detect patients early, intervene, slow down the disease progression, and improve their quality of life.^10,15 Although pulmonary function test is the standard for COPD diagnosis,^12,16 in primary care settings, difficulties exist in performing pulmonary function tests due to insufficient technical skills, limited resources, and lack of equipment.¹⁷ Therefore, it is necessary to choose screening tools that are simple to operate and have high accuracy in primary care settings.

Apart from pulmonary function tests, various screening tools for COPD have been developed domestically and internationally, such as COPD screening questionnaires COPD-6 instrument and peak expiratory flow meters. However, due to differences in the development population, cultural adaptation, and other factors, the effectiveness of these tools may vary in different countries and regions, and there may be limitations in using different screening tools. This study also found differences in the positivity rates when comparing the COPD-SQ questionnaire, CAPTURE questionnaire, and PEF screening methods in pairs. The CAPTURE questionnaire, developed in 2016 by American scholars based on the population of primary healthcare institutions, mainly asks about symptoms in five questions. Its sensitivity is 95.7%, and specificity is 67.8%.¹⁸ In this study population, compared with pulmonary function tests, the sensitivity of the CAPTURE questionnaire was 75.68%, and the specificity was only 36.43%. This result differs greatly from the development result and may be related to the development population, translation of the questionnaire, and cultural adaptability, so the practicality of using the CAPTURE questionnaire alone for COPD screening in primary healthcare institutions in China needs further discussion.

The COPD-SQ questionnaire was developed by Chinese scholars based on data from the domestic population. It is more comprehensive than the CAPTURE questionnaire, including an evaluation of factors such as age, symptoms, body mass index, smoking status, and family history. Its sensitivity is 60.6%, specificity is 85.2%, and the area under the ROC curve (AUC) is 0.812.⁹ In this study, the COPD-SQ questionnaire was the most accurate of the three independent screening tools (Youden index 0.246), had the best specificity (59.69%), and a sensitivity of 64.86% with an ROC curve area (AUC) of 0.744. The development of the COPD-SQ questionnaire was based on data from the domestic population, and according to the results of this study, the questionnaire is considered to have a higher applicability value than the CAPTURE questionnaire in China, and can more fully demonstrate its screening effect on patients with COPD.

Peak expiratory flow (PEF) is one of the commonly used methods for evaluating lung function.¹² Compared with lung function testing, PEF is relatively simple to operate. A study in the UK showed that PEF screening for COPD had a sensitivity of 91% and specificity of 82%,¹⁹ while a study conducted in Guangdong, China in 2012 showed that the sensitivity of PEF screening for COPD was 76.8% and specificity was 81.4%.²⁰ In this study, PEF screening had the highest sensitivity of 83.78%, but its specificity was only 23.26%, and the results differed significantly. This is likely due to the influence of factors such as the cooperation between the operator and the patient, which affect the accuracy of the results.²¹ The operator’s guidance on the subject’s exhalation and the subject’s understanding and cooperation are factors that may contribute to the variability of the results. In this study, all the investigators involved in the screening were trained. Therefore, the large variability in the results may be related to the fact that some of the patients were in the recovery period of COVID-19 during the data collection period (September 2022 to January 2023), which may have increased discomfort such as shortness of breath and difficulty breathing, leading to a decrease in exhalation cooperation and a decrease in screening accuracy.²²

It can be seen that when conducting COPD screening, multiple factors need to be considered comprehensively, including the sensitivity, specificity, and accuracy of the screening tool. Several studies have also investigated the use of combined screening tools to improve screening accuracy. For example, a study by British scholars evaluated the combination of peak flow rate and questionnaire screening and found that its sensitivity was as high as 100%, but its specificity was only 30.7%. A 2021 study on COPD screening strategies based on the Chinese population showed that the combination of the CAPTURE questionnaire and PEF had a sensitivity of 37.4% and a specificity of 94.6%, while the combination of the COPD-SQ questionnaire and PEF had a sensitivity of 19.7% and a specificity of 90.1%.²³

Based on the findings of this study, it is recommended that primary healthcare institutions prioritize the combination of PEF and the CAPTURE questionnaire for COPD screening. This combination demonstrated the highest sensitivity (62.16%) and effectively identified individuals with COPD. Additionally, the combination of PEF screening and the COPD-SQ questionnaire showed the highest accuracy (Youden index of 0.277), providing improved diagnostic accuracy. Furthermore, the ROC analysis using lung function tests as the gold standard confirmed the superiority of the PEF + COPD-SQ questionnaire combination. It yielded an area under the curve of 0.639 (95% CI 0.535, 0.742), indicating its ability to distinguish between individuals with and without COPD. The combination of the CAPTURE questionnaire + COPD-SQ questionnaire also exhibited good accuracy, with an area under the curve of 0.629 (95% CI 0.523, 0.735). It should be noted that in practical applications in China, regional economic and environmental differences may limit the availability of peak flow measurement devices in certain areas, restricting the use of PEF. Therefore, when selecting screening tools, regional considerations should be taken into account. Based on the results of this study, the COPD-SQ questionnaire may be a more suitable choice for primary healthcare institutions as it offers higher accuracy and provides a comprehensive assessment of COPD risk in patients.

There are some limitations in this study. Firstly, we only selected residents in the jurisdiction of primary healthcare institutions in one city in China as study subjects, and the evaluation did not show regional and urban-rural differences, so the results are not comprehensive and objective enough. Secondly, the data collection of this study was conducted during the COVID-19 epidemic prevention and control period, and there may be selection bias and measurement bias in the study subjects. Regarding the choice of screening tools, we carefully considered various options, including the COPD-6 instrument. The COPD-6 instrument is a validated and widely used COPD screening tool in clinical practice. It offers a convenient and effective method for assessing lung function and airflow limitation, measuring parameters such as PEF, FEV1, and other indicators. Its accuracy, ease of use, and cost-effectiveness make it suitable for implementation in primary healthcare settings.²⁴ However, due to resource availability and feasibility constraints, we opted for the selected screening tools used in our study. In light of these limitations, future research should adopt a multi-center and cross-regional research design to obtain more comprehensive and objective data and explore the accuracy of screening tools for COPD. Additionally, with the development of the big data era, after large-scale information sharing, it is possible to evaluate the risk of COPD based on patients’ symptoms, medication history, etc., through the current outpatient electronic medical record system, to actively identify the high-risk population of COPD, in order to improve the screening effectiveness for COPD in China.

Conclusion

When the standard pulmonary function test is difficult to perform for the diagnosis of chronic obstructive pulmonary disease (COPD), the PEF screening combined with the COPD-SQ questionnaire is recommended to improve the accuracy of COPD screening in primary care institutions. In areas where peak flow screening equipment is limited, the COPD-SQ questionnaire can be used alone for screening.