Quantification of inaccurate diagnosis of COPD in primary care medicine: an analysis of the COACH clinical audit

Introduction

The adequacy of the diagnosis of COPD remains a challenge for clinicians and health-care managers. Recent epidemiological studies have reported a considerably high frequency of inadequate diagnosis,¹ despite the well-known practice guidelines with clear recommendations for the diagnosis of the disease,² resulting in two different clinical scenarios. On the one hand, under-diagnosis constitutes a problem already acknowledged by different studies in recent years.^3–5 Receiving inadequate heath care⁶ or suffering from an uncontrolled disease that may also impact other comorbidities⁷ are some of the consequences of this under-diagnosis. On the other hand, over-diagnosis is another problem that frequently occurs in COPD. This over-diagnosis can impact several key aspects of the disease,⁸ including increased exposure to – pharmacological treatment that would not otherwise be needed as well as an increase in the use of health services by the wrong patients, and the performance of a number of diagnostic tests. Additionally, people may be urged to adapt their lifestyle for a disease they do not have, with regular unneeded monitoring which would finally label them as sick individuals. Finally, it clearly impacts the health system leading to potential extra costs.

Beyond these two situations, a number of other clinical scenarios can be identified in which a diagnosis of COPD is determined without confirming the diagnostic criteria, termed as inaccurate diagnosis. A recent review highlighted the possible clinical situations in which an inaccurate diagnosis of COPD could occur.⁹ Notably, the relevance of this inaccurate diagnosis has not been as exhaustively analyzed,^10,11 with only few reports evaluating it, including a recent clinical audit on its prevalence in Wales.¹²

Accordingly, more information on the quantification of inaccurate diagnosis and the associated determinants would be of interest for clinicians and health-care managers. The Community Assessment of COPD Health Care (COACH) study was a national, observational, randomized, non-interventional, multicenter clinical audit aimed at evaluating clinical practice for patients with COPD in primary care medicine in Spain.¹³ The aim of the present study was to use the COACH data to evaluate the degree of inaccurate diagnosis in primary care medicine in Spain and evaluate the determinants associated with it.

Methods

The methodology of the COACH study has been extensively reported.¹³ Briefly, COACH was a national, observational, randomized, non-interventional, multicenter, clinical audit as result of a joint project between the Spanish Primary Care Respiratory Group (GRAP) and the Spanish Society of Pulmonology and Thoracic Surgery (SEPAR). The design was cross-sectional and recruitment of cases and gathering of information was retrospective by nature.

A randomization procedure was used for center selection, as previously explained¹³ using a Microsoft (Microsoft Corporation, Redmond, WA, USA) Excel spreadsheet, aiming to include 10% of all primary care centers (PCCs) in Spain, with replacement in case of refusal to participate.

The first draft of the databases was later discussed in a face-to-face kick-off meeting in Madrid, on September 13, 2014, and by email and teleconferences thereafter. The aim was to benchmark the results with the two most widely used guidelines in Spain: GOLD 2013¹⁴ and the Spanish National Guideline for COPD (GesEPOC).¹⁵

Information was recorded in a centralized database, organized as a hierarchical web-based tool with different levels of data accessibility. Data were collected from January 1, 2015, to December 31, 2016. Since the data was retrospective, the doctors in charge of the patients were not informed that the audit was being carried out. During this period, each participating PCC was told to search the center’s medical records to select all cases with a diagnosis of COPD in no specific order. The local investigator consecutively reviewed each medical record to confirm that the diagnosis was established in the medical record and, if this was the case, the audit information was recorded. Because accuracy of COPD diagnosis was an outcome measurement of the audit, no diagnostic criteria were required for the case to be included in our study. The classification of the patient as having COPD in the medical record, was sufficient to include the case and perform the audit. No exclusion criteria were defined ad-hoc.

For descriptive purposes, age, gender and tobacco history expressed as pack-year were noted. The setting of the PCC (rural/urban) and comorbidities were also noted. Rural areas were defined as areas with a population of <25,000. Comorbidities were evaluated by comorbidity indexes including the Charlson index¹⁶ and the COPD-specific comorbidity test (COTE).¹⁷

For the present analysis, the evaluation of a correct diagnosis was done at two-time points: once upon establishing a COPD diagnosis as indicated in the medical record and again during the current audited visit. Diagnostic criteria were set according to the GOLD document.² Two criteria were used to detect a correct diagnosis during the diagnostic visit: previous exposure to any noxious particles or gases (smoking, biomass and occupational origins) and the detection of an airflow limitation that is not fully reversible according to spirometry results. Although the aim was to assess lung function by using post-bronchodilator spirometry, most cases only had a pre-bronchodilator spirometry results recorded and available for assessment. In these cases, we decided to consider pre-bronchodilator spirometries. To detect a correct diagnosis in the audited visit we also considered the presence of respiratory symptoms. Dyspnea, cough, and expectoration were considered symptoms associated with COPD. Since chronic respiratory symptoms as a requirement for a correct diagnosis was included in GOLD 2017,² which had not been released when the audit took place, we also studied diagnostic adequacy at the audited visit, with and without symptoms.

The study observed the ethical requirements in Spain, including the Declaration of Helsinki, and the Spanish regulation on data protection and confidentiality (Spanish Organic Law 15/1999 of December 13 on the Protection of Personal Data). Main ethical approval was granted by the coordinating center’s Ethics Committee (Comité de Ética de Investigación de los hospitales universitarios Virgen Macarena-Virgen del Rocío, Seville, Spain, approval act 02/2014) and in every participating region according to local regional legislation. Medical records were anonymized in the database. Informed consent was waived due to the anonymization of the data, the lack of active therapeutic interventions, and the retrospective nature of the study. The Ethics Committee approved this procedure, which was clearly specified in the protocol.

Statistical analysis

Statistical analyses were done with IBM SPSS Statistics, version 20.0 (SPSS; IBM Corporation, Armonk, NY, USA). Descriptive analysis used the mean and SDs or absolute and relative frequencies, depending on the nature of the variable. The inter-regional range, representing the highest and lowest mean values from the participating centers at the regional level, was used to reflect variability. The significance of this variability was explored using the ANOVA test (using the Welch test in case of non-homogeneity of the variances) or chi-square test, according to the nature of the variable. The percentage of accurate diagnosis was obtained for each visit dividing the number of cases fulfilling the diagnostic criteria by the total number of audited cases. Accurate diagnosis was also calculated considering whether the patient had had an obstructive result during the diagnostic or audited visit. Once the cases with a correct diagnosis were identified, we evaluated variables associated with inaccurate diagnosis using the unpaired Student t test for numerical variables and chi-squared test for categorical variables.

Multivariate multilevel binomial logistic regression models were used to explore those patient-level and center-level variables found to be significantly associated with the presence of inaccurate diagnosis.¹⁸ We first constructed an empty model, which included only the hospital-cluster effect. Then, we estimated a second adjusted model, which added the variables at the patient-level (model 1). A final model 2 added the variables at the PCC level. The variables tested in the adjusted model were selected using a forward selection procedure based on the Wald test. The results are expressed as OR with 95% CI. P-value was set at 0.05.

Results

The audit included a total of 4,307 cases that were included from 63 PCC in 5 regions of Spain. The description of the audited cases is summarized in Table 1. This was a cohort of patients identified as being diagnosed with COPD, with a considerable number of active smokers and moderate lung function impairment.

Table 1 Characteristics of the 4,307 audited cases

The accuracy of the diagnostic criteria at the time of the initial diagnosis and during the audited visit is summarized in Tables 2 and 3, respectively. Overall, the diagnosis (exposure + obstruction) was correctly established in 16.1% during the diagnostic visit and was correct during the audited visit in 5.0% and 3.6%, depending on symptom criteria. These figures presented a considerable variability. Considering the best possible scenario in which we require previous exposure and any obstructive spirometry at any visit, without full reversibility, the diagnosis was correct in 758 (17.6%) cases, with an inter-regional range from 9.8% to 23.2%. When we added symptoms to the equation as a necessary criterion for a correct diagnosis, the diagnosis would be correct in 369 (8.6%) cases with an inter-regional range from 0.7% to 14.0%. Therefore, taking these 758 cases as correctly diagnosed, the rate of inaccurate diagnosis in PCC in Spain was 82.4% (inter-regional range from 76.8% to 90.2%).

Table 2 Validation of the initial diagnostic criteria upon diagnosis

Table 3 Validation of the diagnostic criteria in the audited visit

Variables associated with inaccurate diagnosis in the crude bivariate analysis are summarized in Tables 1S and 2S in the online supplement. The results of the multivariate multilevel regression analysis are shown in Table 4. Patient-related interventions associated with inaccurate diagnosis were related to active smoking, lung function evaluation with mild disease and specific interventions. Notably, other PCC-level variables were associated with the availability of certain complementary tests and different aspects of the resources available.

Table 4 Multivariable multilevel regression analysis showing factors associated with COPD inaccurate diagnosis

Discussion

The present study analyzes the inaccurate diagnosis of COPD in primary care medicine in Spain, quantifying the degree of inaccurate diagnosis and evaluating the number of factors at the patient and PCC level that are associated with this inaccurate diagnosis. As a result, variables related to smoking status, lung function evaluation and specific interventions were considered important at the patient level. Additionally, the availability of complementary tests and different aspects of the resources available including the presence of primary care trainees, the availability of a tobacco cessation unit or home nebulized therapy were also associated with an inaccurate diagnosis.

The main strengths of this study include the national coverage and evaluation of the different diagnostic criteria both separately and in a single composite index. However, in order to be able to interpret our results correctly, there are some methodological aspects that should be taken into consideration. First and most importantly, an audit aims to assess clinical performance, not the results thereof. Thus, the focus of the audit is to have an idea of the practitioner’s clinical performance, rather than the clinical situations of the patients. Second, the clinical audit data is collected from the contents of medical records and therefore, undocumented interventions are not evaluated. Accordingly, the estimates presented here may have underestimated the real values. Our results further support the need to record every aspect of clinical practice in the medical record.¹⁹ Third, the application and interface to access the data differed depending on the region of the country, which may have influenced the accessibility of the data. Therefore, our first step in the analysis was to evaluate data that was extreme, missing, or inconsistent. This was corrected by the investigators upon request. Fourth, the audit evaluated the medical record at two specific time points, ie, upon diagnosis and during the current visit. It is possible that the visits in between may offer relevant information that has not been considered here.

Although the diagnostic criteria for COPD are quite clear in the current recommendation documents, the adequate diagnosis of this disease is far from being optimal in real life. There are likely several factors that have a complementary influence on this situation. First, the impact of epidemiological studies on the prevalence of the disease may have played an important role. These studies, generally based on population, study the prevalence of COPD by performing spirometry in the population without considering other diagnostic criteria. As a result, prevalence studies often report the prevalence of chronic airflow obstruction rather than COPD, contributing to the confusion.^20,21 Secondly, considering the clinical context, the next key element for diagnosis is the performance of spirometry. This spirometry should be done in a stable clinical situation, at rest, and with a bronchodilator test. The performance of spirometry after the bronchodilator test is essential to confirm the diagnosis, not because of the potential broncho-reversibility that may present, but rather because it is spirometry after bronchodilation that establishes the functional diagnostic criterion.²² Unfortunately, the verification of this diagnostic criterion in primary care medicine is far from optimal in Spain. A study conducted in Spain evaluated the availability and frequency of performing spirometry in primary care medicine in Spain,²³ revealing that most health centers had a spirometer. However, the frequency of performing spirometries was 5.6 per week with an inter-regional range between 2.0 and 8.8 spirometries per week for the study of airway diseases. Considering the population prevalence of COPD, asthma and bronchiectasis – the three main chronic airway diseases by frequency – it can clearly be concluded that the frequency of spirometry in primary care medicine is insufficient. The present study reflects this situation, since the performance of spirometry was the most lacking diagnostic criterion in our cohort. In this regard, there are some recent initiatives to try to improve lung function evaluation in primary care medicine with different devices beyond formal spirometry such as the Piko-6,²⁴ the COPD-6,²⁵ micro-spirometry²⁶ and the peak-flow meter.²⁷ Additionally, several questionnaires have also been evaluated.^28,29 Third, another aspect that may have contributed to the confusion in the diagnosis of the disease is the inclusion of the symptoms.³⁰ Through the 2016 version, the GOLD document was not clearly limited in defining whether symptoms should be a diagnostic criterion in establishing the COPD label. In the 2017 version,² GOLD changed to include symptoms in the concept and the diagnostic criteria. This new approach is accompanied by another debate regarding which symptoms should be considered indicative of the disease. In the present work, we have selected dyspnea, cough and chronic expectoration because they symptoms most frequently evaluated by the clinician, who is familiar with their measurement. However, the clinical expression of the disease is more extensive and symptoms such as fatigue³¹ or limitation of daily activities^32,33 constitute areas of the disease that must be borne in mind.

A recent clinical audit has evaluated the degree of inaccurate diagnosis in PCCs in Wales.¹² These authors evaluated 48,105 patients and found that 13.9% of evaluated cases had compatible post-bronchodilator spirometry. Therefore, by only considering functional criteria, the degree of inaccuracy in this diagnosis was high. Additionally, this figure would likely decrease if documented previous exposure and symptoms were considered diagnostic criteria in the analysis.

Several key aspects at the patient level were associated with inaccurate diagnosis. Active smoking³⁴ and certain features associated with severity, ie, the use of double bronchodilation, lung function evaluation and the use of home-based therapies are likely expected findings. Interestingly, milder COPD was associated with an increased probability of a correct diagnosis, likely because these mild patients need spirometry to be detected.³⁵ The use of long-term oxygen therapy and the availability of computed tomography reflects contact with respiratory clinics, and it is therefore associated with a confirmed diagnosis. Similarly, the association with health-status evaluation may indicate a specific primary care clinic devoted to COPD or feedback from a respiratory clinic, since health status evaluation is relatively uncommon in clinical practice.¹³ Comorbidities were not associated with inaccurate diagnosis. This suggests that the presence of other comorbid conditions is not associated with an erroneous diagnosis. However, they can be associated with under-diagnosis, since other diseases can also explain respiratory symptoms.^36,37

In summary, the present study analyzes the prevalence of inaccurate COPD diagnosis in primary care medicine in Spain. The prevalence data establishes a point of reference in the clinical management of COPD at this level of care. The descriptors of the variables associated with this inaccurate diagnosis can be used to identify cases and centers in which inaccurate diagnosis is occurring considerably. In this way, we can establish targeted interventions that can improve the detection of COPD cases by making health care more efficient.