Real-World Evaluation of an EHR-Enabled Chronic Obstructive Pulmonary Disease Assessment Test

Introduction

Chronic obstructive pulmonary disease (COPD) is characterized by a persistent reduction of airflow, chronic respiratory symptoms including cough, increased sputum production, and dyspnea.¹ COPD is a leading cause of death in the United States, affects over 15.6 million Americans, and results in 1.5 million emergency department (ED) visits and 726,000 hospitalizations annually.^2,3 COPD exacerbations, defined as an event characterized by increased dyspnea and/or cough and sputum that worsens within 14 days,⁴ are associated with reduced quality of life, increased mortality,³ and approximately 50% of the direct costs of COPD care.⁵ History of exacerbations is one of the most important predictors of future exacerbations.⁶

The goals of COPD management are to reduce symptoms and exacerbations.⁴ Spirometry is an important tool for the evaluation of COPD, but at an individual patient level it has only weak to moderate correlation with various COPD symptoms.^7–9 As a result, patient reported outcome measures, such as the COPD Assessment Test (CAT), are recommended to supplement the evaluation of a patient.^4,10 The CAT has demonstrated predictive validity, effectively identifying individuals at increased risk of exacerbations, hospital 30-day readmission, and mortality.^11–13 The CAT has high test–retest reliability (measures ranged from 0.80 to 0.96) and was significantly correlated with other established measures of COPD, such as the St. George’s Respiratory Questionnaire for COPD patients (SGRQ-C) and the Clinical COPD Questionnaire (CCQ).¹⁴ Subsequent research has shown that the CAT can be used as a measure of treatment response with a change of two points on the 40-point scale deemed clinically meaningful.¹⁵ Although the CAT is validated, and its use is consistent with international guidelines, actual utilization in real-world US clinical practice is not well described in the literature.

COPD disease severity has been traditionally determined by pulmonary function testing, but the use of patient reported outcome measures are helpful to address multiple different quality of life symptoms associated with COPD.¹⁶ In 2017, M Health Fairview made a digital format of the CAT available in their electronic health record (EHR). The objective of this research was to evaluate a real-world implementation of the CAT and assess the association of CAT scores with subsequent COPD care and disease outcomes.

Materials and Methods

This was a retrospective observational comparative cohort study that assessed the early experiences of a healthcare system using an EHR-enabled (digital) CAT. Key assessments were among two groups: 1) individuals with COPD who had a complete EHR-enabled CAT score in at least one clinical encounter (case group), and 2) a comparator group of patients with COPD who did not complete an EHR-enabled CAT during any clinical encounter. The study was conducted at M Health Fairview, a partnership between the University of Minnesota, University of Minnesota Physicians, and Fairview Health Services, which provides healthcare services primarily in the state of Minnesota and has more than 3300 providers across more than 100 specialties.¹⁷

COPD Assessment Test Implementation

The CAT is a validated 8-item instrument designed to measure functional limitations associated with COPD symptoms^10,11 (Supplementary Figure 1). CAT scores range from 0 to 40, with higher score indicating a greater symptom burden. M Health Fairview implemented the EHR-enabled CAT in July 2017 with the goal of improving the documentation of patient-reported COPD outcomes. Administration of the CAT by a provider was completely voluntary, no prompts, alerts, or decision support were instituted to suggest its use as part of clinical care, and care teams did not receive formal training in its use.

Figure 1 Study Schematic. Abbreviations: CAT: COPD Assessment Test, COPD: Chronic Obstructive Pulmonary Disorder.

Study Periods

The study period was from July 1, 2016, through February 28, 2022. To ensure adequate follow-up time for the analysis of post-CAT care patterns, the case-finding period was July 1, 2017, through December 31, 2021. The dates were selected to coincide with the implementation of the EHR-enabled version of the CAT within the healthcare system. To understand differences in the populations who did and did not complete the CAT, Baseline included the 12-month period prior to the initial CAT administration (CAT cohort) or the first outpatient COPD encounter (comparator group). To understand the patterns of clinical management and outcomes based on the CAT score, the index date was defined as the encounter with the first CAT administration with a physician or advanced practitioner in pulmonology and primary care. If the patient received only CAT administrations with non-primary care or non-pulmonologists, the first outpatient appointment with any physician or advanced practitioner was used as the index. The 12-month period prior to the index was used to identify a history of COPD exacerbations. The 60-day period following the index was used to identify follow-up care in pulmonology, pulmonary rehabilitation, and intensification of COPD medication regimens (Figure 1).¹⁸

Identification and Selection of Study Participants

To be eligible for the CAT cohort, an individual had to complete a CAT during the case-finding period. To be eligible for the comparator group (No CAT cohort), study participants met each of the following criteria: 1) diagnosis of COPD defined by either “COPD” listed on the Problem List or a history of at least two outpatient visits associated with an International Classification of Diseases Tenth Revision Clinical Modification (ICD-10-CM) of COPD (J44.x) or an inpatient hospitalization associated with ICD-10-CM of J40.x-J44.x in the primary position during the baseline through the case-finding period; 2) at least one visit at an M Health Fairview clinic during the case-finding period; 3) 18 years of age or older as of the first date of baseline, and 4) no evidence of a completed CAT.

Outcome Variables and Covariates

Patient demographic characteristics included in this study were age as of first date of Baseline (years), sex (male vs female), race (white, black/African American, and other), COPD-related diagnoses (COPD, emphysema, and bronchitis) and comorbidities including the modified Charlson Comorbidity Index, which was categorized as 0 vs 1+.¹⁹ Clinical characteristics included pulmonary function test results and COPD exacerbations. The most recent pulmonary function test results within 6 months up to and including the index date were recorded and included forced expiratory volume in one second (FEV₁) measured in liters (L) and % predicted, forced vital capacity (FVC) measured in liters (L) and % predicted, and the ratio of FEV₁/FVC. COPD encounters were defined as completed office visits associated with ICD-10 codes J44.X, J40, J41, J42, and J43 in any position. An exacerbation event was defined by a visit with an ICD-10 code of J44.1 in any position or an ED or hospitalization with an ICD-10 code for COPD (J44.x), emphysema (J43.x) or bronchitis (J40.x, J41.x, or J42.x) in the primary position. Subsequent exacerbation events require a prior two-week period free of ED visits, hospitalizations, or exacerbation codes.

Provider characteristics included specialty (defined by pulmonology, primary care, and others) and degree (MD, DO, NP, etc). Measures related to the CAT included the index CAT scores, proportion of all COPD visits for which a CAT was administered, providers who administered the CAT, and the number of CATs administered per provider. Providers associated with one or more encounters with CAT administration were defined as CAT administrators.

The clinical outcome measures included completion of a pulmonary rehabilitation visit, a pulmonology visit, an intensification of the COPD treatment regimen, and an exacerbation during the follow-up period. Intensification of the treatment regimen was defined as a) the addition of new COPD medication to an existing regimen, including one or more inhaled corticosteroids (ICS), long-acting beta agonists (LABA), long-acting muscarinic antagonists (LAMA), phosphodiesterase inhibitors, oral corticosteroids, two short-acting bronchodilators, or azithromycin, or b) an increase in ICS dose. To avoid confusion with simple refilling of medications, the order for a single short-acting bronchodilator was excluded from the definition of treatment intensification.

Analysis

Descriptive characteristics, including demographics, comorbidities, and pulmonary function test results, were compared between the CAT and No CAT cohorts. The 12-month period prior to initial CAT administration was used to derive these measures for descriptive analysis (CAT cohort only). Chi-square and Student’s t-tests were used to compare the populations for categorical and continuous variables, respectively. We also compared the characteristics of providers associated with administered CATs (Administered CAT) and those not associated with administered CATs (No CAT) using chi-square tests. Differences were considered statistically significant at p <0.05. Exploratory analyses were conducted to evaluate the distribution of the CATs across the study period. This distribution was broken down by the provider (Primary Care Physician (PCP) or pulmonologist) that administered the CAT. The CAT scores were then divided into categories of low impact (<10), moderate impact (10–20) and high/very high impact (21–40). The distribution of pulmonary interventions within 60 days following the CAT index was compared across the CAT score groups. These interventions included COPD medication regimen intensification, pulmonologist visits, and pulmonary rehabilitation. Adjusted analyses accounting for age, sex, and baseline pulmonary rehabilitation and pulmonology encounters were conducted using logistic regression and Cox proportional hazards models. A logistic regression model was developed to predict the likelihood of a medication intensification event within 60 days of the index. In addition, separate proportional hazard models predicting the time to first exacerbation, time to first pulmonary rehabilitation visit, and time to first visit with a pulmonologist within 60 days following index were conducted, controlling for age, sex, and baseline exacerbation. We reported odds ratios (OR) and hazard ratios (HRs) for these respective models tested at a significance level of 0.05.

Ethics Approval

The research protocol received an exemption from IRB review determination from the University of Minnesota Institutional Review Board (STUDY00015433) as this was secondary research for which consent is not required. The study complies with the Declaration of Helsinki. All methods were performed in accordance with privacy and confidentiality guidelines and regulations.

Results

Patients

A total of 11,194 individuals with COPD met the eligibility criteria during the study period, including 821 individuals (7.3%) who were administered the EHR-enabled CAT (cases). Compared to individuals with COPD who were not administered the CAT (comparators), cases were older (66.7 vs 63.9 years; p < 0.05), had higher rates of comorbidities as assessed by a Charlson Comorbidity Index score of 1 or greater vs none (93.9% vs 79.2%, p < 0.05), and had a higher prevalence of emphysema (64.1% vs 39.4%, p < 0.05). With regards to pulmonary function, CAT cases were more likely to have a pulmonary function test within 6 months (29.0% vs 10.0%, p < 0.05) and to have a lower FEV₁/FVC ratio (59.0% vs 62.4%, p < 0.05). CAT cases also had higher rates of exacerbations (0.31 vs 0.14 per patient per year) during the baseline period. The groups did not differ by sex or race (Table 1).

Table 1 Description of Eligible Characteristics, Derived from the Baseline Period, for Patients Who Did and Did Not Complete the COPD Assessment Test

Providers

The types and specialties of providers included in this study are reported in Table 2. Within each specialty, over the study period, 62.0% of all pulmonologists and 11.5% of all PCPs administered the CAT in at least one encounter (Table 3). During the study period, there were 72,201 COPD encounters, 16,698 COPD encounters in pulmonology, and 55,503 COPD encounters in primary care. A total of 1502 CATs were administered over the study period in all specialties: 949 in pulmonology and 491 in primary care. The monthly usage of the CAT for COPD encounters reached a peak of 15% for pulmonologists and remained less than 2% for PCPs, and its use appeared to decline during the coronavirus disease 2019 pandemic period, beginning in early 2020 (Figure 2). The number of CAT administrations by pulmonologists and PCPs were a) one only (6.3% and 4.5%, respectively), b) 2–5 (15.2% and 4.8%), c) 6–10 (11.4% and 1.2%), and d) 11 or more (29.1% and 1.0%) (Table 3).

Table 2 Provider Type and Specialty for CAT and No CAT Providers

Table 3 Uptake of CAT by Pulmonologists and PCPs

Figure 2 Monthly percentage of COPD visits with CAT administration, by provider specialty. Abbreviations: CAT: COPD Assessment Test, COPD: Chronic Obstructive Pulmonary Disorder, PCP: Primary Care Physician.

CAT Scores

At index, a total of 130 (15.8%) patients had a score <10 points, 385 (46.9%) had a score between 10 and 20 points, and 306 (37.3%) scored 21–40 points. During the study period, 386 patients (47.0%) had one CAT administered, 280 patients (34.1%) had 2–3 CATs and 155 patients (18.9%) had four or more CATs (Table 1).

Post-CAT Clinical Changes

Stratified by CAT index provider type, index CAT scores of greater than 20 were associated with intensification of the COPD medication regimen by pulmonologists and PCPs in 23.0% and 28.6% of patients, respectively, for CAT scores of 10–20, 19.2%, and 20.5%, and CAT scores less than 10, 12.7%, and 5.3% (Table 4). Unadjusted odds ratio for having COPD medication intensification 60 days post-CAT for high scores (21–40) and moderate scores (10–20) compared to low scores (less than 10) were 3.19 and 2.57, respectively (p = 0.0001) (Table 5). After adjusting for a history of COPD exacerbations in baseline, individuals with the highest CAT scores (21–40) and moderate CAT scores (10–20) continued to be significantly more likely to have their medications changed compared to those with low CAT scores (<10) (21–40: OR: 4.40, 10–20: OR: 3.17, p = 0.0002) (Table 5).

Table 4 Index CAT Scores, Index CAT Provider Specialty, and Subsequent 60-Day COPD Medication Regimen Changes

Table 5 Logistic Regression Model* Results for Medication Regimen Intensification in the 60 Days Following Index CAT Administration

Individuals with the highest index CAT scores (score: 21–40) were most likely to have an exacerbation event in the subsequent 60 days (24.2%), followed by 17.4% of individuals with an index CAT score from 10 to 20 and 7.7% of individuals with an index CAT score <10 (p < 0.001) (Supplementary Figure 2).

In the 60 days following the index CAT, 30.1% and 3.3% of patients with the highest CAT scores (20+) had a visit with a pulmonologist or pulmonary rehabilitation, respectively. We compared 28.1% and 2.1% for individuals with CAT scores from 10 to 20 and 25.4% and no visits to pulmonary rehabilitation for individuals with CAT scores of <10. While the likelihood of completing a visit with a pulmonologist (CAT score 21–40 compared to CAT score <10, Hazard Ratio = 1.1, 95% CI 0.7,1.7) and pulmonary rehabilitation (CAT 21–40 compared to CAT <10, Hazard Ratio = 1.4, 95% CI 0.5, 3.5) in the 60 days post-index increased with CAT score category, these differences were not statistically significant (Supplementary Figures 3 and 4).

Discussion

In this real-world implementation of the CAT for individuals with COPD, our study documents the natural history of clinical uptake and use of a patient-reported outcome measure in routine clinical operations over a four-year period. While the uptake of the CAT was limited, repeated use was observed in over half of the pulmonologists, while it remained uncommon among PCPs. Patients with elevated CAT scores were significantly more likely to have subsequent medication regimen intensifications and exacerbations, and the results suggest an increase in follow-up visits with a pulmonologist or pulmonary rehabilitation within 60 days.

Implementation of the CAT into routine clinical practice has the potential to systematically identify patients with a higher COPD symptom burden at near-term risk for exacerbation, who may benefit from more intensive management. Our analysis demonstrated that individuals with elevated CAT scores were significantly more likely to undergo COPD treatment intensification as well as suffer COPD exacerbation. While PCPs provide up to 80% of the care for individuals with COPD,²⁰ the evidence of low PCP engagement with CAT in this study suggests both an important gap in care and the need for more robust implementations (ie, training, clinical decision support, and performance tracking) to achieve successful adoption. While the use of pulmonary rehabilitation for individuals post-CAT was low, the trend toward increased referral for higher scores is encouraging, and future research could consider using elevated CAT scores as a trigger for referral to pulmonary rehabilitation.

The incorporation of patient-reported outcomes (PROs), such as the CAT, into routine clinical care represents an opportunity to improve clinical care and enable patient-centered research. There is precedent for electronic capture of PROs for the purpose of public reporting.²¹ Collection of symptom-based PROs augments patient charts with information not documented by physicians, creating a more complete database from which to manage populations, prevent admissions and readmissions, and ensure alignment with guidelines.^22,23

While substantial progress has been made into incorporating PROs²⁴ into care, including for the treatment of COPD,²⁵ there are barriers to adoption. In this study, the barriers included: a) limited knowledge of the CAT availability in the EHR, b) no standardization of in-clinic workflow, c) no associated clinical decision support for providers, d) lack of performance measures with provider feedback, and e) no associated incentives for use. Negative provider perceptions of clinic-based PRO collection can be another barrier to adoption.²⁶ While national guidelines recommend the use of structured symptom assessments to monitor individuals with COPD, the implementation and evaluation of improvement initiatives is needed to optimize data collection and presentation of CAT results to enable medical decision making while avoiding provider and patient burden. Corresponding improvement initiatives could include expanded education and awareness programs for providers, enhanced EHR functionality to link PRO responses to clinical decision support, revised clinical workflows to empower clinic staff to collect PRO data during the rooming process, and integration of CAT PRO data as a health system performance measure.

Global guidelines recommend regular assessments such as the CAT and their use is not limited to primary care or pulmonology visits. Clinical pharmacists and respiratory therapists are strong components of chronic COPD management and are well suited to using PROs to improve care.^27–29 Additionally, programs using the CAT for remote monitoring or self-monitoring³⁰ have the potential to identify worsening clinical status between provider visits. Out-of-clinic use of the CAT has the potential to enable early detection of exacerbations while avoiding provider concerns regarding increased in-clinic burden.

Limitations

These results should be interpreted within the context of the limitations of a retrospective, observational study. COPD diagnosis was based on ICD-10-CM codes and was not independently confirmed by test results. The implementation of the CAT without explicit training, formal workflow modification, performance reporting, or the use of incentives is not ideal for the adoption of electronic tools and may not reflect the experience of others using the CAT. The resulting low rate of CAT administration, although an outcome of the study, also represents an unmeasured selection bias. This study also primarily focused on the implementation of the CAT among prescribing providers, with less attention paid to non-prescribing providers such as pulmonary rehabilitation, where it is often used to document functional outcomes.^31,32 The low uptake and use of the CAT in this system contrasts with a substantially higher uptake and use of the Asthma Control Test, another patient-reported outcome measure, which is a reportable quality measure.⁹ While this study demonstrates an association between increased CAT scores and COPD medication regimen intensification, a causal relationship has not been established.

Conclusion

The implementation of CAT in routine clinical operations for individuals with COPD requires overcoming technical challenges and workflow inefficiency. While adoption of the CAT was limited, higher CAT scores were associated with COPD exacerbations, and subsequent treatment intensification. Repeated use of the CAT was present in over half of pulmonologists and less than 10% of PCPs, suggesting that the use of CAT was valued more by specialists, although most COPD care was delivered in primary care. Further investigation is warranted to optimize CAT implementation into routine clinical practice, especially within primary care, to systematically identify patients with a higher COPD symptom burden and enable proactive management.