Validation of COPDPredict™: Unique Combination of Remote Monitoring and Exacerbation Prediction to Support Preventative Management of COPD Exacerbations

Introduction

Chronic Obstructive Pulmonary Disease (COPD) is a progressive lung condition characterized by persistent respiratory symptoms and life-changing disruptive exacerbations associated with a faster decline in lung function, poorer quality of life and increased mortality.^1,2 Over 70% of patients living with COPD experience exacerbations within 3 years of diagnosis;^3,4 and up to 47% may have two or more exacerbations a year irrespective of airflow severity (so-called ‘frequent exacerbator phenotype’).^5,6 Exacerbations of COPD have a global detrimental and substantial impact on patients, healthcare resources and the wider society. Exacerbations prevent many working-age individuals from working, with billions of dollars lost through reduced productivity.⁷ Severe exacerbations remain the 2nd commonest cause of emergency hospital admissions; with 1 in 3 patients readmitted within 3 months of discharge.^8,9 These hospitalisations account for around two-thirds of all COPD-related healthcare costs.¹⁰ Frequent moderate episodes are community-treated but still consume significant primary care resources.¹¹ The COVID-19 pandemic-induced changes in healthcare provision, amidst ‘lockdown’ restrictions and reduced specialist access, have increasingly shifted service provision and treatment costs of COPD exacerbations onto community care resources.¹²

Changes in dyspnoea, coughing and/or sputum production often precede COPD exacerbations. As symptoms vary within the same day and/or daily, patients cannot easily judge the significance of such changes and so fail to follow physician-recommended ‘action plans’ and respond promptly to exacerbations.^13,14 Reports show that patients do not adhere to their provided action plans in about 50% of exacerbations.¹⁵

Clearly COVID-19 is mandating the transformation of healthcare delivery for chronic disease management such as COPD, with technological advances making it possible for clinicians to monitor patients remotely and safely at home and guide appropriate treatment.¹⁶ Developing models of care based around prevention through a combination of remote monitoring and personalised exacerbation prediction could optimize action plans and tailor exacerbation management, reducing overall service delivery costs. This approach would crucially address patients’ needs for prompt access to specialist care and appropriate treatment as required. When exacerbations are treated promptly, exacerbation severity, recovery time and hospital admissions/readmissions can be reduced.¹⁷ Indeed, there is a time window between the initial exacerbation symptoms/signs and subsequent hospitalisation within which there is an opportunity for preventative intervention.^18–20

COPDPredict™ is a customized digital health application collaboratively designed and developed by clinicians and an “experts by experience” patient group to assist patient self-management of their condition. This collaborative design approach facilitated an appreciation of what was important to patients in experiencing exacerbations and thereof, the inclusion of elements to enable them to identify COPD-related exacerbations early, enabling prompt intervention. COPDPredict™ allows patients to collect patient-reported outcomes and bio-physiological data via an App and connected devices (eg, smart spirometers) and share information securely with their healthcare team. Clinicians can review all patients’ health data in real time using a bespoke decision support dashboard. As there is marked heterogeneity amongst COPD populations, COPDPredict™ is programmed to automatically establish patient-specific critical baseline bio-clinical profiles so that prediction of exacerbation is personalised and clinically meaningful. Indeed, an individualised profile of relevant health indicators which could be tracked over time was identified as important by patients to aid their self-management, thus avoiding the conventional one-size-fits-all approach with static thresholds²¹ or subjective setting of per patient reference values by healthcare teams.²² A systematic review of clinical prediction models for COPD exacerbations observed that when predictor availability and practical applicability were considered, none of the models evaluated at that stage were deemed ready for clinical implementation.²³ With COPDPredict™, patient health data is continuously processed using proprietary cloud-based algorithms, with the driving algorithm decision tree constructed from percentage thresholds for disease state changes in patient-reported wellbeing, forced expiratory volume in one second (FEV₁) and blood-based C-reactive protein (CRP) levels.^24,25 Accordingly, COPDPredict™ aims to 1) determine an individual’s COPD health status in real time, 2) predict imminent exacerbations in an early phase, and 3) provide early warning alerts directly to both patients and clinicians for prompt and adequate action to prevent clinical deterioration and unnecessary hospitalisations.

In this study, we evaluated whether COPDPredict™ can be safely used by patients with COPD and their clinical teams in the everyday management of COPD. More specifically, we determined the sensitivity and specificity of the prediction algorithm to differentiate between the varying health phases of patients with COPD, identifying imminent exacerbations and providing their timely notification to both patients and clinicians for appropriate intervention.

Materials and Methods

Study Design, Population and Sample Size

We performed a 6-month prospective observational study in a North Midlands community-setting (UK) involving 90 patients with COPD between May 2017 and April 2018. Individuals were identified and randomly selected, independent of severity, from the University Hospitals of North Midlands NHS Trust research and outpatient clinic databases (Figure 1). Inclusion criteria included (i) established COPD confirmed by spirometry and categorized according to GOLD guidelines;²⁶ (ii) history of frequent exacerbations with two or more documented acute episodes²⁷ and a minimum of one COPD-related hospitalisation in the preceding 6 months; (iii) clinically stable and exacerbation-free in the six weeks prior to enrolment. Individuals with other respiratory disorders and unstable co-morbidities, an inability to consent and/or inability/unwilling to use COPDPredict™ were excluded. Patients were enrolled using staggered entry points and remained on the study for a maximum of 6 months. Staggered entry enabled the research team to manage effective set-up of COPDPredict™ in the patients’ homes and ensure clear patient understanding of the required data entry. All recruited patients were required to have internet access. However, all patients were given mobile tablets (Galaxy A3 Samsung^®, South Korea) pre-installed with the COPDPredict™ application (App).

Figure 1 Study flowchart total number of patients screened and included/excluded for the study. This image is the property of the author.

Longitudinal recording of bio-clinical metrics in the study cohort allowed stable to prodromal to exacerbation phases to be determined. Within subject differences could be generated and summarised with appropriate 95% confidence intervals; a previous study²⁴ indicated that a minimum sample of 40 COPD patients would be sufficient. Therefore, we concluded that a total sample of 90 patients with COPD and frequent exacerbator phenotype would provide reasonable estimates of population differences and standard deviation across the different disease phase dynamics as well as enough exacerbations for cross-validation of the prediction algorithm.

The study was conducted in accordance with the Declaration of Helsinki. Ethical approval was granted for the study from the North West - Greater Manchester South (12/NW/0623) and Central Research Ethics Committees (15/NW/0638); UK CPMS ID 19566. All patients were provided with a Participant Information Sheet. All patients gave informed written consent to participate and were given no reimbursement.

Data Collection Protocol

All 90 patients used the COPDPredict™ App and a provided BT smart lung monitor (Vitalograph^®, Ireland) to self-monitor their COPD in their homes. Wellbeing self-assessment was based on a set of questions and a choice of answers from which a person could choose how they felt (Figure 2A and B); with responses set out simply using a five-level Likert scale (please see Supplementary Information). An overall wellbeing score was automatically calculated by the App from the answers supplied to 5 key scored components in the self-assessment relating to breathing, cough, sputum and activities of daily living.²⁴ The Bluetooth-enabled spirometer allowed patients to perform lung function at home and the result automatically captured into the App. Blood CRP levels were measured by the research team using finger-prick testing (Eurolyser Diagnostica^®, Austria), with results inputted into the App. Induction was carried out in the patient’s home: Each patient was given an initial face-to-face coaching session by the research team on how to use COPDPredict™. Further support was provided throughout the study for any problems or concerns regarding use of the technology; patients could also contact the team via the secure within-App messaging facility.

Figure 2 (A) Screenshot of the COPDPredict™ Wellbeing self-assessment opening screen. (B) Screenshot of the first page of the Wellbeing self-assessment Both images are the property of the author.

Critical baseline data on wellbeing, physiological (FEV₁) and biomarker (CRP) parameters were established for each patient in the first fortnight post-enrolment. Thereafter, for the remaining study period, patients performed their wellbeing assessment daily and spirometry weekly. The only derogation to this protocol was during algorithm-prompted notifications of a change in health status and/or clinical team instruction for additional wellbeing self-assessment, FEV₁ and/or CRP tests.

Patients could provide any additional information about their health that they thought appropriate in a dedicated free text section. All patients were provided with “standard rescue” medication of a 5-day course of oral steroids (prednisolone 30mg/day) and antibiotics such as doxycycline, amoxicillin, clarithromycin [the antibiotic and dose used was determined by the individual’s previous treatment history], allowing them to self-manage COPD exacerbations in line with their clinician-prescribed action plans.^28,29 Patients were advised the clinical team would remotely monitor COPDPredict™ usage and data entry daily, and would be contacted by clinicians as appropriate.

The clinical team monitored patients’ progress using a web-based decision support dashboard. Exacerbation alerts produced by the prediction algorithm were sent in real time to both patients and clinicians; patients were also directed to self-management action plans. Treatment plan changes could also be activated by clinicians based on their judgement at the time either via text/telephone message or by nurse-led home visits as appropriate. The clinical team consisted of 2 respiratory physicians and 2 respiratory-trained specialist nurses.

Definition of COPD Exacerbations

Both patient-based and clinician-based definitions were used to compare with the COPDPredict™-driven exacerbation alerts.

Current guidelines were used for defining COPD exacerbations as acute events characterized by a worsening of the patient’s respiratory symptoms that were beyond normal day-to-day variations and leading to changes in medication.³⁰ A mild/moderate exacerbation was defined as an increase in respiratory symptoms for two consecutive days or more, with at least two major symptoms (dyspnoea, sputum purulence, sputum volume) or a major plus a minor symptom (wheeze, cold, sore throat, cough)³¹ and requiring a change in usual treatment with introduction of systemic steroids and/or antibiotics by the decision of a clinician; a severe exacerbation was an episode that also required hospitalisation. Defining exacerbations in this way allowed documentation of the number and severity of any COPD-related exacerbations during the study.

Patients were instructed to contact the clinical team every time they felt a worsening of their COPD. On signaling of a “patient-defined exacerbation”, patients were contacted by the clinical team to confirm or otherwise the presence of a COPD-related exacerbation (“clinician-defined exacerbation”). Patients were also provided with advice on taking rescue treatment (steroids and/or antibiotics). Further clinician-led intervention was initiated if patients did not respond to the rescue medication and the monitored health data indicated delayed recovery.

COPDPredict^TM Exacerbation Algorithm

The study design was used for the algorithm decision tree and split sequentially based on routine daily Wellbeing Score entries and weekly spirometry. Derogation to this testing frequency was based on the alerting thresholds for both Wellbeing Score and FEV₁ which could prompt further testing of FEV₁ and/or blood CRP test. One key feature of the exacerbation prediction model was the 2-week learning phase in the first fortnight when baseline data entries required daily Wellbeing Scores, FEV₁ every 3rd day and blood CRP levels at Day 1 and 14. This set the personalised critical stable baseline for each parameter. The percentage differences between the 50th percentile prodromal value and baseline value and absolute value changes were used to set alert thresholds for changes in disease state (eg, stable, prodromal), with receiver operating characteristics (ROC) curves used to define stable to exacerbation alert thresholds. Alerting thresholds were thus defined separately for each parameter and universally set for all patients. Automatic reminders were sent to patients if the wellbeing self-assessment was not completed within 24 hours. However, patients could submit more than one self-assessment daily if they felt their health had changed during the course of a day; with a Wellbeing Score calculated for each generated self-assessment, thus providing real-time information on how the patient was feeling (and any change in circumstance).

Statistical Analysis

Statistical analysis was performed using IBM SPSS Statistics Version 24 (IBM, USA). Parametric data were expressed as mean ± standard deviation, and non-parametric as median, interquartile range (IQR).

All “clinician-defined” exacerbations were cross-compared to the “algorithm-defined” exacerbation notifications to allow for cross-analysis to determine the accuracy, sensitivity, specificity, and the algorithm’s predictive timing for exacerbation detection. All data was cross-referenced with the registered clinical history of each patient to ensure accuracy. The prodrome of an exacerbation was defined as 14 days prior to the onset of a clinician-defined exacerbation. By analyzing a 14-day window prior to the onset of a clinician-defined exacerbation, it was possible to explore whether the notifications generated by the COPDPredict™ model provided an early warning to an imminent exacerbation. Sensitivity, specificity, positive, negative predictive values, positive, negative likelihood ratios, and accuracy were calculated with confidence intervals (CI) using “exact” Clopper–Pearson, Log method and standard logit tests.

For non-parametric data, between-group comparison was performed using Mann–Whitney U-test, and Kruskal–Wallis one-way analysis of variance (ANOVA). For parametric data, between-group comparison was performed using a paired t-test. A p-value of less than 0.05 was considered statistically significant.

Results

Ninety patients with COPD (Table 1: 45 males and 45 females; age range 48 to 91 years; FEV₁ range 15% to 86% predicted) completed the 6-month study; no adverse events or deaths were reported. There was 98% patient compliance with completing the daily wellbeing self-assessments, supported by timed automatic prompt notifications via the App. There were 19 separate instances when patients submitted more than one daily wellbeing assessment; in all these instances, patients had felt a significant worsening in their breathing and/or cough since recording their assessment earlier in the same day. Some patients (12 out of 90) used the free text section at the end of the self-assessment to explain their wellbeing in their own words: Comments included “Much better today on treatment”; “Feeling generally off”.

Table 1 Patient Characteristics at Enrolment (n = 90)

Exacerbations

During the study period, 112 clinician-defined exacerbations (108 mild/moderate; 4 severe) were evidenced in 80 patients; 10 patients did not experience a significant acute episode and self-managed minor symptoms with adjustments to usual inhaler therapy in accordance with their action plans. Of the 80 patients who experienced exacerbations (Table 2), 52 had just one exacerbation, whilst the remaining 28 patients experienced more than 1 acute episode (2.2 ± 0.4 episodes) (Table 3).

Table 2 Characteristics of the Different Patient Cohorts: Non-Exacerbators and Exacerbators (n = 90)

Table 3 Characteristics of Single (n=52) and the Re-Exacerbator Cohort (n=28)

Sixty-two of the 80 exacerbating patients managed their exacerbation events at home using the provided “rescue pack” of oral prednisolone ± antibiotics as directed by the clinicians and action plan which followed best clinical practice in line with national/international guidelines for COPD exacerbation management.^18,26 In this study, antibiotics were considered on an individual patient basis; generally, all patients showing increased sputum purulence and/or high CRP levels were directed to take antibiotics in addition to prednisolone. Fourteen participants did not improve on standard treatment alone and required escalated home treatment involving nebulised salbutamol 2.5mg ± ipratropium bromide 500µg therapy for 7 days using the AKITA^® Jet Inhalation System (Vectura^®, UK). All 14 cases were identified correctly by COPDPredict™ with “high risk” notification alerts displayed against the respective patients on the decision support dashboard.

Hospitalisations

Only 4 of the total 80 exacerbators attended the emergency department, after having already started on their appropriate rescue medication at home: 2 attendees required chest radiograph and ECG tests to exclude pneumonia and a cardiac event, respectively, and were discharged home upon exclusion; the other 2 attendees were admitted for stabilisation of their known respiratory failure and stayed for less than 72 hours. All 4 patients resumed with COPDPredict™ following discharge from emergency room/hospital.

Compared to the total number of in-patient hospitalisations for all patients in the 6 months pre-COPDPredict™, hospitalisations during the 6-month period with COPDPredict™ were reduced by 98% (90 vs 2, p<0.001). All patients were followed up for 30 days following treatment for an index exacerbation; none of the patients who were treated at home or who had attended the emergency room/been hospitalised needed admission or readmission.

Validity of the COPDPredict™ Exacerbation Prediction Algorithm

Collected data from 3257 time-points across the study period allowed validation of the prediction algorithms. Cross-tabulation analysis between “algorithm-defined” and clinician-defined exacerbations demonstrated that COPDPredict™ identified COPD-related exacerbations at 7, 3 days (median, IQR) prior to clinician-defined episodes, sending appropriate alerts to patients and clinicians.

Figure 3 demonstrates the breakdown for the entire exacerbation prediction algorithm. COPDPredict™ advised on the prodromal status of an impending exacerbation in 285 out of 291 timepoints demonstrating a sensitivity of 97.9% (95% CI: 95.7–99.2), specificity of 84.0% (95% CI: 82.6–85.3), with a positive predictive value of 38.4% (95% CI: 36.4–40.4) and a negative predictive value of 99.8% (95% CI: 99.5–99.9). The accuracy for the prediction of an exacerbation was 85.3% (95% CI: 84.0–86.5); a positive likelihood ratio of 6.12 (95% CI: 5.61–6.66) and a negative likelihood ratio of 0.02 (95% CI: 0.01–0.05). Taken together, the high sensitivity and specificity coupled with the high negative predictive value means that COPDPredict™ will not miss an imminent exacerbation or fail to alert the patient and healthcare professional of that pending exacerbation.

Figure 3 Sensitivity and specificity of the exacerbation prediction algorithm. Notes: AECOPD: acute exacerbation of chronic obstructive pulmonary disease, *failure to enter a wellbeing score or perform spirometry; **AECOPD during 2-week baseline phase. The sensitivity and specificity percentages are highlighted in bold. This image is the property of the author.

In contrast, analysis of algorithm outputs utilising only Wellbeing Score and FEV₁ demonstrated that the prediction algorithm advised on imminent exacerbations in only 224 out of 291 timepoints. This reflected a lower sensitivity of 77.0% (95% CI: 71.7–81.7), a specificity of 65.0% (95% CI: 63.2–66.8), and positive and negative predictive value of 18.3% (95% CI: 17.1–19.5) and 96.5% (95% CI: 95.7–97.2), respectively. The accuracy for exacerbation prediction was 66.1% (95% CI: 64.4–67.8) with a positive likelihood ratio of 2.2 (95% CI: 2.03–2.38) and negative likelihood ratio of 0.35 (95% CI: 0.29–0.44). These results support our approach of combining patient-reported outcomes with bio-physiological metrics for optimal precision of the exacerbation prediction algorithms.

Discussion

In this study, we demonstrated that COPDPredict™ can enable safe, reliable continuous self-monitoring and provide early detection of exacerbations in a high-risk COPD population. We have presented a unique bio-clinical profiling approach for the acquisition of patient-specific critical datasets over time, allowing changes in health status (eg, stable to prodrome to exacerbation to recovery) to be determined on a personalised basis and for exacerbation prediction and severity stratification to be clinically meaningful. Combining patient-reported outcomes with objective bio-physiological parameters contributes to the robustness of COPDPredict™’s prediction algorithms, reflected in high sensitivity (98%) and specificity (84%) plus high negative predictive value (99.8%). Accurate exacerbation prediction will likely become a benchmark for home monitoring technologies as the demand for precise personalised care increases: “Developing predictive algorithms with clinically useful levels of sensitivity and specificity is a priority for future development of telemonitoring of COPD” as non-intuitive systems can produce unnecessary alerts causing stress and anxiety in patients.³²

Exacerbations and consequent hospitalisations are outcomes that COPD patients’ rate as most important to them.³³ Current guidelines^18,26,34 recommend an exacerbation action plan based on patient-reported symptoms for all COPD patients. However, our results suggest that action plans supported by digital technologies with embedded exacerbation prediction would improve exacerbation management and patient outcomes. Indeed, many patients are unable to differentiate between “usual” variations in COPD health and discrete increased symptoms, and delay or fail to respond to impending exacerbations adequately.^13,14,35 It follows that at a clinical service level, COPDPredict™ could enhance home-monitoring and optimize self-management for patients, whilst allowing clinicians to review their patients’ progress and initiate tailored intervention and support promptly. Our study demonstrated that COPDPredict™ identified COPD-related exacerbations at 7, 3 days (median, IQR) prior to clinician-defined episodes, sending appropriate alerts to both patients and clinicians. Specifically, the COPDPredict™ alerts provided sufficient time for instigating appropriate treatment at home in most patients, avoiding unnecessary hospital admissions and use of stretched hospital-based resources. Whilst outside this study’s objectives, a preliminary economic impact evaluation suggests that COPDPredict™ could alleviate the burden on stretched hospital-based resources and reduce current costs associated with exacerbation-related hospitalisations and in-patient bed days.

Remote monitoring interventions have been reported as presenting some usability issues in the real-world.³⁶ Our study showed a 98% compliance for patient completion of daily wellbeing self-assessments; similarly, patients complied with performing spirometry and reacted responsibly to notifications. This sustained compliance and positive interaction with the App over the 6 months are likely due to a number of factors: (1) Design and development of COPDPredict™ was driven from a patient’s perspective; patients were intentionally involved throughout the production phases as “experts by experience”;³⁷ (2) Patients were reminded to submit assessments via automatic prompts; (3) Patients were reassured by the interactive functionality of the system: a) their health status could be reviewed by the clinical team in real-time and at any time; b) any changes in health prompted automatic alerts to them and their clinicians; and c) they could communicate directly with clinicians using the App’s messaging facility; patients found this to be most useful for instant clarifications on treatments and health-related queries. Indeed, multicomponent interventions with action plans for exacerbation management are effective in reducing admissions only when they offer iterative feedback to patients.³⁸

Measurement of CRP was a vital input for the robustness of the exacerbation prediction algorithm, influencing its sensitivity and specificity. C-reactive protein testing is practically achievable within the home and community (GP or pharmacy-based), avoiding the need for patients to attend hospital centres for blood tests. Furthermore, point-of-care CRP testing can reduce unnecessary antibiotic prescribing for respiratory tract infections in exacerbations.^39,40

Strengths and Limitations

We are aware of published work using symptoms and physiological parameters for COPD exacerbation prediction,^22,23,41,42 but to the best of our knowledge COPDPredict™ is the first validated digital application offering an exacerbation prediction algorithm driven by a combination of patient-report outcomes (wellbeing scores) and reliable bio-physiological inputs (FEV₁ and CRP levels). A key feature of the algorithm decision tree is the creation of an individual’s stable baseline which accommodates for intra- and inter-person variability in an extremely heterogeneous clinical condition. This improves on prediction tools that rely on fixed thresholds or manual tuning by healthcare professionals.²¹ Our approach also allows for each patient’s “stable state” baseline to be reset as appropriate throughout the possible changing dynamics of their chronic condition. This aspect of COPDPredict™ has not been fully tested within the time limits of the current study and will require further evaluation.

In-depth analyses of the prediction alert data show that COPDPredict™ could identify those patients likely to re-exacerbate within 30 days of an index exacerbation. Preliminary findings indicate that “re-exacerbating” patients had higher wellbeing scores and CRP levels and lowest FEV₁ at the time of exacerbation. This observation requires further evaluation over a longer study period and in a larger patient cohort; this is currently in progress (NCT04136418). If confirmed, COPDPredict™ would additionally predict those patients likely to re-exacerbate and/or be readmitted, prompting higher levels of intervention and/or longer treatment periods.

We have selectively biased towards a COPD frequent exacerbator cohort as our objective was to validate COPDPredict™ as an effective tool for predicting exacerbations. In practice, these patients are frequently hospitalised and have the poorest outcomes. However, COPDPredict™ is not precluded from being used in all COPD patients to enhance their self-management and prevent them from becoming frequent exacerbators.

Conclusion

We have demonstrated COPDPredict™ is safe and can provide a model of care based around prevention, combing remote monitoring and patient-specific exacerbation prediction. The prediction algorithm reliably and promptly informs both patients and clinicians on acute events, providing an opportunity to intervene and reducing unnecessary hospitalisation. We anticipate that as COPDPredict™ accumulates individual-specific datasets over time, enabled machine learning will identify patterns in exacerbation etiology and treatment responses to improve prediction of exacerbations and individual outcomes.