A Predictive Machine Learning Tool for Asthma Exacerbations: Results from a 12-Week, Open-Label Study Using an Electronic Multi-Dose Dry Powder Inhaler with Integrated Sensors

Plain Language Summary

Why was the study done?

An asthma attack is when a person’s asthma symptoms get worse over a short space of time. This study was done to see if we can use information from a digital inhaler to predict when someone may have an asthma attack before it happens. If we can use technology to predict an asthma attack by using inhaler data, then doctors and patients may be able to help to stop the attack from being too bad or prevent it from happening at all.

What did the researchers do and find?

The Digihaler^® is an inhaler that has an electronic sensor built into it. The inhaler can record when a person uses it. It can also record how much air a person can breathe in, and how long and how fast they can breathe in for. These inhalers were used by the patients in this study. Using a computer programme and information from the inhalers, we developed a way of calculating how well we could predict if a patient would have an asthma attack within the next 5 days. Our results showed that our calculation was very good at predicting if a patient would have an asthma attack within the next 5 days. We also found that the number of times a patient used their inhaler in the few days before an attack was important for making a prediction.

What do these results mean?

An inhaler that can record how well a person breathes in can provide important information about that person’s asthma. This information can also help to predict if a person may have an asthma attack. Being able to do this may help doctors and patients to stop asthma attacks from happening.

Introduction

Correct use of asthma medications is important to minimize symptoms and avoid exacerbations.^1,2 Inhalation therapies for asthma are reliant on correct inhaler technique for optimal deposition of medication.^3–5 For dry powder inhalers (DPIs), appropriate technique involves performing a deep and forceful inhalation in order to adequately de-aggregate and disperse the inhaled particles for delivery to the lungs.^6–8 A well-established measure of inhalation technique is peak inspiratory flow (PIF),⁸ which is an important driver of de-agglomeration of particles inside DPIs, and is shown to decline with hyperinflation and airway disease exacerbations.⁹

ProAir^® Digihaler^® (Teva Pharmaceuticals, Israel) is a US Food and Drug Administration approved electronic multi-dose DPI. The Digihaler incorporates integrated sensors that detect device actuation (opening of the cap, which prepares a dose) and inhalation parameters such as PIF, inhalation duration, inhalation volume, and time to PIF.¹⁰ In a study of 150 participants, the Digihaler was shown to provide accurate measurements of inhalation parameters when used by patients.¹¹ The goal of measuring these parameters was to provide information about real-life inhaler technique and information about short-acting beta₂-agonist (SABA) use frequency, which might be useful to healthcare providers (HCPs) and patients.

It was hypothesized that data from the Digihaler could be used to detect deterioration in a patient’s clinical status prior to an asthma exacerbation based on changes in inhalation parameters and albuterol use. The period of an incipient exacerbation (early loss of control) is considered the “yellow zone” in asthma action plans. Asthma action plans have been used for many years to empower patients to detect early declines in lung function coupled with increased asthma symptoms and are part of standard guideline-based asthma care.¹² Identifying the “yellow zone” provides increased awareness of deterioration and, when combined with early treatment adjustments, can pre-empt more severe exacerbations.^13,14 However, asthma action plans and “yellow zone” identification have not been integrated into practice as widely as possible and optimizing them would represent an advance in asthma treatment.¹⁵

Used for a variety of clinical applications,^16–20 machine learning entails the use of computer algorithms to develop a model capable of making accurate predictions based on data inputs.²¹ One machine learning technique is supervised learning, in which the model is “trained” to classify data using input data with corresponding labels (or outcomes), determining if any relationships exist between the input data and the label. After the model has been trained, it will attempt to classify new data based on the previous training data.²² Decision trees, which hierarchically classify data inputs according to pre-specified characteristics, are a well-established approach in computational biology and have proven amenable to adaptation into machine learning algorithms.²³ Gradient-boosting is a supervised machine learning technique that utilizes decision trees to combine and optimize diverse data inputs in an iterative process that seeks to maximize the accuracy of a probabilistic model.²⁴

A number of previous studies have attempted to develop models to predict a patient’s future risk of exacerbations.^25–27 Most existing models currently predict the overall risk of developing exacerbations within a certain time frame (usually 1 month or 1 year) based on various combinations of clinical and billing data but are unable to specifically predict imminent events.^26,27 A model taking into account day-to-day variations in disease activity based on baseline SABA use, inhalation parameters, and symptomatology may allow for more timely identification of potential deteriorations in asthma control and enable pre-emptive intervention in time to prevent a severe exacerbation.

Our aim was to determine if it was feasible to use data from ProAir Digihaler, along with baseline demographic information and clinical data, to develop a machine learning model capable of predicting impending asthma exacerbations as an initial step in developing a more robust system.

Materials and Methods

Study Design and Participants

This 12-week, open-label study was conducted across 45 study centers in the USA between February 2017 and February 2018. The study consisted of a 2-week screening period and a 12-week intervention period (NCT02969408). During the baseline visit (study Day 1), patients were trained on correct inhaler technique. A schematic design of the study is described in Figure 1. Details of sample size selection can be found in the Supplementary Methods.

Figure 1 A schematic overview of the study.

The study population comprised adult patients with a physician diagnosis of asthma, at least one moderate or severe asthma exacerbation over the 12 months prior to screening, and poorly controlled asthma as defined by an Asthma Control Questionnaire-5 (ACQ-5) score of ≥1.5.²⁸ For this study, exacerbations were defined according to the 2009 American Thoracic Society/European Respiratory Society recommendation.²⁹ Moderate exacerbations were those involving worsening asthma and requiring administration of systemic corticosteroids (SCS) above baseline for at least 3 days, or an unscheduled HCP visit (eg, doctor’s office or emergency care) associated with an increase in asthma therapy. Severe exacerbations were those requiring both administration of SCS as above and an unscheduled HCP visit. All patients were required to be on moderate or high doses of inhaled corticosteroids, equivalent to at least 440 µg daily of fluticasone propionate, with or without other asthma maintenance medications (long-acting beta₂-agonist, leukotriene antagonist, long-acting antimuscarinic agent, biologic, or maintenance oral corticosteroids). Patients were excluded if they had any confounding underlying lung disorder other than asthma or had used any investigational drugs within five half-lives of discontinuation.

Patients were required to discontinue all other reliever medication containing SABA or short-acting antimuscarinic agents and replace them with the Digihaler (albuterol, 90 µg/dose) as their reliever medication (1–2 inhalations every 4 hours, as needed) for the duration of the study, alongside their usual maintenance therapy. Patients were supplied with seven Digihaler devices: three were provided on Day 1 and a subsequent four additional devices were provided by courier on Day 21. The Digihaler devices contain sensors within an integrated electronic module that recorded a timestamp with each use, along with PIF, inhalation volume, time to PIF, and inhalation duration. These data were stored within the electronic module in each device and were downloaded directly from the devices at the end of the patients’ treatment period by appropriately designated and trained personnel using extraction software. A companion Digihaler smart phone application (app) is currently available; however, this was not used during this study due to the potential influence on patient behavior. Future studies will assess the usability and benefit of the app in terms of inhaler usage and technique.

As per informed consent, patients were aware of the recording of measurements by the Digihaler. Patients were contacted monthly by phone for collection of information regarding exacerbations, maintenance medication, and adverse events. Instructions for use were also discussed during the monthly call.

Throughout the study period, asthma maintenance medication was continued or altered as per the treating physician’s judgement. Although the Digihaler recorded measurements in real-time, data were only downloaded at the end and were not available during the study; therefore, any patient treatment modifications during the study were based on patient report and not Digihaler data. The use of nebulized albuterol was permitted for the treatment of acute exacerbations at home or at the hospital only, if deemed necessary by the patient or their physician. Additionally, patients could continue use of other medications, with any modifications at the discretion of their treating physician.

Predictive Model

Enrolled patients who completed the study with at least one valid inhalation using the Digihaler were eligible for inclusion in the predictive analysis dataset. A valid inhalation was defined as an inhaler event with a detected PIF ≤120 L/min and no errors in use. Patients who experienced an exacerbation during the first 10 days of the study or made no inhalation during the 4-day period preceding an exacerbation were excluded from the predictive analysis dataset.

The target of the predictive model was defined as the prediction of an exacerbation within the following 5 days. The selection of five days for the predictive window was based on balancing the accuracy of the prediction against having adequate time for treatment intervention. A 3-day model would have been slightly more accurate but would restrict time to intervene before the predicted exacerbation, whereas a 7-day model would allow more time for intervention but would likely be less accurate.

The primary measure for the predictive analysis was albuterol use, and parameters of interest included total number of inhalations in the days preceding an exacerbation peak (defined as the day on which the patient began using SCS), the number of days prior to exacerbation peak during which albuterol use increased, and the amount of albuterol use in the 24 hours prior to a moderate or severe exacerbation.

The model was used each day to predict whether the patient would have an exacerbation within the following 5 days. Predictions where an exacerbation was anticipated to occur within the following 5 days were described as positive predictions and negative predictions were those where no exacerbation was anticipated to occur within the following 5 days. Each day’s prediction was based on features input into the model, including comparisons of data on numbers of inhalations and inhalation parameters during the preceding days (“days prior to prediction”) with the baseline features (Figure 2). Data on actual ACQ-5 scores and lung function, which could have been important additional variables, were not collected during the study. Respiratory symptoms were not collected after enrollment and thus were not used in the predictive model.

Figure 2 A patient example showing prediction periods and confirmed exacerbation in relation to albuterol use between study days 55 and 85, and albuterol use, peak inspiratory flow, and inhalation volume for the same patient over the full study period. Patient was a male (43 years of age) with a body mass index of 31.9 kg/m2. The red vertical dashed line represents a confirmed exacerbation. Notes: *Predictions were made on every study day. Predictions where an exacerbation was anticipated to occur within the following 5 days were described as positive predictions and negative predictions were those where no exacerbation was anticipated to occur within the following 5 days.

To develop the predictive model, machine learning techniques were applied to a combination of case report form data taken on study Day 1 (age, body mass index, blood pressure, previous exacerbations, and the number of exacerbations and hospitalizations in the previous 12 months), data from the Digihaler prior to (and including) the day of the prediction, and patient baseline characteristics from the Digihaler (timestamp of inhalation, inhalation status, PIF, inhalation volume, time to PIF, and inhalation duration). The number of inhalations and mean (standard deviation [SD]) of each inhalation parameter during the first 10 days of the study were considered as baseline features for the predictive model. A feature engineering process was conducted to determine the most relevant features for the model.

As the goal of the model was to predict cases of impending exacerbations rather than the probability of an exacerbation occurring, gradient-boosting trees²⁴ were identified as the most appropriate algorithm to be implemented in the predictive model. Specifically, the XG-Boost³⁰ implementation of gradient-boosting – which utilizes a tree learning algorithm optimized for the handling of sparse data to iteratively combine trees and thereby optimize the predictive model – displayed the strongest performance on the test set and was subsequently evaluated on the validation set.

Following feature engineering to convert the data inputs into explanatory variables structured suitably for predictive modeling, several supervised machine learning algorithms, including logistic regression, random forest, and gradient-boosting trees, were applied. Patients were randomly divided into three groups to train the model (“training set”), test and optimize the model (“test set”), and validate the chosen model (“validation set”). A 4-fold cross validation technique was used to compare the predictive performance metrics of the algorithms. Patients in the training set were randomly partitioned into four mutually exclusive and collectively exhaustive subsets, and the algorithm was trained in four distinct repeats. In each repeat, the algorithm was trained on data from three of the subsets of patients. The predictive performance of the algorithms was evaluated using the receiver operating characteristics (ROC) curve of sensitivity versus specificity. The ROC area under the curve (AUC) value represents the capability of the model to separate between classes. Values for ROC AUC fall between 0 and 1, with 1 representing perfect performance of the model.³¹ The ROC AUC was computed separately for each group and then averaged over the group to provide a single quality measure for the model. The relevance of the features used for the model is given as a percentage. This percentage relates to the variance reduced by data splits in the feature engineering that used this variable among all trees of the model. A large percentage variance reduction when data are split indicates that the feature has a large amount of relevance to the model and, therefore, contributes to the model performance.

Statistical analyses and algorithm development were performed using R Statistical Software (version 3.6.1; R Foundation for Statistical Computing, Vienna, Austria), on a Windows operating system. Descriptive statistics were used to report demographics and outcome measures, but no between-group statistical comparisons were made.

Ethics

This study was conducted in full accordance with the Declaration of Helsinki and International Conference on Harmonisation guidelines for Good Clinical Practice. Written informed consent was obtained from each patient before study participation. All study documents and procedures were approved by Schulman Institutional Review Board (IRB #201606556). The choice of ethics committee was the decision of the clinical research organization who conducted the study.

Results

Study Populations

Overall, 449 patients with asthma were screened and 397 patients were enrolled (intention-to-treat population). Of these, 16 (4%) patients were excluded due to early termination and 21 (5%) were excluded because they did not make at least one valid inhalation using the Digihaler. Patients who completed the study with at least one valid inhalation using the Digihaler (n=360 [91%]) were eligible for inclusion in the predictive analysis dataset. Excluded from the predictive analysis dataset were patients who experienced an exacerbation during the first 10 days of the study (n=6), and patients who made no inhalations from the Digihaler during this period (n=47), during the period after the first 10 days (n=2) or during the 4-day period preceding an exacerbation (n=7). The predictive analysis population, therefore, comprised 298 patients (Figure 3).

Figure 3 Patient disposition. Abbreviation: ITT, intention-to-treat.

Baseline demographics and maintenance medication use of the predictive analysis population are shown in Table 1; 81.2% of patients were female and the mean age was 50.5 years. Patients experienced a mean of 1.5 exacerbations in the 12 months prior to the study. The estimated annualized exacerbation rate for patients during the study was 0.91 exacerbations per patient per year. Baseline demographics and maintenance medication use of the intention-to-treat population are shown in Supplementary Table 1.

Table 1 Baseline Demographics and Maintenance Medication Use

Inhalation Parameters: Predictive Analysis Dataset

Of the 360 patients who made ≥1 valid inhalation and completed the study, and so were eligible for inclusion in the predictive analysis, 64 (18%) experienced a total of 78 moderate/severe exacerbations. Over the full study period, the mean PIF (SD) for all analysis eligible patients was 73.2 L/min (21.0 L/min) (Table 2). For patients with and without exacerbations, the mean PIF (SD) was 73.4 L/min (23.1 L/min) and 73.2 L/min (20.3 L/min), respectively. The mean inhalation volume, inhalation duration and time to PIF were similar in all patients, patients with exacerbations and patients without exacerbations. Among non-exacerbators, the mean number of daily albuterol inhalations over the study duration was 1.14 (SD 2.35). For exacerbators, the mean number of daily albuterol inhalations was 1.87 (SD 2.78) outside of the ±14-day window around an exacerbation and 2.43 (SD 3.67) during the exacerbation window. During the exacerbation window, the mean daily albuterol use increased in the days leading to the exacerbation peak and decreased in the days following (Figure 4). Analysis of percentage changes from baseline showed that PIF and inhalation volume decreased in the days prior to the exacerbation peak and increased after (Figure 5). A representative example of a patient who experienced an exacerbation is shown in Figure 2. This particular patient had an exacerbation on Day 76 and this figure demonstrates the predictive periods leading up to this exacerbation and the changes in inhalation parameters before and after the exacerbation.

Table 2 Inhalation Parameters and Albuterol Use Over the Full Study Period Captured by the Digihaler in Patients Who Completed the Study with ≥1 valid Inhalation

Figure 4 Albuterol use preceding a patient’s first asthma exacerbation during the study. Data are shown for patients who completed the study and made ≥1 valid inhalation from the Digihaler. The vertical dotted line represents a confirmed exacerbation.

Figure 5 Percentage change from baseline in peak inspiratory flow and inhalation volume during ± 7-day window around an exacerbation, compared with the daily mean outside of this exacerbation window, in 52 patients with one exacerbation. Data are shown for patients who completed the study with ≥1 valid inhalation from the Digihaler, experienced one exacerbation, and made ≥1 valid inhalation outside of the ± 7-day window around the exacerbation. The vertical dotted line represents a confirmed exacerbation. Abbreviation: PIF, peak inspiratory flow.

Predictive Factors and Model Validation

The training set comprised 184 patients, the test set comprised 60 patients and the validation set comprised 54 patients. In the validation set, the gradient-boosting model predicted an exacerbation within the following 5 days with an ROC AUC value of 0.83 (95% confidence interval: 0.77–0.90) (Figure 6). Of the features included in the predictive model, the strongest predictive factors for a future exacerbation were the mean number of daily albuterol inhalations during the 4 days prior to the prediction (47%), the inhalation parameters in the four days prior to prediction (PIF, inhalation volume, and inhalation duration; 11%), and comparison to the baseline values for these inhalation parameters (14%; Figure 7). Other features were also input to the model but were not found to contribute to the model performance.

Figure 6 Receiver operating characteristic curve of sensitivity and specificity of a model for predicting upcoming exacerbations within 5 days. Selected risk percentiles shown in table are based on model score, eg, the sensitivity and specificity values for risk percentile 1 are those for the 1% of data points with the highest predicted probability for an exacerbation within 5 days.

Figure 7 Features contributing to predictive model performance. The 4 days prior to prediction refers to the 4 days before the prediction was made and not the days prior to an exacerbation. For the predictive model, baseline features (number of inhalations and inhalation parameters) were those measured during the first 10 days of the study. Features shown are those that were found to contribute to the model performance. Other features were also input to the model but were not found to contribute.

Discussion

Data from adult patients with poorly controlled asthma were obtained via the use of a reliever medication inhaler device equipped with an integrated sensor and, along with demographic and clinical information, were used to develop a predictive model for exacerbations within 5 days. To our knowledge, this represents the first successful attempt to identify clinical deterioration of asthma using data from a reliever medication inhaler device with integrated sensors.

Despite available therapies, many patients report daily symptoms associated with partly- or uncontrolled asthma per the Global Initiative for Asthma 2020 report.¹ Respiratory symptoms in patients with asthma are known to be predictive of exacerbations³² and patients with poorly controlled asthma have an increased risk of exacerbations, poorer outcomes and reduced quality of life.¹ Similarly, it is well known that increased SABA use is an indicator of poor asthma control and exacerbation risk,^33,34 and is widespread among patients with poorly controlled asthma.³⁵ These patients often underestimate their symptom control,^36,37 which may be one barrier to the identification of early deterioration and the implementation of asthma action plans that depend on this recognition.³⁸

The Digihaler pressure sensor was originally designed with the intention of facilitating correct DPI technique and recording medication use. When using DPIs, the delivery of the inhaled particles to the lungs relies on generation of an adequate inspiratory flow to deagglomerate the powder into an emitted dose. The time taken to achieve PIF, duration of inhalation, and inhaled volume are also important considerations³⁹ as a rapid and deep inhalation has been shown to be the most appropriate for using DPIs.⁶ Consequently, measurement of these inhalation parameters can give important information regarding inhaler technique. Furthermore, PIF has been shown to deteriorate with hyperinflation around the time of an exacerbation when assessed in relation to changes in other inhalation parameters.^9,40 Changes in PIF patterns may indicate an impending exacerbation, particularly if identified in combination with other signals such as changes in albuterol use.

Sensor-derived data enabled the observation of SABA use and inhalation characteristics of poorly controlled asthma patients who did and did not experience an exacerbation over the 12 weeks of the study. Despite high baseline SABA use, which was apparent across the study population, increases in daily inhalations during the days preceding an exacerbation were evident. Consistent with previously reported observations, PIF, inhalation volume and inhalation duration were all observed to decrease in the days prior to an exacerbation. Together, these observations suggest that exacerbations are preceded by predictable changes in patient behaviors and inhalation attributes, and that it may be feasible to pre-empt more severe exacerbations through intervention upon detection of these changes.

Achieving a shift towards personalized medicine in asthma will necessitate accurate measurement of physiologic parameters, combined with powerful diagnostic tools that take into account individual variability.⁴¹ Machine learning algorithms designed to classify and optimize large volumes of heterogeneous data are ideally placed to contribute to this shift,⁴² and the adaptive nature of these algorithms is key for personalized predictive modeling.⁴³ Gradient-boosting²⁴ is well-suited for this task; it offers classification and structured prediction approaches with a high degree of flexibility.^31,44

By combining Digihaler data with patient information and applying the gradient-boosting trees algorithm, we were able to develop a model predictive of an impending exacerbation within the following 5 days with an ROC AUC value of 0.83. The most important predictive feature was found to be the mean number of daily inhalations during the 4 days prior to the day the prediction was made. It is, however, important to note the breadth of other features which were utilized in the model and contributed to its overall power. These reflect the highly personalized nature of the exacerbation progression within each patient and their patterns of inhalation parameters. Multiple factors may be predictive of asthma exacerbation risk, including clinical, behavioral, and social factors. When developing this model, a feature engineering process was conducted to determine which features would be most relevant for the model. For this data set, baseline systolic blood pressure was found to add accuracy to the model. It is possible that this would not be the case with a different data set, but the inclusion of systolic blood pressure here highlights the importance of predictive modelling. With predictive modelling, trends may become apparent that are counter-intuitive or unexpected. Lung function and ACQ-5 score data, along with respiratory symptoms data and data on other physiologic measures such as heart rate and respiratory rate, were not used in this model and may have been important additional variables, if available. Future research that builds on this study will help to further understand the role these factors play in predicting asthma exacerbations.

This study represents early progress in the development of a predictive model based on data from a digital inhaler system. As this model is further developed and improved, additional work needs to explore how the predictive capabilities might best be optimized for a clinical setting. The use of a single universal risk threshold to guide decision-making is commonplace in clinical practice,¹⁶ whereas a predictive model has the potential to provide a personalized, dynamic measure of risk that is informed by day-by-day changes in parameters as well as patient-specific characteristics.¹⁶ In practice, however, it is anticipated that implementation will likely entail the prior selection of one or more risk thresholds, balancing sensitivity with specificity as deemed appropriate for the specific use-case.⁴² The algorithm offers the flexibility to support this. The relative impacts of false positives (which could result in inefficient use of healthcare resources) and false negatives (which could result in missed opportunities to intervene to prevent deteriorations) will depend on exactly how this or similar models are implemented.

The study had several limitations. Digihaler data were only downloaded and analyzed at the end of the study, and exacerbations were identified by monthly patient phone calls, potentially resulting in recall bias and unidentified exacerbations, which may have affected the precision of the model. Minor technical issues with the Digihaler may also have affected model precision, mostly as a result of timestamp errors from a later download of the data instead of real-time synchronization with a smart phone app. The digital inhaler system, designed with the capability to transmit data from the sensor to a smart phone app and/or internet dashboard might support real-time or near real-time identification of an exacerbation by a patient or HCP. Additionally, although the protocol discouraged nebulizer use, it is impossible to prohibit its use by all patients, even outside of an exacerbation, and this possible use was not recorded. Therefore, reliever medication use outside of the Digihaler data was not accounted for in the model. Lastly, there was an imbalance in the population demographic, with 80% of enrolled patients being female. Studies have shown that females are more likely to experience an asthma exacerbation⁴⁵ and have lower PIF values on average than are typical in males.⁴⁶ This imbalance in demographic could have had an effect on the overall findings of the study and the influence of sex in the model requires further exploration.

Conclusion

Our findings highlight the potential value of a model predictive of impending asthma exacerbations based on data from the Digihaler. Individualizing treatment regimens is key to optimal asthma management. This initial model represents the first step in refining the algorithm predictive of exacerbations and will inform future development and validation of predictive models from inhaler use data in respiratory care.