Predicting the Risk of Incorrect Inhalation Technique in Patients with Chronic Airway Diseases by a New Predictive Nomogram

Introduction

The burden of chronic airway diseases, such as asthma and chronic obstructive pulmonary disease (COPD), is high and has become a major public health concern worldwide.^1,2 For these patients, inhalation medication is the basic treatment to be considered as its low dose, few side effects and direct drug delivery into the lung.³ Despite a variety of inhalation devices are widely used in patients with chronic airway diseases, such as pressurized metered-dose inhalers (pMDIs), dry powder inhalers (DPIs), and soft mist inhalers (SMIs), incorrect use of inhalation devices is common. Poor inhalation technique can adversely affect clinical efficacy,^4,5 which may compromise drug delivery, resulting in poor outcomes over time including increased risk of hospitalization, additional medical cost and mortality.⁶ A randomized controlled trial showed that COPD patients with incorrect device use had significantly worse forced expiratory volume in the first second in percent predicted values (FEV1%) at baseline and were more likely to experience cough and breathlessness than patients with correct device use.⁷ Furthermore, regular instruction and assessment of inhalation techniques are considered to be essential components of the successful management in chronic airway diseases.⁸

The incorrect use of inhalation devices is influenced by multiple factors, such as patient-related factors (eg, age, gender, or education level) and medicine-related factors (eg, type of inhalation device, duration of drug use, or inhalation device use instruction).^9,10 Given so many associated risk factors, an accurate predictive tool to detect such populations and provide early interventions may be the simplest and most effective action against the incorrect use of inhalation devices. A predictive nomogram may make a difference for patients with chronic airway diseases who present improper inhalation techniques.

Thus, this study aimed to develop a valid but simple prediction tool to assess the risk of incorrect inhalation techniques in patients with chronic airway diseases, which might help healthcare workers to rapidly screen and early intervene in patients with high-risk factors during follow-up visits.

Methods

Patients

Patients with chronic airway diseases such as asthma and COPD, treated with inhalation drugs at the outpatient department of respiratory medicine of the First Affiliated Hospital of Guangzhou Medical University from August 2021 to March 2022, were recruited in our study. And all patients were older than 14 years.

The investigation conformed with the principles outlined in the Declaration of Helsinki and our study was approved by the Medical Ethics Committee of the First Affiliated Hospital of Guangzhou Medical University (2021 No. 93). Written informed consent was obtained by each participant. And patients under the age of 18 years were obtained informed consent from their legal guardian. This study is registered at Chictr.org with identifier number ChiCTR2200056579.

Data Collection

Data such as demographic characteristics, medical history and variables associated with inhalation treatment were collected from questionnaires and medical records, while the results of pulmonary function testing were collected from checklists. Besides, patients’ mastery of inhalation device was assessed by medical professionals.

The variables consisted of the following parts:

1. Demographic characteristics and medical history including gender, age, body mass index (BMI), marital status, and education level (low: junior high school or below, middle: senior high school or junior college, high: bachelor or higher), income level, whether patients settled by medical insurance, dust exposure history, smoking history, family history, and whether patients experienced an exacerbation in the previous 4 weeks.
2. Questions related to inhalation drug including the type of inhalation drug patients were treated [single bronchodilator like inhaled corticosteroid(ICS), long-acting β₂ agonist (LABA), long-acting muscarinic antagonist (LAMA), double bronchodilators like ICS/LAMA, ICS/LABA, LAMA/LABA, triple bronchodilators like ICS/LAMA/LABA, etc], duration of drug use, patients’ cognition of drugs, knowledge of inhalation drugs and self-evaluation of curative effect.
3. Questions related to inhalation device including the type of inhalation device (details are given in Table 1), whether patients had received use instruction of inhalation device before treatment, whether to evaluate after instruction and at return visit, whether patients had recently replaced the inhalation device.
4. Mastery of inhalation techniques were assessed by a scale shown in Table 2 (mastered correctly: 8–10, mastered incorrectly: <8).
5. Medication adherence was divided into low adherence, medium adherence and high adherence.
6. Other questions such as trust in health care workers, family medication supervision, evaluation of pulmonary function before treatment, pulmonary function results, etc.

Table 1 Proportion of Different Inhalation Devices

Table 2 Assessment Scale for Mastery of Inhalation Techniques

Statistical Methods

All the categorical variables were expressed as count (%). Differences between categorical variables were evaluated with Chi-Square test. A two-tailed P value less than 0.05 was considered statistically significant. The optimal features for predicting incorrect inhalation techniques were screened out using the least absolute shrinkage and selection operator (LASSO) method.^11,12 Features with nonzero coefficients in the LASSO regression model were selected. A predicting model was constructed using multivariate logistic regression analysis by combining the selected features, which were considered odds ratios (ORs) having 95% confidence intervals (95% CIs). All tests were two-sided, and a P value less than 0.05 was considered statistically significant. Except the variables associated with inhalation treatment, demographics and clinical characteristics were also included in the model. All potential predictors were applied to develop a model to predict the risk of incorrect inhalation techniques by using the cohort.

The calibration curve was drawn to evaluate the calibration of the nomogram for error inhalation technique risk. A test for statistical significance indicated that the model was not perfectly calibrated. Furthermore, Harrell’s C-index and area under the receiver operating characteristic curve (AUC) were calculated in order to quantify the performance of the nomogram in identifying the risk of incorrect inhalation techniques.¹³ To determine the clinical validity of the nomogram by quantifying the net benefits at different threshold probabilities in the error inhalation technique cohort, decision curve analysis (DCA) was performed in our study.^13,14 The net benefit was measured by subtracting the proportion of all false-positive patients from the proportion of true-positive patients and weighing the relative harm of forgoing interventions against the negative consequences of an unnecessary intervention. All statistical analyses were done with IBM SPSS Statistics Version 20.0 and R software Version 3.6.2 (https://www.R-project.org).

Results

Patients’ Characteristics

In total, 206 patients from August 2021 to March 2022 were included in our study, and the cohort consisted of 129 patients who were able to use the inhalation devices correctly (49 females and 80 males; mean age 52.52±15.53 years) and 77 patients who used the device incorrectly (27 females and 50 males; mean age 59.74±15.31 years). It was to say, approximately 37% (77/206) of the patients performed incorrect use of inhalation devices in our study. The categorical variables of the two groups including demographic characteristics and medical history are presented in Table 3.

Table 3 Categorical Variables for Demographic Characteristics and Medical History Between the Correct and Incorrect Groups

The most frequently prescribed inhalation medications were ICS/LABA (61%), followed by ICS/LAMA/LABA (19%). The most commonly used inhalation devices were DPIs, in 73% of the patients, either single or dual. Unfortunately, our study showed that 52% of the patients had no sense of drug cognition and inhalation medication knowledge, and 9% were absence of received inhalation instruction before treatment. Despite 91% of the patients received inhalation instruction before treatment, only 43% were evaluated after instruction and 17% at return visit. The result of medication adherence showed that merely 16% of the patients had high adherence, 40% had medium adherence, and 44% had low adherence. The categorical variables associated with inhalation treatment are described in Table 4.

Table 4 Categorical Variables Associated with Inhalation Treatment Between the Correct and Incorrect Groups

Development of an Individualized Model for Predicting the Risk of Incorrect Inhalation Techniques

Feature Selection

As shown in Figure 1A, 7 potential predictors were screened out of 26 features on the basis of 206 patients in the cohort and were with nonzero coefficients in the LASSO regression model. As presented in Figure 1B, the optimal parameter (lambda) of the LASSO model was selected using the minimum criterion. The left dotted vertical lines was lambda minimum, which meant lambda with the minimum error, representing the highest degree of model fitting under this lambda value. The value of lambda minimum was 7 in this figure, indicating that 7 predictors could be remained. And these 7 features included age, education level, drug cognition, self-evaluation of curative effect, inhalation device use instruction before treatment, post-instruction evaluation and evaluation at return visit.

Figure 1 Feature selection using the LASSO. Abbreviations: LASSO, least absolute shrinkage and selection operator; SE, standard error. Notes: (A) LASSO coefficient profiles of the 26 features. According to the log (lambda) sequence, the coefficient profile was generated. Vertical line was plotted at the values selected using cross-validation, where the optimal λ resulted in seven features with non-zero coefficients. (B) The optimal parameter (λ) of the LASSO model was selected by sevenfold cross-validation using the minimum criterion. The x-axis represents log (λ). The y-axis represents binomial deviance. There were two dotted vertical lines in this figure. The left one was λ minimum, which meant λ with the minimum error, representing the highest degree of model fitting under this λ value. The right one was λ 1-SE, representing the one standard error of the minimum criteria. The value of λ minimum above was 7, indicating that 7 predictors could be remained.

Multivariate Logistic Regression Analysis

As outlined in Table 5, all the 7 features were included in the multivariate logistic regression analysis.

Table 5 Prediction for Incorrect Inhalation Techniques

The Nomogram for Predicting Incorrect Inhalation Technique Risk

The predicting model that combining the above 7 independent predictors was developed and presented as the nomogram (Figure 2). For example, a female asthmatic patient who had been receiving inhalation therapy permanently, was 50 years old (0 point), with a junior high school education (45 points), had no knowledge about her using drug (70 points), and had a good self-evaluation of curative effect (0 point), and had been received use instruction of inhalation device before treatment (0 point), but had not been received evaluation after instruction (30 points) and at return visit (100 points). These 7 items added up to a total of 245 points, indicating that the asthmatic patient had a high risk of using inhalation device incorrectly, nearly 0.90. In this case, the healthcare workers could rapidly screen and early intervene in patients with high risk during follow-up.

Figure 2 Developed the nomogram for predicting risk of incorrect inhalation techniques. Notes: The nomogram for predicting risk of incorrect inhalation techniques in patients with chronic airway diseases was developed in the cohort, with the risk factors of age, education level, drug cognition, self-evaluation of curative effect, inhalation device use instruction before treatment, post-instruction evaluation and evaluation at return visit.

Calibration of the Nomogram for Predicting Risk of Incorrect Inhalation Techniques

Apparent Performance of the Nomogram

Figure 3 shows that calibration curve of the nomogram used to predict the risk of incorrect inhalation techniques performed good agreement in our cohort. The Harrell’s C-index for the prediction nomogram was 0.814, with a Z value of 10.31 (P<0.001). Furthermore, it was confirmed to be 0.783 by bootstrapping validation, which indicated that the model had good discrimination. In addition, the ROC curve was obtained in our study, as outlined in Figure 4, with AUC of 0.814, the sensitivity of 0.78; specificity of 0.75. That is, in the nomogram for predicting the risk of incorrect inhalation techniques, apparent performance addressed a good prediction capability.

Figure 3 Calibration curve of the risk of incorrect inhalation techniques in nomogram prediction model. Notes: The x-axis represents the probability of incorrect inhalation techniques. The y-axis represents the actual performance of incorrect inhalation techniques. The diagonal dotted line represents a perfect prediction by an ideal model. The solid line represents the apparent performance of the nomogram, of which a close fit to the diagonal dotted line addresses a good prediction capability.

Figure 4 The ROC curve of the nomogram for predicting risk of incorrect inhalation techniques. Abbreviations: ROC, receiver operating characteristic; AUC, area under the curve. Notes: The x-axis represents the false-positive rate of risk prediction. The y-axis represents the true-positive rate of risk prediction.

Assessment of Clinical Validity

The DCA for the nomogram is presented in Figure 5. The net benefit of our model was higher than the extreme curve. Assume a predicted probability of 70% for patients to be considered to use the inhalation devices incorrectly and receive intervention, approximately 30 out of every 100 patients using the model could benefit from it without harming the interests of others. In addition, using this nomogram to predict the risk of incorrect inhalation techniques added more benefits than the scheme.

Figure 5 Decision curve analysis for the nomogram of incorrect inhalation techniques. Notes: The x-axis represents the threshold probability. The y-axis measured the net benefit. The blue line represents the nomogram of incorrect inhalation technique risk. The thin solid line represents the assumption that all patients use inhalation devices incorrectly. The thick solid line represented the assumption that no patient used inhalation device incorrectly.

Discussion

Inhaler use is the main method of medication administration for asthma and COPD patients.^15, However, inhalation devices, such as SMIs, pMDIs and DPIs, require several steps in order to obtain sufficient medication and have a high rate of misuse.^7,16,17 A recent research found that 86.7% of the patients had at least one inhalation technique error, and 76.9% had at least 20% incorrect steps.⁷ And the percentage of inhalation technique errors in our data was approximately 37%.

Previous studies have found that several high-frequency factors were associated with the misuse, such as age, education level and lack of inhaler instruction.^15,18 Our research revealed the proportions of these risk factors that contributed to the incorrect inhalation techniques in quantitative form through a new nomogram prediction model, which is a visualization method of a complicated mathematical model that incorporates multiple risk factors to provide accurate and personalized risk estimates.^19,20

To our knowledge, this is the first study to use this new nomogram prediction model to assess risk factors for incorrect inhalation techniques in patients with chronic airway diseases. Consequently, the risk factors of age, education level, drug cognition, self-evaluation of curative effect, inhalation device use instruction before treatment, post-instruction evaluation and evaluation at return visit were identified as the major predictors to develop the predictive nomogram. Otherwise, the model displayed good discrimination, calibration and clinical validity. Especially the C-index was confirmed to be 0.783 by bootstrapping validation, which meant that the nomogram can be widely and accurately applied in clinical practice.

As can be seen from the above, the predictors of inhalation technique instruction and evaluation take a large proportion in our prediction model. Unfortunately, our study showed that 9% of the patients were never received inhalation instruction, only 43% were evaluated after instruction and 17% at return visit. Instruction by medical workers is an important modifiable factor for reducing inhaler misuse and repeated instruction is necessary.^21–25 Takaku et al proposed that each device required at least three instructions to achieve completely error-free or total error less than 10%.²¹ Klijn et al suggested that periodical intervention reinforcement was necessary to reduce the rate of misuse as educational interventions were effective on the short-term and appeared to wane over time.²⁶ Further support came from a 3-month controlled parallel-group study by Wang et al, which showed that patients in the intervention group received educational interventions such as face-to-face instruction, videos and internet-based education, and follow-up reeducation was more effective in improving inhalation technique than the control group.¹⁸ These findings should encourage health staff to provide instruction and education on proper inhalation techniques and to regularly re-evaluate patients’ mastery of inhalation techniques.

Reduced medication adherence to treatment is considered to be another major issue that significantly impairs pharmacologic treatment effectiveness.^22,27 The result of medication adherence showed that only 16% of the patients had high adherence, 40% had medium adherence, and 44% had low adherence. However, no correlation was found between medication adherence and inhalation technique in our study.

Furthermore, our research presented that the most commonly used inhalation devices were DPIs, accounting for 73%, followed by pMDIs for 23%, either single or dual. However, the two most common critical inhaler errors when using DPIs and pMDIs are uncoordinated actuation and inhalation, and failure to inhale forcefully.^28,29 Currently, several new inhalers with innovative designs are being applied clinically.³⁰ For example, breath-triggered inhaler (BTI), a low-resistance device, can be triggered at a low inspiratory flow rate with merely a slight inspiratory effort. It does not need to inhalation and manual canister compression simultaneously, which may fulfill patient needs by reducing the two critical inhaler errors mentioned above.^31,32 Some new smart inhalers, connected to smartphones, can even provide information about patients’ mastery of inhalation techniques.³³ Additionally, using inhalers according to patient preferences may reduce device handling errors and improve adherence to inhalation medication.³⁴

There are also some limitations in this study. Firstly, assessment scales for mastery of inhalation devices were only administered by health professionals and the objective parameters of corresponding evaluation devices were lacking. Secondly, although our study was internally validated to ensure the reliability of the nomogram model, external validation could not be conducted. Further clinical practice studies are needed to confirm. Thirdly, this is a single-center, cross-sectional study with a relatively small sample size, which limits the generalizability of the findings.

In conclusion, our study had developed and validated a simple nomogram for predicting the risk of incorrect use of inhalation devices among patients with chronic airway diseases. The application of the aforementioned risk factors in the current predictive model is also plausible.

Conclusion

The nomogram incorporating age, education level, drug cognition, self-evaluation of curative effect, inhalation device use instruction before treatment, post-instruction evaluation and evaluation at return visit could be conveniently used to predict the risk of incorrect inhalation techniques in patients with chronic airway diseases.