Predictive Factors of Oxygen Therapy Failure in Patients with COVID-19 in the Emergency Department

Backgrounds

Coronavirus disease 2019 (COVID-19) affects mortality owing to viral pneumonia that progresses to acute respiratory distress syndrome and acute hypoxemic respiratory failure. Early strategies in this new emerging disease are intubation in patients with signs of respiratory distress and avoiding non-invasive ventilation or a high-flow nasal cannula (HFNC) that risks aerosol generation.^1,2 In a study on COVID-19, HFNC therapy was associated with decreased intubation or mechanical ventilator use and mortality.³

A predictor for oxygen therapy is the respiratory rate-oxygenation (ROX) index at 2, 6, and 12 hours after applying HFNC. This index has been validated in treating hypoxemic respiratory failure without COVID-19.⁴ Many predictors of COVID-19 have been used to identify patients who require mechanical ventilation and those at risk of mortality, such as biomarkers and physiological parameters.^5–9 However, the predictors of failure of intubation remain unclear. The optimal timing of intubation in COVID-19 patients who receive oxygen is controversial. Some studies have shown that early intubation reduces patients’ self-inflicted lung injury and adverse outcomes.^1,10–12 In the COVID-19 pandemic, most patients were critical illnesses requiring invasive mechanical ventilation (IMV), leading to the intensive care unit (ICU)‘s overburdening. Therefore, prioritizing patients who require intensive care is important.

This study aimed to evaluate the predictors and develop a predictive model requiring IMV in COVID-19 patients treated with oxygen in the emergency department (ED).

Methods

Study Design and Setting

We performed a retrospective, single-center, at Ramathibodi Hospital, a tertiary care and university hospital in Bangkok. The Ethics Committee of Ramathibodi Hospital, Mahidol University, approved this study (COA. MURA2021/679). The ethics committee did not require consent for this research because the study is based on hospital registry data, and reviewing the medical record is the reason for the waiver and a statement covering patient data confidentiality and compliance according to the Declaration of Helsinki.

Selection of Participants

Patients aged ≥18 years who were diagnosed with COVID-19 and required oxygen therapy in the ED from March 20, 2020, to August 18, 2021, were included in the study. The exclusion criteria included the following: 1) patients who had intubation performed immediately after the ED visit; 2) patients who submitted a do-not attempt to intubation; and 3) missing data.

Definitions

COVID-19 was determined by laboratory-confirmed detection via qualitative reverse transcription-polymerase chain reaction; testing was performed on a nasopharyngeal swab or endotracheal aspirate. Additionally, oxygen therapy, including conventional oxygen therapy, was defined as the delivery of oxygen flow >6 L/min. The inspired oxygen fraction (FiO2) was approximated depending on the oxygen delivery device and oxygen flow delivery. For example, a nasal cannula with a flow of 6 L/min resulted in a FiO2 of approximately 0.44. A non-rebreathing mask with a reservoir bag flow of 10 L/min and 15 L/min resulted in a FiO2 of 0.8 and 1.0, respectively. An HFNC was defined as a device that delivered humidified and heated oxygen at a flow rate >30 L/min. The ROX index was defined as the ratio of oxygen saturation/FiO2 to the respiratory rate. Failure of oxygen therapy was defined as when patients required IMV. The decision to use IMV was dependent on the physician on duty.

Measurements and Clinical Outcomes

Patients included in this study were evaluated by emergency room staff, emergency physicians, and internal medicine residents. We recorded the patients’ demographic information (age, sex, comorbidity, and body mass index), vital signs at initiating oxygen therapy (systolic blood pressure, heart rate, respiratory rate, oxygen saturation, and temperature), and initial oxygen therapy (flow and FiO2). We also recorded the SOFA score without a respiratory score at initiating oxygen, the ROX index at 2, 6, 12 and 24 hours, and laboratory parameters (C-reactive protein (CRP), lactate dehydrogenase, absolute lymphocyte count, D-dimer, albumin, and blood glucose). Furthermore, PF ratio after intubation, time to intubation, and mortality were recorded. The data of participants were identified from Electrical Medical Records (EMR). All patient data were treated with confidentiality following the Declaration of Helsinki.

Our study aimed to assess the predictors and develop a predictive model of oxygen therapy failure within 7 days in patients with COVID-19 in the ED.

Data Analysis

We calculated the sample size required to analyze predictors for oxygen therapy failure. Our hospital data on COVID-19 showed that the intubation rate in patients treated with conventional oxygen therapy and HFNC was 24.9%, and 75.1% of patients were not intubated (N = 198 patients). Therefore, we calculate the sample size by the two-sample proportion test (Pearson’s chi-squared test) of oxygen therapy success and failure. Our calculation showed that 68 patients with COVID-19 who required oxygen therapy in the ED, 17 patients with intubation, and 51 patients with the success of oxygen therapy were needed to provide an adequate sample size for this study (95% power of sample size, 5% alpha error [two-sided test]).

Categorical variables are expressed as absolute values and percentages and continuous variables as mean (standard deviation (SD)) and medians (interquartile ranges (IQRs)). The Cox regression and Harrell’s C were used to assess predictive factors requiring IMV within 7 days in patients with COVID-19. Selected variables in Cox regression with a p-value less than 0.100 were used in multivariable Cox regression to assess the primary outcome. The predictive model was created using variables statistically significant in multivariable Cox regression. Nomogram predictive model was developed from coefficients calculated and survival probability of ventilator-free;¹³ model calibration was assessed using the proportional-hazards assumption. The predictive model was internally validated using bootstrapping, and 1000 bootstrap replications were drawn. Kaplan–Meier curves were used to determine the probability of being ventilator-free within 7 days in the predictive model and ROX index. Decision curve analysis was performed to assess the benefit of using the model. With the intensity of care in the COVID-19 pandemic, we restricted the probability threshold at 20% for requiring IMV within 48 hours.

Data were recorded and statistical analysis by STATA version 16.1 (StataCorp LLC, College Station, TX, USA).

Results

From March 20, 2020, to August 18, 2021, 407 patients were eligible for inclusion in the study. As a result, 74 patients were excluded according to the exclusion criteria, and 333 patients were finally analyzed (Figure 1).

Figure 1 Study flow chart.

Among these 333 patients (Table 1), 256 (76.9%) initially had oxygen therapy with an HFNC, and 77 (23.1%) had oxygen therapy with conventional oxygen. Ninety-seven (29.1%) patients had failed oxygen therapy and subsequently received IMV, and death at 28 days after the ED visit occurred in 31 (63.3%). Prediction performance in each variable was shown in Supplementary File 1.

Table 1 The Baseline Characteristics and Univariable Cox Regression Analysis for Risk Factors Associated with Oxygen Therapy Failure in 7 Days in Patients with COVID-19

In the cohort, multivariable Cox regression showed patient’s age, the ROX index at 2 hours, IL-6 concentrations ≥20 pg/mL, and the SOFA score without a respiratory score were statistically significant independent factors in patients with oxygen therapy failure (Table 2).

Table 2 Cox Proportion Hazard Model for Analyzing Risk Factors Associated with Oxygen Therapy Failure in 7 Days in Patients with COVID-19

The predictive nomogram model was developed with significant factors; the scores for each variable (Supplementary File 2) and ventilator-free survival probability are shown in Figure 2. The proportional-hazards assumption of the predictive model was 0.776. The predictive model was divided into three levels to predict requiring IMV: high risk (score >8; sensitivity 70.10% and specificity 95.76%), intermediate-risk (score 5–8, sensitivity 90.72% and specificity 73.31), low risk (score <5).

Figure 2 The nomogram prediction model for predicting survival probability of ventilator-free within 24 hours, 48 hours and 7 days of COVID-19.

Predicting the Performance of the Predictive Model for Requiring a Mechanical Ventilator in 7 Days in Patients with COVID-19

The predictive model had a better performance than the ROX index at 2 and 6 hours, as shown by Harrell’s C-statistic, p-value <0.001 and 0.334, respectively, Table 3. The internal validation of the predictive model for predicting oxygen therapy failure is shown in Table 3.

Table 3 Prediction Performance of Predictive Model and ROX Index for Requiring IMV in 7 Days in Patients with COVID-19 in Training Cohort and Bootstrapping Internal Validating

Survival Analysis of the Predictive Model and ROX Index and the Probability of Being Ventilator-Free

The high-risk group of the predictive model (Figure 3) and ROX index <5.1 (Figure 4) indicated a significantly lower probability of being ventilator-free in 7 days than the intermediate and low risk.

Figure 3 Kaplan–Meier survival analysis of being ventilator-free for 48 hours with the risk of the predictive model; log rank between predictive model score >8 and predictive model score 5–8 was p <0.001, between predictive model score >8 and predictive model score <5 was p <0.001 and between predictive model score 5–8 and predictive model score <5 was p <0.001.

Figure 4 Kaplan–Meier survival analysis of being ventilator-free for 48 hours with the risk of the ROX index; log rank between ROX index ≤5.1 and ROX index 5.11–9.99 was p <0.001, between ROX index ≤5.1 and ROX index ≥10 was p <0.001 and between ROX index 5.11–9.99 and ROX index ≥10 was p <0.001.

Of 97 patients who required IMV, delayed intubation for longer than 48 hours after oxygen therapy appeared to increase 28-day mortality, not significant [crude HR 1.13 (0.63, 2.03), p-value 0.684]. Decision curve analysis for requiring IMV within 48 hours, the predictive model was superior to the ROX index for predicting IMV associated with a threshold probability between 5% and 20%. (Figure 5).

Figure 5 Decision curve analysis for assessing the net benefit between using risk strategy model of the predictive model and ROX index in requiring invasive mechanical ventilation within 48 hours.

Discussion

In the COVID-19 pandemic, acute respiratory distress syndrome and respiratory failure have affected resources and mortality in the ICU.^1–3 Many predictors have been used to predict worsening outcomes of COVID-19, but many studies only focused on the ICU setting.^3,5,6

In this study on patients with COVID-19 in the ED, we found that elderly, a low ROX index, high IL-6 concentrations, and a high SOFA score without a respiratory score were associated with oxygen therapy failure within 7 days. Therefore, we developed the predictive model, which had a good performance to predict oxygen therapy failure. The cutoff for predicting the probability of requiring IMV was a ROX index of 5.10 and a predictive model score of more than 8.

In a previous study in the ICU, the ROX index <3.5 showed 89% specificity for HFNC failure.¹⁴ Additionally, Zucman et al showed that ROX index >5.37 at 4 hours was a low risk for intubation (66.0% sensitivity and 83.0% specificity).⁶ Outside of the ICU, a low ROX index indicates an increased risk of requiring IMV.¹⁵ Moreover, ROX index <5.9 at 12 hours after HFNC therapy was reported to show 90% specificity for HFNC failure.¹⁶ Therefore, our study supports previous evidence that the ROX index helped predict patients who require IMV. We classified patients with COVID-19 into three groups according to the ROX index. In the high-risk group, ROX index <5.1 at 2, 6, 12, and 24 hours showed 98.31%, 96.61%, 98.31%, and 99.15% specificity, respectively, and had a high probability of requiring IMV. In the moderate-risk group, ROX index of 5.11–9.99 at 2, 6, 12, and 24 hours showed 97.94%, 95.38%, 95.83%, and 100.00% sensitivity for requiring IMV, respectively, which suggested that oxygen therapy should be continued and closely monitored. In the low-risk group, ROX index >10 showed a low probability of oxygen therapy failure.

Many studies have identified potential biomarkers associated with COVID-19 pneumonia with a cytokine storm and worse outcomes.^17,18 Arnold et al found that IL-6 had a good performance (AUROC: 0.770, 95% CI: 0.65–0.88, cutoff at 79 pg/mL) to predict adverse events and the requirement of respiratory support and ICU admission.⁷ In our study, IL-6 was a predictor for oxygen therapy failure and better performance than D-dimer, the absolute lymphocyte count albumin, CRP, and lactate dehydrogenase. A cutoff of IL-6 concentrations >20 pg/mL showed 71.13% sensitivity and 83.90% specificity for requiring IMV.

We developed the predictive model by using clinical parameters and biomarkers to identify patients who require IMV at an early stage. The predictive model had a better performance than the individual ROX index at 2 hours for predicting oxygen therapy failure. Additionally, in decision analysis, the predictive model had a better net benefit of predicting oxygen therapy failure within 48 hours than ROX index at 2 hours. The optimal time for intubation remains unclear in COVID-19 pneumonia. Gattinoni et al advocated avoiding delayed intubation to decrease lung injury if reducing respiratory effort was not possible after resolution of hypoxemia by applying oxygen therapy or a non-invasive ventilator.¹² Bavishi et al found that delayed intubation led to a higher mortality rate (29%) than that (6%) with intubation within 4–24 hours after hospital admission.¹⁹ Our study appeared to show that delayed intubation had an increased risk of mortality, but this was not significant. Based on our results, we strongly support using clinical parameters with laboratory values in patients with COVID-19 pneumonia who require close monitoring and IMV for effective care to decrease adverse outcomes.

Limitations

Our study has several limitations. First, this was a single-center study, which limited the generalization of our results. However, this study had an adequate sample size to assess the outcome. Second, the management of initial oxygen therapy, switching conventional oxygen therapy to HFNC or non-invasive ventilation, and the decision of IMV used were based on physicians in the ED; this condition in medical decisions is a part of real-life practice. In addition, the physicians had been trained in resuscitation in patients with COVID-19 pneumonia. Third, our study excluded most of the patients’ delivery of oxygen flow less than 6 L/min because our setting was categorized as low risk and treated outside the hospital. Fourth, our study did not analyze treatment as the predictor because of varying times in initiated treatment with medication; especially tocilizumab was not the first line in our setting. However, this study was not statistically different from initial medication treatment as remdesivir and intravenous methylprednisolone. Finally, the predictive model only had internal validation performed, and further external validation is required.

Conclusions

This study shows that a low ROX index, high IL-6 concentrations, and a high SOFA score without a respiratory score and elderly are associated with requiring IMV within 7 days. The predictive model has a good performance in discriminating patients who require IMV. The COVID-19 pandemic, which has led to limited available resources, provides a challenge for appropriate allocation resource management of patients. The early identification of patients at risk could guide decision-making and allocation of resources for these patients.