A Nomogram for Predicting Mortality in Patients with Pneumonia-Associated Acute Respiratory Distress Syndrome (ARDS)

Background

Acute respiratory distress syndrome (ARDS) is a common life-threatening clinical syndrome in critically ill patients. It is characterized by bilateral chest radiographical infiltrates with severe acute hypoxemic respiratory failure due to diffuse lung inflammation and edema which is not fully explained by cardiac failure or fluid overload.¹ It is estimated that ARDS is present in about 10% of all patients in intensive care units (ICU) worldwide.² Progress has been made in developing an evidence-based treatment and supportive care for ARDS patients, such as lung-protective mechanical ventilation and limited fluid resuscitation, during the last decades.³ However, no specific pharmacotherapies have been proved to be effective and thus, the mortality of ARDS remains high at approximately 40%.²

Considering the high morbidity and mortality, it is of great significance to identify which category of ARDS patients will have a poor prognosis in advance. The early risk stratification provides an important basis for personalized intervention and treatment, which might help reduce the mortality of ARDS. Several prediction tools are available for predicting prognosis, such as the Acute Physiology and Chronic Health Evaluation II (APACHE II) and Sequential Organ Failure Assessment (SOFA).⁴ However, these widely used scoring systems in ICU are not specific enough for the evaluation of mortality in ARDS patients. The 2012 Berlin definition proposed three categories of ARDS patients based on degree of hypoxemia that correlated with mortality: mild (PaO₂/FiO₂ ≤ 300 mmHg), moderate (PaO₂/FiO₂ ≤ 200 mmHg) and severe (PaO₂/FiO₂ ≤ 100 mmHg).⁵ Nevertheless, it is previously reported that the predictive usefulness of Berlin stages for mortality in ARDS was marginal with an area under the receiver operating characteristic (ROC) curve (AUC) of 0.60.⁶ Therefore, developing novel prediction models of ARDS have a great clinical value for risk assessment and clinical management optimization.

Previous studies have verified that different phenotypes of ARDS have diverse risk factors, microbiology, pathophysiology of lung injury, and response to treatments.⁷ The most common cause of ARDS is pneumonia (accounting for 50–80% of all ARDS), followed by non-pulmonary sepsis, pancreatitis, aspiration and trauma.⁸ Thus, more attention needs to be paid to pneumonia-associated ARDS. However, to our knowledge, there is no predictive tool developed for pneumonia-associated ARDS specifically so far. As a result, the clinical risk classification of these patients is not well defined, which might partly make the treatments challenging. Our study aims to construct a novel, early, precise prediction model for hospital mortality based on baseline clinical data in patients with pneumonia-associated ARDS.

Methods

Study Designs

In this single-center retrospective study, consecutive patients with pneumonia-associated ARDS admitted into ICUs in West China Hospital of Sichuan University in China between January 2012 and December 2018 were enrolled. The study protocol was approved by the West China Hospital of Sichuan University Biomedical Research Ethics Committee (No.2021–828) and was conducted in accordance with the amended Declaration of Helsinki. The requirement for written informed consent from patients was waived due to retrospective design. The study was performed and reported in accordance with the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) reporting guideline.⁹

Patients and Data

The ARDS was defined according to the Berlin definitions: (1) Within 1 week of a known clinical insult or new or worsening respiratory symptoms; (2) Bilateral opacities—not fully explained by effusions, lobar/lung collapse, or nodules; (3) Respiratory failure not fully explained by cardiac failure or fluid overload; and (4) the presence of acute hypoxemic respiratory failure with PaO₂/FiO₂ ≤300 mmHg.⁵ The pneumonia was defined as a new pulmonary infiltrate on chest X-ray or computed tomography and at least one of the following acute lower respiratory infection symptoms: fever, productive cough, purulent expectoration, dyspnea, pleuritic chest pain or focal chest signs on auscultation or abnormal peripheral white cell counts.¹⁰ The etiologies of ARDS were reviewed and determined by two experienced physicians independently. Any disagreement was solved by a third physician or team discussion. Only individuals who were diagnosed with pneumonia-associated ARDS were selected in present study.

Individuals were excluded from the study if they were as follows: (1) under 18 years old; (2) pregnant; (3) severe immunocompromised, defined according to a consensus statement of pneumonia;¹¹ (4) discharged within 24 hours of admission; (5) having incomplete data. Only data related to the first admission were considered if the patient had repeated admission during study period.

Data related to demographic characteristics, comorbidities, vital signs and laboratory examinations during the first 24 hours after ICU admission were extracted and collected. The first value was used for analysis if any data was repeated. The definitions of chronic diseases were shown in Table S1. Two physicians completed the data collection by using a standardized data collection form independently. The included patients were randomly divided into two cohorts (70% in training cohort and 30% in testing cohort) by simple random sampling. The training cohort was used to develop a model, and the testing cohort was used to validate the model. All patients received routine standard care and therapy to the discretion of the ICU attending physician and based on the pneumonia and ARDS guidelines, including lung-protective mechanical ventilation, use of prone positioning or recruitment maneuvers, maintaining of fluid balance, timely administration of appropriate antimicrobial medications and vasopressors, etc.^10,12 The primary outcome was hospital mortality.

Statistical Analysis

The clinical characteristics of patients were expressed as the mean ± standard deviation (SD) or median (interquartile range [IQR]) for continuous variables, and the frequency (percentage) for categorical variables. Independent sample t–test or Kruskal–Wallis test was performed to analyze the differences between the continuous variables as appropriate. The chi-square test or Fisher exact test was used to analyze those categorical variables. The variables missing over 20% of observations were removed to ensure the accuracy of study. After that, multiple imputation for missing variables was employed if missing values were less than 20%. First, we used least absolute shrinkage and selection operator (LASSO) regression to analyze the high-dimensional data in the training cohort. It could eliminate multicollinearity and avoid over-fitting of variables. In the LASSO regression model, the characteristics with non-zero coefficients were selected as potential useful predictors for mortality. Then, the candidate predictors were included into the multivariate logistic regression analysis using forward step: LR to identify the independent risk factors for death. The odds ratios (ORs) with 95% confidence intervals (95% CIs) and P values were calculated. The logistic regression model was evaluated by Hosmer–Lemeshow goodness-of-fit test, Omnibus test and Brier score.

The study established a prediction model based on these identified independent predictors. The model was visualized via a nomogram, which assigns a score to each value level of each predictor according to the degree of its contribution to the outcome. Summing these scores could obtain a total score and correspondingly, the probability of death. We further established an online dynamic nomogram which could transform complex regression equations into a visual graph, making the results of the model more readable. We evaluated the performance of differentiation, calibration, and clinical utility of the nomogram in both training and testing cohorts. The Harrell’s concordance index (C index) was reported, and the ROC curve was plotted to quantify its discriminative performance. The precision–recall (PR) curve, which plots the positive prediction value (PPV) against the true positive rate (TPR) across all thresholds, is another accurate method to assess the discrimination capability of the model. The calibration curve with 1000 bootstrap resampling was plotted to evaluate the agreement between the predicted outcomes and the actual outcomes. The decision curve based on net benefits at different threshold probabilities and the clinical impact curve were both drawn to evaluate the nomogram’s clinical validity and utility. Finally, to make the prediction model more feasible and applicable, the patients were divided into four groups with different risks of death (low, moderate, high and very high risk) according to the nomogram. The observed ORs (95% CIs), P values and the P for trend across these groups were calculated.

In this study, R software version 4.2.1 (R Foundation for Statistical Computing) and SPSS software version 26 (SPSS, Chicago, IL, USA) were used to carry out the statistical analysis. A two-sided P < 0.05 was considered statistically significant.

Results

Baseline Characteristics

A total of 712 individuals with pneumonia-associated ARDS were admitted into our hospital. Among them, 80 individuals were excluded from analysis according to exclusion criteria (Figure 1). The included patients were divided into the training cohort (442 patients) and the testing cohort (190 patients). The hospital mortality was 200 (45.2%) in the training cohort and 85 (44.7%) in the testing cohort. In the training and testing cohort, the median age was 67 (IQR: 51, 78) and 64 (IQR: 52, 76) years and the median PaO₂/FiO₂ ratio was 138 (IQR: 95, 190) and 149 (IQR: 92, 183), respectively. The detailed baseline characteristics were summarized in Table 1. There was no significant difference in the variables (the P values were all above 0.05), which suggested that the baseline clinical characteristics of patients were comparable between the training cohort and the testing cohort.

Table 1 Baseline Characteristics of Pneumonia-Associated ARDS Patients in Training Cohort and Testing Cohort

Figure 1 Study population. Abbreviation: ARDS, acute respiratory distress syndrome.

Selection of Predictors and Development of Nomogram

In the training cohort, the LASSO regression effectively selected 11 potential predictors (Figure S1), including age, chronic cardiovascular diseases, chronic respiratory diseases, chronic renal diseases, total neutrophil count, total lymphocyte count, platelet, albumin, creatinine, D-dimer, and procalcitonin. Then, the multivariate logistic regression analysis further identified the age (OR: 1.04; 95% CI: 1.02, 1.05; P < 0.001), chronic cardiovascular diseases (OR: 2.62; 95% CI: 1.54, 4.45; P < 0.001), chronic respiratory diseases (OR: 1.87; 95% CI: 1.02, 3.43; P = 0.044), lymphocyte (OR: 0.56; 95% CI: 0.39, 0.81; P = 0.002), albumin (OR: 0.94; 95% CI: 0.90, 1.00; P = 0.031), creatinine (OR: 1.00; 95% CI: 1.00, 1.01; P = 0.021), D-dimer (OR: 1.06; 95% CI: 1.03, 1.09; P < 0.001) and procalcitonin (OR: 1.14; 95% CI: 1.07, 1.22; P < 0.001) as independent prognostic factors. The Hosmer–Lemeshow goodness-of-fit test (P = 0.706), omnibus test (P < 0.001), R² (0.346) and Brier score (0.180) indicated that the multivariate logistic model had good performance with acceptable predictive accuracy.

Among these factors, the age, chronic cardiovascular diseases and chronic respiratory diseases were considered to be pre-admission variables. The Sankey diagram was plotted to visualize their distributions (as categorical variables) and relationship with outcomes (Figure S2). The patients with age ≥65 years or with chronic diseases had relatively high mortality. The Spearman correlation analysis was also performed among the remaining after-admission variables (as continuous variables) to explore their potential innate relationships (Figure S3). The procalcitonin were positively correlated with creatinine and was negatively correlated with lymphocytes and albumin (P < 0.05). The D-dimer was also negatively correlated with albumin (P < 0.05).

Thereafter, a prediction model that incorporated the above eight independent predictors was developed and presented as a nomogram (Figure 2A). Meanwhile, a web-based user-friendly dynamic nomogram (https://h1234.shinyapps.io/dynnomapp/) was constructed for conveniently predicting the hospital mortality among patients with pneumonia-associated ARDS (Figure 2B). By entering the information regarding these factors in the web-online tool, we could obtain the predicted hospital mortality of patients.

Figure 2 (A) The nomogram for predicting hospital mortality among patients with pneumonia-associated ARDS. It includes eight factors: age (years), chronic cardiovascular diseases, chronic respiratory diseases, lymphocyte (×109/L), albumin (g/L), creatinine (μmol/L), D-dimer (mg/L) and procalcitonin (ng/mL). (B) A web-based dynamic nomogram (https://h1234.shinyapps.io/dynnomapp/). By entering the information regarding these factors, we could obtain the predicted hospital mortality of patients.

The Performance of Nomogram

The C index of the nomogram was 0.798 (95% CI: 0.756, 0.840) in the training cohort and 0.808 (95% CI: 0.747, 0.870) in testing cohort (Figure 3A and D). In addition, the area under the PR curve was 0.778 and 0.798 in training and testing cohort (Figure 3B and E). These results showed favorable and robust discrimination between survivors and non-survivors of the model. The calibration curve showed satisfactory consistency with the perfect prediction line, which indicated that the predicted mortality curves were close to the observed mortality curves, in both two cohorts (Figure 3C and F). The bias-corrected C index was 0.784 and 0.790 in training and testing cohort, respectively. The decision curve analysis (DCA) of training and testing cohort is shown in Figure 4A and C. The DCA showed that if the threshold probability for patients varied from 10% to 80%, using the prediction model to make the decision of whether to treat could add more net benefits than treat-all-patients or treat-none-patients. The clinical impact curves showed that the predicted probability coincided well with the actual probability in the training and testing cohort, respectively (Figure 4B and D). Thus, the nomogram has good clinical utility.

Figure 3 (A) The ROC curve of nomogram for training cohort. (B) The precision–recall (PR) curve in training cohort. (C) Calibration curve in training cohort. (D) The ROC curve for testing cohort. (E) The precision–recall (PR) curve in testing cohort. (F) Calibration curve in testing cohort.

Figure 4 (A) The DCA of nomogram for training cohort. (B) the clinical impact curve for training cohort. (C) The DCA for testing cohort. (D) the clinical impact curve for testing cohort. Abbreviation: DCA, decision curve analysis.

To better apply this model in clinical practice, a risk classification system was constructed to identify distinct groups of patients with distinct risks of death. The patients were categorized into four risk groups in terms of the total scores calculated by the nomogram: low risk (<140 points; predicted mortality <25%), moderate risk (140–174 points; predicted mortality: 25–50%), high risk (175–209 points; predicted mortality: 50–75%) and very high risk (>210 points; predicted mortality >75%). Compared with patients in low-risk group, the observed ORs (95% CIs) for hospital mortality of patients in moderate-risk, high-risk and very high-risk group were increased gradually in both two cohorts (P for trend < 0.001) (Table 2).

Table 2 The Risk Stratification of Pneumonia-Associated ARDS Patients According to Nomogram

Discussion

In present study, we identified the age, chronic cardiovascular diseases, chronic respiratory diseases, lymphocytes, albumin, creatinine, D-dimer and procalcitonin as independent risk factors for the mortality in patients with pneumonia-associated ARDS. The predictors were all readily available at ICU admission. We developed a simple and clinically beneficial nomogram and evaluated its performance in training and validation cohorts. The dynamic online manner with a user-friendly digital interface may help contribute to better clinical decision-making.

Previously, a large number of studies have investigated the risk factors related to mortality in ARDS patients. Older age has been widely considered to be a powerful risk factor for poor prognosis in various diseases. Increased creatinine is associated with presence of renal injury or renal failure. The retention of metabolic waste products would result in declined ability to manage fluid status and acid-base balance. In a previous study, ARDS patients with the hyper-inflammatory phenotype, including increased age and creatinine, had higher 28-day mortality (39% vs 17%; P < 0.0001) and 90-day mortality (47% vs 22%; P < 0.0001) than those with the hypo-inflammatory phenotype.¹³ Similarly, in coronavirus disease 2019 (COVID-19)-induced ARDS cohort, the subgroup with higher age, creatinine, procalcitonin and lower albumin had higher 90-day and 180-day mortality than other subgroups (P < 0.01).¹⁴ The procalcitonin is a traditional early infectious and inflammatory marker that is often used to assist risk stratification in severe pneumonia. Tseng et al previously reported that procalcitonin within 72 hours of the onset of ARDS could predict mortality in patients with ARDS caused by severe pneumonia.¹⁵ Apart from reflecting the nutritional status, albumin is also associated with liver dysfunction and metabolic processes. Chen et al reported that the elevated PAR (procalcitonin to albumin ratio), which means increased procalcitonin and decreased albumin, is an independent predictor of 28-day mortality of ARDS in a prospective observational study (HR = 3.593, 95% CI: 1.702–6.482).¹⁶ Chronic cardiovascular and respiratory diseases are both frequent comorbidities of ARDS patients. The hemodynamic changes in the setting of ARDS might aggravate pre-existing heart dysfunction or heart failure. Then, this acute exacerbation of chronic cardiovascular diseases could influence the ventilatory strategy, therapeutic effects and clinical outcomes.¹⁷ Indeed, a recent study demonstrated that chronic cardiovascular disease increased the risk of death in ARDS (multivariate adjusted OR 1.54, 95% CI 1.23–1.92, P < 0.001).¹⁸ The comorbidity of chronic respiratory disease might represent long-term respiratory system dysfunction which is often characterized by hypoxemia and impairment of ventilation function or gas exchange. This would cause the decline of the pulmonary function and the antibacterial and recovery capacities. For instance, the chronic obstructive pulmonary disease (COPD) was found to be an independent risk factor associated with 28-day mortality in patients with severe COVID-19 in ICU.¹⁹ Lymphocytes play a protective role in the inflammatory reaction, systemic immune process and host defenses responses of pneumonia-ARDS. The study from Cheng et al revealed that the lower lymphocyte, especially the CD8+ T lymphocyte, was associated with higher severity and early mortality in patients with ARDS caused by Acinetobacter baumannii pneumonia.²⁰ The pathogenesis and pathophysiology of ARDS are complex, including epithelial and endothelial injury, intense inflammatory cascade, coagulation and anti-coagulation disorder, etc. Elevated D-dimer is also observed in ARDS patients, especially those with widespread pulmonary vascular thrombosis. It is demonstrated that the patients with D-dimer levels greater than the median value had markedly increased 28-day mortality than those with low D-dimer levels (P = 0.0001) in COVID-19-associated ARDS.²¹ These above prior conclusions are consistent with our results regarding the selection of predictors.

One unexpected result was that the PaO₂/FiO₂ ratio was not identified as a candidate predictor in the LASSO regression model, indicating that the stratification of severity of ARDS according to Berlin Definition did not completely correlate with the mortality, which agrees with the previous study.⁶ Considering that the condition of ARDS patients is usually complicated, the PaO₂/FiO₂ ratio, which is only based on lung-associated factors, is expected to have an inferior prognostic value than a comprehensive model including various parameters. However, caution is still needed in clinical use of this result. We only used the data of PaO₂/FiO₂ ratio at the first day of admission in the analysis. Nevertheless, we failed to collect the change of PaO₂/FiO₂ ratio or its worst value after several days of admission. The unexpected result could be somewhat attributed to this cause. We acknowledged that the PaO₂/FiO₂ ratio could also be considered by the clinicians in the clinical decision-making.

At present, a variety of prediction models have already been developed for ARDS. Liu et al included 197 ARDS patients and constructed a nomogram based on age, albumin, platelet, PaO₂/FiO₂, lactate dehydrogenase, computed tomography score, and etiology to predict 28-day survival with the AUC of 0.75.²² Another study included 1814 patients from the MIMIC-III Database and developed a nomogram with age, hemoglobin, heart failure, renal failure, Simplified Acute Physiology Score II (SAPS II), immune function impairment, total bilirubin, and PaO₂/FiO₂ to predict mortality (AUC: 0.791).²³ Furthermore, Wang et al enrolled 185 patients with ARDS originating from pulmonary disease and developed a prediction model (AUC: 0.795) consisting of age, sex, C-reactive protein, albumin and multiple organ dysfunction syndrome (MODS).²⁴ However, prior studies included various ARDS patients with heterogeneous causes, including pneumonia, aspiration, sepsis and acute pancreatitis. It has been suggested that the differences in immunological mechanisms, severity of neutrophil infiltration, resolution manner of inflammation, etc. between pneumonia and non-pneumonia -induced ARDS should be considered in the management of patients in the ICU.²⁵ We suspected that the heterogeneity of ARDS patients in prior studies could weaken the accuracy and practicality of models in specific pneumonia-associated ARDS.

The present study was constructed based on an adequate sample size by rigorous methods. Our study population was strictly confined to pneumonia-associated ARDS to decrease the bias originating from heterogeneity of patients and to ensure the predictive value when applied on those specific patients. The LASSO algorithm has been widely applied for the selection of candidate predictors. Furthermore, we used the logistics regression analysis to identify independent predictors after adjustment of confounding factors to ensure the accuracy of nomogram. The model showed satisfactory discrimination, calibration and clinical usefulness after evaluation by multiple methods. The bias between two cohorts was considered to be acceptable. The prevalence of pneumonia-associated ARDS continues to rise and its mortality remains high. The rapid accurate prediction of prognosis is still clinically challenging. An efficient evaluating approach with a high clinical applicability and generalizability is urgently needed. Early screening of high-risk patients could contribute to the decision-making process in the management of patients and might improve their prognoses.

There are some limitations in this study. First, we collected data from a single-center retrospective cohort with inherent limitations. Although we divided the testing cohort to evaluate the stability of prediction model, the model was not externally validated. Then, some other unknown risk factors related to mortality might be excluded from our model, which could cause some deviation of our nomogram. The therapy strategies might have also influenced the outcomes of ARDS. In addition, the type and etiology of pneumonia may result in different outcomes of ARDS patients. However, we could not perform more subgroup analysis due to lack of data. Third, some common risk factors were not evaluated, such as the radiological parameters and the pH values. According to the study design, they were removed because more than 20% of the values were missing. Last, we failed to perform long-term follow-up for these patients.

Conclusions

In conclusion, the current study developed and visualized a convenient and economical prediction model for predicting the hospital mortality in the patients with pneumonia-associated ARDS. The novel nomogram consisting of eight common clinical characteristics has a satisfactory predictive performance and may allow clinicians to better predict the risk of death. Future multicenter prospective studies with larger sample sizes are warranted to confirm our results and validate or improve our nomogram.