Development and Validation of Nomogram for Hospital Mortality in Immunocompromised Patients with Severe Pneumonia in Intensive Care Units: A Single-Center, Retrospective Cohort Study

Introduction

Pneumonia is one of the most common burdens of diseases worldwide.¹ It persistently causes deaths, without significant improvement during the last 34 year period, and a fifth of these patients hospitalized were severe pneumonia need to be admitted to intensive care units.² To identify the population of severe pneumonia and stratify patients, several predicting tools have been clinically applied in community-acquired pneumonia (CAP) patients, such as PSI,³ CURB-65,⁴ and IDSA/ATS 2007.⁵

The immunocompromised hosts (ICHs) are mainly used to describe a group of conditions such as tumor, hematologic malignancies, HIV infection, or treatment with immunosuppressants that lead to an impaired ability to exhibit an appropriate immune response. At present, the population of immunocompromised hosts continues to increase due to the increasing prevalence of the subjects with solid tumors and hematological disease,^6–9 wide clinical application of immunosuppressants, hormones, and chemotherapies,^10,11 organ transplantation, and HIV infection.^12,13 Compared to the non-immunocompromised populations, immunodeficiency populations are more likely to develop multiple, opportunistic, and drug-resistant infections with atypical clinical manifestations,¹⁴ which makes it difficult to identify the cause and causes a delay in diagnosis and treatment. The severity of pneumonia is increased in ICHs, resulting in higher incidence of acute respiratory failure and fatal outcomes,¹⁵ as well as higher demands for ICU admission. However, the guidelines of the Infectious Diseases Society of America and the American Thoracic Society on the management of community acquired pneumonia and hospital acquired pneumonia^11,12 have not addressed community-acquired pneumonia occurring in immunocompromised patients, and the recommendations for hospital-acquired pneumonia stated that they are not intended for ICHs,^5,16 especially for ICHs with severe pneumonia.

Because of the high mortality and poor prognosis in ICHs with severe pneumonia, this retrospective study was mainly performed to discover prognostic factors and report a prognostic nomogram model for predicting hospital mortality of ICHs with severe pneumonia in the ICU.

Materials and Methods

Population

This is a single-center, retrospective cohort study. We retrospectively enrolled ICHs with severe pneumonia who were admitted to the ICU in the West China Hospital of Sichuan University, a national-level center for the diagnosis and treatment of difficult and severe cases in western China, with 4,300 beds in total, between January 2012 and December 2018. According to the Infectious Diseases Society of America/American Thoracic Society (IDSA/ATS) consensus guidelines,⁵ severe pneumonia patients were included by meeting any one of major criteria (septic shock with need for vasopressors; respiratory failure requiring mechanical ventilation), or at least three minor criteria (respiratory rate ≥30 breaths/min; PaO₂/FiO₂ ratio ≥250; multilobar infiltrates; confusion/disorientation; blood urea nitrogen level ≥20 mg/dL; white blood cell count <4,000 cells/µL; platelet count <100,000/µL; core temperature <36°C; hypotension requiring aggressive fluid resuscitation).

Based on previous studies and consensus,^17,18 ICHs were determined by meeting one of the following criteria: 1) a progressive solid tumor or hematological malignancy (such as leukemia, myeloma, or lymphoma) undergoing chemotherapy within 6 months and chest radiation therapy within 3 months of admission or causes neutropenia (neutrophil count < 500 cells/mm³); 2) solid-organ, stem cell, or bone marrow transplant recipients; 3) receiving high-dose (>1 mg/kg/day) steroid treatment or long-term (≥3 weeks) immunosuppressive therapy, including steroids, cyclosporine, azathioprine, or biological modifiers such as etanercept or infliximab, for autoimmune diseases; 4) positive HIV infection. Patients were excluded if they 1) were <18 years of age; 2) had CPR within the first 24 hours or stayed less than 24 hours in the ICU; 3) had a long ICU stay over 90 days because of repeated admission for medical settlement; or 4) patients with >20% variable absence.

Data Collection and Outcomes

Clinical data, including medical history, demographic data, symptoms and vital signs, comorbidities, laboratory findings at ICU admission, clinical managements, and clinical outcomes were extracted from the electronic medical record system. Demographic data included age and sex. Symptoms and vital signs included fever, cough, dyspnea, temperature, mean blood pressure, heart rate, respiratory rate, and the ratio of arterial oxygen partial pressure to fractional inspired oxygen (PaO₂/FiO₂). Laboratory findings involved: complete blood count, coagulation test, liver and renal function, electrolytes, procalcitonin (PCT), and lactic acid. Clinical management consisted of direct ICU admission, the requirement for invasive mechanical ventilation, and the requirement for vasopressors. Sepsis was evaluated by a quick sepsis related organ failure assessment (qSOFA) score which was calculated by assigning one point of low blood pressure (systolic blood pressure ≤100 mmHg), high respiratory rate (≥22 breaths per min), or altered mental state for each. The main outcome was hospital mortality. All data were cross-checked by at least two clinicians or researchers.

The outcome of this study was hospital mortality. Clinical characteristics at ICU admission were compared between the survival and non-survival group. A nomogram for predicting hospital mortality in ICHs with severe pneumonia in the ICU was developed.

Statistical Analysis

The sample size (n=1,933) was derived on the basis of the available data, with mortality of 628, which is greater than the sample size evaluated by event per variables value of ten principal. Baseline characteristics were compared between survivors and non-survivors. Categorical variables were described as frequencies and percentages (%) and were compared using χ² tests. Continuous variables were expressed as median (IQR) and compared using Wilcoxon rank-sum test. Two-tailed P-value <0.05 considered the difference to be statistically significant.

We selected 40 candidate predictor variables based on their clinical value, existing literature, and statistical significance. Candidate variables with more than 20% missing data were excluded. For variables with less than 20% missing data, multiple imputations were performed for the missing values using the “mice” R package.¹⁹ The total patients were randomly split into a 70% training set to build the prediction model and a 30% testing set for independent internal validation using the “caret” R package. The training set was applied to develop a prediction model and the testing set was used to validate the performance of the model. In the training set, we used the least absolute shrinkage and selection operator (LASSO) method to select the candidate predictive features with “glmnet” R package. A 10-fold cross-validation method was applied to find the regularization parameter lambda, which gave the minimum mean cross-validated concordance index. Variables selected by LASSO method were included in the multivariate logistic regression analysis to identify independent risk factors. We use the “rms” R packages to develop the prediction model based on the result of multivariate logistic regression analysis. Finally, a nomogram for predicting hospital mortality as the outcome of interest will be presented based on the predicting model.

We validated the prediction nomogram both in the training set and testing set. The calibration,

discrimination, and clinical usefulness of the nomogram were calculated to evaluate its performance. The prediction accuracy was evaluated by using the Harrell’s concordance index (C-index) and its 95% CI, as well as the area under the curve (AUC) of the receiver operating characteristic (ROC) curve. A calibration curve was used to access the difference between the predicted value conducted by the nomogram and the actual value. Decision curve analysis (DCA) was performed to assess the clinical usefulness and benefits of the prognostic nomogram.

All statistical analyses were done on R Studio, version 4.0.5.

Results

Patient Selection

In total, 6,992 patients with severe pneumonia at ICU admission were extracted from the electronic medical records, 58 patients were excluded for being younger than 18 years old. In the remaining 6,934 patients, 2,122 were immunocompromised patients, including 1,450 patients (68.3%) with active solid tumor or hematological malignancy, 131 patients (6.2%) with transplantations, 696 patients (32.8%) with corticosteroid or biological immune modulators treatment, 33 patients (1.6%) with HIV infection, and 188 patients (8.9%) had more than one immunodeficient factor. For 2,122 ICHs with severe pneumonia, we excluded 45 patients for undergoing a cardiopulmonary resuscitation within the 24 hours before ICU admission or haing stayed less than 24 hours in the ICU, 14 patients for a long ICU stay of over 90 days because of repeated admission for medical settlement, and 130 patients for more than 20% variables being missing. In total, 1,933 patients were included in our analysis. The patients selection is presented in Figure 1.

Figure 1 Patients selection procedure.

Clinical Characteristics

Clinical characteristics at ICU admission between survival and non-survival cohorts are given in Table 1. Male gender was not significantly associated with death group (68.2% vs 65.1%, P=0.207). The median age of ICHs admitted in the ICU with severe pneumonia was 64 years (IQR=50–75), for those who died in the hospital it was 70 years (IQR=57–79) and for those who survived it was 61 years (IQR=48–72; P<0.001).

Table 1 Clinical Characteristics between Survival and Death Groups

Patients who presented with fever (32.5% vs 14.9%), cough (40.6% vs 27.2%), and dyspnea (26.8% vs 11.8%) were more common in the death group than the survival group (P<0.001).

The frequency of comorbidities with chronic renal disease (16.9% vs 6.0%) and chronic pulmonary disease (29.8% vs 21.1%) was significantly associated with the death group (P<0.001).

For vital signs, increased heart rate (102 vs 96 bpm, P<0.001), increased respiratory rate (20 vs 18 rate/min, P<0.001), decreased mean blood pressure (65.00 vs 67.67mmHg, p< 0.001), and decreased Pa02/Fi02 (147.66 vs 181.50, p<0.001) were associated with mortality. Among these, difference in mean blood pressure, heart rate and respiratory rate are small and these factors were unlikely to be clinically meaningful.

Compared to survive group, death group had elevated neutrophils (8.53 vs 7.32×10⁹/L, p<0.001), glucose (7.57 vs 6.89mmol/L, p<0.001), creatinine (81.00 vs 70.00 μmol/L, P<0.001), APTT (34.45 vs 31.00 sec, P<0.001), PT (13.50 vs 12.70 sec, P<0.001), lactic acid (1.60 vs 1.40 mmol/L, P<0.001), PCT (0.72 vs 0.31 ng/mL, P<0.001), and decreased hemoglobin (105.50 vs 113.00 g/L, P<0.001), platelet (137.00 vs 185.00×10⁹/L, P<0.001), lymphocyte (0.78 vs 0.97×10⁹/L, P<0.001), monocyte (0.38 vs 0.43×10⁹/L, P<0.001), and albumin (30.90 vs 32.90g/L, P<0.001). White blood cell count and uric acid had no differences between the two groups.

In our study, we found a higher proportion of death group patients needed a direct ICU admission (46.3% vs 34.3%, P<0.001), required invasive mechanical ventilation (99.5% vs 97.7%, P=0.007), and required vasopressors (85.4 vs 48.1%, P<0.001). Although the quick sepsis related organ failure assessment (qSOFA) score showed significance between the two groups (P<0.001), its 95% CI of 0–1 indicated that qSOFA at ICU admission cannot help predict the mortality in immunocompromised patients with severe pneumonia.

Patients were divided into a training set (n=1,353) and a testing set (n=580). The baseline characteristics between training and testing sets are presented in Table 2. There was no significant difference in these characteristics between the training set and testing set with all (all P>0.05).

Table 2 Clinical Characteristics in Training and Testing Set

Candidate Predictors Selection

On the basis of 1,353 patients being in the training set, 18 potential predictors were selected from 40 features by LASSO, including: age, fever, cough, dyspnea, chronic renal disease, chronic pulmonary disease, respiratory rate, heart rate, PaO₂/FiO₂ ratio, hemoglobin, platelet, neutrophils, lymphocyte, monocyte, APTT, direct ICU admission, the requirement for invasive mechanical ventilation, and the requirement for vasopressors (Figure 2A and B).

Figure 2 Feature selection by LASSO and cross-validation. (A) Coefficient trendlines of 40 variables for hospital mortality. The coefficient profile plot was created against the log (lambda) sequence. The predictor’s selection was according to the 1-SE criteria (B, right dotted line), where 18 nonzero coefficients were selected. (B) Tuning parameter (lambda) selection of deviance in the LASSO regression based on the minimum criteria (left dotted line) and the 1-SE criteria (right dotted line). Abbreviations: LASSO, least absolute shrinkage and selection operator; SE, standard error.

Then, we adopted eight factors through multivariate logistic analysis, with results reported as odds ratio (95% CI), shown in Table 3. Eight risk factors including: age (1.027 [1.018–1.036]), fever (1.957 [1.405–2.726]), dyspnea (2.107 [1.458–3.045]), chronic renal disease (3.468 [2.182–5.511]), PaO₂/FiO₂ ratio (0.997 [0.995–0.999]), platelet (0.997 [0.996–0.999]), neutrophils (1.048 [1.022– 1.074]), and the requirement for vasopressors (7.664 [5.494–10.692]) were independently associated with hospital mortality.

Table 3 Risk Factors Selected by Multiple Logistic Analysis

Development of Predicting Nomogram

Nomogram to predict hospital mortality of ICHs with severe pneumonia in ICU is presented in Figure 3. Each predictive factor was assigned with a single score which was presented on the top line of the nomogram. The total score of each patient is the sum of each single score. On the bottom of the nomogram, the probabilities of hospital mortality were predicted in terms of the total scores. As the nomogram shown, patients with higher age, higher neutrophils count, lower platelet counts, lower PaO2/FiO2 ratio and presented with fever, dyspnea, chronic renal disease and need for vasopressors can get a higher total point with a worse prognosis.

Figure 3 Nomogram to predict hospital mortality of immunocompromised patients with severe pneumonia in the ICU.

Validation of Predicting Nomogram

The ROC curve indicated the predictive accuracy of the nomograms (Figure 4A and B). AUC in the training set was 0.819 [0.808–0.698] and 0.819 [0.802–0.717] in the testing set. To further assess the discriminative ability of the model, we also calculated the C-index (Table 2) both in the training set (0.819 [0.795–0.842]) and testing set (0.819 [0.783–0.855]). Calibration plots indicated that the nomogram had well predictive accuracy for hospital mortality compared to the 45-degree ideal model (Figure 4C and D). The decision curve analysis showed the net benefits obtained from the application of our nomogram both in the training set and testing set (Figure 4E and F).

Figure 4 Performance evaluation of the nomogram in training and testing sets. (A and B) Receiver operating characteristic curve analysis in the training and testing sets, AUC, area under the curve. (C and D) Calibration curve analysis in the training and testing sets. The horizontal axis represents the nomogram-predicted probability of hospital mortality, and the vertical axis represents the actual observed hospital mortality. (E and F) Decision curve analysis (DCA) for the training and validation sets, implicating the net benefit with respect to the use of the nomogram.

Discussion

Based on the population we investigated, there is an approximately 30% burden of immunodeficiency status in critically ill patients with severe pneumonia. Active solid tumor is the major cause of immunodeficiency of nearly 60%, the second most common cause was immunosuppressive therapy. Nearly a third of ICHs with severe pneumonia in the ICU will develop a fatal outcome in hospitalization.

In our nomogram, eight independent risk factors were included for predicting hospital mortality of ICHs with severe pneumonia in the ICU, including age, fever, dyspnea, chronic renal disease, platelet counts, neutrophil counts, PaO₂/FiO₂, and the requirement for vasopressors. Among these predictors, the symptoms, medical history, and other clinical data can be obtained in the shortest time. Meanwhile, the laboratory results mainly cover the blood routine and oxygenation index, which can be obtained relatively quickly. The nomogram is expected to be easy-to-use for doctors to identify the risk factors of mortality, and might be helpful for them to make a quick initial diagnosis.

Several predicting tools for community-acquired pneumonia (CAP) patients like PSI,³ CURB-65,⁴ SCAP score,²⁰ NEWS-L,²¹ IDSA/ATS 2007,⁵ and SMART-COP²² had been approval to be clinically useful for the decision of ICU admission and prediction of mortality. The two most widely used severity assessment tools in CAP, the PSI, and the CURB65 score, perform well to predict 30-day mortality, but are less useful in identifying SCAP requiring ICU admission. The IDSA/ATS 2007 criteria predict both mortality and future requirements for mechanical ventilation and vasopressor support as a surrogate of ICU admission.²³ But none of these tools are mentioned to be useful for immunocompromised patients. For prognostic tools, nomograms are widely used in predicting an individual’s probability of a clinical event using individual variables, especially in oncology.²⁴ As far as we know, this is the first study to build and validate a nomogram for predicting hospital mortality of immunocompromised patients with severe pneumonia in the ICU. According to the evaluation results of the nomogram performance, our model shows a great predictive accuracy with both an AUC and C-index greater than 0.8 in the training and testing sets. The calibration analysis revealed that the predicted hospital mortality according to nomogram was similar to the actual hospital mortality. The nomogram showed good discriminatory ability and calibration. Additionally, the DCA presented that the net clinical benefits were positive with an approximately 80% probability in training set or greater in the testing set for immunocompromised patients with severe pneumonia, which showed the clinical usefulness of the novel nomogram.

Compared to other severity assessment tools in CAP, we found some of the risk factors in our nomogram are similar with factors in previous predictive tools. The requirement for vasopressor was thought to be the most important factor, with an OR of 7.664, which is similar to the major criteria of septic shock with the need for vasopressors in IDSA/ATS 2007 criteria the most pragmatic tool to predict ICU admission for severe community acquired pneumonia.²³ Chronic renal disease was considered the second most significant risk factor, with an OR of 3.468 in our predicting model, which can also be found in common with the minor criteria of blood urea nitrogen level ≥20 mg/dL in IDSA/ATS criteria and the blood urea nitrogen >7 mmol/L in CURB-65 score. Other factors like platelet count, PaO₂/FiO2 ratio, and dyspnea can also find similar expressions in the minor criteria in IDSA/ATS 2007 tools. Age was considered as an independent predictor of mortality in our model, which is not mentioned in the IDSA/ATS 2007 criteria. Differently, factors like fever and neutrophil counts seem to be contrary to the standard of core temperature <36°C and white blood cell count <4,000 cells/µL in IDSA/ATS 2007. Fever was considered as an independent risk factor for hospital mortality in ICHs with severe pneumonia. As an indicator of infection and inflammatory process, fever had been approved to have both protective and detrimental effects in critical infectious disease.²⁵ The survival benefit of fever is mainly performed by triggering a process of immune response and enhancing immune-protective mechanisms during infection.²⁶ However, in immunocompromised patients, the effect might be inefficient or adverse. For the controversial part, more research is needed in the future.

Our study has also investigated different risk factors of mortality in ICHs with severe pneumonia in ICH at the baseline. Factors like chronic cardiac, pulmonary, cerebral diseases, as well as diabetes are associated with higher hospital mortality. Other factors like ICU admission strategy and the need of IMV were related to fatal outcomes for severe pneumonia in ICHs. But none of these was thought to be an independent predictor in our study. Part of these findings are inconsistent with the results of some previous studies. In addition to older age and PaO₂/FiO₂ <100, a previous multinational perspective cohort study had reported that delayed admission to the ICU, day-1 SOFA, and undetermined ARF etiology were independently associated with hospital mortality in non-HIV ICHs with acute respiratory failure.¹⁸ Legrand et al²⁷ discoverd that other factors of neurologic, respiratory, or hepatic dysfunction, and acute noninfectious condition could predict a higher hospital mortality in neutropenic patients with severe sepsis.

Some limitations for interpreting these study results exist. Firstly, our findings are based on a singer-center, retrospective cohort, with selection bias and information bias for data missing that may influence the accuracy of the results and the extrapolation of the findings to other settings must be done cautiously. Secondly, some factors, such as SOFA, APACHII score,²⁸ and objective laboratory testing related to immune status like CD4 cell counts had been approved to be associated with mortality in a critically ill population. However, they were excluded in our study due to incomplete data. Therefore, it could not be estimated for the prognostic value of these factors. For lack of imaging data and antibiotic treatment information in our study, the CURB-65 score and PSI score could not be obtained, we cannot compare these prognostic models with ours. In our study, the predicting performance of nomograms was internally validated by a testing set. However, we lacked an external validation cohort to enhance the credibility and effectiveness.

Conclusion

In summary, our study developed and validated a novel prognostic nomogram based on eight risk factors including: age, fever, dyspnea, chronic renal disease, platelet counts, neutrophil counts, PaO₂/FiO₂, and the requirement for vasopressors, to predict the hospital mortality of severe pneumonia patients in the ICU with immunocompromised status. Validation showed good discriminatory ability and calibration, indicating that the nomogram was expected to be superior predictive clinically useful for stratification patients and might generally be applied after more external validations.