An Individualized Nomogram for Predicting Mortality Risk of Septic Shock Patients During Hospitalization: A ten Years Retrospective Analysis

Introduction

Septic shock is a severe life-threatening disease caused by abnormal low blood pressure and cell metabolism. The condition is aggravated by sepsis and is associated with a high mortality rate. A recent meta-analysis revealed that the 30-day mortality rate among patients with septic shock was 33.7% in North America, 32.5% in Europe, and 26.4% in Australia.¹ Therefore, early identification of septic shock patients at high risk of death and timely intervention could substantially reduce mortality.

According to the third international consensus on the definition of sepsis and septic shock jointly proposed by the Society of Critical Care Medicine and the European Society of Intensive Care Med, the sequential organ failure assessment (SOFA) score and the quick sequential organ failure assessment (qSOFA) score are the widely used score systems used to evaluate the severity of organ dysfunction in patients with sepsis and septic shock.² The systemic inflammatory response syndrome (SIRS) standard applied earlier has been replaced by SOFA and qSOFA scoring systems. The variables applied in the qSOFA scoring system are fewer, but it has the advantage of being simple and fast. However, its usage in evaluating the probability of mortality need to be clarified. SOFA scoring system is more accurate than the qSOFA scoring system in predicting in-hospital mortality of hospitalized patients after severe infection.³ However, the discrimination power of the SOFA scoring system in predicting the mortality risk of septic shock patients is low (less than 0.7).⁴

In recent years, the published predictive models of septic shock based on machine learning algorithms show good predictive performance.^5,6 However, clinical interpretation of machine learning models’ outcomes is often challenging.⁷ Therefore, it is necessary to develop a model with good predictive accuracy for the mortality of patients with septic shock. The model should also be easy to interpret. A nomogram based on the prediction model can show the contribution of each independent risk factor to death. Additionally, it facilitates the selection of individualized interventions according to the specific needs of each patient,⁸ improving the prognosis of the patients. The model also has huge clinical application potential because it is easy to use. In our previous study, a nomogram has been established to predict the mortality risk in patients with sepsis,⁹ but whether it could be used for death prediction in progressed status of sepsis (septic shock) remained unknown. In this study, a model for predicting the death risk of septic shock patients was established by incorporating several variables following hospitalization.

Materials and Methods

Inclusion and Exclusion of Patients

This was a retrospective study. The data for patients included in this study were obtained from their medical records at Affiliated Dongyang Hospital of Wenzhou Medical University, which was constructed under the support of Hangzhou Le9 Health Technology Co., LTD.¹⁰ Only patients hospitalized at the hospital between January 2013 and January 2023 were included in this study. Personal data that could identify the patients were removed from their medical records. Informed consent was waived by the Ethics Committee of Affiliated Dongyang Hospital of Wenzhou Medical University due to retrospective data analysis in this study. The protocol for this study was approved by the Ethics Committee of Affiliated Dongyang Hospital of Wenzhou Medical University (approval number: 2023-YX-033) and was conducted in accordance with the principles of the Helsinki Declaration and its subsequent amendments.

Septic shock was defined based on the third international consensus (Sepsis-3) in 2016. Particularly, septic shock was defined based on the need for vasopressor therapy to maintain the mean arterial pressure above 65 mmHg after adequate fluid resuscitation and serum lactic acid level ≥2.0 mmol/L.² The diagnosis for septic shock was made at emergency department before hospitalization. After receiving clinical intervenes, the patients were hospitalized into different departments according to involved infection sites. Patients with one of the flowing conditions were excluded: 1) With hematological diseases and decompensated liver cirrhosis (patients with leukemia and decompensated liver cirrhosis often have abnormal levels of blood and biochemical indicators); 2) With advanced tumors (patients with advanced tumors are often unwilling to undergo active treatment and respond differently to treatment compared with other patients); 3) Receiving renal replacement therapy before admission (patients on renal replacement therapy have abnormal creatinine levels, blood urea nitrogen levels, among others); 4) With hemorrhagic shock, anaphylactic shock, cardiogenic shock, or other shocks; 5) Under the age of 18 years old; 6) pregnant.

Research Variables Collection

Variables incorporated in the prediction model were all collected from the first examination results after admission, and whether patients died of septic shock during hospitalization was obtained from their hospital discharge records. The presence of common infections sites for sepsis (including abdomen, lung and urinary tract) were obtained from electronic records. The specific variables researched in this study included: the patient’s gender and age, creatinine, the lactic acid and potential of hydrogen (pH) level, prothrombin time, international normalized ratio, serum cholinesterase, C-reactive protein, and procalcitonin levels, systolic blood pressure, diastolic blood pressure, mean arterial pressure, heart rate, respiration, white blood cells, hemoglobin level, hematocrit level, platelet count, pro-brain natriuretic peptide (pro-BNP) level, blood urea nitrogen level, blood phosphorus level, blood magnesium level, total bilirubin level, albumin level, and the presence of respiratory failure (defined as PaO2/FiO2 <300).¹¹ The primary outcome indicator of this study was whether patients with septic shock died or not during hospitalization.

Establishment and Evaluation of the Predictive Model

Data were analyzed using the R software (version 4.2.2). Continuous variables with non-normal distribution were expressed as median (quartile), or as mean ± standard deviation if with normal distribution, while categorical variables were expressed as number with percentages. P<0.05 was considered statistically significant.

The relevant variables were processed as described in our previous work.⁹ Briefly, missing data were interpolated through multiple imputations (Figure S1). Secondly, factors related to the prognosis of septic shock patients were selected by univariate and multivariate analysis, and those with non-linearity to logitp or with multicollinearity among the enrolled variables were excluded. The logistic regression model was evaluated for the predictive power, a goodness of fit and net clinical benefit as previously.⁹ Finally, a nomogram graph was constructed.

The accuracy of the logistic regression model established in this study was compared with those methods established by the SOFA scoring and machine learning models. In the machine learning model, four methods, including random forest, support vector machine (SVM), extreme gradient boosting (Xgboost), and decision tree, were selected to establish the models separately.¹² Subsequently, these four machine learning methods were integrated to establish an ensemble model,¹³ and the performance of the integrated model and other models was compared using the “roc.test” function in the “pROC” package.

Results

Basic Information of the Included Population

A total of 1253 septic shock patients, including 749 females and 504 males, were included in this study. Among them, 384 (30.6%) died. The patients were divided into the modeling group (878 cases, 272 deaths) and the validation group (375 cases, 112 deaths). No significant difference was observed in baseline characteristics between the two groups (P>0.05) (Table 1). In the enrolled patients, lung infection was the most common site, accounting for 37.5% in total cases, following by abdominal infection (22.3%) and urinary tract infection (17.9%).

Table 1 Baseline Characteristics of the Modeling Group and Validation Groupa

Variable Screening and the Establishment of a Logistic Regression Model

Univariate analysis of specific variables for patients in the modeling group revealed that twelve variables (age, gender, creatinine, lactic acid, PH, serum cholinesterase, pro-BNP, blood urea nitrogen, blood phosphorus, blood magnesium, total bilirubin, and respiratory failure) were associated with the prognosis of patients with septic shock (P<0.001) (Table 2). However, the infection site was not significantly associated with mortality (P>0.05). A linear distribution between logitp and continual variables was observed (P>0.05) (Table S1). In addition, no multicollinearity was observed among the variables associated with the prognosis of septic shock patients (VIF value less than 5, Table S2).

Table 2 Univariate Analysis Between Survivors and No Survivors in Modeling Groupa

Logistic regression analysis and stepwise regression analysis in both directions showed that eight variables, including age (OR 1.0199), lactic acid (OR 1.0703), PH (OR 0.1073), serum cholinesterase (OR 0.9998), blood phosphorus (OR 2.3313), blood magnesium (OR 15.7867), total bilirubin (OR 1.0091) and the presence of respiratory failure (OR 7.3364) were independent factors for themortality risk of patients with septic shock (see Table 3 for details).

Table 3 Stepwise Regression Analysis of Independent Risk Factors for Death in Patients with Septic Shock

The Establishment of a Nomogram

Factors incorporated in the logistic regression prediction model were used to construct the nomogram (Figure 1). When using the nomogram, a vertical line was drawn upwards from each variable to the top of the graph, and corresponding points were recorded. The score corresponding to the point of each variable was summed to generate the total score. The death prediction probability corresponding to the bottom of the nomogram charts was obtained based on the total score. Taking an example of a patient with septic shock and with the following variables at admission: age (80 years old), lactic acid (9.1 mmol/L), pH (7.0), serum cholinesterase (2106.0 U/L), blood phosphorus (2.59 mmol/L), blood magnesium (0.57 mmol/L), total bilirubin (34.1 μmol/L), and with respiratory failure, the patient’s total score is 3.79, and his corresponding probability of death is 0.935.

Figure 1 The nomogram of established model for predicting mortality risk in septic shock patients during hospitalization. The enrolled variables were collected for the first time after admission. A patient was displayed as an example, with detailed enrolled variables labelled by red dots. The variables labelled of asterisk indicated significance in the model, *P < 0.05, **P < 0.01, ***P < 0.001.

Evaluation of the Prediction Model in the Modeling and Validation Population

The area under the ROC curve of our model in the modeling population was 0.881 (Figure 2A), indicating that this model had a good discrimination ability. The P value of the calibration diagram was 0.889, suggesting that the model had a good goodness of fit (Figure 3A). The DCA curves of this model were far away from both extreme curves, suggesting its good clinical applicability (Figure 4A).

Figure 2 ROC curves for the logistic model with the modeling groups (A) and the validation groups (B).

Figure 3 Calibration curves for the modeling groups (A) and the validation groups (B).

Figure 4 Decision-curve analysis for the modeling groups (A) and the validation groups (B).

The area under the ROC for the validation population was 0.868 (Figure 2B), the P value of the calibration diagram was 0.900 (Figure 3B). DCA curves were far from both extreme curves (Figure 4B), indicating that the prediction model had good discrimination, goodness-of-fit and clinical applicability.

Comparison to the SOFA Scoring Model and Machine Learning Model

The AUC of the model established by incorporating the SOFA scoring model for the validation set was 0.799 (95% CI: 0.744–0.854) (Figure 5). The AUCs of the random forest, SVM, Xgboost, decision tree, and ensemble model for the machine learning model were 0.865, 0.837, 0.834, 0.835, and 0.863, respectively (Figure 5). Based on the AUCs, the discrimination power of the logistic model was significantly higher than those of the SOFA scoring model, SVM, Xgboost and decision tree model, but was comparable to that of the models established based on random forest and ensemble model (Table S3).

Figure 5 ROC curves for the logistic model, sofa model and five machine learning models to predict in-hospital mortality risk. Abbreviations: Sofa, sequential organ failure assessment; SVM, Support Vector Machine; Xgboost, extreme gradient boosting.

Discussion

Herein, we constructed a model to predict the mortality risk of patients with septic shock by incorporating several parameters. The independent factors related to the prognosis of septic shock included age, elevated lactic acid, blood phosphorus, blood magnesium, total bilirubin, low PH, serum cholinesterase, and the occurrence of respiratory failure. The nomogram established in this study could accurately discriminate high and low death risk in septic shock patients, with good calibration and clinical application potential. Thus, it could be applied to formulating customized clinical interventions for septic shock patients, improving their prognosis.

Septic shock is a systemic dysfunction of multiple organs after infection. This study showed that respiratory failure increased the risk of death in patients with septic shock, consistent with previous findings.^14,15 Moreover, our results revealed that higher total bilirubin levels and low serum cholinesterase activity increased the risk of death in patients with septic shock. Total bilirubin and serum cholinesterase are mainly produced in the liver. Liver function impairment that disrupts bilirubin excretion causes hyperbilirubinemia.¹⁶ Previous studies have shown that hyperbilirubinemia increases the mortality risk in patients with severe sepsis and septic shock.¹⁷ A recent study showed that low serum cholinesterase in patients with septic shock is closely related to high mortality risk,¹⁸ consistent with our findings.

Lactic acid is one of the main indicators of oxygen metabolism, and a higher lactic acid level often indicates circulation or respiratory disorders in patients. Multiple studies related to septic shock have demonstrated that hyper-lactacidemia can significantly increase patients’ mortality risk,^19,20 consistent with our findings. Since the severity of acidosis is mainly evaluated by blood pH, the degree of acidosis increases gradually when the acid released by the body exceeds the body’s buffer capacity.²¹ According to previous data, higher acidosis increases the mortality risk of patients.^22,23

Patients with septic shock are prone to phosphate disorders and magnesium ion metabolism disorders. Our study showed that hyperphosphatemia increases the mortality risk of patients. Severe infection decreases the digestion and absorption of blood phosphorus and increases renal excretion and intracellular phosphorus transfer, aggravating hypophosphatemia.²⁴ On the other hand, patients with septic shock also develop hyperphosphatemia, possibly due to renal insufficiency, iatrogenic phosphate loading, hemolysis, and rhabdomyolysis or lactic acidosis, and other reasons.²⁵ In addition, previous studies have found that hyperphosphatemia increases the mortality risk of patients with septic shock, while hypophosphatemia exerts the opposite effect.^25–27 Magnesium is a cofactor for hundreds of enzymes and modulates the immune response, particularly under severe infection.²⁸ Magnesium level is a novel prognostic indicator. A study showed that lower magnesium levels during hospitalization increased the risk of death of septic shock syndrome patients.²⁹ However, other studies have shown that the mortality rate is higher in hypermagnesemia patients than hypomagnesemia group and those with normal magnesium levels.³⁰ In this study, hypermagnesemia, rather than hypomagnesemia, increased the risk of death of septic shock patients.

Although septic shock is a progressed status of sepsis, they share common mortality-related predictive factors, including respiratory failure, total bilirubin, cholinesterase and lactic acid as described in our previous study.⁹ The different variables enrolled in the final models reflect the disease-status specific factors in predicting the mortality risk. The patients in this study were diagnosed in the emergency department, and several septic shock related variables have been corrected by emergent clinical intervenes, including mean arterial pressure and procalcitonin. Thus, the first indicators following hospitalization from emergency department might not reflect the original status of septic shock, but they could still be adopted to efficiently predict the mortality risk.³¹

Moreover, the original infection at different anatomic sites owns varied ability to result into sepsis or septic shock, which could be explained by site-specific inflammatory factors and metabolite.³² Although several studies have reported the close relationship between the infection origin and prognosis in sepsis patients, different patient inclusion criteria and comorbidities would have non-negligible impact on the observed endpoint event.^32–36 In this study, the infection site was not associated with mortality, which could be partly explained by the late disease status of involved patients (septic shock) in this study other than the early status in previous data (sepsis).^37,38

Many scoring tools to evaluate the prognosis of patients with sepsis or septic shock, such as SOFA, qSOFA, and SIRS standard, exist.² According to previous studies, the SOFA scoring system is more accurate than the qSOFA scoring system and SIRS standard for predicting in-hospital-related mortality of patients after infection. However, the overall accuracy of the SOFA scoring system in predicting the mortality of patients is suboptimal.^3,4 According to a previous report, a nomogram for predicting the mortality risk of patients with severe sepsis and septic shock within 28 days after hospitalization exists, but the predictive accuracy of this model is limited to certain populations.³⁹ The prediction power of the SOFA scoring model for septic shock patients was lower than that of the model established in this study. In addition, a previous study shows that the machine learning model accurately predictthe risk of death of patients with septic shock.⁴⁰ The prediction performance of this model was as good as those of the random forest and ensemble models and better than those of the SVM, Xgboost, and decision tree models. Considering that this model better reflects the contribution of included variables on the outcomes, it is easier to explain the results to the patients and family members.

Limitations of this study: (1) because the data included in this study were collected from a single center, the application of the developed model to other populations could be limited. (2) This was a 10-year retrospective study, and data on some indicators, such as blood zinc ions, prealbumin, and thrombocytocrity for more than 10% of the study participants, was missing. (3) Other infection sites such as central nerve system and cutaneous tissue were not obtained during data collection, and whether the presence of infections at multiple anatomic sites was missing. (4) Septic shock was diagnosed in the emergency department and nearly all the patients have received intervenes, resulting into limited significance for these patients without treatment before hospitalization.

Conclusion

In this study, a model for predicting the mortality risk of septic shock patients during hospitalization was developed. A nomogram was used to describe the contribution of each included variable to the outcomes. The model can effectively predict the mortality risk of patients with septic shock during hospitalization. Thus, it could assist in selecting the best-individualized treatments, which could reduce the mortality rate among septic shock patients.