A Retrospective Cohort Study: Predicting 90-Day Mortality for ICU Trauma Patients with a Machine Learning Algorithm Using XGBoost Using MIMIC-III Database

Introduction

Trauma is a major cause of death in the United States, a worldwide public health issue with serious economic burdens, and an important cause of life expectancy loss.^1,2 It has been reported that trauma is the main cause of death in the first forty years of life. Trauma causes 4.4 million deaths annually, accounting for almost 8% of global deaths.^3,4 Patients with severe trauma usually require admission to the ICU, and trauma is a common disease in the ICU, with variable morbidity and mortality rates.⁵ For the assessment of trauma prognosis, several methods for assessing the severity of injury have been developed over the past decades. Common scoring systems include the Injury Severity Scale (ISS), Revised Trauma Score (RTS), and Trauma and Injury Severity Score (TRISS).⁶ Cook et al compared the Trauma Audit and Research Network (TARN) with the Trauma Mortality Prediction Mode (TMPM). TMPM should be considered a measure of injury severity.⁷

The above scores and models provide clinical importance, but these methods and scores require the assumption of independent and linear relationships between explanatory. The above scores and models and various modifications are evidence-based tools, and some research findings suggest that they may mislead doctors by misclassifying patients’ conditions. However, when there are collinearity, heteroscedasticity, high-order interactions, and nonlinear relationships between variables, the performance of these two types of models is poor.^8,9 Therefore, more valuable and accurate prognostic tools that are not limited to these assumptions are needed to achieve better patient outcomes and maximize resource utilization. The new machine learning technology performs better prediction than traditional prediction methods. Modern ICUs are rich in data through continuous patient monitoring. Advances in computer technology and the establishment of specialized databases such as MIMIC have helped more doctors recognize and focus on machine learning, and machine learning methods are gaining acceptance, providing opportunities for data science and machine learning.^10,11 In addition, machine learning can determine the combination of reliable prediction results by observing patients and automatically calculating important variables and empirical patterns based on a large number of variables.¹²

Machine learning (ML) “learns” models from past data to predict future data.¹³ Learning is one of the key processes in artificial intelligence. ML for predicting and extracting information from data is increasingly being applied in many different fields, from medicine to finance.¹⁴ Due to these algorithms, we can analyze the increasing amount of biological data generated by next-generation sequencing technology, infer the pathogenicity of genomic mutations or cell types,¹⁵ identify therapeutic targets, and design new therapeutic compounds. ML can also be used to extract information from EHR data to predict patients’ diagnosis, post hospitalization risk, or heart failure, and can also support the analysis of emerging diseases, such as the severity of results during COVID-19 infection.¹⁶ Many different statistical, probabilistic, and optimization techniques can be implemented as learning methods, such as logistic regression, artificial neural networks (ANN), K-nearest neighbors (KNN), etc. In this study, XGBoost, logistic regression, random forest, AdaBoost, multilayer perceptron neural networks (MLP), support vector machine (SVM), Light GBM, KNN, and Gaussian Naive Bayes (GNB) machine learning models were used to establish the model. MLP, a neural network (NN) model and other related algorithms used to predict the actual weight of pigs at different stages, which is a ML classification technique based on feedforward neural networks (FFNN), which consists of ordered layers similar to human neural processing.¹⁷ It consists of many neurons that act as processing components and are arranged in a fully connected stack order. As succinctly defined by Laudani et al, the internal architecture framework of FFNN is constructed in such a way that continuous neural layers and interconnections are created using the following guidelines: neurons in each layer are connected to all (and only) neurons in the next layer.¹⁸ In several studies, it is an effective tool in predicting biological activities in vivo.¹⁹ SVM learning is one of many ML methods. Compared with other ML methods, SVM is very powerful in identifying subtle patterns in complex datasets. SVM can be used for handwriting recognition, fraudulent credit card recognition, speaker recognition, and facial detection. SVM is a powerful method for building a classifier.²⁰ It aims to create a decision boundary between two classes that enables the prediction of labels from one or more feature vectors. The application of support vector machine learning in cancer genomics is a popular and successful undertaking.²¹ The KNN method is an instance-based learning method that stores all available data points (instances) and classifies new data points based on similarity measures. The basic idea of the KNN method is to assign new unclassified examples to the classes to which most of its K nearest neighbors belong. The kNN classifier classifies unlabeled observations by assigning them to the most similar labeled instances. Collect observed features for training and testing datasets.²² When the number of samples in the training dataset is large, this algorithm has been proven to be very effective in reducing misclassification errors. Another advantage of the KNN method compared to many other supervised learning methods such as SVM, decision trees, neural networks, etc. is that it can easily handle problems with class sizes 3 or larger. Extreme gradient enhancement (XGBoost) is a machine learning technique that can efficiently and flexibly process missing data and assemble weak prediction models.²³ XGBoost is widely recognized and highly praised in multiple data mining and prediction models.^24–27 Therefore, the purpose of our study was two: first, we attempted to compare the performance of the machine learning XGBoost model with eight other commonly used machine learning models in predicting 90-day mortality in MIMIC-III database trauma patients. Second, we compared the characteristics and efficacy of the traditional logistic regression model and the XGBoost model, obtained clinical features based on the XGBoost model, and explained them according to their importance. XGBoost, logistic regression, random forest, AdaBoost, MLP, GNB, SVM, KNN, and LightGBM models were established by using the training set. The area under the curve (AUC), sensitivity, specificity, positive predictive value, negative predictive value, F1 score, and decision curve analysis (DCA) curve of each model were compared to evaluate and compare the performance of the models. The optimal model was compared with logistic regression in the validation set and test set. Determine the importance of risk factors based on the best model and explain the model.

Materials and Methods

Sources of Data

Our study is based on a retrospective cohort study of a critical care database, Medical Information Mart for Intensive Care III (MIMIC-III, Version 1.4). MIMIC-III is a publicly available single-center intensive care database approved by the Institutional Review Committee of Beth Israel Deaconess Medical Center (BIDMC, Boston, MA, USA) and Massachusetts Institute of Technology (MIT, Cambridge, MA, US), which includes information on 46,520 patients admitted to various ICUs of BIDMC in Boston, Massachusetts from 2001 to 2012.^28,29 The patient dataset contains both static features (age, gender, etc.) and dynamic features (heart rate, blood pressure, etc.) and is suitable for the study of time-varying processes in trauma patients. This database includes demographic, vital signs, laboratory tests, fluid balance, and life state chart events; Document International Classification of Diseases and Ninth Revision (ICD-9) Code; Records the physiological data of the bedside monitor verified by ICU nurses per hour; And store written evaluations of radiology films by experts during the corresponding period. To establish a predictive model of mortality in ICU trauma patients, data from the MIMIC-III database were divided into a training set and an internal validation set.

One author obtained the qualification (Approval Code 48718265) to use the database after completing and passing the online course, and he was responsible for data extraction. Patient-related information in the database was anonymous, and personal informed consent was waived in this study. The study was reported according to the recommendations of the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) statement.³⁰

Study Population

In MIMIC-III, patients were diagnosed with trauma in the ICU. The inclusion criteria were patients aged between 18 and 89 years, who chose the first hospitalization for analysis when admitted to the hospital or ICU multiple times. The exclusion criteria were as follows: (1) no trauma diagnosis within 24 years after admission to the ICU, (2) length of ICU stay less than 1 day, and (3) lack of age or gender information. This study was reviewed and approved by the ethics committee of our institution.

Data Collection and Variable Extraction

The structure query language (SQL) with code in MIMIC Code Repository was used for extracting the raw data. R software (version 3.6.3) was used for processing the data.³¹ All variables included basic characteristics (age, sex, ethnicity, and admission type), comorbidities (congestive heart failure, hypertension, chronic pulmonary disease, renal failure, liver disease, rheumatoid arthritis, obesity, diabetes, and anemia), laboratory tests (anion gap, bicarbonate, creatinine, chloride, glucose, hematocrit, hemoglobin, platelet, potassium, prothrombin time, activated partial thromboplastin time, international normalized ratio, sodium, blood urea nitrogen, white blood cell counts, red blood cell counts, calcium, mean corpuscular volume, red cell distribution width, and vital signs (heart rate, respiratory rate, body temperature, pulse oxygen saturation, diastolic pressure, systolic pressure, and mean arterial pressure). The Glasgow Coma Scale (GCS) score, SOFA score, and SAPS II score were evaluated within the first 24 h after ICU admission. Comorbidity was assessed according to recorded International Classification of Diseases (ICD)-9 and ICD-10 codes,³² and the Charlson comorbidity index was calculated.³³ Variables were reported as the average value within 24 h admission to the ICU.

Data Analysis

A total of 5556 adult trauma patients in the MIMIC-III database, 3754 investigators met our inclusion criteria, with 645 deaths and 3100 patients surviving within 90 days. Patients were divided into two groups based on whether they died or survived within 90 days and variables were displayed and compared between groups. Continuous variables of the study were nonnormally distributed and reported as medians along with interquartile ranges (IQRs). Categorical variables are presented as numbers and percentages. Comparisons between groups were made using the Kruskal–Wallis test or Mann–Whitney U-test for continuous variables and chi-square or Fisher’s exact tests for categorical variables.

In the model construction stage, first, the traditional logistic regression model is carried out with significant variables determined by backward stepwise analysis with the chi-square test. Then we chose an entry probability of < 0.01 by the stepwise selection method. Second, MIMIC-III data were divided into two sets: 80% as the training set and 20% as the internal validation set. Models based on common ML classifiers including XGBoost, logistic regression (LR), random forest, AdaBoost, multilayer perceptron neural networks (MLP), SVM, Light GBM, k nearest neighbors (KNN), and Gaussian naive Bayes (GNB) models, were used to establish a 90-day death prediction model for ICU trauma patients. Receiver operating characteristic (ROC) curves were used to compare the prediction efficiency of these models. To further elucidate the performance of the model, calibration plotting and DCA were performed. The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and F1 score of each model were calculated to evaluate the value of the model. The model with the best performance was selected for comparison with the traditional logistic regression, and then the model with the highest overall diagnostic value was selected for further verification. In addition, the recursive feature elimination (RFE) algorithm based on Shapley Additive explanations (Shap) values was used to identify key features, making the model more suitable for clinical practice. All statistical analyses were conducted using Python (version 3.9.0, Python Software Foundation, www.python.org) and R software (The R software environment is available for download from [https://www.r-project.org/]). A two-tailed test was performed, and P < 0.05 was considered to reflect statistical significance.

Results

Baseline Characteristics

A total of 5556 adult trauma patients in the MIMIC-III database, 3754 investigators met our inclusion criteria (Figure 1), with 645 deaths and 3100 patients surviving within 90 days. Table 1 summarizes the comparison of baseline characteristics, vital signs, and laboratory parameters between the nonsurvivors and the survivors within 90 days. The median length of ICU stay of survivors was 2.64 days [1.66, 5.31] and that of nonsurvivors was 3.77 days [2.0, 7.89] (p<0.001). The nonsurvivors were older and had higher SAPS II and SOFA scores. Among the dead patients, age, sex, ethnicity, congestive heart failure, hypertension, chronic pulmonary, renal failure, diabetes, and anemia, RBC mean, MCV mean, RDW mean, BNU mean, sodium mean, PT mean, INR mean, APTT mean, platelet mean, hemoglobin mean, hematocrit mean, glucose mean, creatinine mean, bicarbonate mean, anion gap mean, BT mean, Respirate mean, Mean bp mean, Dias bp mean, Sys bp mean, SAPS II, SOFA, Elixhauser scores, GCS, MV group, and ICU stay showed significant differences between the two groups. (p<0.05). The median age for survivors was 54 while 77 for nonsurvivors. The male gender account for 2% of 65.645% in the survivor group while 58.605% in the nonsurvivor group. A total of 8.387% survivors presented with congestive heart failure while 22.791% nonsurvivors presented with congestive heart failure. A total of 3.742% survivors presented with hypertension while 10.233% nonsurvivors presented with congestive heart failure. A total of 10.355% survivors presented with chronic pulmonary obstructive disease while 16.279% nonsurvivors presented with chronic pulmonary obstructive disease. A total of 4.419% survivors presented with renal failure while 13.023% nonsurvivors presented with renal failure. A total of 13.484% survivors presented with diabetes while 20.93% nonsurvivors presented with diabetes. A total of 11.065% survivors presented with diabetes while 17.209% nonsurvivors presented with diabetes. The median RBC for survivors was 4.09 while 3.84 for nonsurvivors. The median RDW for survivors was 14.62 while 16.1 for nonsurvivors. The median BNU for survivors was 20.67 while 31.0 for nonsurvivors. The median value of sodium mean for survivors was 141.0 while 142.5 for nonsurvivors. The median value of PT mean for survivors was 14.4 while 15.5 for nonsurvivors. The median value of INR mean for survivors was 1.3 while 1.44 for nonsurvivors. The median value of APTT mean for survivors was 31.03 while 36.2 for nonsurvivors. The median value of platelet mean for survivors was 246.0 while 243.0 for nonsurvivors. The median value of hemoglobin mean for survivors was 12.6 while 11.6 for nonsurvivors. The median value of hematocrit mean for survivors was 36.3 while 34.25 for nonsurvivors. The median value of glucose mean for survivors was 36.3 while 34.25 for nonsurvivors. The median value of chloride mean for survivors was 152.0 while 171.0 for nonsurvivors. The median value of creatinine mean for survivors was 1.0 while 1.4 for nonsurvivors. The median value of bicarbonate mean for survivors was 26.0 while 25.0 for nonsurvivors. The median value of anion gap mean for survivors was 15.0 while 16.4 for nonsurvivors. The median value of BT mean for survivors was 37.45 while 37.39 for nonsurvivors. The median value of RESP mean for survivors was 19.85 while 21.56 for nonsurvivors. The median value of BP mean for survivors was 87.85 while 86.23 for nonsurvivors. The median value of Dias BP mean for survivors was 69.89 while 67.31 for nonsurvivors. The median value of Sys BP mean for survivors was 134.0 while 132.47 for nonsurvivors. The median value of SAPSII for survivors was 134.0 while 132.47 for nonsurvivors. The median value of SOFA for survivors was 134.0 while 132.47 for nonsurvivors. The median value of GCS for survivors was 134.0 while 132.47 for nonsurvivors. The median value of ICU stay for survivors was 5.31 while 7.89 for nonsurvivors. However, admission type, liver disease, rheumatoid arthritis, obesity, calcium mean, WBC mean, potassium mean, chloride mean, SpO2 mean, heart rate mean, and hospital stay showed no statistical significance between the two groups.

Table 1 Baseline Clinical Characteristics of Mimic-III Patients with Trauma Between Survivors and Nonsurvivors at 90 Days

Figure 1 The flow chart of data extraction. Abbreviations: MIMIC-III, Medical Information Mart for Intensive Care-III; ICU, intensive care unit.

Model Comparison

The XGBoost, logistic regression, random forest, AdaBoost, MLP, GNB, SVM, KNN, and LightGBM models were established with the training set; the AUCs of the testing set were 1.00, 0.858, 1.00, 0.884, 0.808, 0.809, 0.422, 0.898 and 0.722, respectively (Figure 2a). AUROC is one of the most used metrics and shows sensitivity against-specificity. AUROC was used in this study to measure the performance of the model. The AUCs of the validation set were 0.857, 0.807, 0.856, 0.857, 0.783, 0.781, 0.412, 0.722 and 0.707, respectively (Figure 2b). Other criteria included sensitivity, specificity, PPV, NPV, and F1-score (Table 2). Evaluate the sensitivity, specificity, sensitivity specificity gap, area under the subject working characteristic curve (AUC), positive predictive value (PPV), negative predictive value (NPV), and Matthews correlation coefficient (MCC) of the model. The sensitivity specificity gap is the linear distance between these two values and explains how far the model is from perfect predictive ability. Among all these models, the XGBoost model has the highest AUROC and accuracy in the training set. In addition, XGBoost’s accuracy in the internal validation set of the MIMIC-III database is second only to that of random forests. Therefore, the XGBoost algorithm was selected as the best model.

Table 2 Model Performance in the Training and Validation Sets

Figure 2 Receiver operating characteristic curves (ROCs) of the XGBoost, logistic regression, random forest, AdaBoost, MLP, GNB, SVM, KNN and LightGBM models. (a) Training sets. (b) Validation sets.

Features Selected in Models

The dataset consists of static and dynamic variables. The commonly used method for processing dynamic patient data is to extract features based on the entire duration of the patient’s hospitalization. In this analysis, we performed a different method of dividing the evolution of the original patient’s vital signs into multiple subsequences with fixed step sizes and window lengths. This method implicitly assumes that short-term patient history is more important for patient outcomes than long-term patient history. The XGBoost model and logistic regression included the most important features (Table 3 and Table 4), which were determined by the results of reverse stepwise regression analysis and were strongly correlated with 90-day mortality, P<0.05. The 10 most important variables in the XGBoost model were age, RDW mean, sodium mean, glucose mean, respiration mean, SAPS II, Elixhauser scores, MV group, hospital interval, and ICU interval. Red blood cell distribution width (RDW) has been used to predict mortality during infection and inflammatory diseases. SAPS II as well as Elixhauser scores have been proposed to be effective tools for patient evaluation. Meanwhile, the mean value of glucose level and respiration rate have also been shown to correlate with the condition of the patient.^34,35 In this study, these selected features were highly associated with the survival of patients.

Table 3 Features Selected in the Conventional Logistic Regression

Table 4 Features Selected in the XGboost Model

Model Evaluation and Validation

The calibration curve is used to determine the degree of consistency between the predicted probability and the observed results. R2 quantifies the goodness of fit of the model, with R2=1 indicating that the regression line fully fits the data. Among the machine learning models, the XGBoost model performs best. To further clarify the performance of the XGBoost model, calibration plots (Figure 3a) and DCA (Figure 3b)³⁶ were conducted. For simplicity, only XGBoost and logistic regression results are demonstrated. The XGBoost calibration curves performed well, and the XGBoost model had a greater net benefit in DCA than the logistic regression model. At the same time, we compared the ROC curve (Figure 4a and b) and learning curve (Figure 4c) of XGBoost in the test set and verification set. Other criteria included sensitivity, specificity, PPV, NPV and F1-score (Table 5). The XGBoost model performed well in terms of the AUROC, sensitivity and specificity in the two sets.

Table 5 XGBoost Model Performance in the Training and Validation Sets

Figure 3 (a) Calibration plots and (b) decision curve analysis (DCA) of the XGBoost model and conventional logistic regression prediction models. The XGBoost calibration curves performed well, and the XGBoost model had a greater net benefit in DCA than the logistic regression model.

Figure 4 Model evaluation and validation in the training and validation sets. Receiver operating characteristic curves (ROCs) of XGBoost. (a) Training sets. (b) Test sets. (c) Learning curve.

Model Interpretation

There is a new interpretation model in ML, called the SHapley Additive exPlanations (SHAP), was first articulated by Lundberg and Lee to explain individual predictions for both kernel-based approaches and tree-based models.³⁷ We use the SHAP value of XGBoost to calculate feature importance, which has the greatest discriminative ability in the validation queue. The Shap values in Figure 5 are evaluated in an internal validation set, outlining the impact of the feature on the final model. Figure 5 provides an overview of the (positive or negative) impact of factors on the XGBoost model. We found that hospital interval, SAPS II and age were the three major risk factors. In addition, Figure 6 illustrates the predicted results for three specific instances. The red and blue bars represent risk factors and protective factors, respectively. Longer lines indicate that the eigenvalues are more important. In the example in Figure 6a, the patient’s condition is moderate, and our model predicts a lower risk than the base value. In the example in Figure 6b, the model predicts a value of risk slightly higher than the base value. In the example shown in Figure 6c, the predicted risk value of the model is significantly higher than the base value, and the largest influence value is RESP mean, SAPS II and age.

Figure 5 Feature importance estimated using the Shapley Additive explanations (SHAP) values. The plot sorts features by the sum of SHAP value magnitudes over all samples. The color represents the feature value (red high, blue low). The x-axis measures the impact on the model output (right positive, left negative). Taking the feature SAPS II as an example, red points are on the right, whereas blue points are on the left. This means prediction scores will be smaller when patients have a lower SAPS II score. Abbreviations: SAPS II, simplified acute physiology scale II; RDW, red cell distribution width; MV, mechanical ventilation.

Figure 6 The predicted results for three specific instances. The red and blue bars represent risk factors and protective factors, respectively. Longer lines indicate that the eigenvalues are more important. This figure shows the explanation for a low-risk instance (a), a medium-risk instance (b) and a high-risk instance (c). Abbreviations: SAPS II, simplified acute physiology scale II; RDW, red cell distribution width.

Discussion

In our study, a 90-day mortality prediction model for trauma patients in the ICU was established based on nine machine learning algorithms. A total of 3745 patients are included in MIMIC-III in the final cohort, in which, 645 patients died after 90 days, and 3100 survived. Of the nine models, the XGBoost model has the best predictive performance in the test set and the validation set. Compared with traditional logistic regression, the calibration curves of the XGBoost model and DCA performed well. Using XGBoost as the main model, 10 characteristics were found to be closely associated with 90-day mortality in trauma patients. We found that hospital stay, SAPS II and age were significant influencing factors. In addition, ICU stay, RDW mean, respiratory rate mean, glucose mean, sodium mean, MV and elixhauser score were closely related to the prediction model. The XGBoost prediction model shows good performance and provides an accurate tool for predicting the 90-day mortality of trauma patients in the ICU. The Shapley Additive Interpretation (SHAP) value illustrates the predicted results, which makes our model clinically interpretable.

This study shows that the importance of the SAPS II score in the model is second only to hospital stay (Figure 5). A. ULVIK et al found that SAPS II was a predictor of 30-day mortality in ICU trauma patients. SAPS II was a stronger predictor than SOFA score and ISS in both univariate and multivariate analyses.³⁸ Our XGBoost model includes RDW, which is known to have shown good benefits in a variety of human diseases, with high negative predictive value for diagnosis. More importantly, RDW is now recognized as a strong independent mortality risk factor in a variety of diseases.³⁹ RDW also has good predictive value in trauma.⁴⁰ Increased RDW was found to be an independent predictor of 28-day mortality in patients with severe trauma by continuous monitoring of RDW.⁴¹ The models in the current study included glucose and sodium biochemical markers, which have been associated with prognosis in clinical practice, and Grant et al considered glucose change to be a highly accurate predictor of infection.⁴² Studies have shown that hyperglycemia is associated with increased mortality in critically ill patients.⁴³ Dominic et al ‘s study showed that 28-day mortality was higher in patients with serum sodium disorder after surgery in the MIMIC II database.⁴⁴ Mechanical ventilation (MV) is a supportive treatment method for patients with various forms of respiratory failure. Although mechanical ventilation has a wide range of applications and obvious benefits, it is not a risk-free intervention measure. At the same time, delayed weaning is associated with an increased risk of incidence rate, mortality, length of stay and discharge from long-term care institutions. For most patients (70%), detachment from mechanical ventilation is a simple process. This usually requires extubation after passing the first spontaneous breathing test (SBT). The remaining 30% of patients are a challenge for ICU doctors. Difficulties typically occur in patients with underlying causes such as chronic obstructive and restrictive pulmonary disease, heart failure, and neuromuscular disorders.⁴⁵ It has also been found that MV is associated with mortality of ICU patients with tauma, which is in agreement with the prediction of the model.⁴⁶ In machine learning algorithms, the interaction between variables is considered when important variables are selected to effectively extract prediction patterns from data. Most of the top 10 variables obtained by using XGBoost as the main model reflect the characteristics of critically ill patients. However, among the current characteristics, the Elixhauser score has rarely been explored in the ICU. Therefore, this study can also provide some enlightenment for further research.

Ahmed et al⁴⁷ used the MIMIC-III database to explore early mortality prediction models for surgical patients in trauma ICUs. Among the Deep FLAIM, GNB Model, Decision Tree Model (CART), KNN Model and Linear Discriminant Analysis (LDA) machine learning models, the results showed that deep-field models perform better than others. Another study used mechanical ventilation patient characteristics in the MIMIC-III database and compared seven machine learning models to establish an in-hospital mortality prediction model for ICU mechanical ventilation patients based on the XGBoost algorithm.⁴⁸ The AUC value of the XGBoost model was 0.8213, similar to the performance of XGBoost in our study. Among the important features, the first three are age, respiratory disease and SAPS II, two of which are consistent with our study. Therefore, our XGBoost model may have implications for predicting ICU patient mortality in other diseases.

Through further interpretation of the model, sample examples were used to predict and explain the results. As shown in Figure 6c, the patient was 72 years old with RDW 14.1, SAPS II 34 points, and respiratory rate 27 times per min. The predicted risk of death was significantly higher than the base value. Respiratory rate and SAPS II were more influential factors. Zhao et al found that coagulation disorders were the most important variable in predicting sepsis-induced coagulopathy.⁴⁹ By explaining the model with examples, the model can be explained in the clinic. A more intuitive understanding of the model’s practicality and interpretability is needed.

A number of model have been employed in the prediction of prognosis of patients in ICU.⁵⁰ For instance, four models (LR, SVM, NN, and TRISS) exhibited a similar high accuracy and sensitivity in predicting the survival of the trauma patients. In the test dataset, the NN model had the highest balanced accuracy and predictive specificity.⁹ In this study, we chose XGBoost model after comparing different models. Our results showed that the calibration curves of the XGBoost model and DCA performed well. The predicted results are illustrated by Shapley Additive explanations (SHAP) values, which makes our model clinically interpretable Our results were in agreement with previous studies, which showed that XGBoost model displayed good recall for the healthy recovery group.⁵¹ In this prior work, data was collected from 5871 patients. Then, two algorithms were created using the Extreme Gradient Enhancement (xGBT) machine learning model. The complete model showed 86% recovery recall, 30% chronic care recall, 67% mortality recall, and 80% complication recall; The short-term model applicable to ED shows a recovery recall rate of 89%, a chronic care recall rate of 25%, and a mortality recall rate of 41%.⁵¹ Wang et al also reported that the XGBoost algorithm is more effective and accurate than logistic regression in predicting mortality in patients with moderate to severe TBI. The XGBoost prediction method is beneficial for doctors to assess the high risk of poor prognosis in TBI patients.⁵² Combined with our findings, it is reasonable to believe that XGBoost algorism was superior to traditional logistic regression in predicting mortality of patients with trauma.

The main advantage of this study is that the XGBoost model was used for the first time to predict the 90-day mortality rate of MIMIC-III patients in ICU, and compared with traditional regression analysis and clinical scoring systems. However, this study has some limitations. Firstly, in addition to the patient’s clinical condition, the decision to send the patient to the intensive care unit may be the result of various other factors, including the doctor’s discretion, institutional policies and procedures, and the hospital’s capabilities, which may bias our predicted scores. Secondly, the data extracted from the MIMIC-III database is distributed over several years (2001–2012), during which there have been significant changes in the treatment of heart failure, which may weaken the application of our model. Thirdly, as a single-center study, the population is relatively small. Although the robustness of our column chart has been extensively tested through internal validation through bootstrap testing, it is still uncertain whether the results of this study can be applied to other populations, and further research is needed on more patients in various clinical settings to confirm our results. Fourthly, data is collected from patient’s medical records, and we rely on the accuracy of the records. Moreover, as this is a retrospective study, we cannot avoid selection bias. In addition, the proposed model is not intended to be validated by developing sets from databases or our clinical data. Nevertheless, we believe that the proposed model may help us further understand the prognosis of patients with trauma in ICU.

The highlights of our study are, first of all, that this is the first predictive model to predict 90-day mortality in trauma patients in the ICU. Second, we used 9 machine learning models for comparison, a 20% subset of the database for internal validation, ROC, and calibration graphs for evaluation of models.⁵³ However, there are several limitations in our study. First, this was a retrospective observational study based only on the MIMIC-III database. Most of the patients were white and there may be potential bias. Second, there are many variable features in the MIMIC-III database, so our study did not explore all the features, and some key features may be omitted. Third, the lack of external validation of our model may reduce the importance and evidence of the model. Therefore, further external validation and prospective studies are needed to evaluate the universal applicability of the model.

Conclusions

In conclusion, this study shows that the prediction performance of the XGBoost model is superior to that of other models by comparing 9 other algorithms. Ten clinical features were found to be closely associated with 90-day mortality in ICU trauma patients. Our study shows that the XGBoost model outperforms other machine learning models in predicting 90-day mortality in trauma patients. This simple yet powerful mortality prediction model can be used to assist clinicians in early identification of mortality risk factors and early intervention to reduce mortality, particularly in patients whose condition is rapidly deteriorating.