Ten-Year Multicenter Retrospective Study Utilizing Machine Learning Algorithms to Identify Patients at High Risk of Venous Thromboembolism After Radical Gastrectomy

Introduction

The incidence of gastric cancer, which ranks as the second highest among all malignant tumors, is increasing yearly due to changes in lifestyle and dietary habits.¹ Patients are frequently diagnosed in advanced stages, resulting in a poor prognosis.² Surgery remains a primary approach to treating gastric cancer.³ Advances in minimally invasive surgical techniques, such as laparoscopic and robotic surgery, have reduced postoperative recurrence rates and minimized patient trauma, thereby enhancing survival and quality of life.^4,5 However, given the intricate anatomy of the stomach and the distribution of surrounding lymph nodes, as well as the high technical demands of radical gastrectomy, life-threatening complications may occur, including anastomotic leakage (AL) and venous thromboembolism (VTE).^6,7 VTE is a serious complication in cancer patients, causing significant noncancer-related mortality.⁸ Although more common in patients with malignancies such as pancreatic, brain, lung, and ovarian cancers, postoperative patients with gastric cancer are also at risk for this complication.⁹ VTE can have serious consequences, including respiratory distress, heart failure, and death, if the thrombus travels to the lungs. In the lower limbs, VTE can cause edema, pain, and even skin ulceration if it affects the deep veins.¹⁰ This complication prolongs hospitalization and imposes a significant financial burden on patients and their families. Prophylactic treatment can effectively reduce the risk of patient death by 50% in high-risk patients.^11,12 Accurate prediction of postoperative VTE in patients undergoing radical gastrectomy and identification of high-risk patients is therefore crucial.

Surgeons often rely on their previous clinical experience to assess the risk of VTE in surgical patients, but this method is often unreliable due to the subjective nature of the surgeon’s experience and its temporal limitations. Ten years ago, some researchers utilized parametric regression methods to predict VTE in surgical patients preoperatively, which were relatively accurate. However, due to the complex relationships between clinical characteristic variables and the multifactorial nature of postoperative complications, using regression models cannot solve the problem of predicting disease. Moreover, when the number of cases for variables in the regression model is small, the predicted results can be highly variable, which limits the usefulness of the regression model.^13–15 In recent years, artificial intelligence (AI) has rapidly advanced in the medical field, with machine learning being a major branch of AI that offers more stable model building and accurate predictions, making it popular among clinicians for clinical prediction and other applications.^16,17 Here, we analyzed the clinical information of gastric cancer patients and used machine learning algorithms to develop a prediction model for VTE after radical gastrectomy. This model can identify high-risk patients for VTE without relying on conventional imaging examinations such as abdominal computed tomography (CT), which can reduce medical costs and assist clinicians in providing timely and precise individualized treatment plans.

Materials and Methods

Study Subjects

In this study, we utilized clinical data obtained from the databases of two medical institutions, namely, Wuxi People’s Hospital affiliated with Nanjing Medical University and Wuxi Second People’s Hospital. Case inclusion criteria were as follows: (1) patients aged 18–80 years with gastric cancer; (2) patients undergoing open radical gastrectomy or laparoscopic-assisted radical gastrectomy; and (3) the surgical team consisted of senior surgeons with the ability to independently perform radical gastrectomy. Case exclusion criteria were as follows: (1) patients with other malignant tumors; (2) patient with a previous diagnosis of VTE; (3) patients diagnosed with hemophilia, thrombocytopenic purpura, vascular purpura, and other disorders characterized by a tendency towards bleeding or abnormal bleeding; (4) patients diagnosed with hematologic conditions such as vascular purpura, platelets, and coagulation factors; (5) patients diagnosed with vital organ conditions such as liver and kidney diseases that are intolerant to surgical interventions; (6) patients on anticoagulant medications such as warfarin and heparin, anti-platelet medications such as aspirin and clopidogrel, and non-steroidal anti-inflammatory drugs such as ibuprofen during the perioperative period; and (7) patients with missing cases, clinical data, or visits. All patients in the study were followed up for at least 3 years after surgery. This study was conducted in accordance with the Declaration of Helsinki and was approved by the Ethics Committee of Wuxi People’s Hospital and Wuxi Second People’s Hospital, with approval number KY22085.

Study Design and Data Collection

Clinical data on gastric cancer patients between 2010 and 2020 were obtained from the databases of Wuxi People’s Hospital and Wuxi Second People’s Hospital. The data included 42 preoperative variables (within 24 h before the day of surgery), intraoperative variables, and postoperative variables (occurring 48 h after the initial surgery). Preoperative variables collected included patient demographic characteristics (gender, age, history of smoking, alcohol abuse, and body mass index), basic clinical characteristics (American Society of Anesthesiologists score, nutrition risk screening 2002 score, history of surgery, adjuvant chemotherapy history, adjuvant radiotherapy history, and use of central venous catheters), basic medical history (anemia, ileus, ulcerative colitis, Crohn’s disease, diabetes mellitus, hypertension, chronic obstructive pulmonary disease, hyperlipidemia, and coronary artery disease), laboratory tests (albumin, carcinoembryonic antigen and carbohydrate antigen 19–9), and tumor characteristics (T-stage, N-stage, peripheral nerve invasion, tumor size, and tumor number). The intraoperative variables collected included the type of surgery, surgical approach, number of lymph nodes dissected, duration of surgery, intraoperative bleeding, intraoperative blood transfusion, intraoperative percutaneous arterial oxygen saturation status, abdominal drainage, whether the patient experienced intraoperative tachycardia, and whether the surgery was an emergency procedure. Postoperative variables collected included laboratory test indices (neutrophil to lymphocyte ratio, procalcitonin, C-reactive protein, and serum amyloid A). The outcome variable of the study was postoperative VTE.

Development and Evaluation of Predictive Models

In the present study, statistical analysis was carried out using SPSS software and R software. The following steps were taken for the construction and evaluation of the clinical prediction models:¹ Data preprocessing. The dataset used in this study comprised patients diagnosed with gastric cancer at Wuxi People’s Hospital between January 2010 and January 2020. This dataset was employed for building the predictive model. The establishment set was randomly divided into a training set (70%) and a test set (30%). A separate dataset of gastric cancer patients from Wuxi Second People’s Hospital during the same period was used as an external validation set to evaluate the model’s performance.² Univariate and multivariate regression analyses were conducted on the model establishment set’s data. The chi-square test was utilized to compare the differences between two groups for categorical variables, while a t-test was applied for continuous variables that followed a normal distribution. For continuous variables that did not conform to a normal distribution, the rank sum test was employed. A p value of less than 0.05 was considered statistically significant. Logistic regression analysis was performed for variables that were significant in the univariate analysis to obtain independent influences on postoperative VTE. Four models, extreme gradient boosting (XGBoost), random forest (RF), support vector machine (SVM), and k-nearest neighbor algorithm (KNN), were used to score the importance of each factor and rank them according to the importance of the weight of the influencing factor. The variables that ranked in the top ten in all four model rankings and were meaningful in univariate and multivariate analyses were selected.³ Evaluate and build prediction models. The filtered clinical variables were incorporated into four machine learning algorithms, including SVM, RF, XGBoost, and KNN. The four models were evaluated using three criteria: discrimination, calibration, and clinical utility, with the best performing model being selected for further analysis. A receiver operating characteristic (ROC) curve was used to calculate the area under the curve (AUC) value and gauge the model’s predictive ability. The calibration curve was plotted to assess the agreement between the predicted and actual results, and decision curve analysis (DCA) was performed to determine the benefit to patients from interventional treatment. Internal validation was conducted through a k-fold cross-validation methodology.⁴ External validation of the best model. The generalizability and predictive efficiency of the model was assessed by applying it to an external validation set and plotting ROC curves.⁵ Model interpretation. The contribution of each feature in the sample to the prediction was obtained through Shapley value-based Shapley additive explanation (SHAP). The ranking of risk factors’ importance was depicted through the SHAP summary plot, and the prediction results of individual samples were analyzed and interpreted through the SHAP force plot.

Results

Clinical Information of the Patients

In this study, we included a total of 1239 patients diagnosed with gastric cancer, among whom 107 patients (8.64%) developed postoperative VTE (Table 1 and Figure 1). The original data presented in the study are included in Table S1.

Table 1 Characteristic Distribution of Data in the Establishment Set and External Validation Set

Figure 1 Flow diagram of patients included in the study. Abbreviation: VTE, venous thromboembolism.

Screening for Risk Factors for Postoperative VTE

The results of univariate and multivariate analyses showed that body mass index (BMI), history of adjuvant radiotherapy, history of adjuvant chemotherapy, albumin (ALB), use of central venous catheters (CVCs), history of hypertension, emergency surgery, duration of surgery, intraoperative bleeding, T-stage, N-stage, and neutrophil to lymphocyte ratio (NLR) were independent influencing factors for VTE (P<0.05) (Table 2). XGBoost, RF, SVM, and KNN algorithms screened for risk factors influencing postoperative VTE included BMI, history of adjuvant radiotherapy, history of adjuvant chemotherapy, use of CVC, duration of surgery, intraoperative bleeding, T-stage, N-stage, and NLR (Figure 2A–D). After comprehensive analysis, the risk factors included in this prediction model included BMI ≥25 kg/m², history of adjuvant radiotherapy, history of adjuvant chemotherapy, use of CVC, surgery time ≥270 min, intraoperative bleeding ≥100 mL, T-stage, tumor lymph node metastasis, and NLR ≥3.

Table 2 Univariate and Multivariate Analyses of Variables Related to Postoperative VTE

Figure 2 The variable influence factor ranking plots of the four models. (A) Variable importance ranking diagram of the XGBoost model. (B) Variable importance ranking diagram of the RF model. (C) Variable importance ranking diagram of the SVM model. (D) Variable importance ranking diagram of the KNN model. Abbreviations: BMI, body mass index; ASA, PNI, peripheral nerve invasion; NLR, neutrophil to lymphocyte ratio; CVC, central venous catheter.

Model Building and Evaluation

The ROC curve results show that XGBoost has an AUC value as high as 0.989 in the training set; the AUC value in the validation set is 0.912, which is the best performance among the four models (Table 3). The Brier score is a significant measure of the accuracy of model prediction, and a small Brier score for all four models signifies an accurate prediction. In this study, the slight deviation of the calibration curves of all four models from the diagonal line suggests their excellent calibration performance, and their predicted probabilities can be relied upon for making decisions. The net benefit curves of all four models showed an “S” shaped pattern, indicating an increase in net benefit with an increase in the probability threshold, followed by a decline after reaching the highest point. The XGBoost model showed the high net benefit among all models, and the optimal threshold point was located within the probability range of the actual disease state (Figure 3A–D). The generalization ability of the four models was compared using k-fold cross-validation. In this study, the test set comprised N=262 cases (30.01%), and the remaining samples were used for 10-fold cross-validation. The XGBoost algorithm exhibited an AUC value of 0.9036±0.0566 in the validation set and an AUC value of 0.9212 in the test set, with an accuracy of 0.8855 (Figure 4A–C). The RF algorithm displayed an AUC value of 0.8820±0.0511 in the validation set and an AUC value of 0.8485 in the test set, with an accuracy of 0.7786. The SVM algorithm showed an AUC value of 0.8788±0.0704 in the validation set and an AUC value of 0.8788 in the test set, with an accuracy of 0.8702. Finally, the KNN algorithm exhibited an AUC value of 0.8263±0.1192 in the validation set and an AUC value of 0.8065 in the test set, with an accuracy of 0.9046. After a comprehensive comparison, the XGBoost algorithm was selected to construct the model in this study.

Table 3 Evaluation of the Four Models

Figure 3 Evaluation of the four models for predicting VTE. (A) ROC curves for the training set of the four models. (B) ROC curves for the validation set of the four models. (C) Calibration plots of the four models. The 45-degree dashed line in each plot represents the ideal correspondence between the predicted (x-axis) and observed (y-axis) probabilities of complications. The closer the distance between the two curves, the higher the predictive accuracy. (D) DCA curves of the four models. The point of intersection between the red curve and the “All” curve represents the baseline or starting point, while the point of intersection between the red curve and the “None” curve indicates the decision node where the corresponding patients may derive benefit. Abbreviations: XGBoost, extreme gradient boosting; SVM, support vector machine; KNN, k-nearest neighbor algorithm.

Figure 4 Internal validation of the XGBoost model. (A) ROC curve of the XGBoost model for the training set. (B) ROC curve of the XGBoost model for the validation set. (C) ROC curve of the XGBoost model for the test set. (D) External validation of the XGBoost model. Abbreviations: AUC, area under the curve; XGBoost, extreme gradient boosting.

Model External Validation

The external validation set yielded an AUC value of 0.85, suggesting that the disease prediction model was characterized by a high degree of accuracy (Figure 4D).

Model Explanation

The SHAP summary plot results demonstrated that the risk factors for VTE after radical gastrectomy were prioritized as follows: surgery time ≥ 270 min, usage of CVC, intraoperative bleeding ≥ 100 mL, history of adjuvant radiotherapy, NLR ≥ 3, gastric cancer of T3 and T4, history of adjuvant chemotherapy, tumor lymph node metastasis, and BMI ≥ 25 kg/m² (Figure 5). The SHAP force plot presents the results of predictive analysis for three patients with VTE using the study model. For patient one, the model predicted disease occurrence with a probability of 0.029, with increased probability associated with gastric cancer of T3 and T4, use of CVC, and surgery time ≥270 min and decreased probability associated with a history of adjuvant chemotherapy. For patient two, the model predicted disease occurrence with a probability of 0.374, with increased probability associated with gastric cancer of T3 and T4, use of CVC, and surgery time ≥ 270 min. For patient three, the model predicted a disease occurrence probability of 0.748, with increased probability associated with NLR ≥ 3, use of CVC, gastric cancer at T3 and T4, and surgery time ≥ 270 min (Figure 6A–C).

Figure 5 SHAP summary plot. The risk factors are ranked on the y-axis according to their significance, which is determined by the mean of their absolute Shapley values. The higher the risk factor appears on the plot, the more crucial it is for the model. Abbreviations: BMI, body mass index; ASA, NLR, neutrophil to lymphocyte ratio; CVC, central venous catheter.

Figure 6 SHAP force plot. The explanatory variables are ordered along the horizontal axis based on the absolute value of their impact, with blue representing features that negatively affect disease prediction, as indicated by a decrease in SHAP values, and red representing features that positively affect disease prediction, as indicated by an increase in SHAP values. (A) Predictive Analysis of Patient I. (B) Predictive Analysis of Patient II. (C) Predictive Analysis of Patient III. Abbreviations: NLR, neutrophil to lymphocyte ratio; CVC, central venous catheter.

Discussion

In this study, we developed risk prediction models using four machine learning algorithms. Despite each algorithm having unique characteristics and potential for various scenarios, we evaluated and determined that the XGBoost algorithm demonstrated the highest accuracy, better stability, and superior generalization ability. Compared to the RF algorithm, the XGBoost algorithm allows for easy assessment of the feature contribution to the model by ranking the importance of the features, providing more intuitive output of the model’s conclusions. Additionally, the XGBoost algorithm offers more options for parameter tuning, which enhances the model’s performance control.¹⁸ The SVM and KNN algorithms have high model complexity, potentially hindering performance on large datasets.¹⁹ Conversely, the XGBoost algorithm has lower model complexity and is better suited for multidimensional studies, reducing computational effort and training time. After comprehensive comparison of the four algorithms, we chose the XGBoost algorithm to construct a model predicting postoperative VTE formation in patients in this study.

Several studies^20,21 have demonstrated the effectiveness of machine learning algorithms in clinical diagnosis and prognosis, as well as their ability to accurately predict adverse outcomes in disease progression compared to traditional diagnostic methods. Machine learning algorithms were also employed in the development of the prediction model in this study. This model can aid clinical decision-makers in accurately identifying patients at high risk for postoperative VTE, enabling patients to avoid unnecessary tests and reducing the financial and physical burdens of diagnostic procedures. Importantly, the model considers risk factors that are often overlooked by clinicians, such as NLR and BMI, and thus may improve patient survival rates. Previous studies^22,23 have focused on hematological indicators such as platelets and fibrinogen to assess the risk of postoperative thrombosis, but with poor accuracy. This study expands the list of risk factors, with the hope of drawing attention to other high-risk patients. The current study visualized the model by SHAP analysis and showed that VTE was strongly associated with BMI, history of adjuvant radiotherapy, history of adjuvant chemotherapy, use of CVC, duration of surgery, intraoperative bleeding, T-stage, N-stage, and NLR.

Obesity is associated with increased surface tension on the abdominal wall, which impedes instrument movement during surgery and compresses the operating space in the abdominal cavity, requiring greater surgical skill. In addition, due to looser vascular tissue in obese patients, tissue resection during surgery is more likely to cause injury and affect blood supply to the trauma and intestine, increasing the risk of postoperative VTE. A recent study by Jeong²⁴ confirmed the strong association between high BMI and surgical trauma and thrombosis in tumor patients after surgery. Obesity is also associated with increased inflammation and impaired endothelial cell function, and the added pressure on veins further increases the risk of postoperative VTE.

A retrospective investigation conducted in the United States, which encompassed a total of 399 participants,^25,26 discovered that the risk of venous thrombosis was six times higher among patients who underwent radiotherapy in comparison with those who did not. The results of the study suggest that radiotherapy is a significant factor that influences venous thrombosis, corroborating the present study’s findings. Radiotherapy exerts a potent effect in impeding tumor growth and killing tumor cells; nevertheless, it also generates severe detrimental effects.^27,28 High-energy particles interact with tissues during radiotherapy, resulting in the production of free radicals that cause harm to cell membranes and other cellular structures, including endothelial cells.^29,30 The damage to endothelial cells expedites the release of pro-inflammatory molecules and triggers the activation of peripheral platelets and leukocytes, leading to thrombosis. Furthermore, impairment of the natural anticoagulant properties of blood vessels can arise due to damage to endothelial cells, amplifying the likelihood of coagulation.³¹ Conversely, radiation therapy can also provoke localized limb swelling, rendering the veins stiff and inelastic, which, in turn, results in venous blood stasis. Curiously, the converse physiological transformations were observed in patients undergoing chemotherapy. While some researchers contend that chemotherapy impairs estrogen-producing cells, diminishing the influence of estrogen on the coagulation system and thus making patients more vulnerable to venous thrombosis,³² chemotherapy is exceedingly efficacious in inhibiting coagulation factors such as thrombin and fibrinogen, natural anticoagulants such as protein C and protein S, and also reduces platelet adhesion. These effects play a crucial role in curtailing the incidence of postoperative venous thrombosis.³³

This study has also identified that, in addition to the impact of radiotherapy on thrombogenesis, surgery heightens the risk of thrombosis, particularly in patients who undergo longer surgeries or experience more intraoperative bleeding. As the duration of surgery increases, tissue and vascular damage can occur, which can result in extended postoperative pain, fatigue, and other adverse effects. Patients may then be hesitant to resume activities due to physical discomfort, further increasing the risk of blood clots in the legs and feet. Furthermore, elevated intraoperative bleeding disrupts the balance of coagulation and anticoagulation factors in the patient’s blood, triggering the body’s procoagulant response.³⁴ Therefore, healthcare professionals should closely monitor patients who undergo longer operative times and experience more intraoperative bleeding and take necessary precautions when thrombosis signs manifest to mitigate the risk of postoperative mortality.³⁵

The findings of this study suggest that patients with more aggressive and invasive gastric cancer have a higher likelihood of developing postoperative VTE, which is likely related to the extent of surgery required to remove the tumor. Advanced gastric cancer typically necessitates more extensive resection and lymph node dissection, making it challenging to prevent postoperative thrombosis.³⁶ Additionally, malignancy can increase the levels of procoagulant substances and inflammatory factors in the blood, as supported by prior research.^37,38 Furthermore, clinicians often use a combination of treatments, including adjuvant radiotherapy and chemotherapy, to eradicate advanced gastric cancer, which can increase coagulation and further elevate the risk of thrombosis. In conclusion, postoperative VTE in patients with advanced gastric cancer is a multifactorial phenomenon. Therefore, clinicians should monitor the preoperative blood profile, liver and kidney function of patients, minimize surgical duration and bleeding volume, and encourage patients to engage in physical activity as soon as possible after surgery to prevent thrombus formation.

This study examines the application of three samples to determine the rationale for models predicting VTE. Analysis of disease prediction in sample II identified the use of CVCs as a significant risk factor. The placement of a CVC in a larger venous vessel can lead to vein trauma and contribute to thrombosis.³⁹ Furthermore, the use of CVC may cause adverse complications such as blood stagnation and bacterial colonization of peripheral veins, increasing the risk of thrombosis. Importantly, patients with CVCs require rest and recuperation, which we believe is the main mechanism of thrombosis.

In recent years, several predictive models for VTE have been constructed successfully by clinical investigators.^38,40 However, these studies usually only include basic clinical characteristics of the patient. The hemagglutination process often triggers the expression of proinflammatory cytokines and signaling molecules, leading to systemic inflammation.⁴¹ Inflammatory cells can be an important predictor of postoperative thrombosis, as suggested by the results of the current study, which indicate that patients with a higher NLR index are more likely to experience VTE in the postoperative period. Neutrophils, a type of white blood cell, play an important role in the inflammatory and immune response. They can cause oxidative stress that damages vascular endothelial cells and inhibits their function while also releasing tissue factors and other procoagulant molecules, increasing the risk of postoperative VTE.⁴² Similarly, lymphocytes have similar functions to neutrophils, and their reduced number usually indicates poorer immune function and a stronger postoperative trigger of infection and inflammation, increasing the risk of VTE in the postoperative period.

Limitations

This study provides a comprehensive evaluation of the discrimination, calibration, and clinical utility of the model, but it also has some limitations. While the study considered multiple risk factors, it did not focus on laboratory test indicators. Additionally, although machine learning algorithms were more accurate, their models were more complex and less interpretable The entire computational and decision-making process of the model runs in a black box, which is not as intuitive and clear as the logistic regression model.^43,44 Furthermore, the current study was a retrospective study, which has inherent limitations such as selection bias and retrospective bias. Therefore, future studies should incorporate multicenter prospective studies to further improve the reliability of the findings.

Conclusion

This study employs the XGBoost machine learning algorithm to construct a model that predicts the risk of postoperative VTE. The model exhibits high accuracy in prediction and clinical utility, providing surgeons with a means to promptly identify high-risk patients. Notably, the model identifies VTE as a significant challenge for patients undergoing radical gastrectomy, with risk factors including BMI, adjuvant radiotherapy and chemotherapy history, use of CVC, surgical time, intraoperative bleeding, T-stage, N-stage, and NLR.