Predicting Preoperative Deep Vein Thrombosis in Elderly Hip Fracture Patients Using an Interpretable Machine Learning Model

Introduction

Hip fractures comprise one of the most prevalent fractures among the elderly population. As the global population ages at an accelerating pace, the prevalence of hip fractures among the elderly has been gradually increasing, resulting in a substantial global public health concern.¹ Consequently, the annual incidence of hip fractures worldwide is projected to increase from 1.6 million in 2000 to a minimum of 4.5 million by 2050. Hip fractures represent a potentially catastrophic event for patients.^2,3 Following injury, approximately 30% of patients die within the first year, while those that survive face a considerable disease burden affecting their future quality of life. Accordingly, socioeconomic costs related to hip fractures are estimated to represent 0.1% of the global disease burden and 1.4% in developed market economies.⁴

Deep vein thrombosis (DVT), a primary manifestation of venous thromboembolism (VTE), is a prevalent preoperative complication of hip fractures that results in a blood clot in a deep vein, usually in the leg of affected patients.^5–7 Lower limb DVT is typically characterized by pain, swelling, and increased soft tissue tension in the affected limb. After detachment, lower limb DVT may migrate to the pulmonary artery and induce pulmonary embolism (PE), which poses a life-threatening risk to patients.^8,9 Song et al found in a previous study that two - thirds of the patients diagnosed with postoperative DVT already had thrombi before surgery.⁶ Research has shown that the incidence of pulmonary embolism is approximately 1.5% among patients with lower - limb DVT formation.¹⁰ In elderly patients with hip fractures, preoperative DVT may result in surgical delays, significantly elevated postoperative mortality and complication rates, and adverse effects on patient prognosis.^9,11 Thus, establishing a predictive model that can identify high-risk patients and predict DVT is likely to aid clinicians in implementing preventive measures to reduce the incidence of DVT.

Machine learning is a subfield of artificial intelligence that employs mathematical algorithms to identify and categorize structures in heterogeneous data for the purpose of decision-making.¹² In recent years, machine learning (ML) algorithms have been increasingly used to predict the risk of diseases and disease-related complications. Accordingly, machine learning has been employed in the development of accurate DVT risk prediction models for specific populations in numerous studies, which have demonstrated exceptional performance. Nevertheless, studies that have utilized machine learning to develop preoperative DVT prediction models for hip fracture patients have been largely limited.^13–17 Although machine learning models exhibit exceptional performance, their black-box nature (ie, the absence of transparency in the process between input data and decision output) restricts their applicability in specific areas.^18,19 Thus, understanding the process by which these models make decisions, as well as the rationale behind them, is crucial to ensure their effective implementation in clinical practice. Accordingly, algorithms that elucidate machine learning models have emerged in recent years, thereby enhancing the trust and understanding of clinicians in these models.²⁰

In the present study, an interpretable machine learning model was developed in order to predict preoperative deep vein thrombosis in elderly hip fracture patients. Additionally, the SHapley Additive exPlanations (SHAP) method was employed to explain the machine learning model utilized.

Methods

Study Population

Due to the retrospective nature of the study, the Institutional Review Board of Yijishan Hospital waived the need of obtaining informed consent. Hip fracture patients admitted at the First Affiliated Hospital of Wannan Medical College were retrospectively analyzed from January 2019 to April 2023 in the current study. The inclusion criteria comprised the following: (1) patients who were 60 years of age or older; (2) patients diagnosed with a hip fracture; (3) patients having complete clinical data. The exclusion criteria comprised the following: (1) patients with multiple fractures, open fractures, or pathological fractures; (2) patients with a history of hip joint surgery; (3) patients with a history of thrombosis; (4) patients with the presence of hematologic disorders or current use of anticoagulants; (5) patients that had an admission period lasting more than fourteen days following injury. Figure 1 illustrates the flowchart of the study cohort selection process.

Figure 1 Flowchart illustrating the patient selection process.

Independent Variables

A total of 42 clinical features were collected, excluding those with missing values exceeding 20%. Overall, a total of 38 variables were classified into the following categories: (1) Demographic data: These included age, gender, smoking habits, alcohol consumption, presence of hypertension, diabetes, cardiovascular disease, cerebrovascular disease, and history of tumors; (2) Fracture-related data: These included information about the type of fracture, the mechanism of injury, the injured side, the time elapsed from the injury to admission, the patient’s activities of daily living, and the Caprini thrombosis risk assessment; (3) Preoperative biochemical parameters: These included albumin, alanine aminotransferase, blood urea nitrogen, creatinine, total cholesterol, triglycerides, C-reactive protein, and erythrocyte sedimentation rate; (4) Preoperative standard blood parameters: These included the quantification of red blood cells, hemoglobin, white blood cells, platelets, as well as absolute neutrophil count and absolute lymphocyte count; (5) Preoperative coagulation parameters: These included the assessment of prothrombin time, international normalized ratio, prothrombin time activity, activated partial thromboplastin time, thrombin time, fibrinogen, fibrinogen degradation products, D-dimer, and antithrombin.

Data Preprocessing

For variables that have less than 20% missing values, analysis of the corresponding data without accounting for missing values may result in biased results. As a result, multiple imputation was used to address missing data in the current study.²¹ In this study, we employed an imputation model based on predictive mean matching. PMM is a commonly used imputation method that can preserve the distribution characteristics of the original data and avoid biases arising from assumptions in parametric models. It selects the closest sample value from the observed data based on the predicted value as the imputation value, ensuring that the statistical properties of the imputed data are similar to those of the original data. After multiple simulations, we found that the standard errors of parameter estimates were basically stable when the imputation was performed 20 times. Therefore, we determined the number of imputations to be 20. Correspondingly, the outliers that persisted after verifying the data source were considered to be the missing values. Subsequently, the StandardScaler was used for standardization. Herein, the average of the data features that were processed by StandardScaler was observed to be 0, while the standard deviation from the mean was observed to be 1. Subsequently, the samples were randomly allocated into training and validation sets using a 7:3 ratio. The workflow of the proposed approach in this study is illustrated in Figure 2.

Figure 2 Flowchart illustrating the methodology used in the current study.

Data Balance

The ratio between the thrombosis group and the non-thrombosis group was roughly 1:7, suggesting an uneven distribution of data. Earlier studies indicate that the use of unevenly distributed data can cause the classification boundary to favor the minority class, leading to reduced sensitivity and increased specificity during model fitting.²² In order to address this issue, the Synthetic Minority Over-sampling Technique (SMOTE) algorithm was used in the current study to equalize the distribution of data in the training set. Accordingly, 527 new data points were generated for the thrombosis group to achieve a rough equilibrium between the non-thrombosis and thrombosis groups.

Weighting and Selection of Analytical Variables

Variable selection, which involves analyzing the dataset and selecting the most important variables, is a critical aspect in predictive problems, as it determines the significance (weight) of variables by ranking them. The most common methods for ranking importance can be primarily divided into two categories: (1) Those that rely on model coefficients, and (2) those that rely on model performance (such as using univariate feature ranking for importance analysis).²³ Herein, the first category method (based on model coefficients) was employed to analyze the importance of variables. Accordingly, machine learning algorithms were utilized to prioritize variables, and significant features were subsequently extracted following an initial screening. Accordingly, three machine learning methods —LR, RF, and Adaboost—were utilized to rank the importance of features.

Comparison of Different Machine Learning Algorithms

In the training set, a total of five machine learning models, including XGBoost, SVM, MLP, AdaBoost, and LR, were used to develop the prediction model. Herein, the model’s performance was optimized by adjusting each classifier and optimizing the hyperparameters during the model construction process. The predictive performance of each model was assessed by calculating the area under the receiver operating characteristic curve. Moreover, the accuracy, sensitivity, specificity, positive predictive value, negative predictive value, and F1 score of the models were also calculated. Since calibration curves closer to the ideal line indicate better calibration capability of predictive models, the correlation between the predicted probabilities of the model and the actual observed results was evaluated using calibration curves in the present study.²⁴ Furthermore, the practicality of the model’s decision-making utility was assessed through decision curve analysis (DCA), which quantified the net benefit at various threshold probabilities.²⁵ Subsequently, the machine learning algorithm with the highest performance was selected on the basis of these results.

Model Evaluation

Based on the optimal machine learning algorithm, the predictive model for preoperative DVT occurrence in elderly hip fracture patients was constructed in the current study. Subsequently, the model was evaluated using the validation set data, and calibration curves were generated to assess the consistency of the predicted results with the actual results. In addition, the model’s utility was evaluated through DCA.

Model Interpretability

SHAP values can demonstrate the positive or negative contribution of each predictive variable to the target variable.²⁶ As a result, the SHAP method was employed to interpret the predictive model in the current study, thus, accurately reflecting the contribution of each feature to the final prediction. Additionally, each observation in the dataset was be accounted for by a specific set of values.

Statistical Analysis

Herein, qualitative variables were represented as ratios or constituent ratios. On the other hand, the Kolmogorov–Smirnov test was used to assess the normality of the quantitative variables. Variables that adhered to the normal distribution were presented as the mean (±SD), while variables that did not fit the normal distribution were presented as the median (25^th percentile [P25] to 75^th percentile [P75]). In addition, the machine learning models were constructed using Python (version 3.8.0; Python Software Foundation) while R (version 4.2.0; R Foundation) was used to carry out the statistical analyses.

Results

Patient Characteristics

The present study comprised 976 elderly hip fracture patients, who were divided into a thrombus group (N = 117) and a non-thrombus group (N = 859) based on preoperative Doppler ultrasound examination data. The incidence rates were consistent with those of previous studies conducted in Asia. Table 1 illustrates a comparison of predictive variables between the thrombus and non-thrombus groups.

Table 1 Patient Demographics and Baseline Characteristics

Feature Variable Selection

Following optimization using the machine learning algorithms LR (Figure 3A), Adaboost (Figure 3B), and RF (Figure 3C), the feature variables were processed in the current study. Each algorithm was employed to determine the ten most significant feature variables in its model. Subsequently, a comprehensive analysis using a Venn diagram resulted in the identification of eight variables (D-dimer, fibrinogen, platelet count, red blood cell count, activities of daily living, Caprini thrombosis risk assessment, time from injury to admission, and age) that were necessary for constructing the model (Figure 3D).

Figure 3 Results of the LR (A), Adaboost (B) and RF (C) machine learning algorithms filter the top 10 important variables. The results are expressed by the coefficient values. (D) Venn analysis of the results of the above three machine algorithms.

Modeling and Parameter Tuning

After the optimal predictive variables were determined in the current study, the XGBoost, SVM, MLP, AdaBoost, and LR models were constructed using the training set. The corresponding AUC values were 0.975, 0.582, 0.713, 0.947, and 0.780, respectively (Figure 4A). The hyperparameters that were adjusted in the five classifiers are listed in Table 2. Among these five models, XGBoost demonstrated the most significant predictive performance in terms of relative value (AUC = 0.975, 95% CI 0.957–0.992). Conversely, the performance of the SVM model was observed to be relatively poor (AUC = 0.582, 95% CI 0.512–0.651). In this study, the poor performance of the SVM model may be due to the insufficient optimization of its parameters, which in turn affected the predictive performance of the model. Table 3 provides the quantitative metrics of the performance of the five machine learning models. As observed, XGBoost outperformed all other ML models, exhibiting an accuracy, sensitivity, specificity, positive predictive value, negative predictive value, and F1 score of 0.923, 0.936, 0.910, 0.909, 0.939, and 0.922, respectively. Furthermore, the calibration curve (Figure 4B) and decision curve of the XGBoost model was found to be superior to all other ML models (Figure 4C). As a result, XGBoost was deemed the optimal classifier in the current study.

Table 2 Hyperparameter Configuration of the Algorithms Used in the Current Study

Table 3 Performance of Each Model Used for Prediction

Figure 4 Performance evaluation of different models (A) Receiver operating characteristics curves. (B) Calibration curves. (C) Decision curves.

Model Construction and Evaluation

The final model was subsequently constructed using the XGBoost algorithm and evaluated using the validation set data. As shown in Figure 5A, the AUC for the validation set was observed to be 0.740 (0.657–0.823). Furthermore, the Brier score was estimated to be 0.105, and the calibration curve of the predictive model on the validation set was found to be well-aligned with the diagonal line (Figure 5B). In addition, the decision curve demonstrated that the model provided a greater net benefit when the threshold probability was between 0.1 and 0.6 (Figure 5C).

Figure 5 Model validation results (A) Receiver operating characteristics curve. (B) Calibration curve. (C) Decision curve.

Interpreting the XGBoost Model Using the SHAP Approach

Herein, the SHAP algorithm was employed to ascertain the significance of each predictive variable in the prediction results of the XGBoost model. The most significant variables were listed in descending order in the variable importance plot (Figure 6A). As observed, D-dimer exhibited the highest predictive value, followed by the time from injury to admission, fibrinogen, and red blood cell, activities of daily living, platelet, age, and Caprini thrombosis risk assessment. In addition, the SHAP values were used to identify thrombotic risk factors and examine the positive and negative relationships between predictive variables and target outcomes. As shown in Figure 6B, the color represented the feature value of each feature within a sample, with red indicating a higher value. In addition, Figure 6C and D illustrate individual force plots for thrombotic and non-thrombotic patients. As observed, the predictive features for each patient and the contribution of each feature to thrombosis prediction were elucidated by the SHAP values. The values beneath f(x) represented the probability predictions, while the baseline value denoted the prediction that was made without considering the model input. In addition, the log-odds ratio for each observation was denoted by f(x). The red features represented characteristics that increased the risk of thrombosis, while the blue features represented traits that mitigated thrombotic risk. Herein, the length of the arrows facilitated the visualization of the magnitude of the impact on the prediction. As observed, the length of the arrow directly correlated with the magnitude of its impact.

Figure 6 Model interpretation. (A) SHAP plot of the 8 key variables. (B) Ranking plot for the 8 key variables based on their importance. (C and D) Patients with predicted outcomes of no thrombus and thrombus, respectively.

Discussion

DVT is a frequent occurrence in patients with hip fractures, which can result in significant rates of incidence, mortality, and disability, thereby endangering patient safety.^9,27,28 In the population of elderly patients with hip fractures, the occurrence of preoperative deep vein thrombosis (DVT) has a significant negative impact. Preoperative DVT requires a comprehensive assessment of the thrombus situation to avoid serious complications such as pulmonary embolism caused by thrombus detachment during surgery, which often leads to surgical delay. Preoperative DVT also causes hypercoagulability of the blood, increasing the risk of postoperative thrombus recurrence and thus significantly raising the postoperative mortality rate. For these patients, treatment follows the principle of individualization. For those with mild conditions and stable hemodynamics, anticoagulant therapy is adopted, such as low - molecular - weight heparin combined with warfarin or new oral anticoagulants. For those with severe conditions and unstable hemodynamics, inferior vena cava filter placement is used. Although there have been advancements in anticoagulation strategies, the occurrence of deep vein thrombosis has not significantly decreased because of the substantial rise in the number of patients with hip fractures.²⁹ Thus, promptly identifying patients at high risk for deep vein thrombosis and implementing preventive measures are crucial to decreasing mortality rates.

Herein, LR, RF, and Adaboost algorithms were employed to identify the most significant risk factors. The findings demonstrated that D-dimer, the duration between injury and admission, fibrinogen, red blood cell count, activities of daily living, platelet count, age, and Caprini thrombosis risk assessment were significant factors in the development of DVT. Based on these key factors, the application of XGBoost, SVM, MLP, AdaBoost, and LR algorithms in predicting preoperative DVT in elderly patients with hip fractures was subsequently compared. As observed from the findings, the XGBoost model outperformed all other models in terms of accuracy, sensitivity, specificity, positive predictive value, negative predictive value, F1 score, and AUC. In addition, the AUC value for this model was observed to be 0.975 (0.957–0.992), suggesting a high level of efficiency and stability. Furthermore, the AUC on the internal validation set was found to be 0.740 (0.657–0.823), indicating that the model performed well. Moreover, the comprehensive evaluation of the model’s performance, utilizing a range of metrics and visualizations such as ROC curves, calibration curves, and decision curve analysis, enhanced the credibility of the research findings. These findings, therefore, suggest that the use of machine learning models can effectively identify elderly individuals with hip fractures who are at risk of developing deep vein thrombosis, potentially reducing the risk of thrombotic complications. However, there is a certain disparity between the AUC values of the training set and the validation set, which may indicate the presence of an overfitting issue. Simultaneously, the absence of cross - validation and external validation may potentially undermine the stability of the model. As evident from earlier studies, Wei et al, reported a machine learning model that accurately predicted the risk of thrombosis in patients with lower limb fractures, demonstrating outstanding performance.³⁰ However, the model’s lack of interpretability restricted its practical use in clinical settings. Hence, the SHAP method was used to elucidate the XGBoost model in the current study, ensuring optimal model performance and accuracy, while also aiding in its clinical interpretability. Not only does this approach aid physicians in better understanding the decision-making process of the model, it also promotes the use of the predictive outcomes for clinical prognosis downstream. Additionally, SHAP values provide critical insights into the magnitude and direction of the impact that each predictive factor has on the target variable. Thus, by analyzing the relationships between the predictive variables and their corresponding outcomes in the XGBoost model, detailed investigation into the role of the predictive variables in preoperative DVT in elderly patients with hip fractures was further explored. Throughout the design phase of the study, every effort was made to collect pertinent injury and clinical data at every stage of the clinical process, encompassing preoperative coagulation indicators, routine preoperative blood tests, and additional indicators that were not previously examined in prior studies. Accordingly, D-dimer was identified as the most crucial predictive variable in the current study. D-dimer is a byproduct of fibrin degradation and serves as a precise indicator of the fibrinolytic process in the body, reflecting the activation status of the coagulation system, thrombus formation, or other coagulation-related diseases.^31,32 An earlier study by Cheng et al, reveal fibrinogen and D-dimer levels to be excellent indicators for predicting thrombosis in patients with lower limb fractures.³³ This is consistent with the findings of the current study, which also revealed a positive correlation between fibrinogen and D-dimer levels, as well as the formation of deep vein thrombosis in the lower limbs. This further validates that predictive models employing machine learning algorithms can accurately capture the influence of various predictive factors on DVT, yielding results that closely align with clinical diagnoses.

However, there are numerous limitations associated with this study. First, the lack of an external validation cohort may limit the applicability of the interpretable machine learning model developed in clinical practice. Secondly, the need to position elderly hip fracture patients correctly during ultrasound procedures can pose a challenge, impacting the accuracy of the results. Although vascular ultrasound examination has become increasingly popular and is widely used, it is not regarded as the “gold standard”. Doppler ultrasound is widely used in DVT diagnosis. As a non-invasive, convenient and cost-effective method, it can clearly visualize lower limb vein morphology, structure and blood flow, serving as a common preliminary screening and diagnostic tool. For more accurate DVT diagnosis, there are other methods. Venography, the “gold standard”, directly reveals venous thrombi. However, its invasiveness, high cost and potential complications limit its widespread use. Hence, this predictive model used in this study is unable to eliminate the possibility of false positive and negative outcomes in the diagnosis of DVT.

Conclusions

An interpretable XGBoost predictive model that demonstrating outstanding performance in predicting preoperative DVT in elderly hip fracture patients was successfully developed in the current study. Additionally, by utilizing a machine learning model that could be easily understood and interpreted, patients who are at a high risk for DVT could be accurately identified, facilitating the implementation of appropriate interventions in a timely and precise manner.