Survival Prediction Model for Patients with Hepatocellular Carcinoma and Extrahepatic Metastasis Based on XGBoost Algorithm

Introduction

Hepatocellular carcinoma (HCC) is a multifaceted disease that requires careful consideration of treatment decisions. Unlike other solid cancers which are mostly treated according to the tumor burden, HCC treatment requires careful evaluation of concomitant liver disease, liver function, and general performance status as well as intra- and extrahepatic tumor burden.^1–4 Also, HCCs have inter-patient and intra-tumoral heterogeneity arising from chronic liver inflammation with complex pathogenesis, accompanied by a range of genetic and epigenetic changes.⁵ Thereby, patients with HCC and extrahepatic metastasis, classified under the same stage, may exhibit varied clinical courses with estimated survival duration of 7–16 months.^2,4,6 However, survival prediction of patients with HCC and extrahepatic metastasis often relies on a few studies analyzed with a limited set of factors, or on physicians’ experience, which is prone to inaccurate. Hence, there has been an urgent need for systematic and personalized survival prediction for these patients.

The eXtreme Gradient Boosting (XGBoost) algorithm, a product of modern information technology, was introduced by Chen and Guestrin in 2016.⁷ This algorithm employs ensemble learning, one of the most widely used machine learning methods, to construct a single generalized model with high predictive ability. It leverages the gradient descent method to generate decision trees for classification and regression, thereby enhancing robustness.⁸ XGBoost is renowned for improving the accuracy and computational speed of machine learning algorithms, and has found widespread applications in various fields, including the medical domain.^7–10 Another strength of XGBoost is its interpretability. The decision-making process of XGBoost is understandable, unlike other machine learning algorithms.^11,12 Utilizing XGBoost in the context of patients with HCC and extrahepatic metastasis can enhance survival prediction performance.

This study aimed to develop a model for predicting the survival of patients with HCC and extrahepatic metastasis using the XGBoost algorithm with risk factors associated with survival.

Materials and Methods

Study Design and Population

Between January 2013 and December 2018, 9083 patients initially diagnosed with HCC and extrahepatic metastasis were recruited from the National Cancer Registry in Korea. The National Cancer Center selected 13 hospitals, of which over 75% of patients with liver cancer in Korea were treated to build the registry. A systematic extraction method was used to sample 10% of the initially diagnosed HCC patients, and their comprehensive medical data were investigated by medical record administrators. Death status and date of death were obtained from the Korean Statistical Office. The diagnosis of HCC was based on radiological hallmarks in multiphasic CT or MRI, arterial phase enhancement, and portal or delayed phase washout appearance, according to the KLCA-NCC Korea practice guidelines.² Of these patients, we excluded the following:1) 255 patients with missing data; 2) 243 patients who died within 30 days after initial HCC diagnosis; 3) 8585 patients without extrahepatic metastasis. The 993 patients were finally enrolled (Figure 1). This study was approved by the Institutional Review Board (IRB) of Catholic Medical Center (IRB No. SC22ZISE0092), and the need for informed consent was waived owing to the use of de-identified data. It was conducted in accordance with the Declaration of Helsinki and Istanbul.

Figure 1 Study flow.

Data Collection

We gathered data on various clinical variables at the time of initial HCC diagnosis that are known to impact prognosis:1) baseline patient-related variables including age, sex, body mass index (BMI), alcohol consumption, smoking, presence of diabetes, hypertension, dyslipidemia, and Eastern Cooperative Oncology Group (ECOG) performance status; 2) liver-related variables obtained for the etiology of liver disease, Child-Pugh class, Model for End-stage Liver Disease (MELD) score, and cirrhosis by image or fibrosis-4 (FIB-4) index ≥ 2.67;¹³ 3) laboratory findings including platelet, prothrombin time, aspartic acid transaminase (AST), alanine transaminase (ALT), albumin, total bilirubin, creatinine, sodium, glucose, and total cholesterol; 4) tumor-related factors including number of tumors, size of the tumor, presence of intrahepatic vascular or bile duct invasion, extrahepatic metastatic site classified as regional lymph node, lung, bone, distal lymph node and others, alpha-fetoprotein (AFP), and initial anti-HCC treatment.

Outcomes

The primary outcome of this study was to identify the clinical factors that affect survival and to estimate the survival of patients with HCC and extrahepatic metastasis at 3, 6, and 12 months after the initial diagnosis. In addition, we sought to develop a personalized model that could accurately predict survival.

Statistical Analysis and Construction of a Prediction Model

Categorical variables were expressed as numbers with percentages and analyzed using the chi-square or Fisher’s exact test. Continuous variables were summarized as mean ± standard deviation (SD) and analyzed using the t-test or Mann–Whitney U-test. Survival along with clinical variables were estimated and compared with the Kaplan-Meier and Log rank test. Traditional Cox proportional hazard models were used to identify factors that could significantly influence survival. The hazard ratios (HR) and 95% confidence intervals (CI) were summarized. Statistical significance was set at P < 0.05.

In addition, we used XGBoost, a gradient-boosted decision tree algorithm, to build a prediction model with complete case analysis. We utilized multiple imputations to estimate missing data with less than 10% of patients who did not introduce bias.¹⁴ Variables with more than 10% of missing data were excluded for model development.

The data was randomly split into a training set and a test set in a 7:3 ratio. The training set was used for model training, and the test set was employed for validation to assess the model’s accuracy. We selected variables that demonstrated statistical significance in multivariable analysis or clinical relevance to include in the model. The response variables, representing survival information at each time point, were obtained.

To build our prediction models, we used the following parameters: number of estimators = 300, maximum depth of each tree = 15, colsample.bytree = 0.75, and gamma = 3.87. We utilized ensemble learning methods such as Bootstrap Aggregating and Boosting to integrate decision trees, effectively reducing both bias-related and variance-related errors.¹⁵ The results generated by XGBoost ranged between 0 and 1, representing the probability of survival. Patients with results greater than 0.5 were classified as dead, while those with results less than 0.5 were classified as alive.

The performance of model was assessed by evaluating its accuracy, precision (positive predictive value), recall (sensitivity), F1 score (harmonic mean of precision and recall), and area under the receiver operating characteristic curve (AUC). An AUC value greater than 0.5 is considered statistically significant in evaluating binary classifiers.¹⁶ Additionally, we analyzed the feature importance and decision trees of each model to elucidate and understand the decision-making process Finally, we utilized a web-hosting service named “Render” to upload our model and make it available on a website for broader accessibility.¹⁷ All statistical analyses were performed with R software version 4.1.3, SAS version 9.4 (SAS Institute, Inc., Cary, NC, USA), and Python 3.9.16 (Python Software Foundation, Delaware, USA).

Results

Characteristics of the Study Population

Of the 993 patients, 695 (70.0%) and 298 (30.0%) were assigned to the training and test sets, respectively. The baseline characteristics of the two groups are shown in Table 1. The mean (± SD) age was 60.8 (± 12.3) years. Males accounted for 85.4% and hepatitis B virus (HBV) infection was the most common cause of HCC (60.1%). Among the study participants, 56.1% had Child-Pugh class A liver function, 48.7% had more than five masses, and 48.9% had the tumors larger than 10 cm in diameter. Intrahepatic vascular or bile duct invasion was observed in 63.3% of the patients. Regional lymph nodes were the most frequent metastatic sites (51.5%), followed by the lungs (39.8%), distant lymph nodes (21.1%), bones (20.0%), and others (17.9%). The mean AFP was 44,108.7 ± 212,265.8 ng/dL. Supportive care (36.8%) was the most common first-line treatment, followed by systemic chemotherapy (28.8%), and transarterial therapy (24.8%). There was no statistically significant difference in the baseline characteristics between the two datasets.

Table 1 Baseline Characteristics of the Study Population

Survival Analysis and Risk Factors Associated with Mortality

During a median follow-up of 4.0 months (range, 2.00–9.02 months), 96.1% of the patients died in our study cohort. Survival rates were 62.5%, 38.6%, 26.8%, 21.7%, and 17.8% at 3, 6, 9, 12, and 15 months, respectively.

In a multivariate analysis, Child-Pugh class (class B: HR, 1.52 [95% CI, 1.31–1.77], P < 0.001; class C: HR, 1.60 [95% CI, 1.18–2.17], P = 0.003), platelets, log₁₀ 1000/mm³ (HR, 1.55 [95% CI, 1.14–2.11], P = 0.005), sodium, mmol/L (HR, 0.97 [95% CI, 0.95–0.99], P < 0.001), number of tumors (≥ 5: HR, 1.35 [95% CI, 1.17–1.56], P < 0.001), size of tumors (5–7 cm: HR, 1.48 [95% CI, 1.01–2.17], P = 0.046; 7–10 cm: HR, 1.64 [95% CI, 1.14–2.35], P = 0.008; > 10 cm: HR, 1.71 [95% CI, 1.2–2.45], P = 0.003), presence of vascular or bile duct invasion (HR, 1.17 [95% CI, 1.01–1.36], P= 0.035), lung metastasis (HR, 1.19 [95% CI, 1.01–1.39], P = 0.032), bone metastasis (HR, 1.36 [95% CI, 1.14–1.62], P = 0.001), serum AFP, log₁₀ ng/dL (HR, 1.08 [95% CI, 1.03–1.14], P = 0.001), and primary anti-HCC treatment other than supportive care (surgical resection: HR 0.24 [95% CI, 0.16–0.35], P < 0.001; liver transplantation: HR, 0.28 [95% CI, 0.09–0.89], P = 0.031; local ablation therapy: HR, 0.35 [95% CI, 0.14–0.87], P = 0.024; transarterial therapy: HR, 0.46 [95% CI, 0.39–0.56], P < 0.001; chemotherapy: HR, 0.67 [95% CI, 0.57–0.8], P < 0.001; radiation therapy: HR, 0.61 [95% CI, 0.44–0.84], P = 0.003) were associated with mortality (Table 2 and Supplementary Figure 1).

Table 2 Univariate and Multivariate Analyses for Overall Survival

Model Construction and Evaluation

Based on survival analysis, the following variables were included to build a model: age, sex, BMI, etiology of liver disease, Child-Pugh class, platelet count, ALT level, sodium level, number of tumors, tumor size, presence of vascular or bile duct invasion, metastatic site, serum AFP level, and primary treatment method for HCCs.

We built survival prediction model for 3, 6, and 12-months by using the training dataset. The performances of these models are summarized in Table 3. Figure 2 shows AUC of each model. The AUC values of the test set at 3, 6, and 12-months were 0.778, 0.794, and 0.784, respectively, demonstrating the discriminative ability. We then created decision trees. A schematic representation of the decision tree of each model is presented in Supplementary Figure 2. Using the XGBoost model, we identified the feature importance for 3-, 6-, and 12-months survival based on the magnitude of the gain value obtained for each variable. As shown in Figure 3, the primary anti-HCC treatment was a discriminative feature for predicting survival.

Table 3 Performance Metrics of 3-, 6-, and 12-Months Survival Prediction Models

Figure 2 Logistic regression analysis in the test cohort. Area under the receiver operating characteristic curve for (A) 3-, (B) 6-, and (C) 12- months.

Figure 3 Feature importance of (A) 3-, (B) 6-, and (C) 12- months survival prediction model in hepatocellular carcinoma patients with extrahepatic metastasis.

Model Application

Our prediction model is available online (https://metastatic-hcc.onrender.com/form). For example, a male patient aged 50 years with HBV-induced, single, and 8.4 cm sized HCC with bone metastasis was part of our cohort. At the time of diagnosis, his BMI was 21.2 kg/m², he had Child-Pugh class A, ALT of 25 IU/L, platelet count of 211x103/mm³, sodium of 142 mmol/L, and AFP level of 16,600 ng/dL. The patient then underwent transarterial therapy. According to our model, the patient survival probabilities at 3, 6, and 12 months were 95.0%, 65.3%, and 17.9%, respectively (Figure 4). He died 9.0 months after the initial HCC diagnosis.

Figure 4 An example of applying our model to estimate survival probability at 3 months.

Discussion

We developed a survival prediction model using XGBoost for patients with HCC and extrahepatic metastasis. We utilized a large volume of qualified data from multi-centers and used variables commonly evaluated in real-world practice. Our model achieved AUC values of 0.778, 0.794, and 0.784 for the 3-, 6-, and 12-month survival probabilities, respectively. Using our model, patients with HCC and extrahepatic metastasis can obtain more detailed information about their prognosis and enable individualized treatment.

Being diagnosed with advanced cancer brings anxiety and uncertainty with the anticipation of suffering and fear of death.^18,19 The patients want to participate in their care process with necessary information for the best decisions. They hope to be prepared for their death making certain arrangements beforehand. The caregivers are also forced to face emotional and practical hardships.²⁰ They feel anxiety and depression affecting mental health before and after the patient’s death. The financial burden caused by medical care and loss of productivity often overwhelms their financial condition. Therefore, the reliable anticipation of life expectancy is crucial for patients and their loved ones.

Despite active surveillance, about 12–18% of patients with HCC are diagnosed with extrahepatic metastasis.^21–24 In our study, 11.6% of patients with HCC had extrahepatic metastasis. The frequency of metastatic sites differs according to the studies. A Japanese study including 151 patients with metastatic HCC reported that the lung (47.0%), regional lymph node (42.4%), and bone (37.1%) were frequent sites for HCC metastasis at 2007.²³ Another study conducted in the USA in 2014 using claims data revealed that the lung (30.8%), peritoneum (19.0%), bone (15.9%), and lymph nodes below the diaphragm (11.2%) were the common sites of HCC.²¹ We found that metastasis to lung and bone had a negative effect on survival. In a European study, it was observed that the presence of lung metastasis reduced survival, whereas bone metastasis did not have a similar effect.²⁵ The association between metastatic site and survival is an intriguing finding, which could be attributed to distinct metastatic mechanisms. Lung metastasis is commonly associated with hematogenous spread through encapsulating tumor clusters and macro trabecular-massive subtypes.^26–28 It is expected to facilitate tumor spread through blood circulation. Whereas, lymph node metastasis occurs through increased lymphangiogenesis of the tumor along with epithelial–mesenchymal transition.²⁹ The metastatic lymph node has high immune cell infiltration with more fibrous tumor stroma than the lung reflecting immune system activation against HCC.³⁰ Lastly, bone metastasis has both hematogenous and lymphatic spread features.²⁸

In our study, liver function and tumor burden played a critical role in prognosis consistent with previous studies.^23,31,32 In particular, various anti-HCC treatment strategies resulted in positive responses. Intrahepatic HCC-directed therapies such as surgery, local ablation therapy, transarterial therapy, and radiation therapy have improved survival and are recommended for patients with HCC and extrahepatic metastasis.^2,3,32–36 A Korean study reported that 13.1% of the patients with HCC and extrahepatic metastasis, who were treated with multimodal strategies obtained objective intrahepatic tumor response and gained improved survival compared with their counterparts.³²

We employed the XGBoost algorithm for survival prediction and model development by leveraging a large patient dataset. XGBoost is a powerful machine learning algorithm that can handle high-dimensional data while reducing the risk of overfitting by automatically selecting and utilizing important factors.⁷ Moreover, it utilizes ensemble learning techniques, which enables decision trees to improve performance and reduce learning time.⁷ XGBoost also provides feature importance and a decision tree that enhances interpretability compared to previous machine learning algorithms.³⁷

There are several limitations in our study. First, the median survival of our patients was only 4.0 months, which is relatively shorter than that reported in previous studies.² This shorter survival time may be attributed to the inclusion of heterogeneous patients, especially those with poorer performance or liver function who were left untreated compared to other studies. Nevertheless, including all patients is reasonable, as our goal was to build a model based on real-world data. Second, our study did not include the latest medications, such as atezolizumab-bevacizumab, durvalumab-tremelimumab, lenvatinib, cabozantinib, or ramucirumab, as our patient cohort was enrolled from 2013 to 2018.^1–4 Due to the inherent limitations of our data, our model may not be directly applicable to patients who have received recent chemotherapeutic agents, and it will need continuous updates, including data from patients undergoing chemotherapy. Third, it may be difficult to generalize our model to different demographic populations because of the lack of external validation. In addition, we were unable to obtain or utilize data that might play a crucial role in survival, such as ECOG performance, cirrhosis, or treatment response after the initial anti-HCC treatment. Therefore, prospective validation and improvement of our model with a more diverse patient population and precise clinical data are required to overcome these limitations.

Conclusion

In conclusion, we successfully developed an accurate and reliable model for predicting the survival probability of HCC patients with extrahepatic metastasis using the XGBoost algorithm. Our model is easy to use and requires simple clinical data, making it accessible to both physicians and patients. We anticipate that our model will aid in individualized survival time estimation and provide valuable information for clinical decision making, ultimately leading to improved survival outcomes.