Development of a Reliable GADSAH Model for Differentiating AFP-negative Hepatic Benign and Malignant Occupying Lesions

Introduction

Hepatic space-occupying lesions are abnormal tissue areas, comprising solid or cystic masses, located with the liver. Depending on the characteristics of the lesion, it can be categorized as either a malignant or a benign hepatic lesion.¹ However, it is essential to note that the treatment and prognosis for these two types of lesions differ significantly. Hepatic malignant occupying lesions (HMOL), primarily hepatocellular carcinoma (HCC), ranked sixth in incidence and third in mortality among global cancer diseases, as reported in the 2020 Global Cancer Statistics.^2,3 The incidence and mortality rates of HCC have shown a consistent increase worldwide over the last two decades.⁴ Late diagnosis often leads to a poor prognosis for patients. Timely detection allows for liver transplant or surgical resection, which can result in a 5-year survival rate exceeding 70%.⁵ In contrast, hepatic benign occupying lesions (HBOL), such as hepatic hemangiomas, generally exhibit slow growth and often require mere observation or local intervention.⁶ Therefore, an accurate differential diagnosis of hepatic occupying lesions is crucial for developing appropriate treatment plans and predicting prognosis.

The Current guidelines recommend screening for HCC using alpha-fetoprotein (AFP) and B-ultrasound.⁷ However, as the proportion of small hepatocellular carcinomas being diagnosed continues to increase, the sensitivity of AFP progressively decreases.^8,9 Furthermore, studies have shown that approximately 30% to 40% of HCC cases are AFP-negative.¹⁰ Additionally, ultrasound has limited sensitivity for assessing HMOL.^11,12 Other imaging modalities, especially in the detection of small HCC and differentiation from other benign lesions, still lead to misdiagnosis and missed diagnoses.^13,14 While histopathological diagnosis remains the gold standard for the diagnosing HMOL, it is an invasive and risky. In contrast, blood diagnostic markers offer a non-invasive, low-cost, objective, and convenient approach to the screening and monitoring liver malignancies. Although new HMOL blood diagnostic markers have been developed, they are still in the research stage and have not been widely implemented in clinical practice.^15–17 Furthermore, the combination of multiple blood markers and demographic characteristics to construct hepatocellular carcinoma prediction models and enhance the early diagnosis rate of HCC has been a popular research topic in recent years.^18–20

Therefore, it is crucial to develop a rapid, convenient, and sensitive screening and surveillance method using a series of clinical examinations, in order to ensure timely detection of malignant lesions in asymptomatic individuals at risk. Machine learning, an emerging medical field of artificial intelligence in medicine, possesses the ability to process large and complex data, and thus can facilitate more precise disease diagnoses and personalize patient treatments.²¹ In the realm of personalized cancer therapy, nomograms serve as statistical tools that provide patients with a personalized prediction based on a set of variables. Nomogram models have been used to diagnose or predict cancer by integrating multiple parameters.^22,23 However, little research has focused on developing a nomogram prediction model for differentiating between HMOL and HBOL for patients with AFP-negative. Therefore, we conducted a retrospective analysis of the clinical data and laboratory parameters from 245 cases of HMOL and 95 cases of HBOL patients admitted to the Guangxi Medical University Cancer Hospital. The objective was to construct a reliable nomogram model, which aims to distinguish AFP-negative HMOL from HBOL by using machine learning algorithms.

Materials and Methods

Patients

This study conducted a retrospective analysis of data from 1647 cases of hepatic occupying lesions collected at the Guangxi Medical University Cancer Hospital between August 2021 and April 2023. Inclusion criteria required patients to have received their initial diagnosis through imaging or pathology prior to surgical treatment.^24,25 Exclusion criteria were as follows: 1. Received any anti-cancer treatment; 2. Co-existence of malignant localization in other organs; 3. Co-existence of benign and malignant occupations; 4. Diagnostic inconsistencies in pathology and imaging; 5. AFP≥20mg/mL cases. Ultimately, 340 cases of hepatic lesions were included in the study, consisting of 245 cases of HMOL and 95 cases of HBOL.

Clinicopathologic Characteristics

General patient data, routine hematological, biochemical and immunological parameters were collected from 340 individuals with hepatic occupying lesions. The collected information included gender, age, blood platelet (PLT), prothrombin time (PT), activated partial thromboplastin time (APTT), thrombin time (TT), fibrinogen (FIB), des-gamma-carboxy prothrombin (DCP), total bilirubin (TBIL), direct bilirubin (DBIL), indirect bilirubin (IBIL), serum ferritin (SF), alanine aminotransferase (ALT), aspartate transaminase (AST), alkaline phosphatase (ALP), lactate dehydrogenase (LDH), gamma-glutamyltransferase (γ-GGT), alpha-L-fucosidase (AFU), 5’-nucleotidase (5’-NT), monoamine oxidase (MAO), total bile acid (TBA), thymidine kinase 1 (TK1), heat-shock protein 90 alpha (HSP90α), complement protein 3 (C3), complement protein 4 (C4); carcinoembryonic antigen (CEA), carbohydrate antigen 125 (CA125), carbohydrate antigen 153 (CA153), carbohydrate antigen 199 (CA199), AFP, lens culinaris agglutinin-reactive fraction of AFP (AFP-L3), lens culinaris agglutinin-reactive fraction of AFP/total AFP (AFP-L3%), hepatitis B surface antigen (HBsAg), and hepatitis B e antigen (HBeAg).

Statistical Analysis

Data Preprocessing

All calculations were performed using R software (version 4.3.1) and relevant packages. Missing data points below 20% were imputed using the predictive mean matching method.²⁶

Model Building

Significant variables in the analysis of differences between the HBOL cohort and the HMOL cohort were screened. The two cohorts were randomly divided into training and validation sets using a 7:3 ratio. In the training set, the least absolute shrinkage and selection operator (LASSO) were used to screen variables.²⁷ To screen the best predictive modeling methods, five common machine learning algorithms were selected, including decision tree (DT), logistic regression (LR), neural network (NNET), random forest (RF), and support vector machine (SVM) to evaluate the performance of the model in the training set.²⁸ The performance of models for the statistically significant features was assessed using area under the curve receiver operating characteristics curve (AUROC). Therefore, univariate and bidirectional elimination logistic regression analyses were conducted on chosen the significant factors to assess the final model. Finally, a predictive nomogram was constructed to discriminate AFP-negative HMOL and HBOL. The LASSO regression analysis applied ten-fold cross-validation, and five machine learning algorithms used ten-fold bootstrap validation.

Model Evaluation

The performance of the nomograms was assessed using AUROC and calibration curve analyses. The calibration capability of the model was evaluated using the Hosmer-Lemeshow test, and the calibration curves were analyzed using 1000 bootstraps. Additionally, decision curve analysis (DCA) was performed to determine the predicted net benefit threshold. These methods for evaluating models were also evaluated by the validation set.

Non-normally distributed data were presented as median and interquartile range. The chi-squared test was used for categorical variables, and the Mann–Whitney U-test was used for numerical variables. Results with a p value of less than 0.05 were considered significant.

Results

Patient Cohorts and Clinicopathologic Features

A total of 1647 cases of liver lesions were subjected to inclusion and exclusion criteria, resulting in a final cohort of 340 cases. The demographics and clinicopathologic characteristics of the 340 patients were listed in Table S1. Among these patients, 95 (27.94%) cases were diagnosed as benign and 245 (72.06%) were diagnosed as malignant. To address missing data, multiple imputation methods were utilized as a preprocessing step. Subsequently, statistical significance tests were performed on all clinical characteristic parameters. The detailed process is presented in a flow diagram (Figure 1). Within the AFP-negative HBOL and HMOL cohorts, there were 30 parameters that exhibited statistically significant variances among a total of 35 parameters (P<0.05, Table 1). Significantly, there is a notable difference in the distribution of benign and malignant liver lesions between genders (P<0.001, Table 1). In each cohort, the majority of patients with HMOL were male (85.31% vs 14.69%), while the majority of patients with HBOL were female (58.95% vs 41.05%), and the median age surpassed 45 years (48 vs 55). Furthermore, several parameters including Age, PT, TT, FIB, D-Dimer, log₁₀(DCP), TBIL, DBIL, SF, ALT, AST, ALP, LDH, γ-GGT, AFU, 5-NT, TBA, TK1, HSP90α, C4, CEA, CA125, CA153, CA199, log₁₀(AFP), AFP-L3, HBsAg, and HBeAg were found to be higher in the HMOL cohort compared to the HBOL cohort (P<0.05, Table 1). Conversely, PLT level was lower in the HMOL cohort (P<0.05, Table 1). The patients were divided into two sets, namely the training set and the validation set, comprising 237 and 103 patients, respectively. Importantly, all thirty variables mentioned above were found to be statistically significant. Notably, there were no significant differences were observed between the parameters in these two sets (P≥0.05, Table S2).

Table 1 Comparison of Variables Between HBOL Cohort and HMOL Cohort

Figure 1 Flow chart of the study process. Abbreviations: AFP, alpha-fetoprotein; HBOL, hepatic benign occupying lesions; HMOL, hepatic malignant occupying lesions; LASSO, least absolute shrinkage and selection operator; ROC, receiver operating characteristic; DCA, decision curve analysis.

Clinical Parameters and Method Selection for the Predictive Model

In the training set, the LASSO regression analysis was performed on 30 variables (Figure 2A and B). The dashed line in Figure 2B represents the maximum value for lambda (λ) within one standard error of the minimum binomial deviation. At λ = 0.061, six variables with non-zero coefficients were identified via ten-fold cross-validated LASSO regression analysis as concise combinations distinguishing AFP-negative HBOL from HMOL. The variables contained gender, age, log₁₀(DCP), SF, log₁₀(AFP), and HBsAg, as presented in Table S3. Subsequently, five machine learning models were analyzed by the above six variables. The receiver operating characteristic (ROC) curve of the training set demonstrated that the area under the curve (AUC) value of DT, LR, NNET, RF, and SVM models in diagnosing AFP-negative HMOL were 0.790, 0.898, 0.775, 0.896, and 0.888, respectively (Figure 2C). Notably, the LR model displayed the most significant diagnostic ability, with an AUC of 0.898 for distinguishing between benign and malignant hepatic lesions.

Figure 2 Results of the LASSO regression and machine learning analyses for predicting hepatic malignant occupying lesions (HMOL). (A) Plot of the LASSO coefficient profiles. (B) The tuning parameter (λ) was selected based on the 10-fold cross-validation error. Two vertical lines mark the selection point, with the right line indicating the selection of six candidates with non-zero coefficients at a mean error of one standard error (λ=0.061). These candidates include gender, age, log10(des-gamma-carboxy-prothrombin) (DCP), serum ferritin (SF), log10(alpha-fetoprotein) (AFP), and hepatitis B surface antigen (HBsAg). (C) ROC curve of five machine learning models used to predict HMOL. Abbreviations: LASSO, least absolute shrinkage and selection operator; ROC, receiver operating characteristic; DT, decision tree; LR, logistic regression; NNET, neural network; RF, random forest; SVM, support vector machines; AUC, area under the curve.

Logistic Regression Analysis and Model Building

Significance variables identified in univariate logistic regression analysis of the training set included gender, age, log₁₀(DCP), SF, log₁₀(AFP), and HBsAg (P<0.05, Table 2), and were subsequently subjected to the multivariate regression analysis. Bidirectional elimination regression analysis revealed that the final model with the smallest Akaike information criterion (AIC), included gender (odds ratio [OR] = 2.85, 95% confidence interval [CI] 1.24–6.57), age (OR = 1.03, 95% CI 0.99–1.07), log₁₀(DCP) (OR = 2.97, 95% CI 1.67–5.28), SF (OR = 1.01, 95% CI 1.01–1.01), log₁₀(AFP) (OR = 4.46, 95% CI 0.74–27.03) and HBsAg (OR = 1.01, 95% CI 1.01–1.01), as shown in Table 2. The AUROCs of gender (AUC=0.727), age (AUC=0.641), log₁₀(DCP) (AUC=0.799), SF (AUC=0.801), log₁₀(AFP) (AUC=0.741) and HBsAg (AUC=0.758) were all above 0.6 (Table S4). The accuracy, sensitivity, and specificity of these parameters can be found in Table S4. Based on these six independent predictors, a LR equation for AFP-negative HMOL diagnosis was established as follows: . The nomogram provided an example of the correctly predicting an AFP-negative HMOL diagnostic result with an optimal cut-off value of 0.62436 (Figure 3, Table S4).

Table 2 Results of Logistic Regression Analysis

Figure 3 Nomogram prediction model for the diagnosis of liver malignant tumors. The model was established in the training set using six parameters: gender, age, log10(des-gamma-carboxy prothrombin) (DCP), serum ferritin (SF), log10(alpha-fetoprotein) (AFP), and hepatitis B surface antigen (HBsAg). The red points on the nomogram represent a case, involving a 71-year-old male with DCP at 1.780 log10 ng/mL, SF at 290 μg/L, AFP at 0.462 log10 ng/mL, and the HBsAg at 0 IU/mL. Utilizing the GADSAH model, the total score for this patient was calculated as 0.681, corresponding to an 85.8% probability of diagnosing hepatic malignant occupying lesions (HMOL). *p<0.05, **p<0.01, ***p<0.001.

Validation of the Model and Evaluation of Clinical Application

The AUC value of the nomogram model in the training set is 0.905, with a sensitivity is 86.6%, a specificity is 80.0% and an accuracy is 84.8%. In the validation set, the AUC value, sensitivity, specificity and accuracy of the nomogram model are 0.880, 82.2%, 86.7%, and 83.5% respectively (Figure 4A and B, Table S4). The GADSAH, serving as a discrimination model, outperformed any single variable model in accurately differentiating various types of focal liver lesions and providing valuable insights for clinical decision-making (Table S4). To further validate the prediction results, we conducted internal validation using the bootstrap approach. The Hosmer-Lemeshow test for the final model demonstrated good fit in both the training and validation sets, indicating a well-fitted model (P>0.05, Figure 4C and D). Moreover, the prediction and calibration curves of the nomogram model closely resembled the ideal curves, confirming its strong predictive ability and consistency (Figure 4C and D). Additionally, decision curve analyses revealed that the patient benefit curve for our model surpassed the two extreme curves (grey and black) under the same risk threshold probability region, suggesting a greater net benefit and clinical utility of the model (Figure 4E and F).

Figure 4 Assessment of discrimination, credibility, and benefit of the nomogram model. The figure includes ROC curves for the model in both the training set (A) and validation set (B), calibration curves for both the training set (C) and validation set (D), and decision curve analysis for both the training set (E) and validation set (F). Abbreviations: ROC, receiver operating characteristic; AUC, area under the curve; CI, confidence interval.

Discussion

In this study, we developed a nomogram model called GADSAH to differentiate between AFP-negative HBOL and HMOL. This model integrates six key predictive factors, encompassing a range of demographic and laboratory parameters including gender, age, log₁₀(DCP), SF, log₁₀(AFP), and HBsAg. Our nomogram model accurately predicted the diagnosis of AFP-negative HMOL with an optimal cut-off value. The calibration curve closely resembled the ideal standard curve, revealing that the nomogram model possessed adequate statistical capability to distinguish between benign and malignant hepatic occupying lesions.

Furthermore, our GADSAH model exhibited excellent performance in diagnosing AFP-negative HMOL, demonstrating heightened sensitivity and specificity. Misdiagnosing some HBOL cases as hepatocellular carcinoma can lead to unnecessary treatment, financial burden, and diminished quality of life for patients.^14,25 Additionally, negative results from AFP tests can complicate the distinction between HMOL and HBOL. Thus, our model can assist surgeons in designing tailored treatment plans prior to initiating treatment. By employing prospective quantitative prediction, the precision of diagnosing AFP-negative HMOL can be enhanced while simultaneously preventing overtreatment of HBOL.

According to relevant studies using world-standard population calculations, the incidence of primary liver cancer in males is 3.28%, surpassing the 1.42% incidence in females.³ Consequently, the incidence of liver cancer is higher in males than in females.²⁹ However, HBOL is less prevalent in men compared to women.³⁰ This difference between gender can be attributed to various risk factors associated with liver cancer, including increased susceptibility to hepatitis B and C viruses, history of smoking and alcohol consumption, higher levels of iron stores, and elevated blood levels of aflatoxin and androgens.³¹ Consistent with this, our study revealed that men have a higher risk of developing HMOL than women.

As a storage form of iron in the body,³² there is increasing evidence that SF is significantly increased in patients with liver cancer, and serves an important marker for its diagnosis, particularly in patients with AFP negative liver cancer.^33–35 In line with this, incorporating SF into the model allows for the detection of AFP-negative HMOL among HBOL.

AFP is commonly used as a tumor marker for HCC screening and clinical diagnosis.³⁶ However, its accuracy as a serological marker for detecting liver malignant tumors is limited.^11,37,38 Previous studies have primarily focused on distinguishing HCC from patients with Hepatitis B-related benign liver disease. Due to the large number of occupying lesions patients admitted to our cancer hospital, the differentiation between AFP-negative HMOL and HBOL is worthy of great attention. Therefore, we excluded cases that were positive for AFP and concentrated on differentiating between AFP-negative HMOL from HBOL. The Food and Drug Administration (FDA) in the United States has approved DCP as a diagnostic marker for liver cancer in East Asian countries. DCP has high specificity in the diagnosis of liver cancer, which is useful in distinguishing between benign and malignant tumors. The combined detection of DCP and AFP is more sensitive than single detection in early HCC detection.^39,40 Our study found that both AFP and DCP were independent risk factors for predicting AFP-negative HMOL, consistent with previous studies.

HBV infection is a significant risk factor for liver cancer development, accounting for more than two-thirds of cases in China.⁴¹ serum HBsAg detection plays a crucial role in HBV infection diagnosis.⁴² Numerous studies have reported the substantial diagnostic and predictive value of assessing serum HBsAg in liver malignancy.^22,43,44 Our findings align with these established conclusions.

The nomogram model we developed, GADSAH depicts AFP combined with DCP as well as other readily available clinical parameters and predicts the probability of AFP-negative HMOL. Previous AFP combined with DCP multiparameter models have also been reported, such as the GALAD (gender, age, AFP-L3, AFP, and DCP) and ASAP (age, sex, AFP, and DCP) models have shown diagnostic value in identifying HCC within the context of chronic liver disease, particularly in cases with negative AFP results. The applicability of these models varies across different patient populations. Currently, there is a scarcity of reports establishing nomograms based on clinical parameters for distinguishing between benign and malignant liver lesions, especially for AFP-negative patients. While certain research studies have demonstrated that radiomics or ultrasound models can effectively discriminate between benign and malignant hepatic occupying lesions, the generalizability of such models in clinical practice is limited due to potential deficiencies in professionalism with new imaging techniques and the high economic burden on patients.^45–47 Our GADSAH model focuses solely on discriminating between malignant and benign liver occupying lesions in AFP-negative patients, providing a noninvasive, cost-effective, and efficient method in diagnosing such cases. However, it is important to acknowledge the limitations of our study, which was a retrospective analysis conducted at a single center and lacks external validation. To address these limitations and further enhance the understanding of our model’s effectiveness and potential limitations, additional studies should be conducted in the future.

Conclusion

In conclusion, we have developed and validated a robust and objective nomogram model, known as GADSAH model which incorporates gender, age, and four routine clinical hematological parameters (log₁₀(DCP), SF, log₁₀(AFP), and HBsAg). This model effectively differentiates AFP-negative HMOL patients from the population with HBOL. The GADSAH model demonstrates excellent consistency, discrimination, and clinical relevance. It offers a rapid and accurate approach for identifying AFP-negative HMOL cases, thereby facilitating the implementation of targeted interventions and treatment protocols.