Machine Learning-Based Model for the Prognosis of Postoperative Gastric Cancer

Introduction

According to the global cancer statistics released by the World Health Organization in 2018, the incidence and mortality rate of gastric cancer (GC) ranked fifth and third respectively. It is common in East Asia, with an incidence rate of 32.1/100,000 people and a mortality rate of 13.2/100,000 people;¹ therefore, the prevention and treatment of gastric cancer should arouse our great attention. In fact, the treatment of tumors mainly depends on the judgment of prognosis prediction. Accurately predicting the prognosis of different individuals is of immense significance for patients with gastric cancer to choose appropriate treatment strategies. Surgery, as the main treatment for gastric cancer, is considered to be the only possible cure method. Although the level of surgery has continuously improved in recent years, the overall prognosis is poor.^2,3 There are many influencing factors, such as gender, age, Eastern Cooperative Oncology Group (ECOG) score, tumor location, tumor size, degree of differentiation, tumor grade, tissue typing, TNM staging, and chemotherapy,^4–6 among which TNM staging is widely used in clinical work. Unfortunately, TNM staging alone cannot accurately predict the overall postoperative prognosis of patients;⁷ therefore, it is very important to establish a reliable model to predict the prognosis of high-risk patients and formulate individualized treatment strategies.

In recent years, scientists have adopted different methods to optimize the early screening of cancer to determine the prognosis in advance through the classification of cancer and, at the same time, develop new targeted cancer treatment strategies. Therefore, machine learning (ML) methods have become an important tool in the field of medical research. ML methods are good at capturing complex, interactive, or nonlinear relationships and can also effectively predict their prognosis.⁸ ML is a subfield of artificial intelligence. The main research focus in this field is artificial intelligence, especially improving the performance of specific algorithms in empirical learning. ML improves the quality of data and analysis results through feature screening, feature extraction, dimensionality reduction, and other processes. Especially, the benefits of dimensionality reduction are more obvious on datasets with large numbers of features, such as in noise reduction and the elimination of low-relevance features.⁹ In addition, various ML methods have been widely used in disease prognosis prediction; especially, the use of supervised learning techniques such as random forest (RF), support vector machine (SVM), decision tree (DT), and convolutional neural network (CNN), to predict cancer prognosis is increasing.^10–13 Different types of prognostic models have also been widely used in cancer treatment.^14–17 By integrating a variety of ML methods to build a prognostic model, we can effectively improve the effectiveness of the model¹⁸ and help clinicians carry out precise and individualized treatments.

At present, there is still a lack of effective machine learning prognosis model in gastric cancer, so we are based on the clinical information of large samples of gastric cancer from the surveillance epidemiology and end results (SEER) database, this study aimed to directly construct a prognostic model using the LASSO regression method and integrate a variety of ML methods to sort and reassemble the importance of clinical features of gastric cancer prognosis. This was aimed at improving the accuracy of prognostic prediction in gastric cancer patients after operation and constructing a highly effective prognostic model. In addition, we used the information of gastric cancer patients collected in the real world to evaluate the accuracy of the constructed model.

We believe that our study makes a significant contribution to the literature because traditional TNM system can no longer meet the precise treatment of patients, whereas in the current study, the ML models established can accurately evaluate the prognosis of postoperative gastric cancer patients, which may be helpful for accurate prediction in the clinical environment.

Materials and Methods

Data Acquisition

Clinical information data related to postoperative patients with gastric adenocarcinoma (STAD) (from January 1, 2002 to April 31, 2014) were obtained from the SEER database (SEER_1975_2016) (https://seer.cancer.gov/). A total of 167,748 postoperative gastric adenocarcinoma patients were obtained, including 11 clinical features: Age at diagnosis, Gender, Primary Site, Grade, Histologic, AJCC Stage Group, T stage, N stage, M stage, Regional nodes examined (RNE), and Lymph node metastasis rate (LNMR). We deleted uncertain data to reduce noise, such as T0, Tis, Tx, N3, Nx, Nxa, Nxr, and Mx. After screening, we obtained 22,112 patients with complete information, which were randomly divided into training and validation sets at a ratio of 4:1. The external validation dataset comes from the clinical information of 955 postoperative patients with gastric cancer collected by the Affiliated Tumor Hospital of Harbin Medical University (from January 1, 2002 to April 31, 2014), including 692 males and 263 females, with a male to female ratio of 2.6:1 and a median age of 58 years old (23–88 years old). The work flow chart is shown in (Figure 1).

Figure 1 Work flow chart.

Construction and Evaluation Index of Key Features of the Clinical Prognosis Model

First, the univariate Cox regression model was used to analyze the relationship between clinical features and survival time, and 11 clinical features were preliminarily screened. The clinical prognosis model was constructed using the LASSO regression model, and the clinical prognostic riskscore (cp-riskscore) was calculated.¹⁹ Taking the median risk score as the dividing point, the patients were divided into low-risk and high-risk groups. The pROC package for R was used to obtain the ROC curve area,²⁰ and the “survivalROC” package for R was used for independent time ROC curve analysis.²¹

Screening of Key Features of Machine Learning Clinical Prognosis Model

As with the clinical prognosis model, the clinical characteristics were screened by univariate Cox regression model. Subsequently, four algorithms—Boruta,²² Elasticnet,²³ SVM,²⁴ and Random Forest²⁵ were used to screen the key features. The importance ranking of clinical features is combined with four algorithms, which are sorted by the sum of ranks. Thereafter, the filtered key features were combined through different ways (at least two features were extracted from the key features each time as the constituent elements of the new feature) to obtain the combined features (first add 1 to the quantized value of each feature, and then replace the value of the newly generated combined features with the product of the quantization value of the features). Univariate Cox regression was used to screen candidate features for constructing prognostic-related models.

Construction and Evaluation Index of Machine Learning Clinical Prognosis Model

We randomly extracted candidate features to count the C-index value obtained by each combination, and to select the feature combination with the highest C-index average in the training and validation sets as the modeling feature for this time. We constructed the machine learning prognostic model using LASSO regression and calculated the machine learning clinical prognostic riskscore (mL-riskscore);¹⁹ we used the median risk score value as the demarcation point to divide patients into low-risk and high-risk groups. The “pROC” package for R was used to obtain the ROC curve area,²⁰ and the “survivalROC” package was used for independent time ROC curve analysis.²¹

Statistical Analysis

The survival curve was generated by Kaplan-Meier method and compared by log-rank. The correlation between the risk score and clinical features was analyzed using the Student’s t-test and Kruskal Wallis test, and all statistical analyses were performed using R (Appendix Material 2) (version 3.6.1). All statistical tests were bilateral, and p < 0.05 was statistically significant.

Results

The Proportion of Survival and Death in Each Subgroup of Clinical Features

The different subtypes of each clinical feature were statistically analyzed to explore the relationship between clinical indexes and death rate. The results showed that the number of survival cases with clinical features Histologic-Carcinoid was much higher than that of deaths, while the number of survival cases with clinical features LNMR-≥70% was much lower than that of deaths (Figure 2).

Figure 2 Survival and death ratio of clinical features of postoperative patients with gastric cancer in the SEER database in each subtype.

Construction and Effectiveness Evaluation of the Clinical Prognosis Model

The patient information in the SEER database was randomly divided into training and validation sets at a 4:1 ratio. After 11 clinical features were analyzed using the univariate Cox regression model, nine features with significant differences were obtained as the modeling features. The corresponding best cp-riskscore was calculated using a LASSO regression analysis for subsequent analyses (Figure 3A–C), cp-riskscore = Age × (−0.4942) + RNE × (−0.1973) + Primary × (−0.0098) + N × (0.0601) + Grade × (0.0947) + Stage × (0.1348) + T × (0.2141) + LNMR × (0.3365) + M × (0.4104). According to the median value of the risk score, the patients were divided into high-risk and low-risk groups, and Kaplan-Meier was used to draw a survival curve. The results showed that the overall survival (OS) of the high-risk group in the training set and test set was significantly lower than that of the low-risk group (Figure 3D and E); the baseline data are shown in (Tables 1–2). In addition, the ROC curve results showed that the AUC values for 1, 3, and 5 years in the training set were 0.76, 0.8, and 0.8, and the AUC values for 1, 3, and 5 years in the test set were 0.77, 0.79, and 0.78, respectively (Figure 3F and G), indicating that the clinical prognosis model had better verification efficiency.

Table 1 Baseline Data of High-Risk Group and Low-Risk Group in the Training Set of Clinical Prognosis Model

Table 2 Baseline Data of High-Risk Group and Low-Risk Group in the Internal Validation Set of Clinical Prognosis Model

Figure 3 Construction of prognostic model of postoperative gastric cancer patients in the SEER database and survival analysis of high and low risk groups. (A) Misclassification error of different quantitative variables revealed by the LASSO regression model. The red dot represents the misclassification error value, gray line represents the standard error (SE), and left and right vertical dashed lines represent the optimal value under the minimum criterion and 1-SE criterion, respectively, and “lambda” is the tuning parameter. (B) Variation in coefficient values (Coefficients) corresponding to the variables with the lambda value of the tuning parameter. (C) Characteristic coefficient diagram of the LASSO regression model. The dots represent the model coefficients and bar graphs represent log2 (HR). (D) Survival analysis of the high and low risk groups in the training set. Yellow represents the high-risk group, and blue represents the low-risk group. (E) Survival analysis of the high and low risk groups in the internal validation set. Yellow represents the high-risk group, and blue represents the low-risk group. (F) ROC analysis of risk score in training set (1, 3, 5 years). (G) ROC analysis of risk score in internal validation set (1, 3, 5 years).

External Verification of the Clinical Prognosis Model

The clinical information of gastric cancer patients collected from the Affiliated Tumor Hospital of Harbin Medical University was used as the external validation data of this model. The survival difference between the two groups was evaluated by Kaplan Meier to explore the stability of the clinical prognosis model. The results showed that in the external validation set, the OS of the high-risk group was significantly lower than that of the low-risk group (Figure 4A), and its baseline data table is shown in (Table 3). The ROC curve was used to evaluate the accuracy of the model’s prediction, and the results showed that the model had strong predictive efficiency. The AUC values for 1, 3, and 5 years on the external validation dataset were 0.81, 0.80, and 0.80, respectively (Figure 4B).

Table 3 Baseline Data of High-Risk Group and Low-Risk Group in External Validation Set of Clinical Prognosis Model

Figure 4 External validation of the clinical prognosis model of postoperative patients with gastric cancer. (A) Survival analysis of the external validation set. Yellow represents the high-risk group, and blue represents the low-risk group. (B) ROC analysis of risk score on external validation (1, 3, 5 years).

Screening of Key Features of Machine Learning Clinical Prognosis Model

Similarly, after analyzing the 11 clinical features of gastric cancer patients from the SEER database through univariate Cox regression, it was found that all the features were significant except for the two clinical features of gender and Histologic (Figure 5A). Furthermore, the remaining nine clinical features were screened using the Borutafit algorithm, Elasticnet, SVM, and Random Forest (Figure 5B–E), and the top five clinical features (the average value of the importance ranking of the four algorithms) were obtained: Age, LNMR, RNE, Grade and T (Figure 5F); thereafter, the selected key features were combined through different combinations methods to obtain combined features. Finally, 31 features were obtained (Appendix Material 1), which were combined with survival information, and the univariate Cox regression analysis was performed to obtain 30 candidate features significantly related to the prognosis.

Figure 5 Feature selection of postoperative patients with gastric cancer in the SEER database. (A) Univariate Cox regression forest plot for feature selection. Regression analysis was performed on each clinical feature using the Cox proportional hazard model. The red dots indicate that the clinical feature is significantly related to survival (p < 0.05). (B) Feature selection of Boruta algorithm. The Boruta algorithm was used to obtain the importance of each clinical feature, and the value of importance was used to reflect the correlation between the feature and survival. (C) Feature selection of Elasticnet algorithm to obtain the importance of each clinical feature and reflect the impact of the feature on survival through the size of risk coefficient of each feature. (D) Feature selection of SVM algorithm. The machine learning method based on SVM sorted the scores of each feature, and finally selected the required features. (E) Feature selection of the Random Forest algorithm. %IncMSE means increase in mean squared error. The larger the value, the greater the importance of the variable; IncNodePurity means increase in node purity. The larger the value, the greater the importance of the variable. This study evaluated the importance of features according to %IncMSE. (F) Ranking summary of the importance of clinical information features of gastric cancer.

Construction and Effectiveness Evaluation of Machine Learning Clinical Prognosis Model

Patient information in the SEER database was randomly divided into training and validation sets at a ratio of 4:1, and then the C-index value of each combination was calculated by randomly extracting candidate features, and the combination feature with the highest C-index mean in the training and validation sets was selected as the modeling feature, and the corresponding best mL-riskscore was calculated through LASSO regression analysis for subsequent analyses (Figure 6A–C). Nineteen combined features were obtained, where Calculated mL-riskscore = Age × (−0.5858) + RNE × (−0.1212) + Age_RNE × (−0.0821) + Grade_RNE × (−0.0197) + LNMR_T × (−0.0176) + Age_Grade_RNE × (−0.0089) + LNMR_RNE_T × (−0.0030) + Grade_LNMR_RNE_T × (−0.0015) + Grade_LNMR_T × (−0.0007) + Age_LNMR × (0.0002) + Grade_LNMR_RNE × (0.0013) + Age_Grade_T × (0.0038) + Grade_LNMR × (0.0051) + Age_LNMR_RNE × (0.0316) + Age_T × (0.0397) + T × (0.0522) + Grade_T × (0.0641) + LNMR_RNE × (0.0919) + LNMR × (0.1571). According to the median value of the risk score, the patients were divided into high-risk and low-risk groups, and Kaplan-Meier was used to obtain a survival curve. The results showed that the OS of the high-risk group in the training and test sets was significantly lower than that of the low-risk group (Figure 6D–E), the baseline data is shown in (Tables 4 and 5). In addition, the ROC curve results showed that the AUC values for 1, 3, and 5 years in the training set were 0.76, 0.79, and 0.79, respectively, and the AUC values for 1, 3, and 5 years in the test set were 0.76, 0.79, and 0.79, respectively (Figure 6F and G), indicating that the machine learning clinical prognosis model had good validation efficiency.

Table 4 Baseline Data of High-Risk Group and Low-Risk Group in Training Set of Machine Learning Clinical Prognosis Model

Table 5 Baseline Data of High-Risk Group and Low-Risk Group in Internal Validation Set of Machine Learning Clinical Prognosis Model

Figure 6 Construction of machine learning clinical prognosis model for postoperative gastric cancer patients in the SEER database and survival analysis of high and low risk groups. (A) Misclassification error of different quantitative variables revealed by LASSO regression model. The red dot represents the misclassification error value, gray line represents the standard error (SE), the left and right vertical dashed lines represent the optimal value under the minimum criterion and 1-SE criterion respectively, and “lambda” is the tuning parameter. (B) Variation in coefficient values (Coefficients) corresponding to the variables with the lambda value of the tuning parameter. (C) The characteristic coefficient diagram of the LASSO regression model. The dots represent the model coefficients and bar graphs represent log2 (HR). (D) Survival analysis of the high and low risk groups in the training set. Yellow represents the high-risk group, and blue represents the low-risk group. (E) Survival analysis of the high and low risk groups in the internal validation set. Yellow represents the high-risk group, and blue represents the low-risk group. (F) ROC analysis of risk score in training set (1, 3, 5 years). (G) ROC analysis of risk score in internal validation set (1, 3, 5 years).

External Validation of Machine Learning Clinical Prognosis Models

The clinical information of gastric cancer patients collected from the Affiliated Tumor Hospital of Harbin Medical University was used as the external validation data of this model. The survival difference between the two groups was evaluated by Kaplan Meier to explore the stability of the prognostic model. The results showed that in the external validation set, the OS of the high-risk group was significantly lower than that of the low-risk group (Figure 7A), and its baseline data are presented in Table 6. The ROC curve was used to evaluate the accuracy of the model’s prediction, and the results showed that the machine learning clinical prognosis model had strong predictive efficiency. The AUC values for 1, 3, and 5 years in the external validation dataset were 0.80, 0.80, and 0.80, respectively (Figure 7B).

Table 6 Baseline Data of High-Risk Group and Low-Risk Group in External Validation Set of Machine Learning Clinical Prognosis Model

Figure 7 External validation of machine learning clinical prognosis model for postoperative patients with gastric cancer. (A) Survival analysis of the external validation set. Yellow represents the high-risk group and blue represents the low-risk group. (B) ROC analysis of risk score in external validation (1, 3, 5 years).

Survival Analysis of Clinical Subgroups Based on Machine Learning and Clinical Features

To further explore the application value of risk scores, this study analyzed the survival of risk scores of different subtypes in each clinical feature. The results showed that except that the groups with LNMR≥70% were high-risk groups, the survival curve could not be constructed, and that there were significant differences in the survival of the remaining 43 clinical feature subtypes between the high-risk group and the low-risk group (Figure 8).

Figure 8 Survival analysis of the clinical features of postoperative patients with gastric cancer in the high and low risk groups in each subgroup in the SEER database.

Discussion

There are many clinical factors affecting gastric cancer. The traditional TNM staging can no longer meet the needs of individualized and precise treatment of patients in the clinic. Some studies have proved that the performance of constructing prognosis model by Cox risk ratio analysis²⁶ and SVM¹¹ is significantly better than TNM staging system. Therefore, selecting more representative features to evaluate the prognosis more accurately is an urgent problem to be solved. Machine learning algorithms have become the first choice to solve this problem. Previously, there was a similar application in the research and development of a prognostic prediction model using clinical data of patients.^27–29 Turkki R applied an artificial neural network and support vector machine in prognostic prediction of breast cancer.³⁰ He selected the cancer recurrence time (local and systemic) and whether a patient died of cancer within 32 months as the outcome variables. The results showed that the model constructed by each algorithm had good efficiency. Wang et al used five classifiers: Naive Bayes, the generalized linear model, the linear discriminant model, glmnet, and the quadratic discriminant model to rank the importance of clinical features of patients with colon cancer after surgery.¹⁸ They averaged the five importance rankings as the final importance ranking of clinical features and successively constructed a clinical prognosis model with good efficiency; however, this has not been verified on external datasets.

First, based on the nine clinical features selected by the univariate Cox analysis, the classic clinical prognosis model was directly constructed by LASSO regression analysis, and the cp-riskcore of each patient was calculated. The results showed that the prognosis model constructed by this method was effective, and the AUC values for 1, 3, and 5 years in the training set were 0.76, 0.80, and 0.80 respectively. The AUC values for 1, 3, and 5 years in the internal and external validation sets were 0.77, 0.79, and 0.78, and 0.81, 0.80, and 0.80, respectively, and there were significant differences between the high-risk group and the low-risk group. Subsequently, we used Boruta, Elasticnet, SVM, and Random Forest as feature selection methods according to the features of different learning methods. The Boruta algorithm can help us understand the influencing factors of the dependent variable more comprehensively. It helps to eliminate the limitations of related variables, prejudice, and unnecessary noise. The stability and scalability of the experimental results were excellent.³¹ Elastic network is a linear regression model that uses L1 and L2 as prior regularization terms to train. While using L1 regularization to eliminate unimportant features, L2 regularization is used to select highly relevant features at the same time, which helps to reduce the risk of overfitting. It is a very effective feature selection method when there are many interconnected features.³² SVM is a supervised learning method that can solve high-dimensional problems, deal with the interaction of nonlinear features, and has strong a generalization ability.³³ The random forest algorithm was proposed by Breiman in 2001,³⁴ which is an integrated tree classifier. The biggest advantage of random forest is that it can be used to analyze data with a much larger number of variables than the sample size, therefore, the overfitting problem does not occur easily; moreover, it can solve the problem of collinearity. It uses unbiased estimation, has an excellent accuracy, and has strong model generalizability. In this study, the features selected by combining the four algorithms have good predictive performance. We calculated the optimal mL-riskscore corresponding to each sample through LASSO regression. The results showed that we also obtained better results in the process of large-sample machine learning. The AUC values for 1, 3, and 5 years in the training and validation sets were 0.76, 0.79, and 0.79, respectively, especially in the external validation queue, the AUC value for 1, 3 and 5 years was 0.8; in addition, the survival analysis between the high-risk group and low-risk group showed significant differences. After comparing the two prediction models, it is found that the AUC values of the clinical prognostic model and the machine learning prognostic model in the training set, internal validation set and external validation set are similar, floating up and down at 0.8, which has strong generalization. Compared with the clinical prognostic model, the machine learning clinical prognostic model does not show its advantages, which may be due to the large sample size of SEER database, The model has strong generalization ability and is less affected by machine learning algorithm. Although the efficiency of the two algorithms was similar, and the model constructed using machine learning algorithms was relatively complex in terms of mathematical calculations, the results obtained by applying the two models in clinical practice were undoubtedly more reliable. In addition, to clarify the clinical application value of mL-riskscore, we analyzed the survival of high-risk and low-risk groups in different subgroups of clinical features. It was found that mL-riskscore could well distinguish the survival rate in each subgroup of clinical features.

To the best of our knowledge, this study is the first to construct a clinical prognosis model of postoperative patients with gastric cancer by fitting machine learning algorithms. The objective of this study was to construct two effective clinical prognosis models for postoperative patients with gastric cancer, and verify them with a large sample of external data, which has strong clinical practicability. However, there are few limitations. First, there may be a sample selection bias due to the retrospective study design. Second, the machine learning prognostic model we constructed does not show obvious advantages compared with the classic prognostic model. We will continue to optimize the model by trying different machine learning algorithms in the future. In addition, this study did not consider molecular features, such as gene mutation, DNA methylation, and transcriptome markers. In a future work, we will integrate molecular features into the model to improve the overall prognostic efficiency of the model. In conclusion, the two prognostic models of gastric cancer that we have established have the characteristics of low cost, high sensitivity, and specificity, and can provide help for clinical accurate and individualized treatment of postoperative patients with gastric cancer.

Conclusion

Based on the public data of postoperative patients with gastric cancer obtained from the SEER database and the clinical data of postoperative patients with gastric cancer collected from the Affiliated Tumor Hospital of Harbin Medical University, we constructed a clinical prognosis model of gastric cancer and constructed a machine learning clinical prognosis model of gastric cancer for the first time. The models were verified on external datasets, and they accurately evaluated the prognosis of postoperative patients with gastric cancer, which may be helpful for accurate and personalized treatment of postoperative patients with gastric cancer in the clinical environment.