Development and External Validation of a Nomogram to Predict Cancer-Specific Survival in Patients with Primary Intestinal Non-Hodgkin Lymphomas

Introduction

The percentage of non-Hodgkin lymphoma (NHL) patients with extranodal lymphoma is currently at approximately 30–50% and rising.^1,2 The intestinal tract is the second most common site of extranodal invasion involving NHL. The clinical features, pathological subtypes, therapies, and prognoses of primary intestinal non-Hodgkin lymphoma (PINHL) differ from those of gastric lymphoma.^3–5 Separate analysis of intestinal lymphoma may improve the pertinence of research and reliability of results.^3,4 However, most studies on the construction of predictive models have focused on a specific pathological type or site in the gastrointestinal tract.^6–13 Therefore, in this study, PINHL was designated as a separate entity.¹⁴

Today, the Ann Arbor and Lugano stages are used to evaluate the progression of lymphoma. Typically only the status and clinical symptoms of lymph node involvement and organ involvement are considered to predict survival and prognosis¹¹ However, various other factors, such as patient age, sex, and race, pathological type, and treatment modality, may affect the overall survival (OS) of patients with lymphoma, where OS is the length of time from diagnosis to death from any cause. Therefore, the Ann Arbor and Lugano stages may not be ideal methods for predicting the prognosis of patients with PINHL.¹³ To improve treatment strategies and prognosis, it is necessary to develop a straightforward, highly accurate predictive model. Thus far, few studies have systematically evaluated PINHL patients, and there are currently no ideal methods for predicting the survival of inpatients or outpatients with PINHL. Nomograms—quick prediction tools in which numerous independent factors are assigned prediction weights and combined—are crucial in clinical decision-making and risk stratification in oncology.^15–18 Therefore, the present study was aimed at constructing and verifying a nomogram to predict the CSS of PINHL patients based on multicenter data.

Materials and Methods

Data Sources

The Surveillance, Epidemiology, and End Results (SEER) Program, an authoritative resource on cancer data in the United States, is a vast source of cancer statistics. In this study, detailed data were extracted from SEER using SEER*Stat software version 8.3.5 (http://seer.cancer.gov/) and from the First Affiliated Hospital of Guangzhou University of Chinese Medicine (FAHGUCM) using the hospital’s medical record system. Patient consent and institutional review board approval for the SEER database were not required. A data-use agreement was submitted in order to gain access to research data, and the username 17,496-Nov2019 was obtained for data analysis. The Ethics Committee of FAHGUCM approved this study, and the requirement for informed consent was waived because patient information was anonymized at every step of the study, including during data cleaning and statistical analysis.

Patient Selection

In this study, PINHL patients who had undergone treatment from January 1, 2004, to December 31, 2015, were extracted from the SEER database. Additionally, PINHL patients diagnosed by pathology at FAHGUCM from January 1, 2010, to December 31, 2020, were extracted from FAHGUCM’s medical record system. These patients were screened with respect to meeting the selection criteria for the study. Details on patient-selection criteria are provided in Figure 1. Patients with an unknown diagnosis, unknown cancer-specific death, survival of less than one month, or autopsy or death certificate were excluded. Patients with a single primary intestinal lymphoma tumor were included in the study. The information collected included age, sex, race, marital status, Ann Arbor Stage, B symptoms, tumor site, histologic characteristics, stage, and treatment methods. Patients were excluded from the study if any of this information was unavailable. Demographic and clinical-pathologic factors were as follows. (1) Marital status of the patient: married or non-married (widowed, divorced, separated, unmarried, or living with a domestic partner). (2) B symptoms: In the SEER cohort, B symptoms were defined as “CS site-specific Factor 2,” where “Code 010” referred to “night sweats, unexplained fever (above 38°C), and unexplained weight loss (generally greater than 10% of body weight in 6 months before admission)”; “Code 020” referred to “pruritis (recurrent and unexplained)”; and “Code 030” referred to “pruritis plus any B symptom(s).” The above codes were used to determine the existing B symptoms. Additionally, “Code 000” referred to “No B symptoms (asymptomatic).” “Code 988, 999” referred to no applicable information or documentation available in the patient’s record. In the FAHGUCM cohort, B symptoms were defined using the same standard. (3) Tumor site. “Small intestine” referred to tumors in the duodenum, jejunum, and/or ileum, overlapping lesion of the small intestine, those in the small intestine, or those not otherwise specified (NOS). “Ileocecum” referred to tumors in the cecum or appendix. “Colon” referred to tumors in the ascending colon, hepatic flexure of the colon, transverse colon, splenic flexure of the colon, descending colon, sigmoid colon, overlapping lesion of the colon, or colon (NOS). “Other” referred to tumors in the rectosigmoid junction, rectum (NOS), anus (NOS), anal canal, and overlapping lesion of rectum, anus, or anal canal. (4) Histological type of PINHL in patients was ascertained based on the International Classification of Diseases for Oncology, 3rd Edition (ICD-O-3). According to ICD-O-3 oncology codes, six histological subtypes of PINHL were classified as follows: diffuse large B-cell lymphoma (DLBCL) (9680), follicular lymphoma (9690, 9691, 9695, 9698), mantle cell lymphoma (9673), Burkitt lymphoma (9687), T-cell lymphoma (9702, 9705, 9714, 9717, 9719), and other lymphomas (9591, 9670, 9671, 9675, 9684, 9688, 9699, 9727, 9728, 9735, 9738, 9811, 9823). (5) Stage: Localized lesions were designated as Ann Arbor Stage I, regional lesions as Ann Arbor Stage II, and distant metastases as Ann Arbor Stages III and IV. (6) Description of treatment: surgery, radiotherapy, and chemotherapy were set as binary variables. Patient-survival information was quantified using OS and CSS. Data on 8398 patients diagnosed with PINHL from 1983 to 2015 were extracted from the SEER 3database. However, information on B symptoms and Ann Arbor Stage were available only from 2004 to 2015. After screening the data, a total of 3386 patients were enrolled in the present study. In the external cohort, 37 out of 49 PINHL patients met the inclusion criteria for the study.

Figure 1 Recruitment criteria for eligible PINHL patients examined in our present study.

Outcomes

The primary outcome was CSS, defined as the time from diagnosis of PINHL to cancer-specific death, and the secondary outcome was OS, defined as the time from the first diagnosis to death from any cause.

Construction and Validation of the Nomogram

Eligible PINHL patients from the SEER database were randomly divided into two cohorts, the training cohort and the internal validation cohort, at a ratio of 7:3. PINHL patients from FAHGUCM were designated as the external validation cohort. The training cohort was used for nomogram construction, and the internal and external cohorts were utilized for nomogram validation. The effects of variables in the training cohorts were calculated using univariable and multivariable Cox regression analyses. Each variable’s impact on CSS was presented as a hazard ratio (HR) (and corresponding 95% confidence interval [CI]) to identify independent risk factors. Based on results of the Cox regression analysis, a nomogram was established for predicting the probability of 1-, 3-, and 5-year CSS. Bootstrapping was used to assess the robustness of this prediction model.¹⁹ The effectiveness of the nomogram was validated using 1000 bootstrap resamples based on the internal and external validation cohorts. The validity of the nomogram was evaluated by detecting the discrimination and calibration characteristics of the model. A receiver operating characteristic (ROC) curve was plotted, and area under the ROC curve (AUC) was measured. AUC, also called the C statistic, is an accurate and comprehensive measure of sensitivity and specificity.^19,20 The accuracy of 1-, 3-, and 5-year predictions of CSS was assessed using AUC. Calibration curves of the nomogram were used to determine the agreement between anticipated and detected survival in order to further assess the prognostic accuracy of the model. Decision curve analysis (DCA) was also carried out. It is a new method for evaluating the potential clinical value of risk-prediction models²¹ and is widely used in studies on predictive models.²²

Nomogram Performance in Risk Stratification

The survival of each patient in the study was estimated based on a total nomogram score. X-tile plots were constructed to divide the training cohort into four groups based on nomogram scores. The most effective cutoff points were determined following correction for the use of minimum P statistics by Miller-Siegmund P-value correction.²³ After the best cutoff values for the training groups were selected, they were applied to the validation cohorts. The cohorts were divided into low, low-intermediate, high-intermediate, and high-risk subgroups based on total nomogram scores of the patients. ROC and Kaplan–Meier survival curves were used to compare the discriminative ability of the nomogram in predicting survival with that of the Ann Arbor Stage.

The net reclassification index (NRI) and integrated discrimination improvement (IDI) of the nomogram were also calculated and used to evaluate the degree of improvement in the sensitivity and specificity of the nomogram compared with those of the Ann Arbor Stage system. NRI was used to assess differences in the number of PINHL patients correctly classified by the nomogram and the Ann Arbor Stage at a given cut-off point, thereby comparing the predictive power of the two models.²⁴ IDI reflects differences in the prediction probability gap between the two models, which is based on prediction probability for each individual patient in the disease model. The criteria for IDI are the same as those for NRI.

Statistical Analysis

Statistical analysis was performed using EmpowerStats (http://www.empowerstats.com, X & Y Solutions, Inc., Boston, MA) and R software (version x64 4.0.1, www.r-project.org). Demographic and clinical variables were contrasted between the training and validation cohorts, using Fisher’s exact test or the Pearson Chi-square test for categorical variables, and independent t-test for continuous variables, as appropriate. The categorical and continuous variables were reported as frequency and percentage, mean ± standard deviation (SD), or median [interquartile range (IQR)], respectively. Survival curves were constructed using the Kaplan–Meier method, and differences among the groups were compared using the Log rank test. Univariate and multivariate analyses were conducted using the Cox regression model. Nomogram construction and validation were performed using the Iasonos guide.²⁵ The R statistical packages “caret,” “rms,” “survival,” and “pROC” were used to randomize the groups, plot the distribution of risk scores, construct the calibration plot and ROC curves, and build the nomogram; “rmda” was used to draw the DCA curves. The “shiny” and “DynNom” packages were used to develop a web-based survival-rate calculator (https://www.shinyapps.io/) for predicting patient survival rates. All statistical tests were two-sided, and P-values of <0.05 were regarded as statistically significant.

Results

Clinicopathological Characteristics and Patient Prognosis in Training, Internal Validation, and External Validation Cohorts

A total of 3423 PINHL patients were retrospectively enrolled from SEER (n = 3386) and FAHGUCM (n = 37). Details on the demographics, clinical-pathologic characteristics, and treatment of patients in the training, internal validation, and external validation cohorts are listed in Table 1. The median ages of PINHL patients in the SEER and FAHGUCM cohorts were 62.0 years old (IQR, 50.0–74.0) and 56.0 years old (IQR, 48.0–65.0), respectively. Baseline characteristics showed no substantive differences between the training and internal validation cohorts. In the SEER cohort, the median follow-up period was 49 months (ranging from 1 to 155 months). During the follow-up period, 769 patients died of cancer, and 363 patients died from other causes. The 5-year CSS rate was 77.1% and the 5-year OS rate 70.7%. In the FAHGUCM cohort, at the last follow-up, 16 patients died of a cancer-specific cause, and 21 patients died from other causes; the median follow-up time was 28 months (ranging from 1 to 124 months). The 5-year CSS rate was 48.4% (Figure 2).

Table 1 Clinicopathological Characteristics of Patients in the Training, Internal and External Validation Cohorts (N(%))

Figure 2 Kaplan–Meier survival curves for OS (A) and CSS (B) in all the 3386 patients from the SEER database.

Univariate and Multivariate Analyses of CSS, and Construction of Nomogram in Training Cohort

Based on results of the univariate analysis in the training cohort, six factors (age, marital status, Ann Arbor Stage, B symptoms, histologic type, and stage) were linked to the primary outcome CSS. In multivariate Cox regression analysis, five variables (age, marital status, Ann Arbor Stage, B symptoms, and histologic type) were determined as independent predictive parameters of CSS in PINHL patients (Table 2). Multivariable Cox regression analysis was used to identify the parameters significantly associated with CSS, after which a nomogram for CSS was developed and is shown in Figure 3. The nomogram indicated that CSS was higher in patients characterized as younger, married, in Ann Arbor Stage I, without B symptoms, or with follicular lymphoma. Moreover, histologic type had the most significant impact on survival, followed by age, Ann Arbor Stage, marital status, and B symptoms. In the nomogram, the risk score for each variable can be obtained from the above ruler and superimposed on the ruler below to calculate 1-, 3-, and 5-year CSS. Detailed scores for each variable in the nomogram are listed in Table 2.

Table 2 Univariable and Multivariable Analyses of Cancer-Specific Survival in the Training Cohort

Figure 3 Newly developed nomogram for predicting 1-, 3-, and 5-year CSS in PINHL patients.

Evaluation and External Validation of Nomogram

Cox ROC analyses were used to assess the discriminative ability of the nomogram in predicting 1-, 3-, and 5-year CSS in PINHL patients. C-indices of the nomogram for predicting the survival of PINHL patients were 0.772 (95% CI, 0.754–0.790), 0.763 (95% CI, 0.734–0.792), and 0.851 (95% CI, 0.755–0.947) in the training, internal validation, and external validation cohorts, respectively. For PINHL patients, the 1-, 3-, and 5-year CSS of the nomogram yielded AUC values of 0.809, 0.802, and 0.804 in the training cohort (Figure 4A), 0.793, 0.794, 0.800 in the internal validation cohort (Figure 4B), and 0.938, 0.798, and 0.754 in the external validation cohort (Figure 4C), indicating that the nomogram showed favorable discrimination. The calibration curve of the nomogram was highly consistent with the standard curve, indicating that the nomogram showed high reliability in predicting 1-, 3-, and 5-year CSS in the training (Figure 5A–C) and internal validation cohorts (Figure 5D–F). The external validation cohort (Figure 5G–I) also showed consistency between the predicted and observed 1- and 3-year CSS. DCA curves for the performance of the nomogram and Ann Arbor Stage in predicting 5-year CSS in the training, internal validation, and external validation cohorts are shown in Figure 6. The red dotted line indicates the performance of the Ann Arbor Stage, and the black dotted line indicates the performance of the nomogram. DCA of the nomogram showed higher net benefits and demonstrated better clinical outcome values than those obtained using Ann Arbor Stage in the training (Figure 6A), internal validation (Figure 6B), and external validation cohorts (Figure 6C). Total NRI and IDI for the 1-, 3-, and 5-year CSS were assessed when the nomogram was compared with the Ann Arbor Stage model. The NRI and IDI of 5-year CSS predicted using the nomogram were 0.791 and 0.162 in the training cohort, 0.74 and 0.172 in the internal validation cohort, and 0.568 and 0.226 in the external validation cohort; these results indicate that the nomogram showed considerable improvement in predicting 5-year survival. The same results were obtained in predicting 1- and 3-year CSS (Supplementary Table 1).

Figure 4 ROC curves for the nomogram’s prediction of 1-, 3-, and 5-year CSS in the training cohort (A), internal validation cohort (B), and external validation cohort (C).

Figure 5 Calibration curve for predicting 1-, 3-, and 5-year CSS of PINHL patients in the training (A–C), internal validation (D–F), and external validation cohorts (G–I).

Figure 6 Decision curve analysis of the nomogram’s ability to predict 5-year CSS in PINHL patients in the training (A), internal validation (B), and external validation cohorts (C). The gray and black lines represent the assumption for all the patients with and without CSS, respectively. The black dotted line represents the nomogram, and the red dotted line represents the Ann Arbor Stage.

Dynamic Web-Based Survival Rate Calculator

Based on the nomogram model, a dynamic web-based survival-rate calculator was built (https://cuifenzhang.shinyapps.io/DynNomapp/) to predict long-term CSS in PINHL patients (Figure 7). For example, for a 68-year-old (42 points) married man (0 points), diagnosed with DLBCL (55 points), having Ann Arbor Stage IV (25 points), and presenting with B symptoms (10 points), the total score was 132, and the 5-year CSS rate was approximately 40% (95% CI: 33–49%). This calculated value can be used by clinicians when making decisions regarding treatment plans and patient counseling.

Figure 7 An example used to illustrate usage of the web-based survival-rate calculator. A 68-year-old married man with PINHL was diagnosed as DLBCL with Ann Arbor Stage IV and presenting with B symptoms (A). The patient’s survival plot according to the web survival-rate calculator (B). The patient’s 5-year CSS rate is approximately 40.0 (95% CI 33.0–49.0) (C).

Nomogram Performance in Risk Stratification

Using the acquired optimal cutoff values based on X-tile plots for the training cohort, patients in the training, internal validation, and external validation cohorts were divided into four groups to assess risk stratification with respect to CSS. According to the cut-off point values, nomogram scores of the patients were divided into low risk (n = 604, 25.46%; score: 0–59), low-intermediate risk (n = 613, 25.84%; score: 60–82), high-intermediate risk (n = 581, 24.49%; score: 83–106), and high risk (n = 574, 24.20%; score: 107–188).

Comparing the Predictive Accuracy of Novel Risk Stratification Model and Conventional Prognostic Scoring Systems

The ROC and Kaplan–Meier survival curves were used to compare the accuracy of the proposed nomogram against that of traditional Ann Arbor staging in predicting CSS. In the training, internal validation, and external validation cohorts, the AUC values of 5-year CSS rate in PINHL patients obtained using the nomogram with its novel risk-stratification model were higher than those obtained using the Ann Arbor Stage system (0.804, 0.800, and 0811 versus 0.630, 0.637, and 0.598) (Figure 8). These results indicate that the proposed nomogram achieved better accuracy than the Ann Arbor Stage system in these cohorts (Figure 9).

Figure 8 ROC curves used to assess the performance of the nomogram and Ann Arbor Stage prognostic systems in predicting 5-year CSS in the training (A), internal validation (B), and external validation cohorts (C).

Figure 9 Kaplan–Meier survival curves showing the risk levels determined using the nomogram and Ann Arbor Stage system in the training (A and B), internal validation (C and D), and external validation cohort (E and F).

Discussion

Clinicians need to predict patient survival in order to optimize treatment strategies. Herein, simple clinicopathological factors were used to develop a novel nomogram for predicting the 1-, 3-, and 5-year CSS of PINHL patients more accurately than the existing Ann Arbor Stage. The variables used in the nomogram can be collected and calculated directly by clinicians, which improves the practicability of this model. The nomogram model developed by Sun et al showed good predictive value for OS and CSS in primary gastrointestinal non-Hodgkin lymphoma patients.¹¹ However, the population examined in that study differed from the population examined in our study. A nomogram developed by Wang et al for predicting OS in adult patients with primary gastrointestinal diffuse large B-cell lymphoma was only verified internally.¹² In our study, a SEER cohort was used for the training and internal validation of the proposed nomogram, and another patient population was used for external validation to further assess the accuracy, applicability, and credibility of the nomogram in a broad population of patients with PINHL. Using the new nomogram model, a C-statistic of 85.1% was obtained in the external cohort, which was higher than the C-statistic of 77.2% obtained in the training cohort, and the C-statistic of 76.3% obtained in the internal validation cohort. These results indicate that the proposed nomogram showed high discriminative ability. The calibration characteristics of the nomogram were also confirmed. The DCA curve and NRI and IDI values demonstrated that the proposed nomogram could potentially be more beneficial than the Ann Arbor Stage system for predicting the CSS rate of PINHL patients.

In our study, five clinical variables were used to construct the nomogram for the prognosis of patient survival: age, marital status, Ann Arbor Stage, B symptoms, and histologic type.

Currently, age is used as a common predictive parameter for various cancers,^26,27 including PINHL. Patients of advanced age tend to have decreased immunity, increased numbers of comorbidities, and increased therapy-related toxicity, contributing to worsened outcomes and prognoses in these patients.^11,12

Marital status is a risk parameter for developing cancers and a significant prognostic indicator in cancer patients,^28,29 as shown by numerous studies discussing this occurrence.^30–33 A systematic review of 18 studies confirmed that unmarried patients with malignancies were more likely to be diagnosed with advanced cancer at the time of diagnosis.³⁴ Le Guyader-Peyrou et al have shown that marital status is independently associated with survival prognosis.³⁵ The findings of our study also indicated that the prognosis of married PINHL patients was better than that of non-married patients. This may be because married patients possibly have better economic resources than unmarried patients, enabling married patients to obtain improved treatment options with the potential improve their prognosis. Additionally, married patients may also receive extra care from their spouses. Furthermore, widowed patients tend to be older, which may increase cancer-related mortality.¹³ Physiologic studies have shown that marriage quality plays a role in improving cardiovascular, endocrine, and immune function.^36,37

Ann Arbor Stage is a clinically recognized classification standard of lymphoma.³⁸ In our study, Cox regression analysis revealed that the Ann Arbor Stage was a separate prognostic index in PINHL patients. This observation, which is similar to those made in several previous studies,¹³ is consistent with an advanced stage being indicative of disseminated disease.

B symptoms, which are often associated with higher histologic grade or distantly disseminated disease, also showed predictive significance³⁹ and were found to be an independent risk factor for poor survival in our study.

In addition to age, marital status, Ann Arbor Stage, and B symptoms, our study showed that histologic type was an independent risk factor in PINHL patients, which agrees with the results obtained by Wang et al.⁵ In our study, histologic type was defined as a significant factor that can be used to design treatment strategies and evaluate prognosis in PINHL patients. The proposed nomogram verified that certain histologic types, such as T-cell lymphoma, were correlated with worsened survival status, necessitating prompt individualized treatment strategies.^40,41 According to Kim,⁴² T-cell lymphoma might have more negative characteristics, such as advanced stage at diagnosis, and its 5-year OS rate was significantly lower than that of B-cell lymphoma.

In our study, no treatments were found to be independent prognostic factors. This may be because the study was retrospective, so the consistency of treatment measures and indications could not be controlled, or because the lack of patient compliance or treatment side effects affected treatment outcomes.⁴³ However, the proposed nomogram can be utilized in patients prior to treatment to obtain a rapid assessment of survival and prognosis. The nomogram integrates the effects of five clinical variables and provides a personalized risk assessment for each patient. The predictive power of parameters and overall clinical performance of the patients are considered in the scoring system of the nomogram. Therefore, the nomogram can provide greater reliability than that of the Ann Arbor Stage criteria in predicting the survival of PINHL patients and in helping to guide treatment decisions. In this nomogram, a high total score indicates a need for increased vigilance on the part of the clinician and appropriate individualized treatment as soon as possible to obtain an improved survival prognosis.

To our knowledge, our model is the first PINHL-specific prognostic nomogram validated internally in a large SEER cohort and validated externally in other patient populations. The number of patients was relatively high in our study, and the use of prognostic factors was similar to that described in other studies.^5,11 The consistency of results obtained in our study with those obtained in previous studies indicates the high feasibility, stability, and performance of the proposed nomogram. This nomogram will help determine the prognosis of PINHL patients in different populations and provide practical guidance to the clinicians treating these patients. In this study, an online tool based on the nomogram was also designed to aid with clinical calculations. Using this tool, clinicians simply need to enter the relevant patient information on the web page, and then the CSS probabilities for PINHL patients are automatically calculated.

Our study had several limitations. First, information on therapies used in patients and some specific indicators, such as specific types of radio or chemotherapy, IPI scores, LDH levels, and performance status, were not available for the SEER cohort. Second, it was not possible to obtain a high number of patients for external validation. Although the ROC, DCA, NRI, and IDI values were satisfactory, the 5-year calibration consistency and discriminative ability of the Kaplan–Meier survival curve were insufficient for the validation of the proposed nomogram. Third, this was a retrospective study based on limited clinical data, and it was not free from potential selection bias. Therefore, the proposed nomogram needs further assessment in prospective multicenter clinical studies to prove its clinical effectiveness.

Conclusion

In conclusion, a novel nomogram for predicting the 1-, 3-, and 5-year CSS probability in PINHL patients was developed and validated. Compared with the Ann Arbor Stage, the proposed nomogram showed superior performance in identification, calibration, and clinical guidance. Moreover, a convenient dynamic web-based survival rate calculator based on the nomogram was constructed for use by clinicians.