Development and validation of a novel diagnostic model for assessing lung cancer metastasis in a Chinese population based on multicenter real-world data

Introduction

Accurate disease staging plays an important role in lung cancer management by informing treatment choices and prognosis.¹ Scagliotti² reported that neoadjuvant chemotherapy could benefit patients with clinical stage IIB/IIIA non-small cell lung cancer (NSCLC) (including T_1-2N_1-2M₀, T_3-4N_0-1M₀), instead of patients with IB/IIA (including T_2a-2bN₀M₀) NSCLC. As the stage of lung cancer advances, the prognosis becomes significantly worse, as evidenced by the 5-year survival rate of 60–92% for localized cancer, but only 13–53%% and 0–10% for advanced and metastatic disease, respectively.^3,4 Obviously, accurate diagnosis of lymph node and organ metastasis is very important in the diagnosis and treatment of lung cancer.

CT scan is the most commonly used noninvasive method for lung cancer evaluation. However, its ability is limited in evaluating the status of lymph nodes due to the fact that the size of mediastinal lymph nodes (LDs) is hardly correlated with tumor involvement.⁵ In several reports, the sensitivity and specificity of chest CT scan for mediastinal lymph nodes' staging were only 52–57% and 81–85%, respectively (1,6). Positron-emission tomography (PET)/CT is another noninvasive technique with improved performance in assessing the status of lymph nodes, with a pooled sensitivity of 74–84% and specificity of 85–89% for assessing lymph node metastasis (1,6). PET/CT is better than traditional CT scan because it can detect the biological activity of cells. However, granulomatous lesions, infections, and other inflammatory diseases may result in false positive results on PET/CT. Furthermore, some well-differentiated low-grade malignancies, particularly bronchioloalveolar cell carcinoma and typical carcinoid tumors, may produce false negative results on PET/CT.^6–9 Additionally, the size of lesion must be adequate for detection by PET/CT because the lower limit of spatial resolution of current generation of PET scanners is approximately 7–10 mm.⁶ The accessibility of this technology may also be limited due to economic cost and radiation exposure. Therefore, a simple, convenient, reproducible, noninvasive method is desirable to facilitate the assessment of lymph node or organ metastasis.

Blood-based biomarkers have been widely used in clinical practice, such as in testicular, prostate, colorectal, breast, and ovarian cancers. The clinical quality requirements and clinical recommendations are published by the National Academy of Clinical Biochemistry (NACB).^10–12 Human epididymis protein 4 (HE4) has been shown to be a new ideal biomarker of lung cancer.^13–15 Numerous studies have confirmed that tumor markers increase significantly in advanced lung cancer, compared with early lung cancer.^16–18 Blood levels of biomarkers are indicative of tumor burden, which is correlated with tumor stage. The prevalence of tumor markers is higher in advanced stage.^19,20 Given the importance of metastasis detection, we investigated the combination of biomarkers and clinical parameters in assessing the state of LDs or organ metastasis on CT. Four nomogram-based models were built to assist assessment of the lesions detected by CT scan. To the best of our knowledge, this study was the first attempt to develop a diagnostic model based on combination of biomarkers and clinical parameters for assessing lung cancer metastasis.

Materials and methods

Overall study design

The current study was designed to evaluate a combination of 5 biomarkers, including CEA, SCC, CYRFRA21-1, ProGRP and HE4, and clinical information to predict the occurrence of LDs or organ metastasis in patients with unidentified lesions (before pathological confirmation) and patients with lung cancer (after pathological confirmation) (Figure 1). To this end, all patients were divided into nodule group and masses group according to the size of the lesion, and four predictive models were developed: two for the pulmonary nodule and the other two for pulmonary masses.

Figure 1 Overall study design. Firstly, all patients were divided into nodule group and mass group according to the size of the lesion; secondly, patients were divided into unidentified lesions group (before pathological confirmation) and lung cancer group (after pathological confirmation).

Patients

A total of 799 participants with pulmonary lesions were enrolled from three centers (Suzhou Hospital Affiliated to Nanjing Medical University, the First Affiliated Hospital of Xiamen University, and Peking University First Hospital) in China from October 2015 to August 2018. The diagnosis of lung cancer was confirmed for all patients by pathology. Lymph node or organ metastasis was confirmed by pathology (surgical resection and/or biopsy). Other metastases were diagnosed by imaging, such as ultrasonic or CT. The specific stages were based on the pathological evaluation and imaging. The staging of NSCLC and small cell lung cancer (SCLC) was determined according to the criteria of AJCC Cancer Staging Manual, 8th Edition and the Veterans Administration Lung Cancer Study Group.^3,30 All patients were naive to antineoplastic therapy, radiotherapy or chemotherapy before surgery or cancer diagnosis. The study protocol was approved by the ethics committees of the hospitals. The institutional review board numbers are K2017002 for Suzhou Hospital Affiliated to Nanjing Medical University, 2017[1294] for Peking University First Hospital and KYX-2015-020 for the First Affiliated Hospital of Xiamen University respectively. All patients signed written informed consents and this study was conducted in accordance with the Declaration of Helsinki. The patients with benign lesion included hamartoma, tuberculoma, intrapulmonary lymph nodes, and fungus ball.

Analysis of tumor markers and image acquisition

A commercial chemiluminescent microparticle immunoassay (CMIA) method was used to test blood samples for CEA, SCC, CYFRA21-1, HE4, and proGRP on ARCHITECT i2000SR (Abbott Laboratories, Chicago, IL). All samples in the training set were run in duplicate. All patients were examined by regular multi‐detector CT (MDCT) scan. The maximum dimension on axial CT images was measured and recorded by two senior consultant radiologists.

Statistical analysis

Continuous variables are described as mean (SD) or median (p25, p75) and count variables are described as frequency and percentage. All analyses were stratified by tumor size and tumor lesions. An independent-sample t-test and paired sample t-test or Mann-Whitney U test was used to compare continuous variables, while count variables were compared by χ2 test or Fisher exact test. Cut-off for each marker was determined by minimum p-value method³¹ with 1000-time bootstrap replications on purpose of overcoming the variability of a single sample. The cut-off of a single bootstrap sample was estimated by minimum p-value approach and the mean of 1000 individual cut-off values as the final cut-off.

Variables including age, sex, CEA, SCC, CYFRA21-1, HE4, and proGRP were involved in building models. Predicting factors with p<0.1 in univariate analysis were used as candidate risk factors for stepwise multivariate logistic regression model (LRM) with a backward selection. A nomogram was elaborated upon LRM coefficients by using RMS package of R. The requirement of sample size was 150 for LRM modeling with 10 variables. A larger sample size will allow a more stable statistical model. The predictive performance of the nomogram was measured by concordance index (C index) and calibrated with 1000 bootstrap samples. Finally, through decision curve analyses (DCAs),^32,33 we evaluated whether the model improved the predictive net benefit. All tests were both sided and 0.05 was set as the P-value for significance. Statistical analyses were performed using R programming language v.3.5.2.

Results

Patient characteristics

A total of 799 patients with pulmonary lesions were enrolled, including 274 patients with benign lesion, 326 patients with primary lung cancer without metastasis, and 199 patients with advanced lung cancer with lymph node or organ metastasis. The patients included 543 patients with pulmonary nodule and 256 patients with pulmonary masses. Lymph node or organ metastasis was confirmed by pathology. The characteristics of the subjects were presented in Table 1.

Table 1 Patient demographic and clinical data

Blood level of tumor markers

Compared with lung cancer without metastasis (including lung cancer and benign lesion) in pulmonary nodules, CEA, CYFRA21-1, Pro-GRP, and HE4 showed higher levels in lung cancer with metastasis. Compared with others in pulmonary masses, only CEA, CYFRA21-1, and HE4 showed higher levels in lung cancer with metastasis (Table 2). When it was compared to patients with lesion without metastasis in pulmonary nodules, CEA, CYFRA21-1, and HE4 showed higher levels in lung cancer with metastasis. While other comparisons in pulmonary masses, CEA, SCC, and CYFRA21-1 exhibited higher levels in lung cancer with metastasis (Table 2).

Table 2 Serum level of tumor markers in different lesions

Cut-off and the distribution of tumor markers

The present study divided patients into four cohorts according to the size of lesion and pathology (Figure 1). Cut-off values of markers for each model were determined by minimum p-value method with 1000-time bootstrap replications to overcome the variability of a single sample. The details of cut-off and distribution of tumor markers based on different cut-off values were shown in Table 3. Univariate analysis was used to screen risk factors, and candidate risk factors were analyzed by stepwise multivariate logistic regression model (LRM) with a backward selection.

Table 3 The distribution of tumor markers based on the cut-off values

Model 1: prediction of lymph node or organ metastasis in patients with unidentified pulmonary nodule (size≤30 mm)

A total of 543 patients with pulmonary lesions were analyzed, including 58 lung cancer patients with metastasis, 282 lung cancer patients without metastasis, and 203 patients with benign nodule. Variables, such as sex, age, size of lesion, CYFRA21-1, HE4, SCC, ProGRP, and CEA were examined in stepwise multivariate analysis. Only size of lesion and CEA were significant independent risk factors for occurrence of lymph node or organ metastasis in patients with unidentified lesion (p<0.05). The result of logistic regression was presented in Table 4. The combination of all independent risk factors can be expressed as:

Table 4 The results of logistic regression based on different lesions

CEA=1, if the level of CEA is over 5.58, otherwise =0. The AUC for model 1 in identifying metastatic lung cancer from unidentified nodule (including benign disease and lung cancer) reached 0.859 (95% CI, 0.827 to 0.887), (Table 4). The prognostic nomogram that integrated all significant independent factors for metastasis was shown in Figure 2A. Calibration curve indicated good agreement between prediction and observation in the probability of metastasis (Figure 2B). DCA was applied to render clinical validity to the nomograms (Figure 2C). It corroborated good clinical applicability of the nomograms in predicting metastasis because the ranges of threshold probabilities were wide and practical.

Figure 2 Development of nomogram to predict the occurrence of lymph node or organ metastasis in patients with nodules. (A) Model 1: for predicting the occurrence of lymph node or organ metastasis in patients with unidentified pulmonary nodule (size≤30 mm). To use the nomogram, an individual patient’s value is located on each variable axis, and a line is drawn upward to determine the number of points received for each variable value. The sum of these numbers is located on the Total Points axis, and a line is drawn downward to Risk axes to determine the risk of metastasis in patients with unidentified lesions. (B) The calibration curve for predicting the occurrence of lymph node or organ metastasis in patients with unidentified pulmonary nodules. (C) Decision curve analysis of nomograms based on model 1. (D) Model 2: for predicting the occurrence of lymph node or organ metastasis in patients with lung cancer (size≤30mm). (E) The calibration curve for predicting the occurrence of lymph node or organ metastasis in patients with lung cancer (size≤30mm). (F) Decision curve analysis of nomograms based on model 2.

Model 2: prediction of lymph node or organ metastasis in patients with lung cancer (size≤30mm)

Different from model 1, patients with benign nodule were excluded. This model was used to predict the occurrence of metastasis in lung cancer patients (size≤30mm). The study population consisted of 340 patients, including 58 lung cancer patients with metastasis and 282 lung cancer patients without metastasis. The result of multivariate analysis was similar to model 1. The size of lesion and CEA were independent risk factors for occurrence of lymph node or organ metastasis in patients with lung cancer (p< 0.05). The result of logistic regression was presented in Table 4. The combination of all independent risk factors can be expressed as:

CEA=1, if the level of CEA is over 6.12, otherwise = 0. The AUC for model 2 in identifying metastatic lung cancer from lung cancer (size≤30mm) reached 0.838 (95% CI, 0.795 to 0.876), (Table 4). The prognostic nomogram that integrated all significant independent factors for metastasis was shown in Figure 2D. Calibration curve showed good agreement between prediction and observation in the probability of metastasis (Figure 2E). DCA was applied to render clinical validity to the nomograms (Figure 2F). It corroborated good clinical applicability of the nomograms in predicting metastasis because the ranges of threshold probabilities were wide and practical.

Model 3: prediction of lymph node or organ metastasis in patients with unidentified pulmonary masses (size >30 mm)

In this model, we analyzed 256 patients with pulmonary masses, including 141 lung cancer patients with metastasis, 44 lung cancer patients without metastasis, and 71 patients with benign masses. Sex, age, size of lesion, CYFRA21-1, HE4, SCC, ProGRP, and CEA were the variables included in stepwise multivariate analysis. Only CEA and CYFRA21-1 were independent risk factors for occurrence of lymph node or organ metastasis in patients with unidentified masses (p<0.05). The result of logistic regression was presented in Table 4. The combination of all independent risk factors can be expressed as:

CEA=1, if the level of CEA is over 3.38, otherwise = 0; CYFRA21-1=1, if the level of CYFRA21-1 is over 1.95, otherwise = 0. The AUC for model 3 in identifying metastatic lung cancer from unidentified masses (including benign disease and lung cancer) reached 0.773 (95% CI, 0.717 to 0.823), (Table 4). The prognostic nomogram that integrated all significant independent factors for metastasis was shown in Figure 3A. Calibration curve showed good agreement between prediction and observation in the probability of metastasis (Figure 3B). DCA was applied to render clinical validity to the nomograms (Figure 3C). It corroborated good clinical applicability of the nomograms in predicting metastasis because the ranges of threshold probabilities were wide and practical.

Figure 3 Development of nomogram to predict the occurrence of lymph node or organ metastasis in patients with mass. (A) Model 3: for predicting the occurrence of lymph node or organ metastasis in patients with unidentified pulmonary mass (size >30 mm). To use the nomogram, an individual patient’s value is located on each variable axis, and a line is drawn upward to determine the number of points received for each variable value. The sum of these numbers is located on the Total Points axis, and a line is drawn downward to Risk axes to determine the risk of metastasis in patients with unidentified lesions. (B) The calibration curve for predicting the occurrence of lymph node or organ metastasis in patients with unidentified pulmonary mass. (C) Decision curve analysis of nomograms based on model 3. (D) Model 4: for predicting the occurrence of lymph node or organ metastasis in patients with lung cancer (size >30mm). (E) The calibration curve for predicting the occurrence of lymph node or organ metastasis in patients with lung cancer (size >30 mm). (F) Decision curve analysis of nomograms based on model 4.

Model 4: prediction of lymph node or organ metastasis in patients with lung cancer (size >30mm)

This model was used to predict the occurrence of metastasis in lung cancer patients (size>30mm), which included 141 lung cancer patients with metastasis and 44 lung cancer patients without metastasis. Patients with benign nodule were excluded. Age, SCC, CYFRA21-1, pro-GRP, and HE4 were independent risk factors for occurrence of lymph node or organ metastasis in patients with lung cancer (size>30mm) (p< 0.05). The result of logistic regression is presented in Table 4. The combination of all independent risk factors can be expressed as:

SCC=1, if the level of SCC is over 1.06, otherwise = 0; CYFRA21-1=1, if the level of CYFRA21-1 is over 2.91, otherwise = 0; pro-GRP =1, if the level of pro-GRP is over 22.12, otherwise = 0; HE4=1, if the level of HE4 is over 96.35, otherwise = 0. The AUC for model 4 in identifying metastatic lung cancer from lung cancer (size>30mm) reached 0.731 (95% CI, 0.661 to 0.793), (Table 4). The prognostic nomogram that integrated all significant independent factors for metastasis was shown in Figure 3D. Calibration curve showed good agreement between prediction and observation in the probability of metastasis (Figure 3E). DCA was applied to render clinical validity to the nomograms (Figure 3F). It corroborated good clinical applicability of the nomograms in predicting metastasis because the ranges of threshold probabilities were wide and practical.

Discussion

Outcome of lung cancer patients are subjected to clinical and pretreatment evaluation, especially for lymph nodes. Such evaluations are important for decisions regarding invasive procedures or treatments, such as neoadjuvant chemotherapy. From 2007 to 2013, among all the reported lung cancer cases, only 16% were diagnosed at early stage when the cancer was limited to primary site. Around 22% were diagnosed after the cancer had spread to regional lymph nodes or directly beyond the primary site. Unfortunately, at least half of cases (57%) were diagnosed at late stage after the cancer had already metastasized.^2,21 In order to improve the clinical efficiency, we developed a diagnosis model to differentiate cancer and benign nodules.²² However, this is far from fulfilling the current clinical need. Accurate clinical evaluation of lymph node or organ metastasis is still a challenge but important for newly diagnosed lung cancers. In the present study, we built a novel diagnostic model to assess lung cancer metastasis based on combination of biomarkers, lesion size, and characteristics of patients, which will provide new decision support in clinical practice. In general, four different models were built in our article. When we found lesions in CT screening test, the nature of the lesion was unclear and Models 1 and 3 can assist physician to assess risk of malignancy and metastasis of the lesions. When the nature of the lesion was identified by biopsy, bronchoscope or other methods, Models 2 and 4 can be used. The current golden standard for metastasis is still pathological examination. PET/CT also provides valuable information. However, the cost is too high. Thus, one important clinical application is to assist the metastasis risk assessment before further clinical decision.

Due to its simple, convenient, inexpensive, and reproducible application, tumor markers have been used in lung cancer assessment for decades. However, most of the previous studies have focused on their diagnostic and prognostic utilities. The implication of these tumor markers in the evaluation of lymph node or organ metastasis has not been systematically evaluated. Thus, this large-scale study further explored the potential clinical application of tumor biomarkers in metastasis assessment of lung cancer. The performances of the five common biomarkers (CEA, CYFRA21-1,Pro-GRP, SCC, and HE4) exhibited different levels among subgroup analysis.

Data from the Dutch Belgian Randomized Lung Cancer Screening (NELSON) trial of CT scan screening showed that malignancy was associated with larger nodule size for solid nodules.²³ Besides, 30mm is the boundary between pulmonary nodules and pulmonary masses.⁶ So we chose 30mm as cutoff. We classified pulmonary lesions into nodules and masses according to the size of lesions. According to different clinical applications, four models to assess lung cancer metastasis have been developed. In pulmonary nodules group and pulmonary masses group, one model was designed to assess the risk of lymph node or organ metastasis in case of unidentified lesions, and the other one was developed to evaluate the risk of lymph node or organ metastasis in lung cancer only. For patients with pulmonary nodules, independent risk factors for metastasis are size and CEA. This may be related to the high percentage of adenocarcinoma patients in the patient pools because this population is known to have high CEA levels. Additionally, the AUC for diagnosing lymph node or organ metastasis is better when the diameter of lesion is smaller than 30 mm. This may be due to the accumulation of tumor biomarkers in late stage tumors regardless of metastasis or not, as the level of tumor marker is correlated to lymph node metastasis and lesion size.²⁰ Although the level of tumor markers is less efficient in predicting lymph node metastasis at later stage, fortunately, metastasis of larger lesions (≥30 mm) is much easier to detect by CT/MRI. Thus, the proposed models in this study can be used as a tool and decision support supplementary to CT/MRI in assessing lung cancer metastasis, especially in smaller lesions (< 30 mm).

For patients with pulmonary masses, independent risk factors for metastasis include CEA, CYFRA21-1, and HE4. Results show that SCC and pro-GRP are in negative correlation to the risk of metastasis. Nevertheless, previous research²⁴ reports that SCC has nothing to do with metastasis. The results may be attributed to selection bias. Besides, pro-GRP is mainly used in the diagnosis and evaluation of neuroendocrine tumors. The number of neuroendocrine tumors included in our trial is small.

Modeling and statistical construction is a common practice to assist diagnosis or estimate prognosis. A growing body of evidence suggests that the combination of biomarkers and tumor morphology, such as size, can improve the accuracy of the model. For example, in the field of lung cancer diagnosis, Yang et al built a diagnostic model based on biomarkers and the most predictive imaging features of lung cancer.²⁵ The model demonstrates better accuracy on risk prediction than the ACCP lung nodule model. In addition, the information provided by the biomarker is no longer limited to its normal range. More useful information can be generated when biomarkers are combined with clinical data. Wang et al developed a prognostic nomogram for resectable intrahepatic cholangiocarcinoma based on tumor markers and clinicopathologic data.²⁶ The model can provide more accurate prediction of patient survival compared with the staging systems currently available.²⁶ The risk prediction models proposed in this study were based on nomogram, which can provide individualized, evidence-based, and highly accurate estimation of cancer metastasis risk by combining multiple independent variables and assigning an appropriate weight to each variable based on its prognostic value.²⁷ Additionally, nomograms are easy to use and can facilitate management-related decision making.^28,29 To the best of our knowledge, this was the first attempt to develop a validated nomogram for assessing the risk of lymph node or organ metastasis in lung cancer based on tumor markers and the size of lung lesions.

There are some limitations to our study. Firstly, the number of patients included in our study was still limited. A larger sample size will provide a more reliable model. Secondly, internal validation was used in the study. The validity of these models needs to be verified further with larger external data. Thirdly, the performance of CT scan alone in diagnosing mediastinal lymph node metastasis was not evaluated in our analysis. We will carry out further randomized prospective study to investigate whether the diagnosis of mediastinal lymph node metastasis can be improved compared with CT alone. Fourthly, only several commonly used tumor markers were analyzed in this study, which may not be enough to reflect the full profile of tumor markers or clinical biomarkers.

In general, a novel pretreatment evaluation nomogram model based on blood level of tumor markers and size of pulmonary lesion was developed and validated. This proposed model is valuable in estimating the risk of lymph node or organ metastasis for unidentified lung lesions or identified lung cancer.