Nomogram For The Prediction Of Malignancy In Small (8–20 mm) Indeterminate Solid Solitary Pulmonary Nodules In Chinese Populations

Introduction

The incidences of solitary pulmonary nodules (SPNs) detected by thoracic computed tomography (CT) performed for lung cancer screening or other reasons are increasingly common.^1,2 SPNs are classifiable into solid or subsolid types according to their CT manifestation. Subsolid SPNs have a significantly higher likelihood of being malignant than solid counterparts;³ the risk of malignancy among subsolid SPNs is over 90%, while that among solid SPNs ranges from 5% to 70% among patients who undergo surgical resection.^3–5 Hence, the timely and accurate identification of SPN type is critical for the management of adults with solid SPNs.

CT-based surveillance is recommended for adults with solid SPNs ≤8 mm because the chances of malignancy in nodules of this size are low. Meanwhile, the guidelines for adults with solid SPNs >8 mm recommend follow-up with CT after 3 months, positron emission tomography (PET)/CT, or tissue biopsy.^2,6,7 Some individuals with solid SPNs >8 mm, particularly those who are anxious, may prefer tissue sampling or PET/CT in order to detect any malignancy as soon as possible.⁸ When tissue sampling is required, guided bronchoscopy technologies or transthoracic needle aspiration is recommended.^9–11 The diagnostic yields of guided bronchoscopy, transthoracic needle aspiration, and PET/CT are reportedly 82.5%, 86.7%, and 84.1%, respectively, for SPNs >20 mm and are 60.9%, 78.9%, and 64.2%, respectively, for SPNs ≤20 mm.^9–12 As such, PET/CT or tissue sampling may not be ideal for evaluating solid SPNs that are 8–20 mm in size, necessitating alternative means of identifying malignancies.

Some prediction models have been developed to estimate the malignancy of SPNs, and have been used extensively in clinical practice.^3,5,12–17 However, these models may not be applicable for solid SPNs that are 8–20 mm, since studies that devised these models may not have necessarily evaluate the patients’ final diagnoses,¹³ may have evaluated both solid and subsolid SPNs together,³ or were based on general lung cancer screenings that may have missed incidental pulmonary nodules.

Hence, the aim of our study was to develop and validate a nomogram for predicting the malignant potential of small (8–20 mm) indeterminate solid SPNs that were detected using routine clinical examinations and CT image data in a Chinese population.

Materials And Methods

The local ethics committee of the Yunnan Cancer Hospital, Xishan Region, Kunming, People’s Republic of China, approved this retrospective study, which complied with the Declaration of Helsinki and good clinical practice guidelines. The requirement for informed consent was waived by the committee owing to the study’s retrospective nature. All the patient data in the survey were anonymized.

Patients

We retrospectively identified consecutive patients with small indeterminate solid SPNs who underwent surgical resection between December 2012 and March 2018 at Yunnan Cancer Center, China. The patients were divided into two separate cohorts: 493 patients treated between December 2012 and December 2016 comprised the training cohort, whereas 216 patients treated between January 2017 and May 2018 comprised the validation cohort. Supplemental Figure 1 shows the study flowchart, including the inclusion and exclusion criteria.

Thoracic CT Scans And Image Analysis

CT was performed with a 128-slice CT scanner (SOMATOM Definition AS+, Siemens Medical Solutions, Forchheim, Germany). CT examination included unenhanced imaging with a tube voltage of 120 kVp (100 mA/s, CARE kV on, CARE dose 4D on, pitch = 0.6, rotation = 0.5 s, display field of view = 42 cm², matrix = 512 × 512, and collimation = 128 × 0.6 mm). Patients were then injected with nonionic contrast material (Ultravist 300, Bayer Schering Pharma AG, Berlin, Germany; or Iohexol 300, Yangtze River Pharmaceutical Group, Taizhou, China) with antecubital venous access at a rate of 3.5 mL/s. A total of 80–100 mL (1.5 mL per kilogram of body weight) was injected with a CT-compatible power injector (Bracco ACIST EZEM, Empower CTA) followed by a 20 mL saline flush at the same rate. Scanning for contrast enhancement was performed 30 and 50–60 seconds after injection. The images were reconstructed using lung- and soft tissue-based algorithms with a thickness of 1 mm, as described previously.¹⁸ The CT features were assessed in the following settings: lung window center, −500 Hounsfield units (HU)/lung window width, 1,500 HU; mediastinal window center, 40 HU/mediastinal window width, 350 HU. All image features were prospectively evaluated by a thoracic radiologist with 11 years of experience who was blinded to the histopathological results of the SPNs.

Variables

The present study analyzed the following variables:

1. Patient characteristics including age, sex, body mass index, smoking history (yes or on), chronic interstitial or obstructive lung disease (yes or no), family history of lung cancer (yes or no), and history of extra-thoracic malignant neoplasm (>5 years ago, yes or no).
2. Image-based features consisted of emphysema (yes or no) and nodule parameters including diameter, upper lobe location (yes or no), marginal spiculation (yes or no), significant lung nodule enhancement (yes [>15 HU] or no [≤15 HU]), and pleural indentation (yes or no) based on previous studies.^5,13,15
3. Tumor biomarkers including carcinoembryonic antigen (CEA, normal [≤5.0 μg/L] or abnormal [>5.0 μg/L]), carbohydrate antigen 125 (normal [≤35.0 kU/L] or abnormal [>35.0 kU/L]), carbohydrate antigen 199 (normal [≤27.0 kU/L] or abnormal [>27.0 kU/L]), carbohydrate antigen 724 (normal [≤6.9 kU/L] or abnormal [>6.9 kU/L]), neuron-specific enolase (normal [≤16.3 μg/L] or abnormal [>16.3 μg/L]), squamous cell carcinoma antigen (normal [≤1.5 μg/L] or abnormal [>1.5 μg/L]), and ferritin (normal [13–150 μg/L] or abnormal [<13 or >150 μg/L]).

SPN Diagnosis

All the SPNs were diagnosed by histopathological examination after surgical resection and were thereby categorized into the benign and malignant groups.

Statistical Analysis

The R software (version 3.4.0; http://www.R-project.org) was used to perform all statistical analyses. The expectation-maximization imputation method was used to impute missing values for covariates with lapses of <10%.¹⁹ The reported statistical significance levels are all two-sided, with statistical significance set at a P-value <0.05.

Mean values ± standard deviations are provided for normally distributed variables, whereas frequencies are provided for categorical data. Differences in characteristics between the training and validation cohorts and between the benign and malignant groups in each of these cohorts were analyzed using the independent-samples t-test or Chi-square test.

Univariate and multivariate logistic regression models were used to calculate the odds ratios and 95% confidence intervals. Independent predictors of malignancy in solid SPNs that were 8–20 mm was determined using the multivariate logistic regression model using the training data (P < 0.05). The prediction model was revised to incorporate only the significant predictors. The model’s performance was evaluated using bootstrapping, in which the dataset was tested 1,000 times with random resampling each time (i.e., patient overlap was possible).

Evaluation Of The Prediction Model

Discrimination

The Harrell concordance index (C-index) was calculated as a measure of the goodness of fit for the prediction model; this index is equal to the area under the receiver operating characteristic (ROC) curve and ranges from 0.5 to 1. The discrimination slope, which is defined as the slope of the linear regression of predicted probabilities for events derived from a binary event status model, is used as a measure of discriminative ability.²⁰

Reclassification

The net reclassification improvement (NRI) and integrated discrimination improvement (IDI), which were calculated to measure the improvement in prediction performance gained by adding a marker, were used to compare the discriminative abilities of two models.²¹

Calibration

Calibration is used to describe the consistency between observed outcomes and predicted values. The calibration curve was represented as predictions on the x-axis and actual outcomes on the y-axis and was evaluated using the Hosmer-Lemeshow test.

Clinical Usefulness

The decision curve analysis (DCA) proposed by Vickers et al is a simple method to quantify the clinical utility of a predictive model (or an extension to such a model).²² Instead of the net benefit (NB), the standardized NB (sNB) was used in this study as a function of the risk threshold in the decision curve, as the sNB was much easier to interpret and compare (sNB values range from 0 to 1). The sNB was calculated at the cut-off of 0.5 of the risk threshold, which implied a relative weight of 1:1 for false positive decisions against true positive decisions.²³ The clinical impact plot was used to estimate the cut-off number that would determine the high risk point for each threshold, and to illustrate their proportions.²⁴ The ROC component plot shows the constituents of sNB; i.e., the true- and false- positive rates, where the former is displayed as a convex function of the risk threshold and the latter as a concave function of the same.²⁴

Nomogram

A nomogram was constructed based on the logistic regression model,²⁵ and the probabilities of malignancy in patients with 8–20 mm solid SPNs who were of different ages and had different imaging features were calculated directly using the predictive model, which was listed as a table for the ease of clinical use.

Results

Clinical Characteristics

The features of patients in the training and validation cohorts are shown in Table 1 and Supplemental Table 1. There were no significant differences between the two cohorts in terms of patient characteristics, image-based features, and tumor biomarkers (P-values ranged between 0.110 and 0.985). There was also no significant difference in the prevalence of malignant solid SPNs between the two cohorts (P = 0.509); the malignancy rates in the training and validation cohorts were 43.41% and 40.74%, respectively.

Table 1 Characteristics Of The Patients In The Training And Validation Cohorts

Identifying Predictors And Developing The Model

Supplemental Table 2 shows the results of univariate logistic regression analyses performed to identify potential clinical and CT-based predictors of malignancy in small indeterminate solid SPNs in the training, validation, and pooled cohorts. These analyses showed that age, diameter, upper lobe rate, marginal spiculation rate, significant enhancement rate, pleural indentation rate, emphysema rate, CEA level, and ferritin level were associated with the probability of malignancy in each of the three cohorts (all P < 0.05).

Multivariate logistic regression analyses revealed that age, marginal spiculation, significant enhancement, pleural indentation, and abnormal CEA levels were independent predictors of malignancy (Table 2). As such, we developed two models: model 1 incorporated age, marginal spiculation, significant enhancement, and pleural indentation, while model 2 incorporated model 1 factors plus CEA level.

Table 2 Predictors Of Malignancy In Indeterminate Solid Solitary Pulmonary Nodules 8–20 Mm In Size

Model Performance

Table 3 shows the C-index, discrimination slope, calibration slope, and sNB of models 1 and 2, respectively, within the training and validation cohorts. The C-index and sNB were not significantly different between model 1 and model 2 in either of the two cohorts.

Table 3 Performance Of The Models Predicting Malignancy In Indeterminate Solid Solitary Pulmonary Nodules 8–20 Mm In Size Within The Training Cohort (n = 493) And Validation Cohort (n = 216)

The integration of CEA into the prediction model showed significantly improved prediction performance in terms of NRI in the validation cohort (0.200; P < 0.001) but did not show significantly improved prediction performance in terms of IDI (0.001; P = 0.941) (Supplemental Table 3).

The Hosmer-Lemeshow calibration test showed no significant difference in the model 1-predicted versus observed malignant SPNs in either the training cohort (P = 0.337) or the validation cohort (P = 0.246) (Figure 1).

Figure 1 Plots depict the calibration of model 1 in terms of agreement between the predicted and observed malignant solitary pulmonary nodules (SPNs) in the training (A) and validation (B) cohorts. The nomogram estimated malignant SPNs are plotted on the X-axis, while the observed malignant SPNs are plotted on the Y-axis. The diagonal dotted line indicates perfect calibration based on an ideal model that would reflect the outcomes perfectly. The solid line indicates the nomogram’s actual performance; a close alignment between the solid and dotted lines indicates better estimation of the actual outcomes.

The DCA plot showed that the probability of malignancy as predicted by each of model 1 and model 2 provided better outcomes than scenarios in which either all patients were treated, or no patients were treated. Model 2 did not outperform model 1, with risk thresholds ranging from 0.10 to 0.85 in both the training and validation cohorts (Figure 2A and B). The clinical impact plot shows that if a 0.5 risk threshold was used in model 1, approximately 382 of 1,000 men would be predicted to have a high risk of malignancy, with approximately 351 of these being actual cases of malignancy in the validation cohort (Figure 2C and D). The true- and false-positive fractions are displayed as convex and concave functions, respectively, of the risk threshold in the ROC components plot for the model in the training and validation cohorts (Figure 2E and F).

Figure 2 Decision curves for two risk prediction models for the malignancy of indeterminate solid solitary pulmonary nodules (A, B). The vertical axis shows the net benefit of standardization. The two horizontal axes display the correspondence between the risk threshold and cost-benefit ratio. The clinical impact curves for model 1 are shown in (C) and (D). Of 1,000 patients, the solid blue line indicates the total number of people who would be considered at high-risk for each risk threshold. The solid gold line indicates the number of true positive cases. True- and false-positive rates, as functions of the risk threshold for model 1, are shown in (E) and (F). The figure shows information similar to that of a receiver operating characteristic curve and also shows the risk threshold corresponding to each true- and false-positive rate. Bands on all plots represent pointwise 95% confidence intervals constructed via bootstrapping. Left: the training cohort (A, C, and E); right: the validation cohort (B, D, and F).

Nomogram

Model 1 was used as a nomogram for easy clinical use (Figure 3). Supplemental Table 4 shows the probability of malignancy among small indeterminate solid SPNs among patients of different ages using eight combinations involving the presence and absence of selected categorical predictors (marginal spiculation, significant enhancement, and pleural indentation), which were derived using model 1.

Figure 3 Nomograms for predicting the malignancy of 8–20 mm indeterminate solid solitary pulmonary nodules. Each variable was assigned a score, the sum of which was converted into a probability on the lowest scale.

Discussion

We developed and validated a nomogram for predicting the malignancy of small (8–20 mm) indeterminate solid SPNs based on routine clinical and CT image data. The nomogram incorporates four items: age, marginal spiculation, significant enhancement, and pleural indentation. This nomogram has similar performance to, but is simpler than, currently available nomograms or models.^3,12–17

To our knowledge, ours is the first nomogram or model that focuses on predicting the malignancy of 8–20 mm indeterminate solid SPNs, which is more challenging to do owing to the smaller nodules and more rigorous research standards required.^3,12–17 First, most previous models focused on the SPNs in general, both solid and subsolid.^3,12–14 However, previous models for subsolid SPNs may outperform those for solid SPNs in terms of predicting the probability of malignancy because subsolid SPNs are significantly more likely to be malignant.³ Second, some previous models focused on solid SPNs, including those 20–30 mm or 8–20 mm in size.^15–17 Models for 20–30 mm solid SPNs may outperform those used for 8–20 mm counterparts in terms of predicting the probability of malignancy because such a probability increases among larger SPNs.¹³ Third, our study focused on indeterminate solid SPNs diagnosed on imaging without identifying their types, such as hamartomas and arteriovenous malformations, as had been performed in previous studies.^15–17

Our study showed that features including age; diameter; rates of upper lobe tumors, marginal spiculation, significant enhancement, pleural indentation, and emphysema; CEA level, and ferritin level are associated with the probability of malignancy in indeterminate solid SPNs 8–20 mm, which is consistent with previous studies.^3,12–18 However, we also found that only age, marginal spiculation, significant enhancement, pleural indentation, and abnormal CEA level were independent predictors of malignancy of small indeterminate solid SPNs, which differs from the results of previous studies.^3,12–18 Smoking history, upper lobe location, and diameter, which were independent predictors of SPN malignancy in most previous studies, were not so in small solid SPNs according to our data. This could be related to differences in characteristics between Chinese and European/American patients, among other reasons. First, the risk of lung cancer owing to air pollution (such as cooking oil vapor and passive smoke experienced by Chinese non-smokers, especially women) is higher than that owing to cigarette smoking among Chinese populations; this is not the case among Europeans and Americans.²⁶ Second, tuberculosis, which often requires a definitive diagnosis because of both individual treatment and public health implications, is often located in the upper lobe, as are malignant SPNs; however, the incidence rate of tuberculosis among Chinese people is higher than that among Europeans and Americans.²⁷ Third, it has been speculated that a correlation exists between marginal spiculation, pleural indentation, and SPN size in terms of prediction values, and the predicted values of marginal spiculation and pleural indentation are higher than that of SPN size; this warrants further investigation.

A strength of our study was that we confirmed the value of significant lung nodule enhancement (>15 HU) for predicting the malignancy of small indeterminate solid SPNs; this was an independent predictor in our model. We utilized clinical and imaging information (including enhancement CT) to the fullest when developing the prediction model, thereby differentiating them from previously described counterparts.^3,12–18 Additionally, our model outperformed those that used PET/CT or tissue sampling for predicting the malignancy of small indeterminate solid SPNs,^9–12 and was therefore more useful for assessing the probability of malignancy in such lesions to enable optimal clinical decision-making. Of note, our model is not necessarily applicable for lung screening.

The CEA level was not included in our nomogram, which may be controversial. A previous study found that incorporating CEA level as an additional predictor only incrementally increased the reliability of solid SPN diagnoses. However, we observed no significant differences in the C-index, calibration, sNB, or IDI between the prediction model that incorporated CEA and the model that did not, although the INR was significantly improved in the former. Our nomogram that does not include CEA enhances the generalizability of our findings to low- and middle-income areas in which tumor biomarkers testing (such as CEA) is difficult or unavailable.

Our study had three major limitations. First, it was a single-center retrospective investigation; the accuracy of our prediction model ought to be confirmed in a multicenter prospective study, which could enable the use of our nomogram as a clinical aid for predicting the malignancy of small indeterminate solid SPNs. Second, we did not use radiomics, which is a popular method that may markedly improve the predictive value of our model. However, models based on radiomics are sometimes limited in their generalizability. In contrast, our current model arguably improves generalizability by using routine clinical and CT image data. Third, the rate of benign lesion is relatively high in our study. It was because that most of these individuals with solid SPNs >8 mm, particularly those who are anxious, may prefer direct surgical resection rather than follow-up CT surveillance to detect any malignancy in the past ten years in Yunnan, China.

Conclusion

Age, marginal spiculation, significant enhancement, and pleural indentation are independent predictors of small indeterminate solid SPN malignancy. We developed and validated a prediction model and nomogram that were more accurate than PET/CT or tissue sampling in terms of predicting the malignancy of such SPNs. Our nomogram is easy to use and may allow clinicians to accurately predict the malignant potential of small indeterminate solid SPNs in Chinese populations.