Development of a Clinical-Radiomics Nomogram That Used Contrast-Enhanced Ultrasound Images to Anticipate the Occurrence of Preoperative Cervical Lymph Node Metastasis in Papillary Thyroid Carcinoma Patients

Introduction

The thyroid gland, composed of two connected lobes, holds importance as a notable endocrine organ. Thyroid lesions, with a prevalence ranging from 4% to 7%, frequently present as asymptomatic conditions with maintained hormone secretion. The majority of these lesions are non-cancerous, encompassing entities like simple cysts, colloid nodules, and thyroid adenomas.¹ However, studies have shown that 80–90% of patients with thyroid cancer (TC) are affected by papillary thyroid carcinoma (PTC), which is a prevalent histological type of TC and the ninth most common type of malignancy worldwide.² Although PTC displays a low death rate and good prognosis, almost 50% of PTC cases show the presence of cervical lymph node metastasis (LNM), which can increase the probability of local recurrence and decrease the overall survival rates.^3–5 Although surgery is still the main form of therapy for PTC, further research is needed to determine whether patients with PTC can benefit from preventive cervical lymph node (LN) dissection due to the lack of accurate preoperative techniques for predicting cervical LNM. While some experts have highlighted the advantages of prophylactic cervical LN dissection in PTC patients with,⁶ a few researchers have stated that this surgery could increase the likelihood of postoperative complications without substantially improving survival rates.⁷ Hence, the accuracy of preoperative LNM evaluation needs to be increased to understand the optimal extent of PTC surgery and improve patient outcomes.

However, the noninvasive assessment of cervical LNM before surgery poses a challenge. Ultrasound (US) imaging is commonly employed and is significant because radiologists can identify cervical LNM by examining the sonographic characteristics of the bilateral cervical lymph nodes.⁸ On the other hand, the conventional US technique displays an unsatisfactory metastatic lymph node detection rate, particularly in central regions.⁹ Radiomics has emerged as a promising noninvasive and accurate diagnostic approach and a potential treatment strategy, as it extracts complete quantitative data from medical images.¹⁰ This enables the development of models that capture the intrinsic heterogeneity of tumors, aiding clinical decision-making.¹¹ Several researchers have used radiomics techniques to predict LNM occurrence from primary tumors in different types of cancers, such as cervical cancer¹² and laryngeal squamous cell carcinoma.¹³ In the case of PTC, several studies have demonstrated the feasibility of utilizing B-mode ultrasound analysis based on ultrasound elastography and B-mode ultrasound (BMUS) to predict cervical LNM.^14,15 However, very few studies have employed contrast-enhanced ultrasound (CEUS)-based radiomics to predict the presence of LNM in PTC. Additional research is needed to determine whether the integration of CEUS-based radiomics and clinical risk factors could increase the accuracy of LNM prediction in PTC.

In this study, a novel nomogram model that combines clinical data and CEUS radiomics was developed and verified for the non-invasive prediction of cervical LNM among patients with PTC to design strategies for individualized treatment. For this purpose, a multivariate regression analysis technique was used to develop a nomogram that combined multiple predictors, which improved the graphical representation of the numerical association between risk factors and disease characteristics. The nomogram employed a scoring system and offered a numerical probability for the occurrence of the final outcome.^16,17 This model could be used by clinicians to enhance their ability to anticipate the occurrence of cervical LNM in patients with PTC, thus facilitating the development of personalized treatment strategies.

Methods

Participants

An ethics waiver for this research was approved by the ethics committee of The First Affiliated Hospital of Wannan Medical College because it is a retrospective study using data routinely collected, and the study population was recruited. However, as this was a retrospective study, written informed consent was not obtained from the PTC patients. This study included patients with thyroid nodules who underwent preoperative ultrasound examination at our hospital between January 2020 and December 2022. The inclusion criteria used in this study consisted of patients with confirmed PTC through postoperative pathology, thyroidectomy, and cervical LN dissection as a form of treatment; single primary thyroid carcinoma; lack of any prior anticancer treatment; and ultrasound examinations that were conducted within 2 weeks before surgery. On the other hand, the exclusion criteria included inadequate visualization of the lesion on ultrasound images because of its large size or poor image quality. This study included 423 patients categorized into LNM and non-LNM groups, depending on their postoperative pathology.

Ultrasound Protocols

Patients in both groups were subjected to ultrasonographic examinations using a GE LOGIQ E9 color Doppler ultrasonic instrument (GE, Chicago, IL, USA) with a 9 L linear array probe (2–9 MHz). The procedure involved BMUS assessment of the thyroid gland, where the largest long-axis section of the lesion was saved as a BMUS image. Real-time CEUS was then performed by switching to the US mode. During CEUS, the patient was instructed to remain calm while the observation sections were not changed. SonoVue (Bracco, Italy) was used as the contrast agent with a bolus injection (2.4 mL, followed by normal saline (5 mL). US physicians recorded the dynamic perfusion procedure of the lesions and the frames at the CEUS peak time were stored. Ultimately, two images of every nodule (CEUS and BMUS images) were exported in the DICOM format.

The US reports that showed the presence of an LNM was regarded as a positive US-reported LN status, whereas the US reports that showed the absence of metastasis were regarded as negative US-reported LN status based on conclusions such as “undetectable LN”, “reactive hyperplastic lymph nodes”, and “visible LN” without metastasis. Suspicious ultrasound signs indicating cervical lymph node metastasis, as outlined in the guidelines presented by the American Thyroid Association (ATA, 2015),¹⁸ included a round shape (aspect ratio > 0.5), hyperechogenicity, cystic alterations, calcifications, and peripheral blood flow signals. If one or more lymph nodes satisfied any of the above criteria, they were classified as positive for lymph node metastasis. The following US characteristics were recorded in this study: lesion site, location within the thyroid gland, tumor size, echogenicity, aspect ratio, margin characteristics, presence of calcifications, and enhancement patterns. Demographic information such as sex and age was also collected.

Segmenting the Images and Extracting All Features

The nodules were segmented with the help of the ITK-SNAP software, and interobserver reproducibility was assessed by two US physicians independently, who segmented a subset of 30 randomly selected cases. The 3D-Slicer software was employed for radiomics feature extraction, with image normalization and resampling to a voxel size of 1 mm × 1 mm × 1 mm. In this study, US physicians extracted 912 radiomics features, including first-order statistics, gray level co-occurrence matrix (GLCM), gray-level dependence matrix (GLDM), gray level run length matrix (GLRLM), and gray level size zone matrix (GLSZM).

Selecting the Features and Constructing the Radiomics Scores

The 423 patients were randomly divided into training (n=282) and validation (n=141) groups using a 2:1 stratified random sampling method. The z-score normalization procedure involved training the set data to normalize the radiomics features. Feature selection and score construction involved several steps: (1) calculating the interclass correlation coefficient (ICC) to determine the reproducibility and retaining features with ICC > 0.75; (2) applying the minimum redundancy maximum correlation (mRMR) algorithm to eliminate redundant and irrelevant features, resulting in the selection of the top 30 features; (3) the least absolute shrinkage and selection operator (LASSO) algorithm to detect the features that could predict the occurrence of LNM; and (4) employing backward stepwise logistic regression (LR) with the Akaike information criterion (AIC) to select the features and develop the LR model. The resulting model scores were defined as the radiomics score (Radscore) and computed separately for the BMUS and CEUS images. The association between Radscore and cervical LNM was studied using the Mann–Whitney U-test to compare the score distribution in the “with LNM” and “without LNM” groups.

Constructing the Clinical and Clinical-Radiomics Nomogram Models

The data from the training set were used to construct the predictive model, whereas those from the validation set were used for evaluation. Univariate analysis was conducted using the training set to determine the relationship between clinical parameters, Radscores, and occurrence of cervical LNM. Clinical parameters (p-value < 0.05) were considered candidate predictors. Stepwise multivariate LR analysis was performed to develop a clinical model based on selected predictors. A backward stepwise selection technique with AIC was used to identify the most relevant predictors in the clinical model. Radscores and clinical risk factors were incorporated into the multivariate LR analysis to develop a combined clinical radiomics model. Backward stepwise selection with AIC was then employed to determine the final set of predictors in the constructed clinical radiomics model. A nomogram based on a clinical radiomics model was developed for individual risk assessment. The nomogram visually represents the LR model and helps estimate an individual’s probability of cervical LNM based on their specific clinical characteristics and radiomic features.

Model Validation

In this study, the predictive capacity of the clinical radiomics nomogram model was used to assess ROC curves, whereas the AUC was used as a measure of discriminative performance. The integrated discrimination improvement (IDI) index was used to evaluate and compare the incremental predictive accuracy of novel clinical radiomics and clinical models. The performance of the clinical-radiomics nomogram model was calibrated using the Hosmer-Lemeshow test and calibration curves, which compared the predicted probabilities with the observed outcomes. Decision curve analysis (DCA) was conducted to calculate the clinical applicability of the clinical radiomics nomogram model, which facilitated the estimation of total advantages at various threshold probabilities within the training set.

Statistical Analysis

The data derived in this study were analyzed statistically with the help of the R software and relevant tools. Descriptive statistics were used to summarize quantitative data, which were presented as mean ± standard deviation (SD) or as median ± interquartile range. The t-test or Mann–Whitney U-test was conducted to compare the variations in the measurement data noted between both groups, while Chi-square or Fisher’s exact tests were employed to compare differences in enumeration data. Statistical significance between the groups was determined at a threshold of p < 0.05.

Results

Clinicoradiological Features

A flowchart of the experiments and the radiomics workflow is presented in Figure 1. The experiments were conducted using 423 patients with solitary papillary thyroid carcinoma (PTC), of which 206 presented with positive cervical LNM, while 217 patients exhibited negative cervical LNM outcomes. These patients were classified randomly in the training (282 cases) and validation (141 cases) sets in a 2:1 ratio. The results also showed that 48.94% of patients with PTC from the training set and 48.23% of patients with PTC from the validation set showed a positive cervical LNM rate, which was not statistically significant (p = 0.891). Table 1 displays the clinical features of PTC patients, who were categorized into the training and validation sets, while the clinical properties of the PTC patients in both sets exhibited no significant difference (p>0.05, for all variables), which indicated comparable baseline data.

Table 1 Patients’ Characteristics of Training Set and Validation Set

Figure 1 The flowchart of the training and validating process for clinical-radiomics nomogram modeling. Abbreviations: BMUS, B-mode ultrasound; CEUS, contrast-enhanced ultrasound; ICC, interclass correlation coefficient; mRMR, minimum redundancy maximum relevance; LASSO, least absolute shrinkage and selection operator; AIC, Akaike information criterion; Radscore, radiomics score.

Developing the Radiomics Score

A total of 912 features from the BMUS images and 775 features from the CEUS images were retained in this study after eliminating less stable features (ICC ≤ 0.75 were eliminated. The mRMR algorithm further selected 30 features from each image. Subsequently, LASSO regression analysis (Figure 2) was conducted to identify one feature from BMUS images and six features from CEUS images. These chosen features were then subjected to backward stepwise LR analysis, resulting in the identification of one radiomic feature from BMUS images and four radiomic features from CEUS images that were associated with LNM. These features were utilized to construct the BMUS and CEUS radiomics scores (ie, BMUS and CEUS Radscores), respectively, using the following formulae:

Figure 2 The method of choosing radiomics features from the training set using the LASSO algorithm. (A and C) show the LASSO coefficient profiles for the BMUS and CEUS features, respectively. In (B and D), it is shown how to apply minimal criteria analysis and 10-fold cross-validation to get the ideal penalization coefficient lambda () for the BMUS and CEUS LASSO models, respectively. The lambda values chosen based on the minimal criterion and one standard error of the minimum criteria are shown as dotted vertical lines.

Constructing and Validating the Model

After backward stepwise multivariate LR analysis, US-reported LN status and age were regarded as significant factors (p < 0.05) associated with LNM (Table 2). The data from the training and validation sets were used to calculate the area under the receiver operating characteristic curve (ROC-AUC) for the clinical model, which showed values of 0.645 (0.501–0.728) for the training set and 0.628 (0.481–0.711) for the validation set (Figure 3).

Table 2 The Results of Stepwise Multivariate Analyses for Prediction of LNM

Figure 3 The ROC curves of the clinical model and clinical-radiomics model in the training set (A) and validation set (B). Abbreviation: ROC, receiver operating characteristic.

Stepwise multivariate LR was conducted with six factors: US-reported LN status, sex, age, tumor size, BMUS Radscore, and CEUS Radscore. The combined clinical-radiomics model was established by integrating multiple factors such as age, sex, CEUS Radscore, and US-reported LN status (Table 2). The analysis revealed that age, US-reported LN status, and the CEUS RadScore were independent LNM-based factors. The combined model showed AUCs of 0.891 (0.633–0.932) and 0.870 (0.610–0.921) in the training and validation sets, respectively (Figure 3). Thereafter, the clinical and proposed clinical-radiomics models were compared, and the IDI index for the clinical-radiomics model showed a significantly better discrimination value [IDI:16.82% (9.33–22.15%) in the training set vs 7.28% (1.21–15.78%) in the validation set]. This suggests that the inclusion of the CEUS Radiomics score enhanced the ability of the proposed clinical radiomics model to discriminate LNM risk compared to the clinical model. A novel clinical radiomics nomogram model was used to visualize the clinical radiomics model (Figure 4A). The results of the Hosmer–Lemeshow test and calibration curves (Figure 4B and C) exhibited a good calibration performance, indicating no significant variation between the actual and predicted probabilities from the clinical-radiomics nomogram (Hosmer–Lemeshow test: P-value = 0.572 in the training set; P-value = 0.560 in the validation set). Furthermore, DCA (Figure 5) indicated that the clinical-radiomics nomogram exhibited a higher net benefit than the treat-all-patients and treat-no-treatment strategies. The clinical radiomics nomogram exhibited a consistently higher net benefit than the clinical model across most threshold probabilities, suggesting its potential clinical utility.

Figure 4 A radiomic nomogram for evaluating cervical LNM based on the clinical-radiomics model (A). Clinical-radiomics nomogram calibration curves in the training set (B) and validation set (C).

Figure 5 Decision curve of clinical model and clinical-radiomics model. The decision curve analysis (DCA) measures the net benefit (y-axis) versus the model’s high-risk threshold (x-axis) for different models.

Discussion

Surgery is the primary treatment strategy used for patients with PTC, but the extent of surgical intervention and the benefits of prophylactic lymph node dissection and complete thyroidectomy are controversial.^19,20 These surgical procedures are associated with significant complications including hypoparathyroidism, cervical hematoma, and recurrent laryngeal nerve paralysis.^21,22 Hence, it is necessary to identify preoperative prognostic markers to evaluate the risk of cervical LNM in patients with PTC to avoid overdiagnosis and to improve patient outcomes.

Herein, a novel clinical radiomics nomogram was constructed and verified to predict the risk of LNM in PTC patients. The nomogram combines radiomics properties and clinical risk factors derived from CEUS images. After comparing the clinical-radiomics and clinical models, the findings implied a significantly enhanced diagnostic performance in LMN prediction, with higher accuracy using data from the training and validation sets. The novel clinical-radiomics nomogram offers a non-invasive and personalized approach to assess LNM risk preoperatively, aiding treatment planning for patients with PTC. Further studies are required to validate the clinical significance of this nomogram and its integration into routine clinical practice.

The TNM staging system of the AJCC 8th edition suggested that a more appropriate prognostic age cutoff for PTC classification systems could be 55 years rather than 45 years.¹⁸ Univariate and multivariate regression analyses showed a significantly negative association between age and LNM, indicating that younger age could serve as an independent risk factor for cervical LNM in patients with PTC. These findings are similar to those published earlier.^23,24 Hence, the LN status during preoperative US examinations must be assessed, especially in young patients with PTC. Male PTC patients were found to have a higher LNM probability than female PTC patients, which is in line with existing studies and may be influenced by sex hormones.²⁵ Gender was included as a risk factor in the final prediction model after a backward stepwise selection. Additionally, both univariate and multivariate analyses demonstrated a significant association between US-reported LN status and LNM, emphasizing the importance of incorporating US-reported LN status into preoperative predictions. However, US-based techniques have limitations in accurately assessing central cervical LNM owing to their deep location and the potential challenges in visualizing it under the thyroid gland. Several meta-analyses have reported low sensitivity of US in detecting central cervical LNM.^26–28 Furthermore, US diagnosis relies on subjective visual judgment, which can be influenced by variations in the experience of the US physicians. Therefore, additional complementary indicators are needed to improve diagnostic efficiency.

In recent years, radiomic analysis has gained significant attention in medical imaging. For instance, a retrospective study developed radiomics with attribute bagging, which leverages BMUS and CEUS images to improve the classification accuracy of breast tumors, with the high accuracy, sensitivity, specificity, and AUC which exceeds all the comparison methods.²⁹ Another study also developed a clinical-radiomics nomogram by BMUS and CEUS images to predict extrathyroidal extension in PTC.³⁰ Furthermore, a noninvasive model combining BMUS and CEUS radiomics features has the potential to accurately distinguish the benign and malignant thyroid nodules, with the better diagnostic performance than single BMUS radiomics model or CEUS radiomics model.³¹ We found that the combined BMUS and CEUS radiomics model exhibited superior diagnostic performance in the prediction of cervical LNM metastasis for PTC patients, indicating the potential significance of multimodal radiomics. Moreover, the nomogram’s performance outperformed other existing models, showing robust results in both the training (AUC = 0.891) and validation (AUC = 0.870) datasets. CEUS, which involves intravenous injection of a contrast agent in the blood pool to visualize tissue microperfusion, has been particularly useful for preoperative LNM prediction in PTC patients with.³² This study aimed to investigate the benefit of combining BMUS and CEUS for radiomics analysis to predict the risk of LMN in PTC. Interestingly, although both the BMUS and CEUS RADscores were significantly associated with LNM in univariate analysis, only the CEUS Radscore was used in the final clinical radiomics model. Our analysis revealed that the CEUS Radscore exhibited better discriminatory ability, which outweighed the contribution of the BMUS Radscore, leading to its exclusion from the final model. Regarding the relatively low efficacy of the clinical model when compared to the CEUS Radscore-based radiomics model, the clinical model in our study was developed based on a selection of relevant clinical variables that were readily available and routinely collected in clinical practice. However, we acknowledge that the inclusion of only a limited number of clinical variables may not fully capture the complexity and heterogeneity of the underlying the cervical lymph node metastasis of PTC. Moreover, the clinical assessments may be subject to inter-observer variability, where different clinicians or radiologists might interpret the same images differently. Thus, this study using the radiomics to extract information from various dimensions, including image features, molecular biology data, and clinical variables, thereby providing the comprehensive information to predict the cervical LNM in PTC. As a result, the radiomics model combined with the clinical information achieves a superior performance compared to the clinical model alone.

In the past, researchers have revealed that CEUS enhancement patterns could be employed to predict the occurrence of LNM in tumors, where hyper- or iso-enhancement could be considered an independent risk factor for LNM.^33,34 However, the results of this study did not show a significant correlation between the CEUS enhancement patterns and LNM. This discrepancy may be due to the differences in the datasets, particularly the small sample size of the highly enhanced cases used in this study. Moreover, the qualitative assessment of enhancement intensity, which is subject to some degree of subjectivity, might have influenced the results. Nonetheless, this study showed that CEUS image-based quantitative radiomics analysis is significantly associated with LNM. Unlike visual inspection of enhancement homogeneity and intensity, radiomics analysis has the potential to quantitatively capture crucial information regarding tumor microcirculation heterogeneity.³⁵ In this study, the CEUS radiomics score consisted of four radiomics features. Wavelet transform can reveal hidden features within medical images at different scales, enhancing the discriminative ability by amplifying heterogeneous information related to tumor texture features. Furthermore, most of the selected CEUS radiomics features in this study characterized the spatial distribution of lesion voxels, indicating that PTCs showing high vascular heterogeneity tend to display more aggressive biological behavior. All of the above radiomics characteristics, which are difficult to visually identify, can be used as non-invasive biomarkers for the preoperative prediction of lymph node status in patients with PTC.

Despite these advantages, this study has several limitations. First, this was a retrospective study conducted at a single center, which could introduce bias. Therefore, prospective multicenter studies should be conducted with larger sample sizes to enhance the generalizability of the findings. Second, the radiomics analysis of CEUS images was conducted using a single-frame image owing to technical constraints, which could potentially lead to the loss of valuable information compared to the assessment of the complete perfusion process. Future studies should focus on feature extraction and processing of dynamic images to improve the accuracy of radiomic analysis. Finally, our radiomics analysis was based on primary tumor images, and very few studies have developed a radiomics model using lymph node sonograms for predicting LNM in patients with PTC. Hence, additional studies should investigate the predictive value and feasibility of the radiomics analysis approach, depending on lymph node sonograms or the integration of lymph node and primary tumor images.

In summary, we developed a novel clinical-radiomics model by integrating the radiomics score with important preoperative clinical characteristics. A nomogram was constructed to improve clinical applications and to visualize the logistic regression model. The findings revealed that The clinical-radiomics nomogram model exhibited better performance than the clinical model in terms of AUC scores and clinical advantages. The incorporation of the major clinical features and CEUS Radscore significantly improved the IDI scores, highlighting the significance of the CEUS-based radiomics technique in predicting the preoperative LN status among PTC patients.