Machine Learning-Based Ultrasound Radiomics Signature: A Retrospective Analysis for Differentiating Benign from Malignant Non-Mass Lesions in Dense Breasts

Introduction

Non-mass lesions of the breast (NML), as a sub classification of breast cancer, account for about 9.21% of all breast abnormalities.^1,2 NML is defined by their morphological features, including diffuse echo texture changes that are different from the surrounding breast tissue, and the lack of well-defined spherical or oval masses with distinct margins, rather than the lack of measurable size.^2–4 Compared with typical mass lesions, its ultrasonic characteristics are not obvious, which brings great challenges to clinical diagnosis. Unlike well-defined mass lesions, NML is characterized by diffuse echo texture changes, which are different from the surrounding breast tissue and lack of unified classification criteria for ultrasound imaging features.^3,5 This ambiguity leads to overlapping benign and malignant manifestations, resulting in low efficiency of clinical diagnosis.

It is worth mentioning that dense breasts, which are more prevalent in Asian women, are mainly composed of fiber and adipose tissue.^4,6 Traditional imaging modalities have inherent limitations, such as reduced sensitivity of mammography to dense breast lesions (especially noncalcified NML), while magnetic resonance imaging (MRI) may not detect low-grade ductal carcinoma in situ.^7–9 In contrast, ultrasound is not affected by breast density, can detect invisible lesions on mammograms, and has the advantage of identifying smaller, lower grade ductal carcinoma in situ, making it the preferred way to screen and diagnose dense breast lesions.^10,11 Moreover, traditional ultrasound diagnosis largely relies on the subjective experience of doctors, resulting in inconsistent judgments and insufficient specificity (previous studies ranged from 6.5% to 42.3%).^12,13 With the progress of artificial intelligence, radiomics has become a powerful tool to quantify high-dimensional imaging features to objectively assess tumor heterogeneity, providing an objective complement to subjective visual evaluation.^14,15 Although radiology based breast NML research has mainly focused on dynamic contrast-enhanced MRI, ultrasound radiology has greater clinical feasibility due to its wide availability, noninvasiveness, and cost-effectiveness.

In view of this, the purpose of this study is to construct ultrasound radiological features based on machine learning, and combine them with clinical and ultrasound features to establish a combined diagnostic model. By evaluating its performance in differentiating benign and malignant NML in dense breast, this study aims to provide a practical and reliable tool for clinical practice and solve the unmet need for accurate diagnosis of this challenging type of lesion.

Methods

Study Population

We retrospectively enrolled NML patients with dense breast admitted to Hubei Cancer Hospital from January 2017 to January 2023. The inclusion criteria were as follows: (1) ultrasound diagnosis of breast NML; (2) Breast density was confirmed by ultrasound or mammography; (3) Complete clinical data and definite pathological results were obtained by puncture biopsy, vacuum assisted biopsy or surgical resection. The exclusion criteria were as follows: (1) preoperative history of neoadjuvant chemotherapy, radiotherapy or endocrine therapy for breast disease; (2) Ultrasonic image quality is poor, too many artifacts or noise interference analysis; (3) Lesions cannot be measured on ultrasound images. Between January 2017 and January 2023, a total of 680 patients with non‑mass lesions of dense breast were initially identified. According to the above criteria, 61 patients were excluded, including 18 with a history of neoadjuvant therapy, 25 with poor ultrasound image quality or severe artifacts/noise, and 18 with lesions that could not be measured on ultrasound. Finally, 619 patients were enrolled in this study.

This study was reviewed and approved by the ethics committee of Hubei Cancer Hospital (approval number: LLHBCH2026YN-026). All research procedures strictly comply with the principles of Helsinki Declaration (2013 Revision) and relevant national medical research ethics regulations. Considering the retrospective nature of the study, the ethics committee agreed to waive the written informed consent of individual patients. Before the start of the study, all patient data were de identified to delete personal identifiers (such as name, medical record number and contact information), and the data were managed and stored in a password protected secure database to ensure the privacy and confidentiality of patient information. Any access and use of the research data is limited to authorized researchers, and the data is only used for the purpose of this study and will not be disclosed to any third party. Upon reasonable request, the de identified data sets used in this study, including clinical, ultrasound and radiomics characteristics and pathological labels, can be obtained from the corresponding authors. The process of patient registration, screening and prediction model establishment is shown in Figure 1.

Figure 1 Flowchart of patient registration, screening and prediction model establishment.

Imaging Acquisition and Processing

We used Ge logic E9 color Doppler ultrasound system with high-frequency linear array probe and mechanical index of 0.16. The patient was in supine position, his upper limbs were abducted, and his breast and armpit were fully exposed; Use side position if necessary. Repeat scanning from multiple angles and directions, adjust the gain, and focus on key areas. The two-dimensional ultrasound images of the longest diameter segment of the lesion were collected, and the ultrasonic characteristics of the lesion range, internal echo uniformity, microcalcification, posterior echo attenuation, and surrounding structure distortion were recorded. Color Doppler ultrasound was used to evaluate the blood flow characteristics.

All initial ultrasound studies were conducted by a committee certified breast ultrasound radiologist (non-technical) with at least 5 years of clinical experience in breast ultrasound imaging. Radiologists from the breast cancer center of Hubei Cancer Hospital were examined. All radiologists received unified training on intensive breast non mass lesions scanning scheme to ensure the consistency of image acquisition. All images are exported in DICOM format for subsequent analysis. Before feature extraction, Gaussian smoothing filter (sigma=0.5) was applied to all ultrasound images to reduce random speckle noise, and all images were clipped to the breast region to eliminate background artifacts. B-spline interpolation (3rd order) was used for image resampling, and the voxel size of each resampling step was 0.1 × 0.1 × 0.1 mm.

Region of Interest Segmentation and Feature Extraction

Due to the complex and diverse ultrasound manifestations of NML, such as patchy hypoechogenicity, ductal patterns, microcalcifications, and glandular structural disorders, we arranged for ultrasound physicians with over 10 years of experience in breast ultrasound to manually delineate the region of interest (ROI) using Itk Snap software (version 3.8.0). Selecting the most prominent area of the lesion for ROI delineation does not rule out cystic changes, necrosis, or calcification within the lesion. ROI segmentation follows standardized standards: the ROI is delineated along the outermost boundary of ultrasound abnormalities in non mass lesions, including all areas of echo abnormalities, microcalcifications, structural distortions, or blood flow abnormalities. Use the longest diameter slice of the lesion identified by ultrasound for two-dimensional segmentation, and segment the original grayscale ultrasound images except for color Doppler images. The complete PyRadiomics configuration file (including discretization, resampling, filtering, and normalization parameters) is provided in Supplementary File 1.

The physician performing ROI segmentation does not know all the pathological results of the lesion to avoid subjective bias. For lesions with unclear boundaries (defined as the boundary where abnormal echoes mix with normal glandular tissue), the boundaries are determined by combining grayscale echo changes and color Doppler blood flow signals, and the segmentation is validated by a senior ultrasound physician with over 20 years of experience to ensure consistency. Evaluate intra observer variability in ROI segmentation: The same physician replicates segmentation of 50 randomly selected lesions every 4 weeks and calculates the intra group correlation coefficient (ICC) for all extracted radiomic features. The results showed that 98% of the features had an ICC>0.75, and 12 features had an ICC>0.85, indicating good consistency.

Data Preprocessing and Feature Selection

We used a random number generator to randomly divide the data into a training set and a validation set in a 7:3 ratio. Normalize all extracted radiomic features using the Z-score method to ensure they are on the same scale. Feature selection consists of three steps: (1) preliminary screening using Student’s t-test for features that show significant differences between benign and malignant lesions; (2) Perform Pearson correlation analysis to remove highly correlated features (correlation coefficient threshold=0.8); (3) Based on the minimum squared error (MSE) criterion, LASSO regression is applied to select the optimal features and retain non-zero coefficient features to construct radiomics features.

All steps of feature selection, including Student’s t-test, Pearson correlation analysis, and LASSO regression, are strictly performed only on the training set and cannot access the validation set data at any stage. The LASSO hyperparameter (λ) selection is performed using 5-fold cross validation on the training set, with the optimal λ determined by the minimum mean square error (MSE) criterion and a standard error rule to ensure model simplicity. The intensity scaling parameters (Z-score normalized mean and standard deviation) are only learned from the training set and applied to the validation set without the need for re estimation to avoid data leakage.

Model Construction

We have established four predictive models: (1) Clinical model: constructed using independent clinical predictive factors determined by multiple logistic regression; (2) Ultrasound model: construction of independent ultrasound prediction factors based on multiple logistic regression; (3) Radiomics model: Developed using LASSO logistic regression combined with selected radiomics features, calculating radiomics score (Rad score) for each patient by adding the product of each feature and its corresponding regression coefficient; (4) Combination model: Integrating Rad scores with independent clinical and ultrasound predictive factors.

Statistical Analysis

We used R software (version 4.3.2) for data statistical analysis and visualization. Quantitative data that conforms to a normal distribution are expressed as mean ± standard deviation (± s) and compared using independent sample t-test. Quantitative data that do not conform to a normal distribution are represented as median (interquartile range) [M (P25, P75)] and compared using Mann Whitney U-test. The categorical data is presented in the form of counts (percentages) and compared using chi square tests. Univariate and multivariate logistic regression analysis are used to determine independent predictive factors for malignant tumors. The diagnostic performance of the model is evaluated through AUC, sensitivity, specificity, accuracy, balance precision, and F-score. Calibration curves are used to evaluate the consistency between predicted probabilities and actual results. Evaluate the clinical utility of DCA by calculating the net benefit at different threshold probabilities. The DeLong test is used to compare the AUC between models. Bilateral P<0.05 is considered statistically significant.

Results

Baseline Clinical and Pathological Characteristics of the Study Population

A total of 619 patients were ultimately enrolled, including 304 with malignant lesions (average age: 47.9 ± 9.4 years, range: 20–79 years) and 315 with benign lesions (average age: 42.2 ± 10.5 years, range: 20–76 years). Among them, the training set includes 207 malignant and 227 benign lesions, while the validation set includes 97 malignant and 88 benign lesions. As shown in Table 1, there was no significant difference in baseline clinical and pathological characteristics (including age, height, weight, menarche age, menopausal status, lesion location, and Rad score) between the training and validation sets (all P>0.05), indicating good homogeneity between the two groups. Overall, the research population maintains a balance between the training and validation sets, ensuring the reliability of subsequent model training and validation.

Table 1 Baseline Clinical and Pathological Characteristics of Patients in the Training and Validation Sets

Identification of Independent Predictive Factors for Malignancy

In the training set, we used univariate logistic regression analysis to show that age, menopausal status, lesion length, lesion width, microcalcifications, peripheral structural deformation, and blood flow characteristics were potential factors associated with the malignancy of NML (all P<0.05); Supplementary Tables 1 and 2). Multivariate logistic regression analysis further confirmed that age (odds ratio [OR]=1.053, 95% CI: 1.027–1.080, P<0.001), lesion length (OR=0.701, 95% CI: 0.573–0.857, P=0.001), microcalcification (OR=8.197, 95% CI: 4.895–14.144, P<0.001), peripheral structural deformation (OR=3.479, 95% CI: 2.044–6.044, P<0.001), and blood flow characteristics (OR=1.195, 95% CI: 1.036–2.408, P=0.042) were independent predictive factors for malignant tumors. In short, age, lesion length, microcalcifications, surrounding structural deformations, and blood flow characteristics have been identified as key independent factors for predicting malignant NML, providing a basis for the construction of clinical and ultrasound models.

Table 2 Univariate and Multivariate Logistic Regression Analyses of Factors Associated with Malignancy of Non-Mass Lesions

Selection of Radiomics Features and Construction of Radiomics Signature

After preliminary screening using Student’s t-test, we retained 433 radiological features with significant differences between benign and malignant lesions. The subsequent Pearson correlation analysis excluded highly correlated features, and LASSO regression further screened out 12 non-zero coefficient features (Table 3), including 1 histogram feature, 2 texture features, and 9 wavelet features. These 12 features are used to construct radiological features and calculate Rad scores for each patient. Finally, 12 optimal radiomics features were determined through multi-step selection, forming a stable radiomics signature that can objectively quantify the imaging features of NML.

Table 3 Selected Radiomics Features and Their Coefficients

Diagnostic Performance of Individual Models

As shown in Figure 2 and Table 4, the AUC of the radiomics model in the training set is 0.782 (95% CI: 0.743–0.821), and the AUC in the validation set is 0.724 (95% CI: 0.659–0.789), with sensitivities of 74.9% and 77.3%, and specificities of 65.6% and 60.0%, respectively. It is gratifying that the ultrasound model showed better performance than the clinical model, with AUC values of 0.816 (95% CI: 0.780–0.853) and 0.754 (95% CI: 0.691–0.817) in the training and validation sets, respectively. The diagnostic efficiency of the clinical model is the lowest, with AUC values of 0.688 (95% CI: 0.644–0.732) and 0.641 (95% CI: 0.571–0.711) for the two groups, respectively. Among the three single models, the ultrasonic model showed the best diagnostic performance, followed by the radiological model, while the clinical model had relatively limited efficacy.

Table 4 Diagnostic Performance of Different Models in the Training and Validation Sets

Figure 2 Receiver operating characteristic (ROC) curves of the four models in the training set (a) and validation set (b).

We combined the Rad score with independent clinical and ultrasound predictive factors in a combined model, and the results showed the highest diagnostic performance among all models. In the training set, the AUC of the combined model was 0.89 (95% CI: 0.86–0.92), sensitivity was 0.811, specificity was 0.789, accuracy was 0.778, balance accuracy was 0.799, and F-score was 0.794. In the validation set, the AUC was 0.83 (95% CI: 0.78–0.89), with corresponding values of 0.773, 0.670, 0.724, 0.721, and 0.746 (Supplementary Table 2 and Supplementary Figure 1). The DeLong test confirmed that there were significant differences between the combination model and the clinical model (Z=−3.974, P<0.001), ultrasound model (Z=3.338, P=0.001), and radiomics model (Z=−3.468, P<0.001) in the validation set. The calibration curve shows that compared with the other three models, the combined model has better consistency between predicted probability and actual pathological results (Figure 3). DCA indicates that the combined model achieved the highest net benefit in both the training set (DCA curve area=0.12) and the validation set (DCA curve area=0.22), demonstrating its superior clinical practicality (Figure 4). The comprehensive model is obviously superior to the single model in terms of diagnostic accuracy, consistency and clinical practicability, making it a promising clinical application tool.

Figure 3 Calibration curves of the four models in the training set (a) and validation set (b). The x-axis represents predicted probabilities of malignancy, and the y-axis represents actual probabilities (true malignancy rate). The diagonal black line indicates perfect calibration. The combined model (red curve) is closest to the perfect calibration line, indicating better consistency between predicted and actual outcomes compared with the other models.

Figure 4 Decision curve analysis (DCA) of the four models in the training set (a) and validation set (b). The x-axis represents threshold probabilities, and the y-axis represents net benefit. The gray curves represent the “treat none” and “treat all” strategies for reference.

Discussion

The diagnosis of NML in dense mammary glands has always been a clinical challenge due to the overlapping imaging features and the blurring effect of dense glandular tissue. In this study, we constructed a machine learning based composite model that combines ultrasound radiological features with clinical and ultrasound features, and demonstrated its superior diagnostic performance in distinguishing benign and malignant NML. The main findings and their clinical significance are as follows. It should be emphasized that radiomics features are complementary to composite models rather than dominant components. In the validation set, the AUC (0.724) of the radiomics model alone was lower than that of the ultrasound model alone (0.754), but the integration of radiomics features with clinical and ultrasound features significantly improved the diagnostic performance of the model (AUC increased from 0.754 to 0.83), the correction rate of misclassified cases by the combined model is shown in Supplementary Table 3. This is because radiomics captures subtle quantitative texture and structural changes that are not visible in visual ultrasound assessment, providing unique supplementary information for traditional clinical and ultrasound features. Among all test models, the combination model had the highest AUC (0.89 for the training set and 0.83 for the validation set), indicating a significant improvement in specificity compared to a single radiomics model. Encouragingly, this advantage can be attributed to the complementary information provided by clinical, ultrasound, and radiological features.

Previous studies have shown that clinical factors such as age reflect the inherent risk of cancer, as aging is associated with metabolic and immune changes, accumulation of primary genes, and insufficient DNA methylation, all of which increase susceptibility to cancer.^16–18 Ultrasound features include microcalcifications, deformation of surrounding structures, and blood flow status, which directly reflect the biological characteristics of the lesion. Microcalcification is associated with high metabolic activity and local ischemia or necrosis of tumor cells. The deformation of the surrounding structure indicates invasive growth. The increase in blood flow reflects tumor induced angiogenesis to support rapid proliferation of tumor cells. On the other hand, radiological features quantify the heterogeneity of lesions at the microscopic level, capturing subtle texture and echo changes that are not visible to the naked eye. However, many previously published predictive models have performed poorly in clinical validation, failing to provide satisfactory predictive performance for lesion features.^19–21 By integrating these multidimensional data, the composite model overcomes the limitations of single modal models, which rely on incomplete information and therefore have suboptimal diagnostic performance.

The identification of independent predictive factors in this study is consistent with previous research results in some aspects, but also shows certain differences. Consistent with previous studies, microcalcifications were found to be the strongest predictor of malignant tumors, highlighting their critical role in the diagnosis of NML.^22,23 Malignant lesions are also associated with older age, which is consistent with the epidemiological characteristics of cancer. However, this study found a negative correlation between lesion length and malignancy, which contradicts previous reports suggesting that the larger the lesion size, the higher the risk of malignancy. This difference may be due to the unique pathological composition of NML in dense mammary glands: benign lesions such as fibrocystic changes may manifest as diffuse large-area lesions, while malignant NML in dense mammary glands may be more localized in the early stages due to the restrictive effect of dense glandular tissue.

In addition, the frequency of structural distortions around malignant lesions observed in this study is relatively high, which is consistent with the invasiveness of cancer. However, due to differences in the study population and pathological types, the degree of association may vary. These findings emphasize the need to develop tailored diagnostic criteria for NML in dense breasts, taking into account the specific characteristics of this type of lesion. The potential risk of overfitting in radiomics models has been effectively addressed through various robust validation methods, such as using 5-fold nested cross validation instead of single random segmentation validation, adding 1000 guided validations to quantify the variability of performance indicators, and training early cohorts and testing late cohorts for time validation. The results show that the model performance of all validation methods is consistent, indicating no significant overfitting and reliable model performance.

In this study, we provided valuable insights into the pathological basis of NML by examining 12 selected radiological features, including histograms, textures, and wavelet features. Among them, histogram features (such as interquartile range) reflect the distribution of grayscale values in lesions, which is related to the uniformity of tissue composition. Texture features, such as the joint mean and mean of the gray level co-occurrence matrix, quantify the spatial relationship of gray values and reflect the structural heterogeneity of lesions. Wavelet features originate from multi-scale decomposition of images, capturing subtle changes in tissue texture at different frequencies, which are related to the microstructural differences between benign and malignant tissues. Integrating these features into radiomics can objectively and quantitatively evaluate the heterogeneity of lesions, reducing the subjectivity of traditional ultrasound diagnosis. This is particularly important for NML, as visual assessment is often challenging due to boundary blurring and feature overlap.

Compared with previous radiomics studies on breast NML, our study has several significant advantages. Most previous studies have focused on DCE-MRI, which is an expensive, time-consuming, and not widely used method in primary hospitals.^24,25 In contrast, ultrasound is a routine clinical examination with high accessibility and low cost, making the proposed model more feasible in clinical practice. In addition, previous ultrasound based NML studies often relied on subjective visual features or a limited number of quantitative indicators, while this study extracted 848 high-dimensional radiological features and used machine learning to select the best features, ensuring the comprehensiveness and accuracy of the features.^26,27 In addition, integrating clinical and ultrasound features into radiological models can improve their interpretability, as independent predictive factors such as microcalcifications and age have clinical significance and are familiar to doctors, which increases the model’s acceptance in clinical practice. It is worth noting that a slight increase in AUC of the combined model compared to the individual ultrasound model does not necessarily mean that its clinical value is small. DCA reflects the actual clinical net benefit of the model, which is a more clinically relevant indicator than AUC. This combination model has significantly higher net benefits within a wide range of clinically relevant threshold probabilities, indicating that compared to traditional clinical combined ultrasound models, it can identify more malignant lesions, reduce more unnecessary biopsies, and has substantial clinical value.

The clinical practicality of this combination model is further supported by its calibration and DCA performance. The calibration curve shows good consistency between the predicted and actual results, indicating that the model can provide reliable probability estimates for individual patients. DCA indicates that within a wide range of threshold probabilities, the combined model achieved higher net benefits than other models, which means its application can help clinicians make wiser decisions, avoid unnecessary biopsies of benign lesions, and ensure timely intervention for malignant lesions. This is particularly valuable in clinical practice, as unnecessary biopsy not only increases the physical and mental burden on patients, but also consumes medical resources. As an auxiliary tool for BI-RADS assessment, this combination model can improve the accuracy of risk stratification for dense breast NML and optimize management strategies for these patients.

Our research inevitably has certain limitations. First, this is a single center retrospective study. The sample size is relatively small, because non breast cancer is rare, which may introduce selection bias and limit the generalization of the results. Future multicenter studies with larger sample sizes are needed to validate this model. Secondly, ROI segmentation is manually performed by a doctor, which may be influenced by subjective differences; In future research, automatic segmentation methods should be explored to improve reproducibility. Thirdly, the model does not include other potential predictive factors such as genetic markers or molecular subtypes, which may further improve its diagnostic efficiency. Despite these limitations, this study provides a promising approach for the diagnosis of dense breast NML and has significant clinical implications.

Conclusion

In summary, our machine learning based combination model combines ultrasound radiological features with clinical and ultrasound features, and has excellent diagnostic performance in distinguishing benign and malignant non mass lesions in dense breasts. It outperforms single modal models in accuracy, consistency, and clinical practicality, providing a reliable and feasible tool for supporting clinical decision-making, optimizing patient management, and improving cancer patient prognosis.