An Overview of Current Trends, Techniques, Prospects, and Pitfalls of Artificial Intelligence in Breast Imaging

Introduction

Artificial intelligence (AI) has become a buzzword that exhilarates the contemporary medical community to both extreme enthusiasm and grave concern about its capacity to surpass human potential. Though AI still lacks a standard definition, in 1955, John McCarthy considered artificial intelligence a multi-disciplinary field and defined it as “the science and engineering of making intelligent machines” in a workshop at Dartmouth Summer Research Project on AI.¹ While radiologists have used computer-aided detection (CADe) and computer-aided diagnosis (CAD) in imaging — especially breast imaging — to assist them in the early detection and diagnosis of lesions for many years, the imprecision and unpredictability of CAD systems have suggested the need for more clinical studies and development of potential imaging applications. Hence, there is tremendous potential for evolution in the applicability of AI to radiological images.²

A Brief Overview of AI

AI is a blanket term that encompasses various subdomains like artificial neural networks (ANNs), natural language processing (NLP), and deep learning (DL) used for predictive modeling.³ Machine learning refers to the techniques used to achieve artificially intelligent models. Machine learning falls into three categories and can be supervised if the machine is trained by providing training data and corresponding labels, such as biopsy-confirmed cancers and benign lesions. It can be unsupervised if no labeling is done, and the AI is expected to figure patterns out by itself. It is called semi-supervised if a combination of labeled and unlabeled images is used.^{4, 5} Supervised learning is useful for two types of problems, depending on whether outputs are real values (regression) or categories (classification). It includes linear/logistic regression, random forest, or support vector machines for these problems. Unsupervised learning is also divided into three categories, namely association, clustering, and dimensionality reduction. It is termed as an association if we were going to find a set of rules that describe our data, clustering if we want to see similar groups in our data like k-clustering and dimensionality reduction for big data visualization, text mining, and image recognition. Neural networks are also such kinds of algorithms, though they have multiple subtypes too: convolutional networks, recurrent networks, and deep feedforward networks, to name a few.^{6, 7}

Using different algorithms specific to particular tasks, computers can organize and scrutinize large amounts of data, including text, medical images, and medical records, in exponentially less time than required for manual processing. The creation of synthetic images from an available database to train an algorithm is a subsequent step to meet the challenges faced due to the shortage of annotated image data. Augmented intelligence, an escalated human intellect with the help of artificial intelligence, uses a multi-perspective and team-based approach for an alliance between technology and physicians to handle vast amounts of imaging data, accelerate workflow, and improve patient care.⁸ Modern-day high-speed computers with cloud computing and enhanced storage capacities facilitated the progress and advancements of the various AI techniques. Dissecting these networks to look at their weights and biases might even help identify patterns, even some that had not been noticed earlier by experts either.^{3, 9} Besides, unlike humans, computers are not tired, bored, or distracted toward the end of a long day of, say, reading mammograms.¹⁰

This review article discusses the current trends, techniques, and prospects of AI in breast imaging, including its shortcomings.

Materials and Methods

A thorough literature search was conducted to determine the prevailing status of AI in radiodiagnosis, in specific reference to breast imaging. Validation from an institutional review board was not mandatory for this literature review as no human data has been used. All English language articles published on AI in the last two and a half decades were searched for authentic research, literature reviews, meta-analysis, and systematic review articles from the databases accessible online. Although this search was wide-ranging due to the dynamic nature of the topic, the author is aware that all of the published data could not be included, though the pivotal issues have been covered.

The articles mainly published during the last decade were considered to summarize recent advances, novel techniques, and future scope, along with the prior work that contributed to the framework of current progress. Artificial intelligence, AI, AI in medical imaging, AI in radiology, AI in breast imaging, computer-aided diagnosis, and computer-aided detection, CAD, neural network, NN, artificial neural network, ANN, convolutional neural network, CNN, machine learning, ML, transfer learning, deep learning, DL, generative adversarial networks (GANs), radiomics, natural language processing, NLP and augmented intelligence were used as the major and minor MeSH headings and keywords to limit the search narrowly.

Results

Initial literature research revealed more than a million articles related to AI in medicine. More than a thousand articles relevant to AI in radiology, primarily breast imaging, were found after applying the appropriate search filters defined above, of which I went through 296. Most of these discussed the current trends, applications, and drawbacks of AI in breast imaging. Various articles exhaustively delineated AI and the techniques used to construct the algorithms for deep learning for most medical specialties, including imaging. AI techniques, along with augmented intelligence, CAD, and radiomics, have been explained outstandingly and seem to be easily discernible in these exhaustive articles by Currie et al,⁴ Arieno et al,⁸ Giger¹¹ and Ribeiro et al.¹² Augmented intelligence has been considered as a personalized tool to aid radiologists and upgrade patient management.⁸ Also, Hosny et al,¹³ Yi et al,⁹ Soffer et al,¹⁴ Ayer et al¹⁵ and Baker et al¹⁶ in their article stress the significance of deep learning, convolutional neural networks, artificial neural networks, computer-aided diagnosis, generative adversarial networks (GANs), and radiomics for radiologists, as their profession mainly constitutes an assessment of visual data. Langlotz et al¹⁷ discussed the need for publicly available, well-annotated, and reusable datasets to evaluate various AI algorithms and techniques. In an article published over two decades ago, the study assessed the inter-and intraobserver variability of radiologists’ interpretations and ANN predictions with Cohen kappa analysis with improved accuracy of mammographic interpretation of breast lesions.¹⁶

The following paragraphs delineate the analytical results of some of the studies regarding the use of subsets of AI — ANNs, CNNs, radiomics, transfer learning, CAD with AI — in breast imaging.

Marchevsky et al suggested that AI applications in breast imaging research are being navigated by clinical interests to upgrade patient care. It used neural networks and logistic regression to infer nodal status in 279 patients with breast cancer according to prognostic determinants such as age, family history, axillary nodal involvement, receptors and tumor type, and grade and size in the surgical specimen. The best neural network model was trained with 224 cases using 19 input neurons. The results – a specificity of 97.2%, sensitivity of 80.0%, positive predictive value of 93.8%, negative predictive value of 87.5% — were satisfactory, accurately classifying 49 of 55 (89.0%) unknown cases.¹⁸ A 2007 article supports using an artificial neural network as a machine learning technique and possibly replacing the more invasive sentinel node imaging for predicting metastatic involvement of axillary lymph nodes in patients with breast cancer.¹⁹ Saritas²⁰ in 2012 commenced a study to determine whether an ANN could anticipate breast cancer from the evaluation of patient age and imaging features of a breast lesion (ie, shape, margin, and density). Confusion matrix and receiver operating characteristic (ROC) analyses were used to investigate the data from the ANN model with the sensitivity of 90.2% and specificity of 81.4%. Conducting a study on 800 patients’ biopsies yielded a disease prediction of 90.5%, thus validating the process. These studies might be useful for patients without a diagnosis who are hesitant to undergo further testing if the probability of malignancy helps decide the need for biopsy. The BI-RADS (Breast Imaging- Reporting And Data System) assessment has been compared with radiologists’ interpretations by using artificial neural networks.

Mohamed et al²¹ used deep learning to classify the mammograms into scattered densities or heterogeneously dense breasts and help assign a BIRADS category. The CNN model was trained from scratch on their mammographic images with an AUC of 0.942. Rodríguez-Ruiz et al²² concluded the improvement in the efficiency of a radiologist in detecting breast cancer with the help of AI in comparison to unaided reading images. 240 digital mammography images with 100 normal, 100 malignancies and 40 false-positive cases were used as the input datasets. DL (CNNs) feature classifiers and image analysis algorithms were used to delineate microcalcifications and soft tissue lesions. The results with AUC (0.89 vs 0.87, P= 0.002), sensitivity (86% VS 83%, P= 0.046), specificity (79% VS 77%, P= 0.06) supported the AI system as an aid to radiologists. Qiu et al²³ exhibited the feasibility of applying 8-layered DL with three pairs of convolutional-max-pooling layers for automatic feature extraction and multiple layer perceptron (MLP) classifier for feature categorization to process 560 ROIs (region of interests) extracted from digital mammograms. Overall, AUC, 0.790 ± 0.019, was found by applying the DL-based CADx scheme to classify and differentiate benign from malignant lesions.

Radiomics score was created based on ultrasound texture feature analysis to discriminate 186 pathologically proven triple-negative breast malignancy from 715 benign fibroadenomas in a retrospective study conducted by Lee et al Triple-negative invasive breast carcinomas might appear, round or oval, enhancing with circumscribed margins, and without spiculation. They can be falsely detected as benign cysts or fibroadenomas by ultrasound if imaging parameters are poorly modulated. Three different ultrasound machines were used to image the lesions and check whether the results’ discrepancies were attributable to image quality differences of the systems. The radiology resident selected an image for each lesion and outlined the mass’s margin to indicate the ROI suggesting the study’s supervised nature. Around 730 features were extracted, including 14 pixel intensity-based features, 132 textural features, and 584 wavelet-based features. Radiomics score showed a substantial difference (p < 0.001) between the cancers and fibroadenomas for all the three ultrasound systems with high diagnostic efficiency (AUC 0.910) after the proper training, validation, and testing of the algorithms.²⁴

The study by Huynh et al²⁵ inferred that transfer learning could improve the current CADx methods by using 219 breast lesions (607 full-field digital mammographic images) and comparing support vector machine classifiers based on the CNN-extracted image features and their previous computer-extracted features (AUC 0.86) to differentiate benign and malignant lesions.

Zelst et al²⁶ inferred the significance of CAD software for automated breast ultrasound (ABUS) images to improve the detection of radiologists screening for breast cancer. The study selected 90 patients, and after follow-up, 40 were normal, 20 were malignant, and 30 were benign. The average area under the curve (AUC) was 0.77 without CAD, which improved to 0.84 (p= 0.001) with CAD. However, no significant difference was discerned in the AUC between the experienced radiologists with and without CAD.

Applying machine learning, Cai et al²⁷ conducted a study and combined morphology of DWI and kinetic features of contrast-enhanced MRI to classify the lesions as benign or malignant using semi-automated segmentation on 234 cases (85 benign and 149 malignant). The sensitivity of 0.85, the specificity of 0.89, the 0.93 accuracy, and AUC of 90.9% were achieved while diagnosing and classifying the breast lesions as benign or malignant accurately.

Statistically significant (p <0.0001) and the accurate prediction was made by the neural networks compared to the Cox proportional hazard model for predicting breast malignancy relapse in high-risk intervals on 3811 patients, in a study conducted by Jerez et al.²⁸ The five input covariates (age, tumor size, lymph node status, tumor grade, type of treatment) with survival time (months) were studied with a three-layer neural network model (each input node corresponded to prognostic factor plus one node for time; one hidden layer, and one output layer) constructed with a software.

A recent study conducted by Mc Kinney et al²⁹ used two large datasets from the United States (US) and United Kingdom (UK) and discussed an absolute reduction of 5.7%/1.2% (US/UK) in false positives and 9.4%/2.7% (US/UK) in false negatives, hence improving the accuracy and efficiency of breast cancer screening. The AUC-ROC for the AI system exceeded that of an average radiologist by 11.5%.

Of the 12 studies described above, AUC (Figure 1) of 7 studies and sensitivity and specificity (Figure 2) of 4 studies were compared by graphical representation. As evident from the bar diagrams, the study by Mohamed et al achieved the highest AUC. Maximum sensitivity was achieved by Saritas, and maximum specificity by Marchevsky et al (Figures 1 and 2).

Figure 1 Comparison of AUC of various studies.

Figure 2 Comparison of sensitivity and specificity of various studies.

Discussion

Rapidly evolving changes in healthcare delivery systems combined with the recent state-of-the-art advances in breast imaging has necessitated a review of the current trends and techniques used for AI in breast imaging. Various international radiologic organizations, such as the American College of Radiology (ACR), and the Canadian, Indian and European associations of radiologists, are progressively and efficiently integrating AI techniques.^30–32

Current Trends and Techniques Used for AI in Imaging

A neural network is gaining popularity as it forms a series of algorithms that simulate the human brain in perceiving the dataset relationship to analyze image feature extraction and identification patterns, sometimes even better than human recognition, if an adequate amount of high-quality data is available.¹¹ Deep learning is preferred as it is performed by neural networks with multiple hidden layers that can determine both simple and complex features like lines, texture, edges, intensity, shapes, and other lesions.³³ In the following paragraphs, the application of various AI techniques (Figure 3) have been discussed briefly.

Figure 3 Various techniques of artificial intelligence.

Convolutional neural networks (CNNs), the type of deep, feedforward artificial neural networks (ANNs), are spontaneous in learning the spatial hierarchies of features through backpropagation algorithms. It constitutes multiple building blocks, namely convolutional layers, pooling layers, non-linearity layers, and fully connected layers, which transform the inputs in the form of images to the output as in classifying the lesion. They are applied for recognizing the images most commonly since they are designed for feature extraction, and their implementation may improve the detection and classification of lesions to be much faster and efficient than conventional ANNs. The data can be used to train (CNNs) as radiological images’ pixel values, orientation, and relation to other pixels can be analyzed, which helps CNNs comprehend lines, curves, and objects within images. Once trained, CNNs can classify the microcalcifications as either benign or malignant on images it has not seen before.^{3, 8, 32, 35} The automatic hierarchical feature learning ability of deep CNNs also helps reduce false-positive rates in breast cancer screening.³⁶ However, owing to the small sample size and variable presentation, it is a herculean task to train the CNNs from scratch for the medical images. Hence comes the use of transfer learning, another approach to machine learning in which knowledge of previously trained models is applied to a different and new task by pretraining the data in a generalized, broad-spectrum way and further fine-tuning the data to with a particular region of interest to re-apply it to some other task.^{25, 37}

The combination of convolutional neural network (CNN) and transfer learning can shorten the data training time, reduce the amount of data and processor requirements, and improve diagnostic accuracy.³⁸

Another AI technique commonly used for language translation and data extraction from text, such as radiology reports, is natural language processing (NLP).³⁹ It can be used to analyze what such reports say and make predictions based on that.

Representation learning is another subset of ML in which input features are not defined clearly, and the computer algorithm trains itself to become proficient and provide classification as the output. While this method gives results only if an adequate amount of high-quality data is available, feature representation learning techniques help reduce the time to train data and are promising ways to facilitate computer-aided-diagnosis systems for breast lesions.⁴⁰ Radiomics, yet another subfield of AI, aids in extracting vast amounts of features, sometimes even indiscernible to the human eye, from radiographic medical images from different imaging modalities, such as mammography, sonography, and MRI, using data characterization algorithms. It operates similarly to computer-aided diagnosis (CAD) and associates these radiological images with digital pathology images, enhancing our knowledge of imaging-pathology concurrence.¹¹

Trending these days is another term; augmented intelligence — an amalgamation of machine intelligence and human ingenuity — has been described as a sequential approach of artificial intelligence in which humans and machines are interknitted in a constant learning and development loop.⁸ Hybrid-augmented intelligence has been recommended to focus on the high levels of human inconsistency and uncertainty by introducing human-in-the-loop human-computer alliances or embedding cognitive models in the machine learning system.⁴¹

Taking a step further, AI algorithms have been developed to create new synthetic mammographic images from digital databases by applying generative adversarial networks (GAN) to train the data and other algorithms and improve the performance of the CNN classifier. Generative adversarial networks (GANs) seem to be especially prospective due to the dearth of publicly available annotated data and its high cost. GANs create synthetic images through the pairing of neural networks, helping bridge these gaps. A GAN contains a generator that generates a new dataset of fake images based on the input training set and a discriminator to evaluate those images for their authenticity. Hence the novel data generated by GANs can be used to improve research, education, and patient management.^{9, 42}

Workflow

As of now, AI is omnipresent in our digital lives, in the form of spam-detecting, voice recognition, video recommendation, and face identification algorithms, among many others. However, in spite of its normal potential, it has not, as of yet, seen widespread use in the medical field. For example, after appropriate training, natural learning processes could serve as a second opinion indicating the final report’s errors. If we know how, where, and when to use AI, parts of the radiological workflow like optimizing patient scheduling, image acquisition, post-processing image segmentation and quantification, automated detection and interpretation of findings, automated radiation dose estimation, automated data integration and analysis, and automated image quality evaluation, to name a few, could be significantly improved.^{2, 7, 43}

Next, we shall discuss the sequence of steps (Figure 4) — data collection, processing, categorization, diagnosis, and management — involved in the lesion detection and management using AI.

Figure 4 Various steps involved in the lesion detection using AI.

Data Collection and Processing by AI Algorithms

Diverse data from multiple vendors or different institutions should be used to prepare algorithms and train them to prevent bias. Proper scrutiny of the data is crucial since it might reveal a fault in the clinical history or improper image quality can be sorted out by segmentation, feature extraction, and computational analysis. Feature extraction is a fundamental element of dimensionality reduction and a mathematical way of obtaining the relevant constituents from the original input image by Principal component analysis (PCA) and Linear discriminant analysis. Segmentation simplifies representation images into a meaningful and easily analyzable form by methods such as edge detection, thresholding, and clustering.³³ The processing of images, primary source data, text, and quantifiable data is possible using these algorithms’ neural network techniques. Proper scrubbing of data should be done to de-identify any protected health information and prevent legal and ethical issues. Overall, a well-annotated and curated image dataset is required to enable radiomics to discern subtle differences in benign and malignant pathologies and train the appropriate algorithms.^{11, 44} Inadequate quality or quantity of data may hinder the study, even if we use AI techniques to plan patient management.^{45, 46}

There should be three datasets: a training dataset to train the model, a validation dataset to evaluate whether changes in the model improve or make it worse, and a test dataset, which should be entirely different from the training dataset and only used for final evaluation. To avoid overfitting and improve the algorithms, successive iterations of training and validation may be needed.^{11, 33, 47, 48}

Categorizing Images and Data

The system of categorizing images and its integration into AI data to differentiate benign and malignant breast lesions is unpredictable due to inadequate image quality, overlapping diagnostic criteria, the difference in the breast density in patients of various age groups, and apprehension of false-positive cases.^{22, 49}

The BI-RADS lexicons of the American College of Radiology have evaluated and quantified the probability of malignancy as class 4 and its smaller subdivisions by assessing the feature analysis and identifying the image pattern using AI algorithms.^{50, 51} Deep learning CNN tools have also been described to classify mammograms according to the breast’s density and thus established the use of BI-RADS–based structured reporting and helped the radiologists and physicians in the management of these lesions.^{29, 52}

Diagnosis and Management

AI can expand the physician’s potential to collect, understand, and interpret enormous amounts of patients’ specific data. Progress in medical image analysis, such as radiomics, data computation, and machine learning, has augmented our understanding of disease processes and their management.⁵³ A well-designed AI algorithm with appropriately selected techniques can influence patient management positively. They recommend a user-friendly computer model that can compare the output results of the CNN with the radiologist’s imaging assessment for further evaluation. Furthermore, information technology experts can be consulted to resolve the problems related to algorithmic failure.¹⁷

A radiomics-based approach using CNNs with radiology and pathology images as input data can facilitate the recognition of patterns differentiating various lesions and their different probabilities of improvement rate and, hence, can help the breast imagers better demonstrate the need for changing or maintaining current clinical management.^{54, 55} AI with ANNs and DL algorithms can be used to access the available reporting systems, databases, and electronic medical records to predict and analyze the detection rates of breast lesions, differentiate benign from malignant lesions, and assess them as per BIRADS for further management decisions.⁵⁶ The case studies can be used as a future dataset, with the objective of the output being the diagnosis of lesions for appropriate patient management and documentation of guidelines.⁵⁷ Moreover, interdependent analyses of large amounts of data and integrated input from a range of medical specialties like imaging, pathology, surgery, and oncology can help ascertain the best management protocol for patients.^{46, 56}

Advantages of AI

The AI-based strategy reduces radiologists’ workload for breast cancer screening programs without affecting the sensitivity attained by the radiologist.⁵⁸ AI offers superior quality of work for radiologists and patients with faster turnaround, better results, and improved diagnostic certainty (Figure 5). By using machine learning techniques that augment their productivity, radiologists get more time for other patient care activities.⁵⁹ AI systems can accurately diagnose a proportion of the screening population as non-cancerous and reduce false positives, hence augmenting the efficiency of screening mammography.⁶⁰ Factors such as high patient volume, complex diagnostic evaluations, responsibility for biopsies, and direct patient communication make AI beneficial in improving the workflow without dropping out the human factor, which is crucial for both the patients and breast imagers. AI helps the radiologists recognize their virtues and constraints, and eventually, deliver the best patient care.^{33, 34, 47, 56, 58}

Figure 5 Advantages of AI.

Standardized views, availability of images for comparison, systematized reporting format, and classifiable outcome constitute the desirable properties that make screening mammography suitable to train AI algorithms. It may help take a second opinion, generate auto reports for typical cases, predict the likelihood of malignancy, and triage the patients. It is instrumental in the densely populated and developing countries with an inequitable supply of medical resources. The use of AI algorithms can mitigate the excessive workload and shortage of doctors to a certain degree.⁶¹

Challenges with AI

AI algorithms might be confused with situations like inadequate data input or suboptimal pattern recognition that may result from an atypical or rare lesion (Figure 6). Features frequently encountered during imaging — like tumor heterogeneity, broader disease spectrum with atypia and carcinoma in situ, inter and intraobserver variability in diagnosing the lesions, and subjective assessment of breast density — along with substandard image quality might stunt the performance of the neural network.^{11, 20, 27, 39, 40} Data extraction challenges include lack of standardization, non-uniformity of stored data fields, erroneous data, limited annotated data, and image quality variation. Obtaining all data from the same source or equipment may also result in overfitting, whereby a model is unable to generalize patterns beyond the training set.^{13, 47, 48, 62} So, while preparing a well-annotated and curated data set to train the algorithms is costly and time-consuming, it is a highly advantageous process too.⁴⁴ Apart from these, ethical and legal issues regarding decision-making constitute a significant concern.⁶

Figure 6 Various challenges faced by AI.

Inadequate availability of training data sets and overfitting act as major stumbling blocks and can be dealt with by data augmentation, an approach to artificially create the data by increasing the number of training images using image rotation and image flipping.¹⁴

Future Perspectives

Radiomics and the creation of synthetic images via GANs are an evolving subset of artificial intelligence that converts images into mineable high-dimensional data to help analyze and predict further management. In the coming times, combining imaging and non-imaging data in electronic medical records will result in analysis and prediction of clinical diagnosis, hence improving the clinical decision making.⁷

The efforts are being made to improve diagnostic accuracy by deep learning-based CAD-enhanced synthetic mammography and deep learning-based CAD systems to classify breasts according to the density and tailor patient’s management.⁶³

Summary

The literature review can only explore what has already been reported, and therefore there is an element of publication bias even before the search has begun. The current literature delineates far-reaching progress in the knowledge and insight of AI in medical imaging, specifically in radiodiagnosis and breast imaging. AI has a tremendous capability of contributing to the categorization and training of data, including high-quality images. Breast lesions can be predicted more effectively by increasing the study’s sample size, adding suitable input parameters, and using appropriate AI techniques. The recent trend of radiomics, augmented intelligence, and the creation of synthetic images via generative adversarial networks (GANs) will also boost up and prepare the radiologists to collaborate with machines and hence improve the diagnosis, management, and workplace productivity in breast imaging practices.^{8, 20, 56, 63, 64}

Conclusion

Computers, employing various algorithms, can systematize and analyze massive data, including text, medical images, and medical records, in exponentially less time than required for manual processing. With the availability of high-quality imaging data in substantial numbers, the creation of synthetic images, the availability of high-speed computers, cloud computing, and efficient computer programs, AI algorithms can help facilitate radiologists’ workflow, increase diagnosing capacity, and improve patient management.

Clinical support and feedback will be invaluable in developing AI to its full potential in revolutionizing breast imaging.