Contrast-enhanced ultrasound for the differential diagnosis between benign and metastatic superficial lymph nodes: a meta-analysis

Introduction

In a wide number of clinical situations, including prognosis prediction, carcinoma staging, selecting, and monitoring treatment for patients with carcinoma and lymphoma, it is crucial to distinguish between benign and malignant lymph nodes (LNs).^1,2 Currently, several diagnostic imaging modalities are applied to characterize LNs, including ultrasound (US), computed tomography (CT), magnetic resonance imaging (MRI), and positron emission tomography (PET). As reported by Liao et al,³ the pooled sensitivity of CT, MRI, PET, and US was 52%, 65%, 66%, and 66%, respectively, while the pooled specificity was 93%, 81%, 87%, and 78%, respectively. However, currently, there is no imaging modality capable of accurately staging nodal disease. In this context, US-guided fine-needle aspiration cytology (FNAC) can improve the diagnostic accuracy; FNAC has proven to be the most accurate imaging modality for the detection of cervical LN metastases.⁴ Compared with other modalities, US is easy to use, widely available, and less expensive and does not involve exposure to ionizing radiation. Currently, US is regarded as a valuable tool for the diagnosis of metastatic LNs,^5,6 which are characterized by hypoechoic mass, the absence of echogenic hilum, round shape, and blood vessels that are predominant in periphery.^6–8

Contrast-enhanced ultrasound (CEUS), a novel technique for evaluating tissue perfusion in real time, has been widely used for the evaluation of lesions in the liver, kidney, pancreas, spleen, ovary, thyroid, breast, and prostate, as well as for sentinel nodes identification in patients with breast cancer.^9–12 CEUS helps radiologists to detect and characterize focal lesions; its contrast pools, as well as its sensitivity and specificity in diagnosing liver lesions, are similar to contrast CT, which is more costly and involves higher levels of exposure to ionizing radiation. Recently, CEUS has been used to distinguish benign from malignant superficial LNs,^13–15 revealing that CEUS has a higher sensitivity in identifying minor regions of hypovascularity or hypervascularity not detectable by Doppler techniques, thus leading to a better evaluation in LNs categorization.¹⁶ Compared with other imaging modalities, such as CT and MRI, gray-scale US combined with CEUS is capable of evaluating the shape, margins, internal structure, and vascularization of LNs.¹⁷ It has improved the accuracy of ultrasonography for the diagnosis of superficial lymphadenopathy compared with conventional US.^18,19 Nevertheless, Schulte-Altedorneburg et al²⁰ have found that the use of US agent does not improve the diagnostic accuracy in identifying malignant LNs. The main limitation of CEUS is the practice pattern because the interpretation of sonographic images depends on individual skill and the experience of the radiologist. In addition, CEUS is not suitable for patients who have allergic reaction to contrast agents or pregnant women.²¹

A series of studies assessing the accuracy of CEUS in distinguishing benign from malignant superficial LNs have been published, with the sensitivity ranging from 76% to 99% and the specificity ranging from 55% to 98%.^{13,15,22–32} The aim of this meta-analysis was to evaluate the overall accuracy of CEUS for the differential diagnosis between benign and metastatic superficial LNs.

Methods

Search strategy

A comprehensive search of PubMed, Web of Science, Medline, Embase, and the Cochrane Library was performed for relevant literature published from January 1980 to April 2018. The search terms were (lymph node OR lymph nodes) and (ultrasound OR sonography OR ultrasonography OR ultrasonic OR US) and (contrast enhanced OR CE OR contrast enhancement OR echo enhanced OR contrast imaging) and (surgery OR biopsy OR pathology OR histology). All references from the selected studies were reviewed to identify additional relevant articles to expand our study. Two authors (Mei Mei and Ligang Ye) independently searched and extracted the data following the same process. All disagreements were resolved by mutual agreement. Articles written by the same authors were scrutinized to avoid duplicate data. In case there was any unclear information, the authors were contacted for further clarification.

The local ethics committee and institutional review board approved this study.

Study selection

The following studies were included: 1) studies using CEUS for differential diagnosis between benign and metastatic superficial LNs among adults with carcinoma; 2) studies using a reference standard of US-guided biopsy or surgical histology; 3) articles written in English; and 4) studies providing adequate data to construct a 2×2 contingency table for true-positive, true-negative, false-positive, and false-negative results.

The following studies were excluded: 1) studies using CEUS for identifying sentinel LNs by subcutaneous injection of contrast agent; 2) studies with incomplete data available to construct a 2×2 contingency table; 3) retrospective studies; 4) studies that recruited patients with lymphoma; 5) studies using CEUS for evaluating LNs in pediatric patients; 6) studies with duplicate data; 7) reviews, editorials, and letters that did not report their own data; and 8) case reports.

Quality of studies

The revised tool for the quality assessment of diagnostic accuracy studies (QUADAS-2) was used to assess the quality of included studies.³³ This tool comprises four domains: patient selection, index test, reference standard, and flow and timing. Two investigators assessed the quality of the selected studies independently. All disagreements were resolved by mutual consensus. For each domain, the risk of bias and concerns about applicability were labeled as low risk, high risk, and unclear risk.

Statistical methods

This meta-analysis of the evaluation of accuracy of CEUS was performed by calculating the pooled sensitivity, specificity, positive likelihood ratio (LR), negative LR, and diagnostic ORs. We constructed a 2×2 contingency table for each selected study, and a value of 0.5 was added to all cells with zero. Pooled results were calculated by using the Mantel–Haenszel method (fixed-effect model) when there was no significant heterogeneity; otherwise, the DerSimonian–Laird method (random-effect model) was used. The heterogeneity among included studies was analyzed by Cochran’s Q test. Inconsistency (I²) was calculated to express the variability attributable to heterogeneity across the studies. I²>25% was considered significant for heterogeneity when the random-effect model (DerSimonian–Laird method) should be used.³⁴ A summary receiver operating characteristic (SROC) curve was produced by the Moses–Shapiro–Littenberg method to explore the nonlinear relationship between sensitivity and 1 – specificity.³⁵ The value of area under the curve (AUC) was calculated; an AUC close to 0.5 indicated a poor test, whereas an AUC of 1.0 indicated a perfect diagnostic test.³⁶

Additionally, we performed the meta-regression analysis to explore the possible sources of heterogeneity among the included studies with the following covariates: number of LNs (<50 vs ≥50), interpretation of CEUS (qualitative vs quantitative), contrast agent (SonoVue vs Levovist), and contrast mode (harmonic vs color/power Doppler). P<0.05 was considered statistically significant. Furthermore, outliers were identified by inspection of the plots of sensitivity, specificity, positive LR, and negative LR, and SROC curve. A subgroup analysis was performed by excluding outliers. Spearman correlation analysis was conducted to assess threshold effect. A strong positive correlation between the log of sensitivity and the log of 1 – specificity manifested the presence of threshold effect.

For studies using both the qualitative and quantitative diagnostic standards, sensitivity analysis was conducted by using either one of the two or both. Deeks’ asymmetry test was used, and a funnel plot of diagnostic log OR vs 1/sqrt (effective sample size) was constructed for the publication bias assessment.³⁷ A slope coefficient with P<0.05 indicated the presence of publication bias.

The pooled sensitivity, specificity, positive LR, negative LR, diagnostic OR, and SROC AUC, and meta-regression results were analyzed using software Meta-DiSc, version 1.4 (Ramón y Cajal Hospital, Madrid, Spain). The publication bias assessment was performed using Stata version 11.0 (Stata Corporation, College Station, TX, USA).

Results

The predefined search strategy yielded 2,665 studies, among which nine were eligible for analysis. No articles were found by searching the reference lists. The detailed flowchart of literature research is presented in Figure 1, while the characteristics of the selected studies are shown in Table 1. A total of 436 LNs analyzed using CEUS were investigated in this meta-analysis. Among them, 319 LNs were in the neck, 100 were in the axilla, and 17 were in the groin region. In addition, 23 LN metastases were from melanoma, 63 were from squamous cell carcinoma, 85 were from breast cancer, 61 were from ENT cancer, 37 were from oral cavity carcinoma, 8 were from adenocarcinoma, 8 were from small-cell malignancy, and 18 were from poorly differentiated carcinoma. SonoVue and Levovist are the two main contrast agents, and various imaging strategies were applied to US including color/power Doppler and contrast harmonic-enhanced US. The selected studies all used intravenous contrast agent. Among the nine studies, three studies used qualitative interpretation of CEUS, including inhomogeneous nature, early impregnation, perfusion defects, peripheral perfusion, or mixed patterns with heterogenic hyperperfusion;^15,26,32 three studies used quantitative interpretation of CEUS;^27,28,31 and the remaining three studies used both the qualitative and quantitative interpretation of CEUS^13,22,25

Figure 1 Flowchart showing the selection process of the eligible studies.

Table 1 Characteristics of the included studies Notes: aThe number of LNs which was analyzed using CEUS. bThe difference of signal intensity in the hyperintense region relative to that in the hypointense region. Abbreviations: CEUS, contrast-enhanced ultrasound; DPI, derived peak intensity; F, female; LNs, lymph nodes; M, male; NA, not available; RBV, regional blood volume; TIC, time-intensity curve; US, ultrasound.

Quality assessment of studies

The quality assessment of the selected studies using QUADAS-2 tool is shown in Table 2 and Figure 2. Concerning the patients’ selection bias, four studies did not explicitly report consecutive patients.^13,22,26,27 Regarding applicability concerns for the patient selection, four studies were labeled as having high bias because only patients with a definite carcinoma were included.^22,25,26,31 Regarding risk of bias for the index test, four studies did not prespecify the diagnostic threshold.^25,27,28,31 All studies met the predefined criteria such as qualitative diagnostic standards, quantitative diagnostic standards, or both. There were no concerns about reference standard applicability because all studies applied pathological evaluation. Nevertheless, two studies were labeled as “unknown” because the blinded status was not specifically mentioned.^15,27 For the flow and timing domain, one study was labeled as having high risk because not all subjects were included in the analysis.²²

Table 2 Quality assessment of the included studies using QUADAS-2

Figure 2 Quality assessment of the included studies using QUADAS-2 tool.

Meta-analysis

The pooled sensitivity (random-effect model) and specificity (random-effect model) of CEUS for the differential diagnosis of benign and metastatic superficial LNs were 0.88 (95% CI, 0.83–0.92) and 0.80 (95% CI, 0.74–0.85), respectively (Figures 3 and 4). Significant heterogeneity was found for the sensitivity (I²=46.6%) and specificity (I²=77.6%). The pooled positive LR (random-effect model), negative LR (random-effect model), and diagnostic OR (random-effect model) for diagnosing LNs by CEUS were 4.36 (95% CI, 2.38–7.99), 0.17 (95% CI, 0.10–0.31), and 32.75 (95% CI, 11.08–96.84), respectively (Figures 5–7). Significant heterogeneity was found for positive LR (I²=75.4%), negative LR (I²=50.0%), and diagnostic OR (I²=64.0%). SROC AUC was 0.9288 (standard error [SE] =0.0379) (Figure 8). The pooled sensitivity, specificity, positive LR, negative LR, and diagnostic OR were similar to the studies reporting application of either one of the two diagnostic standards or both such as the studies by Poanta et al,¹⁹ Ouyang et al,²⁵ and Cui et al.²²

Figure 3 Forest plot showing the pooled sensitivity of CEUS for diagnosis of superficial metastatic LNs. Abbreviations: CEUS, contrast-enhanced ultrasound; df, degrees of freedom; LNs, lymph nodes; LR, likelihood ratio.

Figure 4 Forest plot showing the pooled specificity of CEUS for diagnosis of superficial metastatic LNs. Abbreviations: CEUS, contrast-enhanced ultrasound; df, degrees of freedom; LNs, lymph nodes; LR, likelihood ratio.

Figure 5 Forest plot showing positive LR of CEUS for diagnosis of superficial metastatic LNs. Abbreviations: CEUS, contrast-enhanced ultrasound; df, degrees of freedom; LNs, lymph nodes; LR, likelihood ratio.

Figure 6 Forest plot showing negative LR of CEUS for diagnosis of superficial metastatic LNs. Abbreviations: CEUS, contrast-enhanced ultrasound; df, degrees of freedom; LNs, lymph nodes; LR, likelihood ratio.

Figure 7 Forest plot showing the diagnostic OR of CEUS for diagnosis of superficial metastatic LNs. Abbreviations: CEUS, contrast-enhanced ultrasound; df, degrees of freedom; LNs, lymph nodes.

Figure 8 SROC curve of CEUS for diagnosis of superficial metastatic LNs. Abbreviations: AUC, area under the curve; CEUS, contrast-enhanced ultrasound; LNs, lymph nodes; SE, standard error; SROC, summary receiver operating characteristic.

The Spearman correlation coefficient between the log of sensitivity and the log of 1 – specificity was –0.467 (P=0.205), indicating that there was no significant threshold effect. The meta-regression analysis was performed to explore the potential sources of heterogeneity. Table 3 shows the relationship between the characteristics of studies and the diagnostic OR. These data suggested that there was no relationship between the characteristics of studies and the diagnostic OR. The forest plots (Figures 3–7) showed that the studies by Zenk et al²⁷ and Moritz et al³² were outliers. Therefore, subgroup analysis excluding outliers was performed, which implied that the heterogeneities were almost eliminated in pooled estimates (Table 4). The SROC AUC was 0.8950 (SE =0.0318).

Table 3 Meta-regression analysis of the possible sources of heterogeneity Notes: aWhen the study characteristics indicated a lower diagnostic OR, RDOR is <1; otherwise, it is >1. P<0.05 indicated significant relationship between the characteristics of studies and the diagnostic OR. Abbreviations: LNs, lymph nodes; RDOR, relative diagnostic OR.

Table 4 Subgroup analysis excluding outliers Note: aI2>50% was considered significant for heterogeneity. Abbreviation: LR, likelihood ratio.

Deeks’ funnel plot of diagnostic log OR vs 1/sqrt (effective sample size) did not show significant asymmetry (P=0.409), indicating that there was no significant publication bias in this meta-analysis (Figure 9).

Figure 9 Funnel plot showing the results of Deeks’ asymmetry test for the assessment of publication bias of the included studies (P=0.409). Abbreviation: ESS, effective sample size.

Discussion

The histological status of LNs is a very powerful prognostic indicator for patients with carcinoma and lymphoma. US has proven to be a valuable technique for evaluating superficial LNs. US is superior to CT and MRI in the view of assessing the morphology and internal structure of LNs¹⁷ because it can allow accurate LN characterization including morphology, margin, and internal structure, while color/power Doppler imaging provides further information about vascularization.^6,8,21,38 Until now, the most commonly used approaches to detect LNs metastases were CT and MRI due to their lower interobserver variation, less time consumption, and ease of detection of deeply located LNs. However, imaging alone is not accurate enough to guide clinical staging and treatment decisions for patients with carcinoma and lymphoma. In this context, CEUS is a simple approach which can allow evaluation of tissue perfusion in real time and extends US examination by a few minutes. This tool has also been shown to be very useful in the differential diagnosis of lymphadenopathy.^13–15

Our meta-analysis demonstrated that CEUS, as a diagnostic imaging technique for superficial metastatic LNs, has a high pooled sensitivity (88%) with a fair pooled specificity (80%). No significant publication bias was found using Deeks’ asymmetry test based on the funnel plot. Therefore, we performed meta-regression analysis to find the potential source of heterogeneity among the studies. Our results revealed that there was no relationship between the characteristics of studies and the diagnostic OR. The forest plots (Figures 2–5) showed that the studies by Zenk et al²⁷ and Moritz et al³² were outliers. Therefore, subgroup analysis excluding outliers was performed, which revealed that the heterogeneities were almost eliminated in pooled estimates (Table 4).

CEUS could be performed by color/power Doppler and by contrast harmonic-enhanced US. US contrast agents were first used to accentuate color/power Doppler signals as reported by Moritz et al, obtaining 100% sensitivity and 98% specificity in differentiation of benign from malignant LNs.³² Nevertheless, Doppler imaging has some limitations, especially in cases of LNs situated nearby pulsating tissues such as arterial vessels. Contrast harmonic-enhanced US imaging is a relatively new technique that can detect the microcirculation and parenchymal perfusion of lesions with a very low mechanical index, thus allowing minimum bubble destruction and a prolonged time of microvascularization assessment. Over recent years, contrast harmonic-enhanced US has been most commonly used for diagnosis of LNs.

The enhanced microvascular pattern is a visualized diagnostic standard. Generally, malignant LNs are inhomogeneous, with early impregnation, perfusion defects, peripheral perfusion, or mixed patterns with heterogenic hyperperfusion. These characteristics of microvascular pattern seem to be reliable features of malignant LNs.^{1,14,15,18,19,25,32,39} Yet, these observations largely depend on the individual skills and experience of the radiologist. Although the objective CEUS parameters concerning the kinetics of contrast agent are significant, their predictive ability is not yet reliable enough for routine clinical use. In addition, Scheipers et al have reported that the computer-guided self-learning analysis systems may help to identify investigator-independent classification characteristics.⁴⁰ Automated quantitative software of US signal intensity was used to analyze the difference between maximum and minimum signal intensity.^25,28 It demonstrated that the value of SI_max–SI_min ranging from 24 to 31 had the highest accuracy for differentiation between benign and malignant LNs with a sensitivity of 92% and a specificity of 89%. Consequently, further random, blinded, and multicenter studies should be performed to confirm the optimal CEUS parameters and cutoffs to differentiate malignant LNs from benign LNs before this approach can be applicable in practice. Perfusion software with higher accuracy should be developed to optimize measurement of functional parameters and to improve the accuracy of CEUS in diagnosing superficial metastatic LNs.

Patients with lymphoma, which have a more variable CEUS appearance, are regarded as a special group because the microvascular pattern of lymphoma partially overlaps with reactive and metastatic LNs.^39,41,42 Yu et al have demonstrated that T-cell lymphomas have intense homogeneous mass, while B-cell lymphomas show perfusion defects.³⁹ Controversially, the most often reported pattern is intense homogeneous enhancement, which causes difficulties in distinguishing lymphomatous nodes from reactive LNs.^1,43,44 Thus, we excluded the studies which recruited patients with lymphoma from the present meta-analysis to avoid heterogeneities. Additionally, since there are very few published studies that are in general based on a very limited sample, the CEUS microvascular pattern for lymphomatous nodes still remains controversial. Therefore, further random, blinded, large sample, and multicenter studies are required to assess the CEUS pattern of lymphoma and to assess the ultimate benefit of CEUS in diagnosing patients with lymphoma before this approach can be applied in practice.

The present study has some limitations. Firstly, various diagnostic standards for CEUS were used in the selected studies, as well as different study designs, contrast agents, and contrast modes. Accordingly, it is very plausible that many variations in characteristics of these studies may lead to the heterogeneity. Secondly, we did not include articles written in other languages besides English. Thirdly, some studies were also excluded because of insufficient data; although we tried to contact the authors to obtain additional data, unfortunately the data were still not sufficient.

Conclusion

CEUS, as a novel imaging modality for the characterization of superficial LNs, showed a high sensitivity and specificity in diagnosing superficial metastatic LNs. The different diagnostic standards, contrast agents, and contrast modes contributed to a considerable level of heterogeneity among the studies. A uniform standard for distinguishing benign from metastatic LNs is needed for further clinical application.

Disclosure

The authors report no conflicts of interest in this work.

References

1.		Slaisova R, Benda K, Jarkovsky J, Petrasova H, Szturz P, Valek V. Contrast-enhanced ultrasonography compared to gray-scale and power doppler in the diagnosis of peripheral lymphadenopathy. Eur J Radiol. 2013;82(4):693–698.
2.		Vassallo P, Wernecke K, Roos N, Peters PE. Differentiation of benign from malignant superficial lymphadenopathy: the role of high-resolution US. Radiology. 1992;183(1):215–220.
3.		Liao LJ, Lo WC, Hsu WL, Wang CT, Lai MS. Detection of cervical lymph node metastasis in head and neck cancer patients with clinically N0 neck-a meta-analysis comparing different imaging modalities. BMC Cancer. 2012;12:236.
4.		de Bondt RB, Nelemans PJ, Hofman PA, et al. Detection of lymph node metastases in head and neck cancer: a meta-analysis comparing US, USgFNAC, CT and MR imaging. Eur J Radiol. 2007;64(2):266–272.
5.		Ying M, Ahuja A, Brook F. Sonographic appearances of cervical lymph nodes: variations by age and sex. J Clin Ultrasound. 2002;30(1):1–11.
6.		Ying M, Ahuja A. Sonography of neck lymph nodes. Part I: normal lymph nodes. Clin Radiol. 2003;58(5):351–358.
7.		van den Brekel MW, Stel HV, Castelijns JA, et al. Cervical lymph node metastasis: assessment of radiologic criteria. Radiology. 1990;177(2):379–384.
8.		Dudea SM, Lenghel M, Botar-Jid C, Vasilescu D, Duma M. Ultrasonography of superficial lymph nodes: benign vs. malignant. Med Ultrason. 2012;14(4):294–306.
9.		Halpern EJ, Rosenberg M, Gomella LG. Prostate cancer: contrast-enhanced us for detection. Radiology. 2001;219(1):219–225.
10.		Ladam-Marcus V, Mac G, Job L, Piot-Veron S, Marcus C, Hoeffel C. Performance de l’échographie decontraste en pathologie hépatique: synthèse de la littérature [Contrast-enhanced ultrasound and liver imaging: review of the literature]. J Radiol. 2009;90(1 Pt 2):93–106.
11.		Hirooka Y, Itoh A, Kawashima H, et al. Contrast-enhanced endoscopic ultrasonography in digestive diseases. J Gastroenterol. 2012;47(10):1063–1072.
12.		Cox K, Sever A, Jones S, et al. Validation of a technique using microbubbles and contrast enhanced ultrasound (CEUS) to biopsy sentinel lymph nodes (SLN) in pre-operative breast cancer patients with a normal grey-scale axillary ultrasound. Eur J Surg Oncol. 2013;39(7):760–765.
13.		Poanta L, Serban O, Pascu I, Pop S, Cosgarea M, Fodor D. The place of CEUS in distinguishing benign from malignant cervical lymph nodes: a prospective study. Med Ultrason. 2014;16(1):7–14.
14.		Rubaltelli L, Beltrame V, Scagliori E, et al. Potential use of contrast-enhanced ultrasound (CEUS) in the detection of metastatic superficial lymph nodes in melanoma patients. Ultraschall Med. 2014;35(1):67–71.
15.		Dudau C, Hameed S, Gibson D, et al. Can contrast-enhanced ultrasound distinguish malignant from reactive lymph nodes in patients with head and neck cancers? Ultrasound Med Biol. 2014;40(4):747–754.
16.		Liu H, Xiao J. Flow imaging of residual neck lymph nodes in nasopharyngeal carcinoma by contrast-enhanced ultrasonography after radiotherapy. Zhong Nan Da Xue Xue Bao Yi Xue Ban. 2012;35(8):861–863.
17.		Sumi M, Ohki M, Nakamura T. Comparison of sonography and CT for differentiating benign from malignant cervical lymph nodes in patients with squamous cell carcinoma of the head and neck. AJR Am J Roentgenol. 2001;176(4):1019–1024.
18.		Rubaltelli L, Beltrame V, Tregnaghi A, Scagliori E, Frigo AC, Stramare R. Contrast-enhanced ultrasound for characterizing lymph nodes with focal cortical thickening in patients with cutaneous melanoma. AJR Am J Roentgenol. 2011;196(1):W8–W12.
19.		Poanta L, Serban O, Pascu I, Pop S, Cosgarea M, Fodor D. The place of CEUS in distinguishing benign from malignant cervical lymph nodes: a prospective study. Med Ultrason. 2014;16(1):7–14.
20.		Schulte-Altedorneburg G, Demharter J, Linné R, Droste DW, Bohndorf K, Bücklein W. Does ultrasound contrast agent improve the diagnostic value of colour and power Doppler sonography in superficial lymph node enlargement? Eur J Radiol. 2003;48(3):252–257.
21.		Chong WK, Papadopoulou V, Dayton PA. Imaging with ultrasound contrast agents: current status and future. Abdom Radiol. 2018;43(4):762–772.
22.		Cui QL, Yin SS, Fan ZH, Yang W, Wang S, Yan K. Diagnostic Value of Contrast-Enhanced Ultrasonography and Time-Intensity Curve in Differential Diagnosis of Cervical Metastatic and Tuberculous Lymph Nodes. J Ultrasound Med. 2018;37(1):83–92.
23.		Rubaltelli L, Beltrame V, Scagliori E, et al. Potential use of contrast-enhanced ultrasound (CEUS) in the detection of metastatic superficial lymph nodes in melanoma patients. Ultraschall Med. 2014;35(1):67–71.
24.		Podkrajsek M, Hocevar M. The role of contrast enchanced axillary ultrasonography in early breast cancer patients. Coll Antropol. 2011;35(1):33–37.
25.		Ouyang Q, Chen L, Zhao H, Xu R, Lin Q. Detecting metastasis of lymph nodes and predicting aggressiveness in patients with breast carcinomas. J Ultrasound Med. 2010;29(3):343–352.
26.		de Giorgi V, Gori A, Grazzini M, et al. Contrast-enhanced ultrasound: a filter role in AJCC stage I/II melanoma patients. Oncology. 2010;79(5–6):370–375.
27.		Zenk J, Bozzato A, Hornung J, et al. Neck lymph nodes: prediction by computer-assisted contrast medium analysis? Ultrasound Med Biol. 2007;33(2):246–253.
28.		Rubaltelli L, Corradin S, Dorigo A, et al. Automated quantitative evaluation of lymph node perfusion on contrast-enhanced sonography. AJR Am J Roentgenol. 2007;188(4):977–983.
29.		Schmid-Wendtner MH, Dill-Müller D, Baumert J, et al. Lymph node metastases in patients with cutaneous melanoma: improvements in diagnosis by signal-enhanced color Doppler sonography. Melanoma Res. 2004;14(4):269–276.
30.		Schmid-Wendtner MH, Partscht K, Korting HC, Volkenandt M. Improved differentiation of benign and malignant lymphadenopathy in patients with cutaneous melanoma by contrast-enhanced color Doppler sonography. Arch Dermatol. 2002;138(4):491–497.
31.		Yang WT, Metreweli C, Lam PK, Chang J. Benign and malignant breast masses and axillary nodes: evaluation with echo-enhanced color power Doppler US. Radiology. 2001;220(3):795–802.
32.		Moritz JD, Ludwig A, Oestmann JW. Contrast-enhanced color Doppler sonography for evaluation of enlarged cervical lymph nodes in head and neck tumors. AJR Am J Roentgenol. 2000;174(5):1279–1284.
33.		Whiting PF, Rutjes AW, Westwood ME, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155(8):529–536.
34.		Deeks JJ. Systematic reviews in health care: Systematic reviews of evaluations of diagnostic and screening tests. BMJ. 2001;323(7305):157–162.
35.		Moses LE, Shapiro D, Littenberg B. Combining independent studies of a diagnostic test into a summary ROC curve: data-analytic approaches and some additional considerations. Stat Med. 1993;12(14):1293–1316.
36.		Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29–36.
37.		Deeks JJ, Macaskill P, Irwig L. The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. J Clin Epidemiol. 2005;58(9):882–893.
38.		Ahuja A, Ying M. Sonography of neck lymph nodes. Part II: abnormal lymph nodes. Clin Radiol. 2003;58(5):359–366.
39.		Yu M, Liu Q, Song HP, et al. Clinical application of contrast-enhanced ultrasonography in diagnosis of superficial lymphadenopathy. J Ultrasound Med. 2010;29(5):735–740.
40.		Scheipers U, Siebers S, Gottwald F, et al. Sonohistology for the computerized differentiation of parotid gland tumors. Ultrasound Med Biol. 2005;31(10):1287–1296.
41.		Sidhu PS, Cantisani V, Dietrich CF, et al. The EFSUMB Guidelines and Recommendations for the Clinical Practice of Contrast-Enhanced Ultrasound (CEUS) in Non-Hepatic Applications: Update 2017 (Short Version). Ultraschall Med. 2018;39(2):154–180.
42.		Jiang W, Xue H, Wang Q, Zhang X, Wang Z, Zhao C. Value of contrast-enhanced ultrasound and PET/CT in assessment of extramedullary lymphoma. Eur J Radiol. 2018;99:88–93.
43.		Nakase K, Yamamoto K, Hiasa A, Tawara I, Yamaguchi M, Shiku H. Contrast-enhanced ultrasound examination of lymph nodes in different types of lymphoma. Cancer Detect Prev. 2006;30(2):188–191.
44.		Cui XW, Jenssen C, Saftoiu A, Ignee A, Dietrich CF. New ultrasound techniques for lymph node evaluation. World J Gastroenterol. 2013;19(30):4850.