Back to Journals » Cancer Management and Research » Volume 12

Diagnostic Significance of Downregulated circMORC3 as a Molecular Biomarker of Hypopharyngeal Squamous Cell Carcinoma: A Pilot Study

Authors Guo Y, Huang Q, Zheng J, Hsueh CY, Huang J, Yuan X, Chen H, Zhou L

Received 25 October 2019

Accepted for publication 16 December 2019

Published 6 January 2020 Volume 2020:12 Pages 43—49


Checked for plagiarism Yes

Review by Single anonymous peer review

Peer reviewer comments 2

Editor who approved publication: Dr Sanjeev K. Srivastava

Yang Guo, Qiang Huang, Juan Zheng, Chi-Yao Hsueh, Jiameng Huang, Xiaohui Yuan, Hui Chen, Liang Zhou

Department of Otorhinolaryngology Head and Neck Surgery, Shanghai Key Clinical Disciplines of Otorhinolaryngology, Eye and ENT Hospital of Fudan University, Shanghai, People’s Republic of China

Correspondence: Hui Chen; Liang Zhou
Department of Otorhinolaryngology Head and Neck Surgery, Shanghai Key Clinical Disciplines of Otorhinolaryngology, Eye and ENT Hospital of Fudan University, No 83, Fenyang Road, Xuhui District, Shanghai 200031, People’s Republic of China
Fax +86 21 6437 7134
Email [email protected]; [email protected]

Background: Circular RNAs (circRNAs) have proven to be of great clinical significance as diagnostic biomarkers in various cancers. Here, we investigate the expression of circMORC3 in hypopharyngeal squamous cell carcinoma (HSCC), exploring whether it could serve as a diagnostic marker of HSCC.
Methods: CircMORC3 expression levels were detected in HSCC tissues and adjacent normal tissues using quantitative real-time polymerase chain reaction (qRT-PCR). The relationships between circMORC3 expression levels and clinicopathologic factors were explored. CircMORC3 expression levels in plasma from HSCC patients and non-tumor patients were also detected by qRT-PCR. Finally, receiver operating characteristic (ROC) curves were established to evaluate the diagnostic value of circMORC3 as a potential HSCC biomarker in tissues and plasma.
Results: The expression levels of circMORC3 were significantly lower in HSCC tissues than paired adjacent normal tissues, and the area under the ROC curve was 0.834. The decreased expression of circMORC3 was correlated to T stages and tumor sizes. Similarly, the circMORC3 expression levels in HSCC patient plasma were lower than non-tumor patient plasma, and the area under the ROC curve was 0.767.
Conclusion: Our results indicate that circMORC3 was downregulated in HSCC tissues and plasma, and it could serve as an early diagnostic HSCC biomarker.

Keywords: circMORC3, diagnostic significance, molecular biomarker, hypopharyngeal squamous cell carcinoma


Hypopharyngeal squamous cell carcinoma (HSCC), the most common histological type of hypopharyngeal cancers generated from the hypopharynx, accounts for about 5% of head and neck squamous cell carcinomas.1,2 Due to the special anatomical location, it is not discoverable in the early stages, and most patients are diagnosed at advanced stages.3 Although surgical treatment has developed greatly in recent decades, the quality of life of patients has not significantly improved because many cases are treated at the expense of laryngeal function due to late diagnosis.4 Thus, further exploration of the underlying molecular mechanisms and identification of novel molecular biomarkers for HSCC is of great value for clinical practice.

Circular RNAs (circRNAs) are covalently joined single-strand RNA loops that are abundant in various eukaryotic transcriptomes.5 This unique molecular structure endows circRNAs with stability based on their exonuclease resistance.6 Usually, circRNA expression is cell-, tissue-, developmental stage- and, most importantly, disease-specific.7,8 CircRNAs are abundant in tissues, plasma, saliva and exosomes.911 Thus, circRNAs are ideal molecular biomarkers for various diseases, including diverse cancers of the aerodigestive tract.1214 For example, hsa_circ_00001649, hsa_circ_0000745 and hsa_circ_0000140 were suggested to be novel biomarkers for gastric cancer,1517 and hsa_circ_0001946, hsa_circ_004360318 and hsa_circ_006793419 could serve as biomarkers for esophageal squamous cell carcinoma. At the same time, the clinical application potential of hsa_circ_0000711 and hsa_circ_0004585 as biomarkers for colorectal cancer was demonstrated.20,21 However, few studies have focused on the involvement and clinical significance of circRNAs in HSCC.

MORC family CW-type zinc finger 3 (MORC3) belongs to a new family of ATPases, MORCs, which participate in various cancers.22,23 MORC3 expression changed after anticancer chemotherapy, suggesting that MORC3 was involved in reactions to cellular damage caused by chemotherapy.24 And by recruiting p53 and localizing it to promyelocytic leukemia nuclear bodies (PML-NBs), MORC3 plays essential roles in p53-induced cellular senescence.25 A total of 41 circRNAs could be generated from MORC3 in the human genome, and 3 circRNAs could be produced in the mouse genome according to the records in circBase.26 Hsa_circ_0001189, located at chr21:37711076–37717005 +, was highly conserved compared to mmu_circ_0006475 located at chr16:93847532–93853462 +. Thus, we named this isoform of hsa_circ_0001189 “circMORC3,” and we investigated its expression and clinical importance in HSCC.

Here, we demonstrate that circMORC3 expression levels were significantly lower in HSCC tissues compared to adjacent normal tissues. The relationship between circMORC3 expression and clinicopathological factors in HSCC patients was explored. CircMORC3 was shown to be downregulated in the plasma of HSCC patients compared to non-tumor patients, and the diagnostic roles of circMORC3 were further explored.


Patients and Samples

HSCC tissues and paired adjacent normal tissues were collected in Department of Otorhinolaryngology Head and Neck Surgery, Eye & ENT Hospital of Fudan University from HSCC patients who underwent surgery. Patients who received chemotherapy and/or radiotherapy were excluded from the study. A total of 33 pairs of HSCC tissues and adjacent normal mucosa tissues were included. All HSCC tissues were histopathologically confirmed by pathologists, and pathological stages were determined.27 All samples were stored at −80°C until RNA extraction. This study was approved by the Institutional Ethics Committee of the Eye & ENT Hospital of Fudan University, and all patients signed informed consent forms before sample collection.

Peripheral Blood Collection

Venous blood samples from 22 HSCC patients and age-matched non-tumor (vocal cord polyps) patients were obtained in trisodium citrate anticoagulation tubes. Plasma was extracted within 2 h after blood collection. First, peripheral bloods were centrifuged at 1500 rpm for 10 min to primarily isolate plasma. The plasma was centrifuged again at 12,000 g for 30 min. Finally, the isolated plasma was stored at −80°C until use.

RNA Extraction and qRT-PCR Analyses

Total RNA was extracted using TRIzol reagent (Invitrogen, Carlsbad, California, USA) according to the manufacturer’s instructions. Nanodrop ND-2000 (Thermo Fisher Scientific, Inc) was used to measure the quantity and quality of RNA. PrimeScript™ RT Master Mix (TaKaRa, Dalian, China) was used to perform reverse transcription, and qRT-PCR was performed with SYBR Green Premix Ex Taq™ II (TaKaRa, Dalian, China) according to the manufacturer’s instructions. GAPDH was chosen as the internal standard control, and relative expression of transcripts were compared using the 2−ΔΔCt method. The following primers were used: forward primer of 5′-CGAACGTGATGTTTATCGAC-3′ and reverse primer of 5′-CAGTAACTTCTGTTCCGTGG-3′ for circMORC3; forward primer of 5′-TCCTGATGTGAACGCTAAACAAA −3′ and reverse primer of 5′-GAACCCGACTTGAAGCCATTC-3′ for MORC3; and forward primer of 5′-CAATGACCCCTTCATTGACC-3′ and reverse primer of 5′-TTGATTTTGGAGGGATCTCG-3′ for GAPDH.

Statistical Analysis

Statistical analyses were performed using GraphPad Prism 6.02 (GraphPad Software, La Jolla, CA, USA) and SPSS version 22.0 (SPSS Inc, Chicago, IL, USA). The results of qRT-PCR were presented as mean ± standard  deviation (SD). Paired t-tests were used to compare the relative expression of circMORC3 and MORC3 in HSCC and adjacent normal tissues. Non-paired t-tests were used to compare the relative expression of circMORC3 in plasma from HSCC patients and non-tumor patients. Similarly, non-paired t-tests were used to test the relationships between circMORC3 expression levels and clinicopathological characteristics. Pearson correlation analysis was used to explore the correlation between expression levels of circMORC3 and MORC3 in HSCC tissues. The diagnostic value of circMORC3 was evaluated using the area under the receiver operating characteristic (ROC) curve (AUC). P values < 0.05 were considered as statistically significant.


CircMORC3 Expression Levels Were Downregulated in HSCC Tissues

CircMORC3 was generated from MORC3 consisting of exons 5 to 7. Divergent primers were designed to examine circMORC3 expression levels in HSCC tissues and adjacent noncancerous tissues. CircMORC3 expression levels were markedly downregulated in HSCC tissues compared to matched normal tissues (Figure 1A). Downregulated circMORC3 was observed in 84.8% (28/33) of patients with HSCC compared to corresponding noncancerous tissues (Figure 1B).

Figure 1 (A) CircMORC3 was downregulated in HSCC tissues compared to paired adjacent normal tissues (n = 33, **P < 0.001). (B) CircMORC3 was downregulated in 84.8% (28/33) of HSCC patients compared to corresponding noncancerous tissues.

Correlation of circMORC3 Expression Levels with Clinicopathological Features of HSCC Patients

Considering the obvious downregulation of circMORC3 in HSCC compared to paired normal tissues, we explored correlations between the expression levels of circMORC3 in tissues and the clinicopathological characteristics of HSCC patients. As shown in Table 1, the expression levels of circMORC3 were significantly lower in HSCCs with higher T stages than lower T stages (P = 0.0194), and circMORC3 expression levels were significantly lower in HSCCs with larger tumor sizes compared to smaller tumor sizes (P = 0.0438). Other clinicopathological factors like age, N stage, TNM stage, and pathological differentiation were not associated with circMORC3 expression.

Table 1 Correlation Between circMORC3 Expression Levels and Clinicopathological Features of Hypopharyngeal Squamous Cell Carcinoma Patients (HSCCs)

The Expression Levels of MORC3 and Correlation Between circMORC3 and MORC3 in HSCC Tissues

The cognate linear mRNAs generated from the parental gene of circMORC3 were also quantitively examined using qRT-PCR. Consistent with the downregulation of circMORC3, the mRNA MORC3 was decreased in HSCC tissues too (P < 0.0001; Figure 2A). The positive relationship between circMORC3 expression and MORC3 expression was validated using Pearson correlation analysis (r = 0.415, P = 0.016; Figure 2B).

Figure 2 (A) MORC3, the cognate mRNA of circMORC3, was downregulated in HSCC tissues compared to adjacent normal tissues (n = 33, ***P < 0.0001). (B) CircMORC3 and MORC3 were significantly correlated in HSCC tissues (r = 0.415, P = 0.016).

Potential Diagnostic Value of circMORC3 in HSCC

Based on the significant relationship between circMORC3 expression levels and the clinicopathological characteristics of HSCC patients, ROC curve analysis was used to evaluate the underlying diagnostic value of circMORC3 expression. The AUC for circMORC3 distinguishing HSCC from normal adjacent tissues was 0.834 (95% confidence interval (CI): 0.737 to 0.931; P < 0.0001; Figure 3), suggesting that it could serve as potential HSCC diagnostic biomarker.

Figure 3 The diagnostic potential of circMORC3 in HSCC tissues. The area under the ROC curve (AUC) was 0.834 (95% confidence interval (CI): 0.737 to 0.931; P < 0.0001).

CircMORC3 Expression Was Downregulated in HSCC Plasma

In total, plasma from 22 pairs of HSCC patients and non-tumor patients were collected. CircMORC3 was significantly downregulated in HSCC plasma compared to circMORC3 expression levels in non-tumor patient plasma (P = 0.009; Figure 4A). ROC curve analysis was also used to assess the diagnostic role of circMORC3 in plasma. The results suggested that circMORC3 could distinguish HSCC patients from non-tumor patients with an AUC of 0.767 (95% CI: 0.627 to 0.906; P = 0.002; Figure 4B).

Figure 4 (A) Expression of circMORC3 was downregulated in HSCC plasma compared to non-tumor patients (n = 22, *P = 0.009). (B) The diagnostic value of circMORC3 in plasma. The area under the ROC curve (AUC) was 0.767 (95% confidence interval (CI): 0.627 to 0.906; P = 0.002).


Representing a high malignant level, HSCCs are generated from the upper aerodigestive tract mucosa.2 The diagnosis of HSCC typically comes at a late stage, and the prognosis of HSCC was poor. The potential of circRNA to serve as biomarkers might shed light on the early detection and diagnosis of HSCC.

CircRNAs are a new focus of the non-coding RNA field, and studies suggest that they are involved in different hallmarks of cancers.2830 Generally, circRNAs function by sponging different miRNAs to regulate various target mRNAs.31 CircRNAs could also affect physiopathologic processes by binding to RNA binding proteins or regulating transcription and splicing.12 CircRNAs are extremely stable compared to linear RNAs because they lack free 3′ or 5′ ends. CircRNAs exist widely in various tissues and in saliva and plasma, which allows for non-invasive diagnosis with dysregulated circRNAs.12

Here, we validated the downregulation of circMORC3 in HSCC tissues compared to paired normal tissues, which was consistent with the previous microarray results.32 Further analyses of the relationships between circMORC3 expression levels and clinicopathological features illustrated that the expression levels of circMORC3 were more significantly decreased in HSCC with higher T stages and bigger tumor sizes than those with lower T stages and smaller tumor sizes. Although the detailed mechanisms need more careful study, the results suggested that circMORC3 expression was correlated with HSCC proliferation abilities. ROC curve analysis indicated that circMORC3 could serve as a potential biomarker for HSCC based on the AUC of 0.834.

It should be noted that circMORC3 expression levels were significantly correlated with the cognate linear mRNA MORC3. This might imply that circMORC3 might function partly through modulating MORC3 expression, which is consistent with previous reports. For instance, circSKA3 and circDTL acted as oncogenes by regulating the expression of their host genes in medulloblastoma.33 Similarly, circGFRA1 functioned as a competing endogenous RNA (ceRNA) to regulate GFRA1 expression by sponging shared miR-34a to promote the tumorigenesis of triple negative breast cancer.34 However, the precise molecular mechanisms by which circMORC3 participates in HSCC remain to be explored.

Recent studies demonstrated the existence of circRNA in plasma and saliva, and their non-invasive diagnostic values were also shown.35 For example, hsa_circ_0000190, hsa_circ_0000745, and hsa_circ_0000520 were downregulated in gastric cancer plasma, and they could be used as biomarkers with AUC of 0.60, 0.683, and 0.8967, respectively.16,36,37 Zhao et al found that hsa_circ_0001874 and hsa_circ_0001971 were upregulated in saliva of oral squamous cell carcinoma patients compared to healthy controls, and their AUCs reached 0.863 and 0.845, respectively. Interestingly, the AUC of the two circRNAs in combination reached 0.922.10 To investigate the diagnostic value of circMORC3 in HSCC, the expression of circMORC3 was first shown to be decreased in HSCC plasma compared to non-tumor plasma. ROC curve analysis suggested the potential value of circMORC3 in HSCC diagnosis with an AUC of 0.767.

According to Jones and Athanasiou,38 an AUC of 0.75 to 0.92 is good and an AUC of 0.93 to 0.96 is very good. Thus, circMORC3 could serve as a diagnostic biomarker with good predictability for HSCC in both tissues and plasma. Further studies are still needed to verify the diagnostic potential of circMORC3 in a larger cohort and elucidate the molecular mechanism of circMORC3 in oncogenesis and progression of HSCC.


Our results indicated that circMORC3 was downregulated in HSCC tissues compared to adjacent normal tissues, and its expression levels were correlated with T stages and tumor sizes of HSCCs. CircMORC3 could distinguish HSCC tissues from adjacent normal tissues with an AUC of 0.834. Furthermore, circMORC3 expression in HSCC patient plasma was significantly lower than in non-tumor patients, and ROC curve analysis suggested that circMORC3 could serve as a promising non-invasive biomarker for early HSCC diagnosis. However, the detailed mechanism needs further investigation in the future.


The present study was supported by grants from the Science and Technology Commission of Shanghai Municipality (12J1402100 and 16411950101), Shanghai Shen Kang Hospital Development Center (SHDC12015114) and National Natural Science Foundation of China (81402237).


The authors report no other conflicts of interest in this work.


1. Gourin CG, Terris DJ. Carcinoma of the hypopharynx. Surg Oncol Clin N Am. 2004;13(1):81–98. doi:10.1016/S1055-3207(03)00122-4

2. Qian Y, Liu D, Cao S, et al. Upregulation of the long noncoding RNA UCA1 affects the proliferation, invasion, and survival of hypopharyngeal carcinoma. Mol Cancer. 2017;16(1):68. doi:10.1186/s12943-017-0635-6

3. Zhou L, Cheng L, Tao L, Jia X, Lu Y, Liao P. Detection of hypopharyngeal squamous cell carcinoma using serum proteomics. Acta Otolaryngol. 2006;126(8):853–860. doi:10.1080/00016480500525205

4. Kwon DI, Miles BA, Education Committee of the American H, Neck S. Hypopharyngeal carcinoma: do you know your guidelines? Head Neck. 2018;41(3):569–76.

5. Memczak S, Jens M, Elefsinioti A, et al. Circular RNAs are a large class of animal RNAs with regulatory potency. Nature. 2013;495(7441):333–338. doi:10.1038/nature11928

6. Shang Q, Yang Z, Jia R, Ge S. The novel roles of circRNAs in human cancer. Mol Cancer. 2019;18(1):6. doi:10.1186/s12943-018-0934-6

7. Xia S, Feng J, Chen K, et al. CSCD: a database for cancer-specific circular RNAs. Nucleic Acids Res. 2018;46(D1):D925–D9. doi:10.1093/nar/gkx863

8. Salzman J, Chen RE, Olsen MN, Wang PL, Brown PO. Cell-type specific features of circular RNA expression. PLoS Genet. 2013;9(9):e1003777. doi:10.1371/journal.pgen.1003777

9. Memczak S, Papavasileiou P, Peters O, Rajewsky N. Identification and characterization of circular RNAs as a new class of putative biomarkers in human blood. PLoS ONE. 2015;10(10):e0141214. doi:10.1371/journal.pone.0141214

10. Zhao SY, Wang J, Ouyang SB, Huang ZK, Liao L. Salivary circular RNAs Hsa_Circ_0001874 and Hsa_Circ_0001971 as novel biomarkers for the diagnosis of oral squamous cell carcinoma. Cell Physiol Biochem. 2018;47(6):2511–2521. doi:10.1159/000491624

11. Li Y, Zheng Q, Bao C, et al. Circular RNA is enriched and stable in exosomes: a promising biomarker for cancer diagnosis. Cell Res. 2015;25(8):981–984. doi:10.1038/cr.2015.82

12. Guo Y, Yang J, Huang Q, et al. Circular RNAs and their roles in head and neck cancers. Mol Cancer. 2019;18(1):44. doi:10.1186/s12943-019-1003-5

13. Cai J, Chen Z, Zuo X. circSMARCA5 functions as a diagnostic and prognostic biomarker for gastric cancer. Dis Markers. 2019;2019:2473652.

14. Gu W, Sun Y, Zheng X, et al. Identification of gastric cancer-related circular RNA through microarray analysis and bioinformatics analysis. Biomed Res Int. 2018;2018:2381680. doi:10.1155/2018/2381680

15. Li P, Chen S, Chen H, et al. Using circular RNA as a novel type of biomarker in the screening of gastric cancer. Clin Chim Acta. 2015;444:132–136. doi:10.1016/j.cca.2015.02.018

16. Huang M, He YR, Liang LC, Huang Q, Zhu Z. Circular RNA hsa_circ_0000745 may serve as a diagnostic marker for gastric cancer. World J Gastroenterol. 2017;23(34):6330–6338. doi:10.3748/wjg.v23.i34.6330

17. Li WH, Song YC, Zhang H, et al. Decreased expression of Hsa_circ_00001649 in gastric cancer and its clinical significance. Dis Markers. 2017;2017:4587698. doi:10.1155/2017/4587698

18. Fan L, Cao Q, Liu J, Zhang J, Li B. Circular RNA profiling and its potential for esophageal squamous cell cancer diagnosis and prognosis. Mol Cancer. 2019;18(1):16. doi:10.1186/s12943-018-0936-4

19. Xia W, Qiu M, Chen R, et al. Circular RNA has_circ_0067934 is upregulated in esophageal squamous cell carcinoma and promoted proliferation. Sci Rep. 2016;6:35576. doi:10.1038/srep35576

20. Li J, Ni S, Zhou C, Ye M. The expression profile and clinical application potential of hsa_circ_0000711 in colorectal cancer. Cancer Manag Res. 2018;10:2777–2784. doi:10.2147/CMAR.S172388

21. Tian J, Xi X, Wang J, et al. CircRNA hsa_circ_0004585 as a potential biomarker for colorectal cancer. Cancer Manag Res. 2019;11:5413–5423. doi:10.2147/CMAR.S199436

22. Hong G, Qiu H, Wang C, et al. The emerging role of MORC family proteins in cancer development and bone homeostasis. J Cell Physiol. 2017;232(5):928–934. doi:10.1002/jcp.v232.5

23. Zhang Y, Klein BJ, Cox KL, et al. Mechanism for autoinhibition and activation of the MORC3 ATPase. Proc Natl Acad Sci U S A. 2019;116(13):6111–6119. doi:10.1073/pnas.1819524116

24. Gonzalez-Fernandez R, Morales M, Avila J, Martin-Vasallo P. Changes in leukocyte gene expression profiles induced by antineoplastic chemotherapy. Oncol Lett. 2012;3(6):1341–1349. doi:10.3892/ol.2012.669

25. Takahashi K, Yoshida N, Murakami N, et al. Dynamic regulation of p53 subnuclear localization and senescence by MORC3. Mol Biol Cell. 2007;18(5):1701–1709. doi:10.1091/mbc.e06-08-0747

26. Glazar P, Papavasileiou P, Rajewsky N. circBase: a database for circular RNAs. RNA. 2014;20(11):1666–1670. doi:10.1261/rna.043687.113

27. Brierley JD, Gospodarowicz MK, Wittekind C. TNM Classification of Malignant Tumours. John Wiley & Sons; 2016.

28. Kristensen LS, Hansen TB, Veno MT, Kjems J. Circular RNAs in cancer: opportunities and challenges in the field. Oncogene. 2018;37(5):555–565. doi:10.1038/onc.2017.361

29. Hanahan D, Weinberg RA. Hallmarks of cancer: the next generation. Cell. 2011;144(5):646–674. doi:10.1016/j.cell.2011.02.013

30. Su M, Xiao Y, Ma J, et al. Circular RNAs in Cancer: emerging functions in hallmarks, stemness, resistance and roles as potential biomarkers. Mol Cancer. 2019;18(1):90. doi:10.1186/s12943-019-1002-6

31. Zhong Y, Du Y, Yang X, et al. Circular RNAs function as ceRNAs to regulate and control human cancer progression. Mol Cancer. 2018;17(1):79. doi:10.1186/s12943-018-0827-8

32. Cao S, Wei D, Li X, et al. Novel circular RNA expression profiles reflect progression of patients with hypopharyngeal squamous cell carcinoma. Oncotarget. 2017;8(28):45367–45379. doi:10.18632/oncotarget.17488

33. Lv T, Miao YF, Jin K, et al. Dysregulated circular RNAs in medulloblastoma regulate proliferation and growth of tumor cells via host genes. Cancer Med. 2018;7(12):6147–6157. doi:10.1002/cam4.1613

34. He R, Liu P, Xie X, et al. circGFRA1 and GFRA1 act as ceRNAs in triple negative breast cancer by regulating miR-34a. J Exp Clin Cancer Res: CR. 2017;36(1):145. doi:10.1186/s13046-017-0614-1

35. Bahn JH, Zhang Q, Li F, et al. The landscape of microRNA, Piwi-interacting RNA, and circular RNA in human saliva. Clin Chem. 2015;61(1):221–230. doi:10.1373/clinchem.2014.230433

36. Chen S, Li T, Zhao Q, Xiao B, Guo J. Using circular RNA hsa_circ_0000190 as a new biomarker in the diagnosis of gastric cancer. Clinica Chimica Acta. 2017;466:167–171. doi:10.1016/j.cca.2017.01.025

37. Sun H, Tang W, Rong D, et al. Hsa_circ_0000520, a potential new circular RNA biomarker, is involved in gastric carcinoma. Cancer Biomark. 2018;21(2):299–306. doi:10.3233/CBM-170379

38. Jones CM, Athanasiou T. Summary receiver operating characteristic curve analysis techniques in the evaluation of diagnostic tests. Ann Thorac Surg. 2005;79(1):16–20. doi:10.1016/j.athoracsur.2004.09.040

Creative Commons License © 2020 The Author(s). This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.