Evaluating the Viability of a Smartphone-Based Annotation Tool for Faster and Accurate Image Labelling for Artificial Intelligence in Diabetic Retinopathy
Received 29 October 2020
Accepted for publication 18 January 2021
Published 8 March 2021 Volume 2021:15 Pages 1023—1039
Checked for plagiarism Yes
Review by Single anonymous peer review
Peer reviewer comments 4
Editor who approved publication: Dr Scott Fraser
Arvind Kumar Morya,1 Jaitra Gowdar,2 Abhishek Kaushal,2 Nachiket Makwana,2 Saurav Biswas,2 Puneeth Raj,2 Sharat Hegde,3 Girish Velis,4 Winston Padua,5 Nazneen Nazm,6 Sangeetha Jeganathan,7 Sandeep Choudhary,1 Nishant Parashar,1 Bhavana Sharma,8 Pankaja Raghav1
1Department of Ophthalmology, All India Institute of Medical Sciences, Jodhpur, Rajasthan, 342005, India; 2Radiate Healthcare Innovations Private Limited, Bangalore, Karnataka, 560038, India; 3Prasad Netralaya, Udupi, Karnataka, 576101, India; 4Goa Medical College, Bambolim, Goa, 403202, India; 5St.John’s Medical College & Hospital, Bengaluru, Karnataka, 560034, India; 6ESI PGIMSR, ESI Medical College and Hospital, Kolkata, West Bengal, 700104, India; 7Srinivas Institute of Medical Sciences and Research Centre, Mangalore, Karnataka, 574146, India; 8All India Institute of Medical Sciences, Bhopal, Madhya Pradesh, 462020, India
Correspondence: Arvind Kumar Morya
Department of Ophthalmology, All India Institute of Medical Sciences, Bibinagar, Hyderabad, Telangana, 508126, India
Email [email protected]
Introduction: Deep Learning (DL) and Artificial Intelligence (AI) have become widespread due to the advanced technologies and availability of digital data. Supervised learning algorithms have shown human-level performance or even better and are better feature extractor-quantifier than unsupervised learning algorithms. To get huge dataset with good quality control, there is a need of an annotation tool with a customizable feature set. This paper evaluates the viability of having an in house annotation tool which works on a smartphone and can be used in a healthcare setting.
Methods: We developed a smartphone-based grading system to help researchers in grading multiple retinal fundi. The process consisted of designing the flow of user interface (UI) keeping in view feedback from experts. Quantitative and qualitative analysis of change in speed of a grader over time and feature usage statistics was done. The dataset size was approximately 16,000 images with adjudicated labels by a minimum of 2 doctors. Results for an AI model trained on the images graded using this tool and its validation over some public datasets were prepared.
Results: We created a DL model and analysed its performance for a binary referrable DR Classification task, whether a retinal image has Referrable DR or not. A total of 32 doctors used the tool for minimum of 20 images each. Data analytics suggested significant portability and flexibility of the tool. Grader variability for images was in favour of agreement on images annotated. Number of images used to assess agreement is 550. Mean of 75.9% was seen in agreement.
Conclusion: Our aim was to make Annotation of Medical imaging easier and to minimize time taken for annotations without quality degradation. The user feedback and feature usage statistics confirm our hypotheses of incorporation of brightness and contrast variations, green channels and zooming add-ons in correlation to certain disease types. Simulation of multiple review cycles and establishing quality control can boost the accuracy of AI models even further. Although our study aims at developing an annotation tool for diagnosing and classifying diabetic retinopathy fundus images but same concept can be used for fundus images of other ocular diseases as well as other streams of medical science such as radiology where image-based diagnostic applications are utilised.
Keywords: artificial intelligence, deep learning, referrable diabetic retinopathy
This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.Download Article [PDF] View Full Text [HTML][Machine readable]