Practical Applications of Artificial Intelligence Diagnostic Systems in Fundus Retinal Disease Screening

Introduction

As the global population ages and the incidence of chronic diseases rises, retinal diseases have emerged as a significant public health concern.¹ Among these conditions, proliferative diabetic retinopathy (PDR) and diabetic macular edema (DME) represent prevalent retinal vascular disorders, with their prevalence steadily increasing.^2,3 Early screening and timely intervention have the potential to prevent the ocular damage associated with these diseases.^4,5 However, the uneven distribution of healthcare resources and a notable shortage of ophthalmologists render large-scale manual screenings impractical.⁶ This context has spurred a growing interest in developing reliable and cost-effective screening methodologies for retinal diseases.

The rapid advancements in artificial intelligence (AI) technology, particularly deep learning-based diagnostic systems, have unveiled substantial potential within the medical field.^7–10 These AI systems have been extensively applied in ophthalmic imaging, particularly for the detection of diabetic retinopathy (DR), glaucoma, age-related macular degeneration, and various retinal lesions.^11–15 The integration of AI and deep learning technologies has facilitated routine large-scale DR screenings and screenings in underserved areas.^4,16,17

In this study, the AI technology utilized has undergone prospective validation in a national cohort encompassing over 200,000 individuals.¹⁸ Our artificial intelligence is based on Comprehensive AI Retinal Expert (CARE), a single Convolutional Neural Network (CNN) model that is computationally efficient, offering lower computational costs compared to ensembles of binary Deep Learning Systems (DLSs) for individual abnormality detections. Leveraging this robust foundation, we aim to systematically analyze the application of AI technology in screening for retinal diseases, comparing its diagnostic outcomes with those of expert practitioners. The results of this research will contribute to an efficient, cost-effective, and broadly accessible solution for managing eye diseases among patients in our community.

Materials and Methods

Study Subjects

This study comprised 3076 participants recruited between March 1, 2024, and August 31, 2024. All participants were evaluated using a system based on the previously reported Comprehensive Artificial Intelligence Retinal Expert (CARE) system,¹⁸ and clinical data—including age, gender, history of chronic diseases, and retinal symptoms—were systematically recorded. The exclusion criteria included: (1) patients with corneal diseases, cataracts, vitreous opacities, or other conditions that could compromise the quality of color fundus images; (2) individuals with mental disorders or pregnant women who were unable to cooperate.

All research protocols adhered to the principles outlined in the Declaration of Helsinki. All participants were included in the study after providing written informed consent, and those under the age of 18 were included only after obtaining written informed consent from their parents or legal guardians. This study received ethical approval from the Research Ethics Committee of Tong Ren Hospital, Shanghai Jiao Tong University School of Medicine (Approval Number: K2024-033-01).

Fundus Image Acquisition and Diagnostic Standards

Each participant underwent bilateral fundus examinations conducted using a fully automated non-mydriatic fundus camera (Airdoc, AI-FD16aF) and the images generated by the portable camera are 40-degree field images. The acquired color fundus images were comprehensively assessed in a double-blind manner by two associate chief ophthalmologists specializing in retinal diseases. When their diagnoses were consistent, a final diagnosis was established directly. In cases of diagnostic disagreement, a third chief ophthalmologist specializing in retinal diseases was consulted for a collaborative diagnosis, resulting in a definitive conclusion. The prevalence of fundus diseases was subsequently calculated. Additionally, the diagnostic results from the AI reading system were evaluated for consistency with expert diagnoses, while also assessing the screening performance for fundus diseases and analyzing the performance metrics.

Statistical Analysis

Statistical analyses were conducted utilizing SPSS version 25.0 and R version 3.6.1. Categorical variables were represented as counts and percentages, while continuous variables were reported as means and standard deviations. The diagnostic performance of the screening tests was assessed in terms of sensitivity, specificity, positive predictive value, and negative predictive value. Prevalence rates were determined using point estimates along with 95% confidence intervals, providing a robust evaluation of the diagnostic efficacy.

Results

Demographic and Clinical Characteristics

This study encompassed 3,076 participants, leading to the collection of 6,152 retinal fundus images. Among the included patients, 1,495 (48.7%) reported no comorbid systemic conditions; 742 (24.1%) had diabetes mellitus, 181 had Type 1 diabetes, and 561 had Type 2 diabetes; 586 (19.1%) had hypertension; 191 (6.2%) had heart disease; and 62 (2.0%) suffered from other systemic ailments. The age distribution of the participants ranged from 6 to 95 years, with 2,890 males (49.6%) and 1,960 females (50.4%) (Table 1).

Table 1 Clinical Characteristics of the Patients

Evaluation of Fundus Disease Screening Program

Among all the screened subjects included in this study, and after rigorous evaluation by senior ophthalmologists, the statistics for the positive detection of various fundus diseases are as follows: The positive detection rate for DR is 2.2% (136/6152), for AMD it is 3.9% (238/6152), for possible glaucoma it stands at 4.8% (296/6152), for pathological myopia it is 2.1% (130/ 6152), for retinal vein occlusion it is 0.6% (38/6152), for retinal detachment it is 0.7% (44/6152), for macular hole it is 0.3% (18/6152), for epiretinal macular membrane it is 1% (62/6152), for hypertensive retinopathy it reaches 3.3% (202/6152), for myelinated fibers it is 0.4% (24/6152), and finally, for retinitis pigmentosa, the positive detection rate is negligible at 0% (4/ 6152) (Table 2).

Table 2 Overview of Fundus Disease Screening

AI-Assisted Screening and Evaluation of Ocular Fundus Diseases

The AI screening and evaluation system achieved a sensitivity of 94.12% (95% CI 88.36~97.24) and a specificity of 98.60% (95% CI 98.27~98.88) in diabetic retinopathy (DR) compared to expert diagnoses. Consistency testing yielded a Kappa coefficient of 0.724 (P<0.001), indicating a high level of agreement. The ROC curve was further plotted (Figure 1A), with an AUC of 0.963. For age-related macular degeneration (AMD), the system showed a sensitivity of 89.50% (95% CI 84.71~92.96) and a specificity of 98.33% (95% CI 97.96~98.63) compared to expert diagnoses. Consistency testing yielded a Kappa coefficient of 0.761 (P<0.001), indicating a high level of agreement. The ROC curve was further plotted (Figure 1B), with an AUC of 0.939. In suspected glaucoma, the system showed a sensitivity of 91.55% (95% CI 87.64~94.35) and a specificity of 97.40% (95% CI 96.96~97.79) compared to expert diagnoses. Consistency testing yielded a Kappa coefficient of 0.738 (P<0.001), indicating a high level of agreement. The ROC curve was further plotted (Figure 1C), with an AUC of 0.945. For pathological myopia, the system showed a sensitivity of 90.77% (95% CI 84.10~94.93) and a specificity of 99.10% (95% CI 98.82~99.32) compared to expert diagnoses. Consistency testing yielded a Kappa coefficient of 0.773 (P<0.001), indicating a high level of agreement. The ROC curve was further plotted (Figure 1D), with an AUC of 0.949. In retinal vein occlusion, the system showed a sensitivity of 81.58% (95% CI 65.11~91.68) and a specificity of 99.49% (95% CI 99.27~99.65) compared to expert diagnoses. Consistency testing yielded a Kappa coefficient of 0.611 (P<0.001), indicating a high level of agreement. The ROC curve was further plotted (Figure 1E), with an AUC of 0.905. For retinal detachment, the system showed a sensitivity of 88.64% (95% CI 74.65~95.74) and a specificity of 99.18% (95% CI 98.91~99.39) compared to expert diagnoses. Consistency testing yielded a Kappa coefficient of 0.578 (P<0.001), indicating moderate agreement. The ROC curve was further plotted (Figure 1F), with an AUC of 0.939. In macular hole, the system showed a sensitivity of 83.33% (95% CI 57.74~95.59) and a specificity of 99.80% (95% CI 99.65~99.89) compared to expert diagnoses. Consistency testing yielded a Kappa coefficient of 0.769 (P<0.001), indicating a high level of agreement. The ROC curve was further plotted (Figure 1G), with an AUC of 0.916. For epiretinal membrane, the system showed a sensitivity of 82.26% (95% CI 70.05~90.40) and a specificity of 99.23% (95% CI 98.97~99.43) compared to expert diagnoses. Consistency testing yielded a Kappa coefficient of 0.629 (P<0.001), indicating a high level of agreement. The ROC curve was further plotted (Figure 1H), with an AUC of 0.907. For hypertensive retinopathy, the system showed a sensitivity of 94.55% (95% CI 90.21~97.11) and a specificity of 97.82% (95% CI 97.40~98.16) compared to expert diagnoses. Consistency testing yielded a Kappa coefficient of 0.718 (P<0.001), indicating a high level of agreement. The ROC curve was further plotted (Figure 1I), with an AUC of 0.962. In myelinated fibers, the system showed a sensitivity of 83.33% (95% CI 61.81~94.52) and a specificity of 99.74% (95% CI 99.56~99.85) compared to expert diagnoses. Consistency testing yielded a Kappa coefficient of 0.700 (P<0.001), indicating a high level of agreement. The ROC curve was further plotted (Figure 1J), with an AUC of 0.916. For retinitis pigmentosa, the system showed a sensitivity of 75.00% (95% CI 21.94~98.68) and a specificity of 99.95% (95% CI 99.84~99.99) compared to expert diagnoses. Consistency testing yielded a Kappa coefficient of 0.600 (P<0.001), indicating moderate agreement. The ROC curve was further plotted (Figure 1K), with an AUC of 0.875.(Table 3) Figure 2 demonstrated normal fundus photographs, abnormal fundus photographs, and the analysis results from the AI software. Figure 2A illustrated the AI software’s analysis results for a normal fundus photograph, while Figures 2B and C showed the DR and ERM fundus photographs, respectively, along with the AI software’s analysis results. Figure 2D presented the analysis of a misclassified image: the patient’s left eye with AMD was only identified as having macular abnormalities.

Table 3 ROC Curves for AI Diagnosis of Retinal Diseases

Figure 1 Receiver Operating Characteristic (ROC) curves for AI diagnosis of retinal diseases. (A) Diabetic retinopathy (DR); (B) Age-related macular degeneration (AMD); (C) Possible glaucoma; (D) Pathological myopia; (E) Retinal vein occlusion; (F) Retinal detachment; (G) Macular hole; (H) Epiretinal macular membrane; (I) Hypertensive retinopathy; (J) Myelinated fibers; (K) Retinitis pigmentosa.

Figure 2 Examples of normal fundus photographs and abnormal fundus photographs with the analysis results from the AI software. (A) Analysis of a normal fundus photograph; (B) Analysis of a diabetic retinopathy (DR) fundus photograph; (C) Analysis of an epiretinal membrane (ERM) fundus photograph; (D) Analysis of a misclassified case: the patient’s left eye with age-related macular degeneration (AMD) was only identified as having macular abnormalities.

Discussion

The advent of multi-disease AI screening for fundus pathology marks a significant advancement over traditional single-disease screening approaches, such as those targeting DR.^11,13,19,20 This innovative screening model simultaneously addresses multiple blinding eye conditions, which has not yet been realized in commercially available medical device software, either domestically or internationally. Prior investigations have conducted multi-center validations of artificial intelligence systems for detecting a range of fundus diseases or confirming normal fundus status. For instance, Burlina et al established a deep learning model utilizing a large dataset to detect 12 types of retinal anomalies, achieving diagnostic accuracies comparable to those of clinical experts, with average AUC values surpassing 95%. However, this research was limited by the model’s focus on identifying abnormalities rather than specific retinal diseases, rendering it unsuitable for direct screening applications.²¹ In a more recent study, Lin et al developed a robust deep learning-based system capable of identifying 14 retinal diseases, exceeding the 10 diseases evaluated in our current research.¹⁸ The study utilizes the AI fundus camera for multi-disease screening within a real-world clinical setting, contributing to the validation of multi-disease AI technology in relation to the observed disease distribution in our sample population.

This study demonstrated the AI’s diagnostic performance characteristics across 11 pathologies, showcasing sensitivities ranging from 75.00% to 94.55%, specificities from 97.40% to 99.74%, Kappa coefficients of at least 0.600 (P<0.001), and AUC values of 0.905 or more. These results indicate that AI has the potential to deliver autonomous and automated feedback within healthcare settings and to generate referral recommendations, thus addressing disparities in resource allocation and enhancing the diagnostic capabilities of primary care institutions across various regions. Moreover, the integration of AI technology with the clinical assessments performed by ophthalmologists yielded similar diagnostic accuracy to those obtained through examinations conducted solely by ophthalmologists, while concomitantly significantly reducing examination duration. The CARE system implemented has exhibited robust diagnostic capabilities in multi-disease classification tasks.²² However, it’s recognized that increasing the number of disease categories can lead to a decrease in diagnostic accuracy. To mitigate this, we need to employ advanced techniques such as data augmentation to enhance sample diversity, integrate multi-modal data fusion to leverage complementary information from different data sources, and implement hierarchical classification strategies to systematically categorize diseases at different levels of granularity. These approaches are anticipated to improve the precision of retinal abnormality screening in a multi-disease context, thereby enhancing clinical outcomes.

Nevertheless, this study is not without limitations. Firstly, it is a retrospective, single-center investigation. Secondly, the AI’s diagnostic capabilities are based on analysis of a single fundus color photograph, necessitating future studies to incorporate patient medical history and other multimodal diagnostic modalities for comprehensive diagnosis. Furthermore, the non-mydriatic camera, which provides only a 40-degree field of view, may compromise the diagnostic accuracy for conditions such as RP or partial RD, where the pathology is primarily localized to the peripheral retina while the macula remains unaffected. Lastly, the sample sizes for certain conditions were limited. Consequently, future research should aim to aggregate a larger volume of ophthalmic examination results. By harnessing advanced big data techniques, this endeavor will facilitate the development of a comprehensive deep learning-based fundus disease screening and diagnostic system, continually enhancing the AI diagnostic framework to optimize the efficiency and intelligence of healthcare services.

Conclusion

In conclusion, the AI screening system exhibits elevated sensitivity and specificity for the majority of retinal diseases (with the exception of certain peripheral retinal manifestations), establishing its suitability for screening applications in ophthalmology. This system is particularly beneficial for primary and community healthcare facilities in facilitating the initial diagnosis of fundus conditions, thereby reinforcing the implementation of a tiered ophthalmic care model. Such advancements not only enhance diagnostic precision but also optimize the allocation of healthcare resources, ultimately promoting improved patient outcomes and systemic efficiency.