Automated retinal imaging and trend analysis – a tool for health monitoring

Retinal imaging potential

Retinal imaging offers a unique perspective into the health of our eyes as well as that of our body. Ocular-specific diseases and life-threatening systemic diseases show manifestations in the eye, including age-related macular degeneration (AMD) and glaucoma, two of the leading causes of blindness, as well as diabetes, hypertension, and multiple sclerosis (MS). Additionally, an individual’s lifestyle and environmental factors are reflected in the retinal vasculature.^1,2

Disease detection is oftentimes about identifying changes in the structure of the blood vessels, timing the changes and locating them. Characteristic structural changes can be distinguished by examining different features (optic fundus signs): flame-shaped hemorrhages (nerve fiber layer), vitreous (cloudy) hemorrhages, preretinal (between vitreous membrane and retina) hemorrhages; venous beading (dilation of the vessel walls); hard exudates (lipid deposits) and soft exudates (microinfarction of retinal nerve fiber layer); microaneurysms (MAs); and drusen.³

Knowing when a feature occurred is key. For example, the MA population is dynamic and changes occur in a matter of months.^4,5 For diabetic retinopathy (DR), it has been established that MAs are the earliest lesions visible.⁶ Additionally, MA turnover rates are indicative of early-stage DR as well as the likelihood of DR progression to macular edema.⁷

Even more critical is the location of the features, ie, the particular site where blood vessel changes occur. Changes affecting the area of central vision (fovea/macula) have the most severe effects on vision, eg, edematous changes (swelling, fluid accumulation).⁸ Similar changes occurring in the peripheral retina often go unnoticed by the subject. Indeed, many vision disorders start at the periphery and later spread to the center of the retina.⁹ Therefore vision loss is often considered a “silent” disease, only noticed by patients at late stages. To be able to prevent disease progression, it is critical to detect such changes early on. In addition, retinal vessel patterns can provide further insights. Vessel patterns are unique due to the complex structure of the capillaries, but so far, they are largely understudied. Their diagnostic potential is discussed in a separate paragraph below.

Retinal imaging pipeline and limitations

The eye provides direct noninvasive access to the retina, the blood vessels supporting its function, the nerve fiber layer, and the optic nerve head. However, the imaging set-up required is technically demanding and precise alignment is critical, as light must enter and reach the back of the eye through a limited (pupil) opening. Pupil size varies with age, light/dark settings, and disease states, eg, the pupil diameter is smaller in diabetics and elderly. Increased eye movement in the latter and the existence of “unknown” and reflecting optical elements in the eye, such as the cornea and lens, present additional imaging challenges.¹⁰

The current equipment used is complex and needs a trained specialist to operate. The more accurate a diagnosis needed, the more sophisticated the devices used and the more expensive the examination becomes. The imaging pipeline starts with a handheld ophthalmoscope, which provides a basic view of the central area of the retina. Fundus photography is oftentimes the method of choice, especially for population-based large-scale detection (eg, DR, AMD, glaucoma), and gives a two-dimensional representation of the three-dimensional retinal structure in a digital output.¹¹ Confocal scanning laser imaging allows a more detailed look.¹² However, most imaging techniques facilitate a subjective examination based on the physician’s judgment. The only objective tool to visualize the retina is tomographic imaging via optical coherence tomography, and its introduction has indeed led to improved knowledge of macular disease via enabling quantitative analysis of the nerve fiber layer in glaucoma.^13–15 Current set-ups are further limiting due to the fact that a diagnosis is usually based on a “snapshot” in time (images taken during a single physician visit) or a few images captured years apart at best. Additionally, this limited dataset is kept in the silos of the individual clinics. These shortcomings are not specific to ophthalmology but common to diagnostic tools across different medical disciplines.

Future of retinal imaging care delivery system

The capacity of the medical system is not well equipped to provide diagnostics on a large scale. In addition to the challenges regarding image capture discussed above, current image analysis relies on manual interpretation by a skilled physician, oftentimes via a reading center, and is time consuming and expensive due to labor-intensive interpretation protocols and multiple images per patient. As telehealth efforts are expanding to address the exponentially growing number of people with vision disorders, cheaper and easier to use image capture devices are becoming available, resource demands are increasing, and the shortage of trained expert readers is getting further exacerbated. Computerized automated image analysis programs are an alternative to traditional human grading; however, computational approaches to solve complex image analysis are energy inefficient, difficult to implement; in large, lack regulatory approvals; and usually only work for a specific disease category.

Better infrastructure and new ways of processing images are needed for large-scale public health efforts. A comprehensive approach for large-scale fundus image interpretation, performed by statistical models based on deep convolutional neural networks that are trained using images graded by experts, could radically improve throughput of eye screenings, reduce the dependence on specialists, and provide accurate diagnosis across a wide range of diseases.¹⁶ The performance of these models is currently on par with that of trained experts, and is expected to improve over time with more data. It could provide an effective solution that can empower less trained employees in a point-of-care setting, make it suitable for low resource settings, amplify the impact of the individual physician to have a wider reach, and eventually empower the patient by providing actionable information, eg, advice on when they should consult a specialist for a follow-up examination or treatment.

Indicators: lagging, first, leading

Ocular disease is considered to be a silent disease, oftentimes asymptomatic early on and visual problems are usually only noticed at a late stage. Many systemic diseases with eye manifestations fall in this category of lagging indicators such as diabetes, coagulation disorders, sickle cell disease, syphilis, and toxoplasmosis. Sickle cell anemia causes ischemic vascular obstructions (decreased blood flow) by defective rigid red blood cells; obstructions and their consequences can be evaluated by looking at the retina.^17–19 Diagnosis of DR, a condition that affects one-third of diabetics and leads to legal blindness, considers altered features in the following areas listed according to priority: hemorrhage formation due to glucose damage to the vessel walls; changes in the macula (edema, hard exudates, color), the optic disc (edges, color), the vessels (shine, artery/vein ratio, aneurysm) and in the periphery (hard exudates, new vessels).²⁰ Analysis of the retinal vessels provides a more complete and accurate assessment of the diabetic pathophysiology than blood glucose level measurements at a clinic because the latter only provides a “snapshot” insight, whereas the accumulated past blood glucose level changes are manifested in the retinal vasculature.

Oftentimes, the eye is looked at as a monitoring tool if a disease has set in and is under active management. Instead, let us consider vascular changes in the eye as a first indicator of disease that may be stoppable by early treatment. For example, diabetes type I has a clear onset; however, diabetes type II goes unnoticed for a long time and screening is crucial to prevent macular edema and proliferative DR, the vision threatening aspect of DR. Indeed, 90% of vision loss, as estimated by the National Eye Institute, could be prevented if caught early on via detection of early indicators (such as MAs).

Similarly, signs of other vasculopathies, such as hypertension, collagen problems, and inflammatory diseases, can be detected early on. Distinct features in the retina point toward chronic hypertension, such as straightness of the vessels (vessels become more wiggly and curved), artery to vein ratio, reflection (color, shine), hemorrhages, exudates, macular edema, edema at the optic disc, and signs of pulling of blood vessels at vessel crossings.²¹ Ocular symptoms caused by rheumatoid arthritis can be helpful for diagnosis and monitoring of disease activity and adjustment of medication given to keep the autoimmune response under control.

Infectious diseases also show signs in the eye. In emerging markets, a diagnosis of AIDS is oftentimes made during an eye examination, although signs in the eye are not the first symptoms to arise.^22,23 Detection of a “cheese and ketchup” structure in the retina is a typical trigger for follow-up examinations. Lupus is an autoimmune disease, which can cause vascular inflammation (vasculitis) detectable with an eye examination.^24,25 Indeed, vasculitis is a reliable marker to determine the active stage of the disease.^26,27 Onchocerciasis (river blindness), caused by a filarial infection, can be difficult to diagnose, but has many ocular complications, inflammation being the most devastating one.^28,29

Changes in the retinal structure are also indicative for tumors. Blurred edges of the optic disc and fluid accumulation are oftentimes the first signs of brain tumors. Even for intracranial tumors with asymptomatic increase in intracranial pressure, swelling of the optic disc (papilledema) can oftentimes be observed.³⁰ Lung and breast tumors commonly metastasize to the choroid, which is the most vascularized layer of the eye.^31,32

Increasing research demonstrates that changes in the eye correlate with disease onset. Indeed, the retinal capillaries are among the smallest in our body and changes affecting the vascular system (circulation) are reflected in the smallest blood vessels first.³³ However, subtle early changes oftentimes go undetected, even with specialized high-end devices available nowadays. Ideally, diagnosis should occur at disease onset, which would allow for better disease management and prevention of complications. Considering that the optic nerve presents a direct link to the central nervous system, many neurological diseases affect the eye, eg, patients with MS also develop macular edema.³⁴ Early studies show that ocular manifestations of Parkinson’s and Alzheimer’s diseases can be detected in the eye as well.^35–38 Taking this a step further, (micro) vascular changes in the eye could be proxies for risk assessment of future cardiovascular and cerebrovascular events. A spectral signature of the ending of the nerve fiber layer could present a leading indicator for MS. However, more data are needed to get a better understanding of how such diseases manifest in the eye early on. To achieve this, a new generation of digital eye scans will be necessary to collect a more comprehensive dataset of retinal morphology and function.

Biomarker discovery based on trend analysis

Medical diagnostics today relies on a single data point (“snapshot”) for health assessment. This is especially true for medical imaging and specifically fundus imaging. Diagnosis is primarily based on one static picture taken at a specific point in time or at best a few pictures taken several years apart. The former is comparable to the understanding of a story one gains by looking at one frame (still picture) of a movie versus watching a full movie. The latter only allows for a straight line to be drawn between the (sparse) data points, comparable to the fact that nobody would bet on the stock market based on two data points.

A new class of user-friendly community-operated (retinal) imaging devices, such as the eyeSelfie concept, will enable frequent data capture and improved results.³⁹ Such data capture does not need to be “constant” such as data gathered by most “wearables”. “Interval-based” image acquisition can provide several data points taken frequently over a long period of time. This allows establishing a baseline and timeline (trend) and statistical outliers from the baseline can be eliminated (Figure 1). Deviation from the baseline can be an early disease indicator and interval-based fundus photography can provide the tool to study disease evolution. There are many key insights to be gained from such a dataset. We can detect how diseases change (oscillation), as well as the timeline and a pattern of change. This will allow a better understanding of the pathophysiology, earlier diagnosis, tracking of disease progress as well as prediction of future disease progress (eg, can we predict how quickly a person will go blind?).

Figure 1 The future of health diagnostics. Current diagnostics are based on a “snapshot” in time and limited data points. In the future, large datasets acquired over time through constant monitoring will be analyzed to establish baselines and trends, enabling preventative interventions. Part of this figure reuses a drawing previously published in Swedish et al, 2015. Copyright © 2017 ACM, Inc. Adapted from Swedish T, Roesch K, Lee IK, Rastogi K, Bernstein S, Raskar R. EyeSelfie: Self directed eye alignment using reciprocal eye box imaging. ACM Trans Graph. 2015;34(4):58.39

Studying disease evolution based on fundus images will allow discovery of novel biomarkers of disease based on vessel morphology and/or pattern changes. A simplistic view of biomarkers is that they are indicators of disease trait (risk factor or risk marker) and disease state or rate. Antecedent biomarkers indicate the risk of developing a disease and prognostic markers predict the future disease course. Other markers are distinguished as screening, diagnostic, and staging. A better understanding of disease progress can lead to discovery of novel prognostic markers. Prognosis will be based on the correlation of pattern changes and the velocity of such changes. However, it is not just about correlation, but is also about causality.

Trends detected on plotted graphs of ganglion cell loss can predict glaucoma.^40,41 In the case of diabetes, microvascular damage underlies both retinopathy as well as kidney failure; therefore, the microvasculature could be used as a prognostic marker.⁴² In a similar way, novel markers tracked might be able to predict stroke.^43,44

Identification of novel biomarkers and trends will allow for better health management. Screening biomarkers can identify vulnerable patients. Treatments (eg, dosage of drugs taken) can be adjusted in an informed way and can be followed up more frequently. Diseases can be detected earlier (even if asymptomatic), making treatment easier and more effective. Frequent monitoring is also helpful in determining follow-up care, as the precise time intervals are critical. For example, it is important to define when hemorrhages appear.

Novel information and increased data change the way a physician thinks and performs treatment, eg, the standard triglyceride levels get adjusted (go down) every so often as guidelines for standard of care change.⁴⁵ Additionally, visualization of the health information that lies within us will support individuals’ understanding of their health, provide them with more control of their health, and empower them with a tool for proactive wellness maintenance. When it comes to disease, patient involvement in their treatment regime is critical for a successful outcome.

Finally, novel user-friendly high-throughput instrumentation and data gathering will be applicable to large populations (Figure 1). It will not only allow collecting patterns for one individual, but also allow comparing patterns across subpopulations. Longitudinal and cross-sectional analysis across different ethnicities and geographical areas can be performed. Creation of a large dataset is the cornerstone of building useful statistics and machine learning tools. Big data analytics will not just allow for comparison to an invariable baseline (eg, gender, age) but to a varying multimetric baseline (eg, glucose levels, genotype).

This approach is also very interesting to the drug development process during preclinical and clinical validation. Indeed, there are many drugs that present side effects in the eye as well as many systemic diseases that manifest in the eye. Current trials depend on a clinical setting. More user-friendly devices will allow for the testing to occur outside the clinic environment, independent of long waitlists for appointments at a specialist office. Baselines will allow for establishment of trends and faster detection of changes induced by the drug. Combined, this will simplify and shorten the timelines for drug trials dramatically and will allow for faster iterative improvements. Additionally, it will allow for identification of novel biomarkers that can potentially be integrated into a companion diagnostic tool kit.

Acknowledgments

The authors thank Margaret Church, Paulo Schor, and Thiago Martins for discussions and clinical feedback, and Laura Piraino for illustrations.

Disclosure

The authors report no conflicts of interest in this work.

---

References

1.		Sun C, Wang JJ, Mackey DA, Wong TY. Retinal vascular caliber: systemic, environmental, and genetic associations. Surv Ophthalmol. 2009;54(1):74–95.
2.		Serre K, Sasongko MB. Modifiable lifestyle and environmental risk factors affecting the retinal microcirculation. Microcirculation. 2012;19(1):29–36.
3.		Abramoff MD, Garvin MK, Sonka M. Retinal imaging and image analysis. IEEE Rev Biomed Eng. 2010;3:169–208.
4.		Hellstedt T, Immonen I. Disappearance and formation rates of microaneurysms in early diabetic retinopathy. Br J Ophthalmol. 1996;80(2):135–139.
5.		Klein R, Meuer SM, Moss SE, Klein BE. Retinal microaneurysm counts and 10-year progression of diabetic retinopathy. Arch Ophthalmol. 1995;113(11):1386–1391.
6.		Feman SS. The natural history of the first clinically visible features of diabetic retinopathy. Trans Am Ophthalmol Soc. 1994;92:745–773.
7.		Ribeiro ML, Nunes SG, Cunha-Vaz JG. Microaneurysm turnover at the macula predicts risk of development of clinically significant macular edema in persons with mild nonproliferative diabetic retinopathy. Diabetes Care. 2013;36(5):1254–1259.
8.		Ciulla TA, Amador AG, Zinman B. Diabetic retinopathy and diabetic macular edema: pathophysiology, screening and novel therapies. Diabetes Care. 2003;26(9):2653–2664.
9.		Hartong DT, Berson EL, Dryja TP. Retinitis pigmentosa. Lancet. 2006;368(9549):1795–1809.
10.		Negi A, Vernon SA. An overview of the eye in diabetes. J R Soc Med. 2003;96(6):266–272.
11.		Gullstrand A. Neue Methoden der reflexlosen Ophthalmoskopie. [New Methods for reflex-free Ophthalmoscopy]. Berichte Deutsche Ophthalmologische Gesellschaft. 1910;36. German.
12.		Webb RH, Hughes GW. Scanning laser opthalmoscope. IEEE Trans Biomed Eng. 1981;28(7):488–492.
13.		Huang D, Swanson EA, Lin CP, et al. Optical coherence tomography. Science. 1991;254(5035):1178–1181.
14.		Chen TC, Cense B, Pierce MC, et al. Spectral domain optical coherence tomography: ultrahigh-speed, ultrahigh-resolution ophthalmic imaging. Arch Ophthalmol. 2005;123(12):1715–1720.
15.		Van Velthoven ME, Faber DJ, Verbraak FD, van Leeuwen TG, de Smet MD. Recent developments in optical coherence tomography for imaging the retina. Prog Retin Eye Res. 2007;26(1):57–77.
16.		Kaggle.com [homepage on the Internet]. Kaggle Inc; c2016. Available from: https://www.kaggle.com/c/diabetic-retinopathy-detection. Accessed 16 August 2016.
17.		Jackson H, Bentley CR, Hingorani M, Atkinson P, Aclimandos WA, Thompson GM. Sickle retinopathy in patients with sickle trait. Eye (Lond). 1995;9(Pt 5):589–593.
18.		Akinsola FB, Kehinde MO. Ocular findings in sickle cell disease patients in Lagos. Niger Postgrad Med J. 2004;11(3):203–206.
19.		Lima CS, Rocha EM, Silva NM, Sonatti MF, Costa FF, Saad ST. Risk factors for conjunctival and retinal vessel alterations in sickle cell disease. Acta Ophthalmol Scand. 2006;84(2):234–241.
20.		Fong DS, Aiello LP, Ferris FL 3rd, Klein R. Diabetic retinopathy. Diabetes Care. 2004;27(10):2540–2553.
21.		Wong TY, McIntosh R. Systemic associations of retinal microvascular signs: a review of recent population-based studies. Ophthalmic Physiol Opt. 2005;25(3):195–204.
22.		Cunningham ET Jr, Margolis TP. Ocular manifestations of HIV infection. N Engl J Med. 1998;339(4):236–244.
23.		Holland GN. AIDS and ophthalmology: the first quarter century. Am J Ophthalmol. 2008;145(3):397–408.
24.		Isenberg DA, Rahman A, Allen E, et al. Development and initial validation of an updated version of the British Isles Lupus Assessment Group’s disease activity index for patients with systemic lupus erythematosus. Rheumatology (Oxford). 2005;44(7):902–906.
25.		Sivaraj RR, Durrani OM, Denniston AK, Murray PI, Gordon C. Ocular manifestations of systemic lupus erythematosus. Rheumatology. 2007;46(12):1757–1762.
26.		Nguyen QD, Foster CS. Systemic lupus erythematosus and the eye. Int Ophthalmol Clin. 1998;38(1):33–60.
27.		Peponis V, Kyttaris VC, Tyradellis C, Vergados I, Sitaras NM. Ocular manifestations of systemic lupus erythematosus: a clinical review. Lupus. 2006;15(1):3–12.
28.		Taylor HR. Onchocerciasis. Int Ophthalmol. 1990;14(3):189–194.
29.		Berger IB, Nnadozie J. Onchocerciasis and other eye problems in developing countries: a challenge for optometrist. J Am Optom Assoc. 1993;64(10):699–702.
30.		Whiting AS, Johnson LN. Papilledema: clinical clues and differential diagnosis. Am Fam Physician. 1992;45(3):1125–1134.
31.		Mewis L, Young SE. Breast carcinoma metastatic to the choroid. Ophthalmology. 1982;89(2):147–151.
32.		Kreusel KM, Wiegel T, Stange M, Bornfeld N, Hinkelbein W, Foerster MH. Choroidal metastasis in disseminated lung cancer: frequency and risk factors. Am J Ophthalmol. 2002;134(3):445–447.
33.		Liew G, Wang JJ, Mitchell P, Wong TY. Retinal vascular imaging: a new tool in microvascular disease research. Circ Cardiovasc Imaging. 2008;1(2):156–161.
34.		Saidha S, Sotirchos ES, Ibrahim MA, et al. Microcystic macular oedema, thickness of the inner nuclear layer of the retina, and disease characteristics in multiple sclerosis: a retrospective study. Lancet Neurol. 2012;11(11):963–972.
35.		Archibald NK, Clarke MP, Mosimann UP, Burn DJ. The retina in Parkinson’s disease. Brain. 2009;132(Pt 5):1128–1145.
36.		Lee JY, Ahn J, Kim TW, Jeon BS. Optical coherence tomography in Parkinson’s disease: is the retina a biomarker? J Parkinson’s Dis. 2014;4(2):197–204.
37.		Berisha F, Feke GT, Tempe CL, McMeel JW, Schepens CL. Retinal abnormalities in early Alzheimer’s disease. Invest Ophthalmol Vis Sci. 2007;48(5):2285–2289.
38.		Frost S, Kanagasingam Y, Sohrabi H, et al. Retinal vascular biomarkers for early detection and monitoring of Alzheimer’s disease. Transl Psychiatry. 2013;3:e233.
39.		Swedish T, Roesch K, Lee IK, Rastogi K, Bernstein S, Raskar R. EyeSelfie: Self directed eye alignment using reciprocal eye box imaging. ACM Trans Graph. 2015;34(4):58.
40.		Meira-Freitas D, Lisboa R, Tatham A, et al. Predicting progression in glaucoma suspects with longitudinal estimates of retinal ganglion cell counts. Invest Ophthalmol Vis Sci. 2013;54(6):4174–4183.
41.		Quigley HA. Glaucoma. Lancet. 2011;377(9774):1367–1377.
42.		Fowler MJ. Microvascular and macrovascular complications of diabetes. Clin Diabetes. 2008;26(2):77–82.
43.		Lindley RI, Wang JJ, Wong MC, et al. Retinal microvasculature in acute lacunar stroke: a cross-sectional study. Lancet Neurol. 2009;8(7):628–634.
44.		Baker ML, Hand PJ, Liew G, et al; Multi-Centre Retinal Stroke Study Group. Retinal microvascular signs may provide clues to the underlying vasculopathy in patients with deep intracerebral hemorrhage. Stroke. 2010;41(4):618–623.
45.		Miller M, Stone NJ, Ballantyne C, et al. Triglycerides and cardiovascular disease. Circulation. 2011;123(20):2292–2333.