Evaluation of a smartphone photoscreening app to detect refractive amblyopia risk factors in children aged 1–6 years

Introduction

Amblyopia is the leading cause of monocular visual impairment for children in the US, a leading cause of childhood vision impairment worldwide,^1–5 and affects about 1%–4% of preschool-aged children in the US.⁶ Left untreated, amblyopia can result in permanent vision loss,⁷ but can be effectively treated if detected early.^8,9 Some have shown that photoscreening in children under the age of 3 years can be a reliable method to detect amblyogenic risk factors.¹⁰ Various medical organizations recommend vision screening, but actual screening rates remain low as a result of perceived and real barriers, including parental perceptions that the procedure is too time-consuming or that children are uncooperative.^11,12

Instrument-based photoscreening has been shown to identify risk factors that are likely to lead to amblyopia or poor vision.¹³ Automated photoscreening may be a means to evaluate an even greater number of children without placing an undue burden on the healthcare system or those charged with pediatric care.

The American Academy of Pediatrics (AAP) and the United States Preventative Services Task Force (USPSTF) recommend photoscreening for children under the age of 5 years, but differ on the preferred age to initiate screening, with the former suggesting 1 year and the latter suggesting 3 years.^7,14 The American Association for Pediatric Ophthalmology & Strabismus (AAPOS) recommends beginning instrument-based testing in children between the ages of 1 and 3 years, and places an additional emphasis on higher specificity in younger children and higher sensitivity in older children.⁹ Published recommendations concur that screening techniques should have high sensitivity and specificity.^15–17

This study evaluated the GoCheckKids (GCK, Gobiquity Mobile Health, Scottsdale, AZ, USA) smartphone photoscreening app’s ability to accurately identify children at risk for amblyopia by comparing smartphone photorefraction measurements from four smartphone platforms. One is the Nokia Lumia 1020 (Nokia, Espoo Finland), which has an actual flash. Three others are recent iOS iPhone models 6, 6s, and 7 (Apple, Cupertino, CA, USA) all of which utilize an LED “flash.” Images obtained from these are compared to standard cycloplegic refraction. (Since the time of this study, Gobiquity has chosen to commercialize only the iPhone 7 app due to rapidly changing technologies; as such the remainder of this paper will concentrate only on details and results from the iPhone 7 arm of the study).

Participants and methods

Participants

This single visit study was conducted at four private pediatric ophthalmology practices located in Alaska, Arizona, California, and Pennsylvania. Children aged 1–6 years old were enrolled in the iPhone 7 arm of a multi device study. We recruited one group aged 1–3 years as well as those 3–6 years to parallel USPSTF age classification. Written informed consent was given by a parent or legal guardian. The study was approved by the Salus Institutional Review Board (Austin, TX). This study was registered at ClinicalTrials.gov (NCT03447171); as mentioned above, this is a subset analysis from that study.

Exclusion criteria included any history of ocular surgery (strabismus, congenital cataract, or congenital glaucoma surgery) that could potentially affect photorefraction and/or any condition that would preclude the ability of the investigator to obtain a reliable photo image/measurement (eg, cataracts, corneal opacity) or cycloplegic refraction measurement.

All children who met the enrollment criteria were screened using the GoCheckKids app on the iPhone 7 by an ophthalmic technician. The app provides a template to optimize alignment and focal distance for the child, and then generates a flash image similar to conventional iPhone but with the mandatory red-eye reduction pre-flash overridden. The technician reviewed the image to ensure the child was looking directly at the camera before accepting the image. Images were taken prior to the installation of any cycloplegic drops. Following photoscreening, all children underwent a complete eye exam with cycloplegic refraction.

There was no anticipated early withdrawal, as this was a single-visit study with no treatment regimen or follow-up required. Both the technicians securing the images and those evaluating them were masked to gold standard clinical data.

GoCheckKids Mobile Smartphone app

GCK’s Mobile Smartphone photoscreening app is a U.S. Food and Drug Administration-cleared Class IIe medical device consisting of the native camera with flash configured for eccentric photorefraction and a software app to capture and process images. It is intended for vision screening in children from the ages of 6 months to 6 years. This study employed an Apple iPhone 7 in portrait mode (flash-horizontal and to the right of the lens when viewed by the patient).

Image quality assessment

Following image acquisition by the ophthalmic technician, images were processed using the GCK app’s proprietary Image Processing Algorithm (IPA), which identified and created estimates of eye metrics (such as the first Purkinje image location, pupil diameter, crescent width and limbus diameter) and calculated a photorefraction value. These values were compared to the apps referral criteria and the results were immediately displayed as “risk factors identified,” “no risk factors identified,” or “not gradable.” If a not gradable result is displayed the user is encouraged to “retake” the image, or schedule a rescreening appointment.

All images were later reviewed manually, identifying the same eye metrics using calipers, by trained ophthalmic imaging specialists. All images were saved on a remote server that was Health Insurance Portability and Accountability Act (HIPPA)-compliant. Automated results and manual results were compared and are reported in the analysis.

Statistical analysis

Referral criteria for amblyopic risk factors identified after the mobile app use was established from photorefractive measurement thresholds for hyperopia, myopia, and anisometropia. The gold standard referral criteria for cycloplegic refraction is established by the 2013 AAPOS guidelines.⁹ Comparison of the two methods yielded sensitivity, specificity, Positive Predictive Value, and Negative Predictive Value. This study was designed for two-tailed 95% CI, and maximum CI width of 10%.

The study was conducted in accordance with the protocol, Good Clinical Practice Standards, and applicable regulatory requirements, particularly as they pertain to children.

Results

A total of 287 patients were enrolled for evaluation. The majority of patients were Caucasian (52.6%), followed by Hispanic (16%) and Asian (13.2%). There were slightly more female patients than male patients (50.5% vs 49.5%, respectively). Additional patient demographics are included in Table 1.

Table 1 Patient demographics

Image quality assessment

There were 32/287 rejected images from the manual grading group (11.1%), and 37/287 rejected images in the auto grading group (12.9%). In the auto grading group, the leading reason to reject an image was poor gaze (lack of fixation, not strabismus), while in the manual group, the leading reason to reject an image was that the image was considered blurry (Table 2). As stated, if an image was considered “not gradable,” the user was encouraged to “retake,” or schedule a re-screening.

Table 2 Reasons for rejecting captured images

The manual grading results for the iPhone 7 had a sensitivity and specificity of 76% and 85% (95% CI 71%–81% and 80%–90%), respectively. The automated grading results on the iPhone 7 had a sensitivity and specificity of 65% and 83% (95% CI 62%–68% and 80%–86%), respectively. The prescreening probability for our patients was 38% (see Table 3).

Table 3 Combined sensitivity and specificity

False positives and false negatives

For the manual and automated grading groups, there were 24 and 27 false positives respectively. False negatives were 23 and 33 respectively for manual and automated grading with 12 and 16 respectively due to astigmatism and 11 and 17 due to accommodated hyperopia.

All single-illuminator eccentric photorefractors have a “null zone” that exists between low hyperopia and low myopia refractive values. This null zone is tuned by the manufacturer for the application, based on distance to the subject.^18–20 An accommodating eye will focus (or partially accommodate) at the plane of the camera, the reflected light returns close to the flash source, and no crescent appears in the image of the subjects’ pupil.²¹ When this phenomenon occurs in an eye with hyperopia exceeding the 2013 AAPOS referral criteria, the photorefractor produces measurements below its risk threshold and no referral recommendation is generated. In this study, those images were classified as a false negative.

For the purposes of this study, a false positive occurred when the photorefractor produced measurements above its risk threshold but the child’s cycloplegic refraction was below or exactly at the 2013 AAPOS referral guideline. In this study, we evaluated two separate categories: astigmatism and refractive crescent measurements.

The majority of individual false positives occurred when the same type of refractive error was detected by both photorefraction and cycloplegic refraction, but the severity was above threshold for photoscreening, but below threshold for cycloplegic refraction. In these cases the app overestimated the refractive value. In the manual grading group, an additional two false positives were attributed to extraneous reflections produced by exam room lighting (artifact), and one instance of partial anterior embryotoxon in one eye at 12–3 o’clock. In the auto grading group, there was one instance each of esotropia and blurry image.

There were no reported adverse events.

Discussion

This study evaluated the ability of a smartphone photoscreening app to detect amblyopic risk factors in children aged 1–6 years. The gold standard for detecting refractive errors in children is cycloplegic refraction,²² performed by trained eye care specialists, which requires instilling medication and cooperation on the part of the child.^22,23 Insufficient access and numbers of trained eye care professionals, as well as costs, limit the practicality of exclusively relying on comprehensive eye exams to conduct mass vision screenings in young children.^22,23 In today’s society, smartphones are ubiquitous and relatively inexpensive devices that may help clinicians and other allied healthcare professionals minimize current barriers to screening vision and detection of amblyopia. In this study, the GCK smartphone app had NPV of 80% or greater, performing well in its ability to recognize patients that did not require comprehensive exam by an eye care professional.

The GCK app provides immediate, onsite automated grading and also centralized, subsequent expert manual grading. Different Current Procedural Terminology (CPT) codes can therefore be employed – 99174 for manual grading and 99177 for automated grading. We had image quality issues with each grading method, but better sensitivity with manual grading. The pediatrician considering instrument–based amblyopia screening should balance these issues when selecting which grading paradigm to employ.

A systematic literature review was unable to find a screening test other than instrument-based objective screening that was consistently associated with both high sensitivity and specificity.²⁴ Retinal Birefringence Screening – a highly specialized form of “photo-screening” – has high validation for amblyopia and strabismus.²⁵ Other photoscreening devices have specificity and sensitivity ranges comparable to the GCK.^{15–17,26,27} In this study, the GCK app had a sensitivity of 76% and a specificity of 85%. Peterseim et al²⁸ found a sensitivity of 76% and a specificity of 67% in detecting amblyopia risk factors with the GCK app and a modified Nokia Lumia 1020 cellphone. Differences in how subjects were managed between the two studies, and inclusion of strabismus detection, might account for the differences in specificity. In our study, any image considered “non gradable” was excluded from statistical analysis with a recommendation to re-screen or reschedule the child. In the Peterseim et al study, however, any image considered “not gradable” was included in the analysis as a vision fail. An additional relevant difference between the two studies was the platform on which the app was being evaluated. This study used the Apple iPhone 7, whereas Peterseim et al used a Nokia smartphone.

The AAP has recommended instrument-based vision screening in infants as young as 1 year, with appropriate action to follow, if positive.²⁹ If not identified early, treatments for amblyopia may be ineffective.³⁰ Estimated costs to screen and treat children diagnosed with amblyopia are approximately $1.24 billion annually (presuming about 4.4 million children are screened and about 3% of the children are treated).³⁰

An advantage to the GCK app is the ease with which clinicians can use the application. With the prevalence of smartphone technology even in more remote areas of the world, virtually everyone has the ability to use this app, meaning children in more rural areas can be properly screened for amblyopia risk factors without having to travel great distances.

There are limitations to this study, however. We did not evaluate the GCK app for its ability to detect strabismus, also an AAPOS amblyopia risk factor, for example. Additionally, only a single flash (portrait) image was acquired which limits the ability to detect astigmatism, that requires both a portrait and landscape image. It is possible that the results garnered in this study would not be borne out with much larger cohorts, or in all populations. Similarly, this study evaluated the GCK app on one smartphone device, the iPhone 7. As cellphone manufacturers continue to improve their technologies, this app may function differently on future smartphones.

It is possible that accommodation in hyperopic patients caused the refractive crescent measurement to fall below the referable risk threshold. Also, because the app only measured refractive error in one meridian, it may have underestimated the amount of astigmatism. Lastly, Internet access is necessary, which may preclude this app’s secondary follow-up image interpretation use in the most remote regions of the world. The next generation smartphone app includes this capability.

Conclusion

The GoCheckKids study demonstrated a specificity and sensitivity for identifying refractive amblyopia risk factors with the iPhone 7 app that is within the range of other photoscreening methods. This smartphone photoscreener is a viable photoscreening app for the detection of amblyopia risk factors in children aged 1–6 years.

Acknowledgments

John Tamkin, PhD, a consultant for Gobiquity, provided technical assistance. Michelle Dalton, ELS, provided medical writing assistance. The study was supported by Gobiquity Mobile Health, Inc.

Disclosure

The study was supported by Gobiquity Mobile Health, Inc. The authors report no other conflicts of interest in this work.

---

References

1.		Donahue SP, Ruben JB, American Academy of Ophthalmology, et al. US Preventive Services Task Force vision screening recommendations. Pediatrics. 2011;127(3):569–570.
2.		Agervi P, Kugelberg U, Kugelberg M, Zetterström C. Two-year follow-up of a randomized trial of spectacles plus alternate-day patching to treat strabismic amblyopia. Acta Ophthalmol. 2013;91(7):678–684.
3.		Taylor K, Elliott S. Interventions for strabismic amblyopia. Cochrane Database Syst Rev. 2014;7(7):CD006461.
4.		Richardson SR, Wright CM, Hrisos S, Buck D, Clarke MP. Stereoacuity in unilateral visual impairment detected at preschool screening: outcomes from a randomized controlled trial. Invest Ophthalmol Vis Sci. 2005;46(1):150–154.
5.		Wallace DK, Lazar EL, Melia M, et al. Stereoacuity in children with anisometropic amblyopia. J AAPOS. 2011;15(5):455–461.
6.		Pascual M, Huang J, Maguire MG, et al. Risk factors for amblyopia in the vision in preschoolers study. Ophthalmology. 2014;121(3):622–629.
7.		American Academy of Pediatrics. Visual system assessment in infants, children, and young adults by pediatricians. Pediatrics. 2016;137(1):e20153596.
8.		Afsari S, Rose KA, Gole GA, et al. Prevalence of anisometropia and its association with refractive error and amblyopia in preschool children. Br J Ophthalmol. 2013;97(9):1095–1099.
9.		American Association for Pediatric Ophthalmology and Strabismus. Vision Screening Recommendations; 2014. Available from: http:// www.aapos.org/terms/conditions/131 Accessed July 08, 2018.
10.		Longmuir SQ, Boese EA, Pfeifer W, et al. Practical community photoscreening in very young children. Pediatrics. 2013;131(3):e764–e769.
11.		Kemper AR, Clark SJ. Preschool vision screening in pediatric practices. Clin Pediatr. 2006;45(3):263–266.
12.		Alley CL. Preschool vision screening: update on guidelines and techniques. Curr Opin Ophthalmol. 2013;24(5):415–420.
13.		Peterseim MM, Arnold RW. Vision screening: program models. American Academy of Ophthalmology; 2015. Available from: http://www.aao.org/pediatric-center-detail/vision-screening-program-models. Accessed July 08, 2018.
14.		US Preventive Services Task Force, Grossman DC, Curry SJ, et al. Vision screening in children aged 6 months to 5 years: US Preventive Services Task Force Recommendation Statement. JAMA. 2017;318(9):836–844.
15.		Arnold RW, Arnold AW, Stark L, et al. Amblyopia detection by camera: Gateway to portable, inexpensive vision screening (calibration and validation of inexpensive, pocket-sized photoscreeners). Alaska Med. 2004;46(3):63–72.
16.		Matta NS, Arnold RW, Singman EL, Silbert DI. Comparison between the PlusoptiX and MTI photoscreeners. Arch Ophthalmol. 2009;127(12):1591–1595.
17.		Silbert DI, Arnold RW, Matta NS. Comparison of the iScreen and the MTI photoscreeners for the detection of amblyopia risk factors in children. J AAPOS. 2013;17(1):34–37.
18.		Roorda A, Bobier WR, Campbell MC. An infrared eccentric photo-optometer. Vision Res. 1998;38(13):1913–1924.
19.		Roorda A, Campbell MC, Bobier WR. Slope-based eccentric photorefraction: theoretical analysis of different light source configurations and effects of ocular aberrations. J Opt Soc Am A Opt Image Sci Vis. 1997;14(10):2547–2556.
20.		Fan C-S, Edwards M, Chan Y-C. Vision screening of preschool children by eccentric photorefraction using a digital camera. Hong Kong J Ophthal. 1998;2(1):14–20.
21.		Chen YL, Tan B, Lewis J. Simulation of eccentric photorefraction images. Opt Express. 2003;11(14):1628–1642.
22.		Schimitzek T, Haase W. Efficiency of a video-autorefractometer used as a screening device for amblyogenic factors. Graefes Arch Clin Exp Ophthalmol. 2002;240(9):710–716.
23.		Erdurmus M, Yagci R, Karadag R, Durmus M. A comparison of photorefraction and retinoscopy in children. J AAPOS. 2007;11(6):606–611.
24.		Chou R, Dana T, Bougatsos C. Screening for visual impairment in children ages 1–5 years: systematic review to update the 2004 U.S. Preventive Services Task Force Recommendation. Rockville, MD: Agency for HealthcareResearch and Quality (US); (Evidence Syntheses, No 81); 2011. Available from: https://www.ncbi.nlm.nih.gov/books/NBK52708/. Accessed July 08, 2018.
25.		Jost RM, Yanni SE, Beauchamp CL, et al. Beyond screening for risk factors: objective detection of strabismus and amblyopia. JAMA Ophthalmol. 2014;132(7):814–820.
26.		Kirk S, Armitage MD, Dunn S, Arnold RW. Calibration and validation of the 2WIN photoscreener compared to the PlusoptiX S12 and the SPOT. J Pediatr Ophthalmol Strabismus. 2014;51(5):289–292.
27.		Sanchez I, Ortiz-Toquero S, Martin R, de Juan V. Advantages, limitations, and diagnostic accuracy of photoscreeners in early detection of amblyopia: a review. Clin Ophthalmol. 2016;10:1365–1373.
28.		Peterseim MMW, Rhodes RS, Patel RN, et al. Effectiveness of the GoCheck Kids vision screener in detecting amblyopia risk factors. Am J Ophthalmol. 2018;187:87–91.
29.		Hagan JF, Shaw JS, Duncan PM. editors. Bright Futures: Guidelines for Health Supervision of Infants, Children, and Adolescents. 4th ed. Elk Grove Village, Ill: American Academy of Pediatrics; 2017.
30.		Gibson WE. Economic analysis of the consequences of failure to prevent childhood blindness from Amblyopia; 2013. Available from: http://www.eyeresearch.org/pdf/economic%20impact_of_blindness_1-7.pdf. Accessed July 12, 2018.