Performance of Two Photoscreeners Enhanced by Protective Cases
Authors Martin SJ, Htoo HE, Hser N, Arnold RW
Received 26 February 2020
Accepted for publication 8 May 2020
Published 25 May 2020 Volume 2020:14 Pages 1427—1435
Checked for plagiarism Yes
Review by Single anonymous peer review
Peer reviewer comments 2
Editor who approved publication: Dr Scott Fraser
Samuel Joseph Martin,1 Hser Eh Htoo,2 Nay Hser,2 Robert W Arnold1
1Alaska Blind Child Discovery, Anchorage, AK, USA; 2Jungle School of Medicine Kawthoolei, Karen State, Burma (Myanmar)
Correspondence: Robert W Arnold
Alaska Blind Child Discovery, Anchorage, AK, USA
Tel +1 907561-1917
Background: Photoscreeners provide valid, cost-effective early detection of amblyopia risk factors; however, they require proper illumination, flash intensity, pupil dilation and patient cooperation. The Kaleidos case for the 2WIN infrared photoscreener fixes focal distance, decreases luminance and provides electric power. GoCheck Kids (GCK) has developed a flash-concentrating case for the Apple iPhone 7 Plus smartphone to reduce exposure time.
Methods: In remote Burmese and urban Alaskan clinics, Kaleidos and GCK were used before a confirmatory eye examination using 2013 AAPOS uniform guidelines validation, including some older patients. 2WIN refraction was compared to a cycloplegic examination using J0 and J45 vector transformation.
Results: In total, 48 Burmese patients and 114 Alaskan patients aged 8± 6 years were evaluated utilizing 2013 amblyopia risk factor guidelines. Kaleidos, with 13 Burmese inconclusives, had a sensitivity of 84%, specificity of 65% and positive predictive value (PPV) of 63%. GCK, with only six inconclusives, had a sensitivity of 63%, specificity of 83% and PPV of 73%, with sensitivity improving to 74% with central expert interpretation. Kaleidos closely matched cycloplegic refraction, with intraclass correlations of 0.47 for J0 vector and 0.57 for J45 vector. The protective cases provided clearer images and better pupil dilation than similar devices without cases.
Conclusion: Both devices detected amblyopia risk factors well. GCK gave fewer inconclusive results even with the Asian eyelid configuration, while Kaleidos matched sphere and cylinder refraction. The specialized housing enhanced the performance of the 2WIN and GCK photoscreeners, improving the speed and reliability of amblyopia screening and refraction, even in populous and luminous locations.
Clinical Trials Registry: NCT04068129.
Keywords: pediatric vision screening, amblyopia, instrument-based screening, validation
Photoscreening is an objective technique for detecting amblyopia risk factors early enough to reduce permanent vision impairment.1–4 Amblyopia is deficient acquisition of central nervous system vision during a critical period of a child’s first decade.5 Risk factors for amblyopia include obscured images (ie infantile cataract), constant strabismus and refractive error. Effective treatment of amblyopia includes early reversal of the offending risk factors and directing visual influences to the amblyopic eye. The vision screening committee of the American Association for Pediatric Ophthalmology and Strabismus (AAPOS) has published two related sets of uniform guidelines for the comparative evaluation of cost-effective screening for target levels of the main amblyopia risk factors, the first in 2003,6 followed by the 2013 age-based update.7
Photoscreeners work by shining visible or infrared light at both eyes and then analyzing the pupillary red reflexes. Current instruments can provide instant computer determination as to whether the patient passes or should be referred for confirmatory pediatric ophthalmic examination.8 A dimly lit environment, with children who are cooperative, sit still and have large pupils are the optimal conditions for producing reliable results with photoscreeners. Challenges arise when dealing with patients who cannot look at the fixation targets on the devices, or when their pupils are too small or partially obstructed by eyelids or eyelashes, which has been a notorious photoscreening challenge in Asian children. Two photoscreeners that have been enhanced with special housing to address some of these challenges of pediatric vision screening worldwide are the computer-interpreted Kaleidos for the 2WIN (Adaptica, Padova, Italy) and smartphone-interpreted GoCheck Kids (GCK, aka Gobiquity, Scottsdale, AZ).
2WIN is a computer-interpreted infrared photoscreener with similarly high validity when compared to two other infrared photoscreeners: Plusoptix (Nuremberg, Germany) and SPOT (Welch-Allyn, Skaneateles Falls, NY).9 2WIN was developed by Mario Angi as a component of a portable suite of refractive tools for adults, and also for children. 2WIN refractometry resembles the high quality of Retinomax (Righton, Tokyo Japan).10 A modified housing for 2WIN called Kaleidos has been developed; this is a rectangular tube which, on one end, clamps the 2WIN with additional battery power and tablet remote control, while the other end of the housing provides a darkened tunnel with the desired focal distance that encourages the patient to gaze undistracted at the 2WIN fixation lights, while also naturally dilating the pupils.
In 2013, we discovered that an LED-flash iPhone (Apple, Cupertino, CA) smartphone can provide valid photoscreening if the child sits still.11 GCK then provided interpretive software adapted to evolving models of smartphones allowing cost-effective photoscreening. The actual flash of the Nokia Lumia 1020 (Espoo, Finland) was initially cropped and utilized, with excellent positive predictive value (PPV).12,13 For typical American children showing astigmatism, predominantly, with-the-rule, one-axis exposure had similar validity to slower, two-axis exposure.14 The current GCK app uses an iPhone with an LED source which requires longer than instant exposure to gain sufficient pupillary red reflex.15 GCK has developed an enhanced housing for the iPhone 7 Plus that concentrates the flash by adding a convex lens centered over the LED “flash”, speeding up the process, reducing blur and facilitating two-axis photoscreening. Smartphone software was then adjusted to account for the brighter, more concentrated red-reflex illumination.
We compared the 2WIN in the Kaleidos system and the GCK flash-concentrated housing in a remote clinic in Burma, and also in an Alaskan urban pediatric ophthalmology clinic.
This reliability analysis was covered by the Institutional Review Board at Providence Hospital, Anchorage, Alaska ( Clinical Trials Registry no. NCT04068129). Depending on age, the patients, parents and/or legal guardians provided written informed consent or assent. For Burmese patients, risks and benefits were translated into the native S’Gaw or Po Karen language. The study complies with HIPAA and the Declaration of Helsinki. Patients received medical care but no stipend was paid for research participation. This study of screening tests complies with STARD 2015.
Photoscreening with follow-up confirmatory examinations for referred and passed interpretations was offered to children and young adults at a remote clinic in the Karen State of eastern Burma (Myanmar). The clinic had an intermittent 220-volt power generator, but no internet coverage.
Consecutive patients in a WiFi-equipped pediatric ophthalmology practice in Anchorage, Alaska, underwent photoscreening before confirmatory examination.
Each patient underwent photoscreening with the 2WIN photoscreener installed in the Kaleidos housing. In Anchorage, the device was controlled by the accompanying wireless tablet computer. In Burma, however, despite using a WiFi router not connected to the internet, the tablet computer would not connect to the 2WIN; therefore, the rear trapdoor of the Kaleidos was opened so that the 2WIN could be activated and controlled manually (Figure 1). The 2WIN stored results on a micro-SD memory card, which was eventually downloaded to a computer.
Figure 1 Kaleidos enhanced housing for 2WIN photoscreening, with housing opened to allow manual control. (Parental permission given for use of all photographs).
Each patient also underwent photoscreening with GCK on an iPhone 7 Plus using the enhanced, flash-concentrating cell-phone case. Pupil size and ocular media were viewed in real time after successful image acquisition. In Anchorage, in addition to on-site smartphone interpretation, images were uploaded to the manufacturer’s central reading center for secondary interpretation. GCK as a photoscreener therefore has the unique option to provide on-site smartphone interpretation and also internet-transmitted, expert reinterpretation by the manufacturer; both results were analyzed. In Burma, all images were retained on the smartphone and eventually uploaded for central secondary analysis by GCK after returning to urban internet availability. In Burma, a portable, lightweight tent was used to provide a dim screening environment with less peripheral visual distraction (Figure 2).16 The order of Kaleidos and GCK photoscreening was randomized and the confirmatory examiner was not aware of the screening results at that time.
Figure 2 GoCheck Kids iPhone 7 Plus with flash-concentrating case being used in a tent in Burma.
Confirmatory examinations were performed following 2013 AAPOS uniform guidelines, modified so that older patients were grouped with the older triad of patients (“2013+” AAPOS). Additional analyses were performed with the 2003 guidelines and patients with amblyopia. Amblyopia is defined as visual acuity of 20/40 or worse and/or two-line intereye difference combined with amblyopia risk factors. Cycloplegic refractions were performed 30–40 minutes after instillation of cyclopentolate 1%. Visual acuity was checked with patched, surround HOTV at 3 meters, at a threshold of at least three out of four correct optotypes. Validation was performed with a conventional 2×2 screen-examination matrix, and also with an ABCD 2×3 matrix, which incorporates inconclusive results (no instrument interpretation or unable to gain a reading) which are considered as referrals (Table 1).17 No patients were lost to follow-up.
Table 1 Validation of Kaleidos and GoCheck Kids Compared to Age-Added 2013 Amblyopia Risk Factors and True Amblyopia
Twenty of the Anchorage screenings were compared with prior versions of GCK (iPhone 7 Plus alone, Nokia Lumia 1020) and also a hand-held Adaptica 2WIN without a Kaleidos case.
The sample size for a correlation with type I error alpha of 0.01 and type II error beta of 0.05, expecting a correlation of r=0.6, should comprise at least 40 subjects.
In Burma, 48 patients aged 15±8 years (mean±SD) completed testing with a prescreening probability of 2003 AAPOS risk factors of 8%. These were compared to and added to 114 patients from Alaska, with mean±SD age 7±4 years and prescreening probability of 60%. The median age for all 162 patients was 8.2 years. Only 50 subjects met the exact age ranges for 2013 AAPOS age divisions less than 72 months and therefore we added older patients to the older triad, defined as “2013+.”
Table 1 gives the performance of the Kaleidos and the GCK photoscreeners compared to age-enhanced 2013 AAPOS guidelines and also compared to actual amblyopia. Figure 3 shows receiver operating characteristics (ROC) curves for Kaleidos and flash-concentrated GCK compared to 2013+ AAPOS amblyopia risk factors and also to actual amblyopia, with any inconclusive results considered as a “refer.”
Validation is impacted by how inconclusive interpretations are considered,17 by instrument referral criteria, by prescreening probability and by gold-standard examination criteria. The Supplementary Table shows validation metrics (sensitivity, specificity and PPV) for Kaleidos and GCK housing-modified photoscreeners separated by Burmese and Alaskan cohorts with the corresponding instrument referral criteria. By 2013+ AAPOS amblyopia risk factor uniform guidelines with inconclusives considered as “refer”, the sensitivity, specificity and PPV for Kaleidos in 48 Burmese patients were 75%, 68% and 18% and in 114 Alaskan patients 86%, 62% and 74%, respectively. In comparison, GCK on-site had respective values of 50%, 86% and 25% in Burmese and 64%, 80% and 80% in Alaskan patients, but central interpretation yielded 100%, 60% and 18% in Burmese and 72%, 68% and 74% in Alaskan patients, respectively. When inconclusive interpretations were incorporated as if they were referrals, the performance of Kaleidos suffered more than GCK. The sensitivity of GCK on-site interpretation was increased by remote, central expert reading.
Figures 4 and 5 show refraction estimates of Kaleidos by J0 and J45 power vectors, respectively, compared to retinoscopy. For all patients with cycloplegic refraction sphere hyperopia greater than 0.5 D, the mean±SD hyperopia was 2.1±2.1 D while the reading from the Kaleidos was 0.4±1.4 D. Figures 6 and 7 show corresponding Bland–Altman plots. Table 2 shows Bland–Altman metrics and intraclass correlation coefficients from refractive estimates for hyperopia, myopia and astigmatism degree, and J0 and J45 vectors, with GCK and Kaleidos compared to cycloplegic examination.
Table 2 Correlations Between Kaleidos Refractions and Cycloplegic Examination
Figure 4 Kaleidos J0 vector analysis comparing Kaleidos refractions with cycloplegic retinoscopy.
Figure 5 J45 vector analysis comparing 2WIN photoscreening through Kaleidos protective case with uniform guidelines cycloplegic refraction.
Figure 6 Bland–Altman plot for J0 vectors comparing Kaleidos with retinoscopy.
Figure 7 Bland–Altman plot for J45 vectors comparing Kaleidos with retinoscopy.
The practical ability to obtain focused photoscreen images and yield interpretation in 20 patients was compared with prior instruments lacking the new protective cases. In the daytime clinic in Burma and in the lit examination rooms in Alaska, Kaleidos was easily able to yield focused images with mydriatic pupils, while the hand-held 2WIN would not provide an interpretation owing to either “pupils too small” or excess infrared light levels. Opening the trapdoor back of Kaleidos in Burma produced adequate photoscreening images similar to tablet control in Anchorage, but activating the shutter was not simple. Some Burmese patients could not open their eyelids enough for the 2WIN to yield results with or without the Kaleidos case. In patients who did not easily sit still, prior GCK cameras required more repeat efforts and resulted in more blurred pupils compared to those images produced by the new flash-concentrating case. As with our past experience, the tent enhanced pupil dilation and fixation compared to middle-of-the-room screening efforts with GCK in the Burmese patients.
Protective containers improved illumination and fixation for the 2WIN photoscreener, and the flash-concentrating external case improved image quality by shortening exposure time with the GCK app on the iPhone 7 Plus. Kaleidos refractions closely resembled sphero-cylinder refraction, but underestimated cycloplegic hyperopia, as expected.18 GCK gave fewer inconclusive readings than 2WIN, especially in Burmese patients. Central expert interpretation after internet download to the manufacturer improved validity for sensitive GCK screening by a moderate amount.
Our experience with both devices with protective enhancing cases compared well with other photoscreening studies.3 When inconclusive results are counted as “refer”, in a cohort with 45% prescreening probability of 2003 AAPOS risk factors, Kaleidos had 74% sensitivity and 82% specificity while GCK had 78% sensitivity and 86% specificity. Bare 2WIN, with its CR strabismus tool but no protective case, had 68% sensitivity and 84% specificity in a cohort with 56% prescreening probability of 2003 AAPOS risk factors and 31% developmental delay.10 GCK with the Nokia 1020 smartphone had 65% sensitivity and 83% specificity on site, improving to 76% sensitivity and 85% specificity with expert central reading, in an enriched cohort with 38% prescreening probability.15 In another group with 36% 2013 prescreening probability, GCK Nokia 1020 had 76% sensitivity and 67% specificity on site, and also improved with central expert reading.12 GCK Nokia 1020 referred 5% of 6310 pediatrician screenings for a 2003 AAPOS PPV of 77%.13 In head-to-head screening of Plusoptix compared to SPOT in 270 patients with 63% prescreening probability, the sensitivity and specificity for Plusoptix were 84% and 94% and for SPOT were 82% and 68%, respectively,17 and again with 62 children, the sensitivity and specificity for Plusoptix S12 were 91% and 71%, for SPOT 78% and 59% and for 2WIN 73% and 76%, respectively.9
The prevalence of 2003 AAPOS amblyopia risk factors (prescreening probability) in American communities is about 21%.19,20 Community amblyopia screening excels when the PPV is 80% or greater21 and specificity exceeds 90%.22 Since the prevalence of AAPOS amblyopia risk factors exceeds the prevalence of amblyopia, 100% sensitivity is not preferred, and with early specific instrument referral criteria, a sensitivity near 50% may be adequate.20 By these standards, the protective-cased Kaleidos and GCK photoscreeners performed admirably.
Advantages of this study are the inclusion of remote and urban patient cohorts, one with low and one with high prescreening probability for refractive amblyopia risk factors according to the uniform guidelines. In community screening, strabismus is much less common than in pediatric eye practice; most novel cases of strabismic amblyopia are associated with apparent, large-angle constant strabismus, for which scheduled photoscreening is inferior to immediate parental referral for timely detection.
Weaknesses of this study are the relatively small number of subjects with ages represented by 2013 AAPOS guidelines screening. We also did not screen with and without the protective cases in all patients. When lacking an internet connection in Burma, the Kaleidos system seemed to perform well when the operator manually opened the clamshell, but we cannot be sure that this method truly mirrored the routine use in the Alaskan patients.
Adaptica could improve their results by working to better interpret dark irides with Asian eyelid and eyelash configuration, and by redesigning the camera shutter when internet and WiFi are not available for Kaleidos. GoCheck Kids could improve their screener by improving fixation and accommodation directed at their smartphone lens or the protective case.
Data Sharing Statement
De-identified data can be accessed at http://www.abcd-vision.org/references/Kaleidos%20GCK%20shared%20ABCCD%20database.pdf
We thank medics from Jungle School of Medicine Kawthoolei and Cross Border Clinic as well as Eric Egeland and Sandy Singer for their assistance with photoscreening in Burma.
Dr. Arnold coordinates the Alaska Blind Child Discovery (ABCD) project, which has received discounted vision screen technology from several vendors. He is the President of Glacier Medical Software, which markets the cloud-based NICU monitoring software ROP Check. Dr. Arnold is also the President of PDI Check, which has developed vision screening game for the Nintendo 3DS. Dr. Arnold is an investigator and protocol developer for PEDIG. Dr. Robert Arnold has a patent for PDI Check pending to Robert Arnold and Alex Damarjian. Sam Joseph Martin, Nay Hser and Hser Eh Htoo have no financial conflicts to disclose. The authors report no other conflicts of interest in this work.
1. Kirk VG, Clausen MM, Armitage MD, Arnold RW. Preverbal photoscreening for amblyogenic factors and outcomes in amblyopia treatment: early objective screening and visual acuities. Arch Ophthalmol. 2008;126(4):489–492.
2. US Preventive Services Task Force, Grossman DC, Curry SJ, et al. Vision screening in children aged 6 months to 5 years: US Preventive Services Task Force recommendation statement. JAMA. 2017;318(9):836–844. doi:10.1001/jama.2017.11260
3. Silverstein E, Donahue SP. Preschool vision screening: where we have been and where we are going. Am J Ophthalmol. 2018;194:xviii–xxiii. doi:10.1016/j.ajo.2018.07.022
4. O’hara MA. Instrument-based pediatric vision screening. Curr Opin Ophthalmol. 2016;27:1–4. doi:10.1097/ICU.0000000000000289
5. Wu C, Hunter DG. Amblyopia: diagnostic and therapeutic options. Am J Ophthalmol. 2006;141(1):175–184. doi:10.1016/j.ajo.2005.07.060
6. Donahue S, Arnold R, Ruben JB. Preschool vision screening: what should we be detecting and how should we report it? Uniform guidelines for reporting results from studies of preschool vision screening. J AAPOS. 2003;7(5):314–316. doi:10.1016/S1091-8531(03)00182-4
7. Donahue SP, Arthur B, Neely DE, Arnold RW, Silbert D, Ruben JB. Guidelines for automated preschool vision screening: a 10-year, evidence-based update. J AAPOS. 2013;17(1):4–8. doi:10.1016/j.jaapos.2012.09.012
8. Miller JM, Lessin HR. AAP. Instrument-based pediatric vision screening policy statement. Pediatrics. 2012;130(5):983–986.
9. Kirk S, Armitage MD, Dunn S, Arnold RW. Calibration and validation of the 2WIN photoscreener compared to the PlusoptiX S12 and the SPOT. J Pediatr Ophthalmol Strabismus. 2014;51(5):1–4. doi:10.3928/01913913-20140701-01
10. Arnold SL, Arnold AW, Sprano JH, Arnold RW. Performance of the 2WIN photoscreener with “CR” strabismus estimation in high risk patients. Am J Ophthalmol. 2019;207:195–203. doi:10.1016/j.ajo.2019.04.016
11. Arnold RW, Davis B, Arnold LE, Rowe KS, Davis JM. Calibration and validation of nine objective vision screeners with contact lens-induced anisometropia. J Pediatr Ophthalmol Strabismus. 2013;50(3):184–190. doi:10.3928/01913913-20130402-02
12. Peterseim MMW, Rhodes RS, Patel RN, et al. Effectiveness of the GoCheck Kids vision screener in detecting amblyopia risk factors. Am J Ophthalmol. 2018;187:87–91. doi:10.1016/j.ajo.2017.12.020
13. Arnold RW, Arnold AW, Hunt-Smith TT, Grendahl RL, Winkle RK. The positive predictive value of smartphone photoscreening in pediatric practices. J Pediatr Ophthalmol Strabismus. 2018;55(6):393–396. doi:10.3928/01913913-20180710-01
14. Silbert DI, Arnold RW. Do we need to directly detect astigmatism when photoscreening for amblyopia risk factors (ARFs)? J AAPOS. 2015;19(4):e61.
15. Arnold RW, O’Neil JW, Cooper KL, Silbert DI, Donahue SP. Evaluation of a smartphone photoscreener app to detect refractive amblyopia risk factors in children 1–6 years. Clin Ophthalmol. 2018;12:1533–1537. doi:10.2147/OPTH.S171935
16. Arnold RW, Stark L, Leman R, Arnold KK, Armitage MD. Tent photoscreening and patched HOTV visual acuity by school nurses: validation of the ASD-ABCD protocol. (Anchorage School District- Alaska Blind Child Discovery program). Binocul Vis Strabismus Q. 2008;23(2):83–94.
17. Arnold RW, Arnold AW, Armitage MD, Shen JM, Hepler TE, Woodard TL. Pediatric photoscreeners in high risk patients 2012: a comparison study of Plusoptix, iScreen and SPOT. Binocular Vis Strabismus Quart. 2013;28(1):20–28.
18. Racano E, Alessi S, Pertile R. Comparison of 2Win and plusoptiX A12R refractometers with Retinomax handheld autorefractor keratometer. J AAPOS. 2019;23(5):276e271–276 e275. doi:10.1016/j.jaapos.2019.05.017
19. Varma R. Amblyopia refractive risk factors (letter reply). Ophthalmol. 2012;119(6):1283–1284. doi:10.1016/j.ophtha.2012.02.012
20. Arnold RW. Amblyopia risk factor prevalence. J Pediatr Ophthalmol Strabismus. 2013;50(4):213–217. doi:10.3928/01913913-20130326-01
21. Arnold RW, Donahue SP. The yield and challenges of charitable state-wide photoscreening. Binocul Vis Strabismus Q. 2006;21(2):93–100.
22. VIPS. Comparison of preschool vision screening tests as administered by licensed eye care professionals in the vision in preschoolers study. Ophthalmology. 2004;111(4):637–650. doi:10.1016/j.ophtha.2004.01.022