Pediatric Non-Refractive Vision Screening with EyeSwift, PDI Check and Blinq: Non-Refractive Vision Screening with Two Binocular Video Games and Birefringent Scanning

Introduction

Vision screening is applied to classes of younger students to detect amblyopia early enough for effective therapy, and also to recognize visually significant refractive errors high, symmetric myopia and astigmatism so spectacle or contact lens therapy can allow clear vision for learning.¹ Earlier detection is beneficial for successful amblyopia treatment, but newly detected amblyopic students in third grade and even older can still benefit from occlusion, penalization and spectacle therapy.² The ability of new non-refractive vision screening devices has not yet been compared and validated with standard methods.

Non-refractive devices (Figure 1) require some level of cooperation but could easily be done by students of Kindergarten age or even younger. If such a student is referred to a busy school nurse via conventional visual acuity screening, refractive photoscreening³ or observations by a teacher, what additional information could they practically provide? In this study, each device is used as if in a busy school nurse’s office with the nurse’s attention divided between the enhanced non-refractive device screening and other duties.

Figure 1 Vision screening devices: NovaSight EyeSwift® (upper right), PDI Check on Nintendo 3DS dev kit (lower left) and Rebion blinq (upper left).

A non-refractive portable vision screen video game, PDI Check (Anchorage, Alaska), is played on the autostereoscopic Nintendo 3DS. It can provide, rapid, dynamic stereopsis and monocular screening of near visual acuity and color. PDI Check can be performed by non-English, non-computer literate individuals.⁴ It correlates well with Innova Rabin color test for color deficient individuals.⁵ PDI Check may have merit in screening for some types of amblyopia since it indicates ocular suppression.⁶ The dynamic presentation does not have a testing “floor-effect” so superior visual performance in stereopsis can be monitored.⁷

A novel, non-refractive sensory testing game called EyeSwift® (Nova-Sight, Israel) incorporates eye tracking with an interactive touch video screen. Binocular testing is achieved by placing electronic shutter screen goggles, or red-blue anaglyph goggles over the students’ conventional eyewear, if any. EyeSwift is programmed to perform many tests including reading fixation accuracy and vergence amplitudes, and it can do an automated cover test with reliable results.⁸

The other non-refractive device developed since 1991 by Drs. David Hunter and David Guyton, Rebion blinq (Boston, Mass) was commercially released late in 2019 and utilizes radially oriented birefringence to assess a student’s ability to steadily foveate.⁹ An earlier version (PVS) outperformed the monocular autorefractor Suresight.¹⁰ Using a novel validation rubric based on Multi Ethnic Pediatric Eye Disease Study (MEPEDS) amblyopia risk factors (ARF),¹¹ Bosque et al found sensitivity 100% and specificity 91% with blinq.¹²

This study aims to validate against the most recent standards each novel, non-refractive screening device highlighting aspects as if each was performed by a school nurse or in pediatrician’s office.

Methods

This Alaska Blind Child Discovery study, an evaluation of screening tests, was approved by the institutional review board of Providence Alaska Medical Center; parents and legal guardians provided signed written consent for children younger than 18 years, and older children provided written assent appropriate for age. The study is compliant with HIPAA and the Declaration of Helsinki. De-identified data are available for review at https://www.abcd-vision.org/references/EyeSwift-PDI-blinq.pdf.

Patients were attending a subspecialty pediatric ophthalmology clinic for new or follow-up indications. Each completed a confirmatory exam that included sensory testing with patched, monocular surround distance HOTV visual acuity, Stereo Fly and Worth dot testing distance and nearby. Refraction was done with accommodation-relaxing retinoscopy either with SBARS¹³ or phoropter with fogging and cycloplegia (cyclopentolate 1% 30 min before) was used in younger children. Visual acuity was maximally refined at the phoropter. Diagnosis of amblyopia (Va worse than 0.3 logMAR intereye difference ≥0.2), strabismus (manifest >8 PD), nystagmus and color deficiency were taken directly from the confirmatory exam. The level of cooperation with the exam was subjectively determined as to whether the patient sustained a relaxed gaze at a distant target or not.

The manufacturer’s procedure for each device was carefully adapted to simulate testing a student as if they were referred to a school nurse. Students were encouraged to “play the video game” and follow its instructions seeking assistance and instruction only when needed. Tests were performed in a moderately dim examination room, and lights were turned down with window blinds drawn for blinq testing. Sensory tests were completed in spectacles if patient had them, however the blinq was performed without spectacle correction. The order of testing was randomized between EyeSwift®, blinq and PDI Check. No patient had cycloplegia before screening.

EyeSwift® is a highly specialized tablet computer with built-in eye tracking. The student can wear their own spectacles and then perform some of the tasks with electronic shutter goggles, other tests with provided red-blue goggles and others with no goggles over their conventional eyewear. EyeSwift® can perform many tasks, such as cover testing, assessment of vergence amplitudes and reading accuracy. In this study, we limited our testing to near monocular visual acuity (surround tumbling E optotypes), random-dot stereo, worth-dot near and Ishihara color plate testing. EyeSwift® does not recommend using the device in children less than age 4 years due to cooperation issues; as such this study limited enrollment to children older than age 4 years.

PDI Check (version 2.1.7) is a forced choice, dynamic near vision screening game on the Nintendo 3DS battery-powered video game autostereoscopic barrier screen. The student gazes at the upper 3-dimensional screen and interacts with the lower touch screen in a sturdy, clam-shell case. The game is designed with a familiarization phase for color, Landolt-C near visual acuity and stereopsis wherein a user lag-time is found, then the testing phase presents four quadrants with identical, tasks that transition from fine to coarse resolution until the student recognizes a “different” quadrant. For visual acuity, all four quadrants start with a tiny “C” and then one quadrant, for one eye only switches to “U” configuration and all 4 quadrants get bigger until the student can select the correct one with better monocular acuity associated by rapid, correct detection. For stereo, all 4 quadrants start with bold, black rings all appearing equidistant from the student’s eyes. Then one random quadrant autostereoscopically “levitates” with a good score indicated by rapid lower-screen selection of the corresponding quadrant. Color is monocularly tested with a isoluminence, hue-on-gray presentation with a colored disc (teal, pink or purple) in one random quadrant for just one eye, gradually transitioning to a bold color from uniform gray. If a student cannot recognize the dynamic, correct random quadrant, then a bold, obvious image- observable by left and right eye- is shown to allow scoring- and indicating poor visual performance on that aspect of the screening game. Eventually a set of results is presented with logMAR visual acuity right and left, stereopsis in arc seconds- with no 40 arc-seconds “floor”, and monocular color shown as percent red, green and blue cone function. The time to screen is also recorded.

The Rebion blinq (early serial number, Model BQ830, Pro Version, version A.2.1 B-00027-V, Rebion, Boston, MA) was used according to manufacturer’s guidelines in a dim room, with distance calibration red laser beams directed to the student’s skin at the chin or below (despite COVID-19 masking) and then focal-distance aiming beams concentrated on the bridge of the nose steadily asking the student to gaze at the orange object in the round window on the front of the device. If a “timed-out” result was obtained, then a repeat measure was taken. Using 2003 AAPOS criteria, the blinq has compared favorably with the conventional, multiradial infrared autorefractor Adaptica 2WIN¹⁴ utilizing the corneal reflex and strabismus quantifying function combined with infrared-blocking goggles.¹⁵

Instrument referral criteria: Three criteria were utilized for blinq; 1) any initial screening with a definite “refer-right” or “refer-left” was determined refer and all others including repeated “timed out” considered a “pass,” 2) all initial “refer”- and the four “refer” after an initial “timed out” result were considered the referrals with all others counted as pass and 3) initial screenings that were not a “pass” considered as the referrals. The instrument criteria for EyeSwift® were 1) sensitive with stereo worse than 200 arc seconds or worst monocular logMAR acuity 20/30 or worse and Worth Dot not fusing, 2) more specific with stereo of 800 arc seconds arc or worse or least acuity 20/40 or suppression on Worth Dot and 3) most specific stereo of 800 arc seconds or worse or least visual acuity 20/50 or worse or Worth Dot suppressing. The instrument referral criteria used to generate the ROC curve for PDI Check were 1) more sensitive stereo 100 arc seconds or worse or visual acuity logMAR 0.3 or worse or intereye difference 0.2 or worse, 2) more specific stereo 400 arc seconds or worse or visual acuity logMAR 0.4 or worse or intereye difference 0.3 or worse, and 3) most specific stereo 1000 arc seconds or least visual acuity logMAR 0.5 or intereye difference 0.4 or worse.

Validation targets: Confirmatory examinations were classified as to amblyopia risk factors using the 2021 AAPOS Guidelines for the ages 4 years and older.¹⁶ The refractive components were analyzed separately and then refractive ± strabismic ± visual acuity were also analyzed. The 24-point rubric based on MEPEDS¹² was also analyzed and compared to AAPOS 2021 guideline amblyopia ± strabismus (Table 1).

Table 1 Exam Outcomes for Validation

Sample Size: to detect a difference of 0.25 from a mean of 2.5 for log stereo, given alpha 0.05 and beta 0.8, a sample size of at least 63 is prudent. Consideration of level of cooperation was given before recruitment and young preschool-aged children were not entered, but students with developmental disabilities were not excluded from the study.

Results

Same-day comparison by all three testing devices with confirmatory exam was completed by 77 students, aged 11±4 years (range 4–19), of whom 41 were males. The race/ethnicity mix of the cohort was Pacific Islanders 5, Alaska natives 8, Hispanics 10, black 10, Asian 1 and white 43. Eighteen were treatment naïve. Cooperation was rated good in 63, fair in 13 and poor in 1. Several had developmental delays: 5 ADHD, 6 autism, 5 syndrome, 1 dyslexia, 1 marked prematurity, 1 cerebral palsy and another with epilepsy. Refractive amblyopia had been diagnosed in 21, accommodative esotropia in 6 and nystagmus in 2.

Continuous linear variables visual acuity, inter-eye acuity difference and stereopsis were converted to logarithm and then compared with exam findings versus screening by EyeSwift® and PDI Check (Figure 2). Students did not lean forward with the dynamic presentation of PDI Check, however 14 of the students did lean forward for static acuity screening on EyeSwift®; the results noted these as “invalid.”

Figure 2 Vision games correlation and Bland Altman Plots. Top row (green) with NovaSight EyeSwift® and bottom rows (Orange) PDI Check on autostereoscopic Nintendo 3DS. Columns are compared examination logMAR visual acuity, intereye difference logMAR acuity and stereopsis (log arcseconds). For each row, upper is linear regression and lower is Bland Altman Plot.

Table 1 compares the AAPOS 2021 and also the Bosque-Hunter amblyopia risk factors for which the birefringent vision screening device is designed to target. Figure 3 shows ROC curves validating the three non-refractive devices compared to the target conditions. For the full AAPOS 2021 target, sensitivity/specificity/PPV for EyeSwift® was 82%/56%/77%, PDI Check was 75%/64%/84% and blinq was 75%/48%/78%. For the Bosque-Hunter Referral-Warranted target, EyeSwift® was 74%/62%/37%, for PDI Check 69%/69%/65% and blinq 82%/39%/37%. The area under the curve (AUC) targeting AAPOS 2021 was 0.70 for EyeSwift®, 0.72 for PDI Check and 0.59 for blinq. Targeting the Bosque-Hunter Referral-Warranted, AUC for EyeSwift® was 0.73, PDI Check was 0.68 and blinq was 0.59.

Figure 3 Receiver operating characteristic (ROC) curves validating EyeSwift®, PDI Check and blinq. The exam outcome targets are AAPOS (≥4-year-old 2021) amblyopia risk factors (ARFs) refractive only (upper right) and refractive or strabismus or visual acuity (upper left), strabismus or amblyopia (lower left) and “referral-warranted” diagnosis (lower right). Instrument referral criteria are identified in upper right ROC curves. Prevalence of AAPOS 2021 ARFs is high at 77%.

Screening time: The time to screen students was 375 ± 102 s for Eye Swift® and 96 ± 19 s for PDI Check. The blinq usually took less than 15 s to yield a definite initial result. Definite initial result (24 pass and 32 refer) was 68% for blinq with one “inconclusive” and 3 unable to cooperate. “Timed Out” interpretations were 17 (21%) of initial blinq results. Following manufacturer’s protocol for a “timed out”, repeat screening yielding 4 more “refer,” no “pass” and 13 repeat “timed out” interpretations.

Laterality prediction: The number of patients with Worth-dot suppression and visual acuity consistent ocular dominance was 26. For patients with greater than 1 logMAR line difference between eyes EyeSwift® had 44 with 19 of 26 (19/26 = 73%) correctly matching Exam and 2 mixing dominance prediction. EyeSwift® Worth Dot near suppression declaring laterality in 17 students of which 12 of the 26 (12/26 =46%) correctly predicted exam side and 1 mixed sides. Using 2 logMAR difference in near visual acuity for PDI Check, 65 had a laterality; 20 predicted the 26 (76%) correctly matching the examination and 6 mixing sides. Rebion blinq declared referral side for 53 students of which 26 of 26 (100%) correctly matched the patients with defined exam side.

Seven of these students demonstrated color vision deficiency on exam. With student-directed games, EyeSwift® (Ishihara plates) was able to screen with 86% sensitivity and 84% specificity. The PDI Check monocular isoluminance-gray trichromatic game predicted color with 100% sensitivity and with a specificity of 81%.

Discussion

Three different non-refractive vision screening devices each demonstrated advantages and disadvantages when applied to a high-risk cohort of pediatric students tested as if they were in a school nurse's office.

The 2021 AAPOS Uniform Guidelines for vision screen validation provide not only age-based refractive cut-offs but also strabismus and visual acuity amblyopia risk factor levels.¹⁷ David Hunter, the co-developer of the blinq has recognized the deficiencies of purely refractive amblyopia risk factors.¹⁸ He helped develop a 24-point rubric for determination of “Referral Warranted” pediatric eye disease targeting actual amblyopia and strabismus.¹² In our experience, when the two outcomes, AAPOS 2021 and Bosque-Hunter, were targeted by each device in our high-risk cohort, a similar but not identical set of ROC curves was generated.

Early treatment age is advantageous for effective amblyopia therapy. The PEDIG Amblyopia Treatment Study long-term follow-up found 33% better eventual acuity gain for those enrolled age 3–4 years than those enrolled age 5–6 years.¹⁹ Photoscreened patients may have even better treatment acuity when detected age 1–2 years compared to those photoscreened age 3–4 years.²⁰ The three non-refractive devices in this study do not perform well under the age of 3 years; the video games required more understanding and cooperation while the tiny fixation target in blinq was not obvious enough to trigger consistent, sustained fixation in toddlers. Therefore, these three non-refractive devices will not yet be expected to provide reliable screening for very-early amblyopia detection. Whether this matters in terms of optimal treatment of eccentric fixation, dense strabismic amblyopia remains to be determined.

EyeSwift® is a novel vision video game with eye tracking. This study far from exhausted all of the subtests it can present. Almost all of our students were able, with minimal coaching, to follow the in-game instructions including putting on and taking off shutter and anaglyph goggles. Eyetracking accurately noted when students were too close for valid interpretations; Nova Sight is already developing paradigms to warn and direct participants to be more compliant for more reliable measurement. EyeSwift® is on an articulating stand. Care should be used to avoid knocking it off a desk, table or cart; our device was tough enough to function normally after one such fall. EyeSwift® has battery power for limited testing but it is optimally utilized with A/C power and internet connectivity.

PDI Check works on the battery-powered, autostereoscopic barrier screen of Nintendo 3DS technology so goggles are not required for monocular or stereo presentation. The clam-shell case of the game is very sturdy and easily charged with an inexpensive USB cord. Further calibration efforts with more color deficient and amblyopia patients can further improve so deviation in mean Bland Altman results between device and exam will be minimized. PDI Check is pursuing other autostereoscopic device options. The 4-quadrant, forced-choice presentation of optotypes did not tempt the students to lean too close to the screen.

Rebion blinq is moderately tall and nearly spherical with a narrow base so ample care was taken in earthquake-prone Alaska to avoid having the intricate device roll on the floor. It retains a battery charge for a dozen or more screenings but should remain close to AC charge and ideally Wi-Fi connectivity. The touch screen is clear and helpful but encourages holding the device at the screener’s waist-level; some younger children gazed up at the screener’s face interrupting data acquisition. COVID-19 face masks may have interfered with initial distance calibration for each screening. The tiny, orange fixation light inside blinq’s small, round dark gray front window is not easy to fixate for the 5–25 seconds required for optimal interpretation. Blinq was the fastest of the non-refractive screening devices and scored 100% in determining the laterality of the suppressed, amblyopic eye. However, as with many pediatric vision screening devices, not all interpretations are identical on repeat screening for the same student. Two of our initial “pass” had a “refer” follow screen and one of the “refer” had a “pass” when the blinq was repeated (we counted the score from the initial finding). In addition to our 32 initial definite “refer” and 24 “pass”, 22% of our initial interpretations were “timed out.” On repeat screening, 4 of those 17 had “refer” interpretations. In an older student with residual refractive amblyopia risk factors, successful patching therapy may improve foveation such that blinq may “pass” a student with persistent ARFs. As such, validation on blinq may be optimized with younger, treatment-naïve patients. Further validation with blinq in children with non-enhanced risk factor prevalence is warranted and will help Rebion balance specificity with the high sensitivity produced in earlier studies.^12,21

Each of these devices could fit both space- and time constraints of a school nurse or pediatrician's office and would provide an array of additional vision-related diagnostics for conventional screening. Updated EyeSwift® viewing distance detection could advise patients to move back or forward and resume testing for even better accuracy; the manufacturer is already working on this.

Strength and Weaknesses of the study: A strength is that each device was used concomitantly with confirmatory exam utilizing strict validation methods. The study cohort had diverse racial and ethnic mix and a variety of developmental delays. The reliability for these devices giving a result in patients over age 4 years regardless of developmental delay and level of cooperation was fairly good. A weakness is relatively small sample size so subgroup analysis lacked power to address certain differences such as validation in younger students under treatment-naïve conditions. An additional weakness is that the confirmatory examiner was not always blinded to the interpretations of the device screening. The two video games have additional functionality that was not utilized in this validation test. The variety of instrument referral criteria for the video games was far from exhaustive; many other visual functions could have been selected for EyeSwift®, and two different bilateral color tests were not tested on PDI Check. Fourteen of our students had limited validity for their acuity screening due to leaning forward with our study design presumably not requiring a busy school nurse to monitor and warn every process. The eye tracking kept up with all the student fixation behavior and head position, however.

Conclusion

Students as young as Kindergarten-age were able to complete screening with three non-refractive devices from which regressions, Bland Altman Plots and ROC curves demonstrated good but not perfect validity. Residual risk factors despite successful amblyopia therapy in older patients likely reduced apparent performance. Validation that is exclusively refractive should be avoided with these tools that target amblyopia associated with strabismus and visual acuity loss.