Validation of a Simple-to-Use, Affordable, Portable Wavefront Aberrometry-Based Auto Refractometer in a Paediatric Population

Introduction

Uncorrected Refractive Error (URE) is the leading cause of visual impairment worldwide. At least 2.2 billion are affected of which nearly half could have been prevented or have yet to be addressed.¹ Globally growing rates of URE, especially in the paediatric population, necessitate a scalable vision screening strategy for diagnosis and timely management.² Early screening for RE is important as a child may not be able to express poor vision or a unilateral vision loss can be missed. Uncorrected or inaccurately corrected RE can lead to undesirable consequences, such as subnormal vision, amblyopia, asthenopia, binocular vision anomalies, and poor academic performance affecting the quality of life. Additionally, vision impairment poses an enormous global financial burden to an estimated USD 244 billion annually in productivity losses due to uncorrected myopia alone.¹ Fueling this public health problem is the low number of eye health professionals to screen a large population, especially in low-to-middle-income countries, which have a high burden of vision impairment and blindness.³ Some mass vision screening approaches utilise school teachers, nurses and other health workers trained to screen for eye conditions.^4,5 A reliable refraction tool will be beneficial in such settings.

URE can largely be corrected with a simple solution such as spectacles.⁶ However, currently available bulky, expensive, desktop auto refractometers are cumbersome when screening children. A portable, handheld refractometer, which is robust, portable, easy-to-use and quick, is ideal for screening, especially in young children. There are several clinically validated auto refractometers available primarily in adults.^7–10 In children, studies have reported the measurements without cycloplegia to underestimate hyperopia or overestimate myopia. In comparison with adults, children usually have larger pupils, a higher amplitude of accommodation, and more varying fixation. Therefore, the results of validation of an auto refractometer obtained in an adult population may not be directly applicable to children.

InstaRef R20 (Remidio Innovative Solutions, India) is a handheld, auto refractometer based on Shack Hartmann aberrometry technology that fits the needs for pediatric use.¹¹ The device is portable, and low weight (400g). It is a fraction of the cost of a desktop auto refractometer and is on the lower end of the price range of available portable systems, making it an affordable device with several advantages. Minimal training is sufficient to obtain accurate readings. The infrared imaging feedback allows simple centration of crosshairs over the pupil. Additionally, the device has features such as a built-in audible tilt warning system to ensure correct alignment and accurate cylindrical measurements. The device allows the patient to look at a far distance using the other eye, relaxing accommodation and offers a good range of refractive error measurement (−10.0 D to +10.0D sphere and −5.0D to +5.0D of cylinder). Finally, the readings are taken quickly within three seconds and the results can be immediately printed using Bluetooth-enabled printers or accessed on an EMR system adding to the utility of the device. To ensure good accuracy, an average of three readings is obtained within seconds. We aim to compare the performance of this novel device against gold standard pre- and post-cycloplegic manual retinoscopy and subjective refraction for measuring refractive errors in a paediatric population.

Methods

A prospective study was conducted to evaluate the performance of a newly developed portable auto refractometer against the standard objective retinoscopy and acceptance (pre- and post-cycloplegia) in a paediatric population. The study was conducted in Sankara Eye Hospital, Bengaluru, South India between 15th October and 31st October 2021. Assent from the children and written informed consent were obtained from the guardians of all the subjects who participated in the study. The study was approved by the Institutional Ethics Committee, Sankara Eye Hospital, Bengaluru, India (Approval number: SEH/BLR/EC/2O21/47) and adhered to the tenets of the Declaration of Helsinki.

Inclusion and Exclusion Criteria

The study included subjects 8 to 18 years of age, with BCVA of 20/20 in each eye, and refractive error within ±10.00D for sphere and ±5.00D for cylinder. Excluded were subjects, with the presence of significant media opacity, ocular conditions affecting vision, contraindications to cycloplegic agents, history of eye surgery, active infections or inflammation, and current usage of any drugs that can affect visual acuity.

Sample Size Calculation

For the current study, assuming 90% power, 95% confidence to a difference of 0.5 D between the two instruments, the required sample was 132 participants.

Methodology

All subjects underwent a comprehensive eye examination. An experienced optometrist (>5 years, examiner 1) performed gross clinical evaluation as a pre-screening followed by an objective retinoscopy of the right eye (with the left eye fogged) and the left eye. Subjective refraction was performed next, and the best-corrected visual acuity (BCVA) from this procedure was recorded. A trained optometrist, examiner 2 measured the refractive error using the hand-held study device InstaRef R20 in a different room and masked to examiner 1 and other clinical findings. Proper alignment was ensured by holding the device at the subject’s eye level, and the crosshair on the device screen was centred to the pupil. Right eye measurement was taken with the patient being instructed to view the distant external visual target with the left eye kept open. An average of three readings was obtained. The same process was repeated for the left eye. A unique identification number was assigned to all the participants included in the study, and refractive error measurements were recorded accordingly. Dilation was done by instilling 0.5% proparacaine, followed 30 seconds later with 1% cyclopentolate eye drops (cycloplegic agent) under the supervision of the attending ophthalmologist. Forty-five minutes later, pupillary diameter and light reflexes were checked. Pupillary dilation to 6mm or greater with absent light reflexes was considered adequate cycloplegia. A detailed slit-lamp examination (pre- and post-dilation), posterior segment evaluation by an ophthalmologist and post-dilated retinoscopy and subjective refraction by examiner 1 was performed. Those with any ocular abnormality or other exclusion criteria noted after dilation were excluded. The methodology followed is represented as a flow diagram in Supplementary Figure 1 along with the image of InstaRef R20 device.

In a subset of 20 participants, the repeatability of InstaRef R20 measurements was measured by obtaining three automated measurements under non-cycloplegic conditions similar to previous studies.^12,13 Data were collected from both eyes (total of 40 eyes) of the subject, but only data from the right eye are analyzed and presented here to avoid statistical inconsistencies.¹⁴ Intraclass correlation coefficient (ICC) was calculated and values below 0.5 were considered poor, between 0.5 and 0.75 as moderate, between 0.75 and 0.9 as good and any value above 0.9 as excellent repeatability.¹⁵ The differences between each of the three test readings were averaged to determine the mean difference for sphere, cylinder, spherical equivalent, and the coefficients J0 and J45. Repeated measures analysis of variance (ANOVA) was used to detect differences between the three measures.

Statistical Analysis

All the refraction measurements were transformed using Fourier vector decomposition to spherical and cylindrical scale values using the below formulae. The results are represented in the format of M, the spherical equivalent of refraction (in dioptres), and J0 and J45, the two Jackson’s cross-cylinder astigmatism at 0 degree axis (regular astigmatism) and 45 degree axis (oblique astigmatism). The spherical equivalent was calculated as sphere value + cylindrical value/2. The following equations were used to calculate the vectors:

J0 = (-C/2) cos(2a), J45 = (-C/2) sin(2a) where C = cylinder power, a = axis of astigmatism.

The agreement between the autorefractor, manual retinoscopy and subjective refraction measurements was evaluated using a Bland-Altman (BA) analysis with 95% limits of agreement (LOA) on each power vector component. Agreement within thresholds of 0.5 D for M, J0, and J45 was evaluated. In a BA plot, the mean difference closer to zero indicates no bias in the estimate or no difference in the mean measurements between the devices. Narrower the 95% limits of agreement with a diminished scattering of the points with points lying relatively closer to the line representing mean bias, the better the agreement. The intraclass correlation coefficient (ICC) was calculated for the repeatability of InstaRef R20. Any p-value <0.05 was considered statistically significant. All data were entered, and statistical analysis was performed using Microsoft Excel and XLSTAT 2022.

Results

The study included 132 subjects with a mean age of 12.31 ± 3 years (8, 18), 53% (n = 70) less than 12 years of age, 51.5% (n = 68) girls, and 56% (n = 74) were previous spectacle users. The average values of the sphere, cylinder, M, J0 and J45 of different methods, ie, InstaRef R20, pre- and post-cycloplegic retinoscopy and subjective refraction (acceptance) are presented in Table 1.

Table 1 Data Showing Mean and 95% Confidence Interval (CI) of Refractive Outcomes (Sphere, Cylinder, M, J0 and J45) Measured Using InstaRef R20, Pre- and Post-Cycloplegic Retinoscopy and Subjective Refraction (SR)

InstaRef R20 vs Manual Retinoscopy (Pre- and Post-Cycloplegia)

The mean values for M, J0 and J45 obtained using InstaRef R20 were closest to pre-cycloplegic retinoscopy values. The difference was not statistically significant (Tables 2 and 3). The mean values for M of InstaRef R20 in comparison to post-cycloplegic retinoscopy were statistically significantly different (mean paired difference of −0.22D, 95% LOA 1.17 to −1.62 D, p = 0.002). A statistically significant difference was noted in M values between pre- and post-cycloplegic retinoscopy (mean paired difference of −0.16 D, 95% LOA −1.13 to 0.81 D, p = <0.001). There was no significant difference in J0 and J45 values. InstaRef R20 agreed within 0.5 D of pre- and post-cycloplegic retinoscopy in 67% and 61%, respectively, of the eyes for spherical equivalent power. Tables 2 and 3 show the proportion of values within ±0.50 D for M, J0 and J45 values between InstaRef R20 and pre-post cycloplegic retinoscopy.

Table 2 Paired Mean Difference and Agreement Within 0.50D for Spherical Equivalent (M) Values of Pre- and Post-Cycloplegic Retinoscopy and Subjective Refraction (SR) Compared to InstaRef R20 (Pre-Cycloplegic)

Table 3 Paired Mean Difference and Agreement Within 0.50D for J0 and J45 Values of Pre- and Post-Cycloplegic Retinoscopy and Subjective Refraction Compared to InstaRef R20 (Pre-Cycloplegic)

InstaRef R20 vs Subjective Refraction (Pre- and Post-Cycloplegia)

The mean values for M, J0 and J45 obtained using InstaRef R20 were closest to pre-cycloplegic subjective refraction (acceptance) values in all age groups. The mean values for M of InstaRef R20 in comparison to post-cycloplegic SR were statistically significantly different (mean paired difference of −0.22 D, 95% LOA 0.96 to −1.47 D, p = <0.001). A statistically significant difference in M values was noted pre- and post-cycloplegic SR (mean paired difference of −0.16 D, 95% LOA −1.07 to 0.75 D, p = <0.001). There was no significant difference in J0 and J45 values. InstaRef R20 agreed within 0.5 D of pre- and post-cycloplegic SR in 70% and 63%, respectively, of the eyes for spherical equivalent power. Tables 2 and 3 show the proportion of values within ±0.50 D for M, J0 and J45 values for the difference in instaref vs pre-post cycloplegic subjective refraction.

Tables 2 and 3 present the mean difference and 95% limit of agreement (LOA) for M, J0 and J45 values of InstaRef R20 when compared to pre- and post-cycloplegic retinoscopy and subjective refraction. Results of the agreement between InstaRef R20 and pre-post retinoscopy and SR for M, J0 and J45 are presented in Bland-Altman plots (Figures 1 and 2). All the paired mean differences remained within the range of ±2 SD. Pre-cycloplegic Instaref values for M were noted to have the narrowest LOA when compared against pre-cycloplegic SR (2.16D) and the widest LOA when compared against post-cycloplegic retinoscopy (2.79D). In a sub-group analysis, comparing the study device against retinoscopy and SR based on the type of refractive error (within ±0.50DS (n = 45), Myopia < −3.00D (n = 47), myopia > −3.00D (n = 28) and Hyperopia (n=12)), the mean paired difference was clinically insignificant (<0.50D) for all the M, J0 and J45 values. A similar clinically insignificant difference (<0.50 D) in mean paired difference for all the vector values was noted based on age (less than (n = 70) and greater than 12 years (n = 62) group) and spectacle use (previous user (n = 74) and new user (n = 27)).

Figure 1 Bland-Altman plots showing bias and 95% limit of agreement comparing InstaRef R20 with pre- and post-cycloplegic retinoscopy for M values (A and B), J0 (C and D) and J45 (E and F).

Figure 2 Bland-Altman plots showing bias and 95% limit of agreement comparing InstaRef R20 with pre- and post-cycloplegic subjective refraction (SR) for M values (A and B), J0 (C and D) and J45 (E and F).

Repeatability of InstaRef R20

The refraction of 20 subjects was repeated three times. The average difference between each of the three repeated measures was −0.09 ± 0.14D and 0.10 ± 0.17D for sphere and cylinder values, respectively. The ICC demonstrated good to excellent repeatability (0.843 to 0.998) for Sphere, Cylinder, and M values. Table 4 summarizes the repeatability analysis of three consecutive readings of InstaRef R20 using the Intra-class correlation (ICC) test.

Table 4 Repeatability Analysis of Three Consecutive Readings of InstaRef R20 Using Intra-Class Correlation (ICC) Test (n = 20)

Discussion

The prevalence of undiagnosed and uncorrected refractive error in children is on the rise, and there is a huge disparity between the number that needs screening and available eye care professionals. Automating the measurement process allows for screening to take place outside of a specialist context. Autorefractometers are a rapid, accurate and reliable tool for objective measurement that can be used as a starting point for subjective refraction, eliminating the need for cumbersome manual retinoscopy.¹⁶ The results from our study show that the spherical equivalent (M) and cylindrical components J0 and J45 of a newly developed portable, handheld device have clinically acceptable differences and acceptable agreement compared to standard manual retinoscopy and subjective refraction (SR) both in pre- and post-cycloplegic states in the paediatric population.

The first generations of auto refractometers were heavy, and table mounted. The validity and repeatability of the use of table-mounted autorefraction devices to determine refractive errors have been widely studied in children.^7–10 The current device is a portable, lightweight, simple to use, low-cost, Shack Hartmann aberrometry technology-based auto refractometer. In a review of portable wavefront aberrometry-based autorefractors, it was emphasized that the accuracy of the device is well understood when compared against the gold standard technique, ie retinoscopy and the patient’s final subjective acceptance.¹⁷ It is also well known that children possess a strong accommodative power and hence for an accurate refractive error estimate accommodation needs to be relaxed. Hence, we used two benchmarks to evaluate the performance of the device pre- and post-cycloplegic retinoscopy and pre-post subjective refraction (patient acceptance). Additionally, we also evaluated the repeatability of the tool in this cohort and found the repeatability across the 3 readings for sphere, cylindrical and spherical equivalent values to be excellent.

As expected, due to the accommodative effect, the spherical equivalent values (M) of retinoscopy and study device in a non-cycloplegic state showed a tendency towards minus overcorrection.¹⁶ However, this was not found to be clinically significant. The M, J0 and J45 values of the device in the Indian population paediatric group showed no significant difference compared to standard pre-cycloplegic retinoscopy. A majority of the M, J0 and J45 values (67%, 78% and 80% respectively) agreed within ± 0.50D. In InstaRef R20, the effect of accommodation is controlled as the subject can view a distant object with the non-testing eye during measurements. Another wavefront-based auto refractometer validated in the paediatric population by Gil et al, 2022, using an open-view binocular handheld device and validation studies using the same wavefront device reported similar results.^18–20 Rosenfield and Ciuffreda validated a portable Shack-Hartmann wavefront aberrometer that can be attached to a smartphone against retinoscopy and open-field auto refractometer.²¹ They too found no significant difference in refractive measurements in the paediatric population and recommended the use of this device in vision screenings and as a starting point for refractive assessment in optometry clinics. Table 5 summarises the comparison of spherical equivalent and cylindrical values of wavefront-aberrometry-based handheld, portable auto refractometers against other techniques validated in paediatric population. A recent study (2021) which compared four different refractive error measurement techniques among 234 participants reported wavefront aberrometry-based auto refractometer performance to be better under non-cycloplegic conditions, and the difference was within ± 0.50D for M, J0 and J45 values against standard retinoscopy than when compared to other auto refractometer technologies.¹²

Table 5 Comparison of Different Portable, Handheld Wavefront-Aberrometry-Based Auto Refractometers Under Non-Cycloplegic Conditions Validated in Paediatric Population

Post-cycloplegic retinoscopy is commonly performed in children, especially in a hospital-based setting. We found a statistically significant difference only in M values of the device when compared to post-cycloplegic retinoscopy, but clinically the mean paired difference was acceptable (mean paired difference of −0.22D, 95% LOA 1.17 to −1.62). Additionally, 61% of M values compared to post-cycloplegic retinoscopy were within ± 0.50D. A similar difference was noted in other studies.^18,21 Gil et al reported the average M values under the non-cycloplegic state to be −0.35D and –0.41D for the open-view device and tabletop model auto refractometer, respectively, and found an average hyperopic shift to 0.06D and –0.01D under the cycloplegic state.¹⁸ However, unlike ours, the study used two different cohorts for cycloplegic and non-cycloplegic conditions. Rosenfield and Ciuffreda found an interesting finding, ie, doubling of the M value’s limits of agreement between pre- and post-cycloplegia (post-cycloplegic values greater than pre-cycloplegic values, hyperopic shift).²¹ The authors report it could also be due to the centration of pupils during the assessment, particularly under dilation conditions. A similar pattern was observed in the young adult group evaluated using the same device.¹³ Children usually have large pupil diameter making pupil centration important and frequently need cycloplegic evaluation for an accurate estimate. The current study device has infrared imaging feedback that ensures pupil centration along with a built-in audible tilt warning system for high accuracy of cylindrical values. It is pivotal to validate the device against gold standard retinoscopy under both cycloplegic and non-cycloplegic conditions, especially in the paediatric population.

In a subgroup analysis on the type of refractive error in the current study, no clinically significant difference was noted between the study device when compared against pre-cycloplegic retinoscopy and SR for M, J0 and J45. There was a good distribution of sample size from ±0.5D to >-3D myopia. Another study that used a wavefront aberrometry-based auto refractometer in comparison against pre-cycloplegic retinoscopy in children also found no difference in values in myopic children.⁷ However, there were differences noted in a few studies without cycloplegia with only a few showing acceptable agreement as seen here.^7–10 Several factors such as age (younger or older children), magnitude and definition of refractive errors, optics of the auto refractometer, study design, etc., can influence the above results. We included 8 to 18 years and found no clinically significant difference in refractive error across these age groups.

In most studies, auto refractometers are validated only against retinoscopy, the gold standard for objective measurement. While this is essential, comparing the measurements against the final patient acceptance would be invaluable to move towards a more simplified method of spectacle prescription. Although retinoscopy by skilled licenced practitioners still holds as the gold standard procedure, auto refractometers can be especially useful in large population-based refractive error screening programs in children using non-eye health professionals such as school teachers, nurses and health workers. In the current study device, except for the post-cycloplegic acceptance of M values, J0 and J45 values were not significantly different. Even for M values, the difference between the study device and subjective refraction (acceptance) was within ± 0.50D in a majority of the population (70%). This initial data suggest that the measurements from this device can potentially be used for prescribing spectacles in a majority of the paediatric population. However, this will require further validation in large-scale community-based studies. A similar comparison was performed by Gil et al using open-view auto refractometer M values against SR and found an agreement within 0.5D of spherical equivalent values in 71% of subjects under non-cycloplegic state and 70% of subjects under cycloplegic conditions.¹⁸ Table 5 summarises the comparison of spherical equivalent and cylindrical SR values of wavefront-aberrometry-based handheld, portable auto refractometers against other techniques validated in paediatric population.

The use of these devices is not only in the clinic but could be a valuable addition in screening refractive errors in the community. The cost-effectiveness of devices has been established in an earlier study.²² Instaref could be considered in similar programmes in the future. For non-eye care personnel, we advocate a short training on optimum device utilization. Key components such as the description of the device, usability specifications, functional, operational aspects of the device and operational tips to measure accurate values will be provided to the users. Hands-on training with the device on emmetropes and those with refractive errors are given based on the setting. On average, four hours is sufficient to train a person who is previously a part of eye screening programs. For others, it depends on the level of experience.

On exploring the sources of error, there are a few areas of improvement that can be considered. While open-view design works well to control instrument-induced accommodation in a majority of children, accommodative errors still do occur. Pre-cycloplegic M values provided by Instaref agreed within 0.5 D of pre-cycloplegic SR in 70% of the subjects, which reduced to 63% when compared against post-cycloplegic SR at the same threshold. Emphasizing during training regarding distant target fixation and undertaking cycloplegic measurements can improve the accuracy further. Proper centration and training to correct alignment when the tilt warning system is triggered can improve the performance of cylindrical measurements (pre-cycloplegic J0 and J45 of Instaref agreed within 0.5 D threshold of pre-cycloplegic SR in 77% of subjects). Lastly, further optimization of the wavefront reconstruction can minimise instrument-induced errors.

There are a few limitations of the study that need to be considered. One is the inclusion of children older than eight years only. Those less than 8 years of age may have a highly fluctuating focusing system with a stronger effect of accommodation and hence will require independent validation. Secondly, the number of participants in some refractive error subgroups (hyperopes) was small and hence difficult to draw definitive conclusions. Thirdly, recording post-cycloplegic Instaref measurements would have provided further insight on the control of accommodation the device offers with an ability for a direct pre-post cycloplegic comparison.

Conclusion

This study evaluating a novel wavefront auto refractometer demonstrates that it is comparable to standard objective retinoscopy in the paediatric population. The measurements from this device can be used as a starting point for subjective acceptance. The device being simple to use, portable, reliable and affordable has the potential for large-scale community-based refractive error detection. This device can serve as beneficial to reduce the global burden of paediatric visual impairment arising from uncorrected refractive errors.