Development and Validation of a Web-Based Reading Test for Normal and Low Vision Patients

Introduction

It is a truism that reading is a fundamental activity in everyday life, especially in educational and workplace environments.¹ The ability to read and understand text is known as reading comprehension.^2,3 Several vision-related factors interfere with reading comprehension; among them, the refractive errors, the presbyopia, and the suboptimal macular function.^4,5 Reading directly reflects near vision capacity; therefore, it is included among the standard tests in a routine ophthalmological examination.

The Democritus Digital Reading Acuity Test (DDART) is a digital reading test that was developed and validated recently by our group, which calculates all prevalent reading parameters that evaluate the reading capacity of the patient.^6,7 However, since DDART is a Microsoft (MS) Windows^®-based executable program, it requires installation of MS Windows, which increases the overall cost of the test and excludes its installation in computers with other operating systems. Moreover, potential software upgrades of the underlying executable program cannot be universally applied to all computers that host the DDART. It becomes obvious that a web-based tool for a valid reading test would assist healthcare authorities and care providers in their effort to introduce a reading test to all beneficiaries of a National Healthcare System.

Within this context, the objective of this study was to develop and validate a web-based reading test that is based on the DDART that will provide all benefits of a web-based medical examination, and at the same time demonstrate comparable validity and reliability with the conventional computer-based application.

Methods

Setting

This was a prospective, comparative trial between the validated computer-based reading test DDART and the web-based reading test (wDDART) in order to evaluate the latter’s validity in a sample of normal and low vision subjects. Ethics approval was obtained by the institutional review board of Democritus University of Thrace, while written informed consent was provided by all participants. The study was conducted in accordance with the Declaration of Helsinki at the Department of Ophthalmology in the University Hospital of Alexandroupolis, Greece, between December 2020 and February 2021. Official registration number of the study is NCT04618224.

Development of wDDART

wDDART is a front-end application, using JavaScript, that does not require an application server. Its interface is user-friendly, requiring minimal training for its use (Figure 1). On the other hand, the operator’s interaction with wDDART mimics the conventional reading test examination in a clinical setting (Figure 2).

Figure 1 The initial screen of wDDART.

Figure 2 The control flow of the wDDART application (see the explanation of symbols used at the upper right corner of the figure).

Reading capacity with wDDART is evaluated by six sets of 19 sentences, with size logarithmically decreasing by a factor of 10^−0.1, from an initial logMAR of 1.3 up to –0.5 logMAR with constant step of 0.1. All six sets of sentences can be used interchangeably for the estimation of all major reading parameters: reading acuity (RA), maximum reading speed (MRS), critical print size (CPS) and reading accessibility index (ACC). However, wDDART introduces a series of advanced features that facilitate the testing procedure that are explained below:

Text Size Calculation and Calibration Subsystem

The accurate size and positioning of the displayed text are fundamental for the precise evaluation of reading parameters. By definition, the correct height H of a displayed character that corresponds to logMAR=0 (equivalently Snellen fraction of 20/20), has an angular size of δφ=5 arc minutes when viewed from distance D. The height H is defined over the main body of the character, often called x-height, without considering the ascending and descending height.^8,9 Thus, the height H for any logMAR viewed at distance D is given by:

(1)

In the case of a near-vision test at a viewing distance D=40 cm, the height H of a character corresponding to logMAR=0 is equal to 0.58 mm. In principle, the font size in points (pt), corresponding to height H can be calculated by the definition of 1 inch=72 points.¹⁰ However, the actual displayed size of the text may deviate significantly due to operating system, or browser display settings.

To overcome the aforementioned issue, a text size calibration feature has been implemented in wDDART, consisting of the following steps:

1. The user also sets the distance at which the examination will take place (action “1” in Figure 3) – the default distance value is equal to 40 cm.
2. The user selects the logMAR to be used for the calibration (action “2” in Figure 3). The default value is logMAR=1.3. This step is useful in cases where the size of the screen does not allow the display of the full sentence in high logMARs at certain distances; thus, the examination may start from a smaller logMAR.
3. The system calculates the appropriate length Q of the test sentence and the user is prompted to interactively resize the text by using the scrollbar (noted as “3” in Figure 3), until its measured physical length (in cm) on the screen approaches the correct value.

Figure 3 The text calibration screen.

The length Q of the sentence is more accurate and convenient to measure, instead of character height H, since Q ≫H, and Q and H are linearly connected, according to Q=17.3·H, or Q=20·H – w_space, where w_space is the width of the space character.^11,12

The proposed calibration can be performed very easily without any expert knowledge. It is required only when the display settings of the operating system or the browser are changed. In addition, sample characters for all logMARs (from 1.3 to −0.5) are displayed with the current text calibration settings, to indicate to the user the appearance of each logMAR sentence using the specific computer screen at the selected patient-screen distance.

Computer Vision-Based Estimation of Patient’s Distance

The accurate assessment of the distance between the patient and the screen is essential for reliable assessment of the reading parameters. In wDDART, the distance between the camera (computer screen) and the face of the examinee is continuously estimated using computer-vision-based techniques. During the examination, the patient is being positioned at the correct distance according to the estimation provided by the system. Furthermore, the average distance for each logMAR is reported in the results, allowing the examiner to confirm that the examination was conducted within the appropriate distance range.

It is well known that such a calculation is based on camera calibration, which usually implies recovering the intrinsic (focal lengths, coordinates of focal point and deformation coefficients) and the extrinsic (position, direction of view and rotation round the direction of view) camera parameters. Although this procedure is well documented and automated up to a degree, it requires the use of printed patterns and some expert knowledge. Furthermore, the physical size of the imaged object (in this case, the patient’s face) is also required. A much simpler, yet accurate, approach is employed in the proposed work, based on an efficient face detector.¹³ The proposed camera calibration can be performed by a non-expert very easily.

The simple pinhole camera model is utilized, which is very accurate for a typical web camera.¹⁴ Let w denote the apparent width of the imaged face (in pixels) and L the physical width of the face (in cm). L is unknown and different for each examinee. Let us assume that the face stands at a distance d₁ from the camera and is imaged with a length of w₁ pixels and in another instance, it appears as w₂ pixels in the image when imaged at a distance of d₂. According to the camera pinhole model, if f is the focal length of the camera, then the following hold:

(2)

The geometric setup and the definition of the symbols are provided in Figure 4A. By substituting the constant product L·f, the next relation is obtained:

(3)

Figure 4 (A) The concept of face-camera distance calculation, (B) an exemplar screenshot of the camera calibration screen, with automatic face detection overlaid.

According to the above, if the apparent width of the face in the image can be measured for a known distance d₁, then any distance d₂ can be estimated, provided that the apparent width w₂ on the image can also be measured.

The publicly available deep learning architecture using TensorFlow^TM that has been pre-trained to detect faces in still images and videos is utilized in order to automatically detect the imaged face.¹⁵ As demonstrated in Figure 4, the detected face is in the form of four contours outlining the jaw, two eyes and the central axis of the nose. Typical example of a detected face is provided in Figure 4B. Thus, the apparent width w of the face (in pixels) can always be measured.

Based on the above, when a patient starts the examination, camera calibration can be performed in a straightforward manner: the patient is asked to remain at a predetermined distance, while his/her face is being detected (normally a fraction of a second). Figure 4B shows the camera calibration screen with a typical example of its use.

After the activation of this feature, the algorithm monitors the distance at predetermined intervals (typical values between 100 and 500 ms) and stores the average distance of each logMAR size text that have been read by the patient. The algorithm is initiated and stopped when pressing the “NEXT” and “STOP” button, respectively. The estimated distances for each logMAR size are reported in the results, in order to be taken into consideration in the test evaluation.

Automatic Calculation of Patient Reading Times

wDDART introduces an automatic calculation of reading times using a digital signal processing algorithm to measure the duration of the talk and post-talk delays, formerly described by our group,⁶ similarly to the approach of Radner et al.¹⁶ In detail, the patient’s voice is recorded by a standard microphone with sampling frequency f_s, set in the operating system’s settings (default value 44.1 kHz). Thus, a digital, signed voltage signal x(n) is generated. The recording is automatically initiated as soon as the current sentence appears on the screen and it is terminated when the STOP button is pressed. Then, the following algorithm is applied. The raw signal is down-sampled by summing consecutive blocks of b samples, to reduce memory requirements that may otherwise interfere with the execution of javascript, especially in low-end computers:

(4)

Subsequently, a threshold T is applied to signal y(n), to generate the binary signal z(n) that has only two values: 0 (background noise) and 1 (patient talk). T is set equal to 10% of the maximum absolute value of the patient’s reading recording of the first slide.

(5)

Morphological opening (erosion followed by dilation) is applied to z(n), to remove short duration sounds (less than 0.1 s) and the following time quantities can be easily calculated:

• Total duration, equal to total number of samples divided by the sampling frequency (= total number of x(n) samples/f_s)
• Post-talk delay: the time between the end of reading and the end of sound acquisition, calculated as (N₀ - n₀) · b/f_s, where n₀ is the last sample n₀, such that z(n₀) >0.

The talk duration is easily calculated

(6)

This parameter is necessary for reading speed estimation, as described in the next subsection. The waveform x(n) of a typical acquired sound during patient read out of a random sentence is shown in Figure 5A. The processed signal y(n) (blue curve) and the signal values considered as non-reading are plotted in Figure 5B, in blue and red color, respectively. The post-talk delay is graphically displayed.

Figure 5 (A) Acquired sound x during patient read out of a random sentence. (B) The processed signal y (blue curve) and the signal values considered as non-reading (red color). The post-talk delay is graphically displayed.

Calculation of Reading Parameters

wDDART introduces an automatic calculation of all prevalent reading parameters described below. The digital signal processing algorithm detects the patient’s reading times and the operator enters the number of reading errors detected in each sentence. Reading speed (RS) for each sentence is calculated in words per minute (wpm) by the formula:

(7)

The calculation of wDDART’s reading parameters is performed at the end of the examination, as follows:

• Reading acuity (RA), measured in logMAR, is calculated as:

(8)

where N_s is the number of sentences read and N_err the total number of errors.

• Critical print size (CPS), defined as the smallest print size (measured in logMAR) that has been read with RS no less than the average RS of the larger logMAR sentences minus 1.96 × standard deviation (SD) of the RS of these sentences. More formally, CPS is the smallest logMAR (equivalently the largest k sentence index) that satisfies the following:

(9)

where RS_k is the reading speed of the k^th sentence, and .

• Maximum reading speed (MRS), defined as the average RS of the sentences larger than the CPS, equal to the value of μ in Eq.(9). Thus, MRS is an indication of the patient’s reading speed when reading is not inhibited by small print size.
• Reading accessibility index (ACC), defined as the mean RS of print sizes up to 0.4 logMAR at 40 cm, which correspond to typical text sizes encountered in realistic situations of daily living, divided by 200 wpm.

The results are displayed in tabular form in html format and can be exported into other formats (pdf and MS Excel) to facilitate both clinical use and research purposes. The patient data, examination settings (distance), the information for each read sentence (reading duration, number of errors, reading speed and patient distance) are included in the exported results. A semi-logarithmic plot of the reading speed as a function of logMAR is also generated, with the calculated CPS indicated as a larger circle on the graph. Finally, the resulting values of RA, CPS, MRS, and ACC are also displayed. An example of a typical examination result in html format is shown in Figure 6.

Figure 6 A typical example of wDDART’s output (html format).

Validation of wDDART

wDDART reading parameters (RA, MRS, CPS, ACC) were compared with the corresponding ones from the computer-based DDART reading test in a sample of study participants who were recruited from the outpatient service of our Department in a consecutive-if-eligible basis. Inclusion criteria included a) adults older than 18 years old, b) fluency in written and verbal Greek language. Exclusion criteria included a) difficulty in understanding the objectives of the study, b) neurologic or mental diseases that interfered with the reading process. wDDART’s test–retest reliability for all reading parameters was evaluated in a 15-day time window, using a different set of sentences to avoid the memory effect.

Further to the reading parameters, the accuracy of the wDDART’s automatic estimation of patient’s distance was evaluated, as well, in the following way. Patient’s distance from the screen was objectively measured using a laser distance meter (Stanley TLM99s Laser Distance Measurer, Towson, Maryland, USA) and compared with the corresponding distance values estimated automatically by the wDDART for distances between 35 cm and 100 cm in a 5 cm step (Figure 7).

Figure 7 Typical examples of face detection at different distances (40, 70 and 100 cm), for two individuals with and without spectacles.

Examination Technique

Reading capacity of all participants was examined in the same consistent way, first by the DDART and then by the wDDART. Regarding the DDART reading assessment, a portable computer with 13.3-inch screen diagonal with native resolution of 3840×2160 and a pixel density of 331.3 ppi was used. Following DDART examination, each study participant responded to the wDDART using a similar computer setting. The exact same environmental lighting conditions were applied, while the brightness of the computer screen was set to 200 cd/m², according to the portable Extech Lux Meter EA30 (Extech Instruments Corporation, USA.).

Statistical Analysis

A power of 0.8 at the significance level of 0.05, for an RA-effect size of 0.29 would be achieved using 96 participants, according to an a priori power analysis. The Shapiro–Wilk test was utilized to assess the deviation of the calculated reading parameter values from the normal distribution. The level of agreement between DDART and wDDART for the normally distributed reading parameters (as determined by the Shapiro–Wilk test) was evaluated using Student’s paired sample t-test. For the non-normally distributed reading parameters, the paired samples Wilcoxon test was used. P-values <0.05 were defined as statistically significant.

The level of agreement between the DDART and wDDART reading parameters was also evaluated calculating the intraclass correlation coefficients (ICCs), as well as the corresponding Limits of Agreement (LoAs). A two-way mixed model with measures of absolute agreement was used for the ICC calculation and the reliability was estimated for a single rating. Agreement between the two tests was also assessed using Bland–Altman plots. ICCs and repeatability LoAs were used to evaluate test–retest reliability of wDDART reading parameters. The statistical software MedCalc version 14.8.1 (MedCalc Software, Mariakerke, Belgium) was used to perform statistical analyses.

Results

A total of 100 participants (48.2 ± 13.7 years, 50 men, 50 women) responded to both reading tests. According to their distance best-spectacle-corrected visual acuity (BSCVA), 70 of the participants populated the normal vision group (NVG), while the remaining 30 the low vision group (LVG), with a BSCVA of 0.04 ± 0.20 logMAR and 0.88 ± 0.32 logMAR, respectively. LVG included patients with various diagnoses; among them: age-related macular degeneration (nine), diabetic retinopathy (seven), glaucoma (four), optic neuropathy (two), retinitis pigmentosa (two), retinal detachment (two), epiretinal membrane (one), myopic degeneration (one), macular hole (one), and Stargardt’s disease (one). Demographic characteristics and clinical parameters of the two groups are shown in Table 1.

Table 1 Demographic Characteristics and Clinical Parameters

The reading parameters (RA, MRS, CPS, ACC) estimated by the DDART and the wDDART are presented in Table 2A–C. Differences between DDART and wDDART were non-significant for all reading parameters for all participants as well as for NVG and LVG (p > 0.05 in all cases). Bland Altman plots for RA, MRS, CPS and ACC are presented in Figure 8A–D for both NVG and LVG.

Table 2 Comparison of Calculated Reading Parameters Using DDART and wDDART

Figure 8 Comparison between wDDART and DDART for NVG and LVG using Bland–Altman plots: (A) RA, (B) MRS, (C) CPS, (D) ACC.

ICCs and LoAs for all reading parameters are presented in Table 3. For the majority of parameters, ICC’s indicated sufficient level of agreement (from 0.866 up to 0.984), except for CPS, which exhibited moderate ICC for both groups (ICCs: NVG: 0.686, LVG: 0.660, respectively), but excellent ICC for all participants (0.911).

Table 3 ICCs (Two-Way Mixed Model with Measures of Absolute Agreement – Single Rating) and LoAs of Reading Parameters Between DDART and wDDART for the Different Study Groups

Test–retest reliability results are presented in Table 4. The ICCs ranged from 0.715 to 0.990 which suggest sufficient test–retest reliability.

Table 4 wDDART Test–Retest ICCs (Two-Way Mixed Model with Measures of Absolute Agreement – Single Rating) and Repeatability LoAs of Reading Parameters for the Different Study Groups

Evaluation of the wDDART’s automatic patient-screen distance calculation is presented in Figure 9. The mean distance difference was to 0.1 ± 0.38 cm, whereas the maximum absolute difference was lower than 1 cm at the distance of 35 cm.

Figure 9 The average patient – camera distance estimation error, as a function of distance, with the 95% confidence intervals superimposed.

Discussion

The accurate evaluation of reading is essential in the routine ophthalmological examination, since reading is a fundamental activity of daily living that directly reflects near vision capacity. Numerous ophthalmological conditions reduce reading capacity, such as the suboptimal macular function and the refractive errors, including presbyopia, hyperopia and astigmatism.^17,18 Furthermore, neurological conditions, such as the lesions in the optic pathway and in the occipital lobe also reduce reading capacity.^19,20 Since reading is an advanced cognitive function, it requires optimal functionality of the eye as a refractive medium, of the optic pathway for the transfer of the optic stimuli and of the occipital lobe for the construction of image.

In the majority of the ophthalmological outpatient services, near vision acuity is evaluated by custom Jaeger charts. However, literature suggests that Jaeger charts fail to reflect the whole spectrum of near vision capacity and provide information only on the discriminant ability.⁸ On the other hand, the advanced reading tests like the RADNER, the MNREAD and the DDART reflect accurately the near vision, and their printed and digital versions have been used both in clinical and research settings.^{6,12,16,21,22}

The objective of this study was to describe and validate the wDDART web-based reading test. Despite the fact that web-applications do not offer customization comparable to the conventional installable applications, they present significant advantages; among them, a) they can be operated in any computer regardless of the underlying operating system, b) updates or upgrades to the core service are instantly available, simultaneously to all end users.

However, further to the apparent benefits of a web-based reading test, wDDART introduces the following novel features that facilitate the examination and increase the accuracy of the results:

1. Advanced text calibration, which allows the operator to perform the examination at any patient-screen distance. It also provides a preview of the reading sentences at each logMAR, in order to assess the feasibility of the examination under the current distance setting with the available computer screen.
2. Automatic assessment of the patient-screen distance, using face detection technology. Patient-screen distance is monitored 2–10 times per second during the reading phase, using a single camera, without the need for placing external landmarks on the examinee’s face. Assessment of the distance from the reading test is very important for valid and replicable measurements of the reading indexes. Most of the prevalent digital reading assessment tools rely on the subjective estimation of the distance, or using a manual measuring tool.^16,21,23,25
3. Automatic calculation of the patient’s reading times by processing the acquired patient voice, capable of measuring the duration of the talk and the post-talk delay.

In order to confirm that the advanced features of wDDART actually provide valid reading measurements, we compared wDDART’s outcomes with the corresponding ones from the DDART that has already been validated both for clinical and research settings. Despite the fact that DDART and wDDART share common user-interface and features, they present fundamental differences in their core-design since the former is a conventional computer-based application, while the latter is a high-end, web-based one. Within this context, a validation study was necessary that would confirm wDDART’s accuracy.

Indeed, wDDART and DDART demonstrated sufficient level of agreement for all reading parameters in both normal-vision and low-vision participants. ICCs for RA, MRS and ACC, ranged between 0.866 and 0.971, while ICCs for CPS were 0.685 and 0.660 in NVG and LVG, respectively. The test–retest reliability of wDDART was also very high, with all ICCs ranging between 0.715 and 0.980. Moreover, wDDART’s computer-vision estimation of patient’s distance proved extremely accurate for all distances between 35 and 100 cm with maximum 1 cm deviation.

Literature indicates a number of computer applications that have been developed for the reading assessment, such as an iPad application for the MNREAD test,²¹ IUread,²³ and the Salzburg Reading Desk (SRD).²⁴ However, none of these applications are web-based, or provide the advanced features of wDDART. An early use of a computerized device to perform an accurate visual reading test was reported by Radner et al¹⁶ and its automatic read timing feature was later re-assessed.²⁵ The iPad MNREAD application has been designed for execution on the specific computer device; thus, it does not provide extensive text calibration features.²¹ It allows for a viewing distance between 20–40 cm and 60–80 cm, it provides 5 different sets of reading sentences, allows the user to reverse the polarity of the text and calculates automatically the reading parameters. However, the application supports limited output formats of the results, while it does not support continuous distance estimation.^21,26 The IUread application provides time calculation by manually selecting the limits of the recorded sound waveform, as well as text size calibration, to be able to utilize different display screens.²³ Finally, the Salzburg Reading Desk was proposed for accurate reading visual test by Dexl et al.²⁴ This device was the only one, to our knowledge, that continuously measures distance. For this purpose, it required two firmly embedded cameras that tracked a green dot attached on the nose of the examinee. Reading timing was performed manually by graphically annotating the sound recording.

Conclusions

To our knowledge, wDDART is the first validated ophthalmological reading assessment tool that is available as a web application. Study outcomes suggest its sufficiency for clinical and research settings.

Although wDDART is validated for Greek-speaking populations, its underlying technology supports all languages, provided that the corresponding sets of sentences are used. Within this context, the potential uses of wDDART as a web-based reading assessment tool are numerous, including support for screening initiatives and remote care provision.

wDDART can be accessed at the Democritus University main server (https://ddart.med.duth.gr).