Assessing the Utility and Patient Satisfaction of Virtual Retina Clinics During COVID-19 Pandemic

Introduction

The COVID-19 pandemic, caused by the SARS-CoV-2 virus, has forced ophthalmologists to reduce the number of face-to-face (F2F) appointments they offer and modify the protocol for visits. In particular, the pressure on healthcare services and need for social distancing to protect staff and patients has motivated efforts to explore new methodologies for ophthalmic evaluation, as well as alternative approaches to coding and billing.¹ Technological advances in retinal imaging have allowed virtual retina clinics (VRCs) to become a reality in the last decade. Especially in this last year, the COVID-19 pandemic has transformed this scenario, some leading telehealth platforms now reporting virtual patient visits having increased by 257% to 700%.¹

The first virtual clinic was implemented in 2011 in Britain, by orthopedic specialists.^2,3 After that, in ophthalmology, the glaucoma subspecialty set up virtual clinics.⁴ Now, during the pandemic, care for other ophthalmological areas, such as the cornea, retina, cataract, and pediatric ophthalmology, as well as glaucoma, has been expanded through virtual clinics.^5,6 Specifically, in the case of the retina, a VRC is a platform where patients have their medical history taken and undergo ocular tests (including retina images and macular optical coherence tomography (OCT)), without seeing a doctor face-to-face and the ophthalmologist reports the case in a store and forward strategy.

Furthermore, OCT of the macula has become a cornerstone of the diagnosis and treatment monitoring of patients with retinal disorders like age-related macular degeneration (AMD) and diabetic macular edema, and the comparability of ultra-widefield (UWF) images with the gold standard of Early Treatment Diabetic Retinopathy Study (ETDRS) standard field photography facilitates telemedicine especially in the subspecialty of medical retina.⁷

During the COVID-19 pandemic, retina F2F clinics’ volume has been reduced by cancelling routine appointments in many hospitals in accordance with American Academy of Ophthalmology (AAO) COVID-19 recommendations.⁸ For example, in the US, the pandemic initially reduced patient volumes in outpatient eye-care centers by 79%, the largest decline seen in any specialty.⁹

On the other hand, over time, clinicians have reached a consensus regarding the importance of continuing retinal care while using best practices to mitigate COVID-19 transmission risk.⁹ Notably, patients with certain retinal conditions, such as AMD and diabetic retinopathy (DR) are at high risk of developing severe COVID-19, due to their age and comorbidities.⁹ Therefore, it is important to gather evidence on whether VRCs are a useful option to deal with retina clinics during the pandemic.

Herein, we present the results of non-F2F clinics for the follow-up of various stable retinal diseases during the COVID-19 pandemic at Donostia University Hospital. For more than 10 years, this tertiary hospital has been treating patients via teleretina for the detection of DR in the Diabetic Retinopathy Screening Service (DRSS). With the pandemic, we have expanded the platform for the follow-up of more retinal diseases.

In mid-March 2020 retina clinics were disrupted for non-urgent and stable diseases in line with AAO recommendations.⁸ In this context, we recruited patients for the VRC and organized the platform, launching the clinic in May. It was the key to dealing with a growing waiting list and reducing SARS-CoV-2 exposure for patients at the hospital. To decide whether to maintain the VRC as the new standard for retina care, the results of the clinic needed to be assessed.

The objective of this study was to evaluate the functioning and effectiveness of the VRC in the context of the COVID-19 pandemic.

Materials and Methods

A prospective observational study (gathering real-world data) was conducted from March 2020 until March 2021. From March to April 2020, while retina clinics were cancelled for stable chronic retina diseases due to the COVID-19 pandemic, patients that met all the inclusion criteria and none of the exclusion criteria were recruited for the VRC. Those that agreed over the phone to join the study attended the VRC rather than the F2F clinic, seeking to reduce SARS-CoV-2 exposure. The study was approved by the Ethics Committee of Donostia University Hospital. All participants gave written informed consent before inclusion in the study. The study was conducted in accordance with the principles of the Declaration of Helsinki and ensuring confidentiality in data collection.

In September 2020, when the pandemic was calming down, repeat blind assessments were performed in a group of patients to assess the effectiveness of the VRC.

Study Participants

Inclusion Criteria

The study included patients over 18 years of age, who had a Barthel activities of daily living index ≥60,^10,11 decimal visual acuity (VA) ≥0.3 (6/19 in meters; 20/60 in feet), and retinal pathologies that needed follow-up and had been visually and anatomically stable for three consecutive visits.

Specifically, we included patients with mild-to-moderate non-proliferative DR,¹² dry AMD, retinal vein occlusion (RVO), central serous chorioretinopathy (CSC), choroidal nevus, epiretinal membrane (ERM), vitelliform lesions and/or other conditions, eg, a toxoplasma scar or macular telangiectasia, or who required hydroxychloroquine (HCQ) retinopathy screening.

Exclusion Criteria

Patients were excluded if they were deemed unsuitable for the virtual clinic: wheelchair users, patients with kyphosis or media opacity, urgent referrals from the DRSS or the presence of other ophthalmological conditions that meant they needed to be seen in an F2F clinic (eg, Sjögren’s syndrome or uncontrolled glaucoma, or any pending surgery). Having previously received anti-VEGF or laser therapy was not an exclusion criterion (See Supplementary Table 1 for selection criteria).

Clinical Pathway

Figure 1 shows the possible routes of the patients between the different platforms within the clinical pathways of our retina unit in the Ophthalmology Service. Patients were referred from the DRSS, general ophthalmologist clinic or retinal specialist clinic to the VRC, and vice versa.

Figure 1 Illustrative diagram of the clinical pathways followed by patients in this study and our retina unit in the Department of Ophthalmology at Donostia University Hospital. The arrows show possible outcomes of the clinics.

Our aim was to reflect usual practice. Therefore, no limits were placed on referrals, ie, they could happen in any direction. In our case, meeting inclusion and exclusion criteria were the key factors for preventing unnecessary referrals.

Virtual Retina Clinic

Each VRC appointment consisted of the collection of clinical parameters by a trained optician including ocular history, refractometry, VA, non-contact intraocular pressure, macular OCT (Topcon 3D DRI OCT Triton), widefield retinography in macular diseases or UWF retinography in peripheral retinal diseases (Clarus ultra-widefield camera, Carl Zeiss Meditec) and a 10–2 visual field test (Humphrey 745i analyzer, Zeiss) in cases requiring HCQ screening. After these tests, the patients left the hospital. The medical history and ocular measurement data collected by the optician were recorded in a specifically designed digital VRC form and stored on the hospital’s intranet. Subsequently, a retinal specialist reviewed and reported the case in an asynchronous or store and forward strategy. This retinal specialist was able to access the VRC digital form data, retinography images with the Clarus-Zeiss viewer and macular OCT images with the Topcon viewer through a single computer.

The condition was deemed stable compared to that at the most recent previous examination at the discretion of the ophthalmologist, as in usual practice (in general, if the following criteria were met: VA was maintained; non-contact intraocular pressure was <21 mmHg; and there were no macular edema in the macular OCT, retina neovascularitation, or scotomas in the visual field test; no ERM worsening or lesion enlargement; and no new symptoms reported by the patient). In such cases, the ophthalmologist sent patients a report by post, with an appointment for the next VRC check-up. If the condition had resolved, the ophthalmologist sent them a discharge report, while if there was evidence or suspicion of disease progression, the ophthalmologist made an F2F appointment and patients were invited by phone to attend the F2F clinic. Figure 2 shows what the VRC at Donostia University Hospital is like; notably, all the equipment is in a single room, the patient only having to move once from one chair to another.

Figure 2 Donostia University Hospital VRC photograph. All the equipment is in the same room and the patient only has to move once.

Patient Satisfaction

To assess patient satisfaction and perception of the quality of hospital care, we adapted the SERVQHOS questionnaire¹³ to our context. This questionnaire is based on the SERVQUAL scale,^14,15 a tool for measuring quality of service. SERVQHOS was designed for assessing the perceived quality of the public hospital environment by patients in Spain. According to research by the Preventive Medicine Unit of the University of Alicante, it has adequate psychometric properties to measure this environment.^16,17

For our study, we used the eight SERVQHOS items that best suited our work (see Table 1). First, patients rated six items concerning clinicians’ kindness towards patients, personalized treatment received, confidence (sense of safety), medical equipment and technology, punctuality of consultations and global satisfaction. Unlike in the SERVQHOS questionnaire, these items were rated with a customer satisfaction score (CSAT), which ranges from the care received having been 1: much worse than expected to 10: much better than expected. Additionally, we used a modified version of one other SERVQHOS item, specifically, instead of asking about the perception of the duration of the hospitalization, we asked whether the clinic had lasted more or less than they expected.

Table 1 Adapted SERVQHOS Scale on the Perceived Quality of Hospital Care

Patients were requested to rate all of the aforementioned items twice: first, in relation to the VRC, and second, in relation to previous experience with the F2F clinic. Lastly, patients were asked whether they would recommend the VRC clinic to other patients (response: yes or no) and which type of clinic they preferred. On the other hand, data were recorded for all patients on waiting and consultation times.

Clinical Effectiveness

To evaluate the VRC effectiveness, repeat blind assessments were performed in a subsample of patients. Two different ophthalmologists reported on the same case. On the one hand, the VRC ophthalmologist, analyzed the case with parameters collected by the optician in the VRC, together with the retinography and macular OCT images, and reported whether the patient was suitable to continue in the VRC, should be referred to the F2F clinic due to suspected progression or for some other reason, or could be discharged from the VRC. The information was stored in the VRC form on the hospital’s intranet, but the final outcome was stored in a specific document to keep the study blind.

On the other hand, at a second appointment (within 1 week), a second ophthalmologist explored the same patient in the F2F clinic and concluded whether the patient was stable, had progression with or without the need for treatment or could be discharged. The information was stored in the patient´s clinical folder on the hospital’s intranet, as is usual practice.

Each ophthalmologist was unaware of the other ophthalmologist’s conclusions. We assessed the agreement between the results of VRC and F2F clinics.

Variables Included in the Study

Data were gathered on the following variables: age, sex, retinal disease, preference for VRC vs F2F clinics, whether patients would recommend the VRC to other patients, staff kindness, personalized treatment, confidence, equipment and technology, punctuality, global satisfaction, waiting time, time in the clinic and whether the clinic appointment lasted more or less than the patient had expected.

For the effectiveness study, the variables considered were VRC and F2F clinic outcomes. The predictive values, sensitivity and specificity of the VRC findings were measured comparing the VRC outcome with that from the F2F control assessment.

Statistical Methods

First, a descriptive analysis was performed of the samples of patients seen in the VRC and double-checked in the F2F clinic. The statistical significance of differences between groups was assessed using Student’s t-test for normally distributed data and non-parametric Wilcoxon tests for non-normally distributed and ordinal data. The level of significance was set at p < 0.05. Chi-square tests were used to compare contingency tables. Statistical analysis was performed in an Excel database using IBM SPSS Statistics for Windows version 23.

Second, a multivariate analysis was performed using logistic regression models. We calculated the likelihood of each outcome (remaining in the VRC, being discharged from the VRC and being referred to the F2F clinic) adjusted for age, gender and retinal disease. We built models for each outcome separately to explore whether the likelihood of different outcomes was associated with any of the variables analyzed.

Results

During the yearlong study period, 481 patients were included, 220 (45.7%) men and 261 (54.2%) women. The mean age was 67 years (SD ± 13.9; range: 18–97). Of this total sample, 114 underwent repeat blind assessments at the F2F clinic (23.7%). The most prevalent disease was DR (146 patients, 30.3%), followed by AMD (105 patients, 21.8%). Table 2 details the characteristics of the patients and their diagnoses.

Table 2 Patient Characteristics in the Whole Sample and in the Sample with Repeat Blind Assessment

VRC Outcomes

Regarding retinal clinical assessment, 356 patients (74%) were considered suitable to continue in the VRC and 32 (6.6%) were discharged, while 93 (19.3%) were referred to the F2F clinic. Table 3 lists the outcomes of the VRC appointments by disease.

Table 3 Outcomes of Retina Virtual Clinic Appointments Classified by Disease

The most common reason for referral was disease progression (61.2% of all referred patients), followed by patients being unsuitable for virtual management (22.5%). Only 4 patients (4.3%) were referred because of poor image quality. See Table 4 for all reasons for F2F clinic referrals.

Table 4 Reasons for Face-to-Face Clinic Referrals

Patient Satisfaction

The VRC was given a satisfaction rating of 9.8 out of 10. Notably, 392 patients (81.4%) preferred the VRC to the F2F clinic, while just 23 (4.8%) preferred the F2F clinic and 66 (13.8%) expressed no preference. Almost all (n=467, 97%) would recommend the VRC to others. See Table 1 for details concerning patient satisfaction.

Clinic Workflow and Timing

In usual practice, at the F2F clinic, the patient first undergoes macular OCT, is then seen by the optician, and finally, seen by the ophthalmologist. Each of these consultations lasts 10 minutes. In the VRC, the medical history and all the tests were completed in a single 15-minute consultation.

Concerning appointment timings, for the VRC, tests with the optician were scheduled every 15 minutes and the ophthalmologist reported on 12 cases in 1 hour (5 minutes/case), meaning that 20 minutes of staff time were needed per patient in the VRC. In contrast, in the F2F clinic, patients attended three 10-minute appointments (OCT, optician tests and ophthalmologist clinic), meaning that 30 minutes of staff time were needed per patient. Hence, the VRC reduces staff time required by 33.3% and the ophthalmologist can report on twice as many cases. Furthermore, it halves the time patients spend in the clinic.

Clinical Effectiveness

Overall, 114 patients (23.7%) were double-checked in the F2F clinic after the VRC to assess the capability of the VRC to detect disease progression that warranted an F2F appointment. In the F2F clinic, three patients (2.6%) were found to need treatment: two cases of severe DR and one of CSC. In the VRC, these three patients had been detected and referred to the F2F clinic because of disease progression. Among the 111 true negatives identified in the F2F, 89 were also correctly diagnosed in the VRC, yielding 22 false positives in the F2F. Therefore, the sensitivity of the VRC to detect disease progression was 100%. The specificity was 80.18% (95% confidence intervals: 71.3–86.9%). The positive predictive value was low (12% with 95% confidence intervals: 3.1–32.3%), but given the high sensitivity, the negative predictive value was 100% (See Table 5).

Table 5 Outcomes of Repeat Blind Assessment – Sensitivity of Retina Virtual Clinic to Detect Progression

Regarding patients discharged in the double-checked sample, nine patients (7.8%) were approved for discharge in the F2F clinic and eight (7%) in the VRC. Of these, three patients were discharged from both clinics, six who were not discharged from the VRC being discharged from the F2F clinic and five who were not seen in the F2F clinic being discharged from the VRC.

Table 6 shows the results of the logistic regressions analyzing the likelihood of different outcomes, no significant differences being found by retinal disease, sex or age. The only factor considered that influenced the risk was the monitoring of HCQ treatment, this being associated with a higher OR for remaining in VRC screening and a lower OR for F2F referral.

Table 6 Likelihood of Outcomes by Type of Retinal Disease

Discussion

The greatest contribution of this study is that it evidences that a VRC platform during the COVID-19 pandemic in a tertiary referral center was effective in that no patients who needed treatment were left without it and the patients´ rated the care received as good.

The rate of referrals from the VRC to an F2F clinic in this study differs from that found in other series.^7,18,19 (See Supplementary Table 2 for comparison of our VRC outcomes with those from other studies.) With new referrals, referral rates to F2F clinics of 27.8% and 17.4% were reported by Lee et al¹⁸ and Kortuem et al,¹⁹ respectively, while in a series with internal referrals, Kortuem et al found a higher rate of 30.9%.¹⁹ In our study population with internal referrals, the prevalence of F2F clinic referrals was 19.3%. As the percentages of patients referred to an F2F clinic and discharged vary across series, the percentage of patients that continue to be managed through the VRC also differs. (See Table 3 for referral percentages.) Some of the variability in referral rates may result from differences in selection criteria and between new and internal referrals.

Further, regarding the diseases chosen for the VRC, we restricted DR to mild-to-moderate cases, as severe DR could be expected to have higher referral rates to F2F clinics, as described by Kern et al⁷ and Kortuem et al.¹⁹ In our case, VRC management was deemed suitable for 62% of mild-to-moderate cases of DR. We also considered patients with ERM or who required HCQ retinopathy screening to be candidates for the VRC and the results were positive, 83% and 91% of patients respectively being deemed suitable for VRC management.

In our study, the main reason for referring patients from the VRC to the F2F clinic was disease progression (over 60% of all patients referred). Kern et al⁷ also found this to be the most common reason for referral to an F2F clinic, as did Lee et al.¹⁸ In contrast, the main reasons for referring patients from VRC to F2F clinics in the series of Kortuem et al¹⁹ was poor image quality (34.7%), followed by detection of potentially treatable disease (20.2%). This difference might be attributable to the image acquisition technique and quality in Kortuem’s study having acted as a threshold that decreased F2F referrals.

Research has demonstrated that lesions in the periphery of the retina could be involved with progression of retinal disease like DR, the most prevalent disease in our study.²⁰ This underlines the importance of using a UWF imaging system in the VRC as it enables the visualization of more of the retina than conventional color fundus photography (CFP).²¹ Furthermore, nonmydriatic UWF imaging acquisition is faster as there is no need to dilate pupils. In addition, as these devices use a scanning laser, they are less affected by pupil size and media opacities such as cataracts than traditional CFP.²²

In our case, images were taken for all patients with a widefield or UWF Clarus ultra-widefield camera (Carl Zeiss Meditec), which provides a resolution of 7.3 μm.²¹

To our knowledge, this is the first report of a VRC series using this equipment and this approach made it possible to keep managing 74% of the patients in the VRC. Furthermore, the percentage of patients deemed suitable for VRC management will likely be higher in consecutive virtual appointments, assuming that unsuitable patients have been excluded from the VRC clinic after triage at the first visit. Notably, Kern et al⁷ reported a reduction in F2F referral rate from >30% to 15% at the second virtual visit.

Concerning VRC outcomes, the most prevalent disease was DR, consistent with the findings of Kortuem et al,¹⁹ Kern et al⁷ and Lee et al,¹⁸ in their virtual clinics. Diabetes mellitus has become a global pandemic and the prevalence of DR can be expected to grow and this implies a rising demand for retinal specialists.^23,24 Apart from sight-threatening DR detection platforms that have been shown to be very effective in England,²⁵ we believe that VRCs could be a helpful way to free up capacity in retina clinics in the near future.

Our assessment was supported by feedback from patients on their experience of the healthcare received, which is key for improving services. In our study, we were able to survey 481 patients and it was encouraging to discover that patients had a very favorable opinion of the VRC. Furthermore, they noted that less time was spent in the VRC than the F2F clinic and this was important regarding attendance to appointments during the pandemic, high-risk patients being afraid of COVID-19. Lindeke-Myers et al⁹ highlighted that 14% of the patients surveyed believed SARS-CoV-2 exposure in eye clinics was likely or extremely likely and 22% of the patients were lost to follow-up, this being significantly associated with fear of SARS-CoV-2 exposure.

Our results indicate that less time is needed for VRC than F2F appointments, and therefore, the exposure to SARS-COV-2 has been reduced using the virtual approach. The less time patients spent in the hospital, the lower the risk of SARS-COV-2 exposure, and in our study, we found better punctuality with shorter waiting times for the VRC appointments as well as faster consultations, than in the F2F clinics. The VRC at least halved the time that patients spent at the hospital. Furthermore, in the same amount of time, the ophthalmologist reported on twice as many cases in the VRC as in the F2F. In our series, on average, the ophthalmologist reported on 12 cases in 1 hour, while in an F2F clinic, the ophthalmologist saw just 6 patients in 1 hour. Lee et al¹⁸ also reported that they were able to report twice as many cases in the VRC: 10 cases in an hour in the VRC versus 5 cases in the F2F clinic.

In the literature, we have not found any studies including repeat blind assessment of VRC performance to assess the effectiveness of virtual management. In our study, the most important finding was that in the subsample whose condition was blindly double checked (in an F2F appointment after the VRC) there were no patients who needed treatment but had not been referred from the VRC for an F2F appointment, any misses implying a potential risk of vision loss.

Therefore, we believe that the VRC is a safe approach for managing the stable retinal conditions mentioned in this study, while all patients whose condition seemed to show progression would be referred to an F2F clinic. Nonetheless, the study might still be underpowered and a larger sample is needed to confirm this finding.

Concerning the patients discharged in the blind assessment sample, on the one hand, some patients were discharged from the VRC but not from the F2F clinic, the main reason being that in the F2F clinic they needed macular OCT that was not performed as part of the first F2F appointment, unlike in the VRC which included this type of imaging. Therefore, these patients were not discharged from the F2F clinic and a subsequent appointment was arranged for macular OCT. That is, the VRC included macular OCT, this allowing earlier discharge of such patients (after one fewer appointment).

On the other hand, there were also patients discharged from the F2F clinic and not from the VRC. In this case, the reason was that another appointment in the VRC was deemed necessary to ensure disease stability. In this context, the VRC could be an alternative for a more staggered discharge; that is, before patients are discharged completely from an F2F clinic, they could be seen in VRC.

The VRC has several advantages but also inherent limitations. The doctor–patient relationship may be more difficult in some cases.²⁶ Related to this, our study patients were internal referrals and gave good satisfaction ratings, but satisfaction might be lower if the VRC included newly diagnosed cases where F2F discussion with the patient is crucial. Our VRC showed a sensitivity of 100% to detect retinal disease progression in this context but other findings could be missed. For example, the impact of cataract progression was not specifically studied. Future studies should explore and tackle this issue. Lastly, another limitation that could be addressed in further research is that artificial intelligence systems were not used in our VRC at the time of this study, though they have shown promising results and we are keen to deploy them in our clinic in the future.²⁷

In conclusion, this study provides evidence that a VRC is a useful platform that supports F2F retina clinics and has certain benefits. In particular, the virtual clinic reduces patients’ potential exposure to SARS-CoV-2 in hospital visits, by minimizing time in the clinic, and under these specific conditions, it seems feasible to keep monitoring a high proportion of patients with selected stable retinal diseases in a VRC, freeing up capacity in F2F retina clinics, while patients’ satisfaction with the virtual approach is very high.