Comparison of smartphone ophthalmoscopy vs conventional direct ophthalmoscopy as a teaching tool for medical students: the COSMOS study
Authors Kim Y, Chao DL
Received 15 October 2018
Accepted for publication 17 December 2018
Published 18 February 2019 Volume 2019:13 Pages 391—401
Checked for plagiarism Yes
Review by Single anonymous peer review
Peer reviewer comments 2
Editor who approved publication: Dr Scott Fraser
Yeji Kim, Daniel L Chao
Department of Ophthalmology, Shiley Eye Institute, University of California, San Diego, CA, USA
Purpose: To investigate the utility of smartphone ophthalmology for medical students for learning fundoscopy compared with direct ophthalmoscopy.
Methods: After 1 hour of didactic instruction on ophthalmoscopy, second-year medical students in a small group setting were randomized to start training with the direct ophthalmoscope vs smartphone ophthalmoscope and crossed over to the other instrument through the session.
Main outcome measures: Ability to visualize the optic nerve and retinal blood vessels in an undilated pupil as well as a survey evaluating ease of use, confidence, and ability to visualize the optic nerve with the two instruments.
Results: One hundred and one medical students participated. Significantly more medical students were able to visualize the optic nerve with the smartphone ophthalmoscope vs the direct ophthalmoscope in an undilated pupil (82.3% vs 48.5%, P<0.0001). Students reported a more positive experience with the smartphone ophthalmoscope, specifically regarding ease of use (median of 4 vs 3; P<0.0001), their confidence in performing ophthalmoscopy (median of 4 vs 3; P<0.0001), and their ability to visualize features of the optic nerve (median 4 vs 3; P<0.0001). A significant number of participants preferred the smartphone ophthalmoscope over the traditional direct ophthalmoscope for learning how to identify the optic disc and for evaluating patients (78.2% and 77.2%, respectively; P<0.0001).
Conclusion: Smartphone ophthalmoscopy may serve as a useful adjunctive tool to teach direct ophthalmoscopy as well as being an alternative for examining the fundus for noneye care physicians.
Keywords: smartphone ophthalmoscopy, direct ophthalmoscopy, medical student education
Nonophthalmologist physicians often treat patients experiencing visual symptoms in primary care settings and emergency departments.1,2 Indeed, it has been estimated that 1.5% of all emergency room (ER) visits have an eye complaint as a principal diagnosis.3 Often with limited immediate access to ophthalmologists, it is important for physicians to be able to examine the fundus in order to triage between ophthalmic emergencies, such as optic nerve edema, occluded retinal vessels, or retinal detachment and vision problems that can be referred to an ophthalmologist nonurgently.2,4,5 Since its invention by Charles Babbage in 1847 and reinvention by Hermann von Helholtz in 1851, examination of the fundus by nonophthalmologists has been traditionally performed with the direct ophthalmoscope (DO).1,6 This is a core physical examination skill taught in every medical school curriculum.1,2,7
Although all medical schools include training in direct ophthalmoscopy, this level of education is generally considered inadequate in mastering this technically challenging clinical skill by students and practicing physicians alike.5,8–10 Students and practicing physicians have consistently reported a general lack of confidence in performing a fundus exam using a DO.1,5,9,11,12 The lack of confidence correlates strongly with a lack of competence, which often results in the underutilization of ophthalmoscopy in the primary care and ER setting.5,10,11,13 Indeed, multiple reports have suggested that direct ophthalmoscopy is a forgotten art and given its limited use in practice, the elimination of training in direct ophthalmoscopy for medical students has even been suggested.1,11
In recent years, various improvements and alternatives to direct ophthalmoscopy have been developed to improve the ability of nonophthalmologists to view the fundus. Attachments to the DO, such as the Panoptic DO, allow a larger field of view but are not widely available in clinic rooms. Nonmydriatric fundus cameras placed in primary care clinics and ERs have also demonstrated utility in telemedicine screening for diabetic retinopathy as well as detecting fundus findings in the ER setting.14,15 However, nonmydriatric cameras are not portable and widely available, thus limiting their utility. Recently, numerous smartphone ophthalmoscopes have been developed, which are able to capture images of the fundus simply using a smartphone camera as well as an adapter.16–18 These devices have great potential for use for telemedicine and clinical examination skill given their portability, ubiquitous presence, ease of use, ability to photographically document and easily send to peers using Health Insurance Portability and Accountability Act of 1996 (HIPAA) compliant methods, as well as their low relative cost.
Although smartphone ophthalmoscopy represents an exciting tool for telemedicine and fundoscopy, there are limited studies assessing the utility of smartphone ophthalmoscopy in teaching medical students to visualize the fundus as well as a clinical tool to detect fundus findings. Smaller scale studies published after the completion of our own suggests an overwhelming number of medical students prefer smartphone ophthalmoscopes over the traditional DO.19,20 One of those studies demonstrated more accurate clinical descriptions of eye pathology by medical students using smartphone ophthalmoscopy over the DO suggests that smartphone ophthalmoscopy is indeed a viable alternative to the DO.19
To further investigate the utility of smartphone ophthalmoscopy as a teaching tool for fundoscopy for medical students on a larger scale, we initiated the Comparison Of Smartphone ophthalMoscopy vs conventional direct Ophthalmoscopy as a teaching tool for medical Students (COSMOS) Study, a prospective randomized study to determine whether smartphone ophthalmoscopy could serve as an effective teaching tool for fundus imaging for medical students and compare its ease of use and effectiveness compared with standard direct ophthalmoscopy. We used the D-EYE digital ophthalmoscope (D-EYE Srl, Padova, Italy), a magnetic lens that attaches to the smartphone via magnets to transform the smartphone into a portable fundus camera (Figure 1). The D-EYE is Food and Drug Administration approved as a fundus camera and also has a secure HIPAA compliant cloud software to store images.21 Clinical studies using the D-EYE have found it to be superior in detecting fundus findings of acute hypertension compared with direct ophthalmology in the ER setting, showed significant agreement with slit-lamp biomicroscopy in measuring vertical cup to disc ratios, and was comparable with indirect ophthalmoscopy in detecting fundus findings in pediatric patients.17,18,21,22
This prospective randomized study was conducted at the University of California, San Diego (UCSD) School of Medicine. The study was approved and exempted from Institutional Review Board (IRB) review by the UCSD IRB under 45 CFR 46.101(b), category 2: Research involving the use of educational tests (cognitive, diagnostic, aptitude, and achievement), survey procedures, interview procedures, or observation of public behavior. The described research adhered to the tenets of the Declaration of Helsinki.
Participant recruitment and randomization
This study took place within the context of a half-day clinical skills session to teach the ophthalmic exam that is part of the UCSD second-year medical student curriculum. All UCSD second-year medical students during the academic year 2017–2018 were eligible to participate in the study. All eligible participants were notified of the study prior to the clinical skills session through an email as well as on the day of the study via an in-class announcement. A copy of the informed consent was distributed to all 130 eligible participants during the small groups workshop. In order for the survey responses to remain anonymous and to better protect subject confidentiality, documented consent was waived for the study. However, to fully inform subjects, a consent form was included with every survey.
After a 1-hour didactic lecture on the eye exam, students were divided into small groups consisting of 6–11 students for 1.25 hours to work directly with ophthalmologists and ophthalmology residents to practice direct ophthalmoscopy and other parts of the eye exam. One-third of each small group was randomized to start with the D-EYE (Group B) and the other two-third started with the DO (Group A) with the groups then switching to the other instrument halfway through the session (Figure 2). Students spent 45 and 30 minutes with the DO and D-EYE, respectively. The rationale behind this 2:1 randomization and more time dedicated to the DO was because of the strong recommendation from UCSD medical education directors to focus the session on direct ophthalmoscopy, as this is the traditional method of examining the fundus.
Each group received a 10-minute demonstration of how to use the direct and digital ophthalmoscope by a trained ophthalmologist followed by time for the students to practice using the ophthalmoscopes on their fellow students who volunteered within each group. The DO was either a wall mounted or portable DO (Welch Allyn model #11720). Each group had access to two DOs and one D-EYE digital ophthalmoscope during the workshop. Examinations occurred through both dilated and undilated pupils. Ophthalmologist preceptors were given a 20-minute training on how to use the D-EYE prior to the workshop.
All participants were asked to complete a questionnaire after finishing the clinical skills session (Figure S1). The questionnaire was specifically created for this study and had not been used in prior studies. This questionnaire included basic demographic information (age and gender), as well as objective outcomes to compare ability to visualize the optic nerve and blood vessels using both ophthalmoscopes and subjective observations related to ease of use, confidence in performance, and self-reported competence in ophthalmoscopy. Subjective ratings were provided on a scale of 1 (“negative”) to 5 (“positive”). The primary outcomes of the study were the ability to visualize the optic nerve and blood vessels during the session. Student abilities in visualizing these structures with the traditional ophthalmoscope were compared with their abilities with the smartphone ophthalmoscope using the chi-squared test. Secondary outcomes were subjective questions related to ease of use, confidence, and preference of using the ophthalmoscopes. These subjective ratings between the two ophthalmoscopes were compared using the paired t-test, whereas student preferences between the two ophthalmoscopes were compared using the chi-squared test. All questionnaires in which the participants did not indicate whether they were in Group A or Group B were excluded from the study.
Table 1 shows the basic demographic information between the two groups. There was no difference in gender between the groups, as well as previous experience in ophthalmoscopy.
Table 1 Baseline characteristics of the study population. Data presented as means (SD) or numbers (%)
We first wished to examine to compare the ability of students to visualize the optic nerve and blood vessels using both of these ophthalmoscopy techniques (Table 2). As the instructor cannot see what the student is able to see with the DO, the students were asked whether they were able to visualize the optic nerve and retinal vessels and indicate this on the questionnaire. Students received a 1-hour lecture on ophthalmoscopy immediately prior to the skills session with representative pictures of the optic nerve using a DO for reference. For the smartphone ophthalmoscopy, the instructor was able to visualize whether the correct structure was being visualized and marked on their questionnaire. Significantly more participants were able to visualize the optic nerve using the D-EYE compared with the DO (Figure 3). In dilated eyes, 61 (85.9%) students reported visualization of the optic nerve using the D-EYE, compared with 53 (65.4%) students using the DO (chi-squared test P=0.0036). In undilated eyes, the ability to see the optic nerve was even more pronounced with 65 (82.3%) students able to see the optic nerve with the D-EYE, compared with 43 (48.5%) students with the DO (chi-squared test P<0.0001). However, the majority of students were able to visualize retina vessels in both undilated (69 students, 80.2% vs 69 students, 87.3%, chi-squared test P=0.2175) and dilated (72 students, 88.9% vs 64 students, 91.4%, chi-squared test P=0.6029) eyes using either the DO or D-EYE, respectively. This resulted in no significant difference between the two ophthalmoscopes in the visualization of retina vessels.
Table 2 COSMOS study primary and secondary outcomes comparing the traditional and smartphone ophthalmoscopes
Secondary outcomes included subjective measures of the student’s experience in using these two types of ophthalmoscopy using a rating scale of 1 (“negative”) to 5 (“positive”) (Table 2). After this single session, students found the D-EYE easier to use and felt more confident in using it (Figure 4). When rating the ease of use and confidence in using the two ophthalmoscopes, participants rated the D-EYE median (interquartile range [IQR]) scores of 4 (4–5) and 4 (3–4), respectively, which are higher than the median scores 3 (2–3) and 3 (2–3) for the DO (paired t-test P<0.0001 for both measures).
Students also felt more confident in visualizing blood vessels and the optic nerve with the D-EYE (Figure 4). Participants reported a significantly higher mean rating for the D-EYE compared with the DO in visualizing both the blood vessels (5, IQR =4–5 vs 3, IQR =3–4) and features of the optic nerve (4, IQR =4–5 vs 3, IQR =2–3) (paired t-test P<0.0001 for both measures).
When asked to state a preference between the two ophthalmoscopes, 79 of the participants (78.2%) preferred the D-EYE over the DO for learning how to identify the optic disc, whereas 78 (77.2%) preferred the D-EYE to evaluate patients (chi-squared test P<0.0001 for both measures). Students were also asked to rate on a scale of 1 (“negative”) to 5 (“positive”) if they believed that using the D-EYE helped to improve their ability to use the traditional ophthalmoscope. Students found the D-EYE to be slightly beneficial with a median rating of 4 (IQR =3–4).
We also compared the differences in responses between the two randomized groups (Group A and Group B). While comparing objective and subjective responses between the two groups, most comparisons did not yield a statistically significant difference (Table 3). However, objectively, 79% of students in Group B compared with 39% from Group A were able to confirm visualization of the optic nerve in an undilated eye using the traditional ophthalmoscope (unpaired t-test P=0.0008). Furthermore, subjectively, Group B rated their ability to identify the optic nerve using the traditional ophthalmoscope as significantly higher than Group A (unpaired t-test 3, IQR =3–4 vs 2, IQR =1–3 respectively, P=0.0026).
In the COSMOS study, we conducted a prospective randomized clinical study comparing smartphone ophthalmoscopy with direct ophthalmoscopy in teaching second-year medical students to visualize the fundus. We found that at the end of the clinical skills session, significantly more students were able to visualize the optic nerve and retinal vessels in the undilated pupil using the D-EYE compared with the DO, and subjectively, students felt that the D-EYE was easier to use, had more confidence in using the D-EYE to visualize the fundus, that it was easier to use, and preferred to use the D-EYE as a clinical tool. We confirmed that medical students prefer the D-EYE over the DO with a sample size of medical students four times larger than the prior studies.19,20
As visually related complaints are quite common in primary care and ER settings, examination of the fundus by noneye care providers should be an important skill as part of the physician’s physical examination. Indeed, direct ophthalmoscopy is generally considered a core physical examination skill for medical student and is taught to some degree in all medical school curriculums in the USA.1,2,7 However, given the limited amount of time dedicated to ophthalmology education in medical school, it is no surprise that physicians and medical students lack confidence and competence in this technically challenging skill. In a recent survey, 84% of medical schools offering clinical skills sessions offered clinical skills session related to ophthalmology.23 At UCSD, medical students receive 1 hour of clinical skills instruction with the DO on models in their first year, and a half-day clinical skills session in their second year to learn direct ophthalmoscopy. There is no examination or testing of their skills in their clinical years. Also, there are no guidelines for ophthalmology education in the Liaison Committee on Medical Education guidelines. Numerous reports have demonstrated the lack of competence and confidence in direct ophthalmoscopy use in practitioners.
Given the difficulties with direct ophthalmoscopy, alternative approaches have been investigated to teach medical students the fundus exam. The ToTEMS study used fundus photos as an adjunct to detecting eye diseases. Students were found to prefer learning ophthalmic disease using fundus photographs and found that it significantly improved diagnosis of eye diseases.24,25 Other methods to improve training with direct ophthalmoscopy include peer fundus photo matching and various simulators of direct ophthalmoscopy.26–32 However, few comparative studies have been performed to assess their utility to effectiveness in improving competency with the DO.27,29,30,32
Improving accessibility and confidence of nonophthalmologist physicians in evaluating the fundus could have significant effects of healthcare resources. In a recent paper, it was demonstrated that half of all eye-related ER visits were nonemergencies.33 In general, nonurgent care costs are more than doubled if seen in the ER compared with being seen in other healthcare settings for the same problem.34 Thus, the ability of nonophthalmologist physicians to confidently evaluate the fundus could thus have large implications for healthcare utilization and costs.
Given the ubiquitous presence of smartphones, their portability, the ability for both the student and teacher to visualize the same image, as well as the ability to transmit information in an HIPAA compliant method, the smartphone serves as an attractive instrument for fundoscopy. Many versions of smartphone fundus cameras are now present in the marketplace. Smartphone fundus cameras have been demonstrated to take anterior and posterior segment images that are good enough to exclude urgent ocular findings in developing countries,35 and have been investigated as a proof of concept for diabetic retinopathy screening36–40 as well as acute hypertension.22 In our study, we demonstrated that smartphone ophthalmoscopy is helpful in teaching fundoscopy to medical students.
As suggested by other studies, another consideration is, whether smartphone ophthalmoscopy could replace direct ophthalmoscopy as a fundus imaging modality for nonophthalmologists and medical students. This question is out of the scope of the current study but given the ubiquitous presence of smartphone, the relative low cost of smartphone fundus cameras compared with DOs, and the preference of medical students to use the smartphone ophthalmoscope compared with the DO as demonstrated in our study and others,19,20 this is a question worthy of future research.
Strengths of this study include the prospective randomized design, the large number of participants in the study, and the use of both objective and subjective measures of fundoscopy assessment of patients.
There are multiple limitations to the study. This is a short-term study based on a single skills session. Further studies will be needed to determine whether smartphone ophthalmoscopy can improve medical students’ ability to examine the fundus in the long term. One caveat is that the questionnaire we used has not been previously validated, as this is a novel study. The study was also limited by only having a single point of measurement at the end of the teaching session, which does not allow us to see if there were any differences in the two groups after exposure to the first ophthalmoscope.
Given the subjective nature of the questionnaire, the results may be subject to biases, such as recall or social desirability bias, but randomization of participants, the crossover study design, and anonymity of the survey were utilized to control for biases. Also, some of the objective data (ie, ability to see nerve with the DO) is self-reported as this cannot be easily assessed by an instructor. In addition, ability to visualize the optic nerve may not reflect ability to observe abnormal findings of the optic nerve. Different volunteers were used for examination in each group, who may have had variable pupillary dilation at baseline. Indeed, there may be a selection bias where students with larger pupillary dilation at baseline were used as volunteers for ophthalmoscopy, in order to enhance learning. We believe this would only serve to underestimate the differences between ease of use in direct ophthalmoscopy and D-EYE, as volunteers were likely biased to larger pupillary dilation, which would make direct ophthalmoscopy easier. We attempted to control this by having a large number of groups and volunteers. Moreover, not every medical student that participated answered every question of the survey, causing great variability on the total number of responses per question. Implementing measures to ensure all survey questions are fully answered by a participant may be necessary in future studies.
The groups were randomized in a 2:1 fashion to start with direct vs smartphone ophthalmoscopy and had more time with the DO compared with the D-EYE. This imbalance was dictated by the UCSD Medical Education Course directors to focus the session on direct ophthalmoscopy, as this is the standard typical instrument used in the medical teaching setting as well as in the clinic. Interestingly, we observed that the group that started with the D-EYE (Group B) had significantly more students reporting visualization of the optic nerve in an undilated eye using the DO compared with those who started with the DO first (Group A). This may imply that using the D-EYE first may improve the student’s ability to use the DO, suggesting that the D-EYE may serve as an adjunctive tool to help learn direct ophthalmoscopy.
In conclusion, we have demonstrated that significantly more students were able to visualize the optic nerve in an undilated pupil using smartphone ophthalmoscopy compared with direct ophthalmoscopy. In addition, medical students felt more comfortable and confident in using the D-EYE vs the DO. More students would also prefer to use the D-EYE on patients in future clinical settings compared with the DO. Further research is needed to assess the place of smartphone fundoscopy in teaching fundoscopy to medical students as well as a possible adjunct or replacement to direct ophthalmoscopy.
We thank Drs Amir Marvasti, Rohan Verma, Sally Baxter, Landon Grange, Tommy Apara, Heather Chen, Christopher Toomey, Clara Men, Jaimie Walsh, James Monroe, Norman Rozansky, and Derek Welsbie for their help in precepting the ophthalmology skills sessions.
Neither Dr Daniel L Chao nor Yeji Kim have a financial interest with D-EYE. D-EYE had no input in the design of the experiment or the analysis of this study. We thank D-EYE for graciously providing extra D-EYE loaner fundus cameras for this study. The authors report no conflicts of interest in this work.
Mackay DD, Garza PS, Bruce BB, Newman NJ, Biousse V. The demise of direct ophthalmoscopy. Neurol Clin Pract. 2015;5(2):150–157.
Yusuf IH, Salmon JF, Patel CK. Direct ophthalmoscopy should be taught to undergraduate medical students-yes. Eye. 2015;29(8):987–989.
Vaziri K, Schwartz SG, Flynn HW, Kishor KS, Moshfeghi AA. Eye-related emergency department visits in the United States, 2010. Ophthalmology. 2016;123(4):917–919.
Liu Y, Wu F, Lu L, Lin D, Zhang K. Examination of the retina. New Engl J Med. 2015;373(8):e9.
Roberts E, Morgan R, King D, Clerkin L. Funduscopy: a forgotten art? Postgrad Med J. 1999;75(883):282–284.
Mcmullen WH. The evolution of the ophthalmoscope. Br J Ophthalmol. 1917;1(10):593–599.
Mottow-Lippa L. Ophthalmology in the medical school curriculum: reestablishing our value and effecting change. Ophthalmology. 2009;116(7):1235–1236.
Dalay S, Umar F, Saeed S. Fundoscopy: a reflection upon medical training? Clin Teach. 2013;10(2):103–106.
Gupta RR, Lam WC. Medical students’ self-confidence in performing direct ophthalmoscopy in clinical training. Can J Ophthalmol. 2006;41(2):169–174.
Shuttleworth GN, Marsh GW. How effective is undergraduate and postgraduate teaching in ophthalmology? Eye. 1997;11(5):744–750.
Purbrick RMJ, Chong NV. Direct ophthalmoscopy should be taught to undergraduate medical students – no. Eye. 2015;29(8):990–991.
Schulz C, Hodgkins P. Factors associated with confidence in fundoscopy. Clin Teach. 2014;11(6):431–435.
Bruce BB, Lamirel C, Wright DW, et al. Nonmydriatic ocular fundus photography in the emergency department. New Eng J Med. 2011;364(4):387–389.
Bruce BB, Thulasi P, Fraser CL, et al. Diagnostic accuracy and use of nonmydriatic ocular fundus photography by emergency physicians: phase II of the FOTO-ED study. Ann Emerg Med. 2013;62(1): 28–33.e1.
Mansberger SL, Sheppler C, Barker G, et al. Long-term comparative effectiveness of telemedicine in providing diabetic retinopathy screening examinations. JAMA Ophthalmol. 2015;133(5):518–525.
Ludwig CA, Murthy SI, Pappuru RR, Jais A, Myung DJ, Chang RT. A novel smartphone ophthalmic imaging adapter: user feasibility studies in Hyderabad, India. Indian J Ophthalmol. 2016;64(3):191–200.
Russo A, Mapham W, Turano R, et al. Comparison of smartphone ophthalmoscopy with slit-lamp biomicroscopy for grading vertical cup-to-disc ratio. J Glaucoma. 2016;25(9):e777–e781.
Balland O, Russo A, Isard P-F, Mathieson I, Semeraro F, Dulaurent T. Assessment of a smartphone-based camera for fundus imaging in animals. Vet Ophthalmol. 2017;20(1):89–94.
Mamtora S, Sandinha MT, Ajith A, Song A, Steel DHW. Smart phone ophthalmoscopy: a potential replacement for the direct ophthalmoscope. Eye. 2018;32(11):1766–1771.
Wu AR, Fouzdar-Jain S, Suh DW. Comparison study of funduscopic examination using a smartphone-based digital ophthalmoscope and the direct ophthalmoscope. J Pediatr Ophthalmol Strabismus. 2018;55(3):201–206.
Dickson D, Fouzdar-Jain S, Macdonald C, et al. Comparison study of funduscopic exam of pediatric patients using the D-EYE method and conventional indirect ophthalmoscopic methods. Open J Ophthalmol. 2017;7(3):145–152.
Muiesan ML, Salvetti M, Paini A, et al. Ocular fundus photography with a smartphone device in acute hypertension. J Hypertens. 2017;35(8):1660–1665.
Shah M, Knoch D, Waxman E. The state of ophthalmology medical student education in the United States and Canada, 2012 through 2013. Ophthalmology. 2014;121(6):1160–1163.
Kelly LP, Garza PS, Bruce BB, Graubart EB, Newman NJ, Biousse V. Teaching ophthalmoscopy to medical students (the TOTeMS study). Am J Ophthalmol. 2013;156(5):1056–1061.
Mackay DD, Garza PS, Bruce BB, et al. Teaching ophthalmoscopy to medical students (TOTeMS) II: a one-year retention study. Am J Ophthalmol. 2014;157(3):747–748.
Kwok J, Liao W, Baxter S. Evaluation of an online peer fundus photograph matching program in teaching direct ophthalmoscopy to medical students. Can J Ophthalmol. 2017;52(5):441–446.
Ricci LH, Ferraz CA. Ophthalmoscopy simulation: advances in training and practice for medical students and young ophthalmologists. Adv Med Educ Pract. 2017;8:435–439.
Yusuf IH, Ridyard E, Fung THM, Sipkova Z, Patel CK. Integrating retinal simulation with a peer-assessed group OSCE format to teach direct ophthalmoscopy. Can J Ophthalmol. 2017;52(4):392–397.
Schulz C, Moore J, Hassan D, Tamsett E, Smith CF. Addressing the “forgotten art of fundoscopy”: evaluation of a novel teaching ophthalmoscope. Eye. 2016;30(3):375–384.
Hoeg TB, Sheth BP, Bragg DS, Kivlin JD. Evaluation of a tool to teach medical students direct ophthalmoscopy. WMJ. 2009;108(1):24–26.
Mccarthy DM, Leonard HR, Vozenilek JA. A new tool for testing and training ophthalmoscopic skills. J Grad Med Educ. 2012;4(1):92–96.
Milani BY, Majdi M, Green W, et al. The use of peer optic nerve photographs for teaching direct ophthalmoscopy. Ophthalmology. 2013;120(4):761–765.
Channa R, Zafar SN, Canner JK, Haring RS, Schneider EB, Friedman DS. Epidemiology of eye-related emergency department visits. JAMA Ophthalmol. 2016;134(3):312–319.
Baker LC, Baker LS. Excess cost of emergency department visits for nonurgent care. Health Aff. 1994;13(5):162–171.
Ludwig CA, Newsom MR, Jais A, Myung DJ, Murthy SI, Chang RT. Training time and quality of smartphone-based anterior segment screening in rural India. Clin Ophthalmol. 2017;11:1301–1307.
Toy BC, Myung DJ, He L, et al. Smartphone-based dilated fundus photography and near visual acuity testing as inexpensive screening tools to detect referral warranted diabetic eye disease. Retina. 2016;36(5):1000–1008.
Rajalakshmi R, Arulmalar S, Usha M, et al. Validation of smartphone based retinal photography for diabetic retinopathy screening. PLoS One. 2015;10(9):e0138285.
Ryan ME, Rajalakshmi R, Prathiba V, et al. Comparison among methods of retinopathy assessment (CAMRA) study: smartphone, nonmydriatic, and mydriatic photography. Ophthalmology. 2015;122(10):2038–2043.
Adam MK, Brady CJ, Flowers AM, et al. Quality and diagnostic utility of mydriatic smartphone photography: the smartphone ophthalmoscopy reliability trial. Ophthalmic Surg Lasers Imaging Retina. 2015;46(6):631–637.
Russo A, Morescalchi F, Costagliola C, Delcassi L, Semeraro F. Comparison of smartphone ophthalmoscopy with slit-lamp biomicroscopy for grading diabetic retinopathy. Am J Ophthalmol. 2015;159(2):360–364.e1.
Figure S1 COSMOS questionnaire.