Eyes for Ears: Usage and Efficacy of a Podcast for Ophthalmic Education

Introduction

Medical education innovations beyond traditional lectures and textbooks have seen recent implementation in ophthalmology, including case-based learning and the “flipped classroom” model, among other strategies.¹ Further, new remote teaching modalities such as blogs, virtual surgical simulation, online seminars, and podcasts have emerged in general medical education.^2–4 The added utility of these new modalities was made more obvious than ever by the COVID-19 pandemic’s alterations to traditional medical education.^5–8

In particular, the use of podcasts as an educational tool has also been adopted by many specialties.^3,9–11 Formal incorporation of podcasts in ACGME Emergency Medicine program requirements as “asynchronous learning” has been proposed.¹²

Although there were ophthalmology-focused podcasts in 2019, at that time, few were dedicated specifically to resident and trainee ophthalmic education.¹³ In response to this, the authors created Eyes for Ears, a podcast dedicated to ophthalmic education and test preparation. We previously described this podcast, which provides a free, open-access supplemental review of the Basic and Clinical Sciences Course (BCSC) curriculum fundamentals, and its widespread uptake and use among trainees.¹⁴

Therefore, the aim of our study was to evaluate the demographics, perceived effectiveness, and lister usage of Eyes for Ears, as well as focus on comparison of the podcast to other resources and benefit of using the podcast for wellness. We also report listener usage metrics as tabulated by the podcast host.

Methods

Podcast Creation

The Eyes for Ears podcast was launched in January 2019, and the details of its production have been detailed elsewhere.¹⁴ The authors (BY, AP and AR) co-host the production and release episodes biweekly. Each episode focuses on a testable subject covered in the BCSC curriculum. Episodes are roughly 20–30 minutes in length. The podcast is independently funded and maintained by the authors.

Podcast Metrics

Listenership (usage) metrics are continually and automatically tabulated by the podcast hosting website (Podbean, New York NY), and include how many individual listens/downloads the podcast has received as well as geographic data. This geographic data tracked the country from which listeners accessed the podcast, and within the United States and Canada, also tracks the state or province from which the listener accessed the podcast. The geographic data is only available from the podcast host for the most recent 24 months prior.

Survey Creation and Distribution

An 8-question survey was created by the authors, and was then reviewed and piloted by a contributing author who did not take the survey (BKY) (Appendix 1). Residents and students did not pilot the survey.

The final survey tool elicited information regarding 1) An assessment of the respondent’s eyecare background and level of training, 2) Preferred podcast use (ie number of podcasts and place of use), 3) Perceived improvement of knowledge, 4) How the podcast compared to other current educational resources, and 5) If listening to podcast increased time for self-care. For the survey questions that asked respondents to apply a quantitative estimate for their impressions, a modified Likert scale ranging from 0 to 100 was used, with 0 indicating no preparedness, and 100 indicating complete preparedness.

The survey was then distributed via the Qualtrics survey platform (Qualtrics, Provo UT) between July 2019 – January 2020. The survey was distributed using the podcast itself, through the podcast website, and Twitter (Appendix 2). Inclusion criteria were listeners who self-identified on the survey as ophthalmologists, optometrists, ophthalmic technicians, and their respective trainees that had listened to at least one episode of Eyes for Ears. Exclusion criteria included respondents reporting no background in eye care. Responses that were likely erroneous or duplicate submissions were identified by near-similar survey submission timestamps within 10 seconds of each other that also had exact-match corresponding survey answers and were removed such that only a single entry of the two (or multiple) submissions remained.

The study protocol was approved by the Yale School of Medicine Institutional Review Board, and survey takers were first presented with an informed consent. There were no additional risks or benefits to survey takers.

The primary outcomes were user confidence in the Ophthalmic Knowledge Assessment Program (OKAP) or written board exams before and after listening to the podcast. Secondary outcomes included when and how much respondents listened to the podcast, if the podcast increased time for self-care activities, and comparison of helpfulness of the podcast for clinical or test preparation compared to other educational modalities.

Statistical Analysis

We analyzed the results of preferred podcast use and perceived change in confidence on clinical test performance before and after podcast use. This was based off of the Kirkpatrick’s Training Evaluation Model Level 1.¹⁵ Descriptive statistics were used to summarize survey data with summary statistics reported in percentage of respondents.

Paired data was compared using Wilcoxon signed-rank test. All statistical analyses were done using STATA (StataCorp LLC, College Station TX, USA). P-values < 0.05 were considered statistically significant.

Results

Podcast Metrics

Data from the podcast host indicate that between the time when Eyes for Ears was launched on January 10, 2019 and to September 2, 2021, the podcast was downloaded over 422,000 times, averaging about 4442 downloads per episode. Geographic data listing the number of downloads per country during the 2-year period of September 2019 to August 2021 (a period of 378,900 total downloads) is partially listed in Table 1. Overall, individuals from 110 countries had accessed the podcast at least 10 times during this period (Table 1). In the United States specifically, download numbers for the same period per top 10 individual states are shown in Table 2. The states with the most downloads were California (9.64% of all downloads), New York (8.87%), and Texas (5.07%).

Table 1 Eyes for Ears Downloads by Country, During September 2019 to August 2021 (Top 10 Only)

Table 2 Eyes for Ears Downloads by US State, During September 2019 to August 2021 (Top 10 Only)

Survey Results

The Qualtrics survey received 256 responses. Of these, 47 were excluded from analysis: 28 were incomplete, 7 were from self-identified laypersons with no background in eyecare, 4 were from physicians of other non-eyecare related specialties, and 8 were identified as duplicate entries. After exclusion, 209 responses remained for analysis. Due to the method in which the survey was distributed, and that all podcast users would not meet inclusion criteria, it was unknown how many includable users were aware of the survey, so a precise survey response rate was not calculable. As the target audience is primarily PGY2-4 ophthalmology residents, a lower estimate can be generated by assuming all US PGY2-4 ophthalmology residents were aware of the survey in July 2019, resulting in a response rate of 7.9% (111/1401).

Respondent training level is summarized in Table 3; the majority were residents (126/209, 60.3%), representative of the target audience of the podcast. The other listeners were in large majority other ophthalmology or optometry learners—29.2% (61/209) identified as medical students, ophthalmology fellows, or optometry students and residents.

Table 3 Level of Training Across All Respondents; Only Ophthalmology Training Pathway Respondents (Medical Students, Residents, Fellows and Fully Trained Ophthalmologists) Were Included in Analysis Described in Text

Responses to categorical questions are summarized in Table 4. Most listeners use this podcast while commuting (81.3%), though almost a quarter listened while exercising (47/209, 22.5%). A majority also felt the podcast increased the time they could spend on self-care (132/209, 63.2%), and thought it improved their fund of knowledge (207/209, 99.0%).

Table 4 Categorical Responses to Survey Questions Over Included Respondents

Responses to modified Likert scale questions are summarized in Figures 1 and 2. Respondents recalled feeling better prepared for the OKAP or written boards after using this podcast (Figure 1, before median = 50/100 on modified Likert scale, IQR 30–65 vs median = 70/100, IQR 56–81, p <0.001). Of educational materials, including question banks, podcasts, traditional lectures, literature articles, and textbooks, respondents identified question banks as the most helpful for OKAP studying, followed by podcasts (Figure 2). Podcasts were preferred over articles and lectures in terms of helpfulness (p < 0.001), and there was no statistical significance between podcast and textbook helpfulness (podcast median = 75/100 on modified Likert scale, IQR 62–90 vs textbook median = 71/100, IQR 59–84, p=0.069).

Figure 1 Respondents’ recall of their feeling of preparedness from 0–100, before and after using this podcast. Paired analysis done using Wilcoxon signed-rank test.

Figure 2 Respondents’ preference for board review materials on a scale of 0–100. Paired analyses done using Wilcoxon signed-rank test.

Discussion

The Eyes for Ears podcast achieved a robust listenership audience per the podcast host’s usage and geographic analytics data. Further, the sample of the surveyed audience indicates that these listeners felt the podcast to be useful for their education and well-being.

This work is reflective of that in other specialties done to characterize trends in medical trainees’ use of educational resources.^16,17 This is of particular interest given that the recent proliferation of nontraditional resources (including podcasts) has attained such momentum that there is now a movement towards labeling them as “Free Open Access Medical (Education)”, abbreviated FOAM.^18,19 Social media platforms such as Twitter and Instagram allow for the curated targeting of “#FOAM”-designated content, with significant uptake among students and trainees for their informal medical education.^20–22

Most survey respondents reported listening to the podcast while commuting. This is consistent with other reports that found up to three quarters of surveyed medical trainees reporting the same.²³ Gottlieb et al assessed emergency medicine residents who utilized educational podcasts during their commute with a validated knowledge test before, immediately after, and a month after this practice and found that driving while listening did not seem to negatively affect retention when compared to a control group.²⁴ This suggests that listening to podcasts while driving is an effective way to increase learning efficiency because that time would otherwise be unusable for training. Having a mechanism to use that time for learning could enhance trainee well-being; 63.2% of our survey respondents felt that the podcast had allowed them more time for self-care activities.

A previous survey of members of retina societies found that respondents in general preferred traditional medical education over a retina podcast.¹³ However, they found that younger members were more likely to prefer the podcast medium. Similarly, in this survey of trainees, podcasts were the second highest ranked preferred material, over traditional didactic lectures and textbooks. Although these differences were statistically significant, our results are affected by selection bias, as the survey respondents were existing users of the podcast. Our results do not indicate a devaluation of didactic lectures, textbooks or primary literature, but do suggest that the podcast is a valuable supplement in this group of learners.

Our study is not without limitations. The first of these is selection bias, as those that view the podcast more favorably or use the podcast more often are more likely to respond to the survey, as it was distributed using the podcast, the podcast website, and the podcast social media. Therefore, respondents may also be more skewed to younger generations of ophthalmology providers. Also, it is likely that those that responded are more likely to view the podcast favorably and find it more helpful, as this survey was done with a convenience sample rather than a more objective sampling method.

In addition, a survey response rate is difficult to calculate. As the podcast was designed for residents in the US, it can be argued that the target audience of the survey should be all US ophthalmology residents. However, the survey shows that students, fellows, and even attendings also listen to the podcast. The number of podcast downloads is also not a good estimator of target audience, as multiple downloads can be from the same individual. Furthermore, a survey response rate is difficult to estimate given the inability to detect what percentage of our known audience met exclusion criteria. Finally, while individuals may listen to the podcast, they may not always access the podcast website or Twitter, where direct links to the survey were provided.

Another limitation is that this study could not assess the relationship between podcast use and true knowledge retention, as preparedness for exams was a subjective response.

In spite of these limitations of the survey, the continued increased uptake of the podcast by itself, with no implementation of marketing strategies or advertisements, suggests trainees find it a valuable tool.¹⁴ Future directions will include assessing efficacy amongst resident trainees at university residency programs specifically in coordination with program directors, as well as potentially evaluating objective metrics such as OKAP scores for a possible association with podcast usage.

Conclusion

In summary, the Eyes for Ears podcast has achieved a broad international audience with widespread usage amongst ophthalmology students and trainees. Our survey data indicate that those who listen find it to be an effective educational tool that can improve time management efficiency and trainee well-being. This has important implications for improving medical training in ophthalmology, especially given the recent experiences from the COVID-19 pandemic that required the implementation of distance learning in some circumstances.