Program-Specific Podcasting as a Supplemental Learning Tool for Subspecialty Fellows

Introduction

Subspecialty fellowship training is a tremendous educational undertaking during which medical knowledge and skills are honed to address a particular focus within medicine. Even the most rigorous and comprehensive program cannot account for the entirety of its field during training. Thus, there exists gaps in knowledge, experience, and expertise for certain disease processes unique to each fellowship training program.¹

When attempting to improve specific knowledge gaps, program interventions are limited by various barriers including time constraints, schedule coordination, and available learning space.^2,3 Over the past few decades, the global movement of free open-access medical education (FOAM) has allowed online multimedia tools to gain popularity by addressing some of these barriers. Podcasts are one example of the asynchronous learning medium integral to the FOAM movement, and as a means of education are able to clear many of the hurdles that exist to other methods of teaching.⁴ Without the need to coordinate schedules or secure physical space, this educational method is far more accessible to its learners when compared to traditional lecture formats.

Podcasts first began to manifest in the mid-2000s and have since grown in audience and scope.⁵ Now, an incredible number of medical learners use this accessible means to support their education. Medical podcasting specifically has been shown to stimulate high-level cognitive skills in addition to the expected passive knowledge transfer.^6–9 However, formalized inclusion of these digital tools into the curricula of medical trainees is not yet widespread. With this paradigm shift, clinician educators need to adapt their teaching methodology to integrate this digitized content into existing curricula. Vital to this is the transformation of these tools into evidence-based, theory-driven means of teaching that meet the same standards as traditional medical education.

Nationally, many pulmonary and critical care medicine fellowship training programs have curricula that remain inpatient-focused. This has led to fellow-perceived gaps in ambulatory training.¹⁰ Few standardized outpatient curricula for subspecialty fellowship programs have been developed, and there have been no published data on the utilization of medical podcasts to supplement outpatient pulmonary education in a tailored manner. We aimed to use this tool to address gaps in our fellowship’s outpatient curriculum, and hypothesized that using podcasts to deliver this material would be an effective tool for our fellows. If successful, our goal would then expand to the creation of a reproducible model for academic podcasting that other subspecialty programs could use for their own purposes.

Methods

Study Design and Population

This was a prospective observational cohort study assessing the impact of a podcast curriculum on PCCM fellows from July 2022 to June 2023 at University of California San Diego (UCSD) Healthcare System. First, second and third-year fellows from UC San Diego Health’s PCCM fellowship program were eligible to participate in this podcast curriculum during the 2022–2023 academic year. The fellows volunteered to participate in this podcast curriculum and were provided with an online gift card for their participation. No formal informed consent was used in the collection of participant responses or other aspects of this study, which was determined to meet exemption status by the Institutional Review Board of UC San Diego Health (Exempt #802268).

Needs Assessment

The course was developed using the Kern Model for curriculum development.¹¹ We designed a targeted needs assessment to evaluate the impact of the current curriculum on general knowledge of outpatient pulmonary medicine and to identify subject material that would benefit from additional instruction (Supplementary Figure 1). This survey was completed by 15 graduates who completed UCSD’s PCCM fellowship at the end of the 2020–2021 academic year. All respondents reported a need for additional teaching in outpatient pulmonary medicine topics. Based on the survey results, eight topics were selected for the initial batch of podcast episodes. One of these topics was adjusted based on program leadership input, patient population needs, and other planned educational interventions. The selected topics include severe asthma management, severe chronic obstructive pulmonary disease (COPD) management, interstitial lung disease (ILD), idiopathic pulmonary fibrosis (IPF), respiratory failure in neuromuscular disease, evaluation of chronic cough, evaluation of wheezing, and pre-operative assessment for lung resection.

Curriculum Development

We designed a formal curriculum that included 8 podcasts on the selected topics. Specifically, we identified a faculty member with expertise focused on each of the topics. Interview questions were drafted for each session and shared with their respective faculty beforehand for review and adjustment if needed. These questions were designed to address content at the level of a subspecialty fellow and aligned with learning objectives based on subspecialty board composition, recent advances in the field, and expert consultation (Supplementary Interview TemFigure). Episodes used a conversational question and answer format. These one-on-one interviews were conducted in a quiet, private environment and recorded with multiple takes to ensure optimal sound quality and clarity. Each podcast covered the selected topic over 10 to 20 minutes with the recommended length of time selected based on prior studies.¹² Equipment consisted of an Audiotechnica microphone, Moukey isolation shield, and Apple iPhone 12. These recordings were then transferred to personal computers as MP3 files, where they were edited using Audacity’s digital audio editing software. This process included the removal of repeated takes as well as further refinement to reduce ambient or distracting noise and correct to volume differences. The finalized episodes were uploaded to the university’s online educational platform (Canvas), where they could be accessed by learners. In addition to the interviews themselves, additional educational resources supplemented each episode, including landmark trials and corresponding patient cases from a previously-published curriculum.

Implementation

Fellows were first provided access to this podcast curriculum through Canvas at the start of the academic year following fellowship orientation. Before listening to the episodes, fellows were asked to complete a pre-course questionnaire. Fellows were provided protected time during two of their Pulmonary Clinics to listen to these lectures and review the associated supplemental material. Supervising attendings were present to answer any questions regarding the podcasts, landmark trials, and the patient cases. Some fellows elected to listen to the podcasts during these clinics while others elected to listen to them at other more convenient times. Following completion of all eight episodes, a post-test was administered, roughly six months after the pre-course questionnaire was taken and the series launched.

Data Collection

Pre- and post-course questionnaires were constructed using questions that had been reviewed by a small group of pulmonologists to assess for validity, suitability and completeness. As questions were reviewed by the study design team, a pilot validation study was not performed. The pre-course questionnaire addressed both medical knowledge and comfort level with these eight disease processes (Supplementary Figure 2). Fellows were asked what their comfort level was managing patients with these conditions using a five-point Likert scale ranging from extremely uncomfortable to extremely comfortable This was used as a baseline for content reaction (Kirkpatrick level 1), while multiple-choice questions directed at covered content were used to establish knowledge (Kirkpatrick level 2) preceding the course material. The post-course survey assessed their overall satisfaction and perceived effectiveness of the podcasts and also re-evaluated their medical knowledge and comfort level in these selected topics (Supplementary Figure 3). Both surveys were distributed using an online secure resource (Qualtrics.com).¹³

Statistical Analysis

Statistical analysis of the pre- and post-test results was performed using both descriptive statistics and the two-tailed T-test. The mean score with standard deviation (SD) was calculated. Two-sided P value <0.05 was considered statistically significant.

Results

Of the twenty-two fellows enrolled in the PCCM fellowship program at UCSD during the 2022–2023 academic year, sixteen completed the pre-course questionnaire. Of these sixteen individuals, eleven completed the full podcast series and post-course questionnaire. To better evaluate the impact of the curriculum and content itself, our analysis included only the data of those eleven participants who completed the full series and questionnaire set.

Figure 1 demonstrates participant comfort with each addressed topic, both before and after completion of the podcast series. Fellow comfort improved for all topics, with statistical significance achieved in all but COPD and chronic cough. Respiratory failure in neuromuscular disease had the largest increase as measured by average responses to the Likert-scale questions (P = 0.0077), followed by pre-operative assessment for lung resection (P = 0.0072) and severe asthma management (P = 0.0016).

Figure 1 Reported comfort levels by participants using Likert-like scale, before and after podcast curriculum. 1: Extremely uncomfortable, 2: Somewhat uncomfortable, 3: Neutral, 4: Somewhat comfortable, 5: Extremely comfortable. *denotes statistical significance via two-tailed paired T test.

Figure 2 shows changes in participant knowledge on each topic, again before and after completion of the podcast series. Fellow knowledge improved in all topics other than for ILD, where it had no change. However, statistical significance was only achieved for severe COPD management (P = 0.0016), respiratory failure in neuromuscular disease (P = 0.0111), and pre-operative assessment for lung resection (P = 0.0026).

Figure 2 Percent of knowledge assessment questions (3 per module) answered correctly by participants, before and after podcast curriculum. *denotes statistical significant via two-tailed paired T test.

Discussion

We utilized this popular digital tool to address perceived knowledge gaps in fellowship education at our training program. This podcast successfully achieved our goal of supplementing this knowledge on outpatient pulmonary medicine topics. We demonstrated that formal incorporation of podcasts into a curriculum is both feasible and well-received. Our study adds to the existing knowledge on medical education podcasts.

Advantages

The popularity of medical podcasts as a form of supplemental educational material continues to rise and can be increasingly utilized in subspecialty education to improve learner proficiency and comfort with subspecialty subject matter. Specifically, our study showed that fellow comfort and knowledge of disease processes improved for the majority of addressed topics. Other advantages to the educational format include accessibility to larger audiences and flexibility to learners’ schedules. Content can be consumed while engaging in other activities that allow for split focus and require only a phone and headphones from the perspective of the learner. Beyond the expected passive absorption that listening affords, a systematic review examining the most popular medical education podcasts in the United States suggested that the format stimulates higher-level cognitive skills rather than simply serving as a transfer of knowledge.⁶ Knowledge of pathophysiology and pharmacology for a particular disease process is vital to its management, but it is the improvements felt by our learners in their comfort caring for these conditions that we feel best reflect that described benefit to skills such as critical thinking and clinical reasoning.

Limitations

However, despite its numerous benefits, limitations do exist for this modality of curricular support. This format is not accommodating for learners with hearing impairments, or those who need hands-on or primarily visual teaching to support their learning, and requires some form of playback device with internet connectivity to consume recorded material. Also, certain types of content such as procedural training, do not lend themselves well to an auditory-only format of teaching. Studying specific program interventions at the fellowship level too can be difficult, given how small specific program populations often are. In the case of our study, only eleven participants completed the full curriculum and surveys, thus introducing the potential for type I error as well as selection bias. In addition to study power, alignment between podcast content and survey questions, ceiling effect, or other factors may have contributed to the fact that not all improvements in topic comfort and knowledge reached statistical significance. These limitations will likely affect others implementing similar curricula in programs of their own. Repeated evaluations over years may help to improve the power of quantifying the impact in these interventions.

Implementation Strategies

Reflecting on our own experience in developing a podcast curriculum, we identified several key aspects that made our intervention successful. The first step for creating tailored educational materials include identifying an educational need or specific area of concern to focus your subject matter. This information can be solicited through a targeted needs assessment questionnaire (Supplementary Material: Supplementary Figure 1). Secondly, podcasts in particular require thoughtful consideration of structure and tone. We selected an interview format for our podcast, as we felt that it would best highlight necessary material based on the learning objectives. Additionally, it is important to develop a high-quality recording process, familiarizing oneself with the equipment and software necessary to capture and edit discussions. Creating a podcast requires familiarity with these tools to ensure optimal sound quality, as audio quality significantly impacts listener enjoyment and perceived credibility. Once these sessions are completed, they will need to be made available for your learners to download and consume on their own time. Therefore, a distribution model needs to be considered. Finally, by implementing a survey or other assessment tools before release, you can measure the impact of your curriculum and identify areas for improvement and expansion in the future. Ensuring that the content of a podcast remains accurate and relevant is crucial to successfully implementing podcasts as part of a curriculum. While we only captured a single year in our study, a large potential benefit of the medium is the ability to update material over time to accommodate the changing needs of future learners.

Future Directions

Moving forward, we plan to expand our library of podcast episodes following collection of updated needs assessment data. We are also interested in the long-term impact of this curriculum, and will evaluate this with follow-up surveys for graduates of the fellowship program. This will allow us to analyze not only retention, but even more meaningful Kirkpatrick level 3 and potentially even 4 outcomes as well. It would be of great benefit to evaluate impact of tailored medical podcasting over time, and even more so its effect on patient outcomes. Most important for this educational tool’s development however, will be assessing it’s impact on other fellowship, residency, or additional training programs.

Conclusion

Podcasts are a supplemental learning tool that can be tailored to the specific needs of an educational program. Our study demonstrated that when implemented following targeted needs assessment, this teaching modality can improve trainee comfort and knowledge in topics previously identified as “knowledge gaps” for a particular medicine fellowship training program. Beyond intentional selection of material and recruitment of expert discussants, effective implementation of a program-specific podcast requires only a small number of relatively low-cost technological purchases, some familiarity with digital editing, and a means to access and listen to episodes for its learners. We believe that our method can be used as a framework for other fellowship programs to initiate their own supplemental learning podcasts. With more widespread use and further study, we may discover additional applications or benefits of podcasting in medical education in the future.