An active learning curriculum improves fellows’ knowledge and faculty teaching skills

Introduction

Teaching specialty fellows, who have competing demands for their time, presents unique challenges. Traditional didactic lectures allow teachers to convey important information in a set amount of time, yet actively engaging learners may lead to better conceptual understanding and allow the learner to develop skills needed to solve problems later on in life.^1–4 To build a strong knowledge base while simultaneously mastering the art of clinical consultation and procedures, fellowship programs should encourage educational conferences that are active, high-yield, and learner-centric.

At our institution, educational conferences are predominantly lecture based, with little or no active learning components. We created an active learning curriculum for gastroenterology fellows and coached the faculty presenters on techniques to encourage active learning. Our goals were the following: 1) to evaluate whether a new active learning curriculum improved self-reported learning and educational satisfaction among fellows, 2) to determine whether coaching faculty members and giving them tools to create an active learning presentation improved their perceived teaching and presentation skills, and 3) to assess whether the fellows’ performance on the annual American Gastroenterological Association (AGA) GI Training Exam (GTE), an annual online multiple-choice assessment tool for Gastroenterology Fellowship programs, improved after the implementation of this active curriculum.

Methods

This curriculum was initially implemented in November 2014 at Brigham and Women’s Hospital (BWH), a large academic teaching hospital in Boston, MA. The curriculum was directed by the authors and was given to 15 BWH gastroenterology fellows, representing all 3 years of the fellowship. Ethical approval was obtained from the Partners Institutional Review Board.

Curriculum format

The curriculum was composed of monthly 1-hour presentations. Each presentation was an integrated one, given by one author and an invited faculty member from BWH or Beth Israel Deaconess Medical Center (Boston, MA), who were selected on the basis of their area of expertise. The first 2 years of the curriculum covered the following topics (Table 1).

Table 1 Curriculum topics for first and second year of implementation

The topics covered were not presented in other didactic conferences, or were areas of fellowship weakness as identified by the annual GTE or by annual fellowship evaluation surveys. Presentation content was guided by the AGA Fellowship curriculum, clinical guidelines, and current literature.

Each 60-minute session had a set template as given in Table 2.

Table 2 Template for each 1 hour session

Specific educational tools were consistently utilized with each session. First, fellows answered the pre- and posttest multiple-choice questions using an Immediate Feedback Assessment Ticket (IF-AT; Epstein Educational Enterprises, Cincinnati, OH, USA; Figure 1). The multiple-choice questions and IF-AT tickets were used for the purpose of self-assessment, so fellows could briefly gauge their knowledge of the topic at hand. The IF-AT is a multiple-choice answer card with a thin opaque film covering four answer options. Each fellow independently scratched off his or her answer, as if scratching a lottery ticket, until he or she reached the correct answer, which is marked with an asterisk. These IF-AT cards were not collected and were used for self-assessment only. The authors have no record of how many attempts each fellow made to reach the correct answer.

Figure 1 IF-AT (Epstein Educational Technologies) scratch card. Notes: Each fellow was given an IF-AT scratch card at the beginning of each session to answer pre- and posttest multiple-choice questions. The cards were not collected for review. Fellows were encouraged to scratch off their answer choice with a coin, similar to a lottery ticket. An asterisk indicated the correct answer. Fellows were asked to keep scratching boxes until the correct answer was reached. Reproduced with permission from Colbert J, Pelletier S, Xavier-Depina F, Shields H. A pilot study of team learning on in-patient rounds. Clin Teach. © 2015 John Wiley & Sons Ltd.5 Abbreviation: IF-AT, Immediate Feedback Assessment Ticket.

New pre- and posttest multiple-choice questions were created for every session. Pretest multiple-choice questions focused on physiology, anatomy, and histology, to be covered in the first 10 minutes of the session. Posttest multiple-choice questions focused on pathophysiology, diagnosis, and management, which were covered by the expert lecturer.

A representative pretest question from “Disorders of Acid Secretion” was as follows:

1. The production of bicarbonate, which is an important part of gastric mucosal defense, is enhanced by the secretion of:

   1. Mucin

   2. Pepsin

   3. Prostaglandins

   4. Histamine

   The correct answer is c.

   A representative posttest question from “Disorders of Acid Secretion” was as follows:

2. A 50-year-old woman is referred to you by her PCP due to a positive Helicobacter pylori serology. She has a history of peptic ulcer disease with biopsy-confirmed H. pylori. She was previously treated with a proton pump inhibitor (PPI), amoxicillin, and clarithromycin. She is currently asymptomatic and on no medications. What is your recommendation?

   1. Confirm persistence of H. pylori by stool antigen test or urea breath test

   2. Re-treat with PPI, bismuth, and tetracycline for 14 days

   3. No further action is necessary since she is asymptomatic

   4. Upper endoscopy with biopsy

   The correct answer is a.

Second, a double-sided 8×10 inch color “Take Away” handout was provided to the fellows after each presentation, which served as a quick reference and summary for important tables, figures, algorithms, and images. Each presentation and associated “Take Away” were available to the fellows online for future reference. The “Take Away” was discussed during the last 5 minutes of the presentation. Each figure or image was explained, and questions were solicited from the fellows to make sure they understood the key concepts. Another document with references and detailed answers to the multiple-choice questions was also provided.

Each invited faculty expert had three 1-hour meetings with the series directors prior to their presentation. The first meeting was a month before the presentation, where the presentation format stressing active learning techniques was introduced and discussed in detail. This meeting also served to discuss presentation content. The second meeting was 2 weeks before the presentation and served as an initial review of the faculty member’s PowerPoint slide set. During this meeting, faculty members were coached to ensure that their presentation actively involved fellows. Content was presented in a case-based manner, and a variety of types of questions were deliberately scattered throughout the lecture to encourage fellow participation and discussion. Faculty were coached to use open-ended questions, as well as diagnostic, asking for information, priority, prediction, and action questions.⁶ In addition, presenters were asked to remove slides with dense text and replace these with easier-to-grasp algorithms or high-yield diagrams. Animated slides were encouraged, as this increases learner participation and discussion before information is revealed. The last meeting was a day or two before the presentation, where the faculty member gave a “dress rehearsal” of his or her presentation. Faculty members were coached to speak more slowly and clearly, make eye contact, stand up while presenting, and project their voices better.

Assessment

Three sets of GTE scores before (2012–2014) and two sets of GTE score after (2015–2016) the implementation of this active curriculum were compared as measures of improved knowledge. In addition, each fellow was invited to answer an anonymous paper evaluation with free-text response and Likert scale questions (Likert rating: 1= not at all, 2= somewhat, 3= very, and 4= extremely) after every presentation. Faculty presenters were also invited to answer an online anonymous evaluation (Qualtrics Inc., Provo, UT, USA) at the end of the academic year. All responses were anonymous and were sent to the statistician for subsequent analysis. GTE scores before and after the implementation of this curriculum were compared using a t-test. A p-value <0.05 was considered significant. This study was approved by the Partners Healthcare Institutional Review Board in 2014. This protocol was classified as exempt, and informed consent was not required from the fellows or faculty.

Results

Fellow responses regarding utility of curriculum

The overall survey response rate from the fellows was 47% (85/180) over 2 years. All the learners rated the 12 lectures as “very good” or “excellent,” with the mean Likert scale score of 3.74 and 3.96 for the first and second years, respectively (p=0.005). Learners agreed that the “Take Away” handouts were either “very good” or “excellent,” with mean Likert scale scores of 3.63 and 3.86 for the first and second years, respectively (p=0.015). All learners also felt that they were either “somewhat more” or “a lot more” prepared to take the GTE as a result of the lectures, with mean Likert scale scores of 3.67 and 3.81 for the first and second years, respectively (p=0.156) (Table 3).

Table 3 Fellows’ anonymous paper evaluation of active curriculum’s utility over a 2-year period Notes: A total of 15 fellows were invited to each session. Each fellow who attended the session was asked to fill out a paper evaluation form. A total of 85 surveys were completed by fellows, out of a possible 180 surveys (47.2%). aLikert rating scale: 1= not at all, 2= somewhat, 3= very, 4= extremely. bLikert rating scale: 1= not at all, 2= not too much, 3= somewhat, 4= a lot more. Abbreviation: GTE, GI Training Exam.

The authors asked the fellows the following question on the anonymous survey following each session: “Did this lecture format (questions, lecture, and a “Take Away” handout) make you feel that you understood/learned the material better than the ‘usual’ lecture format?” All survey responses indicated that fellows preferred this new format of learning over the traditional didactic style. Fellows commented, “The interactive nature of the lecture was very helpful and makes us think. The case-based approach was excellent... and the format was very helpful. I will be modeling this format for a resident review on GI emergencies.”

Faculty responses regarding utility of curriculum

The response rate for faculty evaluations was 100% (12/12). All faculty members responded that teaching is part of their current position and they teach fellows “a lot” (50%, 6/12) or “somewhat” (50%, 6/12). All faculty members were involved on a local, regional, and/or national basis. All faculty members agreed that they acquired new teaching skills, improved their teaching skills, improved presentation/speaking skills, received useful feedback during the drafting process, and learned new resources and methods for teaching that would help future teaching; furthermore, they said they valued continued meetings and communication with the authors. All faculty said they would participate again. Further details are given in Table 4.

Table 4 Faculty’s anonymous qualtrics questionnaire regarding utility of being coached for participation in an active curriculum Notes: Likert scale: 1= strongly disagree, 2= disagree, 3= neither agree nor disagree, 4= agree, 5= strongly agree. Abbreviation: AGA, American Gastroenterological Association.

Effect on GTE scores

GTE scores were significantly improved in the areas related to the stomach (p=0.001), general gastroenterology (p=0.005), esophagus (p=0.001), and small bowel (p=0.001), and the total score (p=0.001) between 2012–2014 and 2015–2016 for all fellowship years combined. Scores in hepatology and the study of the biliary, pancreas, and colon showed a trend toward improvement (p=0.05). Table 5 shows further details regarding the change in GTE score percentage after this curriculum was implemented.

Table 5 Change in GTE score percentage between 2012–2014 and 2015–2016 Notes: There were three sets of scores between 2012 and 2014 and two sets of scores between 2015 and 2016. Fifteen fellows took the GTE each year. “+” reflects an improvement in GTE score. “–” reflects a decline in GTE score. “*” reflects a significant improvement in score. Abbreviation: GTE, GI Training Exam.

Conclusion

Gastroenterology fellows preferred this new active learning curriculum over traditional didactic lectures, finding the sessions engaging, high-yield, and educationally stimulating. The lecturers and the educational curriculum received excellent ratings from the fellows. In addition, faculty valued the three coaching sessions and agreed that they improved their teaching and presentation skills as a result. This novel active curriculum resulted in fellows reporting that they felt more prepared to take the GTE and that they understood the material better than they did in the traditional didactic lectures. The GTE scores improved in certain areas after the introduction of this curriculum. However, a definite relationship between the score improvement and the didactic series cannot be proven.

Learners today prefer “active learning,” classically defined by Malcolm Knowles. His learning theory is based on the principles that adults need to know the reason for learning something and that they need to be driven by internal motivators. Learners need problem-centered learning rather than memorization, and topics to be learned must have immediate relevance to their life. Learners also must be involved in the planning and evaluation of their learning.^7–9

Active learning requires students to be part of the discussion and to analyze, synthesize, and evaluate information and attempt to solve problems.^10–12 When students in active learning sessions begin to explore topics off target, it is the responsibility of the facilitator to bring them back to the main objectives. Currently, examples of active learning include web-based modules, case-based sessions, classroom response systems such as “Poll Everywhere,” the use of a variety of questions to engage the audience, and IF-AT scratch tickets.^10,11,13,14, We chose several of these active teaching strategies to make our sessions active rather than passive.

Our findings that fellows preferred this method of active learning over a more traditional lecture-based format support what has been previously reported in the literature for residents.^10,11,15,16 We are unaware of studies showing the practical benefits of an active learning curriculum in a specialty fellowship training program. Our fellows perceived that they were more prepared for future examinations and that this format enhanced their learning and engagement.

Traditional didactic lectures, while leading to short-term gains in knowledge, have not been shown to lead to long-term retention of knowledge.^1–4 Active participation in educational sessions has been reported by residents to increase attention and engagement learning.¹⁷ The format of active learning sessions is important. Residents report that the most effective teaching sessions are shorter than the traditional 60-minute lecture and focus on a few high-yield concepts that are structured around cases and questions.

We recognize that active learning sessions may be more time consuming for the presenter to create since these sessions involve using new interactive techniques. Despite the time commitment, the literature shows the benefit of faculty development, including peer-to-peer coaching skills, as our data show.¹⁸ Faculty development programs have been shown to be essential in boosting teachers’ confidence and creating awareness of effective educational practices.¹⁹

Our active learning curriculum has several strengths, including IF-AT scratch cards to involve all learners in self-assessment. The curriculum also includes an easy-to-reference “Take Away” handout that summarizes key points that can be used as a future reference. In addition, all materials are available on a password-protected website for the fellows. Faculty members enjoy receiving one-on-one coaching. Finally, this curriculum template may be generalizable to trainees in all subspecialties. The limitations of our study include the fact that the pilot curriculum was performed at a single institution with a small number of participant fellows and faculty. Second, it is not easy to carve out dedicated time for faculty development. Third, the objective improvement on GTE scores before and after the implementation of this curriculum could be due to other factors that were not known or assessed. Fourth, most faculty members are comfortable giving traditional didactic lectures, and may be resistant to change, although this was overcome by personalized coaching sessions.

In summary, this active learning curriculum was effective in transmitting important and high-yield information to busy Gastroenterology fellows and received excellent feedback from both fellows and faculty. Coaching sessions were important for teaching excellence and were perceived as benefitting both the coached teacher and the learner. On the basis of these data, this educational template for 1-hour, active learning sessions may be a useful tool for other subspecialty fellowship programs.

Acknowledgments

This work was funded by the Division of Gastroenterology, Hepatology and Endoscopy and Division of Medical Communications; Brigham and Women’s Hospital.

This work was presented at Digestive Diseases Week 2016 (San Diego, CA, USA) at an educational symposium sponsored by the American Gastrointestinal Association (AGA) on May 23, 2016, and at a medical education poster session on May 24, 2016.

Author contributions

All authors met the following criteria for authorship: 1) substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data; 2) drafting the article or revising it critically for important intellectual content; 3) final approval of the version to be published; and 4) agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Disclosure

The authors report no conflicts of interest in this work.

References

1.		Warner S, Williams DE, Lukman R, Powell CC 2nd, Kundinger K. Classroom lectures do not influence family practice residents’ learning. Acad Med. 1998;73(3):347–348.
2.		Picciano A, Winter R, Ballan D, Birnberg B, Jacks M, Laing E. Resident acquisition of knowledge during a noontime conference series. Fam Med. 2003;35(6):418–422.
3.		McDonald FS, Zeger SL, Kolars JC. Factors associated with medical knowledge acquisition during internal medicine residency. J Gen Intern Med. 2007;22(7):962–968.
4.		Winter RO, Picciano A, Birnberg B, et al. Resident knowledge acquisition during a block conference series. Fam Med. 2007;39(7):498–503.
5.		Colbert J, Pelletier S, Xavier-Depina F, Shields H. A pilot study of team learning on in-patient rounds. Clin Teach. 2016;13(1):38–42.
6.		Shields HM, Guss D, Somers SC, et al. A faculty development program to train tutors to be discussion leaders rather than facilitators. Acad Med. 2007;82(5):486–492.
7.		Knowles M. Self-Directed Learning. Chicago, IL: Follet; 1975.
8.		Knowles M. The Adult Learner: A Neglected Species. 3rd ed. Houston, TX: Gulf Publishing; 1984.
9.		Knowles M. Andragogy in Action. San Francisco, CA: Jossey-Bass; 1984.
10.		Gonzalez-Sancho JM, Sanchez-Pacheco A, Lasa M, Molina S, Vara F, del Peso L. The use of an active learning approach to teach metabolism to students of nutrition and dietetics. Biochem Mol Biol Educ. 2013; 41(3):131–138.
11.		Thistlethwaite JE, Davies D, Ekeocha S, et al. The effectiveness of case-based learning in health professional education. A BEME systematic review: BEME guide no. 23. Med Teach. 2012;34(6):e421–e444.
12.		Smith AR Jr, Christie C. Facilitating transdisciplinary teamwork in dietetics education: a case study approach. J Am Diet Assoc. 2004;104(6):959–962.
13.		Steele DJ, Johnson Palensky JE, Lynch TG, Lacy NL, Duffy SW. Learning preferences, computer attitudes, and student evaluation of computerised instruction. Med Educ. 2002;36(3):225–232.
14.		Bernardo V, Ramos MP, Plapler H, et al. Web-based learning in undergraduate medical education: development and assessment of an online course on experimental surgery. Int J Med Inform. 2004;73(9–10):731–742.
15.		Freeman S, Eddy SL, McDonough M, et al. Active learning increases student performance in science, engineering, and mathematics. Proc Natl Acad Sci U S A. 2014;111(23):8410–8415.
16.		Mishra K, Snow-Lisy DC, Ross J, Goldfarb DA, Goldman H, Campbell SC. Evaluation of a case-based urology learning program. Urology. 2013;82(6):1207–1210.
17.		Sawatsky AP, Zickmund SL, Berlacher K, Lesky D, Granieri R. Understanding resident learning preferences within an internal medicine noon conference lecture series: a qualitative study. J Grad Med Educ. 2014;6(1):32–38.
18.		Steinert Y, Mann K, Centeno A, et al. A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME guide no. 8. Med Teach. 2006;28(6):497–526.
19.		Steinert Y, Mann K, Anderson B, et al. A systematic review of faculty development initiatives designed to enhance teaching effectiveness: a 10-year update: BEME guide no. 40. Med Teach. 2016;38(8):769–786.