Flipped classroom model for learning evidence-based medicine

Introduction

Evidence-based medicine (EBM) is “the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients.”¹ There are many aims of evidence-based learning. Staying current with medical literature, improving the understanding of research and improving patient care are just a few.² While the medical literature shows a plethora of venues to teach EBM skills, these skills are frequently taught in the context of journal club (JC).^3,4

JC as a pedagogical strategy has long been used in graduate medical education (GME). Described as “a group of individuals who meet regularly to discuss critically the clinical applicability of articles in the current medical journals,” JC goes back to the early 18th and 19th centuries.^5–7 As EBM becomes a mainstay in GME, traditional models of JC present several insufficiencies. First, in traditional JC models, antiquated teaching strategies disengage and stifle resident learning.^8,9 Second, in a teaching method that fails to engage residents, learning critical appraisal skills is often limited.⁴ Finally, traditional JC models often lack the bridging between academic journals and professional practice.^9,10

The desire to connect evidence to practice while creating engaged and technologically literate physicians calls for new learning methods and strategies. A flipped classroom model appears to be an ideal strategy to meet such demands. As an instructional model, the flipped classroom leverages technology outside the class time to enhance engagement and learning in class.¹¹

There continues to be a lack of clarity on how a flipped classroom model can be implemented in the context of residency programs. In this article, we present the experience of implementing a flipped classroom model in a family medicine residency to demonstrate a working example. We describe the flow of learning activities during the online and face-to-face instruction, and then we highlight specific considerations for implementing a flipped classroom model.

Developing the flipped classroom model

The IU family medicine residency (IU-FMR) is a university-based training program. In each academic year, 13–14 new interns join the residency. In 2014, the IU-FMR program received an internal grant (US$50,000 a year over 2 years) to restructure learning from JC into a flipped classroom model. The new model built on an established JC at the residency which revolved around dialogical learning.¹²

The flow of JC in the flipped classroom model

JC class sessions occur monthly on Thursday afternoons from 12:30 pm to 2:00 pm. Family medicine residents from all three cohorts, approximately 39 residents, attend JC sessions. Considering the multiple other duties that residents participate with during their training, between 15 and 25 participants are usually present at every class session. The class discussions build on the learning that took place online. On the other hand, the online materials are usually completed in 15–30 minutes. Figure 1 shows the flow of activities in the flipped classroom model.

Figure 1 The flow of the activities in a flipped classroom model for EBM. Abbreviation: EBM, evidence-based medicine.

The flow of JC in the flipped classroom model occurs as follows. At the beginning of the year, residents are grouped into presentation teams. Each team consists of two to three residents. Each month, two teams independently select one article each from refereed journals. Thus, each JC session is dedicated to the review and critique of two articles. Teams are responsible for facilitating discussion and critique of articles during the JC session. All residents receive a notification to complete the online modules 1–2 weeks in advance. By using the flipped classroom model in JC, our aim was to have residents prepare and engage in JC in different ways. Our goal was for residents to be prepared with materials “before” they came to the classroom.

The online learning activities

Online activities aim to initiate the residents’ engagement with the article, allows residents to practice and develop critical appraisal skills, and prepares residents for in-class JC session. Residents complete the online modules asynchronously at their own pace before the class session. All residents receive a notification to complete the online modules 1–2 weeks in advance. Each module consists of an introductory page followed by three discussion forums. The introductory page presents a clinical scenario focusing on an area relevant to the article. This scenario encourages residents to reflect on their typical practice before engaging with the articles. Figure S1 shows the examples of these clinical vignettes.

In the first discussion forum, the residents are given a list of questions to assess the study’s validity. The second forum focuses on the results presented in the article and their statistical significance. Finally, the third forum enables the residents to explore the applicability of the study. In each one of these forums, residents are responsible for two tasks: answering proposed questions and responding to colleagues’ remarks. First, each resident must post their answers in the module before he or she can read other residents’ answers. After residents post their answers, they must choose either to ask a question to one of their colleagues’ answers or to present an answer to the colleagues’ proposed questions.

The validity discussion forum

A list of questions about the validity criteria is given to residents. The questions are based on the work of Straus et al.¹³ Residents are asked to answer two of the listed questions and to advance the discussion by either asking a colleague separate questions about his or her posts or by addressing one of their peers’ questions. Figure S2 shows the examples of validity questions created for four types of study’s main questions: therapy, prognosis, etiology, and diagnosis.

The results discussion forum

The discussion forum of the results addresses the results of selected journal articles. It is intended to support residents as they think critically about the clinical and statistical significance of the findings. The residents are asked to answer one of two proposed questions regarding the results. The questions are proposed in a way that encourages residents to evaluate the findings through summarizing the tables, explaining the paragraphs, or conducting calculations to check the accuracy of the reported results. Examples of the questions are shown in Figure S3.

The applicability discussion forum

The applicability forum is intended to advance discussion on the practical impact of the study in the residents’ own practices. Since the study would be applied to individual patients, the residents are asked to think of a patient they have encountered and to whom the study may apply. Then, they are asked to describe briefly the patient and answer one of four questions on applicability. Figure S4 shows the list of the applicability questions.

In-class learning activities

The in-class learning takes place in a large, group discussion format. Two residents facilitate the discussion as they lead the critical appraisal of the articles. During the presentations, the facilitating residents encourage the discussions by opening with a 1–3-minute overview of the topic and its relevance. In-class JC session presentations followed a common format when critically appraising articles: 1) introduction, 2) discussion of validity, 3) discussion of results, and 4) discussion of applicability to practice.

The introduction

The session starts with a brief segment introducing the presenters and background of the study. In this segment, presenters review basic information about the article such as the relevance of the topic, and prior knowledge. In these few minutes, the presenters frame and introduce the topic by reviewing what is known in the literature, what is left unknown, and what the study needs to clarify. Furthermore, during the introduction, the group discusses the framework of the articles including the purpose of the study, study participants, and study design. While didactic review and large group activity are most commonly used in this segment, residents occasionally use skits, performances, or videos to engage the audience during the introduction.

The discussion of validity

In this segment, presenters facilitate a discussion on points of the study’s validity. The goal of this session is to advance residents’ understanding of the skills required to critique the quality of empirical medical research. Since most the issues related to validity are addressed online, the audience often focuses on what has not been addressed, what needs clarification, and what provoked controversy online. The results are then reviewed by examining the tables and graphs in the journal articles.

The discussion of results

The results segment is dedicated to reviewing the residents’ understanding of study results through analyzing graphs and reviewing tables. This segment requires residents to use skills in quantitative analysis, including presenting results and evaluating statistical significance, reporting clinical significance, and analyzing visual diagrams.

The discussion of applicability to practice

This segment usually brings forth the liveliest discussion during presentations. Presenters facilitate dialog on applicability of the study within the context of the residents’ own clinical site and practice. The group discussion opens to highlight how the journal articles impact professional practice. Residents use their personal knowledge of clinical patient demographics (e.g., socioeconomic status, race, gender, and age) and discuss how they would use the articles to influence professional practice. For example, two JC sessions have presented articles focusing on smoking cessation. In each of these sessions, affordability and clinical patient demographics were brought to the forefront. Residents reflected on their patients’ backgrounds and lifestyles and discussed if the study (participants, study design, and results) would be best suited for the clinic.

Consideration in implementing a flipped classroom model

Utilizing a flipped classroom approach in GME is a task in which both faculty and residents must be prepared to engage. It requires a willingness to engage in new learning techniques, as well as democratic learning foundations. These strategies may challenge traditional learning and power roles anticipated in GME. In this section, we address topics of consideration for faculty interested in implementing a flipped classroom.

Technological support

One of the critical components of a flipped classroom is the capability to blend technology with face-to-face instruction. The IU-FMR utilized a content management system called Canvas; one of the benefits of being affiliated with a larger university is access to programs and software which can facilitate incorporating in-class and online learning. However, for smaller residencies that may not have the budgets or streams of income to accommodate such programs, this can be a challenge.

Preparing residents

A flipped classroom approach may seem challenging to many residents unfamiliar with the types of participation necessary for this style of teaching to be successful. Faculty and residencies interested in the flipped classroom should work to create and reinforce a culture of participation in face-to-face instruction and in online formats. Encouraging residents to engage in material, as well as dialogue with colleagues, requires time and patience. Learning environments must be made into safe spaces where residents can trust and feel safe to participate.

Building comfort with teaching

One of the strengths of the flipped classroom is that it allows for several points of instruction for medical residents. In addition to faculty guidance, residents work to teach one another EBM concepts. The IU-MFR prepares residents by scheduling faculty and academic chief resident meetings with resident presentation groups. In these meetings, residents discuss ways to prepare for their presentations. Topics include the following: reviewing presentation objectives, creating discussion questions, and organizing presentations.

In the online forum, building comfort in helping others to learn can develop more organically and sustainably. Residents use the forum to provide feedback on ways to better engage with each other. This feedback can pertain to residents co-constructing the learning space and process for the sake of available content in an online environment. Or, it can pertain to residents co-constructing a safe space for learning and discussion.

Teaching tools/techniques

Throughout all stages of the in-class session, a variety of teaching tools and strategies are used to engage residents. Commonly used pedagogical tools such as “think-pair-shares” and heterogeneous grouping (jig-sawing) assist residents in structuring content collaboratively. These are also called cooperative learning techniques, which allow residents to share information and learn from one another in a timely manner.

Discussion

This article provided an insight into the logistics of implementing a flipped classroom instructional design. This article aims to cover a significant literature gap by detailing how these initiatives take place. Therefore, one goal of this article is to share the experience of the IU-FMR to help diminish the “mystery” behind how a residency implements a flipped classroom approach. We hope that our reflective teaching strategy will assist other residencies interested in utilizing this learning approach.

While significant learning is expected to take place in the classroom, it is well documented that learning in the classroom can often be hindered by poor attendance, poorly planned sessions, and lack of engagement.^14,15 In didactics, the flow of learning traditionally goes from faculty to residents. Using traditional lecture styles, faculty members control the pace of learning, the amount of information provided, and verbal interactions. As a result, learners are often passive, disinterested, and unengaged. With this passive engagement, learning is at best mediocre.¹⁶ On the other hand, active learning has shown to increase learner’s performance. Active learning, on the other hand, has proven to increase learner’s performance.¹⁷

Integrated technology in education has transformed learning and provides many opportunities in GME.¹⁸ Furthermore, it is well documented that collaborative learning and group work supports effective learning.¹¹ Most didactic classroom learning provides an inadequate experience by which to engage learners.⁹ A flipped classroom utilizes a learner-centered approach that fosters learners’ involvement and is regarded as having a positive impact on learning.¹⁹ When flipped classrooms were introduced, the lectures were moved outside the classroom via online forums while exercises were moved into the classroom via learning activities. Because technology supports a plethora of active learning opportunities, passive lecturing has given way to more engaging activities. Current working models of a flipped classroom engage learners in independent activities at home and in group activities in class.

Implementing a flipped classroom model to teach EBM in a residency program not only is possible but also may constitute improved learning opportunity for residents as it is founded on sound pedagogical principles. This article aimed to describe a working example. It has the limitation of not examining whether the model improves its intended outcomes. Follow-up work is needed to evaluate the effectiveness of this model on both learning and clinical practice. Evaluating resident learning and use of EBM skills can be examined using quantitative and qualitative approaches.²⁰ The quantitative approaches leverage validated tests to assess skills and knowledge and surveys to assess the use of skills in practice. Qualitative research approaches should explore how learning takes place in the online space and in class. Furthermore, it could use direct or video-recorded observations to explore how learning transfers into the bedside.

Disclosure

The authors report no conflicts of interest in this work.

References

1.		Rohwer A, Young T, van Schalkwyk S. Effective or just practical? An evaluation of an online postgraduate module on evidence-based medicine [EBM]. BMC Med Educ. 2013;13:77.
2.		Bowles P, Marenah K, Ricketts D, Rogers B. How to prepare for and present at a journal club. Br J Hosp Med. 2013;74(suppl 10):C150.
3.		Mullen SJ, Sabri K. Role of journal club in Canadian ophthalmology residency training: a national survey of program directors. Can J Ophthalmol. 2016;51(3):226–231.
4.		Honey CP, Baker JA. Exploring the impact of journal clubs: a systematic review. Nurse Educ Today. 2011;31:825–831.
5.		Arif S, Gim S, Nogid A, Shah B. Journal clubs during advanced pharmacy practice experiences to teach literature-evaluation skills. Am J Pharm Educ. 2012;76(5):88.
6.		Webber E, Saysana M, McKenna M. Journal club blog for faculty paediatricians. Med Educ. 2013;47(11):1128.
7.		Choi R, Tabby D, Rosso A, Mueller P, Schwartzman R. Current status of neurology grand rounds. Neurologist. 2010;16(5):308.
8.		Montori VM, Guyatt GH. Progress in evidence based medicine. JAMA. 2008;300(15):1814–1816.
9.		Ali KA, Saikumar H. Not so grand rounds. J Grad Med Educ. 2015;7(2):304–305.
10.		Ilic D, Nordin RB, Glasziou P, Tilson JK, Villanueva E. A randomised controlled trial of a blended learning education intervention for teaching evidence-based medicine. BMC Med Educ. 2015;15(1):39.
11.		Mazur AD, Brown B, Jacobsen M. [Learning Designs using Flipped Classroom Instruction] Conception d’apprentissage à l’aide de l’instruction en classe inversée. [Can J Learn Technol] La revue canadienne de l’apprentissage et de la technologie. 2015;41(2):1–26.
12.		Al Achkar M. Redesigning journal club in residency. Adv Med Educ Pract. 2016;7:317.
13.		Straus SE, Glasziou P, Richardson WS, Haynes RB. Evidence-Based Medicine: How to Practice and Teach. EBM. Fourth ed. Edinburgh: Churchill Livingstone; 2010.
14.		Agee N, Komenaka IK, Drachman D, Bouton ME, Caruso DM, Foster KN. The effectiveness of grand rounds lectures in a community-based teaching hospital. J Surg Educ. 2009;66(6):361–366.
15.		Morrison LJ, Portenoy R. Giving a grand rounds presentation. J Palliat Med. 2010;13(12):1477–1484.
16.		Clark KR. The effects of the flipped model of instruction on student engagement and performance in the secondary mathematics classroom. J Educ Online. 2015;12(1):91–115.
17.		Freeman S, Eddy SL, McDonough M, et al. Active learning increases student performance in science, engineering, and mathematics. Proc Natl Acad Sci U S A. 2014;111(23):8410–8415.
18.		Shaikh AT, Ferland L, Hood-Cree R, Shaffer L, McNabb SJ. Disruptive innovation can prevent the next pandemic. Front Public Health. 2015;3:215.
19.		McCallum S, Schultz J, Sellke K, Spartz J. An examination of the flipped classroom approach on college student academic involvement. Int J Teach Learn Higher Educ. 2015;27(1):42–55.
20.		Thomas RE, Kreptul D. Systematic review of evidence-based medicine tests for family physician residents. Fam Med. 2015;47(2):101–117.

Supplementary materials

Figure S1 Examples of the case studies and the discussed articles. Abbreviations: ASCVD, atherosclerotic cardiovascular disease; LDL, low-density lipoprotein; MI, myocardial infarction.

Figure S2 The validity questions for different study designs. Abbreviation: RCT, randomized controlled trial.

Figure S3 Examples of questions for the critical appraisal of the importance of the study results. Abbreviation: ROC, receiver operating characteristic.

Figure S4 Questions for applicability.

Reference

1.		Cannon CP, Blazing MA, Giugliano RP, McCagg A, White JA, Theroux P, Darius H, Lewis BS, Ophuis TO, Jukema JW, De Ferrari GM. Ezetimibe added to statin therapy after acute coronary syndromes. N Engl J Med. 2015;372(25):2387–2397.
2.		Pursnani A, Massaro JM, D’Agostino RB, O’Donnell CJ, Hoffmann U. Guideline-based statin eligibility, coronary artery calcification, and cardiovascular events. Jama. 2015;314(2):134–141.