Use of the Community of Inquiry Framework to Measure Student and Facilitator Perceptions of Online Flipped Classroom Compared with Online Lecture Learning in Undergraduate Medical Education

Introduction

Active learning modalities, such as problem-based learning or team-based learning, usually involve high levels of student engagement, reflection, and discussion. In these modalities, learning is designed to be student-led and teacher-facilitated.¹ In contrast, traditional lecture-only formats lead to more passive student learning even when students are encouraged to participate and ask questions.^1,2 Over the last decade, undergraduate medical education has increasingly adopted the flipped classroom (FC) model to teach both pre-clinical and clinical curricula.^2–6 In the FC model, students complete online videos or reading material prior to class and engage in collaborative, active learning activities with other students during in-class time.^4,7–9 The role of the teacher during FC becomes to organize and guide students through activities, provide expert insight and feedback during discussion, and keep students on track.^4,7,8 While FC interventions in health professions education have shown increased student engagement, greater satisfaction in class, and improved exam scores, FC activities prior to the COVID-19 pandemic were conducted mostly face-to-face.^3,5,8 In 2020 with the rise of COVID-19 transmission, nearly all pre-clinical medical education curricula, including FC and other active learning sessions, shifted to the online setting out of public health necessity.

Since the pandemic, some studies have also shown online FC to be a feasible learning model with similar benefits of higher student engagement and test performance compared with traditional online learning formats.^10–13 Still, lecture-based learning remains a common instructional method in undergraduate medical education. It requires fewer resources and less organization and is easier to implement online.^14,15 In addition, not all students prefer active learning modalities, and some students find them less efficient.^16,17 Social interaction may also feel less natural in online class settings, even with active or FC learning models.^11,18

Though most medical schools did not exclusively use online instruction prior to the COVID-19 pandemic, online learning formats have been utilized and studied since the 1990s as the Internet became more widely available.¹⁹ In 2000, Garrison and colleagues established the Community of Inquiry (CoI) model to inform the development of online and blended learning environments.²⁰ CoI describes the educational process as a collaboration of individuals engaging in purposeful discourse to achieve meaning and understanding.²¹ The CoI conceptual framework focuses on three interdependent elements that are critical in developing a meaningful online learning experience: cognitive, social, and teaching presence. Cognitive presence is defined as the extent to which learners construct meaning through sustained communication. Social presence is how learners may be able to show their “true selves” and develop trust with others in the community. Teaching presence describes the teacher’s direction and facilitation of the course to achieve learning objectives.^20,21 Validated surveys have been developed to evaluate the effectiveness of online learning based on these three elements.^21–23

Only a few studies have used the CoI framework in medical education with none specifically evaluating online undergraduate medical school curricula.^24,25 Understanding which elements work well and which do not may help medical schools incorporate and modify online learning modalities more effectively. While social interactions between students and facilitators may have some challenges in the virtual environment, we wanted to measure if the online FC model would still foster more collaborative, sustained discourse and improve engagement and interaction within the learning community as compared with online lectures, the predominant form of instruction at our medical school during the early part of the COVID-19 pandemic. The CoI framework of cognitive, social, and teaching presence serves as a valuable approach to evaluate these two methods of online learning in greater detail.

Using a validated survey instrument based on the CoI framework, we therefore sought to compare medical student perceptions of an online FC tutorial with their experiences of online lecture at that time. The topic of the FC tutorial was a case study of endocrine physiology. We also sought faculty perceptions of the online FC format in comparison with online lectures and in-person formats.

Methods

Study Design and Population

The study was an observational survey design measuring medical student and facilitator perceptions of an online FC endocrinology tutorial in comparison with their own previous online lecture experiences. The study was carried out at Stanford School of Medicine, the medical school of a large US (United States) academic medical research center in September 2020. As the tutorial was a required part of the second-year student curriculum, all second-year medical students were expected to attend the session. The study population was second-year medical students and faculty facilitators who participated in the tutorial and completed the post-tutorial surveys. Facilitators were endocrinology physicians affiliated with the medical school.

Synchronous Online Flipped Classroom, Case-Based Tutorial

The second-year medical student thyroid pathophysiology class session was redesigned from a traditional two-hour, large-group lecture to a two-hour, small-group FC tutorial as part of a shift toward an active learning-based curriculum at our medical school. Other modules in the endocrine block had already been converted to FC tutorials, and feedback from students and faculty also favored changing to this teaching modality for the thyroid module. This redesign from lecture to FC was planned prior to the COVID-19 pandemic and was originally intended to be held in-person. However, due to the public health emergency, the FC tutorial was held entirely online in September 2020. It was the first time that Stanford had sponsored an FC tutorial fully online.

A schematic of the online FC tutorial is shown in Figure 1. We conducted the online FC tutorial via eight small group Zoom sessions occurring simultaneously. Each small group consisted of 10–12 second-year medical students and one faculty facilitator. Students were expected to watch five short, pre-recorded lecture videos of 10–15 minutes each (approximately 60 minutes total) on thyroid pathophysiology prior to the tutorial. Four clinical cases of thyroid disease were discussed during the Zoom session, with each case discussion expected to take 25 minutes. Smaller breakout Zoom rooms of 5–6 students were used to spur further discussion during each case before rejoining at the end of the case. Facilitators moved between breakout rooms to observe and guide discussion if needed. Cases were not made available to students prior to the tutorial. Students were asked but not required to have their cameras turned on during the tutorial. Students were encouraged to participate by audio if cameras were off.

Figure 1 Online Flipped Classroom Model. In September 2020, second-year medical students (N = 92) at Stanford School of Medicine prepared for an online-only, flipped classroom thyroid tutorial by watching short didactic videos prior to class. Links to cases and questions were sent out to the students in Zoom chat at the beginning of the tutorial. Four clinical cases were discussed in depth during the tutorial.

Student and Facilitator Evaluation Surveys

Students and facilitators received surveys via email immediately following the end of the tutorial. For students, we used a validated survey from Garrison et al (2004) originally designed to ask learners to compare their online learning experience to prior face-to-face learning experiences.²² The survey queried 28 areas of cognitive, social, and teaching presence based on the CoI framework. For our study, we adapted the survey to ask students to compare the current online FC tutorial experience to recent online lecture experiences from other courses (Supplementary File 1, copy of student survey). At that time, all required courses had moved to online modalities, predominately lecture-based. We felt confident that participants would compare this online FC experience with traditional online lectures in the MD program.

A separate survey was developed by the study authors and disseminated to facilitators (Supplementary File 2, copy of facilitator survey). Facilitators were asked to agree or disagree with a series of statements regarding the general effectiveness of the online FC tutorial, its effectiveness relative to an online lecture and in-person FC tutorial, teaching satisfaction, and perceived student engagement and learning. Though the facilitator survey was not validated prior to dissemination, the survey questions were direct and worded clearly, rendering them simple to understand and answer.

The study was reviewed by the Stanford University Institutional Review Board prior to the tutorial and was deemed exempt from further review. The purpose of the study with an explanation of benefits and risks was provided to study participants (Supplementary File 3, IRB Research Information Sheet) prior to survey initiation. Consent was obtained from each survey participant. All surveys were anonymous with no identifying data to link back to the individual. Demographic information of participants was not collected. Participation in the survey was completely voluntary. No incentives were offered to students or facilitators to complete the survey. Surveys were designed and distributed via Qualtrics (Provo, UT).

Survey Ratings and Data Analysis

Students rated survey items along a 5-point Likert scale of much worse (1), worse (2), same (3), better (4), much better (5), comparing the FC tutorial to online lecture experiences. Facilitators rated survey items along a 5-point Likert scale of strongly disagree (1), somewhat disagree (2), neither agree nor disagree (3), somewhat agree (4), strongly agree (5). Favorable ratings were defined as a mean rating of 3.5 or higher out of 5.0 as the halfway point between the neutral category (3) and the first positive category (4) with more than 50% of respondents rating the item a 4 or 5. Descriptive statistical analysis was used for survey responses, and data are presented as mean scores with standard deviation (SD) or measure of frequency of response scale ratings to the survey question item.

Results

Ninety-two second-year medical students and eight facilitators participated in the online FC tutorial on thyroid pathophysiology. Fifty-three students (58%) and seven facilitators (88%) completed surveys following the tutorial.

Student Evaluation Survey

In comparison with online lectures, students rated the online FC tutorial favorably (defined as a mean rating of ≥3.5 out of 5 on the Likert scale and >50% of respondents rating an item a 4 or 5) in 24 out of 28 areas (Table 1). The mean (± SD) rating was 3.9 (± 1.0) for questions related to cognitive presence, 3.6 (± 0.9) for questions related to social presence, and 3.7 (± 0.9) for questions related to teaching presence.

Table 1 Student Survey Responses on Effectiveness of Online Flipped Classroom Tutorial Based on CoI Framework

The online FC tutorial received the highest scores for “engagement in exchange of ideas”, “stimulation of curiosity”, “application of ideas or concepts”, “satisfaction with teacher interaction”, and “identification of relevant new information”. Mean scores for these survey items were at least 3.9, and more than 70% of students felt that the online FC tutorial was superior (score rating of either 4 or 5) to an online lecture in these fields. These highly rated items were associated with cognitive presence, except for satisfaction with teacher interaction, which was associated with teaching presence.

In general, most items within social and teaching presence were rated favorably, but the lowest-rated items with a mean rating of <3.5 were also within these domains. The online FC tutorial achieved relatively low scores regarding students’ “ability to be open” (ie, disclosing personality), “expression of emotions”, and “clear communication of expectations”. All three items had a mean score of 3.4 (± 0.9). The majority of students considered the online FC tutorial and an online lecture format to be either similar or the tutorial to be less effective in these three areas. In addition, “summarization of discussion” had a mean rating of 3.5, but less than 50% of the student survey participants rated it a score of 4 or 5.

None of the surveyed questions had a mean rating of less than 3.0 to indicate an unfavorable comparison of online FC tutorial versus online lecture. No survey item had more than seven students (13%) rate the online FC tutorial as worse or much worse (score rating of 2 or 1) than an online lecture.

Facilitator Evaluation Survey

Facilitator survey responses are displayed in Table 2. All facilitator survey respondents either agreed or strongly agreed that the FC tutorial was effective in an online format and was also more effective and enjoyable to teach than an online lecture format. Facilitators also felt that students were engaged and learned the basics of thyroid clinical principles by the end of the tutorial. None of the facilitators felt that the FC tutorial was more effective online than in-person. Five of seven facilitators (71%) also strongly or somewhat disagreed that FC tutorials should be delivered in an online setting more frequently.

Table 2 Facilitator Survey Responses on Effectiveness of Online Flipped Classroom Tutorial

Discussion

During the period of this study (September 2020), in-person instruction was not permitted at our medical school. Instruction was exclusively online due to the risk of disease transmission during the early stages of the COVID-19 pandemic. Therefore, our study was undertaken to compare the effectiveness of an online, small-group FC tutorial with an online lecture experience, which remained a predominant method of teaching at our institution. Students were asked a series of questions that explored elements of cognitive presence, social presence, and teaching presence as described in the CoI framework for online learning environments. When compared with a traditional online lecture model, the online FC tutorial was particularly impactful in areas of cognitive presence and less so in areas of social and teaching presence, based on student perceptions.

The favorable scores within the cognitive presence domain for the online FC tutorial are consistent with findings from other FC-oriented studies, both for in-person and online FC compared to traditional learning formats.^3,6,9–11 Students reported feeling more engaged in discussing ideas, more able to identify relevant points from the thyroid case vignettes, and more ready to apply learning concepts than after online lectures. This is perhaps not too surprising given the inherent nature of the FC model in which students have already reviewed resources and video recordings before class and are primed to dive into extended discussion on class arrival. Likewise, our thyroid cases were specifically written and designed to stimulate critical thinking and provide examples of pathophysiology in a “real world”, patient-care narrative. The format and purpose of FC model is not changed, whether online or in-person. Cognitive presence elements also scored highly in other studies examining online health professions’ education courses using the CoI framework.^24,26 Thus, in terms of factors related to cognitive presence, the online FC remains a more favorable instructional method over the online lecture.

Based on the more intimate nature of small group interactions, we were surprised to see several elements of social presence rated less highly in the online FC tutorial compared with online lecture experiences. Fewer students in our study felt that they could be more “open” or express emotion through the online small-group sessions. One potential explanation may be that students had the sense of being monitored more closely by faculty in a smaller setting, thus needing to be on their best behavior. Whether these elements of social presence would have been improved during in-person FC tutorials is unclear, as facilitators are also present during in-person tutorials.

A small minority of students also kept their cameras off, often due to technical difficulties. Having the camera off may also be due to a sense of discomfort or invasion of privacy in one’s personal (eg, home) space. However, given that the majority of students tend to turn off their camera during online lectures, it seems less likely that cameras being off for a few students in our online FC had a significant impact on overall social presence. With encouragement, most students had their camera turned on during our tutorial.

The Zoom breakout rooms were used to try to maximize discussion and interaction among students, though faculty still entered these breakout rooms for short periods of time to observe and support discussion. For future tutorials, faculty may be advised to stay in the main Zoom room during the smaller breakout sessions to allow students to feel “open” and show their “true selves”. One concern is that if facilitators remain in the main room, they may not be able to assist when students get stuck or correct misunderstandings in the breakout group. Breakout rooms also have other challenges common to traditional small groups, such as when one student dominates the conversation or when some students refuse or are hesitant to contribute. Rotating the students into different small groups may help mitigate differences in student participation, but it may also decrease sense of community and trust if students are unfamiliar with each other.¹⁸

Perhaps more importantly, simultaneous or private conversations between different groups of individuals are not possible on Zoom as they are in-person. This limits the number of social interactions in the online setting, regardless of learning format. Thus, online FC likely feels more similar to an online lecture for these elements of social presence compared to the difference felt between in-person FC and lecture learning.

Some areas of teaching presence, such as understanding the organization of the format and expectations of the class and summarization of discussion, were also rated lower or equivalently in the online FC tutorial compared with online lectures by students. As this was our first time teaching the thyroid pathophysiology curriculum through an FC model, our facilitators may not have communicated the structure and learning objectives of the session as effectively as with other classroom sessions. We attempted to overcome this potential shortcoming by holding a facilitator training session before the tutorial, which was well attended by facilitators, but recognize this is an area of needed improvement as we continue to refine online learning activities in subsequent sessions. Dedicating a few minutes at the beginning of the online FC to orient students and set expectations as well as a few minutes at the end of each case to summarize the discussion and solicit questions may also enhance teaching presence in future years. Lastly, time management through the four different cases in our tutorial may have been an issue, and this may be improved by editing and shortening the length of the case studies.

Many elements of social presence were still rated positively in our tutorial, particularly the feeling of being a part of the class community. This was encouraging, as this area was highly likely to be adversely impacted by the physical distancing restrictions during the COVID-19 pandemic. Online synchronous learning allows students to still meet and interact regularly, often with the same group of students, as was the case in our online FC and other FC modules within the endocrine block. Questions regarding actual teacher interaction and teacher feedback were also rated highly overall. Other studies that have evaluated in-person FC or online active learning modalities with the CoI framework have shown high ratings within social and teaching presence,^25,27 so our mixed ratings in the social and teaching domains may be more due to differences in faculty involvement and familiarity with the tutorial structure.

Facilitators were in striking agreement in their evaluation of the FC tutorial. All agreed that an online FC tutorial was a successful teaching method and more effective and enjoyable to teach than an online lecture format. Facilitators felt students to be engaged and to understand course material (cognitive presence). Overall, this is a valuable finding as FC formats require more planning and time for course facilitators and is often cited as a barrier to changing to the FC model.^14,15 The overall positive response toward the FC format from facilitators thus signifies that shifting toward active learning modalities is likely worthwhile if facilitators feel supported and resources are available.

By contrast, all facilitators felt tutorials should not be held online more often or held online instead of by in-person instruction. Strong faculty preference for in-person instruction was also seen in a study by Joji et al, which evaluated student and faculty perceptions of an online microbiology laboratory course.²⁸ Our studies differ in that in-person demonstration is usually necessary for wet lab microbiology sessions whereas this is not required for a thyroid endocrinology tutorial. Still, regardless of course subject, in-person interaction may lead to more natural, spontaneous discussion, and conversations can continue uninterrupted before and after set class times. This freedom outside of the online class structure may further strengthen a sense of community among students and educators (social presence). Another advantage of the in-person FC model is that teachers can spend time with struggling students while letting top students move through the activity at their own pace.⁷ As mentioned earlier, this advantage fades in online FC as all questions and conversations are overheard by the whole group, similar to the online lecture format. Facilitators may find it more difficult to identify struggling students, especially if students feel too self-conscious to speak up during the online FC (teaching presence). Therefore, certain aspects related to social and teaching presence of a learning community may be more difficult to achieve with FC in online settings from the facilitator perspective.

Strengths

A particular strength of this study is the novel use of the validated CoI survey instrument to evaluate the effectiveness of this online endocrinology FC tutorial compared with other online learning formats. The survey comprehensively measured several aspects of online learning within the core cognitive, social, and teaching presence domains. Higher and lower ratings are grouped together within a certain presence, highlighting strengths and the areas needed for future improvement. Another strength is the measurement of facilitator perceptions of the online FC tutorial as most FC studies have focused on student perceptions.

Limitations

There are several limitations to our study. Our study was observational, and students were asked to compare their online FC experience with their own recent online lecture experiences. Ethical considerations around implementing an untested curriculum to half of the students precluded conducting an experimental or randomized control study design (ie, students split between a traditional pathophysiology online lecture and an online FC tutorial) at our institution. It is also unknown exactly how many students watched the pre-tutorial videos; only four of seven facilitators felt that students consistently watched the videos. Other limitations were the suboptimal student response rate to the survey (only 58% of the class) and the relatively small number of students and facilitators. With solicitation of student feedback already occurring at several points during the year, survey fatigue may have led to the lower survey response rate. Use of a validated facilitator survey would have also strengthened the study’s rigor and the reliability of the facilitator results. Lastly, our observations may be limited to the topic area of instruction of endocrine physiology and may not directly apply to the instruction of other disciplines.

Future Directions

Direct comparison of learning outcomes using the FC tutorial model with online lecture or in-person learning modalities was not conducted in our study but may be a focus in subsequent studies. Though we observed that overall exam scores in our endocrine block in 2020 were similar to previous years, specific performance on thyroid-based exam questions was not examined. In general, exam performance has been observed to be stronger after online FC modalities compared with online lectures.^10,11,13 On the other hand, a recent study examining online FC versus in-person FC formats in a physical therapy course demonstrated that exam performance and total course grade were similar between the two formats.²⁹ Thus, in-person and online FC formats for our thyroid pathophysiology tutorial might have resulted in similar test performance as well. Whether it is worthwhile to continue online small group or FC learning beyond the COVID-19 pandemic remains a largely unanswered question in medical education, and many schools are returning to mostly in-person instruction. Future studies using the CoI framework may involve comparison of student and facilitator perceptions of in-person versus online FC tutorials.

Conclusions

In our study, an online, small-group FC tutorial on thyroid pathophysiology was well received and viewed favorably compared with an online lecture by both students and facilitators. The CoI framework highlighted online FC to be particularly impactful in areas of cognitive presence, such as application and integration of knowledge. Online FC tutorials may continue to be an attractive teaching option within undergraduate medical education and other health educational settings beyond the COVID-19 pandemic, such as for remote and distance learning. However, there are certain valued aspects of in-person instruction that are not replicated well by online models. Some elements of social and teaching presence were similar in both online formats, suggesting that potential advantages of small group and FC learning were mitigated when moved online. In-person FC instruction or other interactive online learning formats may remain more advantageous for inter-student and facilitator interaction. Overall, the CoI conceptual framework is a useful tool for educators to examine which elements of the learning experience are enhanced by certain online learning formats and to identify areas of relative weakness that need improvement.