Student Satisfaction and Self-Efficacy in a Novel Online Clinical Clerkship Curriculum Delivered During the COVID-19 Pandemic

Introduction

The COVID-19 pandemic has caused significant disruption to medical education and clinical training.^1,2 This disruption has not only affected delivery of clinical curricula but also resulted in social and economic stressors which may impede learning.^3,4 Most medical schools had already implemented some degree of online learning and assessment in basic sciences prior to COVID-19, and even more classes moved to a virtual platform during the pandemic.^2,5,6 However, there are debates regarding the effectiveness of virtual clinical education in hospital settings. Does virtual learning provide meaningful clinical experiences, meet clinical learning outcomes, and satisfy licensing and accreditation requirements? These important questions remain unanswered, including at the Faculty of Medical Sciences (FMS) at the University of the West Indies in Barbados.^7,8

The FMS offers the Bachelor of Medicine, Bachelor of Surgery (MBBS) program enrolling primarily students from Caribbean countries. Clinical clerkship rotations (CRs) generally commence in the third year of medical school and are a vital component of MBBS program. During CRs, students are integrated into clinical teams and learn by direct observation and subsequent practice of clinical skills. These experiences continue for a total of 18 months with students being exposed to pathology and microbiology, public health, psychiatry, pediatrics, obstetrics and gynecology, and medical and surgical subspecialties. CR performance is normally assessed by written, research, and practical examinations including clinical OSCE examinations. Grades contribute to overall academic GPA. Upon successful completion of all 20 CRs, students are required to sit exit examinations in the three major disciplines of clinical medicine – Medicine and Therapeutics, Obstetrics and Gynecology, and Surgery. Successful candidates are then eligible for registration for a 12-month internship in most English-speaking Caribbean countries.

The execution of CRs by the FMS-UWI Cave Hill was disrupted by the COVID-19 pandemic, which precipitated a mandatory lockdown in Barbados from late March 2020, requiring an emergency shift to online learning for the remainder of the first half of the 2020/21 academic year. Some CRs with minor patient interaction components (eg, Social and Preventive Medicine (Public Health) and Pathology and Microbiology) transitioned quickly to online delivery; others requiring greater face-to-face clinical observation or direct patient–student interaction were delayed until August 2020 and conducted in hybrid format. These changes were in keeping with the March/April 2020 recommendation of the Association of American Medical Colleges (AAMC) that “medical students not be involved in any direct patient care activities” and that “medical students’ participation in direct care of patients with or without known or suspected COVID-19 must be voluntary, not required.”⁹

The FMS followed pedagogical best practice in using web-based cyber classrooms, videotaped vignettes, audiotaped recordings, virtual patients, online chat rooms, videoconferencing, and telemedicine sessions to replace in-person clinical teaching.^8,10 However, students nonetheless missed some CRs, had limited exposure to various specialties, and missed the opportunity to benefit from networking and guidance provided by live interactions with faculty.¹¹ The extended suspension of the full face-to-face CRs during the 8 months prior to the review had critical impact, as some students failed to acquire key clinical skills.

Previous studies have reported that that COVID-19-related academic delays and CR cancellations, along with social distancing policies and physical isolation, have contributed to a number of psychological consequences, including emotional distress, fear and panic, and formal mental health diagnoses among medical students.^3,10,12 Therefore, the purpose of this study was to assess student satisfaction and self-efficacy in novel online clinical clerkship curriculum in the Faculty of Medical Sciences, UWI, Cave Hill Campus, Barbados, delivered during the COVID 19 pandemic, as well as to identify areas for improvement in future clerkship iterations.

Materials and Methods

A cross-sectional design was employed. In January 2021, an online questionnaire was emailed to all students (n = 131) registered in years 4 and 5 of the MBBS program through campus information technology services. Study data were collected and managed using REDCap v10.3.4.

Student Satisfaction Measure

Participants rated their satisfaction with online learning using a Likert-type student satisfaction tool developed for the current study.¹³ The participants were divided into a high satisfaction group (HSG) and low satisfaction group (LSG) based on their response to the final item tapping overall satisfaction. The mean for each of the remaining items was compared by HSG and LSG and by gender using independent sample t-tests and 95% confidence intervals (CI). Additionally, students were asked to rate their enjoyment of online learning using the question, “How much did you enjoy online learning classes during the pandemic”. Responses ranged from 1 (extremely unenjoyable) to 5 (extremely enjoyable). Students who answered 4 or 5 on the scale were considered to have enjoyed online learning classes.

The Modified Online Learning Self-Efficacy Scale

We used a modified version of the Online Learning Self-Efficacy Scale (OLSES) developed by Zimmerman and Kulikowich.¹⁴ The scale contains 6-point Likert-type items; the original form consists of three factors and 22 items. For our setting, 17 OLSES items were relevant for measuring the three factors and, therefore, included in our questionnaire. For each item, we calculated the mean score and 95% confidence intervals. For the HSG and LSG groups, we calculated mean self-efficacy scores and used t-tests to determine if there was a statistically significant difference in mean scores between groups. Given the modifications made for this population, we calculated the internal consistency for subscales of the OLSES using Cronbach’s alpha. Differences in mean self-efficacy score by age (continuous variable), gender (male/female), and perceived effectiveness were explored using linear regression analyses. Unadjusted associations were explored for each variable; the association between mean self-efficacy score (outcome) and perception of online effectiveness was quantified with adjustment for age and gender.

Online versus Face-to-Face Learning

We compared online and face-to-face learning using a measure developed by Bączek et al.¹⁵ We calculated means and standard deviation for age, proportions for nationality, and the following outcomes: (1) perceived effectiveness of face-to-face and online learning and (2) satisfaction with the online learning experience. Since these were measured using Likert scales, we utilized medians and non-parametric tests to examine associations between these outcomes and age, gender, and nationality.¹⁶ We used the Wilcoxon rank test to compare perceived effectiveness of face-to-face vs online learning. Ordinal multivariable logistic regression analyses were used to explore associations between satisfaction with online learning and perceived effectiveness after adjustment for age (continuous variable) and gender (male/female). All analyses were conducted using Stata statistical software version 16.

Ethical Approval

The study protocol was approved by The University of the West Indies, Barbados Ministry of Health Research Ethics Committee/Institutional Review Board (IRB) (IRB No. 201103-B). Each participant gave consent, and the study complied with the Declaration of Helsinki.

Results

Eighty-eight students (response rate = 67%) provided responses to the questionnaire. Participants ranged in age from 20 to 42 years, and both males and females had a mean age of 23 years. Most respondents were from Barbados (45.4%) or Trinidad (50%) (Table 1).

Table 1 Description of Participants

Student acceptance of online learning was generally low among both males (21.4%) and females (36.5%). Fewer than half of students (46%) believed online learning effectively increased their knowledge compared to face-to-face learning (56%) (Table 1). The largest perceived difference in effectiveness of online and face-to-face learning was in the area of clinical skills: 18% vs 89% (p < 0.0001). The difference in perceived effectiveness for learning social competencies in online (27%) vs face-to-face (67%) teaching was also statistically significant (p < 0.001).

Scores on the student satisfaction measure (Table 2) indicated that students were most satisfied with the provisions made for group interactions in the clerkships and, more specifically, with encouragement received to participate in clerkship sessions. This subscale demonstrated the strongest internal consistency (α = 0.82). The lowest scoring item was “My interest and motivation in learning have increased as a consequence of this online clerkship placement” (mean <2). Further, overall satisfaction with the clerkship was relatively low (2.48 95% CI (2.30, 2.66)).

Table 2 Mean Scores by Satisfaction Group and Gender for the Student Satisfaction Scale

We analyzed the satisfaction scores by high/low satisfaction groups and gender. The high-satisfaction group scored higher on all scale items except for “I have found online clerkship teaching intellectually challenging and stimulating.” The differences in mean satisfaction levels between high- and low-satisfaction groups were statistically significant for thirteen of sixteen items (p < 0.05). There were no significant differences in overall satisfaction by gender (Table 2). While none of the items on the scale were statistically significantly different by gender, there was a trend of females being more satisfied with “online clerkship teaching” and males more satisfied with (1) “enthusiasm of lecturers”, (2) “group interactions in online clerkship sessions”, and (3) “individual rapport in online clerkship sessions during COVID-19”. The 17-item scale demonstrated strong internal consistency (α = 0.87).

The high-satisfaction group had higher mean self-efficacy scores for sixteen of seventeen items on the OLSES. For eight items, the difference was statistically significant (Tables 3 and 4 shows means of OLSES subscales). Scores were highest for the learning subscale and lowest for time management, suggesting that time management was the most challenging aspect of self-efficacy for students.

Table 3 Online Learning Self-Efficacy Scale (OLSES) Scores by Satisfaction Group

Table 4 Mean Scores and Internal Consistency for OLSES Subscales

Table 5 shows that students satisfied with online learning were more likely to be female (OR = 2.6), and with each year increase in age, respondents were more satisfied with online learning (statistically significant after adjustment). Students satisfied with online learning were more likely to perceive the modality as effective for increasing knowledge, improving clinical skills, and facilitating social competencies. Mean self-efficacy scores were higher for students who perceived online teaching to be effective at increasing knowledge (8.98 points higher after adjustment), improving clinical skills (6.14 points higher after adjustment), and developing social competencies (7.81 points higher after adjustment). Students’ perceptions of online learning were strongly associated with online self-efficacy.

Table 5 Satisfaction with Online Learning

Discussion

This study aimed to examine medical students’ satisfaction, acceptance levels, and perceived online learning self-efficacy for a novel online clinical curriculum during the COVID19 pandemic. The study findings indicate that, although the students’ general acceptance of the online curriculum was low, they reported relatively higher levels of satisfaction and self-efficacy with online learning. Acceptance of online learning was generally low (33.4%), and they felt that face-to-face teaching increased their knowledge to a greater degree than online learning (56% vs 46%). The difference in perceived effectiveness of online learning compared to face-to-face teaching was most significant for both clinical skills (18% vs 89%) and social competencies (27% vs 67%). We also found a relatively low mean score (2.48 out of 6) in overall student satisfaction with the clerkship. Our findings are consistent with those of Bączek et al,¹⁵ who found that online learning was less effective than face-to-face learning for developing clinical and social skills among Polish medical students. If online clinical learning is necessary, web-based video teaching and use of remote standardized patients may be useful for teaching clinical and social skills.^13,15

Although rapid implementation of online learning was a necessity during the COVID-19 pandemic, further research is needed to understand and enhance effectiveness of online pedagogical methods.¹⁷ Studies have identified key barriers to online learning, including lack of eye contact, delayed feedback to students, absence of bedside teaching, technical issues, and lack of previous online learning experience.^18,19 In previous studies, medical students in clinical years have indicated that “some educational opportunities were lacking with the remote learning platform” and that “the COVID-19 pandemic had a negative impact on their medical education.”¹⁹

Satisfaction with and acceptance of online learning are key to successful virtual clerkships. Respondents who were satisfied with online learning were far more likely to perceive the modality as effective for increasing knowledge, improving clinical skills, and developing social competencies. Further, perceived effectiveness was correlated with satisfaction with online learning. Importantly, recent studies of medical students’ satisfaction with online learning have reported variable findings. A study in Jordan also reported that only 26.77% of clinical students were satisfied with online learning.²⁰ In another study, medical students were generally satisfied (52.3% satisfied and 20.7% very satisfied), but 83% experienced difficulties with online learning.²¹ Student satisfaction is an important metric to evaluate online clinical teaching adopted during the pandemic.²² Previous studies found that student satisfaction with current online education is positively associated with prior experience with online learning.²³ Foo et al²⁴ reported significantly lower satisfaction scores for online learning than for face-to-face teaching of problem-based learning tutorial during the COVID-19 pandemic. The authors also noticed significantly lower scores in the online learning group for all five areas of proficiency: participation, communication, preparation, critical thinking, and group skills.²⁴ Pedagogical and technical challenges related to online learning may contribute to poor student performance.^24,25

Learning self-efficacy, defined as learners’ confidence in their capability to learn specific subjects, is crucial for the enhancement of academic progress, because it is positively correlated with academic achievements and effective use of learning strategies.¹⁴ In the current study, total online learning self-efficacy scores ranged from 47 to 94 with a mean score of 73.6. Online self-efficacy was strongly associated with the students’ perception of online learning. Studies have shown that perceived self-efficacy and clinical performance are closely related.^26,27 Findings regarding gender differences in perceived self-efficacy have been inconsistent in previous studies.^28–31 Some researchers^28,29 reported higher self-efficacy scores among male students. However, our findings that self-efficacy scores were unaffected by gender are consistent with other reports.^30,31 Venkatesh et al³² reported significantly higher computer self-efficacy and overall learner satisfaction ratings for males in a blended e-learning program.³²

Limitations of the Study

This was a single-site study that relied on students’ self-assessment. Self-report may be influenced by cultural factors and unique characteristics of the UWI learning environment. The study was further limited by small sample size. For these reasons, results may not be generalizable to other settings.

Conclusion

Our institution adopted online learning by necessity for clinical teaching during the pandemic, as did many medical schools. Student satisfaction with online learning and perceived self-efficacy in online learning were higher than acceptance of the online clerkship curriculum delivered. Online learning may be needed to teach knowledge as well as develop clinical and social skills in both the short and longer terms. Understanding student experiences and unmet needs will help faculty and policymakers design better clerkship curricula and provide support for students during the COVID-19 pandemic and beyond.