Assessment of Factors That Students Perceive to Affect Their Virtual Learning of Clinical Skills for OSCE

Plain Language Summary

One of the main examinations that students must pass in medical school is known as the OSCE. The OSCE is a face-to-face practical assessment of all the skills a student acquires during medical school. It resembles a mock clinic, where the student must investigate, diagnose, or manage a series of actors, real patients, or clinical tasks, under supervision, to pass. Before the COVID-19 pandemic, due to the practical nature of this examination, students mostly learned such clinical skills in face-to-face sessions. However, the pandemic-induced lockdown forced many institutions and student organisations to shift to virtual OSCE teaching.

The study evaluated factors that students perceive to affect their virtual learning of clinical skills for OSCE. It was found that the use of online technology offers important advantages to medical students such as flexibility and convenience. Therefore, our study highlights the advantages of online teaching and its potential to be used in combination with face-to-face teaching to deliver medical education in the post-pandemic era.

Introduction

The Objective Structured Clinical Examination (OSCE) is an examination that is a widely accepted evidence-based model of objectively and reliably evaluating a medical student’s competence in performing a range of clinical skills.¹ The OSCE comprises various clinical ‘stations’ which seek to test a medical student’s competence in skills required to be a proficient medical doctor (eg history taking, data interpretation and clinical examination). In the traditional medical curriculum before the COVID-19 pandemic, teaching in preparation for OSCE such as clinical skills teaching and simulation-based teaching was predominantly delivered face-to-face.² This also includes bedside teaching, which was conducted in clinical environments to use real patient cases to facilitate clinical learning.³

The circumstances of the COVID-19 pandemic have depleted bedside learning opportunities because of restrictions for reducing COVID-19 spread, prompting the shift of education to a virtual format.^4,5 In order to reduce the impact of the lack of learning opportunities helping students prepare for their OSCEs, universities have used virtual delivery of OSCE-specific knowledge as an alternative to traditional face-to-face teaching.⁶ This may even be in the form of clinical videos used for clinical skills training.⁷ Even though the disruption to face-to-face teaching brought upon by the COVID-19 pandemic has begun to dissipate and the periods of lockdown are not very recent, virtual teaching methods have remained in use.

The study was conducted in the academic year 2020–2021 by the founders of OSCEazy, a medical student organization dedicated to providing free online medical education to students across the world. They conduct weekly peer-led teaching sessions in the form of lectures on various domains of medical education, and one of note is OSCE teaching. The OSCE teaching sessions comprise interactive ‘spot diagnoses’, clinical cases consisting of history taking, data interpretation and clinical examination, and single best answer (SBA) questions. They are conducted to aid students in preparation for their OSCE.

Several studies have made use of student-led online OSCE teaching sessions/workshops to assess participants’ confidence across various domains of an OSCE (eg history-taking, communication, and data interpretation).^6,8,9 One such study reported a significant improvement in the participants’ confidence across the domains assessed and most participants believed virtual OSCE teaching would be useful preparation for an in-person OSCE.⁶ This highlights the usefulness of virtual OSCE teaching and how it may aid in providing students with learning opportunities that allow for adequate preparation for an OSCE. While studies have previously evaluated factors that may impact online learning in various domains of medicine such as anatomy,¹⁰ little is known about factors that may affect virtual student learning for an OSCE. Therefore, this study evaluates factors that students perceive to affect their virtual learning of clinical skills for an OSCE.

Materials and Methods

Teaching Session Delivered

“The Respiratory Station” was an OSCE revision session that the medical student education organisation, OSCEazy, designed and delivered online via Zoom^TM. The broad learning outcomes of the session included the basics of respiratory history-taking and examination, relevant data interpretation (eg chest X-rays), and an overview of clinical skills and investigations relating to respiratory medicine. OSCEazy social media platforms publicised the session and over 500 students from around the world attended it at the same time. The live session lasted for approximately 1.5 hours, was not repeated, and was conducted in March 2021 during the maximum lockdown period of the COVID-19 pandemic. The session content was peer-reviewed by three founders of OSCEazy and was delivered by two other founders.

Questionnaire Design and Dissemination

A questionnaire was designed on Google Forms^TM (see Supplementary Material) mixing 5-point Likert Scales (1 = strongly disagree, 5 = strongly agree) and free-text boxes. An itemised list of perceived barriers to virtual OSCE teaching was generated by the authors of the study (see Supplementary Material). Students who felt that their concerns were not included in the itemised list were allowed a free-text response. An additional comments section at the end sought further opinions. Students were informed on the questionnaire that, through completion, they were consenting to the use of their anonymised data in future publications (see Supplementary Material). All data were securely held in a password-protected drive and could only be accessed by the founders of OSCEazy. A cross-sectional approach was taken to limit duplicate responses and maximise the sample size.

Ethical Considerations

This study was an evaluation of student perceptions of factors impacting the delivery of virtual OSCE teaching. According to advice obtained from the NHS Health Research Authority’s online decision tool, the study did not require formal ethics committee approval.¹¹ The guidelines outlined in the declaration of Helsinki do not apply to our study as it is a service evaluation of a single event teaching session, and it does not involve biomedical research. Students were made aware that taking part in the survey was voluntary. All data were anonymised according to General Data Protection Regulation (GDPR) guidelines. The service evaluation questionnaire and presenter feedback form were paired and distributed to all the attendees, and the slides of the teaching session were sent out to those that filled in the presenter feedback form. Provision was made to ensure that those who did not want to complete the service evaluation questionnaire were provided with the presenter feedback form.

Statistical Analysis

All data were analysed using Microsoft (MS) Excel (Mac, Version 16.47.1, Washington, United States) and Statistical Package for the Social Sciences (SPSS) (Mac, Version 26, Illinois, United States). GraphPad Prism (Mac, Version 9, California, United States) was used to generate Figure 1. The wordcloud (Figure 2) was generated using an online word cloud generator tool. A related-samples Wilcoxon signed rank test was used to determine differences in proportions between groups. A p-value threshold of <0.05 was considered significant.

Figure 1 Percentage of participant responses to the different challenges of virtual OSCE teaching.

Figure 2 Wordcloud comprising most common comments made by the participants.

Results

There were a total of 556 responses to this survey and most respondents were female (76.6%) with the mean age of the participants being 24 years. Most students preferred face-to-face OSCE teaching (67%). However, most students agreed that online teaching offers more flexibility and convenience (median: 5, IQR: 4–5), and their likeliness to ask questions was similar in both formats (median: 4, IQR: 3–5 vs median: 4, IQR: 3–4, p value: 0.94).

Students also felt that the efficacy of virtual OSCE teaching was enhanced by the use of visual aids (median: 5, IQR: 4–5) and breakout rooms (median: 3, IQR: 2–4), a function on Zoom that enables small groups of attendees to interact. The major concern regarding online OSCE teaching was found to be having access to a stable internet connection (69.1%), followed by body language (35.8%), clarity of speech (27.5%), factors in the students’ immediate environment such as background noise (27.5%), tutor’s confidence in using online technology (18.9%) and use of online applications such as Zoom and MS Teams (16%) (Figure 1).

Some positive comments mentioned about the session delivered in the additional comments section of the questionnaire include “very good lecture”, “excellent teaching”, “great slides that are easy to follow” and “very helpful and informative”. However, some comments made on the improvement of the delivery of the virtual teaching included the use of polling instead of chat function for interactive spot diagnosis, conducting a shorter 1-hour session and change of colour scheme to accommodate for visual learners. The wordcloud (Figure 2) highlights the most common comments made by the participants.

Discussion

The results of the study have highlighted that the participants believe that virtual OSCE teaching offers more flexibility and convenience in comparison with traditional face-to-face OSCE teaching. Virtual teaching offers students the opportunity to access content from any location across the world, with larger capacity compared to in-person sessions, considering restrictions on gathering sizes.¹² This had become even more apparent during the COVID-19 pandemic considering restrictions on gathering sizes brought upon by social distancing guidelines. Furthermore, it enables medical students to fill gaps in their knowledge resulting from the paucity of learning opportunities available in clinical areas due to the COVID-19 pandemic restrictions.

The decline in bedside OSCE teaching is apparent and as a result it might particularly impact the ability of medical students to recognise key clinical manifestations of disease. However, virtual OSCE teaching has been able to bridge this gap to a large extent via the use of visual cues/diagrams demonstrating key clinical concepts to allow for the effective consolidation of important information, and use of polls to allow for attendees to engage anonymously with tutors to facilitate a greater learning experience. The use of features such as breakout rooms allow for students to receive more individualised attention from the tutor during small group learning sessions and engage with their peers in constructive discussions.¹³ However, there was no significant difference in the participants’ likelihood of asking questions in virtual OSCE teaching compared with a face-to-face equivalent, suggesting no effect of mode of content delivery on this. This contrasts with studies which show that a higher degree of participation and interaction can be observed due to the less intimidating nature of the virtual environment.¹⁴

Grover et al discussed the usefulness of employing a hybrid approach, involving both face-to-face and virtual formats for the delivery of OSCE teaching.⁶ Our study has identified many factors of virtual OSCE teaching deemed to be advantageous by medical students (eg flexibility, convenience and attractive features). This is important as these factors specifically relating to virtual OSCE teaching have not been previously explored in the literature. Therefore, the identification of these factors may allow for the use of virtual OSCE teaching in combination with face-to-face OSCE teaching to deliver quality teaching to a global audience, especially as the restrictions brought upon by the COVID-19 pandemic ease and in-person teaching sessions resume. Moreover, other medical education institutions/organisations may even be able to develop a rough framework for the delivery of virtual OSCE teaching by considering the factors we have identified in the study (eg providing offline resources to their attendees to tackle the issue of poor internet connectivity). However, they must tailor it to their sessions according to their audience to enhance student engagement and promote both active and passive learning.

Our results demonstrate that there are many barriers to delivering virtual OSCE teaching, mainly related to the use of information technology (IT). The majority of participants believed that the most apparent barrier to virtual OSCE teaching is poor internet connectivity. This is a practical concern dependent on factors such as location, other users, and devices utilized.¹⁵ It is imperative to be aware of these barriers so that appropriate measures can be made to ensure the efficient delivery of virtual OSCE teaching with minimal disruptions. For example, the problem of poor connectivity can be addressed by providing offline access to various resources or providing a recording of the teaching events so that they can be referred to later when there is better internet connectivity. Furthermore, it is important to tailor learning involving the internet towards different learning styles so that it is more impactful.¹⁶ It would also be advantageous for tutors to receive appropriate training in the use of online applications and devices to minimise the disruption that may occur during virtual OSCE teaching.

Some limiting factors of the study must be acknowledged. As mentioned previously in the materials and methods section, a cross-sectional approach was taken to limit duplicate responses and maximise the sample size, and the study only evaluated data obtained during a single teaching session. Therefore, a longitudinal study may have improved the reliability of results as further changes in opinions on the method of delivery of OSCE teaching could have been observed.

Conclusion

In conclusion, virtual OSCE teaching was deemed to be more flexible and convenient by the participants despite the majority stating their preference for face-to-face OSCE teaching. Virtual OSCE teaching via platforms such as Zoom™ can be utilised as an adjunct to supplement face-to-face OSCE teaching as social distancing eases, allowing the delivery of quality teaching to a wider audience from any location around the world in an interactive environment. Features such as interactive polling, breakout rooms and visual aids can be used to enhance the delivery of virtual OSCE teaching. These attractive features may even be adapted by other medical education organisations/institutions to enhance the delivery of virtual OSCE teaching and promote student engagement, but they must be tailored according to their audience.