Comparing Online and Face-to-Face Performance in Scientific Courses: A Retrospective Comparative Gender Study of Year-1 Students

Introduction

In January 2020, the World Health Organization (WHO) announced worldwide the outbreak of COVID-19, which is a new strain of coronavirus. Two months later, in March 2020, the disease was declared as a pandemic.¹ COVID-19 has thrown its immense impact on all areas of our life. The field of learning and teaching is of no exception. Schools and universities found themselves compelled to adopt new ways of teaching, as students were unable to come to classes and share the space and place with other colleagues.² From that time on, the COVID-19 pandemic has wreaked havoc in the field of education and health care delivery systems worldwide. Almost all educational institutions applied distant or virtual learning, ie, online learning as a substitute to face-to-face learning³ So, it is important to evaluate the impact of this phenomenon on different aspects of higher education industry including students’ performance in response to this unprecedented event.

The College of Medicine and Medical Sciences (CMMS) at the Arabian Gulf University (AGU). adopted the Problem Based Learning (PBL) method for its medical program.^3–5 The AGU students are mainly from the Arabian Gulf region of the world. Because of COVID-19 pandemic, virtual classes for medical undergraduate students at AGU were commenced on 21st March 2020, as almost all AGU students had no access to AGU campus or face-to-face learning after imposing the new rules due to this recent situation.

There are several online- teaching-barriers that could encounter any new applicant of this system One barrier was the psychological impact on the students, as fear of contracting COVID-19 by the students themselves or exposing their families had led to great impact on the students.^6–8 Other barriers in online teaching are: institutional and community barriers, limited IT resources or infrastructure, social inequality, as certain students could not afford buying computers, technological barriers, ie, internet access, lack of student-instructor contact, and differences in individual standards in how to apply online learning.^6–14

Online learning made the process of communication between students and their lecturers easier and more convenient, as students started receiving their medical education while staying at home.^15–17 Previous studies on the efficiency of online learning stated that this method of teaching should be consistently evaluated to potentially enhance e-learning.¹⁸ Recent studies were carried out on the impact of COVID- 19 on clinical practice by Felder et al and Tabataba^19,20 on how e-learning is considered a conducive way and an interactive method during the COVID-19 pandemic.^19,20

According to research, a significant challenge is how to apply suitable methods of examination and assessment in all courses.^18,21 Assessment can measure student’s performance, learning gaps, capability in learning, readiness to progress and tools of professional activities.²¹ Assessment helps prepare students for the future practice of medicine and to develop important professional values in later life²² and it is a learning instrument which can be considered a portfolio or a reflection exercise to improve students’ cognitive abilities and self- regulate their learning.²² As online experience of teaching is a novel method of teaching at AGU, it is important and beneficial to assess and examine the outcome of year 1, three semesters of online learning at AGU. In addition, the opportunity to compare males and females’ performances in the two ways of learning was seized, especially that this situation is a novel teaching experience in the history of the university.

As online teaching was a novel teaching experience in the history of the university, we seized the opportunity to compare male and female performance. A study by Kozuh et al²³ showed that there were no gender differences in problem solving abilities and performance. Other studies on Pakistani students by Faisal et al,²⁴ where academic performances of males were compared with female undergraduate medical students in pharmacology examinations, had revealed that there was no significantly difference in performance in terms of gender in both short essay and multiple-choice questions. Data obtained from a study by Preven et al²⁵ revealed that there were few significantly differences between male and female in regards to aspects of their training and expectations. In this study, we aimed to assess the academic performance of year 1 medical students, and to measure the academic performance based on gender in pre and during COVID-19.

Materials and Methods

Setting

The AGU is a regional institution based in Bahrain that offers medical education for citizens and residents of all Gulf Cooperation Council (GCC) states, namely, Saudi Arabia, Bahrain, Kuwait, United Arab Emirates, Oman and Qatar.²⁶ The MD program is fully integrated, problem-based approach and divided into three phases: Phase I, as year one with basic and fundamental courses in science (Biology, Physics, Biochemistry), Psychology, Sociology, and English. Phase II, which is given in a three-year period; from year 2 till year 4, consists of 9 units with each unit contains several cases or disease-scenarios that tackle a specific system, ie, cardiovascular, respiratory, gastrointestinal, renal.etc. The clinical phase ie, Phase III is given in year 5 and year 6.^2,4,5 The College of Medicine and Medical Sciences (CMMS) at Arabian Gulf University (AGU) has applied the virtual teaching since March 2020 as an alternative to face-to-face teaching. The university used Moodle and Zoom as an online way of teaching. In this study, during the academic year 2020–2021, year 1 students took Phase I as online learning in the first and second semesters. In addition, the first semester of the academic year 2021–2022 was also taken in an online way of teaching. However, prior to the COVID-19 pandemic, the students had taken their lectures in phase I in a face -to-face method, which triggered us to study the students’ performance under these two ways of learning. The students’ performance in the same three semesters during online period of teaching will be compared with the same three semesters performance of the conventional face-to-face way of teaching prior to COVID-19. In pre COVID-19 students took the exams on campus marking their answer on computer sheets, whereas during COVID-19, exams were taken online, as students used their own personal computers or iPads.

Study Design and Data Collection

This is a retrospective comparative cross-sectional study conducted on 402 year 1 medical students who were enrolled in the College of Medicine and Medical Science at Arabian Gulf University in the two academic years: 202 Year 1 medical students in the academic year 2018–2019 and 200 Year 1 medical students in the academic year 2020–2021, in order to compare the academic performance in science courses before and during COVID19. The sample: 402 student grades were obtained from the assessment office at CMMS-AGU. Our attempt was to shed some light on the efficiency of AGU experience in online learning for year 1 students (Phase I) and to evaluate the outcome of online method of teaching.

Exams were given as multiple-choice questions (MCQ). The grades were retrieved from “Exam Soft” by the assessment office at AGU. Data were installed on EXCEL sheets after the approval and verification of the assessment unit. The final grade of each course was approved by the college council. Students were allowed to have re-sit exams, only if a student fails in three courses or less. Students who fail in four courses were not allowed to take re-sit exams. Any student fails in only one course after re-sit exams will be dismissed from AGU. There is no repeat for year one.

Study Participants

The study sample consisted of 402 medical students who completed year 1 (phase 1) for pre COVID-19 period from 2018 to 2019 and during COVID-19 period from 2020 to 2021. The final course grades of the participants served as the primary comparative factor in assessing performance differences according to face-to-face learning (pre COVID-19) and online learning (during COVID-19). Of the 402 total participants, 202 (50.25%) were online learning while 200 (49.75%) were face-to-face learning. These two cohorts were considered as a limitation of the study. Of the 402 medical students, 118 were male (29.35%), while 284 (70.65%) were female. The study also used students from different GCC countries. There were 100 Saudi, 74 Bahraini, 182 Kuwaiti, 43 Omani, and only 3 students from other countries. This was a convenience, nonprobability sample. There were no changes in the curriculum, text books, or teaching materials. Also, the teaching staff for all the courses were the same in both pre and during COVID-19 periods.

Test Instruments

In this study, medical students’ performance was operationalized by final grades of the basic science courses (Biology I, Physics, Biology II, Biochemistry, and Biochemistry). The Test of all courses consists of a hundred A-types multiple-choice questions. The evaluation method for year-1 students depends on the students’ performance in two examinations, a med-term exam and a final exam which counts for 40% and 60% of the final grade, respectively. The two assessments were valid and relevant; they were useful in gauging student ability and generating objective performance measurements. The final grades were calculated as the sum of a mid-term exam (40%) and a final comprehensive exam (60%). Exams were item analysis of type-A MCQ’s (ie, with 5 multiple choices) and the best answer should be chosen.

The tests in both pre-and-during COVID-19 periods where prepared by the instructors of the subjects and reviewed by more than two specialized doctors in the same field. 50% of the questions were chosen from the university bank and 50% were newly written and reviewed questions by the instructor and reviewers. Serious attention was taken to make the exams with almost the same level of difficulty in both periods. The difficulty indices for all the questions were checked after the exams to detect the fairness of the questions.

As for the practical exercises videos were prepared for the students with all the necessary laboratory techniques and information the students need to know.

Extra care was taken in proctoring online exams. All year-1 exams were taken online using EXAMSOFT to ensure Exam Integrity.

The software, Examplify, ensures fair exams by a powerful proctoring action by disabling Wi-Fi, forceful full screen mode and blocking Buttons access. Examplify uses EXAM ID and EXAM Monitor. Both form a comprehensive proctoring solution.

EXAM ID does the Face Matching, while the EXAM Monitors, record the entire video of Exam taker and the Exam screen and also highlight any face or surrounding movements. The EXAM is timed and password protected.

Statistical Analysis

Quantitative variables were presented as mean and standard deviation. Bar graph and plot with interleaved error bars were used to represent the mean and the significance of the differences in the mean performance of year one medical students in all basic scientific courses (Phase I). unpaired T-test was used to compare medical students’ performance on all basic scientific courses before and during coronavirus pandemic. Effect Size was measured using Cohens d to identify the size of the impact of the independent variable (Two groups: before and during coronavirus pandemic) on the change in the performance of medical students. The conventional effect sizes proposed by Cohen’s d are 0.20 for small effect, 0.50 for moderate effect and 0.8 for large effect.²⁷ Statistical analysis was conducted using the Statistical Package for Social Sciences (SPSS) version 28. A p-value of < 0.05 was used to infer statistical significance.

Ethical Considerations

The study was approved by the Research and Ethics Committees of the College of Medicine and Medical Science (CMMS) at Arabian Gulf University (AGU) (approval number: (E28-PI-5-22).). The names of all students were kept anonymous. All data were kept confidential.

Results

The results showed that the mean performance of year one medical students in all basic scientific courses (Phase I) during the coronavirus pandemic was greater than the mean performance before the pandemic with the exception of the Biostatistics course (Table 1, Figure 1). Unpaired T-test results revealed that there were statistically significant differences between the mean performance of medical students before the coronavirus pandemic and their performance during the coronavirus pandemic in the following basic scientific courses: (Biology I, t=4.948, p<0.001; Physics, t=3.588, p<0.001; Biology II, t=2.678, p <0.01; and Biochemistry, t=3.661, p <0.001). To identify the strength of the performance mean difference between the two groups: before coronavirus pandemic (academic year 2018/2019), and during coronavirus pandemic (academic year 2020/2021) which showed statistically significant difference. The results of the Cohen test indicated that the effect size for each course: Biology I, Physics, Biology II, and Biochemistry were (Cohen’s d =0.494, 0.358, 0.267 and 0.365), respectively, which is indicated to moderate effect.

Table 1 Comparison of the Performance of Year One Medical Students Before and During the COVID-19 Pandemic

Figure 1 Comparison between the mean performance of year one students in basic science courses before and during the coronavirus pandemic (**Highly significant difference, *Significant difference, ns: not significant difference).

Regarding the Biostatistics course, the results indicated that there were no statistically significant differences between the mean performance of year one medical students before and during the coronavirus pandemic (t=1.866, p=0.063).

Before COVID-19

Comparative analysis was carried out to identify the performance of year one medical students in all basic science courses before the coronavirus pandemic, according to gender. The results showed that the mean performance of females in all courses was better than the mean performance of males (Table 2). To verify the significance of these differences, the results of the unpaired t-test indicated that there are significant differences between the mean performance of students in the physics course (t=2.229, p=0.027) and biostatistics course (t=2.021, p=0.045) favoring females’ performance (Table 2).

Table 2 Comparison of the Performance of Year One Medical Students Before and During the Coronavirus Pandemic According to Gender

During the Coronavirus Pandemic

The results showed that the mean performance of females in all basic science courses during the coronavirus pandemic was more than the mean performance of male students in all courses (Table 2). To verify the significance of these differences, the results of the unpaired t-test indicated that there are statistically significant differences between the mean performance of students in biology I course (t=2.535, p=0.012) which favors females (Table 2).

The results by gender showed that the mean performance of females was better than males across all scientific courses before coronavirus. Also, during the COVID pandemic, the mean performance of females was better than males in all basic science courses.

Discussion

As online teaching is a novel way of teaching in the history of AGU, it was of great interest to evaluate this new experience. The main two central issues of this study were: to assess the academic performance of year 1 medical students in pre and during COVID-19, and to evaluate the academic performance based on gender in pre and during COVID-19.

Online Performance of Yeat-1 in Scientific Courses as Compared to Face-to-Face

Results from Table 1 and Figure 1 showed that year one students’ performance in the scientific subjects before COVID-19: Biology I, Physics, Biology II and Biochemistry was significantly different when compared to the results of the same subjects during COVID-19 online learning. Online results were significantly higher than face-to-face results. Even though the online performance of the students in the biostatistics course was better but it was not significantly different than face-to-face learning as compared to online.

The possible explanation for the better performance in online exams could be due to the qualitative method of teaching as student have the videos of all the lectures which allow them to repeat seeing and revising the lectures more than once at their spare time. This fact helps them memorizing and recalling the information given by the instructors.

Another possible reason could be attributed to the fact that students could spare the time of commuting in studying, ie, the time spent in trips to-and-from the campus is invested in revision and preparing for the exams which could be the reason why the majority of third and fourth- year undergraduate medical students at AGU preferred online lab teaching method as revealed by a study by Joji et al.²

A study By Yazici et al, examined the prevalence of cheating behavior among university students before and during the COVID-19 pandemic revealed that there was an increase of cheating incidents in online education. In spite of the use of the software and applying serious measure to prevent cheating and invigilation, a high cheating rate was reported if conducted on campus²⁸ or in remote online exams.²⁹

The possibility of cheating could not be ruled out despite the application of draconian measures of proctoring by the IT department at AGU. This fact is supported by various previous studies.^28–31 The incidence of cheating increases during online examinations, and restrict guidelines must be imposed.³⁰ Data on exam cheating collected in Spain revealed a significant increase exam cheating during online exams held during the COVID19 pandemic.³⁰

In spring 2020, with sudden transition to online teaching and assessment at AGU, several obstacles were encountered by the faculty as well as by students, as neither were prepared to online teaching, learning, and the new assessment methods, such as problems with web camera’ operation. The drastic and prompt change in the method of teaching, learning and assessment created a tremendous challenge for the teaching faculty, students and the IT technicians at AGU. It took us some time to get acquainted with the new information technology methods and how to overpass the hurdles of applying these new methods of teaching. Faculty and administrators were more geared up with all the necessary infrastructure needed for online teaching and assessment. The results of the study showed that the students’ performance indicators before and during COVID19 were almost similar.

Even though the factor of cheating could not be ruled out as a reason for this improvement in performance, the fact that our results were within the expected range in online method of teaching and also, were on par with the face-to-face method of teaching makes cheating possible but a remote possibility. Also, the difficulty indices for all the questions were checked after the exams to detect the fairness of the questions for both periods.

Our results comply with results obtained by other studies tackling the same subject. Studies by Zheng et al³² and Hays et al³³ stated that online learning during COVID-19 produced equivalent or better performance as compared with pre-pandemic among dental school students.

Whereas a study by Joel Faidley,³⁴ on principle of accounting courses indicated that students performed significantly better in the face-to-face classes than the online section.

Results from a study by Albalushi et al¹⁸ indicated that online performance and assessment is equivalent to face-to-face in anatomy courses in Sultan Qaboos University- College of Medicine in Oman.

Other studies about online teaching in entry-level physiotherapy Rossttini et, al, and on massive online courses in health professionals’ continuous education by Longhini et al showed that online teaching is feasible as an option to face the teaching dilemma during COVID-19 pandemic.^10–13 Both, face -to-face and online methods of teaching lead to similar performance.¹⁰

Performance According to Gender

When comparing the mean performance of year one medical students in basic science courses according to gender before the coronavirus pandemic, the results showed that the difference between male and females’ performance is not significantly different in biology I, biology II and biochemistry, even though the mean is higher for females than males.

In physics, female students scored higher grades than males with a significant difference (p=0.027). Also, in biostatistics, female students scored higher grades than males with significant difference (p=0.045).

During COVID19 results showed that there is not significantly difference between female and male students’ grades, even though the mean value is higher in females than males. Only in biology I, female student scored significantly higher grades as compared to males (p=0.012).

Our results are in congruence with other studies carried by Kozuh et al²³ where their analysis showed that there were no gender differences in problem solving abilities and performance. Two other studies on Pakistani students by Faisal et al,²⁴ where academic performances of males were compared with female undergraduate medical students in pharmacology examinations had revealed that there was no significantly difference in performance in terms of gender in both short essay and multiple-choice questions.

Contrary to our findings, the data obtained from a study by Preven et al²⁵ revealed that there were few significantly differences between male and female in regards to aspects of their training and expectations.

Albeit, our results showed that there were no significant differences between male and female students’ performance in some courses, females students scored better than male students in at least some other courses. These findings were supported by the findings of another study at East Tennessee State University where the results of the study showed that female students scored significantly higher than male students in both methods of learning: face-to-face and online.³⁵

Limitation

The study was limited by the retrospective character of the cross-sectional design which might lead to a recall bias. In addition, this study centered around the method of selecting the study sample, because this was a convenient, nonprobability sample, the independent variables were not adjusted accurately, and it is limited to first-year medical students (phase 1). Moreover, the study is limited only to basic science courses and not to all the curriculum courses and the ratio is almost two and a half more females than males. Also, the students were mixture of more than one country with different social background.

Conclusion

Year-1 medical students’ performance during the coronavirus period seems better than pre COVID-19 era. Females’ performance was better than males’ in both periods; pre and during COVID-19 periods. Therefore, it is feasible to apply a hybrid way of teaching in the educational practice at AGU in the future.

Recommendations

Hybrid modes of teaching as a combination of both ways in delivering science courses for year-1 medical school students are recommended. In this way both the students and faculty can save time and efforts in learning and teaching, respectively. The results of this research gave us the importance of applying online learning as a conducive tool and as an alternative for face-to-face learning.