Prediction of Graduate Learners’ Academic Achievement in an Online Learning Environment Using a Blended Trauma Course

Introduction

The modern concept of distance and hybrid “blended” learning started with the progress of postal services in the 19th century. The first offer of distance degrees was in the University of London, in 1858.¹

Distance education is an education that emphasizes using of technology for learning purposes, without being physically present in a traditional classroom. It is the creation of learning access when the teacher and the learners are detached by time and distance.²

Hybrid learning is an approach that combines face-to-face with internet-mediated instructions. Hybrid learning is also called blended learning, and mixed-mode learning. Hybrid learning has the advantages of flexibility in time, pace, and place to learn. Moreover, hybrid learning keeps the advantages of face-to-face learning which is still important in medicine to achieve the proposed clinical competency.²

In 2012, the concept of Massively Open Online Courses (MOOCs) was launched. Large learning management systems such as Blackboard or MOODLE have been used for online learning purposes. Two categories of learning management systems are described: commercial learning management systems and open-source learning management systems. The commercial LMS are paid platforms like Blackboard, SchoolGen, and WebCT. However, the non-commercial LMS are free open-source software as MOODLE, Claroline, and Bazaar.³

Higher education institutions have used Virtual Learning Environments (VLEs). A VLE can be defined as a group of software tools that reinforce administrative activities, learning, and scientific research utilizing the World Wide Web. Various recognized VLEs use built-in social media formats within their design.⁴

Regarding the Oxford University computing services, web learning can be divided into 5 main pillars: Information for news sharing, notifications, and other information such as policies and rules, programs and courses specifications as well as schedules; Content for providing various resources like lecture notes, e-books, educational videos and games, online simulations, and hyperlinks to external online sources; Communication that can be achieved through various tools such as e-mail, discussion forum, messaging applications, and wikis; formative and summative assessments that can be performed in various forms as quizzes and assignments; Management tools that can maintain planning and delivery of online courses as registration for courses, monitoring of enrolled students, and the processing of students’ grades.⁵

A modular object-oriented dynamic learning environment (MOODLE) is the officially adopted online learning management system in our institute. MOODLE is an open-source LMS. The instructors of the online courses can monitor the learners’ activities on the LMS.⁶

The virtual learning environment and the learning management systems can offer a central focus for both students and staff, wherever their geographic location. This allows the flexibility of time giving the two options of synchronous and asynchronous learning.^4,7

Predicting learners’ academic achievement in online education is challenging. However, it warns educators for the less engaged and struggling learners who are liable to fail or get low scores. Various variables may predict the achievement of learners in the online environment as the age of learners, their previous participation in online learning, marital status, previous academic achievement, and online activities during the course to measure the learners’ engagement. Learning analytics is collecting and analyzing data about the performance of learners in the online environment, for improving learning through detecting learners at-risk of low academic achievement. Online predictors can be the number of times the learner logged into the course, the number of activities that the learner completed on the platform, and the time spent on the platform.⁸

As continuous monitoring is an important aspect of effective online teaching, it was recommended in the literature to carry out more studies to explore the predicting factors either personal or educational variables. Studying the variables that can predict the academic achievement of learners in distance education can improve the achievement of learners. This goes with the concept of “prevention is better than cure”. It can help the online instructors design corrective educational strategies for the students who are expected to get poor academic performance. As there is no consensus regarding the early predicting factors of the academic achievement of learners in online education, our focus in this research is to explore the possibility of the prediction of the academic achievement of learners early in the hybrid course.^9–11

Methodology

Predictive correlation design was used in this research. Thirty-three postgraduate learners were enrolled in a hybrid trauma b course. The minimum sample size needed was calculated using the “G- power” software for calculating the sample size. Using a study power of 90%, type I error of 1%, and effect size of 0.85. Online learning was in the form of synchronous and asynchronous sessions. The online aspect was created on Moodle. The face-to-face aspect was in the form of clinical ward rounds, clinics, and operative theater.

The course was organized into themed weeks. Week one for orientation to the course, the online platforms, and the applications to be used in the online aspect of the course. In week 2 was the biomechanics of fractures. Week 3 exposed the learners to pathological fractures. Week 4 discussed the periprosthetic fractures. Week 5 was the time for osteoporotic fractures. Childhood injuries were discussed in week 6. The theme of week 7 was polytrauma patients. In week 8, joints dislocation was introduced. Week 9 was the time for discussing implant failure in trauma cases. In week 10, the compartmental syndrome was the learning theme. During week 11, learners learned about injuries in elderly patients. And lastly, sports injuries were introduced in week 12.

The online contents were presented as small chunks with descriptive headings and subheadings, to avoid cognitive load, respecting the attention span of learners. The arrangement of the course materials offered a simple format and guaranteed easy navigation. Moreover, each section (weekly contents) was on one screenshot (no need to scroll down the page), not to distract the learners (Figure 1). The e-learning materials including a schedule, course objectives, assessment plan, lecture notes, books, links for e-books, links for videos, links for gaming, assigned tasks, electronic discussion groups and quizzes were uploaded to the Moodle. They were organized into sections aligned with the sequence of the course.

Figure 1 Trauma B online platform on MOODLE.

A detailed plan for each topic was created to fulfill the learning outcomes. Touch surgery application was used for gaming. Touch surgery offered quizzes with immediate feedback for the learners. In addition, Vumedi website was used to offer videos of detailed surgical approaches, procedures, tips, and tricks.

As regards the assignments, instructions on how to complete the assignments were created and uploaded online. Due dates for submitting the online assignments were clear. Feedback and grading of the assignments were delivered regularly. These online learning activities exposed the learners to various experiences as answering script concordance test,¹² creating scripts in script-based learning activities,¹³ playing a game then taking a test, quizzes, and discussions. Feedback was delivered by the instructor regularly. The format and the maximum size of the assignment files were specified to the learners for each activity (500 MB).

WhatsApp group was created. Through this group, notifications, reminders, and links for the synchronous online meetings were sent. Learners were allowed to post on the group. They were welcomed to send their inquiries and suggestions.

Online synchronous sessions were delivered using zoom platform. An orientation session was delivered online. The netiquette was clarified to the learners. The link for the zoom meetings was sent to the learners via WhatsApp, and it was launched on Moodle.

Each online synchronous session was 1–2 hours long. Sessions were recorded and shared with learners after the session. The online whiteboard of each meeting, where ideas were recorded, was saved, and shared with the learners. Moreover, the chat was saved and shared. Sessions included discussion of approaching, diagnosis and management of clinical cases. Learners were asked to complete the recommended readings and watch the recommended videos before the synchronous meeting. After the session, learners worked on the assigned task.¹⁴

For example, the “Pathological fractures” session:

• A clinical problem about pathological fracture was presented.
• Learners had to read the assigned materials first.
• Learners were asked to read the assigned problem.
• Learners were asked to create a script for the case. In the script, the learner had to decide which of the provided information helped him reach the diagnosis. What was the surgical and/or non-surgical treatment appropriate for the case, instruments needed, relevant anatomy, key surgical steps, key points of assistant performance, post-operative care, and postoperative information supplements?

• Learners received feedback from the instructor.
• Synchronous discussion on zoom was offered to resolve any misunderstanding.

Regarding the asynchronous discussions, a discussion forum was used on Moodle. Probing question or statement was provided, and the learners were invited to discuss. Netiquette for using the discussion was clarified.

Age of learners and their previous experience in online courses were collected using online short form that is nominated (to be able to correlate data of learners to their grades). We used the number of logs into the online platform of the course (Moodle) in the first three weeks (learning analytics). The online pre-test that was used to explore the level of knowledge before taking the course was also considered as a predicting variable. Moreover, learners’ grades in the written exams were used in the analysis as the academic achievement indicator. The written exams were in the form of continuous assignments and a final test. The continuous assignments were intermediate-stake assessment points. Marks of the assignments accounted for 40% of the total written-assessment marks. However, the marks of the final written exam were 60% of the total written-assessment marks. The final exam contained 50% MCQs and 50% modified essay questions.

This research work aimed at answering the following question:

What are the predictors of learners’ academic achievement in online environment?

Quantitative data were coded, checked for errors, and entered Statistical Package for the Social Sciences (SPSS) Version 22.0 Software for statistical analysis. Shapiro–Wilk test was used to evaluate normality of distribution of data. Spearman and Pearson's correlation were used to study the association between variables. Regression analysis was used to formulate the prediction model.

Ethical Considerations

• Study procedures were reviewed and approved by the Alexandria Faculty of Medicine research ethics committee.
• Informed consent was obtained from the study participants prior to study commencement.

Results

Shapiro–Wilk test of normality showed that grades of written exams (W1), pre-test scores (W2), and number of logs (W3) were normally distributed (Normally distributed data by Shapiro–Wilk test (W1 = 0.97, p1 = 0.37), (W2 = 0.97, p2 = 0.44), (W3 = 0.96, p1 = 0.65)). However, age and number of previous online courses were not normally distributed. Statistical significance was set at the level of p ≤ 0.05 in all the statistical tests.

No statistically significant correlation was found between age of learners and the academic achievement [Spearman correlation coefficient (r) = −0.61, p = 0.74].

There was no statistically significant correlation between number of previous online courses and the academic achievement [Spearman correlation coefficient (r) = −0.042, p = 0.82]. Additionally, the correlation between pre-test scores and the academic achievement by the end of the course was not significant [Pearson’s correlation coefficient (r) = −0.89, p = 0.81].

There was not statistically significant correlation between the number of online logs in the first week of the course and the academic achievement [Pearson’s correlation coefficient (r) = 0.61, p = 0.58]. Similarly, correlation between the first 2 weeks of study and the academic achievement was statistically insignificant. [Pearson’s correlation coefficient (r) = 0.53, p = 0.52]. There was a significant correlation between the number of online logs in the first three weeks of the course and the academic achievement [Pearson’s correlation coefficient (r) = 0.89, p ˂ 0.01].

As there was no statistically significant correlation between age, number of previous online courses, pre-test scores, number of logs in the first week and number of logs in the first 2 weeks, and the academic achievement of learners. These variables were excluded from our assumptions as being predictors for the academic achievement. They were not included in the regression model. The number of logs in the first 3 weeks was the variable under study as a predictor for the academic achievement in the linear regression model.

On examining the residuals of the distribution of the number of the three-weeks logs, the scatter plot showed heteroscedasticity (Figure 2). Heteroscedasticity was fixed using weighted regression. Moreover, the Durbin-Watson statistic was 2.17 (between 1.5 and 2.5) and therefore the data were not autocorrelated.

Figure 2 Residual plot for assessing homoscedasticity of data.

On testing the other assumptions of linear regression, Cook’s values for all learners were below the cutoff point (4/n = 0.12). This means that there were no influential outliers in the predictor variable (count of logs) (Figure 3).

Figure 3 Cook’s distance for individual learners.

Weighted linear regression was used to test if count of logs into the online platform in the first 3 weeks of the course significantly predicted the academic achievement of learners (scores in written exams).

The fitted regression model was academic achievement (exam scores) = 3.48 + 0.75*(count of logs into the online platform in the first 3 weeks of the course).

The overall regression was statistically significant (R² = 0.89, F (1, 31) = 257.86, p = ˂0.00).

It was found that the count of logs into the online platform in the first 3 weeks of the course significantly predicted the academic achievement of learners (β = 0.75, p = ˂0.00) (Table 1).

Table 1 Regression Model Between Count of Three-Weeks Log into the Online Platform and the Academic Achievement of Learners

Discussion

Prediction of academic achievement in online environment can help the instructors to detect learners liable to get low scores. Our hypothesis was the creation of a prediction model for the academic achievement in online education through studying the age of learners, number of previous online courses, pre-test scores and number of logs into the online platform of the course. The age of learners was excluded from the model as it was not correlated with the learners’ grades. This finding was consistent with that of Amro et al.¹⁵ Similarly, Kovačić¹⁶ reported that age does not have a significant impact on predicting academic success. This can be explained by the impact of confounders that affect this relationship. Confounders as ability to deal with technology, willingness to learn new skills, time spent studying, social duties and responsibilities and previous experience in online learning. With larger sample, there could be a significant correlation between age and academic achievement. However, the meta-analysis study of Sturman¹⁷ revealed that age was correlated with the academic performance without studying the impact of age on the academic achievement. Additionally, Park et al¹⁸ revealed that the age of the students significantly predicted their academic achievement. They revealed that those with younger age who are familiar with the technology performed better in the online courses. Regarding the association between previous online courses and the academic achievement of learners in our study, there was no significant correlation between the number of previous online courses and the academic achievement in the hybrid trauma course. This finding is consistent with that of Nagel¹⁹ who reported that there was no statistically significant impact of previous experience of online courses on the academic achievement of learners. This finding may be explained by that, academic achievement is affected by the design of the course, the ability and preferences of the learner rather than his previous experience.²

As regards the previous experience in online learning, Regarding the association between previous online courses and the academic achievement of learners in our study, there was no significant correlation between the number of previous online courses and the academic achievement in the hybrid trauma course. This finding is consistent with that of Nagel¹⁸ who reported that there was no statistically significant impact of previous experience of online courses on the academic achievement of learners. This finding may be explained by that, academic achievement is affected by the design of the course, the ability and preferences of the learner rather than his previous experience.²⁰

On studying the association between the pre-test scores and the academic achievement (post-test scores), there was no statistically significant correlation. This finding is consistent with that of Mitra et al.²¹ This can be explained by the uniformly higher range of posttest score that would not correlate well with the pre-test score. Due to insignificant correlation, the previously mentioned variables were excluded from the prediction regression model.

The correlation between number of logs into the online platform of the course and the academic achievement became statistically significant after the third week of the course. Additionally, count of logs into the online platform in the first 3 weeks of the course significantly predicted the academic achievement of learners. Guidry K.¹⁰ series showed consistency with our results as regards the correlation between the online activity and the academic performance. Moreover, Rajabalee et al²² revealed consistent results as regards the association between the online activities and the academic achievement. This can be explained by the internal motivation of students to learn. The more motivated the learner, the more login activities to the platform of the course.

Limitations

The sample size was relatively small. However, the whole population (learners enrolled in the online Trauma b course were involved). For this sample size (33 participants) only 3 predicting variables could be considered understudy but only one variable was eligible for inclusion. Then the small sample size was considered.

Conclusion

This research work concludes that the records of the online activity of learners in the first three weeks of study can help in early prediction of their academic achievement. Age, previous online education, and pretest scores were not statistically significant predictors. Monitoring the online activities of learners on the educational platforms can enable the instructors to pick the struggling learners. This could help improving the educational impact and the outcomes of the online education. Future research with larger sample size is recommended. Additionally, studying the time spent on the online platform of the course as a predicting factor for the academic achievement of learners could be an interesting point of research that adds to the literature.