Comparing Knowledge Acquisition and Retention Between Mobile Learning and Traditional Learning in Teaching Respiratory Therapy Students: A Randomized Control Trial

Introduction

The twenty-first century has shown a major shift in technological applications through all sectors, including teaching and learning.¹ The use of mobile devices is the leading technological advancement of the century; where according to worldometer, there were 7.33 billion mobile devices in use by the start of February 2023 against the global population of 8 billion people, showing a high penetration rate.² Mobile devices have influenced a shift in learning techniques such as mobile learning (M-Learning). M-learning is a subset of electronic learning and is considered the latest type of learning introduced as a result of the technological revolution.¹

When reviewing the literature, one can find many definitions for M-learning. However, the most common cited one, defined M-learning as

an any sort of learning that happens when the learner is not at a fixed, predetermined location, or learning that happens when the learner takes advantage of the learning opportunities offered by mobile technologies.³

Today, learners can use technological developments, such as tablet personal computers and smartphones to access learning resources, gain knowledge, and sharing information. Learners can access the educational contents when using the mobile through two main resources: applications and social media. Mobile applications consist of a software/set of programs that runs on a mobile device and perform certain tasks for the user.³ According to a study, system, intention, and user satisfaction were revealed to positively and significantly influence the use of M-learning systems.⁴

Literature has reported that m-learning tends to increase the interest in learning among students. This factor can promote the acquisition and retention of knowledge compared to traditional learning. Wilkinson and Barter⁵ supported that students tend to be motivated by the application of mobile devices in their learning. The researchers compared seminar groups taught using tablet devices and traditional methods when assessing the feedback from two cohorts through two academic years. The results indicated that using tablets increased attendance, progression, and achievement, highlighting the need to create a framework for incorporating M-learning to increase learners’ experience. Research conducted by Hosseini et al⁶ also supported that students’ interaction is improved through M-learning. This factor tends to improve knowledge retention since learners are free to engage themselves in discussion groups where activities such as research can be conducted easily. Such considerations have resulted in improved motivation and participation among students.

M-learning can increase students’ motivation, engagement and interaction. These human aspects are key to improving knowledge acquisition and retention. On the contrary, traditional learning techniques consider the teacher as the center of scientific knowledge, limiting the amount of knowledge acquired and retained. Abate⁷ conducted research to demonstrate the retention of knowledge among students subjected to mobile learning, with nursing students as the primary participants. The study confirmed that the podcast lecture group recorded higher scores than students who used traditional learning methods. Similarly, Ashiyan and Salehi⁸ analyzed the benefits of mobile learning. The research supported that the application of platforms such as WhatsApp on mobile learning tends to improve the retention of knowledge among Iranian English as a Foreign Language’s students because of the enhanced capacity to discuss and raise concerns, which may interfere with knowledge acquisition.

Although these studies supported that m-learning is associated with additional tools that aid in the retention of knowledge compared to traditional learning, there is limited research examining the effect of using mobile learning as a teaching modality on knowledge acquisition and retention in Saudi Arabia. Since the outbreak of COVID-19, major learning improvements have taken place where institutions have intensified their use of e-learning techniques to access students. Reports have indicated that, unlike traditional learning, M-learning has a higher acceptance rate because of accessibility, engagement, and personalization. Studies undertaken to examine the significance of quality metrics in increasing the usability of m-learning systems during COVID-19 found that the most significant aspects affecting learners’ satisfaction with mobile learning include service quality, information quality, and system quality.^9,10

Most of the existing literature conducted on Saudi Arabia’s educational system regarding M-learning focused on students’ readiness and attitude toward m-learning. Almutairy et al¹¹ investigated the readiness to apply M-learning among 131 students studying for bachelor, master, or PhD degrees in Engineering, Education, or Health Sciences through an online survey. The study found that most students globally are ready to use mobile phones for their studies since they use them for internet browsing, reading lectures from university websites, reading e-books, taking notes, learning without a teacher, translating, and knowledge attainment through information sharing. Students also believe that m-learning is more flexible than traditional learning because it eliminates the time and place limitations. Participants also pointed out that M-learning can bring new learning opportunities, save time, and enable them to get feedback from lecturers quickly and easily. Alanazi¹² found that students who graduated with Diploma in Education from Al Jouf University used mobile technology to support learning, allowing easier access to learning materials, content, and instructor interaction. These applications made learning easier, accessible, and customizable.

Although the students, according to the literature, are ready to use mobile devices in their learning and showed positive attitude toward it, limited studies are conducted to measure the effect of M-learning on knowledge acquisition and retention. Also, none of these limited studies was targeting respiratory therapy profession. Making it inaccurate to use their findings to make inferences on the suitability of M-learning as a teaching method. Thus, this study will evaluate the effect of using M-learning as a teaching modality on respiratory therapy students’ knowledge acquisition and retention. The study was based on behaviorism and constructivism learning theory to demonstrate how M-learning can be used as learning method.¹³ This study compared knowledge acquisition and retention among respiratory therapy students in their third year of study using traditional and integrated M-learning. The findings of this study will be used in making inferences during curriculum development and technology integration in learning in the health-care professions field.

Materials and Methods

Study Design

The study used a randomized pretest-posttest control group design. This type of study design is appropriate since it allows to meet the objectives of the study by allowing to compare participant groups (traditional lecture and mobile application lecture) and measures the degree of change in students’ performance on test items, occurring as a result of the intervention (mobile learning). The study period, including pretest, intervention, and posttest take over two weeks.

Participants and Setting

The study was conducted under respiratory therapy program at one governmental university in Saudi Arabia. The undergraduate respiratory therapy program consists of four years of study and one-year internship. The first two year (1st and 2nd) of the program are called pre-professional years, where the students study the general courses like biology, chemistry, physics, etc. The second two years (3rd and 4th) are called professional years, where the students study the respiratory therapy courses.

The non-probability convenience sampling method was used to include a cohort of 46 undergraduate respiratory therapy students (males and females) in their 3rd year of respiratory therapy program in fall 2019. A convenience sampling is defined as participants who are often readily and easily available to the researchers. It is considered as a type of non-probability sampling because not every member in the target population has an equal chance to be selected.¹⁴

Fourth year respiratory therapy students were excluded since they have taken the topic that has been used in the lecture, whereas 1st and 2nd year respiratory therapy students were excluded since they are in the pre-professional phase of curriculum and they are not ready yet to take the topic of the lecture used in this study.

Randomization

A random number was assigned to each student volunteered to participate in the study. Then, the 46 participants were randomly allocated (1:1) to either the control group (traditional learning; group 1, n = 23) or the intervention group (mobile learning; group 2, n = 23) using SPSS software. The randomization was carried out by a research assistant and took place after written informed consent had been obtained from the participants. The participants were unaware of the allocation until immediately before the start of the lecture.

Study Intervention

The intervention used in this study is the m-learning. The m-learning was conducted in this study using two mobile applications that are free and can be downloaded using either IOS or Android operating system. The first app is called ABG Book and it has the theoretical knowledge needed for the lecture topic (Figure 1).

Figure 1 Screen snapshot of the ABG Book App.

The second app is called Arterial Blood Gas (Lite) and it is an interactive app to practice interpreting arterial blood gas results (Figure 2).

Figure 2 Screen snapshot of ABG Analyzer (Lite).

Each two students are asked to read and understand the theoretical material provided in the first mobile app. After each paragraph, one of the students is asked to explain what he/she understands to others. If the understanding is correct, the instructor restates the explanation. If the understanding was not right, a chance is given to another student. If it is still not right, the instructor will explain it to the students. Constructivism theory is implemented here since students work and interact together to construct new idea based on what they read in the first app. After going through all the theoretical material, the students are asked to use the second app to practice interpreting arterial blood gas results. The instructors gave the students the parameter values to be entered into the app and then the students will interpret the arterial blood gas results without clicking analyze button. After that, the student will click the analyze button to see the correct answer and compare it with what he/she did. Same scenario is repeated to practice interpreting other arterial blood gas results. Behaviorism is applied here since the second app provides feedback to the students and the students can practice interpreting arterial blood gas results as much as they would to absorb the material.

Instrument and Study Outcomes

The study outcomes are (1) knowledge acquisition, and (2) knowledge retention. To assess the acquisition and retention of knowledge of the participants concerning teaching methods delivered by the researcher, a test consisted of 20 multiple-choice questions was constructed. The test questions measure the ability of participants to interpret arterial blood gas results and can be completed in a period of 30 minutes. The questions of the test were constructed by the first author and validated by 3 experts from the respiratory therapy department. The Cronbach Alpha was 0.671 which is considered moderate reliability.

The same test was administered for both control and intervention groups at three points of time: before and after the lecture and 2 weeks after the lecture (Figure 1). The pre-test was used to evaluate the knowledge base before the lecture. The post-test, taken immediately after the lecture, was used to evaluate initial knowledge acquisition. Additionally, two weeks after the lecture, another post-test was given to evaluate the amount of knowledge retained. Identical test was repeated three times to control confounding variables related to changing the questions itself. Efforts were made to reduce bias. Students were neither informed that the test contain same questions, nor the test will be retaken after 2 weeks of the lecture.

The tests were corrected by two independent faculty members to minimize any possible biases. When the participant’s answer to a question was correct, they were given 1 mark. When the participant’s answer was incorrect, they were given 0 to that question. The resulting score (the main outcome) was accumulated by the researchers with a minimum of 0 point and a maximum of 20 points.

Procedure

Upon receipt of the study approval from the Institutional Review Board (IRB) with number IRBC/1947/19, all participants received a briefing session presenting the necessary information regarding their participation, the purpose and significance of the study as well as their rights for participation. Students were reassured that the test results used to measure the outcomes of this study would not be used in their course evaluations. Each participant was given a random number and the participant wrote this number in their test instead of their name. Informed consent was obtained from the participants.

Forty-six participants were randomly assigned to either group 1, n = 23 or group 2, n = 23 (Figure 3). In the first day of the study, the Group 1, who was the control group, completed a pre-test then was given a lecture about the arterial blood gas interpretation using traditional learning methods for 2 hours. The traditional learning method is centered on the teaching faculty in which the faculty provides information using the PowerPoint slides and students mainly listen and takes note during the lecture. When the lecture ended, the participants completed a post-test that contains same questions as in the pre-test.

Figure 3 Flow diagram of the study procedure.

In the second day of the study, the Group 2, who was the intervention group, completed a pre-test then they use the two mobile apps to acquire theoretical knowledge and practicing interpreting arterial blood gas results. The theoretical knowledge learned by mobile apps was the same content as in the PowerPoint slides of the control group. Refer to the intervention section for more detail about m-learning method. When the 2 hours ended, the participants completed a post-test that contains same questions as in the pre-test.

Two weeks after the lecture, another post-test, with the same questions as in the pre-test, was given to the participants in both control and intervention groups.

Data Analysis

The collected data (Raw scores for pre and post-assessments) were checked for completeness, coded then entered into Statistical Package for Social Sciences (SPSS), version 23 for data analysis. Descriptive statistics were used to describe participants’ demographic data. Percentage and frequency were used for categorical variables like gender, whereas the mean ± standard deviation was used for numerical data like age, final GPA, and total quiz scores (Supplementary material).

In terms of inferential statistics, an independent t-test was used to see the difference of quiz scores of students’ knowledge acquisition between the control and intervention groups before and immediately after the lecture and of knowledge retention between the control and intervention groups immediately after the lecture and 2 weeks after the lecture. Repeated measured ANOVA was used to determine the difference in quiz scores within the control group or within the intervention group among the three-time points: (1) prior lecture, (2) immediately after the lecture, and (3) 2 weeks after the lecture. For all the statistical analyses, the significance is defined as p < 0.05.

Results

Characteristics of Participants

Forty-five out of 46 participated in this study with a response rate of 98%. Twenty-seven (60%) of them were female, whereas 18 (40%) were male. The average GPA was 3.59 ± 0.90. The intervention group consisted of 22 participants in which 11 (50%) of them were females and the other 11 (50%) were males and their average GPA was 3.78 ± 0.82. On the other hand, the control group consisted of 23 participants in which 16 (70%) of them were males, whereas the other 7 (30%) were females and their average GPA was 3.41± 0.96.

Levene’s test was performed to check the homogeneity between the intervention and control groups regarding the pre-test score and academic achievement using GPA at the beginning of the study. Levene’s test showed no statistically significant difference between the two groups in regards to the pre-test and GPA (p= 0.537, p= 0.208, respectively). Therefore, the two groups are considered homogenous.

Comparing Knowledge Acquisition and Retention Between Using Traditional Learning and m-Learning

The independent t-test showed that there was no statistically significant difference between the m-learning and traditional learning on knowledge acquisition (P= 0.305, Cohen’s d=0.309) and retention (p= 0.904, Cohen’s d=0.034). (Table 1).

Table 1 Comparing the Knowledge Acquisition and Knowledge Retention Between Traditional and Mobile Learning Groups Using Independent t-test

Comparing Pre-Test, Post-Test, Post-Test 2 for Each Group: Examining the Effect of Each Type of Learning

The repeated measures ANOVA showed that there was a statistically significant difference among pre-test, post-test, and post-test 2 within m-learning (p < 0.0001, partial η²=0.036) (Table 2).

Table 2 Comparing Pre-Test, Post-Test, and Post-Test 2 for Traditional and Mobile Learning Groups

Discussion

The purpose of this study was to compare the knowledge acquisition and retention of the 3rd year respiratory therapy students using traditional learning and m-learning. Results showed no significant difference in the knowledge scores of respiratory students in traditional and m-learning after the intervention. This study found that both m-learning and traditional learning are useful in improving knowledge acquisition and retention. However, no one is better than the other.

The findings of this study are supported by Fernández-lao et al¹⁵ which investigated the usefulness of mobile learning as a complement to traditional learning for physiotherapy students and obtained no significant difference in the acquisition of the theoretical knowledge between the two groups. Fernández-lao et al¹⁵ concluded that m-learning can be useful in supporting the learning process. Similarly, Kim et al¹⁶ evaluated the efficacy of a mobile application by assessing nursing students’ content knowledge, practical skills, and confidence in the simulated situations while providing care to infants with airway obstruction. In their randomized pre- and post-test design, study authors developed a test consists of MCQs and True or false questions. Though the knowledge scores in the mobile group were higher than the traditional group, no difference was found in the knowledge acquisition scores between the two groups.

While the aforementioned studies found no significant difference in the knowledge acquisition between traditional and mobile learning, other studies found a significant difference between the two types of learning. Nouriasl et al¹⁷ investigated the effect of smartphones application on the understanding of undergraduate medical students. In a semi-experimental design, medical students were randomly divided into experimental and control groups. They identified a significant difference in knowledge acquisition between the two groups. Therefore, the study concluded that mobile applications can be used as an additional method for traditional learning. Additionally, Albrecht et al¹⁸ conducted a study in Germany to compare the impact of mobile augmented reality on learners, especially in subjects of ethical sensitivity for instance forensic medicine. The study concluded that there was a statistically significant difference in the knowledge gain between the two groups (p=0.03), but both learning methods showed an improvement in the knowledge gain which suggests that m-learning has no significant harmful effect on students concerning knowledge acquisition. All the mentioned studies used MCQs or True or False questions to assess knowledge acquisition and retention. However, they argued the insensitivity of multiple-choice tests to assess the depth of content understanding.^19,20 The post-test 2, which we conducted two weeks after the post-test, did not reflect a significant difference in the learning outcomes among both groups. However, this study revealed significant differences between the mean scores for the pre-test, post-test, and post-test 2 encounters. Furthermore, a marginal drop from the post-test to the post-test 2 was detected for students in the m-learning. The findings indicated that the control group performed better in long-term retention of knowledge from post-test to post-test 2 than the students in the experimental group. These results emphasize the need to assess the possible factors influencing the m-learning and to design mobile learning module by limiting written text on the screen, improving audio capability, and using high-quality graphics are essential to the success of any mobile module²¹ and leading to better learning retention.^22,23

Limitations

This study has certain limitations: The results of this study cannot be generalized due to the small sample size since it is limited to the 3rd year respiratory therapy students from one institution. In addition, the study had not used standardized measures for knowledge acquisition and retention as well as no qualitative perspective has been taken into account. The fact that we did not examine learners’ attitudes toward mobile learning, which might sometimes be a barrier, was another limitation. Because they may use them for entertainment rather than learning, students may not realize the benefits of using mobile devices in the classroom.

Conclusion

The study assessed the knowledge acquisition and retention between m-learning and traditional learning and concluded that both m-learning and traditional learning are effective in increasing knowledge acquisition and retention of students. However, we did not find evidence for one approach being more effective. Further, the conclusion is difficult to reach because of the limited data on retention, which emphasizes the need for further studies with a larger sample size through a multi-institutional study to validate these findings. We believe that studies should be carried out on effective education communication devices which might assist in the learning process due to the ongoing advancements in electronic devices, including mobile devices, in particular.