Assessment of Radiological Sciences Students’ and Interns’ Long-Term Retention of Theoretical and Practical Knowledge: A Longitudinal Panel Study

Introduction

Knowledge retention can be defined as “the proportion of information an individual retains over a certain period (ie, interval between an original learning test and retention test)”.¹ In general, the retention of knowledge declines to 75–89% of its original level after only a short period of time.² Additionally, retention rates decrease linearly over time in relation to the retention interval length. In various fields of study, retention rates of 85% after four months, 80% after 11 months, and 75% after 24 months have been reported.^2–4 Long-term knowledge retention across disciplines can be significantly affected by a wide array of factors, including the type and content of the material to be learned, the amount of original learning received, the instructional approaches used, the exam strategies employed, the retention interval, and the individuality of subjects.^1,2

Different educational methods (ie, theoretical, and practical) encompassing different instructional strategies may result in varying degrees of knowledge retention on the part of learners. In continuing medical education, students who attended lecture classes performed better on the three-month follow-up exam than those who attended small group classes.⁵ Conversely, classes with small groups promote more effective learning and better retention of information than lecture-based classes for students in their second year of medical school.⁶ Moreover, there was no statistically significant difference in the one-year knowledge retention among medical students who took lectures or self-study classes.⁷

Several tools have been reported for evaluating knowledge retention of students, interns, and health workers, including pre-posttest designs of identical questionnaires.^8–13 Conflicting findings have been reported regarding the effect of question type on retention rates. The one-month retention rates were equal for various college graduates taking multiple-choice-questions (MCQ) or short-answer exams.¹ Another study suggested that the retention rate for MCQs was higher than that for open-ended questions.¹⁴ On the other hand, a different study found that students performed better on open-ended questions than on MCQs on the retention test.⁴

The curriculum of Radiological Sciences (RADS) program at the College of Applied Medical Sciences (CoAMS) in King Saud bin Abdulaziz University for Health Sciences (KSAU-HS), Kingdom of Saudi Arabia (KSA) offers high-quality undergraduate education by combining intensive theoretical instruction and practical training to maximize the students’ potential.¹⁵ Although RADS programs’ curriculum incorporates passive traditional instructional methods; it also promotes blended, active, and collaborative learning environment through the implementation of various teaching strategies such as problem-based learning (PBL), and small group discussions. Additionally, RADS programs’ curriculum offers various field experience courses that are introduced to students throughout the third and the fourth years of the program and aimed to provide hands-on training.

The ability of RADS students to retain previously taught material is extremely important. In the absence of a deep understanding of RADS’ basic principles, the student’s clinical competence in the future may be questioned, as proficiency is highly dependent on knowledge and skills. During a student’s undergraduate education, longitudinal assessment of knowledge retention is essential to identify areas for improvement, and to allow students to recognize their strengths and weaknesses in various aspects of radiological sciences. To the best of the author’s knowledge, no studies have been conducted to determine how well RADS theoretical and practical knowledge are retained throughout subsequent years of study among Saudi male and female undergraduate students and interns. Thus, the aim of this study was to examine the long-term retention of radiological sciences’ theoretical and practical knowledge among two cohorts of fourth-year male and female students and interns at KSAU-HS, Jeddah campus. We further determine whether instructional delivery methods affect the one-to-two-year retention of radiological sciences’ knowledge.

Materials and Methods

Participant and Procedure

Since our study aimed to collect data from the same sample at different points in time, we implemented quantitative, non-experimental longitudinal panel study design. Using non-probability convenient sampling technique (ie, “strategy where participants are selected in an ad hoc fashion based on their accessibility and/or proximity to the research”),¹⁶ we invited the whole population of 42 fourth-year radiological sciences undergraduate students and interns to voluntarily participate in this study. The following intervention sequence was then followed: First, two cohorts of fourth-year radiological sciences undergraduate students (n = 19; 7 males, 12 females), and Interns (n = 16; 6 males, 10 females) at KSAU-HS-Jeddah campus agreed to voluntarily retake three onsite MCQs midterm exams after one-year interval from the initial midterm exam for fourth-year students (ie, when the fourth-year students were in the third year) and after two years interval from the initial midterm exam for interns (ie, when the interns were in the third year); Second, the first midterm exam was for a practical course (ie, Radiographic procedure), the second midterm exam was for a theoretical course (ie, Radiation Protection & Radiobiology) incorporating blending learning approach (ie, traditional face-to-face lectures, problem-based learning (PBL) instructional strategy where students are introduced to clinical scenarios pertaining to radiation protection and radiobiology to acquire knowledge, which is then applied to improve problem-solving abilities, allows students to develop independence, stimulates reflection, self-directed learning, critical thinking, and using online resources provided by the instructor for independent study and to reinforce concepts), and the third midterm exam was for a theoretical course (ie, Radiologic Imaging) incorporating passive learning approach (ie, traditional face-to-face lectures); Third, the number of midterm MCQs’ items for each exam were forty; Fourth, to assess the knowledge gained, the fourth-year students’ and interns’ initial midterm exams’ scores were retrieved from the assessment unit digital records; Fifth, to assess the long-term knowledge retention (ie, one-year retention for fourth-year students, and two-years retention for interns), the same 40-MCQs midterm exams were conducted in one setting, and participants were unprepared; Sixth, The numerical retake midterm test results were then calculated and imported along with the initial retrieved midterm test results in a Microsoft Excel spreadsheet.

Ethical Consideration

Written Informed consent was obtained from participants. Students’ academic evaluation was unaffected by participating or refusing to participate in the study. Data anonymization was maintained throughout. Neither participant’s names nor other identifying markers were on the test score sheets, only the students’ serial numbers were included. Participants’ confidentiality was maintained, and student’s performance was kept confidential during the study analysis. the participant’s test scores were kept in a secure encrypted Excel file.

Statistical Analyses

First, descriptive analysis showing demographics (ie, numbers, percentage), and students’ test scores (ie, sample mean (), standard deviation (SD), percentage of decrease, and percentage of knowledge retention) were generated. Second, Shapiro–Wilk test was used to examine if variable (ie, students’ test scores) were normally distributed. Third, since the students’ test scores were normally distributed, paired samples t-test was used to compare paired data (ie, students’ test scores at occasion 1 (ie, knowledge gained) and occasion 2 (ie, 1-year and 2-years knowledge retained). Fourth, a statistical comparison of the exam results (ie, initial exams and the repeats) between females and males was performed using independent samples t-test. Fifth, one-way analysis of variance (ANOVA) with Tukey post hoc test was used to examine differences in retention percentages between the 3 courses and among RADS male and female students and interns. The level of significance (α) was set as <0.05 and all data analysis were carried out using the statistical package SPSS version 23.

Results

Out of all 42 fourth-year students and interns, 35 (83.3%) participated. The numerical data for RADS fourth-year students and interns midterm exam scores for the practical course is listed in Table 1, and illustrated in Figure 1A and B. First, in paired comparisons among cohorts, the midterm 1-year retake exam mean scores were significantly lower than the initial mean scores (P = 0.04 for male and <0.0001 for female). Additionally, the midterm 2-years retake exam mean scores were significantly lower than the initial mean scores (P < 0.0001 for male and female). Second, the retention percentage of practical knowledge after one year was significantly higher for male students (78.6%) compared to female students (62.9%), with a p-value of 0.02. Third, there was no significant difference in the 2-years retention percentages of practical knowledge between male (64.7%) and female interns (66%) (P = 0.6). Furthermore, there was no significant difference between the 1-year and 2-years retention of practical knowledge among male and female fourth-year students and interns (P = 0.1 for male; P = 0.8 for female).

Table 1 Descriptive and Inferential Statistics: RADS 4th-Year Students & Interns Mean Midterm Exam Scores for the Practical Course

Figure 1 (A and B) Bar graphs showing RADS 4th-year students and interns mean midterm exam scores for the practical course.

The numerical data for RADS fourth-year students and interns midterm exam scores for the theoretical course (ie, blended learning instructional approach) is listed in Table 2, and illustrated in Figure 2A and B). First, in paired comparisons among cohorts, the midterm 1-year retake exam mean scores were significantly lower than the initial mean scores (P = 0.01 for male and <0.0001 for female). Additionally, the midterm 2-years retake exam mean scores were significantly lower than the initial mean scores (P = 0.002 for male and <0.0001 for female). Second, the retention percentage of theoretical knowledge via blended learning instructional approach and after one year was significantly higher for male students (78.3%) compared to female students (61.9%), with a p-value of 0.003. Third, there was no significant difference in the 2-years retention percentages of theoretical knowledge via blended learning instructional approach between male (70.9%) and female interns (61.5%) (P = 0.18). Furthermore, there was no significant difference between the 1-year and 2-years retention percentages of theoretical knowledge via blended learning instructional approach among male and female fourth-year students and interns (P = 0.3 for male; P = 0.9 for female).

Table 2 Descriptive and Inferential Statistics: RADS 4th-Year Students & Interns Mean Midterm Exam for the Theoretical Course (Blended Learning)

Figure 2 (A and B) Bar graphs showing RADS 4th-year students and interns mean midterm exam scores for the theoretical course (using blended learning instructional approach).

The numerical data for RADS fourth-year students and interns midterm exam scores for the theoretical course (ie, passive learning instructional approach) is listed in Table 3, and illustrated in Figure 3A and B). First, in paired comparisons among cohorts, the midterm 1-year retake exam mean scores were significantly lower than the initial mean scores (P = 0.001 for male and <0.0001 for female). Additionally, the midterm 2-years retake exam mean scores were significantly lower than the initial mean scores (P < 0.0001 for male and female). Second, the retention percentage of theoretical knowledge via passive learning instructional approach and after one year was significantly higher for male students (64.1%) compared to female students (47.6%), with a p-value of 0.04. Third, there was no significant difference in the 2-years retention percentages of theoretical knowledge via passive learning instructional approach between male (56.9%) and female interns (52.6%) (P = 0.5). Furthermore, there was no significant difference between the 1-year and 2-years retention percentages of theoretical knowledge via passive learning instructional approach among male and female fourth-year students and interns (P = 0.4 for male; P = 0.7 for female).

Table 3 Descriptive and Inferential Statistics: RADS 4th-Year Students & Interns Mean Midterm Exam for the Theoretical Course (Passive Learning)

Figure 3 (A and B) Bar graphs showing RADS 4th-year students and interns mean midterm exam scores for the theoretical course (using passive learning instructional approach).

In the RADS fourth-year students’ cohort, females had significant better mean scores than males in all initial exams (P < 0.0001 for the practical exam, P = 0.002 for the theoretical exam (ie, blended learning instructional approach), and P = 0.004 for the theoretical exam (ie, passive learning instructional approach)). Additionally, there were no significant differences in mean scores between males and females in all one-year retention exams (P = 0.07 for the practical exam, P = 0.3 for the theoretical exam (ie, blended learning instructional approach), and P = 0.7 for the theoretical exam (ie, passive learning instructional approach)).

In the RADS interns’ cohort, there were no significant differences in mean scores between males and females in all initial exams (P = 0.4 for the practical exam, P = 0.37 for the theoretical exam (ie, blended learning instructional approach), and P = 0.1 for the theoretical exam (ie, passive learning instructional approach)). Additionally, there were no significant differences in mean scores between males and females in all two-year retention exams (P = 0.6 for the practical exam, P = 0.1 for the theoretical exam (ie, blended learning instructional approach), and P = 0.24 for the theoretical exam (ie, passive learning instructional approach)).

The numerical data for the comparison of retention percentages between the three courses and for RADS cohorts is listed in Table 4, and illustrated in Figure 4A and B. First, for the fourth-year students’ cohort, the 1-year theoretical knowledge retention via passive learning instructional approach (53.1%) was significantly less than practical knowledge retention (68.2%) and theoretical knowledge retention via blended learning instructional approach (68.1%), ANOVA (F (2,53) = 5.949, p = 0.005). Second, for fourth-year female students, the 1-year theoretical knowledge retention via passive learning instructional approach (47.6%) was significantly less than practical knowledge retention (62.9%) and theoretical knowledge retention via blended learning instructional approach (61.9%), ANOVA (F (2,34) = 8.153, p = 0.002). Third, for fourth-year male students, the 1-year theoretical knowledge retention via passive learning instructional approach (64.1%) was less, but not significant compared to practical knowledge retention (78.6%) and the retention of theoretical knowledge via blended learning instructional approach (78.3%) (ANOVA (F (2,15) = 1.364, p = 0.29). Fourth, for the interns’ cohort, the 1-year theoretical knowledge retention via passive learning instructional approach (54.4%) was significantly less than practical knowledge retention (65.6%) and theoretical knowledge retention via blended learning instructional approach (65%), ANOVA (F (2,40) = 4.479, p = 0.018). Fifth, for female interns, the 2-years theoretical knowledge retention via passive learning instructional approach (52.6%) was significantly less than practical knowledge retention (66%) (ANOVA (F (2,28) = 3.391, p = 0.04) and less, but not significant compared to the retention of theoretical knowledge via blended learning instructional approach (61.5%) (P = 0.08). Sixth, for male interns, the 2-years theoretical knowledge retention via passive learning instructional approach (56.9%) was less, but not significant compared to practical knowledge retention (64.7%) and the retention of theoretical knowledge via blended learning instructional approach (70.9%) (ANOVA (F (2,15) = 2.680, p = 0.1).

Table 4 Numerical Data for the Retention Percentage Among the 3 Courses and for RADS Cohorts

Figure 4 (A and B) Bar graph showing the retention percentage among the 3 courses and for RADS 4th-year students and interns.

Discussion

The purpose of the current study was to assess the long-term retention of theoretical and practical radiological sciences’ knowledge for two cohorts of fourth-year male and female students and interns. The one-to-two years retention of practical knowledge and theoretical knowledge via a blended learning instructional approach was 61.5–78.6%, whereas the one to two years retention of theoretical knowledge via a passive learning instructional approach was 47.6–64.1%. The one-year retention of practical and theoretical information was significantly higher among fourth-year male students (64.1–78.6%) than females (47.6–62.9%). Additionally, despite the fact that different cohorts were studied, the long-term retention of theoretical and practical radiological sciences’ knowledge appears to plateau between one to two years from the time of acquisition.

To the best of our knowledge, this is the first study to assess long-term radiological sciences knowledge retention among Saudi male and female undergraduate students and interns. Several previously published studies have focused in assessing long-term retention of basic science knowledge learned in medical school using MCQs as the type of retention test questions and for a retention interval of 1–8 years.^17–26 The retention of anatomy knowledge among medical students was 75–83% after one to two years.^20,22 Similarly, the one- to two-year retention of knowledge among medical students and across various basic science domains was 66–70% for biochemistry, 88% for microbiology, 94% for physiology, 38% for neurosciences, and greater than 100% for pharmacology, pathology and behavioral sciences due to rehearsal and reinforcement during the retention interval.^{18–21,25,26} Furthermore, previous study examined the retention of biochemistry knowledge among 209 second, fourth, and sixth-year medical students, as well as 20 postgraduate medical residents, reporting retention of 51–52% over three to five years, and of 40% after eight years.¹⁹ Such variability in the long-term retention of knowledge across disciplines may be attributed to the type and content of the material to be learned, the retention interval, and the participants’ individuality.^1,2

Our study found no statistical difference between the one-year knowledge retained by RADS 4th year students and two-year knowledge retained by RADS interns; this is in agreement with a single study that found no differences between one-year and two-year anatomy knowledge retention among 304 second and third-year medical students;²² and in a disagreement with another study that found little to no decline in the general basic science one-year knowledge retention and significant decline in the two-year knowledge retention among 75 fourth-year medical students.²⁴ Further evidence that the type and content of the learned material, the retention interval, and the individual characteristics of the participants may impact long-term knowledge retention across disciplines.^1,2

The findings of our study suggest that gender significantly affects the one-year retention of radiological sciences’ theoretical and practical knowledge; male students retain more information, compared to their female counterparts. On the contrary, some studies have shown that female students perform better academically and retain more information than their male counterparts.^27,28 Nevertheless, the findings of these studies were inconsistent and require further investigation.^14,29 Among interns; however, gender was not significantly associated with the two-year retention of theoretical and practical radiological sciences’ knowledge in this study, in agreement with another study that found no gender influence on the long-term retention of physiology knowledge among Saudi medical interns.¹⁴

The findings of this study revealed that the blended learning environment using PBL as an instructional pedagogy along with the use of online resources to reinforce concepts learned in the classroom in addition to face-to-face learning stimulate long-term retention of theoretical radiological sciences’ knowledge compared to passive learning environment. While it appears that the teaching methods may also depend on individuals’ characteristics, and may be favored by RADS female groups only, further research must be carried out. Blended learning has the potential to improve retention, completion rates, and student satisfaction, as shown in several studies.^30–35 Similarly, the long-term retention of radiological sciences’ practical knowledge was superior to that of theoretical knowledge acquired through passive learning. This can be attributed to the practical knowledge delivery methods, which included face-to-face teaching, paired with practical hands-on training and hospital visits and observations. A hands-on approach to learning usually results in a deeper understanding and more ingrained knowledge, as well as better retention of information quickly and for a longer period of time, since such learning is often more based on real-life observation and experience.³⁶

This study suggests that clinical relevance and a blended learning environment can contribute to a better retention of radiological sciences’ knowledge, therefore curriculum planners may wish to consider incorporating these elements into the teaching of third- and fourth-year radiological sciences courses.³⁷ It is imperative that radiological sciences’ courses offer students the opportunity to acquire clinical reasoning skills, evaluate radiologic procedures and interventions critically, and analyze processes related to improving patient care. Although students are still expected to memorize much of what they are taught, the most important step in applied medical sciences’ learning is to create “thinkers” rather than “memorizers”.¹⁴

The cognitive load theory states that an individual’s intrinsic cognitive load is determined primarily by the volume of new information received and that learning is hindered when working memory capacity is exceeded in a learning task.³⁸ While reducing extraneous cognitive load of subjects such as radiographic procedure, radiation protection and radiobiology, and radiologic imaging might not be effective, cognitive load theory can be useful for explaining the challenges and successes in the education of radiological sciences professionals.³⁹ There is a greater chance of storing and retaining more information in the long-term memory for students who are enrolled in an educational program with an integrated approach.³⁹

Limitations

A possible limitation of our study may be that students who participated were less motivated to perform well in our knowledge retention exams since there was no reward for passing or consequence for failing. However, this is less of an influence on MCQs retention tests, but it has a greater impact on non-MCQs because they require more effort and time to complete.⁴⁰ Additionally, when explaining the research idea and introducing the consent form, most students expressed interest in knowing how much they remembered, which suggests they were motivated to complete the tests with their best efforts. Although 83.3% of all RADS 4th-year students and interns have participated in this study, the findings may need to be interpreted with caution due to the small sample size, and therefore may not be generalized to all radiological sciences’ courses. Since this study assessed two considerably different groups and in addition to the fact that the fourth-year female students’ group performed significantly better in the initial exams compared to males, while the male intern group performed better in the initial exams compared to females, the one-to-two-year retention findings should be interpreted with caution. Another limitation is the potential selection bias associated with the use of non-probability convenient sampling technique, as the research question addressed by this strategy is limited to the sample itself and therefore lacks clear generalizability.¹⁶ Additionally, the study focused on a single characteristic of the participants (ie, sex), and there may be many more confounding variables that may affect the findings of this study such as demographics (ie, students’ age, and socio-economic background), and study techniques. Furthermore, the lack of MCQs validity and reliability testing may raise concerns about the findings of this study.

Recommendations for Practice and Future Studies

A good recommendation for future research would be to repeat the study again with the next cohort of students and interns and evaluate the impact of different groups on the outcomes and also investigate the effect of demographics (ie, students’ age, and socio-economic background), and study techniques on the long-term knowledge retention.

Conclusions

After one to two years from the time of knowledge acquisition, radiological sciences’ fourth-year students and interns retained 61.5–78.6% of the practical knowledge and theoretical knowledge via a blended learning instructional approach and retained 47.6–64.1% of theoretical knowledge via a passive learning instructional approach, with male students showing higher long-term knowledge retention than their female counterparts. The radiological undergraduate curriculum should incorporate innovative teaching concepts that promote long-term knowledge retention, such as incorporating blended and active learning strategies (eg, enabling students to navigate and read medical images using diagnostic software) while avoiding information overload, enhancing self-directed learning, and promoting critical thinking, thereby making students active participants in their own learning and instructors facilitators of the learning experience. Further research is vital to confirm the advantages of the teaching methods discussed in the results and the differences observed in retention between males and females.