Reliability of didactic grades to predict practical skills in an undergraduate dental college in Saudi Arabia

Introduction

The dental program is an opportunity for dental instructors to gradually develop students’ knowledge, skills, and attitudes with the aim of graduating a competent general dentist. As students proceed through the dental curriculum, the basic knowledge is progressively gained from lectures and seminars. The development of manual skills on the other hand occurs initially with preclinical bench-top procedures (eg, typodont exercises) and progresses to mannequin simulation exercises before the student starts treating patients under instructor supervision.

The final grade of each course at the Faculty of Dentistry, King Abdulaziz University, is the combined grades of both didactic and practical scores, even though they are graded independently. For example, a high didactic grade and a poor practical grade would yield an average grade and this could bias the student’s actual practical performance.

To develop the qualification processes, it is imperative that dental schools are able to predict both the prospective academic performance and the clinical competencies of their students.¹ There are numerous articles in the literature that investigated the effects of different variables on predicting the performance of dental students. These studies assessed the relationship between predictor variables and student performance in preclinical laboratory courses, but there have been few studies highlighting the overall clinical performance. Preclinical predictors have mostly shown insignificant correlation with preclinical laboratory course performance.^2–9 Nonetheless, there is limited research correlating both the didactic and the practical components of these courses. In a recent study, Afify et al¹⁰ correlated didactic and practical performance of dental students. They found no significant correlation between the practical and didactic scores. However, they only followed one group of students and the sample size was relatively small to draw any significant conclusions. Hence, a follow-up study was recommended to evaluate a larger group in different courses.

The aim of this study was to investigate the correlation between the didactic grades and practical performance and to evaluate if didactic grades can in fact predict the practical performance of dental students.

Materials and methods

Didactic and practical scores of graduates from the Faculty of Dentistry, King Abdulaziz University, between the years 2009 and 2011 were collected. Four courses were selected that represented different levels of preclinical practical skills: Dental Anatomy, Operative Dentistry, Orthodontics, and Prosthodontics.^1,11–13 These courses were selected based on availability of complete lists of calibrated grading materials and similar weight distribution of marks for didactic and practical scores.

Students who repeated any of the three courses or did not take any of the four courses consecutively were eliminated. The final sample size consisted of 320 students, 137 males and 183 females. The study was reviewed and approved by the Research Ethics Committee at the Faculty of Dentistry, King Abdulaziz University.

Each of these courses required the completion of a didactic and a preclinical practical component, and scores were expressed in percentages. Didactic grades for all four courses were represented by the final grade of multiple-choice questions at the end of the course which constituted 60% of the final grade. The approximate percentages of multiple-choice questions from each domain were: knowledge =40%, comprehension =40% and application/analysis =20%. The practical modules represent an assessment of each student’s hand skill and dexterity as well as fine hand-eye coordination, and comprised 40% of the final grade. These practical modules comprised carving teeth from plaster and teeth set-up in the Dental Anatomy course; cavity preparation and restoration in the Operative Dentistry course; wire-bending exercises in the Orthodontic course; and fabrication of partial and complete dentures in the Prosthodontic course.^1,11–13

Illustrated handouts for each practical exercise were distributed to the students 1 week before the session and included the assessment and grading criteria for the upcoming exercise. Instructors for each practical session carried out calibration sessions to establish an agreement when grading. Three faculty members completed the grading of each student’s exercise independently and then averaged to obtain the grade for each exercise. Each faculty member was unaware of the student’s score that was graded by the other two faculty members.

Statistical analysis

Pearson product-moment correlation between practical scores and didactic scores were performed. Cronbach’s alpha test was used to evaluate the reliability and absolute agreement between didactic and practical scores for each subject. The analysis was conducted for the total sample then repeated for males and females separately. To correct for type 1 error, statistical differences were considered significant at an alpha level of less than 0.01. Moreover, the following guidelines were used when interpreting the Pearson’s correlation coefficient (r) for the strength of the association: r<0.3= weak; r=0.3 to <0.5= moderate; and r≥0.5= strong. Statistical analysis was performed using Statistical Package for the Social Sciences Version 20.0 (IBM Corporation, Armonk, NY, USA).

Results

The Pearson product-moment correlation coefficient showed that there was only a statistically significant but moderate correlation between the Dental Anatomy’s didactic grade and practical performance (P<0.01) for the total sample (Table 1). No significant correlation existed between the didactic and practical grades in the other three courses.

Table 1 Pearson product-moment correlations between practical scores and didactic scores for both females and males Notes: *Weak and statistically significant correlation at the 0.01 level (two-tailed); **moderate and statistically significant correlation at the 0.01 level (two-tailed); ***strong and statistically significant correlation at the 0.01 level (two-tailed).

When analyzing the outcome for males, a statistically significant but weak correlation was observed between the didactic grades and the practical performances of both Dental Anatomy and Operative Dentistry (P<0.01); however, no correlation was observed in Prosthodontics and Orthodontics courses (Table 2).

Table 2 Pearson product-moment correlations between practical scores and didactic scores for males Notes: *Weak and statistically significant correlation at the 0.01 level (two-tailed); **moderate and statistically significant correlation at the 0.01 level (two-tailed); ***strong and statistically significant correlation at the 0.01 level (two-tailed).

For the female graduates, there was a statistically significant but moderate correlation between the didactic grades and the practical performances for Dental Anatomy course only, P<0.01 (Table 3).

Table 3 Pearson product-moment correlations between practical scores and didactic scores for females Notes: *Weak and statistically significant correlation at the 0.01 level (two-tailed); **moderate and statistically significant correlation at the 0.01 level (two-tailed); ***strong and statistically significant correlation at the 0.01 level (two-tailed).

Table 4 shows the Cronbach’s alpha reliability and absolute agreement results between didactic and practical scores in each subject for the total sample. There was a poor degree of reliability between didactic and practical scores in all subjects. When evaluating the results of male and female students separately, only female students showed an acceptable agreement between didactic and practical scores in the Dental Anatomy course only (Table 4).

Table 4 Reliability analysis between didactic and practical scores

Discussion

The aim of this study was to investigate whether there is a correlation between the performance of dental students in didactic and practical parts of four dental courses. In the Faculty of Dentistry at King Abdulaziz University, students are introduced to the anatomic forms and shapes of deciduous and permanent dentition during the Dental Anatomy course. This course, as well as the other three courses, has a didactic part in the form of lectures, presentations, and seminars, and a practical part where students learn how to carve and shape different teeth using different types of waxes. In the following year, in the Operative Dentistry course, students are taught how to perform different classes of cavity preparations on typodonts and use a variety of dental materials (eg, amalgam, composites, and glass ionomer cements). The practical component of the Prosthodontic course comprises tooth set-up for partial and complete dentures. In the Orthodontic course, different skills are introduced (eg, bending wires). In all of these above-mentioned courses, students are taught different psychomotor skills and have to apply what was learnt during the didactic part to their practical exercises.

Recently, one study could not demonstrate any relationship between didactic scores of dental students in the same dental college and their psychomotor skills.¹⁰ Their findings are in opposition to the findings of the present study in which a significant correlation was found only in the Dental Anatomy course. This was also true where the reliability between didactic and practical scores were only acceptable in the Dental Anatomy subject and, in particular, more in the female students. These differences can be explained by two factors. The first being the small sample size in the previous study (86 students) compared to the current one (320 students). The other factor is that the previous study was a cross-sectional study that correlated didactic and psychomotor performance of one group of students, while this was a longitudinal study that evaluated three groups of students.

It was noteworthy to find that even though a statistically significant correlation between didactic and practical scores did exist; the Pearson’s coefficient was rather low or weak for the most part (Tables 1–3). This was substantiated by the Cronbach’s alpha test where all courses except Dental Anatomy demonstrated poor reliability to predict students’ practical performance from their didactic grade (Table 4).

Furthermore, the results of this study did not show significant sex differences in the relationship between the didactic grades and practical performances.

Knowledge on how to perform certain procedures is definitely reflected in their application in the laboratory or clinical setting. Students who are not aware or do not quite understand the shape of a tooth will definitely have difficulties carving its proper shape/form in wax or even reproducing the correct anatomic and physiologic aspects of different restorations. Students who cannot recall the different steps of cavity preparations or how different dental restorations require certain cavity designs will not be able to work on patients or perform their procedure properly and according to the required standards.

Feil,¹⁴ in a study assessing the application of psychomotor learning theory constructs in preclinical laboratory instruction, discussed how generation of knowledge is an important facet in psychomotor skills acquisition. Feil and Gatti¹⁵ examined the use of a validation technique called “Verbal self-reporting” and how it can be used in dentistry. Students were taped while preparing teeth. Later, the authors played the tape for the subjects and asked them to recall the mental process, which happened during the preparation. Subjects reported constant knowledge recall and generation during the process. The results of the study showed that didactic performance (recall of knowledge, cognitive thinking, and application of knowledge) could be mirrored in psychomotor performance. However, the present study contradicts this theory.

The majority of research in this field concentrates on how dental admission tests (perceptual aptitude test, manual dexterity test, etc) can predict students’ performance in both preclinical and clinical courses in dental school.^1,4,13,16 However, studies correlating didactic scores to psychomotor performance, which was the focus of this study, are scarce. There is a need to further test such association in an attempt to further improve the educational experience in dental schools, and to develop new techniques of incorporating general and specific didactic dental knowledge toward acquiring and improving different psychomotor skills required for different dental specialties.

The dental field requires fine manual dexterity that is gained from persistent practical and clinical teachings, which should be preceded by fundamental knowledge that is obtained during didactic courses, and both of these components should relate to one another. This study investigated the relationship between didactic grades and practical skills among dental students and whether didactic grades can predict students’ practical performance. The hypothesis was that if a student scored high in his/her didactic courses, he/she would perform equally in practical exercises. It was of interest to find that not all didactic scores can reliably predict the practical performance of a dental student.

Conclusion

Based on the findings of this study, the relationship between didactic grades and practical skills is course specific. Didactic grades do not predict the student’s practical skills. Measuring practical performances should be performed independently from didactic grading.

Disclosure

The authors report no conflicts of interest in this work.

---

References

1.	Gansky SA, Pritchard H, Kahl E, et al. Reliability and validity of a manual dexterity test to predict preclinical grades. J Dent Educ. 2004;68(9):985–994.
2.	Boyd MA, Wood WW, Conry RF. Prediction of preclinical operative dentistry performance in two instructional methods. J Dent Educ. 1980; 44(6):328–331.
3.	Boyle AM, Santelli JC. Assessing psychomotor skills: the role of the Crawford Small Parts Dexterity Test as a screening instrument. J Dent Educ. 1986;50(3):176–179.
4.	Coy K, McDougall H, Sneed M. Issues regarding practical validity and gender bias of the Perceptual Abilities Test (PAT). J Dent Educ. 2003; 67(1):31–37.
5.	Dworkin SF. Dental aptitude test as performance predictor over four years of dental school: analyses and interpretations. J Dent Educ. 1970; 34(1):28–38.
6.	Gray SA, Deem LP. Predicting student performance in preclinical technique courses using the theory of ability determinants of skilled performance. J Dent Educ. 2002;66(6):721–727.
7.	Kramer GA, Kubiak AT, Smith RM. Construct and predictive validities of the Perceptual Ability Test. J Dent Educ. 1989;53(2):119–125.
8.	Manhold JH Jr, Manhold BS. Final Report of an Eight-Year Study of the Efficacy of the Dental Aptitude Test in Predicting Four-Year Performance in a New Dental School. J Dent Educ. 1965;29:41–45.
9.	Phipps GT, Fishman R, Scott RH. Prediction of success in a dental school. J Dent Educ. 1968;32(2):161–170.
10.	Afify AR, Zawawi KH, Othman HI, Al-Dharrab AA. Correlation of psychomotor skills and didactic performance among dental students in Saudi Arabia. Adv Med Educ Pract. 2013;4:223–226.
11.	Boushell LW, Walter R, Phillips C. Learn-a-prep II as a predictor of psychomotor performance in a restorative dentistry course. J Dent Educ. 2011;75(10):1362–1369.
12.	Curtis DA, Lind SL, Brear S, Finzen FC. The correlation of student performance in preclinical and clinical prosthodontic assessments. J Dent Educ. 2007;71(3):365–372.
13.	Kao EC, Ngan PW, Wilson S, Kunovich R. Wire-bending test as a predictor of preclinical performance by dental students. Percept Mot Skills. 1990;71(2):667–673.
14.	Feil P. An assessment of the application of psychomotor learning theory constructs in preclinical laboratory instruction. J Dent Educ. 1992;56(3):176–182.
15.	Feil PH, Gatti JJ. Validation of a motor skills performance theory with applications for dental education. J Dent Educ. 1993;57(8):628–633.
16.	Gray SA, Deem LP, Straja SR. Are traditional cognitive tests useful in predicting clinical success? J Dent Educ. 2002;66(11):1241–1245.