The effect of e-learning on the quality of orthodontic appliances
Authors Schorn-Borgmann S, Lippold C, Wiechmann D, Stamm T
Received 8 December 2014
Accepted for publication 16 June 2015
Published 27 August 2015 Volume 2015:6 Pages 545—552
Checked for plagiarism Yes
Review by Single anonymous peer review
Peer reviewer comments 3
Editor who approved publication: Dr Md Anwarul Azim Majumder
Stephanie Schorn-Borgmann,1 Carsten Lippold,2 Dirk Wiechmann,3 Thomas Stamm2
1Private Practice, Nordkirchen, Germany; 2Department of Orthodontics, University of Münster, Münster, Germany; 3Department of Orthodontics, University of Hannover, Hannover, Germany
Purpose: The effect of e-learning on practical skills in medicine has not yet been thoroughly investigated. Today’s multimedia learning environment and access to e-books provide students with more knowledge than ever before. The aim of this study is to evaluate the effect of online demonstrations concerning the quality of orthodontic appliances manufactured by undergraduate dental students.
Materials and methods: The study design was a parallel-group randomized clinical trial. Fifty-four participants were randomly assigned to one of the three groups: 1) conventional lectures, 2) conventional lectures plus written online material, and 3) access to resources of groups one and two plus access to online video material. Three orthodontic appliances (Schwarz Plate, U-Bow Activator, and Fränkel Regulator) were manufactured during the course and scored by two independent raters blinded to the participants. A 15-point scale index was used to evaluate the outcome quality of the appliances.
Results: In general, no significant differences were found between the groups. Concerning the appliances, the Schwarz Plate obtained the highest scores, whereas the Fränkel Regulator had the lowest scores; however, these results were independent of the groups. Females showed better outcome scores than males in groups two and three, but the difference was insignificant. Age of the participants also had no significant effect.
Conclusion: The offer that students could use additional time and course-independent e-learning resources did not increase the outcome quality of the orthodontic appliances. The advantages of e-learning observed in the theoretical fields of medicine were not achieved in the educational procedures for manual skills. Factors other than e-learning may have a higher impact on manual skills, and this should be investigated in further studies.
Keywords: e-learning, orthodontics, laboratory, skills, appliances
Quality and innovation in university teaching are continuously evaluated. Currently, these aspects are dominated by the Internet because devices such as smartphones and tablet PCs have become part of the daily lives of students. This ever-increasing mechanization is also present at the university when students are taught over the Internet. It has been shown that from 2008 to 2012, the use of mobile devices for learning increased fivefold.1 The task of any academic institution is therefore to evaluate the benefits of such media tools with respect to universities and their students.
Numerous studies have been conducted on e-learning in the field of medical and dental training in which participants are asked about their attitudes toward this subject. Results of these studies show that the respondents enjoy taking new media course offerings,2–7 without missing the traditional style of teaching.8
The majority of studies focus on theoretical education. There is a positive trend that e-learning leads to an increase in theoretical knowledge,9–14 and only a few studies have shown no difference in terms of knowledge improvement15 or that e-learning alone leads to poorer results.16
A questionnaire on the topic of e-learning is a valuable tool but does not reflect the quality of the practical outcomes in medicine and dentistry. Only a few studies have assessed the transition from e-learning to practical skills. Investigations of life support training show similar results17 or moderate improvements18 in practical outcomes when e-learning is compared with traditional didactic teaching. In physiotherapy education, students with access to e-learning sources needed less time for musculoskeletal palpation but more time in obtaining ultrasound images.19 Dental students showed better results in carious lesion excavation when traditional lectures are accompanied with live demonstrations.20
The effect of e-learning on practical skills is yet not well known. There is a lack of data, specifically in disciplines where manual dexterity is required. Therefore, the present investigation aimed to evaluate the influence of online video material on students’ manual skills in manufacturing orthodontic appliances. The hypothesis is that live demonstration plus access to an online demonstration would be more effective than live demonstration alone.
Materials and methods
The ethics commission of the Medical Faculty reviewed the research project (Ref No: 2014-524-f-N) and gave permission to commence the study. A copy of the Institutional Review Board letter was sent to the Editor-in Chief. All participating students freely gave their informed consent to participate in this trial.
This study used a semi-blind three-parallel-group randomized controlled trial. The primary outcome was the students’ practical skill measured by the quality of the appliances that were manufactured during a preclinical orthodontic course.
The inclusion criterion for study enrolment was that students were eligible to attend the first preclinical course in orthodontics at the Dental School of the University of Münster, Germany. This included all students in the sixth semester who had passed the Physikum (preliminary examination). The exclusion criteria were failing the course (excluded from analysis), repeating the course, and violation of the course guidelines (Figure 1). All course students gave their informed consent to participate in the study prior to the beginning of the trial.
Figure 1 Flow diagram of the assessment for eligibility.
Regular attendance at the lectures was mandatory. Conventional lectures on orthodontic theoretical background and live demonstrations about appliance manufacturing were presented to all students. Lectures and live demonstrations were held in the lecture hall of the Dental School. Manufacturing of the appliances by the students took place in the student laboratory also located in the Dental School.
During the course, students were required to make three orthodontic appliances (Figure 2). The manufacturing of the appliances was demonstrated by an experienced dental technician. The live demonstrations were supported with cameras and monitors installed in the lecture hall. All procedures were video recorded by professional camera operators and prepared for viewing via the e-learning portal of the department. The video material was divided into sections and labeled for clarity (Table 1).
Figure 2 Appliances that students were required to make during the course. (A) Schwarz Plate (B) Karwetzky`s U- Bow Activator Type 1 (C) Fränkel`s Functional Regulator Type 3.
Table 1 Film processing
At the beginning of the course, participants were randomly divided into three groups. Randomization was performed by the investigator SS-B using a dice roller. Group one received conventional lectures and live demonstrations during the course schedule. Group two attended the lectures and also had access to an online blog. The blog contained general information about the course, handouts of oral lectures, links to relevant literature and a step-by-step description of the appliances’ manufacturing process.
Group three had access to all resources of groups one and two plus access to the online video material. Students in group three were allowed to download the videos on their devices for offline use.
At the end of the course, all custom-built appliances made by the students were collected and pseudonymized by the investigator SS-B. Tutors (postgraduate students of the Department of Orthodontics), who were blinded to the groups and the students’ names, rated the appliances according to a predetermined point system. It was based on a 0–15 point scale in which 0 represents the lowest and 15 the highest achievable score (outcome measure). The evaluation protocols for the different appliances are shown in Tables 2–4.
Table 2 Grading system Schwarz Plate
Table 3 Grading system Karwetzky’s U-Bow Activator Type 1
Table 4 Grading System Fränkel’s Functional Regulator Type 3
Eighteen pseudonymized appliances were randomly selected and both tutors evaluated them independently according to the evaluation protocol.
The statistical analysis of the data was performed using the statistical program GraphPad Prism 4 (GraphPad Software, San Diego, CA, USA). Variance analysis and a non-parametric t-test were utilized. To estimate inter-rater agreement, Cohen’s kappa values were calculated. To further differentiate Cohen’s kappa test, Spearman’s rank correlation coefficient was calculated. Means were compared using Dunnett’s test. A significance level with a 5% error probability level (P<0.05) was observed.
The first orthodontic course at the Dental School has a fixed group size of 55 students. All students were included in the study and randomized into one of the three groups. The trial started with the first lecture of the course and ended on the closing date for the completion of the appliances. During the course, five students violated the course guidelines and were excluded from the analysis. There were no further dropouts. There are no significant group differences concerning age and sex (Table 5).
Table 5 Representation of group distribution
Cohen’s kappa for inter-rater agreement was 0.212, different from 0, which means that this is not a random match (P=0.029). According to Landis and Koch,21 this is a fair agreement. However, a more reliable measure for ordinal data is the Spearman’s rank correlation coefficient, which was 0.834 (P<0.001). Cohen’s kappa measures the agreement between the evaluations of two raters when both are rating the same object on categorical scale. The Spearman’s coefficient is a parameter-free measure of correlations, which means it measures how well an arbitrary monotonic function to describe the relationship between two variables, without making any assumptions about the probability distribution of the variables.22
Concerning the different appliances made, there was no significant difference in the outcome score for all students (Table 6). The distribution according to the appliance type and group is shown in Figures 3–5.
Table 6 Mean outcome scores and standard deviations concerning groups and appliances
Figure 3 Result-specific differences in the Schwarz Plate on a 15- point scale.
Figure 4 Result-specific differences in Karwetzky`s U-bow on a 15- point scale.
Figure 5 Result- specific differences in Fränkel`s functional regulator Type 3 on al 15- Point scale.
The Schwarz Plate received the highest score, whereas the Fränkel Regulator had the lowest score (Figure 6). Within the groups, there was a similar pattern without any significant difference.
Figure 6 Device-specific differences on a 15- point scale.
Considering the different appliances, the results show that group two had slightly higher scores than the other groups, except for the Fränkel Regulator. However, the differences are not significant.
Concerning sex differences, females showed better outcome scores than males in groups two and three. The highest difference was found in the Fränkel group with 4.3±1.2 points for males and 8.3±2.0 points for females. Given the small group size, a statistical test was not applied.
To assess whether age has an effect on the outcome scores, the data were split according to participants’ age: ≤24 years (n=23) and ≥25 years (n=27). Although there are differences in the outcome scores because of the small numbers of the subgroups, the differences are not significant.
In this study, we assessed the effect of e-learning resources on the outcome quality of orthodontic appliances manufactured by dental students in their first orthodontic course. Development of e-learning content is a time- and resource-consuming process aimed to improve students’ theoretical and manual skills in various fields of education. It is therefore mandatory to evaluate the benefit of new teaching methods. Particularly in disciplines where manual dexterity is required, we think that additional resources like e-learning content can improve students’ ability to convert what they learn during lectures into practical knowledge. Compared with theoretical knowledge, manual outcome quality in medical education is difficult to measure and there is a lack of data in the literature.
Strengths and limitations of the study
Chen et al23 discussed teaching studies and suggested that methods are valid only if a) students are assigned randomly, b) quality and performance of the demonstrators are identical, and c) the content of the lectures and the learning environments are the same. All of these conditions were ensured in this study.
However, one limitation of the study may be the restricted number of participants assessed. The first orthodontic course at the Dental School in general is limited to students in the sixth semester who have passed the Physikum (preliminary examination). The course size depends on the number of laboratory work places available. Allocating students from higher semesters would bias the study because of their gained knowledge and skills in orthodontics. For example, Foley24 showed that the outcome quality between semesters varies. On the other hand, restriction to beginners could be seen as strength of the study because they have the same level of knowledge. Johnson et al showed that when students have a common subject but come from other disciplines, this leads to different behaviors in terms of the commitment to the learning progress.25
Randomization of participants is also a strength, but the procedure used should be discussed. Dice tossing is not truly random but pseudorandom.26 However, in this study, the bias of pseudo-randomization should be neglected.
Grading systems that are based on point scales are used in orthodontics to assess malocclusions and treatment outcomes.27,28 There are no validated indices available to evaluate the quality of orthodontic appliances. However, the validated American Board of Orthodontics Objective Grading System has a comparable reliability to our score (Table 7). The 15-point scale has been used for years at the Dental School, and this score is equivalent to a passing grade in the overall school system. Rater and participating academic staff were familiar with the application of the score. We therefore assume that the score system did not bias the results.
Table 7 Comparisons of reliability measurement between the American Board of Orthodontics Objective Grading System and a 15-point scale system
Students were explained the study design and the hypothesis of the study to be able to give their free informed consent to participate. This involves the risk of bias with regard to the Hawthorne effect. We assume that this type of bias was minimized by the fact that the evaluation of the appliances (score) had no effect of passing the course. Moreover, participants were told that they will be unaware of their particular score. Another discussion point would be whether the Hawthorne effect may have been responsible for the student performance in the courses. On the one hand, it was inevitable in this study to let the students ignorant as you could communicate with each other during the course. On the other side, it is also improbable that the knowledge has changed over the study outcome, as the students were told that their performance will have no benefit for them. Which in turn leads to the hypothesis that the students carefully carried out the production of the devices using online courses when no advantages in terms of knowledge queries or university classroom benefits were present. In the sixth dental semester at the University of Münster, students are required to undertake many tasks and manufacturing devices is only a small part of their training. The students were not informed that the manufactured devices had been evaluated, or that a knowledge test of the manufactured equipment was not performed. Thus, whether a knowledge check would lead to significant differences in this field remains unclear. The theory was that students would prefer to use only the easily accessible, quickly viewable online answers for their exams.
A further bias could be the knowledge exchange between students because it was not possible to separate participants during the course. This problem was encountered by explaining the study protocol and behavior during the course prior to the study.
A further strength of the study could be seen in the low dropout and the statistical power, although fixed group size was fixed. Age and sex of the groups did not differ significantly, and all participants have completed the same basic studies in dentistry and have technical courses of undergraduate successfully passed. Further, all participants were skilled in the use of the e-learning resources that were maintained by the university’s IT structure.
E-learning is more than desire among students. This aspect was evident in the surveys of students where over 50% of respondents took advantage of the e-learning offer29 and 95% responded that e-learning was helpful.30
Moreover, the economic benefit of e-learning has been shown. The University of Chicago conducted a study to determine whether online lectures about sleep medicine could replace the traditional classroom setting. After 1 year, the cost of the online course was paid off, while the cost of employing the teaching staff was incurred repeatedly.31
Cohen et al32 conducted a meta-analysis on the impact of visual-based instruction in education. The authors compared the effects of visual-based and conventional teaching in five areas: achievement, retention, aptitude-achievement correlation, student rating, and course completion. In the area achievement 37 of 65 studies, visual based was better than examination performance in a conventional class and a clear majority of studies favored visual based. In the area retention, the difference, based on six studies, was not statistically significant. By 10 aptitude-achievement studies no reported a significant difference. In the point student attitudes five of six studies favoring visual based. Only 10 studies examined the effect of visual based on course completion. The average withdrawal rate was 13.1% in visual based, and the average rate in the conventional classes was 13.2%. Given these results, it could be speculated that information technology development facilitates e-learning.
However, there are only a few studies that evaluate e-learning with respect to teaching practical skills in comparison with traditional lectures. Preston et al33 divided graduate-level physiotherapy students into two groups to evaluate whether e-learning leads to an improvement in the practical skills applied to patients when coupled with traditional teaching methods. Although the results showed no significant differences, the group with the additional e-learning module tended to achieve slightly better results.33
Hards et al17 divided 20 anesthesia residents into two groups to examine the management of maternal cardiac arrest by using high-fidelity simulation and comparing the subsequent performance following either didactic teaching or e-learning. The results showed no significant differences; however, they discovered that e-learning and didactic teaching offer comparable benefits.17
In a study by Perkins et al,18 3,732 health care professionals received an advanced life support. One group of participants completed a 1-day course with e-learning; the other group participated in a conventional 2-day course. In comparison with traditional advanced life support, the e-learning group had a lower pass rate for cardiac arrest on the simulation tests, equal scores on a knowledge test, and diminished costs.18
Rosa et al20 assessed the influence of e-learning on students’ ability to remove artificial decayed dentine by hand. Twenty-seven students were divided into two groups in order to compare the traditional cavity removal lecture with a traditional lecture that included a practical, live demonstration. An evaluation of the practical cavity removal skills between the two groups showed no significant difference, although the group that received the live demonstration enjoyed a temporary benefit. However, students who participated in the live demo were able to remove nearly twice as much carious dentine in the same time amount of time as the other group.20
Chen et al23 investigated two groups of 16 dental assistants who were educated at the orthodontic department of the University of London. A test was conducted to determine whether the dental assistants obtained a better understanding by watching a 15-minute video demonstration on practical bracket positioning instead of listening to a 30-minute lecture.
No significant difference was found between the two groups, although with respect to bracket angulation, the group that viewed the video demonstration tended to show better results. Chen et al concluded from this study that it is slightly more effective to show video demonstrations of practical movements.23
There is hardly any published study that addresses e-learning effects on the quality of manufacturing processes in education. The results of the few studies on the practical outcome quality suggest that e-learning has only minor effects. Further studies will be necessary to proof or disproof the benefits of e-learning on manual skills.
Our study suggests that providing students with time and location-independent e-learning resources additional to the classical lectures did not increase the outcome quality of the orthodontic appliances. The advantages of e-learning observed in the theoretical fields of medicine were not achieved in the educational procedures for manual skills. Factors other than e-learning may have a higher impact on the outcome quality. Further studies will be necessary to proof or disproof the benefits of e-learning on manual skills.
We acknowledge support by Deutsche Forschungsgemeinschaft and Open Access Publication Fund of University of Münster.
SS-B and TS designed the study. SS-B, CL, DW, and TS analyzed the data. CL and DW contributed to writing the paper. CL and TS supervised the clinical procedures and data collection. SS-B and TS wrote the main part of the paper. All authors gave useful comment on the analysis of data and text of the manuscript. All authors approved the final version of the manuscript.
The authors report no conflicts of interest in this work.
Eggermont S, Bloemendaal PM, van Baalen JM. E-learning any time any place anywhere on mobile devices. Perspect Med Educ. 2013;2(2):95–98.
Jang HW, Kim KJ. Use of online clinical videos for clinical skills training for medical students: benefits and challenges. BMC Med Educ. 2014;14:56.
Goh J, Clapham M. Attitude to e-learning among newly qualified doctors. Clin Teach. 2014;11(1):20–23.
Meckfessel S, Stühmer C, Bormann KH, et al. Introduction of e-learning in dental radiology reveals significantly improved results in final examination. J Craniomaxillofac Surg. 2011;39(1):40–48.
Longmuir KJ. Interactive computer-assisted instruction in acid-base physiology for mobile computer platforms. Adv Physiol Educ. 2014; 38(1):34–41.
Albarrak AI, Mohammed R, Abalhassan MF, Almutairi NK. Academic satisfaction among traditional and problem based learning medical students. A comparative study. Saudi Med J. 2013;34(11):1179–1188.
Davis CR, Bates AS, Ellis H, Roberts AM. Human anatomy: let the students tell us how to teach. Anat Sci Educ. 2014;7(4):262–272.
Nkenke E, Vairaktaris E, Bauersachs A, et al. Acceptance of technology- enhanced learning for a theoretical radiological science course: a randomized controlled trial. BMC Med Educ. 2012;12:18.
Moazami F, Bahrampour E, Azar MR, Jahedi F, Moattari M. Comparing two methods of education (virtual versus traditional) on learning of Iranian dental students: a post-test only design study. BMC Med Educ. 2014;14:45.
Worm BS, Jensen K. Does peer learning or higher levels of e-learning improve learning abilities? A randomized controlled trial. Med Educ Online. 2013;18:21877.
Hearty T, Maizels M, Pring M, et al. Orthopaedic resident preparedness for closed reduction and pinning of pediatric supracondylar fractures is improved by e-learning: a multisite randomized controlled study. J Bone Joint Surg Am. 2013;95(17):e1261–e1267.
Kononowicz AA, Krawczyk P, Cebula G, et al. Effects of introducing a voluntary virtual patient module to a basic life support with an automated external defibrillator course: a randomised trial. BMC Med Educ. 2012;12:41.
Morgulis Y, Kumar RK, Lindeman R, Velan GM. Impact on learning of an e-learning module on leukaemia: a randomised controlled trial. BMC Med Educ. 2012;12:36.
DeBate RD, Severson HH, Cragun D, et al. Randomized trial of two e-learning programs for oral health students on secondary prevention of eating disorders. J Dent Educ. 2014;78(1):5–15.
Peroz I, Beuche A, Peroz N. Randomized controlled trial comparing lecture versus self studying by an online tool. Med Teach. 2009;31(6):508–512.
Bains M, Reynolds PA, McDonald F, Sherriff M. Effectiveness and acceptability of face-to-face, blended and e-learning: a randomised trial of orthodontic undergraduates. Eur J Dent Educ. 2011;15(2):110–117.
Hards A, Davies S, Salman A, Erik-Soussi M, Balki M. Management of simulated maternal cardiac arrest by residents: didactic teaching versus electronic learning. Can J Anaesth. 2012;59(9):852–860.
Perkins GD, Kimani PK, Bullock I, et al. Improving the efficiency of advanced life support training: a randomized, controlled trial. Ann Intern Med. 2012;157(1):19–28.
Arroyo-Morales M, Cantarero-Villanueva I, Fernández-Lao C, Guirao-Piñeyro M, Castro-Martín E, Díaz-Rodríguez L. A blended learning approach to palpation and ultrasound imaging skills through supplementation of traditional classroom teaching with an e-learning package. Man Ther. 2012;17(5):474–478.
Rosa QF, Barcelos TM, Kaizer MR, et al. Do educational methods affect students’ ability to remove artificial carious dentine? A randomised controlled trial. Eur J Dent Educ. 2013;17(3):154–158.
Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–174.
Spearmann C. The proof and measurement of association between two things. Int J Epidemiol. 2010;39(5):1137–1150.
Chen MS, Horrocks EN, Evans RD. Video versus lecture: effective alternatives for orthodontic auxiliary training. Br J Orthod. 1998;25(3):191–195.
Foley JI. Dental students consistency in applying the ICDAS system within paediatric dentistry. Eur Arch Paediatr Dent. 2012;13(6):319–322.
Johnson S, Purkiss J, Holaday L, Selvig D, Hortsch M. Learning histology – dental and medical students’ study strategies. Eur J Dent Educ. 2015;19(2):65–73.
Nagler J, Richter P. How random is dice tossing? Phys Rev E Stat Nonlin Soft Matter Phys. 2008;78(3 Pt 2):036207.
Song GY, Baumrind S, Zhao ZH, et al. Validation of the American Board of Orthodontics Objective Grading System for assessing the treatment outcomes of Chinese patients. Am J Orthod Dentofacial Orthop. 2013;144(3):391–397.
Bergold M, Strametz R, Weinbrenner S, et al. Evidence-based Medicine online for young doctors – a randomised controlled trial. Z Evid Fortbild Qual Gesundhwes. 2013;107(1):36–43.
Levine A. Evaluation of World Wide Web-based lessons for a first year dental biochemistry course. Med Educ Online. 2002;7:13.
Warnecke E, Pearson S. Medical students’ perceptions of using e-learning to enhance the acquisition of consulting skills. Australas Med J. 2011;4(6):300–307.
Bandla H, Franco RA, Simpson D, Brennan K, McKanry J, Bragg D. Assessing learning outcomes and cost effectiveness of an online sleep curriculum for medical students. J Clin Sleep Med. 2012;8(4):439–443.
Cohen PA, Ebeling BJ, Kulik JA. A meta-analysis of outcome studies of visual-based instruction. ECTJ. 1981;29(1):26–36.
Preston E, Ada L, Dean CM, Stanton R, Waddington G, Canning C. The Physiotherapy eSkills Training Online resource improves performance of practical skills: a controlled trial. BMC Med Educ. 2012;12:119.
This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.Download Article [PDF]