Virtual Reality and Physical Models in Undergraduate Orthopaedic Education: A Modified Randomised Crossover Trial

Introduction

Physical models represent effective teaching tools in orthopaedics, but are rarely utilised at undergraduate level. With developments in virtual reality simulation, it is possible to simulate high-fidelity scenarios. Virtual reality has been demonstrated to be an effective teaching tool in orthopaedics,¹ but the extent of its usefulness has not yet been comprehensively explored.² Al Nammari et al³ found that the average duration of orthopaedic education in UK medical schools was around 2.65 weeks, and that medical schools may not be providing their students with a basic competence in musculoskeletal medicine. They also found that students with an interest in orthopaedics had a statistically significantly better understanding of musculoskeletal medicine than their counterparts. In 2010, the prevalence of musculoskeletal conditions in primary and secondary care in the UK was just over two in ten,⁴ and were responsible for 663,451 hospital admissions in 2018–19.⁵ This is not reflected in the UK medical school curriculum, and time spent teaching students about musculoskeletal conditions is disproportionate to their prevalence in clinical practice. The need for change in undergraduate orthopaedic education is clear, and with the rapidly advancing capability of virtual reality, it may present a novel, effective teaching tool for undergraduates in the UK. This study quantifies and compares the effectiveness of virtual reality and physical models in teaching undergraduate orthopaedic concepts.

Fracture fixation is an important skill for doctors, which is not often taught in UK medical schools. Open reduction and internal fixation (ORIF) is an important method of fracture fixation, but the concepts and practical skills are not often taught in UK medical schools.

The concepts of deformity are complex, and require specialist knowledge.⁶ For this reason, osteotomy and deformity principles are often not taught very well during orthopaedic training, even though these principles apply to all orthopaedic procedures. Malunion and non-union following fracture fixation are common, with incidence estimates ranging from between 5% and 10%.⁷ Therefore, it is important for undergraduates to have some understanding of osteotomy as a deformity correction measure following fracture malunion. That is why it is important that these principles are well taught, and is why a series of 3D-printed osteotomy models have been produced to improve learning and understanding of the osteotomy rules.

To the authors’ knowledge, this is the first study to assess the efficacy of virtual reality and physical models on long-term knowledge retention of orthopaedic concepts for undergraduate medical students.

This study intends to evaluate the effectiveness of high-fidelity virtual reality simulation in teaching undergraduate orthopaedic concepts, compared to demonstration using physical models, a more classical teaching modality.

Methods

A total of 49 medical students were invited to take part in a surgical simulation event at an institution in the UK. Students ranged from first year to final year. They were allocated randomly to one of the two groups: Virtual reality or physical models. An A-Priori power calculation was conducted using anticipated means of score improvement, whereby both groups would improve their post-training test scores by 50% compared to their pre-test scores. With an alpha of 0.05 (5%) and a beta of 0.20 (2%) resulting in a power of 80%, the sample size calculation resulted in a minimum number of 9 per group. Computer-based randomisation was employed for randomising delegates to each group. In order to be eligible, delegates must have been current students, studying medicine at an institution.

Both groups were asked to complete a timed single best answer pre-test, in order to assess baseline knowledge of fracture fixation and osteotomy. Students were not made aware of the topics they would be taught before the event to ensure that they would not be able to prepare for testing in advance. The questions asked were all approved by a consultant trauma and orthopaedic surgeon. The test included 10 multiple choice questions on the principles of fracture fixation, and 10 multiple choice questions on the principles of osteotomy. There was a total of 20 questions. Each correct answer was worth 1 mark.

The students then received an introductory 5-min case-based scenario before splitting off to receive further teaching. The groups remained separate after the pre-test. They were initially taught the principles of open reduction and internal fracture fixation. Both groups were taught using the same script, but teaching was delivered either via a virtual-reality video, or demonstration on physical models.

In the VR group, the teaching was delivered utilising a pre-recorded virtual simulation of principles of fracture fixation and osteotomy via Google Cardboard. This device constitutes a self-assembled carton headset with lenses and a strap which upon insertion of a participant’s smartphone is converted into a low-cost virtual-reality headset, however not providing a haptic feedback. Participants watched a virtual reality 360° video of an animated 3D model, with an audio overlay explaining the principles of fracture fixation and osteotomy. Students could see the virtual-reality models shown in the video from any viewpoint they desired and rotate the models, re-wind or watch the video again during the teaching period. They had 15 min to watch the video.

The group receiving teaching from physical models had the principles of fracture fixation demonstrated to them by a demonstrator on a model, using the same script used in the video, over the same time period. They were also able to practice the principles of fracture fixation themselves. Demonstrators rehearsed beforehand to ensure that their demonstrations were as close to 15 min as possible.

Both groups were then immediately given a timed post-test on fracture fixation, using the same relevant 10 questions from the pre-test. This test was given immediately after they had received the teaching to ensure that the two groups did not mix. They then moved on to the principles of osteotomy. Again, the same script was used for both groups, but this was delivered via either a virtual-reality video, or via physical models. Again, both methods of teaching lasted for 15 min.

Both groups were then immediately given a timed osteotomy post-test, using the same relevant 10 questions from the pre-test. All participants were then sent a follow up test at 6 weeks to complete at home. The same questions were used. The pre-test, post-test and follow-up test results were analysed for the normality of distribution and appropriate statistical tests were used to compare the means. All of the statistical analyses were performed using IBM^® SPSS^® Statistics (Version 25) (IBM, Armonk, NY, USA). As this study did not involve patients or biological products, ethical approval was not required.

Osteotomy

An osteotomy involves changing the angle of one segment of bone relative to another segment around an axis, known as the Angulation Correction Axis, or ACA.⁶ The point of intersection of the proximal and distal segments of a deformed bone is known as the Centre of Rotation and Angulation, or CORA.⁶ The CORA refers to the location of the deformity. Lastly, the osteotomy line is used to describe the location of any cuts to be made through the bone. The three rules of osteotomy determine the location of the ACA in relation to the CORA, which will in turn determine where cuts are made.⁶ The three rules are:

1. Both the osteotomy line and the ACA must pass through the CORA.⁶
2. The ACA intersects the CORA, but the osteotomy line does not.⁶
3. Neither the ACA nor the osteotomy line intersects the CORA.⁶

The choice of osteotomy rule used will depend on parameters such as the location of the deformity, and whether lengthening or shortening is required.

A wedge cut technique can be applied to each rule in one of 3 possible ways. Firstly, an opening wedge may be used, in which a cut is made through the bone, and the two segments angulated apart, resulting in a wedge-shaped gap within the bone.

Secondly, a neutral wedge can be applied, which involved removing a small wedge of bone from one side of the bone, and angulating the 2 segments, leaving a small wedge-shaped gap on the opposite side.

Thirdly, a closing wedge may be used, in which a wedge-shaped piece of bone is removed from the bone, and the two segments angulated, leaving no gap in the bone.

Materials

To demonstrate fracture fixation principles, hand models were used with compression plates and screws (Figure 1). A virtual-reality counterpart was also developed by the authors (Figure 2), using Rhino 5 3D design software, and Blender (a CAD/animation tool). To demonstrate the principles of osteotomy, the authors designed and 3D printed plastic models using Rhino 5 3D design software, and a Lulzbot TAZ 5 3D printer, in PLA plastic (Figure 3), demonstrating Rules 1, 2 and 3, respectively. The authors also developed a virtual-reality counterpart to demonstrate the concepts of osteotomy, also made with Rhino 5 and Blender (Figure 4).

Figure 1 Physical hand models.

Figure 2 Virtual-reality image of plating for fracture fixation.

Figure 3 3D printed osteotomy physical models.

Figure 4 Virtual-reality osteotomy model.

Results

All of the delegates enrolled at the start completed the study. Both the virtual reality and physical models’ groups demonstrated an improvement in the mean test scores between pre-training tests and post-training tests. After assessing the data for the normality of distribution, a non-parametric Mann–Whitney U-test was used. On average, participants in the virtual-reality group improved by 2.67 points (106.4%, [U=79.00; p<0.0001]) in fracture fixation test scores, and by 4.46 points (369.2%, [U=20.00; p<0.0001]) in osteotomy test scores. On average, the physical models group improved by 2.12 points (67.1%, [U=144.00; p=0.001]) in fracture fixation scores, and by 5.12 points (400%, [U=31.50; p<0.0001]) in osteotomy test scores. The test scores for fracture fixation in both virtual reality and physical model groups did not achieve statistical significance at the 6-weeks follow-up (p=0.331, p=0.062, respectively). However, there was a statistically significant retention at 6 weeks for osteotomy knowledge in both virtual reality and physical model groups (p<0.01, p<0.01, respectively).

Comparing the VR and the physical models groups overall, there was no statistically significant difference in the total means of post-training tests between the two study groups (U=260.5; p=0.4354). The total post-training mean score for the virtual-reality group was 10.83 out of 20, and the total post-training mean score for the physical models group was 11.68 out of 20.

The means for pre-training tests, post-training tests and 6-weeks follow-up tests for both groups and each training topic is summarised in Table 1. The response rate for the 6-weeks follow-up test was 67% in the virtual-reality group, and 68% in the physical models group.

Table 1 Mean Results for Pre-Test, Post-Test and 6-Week Follow-Up Tests Comparing Virtual Reality and Physical Models Groups

Figure 5 shows a box plot of test scores for VR fracture fixation, from pre-testing, post-testing and 6-week follow-up testing. Figure 6 shows a box plot of test scores for VR osteotomy, from pre-testing, post-testing and 6-week follow-up testing. Figure 7 shows a box plot of test scores for physical model fracture fixation, from pre-testing, post-testing and 6-week follow-up testing. Figure 8 shows a box plot of test scores for physical model osteotomy, from pre-testing, post-testing and 6-week follow-up testing.

Figure 5 Median scores, upper and lower quartiles, and minimum and maximum scores for pre-test, post-test and 6-week follow-up tests for fracture fixation taught using virtual reality.

Figure 6 Median scores, upper and lower quartiles, and minimum and maximum scores for pre-test, post-test and 6-week follow-up tests for osteotomy taught using virtual reality.

Figure 7 Median scores, upper and lower quartiles, and minimum and maximum scores for pre-test, post-test and 6-week follow-up tests for fracture fixation taught using physical models.

Figure 8 Median scores, upper and lower quartiles, and minimum and maximum scores for pre-test, post-test and 6-week follow-up tests for osteotomy taught using physical models.

Discussion

Virtual-reality promises to be an effective resource for supplementing surgical education within orthopaedic surgery. In a systematic review of virtual reality training in orthopaedic surgery, Aïm et al⁸ found existing studies to be almost exclusively in the field of arthroscopy for orthopaedic trainees. However, it identifies that only one of the included studies employ robust methodology focusing on improvements in operative technique. The study in question was a randomized-blinded study of 48 postgraduate year-3 residents training in arthroscopic knee surgery that demonstrated significant improvements when rated according to their procedural checklist (p=0.031) and in their probing skills (p=0.016).⁹ Whilst our study explores knowledge rather than technical skills, the finding of better knowledge of the overall procedure would be in keeping with our finding of improved knowledge of both fracture fixation and osteotomy.

Vaughan et al¹⁰ conducted a review of virtual-reality trainers within orthopaedic surgery and identify nine different simulators being used for hip trauma and fracture simulation, and only one available simulator for osteotomies. However, it provides more of an overview of the technical aspects of these simulators, rather than an evaluation of the effectiveness for training.

In terms of knowledge, Hariri et al¹¹ explore the effectiveness of virtual reality in teaching anatomy to medical students, and found that simulation was at least as effective as textbook images for learning anatomy. Furthermore, there was a 20% improvement in reported likelihood to engage with the simulator for learning than the group who used textbooks (p=0.02), suggesting that students are more likely to engage in more novel and innovative methods of learning than with traditional learning methods such as textbooks.

Sugand et al¹² found that virtual reality was effective in improving pre-post questionnaire scores by 11%, as well as performance for intramedullary femoral nailing in a study of 27 medical students. Whilst intramedullary femoral nailing is more of a specialist skill for orthopaedic trainees, the methodology employed is similar to our study, and osteotomy can be regarded as a similar type of skill. Their study corroborates our findings of improvements in knowledge retention immediately post of 27%, with sustained increases of 7.5% at 6 weeks.

The same test was used throughout the study, in order to ensure that the tests were of the same difficulty, and were assessing the same knowledge throughout. However, this may have sensitized the participants to the concepts they were being assessed on, meaning that they focussed more on certain parts of the teaching than others, thus potentially improving their post-test scores. However, this would be expected to be the same in both groups, so should not confound our comparison between VR and physical models. In addition, the students were sent the 6-week follow-up test to complete at home, meaning that they could not be assessed under exam conditions. This means that there is potential for students to look up the answers to the questions, however, given the strict time limit this would have been difficult to do.

The study was undertaken at an all-day surgical conference at St George’s University of London, and participants were rotated through various surgical speciality stands of which orthopaedics was one. This meant that the teaching sessions had to be given a number of times throughout the day, meaning there may have been some variation between teaching sessions. However, the same script was used throughout the day in an effort to minimise variation in the content being delivered.

There is also the concern of fatigue; of both teachers and participants. As the event was an all-day event, both teachers and participants may have become fatigued as the day went on, which may have affected test scores. However, this effect would be expected to be the same across the VR and physical model groups, so should not impact on the comparison between the two.

The VR and physical model groups were kept separate during the teaching session, but it was not possible to keep them separated throughout the day, nor was it possible to keep them separated from people who had not yet completed the orthopaedics session. This meant that some people may have known what would be assessed prior to testing, and how the teaching content may have been delivered, which may have influenced their test results. Again, we would expect this to affect both groups, in the same way, so should not influence our comparison of VR and physical models.

Whilst there is a relative paucity of literature on use of physical models for orthopaedic training, the above comparisons demonstrate that this study’s finding is broadly in agreement with the existing literature. This study explores different skills including fracture fixation, and osteotomy, and provides a longer-term follow-up for assessment of knowledge retention in medical students. There is future scope for work-place based assessments to explore transfer-validity of these skills in triage and clerking. However, it is fundamentally important that orthopaedic training equips tomorrow’s doctors with the ability to recognise emergencies and know when to appropriately refer, therefore both virtual reality and physical models can be regarded as effective adjuncts to the undergraduate curriculum.

Conclusion

Both virtual reality and physical models have been shown to be effective techniques for demonstrating the concepts of fracture fixation and osteotomy to undergraduates. Virtual reality is an expanding field, and it would be useful to investigate further its role in orthopaedic education. There is future scope for assessing the role of physical models and virtual reality in developing practical surgical skills of orthopaedic registrars and undergraduates.