Predictors of virtual reality simulation bronchoscopy performance among novice bronchoscopists

Introduction

Bronchoscopy is a central and important clinical procedure used in various specialties such as anesthesiology, critical care medicine, pulmonology and thoracic surgery.¹ Increased concerns for patient safety have prompted an ongoing shift from apprenticeship models of medical education to approaches that insulate patients from the initial learning phase in procedural training.² Simulation-based education appears ideally suited to offer effective training in a zero-risk environment. The use of simulator-based training in the field of bronchoscopy has been found to be better than traditional apprenticeship training upon comparing novices’ performance on patients.³ A comprehensive analysis of evidence regarding the efficacy and key features of simulator-based bronchoscopy training would facilitate decision-making among educators. Several recent reviews of bronchoscopy training have offered such summaries.^4–6 We believe that in order to better plan instruction and assess the efficacy of curricular interventions, valid and reliable assessments of technical skills are needed. One such assessment that has not been widely discussed in the literature is toward understanding the predictors of bronchoscopy performance among novices and their effect in the acquisition of basic bronchoscopy skills in a simulator-based training program. A recent study by a Danish group investigated whether male and female medical students performed differently while training in a bronchoscope simulator.⁷ They opined that gender differences should be taken into consideration, when planning and organizing medical education and assessment in simulation training. Along similar lines, we wish to explore the predictors of bronchoscopy performance among novices.

Materials and methods

Sitting and study design

This is a descriptive observational cohort study conducted between January 2018 and April 2018 at King Abdulaziz University Clinical Skills and Simulation Center, Jeddah, Saudi Arabia.

Participants and procedure

All internal medicine residents in training at King Abdulaziz University Hospital were invited to participate in this study. Out of the 57 residents invited, 53 residents consented to participate. All residents were novices’ bronchoscopists without any previous bronchoscopy or simulation experience. Each resident filled a questionnaire about demographic data. Thereafter, all residents received a pre-study orientation about the requested tasks. We used the Simbionix bronchoscopy simulator (Cleveland, OH, USA). Each resident was required to do two tasks. The first task is the basic scope manipulation task that is designed to make users get accustomed to navigating the bronchoscope in a cyber environment to further develop hand–eye coordination. The resident will navigate the scope in a narrowing industrial lumen, following a guiding light and is then asked to keep the insertion tube straight and use the control unit’s roll and lever to keep the scope’s tip in mid-lumen and avoid wall contact. The score will drop with each wall contact based on lumen’s width. Each resident performed this task three times, and the software randomly selects the path each time the resident made an attempt. Metrics and results about the time of the procedure, wall contacts and final scores were retrieved from the simulator. Mean score of the three attempts was analyzed.

The second task was the guided anatomical navigation task. Each resident was asked to perform bronchial navigation maneuver with directional guidance and get performance feedback to accurately examine as many lung segments as possible. The allocated time given for this task was 5 minutes for each resident. The metrics and results in regard to the number of segments examined and the percentage of time with clear visibility were retrieved from the simulator.

This study was conducted by classifying the participatory residents based on seven categories (gender, residency levels, handedness, smoking habit, cumulative grade point average [GPA], interest in a procedure-based career and gaming experience). We included smoking status because different studies have shown that smoking has negative effects on hand coordination, speed, alertness and attention.^8,9 Smokers were defined as those who regularly smoked at least one cigarette per day for at least 1 year. Video game players were defined as those who played video games daily or occasionally, whereas residents who never played video games were considered non-video game players.¹⁰ The outcomes measured were time taken to finish the first basic scope manipulation task, total score for basic scope manipulation task, percentage of time spent at mid lumen, percentage of time spent in contact with the bronchial wall, number of lung segments examined and percentage of time with clear visibility of lung segments. Each resident performed both tasks individually, and no interaction between the researcher and the candidate was allowed during task performance.

Ethical considerations

Informed written consent was obtained from all the residents prior to recruitment. Participants were assured of the confidentiality of their responses and that all findings would be used solely for research purposes. No incentive was provided for their contribution. Permission to conduct this study was granted by the Biomedical Ethics Research Committee of King Abdulaziz University.

Statistical analysis

All the data were collected and entered into the computer. Statistical analysis was done using Statistical Package of Social Science Version 16 (SPSS Inc., Chicago, IL, USA). The quantitative data were presented in the form of mean and SD. Student’s t-test was used for quantitative data of two groups and one-way ANOVA for more than two groups. The qualitative data were presented in the form of number and percentage. A P-value of <0.05 was considered to be statistically significant.

Results

Fifty-three internal medicine residents from different residency levels participated in this study, wherein 31 (58.5%) of them were males (Table 1). Residents were predominately (90.6%) right handed. There were only 7 (13.2%) smokers, and 23 (43.4%) residents had an experience of playing video games. Among all residents, only 14 were interested in pursuing a procedure-based career, whereas 39 residents did not have such interest.

Table 1 Demographic data of the residents in this study, n=53

Male residents achieved significantly higher score in the basic scope manipulation task than female residents (65% vs 46%, P<0.001; Table 2). Furthermore, smokers needed more time to finish the first task than non-smokers (mean 2.5 minutes vs mean 1.1 minutes, P=0.005; Table 2). On the other hand, the remaining parameters – residency level, hand orientation, cumulative GPA score, interest in a procedure-based career and video games playing – were found to have no significant effect on the scores and time taken to finish the basic scope manipulation task (Table 2).

Table 2 Results from the basic scope manipulation task Abbreviation: GPA, grade point average.

Furthermore, Table 3 shows that the percentage of time spent at mid lumen during the scope manipulation task was significantly higher for males as compared to female residents (48% vs 37%, P=0.003), and they displayed significantly less time in contact with the walls during the procedure (13% vs 20%, P=0.004). Residents who were interested in pursuing procedure-based specialty training spent significantly less time in contact with the wall (14.6% vs 20.3%, P=0.045). On the other hand, the other parameters – residency level, hand orientation, smoking habit, cumulative GPA score, interest in a procedure-based career and video games playing – do not significantly affect the bronchoscopy simulation procedure performances of participatory residents.

Table 3 Time spent at mid lumen or in contact with the wall during the basic scope manipulation task Abbreviation: GPA, grade point average.

Table 4 shows the number of collisions during scope manipulation across different lumen widths among different residents. Significantly less number of collisions were observed in the wide lumen for the male residents as compared to the female residents (P<0.001). Furthermore, in wide lumen, second-year residents demonstrated significantly less number of collisions than any other residency levels (P=0.018). Meanwhile, it was noted that there was no significant variation in the collision data with respect to different resident categories.

Table 4 Results from collision data derived during the basic scope manipulation task Abbreviation: GPA, grade point average.

The second task demonstrated the number of segments examined by each resident and the percentage of time with clear visibility during the 5-minute session. It was observed that there were no significant differences in the number of segments examined and the time with clear visibility across all variables (Table 5).

Table 5 Summary of examination of lung segments performed during the guided anatomical navigation task Abbreviation: GPA, grade point average.

Table 6 shows multiple linear regression (Enter model) for the prediction of higher score for basic scope manipulation task. The significant factors were determined by univariate analysis. Enter method was used to determine the most significant factors that predict the higher performance score. The significant factors were determined by univariate analysis. The only significant predictor for higher score was observed in male gender with high β coefficient and P-value of 0.000. Table 7 shows multiple linear regression (Enter method) for the prediction of time needed to finish the basic scope manipulation task. Enter method was used to determine the most significant factors that predict the time needed to finish the task. The significant predictors for shorter time are non-smoking and playing video games with P-values of 0.003 and 0.03, respectively.

Table 6 Multiple linear regression model (Enter method) for predictors of higher score of basic scope manipulation task Abbreviations: SE, standard error; Sig., significance.

Table 7 Multiple linear regression model (Enter method) for predictors of time needed to finish the basic scope manipulation task Abbreviations: SE, standard error; Sig., significance.

Discussion

Among all potential variables, this study demonstrates the significant influence of gender, smoking habit and interest in a procedure-based career on the bronchoscopy skills of residents. Our findings suggest that simulation bronchoscopy performance in regard to basic scope manipulation task was better among males, non-smokers and residents interested in pursuing a procedure-based career. However, the performance of all residents was similar while undertaking the guided anatomical navigation task. The higher residency levels or cumulative GPA did not give any advantage while performing the bronchoscopy simulation procedure (Table 2). This result can be interpreted to mean that the theoretical knowledge of a procedure and the actual performance are two completely different aspects (Table 2). The duration of time spent at mid lumen during the scope manipulation and the duration of time spent in wall contact signify the effectiveness of the bronchoscopy simulation procedure performed by participatory residents.

Overall, male residents have demonstrated better performance than female residents in basic scope manipulation task, where males took less time to finish the task and also achieved higher score. Furthermore, male residents also performed better with spending more time at mid lumen and lesser time spent in contact with the bronchial wall than that of female residents. These results are in agreement with earlier reports that establish the fact that male and female medical students differ in their visuospatial abilities, interest in surgery and technical experience.^11,12

In order to perform in a given task, an important factor driving force is motivation that is necessary in an acquisition task.¹³ Better performance of the males as compared to the females can possibly be attributed to the higher motivation in males as compared to the females in a competitive setting, which was also observed in a previous study.¹⁴ Even the number of male residents who participated in this study was higher than the females, which also points toward higher motivation among male students. But, parameters such as time taken to complete a task cannot be considered as the only parameter as it usually favors men.¹⁵ Therefore, it is also imperative to design studies in such a way that appeals to both genders.

Smokers were observed to poorly fare in the basic manipulation tasks than non-smokers (Table 2). This result is in line with studies that report on the negative effects of smoking on hand–eye coordination. It has been reported that smoking can affect the speed and coordination of upper limb in a negative way and cause its deterioration.⁹ A previous study also suggest that attention and alertness may be significantly impaired in smokers compared to non-smokers.⁸ However, our observation that non-smokers performed better in the scope manipulation tasks goes against counter-intuitive reports that show performance-enhancing effects of smoking habits.¹⁶

We found that factors such as interest in a procedure-based career and prior video-gaming experience, which would otherwise be considered trivial, have a significant impact on the simulator bronchoscopy performance, as supported by earlier reports.^10,17 Put together, these observations could not stress enough the importance of practice and guidance in acquiring bronchoscopy competence.

Interestingly, no statistically significant differences were noted in the performance of the residents in guided anatomical navigation task (Table 5). All data obtained for all the residents showed similar results in terms of the number of segments visualized and the percentage of the time with clear visibility within the allocated time duration (Table 5). It is possible that resident’s skills improved during the second task as they became more comfortable with the simulation setup during the second task. These results contradict a previous study that revealed that the male students were able to examine more segments than female students.⁷

Despite providing many interesting observations, this study has certain limitations. First, the gender differences that we observed in the simulation bronchoscopy performance may be perceived as biased as a result of unconsciously or purposely helping a specific gender and not the other during the procedure. However, prohibiting any interaction between the candidates and the investigator minimized this limitation. Furthermore, all procedure results were obtained objectively from the simulator. This was an effort to overcome information bias, which is a commonly encountered problem in observational studies. Additionally, we need to consider one of the fundamental limitations of observational research, which is the inability to attribute causation to the corresponding effect.

Although this study sheds light on several interesting aspects related to the predictors of performance of simulation bronchoscopy, systematic reviews such as the recent reviews on laparoscopy performance skills^18,19 will shed more light on the efficient translation of the procedural skills from the training program to the real-time patient-based setting. A recent review has also addressed the financial outcomes of the growing trend of simulation-based medical education.²⁰ All these factors must be considered before any general conclusions can be derived regarding the demographic features that bring about competence in bronchoscopy simulator performance.

Conclusion

In this study, we found that simulation bronchoscopy performance in regard to basic scope manipulation task was better among males, non-smokers and residents interested in pursuing a procedure-based career. When organizing future simulation-based teaching, it is imperative that simulation trainers and medical institutions/departments should be made aware of the various demographic factors that affect the simulation performance in order to make it effective.

Acknowledgment

I would like to thank the staff at the Clinical Skills & Simulation Center at King Abdulaziz University, Jeddah, Saudi Arabia.

Disclosure

The author reports no conflicts of interest in this work.

References

1.		Casal RF, Ost DE, Eapen GA. Flexible bronchoscopy. Clin Chest Med. 2013;34(3):341–352.
2.		Ziv A, Wolpe PR, Small SD, Glick S. Simulation-based medical education: an ethical imperative. Acad Med. 2003;78(8):783–788.
3.		Ost D, DeRosiers A, Britt EJ, Fein AM, Lesser ML, Mehta AC. Assessment of a bronchoscopy simulator. Am J Respir Crit Care Med. 2001;164(12):2248–2255.
4.		Davoudi M, Colt HG. Bronchoscopy simulation: a brief review. Adv Health Sci Educ Theory Pract. 2009;14(2):287–296.
5.		Stather DR, Lamb CR, Tremblay A. Simulation in flexible bronchoscopy and endobronchial ultrasound: a review. J Bronchology Interv Pulmonol. 2011;18(3):247–256.
6.		Nilsson PM, Naur TMH, Clementsen PF, Konge L. Simulation in bronchoscopy: current and future perspectives. Adv Med Educ Pract. 2017;8:755–760.
7.		Konge L, Ali A, Sorensen M, Bitsch M. [Gender differences among medical students in the approach to simulation]. Ugeskr Laeger. 2011;173(49):3170–3173.
8.		Bashir S, Alghamd F, Alhussien A, et al. Effect of Smoking on Cognitive Functioning in Young Saudi Adults. Med Sci Monit Basic Res. 2017;23:31–35.
9.		Kumar N, Khan M, Singh N, Singh S. Impact of smoking on speed and coordination of upper limb movement. Addict Health. 2014;6(3–4):155–158.
10.		Enochsson L, Isaksson B, Tour R, et al. Visuospatial skills and computer game experience influence the performance of virtual endoscopy. J Gastrointest Surg. 2004;8(7):876–882; discussion 882.
11.		Thorson CM, Kelly JP, Forse RA, Turaga KK. Can we continue to ignore gender differences in performance on simulation trainers? J Laparoendosc Adv Surg Tech A. 2011;21(4):329–333.
12.		Ali A, Subhi Y, Ringsted C, Konge L. Gender differences in the acquisition of surgical skills: a systematic review. Surg Endosc. 2015;29(11):3065–3073.
13.		Illeris K. Læring. 2. reviderede udgave ed. Frederiksberg: Roskilde Universitetsforlag; 2006.
14.		Lucas K, Sherry JL. Sex Differences in Video Game Play:A Communication-Based Explanation. Communication Research. 2004;31(5):499–523.
15.		Grantcharov TP, Bardram L, Funch-Jensen P, Rosenberg J. Impact of hand dominance, gender, and experience with computer games on performance in virtual reality laparoscopy. Surg Endosc. 2003;17(7):1082–1085.
16.		Heishman SJ, Kleykamp BA, Singleton EG. Meta-analysis of the acute effects of nicotine and smoking on human performance. Psychopharmacology (Berl). 2010;210(4):453–469.
17.		Schlickum MK, Hedman L, Enochsson L, Kjellin A, Fellander-Tsai L. Systematic video game training in surgical novices improves performance in virtual reality endoscopic surgical simulators: a prospective randomized study. World J Surg. 2009;33(11):2360–2367.
18.		Dawe SR, Windsor JA, Broeders JA, Cregan PC, Hewett PJ, Maddern GJ. A systematic review of surgical skills transfer after simulation-based training: laparoscopic cholecystectomy and endoscopy. Ann Surg. 2014;259(2):236–248.
19.		Dawe SR, Pena GN, Windsor JA, et al. Systematic review of skills transfer after surgical simulation-based training. Br J Surg. 2014;101(9):1063–1076.
20.		Zendejas B, Wang AT, Brydges R, Hamstra SJ, Cook DA. Cost: the missing outcome in simulation-based medical education research: a systematic review. Surgery. 2013;153(2):160–176.