Assessment of reaching proficiency in procedural skills: fiberoptic airway simulator training in novices

Xinyuan Duan¹, Dongfeng Wu¹, Alexander F Bautista², Ozan Akca², Mary B Carter³, Rana K Latif^2
1Department of Bioinformatics and Biostatistics, School of Public Health and Information Sciences, University of Louisville, Louisville, KY, USA; ²Paris Simulation Center, Department of Anesthesiology and Perioperative Medicine, University of Louisville School of Medicine, Louisville, KY, USA and Outcomes Research Consortium, ³Medical Education Research Unit, Office of Medical Education, University of Louisville School of Medicine, Louisville, KY, USA

Background: The importance of fiberoptic bronchoscopy (FOB) and intubation (FOI) is well established to reduce anesthesia-related morbidity and mortality during airway management in clinical practice. We conducted a trial to determine whether simulation-based training of novices can increase FOB and FOI skills to an expert level.
Methods: Eight anesthesiologists as experts and 15 fourth-year medical students as novices were enrolled. The computerized AccuTouch^® Bronchoscopy Simulator (ATBS, Immersion Medical, Gaithersburg, MD) and the Human Airway Anatomy Simulator (HAAS, Medical Plastics Inc, Gatesville, TX) were used for FOB and FOI training and testing, respectively. Data included three discrete variables: the numbers of airway collisions, oral passes, and nasal passes. Experts were tested, while novices were tested, trained, and retested. Twelve novices returned after 2 months and were again tested, retrained, and retested. Data were analyzed by a newly proposed likelihood ratio test based on the assumption that the data follow a Poisson distribution. A testing procedure was derived to compare performance between experts, novices pre- and post-training, and novices after the 2-month lag period.
Results: As expected, experts outperformed pre-trained novices on all variables (P < 0.0001). Post-training, novices’ skills improved (P < 0.0001) to the point that there were no significant differences between experts and novices post-training (P = 0.1709, 0.9857, and 0.5014 for collisions, and oral and nasal passes, respectively). After 2 months there was a significant decay in FOB skills among novices compared to the prior level achieved (P < 0.0001), but brief retraining brought their performance back to the expert level (P = 0.2207).
Conclusion: Due to the small sample size, normal approximation is not applicable; therefore many existing tests are not appropriate. Our novel likelihood ratio test provided a powerful tool in analyzing the small samples of discrete training data. Simulation can be used to train novices in fiberoptic airway skills to an expert level in a relatively short period of time.

Keywords: fiberoptic intubation skills, simulator, medical simulation, medical education, maximum likelihood estimate, likelihood ratio test