Improvement in Validity and Reliability of Airway Assessment Using Ultrasonography by Novice Personnel: A Learning Cohort Study
Received 22 June 2022
Accepted for publication 9 September 2022
Published 22 September 2022 Volume 2022:13 Pages 1103—1111
Checked for plagiarism Yes
Review by Single anonymous peer review
Peer reviewer comments 2
Editor who approved publication: Dr Md Anwarul Azim Majumder
Sumidtra Prathep,1 Pawarit Sikarettrakul,2 Sasikaan Nimmaanrat,1 Wipharat Juthasantikul,1 Orarat Karnjanawanichkul,1 Nalinee Kovitwanawong1
1Department of Anesthesiology, Faculty of Medicine, Prince of Songkla University, Songkhla, Thailand; 2Department of Anesthesiology, Faculty of Medicine, Police General Hospital, Bangkok, Thailand
Correspondence: Sumidtra Prathep, Department of Anesthesiology, Faculty of Medicine, Prince of Songkla University, 15 Kanjanavanich Road, Hat Yai, Songkhla, 90110, Thailand, Tel +66 887523423, Email [email protected]
Purpose: To assess improvements in the validity and reliability of novices’ skills in performing ultrasonography for airway assessment.
Patients and Methods: A learning cohort study was conducted with 20 anesthesiology residents and 10 volunteers in the Department of Anesthesiology, Faculty of Medicine, Prince of Songkla University. The four parameters of airway assessment were soft tissue thickness at the level of 1) hyoid bone (STT-HY), 2) true vocal cords (STT-VC), 3) thyroid isthmus (STT-TI), and 4) suprasternal notch (STT-SN). The magnitude of discrepancies between the residents’ and experienced anesthesiologists’ measurements was evaluated over the sequence of measurements in the 10 volunteers.
Results: The mean ultrasonic measurements of STT-HY by the experienced anesthesiologists and residents were significantly different (11.09 ± 3.14 mm vs 8.53 ± 3.02 mm, respectively; P = 0.008), whereas measurements of STT-VC, STT-TI, and STT-SN were not (7.18 ± 1.70 vs 7.14 ± 1.93, P = 0.32; 7.81 ± 2.14 vs 7.73 ± 2.19, P = 0.62; and 11.32 ± 3.33 vs 10.30 ± 3.02, P = 0.35, respectively). The mean discrepancy between the residents’ and experienced anesthesiologists’ measurements was close to zero throughout the sequence of measurements of STT-TI and STT-VC. However, the residents’ measurements of STT-HY and STT-SN were considerably lower than those of anesthesiologists. The range of discrepancies between residents and experienced anesthesiologists in each sequential measurement was wide for all measurements, particularly for the measurement values of STT-HY, and the standard deviation of the discrepancies did not decrease over the sequence of measurements.
Conclusion: Over the sequence of measurements for airway assessment in 10 volunteers by 20 residents in this learning trial, we found no evidence of improvement in measurement accuracy. Discrepancies between the residents’ and anesthesiologists’ measurements and the variability in discrepancy across residents were greatest in the measurement of STT-HY.
Keywords: ultrasound, ultrasonography, anesthesiology, airway management, learning curve
Complications in airway management are often unanticipated and can lead to disability or death. The most common complication is an unexpected encounter of a difficult airway, posing challenges in achieving adequate intubation and ventilation by anesthesiologists. Thus, accurate airway assessment is necessary for predicting difficulties in intubation and ventilation, as well as preparing the assistants and the equipment needed to secure the airway. The use of a correct technique for airway management can reduce the risk of severe complications.1–3
Airway can be evaluated by physical examination, including determining the Mallampati classification score (sensitivity, 43–49%), interincisor gap (sensitivity, 46%), thyromental distance (sensitivity, 20–25%), and sternothyroid distance (sensitivity, 62%), as well as performing the upper lip bite test (sensitivity, 7–19%) and flexion and extension of the neck (sensitivity, 10–16%).4,5 However, these modalities have a low sensitivity, which can lead to inaccurate prediction of difficulties in intubation and ventilation.
Ultrasonography is used in patients with trauma or critical illnesses for vascular assessment, diagnosis, regional anesthesia, and airway management. Ultrasonography for airway management can be performed in the pre-intubation phase to evaluate the pathology and measure the trachea width to estimate the tube size,6–8 intubation phase which detect endotracheal in situ faster than lung auscultation or end tidal CO2 monitoring,9,10 and extubation phase to measure diaphragm excusion and predict the extubation success.11
However, the technique of using ultrasonography for airway assessment is new, and only a few experts use ultrasonography for airway assessment in Thailand. Therefore, skill development in novice personnel pertaining to the use of ultrasonography for airway assessment needs to be investigated. We aimed to assess improvements in the validity and reliability of novices’ skills in performing ultrasonography for airway assessment.
Materials and Methods
This learning cohort study was conducted after receiving approval from the Human Research Ethics Committee, Faculty of Medicine, Prince of Songkla University (REC 61-420-8-4). Twenty anesthesiology residents with no experience in using ultrasonography for airway assessment were included in this study. Four ultrasonography parameters of airway assessment were analyzed in 10 volunteers, random number generation from all nurse assistants, by the 20 anesthesiology residents from one institute. This analysis was first performed by one of two anesthesiologists with experience in airway ultrasonography. The exclusion criteria for the volunteers were presence of a neck mass, history of radiation therapy or surgery, wound in the head and neck region, and/or infection in the neck region.
After obtaining informed consent from the residents and volunteers, their demographic data were recorded. A Philips Affiniti 50 (Philips, Amsterdam, the Netherlands) ultrasound machine with a linear probe (L12-5) was used in this study (Figure 1).
Figure 1 Philips Affiniti 50 ultrasound machine with linear probe (L12-5).
The two anesthesiologists who had the experience of airway ultrasound for 3 years performed ultrasonography three times for airway assessment in each volunteer. These values were averaged to obtain the “gold standard” measurement values for each parameter and standardized by interpersonal validity.
Before the residents performed any measurements, the method of performing ultrasonography for airway assessment was explained, including the assessment of the following four parameters: 1) soft tissue thickness at the level of hyoid bone (STT-HY, distance between the skin and the upper border of hyoid bone; Figure 2); 2) soft tissue thickness at the level of true vocal cords (STT-VC, distance between the skin and the anterior commissure of true vocal cords; Figure 3); 3) soft tissue thickness at the level of thyroid isthmus (STT-TI, distance between the skin and the anterior surface of tracheal cartilage at the level of thyroid isthmus; Figure 4); and 4) soft tissue thickness at the level of the suprasternal notch (STT-SN, distance between the skin and the anterior surface of tracheal cartilage at the level of the suprasternal notch; Figure 5). The method of performing ultrasonography for airway assessment was explained via Supplementary Video before the time of first measurement by each resident. Measurements for each parameter were recorded thrice in millimeters, and the mean was calculated and recorded.
Figure 2 STT-HY, distance between skin and upper border of hyoid bone.
Figure 3 STT-VC, distance between skin and anterior commissure of true vocal cord.
Figure 4 STT-TI, distance between skin and anterior surface of tracheal cartilage at the level of thyroid isthmus.
Figure 5 STT-SN, distance between skin and anterior surface of tracheal cartilage at the level of suprasternal notch.
All 20 residents performed ultrasonography for airway assessment in all 10 volunteers, but in a different order. After the first measurements were performed by each resident, one of two experienced anesthesiologists provided feedback to the residents regarding any errors in the technique, though the discrepancy from the “gold standard” value was not disclosed. All subsequent measurements by the residents on the same and subsequent volunteers were performed on their own, without any feedback by any experienced anesthesiologist. Three measurements were performed for each parameter by each resident on the same day in a particular volunteer. The average values were used to calculate the discrepancy from the “gold standard” value for that parameter in that volunteer.
Statistical analyses were performed using RStudio (version 2.13.0; R Foundation for Statistical Computing, Vienna, Austria) and Stata (version 14.2; StataCorp, College Station, Texas, USA). The magnitude of discrepancies between the measurements performed by residents and anesthesiologists was calculated over the sequence of measurements in the 10 volunteers. Change in discrepancies over the sequence of measurements was evaluated graphically and using mixed-effects random-intercept linear regression models, in which the residents were considered as random elements and the interaction between discrepancy and sequence was considered a fixed effect.
The required sample size was estimated to achieve a power of 80% in detecting a significant difference in the size of the standard deviation (SD) of discrepancies between the first and last measurements in the sequence of measurements at an alpha of 0.05. The sample size for the test of the difference between the two variances was used.
Consequently, a study population of 20 residents was estimated to provide an adequate sample size.
The mean age of the residents was 27.4 ± 2.1 (mean ± SD) years. Of the 20 residents, 4 were male, and 8 were first-year, 6 were second-year, and 6 were third-year residents (Table 1).
Table 1 Demographic Characteristics of 20 Residents
The mean age of the volunteers was 30.9 ± 3.9 years, and the mean height, body weight, and body mass index were 160.3 ± 4.8 cm, 62 ± 15.1 kg, and 24 ± 4.8 kg/m2, respectively (Table 2). The mean ultrasonic measurements of STT-HY by the experienced anesthesiologists and residents were significantly different (11.09 ± 3.14 mm vs 8.53 ± 3.02 mm, respectively; P = 0.008), whereas those of STT-VC, STT-TI, and STT-SN were not (7.18 ± 1.70 vs 7.14 ± 1.93 mm, P = 0.32; 7.81 ± 2.14 vs 7.73 ± 2.19 mm, P = 0.62; and 11.32 ± 3.33 vs 10.30 ± 3.02 mm, P = 0.35, respectively; Table 3).
Table 2 Demographic Characteristic of 10 Volunteers
Table 3 Parameters Performing by Experienced Anesthesiologists and Residents (mean±SD)
The mean discrepancy between measurements performed by residents and experienced anesthesiologists was close to zero throughout the sequence of measurements of STT-TI and STT-VC. However, the measurement values of STT-HY and STT-SN were considerably lower when performed by residents than when performed by experienced anesthesiologists throughout the sequence of measurements in the 10 volunteers. The range of discrepancies between measurements performed by residents and experienced anesthesiologists for each sequential measurement was wide for all measurements, particularly for the measurement of STT-HY, and the SD of discrepancies did not decrease over the sequence of measurements (Figure 6). Regression models including sequences revealed no significant decrease in the discrepancy in any parameter over the sequence of measurements.
Figure 6 Mean discrepancy and standard deviation (SD) value of each parameter of performing airway ultrasonography by twenty residents compared with two experienced anesthesiologists in ten volunteers over the sequences.
Accurate airway assessment is essential to predict difficulties in intubation and ventilation. We evaluated the sensitivity of parameters used in this study and compared it with that of parameters such as Mallampati classification (sensitivity, 43–49%), interincisor gap (sensitivity, 46%), thyromental distance (sensitivity, 20–25%), sternomental distance (sensitivity, 62%), upper lip bite test (sensitivity, 7–19%), and flexion and extension of neck (sensitivity, 10–16%).4,5 The sensitivity of these parameters is low. Therefore, we need tools with a higher sensitivity for promptly securing the airway when a difficult airway is predicted.
Ultrasonography has played a key role in many investigations and procedures. Airway ultrasonography is non-invasive, easy to perform, and useful in preparing for the management of difficult airways. Ultrasonography for airway management can be performed in the pre-intubation phase (airway assessment and prediction of tube size),6–8 intubation phase (ultrasound-guided intubation and confirmation of tube placement),9,10 and extubation phase (estimation of diaphragmatic function and examination of lung pathology).11 In this study, we focused on performing ultrasonography for airway assessment in the pre-intubation phase.
We hypothesized that performing airway ultrasonography in at least 10 cases would improve the physician’s skill because the technique is not difficult. We designed a study to compare the SD of measurement values for each parameter determined by each resident with those determined by experienced anesthesiologists. Computed tomography (CT) is the gold standard for airway imaging.12 Sustic et al observed a strong correlation between tracheal width and left main bronchus width with ultrasonography and not CT.13 However, Prasad et al14 found that the parameters of hyomental distance, depth of the epiglottis below the hyoid, thyrohyoid distance, and depth of the arytenoid cartilage did not differ significantly between sonographic and CT measurements.
Healthy volunteers exposed to radiation while undergoing CT of the head and neck region are at an increased risk of cataract and malignant tumor development in radiosensitive organs such as the salivary gland, thyroid gland, and retina.12 It would have been unethical to expose volunteers to unnecessary radiation for this study. Thus, we designed this study to compare the learning curve of novices with that of experienced anesthesiologists to avoid the possible adverse effects of radiation exposure by CT.
Although CT is the first-choice imaging modality for the larynx and hypopharynx, the values of some parameters measured by CT are significantly different from those measured by sonography, such as the distance to the posterior surface of the tongue, thickness of the submental region, depth of the epiglottis from the skin, and fat pad thickness at the level of the thyroid cartilage.14
Over the sequence of measurements for airway assessment using ultrasonography in 10 volunteers by 20 residents in this learning trial, we did not find any evidence of increasing validity or reliability compared to that of experienced anesthesiologists. A possible explanation could be that the residents were not given adequate feedback after each measurement. No feedback was provided regarding the magnitude of the discrepancy in measurements. Experienced anesthesiologists provided some feedback about the technique for identifying the structure of the airway for the first measurement on the first day, but not for subsequent measurements. Consequently, the residents never understood whether their measured values were more or less than those of experienced anesthesiologists. Another explanation could be that 10 cases were not adequate to observe the improvement in validity or reliability.
STT-HY measurements can accurately indicate the possible difficulties in laryngoscopy (1.69 cm, 95% confidence interval [CI] = 1.19–2.19 in the difficult laryngoscopy group vs 1.37 cm, 95% CI = 1.27–1.46 in the easy laryngoscopy group).15 STT-HY measurements can also indicate the possible difficulties in mask ventilation (0.88 ± 0.3 cm for easy mask ventilation vs 1.4 ± 0.019 cm for difficult mask ventilation).16 STT-VC > 23 mm has been associated with difficulties in intubation (sensitivity, 85.7%; specificity, 57%; and area under the curve, 0.73).7 STT-TI ranged from 0.78 ± 0.35 cm to 1.08 ± 0.33 cm but could not accurately predict difficulties in mask ventilation and laryngoscopy.16 STT-SN has been used to verify the placement of the tube at the correct depth after endotracheal intubation.17 STT-VC, STT-TI, and STT-SN were found to be simple measurements for novices. The measurement values recorded by residents were similar to those recorded by experienced anesthesiologists.
The mean and SD of measurement values for each parameter determined by the residents over the sequence of measurements, as compared with those determined by experienced anesthesiologists, showed that the range of discrepancies was wide and did not change over the sequence. The residents’ measurements of STT-HY were significantly lower than those of experienced anesthesiologists. However, the values of STT-VC, STT-TI, and STT-SN were not significantly different over the sequence between the measurements recorded by residents and experienced anesthesiologists. This result can be explained by the compression due to the ultrasound probe. In volunteers with a greater thickness of soft tissue, the ultrasound probe could have led to greater compression. As the residents had no experience in performing ultrasonography for airway assessment, they may have used the ultrasound probe more aggressively than experienced anesthesiologists.
This study has some limitations. First, the residents were not given adequate feedback. If they had received qualitative and adequate feedback for each measurement, the discrepancy in the measurement values between residents and experienced anesthesiologists may have improved in each sequence. Second, the small number of volunteers did not permit any analysis of volunteer characteristics that may have affected the accuracy of the residents’ measurements. Third, the mean of measurements performed by experienced anesthesiologists using ultrasonography was considered the gold standard. Ultrasonography is an operator-dependent procedure; thus, values measured by anesthesiologists may have been inaccurate. However, both anesthesiologists were validated in the use of ultrasonography for airway assessment using the parameters used in this study. In future studies, values obtained by other imaging modalities, such as CT and magnetic resonance imaging, may serve as a more suitable gold standard.
Over the sequence of airway assessment in 10 volunteers by 20 residents in this learning trial, we did not find any improvement in measurement accuracy. Discrepancies between the residents’ and anesthesiologists’ measurements and the variability in discrepancy across residents were greatest in the measurements of STT-HY.
STT-HY, soft tissue thickness at the level of hyoid bone; STT-VC, soft tissue thickness at the level of true vocal cords; STT-TI, soft tissue thickness at the level of thyroid isthmus; STT-VN, soft tissue thickness at the level of suprasternal notch; SD, standard deviation; CT, computed tomography.
We would like to thank and express our sincere gratitude to Dr. Alan Geater, Epidemiology unit, Faculty of Medicine, Prince of Songkla University, for providing advice regarding statistical analysis. This study was funded by our institution.
The authors report no conflicts of interest in this work.
1. Cook TM, MacDougall-Davis SR. Complications and failure of airway management. Br J Anaesth. 2012;109(suppl 1):i68–i85. doi:10.1093/bja/aes393
2. Frerk C, Mitchell VS, McNarry AF, et al. Difficult airway society 2015 guidelines for management of unanticipated difficult intubation in adults. Br J Anaesth. 2015;115(6):827–848. doi:10.1093/bja/aev371
3. Apfelbaum JL, Hagberg CA, Caplan RA, et al. Practice guidelines for management of the difficult airway: an updated report by the American Society of Anesthesiologists task force on management of the difficult airway. Anesthesiology. 2013;118(2):251–270. doi:10.1097/ALN.0b013e31827773b2
4. Selvi O, Kahraman T, Senturk O, Tulgar S, Serifsoy E, Ozer Z. Evaluation of the reliability of preoperative descriptive airway assessment tests in prediction of the Cormack-Lehane score: a prospective randomized clinical study. J Clin Anesth. 2017;36:21–26. doi:10.1016/j.jclinane.2016.08.006
5. El-Ganzouri AR, McCarthy RJ, Tuman KJ, Tanck EN, Ivankovich AD. Preoperative airway assessment: predictive value of a multivariate risk index. Anesth Analg. 1996;82(6):1197–1204. doi:10.1097/00000539-199606000-00017
6. Pourmand A, Lee D, Davis S, Dorwart K, Shokoohi H. Point-of-care ultrasound utilizations in the emergency airway management: an evidence-based review. Am J Emerg Med. 2017;35(8):1202–1206. doi:10.1016/j.ajem.2017.02.032
7. Reddy PB, Punetha P, Chalam KS. Ultrasonography – a viable tool for airway assessment. Indian J Anaesth. 2016;60(11):807–813. doi:10.4103/0019-5049.193660
8. Baker P. Assessment before airway management. Anesthesiol Clin. 2015;33(2):257–278. doi:10.1016/j.anclin.2015.02.001
9. Ramsingh D, Frank E, Haughton R, et al. Auscultation versus point-of-care ultrasound to determine endotracheal versus bronchial intubation: a diagnostic accuracy study. Anesthesiology. 2016;124(5):1012–1020. doi:10.1097/ALN.0000000000001073
10. Das SK, Choupoo NS, Haldar R, Lahkar A. Transtracheal ultrasound for verification of endotracheal tube placement: a systematic review and meta-analysis. Can J Anaesth. 2015;62(4):413–423. doi:10.1007/s12630-014-0301-z
11. Farghaly S, Hasan AA. Diaphragm ultrasound as a new method to predict extubation outcome in mechanically ventilated patients. Aust Crit Care. 2017;30(1):37–43. doi:10.1016/j.aucc.2016.03.004
12. Yabuuchi H, Kamitani T, Sagiyama K, et al. Clinical application of radiation dose reduction for head and neck CT. Eur J Radiol. 2018;107:209–215. doi:10.1016/j.ejrad.2018.08.021
13. Sustić A, Miletić D, Protić A, Ivancić A, Cicvarić T. Can ultrasound be useful for predicting the size of a left double-lumen bronchial tube? Tracheal width as measured by ultrasonography versus computed tomography. J Clin Anesth. 2008;20(4):247–252. doi:10.1016/j.jclinane.2007.11.002
14. Prasad A, Yu E, Wong DT, Karkhanis R, Gullane P, Chan VWS. Comparison of sonography and computed tomography as imaging tools for assessment of airway structures. J Ultrasound Med. 2011;30(7):965–972. doi:10.7863/jum.2011.30.7.965
15. Adhikari S, Zeger W, Schmier C, et al. Pilot study to determine the utility of point-of-care ultrasound in the assessment of difficult laryngoscopy. Acad Emerg Med. 2011;18(7):754–758. doi:10.1111/j.1553-2712.2011.01099.x
16. Alessandri F, Antenucci G, Piervincenzi E, et al. Ultrasound as a new tool in the assessment of airway difficulties: an observational study. Eur J Anaesthesiol. 2019;36(7):509–515. doi:10.1097/EJA.0000000000000989
17. Abbasi S, Farsi D, Zare MA, Hajimohammadi M, Rezai M, Hafezimoghadam P. Direct ultrasound methods: a confirmatory technique for proper endotracheal intubation in the emergency department. Eur J Emerg Med. 2015;22(1):10–16. doi:10.1097/MEJ.0000000000000108
© 2022 The Author(s). This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.