Difficult Laryngoscopy Prediction Score for Intubation in Emergency Departments: A Retrospective Cohort Study

Background

Endotracheal intubation remains the gold standard for emergency airway management, providing oxygenation, ventilation, and airway protection from aspiration in acutely ill patients. Neuromuscular blocking agents are often required to encourage optimal visualization of glottic structures and improved intubation success. However, apnea caused by intubation failure may result in oxygen desaturation, increasing morbidity and mortality.^1–3 Therefore, predicting a difficult airway is essential to assist with risk-benefit analysis and establish the optimal approach to airway management.

In the emergency department (ED), emergency physicians (EPs) are responsible for managing airway problems. Emergency intubation is more complicated than elective intubation in the operating room setting because the clinical condition of acutely ill patients in the ED is unpredictable and the setting is less controlled.^4–6 The proportion of difficult intubation ranges from 10% to 27% in the ED setting^7–11 but from only 1% to 9% in the anesthesia setting.^12,13 The incidence of difficult intubation increases with the number of intubation attempts.¹⁴ Repeated intubation attempts increase the risk of adverse events such as cardiac arrest, arrhythmia, regurgitation, and airway trauma.^14–16 Early recognition of difficult intubation is helpful for early detection and preparation for further management.

Many large multicenter studies have provided data on emergency airway management in the ED.^8,9,14,17 One study demonstrated that predicted airway difficulty was the main factor associated with successful intubation on the first attempt.¹⁷ The QLD Swapnet group has developed a document with peer-reviewed and accepted definitions of difficult intubation where intubation requires multiple attempts or additional equipment.¹⁸ Although various difficult airway prediction tools have been proposed and evaluated, each has limitations in its sensitivity and specificity.^8,17,19–21 Difficult airway assessment tools such as “LEMON” (Look externally, Evaluate the 3-3-2 rule, Mallampati score, Obstruction, Neck mobility) combine intuitive elements (eg, “look externally”) with assessments developed in the operative setting that requires the patient to be awake and cooperative.⁸

The main objective of the present study was to develop and validate a model for predicting difficult laryngoscopy in patients undergoing emergency tracheal intubation in the ED. Clinical factors associated with difficult laryngoscopy were evaluated, and the scores used to predict difficult laryngoscopy by simple physical evaluation were examined. Such a Difficult Laryngoscopy Score is expected to benefit the community or hospitals where no EP specialists to prepare for difficult laryngoscopy.

Methods

Study Design

This was a diagnostic, retrospective cohort study using an exploratory model. It was conducted at the ED of Ramathibodi Hospital, a university-affiliated super-tertiary care hospital in Bangkok, Thailand. The data were collected from the Ramathibodi Hospital database and emergency medical record system from June 2018 to July 2020.

Sample Size

The data were collected from June 2018 to July 2020 and analyzed by STATA version 16.0 (StataCorp, College Station, TX, USA). A pilot study calculated the sample size by employing a two-sample comparison of difficult laryngoscopy (Cormack–Lehane grade III or IV) and non-difficult laryngoscopy (Cormack–Lehane grade I or II). We used the probability of the expected value (p) = 0.12 and a two-sided test, and the minimum sample size was determined to be 281 patients.

Participants

Patients were required to be >15 years of age and undergo emergency intubation in the ED. We excluded patients who had been intubated by a non-EP (general practitioner) and those who had developed cardiac arrest. 281 patients who underwent emergency intubation in the ED were enrolled in this study from the minimum sample. The study period was 2 years (from June 2018 to July 2020).

Definition of Difficult Laryngoscopy (Reference Standard)

The definition of difficult laryngoscopy is according to the structures visualized and identified by laryngoscopy. Using the 4-grade classification of the laryngoscopic view defined by Cormack and Lehane,²² intubation was defined as easy (grade I or II) or difficult (grade III or IV). The laryngoscopic view was assessed and graded according to the final intubation after each intubation was finished.

Definitions of Predictors

Intubation Method

An “intubation method” was defined as one set of medication or devices, such as rapid-sequence intubation with direct laryngoscopy.

Intubation Attempt

An “intubation attempt” was defined as one effort to place an airway. Each attempt could be performed using one or more methods, and each method could have one or more attempts. After each intubation was finished, the clinician entered all data in the medical record form.

Operator Level

We classified the operator’s level of training into three groups: low experience (general practitioners), moderate experience (first-year residents in emergency medicine and first-year residents in general medicine), and high experience (second-to third-year residents in emergency medicine and emergency attending staff).

Failed Intubation Attempt

We defined a “failed intubation attempt” as multiple efforts to place an airway (more than one effort).

Indicators of Difficult Laryngoscopy

We used the difficult intubation assessment tool “LEMON” and difficult ventilation assessment tool “MOANS” (Mask seal, Obesity, Age (elderly), No teeth, Stiffness) to evaluate the difficult laryngoscopy undergoing endotracheal intubation.^8,18

For the “Look externally” criterion of “LEMON,” we assessed any significant facial injury, large incisors, significant beard or mustache, and large tongue. For the “Evaluate the 3-3-2 rule” criterion, we assessed a mouth opening of less than three finger breadths, a hyomental distance (the distance between the hyoid bone and the tip of the chin) of less than three finger breadths, and a thyrohyoid distance (the distance from the chin to the top of the notch of the thyroid cartilage) of less than two finger breadths. For the “Obstruction” criterion, we record all conditions that may make the laryngoscopic view more difficult, such as a mass, hematoma, or massive bleeding. We assessed limited neck mobility for the “Neck mobility” criterion, such as collar immobilization.

Concerning the “Mallampati score” criterion of the “LEMON” assessment tool, we did not record the Mallampati score because it requires patient cooperation limited in critically ill patients in the ED.

Data Collection and Study Variables

We collected data regarding the patients’ characteristics, including sex, age, Glasgow coma scale score, the main indication for intubation, initial method of intubation, the dosage of any medications used, operator level of training, number of attempts, success or failure, and structures identified by direct and indirect laryngoscopy. The patients were divided into the difficult laryngoscopy group and the non-difficult laryngoscopy group. We then used a multivariable regression model to identify significant predictors of difficult laryngoscopy. The data were collected from June 2018 to July 2020, and after excluding patients who did not meet the study criteria, the total sample size was 617 patients. The protocol is illustrated in Figure 1.

Figure 1 Data collection protocol.

Outcome of Interest

The outcome of interest was a positive difficult anatomical laryngoscopic view according to the final intubation after each set of intubation attempts was finished. Patients in the difficult laryngoscopy group had a Cormack–Lehane grade III or IV laryngoscopic view, and those in the non-difficult laryngoscopy group had a Cormack–Lehane grade I or II laryngoscopic view. Finally, we used the patients’ data to develop a prediction score to predict difficult laryngoscopy in the ED.

Statistical Analysis

The data were analyzed using STATA version 16.0 (StataCorp). Descriptive statistics compared all variables between the difficult laryngoscopy and non-difficult laryngoscopy groups. The predictive power of each variable for a positive difficult laryngoscopic view was calculated using univariable logistic regression and is presented as the area under the receiver operating characteristic curve (AuROC) with the 95% confidence interval (CI). A multivariable stepwise backward logistic regression model was used to develop the predictive model. Predictors with a cut-off P-value of 0.05 after the univariable analysis were included in the model and eliminated with a significance level of 0.001. Regression coefficients for each level of each clinical predictor were divided by the smallest coefficient of the model and rounded to the nearest 0.5, resulting in an item risk score. According to this score, the coefficients were then changed into item scores and added together into a single score, and the patients were thus classified into low-, moderate-, and high-probability categories. Discrimination of the airway assessment scores is presented as the AuROC and 95% CI for the clinical risk score of difficult laryngoscopy. Calibration of the prediction was performed using the Hosmer–Lemeshow goodness-of-fit test. The score-predicted risk of difficult laryngoscopy and the observed risk was then compared in a graph. The number of reports and percentages of each group are presented with the positive likelihood ratio, 95% CIs, and P-values.

Results

A total of 790 patients underwent emergency endotracheal intubation in the ED of Ramathibodi Hospital. We excluded patients intubated by general practitioners (n = 80), patients who developed cardiac arrest (n = 75), and patients with incomplete data (n = 18). The remaining 617 intubations were included in the analysis.

The patients’ overall baseline characteristics are presented in Table 1. Their mean age was 68.66 years, and 56.56% were male. The proportion of men was significantly higher in the difficult laryngoscopy group than in the non-difficult laryngoscopy group (63.86% vs 55.43%, respectively; P = 0.155). The proportion of patients with trauma as the reason for intubation was higher in the difficult laryngoscopy group than in the non-difficult laryngoscopy group (9.64% vs 5.06%, respectively; P = 0.120). The most common main indication for intubation was the failure of oxygenation and ventilation, which nearly range in both groups. Concerning the method of intubation, the number of patients who underwent rapid-sequence intubation with a neuromuscular blockade agent before intubation was significantly higher in the non-difficult laryngoscopy group than in the difficult laryngoscopy group (56.10% vs 18.10%, respectively; P = 0.001). The difficult laryngoscopy group had a significantly higher number of intubations attempts than the non-difficult laryngoscopy group (mean number of attempts: 2 vs 1, respectively; P < 0.001) and a higher of failed intubation attempts (66.27% vs 9.93%).

Table 1 Clinical Characteristics of Patients with Difficult and Non-Difficult Laryngoscopy, and Discrimination Power (AuROC) Under Univariable Analysis

Table 2 shows the results of the multivariable stepwise backward logistic regression analysis, including variables associated with the prediction of difficult laryngoscopy. The variables from Table 1 with a P-value of <0.05 were included in the initial model analysis. Multivariable logistic regression analysis confirmed that each of these was predictive of difficult laryngoscopy, limited mouth opening with an adjusted OR of 1.47 (95% CI: 0.36, 6.03), presence of obstructed airway with an adjusted OR of 5.16 (95% CI: 2.83, 9.46), poor neck mobility with an adjusted OR of 3.59 (95% CI: 1.48, 8.66), large tongue with an adjusted OR of 1.97 (95% CI: 0.86, 4.51), and short hyomental distance with an adjusted OR of 6.19 (95% CI: 1.52, 25.19).

Table 2 Best Multiple Clinically Significant Predictors of Difficult and Non-Difficult Laryngoscopy

The final model contained the significant predictors of difficult laryngoscopy with a P-value of <0.001. Each of the significant predictors was assigned a score based on its beta coefficient value, including limited mouth opening (1 point), a large tongue (2 points), poor neck mobility (3 points), an obstructed airway (4 points), and a short hyomental distance (5 points). The difficult laryngoscopy score model with a score ranging from 0 to 15 was then created and applied to all patients.

The AuROC was 89% (95% CI, 0.860–0.926) for the ability of the clinical risk score to predict difficult laryngoscopy (Figure 2). The data distribution measures presented in Figure 3 show the distribution plot of the clinical risk score for difficult laryngoscopy (n = 83) and non-difficult laryngoscopy (n = 534). The measures of calibration presented in Figure 4 show the observed risk (circle) and score-predicted risk (solid line) for difficult laryngoscopy. The score-predicted risk increased in close association with the observed risk. Finally, the risk scores were categorized into three groups: scores of <0 (low risk), the score of 1 to 4 (moderate risk), and the score of 5 to 15 (high risk). The positive likelihood ratio of difficult laryngoscopy in the high-risk group was 7.62. The risk score was named the MONTH Difficult Laryngoscopy Score. M: limited mouth opening, O: presence of obstructed airway, N: poor neck mobility, T: large tongue, and H: short hypo-mental distance.

Figure 2 Area under receiver operating characteristic curve (AuROC) of clinical risk score and 95% confidence interval (CI) for prediction of difficult laryngoscopy.

Figure 3 Distribution plot of clinical risk score for difficult laryngoscopy (n = 83) versus non-difficult laryngoscopy (n = 534).

Figure 4 Observed risk (circle) versus score (solid line) of difficult laryngoscopy prediction.

Discussion

As confirmed by analysis of anatomical laryngoscopic views, predicting difficult laryngoscopy in the emergency setting is clinically challenging. Underprediction can delay further management of patients and preparation of alternative airway equipment, and repeated intubation attempts are associated with an increased risk of adverse events such as cardiac arrest, arrhythmia, regurgitation, and airway trauma.^14–16

Management of difficult laryngoscopy in the ED has been less thoroughly studied than that in the operating room. Several difficult laryngoscopy prediction scores have been less reported in Asian countries for determining the risk of difficult laryngoscopy in the ED setting. Several small studies in Thailand focused on the patients’ characteristics, intubation methods, and success rates on the first attempt.^19,23,24 However, they had limited information regarding predictors of multiple attempts at intubation of patients with difficult airways in the ED setting.

One study showed that the proportion of difficult intubations in the ED setting was 5.4% to 27%,^6–8 higher than the incidence of elective intubation in the operating room setting (0.3%–13.3%).^25,26 Our study showed that the rate of difficult laryngoscopy was 13.45%, which is similar to that of the prior study. Existing difficult airway prediction tools such as LEMON have been proposed and evaluated, but each has limited sensitivity and specificity (23.3% and 91.3%, respectively, for LEMON).¹⁷ Furthermore, some assessment predictors were collected from studies in the operative setting. Most of the patients in these studies were awake and cooperative, and they were examined for preoperative preparation by anesthesiologists (in contrast to examination by EPs in the ED setting). The validation of the prior score (obtained by modified LEMON criteria to predict difficult intubation in the ED) showed high sensitivity and negative predictive value (85.7% and 98.2%, respectively) but a poor specificity and positive predictive value (47.6% and 8.9%, respectively).⁶ Conversely, our study demonstrated poor sensitivity (57.8%) but high specificity (92.7%). Previous combinations of multivariable risk scores, such as the Naguib score and intubation difficulty scale^27,28 have been presented and assessed in the operative setting to improve the ability of bedside testing. However, many predictors cannot be used to evaluate critically ill patients in the ED, and these scores also tend to be time-consuming and challenging to apply in clinical practice.

Our simple model, MONTH Difficult Laryngoscopy Score, effectively predicted difficult laryngoscopy in our study population (AuROC, 0.893; 95% CI, 0.860–0.926). We identified five predictors of difficult laryngoscopy similar to those found in previous studies:^5,8,19 a short hyomental distance, an obstructed airway, poor neck mobility, a large tongue, and limited mouth opening (Table 2). Our findings showed that the weighted combination of these five independent predictors could help discriminate between patients with and without difficult laryngoscopy among adults undergoing intubation by either conventional direct laryngoscopy or indirect laryngoscopy in the ED.

Our score is also a more user-friendly probability score that can be used without the patient’s cooperation. Difficult laryngoscopy was found in patients with suspected difficult airway anatomy who had moderate scores (1–4) or high scores (5–15) (Table 3). A MONTH Difficult Laryngoscopy Score of >5 points indicates a risk of difficult laryngoscopy, and clinicians should immediately consult a specialist for appropriate alternative management. If the score is >1 point, difficult laryngoscopy should be anticipated, and a moderately or more highly experienced intubator should be consulted. This score also provides sufficient justification to allow intubators with low experience (medical students in their last year of training) to encounter difficult laryngoscopy conditions.

Table 3 Score Categorized Probability Groups, Likelihood Ratio and 95% Confidence Interval of Difficult Laryngoscopy

We identified five predictors of difficult laryngoscopy similar to those found in the previous studies:¹⁹ a short hyomental distance, an obstructed airway, poor neck mobility, a large tongue, and limited mouth opening (Table 2). However, unlike in other reports, male sex was not a significant predictor of difficult laryngoscopy. The lack of statistical significance for the male sex may be explained by the unequal ratio of men and women between the difficult laryngoscopy and non-difficult laryngoscopy groups. The differences in significant predictors between previous studies and our study may be explained by this information not being recorded because of the retrospective data collection.

We found that some data differed from our objective regarding the observed risk versus difficult laryngoscopy prediction score (shown in Figure 4). One patient had a high MONTH Difficult Laryngoscopy Score (13 points), but this patient had been recorded as having a Cormack–Lehane grades I laryngoscopic view (non-difficult laryngoscopy). We could explain this by the fact that different individuals performed the evaluations. This problem could be solved by using a more accurate model, the Percentage of Glottic Opening scale,²⁹ to evaluate the anatomical laryngoscopic view.

We found that an average of 82% of the intubated patients in this study were intubated by using direct laryngoscopy (intubating protocol before the covid-19 pandemic from June 2018 to December 2020). After the covid-19 pandemic (after Jan 2020), the intubating protocol was changed to video laryngoscopy. There are no selection biases in this study in choosing the method of intubation due to the different protocols.

This study has a few limitations. First, this was a retrospective study. The patients’ characteristics and clinical indicators were only taken from their medical records, and patients with incomplete data had to be excluded, limiting the study’s accuracy. However, we solved this problem by increasing the capture rate of data collection (97.74%). Another limitation is that we collected the data from the medical record form completed by the intubator after finishing each intubation, and observer bias might have occurred if the intubation was incorrectly assessed. The patient’s indicators of difficult laryngoscopy were inaccurate. We minimized this bias by administering patient selection in the exclusion criteria process. We excluded patients who were individuals with low experience intubated. Finally, this study only included adult patients whose anatomical appearances differ from children. Therefore, the results of this study cannot be applied to children.

In conclusion, the MONTH Difficult Laryngoscopy Score was associated with a more difficult laryngoscopy and a decrease in intubation success as defined by endotracheal intubation on the first attempt. These associations persisted despite adjustment for multiple difficult laryngoscopy covariables. These data suggest the potential utility of the MONTH Difficult Laryngoscopy Score as a tool for difficult laryngoscopy prediction in critically ill patients in the ED setting. It can classify patients into low-, intermediate-and high-risk subgroups, which will help clinicians improve decision-making on appropriate preparation and alternative management, select the most effective method for ED intubation, and rescue the emergency airway decision.