Development and Validation of a Risk Nomogram Model for Perioperative Respiratory Adverse Events in Children Undergoing Airway Surgery: An Observational Prospective Cohort Study

Introduction

Perioperative respiratory adverse events (PRAEs) in patients undergoing airway surgery cover a wide range of clinical manifestations. The main respiratory adverse events included laryngospasm, bronchospasm, and pulse oximetry value (SpO₂) below 95%, prolonged cough, airway obstruction, and stridor,^1–4 that are associated with high morbidity and mortality rates, and the most common adverse events in the perioperative period.⁵ The incidence of PRAEs was 15% in a general pediatric population, however, a substantial proportion of children undergoing tonsillectomies experience PRAEs, with a prevalence of up to 50% in children.⁶ Although PRAEs can be easily detected and treated, they still progress to life-threatening complications. Therefore, a simple, easy, and rapid prognostic model would have a huge clinical impact on the prognoses of patients with PRAEs.

Common risk factors for PRAEs include age, upper respiratory tract infection, history of passive smoking, eczema, obesity, anesthetist experience, premedication, stuffy nose, coughing, runny nose, allergy, anesthesia method, ASA status, previous lung disease, and airway devices (facemask, laryngeal mask or endotracheal tube).^1,7–10 Fewer prognostic models based on clinical and procedural variables have yet been established to predict the outcomes of PRAEs in China. A quantitative metric currently used in clinical practice for predicting PRAEs was the COLDS scale by Lee⁸ in 2014. The name of the scoring system is an acronym of risk factors for perioperative respiratory adverse events that comprise the COLDS score: (C stands for current signs and symptoms; O stands for the onset of symptoms; L stands for lung disease; D stands for the device to be used for airway management; and S stands for surgery type (whether it involves the airway or not, then further subclassified to major and minor airway). However, COLDS score is a heuristic decision tool rather than a statistically validated risk predictor, which assesses the current signs and symptoms, onset of symptoms, presence of lung disease, airway devices, and surgery, other known risk factors are not included in it (age, obesity, and anesthetist experience), so it is a possibility for improving on this scale. Other more studied models are not convenient for wide clinical application. However, to date, a nomogram model with adequacy to detect the probability of intrahospital morbidity in PRAEs pediatric patients has not yet been developed. PRAEs in pediatric anesthesia can lead to prolonged hospitalization, higher excess hospital costs and risk of death, and varying degrees of postoperative quality of life and recovery.¹¹ Therefore, accurate prediction of PRAEs is essential for preprocedural informed consent and appropriate therapy selection. The aim of the present study was to investigate the risk factors of PRAEs and develop a prediction model for the PRAEs in patients undergoing airway surgery.

Materials and Methods

This is a single-center, prospective observational cohort study, which has been approved by the Research Ethics Committee of the Affiliated Hospital of Xuzhou Medical University (Xuzhou city, Jiangsu, province, China) (XYFY2021-KL018-01), and written informed consent was obtained from all subjects participating in the trial before the operation (Written parental or guardian consent was obtained before enrolment). The trial was registered in the Chinese Clinical Trials Registry (ChiCTR2000038100). From November 2020 to July 2021, data on consecutive patients who underwent non-cardiac surgery were prospectively collected at the Affiliated Hospital of Xuzhou Medical University.

Pediatric patients who met the following inclusion criteria were included in the study: (1) Chinese-speaking patients aged ≤18 years, scheduled for elective airway surgical under general anesthesia. The specific types of airway procedures performed in this study were shown in Table 1; (2) Physical status of American Society of Anesthesiologists (ASA) I–II. Exclusion criteria were: (1) Known cardiopulmonary diseases (uncorrected congenital heart disease, primary or secondary pulmonary hypertension, tumors, structural lung diseases); (2) Neuromuscular diseases; (3) parents with severe hearing or a visual impairment, unable to communicate effectively with the physician; (4) unable to understand the content of the scale and refusing to participate. Elimination criteria were: (1) those who could not continue to participate in the trial due to objective factors during the follow-up; (2) those who were lost to follow-up. As the data were susceptible to incorrect notation by the researcher; data cleansing and editing that consisted of removing typographical errors and reviewing data integrity/quality in data reporting were performed by a second researcher to avoid a flawed model training process.

Table 1 Demographics and Clinical Characteristics of the Training and Validation Cohorts

Data Collection

The study populations were identified by a study team member from the elective surgery list based on inclusion criteria. A homemade case report form (CRF) was used to collect demographic and clinical data. The medical records were obtained from the parent or caregiver of every enrolled child in the study and assessed by a research assistant from the interview with parents or guardians immediately after enrolment. Parents/guardians were provided with a detailed explanation of what the trial and their participation entailed. Finally, written consent was provided by the parents/guardians and child assent was sought when applicable. Potential predictive factors were filtrated based on the literature and clinical practice. Preoperative variables included demographics [sex, age, body mass index (BMI), ASA status, history of diseases, history of medication, history of surgery, family history, allergy, abnormal examination or test result], preoperative hospital stay. Intraoperative variables included anesthetist experience (attending physician or chief physician), anesthesia method, operation method, operation time, anesthesia time, preoperative medication; respiratory tract symptoms and the time of their appearance, anesthetist experience, perioperative vital signs (blood pressure, pulse rate, oxygen saturation); fluid intake and output, other perioperative medication types and doses (including perioperative sedation). Postoperative variables included the duration stayed in PACU, postoperative hospital stay. Due to the specific nature of the procedures, we have chosen tracheal intubation for all of them. The anesthetic protocol for general anesthesia was not standardized. However, any decision to reschedule patients was the result of a discussion between the anesthetist, the operator, and the family.

Diagnostic Criteria for PRAEs

Perioperative respiratory adverse events included pulse oximetry value (SpO₂) below 95%, laryngospasm, bronchospasm, prolonged cough, airway obstruction, and stridor.^1–3 Table 2 provided the types and definitions of PRAEs.² PRAEs in participants were recorded by the attending anesthetist during induction, maintenance, and the emergence of anesthesia, and by specialized nurses during recovery in the post-anesthesia care unit.

Table 2 Definition Used for Respiratory Complications Recorded

Statistical Analysis

Continuous variables were presented as the medians (interquartile ranges [IQR]) and categorical variables as the numbers (%). Differences in baseline characteristics between groups were analyzed using independent sample t-tests, with Mann Whitney U-tests used for continuous variables and Chi-square or Fisher’s exact tests used for categorical variables. Based on previous studies in the literature, the variables associated with PRAEs were obtained, and thus correlation statistics were drawn to indicate the correlation between the variables.

We used the LASSO regression and multivariate logistic regression approach with the Akaike information criterion (AIC) to select the best predictive features of morbidity in the training cohort using the backward selection method that included variables with P<0.05, and AIC was used to control the over-fitting of the model. The characteristic of the nonzero coefficient in the cable regression model was selected through LASSO analysis and was put into the multivariate logistic regression analysis to identify the independent risk factors associated with PRAEs. And these risk factors were considered based on odds ratios (ORs) with 95% confidence intervals (CIs) and P-values. A nomogram model for predicting perioperative respiratory adverse events was developed based on the multivariable logistic regression analysis using rms package and foreign package in R software. TO assign a point value for significant predictors to use the nomogram, and then it needs to be added to generate a total score and converted into an individual probability of PRAEs. The predictive performance of the nomogram was measured by the C index, ROC curve, and calibration with 1000 bootstrap samples to decrease the over-fit bias based on the data from the training cohort and validation cohort.¹² The C-index of 0.5–0.7 indicates poor model predictive power, 0.7–0.9 indicates good model predictive power and greater than 0.90 indicates excellent predictive power. Decision curve analysis (DCA) was conducted to assess the predictive nomogram’s clinical usefulness by quantifying the net benefits at different threshold probabilities. Finally, the clinical impact curve (CIC) was plotted to evaluate the model’s clinical usefulness and applicability with net benefits with the best diagnostic value.

Risk predictive model validation: data of the validation group were substituted into the established logistic regression model, and the sensitivity and specificity of the model in the validation group of patients were calculated by comparing with the final follow-up results.

Statistical analysis was performed with R software (version 4.1.1; https://www.R-project.org). The reported statistical significance levels were all two-sided, with statistical significance set at 0.05.

Results

Participant Characteristics

The flow chart of patient inclusion is presented in Figure 1. In this study, a total of 747 children with airway surgery were collected according to the inclusion criteria, excluding 3 patients with atrial septal defect, 8 patients refused to participate because they did not understand the content of the scale, 15 patients changed the anesthesia mode, 9 cases of suspension of surgery due to heavy cold, and 3 patients failed to complete the follow-up. Finally, the data of 709 patients were analyzed in the present study. Simple random sampling was performed with the random sampling function in version 4.1.1 of R software, and patients were randomly divided into the training (n=496) and validation groups (n=213) by a ratio of 7 to 3 with group cross-validation LGOCV.¹³ All characteristics were no statistically significant differences between training cohort and validation cohort (all P > 0.05) (Table 1). In our study, we observed a post-operative hospital stay of 2 days, one day of observation after routine surgery at our institution, To facilitate timely management of complications such as postoperative bleeding and postoperative edema and to reduce the incidence of adverse postoperative events.

Figure 1 Flow chart outlining the patient inclusion process.

According to LASSO analysis, in the training cohort, twenty potential risk factors from perioperative clinical indicators were included in the LASSO regression analysis (Supplemental Figure 1A and B). Correlation analysis and dot plots were performed to visually detect correlations between all the risk factors or independent variables and PRAEs (Supplemental Figure 1C), showing that age was negatively correlated with obesity, the degree of upper respiratory tract infection was more positively correlated with time to infection, and the strongest correlation was between time to anesthesia and time to surgery. The degree of upper respiratory tract infection is defined by the patient’s symptoms,⁸ mild examples: parent confirms URI and/or congestion, rhinorrhea, sore throat, sneezing, low fever, or dry cough; moderate/severe examples: purulence, wet cough, abnormal lung sounds, lethargy, “toxic” appearance, or high fever. We selected 12 non-zero characteristic variables including gender, age, obesity, allergy, smoking, URTI-mild, URTI-moderate/severe, fever, runny nose, induction, midazolam, dexmedetomidine (Table 3). The results of the multivariate logistic analysis are presented in Table 4, Figure 2. On multivariate analysis, with results reported as odds ratio (95% CI), URTI-mild [1.626 (1.053–2.512)], URTI-moderate/severe [2.270 (1.137–4.533)], passive smoking [1.936 (1.310–2.861)], obesity [1.794 (1.159–2.777)], midazolam [1.662 (1.048–2.635)], dexmedetomidine [0.429 (0.268–0.687)], age (increase) [0.920 (0.863–0.981)] were independently associated with PRAEs. PRAEs occurred in 45.6% (training cohort) and 41.3% (validation cohort). The details of adverse events and their timing are displayed in Table 5. The highest incidence of oxygen desaturation (5.8% in the induction period and 32.7% in the PACU period) and a low incidence of PRAEs in the maintenance period, associated with sound anesthetic management in the maintenance period.

Table 3 Coefficients and Lambda.Min Value of the LASSO Regression

Table 4 Multivariate Logistic Regression Analysis of PRAE Presence Based on Perioperative Data in the Training Cohort

Table 5 Perioperative Adverse Events Occurring During Induction, Maintenance, and the Postoperative Acute Care Unit. (Only the Occurrence is Recorded, No Count is Accumulated)

Figure 2 Forest plot of odds ratio (OR) with confidence intervals.

Development and Validation of the PRAEs-Predicting Nomogram

These independently associated risk factors were used to build a PRAEs risk prediction model (Figure 3). For example, if the child is 4 years old, obese, mild URTI, preoperative sedation with dexmedetomidine, and a history of passive smoking, the patient’s scores for each risk factor are 77.5, 38.75, 32.5, 0, and 43.75, giving a total score of 192.5, which corresponds to a PRAE probability of approximately 60%, thus making it easier to assess the patient’s condition. The resulting model was internally validated using the bootstrap validation method and group cross-validation LGOCV. Discrimination of the nomogram was measured by calculating the unadjusted C-index, which was 0.760 (95% CI, 0.725–0.810), indicating good predictive power in the training cohort and a bootstrap-corrected C index of 0.71 (Supplemental Figure 2A). The validation cohort also confirmed the nomogram’s calibration with an unadjusted C-index of 0.777 (95% CI, 0.714–0.841) and a bootstrap-corrected C index of 0.75 (Supplemental Figure 2B). The C-index indicated good model predictive power. Besides, the calibration plots graphically demonstrated good agreement on the presence of PRAEs between the risk estimation by the nomogram and analysis results of actual clinical data (Supplemental Figure 2C and D). As a C-index >0.75 is generally considered to indicate reliable discrimination, this nomogram performed well in terms of discrimination and calibration in both the training and validation cohorts.¹⁴ A decision curve analysis (DCA) was applied to assess the clinical validity of the nomogram (Supplemental Figure 2E and F). This showed the ability of the nomogram to predict PRAEs because the range of high-risk threshold probabilities was wide and applicable to both the training and validation cohorts. From the decision curves, the net benefits of the nomogram and the internal validation set were significantly higher than those of the two extreme cases.

Figure 3 Nomogram to estimate the risk of PRAE in children undergoing airway surgery. To use the nomogram, find the position of each variable on the corresponding axis, draw a line to the points axis for the number of points, add the points from all of the variables, and draw a line from the total points axis to determine the PRAE probabilities at the lower line of the nomogram.

Discussion

In the present study, we developed and validated a prediction model for PRAEs in training and validation cohorts in children undergoing airway surgery. The proposed nomogram, which incorporated 5 comprehensive and easily available perioperative variables: age (increase), obesity, degree of URTI, premedication, passive smoking, performed well as supported by the C index values of 0.760 and 0.777 in the training and validation cohorts, respectively, and the optimal calibration curves demonstrating the agreements between prediction and actual observation. In children, PRAEs are the most common adverse events in patients undergoing anesthesia, the incidence of URTI, obesity, and airway surgery presenting for anesthesia were high.¹⁵ The incidence of PRAEs in previous studies was 13% to 15%, and in airway surgery, the incidence can be up to 50%.^4,6,9 The prevalence was 44.29% since this study population was all undergoing airway surgery. The incidence of PRAEs was 45.6% (training cohort) and 41.3% (validation cohort). The prevalence rate is increasing every year, and its high rates affect patients’ quality of life.

In the PRAEs risk estimation nomogram, age (decrease), obesity, URTI, premedication (midazolam), passive smoking have been reported to increase the possibility of PRAEs.^1,4,8,16 In addition, we demonstrated that dexmedetomidine was associated with a decreased probability of PRAEs in airway surgery children patients.

The incidence of PRAE increases with decreasing age, especially for laryngospasm, with the relative risk decreasing by 11% for each yearly increase in age,³ The present study also confirmed that the incidence of PRAEs tended to be higher among the younger age group. Compared to adults it is the increased/more sensitive respiratory reflexes that lead to higher PRAE especially laryngospasm in infants/preschoolers. General anesthesia is inherently riskier in the pediatric population and respiratory complications are common in the pediatric perioperative period, making it necessary to establish a practical clinical prediction tool.

The increasing prevalence of overweight and obesity among children has become a significant medical and societal issue worldwide.¹⁷ Anesthesiologists are likely to see a corresponding increase in the proportion of these patients who present for anesthesia and surgery. However, there is a paucity of information regarding the outcome in overweight or obese children who present for anesthesia and surgery. Our study reported an increase in PRAEs in obese children compared to normal-weight children, these results were consistent with the results described in previous studies.¹⁸

PRAEs are major complications during the intraoperative and postoperative periods in children with upper respiratory tract infection (URTI).¹⁹ Previous studies have found that approximately 30% of children undergoing elective surgery have active URTI symptoms.²⁰ In agreement with previous studies that present or recent URTI is associated with an increased risk for PRAEs, the most common reason for delaying surgery in children is due to upper respiratory tract infection.^21,22 However, no clear consensus or formal recommendations exist to assist physicians in deciding whether to proceed with or to reschedule anesthesia. The incidence of URTI in this study was 35.8%, which is a major risk for morbidity during pediatric anesthesia. Using a multivariate analysis by logistic regression, the moderate or severe URTI (OR = 2.270, 95% CI [1.337, 4.533], P = 0.020), the largest OR of all factors, indicating that this factor had the strongest association with outcome. Previous studies have shown a higher incidence of PRAE with URTI <2w, but there was no statistical difference in this study, which may be due to sample variability, the fact that the study institution was a large general tertiary care hospital and all anesthesiologists are professional pediatric anesthesiologists with rich clinical experience and the effect of other factors weakening its effect. The incidence of PRAEs is reported to be two to seven times higher in children with URTI than in children without URTI, with an 11-fold increase when the trachea is intubated.²² This apparently higher incidence of PRAEs may be attributed to alterations in lung function, including reduced diffusion capacity, reduced lung compliance, increased airway resistance, and abnormal lung clearance mechanisms due to lower airway involvement.^1,4

In recent years, the use of pediatric preoperative sedation has been widely recommended and used. Midazolam and dexmedetomidine as the most common preoperative sedatives in children, it can be achieved by intravenous, oral, nasal drops, etc. According to previous studies, intranasal premedication is more acceptable to children than oral and intravenous methods.^23–25 In this study we investigated that dexmedetomidine has a protective effect, consistent with previous studies.^26–28 Dexmedetomidine can facilitate tolerance of the endotracheal tube, suppress a sharp increase in heart rate during extubation and have a protective effect on the occurrence of PRAEs.^29–31 One study showed that preoperative midazolam use appeared to increase the incidence of PRAEs.³² However, a 16-center trial in France was shown to have a preventive effect on PRAEs.¹ In our study, children who were sedated with midazolam preoperatively had a higher risk of PRAEs (OR=1.662, CI:1.055–2.651, p=0.029). Previous results are contradictory, and different routes of administration, sedation time, dose and depth of sedation, and intraoperative management strategies may all affect the outcome.³³ And the mechanism of this behavior is still unclear, and we will do more study of the contradictory reaction to midazolam in the future trials.

Environmental Tobacco Smoke (ETS) or passive smoking exposure is a major public health problem today. Compared to adults, children have immature immune systems and are more susceptible to the effects of various physiological systems, most notably the respiratory system, which worsens over time. Previous studies identified exposure to tobacco smoke as a risk factor for perioperative respiratory adverse events.^4,22,34 Lewis³⁵ reported that the risk of wheezing in children was proportional to the number of people living in a smoking environment. Data from this study do not provide further insight into the effect of the smoking habits of different family members. This is because the study population lives in an urban environment where the majority of fathers smoked.

The LASSO regression is a popular method for variable selection in fitting a high-dimension generalized linear model, which can get a more refined model by constructing a penalty function to reduce the variable numbers and effectively avoid overfitting.^36,37 The Lasso regression was used for our data dimension reduction and feature selection, then multivariable binary logistic regression was used to build a predictive model with regression coefficients.

Accurate estimates of the likelihood of treatment success, complications, and long-term morbidity are essential for counselling and informed decision-making in patients. Informed consent that is based on accurate estimation of the likelihood of various treatment outcomes might improve treatment satisfaction, particularly when complications arise.³⁸ There are several options for risk estimation, including nomogram, risk groupings, webpage calculator, and scorecards. Some studies present a long and complex equation to link the various independent factors, so it is essential to create a nomogram.^3,22,39 The CLODS scale is now the most commonly used for predicting pediatric PRAEs.⁸ Despite its apparent logic, this method is statistically inefficient and might reduce the accuracy. Its flaws consist of the assumption that all patients within a risk group are equal, but risk groups might be heterogeneous.²⁰ Various studies have documented the superior performance of nomogram compared to risk-grouping.^40,41 More and more studies have used nomograms to visualize the results of multifactorial analysis, largely facilitating the early screening of people at high risk for a particular adverse event.^42–44 The statistical definition of a nomogram is a graphical representation of a mathematical formula. The predictors of such a formula might be modeled as continuous or categorical variables to predict a particular endpoint. The statistical methods can consist of multivariable logistic regression or Cox proportional hazards analyses. To use the nomogram, find the position of each variable on the corresponding axis, draw a line to the points axis for the number of points, add the points from all of the variables, and draw a line from the total point’s axis to determine the PRAEs probabilities at the lower line of the nomogram. Nomogram provides the probability of a particular outcome on a continuous scale, which is usually 0–100%. Previous studies have created nomograms, but only included people up to 12 years of age, and the most significant risk factors such as URTI were not ultimately included in the model. The method of screening predictor variables was not as good as LASSO.⁴⁵

The limitations in the present study include: (1) this was a single-center prospective study, which may introduce bias. The results of this study need to be verified by data from other centers (2) the risk factors of PRAEs are complex and there are far more factors to be investigated and used to predict the prognosis of PRAE. The different sources of data and the low prevalence of some risk factors, combined with the different literacy levels of the guardians, resulted in some predictors showing no difference. (3) There may be some homogeneity in the type of disease in this trial, mostly tonsillar adenoid surgery. The scale is to estimate PRAEs in children undergoing airway surgery. It might not predict PRAEs in a cohort with abdominal or urological surgery for example. Further research is needed in the future. (4) Inhalation anesthesia maintenance was not included in the maintenance method of this trial. It is known from other studies that this increases the risk of PRAEs compared to propofol. This needs further research.

Conclusion

In summary, we developed and validated a nomogram that will help predict PRAE allowing a better estimation of risks and discussion with patients and carers. It could be a rapid risk-scoring system to accurately assess anesthesia risks before surgery. At the same time, intraoperative strengthening of anesthesia management is targeted to reduce the risk of postoperative complications, shorten the length of hospitalization, and improve the patient’s hospital satisfaction rate. Additionally, it is imperative to confirm these findings through prospective, multicenter studies.