Nomogram Models for Predicting Delirium of Patients in Emergency Intensive Care Unit: A Retrospective Cohort Study

Introduction

Although advancements in the field of scientific medicine have greatly improved treatment of critical diseases, ICU delirium still occurs in ICU.¹ Delirium refers to an acute and volatile change in mental state, which is characterized by disturbance of consciousness, and is often accompanied by sleep-wake cycle disorders, varying degrees of attention deficiency, as well as cognitive and affective disorders.² Delirium that occurs in ICU patients is known as ICU-delirium.

Previous studies have shown that the incidence of ICU-delirium is high, with a gradual growth trend.^3,4 This incidence also varies across different patients. General ICU patients have an ICU-delirium incidence of 15–80%,^5,6 whereas those under mechanical ventilation (MV) have a 60–80%.^7–10

Generally, the occurrence of ICU-delirium is associated with various adverse events.^11,12 For example, patients experience short-term disease recovery time and long-term mental health problems,¹³ families encounter a high burden of disease and psychological pressure,¹⁴ while medical resources are subjected to increased physical and mental pressure for workers as well as high medical costs.¹⁵ Therefore, evaluating potential risk factors of ICU-delirium, as well as constructing prediction models, and development of prevention approaches are imperative to management of this condition and alleviation of suffering.

Although the risk factors and predictive models of ICU-delirium have been previously reported, little was known regarding the occurrence of ICU-delirium in EICU patients owing to the fact that most focus has been on ICU and coronary care (CCU) wards. EICU is a new organization and management model, an extremely important part of emergency medicine, and a brand-new sign of the modernization of the emergency department.^16,17 In addition, several diseases have been reported in EICU patients, while composition of systemic diseases is different from that in ICU and CCU.¹⁶ Since the existing research evidences may not be suitable for EICU patients, we sought to identify risk factors of ICU-delirium within 24 hours of admission, and construct a nomogram model for predicting the risk of EICU patients, with the aim of providing a reference for early identification and intervention.

Methods

Study Design and Data Sources

This was a retrospective cohort study, comprising data collected from medical records of patients in the EICU at the Second Affiliated Hospital of Xi’an Jiaotong University between July 1st 2020 and December 1st 2020. The evaluation method used herein was one of the daily tasks performed in the ICU. All study procedures were performed in accordance with the guideline of the ethics committee of the Second Affiliated Hospital of Xi’an Jiaotong University, approval number 2021006. The research complied with the Declaration of Helsinki. In addition, the detailed information of participants was not collected in our research database to ensure anonymous usage data.

Inclusion and Exclusion Criteria

Participants who met the following criteria were enrolled in the study: 1) aged ≥18 years old; and 2) had EICU stay time ≥24 h. Conversely, those who met the following criteria were excluded: 1) had a history of mental illness or dementia; 2) exhibited severe hearing or visual impairment; 3) had severe mental retardation or aphasia; 4) those who were continuously in a coma or deep sedation, with a Glasgow Coma Scale (GSC) score less than 8 points; and 5) those who already had delirium when entering the EICU.

Participants

A total of 450 patients were screened, between July 1st to December 1st, 2020, of which 131 were excluded because of the following reasons: they were aged <18 years old (N = 5); had EICU stay times <24 h (N = 14); were severely mentally handicapped or had aphasia (N = 4); were in deep coma (N = 30); exhibited delirium prior to admission in the EICU (N = 64); and were missing important data (N = 14). Finally, a total of 319 cases were included in our study (Figure 1).

Figure 1 Flow chart of patient recruitment.

Calculation of Sample Size

Before the formal study, we conducted a pre-experiment with 50 samples and predicted that 5 factors would be required to establish the ICU delirium prediction model. Each factor was validated with a minimum of 5–10 cases. Based on previous data, the proposed incidence rate is 20%, and assuming that the follow-up loss rate was 10%, the minimum sample size required for this study was 278 ((5*10/0.2)/0.9=278). A total of 319 samples were included in this study, which met the requirements.

Data Collection

Each patient’s relevant medical records were collected, within 24 h of admission, and the following information recorded: (1) Demographic and clinical data, such as age, sex, diagnosis type, educational level, body mass index (BMI), smoking, drinking, diabetes, hypertension, sepsis, heart failure, and electrolyte disorders; 2) previous clinical treatment measures, including stomach and urinary tubes, central venous catheter (CVC), sedative, MV, and vasoactive drugs; and 3) disease severity index: Acute Physiology and Chronic Health Evaluation (APACHE-II) score and Sequential Organ Failure Assessment (SOFA) score.

Assessment of Delirium

Delirium was assessed using the Richmond Agitation-Sedation Scale (RASS) score and Confusion Assessment Method for ICU (CAM-ICU). Particularly, we assessed delirium by CAM-ICU if a patient’s RASS score was −3 to +4, but not in those with a score of −4 to −5. Delirium assessment based on CAM-ICU was performed based on four aspects, namely acute onset and fluctuating causes of symptoms, inattention, disorganized thinking and altered level of consciousness. A patient that met the first and second aspects, and either the third or fourth, was considered positive for delirium. This method was used to evaluate patients every 24 h, until their discharge or death.

Quality Control

RASS score and CAM-ICU are commonly used as evaluation tools in the world.^18,19 Every patient was independently assessed by doctors and nurses, and in case of a disagreement, they would consult a third person in our team. All research in the patient study underwent specialized training and passed the assessment prior to evaluation.

Statistical Analysis

Missing value were less than 5% in all variables in the present study. These values had minimal influence on research conclusions, hence we did not special consideration during statistical analysis. Continuous variables were expressed as means ± standard deviations ( ± s), whereas counts were expressed as numerical values and percentages. Variables with p < 0.05, after univariate analysis, were selected for multivariate logistic regression. Similarly, those with p < 0.05 after multivariate logistic regression analyses were selected for model construction. We adopted a bootstrapping approach for internal validation (resampling 500 times), due to the relatively small sample size of our study. All statistical analyses were performed using EmpowerStats (http://www.empowerstats.com, X&Y Solutions, Inc, Boston, MA) and packages implemented in R software (http://www.R-project.org, The R Foundation).

Results

Clinical Characteristics of Patients

A summary of patients’ clinical characteristics is provided in Table 1. Briefly, a total of 319 patients met our inclusion and exclusion criteria, and were therefore included in the study. Before leaving the EICU, 96 (30.09%) patients developed delirium, while 223 (69.91%) did not. Their average age was 60.46 ± 16.47 years, while treatment measures included stomach tube (n = 41), urinary tube (n = 147), CVC (n = 185), sedative (n = 78), MV (n = 51), and vasoactive drugs (n = 65). APPACHE-II and SOFA scores were 10.82 ± 5.01 and 4.98 ± 2.55 points, respectively (Table 1).

Table 1 The Clinical Characteristics of Patients

Univariate Analysis Results

Univariate analysis revealed that age, hypertension, stomach tube, urinary tube, sedative, MV, APACHE-II score and SOFA score were significantly associated with delirium (p < 0.05), while the other variables had no significant relationship with delirium (p > 0.05) (Table 2).

Table 2 The Results of Univariate Analysis

Results of Multivariate Logistic Regression Analyses

Statistically significant variables (p < 0.05) from univariate analysis were selected for multivariate logistic regression analyses. Since the APACHE-II score included assessment of age and blood pressure, we only included stomach and urinary tubes, as well as sedative, MV, APACHE-II and SOFA scores for multivariate analysis in order to avoid over-fitting of the model. The principle of covariate screening was that after introducing the variable into the basic model, or excluding it from the complete model, the variable whose influence on the regression coefficient of the independent variable exceeds 10%. Adjusting covariates, including vasoactive drugs, heart failure, diagnosis and type, revealed that stomach and urinary tubes, sedative, mechanical ventilation and APACHE-II scores were associated with delirium of patients (Table 3).

Table 3 Multivariate Logistic Regression Analyses for Delirium

Construction of a Prediction Model

The aforementioned five variables, with P < 0.05, from the multivariate logistic regression analyses above were selected and used for model construction. The areas under curve (AUC) of the receiver operating characteristic (ROC) were 0.76 (0.70, 0.82), 0.75 (0.69, 0.81) and 0.75 (0.69, 0.81) for Multiple Fractional Polynomial (MFP), full and stepwise models, respectively. Non-elderly patients, aged <60 years old, recorded AUCs of 0.82 (0.73, 0.91), 0.83 (0.74, 0.91) and 0.82 (0.73, 0.91) for MFP, full and stepwise models, respectively. On the other hand, elderly patients aged ≥60 years old had AUCs of 0.82 (0.73, 0.91), 0.71 (0.62, 0.79) and 0.71 (0.63, 0.78) for MFP, full and stepwise models, respectively (Table 4 and Figures 2–7).

Table 4 Results of the Constructed Predictive Models

Figure 2 ROC curves of the models in all patients.

Figure 3 A MFP Nomogram model for all patients.

Figure 4 ROC curves of the models in non-elderly patients.

Figure 5 A MFP nomogram model for non-elderly patients.

Figure 6 ROC curves of models in elderly patients.

Figure 7 A MFP nomogram model for elderly patients.

Validation of the Prediction Model

We adopted a bootstrapping approach for internal validation of our models, owing to the relatively small size of our study. The full bootstrap model (BS full) and stepwise bootstrap model (BS stepwise) were used to verify the accuracy and value of the full model and stepwise model. The AUCs of 0.75 (0.69, 0.81) and 0.75 (0.69, 0.81) were recorded for BS full and stepwise models, respectively. In patients aged <60 years old, the model recorded AUCs of 0.83 (0.74, 0.91) and 0.82 (0.73, 0.91) for BS full and stepwise, respectively, while in elderly patients (≥60 years old), AUC values were 0.71 (0.62, 0.79) and 0.71 (0.63, 0.78) for BS full and stepwise, respectively (Table 4 and Figures 2–Figures 7).

Decision Curve Analysis of Prediction Models

We performed decision curve analysis (DCA) of constructed models. The results showed that, whether it was a model constructed by all patients or models constructed by age stratification, there was a certain distance between the nomogram and the standard line, which also confirmed the good clinical practicability of models (Figures 8–10).

Figure 8 DCA of the model in all patients.

Figure 9 DCA of the model in non-elderly patients.

Figure 10 DCA of the model in elderly patients.

Discussion

Results of the present study revealed that the incidence of delirium was 30.09%, which was consistent with results from previous epidemiological surveys.^5,6 However, incidence from our study was slightly higher than that reported by Tsuruta,²⁰ possibly due to the high use of sedatives in the present study. To date, numerous studies have demonstrated that sedatives can be used to increase incidence of delirium in patients.²¹ Our results further revealed a slightly lower delirium incidence relative to that reported by Yang,²² which might be due to differences in the average age of patients and APACHE scores between the study. Generally, older patients are more likely to develop ICU delirium.²³

Our results revealed five variables, namely stomach and urinary tubes, as well as sedative, mechanical ventilation and APACHE-II scores, that were significantly associated with delirium. We used these variables to construct 3 predictive models in the entire patient cohort, comprising elderly (≥60 years old), and non-elderly (<60 years old) patients. We internally validated these models using a bootstrapping approach. Results revealed a simple and highly accurate model for predicting delirium. The model could accurately predict delirium in both young and elderly patients, as evidenced by high AUC values across these groups.

APACHE – II, a tool used to analyze disease severity and prognosis of critically ill patients, has been widely used in EICUs worldwide.^24,25 Results of the present study showed that patients with high APACHE-II scores had a higher risk of delirium, consistent with previous, studies that have shown that this variable is an independent risk factor for patients with delirium.^26,27 For example, results of a previous meta-analysis,²¹ with a sample size of 2440, showed that the APACHE-II score was significantly higher in the delirium than non-delirium group.

In addition, Margaret study²⁸ suggested that use of sedative was a controllable risk factor for delirium. In 2013, the guidelines for the management of pain, agitation/sedation, and delirium issued by the Society of Critical Care Medicine (SCCM) also emphasized the influence of sedatives on delirium.²⁹ Results of the present study showed that use of sedative was an independent risk factor for delirium, consistent with the findings of Huang³⁰ and Pan,²¹ who concluded this in ICU patients.

Although MV is a common treatment strategy with promising therapeutic effects in EICU patients, it is often associated with occurrence of physical and psychological discomfort. Results of the present study showed that MV was an independent risk factor for delirium, consistent with a meta-analysis by Zhang³¹ who found that MV could cause sleep disorders in patients and increase the risk of delirium in this group of patients. Particularly, patients with mechanical ventilation were more likely to develop lung infections and hypoxemia, which exacerbated the risk of delirium.³²

Insertion stomach and urinary tubes are common invasive treatment operations in EICU. Long-term indwelling of foreign bodies in the patient’s body often predisposes a patient’s body to a state of stress and causes bad moods, which increase occurrence of delirium. Our results showed that stomach and urinary tubes were independent risk factors for patients with delirium, hence their inclusion in construction of predictive models.

All five variables identified herein were closely associated with delirium, consistent with previous studies. These factors, coupled with application of multiple methods for controlling the influence of collinearity between variables, allowed us to construct an accurate and reliable model for predicting delirium.

We successfully constructed 3 optimal predictive models for all patients, as well as those below and above 60 years. The MFP model for all patients had an AUC of 0.76 (0.70, 0.82), and comprised 4 variables, namely stomach and urinary tubes, as well as sedative, and APACHE-II scores. The full model for non-elderly patients (<60 years old) had an AUC of 0.83 (0.74, 0.91), and comprised all 5 aforementioned variables. In addition, the MFP model for elderly patients (≥60 years old) had an AUC value of 0.82 (0.73, 0.91), and comprised 4 variables, namely stomach and urinary tubes, as well as sedative, and APACHE-II scores. All these models were highly accurate in predicting delirium in patients.

Nomogram Application

Each variable in the figure is marked with a scale on the line segment, representing their value range, with the length of the line segment reflecting its contribution to the outcome event. The point in the figure represents a single score, and their single scores are added to obtain total points.^33,34

Limitations of the Study

Firstly, our prediction model requires additional validation, since it was constructed and validated from a single-center, and the verification was also internal, which may cause its prediction value to decline outside this hospital. Secondly, we used a small sample size.

Conclusion

We developed a five-marker-based prognostic tool, for effective predicting occurrence of delirium in EICU patients. This tool, if prospectively validated, could provide individualized risk estimation of delirium patients in EICU.