Dynamic Nomogram for Subsyndromal Delirium in Adult Intensive Care Unit: A Prospective Cohort Study

Introduction

Delirium is a common clinical syndrome in the ICU, characterized by fluctuating consciousness disorders, attention disorders, changes in consciousness levels or thinking disorders, with an incidence rate ranging from 23% to 54.9%.^1,2 However, in medical practice, patients usually show one or more delirium symptoms but do not meet the complete diagnostic criteria for delirium.³ In the 16th century, Guainerio recorded the occurrence of prodromal symptoms of delirium in patients and pointed out that whether the state progresses to delirium will depend on the patient’s constitution, the nature of the underlying disease, and the treatment used.⁴ In 1990, Lipowski also described the concept of a precursor to delirium, where patients exhibited at least one symptom of delirium but did not progress to delirium.⁵ In 1996, Levkoff formally proposed the concept of “subsyndromal delirium” after verifying the difference between subsyndromal manifestations and delirium.⁶ At present, scholars tend to believe that delirium and subsyndromal are two different states of one disease, both of which belong to the same disease spectrum.⁵

Subsyndromal delirium (SSD), is common in ICU patients which refers to patients presenting with one or more core delirium symptoms without meeting all the delirium criteria.⁴ A survey of ICU patients found that the incidence of SSD was 33.9%, of which 9.5% of patients progressed to delirium.⁷ The incidence of SSD in ICU patients has been reported to vary from 12.5% to 85.5%, depending on different studies.^7–9 Studies have shown that delirium is a powerful predictor of cognitive decline.¹⁰ Another prospective multicenter study found that patients with SSD had cognitive decline from admission to 3 months of follow-up, and early SSD was linked to cognitive impairment.¹¹ SSD is also associated with several adverse outcomes, including increased in-hospital mortality rate, prolonged hospitalization, and delirium, which can increase the burden on family caregivers and the healthcare system.^5,12

However, there is currently no effective clinical model to predict the occurrence of SSD in ICU. Japanese scholars have used the delirium model to predict the probability of ICU subsyndromal delirium. The results show that the sensitivity of the delirium model is 94.3% and the specificity is 57.1%, which means that the model cannot effectively distinguish patients with SSD and has a high misdiagnosis rate.¹³ Whether the delirium prediction model is suitable for patients with SSD remains to be further studied. Therefore, it is necessary to construct SSD prediction models to help healthcare professionals identify high-risk populations in the early stages and improve patient prognosis by reducing risk factors.⁴

A nomogram is a simple statistical visualization tool that graphically shows the influence of each predictor variable on the results. It is now widely used in prediction models.¹⁴ Based on the ordinary nomogram, the dynamic nomogram realizes the real-time online calculation of the prediction probability of different patient outcomes, which significantly improves the promotion value and clinical practicability of the nomogram.¹⁵ The primary purpose of this study was to develop and validate a dynamic nomogram model for the early prediction and timely management of SSD in ICU patients. We hope that the model will help healthcare professionals quickly and accurately identify SSDs in the future and provide a suitable tool for later SSD management.

Materials and Methods

Study Design and Patient Selection

A prospective cohort study was carried out. The study was conducted in four ICUs of a tertiary hospital in Zhejiang Province. Patients admitted to the ICU between September 2021 and June 2022 were selected for the study. The inclusion criteria were as follows: ① age ≥ 18 years; ② length of ICU hospitalization ≥ 24 h. The exclusion criteria included the following: ① patients with delirium before admission to ICU; ② patients with a history of mental illness; ③ patients with communication difficulties;④ patients in deep sedation during hospitalization using the Richmond Agitation Sedation Scale (RASS) score of −4 or −5; ⑤ Patients who are unable to carry out the instructed actions; ⑥ patients with incomplete data.

Diagnosis of SSD

The delirium assessment tool, tested for reliability, can be used to diagnose SSD. Based on the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV), the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU) was designed as a delirium assessment tool specifically for ICU patients.¹⁶ The Chinese version of CAM-ICU has a sensitivity of 93.4%, a specificity of 90.8%, and good reliability and validity.¹⁷

It mainly evaluates four core symptoms of delirium, namely, acute onset of mental status changes or fluctuating course (feature one), inattention (feature two), disorganised thinking (feature three) and altered level of consciousness (feature four). The diagnostic criteria for delirium are as follows: feature one and feature two were positive, and any feature three or feature four was positive. The diagnostic criteria of SSD are at least one core symptom but failing to meet the diagnostic criteria of delirium.¹⁸ Thus, a diagnosis of SSD can be made when any of the features is positive, excluding delirium and four features are negative.

Data Collection

The ICU patients who met the inclusion criteria were selected by the research group, including two researchers and one critical care expert. Both researchers passed unified training and guidance before the start of the study. The purpose and significance of this study were introduced to the patients by unified guidance. After the informed consent of the patients was obtained, the medical history data of the patients within 24 hours of entering the ICU were collected from the electronic information platform. CAM-ICU was used to evaluate the patients at 6 p.m. every day. The time to start the assessment was 24 hours after the patient entered the ICU. The endpoint of the assessment is the transfer of the patient out of the ICU or the development of delirium or death. When the judgment of the two researchers was consistent, the corresponding results would be recorded. If the judgment of the two researchers was inconsistent, they sought the screening judgment of the critical care expert.

Based on reviews and expert opinions in ICU, 28 variables were identified. These variables were age, gender, educational level, marital status, body mass index (BMI), smoking history, drinking history, hypertension history, diabetes mellitus history, cardiovascular disease history, cerebrovascular disease history, indwelling urinary catheter, indwelling gastric tube, mechanical ventilation, glucocorticoid drugs, constraint, Acute Physiology and Chronic Health Evaluation II score (APACHE II score), pain score, respiratory failure, blood transfusion, metabolic acidosis, dehydration, dexmedetomidine, propofol, midazolam, C-reactive protein level, red blood cell count and lymphocyte count. The patient’s plasma osmolality was calculated to determine patient dehydration and plasma osmolarity was calculated (pOsm_c) as follows: pOsmc=1.86*([Na⁺]+[K⁺])+1.15*[glucose]+[urea]+14 (all measure in mmol/L).¹⁹ Dehydration is defined as pOsmc≥300mmol/L.

Sample Size

The sample size was calculated using the “pmsampsize” package in R.²⁰ Through literature review,^5,7,21,22 28 candidate factors were identified, and the incidence of ICU subsyndromal delirium was 30%. According to the area under the ROC curve (AUC) of the previous delirium prediction model, the C index was determined to be 0.8, and the outcome was whether SSD occurred.

All patients who met the inclusion and exclusion criteria in the ICU of a tertiary hospital in Zhejiang Province from September 2021 to June 2022 were collected. The sample size was about 1000 patients and the patients were randomly divided into a training set and a validation set at a ratio of 7:3. The minimum sample size was determined to be 963 cases.

Statistical Analysis

Statistical analysis was performed using IBM SPSS (Version 25.0) and R statistical software (version 4.1.2). We adopt a random split method for internal validation of the model, with 70% of the data used for modeling and 30% for model validation. Randomly divide the patient’s data into training and validation sets in a 7:3 ratio, and compare the differences between the training and validation sets. Continuous variables are expressed as mean ± standard deviation (SD) or median and interquartile range (IQR). Categorical variables are presented as frequency and percentage. The t-test or Mann–Whitney U-test and Chi-square or Fisher’s exact tests were used to investigate the differences in quantitative and categorical variables between these groups.

The least absolute shrinkage and selection operator (LASSO) regression was used for the preliminary screening of risk factors. The variables screened by LASSO regression were included in multivariate logistic regression to screen the variables further. Use R software to draw logistic regression forest plots and continuous variable probability plots. The optimal variables obtained by LASSO and multivariate logistic regression were employed for the prediction model construction. The “replot” and “DynNom” packages in R were employed to generate an ordinary nomogram and a dynamic nomogram, respectively.

We assess the performance of the SSD model from three aspects, namely, discrimination, calibration and clinical usefulness. The model’s discrimination was evaluated using the area under the ROC curve (AUC). The GiViTI calibration band was used to evaluate the calibration of the model, that is, the relationship between the prediction and observation probability. Decision curve analysis (DCA) was used to evaluate the potential benefits of predictive models in clinical practice. P value <0.05 was considered statistically significant.

Results

Baseline Patient Characteristics

A total of 1613 patients were in the adult ICU from September 2021 to June 2022. Approximately 613 patients were excluded for the following reasons: delirium (n=356), history of mental illness (n=37), communication difficulties (n=12), RASS = −4 or −5 (n=82), patients who are unable to follow instructions in physical activities (n=75), and incomplete data (n=51). Finally, 1000 patients were included in the study, of which 345 were diagnosed with SSD; thus, the overall incidence of SSD in adult ICU was 34.5%. The flowchart of the study is shown in Figure 1.

Figure 1 Research object flow chart.

The training set consisted of 700 patients which was divided into the SSD group (n=246) and the non-SSD group (n=458). There were 300 patients in the validation set, including 103 SSD patients and 197 non-SSD patients. The median age of 300 patients was 64 years old, and the median BMI was 23.20. Among them, there were 185 male patients (61.7%) and 115 female patients (38.3%). The main baseline characteristics of patients are shown in Table 1. The incidence of SSD was comparable between the training set and validation set (34.6% vs 34.3%; P=0.942). Baseline data showed a statistical difference in cardiovascular disease history between the two groups (P=0.027), but no difference was found in other indicators (Table 1). In general, the distribution of variables in the training and validation sets was the same, and grouping was utterly random to avoid the deviation of uneven distribution.

Table 1 Demographic and Clinical Characteristics of Study Population in the Training and Validation Cohorts

Nomogram Development

A total of 28 risk indicators were taken for each patient. Using LASSO regression, predicted variables were selected from 28 candidate variables. There are two results by using LASSO. When the lambda value was selected as lambda.1se (0.03048), 13 variables with nonzero coefficients were screened out. When the lambda value was selected as lambda.min (0.00755), 23 variables with nonzero coefficients were screened out (Figures 2A and B). Considering the simplification of the model, we selected thirteen variables. The thirteen variables were age, drinking history, C reactive protein level, APACHE II, indwelling urinary catheter, indwelling gastric tube, mechanical ventilation, cerebrovascular disease, respiratory failure, constraint, dexmedetomidine, propofol, and midazolam.

Figure 2 Identification of significant predictors for the SSD in ICU patients. Notes: (A) LASSO coefficient profiles of the candidate predictors. (B) Selection of the optimal penalization coefficient in the LASSO regression.

Multivariate logistic regression analysis was performed on the above variables. The final logistic model incorporated eleven independent predictors: age, drinking history, C reactive protein level, APACHE II, indwelling urinary catheter, mechanical ventilation, cerebrovascular disease, respiratory failure, constraint, dexmedetomidine, and propofol. The results of multivariate logistic regression analysis are shown in Figure 3.

Figure 3 (A) The result of multivariate logistic regression. (B) The probability plots of age. (C) The probability plots of APACHE II score. (D) The probability plots of C reactive protein level.

In addition, age, APACHE II score, and C-reactive protein level were continuous variables. The probability plot was used to visualise the relationship between continuous variables and the risk of SSD. The results showed that the risk of SSD increased with the increase in continuous variables (Figure 3).

The nomogram is shown in Figure 4. In addition, a dynamic nomogram, anchored on the network, has been developed. To obtain the dynamic nomogram, please visit https://dududu.shinyapps.io/DynNomapp/. Opening the website for the first time takes 1–2 minutes. After each use, please click the quit button, otherwise an error will be reported the next time you use it.

Figure 4 Construction and validation of the predictive nomogram for the subsyndromal delirium in intensive care unit patients.

Evaluation and Validation of the Dynamic Nomogram

The AUC of the training set was 0.902 (95% confidence interval: 0.879–0.925), the best cutoff value was 0.264, the specificity was 78.4%, and the sensitivity was 88.0%, suggesting that the nomogram prediction model had excellent discrimination. The AUC of the validation set was 0.888 (95% confidence interval:0.850–0.930), the best cutoff value was 0.543, the specificity was 94.9%, and the sensitivity was 70.9%. Figure 5 shows further details.

Figure 5 (A) Receiver operating characteristic (ROC) curves of the nomogram in the training cohort. (B) ROC curves of the nomogram in the validation cohort. (C) Calibration plots of the nomogram in the training cohort. (D) Calibration plots of the nomogram in the validation cohort. (E) Decision curve analysis in the training cohort. (F) Decision curve analysis in the validation cohort.

The GiViTI calibration curve is shown in Figure 5. The two groups’ P-values in the GiViTI calibration test were 0.537 and 0.241, respectively. Thus, the predicted probability of the model was consistent with the actual probability, which suggested that the prediction model had strong concordance performance, and the calibration of the prediction model in both groups was perfect.

The DCA curves of the training and validation sets are shown in Figure 5. In addition to the red model curve, two extreme lines are shown. The grey horizontal line represents that all samples were negative, indicating that the probability of SSD was less than the threshold probability. All people did not need to accept SSD intervention, so the overall net benefit was 0. The grey curve represents the opposite, indicating that everyone must accept SSD intervention, and the net benefit was an anti-slash with a negative slope. The prediction curves of both were far from the extreme curves, indicating that the clinical effectiveness of the model was sound and patients could benefit greatly.

Discussion

Based on the ICU population, we developed and verified a model for predicting the probability of individual SSD. We reduced the dimension of SSD-related factors in ICU patients through LASSO regression, employed iterative calculation methods to construct the penalty function, and cross-validated LASSO, taking one times the standard error and excluding variables with weak correlation from the model to avoid the problem of multicollinearity. Finally, independent risk factors were determined through logistic analysis and a predictive model was constructed.

The advantage of the prediction model we constructed is that it can be used directly in combination with online tools, and directly access the website through a computer or mobile phone. The data can be directly input, which reduces the calculation difficulty of continuous variables, avoids the calculation error of ordinary nomograms, and is conducive to medical staff evaluating the probability of SSD in patients.

ROC analysis of the training set showed that the AUC was 0.902, and the AUC of the validation set for internal validation was 0.888, which indicated that the constructed model had good discrimination ability. The GiViTI calibration curve bands in the training and validation sets did not pass through the grey area, indicating that the observed probability was in good agreement with the predicted probability of the model, which ensured the repeatability and reliability of the constructed model. Thus, the prediction performance of the model was satisfactory. DCA was used to assess the model’s expected net benefit across all possible risk thresholds, thereby assessing the influence of various risk thresholds.²³ The DCA curves of the training and validation sets showed significantly better net benefits. Both curves (red) were far from the extreme (grey) curve, and almost all were above the extreme curve. Therefore, the nomogram demonstrated excellent clinical practicability.

According to the findings of our study, the total incidence of SSD among adult ICU patients amounted to 34.5%, a rate closely corresponding to that of Gao’s study (32.0%) and Azuma’s study (31.4%).^4,13 In addition, the logistic regression results were drawn into a forest plot, where the right side of the 0 scales was a risk factor for SSD, and the left side of the 0 scale was a protective factor. In this study, eleven predictors were all located on the right side of the 0 scale, that is, they were all risk factors for SSD.

In the forest plot, age, C reactive protein level and APACHE II score were continuous variables, and the OR value and confidence interval were small. However, from the probability map, we found that the risk of SSD also increased and approached 100% as the value of these variables increased. The main reason for the higher incidence of older patients is the change in neuronal morphology, white matter and substantia nigra volume reduction, and cognitive decline, especially in attention, memory and other aspects,²⁴ thereby forming the basis of delirium symptoms. APACHE II score as an indicator to determine the severity of the disease is closely related to the severity and prognosis of the disease. In previous delirium prediction models, the APACHE II score was also repeatedly reported as an independent predictor.^25,26

In addition, a cohort study found that advanced age and higher APACHE II scores influence the progression of SSD to delirium.⁷ Thus, both have a high predictive value for the occurrence of delirium symptoms. A cohort study of 380 ICU patients showed that C-reactive protein is a risk factor for developing SSD in ICU patients.⁷ Infection or tissue injury causes an inflammatory response. It activates the release of inflammatory factors, such as interleukins and interferons, which cross the blood–brain barrier and lead to central nervous system dysfunction.²⁷

One of the potential mechanisms of delirium is the change in neurotransmitters in the central nervous system. The use of indwelling urinary catheters and sedative drugs can interfere with the normal release and transmission of neurotransmitters (such as acetylcholine), leading to an imbalance of neurotransmitters and subsequently causing dysfunction in central nervous functions.²⁸ A study of 200 medical structures showed that indwelling catheters are significantly associated with delirium symptoms such as impaired attention.²⁹ Thus, we recommend reducing unnecessary catheterisation and removing the catheter as soon as possible.

In this study, sedative drugs were classified, and the effects of different sedative drugs on SSD were discussed. Dexmedetomidine, propofol, and midazolam were mainly used in the study hospital. We found that the first two are predictors of SSD. In particular, dexmedetomidine is often considered a protective factor in reducing the incidence of delirium symptoms.³⁰ However, our study found that dexmedetomidine is a risk factor rather than a protective factor for SSD. The possible reason is that dexmedetomidine protects patients at high risk of delirium from reaching the threshold but meets the SSD threshold. Sedation medications are a modifiable iatrogenic factor in the ICU. Medical staff should adhere to the relevant sedation guidelines, follow an evidence-based practice where possible, avoid excessive use of sedatives and reduce drug accumulation to promote patient recovery.

Constraint enables patients to maintain a fixed posture for a long time, increases discomfort, easily triggers negative emotions such as anxiety and anger, affects sleep quality, and leads to abnormal melatonin secretion.³¹ Melatonin secretion is a peptide hormone secreted by the pineal gland. It has many functions, such as regulating circadian rhythm, anti-oxidation and immune function.³² The abnormal secretion of melatonin leads to a disorder of the sleep cycle, which may increase the risk of SSD. In the ICU, constraints are often required to avoid adverse consequences ensure patient safety and prevent accidents.³³

However, the utilisation rate of constraints among ICU patients varies significantly from 32.9% to 59.1% in different countries or regions.^34–36 The high constraint rate in some countries or regions is due to the shortage of human nursing resources or substandard nursing procedures. Medical staff should minimise the use of constraints, such as the duration of constraints, frequency and instruments, and actively take alternative measures.

Respiratory failure is one of the common causes of ICU admission, and the incidence of SSD is also high.^9,37 A prospective cohort study found that 86% of the 821 patients admitted to the ICU due to respiratory failure or shock developed SSD, and 3/4 of them progressed to delirium.⁹ Studies have shown that decreased oxygen saturation and hypoxemia are associated with delirium symptoms.³⁸ Patients with respiratory failure usually have decreased arterial oxygen partial pressure (PaO₂). The lower the PaO₂, the more pronounced the imbalance of oxygen supply, the more serious the hypoxia of the patients, the more pronounced the disorder of the internal environment, and the lack of oxygen supply to the brain tissue, resulting in brain oedema.³⁹

Mechanical ventilation is an effective means of treatment of respiratory failure; the treatment of respiratory failure in patients has a significant effect.³⁸ Our study found that patients receiving mechanical ventilation were more likely to develop SSD. Patients with mechanical ventilation struggle to express their complex conditions due to severe illness, tracheal intubation and other reasons.³³ They are relatively easy to be ignored in the ICU, and the required intervention measures are often delayed. In addition, sound and light stimulation, noise, and mechanical ventilation lead to sleep deprivation, which is prone to anxiety, fear, anger, and other negative psychology. Mental stress increases, and the stress response occurs, resulting in continuous excitation of sympathetic nerves, increased oxygen consumption, accelerated heart rate, acute imbalance of advanced nerve centres, and induced SSD.³³ Patients with mechanical ventilation are prone to aspiration, which may lead to hypoxemia or pulmonary infection and increase susceptibility to SSD. Medical staff should withdraw the machine or remove the tracheal intubation immediately. According to the patient’s condition, high-flow oxygen inhalation or nasal catheter inhalation should be selected, and different types of respiratory failure should be given varying oxygen infusion methods.

Although drinking history is a risk factor for SSD, drinking is controllable. In a multicenter prospective cohort study in which a history of alcohol abuse was a risk factor for delirium, the risk of delirium in patients with a history of alcohol abuse was 1.657 times higher than that in patients without a history of alcohol abuse.⁴⁰ In our study, patients with a history of alcohol consumption had a 2.007 times higher risk of SSD than those without a history of alcohol consumption. Compared with delirium, drinking history has a more significant impact on SSD. There is substantial evidence that alcohol is neurotoxic to the brain.^41,42 Repeated measurements of weekly alcohol intake and cognitive ability in a 30-year large cohort study concluded that even mild or moderate levels of alcohol consumption are associated with poor brain outcomes.⁴² This finding may constitute the basis of brain function changes and increase the risk of SSD. The implication for patients is to avoid drinking as much as possible and reduce alcohol intake.

Compared with regular patients, patients with a history of cerebrovascular disease may have impaired cerebral blood circulation, brain tissue ischemia and hypoxia, and brain function decline; these events are often accompanied by changes in cognitive function, slow action, and being prone to delirium symptoms.²⁷ Thus, medical staff should pay attention to patients with a history of cerebrovascular disease, improve the alertness of SSD in such patients, and take early preventive measures if necessary.

Limitations

The study had some limitations. First of all, the assessment tools and diagnostic criteria for SSD are not uniform. The assessment tool used in this study is CAM-ICU, which is the most widely used and highly reliable delirium assessment tool. However, CAM-ICU can only assess whether the patient is SSD and cannot distinguish the severity of SSD. Due to the fact that the outcome variable of the prediction model is binary data, CAM-ICU is suitable for research on prediction models. Some of the current studies on SSD use ICDSC as the assessment tool. If follow-up studies apply the model or validation studies and adopt other assessment tools or diagnostic criteria, the results may show some deviation. Second, the frequency of daily assessment was limited. Given the volatility of delirium symptoms, the best timing of assessment may be missed and the actual incidence of SSD may be underestimated. Third, the study was developed based on ICU data from a hospital, which may limit the promotion of research results. Subsequent multi-centre external studies are needed to verify further or correct the model.

Fourth, although sedative drugs were classified, the results also showed that dexmedetomidine and propofol were risk factors for SSD, but the influence of midazolam on patients could not be excluded. In this study, only 18 patients used midazolam, which may be too small a sample size, affecting the objectivity of the results. In the future, the relationship between midazolam and SSD needs to be further studied. Fifth, in determining whether a patient has a history of mental illness, we primarily rely on the patient’s hospital records. Patients with a history of mental illness can be challenging to assess for SSD, as their symptoms can be easily mistaken for delirium. Hence, this exclusion criterion has been utilised in some studies.⁴³ In addition, we did not exclude patients with neurodegenerative diseases, postoperative patients, and the elderly population who may have dementia. Nevertheless, it is essential to note that this data collection method may introduce bias.

Conclusion

The dynamic nomogram we have developed is a web-based calculator based on the nomogram and designed using the Shiny program. It exhibits strong predictive ability and holds clinical significance. Clinical healthcare professionals can access the website through computers and other devices to assess the probability of patients suffering from SSD. In the future, the dynamic nomogram can be integrated into the medical system, thereby enhancing the practicality and usability of this tool.