Development and Validation of a Postoperative Delirium Prediction Model for Patients Undergoing Gastrointestinal Surgery

Introduction

Postoperative delirium (POD) refers to a variable and transient impairment of consciousness and cognition, defined by a sudden onset of inattention, disorganized thought, and changes in the levels of awareness. It is a major complication after gastrointestinal surgery, especially in elderly patients,¹ with prevalence of POD varying from 8.2% to 54.4%.^2,3 POD is associated with an increased incidence of complications, higher mortality rates, longer hospitalization, higher healthcare expenditure, and potential long-term cognitive impairment.^4,5 This issue of early detection of patients at risk was particularly critical to clinical practice because it allows implementing effective interventions to prevent POD.

Even though substantial progress has been made in the development of POD risk assessment tools for abdominal surgery,^6,7 there is still a substantial gap in research on the topic of gastrointestinal surgery. Existing abdominal surgery prediction models that usually incorporate multimodal factors are characterized by complexity and non-specificity to patients undergoing gastrointestinal surgery. Intraoperative factors are included in some models, which are hard to control due to inherent clinical variability. Models, based on intraoperative or postoperative variables, not exclusively on preoperative variables, can relatively poorly predict POD risk preoperatively. Excessive variable inclusion in most models further impedes routine clinical application.⁸ Studies of gastrointestinal surgery have mainly focused on the identification of risk factors individually instead of developing predictive models that incorporate the risk factors.⁹

To fill these research gaps, we carried out a retrospective cohort study, in which we developed and validated a parsimonious, preoperative prediction model of POD in patients undergoing major elective gastrointestinal surgery, intended to present a clinically useful tool, which will surpass the complexity and poor specificity of available models.

Methods

Ethics Approval

This retrospective single-center study was approved by the Ethics Committee of Ruijin Hospital, Shanghai Jiao Tong University School of Medicine on May 13, 2021 (Ethics Approval Number: (2021) No. 86; Registration No.: ChiCTR2100047405, Chinese Clinical Trial Registry). Since the retrospective design and the anonymized patient data were used, the Ethics Committee specifically requested an informed consent to be waived as part of the approval. All procedures were conducted in compliance with Declaration of Helsinki (World Medical Association, 2013), local privacy regulations and institutional policies. Sensitive information was kept in a password-protected database that could only be accessed by the authorized personnel.

Patients and Study Design

Between July 2019 and December 2020, 417 patients aged ≥18 years who underwent elective major gastrointestinal surgery (open or laparoscopic approach) at our center were initially screened. Of these, 369 met the inclusion criteria and were included in the final analysis. The inclusion criteria were: (1) elective gastric and colorectal surgery; (2) American Society of Anesthesiologists (ASA) physical status classification I–IV; (ASA I: a healthy patient; ASA II: a patient with mild systemic disease; ASA III: a patient with severe non-incapacitating systemic disease; ASA IV: a patient with incapacitating systemic disease posing a constant threat to life). (3) no prior history of delirium; (4) no intraoperative serious adverse cardiovascular events. Exclusion criteria included: (1) preoperative cognitive impairment and severe mental illness; (2) severe cardiovascular and cerebrovascular diseases history; (3) immediately postoperative ICU admission; (4) re-operation; and (5) incomplete medical history records. A detailed flowchart of the sample selection process is shown in Figure 1.

Figure 1 Study flow chart.

Data Collection

The Confusion Assessment Method (CAM) was used to diagnose POD, which is characterized by acute onset of inattention, fluctuating consciousness, and cognitive impairment.¹⁰ Assessments were performed twice per day and also to the case of acute changes in mental status occurring during the postoperative days (1 to 7 days).

The hospital information system (HIS) was used to extracted data retrospectively. The demographic variables included age, sex and educational level. Clinical variables included body mass index (BMI), smoking and alcohol use history, psychiatric and cerebrovascular diseases history, habitual use of sleeping drugs, and the American Society of Anesthesiologists (ASA) physical status classification. HIS also had preoperative laboratory data, such as albumin, hemoglobin, glucose, white blood cell count, total bilirubin, urea, creatinine, sodium, and phosphorus levels. The anesthetic records provided intraoperative variables, such as surgical site, intensive care unit (ICU) admission, duration of operation, urine output, total fluid intake, blood loss, blood transfusion, and exposure to medications (including benzodiazepines, opioids, and muscle relaxants).

Statistical Analysis

The present study was a retrospective observational research. The patients were classified into POD and non-POD according to Confusion Assessment Method (CAM) diagnosis during postoperative days 1 to 7. Univariate logistic regression was used to identify candidate variables associated with POD. Variables showing statistical significance in the univariate analysis (p < 0.05) were subsequently included in multivariate logistic regression to develop the prediction model. A nomogram was constructed for model visualization.

The receiver operating characteristic curve (AUC) with 95% confidence intervals (CI) was used to determine model performance. The cut-off was optimized to maximize the Youden index and the sensitivity, specificity and Youden’s index were presented. Goodness-of-fit was assessed using the Hosmer–Lemeshow test. Internal validation carried out with 5-fold cross-validation (repeated 20 times), in which the dataset was split into 4 training sets and 1 validation subset.

All statistical tests were 2-sided, with statistical significance set at P < 0.05. R software (version 3.6.1, Lucent Technologies, USA) was used to perform statistical analyses.

Results

Study Population

Of the 417 patients initially screened for elective major gastrointestinal (GI) surgery, 369 patients were included in the final analysis (Figure 1). Among them, 22 (6.0%) developed POD (POD group) and 347 (94.0%) did not (non-POD group). The baseline features of the 369 comprised patients are described in Supplementary Table 1. Its median age was 62.0 years (20–86 years), with 226 (61.2%) males. Over 67% of patients completed junior high school or above education,17.6% (65/369) were current smokers (defined as ≥ 10 cigarettes/day for 3 consecutive years), and 17.9% (66/369) reported regular alcohol use (≥200 mL/week). Comorbidities conditions consisted of cerebrovascular disease history (11.4%, n=42) and increased preoperative serum glucose (18.7%, n=69). The overall incidence of POD was 6.0% (22/369). Notably, POD risk increased significantly with age: 2.75% (6/218) in patients aged <65 versus 10.60% (16/151) in those aged ≥65 years (P<0.05). In terms of surgery type, 284 (77.0%) patients were subjected to gastric surgery, and 85 (23.0%) to colorectal surgery.

Risk Factors Associated with POD

We evaluated variables related to POD risk. Univariate analysis identified four preoperative factors significantly linked to POD: older age (odds ratio [OR] 4.19, 95% confidence interval [95% CI] 1.68–11.91; P = 0.004), lower education level (OR 0.25, 95% CI 0.10–0.60; P = 0.002), history of cerebrovascular diseases (OR 6.59,95% CI 2.55–16.47; P < 0.001), and elevated preoperative serum glucose level (OR 2.68 95% CI 1.03–6.54; P = 0.034). Of them, only lower education level (OR 0.26, 95% CI 0.09–0.65; P = 0.005) as well as history of cerebrovascular disease (OR 4.88 95% CI 1.73–13.51; P = 0.002) remained statistically significant in the multivariate analysis (Table 1). No intraoperative factors showed significant association with POD in univariate analysis (Table 2).

Table 1 Association of Preoperative Patient Characteristics with Postoperative Delirium

Table 2 Association of Intra-Operative Patient Characteristics with Postoperative Delirium

Development of the POD Prediction Model

The POD prediction model was developed using four preoperative factors identified as statistically significant in univariate analysis (Table 3). These variables were combined into a multivariable logistic regression model, and a nomogram was developed to visualize the prediction model (Figure 2). The performance of the model was assessed by the analysis of a receiver operating characteristic (ROC) curve with an area under the ROC curve (AUC) of 0.810 (95% CI: 0.700–0.921). The model exhibited Youden’s index of 0.500, sensitivity of 0.863, and specificity of 0.634, which showed strong discrimination. The Hosmer–Lemeshow test was also used to estimate the calibration of the model, and the result was non-significant (P=0.436), indicating good agreement between predicted and observed probabilities (Figure 3).

Table 3 Coefficients of the Selected Prediction Model

Figure 2 Nomogram model for predicting the risk of POD in patients underwent gastrointestinal surgery.

Figure 3 AUC curve for the POD prediction model.

POD Prediction Model Validation

In order to evaluate the generalizability of the developed prediction model, we performed internal 5-fold cross-validation 20 times with 20 validation sets. We calculated AUC for each set of validation. The average cross-validation AUC was 0.774 (95% CI: 0.584–0.965) and was rather similar to the AUC of the entire dataset (0.810), which proved the model’s stable performance.

Clinical Use

The following formula may be used to calculate the model-predicted probability of POD:

logit (P) = −1.640 + 0.744 × advanced age − 1.366 × lower education level + 1.585 × cerebrovascular disease history + 0.748 × elevated preoperative serum glucose.

(Note: Binary variables will be coded 1=presence, 0=absence).

The ideal cutoff of 0.162 identifies those whose predicted probability of acquiring delirium exceeds 0.162 as having high-risk delirium. At this cutoff, the model had sensitivity of 0.909 and specificity of 0.579, and higher predicted probabilities are associated with more POD risk.

Discussion

As a retrospective study, we developed and validated a preoperative prediction model on postoperative delirium (POD) specifically for patients after elective gastrointestinal (GI) surgery. The model uses four readily available variables, which are advanced age (≥65 years), lower education level, a history of cerebrovascular disease, and elevated preoperative serum glucose levels. It showed strong discriminative capacity (AUC 0.810) and good calibration, and its performance was the same on internal validation with a mean cross-validation AUC of 0.774. To our knowledge, this is the first easy-to-use, preoperative, GI surgical population POD prediction tool.

Interpretation of Predictors

The model promotes a lot of well-known risk factors. Advanced age is indicative of the increased susceptibility of the aging brain. Having a history of cerebrovascular disease implies the presence of neural compromise and signifies prior injury, which will lower the threshold of subsequent delirious attacks. Elevated preoperative glucose level may cause a pro-inflammatory condition that promotes neuronal dysfunction. In our final model, a lower educational level was independently associated with POD. This association was confirmed by a sensitivity analysis which proved that the elimination of this factor decreased the model’s AUC from 0.810 to 0.727, thereby confirming the specificity of the predictive power. This finding is in line with previous evidence indicating that lower educational level increases the risk of POD,^11,12 while higher educational level offers a protective effect.¹³ The inclusion of educational level as an independent factor in POD prediction nomogram for hip arthroplasty further supports its clinical significance in risk assessment.¹⁴

The primary explanatory factor is the cognitive reserve hypothesis,¹⁵ that the higher educational level, the better the neural efficiency and adaption to the perioperative stress. Moreover, education level is an applicable surrogate of health literacy and self-management capacity,^16,17 potentially supporting health-promoting behaviors, such as proper lifestyle habits and effective patient-clinician communication. All these aspects have the potential to enhance compliance to some of the most important perioperative practices such as glycemic control and early mobilization, thus indirectly lowering the risk of delirium. The overall incidence of POD in our cohort was low at 6.0%, which may be attributed to the broader age group (59% below 65 years of age) and higher educational level compared to studies conducted in the high-risk groups of the elderly.^18,19

Our model integrating educational background to assess cognitive resiliency and health habits with traditional clinical variables would offer a more comprehensive preoperative risk assessment, enabling more proactive and more personalized preventive actions.

Comparison with Existing Models

The main strengths of our model are its preoperative exclusivity and parsimony. This restrictive attention on preoperative variables coincides with our elective, comparatively lower-risk cohort, where patient susceptibility at the time of baseline is a more important factor than usual intraoperative variability. This distinguishes it from many existing tools.

First, models based on high-performing ICU populations frequently depend on postoperative data (eg, SOFA score) or complex intraoperative measures (eg, duration of mechanical ventilation), restricting preemptive risk stratification.^20,21 Second, other models for abdominal surgery incorporate intraoperative scores such as the Surgical Apgar Score, therefore postponing risk stratification until the postoperative period.¹⁸ Third, our four-factor nomogram is parsimonious, which is less than five minutes to compute using standard record, compared to comprehensive but complex instruments such as POSSUM or models based on subjective questionnaires (eg, IQCODE). Lastly, populations-specific models (eg, orthopedic or hip fracture patients) contain risk factors (eg, prolonged immobilization, osteoporosis) that are not consistent with the primary comorbidity profile of GI surgical patients (eg, metabolic disorders, cerebrovascular disease).²²

Therefore, by uses only four readily available preoperative variables, our model offers a rapid point-of-care risk assessment tool specifically designed to address the needs of the elective GI surgical population, filling a key gap in clinical practice.

Clinical Implications

Since our model detects risk depending on preoperative patient vulnerability, it allows proactive interventions. The main clinical benefit is its feasibility: all variables can easily be extracted from electronic health records, and risk stratification can be conducted within a few minutes during the preoperative assessment. The tool is especially appropriate in high-volume surgical centers, due to the efficiency of this approach supported by an intuitive visual nomogram.

In high-risk patients group, as defined by our model, we recommended the use of a specific, multicomponent preventive bundle. These include preoperative education, careful glycemic control, maintenance of cerebral perfusion pressure and oxygenation, avoidance of deliriogenic medication during operation, as well as structured postoperative interventions (eg, frequent reorientation, sleep hygiene and early mobilization). With this combination strategy, we are likely to improve personalized and anticipatory care while optimizing the allocation of preventive resources.

Limitations

This study has limitations. To begin with, its retrospective, single-center nature, moderate sample size and a relatively small numbers of POD events (n=22) restrict its statistical power. Second, despite the fact that standard CAM criteria were involved in the process of POD assessment, the retrospective nature of data collection is likely to have missed short episodes. Most importantly, only internal validation was performed. Prospective, multicenter cohorts external validation is necessary. Future studies should be directed to such validation, intervention trials to assess the clinical impact of the model, and investigation of the biological pathways between socioeconomic variables including education and POD susceptibility.

Conclusions

We developed and initially validated a parsimonious, preoperative POD prediction model on GI surgery patients. The model is highly clinically practical and predictive as it involves age, educational level, cerebrovascular disease history, and preoperative glucose, which allows early detection of high-risk patients and facilitates a transition to personalized preventive measures.