Using Machine Learning Algorithms to Predict High-Risk Factors for Postoperative Delirium in Elderly Patients

Introduction

With the development of the surgical level, clinicians have fully explored the effectiveness of surgery while also discovering the serious complications that accompany patients after surgery. However, physicians often focus on the complications arising from the lesion itself, ignoring disease-related neurological complications such as postoperative delirium (POD) and impaired consciousness. POD is a common postoperative complication in elderly patients, who often have symptoms such as impaired consciousness and behavioral changes, and the disease is acute, reversible, and fluctuating.¹ POD increases postoperative mortality, prolongs hospital stays, and increases the financial burden on patients’ families.^2,3 At the same time, the cognitive function of some delirium patients is also impaired to varying degrees. Once patients develop POD, they cannot be effectively treated by drugs and other means, so currently clinicians are focusing on the prevention aspect of POD.^4,5

Some studies^6–8 have demonstrated that early interventional treatment of patients with a 30% risk of POD can effectively avoid such complications. Therefore, it becomes crucial to find the correlates that lead to the development of POD in elderly patients and to predict its occurrence. In recent years, artificial intelligence (AI) has been developing rapidly in the medical field. Machine learning, as a major branch of AI, has the advantages of more stable model building and more accurate prediction, and is favored by clinicians and is widely used in clinical prediction and other aspects. In this study, we retrospectively collected clinical information from elderly patients and applied machine learning algorithms to establish risk factor prediction models, which help clinicians to develop accurate individualized treatment plans for patients in a timely manner and improve their prognosis.

Materials and Methods

Study Subjects

In this study, we used data from a clinical database of elderly patients from Wuxi People’s Hospital, affiliated with Nanjing Medical University. Case inclusion criteria were as follows:¹ patients aged >60 years;² patients undergoing nonbrain surgery; and³ patients meeting the diagnostic criteria for delirium. The exclusion criteria were as follows:¹ patients with a clear preoperative history of psychiatric disorders;² patients with combined neurological disorders such as cognitive dysfunction;³ patients with severe organ insufficiency such as liver and kidney insufficiency;⁴ patients with clear preoperative intracranial organic lesions;⁵ patients with perioperative use of benzodiazepines and cholinergic inhibitors; and⁶ case records with missing data.

Study Design and Data Collection

A total of 30 preoperative variables (within 24 h before the day of surgery), intraoperative variables, and postoperative variables were collected. The preoperative variables collected included patient demographics (gender, age, smoking history, alcohol history, and body mass index), basic clinical characteristics (American society of anesthesiologists score and surgical history), basic medical history (anemia, diabetes, hypertension, chronic obstructive pulmonary disease, and coronary artery disease), and laboratory tests (albumin). The intraoperative variables collected included surgical approach, anesthesia method, type of surgery, operative time, intraoperative bleeding, intraoperative temperature, bispectral index value, and intraoperative blood transfusion. Postoperative variables collected included laboratory tests (albumin), postoperative stay in the intensive care unit, and postoperative complications (sleep disorders, fever, pulmonary infection, wound infection, stroke, transient ischemic attack, and POD).

Diagnosis of POD

The diagnosis of delirium in postoperative patients in this study was based on the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5),⁹ issued by the American Psychiatric Association (APA). DSM-5 contains five criteria for the diagnosis of delirium: (1) impaired attention and consciousness; (2) impaired attention and consciousness occurring over a short period of time and fluctuating; (3) additional cognitive impairment; (4) the above criteria cannot be explained by other pre-existing neurocognitive disorders; (5) the basis for delirium can be found in the patient’s medical history and examination findings. Delirium can be diagnosed in patients who meet the above five criteria.

Development and Evaluation of Predictive Models

SPSS version 26, R version 3.6.3, and Python version 3.7 were applied for the construction of clinical prediction models.¹ Univariate and multivariate regression analyses were performed. The chi-square test was applied to categorical variables to compare the differences between the two groups; a t-test was performed for continuous variables that conformed to a normal distribution; and the rank sum test was selected for continuous variables that did not conform to a normal distribution. P < 0.05 indicated that the differences were statistically significant. Logistic regression analysis was performed on the variables that were significant in the univariate analysis to obtain the independent influences of POD.² Evaluate and build the prediction model. Elderly patients from January 2015 to January 2021 were selected as the internal validation set, and elderly patients from February 2021 to June 2022 were selected as the external validation set. The internal validation set was randomly divided into a training set (70%) and a test set (30%). The independent impact factors derived from the regression analysis were incorporated into three machine learning prediction models: multilayer perceptron (MLP), extreme gradient boosting (XGBoost), and the k-nearest neighbor algorithm (KNN). The three models are evaluated by three aspects, ie, discrimination, calibration, and clinical usefulness, and the best model is selected for prediction analysis. The ROC curve was plotted to obtain an area under the curve (AUC) value and determine the predictive efficiency of the model; the calibration curve was plotted to evaluate whether the model prediction results were in good agreement with the actual results; and decision curve analysis (DCA) was plotted to evaluate the benefit of the patient after interventional treatment. The intersection of the red curve and the All curve is the starting point, and the intersection of the red curve and the None curve is the node, within which the corresponding patients can benefit. The k-fold cross-validation method was used to complete the internal validation.³ External validation of the best model using an external test set, plotting the ROC curve, and determining the generality and predictive efficacy of the model.⁴ Model interpretation. The contribution value of each feature in the sample to the prediction, ie, the Shapley value, was obtained by SHAP analysis. A SHAP summary plot was constructed, and risk factors were ranked in importance based on Shapley values.

Results

Clinical Information of the Patients

A total of 950 patients were included in the study, including 131 (13.79%) patients with POD (Figure 1, Table S1).

Figure 1 Flow diagram of patients included in the study.

Risk Factor Analysis of the Patients

The results of the univariate analysis showed that there were significant differences (P<0.05) between the delirium group and the nondelirium group in terms of age, smoking history, history of alcoholism, history of hypertension, history of chronic obstructive pulmonary disease (COPD), time of surgery, and postoperative stay in the intensive care unit (ICU) as suspected influencing factors of postoperative delirium. The indicators with significant differences in the univariate analysis were included in the multivariate analysis, and the results showed that age, history of smoking, history of alcoholism, history of hypertension, history of COPD, time of surgery, and postoperative stay in the ICU were independent influencing factors for POD (P<0.05) (Table 1).

Table 1 Univariate and Multivariate Analyses of Variables Related to Postoperative Delirium

Model Building and Evaluation

The ROC curve results showed that XGBoost had an AUC value as high as 0.982 in the training set; the AUC value in the validation set was 0.924, both of which performed the best among the three models (Figure 2A and B, Table 2). The results of the calibration curves showed that the calibration curves of the two models were similar to the ideal curves except for MLP, and there was high agreement between the predicted and actual results of the models (Figure 2C). The DCA curves achieved a net clinical benefit relative to the full treatment or no treatment plan for all models except for MLP (Figure 2D). The k-fold cross-validation method was applied to the three models, the test set N=255 cases (30.04%) was taken, and the remaining samples were used as the training set for 10-fold cross-validation. The results show AUC=0.9283±0.0745 for the XGBoost validation set, AUC=0.9421 for the test set, accuracy=0.9294; AUC=0.9023±0.0981 for the KNN validation set, AUC=0.8949 for the test set, accuracy=0.9333; AUC=0.5060± 0.2025 for the MLP validation set, AUC=0.3143 for the test set, accuracy=0.2275 (Figure 3A–C). The prediction model was built using XGBoost after the combined comparison.

Table 2 Evaluation of the Three Models

Figure 2 Evaluation of the three models for predicting postoperative delirium. (A) ROC curves for the training set of the three models. (B) ROC curves for the validation set of the three models. (C) Calibration plots of the three models. The 45° dotted line on each graph represents the perfect match between the observed (y-axis) and predicted (x-axis) complication probabilities. A closer distance between two curves indicates greater accuracy. (D) DCA curves of the three models. The intersection of the red curve and the All curve is the starting point, and the intersection of the red curve and the None curve is the node within which the corresponding patients can benefit.

Figure 3 Internal validation of the XGBoost model. (A) ROC curve of the XGBoost model for the training set. (B) ROC curve of the XGBoost model for the validation set. (C) ROC curve of the XGBoost model for the test set. (D) External validation of the XGBoost model.

Model External Validation

The results of the ROC curve showed that the AUC value of the external validation set was 0.880, indicating that the prediction model was highly accurate in determining the disease (Figure 3D).

Model Explanation

The SHAP summary plot results showed that the seven risk factors for postoperative delirium were ranked in order of influence as time of surgery, history of hypertension, postoperative stay in ICU, history of alcoholism, history of smoking, history of COPD, and advanced age (Figure 4).

Figure 4 SHAP summary plot. Risk factors are arranged along the y-axis based on their importance, which is given by the mean of their absolute Shapley values. The higher the risk factor is positioned in the plot, the more important it is for the model.

Discussion

In this study, XGBoost was used to construct the prediction model, which has the characteristics of high efficiency, flexibility, and extrapolation. The XGBoost algorithm also takes into full consideration the regularization problem and reduces the computational effort, thus it can effectively avoid model overfitting. The prediction model of POD in elderly patients has at least two purposes; on the one hand, the model can clarify the risk factors for the occurrence of POD; on the other hand, it can prompt clinicians to take timely interventions for high-risk patients to reduce the risk of postoperative death. In this study, SHAP analysis was used to interpret the model, and the results showed that operative time ≥270 min, history of hypertension, postoperative stay in ICU, history of alcoholism, history of smoking, history of COPD, and advanced age were risk factors for POD.

The incidence of POD in elderly patients in different studies also varies. On the one hand, this is because the investigators did not adopt uniform diagnostic criteria for POD, and on the other hand, the patients included in the study had different clinical characteristics. Choi et al¹⁰ concluded that the incidence of delirium in patients was 26.1%, while the incidence in the current study was 13.79%. The mechanism of POD is still unclear, but the numerous risk factors that induce delirium are mainly reflected in three aspects. One of them is the patient’s preoperative physical status. Physiological changes in the cardiovascular and respiratory systems can increase the risk of POD, and certain psychological disorders can also cause related complications to a certain extent. Secondly, the patient’s intraoperative bleeding, operating time and anesthesia. Finally, there is the postoperative care and recovery of the patient. Therefore, this study analyzes and discusses these three aspects, and builds a model based on machine learning to predict the occurrence of POD.

In the preoperative aspect, the results of this study found that elderly patients with a history of hypertension and COPD were more likely to have POD. The blood vessels of hypertensive patients are under high pressure for a long time, and the walls of the vessels have fewer elastic fibers and more collagen fibers. The walls of cerebral vessels are sclerotic and poorly expandable, which affects the blood supply to the brain. At the same time, surgery as a special form of trauma can further cause a decrease in the patient’s blood volume and affect the circulatory metabolism of the brain. On the other hand, patients with hypertension have their cerebrovascular autoregulation function partially affected and are more sensitive to changes in blood pressure, and such patients are more likely to induce hypoxic-ischemic brain injury. All these intracranial pathological changes increase the risk of POD to some extent. Elderly patients with COPD experience the same pathological changes. Patients have poor lung function and are more prone to hypoxia and carbon dioxide storage. On the one hand, because the brain is highly sensitive to carbon dioxide, the elevated concentration of carbon dioxide in the cerebral vasculature will inhibit the synthesis of neurotransmitters such as acetylcholine, causing a weakening of the regulatory mechanism of the internal environment homeostasis and making it more prone to neurological complications;^11–13 on the other hand, the increase of carbon dioxide concentration in the blood will also stimulate the hematopoietic system, causing an increase of blood viscosity and a decrease of oxygen transport capacity of the red blood cells, which will eventually affect the normal physiological function of the patient’s brain cells and make the patient more likely to have delirium after surgery. The normal physiological function of the brain cells of the patient will be affected, and the patient will be more likely to have POD. In addition, several studies^4–6 have demonstrated that elderly patients are more likely to have POD, which is consistent with the results of this study. It is clinically believed that this is mainly related to the atrophy and functional degeneration of the brain in elderly patients. Thomas suggested that these physiological changes are the cause of postoperative delirium and tremor.¹⁴ The preoperative risk factors included in this study suggest that patients with poor underlying conditions such as hypertension and COPD should be strictly monitored during the perioperative period, and medications should be taken regularly to control blood pressure and blood glucose. Appropriate psychological counseling should be given to patients to prevent fluctuations in blood pressure caused by sympathetic excitation, and antibiotics can be applied prophylactically to prevent postoperative pneumonia as well as wound infection. In some studies,¹⁵ it was considered that the high-risk group for POD was male patients, but in the univariate and multivariate regression analysis of the current study, male patients were not independent risk factors for delirium, while history of smoking and alcohol abuse were independent risk factors. Therefore, the author deduced that the reason for the prevalence of POD in male patients should be closely related to their long-term smoking, alcohol abuse and life stress. Alcohol affects dopamine and its receptors, which in turn inhibits the physiological functions of the nervous system. Also, such patients often have withdrawal reactions triggered by abstinence from alcohol during the perioperative period, which predisposes them to postoperative psychiatric symptoms such as tremor and delirium. On the other hand, Tatsuya et al^16,17 concluded that melatonin can effectively prevent neurological symptoms such as delirium, whereas unhealthy lifestyles such as smoking and alcohol abuse can cause disruption of melatonin secretion in patients, which in turn increases the likelihood of postoperative delirium. It is suggested that clinicians should provide health education to patients before surgery, adjust their living habits, and ensure that patients get sufficient rest and sleep. Strictly control the patient’s perioperative medication to prevent the occurrence of withdrawal reactions. At the same time, the attending physician can conduct a delirium risk assessment of the patient early on in conjunction with a neurologist. Low-dose haloperidol can be applied prophylactically to patients at higher risk. Such antipsychotic drugs have a protective effect on nerve cells while blocking dopamine and are effective in preventing POD.^18–20

In terms of intraoperative aspects, the present study found that longer operative time was an independent influencing factor for POD. Some elderly patients have poor organismal conditions, and the prolongation of the operation time further increases the damage of the organism. The organism produces a large amount of inflammatory factors in response to strong traumatic stress, increasing the permeability of the blood-brain barrier and disrupting the normal immune and nervous systems of the organism.²¹ Moreover, the longer the duration of surgery the more likely it is to cause intraoperative trauma, raising the risk of increased intraoperative bleeding. Excessive intraoperative blood loss can cause a decrease in the patient’s hemoglobin and a decrease in the blood’s ability to transport oxygen, causing different degrees of neurological damage.^22–24 On the other hand, a longer operative time means longer anesthesia and more anesthetic drugs. Human cognitive and behavioral functions are related to neurotransmitter transmission, and certain anesthetic drugs can cause neurological dysfunction by affecting the balance of transmitter transmission, triggering POD.²⁵ Moreover, some studies have pointed out that anesthetic drugs can also cause systemic hemodynamic abnormalities in patients, with the most significant intracranial changes.²⁶ Some patients do not show hypotension after anesthesia, but their cerebral blood perfusion decreases and their intracranial pressure drops significantly.^26,27 Meanwhile, compared with young patients, elderly oncology patients metabolize anesthetic drugs more slowly, which means that such patients are more likely to have concurrent toxic side effects of anesthetic drugs. In summary, it is clear that prolonged surgery can cause multiple pathological changes in patients, and multiple factors work together to cause the development of POD. This suggests that the surgical team should reasonably plan the surgical plan before surgery and give preference to minimally invasive surgical approaches, thus reducing patient trauma and postoperative stress. Intraoperatively, surgeons work together and cooperate appropriately to improve surgical efficiency. While ensuring the quality of surgery, the operating time is shortened and the amount of surgical bleeding is reduced. At the same time, the surgeon should follow the principle of no tube, reduce braking treatment, and avoid the patient’s discomfort reaction due to the retention of gastric tube, drainage tube and other tubes. There is no unified clinical opinion on the relationship between the mode of anesthesia and postoperative delirium. One study²⁸ concluded that local anesthesia, as opposed to general anesthesia, prevents patients from developing neurological symptoms such as POD. In contrast, Sivarajah²⁹ concluded that the mode of anesthesia is not a risk factor for the development of postoperative delirium, which is consistent with the results of the current study. Moreover, Radtke et al³⁰ applied a depth of anesthesia detector (BIS) to monitor the changes of indicators in surgical patients based on this study, and found that patients with deep anesthesia were more prone to delirium postoperatively. It suggests that anesthesiologists should use simple anesthesia for patients as much as possible intraoperatively to reduce the depth of anesthesia and reduce the possibility of nerve cell damage. On the other hand, anesthesiologists are also required to closely monitor changes in various indicators such as patient temperature and blood oxygen saturation to detect abnormalities in a timely manner and facilitate early intervention.

In the postoperative aspect, several postoperative complications were included in this study to explore the risk factors for POD. No correlation was found between postoperative complications and POD, but it was found that patients admitted to the ICU after surgery were more likely to develop delirium. On the one hand, some elderly patients are admitted to ICU due to multi-organ failure, and the inflammatory response of patients is more intense, which can easily trigger neurological dysfunction. Pro-inflammatory factors can increase the permeability of the blood-brain barrier and stimulate microglia activation, causing abnormal neurotransmitter release, which in turn triggers postoperative delirium.^31–33 In addition, Maclullich et al³³ suggested that postoperative delirium in ICU patients is closely related to the high level of corticosteroids in the patient’s body, and the noise of the instruments in the ICU ward and the frequent operations of the medical staff are likely to trigger a stress response in the patient and stimulate abnormal glucocorticoid secretion in the body. Fat-soluble glucocorticoids can cross the blood-brain barrier, and long-term accumulation can cause cognitive dysfunction in patients.³⁴ It is suggested that the medical staff in ICU should promptly assess delirium in awakened postoperative patients and intervene early to treat patients with high scores. Moreover, a study³⁵ pointed out that the prodromal manifestation of some delirium patients is insomnia, so the monitoring of such patients should be focused on giving appropriate care to alleviate negative emotions such as fear and anxiety and to prevent patients from stress reactions.

In recent years, several predictive models related to POD have been successfully constructed. For example, Ouimet³⁶ suggested that the important risk factors for POD were a history of alcohol abuse and hypertension, and we incorporated more preoperative, intraoperative, and postoperative risk factors for analysis to obtain more accurate conclusions. In addition, previous studies^37,38 have often used parametric regression to construct models. Clinical characteristic variables have intricate relationships with each other, regression models are built based on linear relationships between variables, and the prognosis of patients cannot be accurately predicted using regression models alone. The study builds models based on machine learning algorithms to meet the needs of actual clinical decision making and more accurate prediction of diseases. On the other hand, neurological disorders such as POD in elderly patients are gradually becoming a more serious public health problem worldwide, which to a certain extent increases the global medical burden. The predictive model developed in this study can screen high-risk patients for postoperative delirium without using imaging examinations (eg, cranial computed tomography, etc.), which greatly reduces medical costs for hospitals. Of course, after the model identifies high-risk patients, clinicians will monitor such patients in more detail and individualize their treatment to improve their survival rates.

Limitations

This study also has some limitations. Although multiple aspects of risk factors were included, we did not focus on aspects such as laboratory indicators. Second, the study only included data from a single center, and the sample size of the study was small, which may have distribution bias. It is necessary to include case data from more regions in future studies to further improve the study.

Conclusion

In this study, a model based on XGBoost machine learning algorithm was developed to predict the risk of delirium in patients after surgery. The model has good prediction accuracy and clinical utility, which facilitates physicians to diagnose and treat patients in a timely manner and improves the prognostic outcome of patients. The model pointed out that postoperative delirium remains one of the major problems affecting patients’ prognosis and is multifactorially associated with age, history of smoking, history of alcohol abuse, COPD, history of hypertension, prolonged surgery time and postoperative admission to ICU.