Development and Validation of a Delirium Risk Prediction Model for Elderly Patients Undergoing Elective Orthopedic Surgery

Background

Post-operative delirium (POD) is a common neurocognitive disorder after surgery characterized by acute and fluctuating impairment in attention and awareness, along with disorganized thinking.¹ The incidence of POD varies based on surgical procedures and ranges between 3.6% and 28.3% in orthopedic patients,² even up to a 70% in patients who underwent femoral neck surgery.^2,3 It is most prevalent in older patients, those with existing neurocognitive disorders, and those undergoing complex or emergency procedures. Delirium in hospitalized patients is associated with poor cognitive and functional recovery, increased morbidity and mortality, resulting in longer hospital stays and greater healthcare resource expenditure.⁴ Furthermore, POD is considered a harbinger of early POCD, which has a negative impact on the long-term health of the patient.⁵ POD holds substantial public health relevance as a target for interventions to prevent its associated burden of downstream complications and costs.

To date, the exact pathophysiology and etiology of POD remain unclear, and no novel treatments are being proposed and developed. Also, there are currently limited treatment options available for clinical use. It is encouraging that multi-component interventions targeted at risk factors have been shown to reduce delirium in hospitalized patients and 30–40% of cases could be preventable.⁶ Therefore, prevention is the most effective strategy for managing POD, with a risk assessment being the first step.

Previous studies have determined several risk factors for POD in elderly elective orthopedic surgery patients, including predisposing factors such as pre-existing dementia, drug or alcohol abuse, and abnormal sodium and potassium levels as well as precipitating factors such as postoperative pain, surgery, anesthesia, and hypoxia.^7–9 As the cause of POD is multifactorial, a single risk factor cannot predict the risk of POD. Although some researchers have developed risk prediction models, the effectiveness of these models in predicting outcomes varies greatly, and the majority of them are only applicable to patients undergoing hip fracture surgery.^10,11 Up to now, no predictive model has been available to all patients undergoing elective orthopedic surgeries. Considering the constantly growing orthopedic population, especially in China, with over 4.7 million by 2022, a clinical prediction model which could combine multiple predictors to provide insight into the relative effects of predictors in the model and absolute risk estimates for individual patients is still lacking.

A nomogram is a graphical predictive model that incorporates clinical variables and allows the user to predict the risk of a specific event quantitatively. Our study aimed to develop and validate a nomogram-based preoperative clinical prediction model to estimate the probability of delirium in elderly Chinese patients undergoing elective orthopedic surgery.

Materials and Methods

Study Design

The prospective observational study was designed according to the framework proposed by Ewout W et al.¹² We first developed the prediction of delirium for elderly orthopedic surgery patient model based on a prospective cohort study, in which the consecutive elderly orthopedic surgery patients in the affiliated Yixing Hospital between March 2021 and May 2022 were recruited. Various orthopedic surgery types, such as spinal trauma surgery, cervical/lumbar spine disease, extremity fracture surgery, arthroplasty, and arthroscopic surgery were included and the anesthesia protocol, eg general anesthesia, nerve blocks, and epidurals was determined by the attending anesthesiologist dependent of the patient conditions. Bootstrap methods were then used to demonstrate the model’s reproducibility for internal model validation.¹³ We next externally validated the delirium prediction model with data from Wuxi Second People’s Hospital between January 2022 and May 2022.

This study was approved by the ethics committee of Affiliated Yixing Hospital of Jiangsu University (approval number IRB-2022-ARTICLE-107) and performed consistent with the principles of the Helsinki Declaration on Human Experimentation. All participants provided written informed consents.

Patients

The following were inclusion criteria: (a) patients scheduled for elective orthopedic surgery; (b) age range from 65 to 80; (c) expected to stay in the hospital for at least 48 hours after surgery. We excluded patients if they were diagnosed with pre-existing delirium, surgical suspension and postponement, severe visual or hearing impairment, preoperative coma, or mechanical ventilation, data missing, and a decline to participate in the study.

Identification of Risk Factors

A literature search was done using the keywords “orthopedic surgery AND POD AND risk variables” to find potential risk factors for the onset of delirium in patients who had undergone orthopedic surgery. Following that, 46 potential risk factors were identified and examined to construct a preoperative predictive model.

Study Protocols and Data Collection

All patients were interviewed the day before surgery for the demographic characteristics, ie, age, gender, BMI, education level, American Society of Anesthesiologists Physical Status Classification (ASA class),¹⁴ and comorbidities, such as, high blood pressure(HBP), diabetes mellitus(DM), coronary heart disease(CHD). Mini-Mental State Examination (MMSE)¹⁵ and Pittsburgh sleep quality index (PSQI) in a Chinese version were used by an independent team member to assess cognitive function and sleep disorder respectively. Pre-operative laboratory tests associated with delirium, including hemoglobin(Hb), white blood cells(WBC), blood glucose(Glu), glycosylated serum protein(GSP), glycosylated hemoglobin (HbA1c), lipoprotein A(LPA), apolipoprotein E(APOE), apolipoprotein B(APOB), high density lipoprotein(HDL), low density lipoprotein(LDL), serum creatinine(SCR), sodium(Na⁺), potassium(K⁺), calcium(Ca⁺), C-reactive protein(CRP), albumin(Alb), total cholesterol(TC), glomerular filtration rate(GFR), blood urea nitrogen(BUN), direct bilirubin(DBil), indirect bilirubin(IBil), sedimentation(ESR), D-dimer, and ejection fraction(EF) were stratified on the basis of our hospital laboratory test normal range. The peri-operative data consisted of the type and duration of surgery and anesthesia, the blood loss and transfusion as well as the delirium-related drugs use (dexmedetomidine, benzodiazepine, and atropine). All laboratory results and peri-operative parameters were collected from each patient’s electronic medical records.

Diagnosis and Treatment

Delirium was diagnosed using the Confusion Assessment Method (CAM),¹⁶ which was developed as a screening instrument to detect delirium based on diagnostic and statistical manual of mental disorders (DSM)-III-R criteria for the use of non-psychiatric clinicians with high sensitivity (94–100%) and specificity (90–95%).¹⁶ CAM includes four characteristics: (A) a sudden start and variable mental state, (B) inattention, (C) disordered thinking, and (D) a change in the degree of consciousness. When characteristics (A) and (B) are met fundamentally and (C) or (D) selectively, delirium is diagnosed.¹⁷ The POD diagnosis team comprised two researchers and one professional psychiatrist. Each researcher visited patients twice daily (8:00 am and 16:00 pm) from post-operative day 1 to 7. Once both agreed each other, the judgement was made. If the diagnosis from these two researchers were inconsistent, the final diagnosis was confirmed by the professional psychiatrist. If the patient was discharged within 7 days after surgery, the researchers assessed the patients for delirium by phone or in a scheduled in-person interview.

Statistical Analysis

R software was used to statistically analyze all the data.

To determine the association between patient characteristics and delirium, we utilized univariate analysis. To evaluate variations in patient characteristics between groups, categorical variables were subjected to a chi-square test or Fisher’s exact test. Continuous variables in the data were categorized according to clinical need. These choices were made in advance of the modeling.

Using the least absolute shrinkage and selection operator (LASSO) method,¹⁸ the best predictive characteristics data from the collected data on potential risk factors for the occurrence of POD in elderly patients undergoing elective orthopedic surgery were chosen. Then predictors with nonzero coefficients in the LASSO regression model were chosen. Prediction models were then developed by combining binary logistic regression analysis with the best predictors.¹⁹ The characteristics were taken into account as odds ratios (OR) with a 95% confidence interval (CI) and as a P-value. The probability of POD in elderly individuals following elective orthopedic surgery was predicted using all factors.

To evaluate how well the delirium prediction nomogram performed, the calibration curve was plotted. After a significant test statistic showed that the model’s calibration was not perfect, we created the C-index.²⁰ After that, we must perform a bootstrap validation of the delirium nomogram, which entails selecting some data at random from all of the data for validation to calculate the C-index obtained from the calibration, and finally, we perform a clinical decision curve analysis to determine the actual validity of the delirium nomogram.

Results

Patients’ Characteristics

474 of the 523 individuals in the primary cohort who underwent elective orthopedic surgery during the study period were eligible for inclusion and were thus part of the study. 49 individuals were eliminated from the research as a result of 12 patients having their surgeries postponed or suspended, 3 patients having preoperative coma, 2 patients having preexisting delirium, 25 patients with missing data, and 7 patients refusing to participate. For the validation cohort, we finally included 153 homogeneous patients after excluding 12 patients met the exclusive criteria in a total of consecutive 165 patients. Within seven days following surgery, delirium was diagnosed in 58 (12.2%) patients in the primary cohort and 19 (12.4%) patients in the validation cohort. A study flowchart diagram is shown in Figure 1. The characteristics of possible risk factors for delirium in the primary and validation cohorts are listed in Table 1.

Table 1 Demographics and Clinical Parameters of Patients in Primary and Validation Cohort

Figure 1 Flow Chart.

Feature Selection

From each patient, a total of 46 risk indicators were taken. 46 variables were utilized in the main cohort of the LASSO model to determine the key risk factors associated with POD in elderly patients receiving elective orthopedic surgery. The results revealed that a total of six variables with nonzero coefficients were screened out when the lambda value was selected as lambda. min (0.02332)(Figures 2A and B).

Figure 2 Selection of risk factors for POD using the LASSO binary logistic regression model. Abbreviations: LASSO, least absolute shrinkage and selection operator; SE, standard error. Notes: (A) The LASSO model’s optimal parameters (lambda) were selected using five-fold cross-validation with the minimum criteria. The logarithm of the partial likelihood deviation (binomial deviation) curves was shown. Using the minimal criteria and its one standard error, dashed vertical lines are displayed at the optimal values (1-SE criterion). (B) Profiles of the lasso coefficients for 46 characteristics. log(λ) sequences were used to plot coefficient profiles. The optimum lambda produces six features with non-zero coefficients, which are represented as vertical lines at values chosen using five-fold cross-validation.

Age, MMSE, sleep disorders, neurological disorders, preoperative serum creatinine (Pre-SCR), and ASA classification were the six risk factors examined.

Establishment of Nomogram

The results of the regression analysis for age, MMSE, sleep disorders, neurological disorders, Pre-Scr, and ASA classification are shown in Table 2. Models with the independent predictors mentioned above were created and shown as a nomogram (Figure 3). This model’s c-index value was 0.928 (95% CI: 0.898–0.957). In this cohort, there was good agreement in the calibration curves for the nomogram used to estimate the likelihood of POD in elderly patients receiving elective orthopedic surgery (Figure 4A).

Table 2 Predictive factors for POD

Figure 3 Developed POD nomogram. Abbreviations: Scr, serum creatine; ASA, American Society of Anesthesiologists. Notes: Creating a nomogram that predicts the likelihood that older patients undergoing elective orthopedic surgery would experience POD, taking into account factors including age, MMSE, sleep disorder, Neurological disorders, Pre-Scr, and ASA classification.

Figure 4 Calibration curves and decision curve analysis for the POD nomogram. Notes: (A) The x-axis indicates the predicted POD risk. The y-axis indicates the inconsistency of the actual diagnosis. Ideal indicates the correction curve of the perfect prediction model. Apparent indicates the performance of the training set. Bias-corrected indicates the performance of the model after repeated self-sampling, where closer to the diagonal dashed line indicates better prediction. (B) The horizontal coordinate of the graph is the Threshold Probability. The vertical coordinate is the net benefit after subtracting the disadvantage from the benefit.

Internal and External Validation of a Nomogram

The model underwent both internal and external validation. External validation was performed using external hospital data for spatial validation with a C-index of 0.921, while internal validation used bootstrap validation with a C-index of 0.912, suggesting the model’s good discrimination.

Clinical Use

Figure 4B displayed the decision curve analysis for the nomogram. The POD prediction nomogram is represented by the blue line. The narrow grey line represents the idea that all surgical patients experience POD. The theory that no patients have POD following surgery is shown by the solid black line. The decision curve shows that utilizing this nomogram to estimate the likelihood of POD in elderly patients receiving elective orthopedic surgery enhances the benefit over the intervention all-patients scenario or the no-intervention scenario within a given interval.

Discussion

In the current study, we developed and validated a nomogram-based model for predicting POD. Our POD model, incorporating the age, MMSE, sleep disorder, neurological disorders, pre-SCR, and ASA classification, reliably identifies patients at high risk of delirium recurrence. This is the first delirium prediction study for all elective elderly orthopedic surgery patients, not limited to hip fracture surgery. The nomogram provided better predictive accuracy than the risk factor-based model, demonstrating the incremental value of the nomogram to the current diagnostic management of POD. Moreover, our nomogram is easy to use, and it could serve as a preoperative tool for the individualized prediction of delirium.

The incidence of POD in our study was 12.2% (development group) and 12.4% (validation group), values that are similar to Yang’s study (13.6%)²¹ and Zhao’s study (12.2%;).²²

Risk Factors

We identified potential risk factors for POD based on a thorough literature review. We identified six independent risk factors using lasso regression and logistic analysis, and we developed a nomogram that can more accurately predict the occurrence of POD. Our research showed that delirium is more common in people above the age of 85. An established and well-known risk factor for the onset of delirium is advanced age.²³ Our systematic search of meta-analyses of POD prediction models indicated that sleep disturbance was also considered a risk factor for the development of delirium.²⁴ The presence of previous cognitive impairment indicated by a low MMSE score has also been identified as a risk for developing delirium.^25,26 It was also thought that a history of neurological disorders constituted a separate risk factor for the emergence of POD. According to earlier research, substantial renal insufficiency or elevated blood creatinine levels can further increase the chance of developing POD.^27,28 The European Society of Anesthesia’s Evidence-Based and Consensus Guidelines for POD state that there is a connection between the prevalence of POD and ASA classification.²⁹

Results of the model development showed that the model has good performance with the c-index for the training set being 0.928 and the c-index for the validation set being 0.921.

The fact that this model only uses six variables to measure risk suggests that additional factors may be equally essential but were overlooked. But when combined with the six predictors, the additional risk factors identified in this study did not improve the prediction accuracy of POD. Additionally, it would be better if a simple and accurate model could predict the onset of POD.

Our study’s incidence of delirium was 12.2% in the development group and 12.4% in the validation group, which is similar to earlier studies that revealed a delirium incidence of 5% to 45% following emergency and elective orthopedic surgery.³⁰ Furthermore, studies have revealed that between 25 and 50% of senior people hospitalized for medical issues also have acute delirium, which may be connected to regional variations in POD diagnosis or institution disparities.³¹

Clinical Relevance

Since the pathophysiological mechanisms of delirium occurrence are still unclear, and because therapy for delirium is incomplete once it has happened, primary delirium prevention is the most effective method. The prevention strategies for delirium are mainly pharmacological and non-pharmacological prevention. Evidence-based recommendations for POD therapy are provided by medical organizations, including the American Geriatrics Society, the European Society of Anesthesiology, and the UK’s National Institute for Health and Care Excellence. Finding and treating precipitating etiologies are the primary goals of the first phases.³² The nomogram screens individuals who are at a greater risk of POD for targeted preoperative management and prevention, thereby reducing the risk of developing POD and ensuring efficient use of resources.

A literature search revealed that the majority of existing prediction models for POD had only focused on ICU patients or patients having hip fracture surgery. Some prediction models are only accessible to ICU patients, and it is unknown if they can be used with other patients.^33,34 There are some studies only on patients with hip fractures.²¹ We cannot rule out the probability of POD in older orthopedic patients undergoing spine or extremities surgery, even though studies have shown a significant likelihood of delirium in patients with hip fractures. Chen et al³⁵ showed that patients were evaluated only 3 days postoperatively, but the current POD evaluation should be extended to the seventh postoperative day. Kalisvaart et al³⁰ included patients undergoing both emergency and elective orthopedic surgery. The model cannot reliably predict POD in patients undergoing elective orthopedic surgery since existing studies reveal a lower prevalence of delirium following elective surgery than following emergency surgery.³⁶ Given this, we created a POD model that was not limited to ICU patients and specific types of orthopedic surgery and diagnosed delirium in elderly patients undergoing elective orthopedic surgery within seven days postoperatively. The model was externally evaluated to show its utility and included only patients who underwent elective surgery, eliminating the effect of patients who underwent emergency surgery on morbidity. The model is displayed as a nomogram for ease of use by clinicians.

Limitations

The study has the following drawbacks. Firstly, confounding bias is inevitable because the model uses data from only one center. Secondly, we defined the normal range of blood parameters on the basis of our hospital laboratory test. Considering the difference in lab examinations among individual hospitals, we acknowledged it as a shortcoming in our study. Thirdly, the lack of information regarding pre-operative function status and frailty is another issue. There is increasing evidence that these variables might be the best predictors of post-operative complications and delayed function recovery. Fourthly, the researchers visited the patients twice a day. However, the state of delirium was fluctuating, and there may have been false negative assessments. Furthermore, although it is recommended that they be incorporated into future studies, the severity and length of delirium were not considered when constructing the prediction model for this study. Given the inclusion of the type of anesthesia and type of surgery as one of the variables in the study, although no statistical significance was found in the univariate analysis, future studies with large samples are needed to determine whether the type of anesthesia as well as the type of surgery has a significant effect on delirium.

Conclusion

Our nomogram model constructed based on age, MMSE, sleep disorders, neurological disorders, Pre-Scr, and ASA classification, shows good discriminatory performance and significant clinical efficacy, and could thereby facilitate more precise prediction and better management of POD. These six predictors are six well-defined clinical variables are easy to assess and detect during the preoperative period, and thus allows clinicians to not only identify and prevent those with a high risk of POD, but also to explain the risks in advance to patients and their families, helping to better understand the possible results.