Development of a Brief Cognitive Screening Tool for Predicting Postoperative Delirium in Patients with Parkinson’s Disease: A Secondary Analysis

Introduction

Millions of people undergo life-saving surgery worldwide each year to regain health. However, many surgical patients frequently develop postoperative delirium (POD), which is characterized by acute changes in attention and consciousness and is associated with increased morbidity and mortality.¹ Preoperative cognitive impairment (CI) also be reported significantly increases the risk of POD.² The Mini-Mental State Examination (MMSE)³ and Montreal Cognitive Assessment (MoCA)⁴ are the most commonly used scales to detect CI and have been commonly used in preoperative settings. Preoperative MMSE or MoCA scores are negatively associated with high incidences and severity of POD.^2,5

The MMSE takes 10 min, MoCA takes 15 min,⁶ and the time required for elders might be prolonged. These tests require a substantial amount of time when implemented for the anesthesiologists during the preoperative visit and may not be practical for routine preoperative cognitive screening. On the other hand, some patients are not able to complete the standard MMSE or MoCA due to discomfort caused by illness (fractures, weakness, and pain) or are limited by education level, as well as other clinical conditions: for example, Parkinson’s disease (PD) with muscular/neurological deficits. Therefore, a simple, rapid, and effective cognitive screening test appropriate for fast-paced settings with limited resources and staff is essential, especially preoperatively.

Notably, not all MMSE or MoCA subtests are required to identify CI,^7,8 and redundant items could cost time and human resources. Therefore, less useful items were excluded, and those with a high discriminative value were used to construct the adapted versions of the cognitive tests. Brief cognitive screening instruments have been developed and validated in non-surgical populations.^9,10 Herein, we aimed to develop and test a short cognitive test for predicting POD in patients with Parkinson’s disease (PD) undergoing elective surgery.

Materials and Methods

Study Design and Population

This study was approved by the Ethics Committee of Yuquan Hospital of Tsinghua University (20190014). All patients enrolled have signed informed consent. The clinical trial registration number was ChiCTR1900027210.

This study comprised patients with PD scheduled for deep brain stimulation of the subthalamic nuclei (STN-DBS) between November 2019 and October 2020. The inclusion criteria were patients meeting the criteria and elected to receive DBS surgery (primary PD, hereditary PD or various genotypes PD, responds well to compound levodopa; drug efficacy has decreased significantly or obvious motor complications affect the patient’s quality of life; adverse drug reactions that cannot be tolerated and affect the efficacy of drugs; and there are tremors that cannot be controlled by drugs.). The exclusion criteria were pre-existing significant CI (dementia), severe (refractory) depression, anxiety, schizophrenia, other mental diseases, and medical comorbidities that affect surgery or survival.

The baseline characteristics, including age, sex, body mass index, the highest level of education, American Society of Anesthesiologists (ASA) functional status, and preoperative complications, were collected. Bilateral STN-DBS treatment was initiated according to the Movement Disorders Society guidelines.¹¹

Preoperative Cognitive Evaluation

All participants were Chinese speaking and were screened using the Chinese version of the MMSE¹² and a revised Chinese version of the MoCA (the Peking Union Medical College Hospital version of the MoCA).⁸ The MMSE and MoCA were administered by a trained geriatrician during the preoperative visit.

The MMSE is a 30-point scale that assesses 11 cognitive domains: orientation to time (range 0–5), orientation to place (range 0–5), registration (range 0–3), attention and calculation (range 0–5), recall (range 0–3), naming (range 0–2), repetition (range 0–1), comprehension (range 0–3), reading (range 0–1), writing (range 0–1), and drawing (range 0–1).¹² The MoCA is a 15-min, 30-point cognitive screening test designed to detect CI. It encompasses seven cognitive domains: visuospatial and executive function (range 0–5), naming (range 0–3), attention (range 0–6), language (range 0–3), abstraction (range 0–2), recall (range 0–5), and orientation (range 0–6).⁸ The optimal MMSE and MoCA cut-off scores for detecting PD-MCI was 27 and 23 respectively.^10,13–15

POD Assessment

POD was assessed twice a day on the first postoperative day until discharge, once in the morning and once in the evening, by research members trained before the study and blinded to patient’s preoperative cognitive function. It was identified using the confusion assessment method (CAM).¹⁶

Sample Size Calculation

A sample size of 36 (18 POD and 18 non-POD) provides precision on a 95% confidence interval (CI) of the area under the curve (AUC) to be approximately ±0.15 if a discrimination magnitude of approximately 0.80 is assumed. Given an anticipated dropout rate of 20%, 23 POD patients are required. A previous study showed that approximately 20% of PD patients developed POD,¹⁷ and thus, a minimum sample size of 115 PD patients scheduled for STN-DBS is required.

Statistical Analysis

Statistical analyses were performed with SPSS 24 (IBM Corp., Armonk, NY, USA). Categorical variables are presented as n (%). Normally and non-normally distributed continuous data are expressed as mean±standard deviation (SD) or as median with first and third quartiles respectively. χ² test or Fisher’s exact test were used to compare categorical data between patients with and without POD. Unpaired Student’s t-test or Mann–Whitney U-test were used to compare continuous data.

In order to organize the reduced versions of the MMSE or MoCA, we performed a backward conditional logistic regression analysis with all MMSE or MoCA items used as independent variables and POD diagnosis as a dependent variable;¹⁸ the probability to be removed from the model was set at 0.05. The variables retained in the model were used to compose the reduced versions of the cognitive screening tests.

Pearson’s correlation coefficient was used to evaluate the correlation between the short cognitive screening tests and the original tests. Receiver operating characteristic (ROC) curve analysis was used to determine the ability of the original and short cognitive tests to discriminate between POD and non-POD individuals. Next, the DeLong test was used to examine the difference between the AUCs. AUC values closer to 1 representing a high predictive accuracy. Logistic regression was used for developing the models for POD prediction. The c-statistic (equivalent to the AUC) is a standard measure of the predictive accuracy of a logistic regression model. The level of significance was set at α=0.05.

Results

Subject Characteristics

A total of 128 consecutive PD patients who underwent bilateral STN-DBS fulfilled the inclusion criteria. Three patients were excluded due to reoperation. Among the remaining 125 patients included in this analysis, 27 (21.6%) developed POD (Figure 1, flow diagram). No statistically significant group differences were detected in demographic data or baseline characteristics other than age and education (Table 1). Compared to individuals without POD, those with POD were older, with a mean age of 57.7±9.4 and 65.6±6.5, respectively. POD patients had lower education than those without POD, with the mean value of 6.9±3.1 and 11.4±2.8, respectively.

Table 1 Demographic and Baseline Characteristics of Patients

Figure 1 Flow chart of the study. Abbreviations: PD, Parkinson’s disease; DBS, deep brain stimulation; BMI, Body Mass Index; ASA, American Society of Anesthesiologists; MMSE, Mini-Mental State Examination; MoCA, Montreal Cognitive Assessment; POD, Postoperative Delirium.

MMSE Scores and MMSE Reduced Version

POD patients had lower MMSE scores than those without POD (Table 2). The registration, naming, repetition, and reading did not differ between the groups. All the remaining MMSE subtests helped to discriminate between the POD and non-POD groups. The backward conditional logistic regression excluded some of the redundant items. Thus, orientation to time, attention and calculation, and comprehension comprised the MMSE reduced version. A strong positive correlation was established between the brief version and the standard MMSE (Pearson r=0.910, p<0.001).

Table 2 Total and Subtest MMSE or MoCA Scores of Patients with and without POD

MoCA Scores and the MoCA Reduced Version

The POD group presented significantly lower scores than the non-POD group regarding MoCA (Table 2). All MoCA subtests but naming and abstraction could discriminate between POD and non-POD individuals. After the backward conditional logistic regression, the following subtests in the model were used to construct the MoCA reduced version: visuospatial and executive functioning, attention, and orientation. A strong positive correlation was established between the MoCA reduced version and the original MoCA (Pearson r=0.884, p<0.001).

Ability of Short Cognitive Tests to Predict Postoperative Delirium

Compared to non-POD individuals, those with POD had lower scores in the MMSE reduced version (t= 5.574, p<0.001), with the mean values of 11.8±1.6 and 9.1±2.4, respectively. Moreover, POD patients had lower scores on the MoCA reduced version than those without POD (t= 6.007, p<0.001), with mean values of 10.3±3.5 and 14.5±1.9, respectively.

As shown in Table 3, the original cognitive and short tests were associated with POD. All the cognitive tests showed good performance in the prediction of POD (AUC: 0.800–0.826; sensitivity: 0.70–0.78; specificity: 0.68–0.90). The short MMSE demonstrated performance (0.818, 95% CI: 0.722–0.915) similar to the original MMSE (0.800, 95% CI: 0.693–0.906) (Figure 2A, z=0.820, p=0.412). Using the cut-off for a MMSE-R of 11.5 (total score 13), sensitivity was 0.778 and the specificity 0.684. The AUCs of the original and the short MoCAs were 0.808 (95% CI: 0.717–0.899) and 0.826 (95% CI: 0.728–0.924), respectively. The differences in the AUC values between the tests were not significant (Figure 2B, z=0.561, p=0.575). A MoCA-R cutoff of 12.5 (total score 17) had a sensitivity of 0.704, and sensitivity was 0.898.

Table 3 Ability of the Original and Short Cognitive Tests to Predict POD

Figure 2 (A) ROC curve for the prediction of POD based on preoperative MMSE scores. (B) ROC curve for the prediction of POD based on preoperative MoCA scores. Abbreviations: MMSE, Mini-Mental State Examination; MMSE-R, the MMSE reduced version; MoCA, Montreal Cognitive Assessment; MoCA-R, the MoCA Reduced version.

Risk Prediction Models for Postoperative Delirium

Table 4 summarizes the results of the four models to predict POD. Age, education, and total MMSE scores were independent predictors in the multivariate model (c-statistic: 0.922). The c-statistic of the model (age, education, and short MMSE scores) were 0.930. The comparison of the two predictive models did not show a significant difference (Figure 3A, z=1.270, p=0.204).

Table 4 Risk Prediction Models for Postoperative Delirium

Figure 3 (A) ROC curve for POD based on risk prediction models containing baseline characteristics and preoperative MMSE scores. (B) ROC curve for POD based on the risk prediction models containing baseline characteristics and preoperative MoCA scores. Abbreviations: MMSE: Mini-Mental State Examination; MMSE-R: the MMSE reduced version; MoCA, Montreal Cognitive Assessment; MoCA-R, the MoCA reduced version.

Moreover, age, education, and total MoCA scores predicted the POD (c-statistic: 0.907). The c-statistic of the model (age, education, and short MoCA scores) was 0.921. The prediction performance of the two models was similar (Figure 3B, z=1.750, p=0.080).

Discussion

In this secondary analysis, we developed and examined the accuracy of a short variant of MMSE and a short form MoCA in the prediction of POD. The short cognitive tests were short and showed a predictive performance similar to the original tests.

The prevalence of PD in people over 65 years of age is approximately 1.0–3.0%.¹⁹ STN-DBS is a effective treatment for advanced PD patients.²⁰ However, POD is a common complication after DBS surgery, and 27/125 (21.6%) of our study population developed POD, which was consistent with the previous report.¹⁷

As very common in the elderly, POD is a serious, costly, under-recognized and often fatal complication.¹⁶ Acute onset, fluctuating symptoms, inattention, impaired consciousness, and disturbance of cognition (for example, disorientation, memory impairment, and alteration in language) are all key diagnostic features of POD.²¹ PD patients need special attention for increased risk of delirium.²² An accurate and timely preoperative identification of high-risk patients is an effective and easy method for the prevention of POD.

Preoperative CI is a risk factor of POD.^23–25 CI includes mild CI (minimally affects the activities of daily living) and dementia or major neurocognitive disorder where ADLs are compromised.²⁶ CI is defined as abnormal scores on cognitive screening tests, such as MMSE and MoCA.²

Cognitive impairment is a common non-motor symptom of PD.²⁷ Cognitively impaired patients are at increased risk of adverse outcomes following surgery.² High proportion of CI or dementia patients were unrecognized preoperatively,²⁸ which could be because surgeons do not focus on preoperative CI. On the other hand, it is also a fact that the existing cognitive screening scales require large amounts of human resources and time,⁶ thus limiting the routine preoperative cognitive screening. Therefore, a simple, fast, and effective cognitive screening test is required.

In this study, we found that some subtests can have a high predictive diagnostic value in predicting POD and proposed a simplified MMSE and a reduced version of MoCA. Our simplified MMSE is composed of time orientation, attention/calculation, and verbal comprehension, and the simplified MoCA contains executive/visuospatial, attention, and orientation domains. These tests measured the cognitive dimensions most prevalent in PD patients with mild CI (PD-MCI).¹⁰ Also, an overlap was detected between these domains and the key features of POD (acute changes in attention and consciousness, disturbance of orientation, and alteration in language).²¹ These phenomena supported the rationality of our simplified MMSE or MoCA.

Our simplified MMSE or MoCA contained three domains each, which were greatly reduced in length compared to the original tests. These short tests require approximately 5 min, and implementation of routine preoperative cognitive screening using these short tests is feasible. Notably, the short cognitive tests showed good performance in the prediction of POD. The AUC values of the short MMSE and MoCA were 0.818 and 0.826, similar to the original tests. The prediction models for POD containing the short tests performed similarly to the models containing the original tests. Therefore, these tests were short and effective, which helps to identify risk patients and implement appropriate management strategies to reduce adverse postoperative outcomes.

At present, age as an important independent risk factor for POD has become a consensus in the field of neurocognitive research.^29,30 Education years might be negatively associated with postoperative delirium.³¹ The presumed reason is hippocampal atrophy in people engaged in complex mental activities are much less.³² This study also showed age and education could also predicted the POD.

To the best of our knowledge, this is the first study to develop a short and effective cognitive screening tool appropriate for preoperative settings. Our simplified MMSE and MoCA perform adequately in the prediction of POD and may serve as a viable alternative for detecting CI and predicting POD when the original tests could not be feasibly administered in busy clinical services. Moreover, the short cognitive tests were created in PD patients scheduled for surgery; the application of these short forms in other surgical patients needs further study.

Nevertheless, the present study had several limitations. Firstly, it was a single-center study with a small sample size as only a few hospitals currently perform the DBS surgery. Secondly, similar to the development of many abbreviated test versions, the short cognitive screening tests were also derived from the standard version,^8,18 and the predictive performance of these short cognitive tests requires further validation.

Conclusion

Our simplified MMSE and MoCA could be used to predict POD efficiently in PD patients scheduled for surgery. To the best of our knowledge, this is the first study to develop short and effective instruments for predicting POD appropriate for fast-paced preoperative settings with limited resources and staff. Nonetheless, additional studies are required to validate our findings in various surgical patients.