Development and Validation of Nomogram Prediction Model for Postoperative Sleep Disturbance in Patients Undergoing Non-Cardiac Surgery: A Prospective Cohort Study

Introduction

In recent years, with the development of comfortable medical concept, and the growth of people’s demand for health level, sleep disturbance has evoked much attention than before. PSD is a common postoperative complication and patients may report decreased sleep time, increased numbers of arousals or awakening, frequent nightmares, fragmented sleep and lowered sleep quality after operation.¹ In previous studies, the prevalence of PSD was 65.7%, and even up to approximately 95% in severe cases.^2,3 About 25% of the patients reported lack of sleep again on the 15th day after surgery, and 24% of them needed medication to correct sleep disturbance.⁴ 10–61% of the patients even had sleep problems over 6 months after discharge, which seriously affected the long-term recovery.⁵ Moreover, PSD are associated with the development of episodic hypoxemia, hemodynamic instability, early postoperative fatigue, cognitive impairment, pain sensitivity and mood disturbances, all with a potential adverse effect on postoperative outcome.^6,7

Figure 1 Flow chart of patients screening and recruitment.

Figure 2 Demographic and clinical feature selection using the Least Absolute Shrinkage and Selection Operator (LASSO) regression. (A) The selection of the tuning parameter (lambda) in the LASSO model used 5-fold cross-validation with the minimum criteria. The relationship curve between partial likelihood deviation (binomial deviation) and log(lambda) was plotted. Dotted vertical lines were drawn at the optimal values by using the minimum criteria and the 1 standard error (SE) of the minimum criteria (the 1-SE criteria). (B) LASSO coefficient profiles of the 16 features. A coefficient profile plot was produced against the log(lambda) sequence. Vertical line was drawn at the value selected using 5-fold cross-validation, where optimal lambda resulted in 16 features with non-zero coefficients.

Figure 3 Nomogram for perioperative estimation of PSD risk. To use the nomogram, find the position of each variable on the corresponding axis, draw a line to the points axis for the number of points, add the points from all of the variables, and draw a line from the total points axis to determine the PSD probabilities at the lower line of the nomogram.

Figure 4 The predictive performance of PSD risk nomogram. (A) The receiver operating characteristic (ROC) curve was compared between training cohort and validation cohort. The area under the curve (AUC) of training cohort and validation cohort were 0.82 and 0.80, respectively. (B) Validity of the predictive performance of the nomogram in estimating the risk of PSD presence in the training cohort (n = 617). (C) Validity of the predictive performance of the nomogram in estimating the risk of PSD presence in the validation cohort (n = 264).

The mechanisms leading to sleep disturbance in patients are complex, with relevance to the environmental, pharmaceutical and medical factors, such as surgical inflammatory response, severity of surgery, type of anesthesia, pain, anxiety, as well as their interactions.^6,8,9 While PSD itself is well recognized, perioperative management of patients is a relatively neglected field of research. Currently, the treatment strategies for PSD include dexmedetomidine, zolpidem, melatonin, enhanced recovery after surgery (ERAS) protocol and improving of ward environment.^10,11 However, these research perspectives of sleep disturbance during perioperative period were emphasizing intervention rather than focusing on risk prediction, especially the clinical modifiable risks.

Medical staff should pay attention to the prevention of sleep problems in patients.¹² The aim of the present study was to investigate risk factors of PSD, and develop a prediction model for the accurate prediction of PSD in patients undergoing non-cardiac surgery.

Patients and Methods

This is a prospective cohort study, which has been approved by the Research Ethics Committee of the Affiliated Hospital of Xuzhou Medical University (Xuzhou city, Jiangsu province, China) on 28 May (XYFY2020-KL057-01) and registered in the Chinese Clinical Trial Registry (ChiCTR1900024172). Written informed consent was obtained from all study participants before the operation.

Participants

From June 2020 to April 2021, data on consecutive patients who underwent non-cardiac surgery were prospectively collected at the Affiliated Hospital of Xuzhou Medical University.

The study had the following inclusion criteria: (1) Chinese-speaking patients aged ≥18 years with the ability to provide informed consent, scheduled for elective non-cardiac surgical under general anesthesia; (2) American Society of Anesthesiologists physical status classification (ASA) grade I–III. Patients were excluded if they: (1) had preoperative sleep disorders [Pittsburgh Sleep Quality Index (PSQI) score > 7]; (2) received brain surgery; (3) received daytime surgery; (4) had cognitive dysfunction, psychiatric disease, or motor dysfunction; (5) had long time use of sedatives, antidepressants, drugs or alcohol dependence. Patients were eliminated if they: (1) admitted to intensive care unit (ICU) after surgery; (2) cancelled the operation; (3) changed the anesthesia method; (4) required reoperation; (5) failed to complete the follow-up.

Clinical Variables

Potential predictive factors were filtrated based on the literature and clinical practice. Preoperative variables included demographics [sex, age, body mass index (BMI), educational level, alcohol or smoking consumption], past medical history [obstructive sleep apnea hypopnea syndrome (OSAHS), hypertension, diabetes mellitus, anxiety, depression, sleep disorders], NRS score, ASA grade, satisfaction of ward environment, preoperative hospital stay. Intraoperative variables included anesthesia method, operation method, operation time, anesthesia time, amount of fluid infusion and blood loss, urine volume. Postoperative variables included the duration stayed in PACU, the use of patient-controlled intravenous analgesia (PCIA), post-surgical complications (fever, hypoxemia, hypotension, PONV), postoperative NRS score, postoperative hospital stay.

Clinical data were collected continuously during hospital stay and at the follow-up visit 1 week postoperatively. Communication methods between the researchers and patients included face-to-face and telephone contacts. The average score of postoperative NRS for three consecutive days was defined as the postoperative NRS score in this study.

Development and Internal Validation Population

With the aid of R software (version 3.6.2; https://www.R-project.org), a total of 881 patients were enrolled in total and were separated randomly into training cohort (n=617) and validation cohort (n=264) at a theoretical ratio of 7:3.

Evaluation of Sleep Disturbance

Sleep quality was assessed using the PSQI, a well-validated self-report questionnaire designed to assist in the diagnosis of sleep disorders.¹³ The PSQI has 7 components including sleep latency, sleep duration, habitual sleep efficiency, sleep disturbances, daytime dysfunction, and hypnotic drugs which range 0–3, respectively. The global PSQI score ranging from 0 to 21 is generated by summing up all the seven component scores, where 0 indicates no difficulty and 21 is severe difficulty in all areas. A global cut-off score of PSQI greater than 7 is used to distinguish poor sleepers from good sleepers. In this study, experienced investigators used PSQI questionnaire to evaluate the sleep quality of non-cardiac patients before and 1 week after the operation.

Statistical Analysis

For the normal distribution data, the data were expressed as mean ± standard deviation (SD) and compared using the independent sample t-test. For the categorical data, the data were presented as number and percentages, and compared using the Chi-square test or Fisher’s exact test.

LASSO regression and multivariate logistic regression analysis were utilized to filter possible indicators in the training cohort. Non-zero characteristic factors selected through LASSO analysis were put into the multivariate logistic regression analysis to identify the independent risk factors associated with PSD. A nomogram was formulated based on the results of multivariate logistic regression analysis by using the rms package of R software. The backward stepdown process based on the Akaike information criterion (AIC) was used to control the over-fitting of the model. The predictive performance of the nomogram was measured by C index, ROC curve and calibration with 1000 bootstrap samples to decrease the over-fit bias based on the data from training cohort and validation cohort.¹⁴

For clinical use of the model, the total points of each patient were calculated based on the proposed nomogram. ROC curve analysis was used to calculate the optimal cutoff values that were determined by maximizing the Youden index. Accuracy of the optimal cutoff value was assessed by the sensitivity, specificity, predictive values, and likelihood ratios.

Statistical analysis was performed with R software (version 3.6.2; https://www.R-project.org). The reported statistical significance levels were all two-sided, with statistical significance set at 0.05.

Results

Participant Characteristics

We initially assessed 1360 patients for eligibility to participate in our study (Figure 1). Of these, 125 patients refused to participate, 203 patients did not meet the inclusion criteria and the remaining 1035 patients enrolled to the study. Following the completion of the study, 154 patients were excluded from the study: 48 patients admitted to ICU after operation, 8 patients cancelled the operation, 16 patients changed the anesthesia mode, 13 patients required reoperation, 69 patients failed to complete the follow-up. Finally, the data of 881 patients were analyzed in the present study. The training cohort composed of 617 patients. The average age was 54.62±12.91 years, 41.3% were male and 71.8% had PSD. The validation cohort consisted of 264 patients. The average age was 54.53±14.30 years, 42.8% were male and 72.0% had PSD. The prevalence of PSD among all patients was 71.9%. All characteristics were no statistically significant differences between training cohort and validation cohort (all P > 0.05) (Table 1).

Table 1 Participant Characteristics

Construction of Prediction Model in the Training Cohort

Thirty-two potential risk factors from perioperative clinical indicators were included in the LASSO regression analysis (Figure 2A and B). We selected 16 non-zero characteristic variables including gender, ASA, hypertension, anxiety, satisfaction of ward environment, surgical classification, combined regional nerve block, PCIA, PONV, duration stayed in PACU, dexmedetomidine, midazolam, sufentanil, postoperative NRS score, postoperative hospital stay, infusion (Table 2).

Table 2 Coefficients and Lambda.Min Value of the LASSO Regression

The results of multivariate logistic analysis are presented in Table 3. On multivariate analysis, with results reported as odds ratio (95% CI), male sex [0.54(0.32–0.88)], anxiety [4.69(1.83–14.64)], satisfaction of ward environment [0.38(0.17–0.80)], combined regional nerve block [0.46(0.28–0.75)], PONV [1.92(1.11–3.41)], the longer duration stayed in PACU [1.02(1.01–1.03)], the higher dose of midazolam [0.62(0.49–0.79)] and sufentanil [0.97(0.95–0.99)], the higher postoperative NRS score [1.59(1.23–2.07)] were independently associated with PSD.

Table 3 Multivariate Logistic Regression Analysis of PSD Presence Based on Perioperative Data in the Training Cohort

Development and Validation of the PSD-Predicting Nomogram

These independently associated risk factors were used to build a PSD risk prediction model (Figure 3). The resulting model was internally validated using the bootstrap validation method. The nomogram showed good accuracy in estimating the risk of PSD, with an unadjusted C index of 0.82 (95% CI, 0.78–0.85) and a bootstrap-corrected C index of 0.79 (Figure 4A). Besides, the calibration plots graphically demonstrated good agreement on the presence of PSD between the risk estimation by the nomogram and analysis results of actual clinical data (Figure 4B). In the validation cohort, the nomogram displayed an unadjusted C index of 0.80 (95% CI, 0.74–0.85) and a bootstrap-corrected C index of 0.80 for the estimation of PSD risk (Figure 4A). There was also a good calibration curve for the risk estimation (Figure 4C).

Risk of PSD Based on the Nomogram Scores

The optimal cutoff value of the total nomogram scores was determined to be 262. The sensitivity, specificity, positive predictive value, and negative predictive value when used in differentiating the presence from absence of PSD were 74.0%, 79.3%, 90.1%, and 54.5% in the training cohort, and 75.3%, 70.3%, 86.7%, and 52.5% in the validation cohort, respectively (Table 4).

Table 4 Accuracy of the Prediction Score of the Nomogram for Estimating the Risk of PSD Presence

Discussion

Of the currently available prediction tools, the nomogram is a superior visual tool with the user-friendly display, easily understand, high accuracy and good discrimination characteristics in predicting results.^15–17 Here, we generated and validated a novel nomogram to accurately forecast PSD risk in non-cardiac surgery patients. As far as we know, this is the first attempt to predict PSD in non-cardiac surgery patients. The proposed nomogram, which incorporated 9 comprehensive and easily available perioperative variables: gender, anxiety, satisfaction of ward environment, combined regional nerve block, PONV, the duration stayed in PACU, midazolam, sufentanil and the postoperative NRS score, performed well as supported by the C index values of 0.82 and 0.80 in the training and validation cohorts, respectively, and the optimal calibration curves demonstrating the agreements between prediction and actual observation.

In the PSD risk estimation nomogram, pain, absence of combined regional nerve block, anxiety, PONV and dissatisfaction of ward environment have been reported to increase the possibility of sleep disturbance.^18–26 Our study demonstrated that these factors were also significantly associated with PSD in non-cardiac surgery patients. In addition, we demonstrated that female sex, the longer duration stayed in PACU, and the lower dose of midazolam were associated with an increased probability of PSD in non-cardiac surgery patients.

Midazolam is a short-acting water-soluble benzodiazepine sedative. Midazolam produces sedative, hypnotic, anticonvulsant, and anxiolytic effects by altering the configuration of the GABAA receptor complex and opening the chloride channel. The effect of antegrade amnesia, combined with opioids, etomidate, propofol drugs can produce synergistic or additive effects, resulting in analgesic effect.^27,28 Previous studies have found that midazolam could shorten sleep latency, reduce wakefulness frequency, improve objective sleep evaluation and reduce daytime sleepiness.²⁹ However, some scholars have also pointed out that long-term application of benzodiazepines could lead to an increase of shallow sleep and increased risk of adverse effects such as cognitive impairment and falling, especially in the elderly.^22,30 Despite dexmedetomidine showed the characteristic of a potential protective effect on sleep, this effect did not occur in our study.¹⁸ Therefore, it is necessary to find a balance between clinical treatment and sleep.

Although the incidence of PSD was higher in women than in men, the reasons for this gender difference were not clear. A possible explanation about the gender difference is the fact that women have higher rates of both acute and chronic insomnia than men from general population studies regardless the setting.^31,32 Stein et al reported that in the general population, women suffer insomnia at higher rates than men and onset is often associated with physical problems of aging.³³ These issues deserve further investigation, ideally with longitudinal or experimental study designs to establish the direction of the effects amongst gender, age, sleep disorders and surgery. In most instances, an extended PACU stay from anesthesia was attributed to longer anesthetic exposure and bigger surgical trauma. Residual anesthetic, analgesic medications and surgical trauma could potentially lead to worse sleep quality of patients.³⁴

With higher degree of internal homogeneity, overall consistency and clinical validity than any other test available, the PSQI originally described by Buysse in 1989, has become the main tool for the assessment of quality of sleep.¹³ Generally speaking, the reporting period of PSQI is 1 month, but the PSQI was used for short-term evaluation for 1 week in this study. Broderick et al once had evaluated the accuracy of PSQI across different lengths of reporting periods (3-, 7- and 28-days). They found there was no significant difference in item scores when electronic daily ratings were compared with recall ratings regardless of the length of the reporting period.³⁵ In addition, some previously studies investigated the short-term sleep quality using PSQI.^36–38 Therefore, PSQI can be also administered confidingly for weeklong reporting periods between subject analyses.

LASSO regression is a popular method for variable selection in fitting high-dimension generalized linear model, which can obtain a more refined model by constructing a penalty function to reduce the variable numbers and effectively avoid overfitting.^39,40 Therefore, we applied the LASSO regression for feature selection and build prediction model in this study.

The nomogram we proposed has several advantages. The selected variables were common and easy to obtain from clinical practice, helping the perioperative management of surgeon. The scoring system of nomogram was simple and comprehensible which achieved good predictive accuracy and favorable stability. By producing an accurate prognosis, this nomogram could help surgeons to stratify patients and make individual clinical treatment strategies under variable conditions.

However, there were still some limitations in the present study. First, this analysis was based on data from a single institution, it is necessary to validate the results from other centers. Second, the risk factors of PSD are complex and there are far more factors to be investigated and used to predict the prognosis of PSD. Third, the follow-up duration was relatively short, and a large prospective study with a long-term follow-up should be performed in the future. Thus, the accuracy of this prediction model will be improved to a certain extent.

Conclusion

In short, this study constructed a predictive model which was presented as nomogram to quantify the risk of PSD in non-cardiac surgery patients. The nomogram showed a powerful predictive ability and could help clinicians make better therapeutic strategy.