Can we predict postoperative complications in elderly Chinese patients with hip fractures using the surgical risk calculator?
Received 27 May 2017
Accepted for publication 23 August 2017
Published 22 September 2017 Volume 2017:12 Pages 1515—1520
Checked for plagiarism Yes
Review by Single anonymous peer review
Peer reviewer comments 2
Editor who approved publication: Dr Zhi-Ying Wu
Xiao Wang, Bin Jiang Zhao, Yue Su
Department of Anesthesiology, Beijing Shijitan Hospital, Capital Medical University, Beijing, China
Purpose: Hip fractures are associated with poor prognosis in elderly patients partly due to the high rate of postoperative complications. This study was aimed to investigate whether the surgical risk calculator is suitable for predicting postoperative complications in elderly Chinese patients with hip fractures.
Methods: The incidence of postoperative complications among 410 elderly patients with hip fractures was predicted by the surgical risk calculator and then compared with the actual value. The risk calculator model was evaluated using the following three metrics: Hosmer–Lemeshow test for the goodness-of-fit of the model, receiver operating characteristic curve (ROC) (also referred as C-statistic) for the predictive specificity and sensitivity, and the Brier’s score test for predictive accuracy.
Results: Preoperative risk factors including gender, age, preoperative functional status, American Society of Anesthesiologists grade, hypertension, dyspnea, dialysis, previous cardiovascular history, and cerebrovascular disease were positively correlated with the incidence of postoperative complications in elderly patients with hip fractures. The predicted complication incidence rate was well matched with the actual complication rate by Hosmer–Lemeshow test. The model had high sensitivity and specificity for predicting the mortality rate of these patients with a C-statistic index of 0.931 (95% CI [0.883, 0.980]). The surgical calculator model had an accuracy of 90% for predicting the reoperation rate (Brier’s score <0.01).
Conclusions: The surgical risk calculator could be useful for predicting mortality and reoperation in elderly patients with hip fracture. Patients and surgeons may use this simple calculator to better manage the preoperative risks.
Keywords: hip fracture, femoral head fracture, elder age, surgical risk calculator, ACS-NSQIP
Approximately 1.6 million hip fractures occur annually around the world, and this number is increasing by 25% every 20 years due to population growth.1 From 1990 to 2006, the number of hip fractures increased by 2.76 times in females and 1.61 times in males in the ≥50-year-old population in China.2 Due to the rapid aging of the population, this number is expected to continue growing.3 The poor prognosis of hip fracture in elderly patients is related to the high incidence of postoperative complications. One-third of elderly patients with hip fractures died within 1 year after injury, resulting in a heavy financial burden for individuals and society.1,4–6 Therefore, a method is urgently needed by which clinicians can accurately and objectively evaluate and predict the incidence of postoperative complications and reduce the perioperative risk in elderly patients with hip fractures.
The American College of Surgeons National Surgical Improvement Program (ACS-NSQIP) Surgical Risk Calculator was developed in 2013.7 Personal data from the preoperative period to 30-day post-operative complications were collected from >1.4 million patients from 393 hospitals in the USA between 2009 and 2012. The calculator was produced using the database generated from these high-quality data.8 Surgeons can calculate the chance of postoperative complications for a patient given the Current Procedural Terminology code (CPT code) and the patient’s preoperative data.
Several orthopedic-related articles have applied the ACS-NSQIP surgical risk calculator to identify preoperative risk factors that increase the risk of postoperative complications in patients.9,10 However, these studies mostly focused on a single risk factor. A recent study evaluated the ability of the surgical calculator to predict complications within 30 days in patients with full-hip arthroplasty.11
In the present study, we identified the preoperative risk factors, compared the predicted and actual postoperative complication rates, and evaluated the goodness-of-fit, sensitivity, specificity, and accuracy of the prediction model in a Chinese population of elderly patients who underwent surgical treatment for hip fracture.
Materials and methods
This study retrospectively analyzed 417 cases of hip fracture in elderly patients who underwent surgical treatment in our hospital from January 2014 to December 2016. Seven cases were lost during the follow-up. The inclusion criteria were age ≥60 years, unilateral femoral neck fracture, and artificial femoral head replacement (CPT code 27,125). The exclusion criterion was a Glasgow Coma scale <3. This study was approved by the ethics committee of Beijing Shijitan Hospital, affiliated to Capital Medical University. All procedures performed in studies involving human participants were in accordance with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. Written informed consent was obtained from patients or their legal guardians.
A total of 21 preoperative entries, including the patient’s basic profile (age, gender, weight, height, functional status, and smoking history), comorbidities (American Society of Anesthesiologists grade, hypertension, diabetes, congestive heart failure, ascites, hormone therapy, COPD, dialysis, renal failure, sepsis, ventilation support, and metastatic cancer), and the nature of the procedure (CPT code, emergency surgery or elective surgery), were entered into the surgical risk factor calculator. Body mass index (BMI, kg/m2) was divided into six categories: underweight (BMI <18.5), normal (18.5≤ BMI ≤25), overweight (25< BMI ≤30), grade 1 obesity (30< BMI ≤35), grade 2 obesity (35< BMI ≤40), and grade 3 obesity (BMI >40).
After entering the patient’s data into the calculator, the predicted incidence of complications within 30 days after surgery was calculated. Postoperative complications defined by this calculator include “major complications” (cardiac arrest, myocardial infarction, pneumonia, progressive renal insufficiency, acute renal failure, pulmonary embolism, deep venous thromboembolism [DVT], reoperation, deep incisional surgical site infection [SSI], organ space SSI, systemic sepsis, unplanned intubation, urinary tract infection [UTI], wound disruption) and “any complication” (superficial incisional SSI, deep incisional SSI, organ space SSI, wound disruption, pneumonia, unplanned intubation, pulmonary embolism, DVT, ventilator use >48 h, progressive renal insufficiency, acute renal failure, UTI, stroke, cardiac arrest, myocardial infarction, reoperation, systemic sepsis). All of the predicted risk ratings in this study were set as “1”. The predicted incidence of postoperative complications was compared with the observed values to determine the accuracy of this calculator in the population of elderly Chinese patients with hip fractures.
Statistical analysis was performed using Microsoft Excel (Redmond, WA, USA) and SPSS 22.0 (SAS Institute, Cary, NC, USA). Quantitative data are presented as average ± SD. The predicted values and the actual values of the count data were analyzed using paired t-test. All preoperative risk factors were separately compared for each major complication and any complication incidence rate using bivariate correlation analysis. P<0.05 was considered statistically significant. The Hosmer–Lemeshow test was used to evaluate the goodness-of-fit for the logistic regression model, and P>0.05 was considered good fit. The receiver operating characteristic curve (ROC) (also referred as C-statistic) was used to determine the predictive sensitivity and specificity of the model. When the C-statistic is >0.83, its predictive ability is considered as mid range, indicating that the model has a good ability to predict the incidence of complications, and when the C-statistic is <0.5, the model is considered not specific or sensitive.12 The Brier’s score is defined as the average squared difference between the patient’s predicted probabilities and observed outcomes (1 or 0 depending on event or non-event). If the predicted value is close to the observed value, the Brier’s score approaches 0.0.7,13,14
A total of 410 patients underwent artificial femoral head replacement, including 132 men and 278 women, with an average age of 80.32±7.514 years (range, 60–100 years). The basic characteristics of the patients are summarized in Table 1. Bivariate correlation analysis showed that gender, age, preoperative functional status, ASA grade, hypertension, dyspnea, dialysis, previous cardiovascular history, and cerebrovascular disease were associated with postoperative complications (P<0.05). Although age, preoperative functional status, ASA grade, dyspnea, and cerebrovascular disease were associated with a higher incidence of postoperative complications, none of them were significantly correlated (Figure 1).
Table 1 Patient demographics and risk factors as recorded in the risk calculator and bivariate analysis for complications
Figure 1 Correlation between preoperative risk factors and postoperative complication rates.
The actual incidence rate of complications within 30 days of surgery is summarized in Table 2. The incidence rates of major complications and any complications were 32.7% and 40.7%, respectively. Complications with a high incidence rate included pneumonia (23.7%), heart complications (17.1%), and UTIs (10.2%), and others with a lower complication rate were thrombosis (4.6%), renal failure (4.6%), rehospitalization (4.6%), SSI (1.5%), mortality (1.5%), and reoperation (0.5%). The predicted average incidence rates of major complications and any complications were 13.26%±6.05% and 14.06%±6.58%, respectively. There was a significant difference between the actual incidence rates of major complications and any complications (P<0.01). The mean values of hospitalization days and actual hospitalization days were 5.74±1.92 days and 19.65±16.31 days, respectively (P<0.01).
Table 2 Event occurrence rates, Hosmer–Lemeshow test values, and C-statistic analysis results for postoperative complications
The Hosmer–Lemeshow test was used to evaluate the postoperative complication rate and the actual observed value of the surgical risk calculator. Our results showed that the predictive morbidity rate of the surgical risk calculator was well fitted with the actual complication rate (P>0.05; Table 2).
Based on the ROC analysis (Table 2), the C-statistic for mortality was 0.931 (95% CI [0.883, 0.980]), indicating that the model was sensitive and specific for predicting mortality within 30 days postoperation in elderly patients with hip fracture (Figure 2). The C-statistic predicted values for remaining outcomes were <0.83.
Figure 2 The area under the ROC curve for mortality within 30 days postoperation.
The Brier’s score for the incidence of reoperation was <0.01, suggesting that the surgical calculator model had an accuracy of 90% for predicting reoperation in elderly patients with hip fractures. However, the Brier’s scores for other complications were >0.01, indicating that the predictive accuracy of this model was low for other postoperative complications (Figure 3).
Figure 3 The Brier’s scores for the predicted and observed values of outcome measurements.
Hip fracture is a common type of fracture in elderly people, who are at high risk of postoperative pulmonary infection, DVT, and other complications, resulting in a significant economic burden and high mortality. If the patient’s physical condition allows, surgical treatment should be performed as soon as possible. However, elderly patients often have complications such as a variety of internal diseases, and a surgical risk calculator is a simple and cost-effective assessment method that can be helpful since the perioperative assessment of the disease could be difficult. Thus, in this study, we investigated whether the surgical risk calculator is suitable for predicting the incidence of postoperative complications in elderly Chinese patients with hip fractures. Our results suggested that a variety of preoperative risk factors were positively or negatively correlated with the incidence of postoperative complications. The predicted complication incidence rate by surgical risk calculator was well matched with the actual complication rate. The surgical risk calculator showed high sensitivity and specificity for predicting 30-day mortality in elderly patients with hip fractures, and the accuracy of the surgical risk calculator for predicting reoperation was 90%.
In elderly patients, multiple organ dysfunctions bring a greater surgical risk. Unfortunately, this calculator does not include preoperative risk factors for elderly patients with a history of common cerebrovascular diseases and heart diseases. Among the risk factors for additional assessment in this study, 42.4% of the patients had a history of heart disease and 29.5% had cerebrovascular disease. Our data also confirmed that cardiovascular disease history and cerebrovascular disease history were correlated with postoperative complications. The lack of preassessment of these risk factors may be the cause of significant differences between the predicted and actual observed values of postoperative complications and any complication rates, as well as the correlation analysis of preoperative risk factors and postoperative complication rates. In addition, one of the common complications in surgical treatment, especially orthopedic surgery, is thrombosis,15,16 and preoperative risk factors in the surgical risk calculator do not include thrombosis-related risk factors, such as the international normalized ratio value, which may also reduce the predicted incidence rate of venous thrombosis in elderly patients postsurgery. Arce et al also noted that several previously identified variables were not encompassed in the surgical risk calculator, which could explain the failure of this calculator to predict the risk of 30-day postoperative complications in a cohort of patients who underwent head and neck reconstruction surgery.17
Similar to the previous finding that this universal risk calculator lacks validity in arthroplasty, pulmonary surgery, and orthopedic surgery,11,17–19 our study also suggested that the mean incidence rates of postoperative major complications and any complication were significantly different from their respective actual observed values. However, the Hosmer–Lemeshow test showed that the morbidity rate predicted by the surgical risk calculator fit well to the actual complication rate. Moreover, C-statistic analysis showed that this model was good at predicting mortality, and Brier’s score showed 90% accuracy for predicting reoperation in the elderly patients with hip fractures. These contradictions suggest that the sensitivity, specificity, and accuracy of the surgical risk calculator for predicting the incidence of other postoperative complications of hip fracture are not good enough. The possible explanations are as follows. First, this calculator does not include preoperative risk factors for elderly patients with a history of common cerebrovascular diseases and heart diseases or risk factors for orthopedic surgeries.20,21 Second, there is variation among surgeons for evaluation of the risk factor ratings due to the lack of preoperative risk factors in the calculator. There could be Class 1 or Class 2 standard deviation even if adjustment is applied.8 Future studies with further subdivision of the risk rating would result in a more objective evaluation of the model. Finally, there was a significant difference between the predicted number of days of hospitalization and the actual number of days of hospitalization, which could be related to the differences between countries. In China, patients need to be hospitalized until the removal of stitches, which could contribute to the low hospital readmission rate, because it is easy to notice and immediately treat any postoperative complication during the weeks of hospital stay.
This study has several limitations. First, the elderly patients included in this study were likely critically ill patients, since the authors worked at a large tertiary hospital. This could result in a higher incidence of postoperative complications than the model determination. Second, the sample size in this study was relatively small. Third, there are differences in medical technology level, population characteristics, and patient composition ratio between China and the country in which this surgical risk calculator was developed, and this could result in a contradiction in the evaluation of the predictive ability.
In conclusion, although the predictive accuracy and power of the surgical risk calculator need to be improved, its value as a simple and inexpensive risk assessment tool for patients and doctors in developing countries is undeniable. Future studies with larger sample sizes and more risk stratifications are desired.
The authors report no conflicts of interest in this work.
Johnell O, Kanis JA. An estimate of the worldwide prevalence, mortality and disability associated with hip fracture. Osteoporos Int. 2004;15(11):897–902.
Xia WB, He SL, Xu L, et al. Rapidly increasing rates of hip fracture in Beijing, China. J Bone Miner Res. 2012;27(1):125–129.
Finsen V. Improvements in general health among the elderly: a factor in the rising incidence of hip fractures? J Epidemiol Community Health. 1988;42(2):200–203.
Maradit Kremers H, Visscher SL, Moriarty JP, et al. Determinants of direct medical costs in primary and revision total knee arthroplasty. Clin Orthop Relat Res. 2013;471(1):206–214.
Brauer CA, Coca-Perraillon M, Cutler DM, Rosen AB. Incidence and mortality of hip fractures in the United States. JAMA. 2009;302(14):1573–1579.
Boddaert J, Cohen-Bittan J, Khiami F, et al. Postoperative admission to a dedicated geriatric unit decreases mortality in elderly patients with hip fracture. PLoS One. 2014;9(1):e83795.
Bilimoria KY, Liu Y, Paruch JL, et al. Development and evaluation of the universal ACS NSQIP surgical risk calculator: a decision aid and informed consent tool for patients and surgeons. J Am Coll Surg. 2013;217(5):833–842.e1–e3.
Weintraub S, Ross-Richardson C, Poster M, Bantus C, Babic B, Singh R. The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) supports the use of an early warning system in identifying postoperative patients at risk for 30-day outcomes. J Am Coll Surg. 2014;219(4):e151.
Marchant MH Jr, Viens NA, Cook C, Vail TP, Bolognesi MP. The impact of glycemic control and diabetes mellitus on perioperative outcomes after total joint arthroplasty. J Bone Joint Surg Am. 2009;91(7):1621–1629.
Gholson JJ, Pugely AJ, Bedard NA, Duchman KR, Anthony CA, Callaghan JJ. Can we predict discharge status after total joint arthroplasty? A calculator to predict home discharge. J Arthroplasty. 2016;31(12):2705–2709.
Edelstein AI, Kwasny MJ, Suleiman LI, et al. Can the American college of surgeons risk calculator predict 30-day complications after knee and hip arthroplasty? J Arthroplasty. 2015;30(9 Suppl):5–10.
Mehdi T, Ahmadi BN. Kernel smoothing for ROC curve and estimation for thyroid stimulating hormone. Int J Public Health Res. 2011:239–242.
Szender JB, Frederick PJ, Eng KH, Akers SN, Lele SB, Odunsi K. Evaluation of the National Surgical Quality Improvement Program Universal Surgical Risk Calculator for a gynecologic oncology service. Int J Gynecol Cancer. 2015;25(3):512–520.
Cohen ME, Bilimoria KY, Ko CY, Hall BL. Development of an American College of Surgeons National Surgery Quality Improvement Program: morbidity and mortality risk calculator for colorectal surgery. J Am Coll Surg. 2009;208(6):1009–1016.
Akpinar EE, Hosgun D, Akan B, Ates C, Gulhan M. Does thromboprophylaxis prevent venous thromboembolism after major orthopedic surgery? J Bras Pneumol. 2013;39(3):280–286. Portuguese.
Dixon J, Ahn E, Zhou L, Lim R, Simpson D, Merriman EG. Venous thromboembolism rates in patients undergoing major hip and knee joint surgery at Waitemata District Health Board: a retrospective audit. Intern Med J. 2015;45(4):416–422.
Arce K, Moore EJ, Lohse CM, Reiland MD, Yetzer JG, Ettinger KS. The American College of Surgeons National Surgical Quality Improvement Program surgical risk calculator does not accurately predict risk of 30-day complications among patients undergoing microvascular head and neck reconstruction. J Oral Maxillofac Surg. 2016;74(9):1850–1858.
Samson P, Robinson CG, Bradley J, et al. The National Surgical Quality Improvement Program risk calculator does not adequately stratify risk for patients with clinical stage I non-small cell lung cancer. J Thorac Cardiovasc Surg. 2016;151(3):697–705. e1.
Slump J, Ferguson PC, Wunder JS, et al. Can the ACS-NSQIP surgical risk calculator predict post-operative complications in patients undergoing flap reconstruction following soft tissue sarcoma resection? J Surg Oncol. 2016;114(5):570–575.
Boddaert J, Raux M, Khiami F, Riou B. Perioperative management of elderly patients with hip fracture. Anesthesiology. 2014;121(6):1336–1341.
Xu B, Han L, Liu H, et al. Cardiovascular disease and hip fracture among older inpatients in Beijing, China. Biomed Res Int. 2013;2013:493696.
This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.