Development and External Validation of a Nomogram for Predicting Acute Kidney Injury in Cardiogenic Shock Patients in Intensive Care Unit

Introduction

Cardiogenic shock (CS), which is distinguished by inadequately severe decrease in cardiac output resulting in hypotension and signs and/or symptoms of end-organ hypoperfusion, is a cardiac emergency and urgently demanded for pharmacological and/or mechanical intervention.^1,2 Despite great progress in the diagnosis and aggressive medical therapy and early reperfusion therapy, CS patients are still characterized with high morbidity and mortality.^3–6 Moreover, acute kidney injury (AKI) in the setting of CS, known as type 1 form of cardiorenal syndrome, is increasing in the risk of adverse clinical and appears to be predictive risk factor for the prognosis of CS patients.^7–10 Mechanisms involved include reduced cardiac output, venous congestion, systemic inflammation, and release of vasoactive mediators. Hence, identification of patients at high risk of AKI is of great concern for clinicians and patients to manage these patients so as to reduce the morbidity and mortality.

As a reliable and simplified scoring system, the nomogram predictive model could accurately predict the risk of mortality of patients and has been widely applied in previous studies, especially in the prognosis of cancer patients.^11–13 Nowadays, a plenty of risk factors for CS-AKI have been identified,^3,9,10,14 however, the predictive model is rare especially in CS patients in intensive care unit (ICU). Therefore, in this study, we performed a retrospective study to determine the independent predictors for CS-AKI and develop a nomogram backed by external validation for better clinical assessment.

Materials and Methods

Data Source

The information of all CS patients in the current study was obtained from two large US-based critical care database named Medical Information Mart for Intensive Care IV version 1.0 (MIMIC IV v1.0) and the eICU Collaborative Research Database (eICU-CRD version 2.0).^15,16 Both of them are publicly and freely accessible database containing clinical information of all ICU patients. After successfully accomplished their training course and examination, our team got the access to extract data from this database. Moreover, this study was approved by the Ethics Committee of the Wuhan No.1 hospital and the informed consent was waived, considering that all patients in this database were de-identified.

Definition

The diagnosis of CS based on the ninth or tenth revision of International Classification of Diseases (ICD-9/10) code (78551, 99801, R570, T8111, T8111XA, T8111XD, T8111XS for the MIMIC IV database and 785.51, R57.0 for eICU database). The diagnosis of CS-AKI based on the KDIGO-AKI criteria of serum creatinine in the first 48 hours of their ICU admission.¹⁷

Selection of Participants

The inclusion criteria in this study were as follows: ICU patients who were diagnosed with CS based on the ICD code. We further excluded patients with repeat ICU stays, a hospital stays less than 48 hours and patients with incomplete clinical data (variables with >20% missing values) were also excluded in this study. Furthermore, patients who had a history of end-stage renal disease (ESRD) were excluded in the study. Finally, a total of 2297 patients (1254 patients were from the MIMIC-IV database and 1043 patients were from the eICU-CRD database) were enrolled in this retrospective study and were randomly assigned as training set (n = 899) and internal validation set (n = 355) based on the ratio of 7:3 and patients in the eICD-CRD were appointed as external validation cohort (n = 1043).

Variable Extraction

Demographic characteristics including age, gender, body mass index, comorbidities including diabetes, hypertension, chronic kidney disease (CKD), congestive heart failure (CHF), myocardial infarct, coronary artery disease (CAD), atrial fibrillation, valvular disease, peripheral vascular diseases (PVD), chronic obstructive pulmonary disease (COPD), liver disease, and personal history of stroke, coronary artery bypass grafting (CABG) were obtained based on the ICD codes. Pharmacologic information and severity of disease were also extracted from these two databases. Use of mechanical ventilation (MV), vasopressors and renal replacement therapy during the first two days of their ICU admission were also recorded in this study. Moreover, initial vital signs and laboratory results were also extracted by structured query language with PostgreSQL 9.6.

Estimate glomerular filtration rate (eGFR) was calculated by the CKD-EPI formula.¹⁸ Accumulated fluid balance was registered at 48 hours after their ICU admission. Cumulative fluid balance was calculated by subtracting the total fluid output from the total intake and all patients were stratified into two groups, based on the presence or absence of positive cumulative fluid balance after 48 hours of continuous fluid balance data collection.

The primary outcome in this study was the incidence of CS-AKI according to the KDIGO-AKI criteria.

Development of the Nomogram

The predictive nomogram was developed as follows and could be applied since the ICU admission. First of all, we applied LASSO regression to determine the significant differential factors associated with CS-AKI in the training set. Secondly, recursive feature elimination for support vector machines (SVM-RFE) regression model was also used to rank the informative indexes on the basis of their permutation importance in the training cohort. To avoid the bias which may cause for single regression model and to give a relatively simplified and easily accessible predicting model to clinicians, the overlapping clinical features of these two models were applied to establish the predictive nomogram for CS-AKI, which could provide the clinicians with an intuitive and quantitative method to determine the ICU patients with high risk of CS-AKI. Finally, we validated the predictive efficiency and clinical ability of the nomogram in validation cohort of patients from eICU-CRD.

Statistical Analysis

Continuous variables were expressed as mean (standard deviation, SD), categorical covariates were reported as number (percentage). The receiver operating character (ROC) curve was used to determine the optimal cut-off of factors based on the Youden index. Discriminative ability of the predictive nomogram was evaluated as the C-index and calibration plot in the training set and in the validation set. The overall performance of the prediction model was assessed as the scaled Brier score, which is the mean squared difference between the observed and the predicted outcome. When the risk of outcome is 50%, the Brier score ranges from 0.0 (perfect) to 0.25 (worthless).¹⁹ Moreover, the decision curve analysis (DCA) was also conducted to evaluate the potential clinical usefulness of the predictive nomogram. All analyses were conducted using R (version 4.1.0) and p < 0.05 was considered statistically significant.

Results

Patients’ Characteristics

A total of 2297 patients who were diagnosed with CS were finally analyzed in this study (899 patients in the training cohort, 355 cases in the internal validation cohort and 1043 participants in the external validation) (Figure 1). Among them, 483 (53.7%) patients in the training set, 192 (54.1%) cases in the internal validation cohort and 483 (46.3%) patients in the external validation were developed CS-AKI during their first 48 hours after ICU admission. Furthermore, the detailed information of these cohorts is described in Table 1.

Table 1 Comparisons of Baseline Characteristics Between the Training Cohort and Validation Sets

Figure 1 The flow chart of this study.

Determination of Significant Features

LASSO regression was firstly performed to determine significant factors associated with CS-AKI in the training group. As described in Figure 2A, Charlson Comorbidity Index (CCI), usage of diuretic, mechanical ventilation and vasopressors, SOFA score, white blood cell, albumin, eGFR, anion gap, sodium, and positive fluid balance were significant risk fractures for predicting CS-AKI. Considering that an easy-to-use predictive model with relatively high accuracy could be more helpful in clinical practice, we also performed the SVM-RFE model to obtain the significant indexes associated with CS-AKI and the SVM-RFE algorithm demonstrated that thirteen clinical variables were filtered out, including hypertension, chronic kidney injury, CCI, usage of mechanical ventilation, statin, and proton pump inhibitors, SOFA score, white blood cell, neutrophils, albumin, eGFR, anion gap, and positive fluid balance (Figure 2B).

Figure 2 Selection of significant factors associated with cardiogenic shock acute kidney injury patients. (A) LASSO Cox regression model. (B) Support vector machine model. (C) The overlapping features identified by the two models.

Development and Validation of the Predictive Nomogram

Only the overlapping clinical risk factors both in the LASSO regression and in the SVM-RFE model were followed in this predictive nomogram (Figure 2C). Based on the results of LASSO and SVM-RFE, eight features were finally included in the predictive nomogram for CS-AKI (CCI, usage of mechanical ventilation, SOFA score, white blood cell, albumin, eGFR, anion gap, and positive fluid balance) (Figure 3). Furthermore, the C-index of the nomogram for CS-AKI was 0.823 (95% confidence index, 95% CI 0.798–0.849) in the training group and 0.819 (95% CI 0.769–0.849) in the internal validation group and 0.733 (95% CI 0.704–0.763) in the external validation cohort, which indicated that the nomogram had a relatively great model discriminative capacity. The calibration curve for the predictive nomogram exhibited a high agreement between the actual probability and the predicted probability of CS-AKI in the training set, in the external validation set and in the external validation cohort (Figure 4A–C). Decision curve analysis (DCA) was also applied to determine the clinical utilities of the predictive nomogram for CS-AKI. The DCA curve also demonstrated that the predictive nomogram derived from the training set was clinically useful in the training set, in the external validation set and in the external validation set (Figure 4D–F). Finally, precision, as measured by the Brier score, was 0.149 in the training set, 0.164 in the internal validation set and 0.198 in the external validation cohort.

Figure 3 The predictive nomogram for cardiogenic shock acute kidney injury.

Figure 4 Calibration and clinical utility of the predictive nomogram. The predictive nomogram exhibited a high correlation between the actual probability and predicted probability in the training cohort (A), internal validation cohort (B) and external validation cohort (C). Decision curves analysis for the predictive nomogram to predict the persistent acute kidney injury in the training cohort (D), internal validation cohort (E) and external validation cohort (F).

Discussion

In this retrospective study based on the two big public databases, we developed a predictive nomogram for CS-AKI based on the overlapping risk factors from LASSO and SVM-RFE models. This predictive nomogram possessed great discriminative capacity, precision and clinical usefulness. To verify the generalizability of this nomogram, we further validated our findings in another clinical database. Hence, these findings demonstrated that this predictive nomogram may be a useful method for clinicians to determine patients at high risk of CS-AKI among ICU patients.

AKI is one of the most common and inauspicious predictors of mortality for patients with CS and the morbidity rate of CS-AKI ranges from 13.0% to 60.4% in previous studies.^9,20 Using the data from the population-based medical registries from Danish, Marie et al concluded that the incidence of AKI treated with renal replacement therapy was 13.0% (660/5078) in patients with first-time myocardial infarction-related CS.²¹ Moreover, Reshad et al performed a retrospective study of 293 consecutive CS patients treated with short-term mechanical circulatory support and demonstrated that the morbidity rate of AKI was 60.4% (177/293) and 113 (38.6%) patients were classified with stage 3.²² Compared with those studies, the overall incidence of CS-AKI in this study was 50.4%, which was in accordance with previous study and the difference of the morbidity in different studies may at least partly explained by differences in patient populations and severity of disease. Furthermore, considering that AKI could independently increase the prognosis of CS patients,^22–24 a novel biomarker or predicting model that could accurately identify patients at high risk of CS-AKI earlier might result in better clinical prognosis and lower hospital care cost.

Several potential risk factors of AKI had been identified in previous studies. The SOFA score is one of the most common severity scores in ICU patients worldwide and has been demonstrated to be associated with adverse clinical outcomes including AKI in different populations.^25,26 Marenzi et al conducted a prospective single-center study of 97 patients with ST-elevation acute myocardial infarction (STEMI) complicated by CS and found that SOFA >2 points was associated with increased incidence of AKI (adjusted relative risk, 1.72, 95% CI 1.01–2.93, P = 0.026).²⁷ Similarly, in this study, SOFA score also included both in the LASSO regression and in the SVM-RFE model and was associated with 1.2-fold increased incidence of AKI when used as continuous variable and SOFA≥ 6 points had 3.4-fold increased incidence of AKI when applied as categorical variable after adjusting for confounding factors. Mechanical ventilation (MV) is one of the most significant support for patients with CS especially in ICU and could serve as an independent risk factor for patients with different types of diseases.²⁸ A meta-analysis that consists 23 studies of 10,333 patients with critical ill and concluded that MV was associated with 3-fold increased incidence of AKI (95% CI 1.85–6.92, P < 0.001).²⁹ For patients with CS, Giancarlo et al also found a positive correlation between the usage of MV and incident AKI.²⁷ Our findings added the evidence that need for ventilation was still a robust risk factor for patients with CS. eGFR was widely used in clinical practice to evaluate the baseline kidney function and has been identified as the great predictor for kidney outcomes as well as clinical outcomes.^30,31 Using the data from the prospective multicenter CardShock study, Tuukka et al found a significant relationship between the low baseline eGFR and the incident AKI in 143 CS patients.³² Along with this study, our study also found that patients with lower eGFR (<69.1mL/min) had a higher proportion of incident AKI compared with patients with higher baseline eGFR (65.0% vs 34.0%, P < 0.001) and concluded that baseline eGFR could independently predict the incident AKI in CS patients in ICU. Early reperfusion therapy is essential for the prognosis of patients with CS; however, positive fluid balance may lead to edema in the kidneys and other organs and thus increase the incidence of complications including AKI.^33–35 Gilad et al performed a retrospective study of 84 adult patients with STEMI complicated by CS and found a positive correlation between the amount of fluid accumulated and the rise in serum creatinine (r = 0.42, P = 0.004) and more importantly, they further concluded that for every 1L increase in positive fluid balance, the adjusted possibility for recovery of renal function decreased by 21%.²³ In this study, we calculated the sum volume of input and output of CS patients during the first 48 hours of their ICU admission and concluded that positive fluid balance was an important risk factor for AKI (OR = 2.55, 95% CI 1.93–3.35, P < 0.001).

Nomogram is an easy-to-use tool for diagnostic and prognostic prediction of different types of diseases and had been widely applied in cancer prognosis recent years, at least partly due to the ability to calculate the predictive model of several variables as a numerical probability of the clinical outcomes, which are customized to the characteristics of individual patient. Furthermore, nomogram can transform complicated regression equations into visual graphics, which are easily performed and are more convenient for clinicians to assess the diagnosis or prognosis of individual patient.³⁶ Considering the aforesaid superiorities, we established the predictive nomogram that selected by both the LASSO regression and SVM-RFE model to make the prediction of CS-AKI relatively precise and easily accessible and therefore, clinicians can easily and quickly determine whether early intervention should be performed in patients with high risk of CS-AKI. Fortunately, in the current study, our nomogram exhibited good predictive value in predicting CS-AKI in ICU patients in the training cohort, in the internal validation cohort as well as in the external validation cohort.

Despite the large population of CS patients and relatively great predictive value of this nomogram for CS-AKI, there were still some limitations in this retrospective study. First of all, this retrospective study is based on the two public databases from America, which may lead to several intrinsic or selection biases. Secondly, due to the database limited, some important risk factors, for example, central venous pressure, and so on, had a lot of missing value or did not obtain in this study. Thirdly, we assessed AKI based on the creatinine only for the first 48 hours of their ICU admission. However, most of AKI in ICU patients developed within the first 24 hours³⁷ and in this study 59.6% of patients occurred AKI within the first day of their ICU admission. Fourthly, variables regarding the characteristic of cardiogenic shock were not reported. For instance; cardiac ejection fraction, cause of cardiogenic shock, intervention for cardiogenic shock management. All of these variables may promote AKI more specific other than general ICU risk factors. Furthermore, no data on creatinine before this hospital admission was accessible in this study, thus the baseline eGFR was calculated based on the first measurement of their admission. In addition, the European Renal Best Practice advocates that applying the baseline rather than historical creatinine measurements or a calculated value based on an assumed GFR of 75 mL/min.³⁸ Finally, this was a retrospective multi-center study, despite validated in another big database, more prospective multi-center studies are still proposed to further verify our findings.

Conclusions

CCI, usage of mechanical ventilation, SOFA score, white blood cell, serum albumin, baseline eGFR, serum anion gap, and positive fluid balance of first 48 hours were closely associated with CS-AKI in patients in ICU. The predictive nomogram for CS-AKI manifested well predictive ability for the identification of ICU patients with CS-AKI. This nomogram may be a helpful method for clinicians to stratify the risk of CS-AKI and to plan treatment strategies among ICU patients.