Diagnostic Performance of a Blood Urea Nitrogen to Creatinine Ratio-based Nomogram for Predicting In-hospital Mortality in COVID-19 Patients

Background

At the end of 2019, a novel coronavirus pneumonia (COVID-19) broke out and spread rapidly throughout China and other countries.¹ The COVID-19 pandemic, resulting from the SARS-CoV-2 virus, is providing significant challenges to global public health systems and clinical management.^2,3 It is spread through respiratory droplets and contact,⁴ and elderly people and those with comorbidities are thought to have poorer prognosis.^5,6 Although the majority of COVID-19 patients have mild disease, severe and critically ill patients can rapidly progress to acute respiratory distress syndrome, septic shock, multiple organ failure and even death.¹ It is therefore imperative to identify prognostic indicators of disease severity, to assist the implementation of early measures to prevent the deterioration and death of critically ill patients. Routine clinical laboratory tests on blood may provide such prognostic factors, as they can reflect the altered biology of critical illness.⁷

Blood urea nitrogen (BUN) represents a surrogate marker for predicting persistent organ failure after 48 h of hospital admission, above and beyond its role in the estimation of renal function.^8,9 Elevated BUN level has been reported as a predictor of worse outcomes in patients with heart failure and BUN level with more than 7 mmol/L was one of the criteria of CURB-65 scoring for community-acquired pneumonia.^10,11 Recent studies have shown that the BUN to serum albumin ratio is an important prognostic factor of mortality and disease severity in patients with aspiration, hospital-acquired, and community-acquired pneumonia.^12–14 Several studies have suggested that an elevated ratio of BUN to creatinine (BCR) is associated with prolonged intensive care and mortality in patients with critical illness.^7,15 In addition, BCR is a poor prognostic risk factor for chronic heart failure, ischemic stroke and serious disease that persists after major trauma.^7,16,17 Of note, a recently published article indicated that the combination of chronic kidney diseases on admission or the development of acute kidney injury during hospitalization in patients with COVID-19 was significantly associated with in-hospital mortality.¹⁸ Therefore, it is hypothesized that critically ill patients with COVID-19 could have changes in BCR during hospitalization, and that BCR may be considered as a prognostic marker of severe disease and mortality. In this study, we systematically quantify the various levels of serum BCR in COVID-19 patients, and explore its relationship to disease severity and prognosis. As normal reference levels for BCR are not yet established, we determined the distribution of BCR values from a general “healthy” population, and assessed the degree to which our patients with COVID-19 were outside of the normal range of BCR.

Patients and Methods

Study Participants

Members of the general population (aged 14–75 years) were enrolled, who underwent a routine (nonurgent) medical health check between January 1 and December 31, 2016, at the Health Management Center of Tongji Hospital in Wuhan (n=26,524). As part of the health check, all individuals had anthropometric measurements taken, and they provided overnight fasting blood samples and information regarding their medical history. We excluded individuals with diabetes mellitus, hypertension, hypercholesterolaemia, heart failure, BMI <18.5 kg/m² or BMI ≥25 kg/m², glomerular filtration rates (eGFR) of <60 mL/min per 1.73 m², glomerulonephritis or chronic renal diseases, pregnant women, tumor. After exclusions, a total of 9165 subjects were included in the study. The BCR distribution across age and sex groups was analyzed (see below), thereby providing normal ranges. This study was approved by the Ethics Committee of Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology (the Institutional Review Board approval number: TJ-C20160115). The study conforms to the principles outlined in the Declaration of Helsinki, and written informed consent was obtained from all participants.

We collected data on 442 COVID-19 patients admitted to hospital from January 10, 2020 to February 27, 2020 treated at Tongji Hospital in Wuhan, all patients older than 14 years of age. We excluded patients who lacked urea nitrogen or creatinine biochemistry measurements (n=8), those with influenza A or influenza B virus (n=5), those with uremia with maintenance hemodialysis or peritoneal dialysis (n=2), those with cachexia status (n=4), those with others (such as active gastrointestinal bleeding, active glomerulonephritis, urinary tract obstruction, n=0) and those who died within 48 h of admission (n=7). We collected COVID-19 patients who went to the designated hospital directly after the onset of fever and were admitted to the hospital without any other treatment after the diagnosis was confirmed. A total of 416 patients with COVID-19 were included in our study. The patients were further divided in the light of the different branch of Tongji Hospital, forming a training set (n=337) for nomogram construction and a validation set (n=79) for external verification. Diagnosis and clinical types of COVID-19 were classified according to clinical guidelines (version 5 trial) developed by the National Health Committee of the People’s Republic of China.¹⁹ All SARS-CoV-2 infections were confirmed by real-time-PCR (RT-PCR) from upper respiratory throat swab samples provided by the patients. Epidemiological, demographic, clinical, laboratory, management, and outcome data were obtained from patients’ medical records. In addition, all patients received a chest computerized tomography examination. The BCR of 416 patients at admission was calculated. The estimated glomerular filtration rate (eGFR) was calculated using the Chinese improved modification of diet in renal disease formula eGFR (mL/min per 1.73 m²)=175×serum creatinine (Scr) ^–1.234×age^–0.179×0.79 (if female). The prognostic significance of BCR below and above the normal range was assessed using mortality as the endpoint. This study was approved by the Research Ethics Committee of Tongji Hospital (cord number: TJ-IRB20200353). The written informed consent was waived by the Ethics Commission for emerging infectious disease. Because COVID-19 is highly infectious, patients with the COVID-19 disease need to be isolated, and it was not practical to sign informed consent. Therefore verbal consent was obtained or given by the next of kin from each patient.

Statistical Analysis

Analyses were performed using SPSS 23.0 (IBM Corporation, Armonk, NY, USA) and R (version 3.6.2) statistical packages. All statistical tests were two-sided and P-value <0.05 was considered statistically significant.

A linear regression model was fitted to the BCR data for the general population (n=9165) with age and gender, thereby allowing the direct inference of 95% prediction intervals that reflect normal range values. On the basis of their BCR values, 416 COVID-19 patients were then assigned being below, within, and above the normal ranges. Categorical variables were expressed by number (%), and continuous variables were expressed by median IQR. The relationship between baseline characteristics (eg age) and each BCR-based group was compared by using the Kruskal–Wallis test, chi-squared or Fisher's exact probability tests, where appropriated. The main outcome was mortality, and the least absolute shrinkage and selection operator (LASSO) regression method was used to select the most useful predictive features from the primary data set. Prognostic models were fitted multivariable logistic regression models, which included the BCR-based groups and other factors. The nomogram for training cohort was established with the rms package in R statistical packages and the predictive accuracy of nomogram was evaluated by calculating the area under the curve (AUC) through the receiver operating characteristic (ROC) curve. To correct overfitting bias, a corrected AUC was calculated using bootstrapping validation (1000 bootstrap resamples) in the training cohort and validation cohort. Calibration of the nomogram was assessed by the calibration curve with Hosmer–Lemeshow test. Moreover, the verification cohort for external verification was employed to calculate the total score of each patient in the verification cohort according to the established nomogram, and constructed the ROC curve and calibration curve. Finally, we evaluated the clinical usefulness and net benefit of the new predictive models using decision curve analysis (DCA) and clinical impact curve analysis (CICA).

Results

Normal Range of BCR in General Population

The median BCR in general cohort (n=9165) was 15.5 (IQR: 12.9–18.8), and lower than in the COVID-19 cohort (17.8; IQR: 13.7–24.5) (P<0.001) (Figure 1A). Using the general cohort data, linear regression models were fitted and revealed a significant age–gender interaction effect on log BCR (P<0.001). The models were therefore fitted separately for each gender. The log BCR increased with age in both sexes, but in women increased more (Figure 1B). The gender specific models provided inference on age effects with 95% prediction intervals (a normal range) (Supplementary Table 1).

Figure 1 The distribution characteristics of blood urea nitrogen (BUN):creatinine ratio in general healthy population and COVID-19 patients. (A) Baseline blood urea nitrogen (BUN):creatinine ratio in general healthy population (normal) and COVID-19 patients. The box represents IQRs, the horizontal line in each box represents the median and the whiskers show the 10–90 percentile range. Scatter plot of association between ages vs log BUN:Creatinine ratio by gender in general healthy population (normal) (B) and COVID-19 patients (C). Solid lines express predicted log BUN:creatinine ratio by age and gender with 95% prediction intervals (shaded area) for each gender.

Clinical Characteristics of COVID-19 Patients

In COVID-19 patients, log BCR also varied widely and increased with age in both sexes (Figure 1C). The upper and lower 95% prediction limits were calculated from age and sex for each patient. According to established normal values of BCR in the general population, COVID-19 patients (n=337) in the training cohort were divided into three groups: higher than normal range of BCR group (n=45; 13.4%), lower than normal range of BCR group (n=38; 11.3%) and BCR within normal range (n=254; 75.4%). The proportion of patients with critical COVID-19 is higher in above normal range of BCR (P<0.001). More patients in the above normal BCR group received mechanical ventilation. The BCR higher than normal range group was associated with male, diabetes and cardiovascular disease (P<0.05), see Table 1 for baseline characteristic analyses. Whilst, white blood cell count, neutrophil count, D-Dimer and BUN in those above the normal BCR range were significantly higher than other groups, and the lymphocyte and platelet counts were significantly lower (P<0.05). However, there was no significant statistical difference in either creatinine or eGFR among the groups. The rate of fever, highest temperature, and cough showed no significant difference among the three groups (Table 1).

Table 1 Baseline Characteristics of Each BUN:Creatinine Ratio Group in Patients with COVID-19

Kinetic Analysis of BCR Levels in the Serum of COVID-19 Patients

The baseline levels of BCR were higher in severe and critical cases, compared to the common cases (25.9±11.9 vs 14.2±2.9; P<0.0001). In severe and critical cases, the BCR level increased significantly on the day of transfer to intensive care unit as compared to the day of admission (25.9±11.9 vs 32.7±10.8; P<0.001). However, no significant differences in BCR levels were observed during the whole course of the common cases (Figure 2).

Figure 2 Kinetic analysis of blood urea nitrogen (BUN):creatinine ratio levels of COVID-19 patients. The levels of BUN:creatinine ratio in the serum of mild cases (red line) and severe and critical (blue line) COVID-19 patients were analyzed at different time points after hospital admission, *P<0.001, **P<0.0001. Green dotted line presents upper limit value of BCR reference range, black dotted line presents lower limit value of BCR reference range. Abbreviation: NS, nonsignificant.

Association Between BCR Groups and Outcomes of COVID-19 Patients

The majority of COVID-19 patients in the above normal range BCR group died (88.9% vs 43.5% for non-high). Kaplan–Meier curves for all-cause mortality outcomes showed that those patients with above normal range of BCR had worse prognosis (P<0.0001). There was no significant difference in all-cause mortality between the lower and normal BCR groups (P=0.081) (Figure 3). Univariate logistic regression models found 23 variable baseline characteristics related to death (Supplementary Table 2). Further application of the LASSO regularized regression approach with all these variables found that only age, neutrophil count, platelet, lactic dehydrogenase, C-reactive protein (CRP), random blood glucose, BCR and BUN were the predictive factors for death incidence when the lambda was one standard error (Supplementary Figure 1). These eight factors and gender were included in multivariate logistic regression model. It was found that above normal range of BCR was associated with significantly higher risks of death after adjusting for sex, age, neutrophil count, platelet, lactic dehydrogenase, CRP, random blood glucose, cardiovascular disease and diabetes (OR: 6.20; 95%CI: 1.18–32.58; P=0.031), but not after further adjusting for BUN (P=0.086) (Table 2). These results suggested that BCR was one of prognostic factors affecting the prognosis for poor outcome.

Table 2 Logistic Regression Analysis for Outcomes in Patients with COVID-19

Figure 3 Kaplan–Meier survival curves for all-cause in-hospital mortality for the three BUN:creatinine ratio groups in patients with COVID-19.

Development of an Individualized Prediction Nomogram Based on BCR

To provide the clinician with a quantitative tool to predict individual probability of death, the nomogram was built on the basis of multivariable logistic analysis results, which integrated age, sex, BCR in the training cohort (Figure 4A). In the training cohort, the calibration curves showed good agreement between predictions and observation in the training cohort (Figure 4B), and the statistical data obtained by the Hosmer–Lemeshow test is not significant (P=0.828), which indicated that there was no deviation from the perfect fit.

Figure 4 Development and calibration curves of a prediction nomogram in patients with COVID-19. (A) The nomogram was developed in the training cohort for predicting mortality, with the age, gender and BUN:creatinine ratio incorporated. (B) The calibration curve for the nomogram in the COVID-19 training cohort. (C) The calibration curve for the nomogram in the COVID-19 validation cohort. Calibration curves depict the predicted probability of death compared well with the actual death.

Performance, Validation and Clinical Utility of Nomogram Based on BCR

Furthermore, an independent validation of the nomogram performance was tested in the validation cohort. Except the neutrophil count, there were no significant differences in age, sex, comorbidity, BUN, BCR, CRP, and disease types between the training cohort and validation cohort (Table 3). The calibration curves for predicting death showed that the nomograms were also in good agreement between prediction and observation in the validation cohort (Figure 4C). To compare the death predictive values of the nomogram, ROC analysis was applied in the training cohort, resulting in the c-index of 0.838 (95%CI, 0.795–0.880), which was confirmed to be 0.834 via bootstrapping validation (Figure 5A). Using maximum Youden index, the cutoff values of 59.8 points, the sensitivity and specificity in predicting mortality were 75.4% and 81%, respectively. Similarly, the c-index of the nomogram in the validation cohort was 0.929 (95%CI, 0.869–0.989), and the Hosmer–Lemeshow test yielded a nonsignificant statistic (P=0.439) (Figure 5B). While, the predictive ability of BCR alone for the incidence of in-hospital mortality was modest (AUC: 0.768; 95%CI: 0.717–0.819, P<0.001), at a BCR cutoff value of 18.6, the sensitivity and specificity in predicting mortality were 73% and 77.6%, respectively (Figure 5C). The above normal range of BCR also had good predictive ability for death (AUC: 0.927; P<0.001) (Supplementary Figure 2). In addition, the clinical utility of the nomogram was analyzed by the DCA and CICA. DCA showed that the threshold probabilities of 0.1–1.0 was the most beneficial for predicting death with nomogram in the training cohort (Figure 5D). Then, further performed the plotted CICAs to evaluate the clinical impact of the nomograms to help us more intuitively understand its substantial value. CICAs of the nomogram in the training cohort and validation cohort (Figure 5E and F) showed that the nomogram had remarkable predictive power when the risk threshold was in the range of 0–0.65, and the net benefit would be satisfied in the same range.

Table 3 Baseline Characteristics of the Study Cohort

Figure 5 The receiver-operating characteristics (ROC) curves analysis, decision curve analysis and clinical impact curve analysis of the nomogram in COVID-19 patients. (A) ROCcurve analysis on the nomogram for in-hospital mortality in training cohort. (B) ROC curve analysis on the nomogram for in-hospital mortality in validation cohort. (C) ROC curve analysis on the BUN:creatinine ratio for in-hospital mortality in training cohort. (D) The decision curves of the nomograms for predicting in-hospital mortality in the training cohort was plotted. The y-axis represents the net benefit. The x-axis shows the threshold probability. The horizontal solid black line represents the hypothesis that no patients experienced death, and the solid gray line represents the hypothesis that all patients met the endpoint. The clinical compact curves of the nomogram to predict in-hospital mortality in training cohort (E and F) in validation cohort are shown.

Discussion

Despite global control measures, the number of COVID-19 diagnoses and deaths has continued to rise. While there is a crude mortality rate of ~2.3%, for patients developing into critical cases the rate is 49%.⁵ Determining the factors that identify severe and critical patients early will improve the recovery rate and reduce mortality. We investigated the potential predictive ability of BCR, which has been associated with prolonged intensive care and mortality in patients with critical illness. Because there are no BCR reference intervals for the Chinese population, we determined these in a cohort undergoing a routine medical health check. Our findings are in line with previous studies in other populations for evaluating the normal values of BCR.²⁰ Our analysis revealed clear gender differences in age-related changes in BCR, which was accounted for using linear regression analysis, and allowed us to create a normal BCR range, independent of disease severity outcomes. Other studies have used the severity of the disease or quantiles of target parameters to create BCR groups, which leads to relative results that may not be robust recognized reference ranges for clinical practice.

In the present study, we showed several novel findings regarding BCR both in the general population and in patients who were hospitalized for COVID-19. Compared with men, BCR was higher in women and gradually increased with age. BCR was higher in COVID-19 patients compared with general population, and 35 (14.4%) of the patients with COVID-19 had an above normal range of BCR. The proportion of critical COVID-19 patients was also highest in those with above normal range values. More generally, higher BCR was related to the severity of COVID-19, and in a Kaplan–Meier analysis, poorer survival and prognosis. In addition, we established a nomogram including age, sex and BCR, which can more intuitively assess an individualized COVID-19 patient’s risk of death. In particular, in the training cohort and verification cohort, the c-index of the nomogram are 0.838 and 0.929, respectively. The diagnostic efficiencies were a better comparison with BCR alone with AUC value of 0.768 (0.717–0.819). DCA is a new method for evaluating diagnostic tests, predictive models, and molecular markers. This method can also be easily extended to many commonly used applications for predictive model performance measurement.²¹ Therefore, DCA and CICA were carried out in this study, which further showed that our nomogram provided significant clinical net benefit, which is of great value for accurate individualized assessment of the incidence of severe COVID-19.

Many studies have reported an association between BUN:creatinine ratio and worse prognosis of acute or chronic heart failure.^16,20 Elevated admission BUN:creatinine ratio is strongly associated with increased mortality of renal dysfunction with decompensated heart failure and as novel biomarker of critical illness-associated catabolism.^7,22 A recent study reported that the BCR was associated with poorer survival outcomes in hospital-acquired pneumonia and aspiration pneumonia, where the AUC of predicting the 30-day mortality and 28-day mortality were less than 0.7.^13,14 In this study, the AUC of BCR predicting in-hospital mortality was 0.768. Especially, the above normal range of BCR had greater predictive ability, with higher AUC values in predicting mortality in patients with COVID-19 pneumonia. This study is the first to focus on BCR as a prognostic factor for viral pneumonia. Further, as BUN and creatinine are routinely and rapidly measured in patients in a hospital setting, BCR would be an easy-to-obtain routine indicator and a useful early marker in COVID-19 patients, who require rapid and timely decision making. In our study, a nomogram was proposed and verified in accordance with age, gender, and BCR, which had the ability to predict the risk of death in a single patient. Both doctors and patients can use this easy-to-use scoring system to personally predict the risk of COVID-19 death in a timely manner and make early clinical decisions about patient treatment.

Nevertheless, this study has several limitations. First, the study was conducted a single-center retrospectively, and there is a need to validate our findings in an external cohort. Second, the study sample size is not large enough to sufficiently represent all COVID-19 patients. The number of patients classified as BCR higher than normal range is small. Finally, the study does not have a long time follow-up, and it is possible there may be poor outcomes in those discharged from hospital who have normal levels of BCR. These limitations aside, our work has derived a set of normal ranges for BCR, and revealed its potential prognostic value in COVID-19 patients.

Conclusions

BCR is an age- and gender-related bio-measure that varies widely in the general Chinese population. The BCR was a useful prognostic factor affecting the prognosis for patients with COVID-19. We established a practical nomogram on the grounds of age, sex and BCR, and internal cohort validation results demonstrate that the model can identify and exactly predict the risk of COVID-19 death. The nomograms should therefore help clinicians to improve individual treatment, make clinical decisions timely and early. Overall, our study shows that BCR may be an informative marker in patients with COVID-19 disease monitoring.