Development and Validation of a Rapid and Efficient Prognostic Scoring System for Sepsis Based on Oxygenation Index, Lactate and Glasgow Coma Scale

Introduction

Sepsis is a life-threatening systemic inflammatory response syndrome caused by infection.¹ It is estimated that tens of millions of severe sepsis cases occur worldwide each year and at least more than 5 million people die from sepsis.^2,3 As the world ages, the morbidity and mortality of sepsis are likely to increase further.^3–5 Early recognition and appropriate management of sepsis can improve the prognosis.⁶ Therefore early assessment and stratified management of sepsis are important.

Since sepsis is a time-related disease and the first medical contact for most of these patients is in the emergency department (ED).^7,8 Therefore, identifying and stratifying the management of septic patients with poor outcomes and high risk of death at an early stage is what emergency departments need to face.

Systems commonly used to assess the severity of sepsis include the Acute Physiologic and Chronic Health Evaluation (APACHE) II,⁹ Sequential Organ Failure Assessment (SOFA),¹⁰ quick Sequential Organ Failure Assessment (qSOFA),² Mortality in Emergency Department Sepsis Score (MEDS),¹¹ Systemic Inflammatory Response Syndrome (SIRS) score, and Modified Early Warning Score (MEWS).¹² APACHE II and SOFA are often used to assess the risk level of patients in the ICU but are rarely used for early assessment in the emergency department outside the ICU. It is also not easy to perform rapid and reproducible measurements and assessments due to the number of items involved. MEWS involves more items and it does not perform better than qSOFA.¹³ Previous studies have shown that qSOFA is more specific than SIRS but less sensitive and not suitable as a screening tool at the emergency bedside.^14–16 MEDS has a good prognostic value for sepsis, but the overall assessment ability is average and involves laboratory tests and imaging, which require more time to complete.^17–19 Therefore, there is no efficient, concise, and rapid assessment method to meet the need for rapid and accurate assessment of sepsis in the emergency department.

Previous studies have shown a strong correlation between lactate(Lac),^20–22 oxygenation index²³ and Glasgow coma scale score (GCS) score^24,25 and sepsis prognosis. Of these, Lac and oxygenation index can be obtained within minutes by bedside arterial blood gas analysis, while Glasgow coma scale score GCS scores can be obtained quickly when the physician examines the patient. Combined with our previous experience in combining indicators from bedside arterial blood gas analysis to rapidly assess disease prognosis in our study.²⁶ Therefore, we attempted to combine Lac, oxygenation index and, GCS scores to assess the prognosis of sepsis.

Materials and Methods

Study Design

This is a single-center retrospective cohort study. The study complied with the Declaration of Helsinki and the study protocol was approved by the Human Ethics Committee of West China Hospital, Sichuan University. We followed up all included patients by telephone.

Study Population

The study used data from the West China Hospital retrospective sepsis database. This single-center database retrospectively included data from all adult (≥18 years old) patients who developed sepsis in the ED of West China Hospital of Sichuan University between July 2015 and June 2016. In this study, only patients with a first diagnosis of sepsis according to sepsis-3 were eligible for inclusion. Patients were excluded if they were pregnant, in cardiac or respiratory arrest, or taking vasoactive drugs before admission.

Data Collection

Demographic data, in addition to data on vital signs at admission, medical history, laboratory data and, final diagnosis, were obtained from the patient’s electronic medical records in the ED. These data have been reviewed by a trained study coordinator. Hematological analytes including PH, pCO2, pO2, lactate (Lac) white blood cell count (WBC), PLT, hemoglobin and hematocrit (HCT) have been analyzed using the Cobas-b-123 system (Roche) (ABG analyzer) and have been analyzed using the automated hematology analysis system Beckman Coulter LH750 (Beckman Coulter Corporation, Brea, CA, USA) for analysis. Albumin, urea nitrogen, creatinine, cystatin C (Cys-C), total bilirubin, alanine aminotransferase (AST) and aspartate aminotransferase (ALT) have been analyzed using an Architect c16000 analyzer (Abbott Diagnostics Inc.).D-dimer has been analyzed using a Sysmex CA-7000 analyzer (Siemens Medical Diagnostics Inc.) The D-dimer values shown in this study are in D-dimer units, not fibrinogen equivalent units. PCT had been measured with a Cobas S6000 Hitachi (Roche Diagnostics Inc.). Oxygenation index has been calculated from arterial partial pressure of oxygen/inhaled oxygen concentration. Physicians stratified risk for sepsis according to APACHE II and SOFA based on the patient’s baseline clinical characteristics.

Study Follow-Up and Primary Endpoints

All patients with sepsis received a 28-day follow-up. The primary endpoint was all-cause mortality during follow-up. Secondary endpoints were adverse outcomes, including mechanical ventilation (MV) and admission to the intensive care unit (AICU). All sepsis patients underwent a structured telephone interview with an emergency physician to determine all-cause mortality and other adverse outcomes at 28 days. All in-hospital data were matched to hospital records (Figure 1).

Figure 1 Flow chart of the study. Abbreviations: AUROC, area under the receiver operating characteristic; ROC, receiver operating characteristic curve; GCS, Glasgow Coma Scale; APACHE II, Acute Physiology and Chronic Health Evaluation II; SOFA, sequential organ failure assessment.

Statistical Analysis

Categorical variables were presented as frequencies and percentages, whereas continuous variables were presented as means ± standard deviation (SD) for normally distributed data or medians and interquartile range (IQR) for abnormally distributed data. Construct time-dependent receiver operating characteristic (ROC) curves to compare prognostic performance and determine optimal cut points for different metrics. The area under the curve (AUC) was compared using the non-parametric method proposed by Hanley and McNeil.²⁷ Construction of a new scoring system using the two highest AUC values from arterial blood gas analysis together with GCS. Univariate analysis was performed with the new score as a variable along with other variables between the survival and death groups. The indicators with significant differences in the univariate analysis were then subjected to multivariate logistic regression analysis to determine whether the new score was an independent predictor of mortality. Discriminative power analysis with AUROC; calibration capability analysis with Hosmer-Lemeshow. The newly constructed scoring system was then used to compare patient characteristics. Parametric patient characteristics were compared using one-way ANOVA tests, whereas nonparametric characteristics were compared using Kruskal–Wallis H-tests; categorical data were compared using χ²-tests. The relationship between the new scoring system and the various endpoints was examined using multivariate logistic regression models. Decision curve analysis was used to quantify the clinical utility of the new scoring model.²⁸ p-values <0.05 were considered statistically significant. Data analyses were performed using SPSS version 25.0 (IBMCorp, Armonk, NY, USA) and Stata version MP17.0 (StataCorp LP, College Station, TX, USA). Graphing with GraphPad Prism version 9.4.1.681.

Results

Clinical Characteristics of the Derivation Group

A total of 841 patients with sepsis were enrolled in the study; however, 20 patients were lost to follow-up and were therefore excluded from the study population. Thus, the final study population consisted of 821 patients; the mean age at admission was 57 ± 18 years, and 64.3% of the patients were male. The first 608 patients were then included in the derivation group according to the order of admission, and the remaining 213 patients were included in the validation group.

In the derivation group, the mean age was 58 years, of which 64.1% were men. The median SOFA and APACHE II scores were 5 (IQR 3–7) and 15 (IRQ 12–20), respectively. During the 28-day follow-up period, there were 131 deaths (Table 1).

Table 1 Comparison of Base Information of Derivation and Validation Groups

Establishment of the New Scoring System

The area under the ROC curve (AUC) for predicting 28-day mortality using Lac (0.645, P<0.001) or oxygenation index (0.678, P<0.001) was greater compared with other indices from arterial blood gas analysis (<0.6). In addition, the GCS score, one of the components of SOFA and APACHE II, is an important predictor of disease prognosis, and the AUC for a GCS score less than 15 to predict 28-day mortality in this study was (0.604, P<0.001). Since the GCS score is also simple and easy to obtain, a new scoring system was formed using the oxygenation index, Lac, and GCS scores, and named GOL based on the initials of these three indicators. The calculation of GOL is shown in Table 2.

Table 2 Weight and Score of Scoring Items in the New Scoring System (GOL)

Clinical characteristics among sepsis patients according to GOL are shown in Table 3. Heart and breathing rates, Lac, urea nitrogen, AST, and PCT were significantly increased with increasing GOL. Conversely, systolic blood pressure (SBP), diastolic blood pressure (DBP), SPO2, PH, Oxygenation index, Albumin, and Fibrinogen significantly decreased with decreasing GOL. The other clinical characteristics did not vary significantly based on GOL (p > 0.05).

Table 3 Relationship Between Clinical Characteristics and GOL Scores of Patients with Sepsis in the Derivation Group

Then, we performed univariate analysis of GOL, SOFA, APACHE II, age, gender, vital signs, GCS, arterial blood gas analysis indexes and laboratory indexes, and then those with significant differences (p < 0.05) were then subjected to multivariate logistic regression analysis. The results showed that GOL, SOFA, APACHE II, GCS, Lac, and albumin were independent predictors of 28-day mortality in sepsis (p < 0.05) (Table 4).

Table 4 Univariate Analysis and Multivariate Logistic Regression Analysis Between Survival and Death Groups in the Derivation Group

The AUROC value for GOL predicting 28-day mortality was 0.723 (95% CI: 0.672–0.775, OR=2.916, P<0.001) (Figure 2), and its Jorden index reached a maximum of 0.364 when GOL ≥2(Sensitivity, 54.2%; Specificity, 82.18%). The logistic regression analysis Hosmer-Lemeshow goodness-of-fit test shows that GOL has good calibration ability (p=0.327).

Figure 2 ROC curves of GOL, SOFA and APACHE II in the derivative group. Abbreviations: ROC, receiver operating characteristic curve; GOL, the new scoring system developed in this study; APACHE II, Acute Physiology and Chronic Health Evaluation II; SOFA, sequential organ failure assessment.

GOL and the Severity of Sepsis

Sepsis patients with higher GOL had higher SOFA and APACHE II scores when compared to patients with lower GOL. For those with GOL of 0, 1, 2, and 3, the respective median SOFA scores were 3 (IQR 1–5), 5 (IQR 3–7), 7(IQR 5–10), and 14 (IQR 10–17) (p < 0.001), and the respective mean APACHE II scores were 12 (IQR 9–16), 16 (IQR 12–20), 19 (IQR 14–24) and 25 (IQR 23–34) (p < 0.001) (Figure 3).

Figure 3 SOFA (A) and APACHE II (B) levels in different OLG scores. Abbreviations: GOL, the new scoring system developed in this study; APACHE II, Acute Physiology and Chronic Health Evaluation II; SOFA, sequential organ failure assessment.

GOL and in-Hospital Adverse Outcomes

The 28-day mortality, MV, and AICU rates for patients gradually increased as GOL increased. The 28-day mortality rate of patients with scores of 3 or 2 or 1 was 8.2 or 4.3 or 2.1 times higher than that of patients with scores of 0, respectively (Figure 4A). The MV rate of patients with scores of 3 or 2 or 1 was 6.0 or 4.9 or 3.0 times higher than that of patients with scores of 0, respectively (Figure 4B). The AICU rate of patients with scores of 3 or 2 or 1 was 5.2 or 4.3 or 2.6 times higher than that of patients with scores of 0, respectively (Figure 4C).

Figure 4 The 28-day mortality (A), MV (B), and AICU (C) with different OLG scores. Abbreviations: MV, mechanical ventilation; AICU, admission to intensive care unit; GOL, the new scoring system developed in this study.

Univariate logistic regression models indicated that the GOL was positively associated with the risks of mortality, MV, and AICU. After adjusting for these potential confounders in the multivariate logistic regression analysis, the GOL was independently associated with MV and AICU. However when adjusted for risk factors, GOL=1 was not an independent predictor of 28-day mortality, but only when GOL≥2 was its independent predictor. (Table 5).

Table 5 Logistic Regression Analysis Regarding Correlations Between Clinical Outcomes and GOL

Comparison Between GOL and Other Prognostic Scores

We assessed the performances of qSOFA, MEWS, MEDS, APACHE II and SOFA in the prediction of 28-day mortality. The C-index for the GOL was significantly better than that of qSOFA (0.675, 95% CI =0.636–0.712), MEWS (0.621, 95% CI = 0.581–0.659), and MEDS (0.634, 95% CI =0.594–0.672). In addition, there was no significant difference between GOL and APACHE II scores (0.701, 95% CI =0.663–0.737) and SOFA scores (0.764, 95% CI=0.728–0.797). Decision curve analysis showed that for threshold probabilities above 0.18, GOL had a higher net benefit compared to APACHE II, and for threshold probabilities above 0.43, GOL had a higher net benefit than SOFA (Figure 5).

Figure 5 Decision curve analysis for the outcome of 28-day mortality. For clinically relevant threshold probabilities (above 18% and 43%), OLG was superior to APACHE II and SOFA, respectively. Abbreviations: GOL, the new scoring system developed in this study; APACHE II, Acute Physiology and Chronic Health Evaluation II; SOFA, sequential organ failure assessment.

Validation of the Ability of GOL to Predict the Prognosis of Sepsis

In the validation group, the mean age was 56 ±19 years, of which 64.8% were male. The median SOFA and APACHE II scores were 5 (IQR 2–8) and 15 (IRQ 10–20), respectively. There were 42 deaths during the 28-day follow-up period (Table 1). OLG score has good discrimination (AUROC=0.785, p<0.001) and calibration (Hosmer-Lemeshow goodness-of-fit test, p=0.774) in the validation group. We compared the performances of GOL, APACHE II, and SOFA in the prediction of 28-day death, AICU, and MV. We found that GOL was superior in predicting AICU (p= 0.030) and MV (p= 0.008) compared with APACHE II, while both were comparable in predicting 28-day death (p= 0.902). The GOL and the SOFA were comparable in their ability to predict 28-day death (p= 0.735) and AICU (p= 0.498), while the SOFA outperformed the GOL in predicting MV (p= 0.011) (Table 6).

Table 6 AUC Comparison of Adverse Outcome Prediction Based on GOL, APACHE II, and SOFA in Validation Cohort

Discussion

In the present study, we derived and validated a new scoring system, GOL, to assess sepsis prognosis. We found that higher GOL scores were associated with more severe sepsis and more poor prognosis. After adjusting for confounders, when the GOL score was greater than 1 still independently predicted mortality, AICU, and MV within 28 days of admission. And in subgroup analysis, GOL was also an independent risk factor for poor prognosis. In addition, we compared GOL with other scores and found that GOL had better predictive power than qSOFA, MEWS, and MEDS, and was comparable to APACHE II and SOFA. The performance of GOL was also validated in the validation group.

In the validation group, GOL predicted 28-day death and AICU as well as APACHE II and SOFA, and GOL demonstrated better performance than APACHE II in predicting MV.

In addition, the GOL score consists of only three indicators: oxygenation index, Lac, and simplified GCS score, and its greatest advantage is its simplicity and speed. The presence of each variable contributes one point to a total 3-point score. Among them, the oxygenation index is the ratio of arterial partial pressure of oxygen to the concentration of inhaled gaseous oxygen (PaO2/FiO2), which is an important indicator for assessing acute lung injury (ALI) or acute respiratory distress syndrome (ARDS),²⁹ a common clinical complication of sepsis.³⁰ Previous studies have shown that cytokines in the course of sepsis can mediate the aggregation of large numbers of immune cells, activating intracellular signaling pathways and further releasing large numbers of cytokines. Inflammatory cells are continuously activated, creating a vicious cycle that eventually leads to a cytokine storm. The lung epithelial cell injury and endothelial cell injury caused by these cytokine storms may ultimately lead to ALI/ARDS.^31–34 ARDS is classified into different levels of severity depending on the oxygenation index: mild (200 mm<PaO₂/FiO₂≤300mmHg), moderate (100mmHg<PaO₂/FiO₂≤200mmHg), and severe (PaO₂/FiO₂<100 mmHg).²⁹ In this study, we found that the oxygenation index showed a significant inverse relationship with poor prognosis and a cut-off value of 271, with a significantly higher incidence of poor prognosis when the oxygenation index was less than 271. This is similar to the SOFA score, where the lower the oxygenation index, the higher the SOFA score and the higher the incidence of poor prognosis.

Previous studies have shown that early lactate levels are associated with organ dysfunction and mortality in the ICU and emergency department,^35–37 with elevated lactate levels positively correlating with increased mortality.³⁸ Serum lactate levels >2 mmol/L are recommended in sepsis 3.0 as the main criteria for clinical differentiation of septic shock.¹ The higher the lactate level, the worse the outcome.³⁹ Earlier studies showed a specificity of 96% for lactate concentrations >4 mmol/L in predicting in-hospital mortality in non-hypotensive patients.^2,40 Our study showed that lactate was an independent risk factor for 28-day mortality in sepsis, and mortality was significantly higher when serum lactate was greater than 2.3 mmol/L.

As two objective indicators in the new scoring system. Both arterial partial pressure of oxygen and Lac in the oxygenation index can be quickly obtained within minutes by a bedside arterial blood gas analyzer.

The level of consciousness is often used as an indicator to assess the prognosis of a disease. The GCS score, as a common assessment tool for the level of consciousness, has been used directly or indirectly to assess the risk level and prognosis of a variety of diseases.^{2,9–12,41,42} As the only subjective indicator in the new scoring system, we found a significant difference between a GCS score equal to 15 and less than 15 in predicting 28-day death in patients with sepsis. Therefore, we simplified the assessment process of the GCS score by contributing 1 point to the new scoring system once the GCS score was less than 15.

Limitations

The present study has several limitations. One, the study design was single-center and retrospective, and thus vulnerable to potential selection bias. Second, because of the increasing attention and motivation for the treatment of sepsis, we collected relatively old data on sepsis cases, which may be biased for current use. Third, in this study, the new scoring system was no worse than the APACHE II system, but slightly inferior to SOFA in predicting MV. Fourth, the GCS score is a subjective score, which has implications for the rigor of the new scoring system. Therefore, further multicenter prospective studies are needed to further validate the results of this study.

Conclusions

The new scoring system GOL, created in this study by combining Lac, oxygenation index, and GCS scores, can effectively predict 28-day mortality in patients with sepsis with significantly better performance than SIRS, qSOFA, MEWS, and MEDS, and comparable performance to APACHE II and SOFA. Its advantage is that it is rapid, concise, and can be used as a reference tool for prognostic assessment of sepsis in the emergency department.