A New Scoring System for Predicting Mortality in Hematological Malignancies with Sepsis: A Derivation and Validation Study

Introduction

Sepsis is a dysregulated immunological response to infection that can cause life-threatening organ dysfunction. In the intensive care unit (ICU), sepsis is major cause of mortality for critically ill patients. In 2020, there were 20.6% sepsis cases in ICUs in China, the mortality of sepsis was 35.5%, and the mortality of severe sepsis was over 50%.¹ Early diagnosis with prompt intensive care and infection source control before organ dysfunction is the acknowledged paradigm in sepsis care.

Patients with hematologic malignancies (HMs) are more susceptible to infections that spread quickly and progress to sepsis. Immune dysfunction or neutropenia brought on by the underlying malignancies and cytotoxic chemotherapeutic medications both play a role in this.² Compared to patients without malignancies, patients with HMs have an up to 10- to 15-fold increased risk of developing sepsis, who frequently necessitates transferring them to the ICU.^3,4 In particular, sepsis in patients with neutropenia was more common (ranges from 7% to 42%)⁵ and was associated with a higher mortality rate (greater than 30%).⁴ Due to the individuals with HMs may have a multitude of comorbid ailments and underlying diseases, the signs and symptoms of sepsis may vary considerably or be less noticeable, making early diagnosis and treatment of the sepsis very challenging. However, specific clinical scoring system can predict sepsis risk before it becomes a serious condition. The SOFA score was suggested as a useful tool to evaluate organ failure and sepsis prognosis in the Third International Consensus Definitions for Sepsis-3.0 in 2016.⁶ The SOFA score should change by at least 2 points from baseline to indicate organ dysfunction. The SOFA score is a crucial measure for assessing the seriousness of the patient’s condition and forecasting the early death.^7,8 The predictive validity of SOFA for in-hospital mortality among patients with infection in ICU was comparable with the more complex LODS and was statistically greater than systemic inflammatory response syndrome (SIRS), supporting its use in clinical sepsis criteria.⁹ However, the clinical use of SOFA score is still constrained and cannot be applied to HM patients without limitations: Due to thrombocytopenia caused by chemotherapy or tumor, platelet (PLT) cannot be employed; additionally, limited assessment is possible due to liver and kidney impairment brought on by chemotherapy.⁵ In addition, the Acute Physiology and Chronic Health Evaluation II (APACHE II) system is frequently employed to determine the severity of the patient’s condition in ICU. However, it underestimated the risk of death for septic patients.¹⁰ Thus, in order to predict mortality rate and risk stratification in patients with HMs and sepsis, we set out to develop a new scoring system.

Although coagulation and inflammation are necessary defenses against infection, sepsis-induced stimulation of these processes also results in life-threatening organ malfunction. A fatal complication of sepsis is known as sepsis-induced coagulopathy (SIC), which can range from thrombocytopenia to disseminated intravascular coagulation (DIC).¹¹ SIC scores are based on parameters including PLT count, PT-INR and SOFA score, are easy to calculate and have a higher predictive value for 28-day mortality. However, it still is not a reliable scoring system for patients with HMs. Thrombocytopenia, as one of its main assessment indicator, can also be caused by chemotherapy and tumors, and this condition decreased its prediction accuracy. In light of this, we speculate that SOFA score in conjunction with markers of coagulation function such as PT, international normalized ratio (INR), or activated partial thromboplastin time (APTT) may be more accurate at predicting the prognosis of patients with HMs and sepsis.

Up to now, the SOFA score has been served for decades as the industry benchmark for determining the prognosis of sepsis. The SOFA score may be more helpful in the diagnosis and prognosis of sepsis patients when paired with a variety of biomarkers, such as procalcitonin (PCT),^12,13 age, albumin level, and international normalized ratio,¹⁴ among others, according to studies published in recent years. Additionally, the ability of the SOFA score to predict outcomes may also be impacted by indications like coagulation indicators, blood lactate level and various underlying disorders. Those indexes and score models’ predictive significance have not, however, been systematically verified in patients with HMs and sepsis. We do not yet know whether patients with HMs and sepsis have better prognostic predictive value when these markers are combined with SOFA scores. To the best of our knowledge, there has been currently no reliable scoring system for determining when patients with HMs and sepsis should be admitted to the HICU or for early identification of severe sepsis. The aims of our study are to develop a new prognostic scoring system to identify patients with HMs and sepsis who have a high mortality rate.

Methods

Patients

A single-center retrospective study with medical chart review was performed. Data was gathered between October 2017 and May 2023. Three hundred and twelve patients diagnosed with HMs and sepsis who were admitted to the HICU, an 8-bed ICU, were identified, in the first affiliated hospital of Harbin Medical University, China. All patients were admitted directly from the hematology department ward. During the study period, only the values from their first 24-hour HICU admission were included in this analysis. All related information was gleaned from the medical records of our institution. We did not have any established criteria for admission to HICU, decisions were made in consultation between the attending HICU specialist and hematologist.

Patients from the following categories were included: adults were identified by a medical record search for all patients ≥18 years presenting with HMs and sepsis; evidence of documented infection and organ dysfunction represented by SOFA score of 2 points or more; those were admitted to the HICU and received early goal directed therapy (EGDT);¹⁵ within 28 days of receiving HICU medical care, patients who passed away or survived have been included.

Exclusion criteria for patients: patients’ medical care was discontinued or they were sent to another medical institution with an unclear prognosis; acute promyelocytic leukemia; COVID-19 and other deadly virus infections; non-septic related coagulation disorders, such as hemophilia, anticoagulant application or vitamin K deficiency in chronically undernourished patients, and so forth; non-septic related liver dysfunction was also ruled out.

Finally, 199 patients were eventually enrolled. With patient numbers of about 1:1 ratios, we randomly assigned all of the recruited patients to the “validation cohort” and the “derivation cohort”. Ninety-nine patients were included in the derivation group, and 100 patients were included in the validation group (Figure 1). The Harbin Medical University Ethics Committee gave its approval to this study, which was carried out in accordance with the Helsinki Declaration.

Figure 1 Flow diagram illustrating selection of patients.

Study Variables

For a follow-up of 28-day mortality, a retrospective study of medical records was completed. The 28-day survival rate was used to split all study participants into groups of survivors and non-survivors. The patient’s demographic information, vital status, clinical signs and parameters (peripheral blood cell count, coagulation indicators, inflammatory biomarkers, blood lactate levels, organ function characteristics, pathogens, SOFA scores and quick SOFA (qSOFA) score, etc.), as well as discharge follow-up clinical data, were extracted from our electronic medical record system within the first 24 hours of HICU admission.

Definitions and Criteria

The diagnosis of sepsis was confirmed genetically based on the Sepsis-3.0; qSOFA score is a bedside prompt that identifies patients at greater risk of death or prolonged HICU stay. Low systolic blood pressure (≤100mmHg), a rapid breathing rate (≥22 breaths per minute), and disturbed mental status were given the qSOFA score (3 points, one point each).¹⁵ Septic shock was recorded if the patient met the criteria for sepsis and had persistent hypotension after receiving enough intravenous fluids. When the absolute neutrophil count fell below 500 cells/mm,³ it was considered neutropenia.

Statistical Analysis and Calculation of the New Scoring System

IBM SPSS Statistics Version 25.0 (Chicago, IL, USA) was used to analyze the data. Give statistical advice and seek the counsel of qualified statisticians.

In the derivation study, univariate analysis was utilized to identify variables strongly associated with the 28-day mortality rate in patients with HMs and sepsis. The continuous variables with normal distribution were reported as mean ± standard deviation (SD) and compared using a t-test; the continuous variables with abnormal distribution were presented with medians and interquartile ranges and compared using a t-test; categorical variables were expressed as frequencies and compared using the chi-square test. In all statistical analyses, a p < 0.05 was considered statistically significant. Variables discovered in the univariate analysis were subjected to a logistic multivariate regression on a dichotomous dependent variable and were used to assess the odds ratio (OR) and p value for discovering independent predictors and were adjusted for confounding factors. Variables with p > 0.05 were taken into elimination criterion.

Creating an ROC curve and determining the optimum cut-off points for each independent variable. The overall discriminating accuracy for the link between each predictor and mortality was assessed using the AUROC, which is made up of the relationship between “sensitivity” and “1-specificity”. The optimum cut-off risk prediction points, sensitivity, specificity, odds ratio (OR), and specific AUROC (95% CI) values were calculated. AUROC values 0.60 are regarded as poor, 0.60–0.80 are regarded as moderate, and AUROC values >0.8 are viewed as good.¹⁶

Building a new scoring system and stratifying patients based on the optimum cut-off point. For each of the independent predictor, the optimum cut-off point was identified and the mortality OR and 95% confidence interval (CI) were calculated. Each predictor was dichotomized for the new scoring system according to the optimum cut-off point and given a score that was weighted according to risk prediction function,¹⁷ and a new scoring system was built using the sum of the component values. Then, the risk stratification was performed according to the new score and risk factor characteristics. The mortality in sepsis in the new scoring system was calculated for each patient by adding the weighted scores. Finally, we compared the performance of the new scoring system in predicting 28-day mortality with the performance of the SOFA scoring criteria.

Furthermore, we validated the results in the validation study, the new scoring system’s distinction and calibration were applied to the validation data. The Hosmer–Lemeshow X² statistic test was used to gauge the degree of calibration. Using Log rank testing, Kaplan–Meier curves for the new scoring system were analyzed.

Results

Cohort Characteristics

During the five years of the study, there were 312 patients with HMs admitted to the HICU who also met the Sepsis-3.0 diagnostic criteria for sepsis. One hundred and thirteen patients were excluded because they failed to meet the inclusion criteria. Ultimately, this study covered 199 patients with HMs and sepsis. The 199 patients were assigned randomly to the validation cohort which included the 99 patients and the derivation cohort which included the remaining 100 patients (Figure 1). The characteristics of the patient cohort were acquired, as shown in Table 1.

Table 1 Univariate Analysis of Variables in the Derivation Data Between Survivors and Nonsurvivors (N = 99)

Derivation Study

Among the derivation study of 99 patients with HMs and sepsis, with a median age of 58 years (range 18–90 years), 47 patients died during a 28-day hospital stay (47.47%, 49/99), with a median age of 61 years (range 18–90 years), and 52 patients survived (52.52%, 52/99), with a median age of 55 years (range 18–78 years). Age differences between the non-survival and survival groups were significant (p = 0.04). The most frequent diagnosis was leukemia (43.43%, 43/99), which was followed by Lymphoma (26.26%, 26/99), MDS (19.19%, 19/99), and MM (11.11%, 11/99). Leukemia had the highest fatality rate (17/47, 36.17%). There was, however, no statistically significant difference (P = 0.306). Diabetes (p < 0.001), SOFA Scores (p = 0.009), PT (p < 0.001), and DD (p = 0.026) all showed significant differences. At the time of admission to the HICU, there were no appreciable differences in the gender distribution, pathogen, WBC, neutrophils, HGB, PLT, or APTT, FIB, DD, Cr, CRP, PCT and blood lactate levels between the two groups (All p > 0.05, Table 1).

Age (P = 0.045), SOFA score (P < 0.001), and PT (P < 0.001) were shown to be significantly different between survivors and nonsurvivors in the derivation study of 99 patients dataset by binary logistic regression analysis (Table 2). PT (OR: 1.213; 95% CI: 1.030–1.430), age (OR: 1.242; 95% CI: 1.026–1.503), and SOFA score (OR: 1.442; 95% CI: 1.035–2.009) were all identified as independent risk factors by multivariate Cox regression analysis (Table 3).

Table 2 Univariate Cox Proportional Hazards Analysis for 28-Day Mortality

Table 3 Multivariate Cox Proportional Hazards Analysis for 28-Day Mortality

The optimum cut-off points for each independent variable were established according to ROC curves. The optimum cutoff points for SOFA score, new scoring system, age and PT were 9 points, 4.5 points, 60 years and 16s, respectively. For each independent variable, the odds ratio for mortality and the 95% confidence interval (CI) were calculated (Table 4). Each independent risk factor was given a score according to risk prediction function¹⁷ (Table 5). A new scoring system was devised, with scores ranging from 0 (low risk) to 7 (very high risk); Additionally, when the PT variable assigned 0 scores, the score was 1, 2, or 3, which is classified as medium risk; when the PT variable assigned points, the score was 4, 5, or 6, which is classified as high risk. Table 6 details the New scoring system’s risk stratifying patients based on the optimum cut-off point. ROC curves were drawn for New scoring system and SOFA score (Figure 2). In comparison to SOFA score, the new scoring system showed the greater AUROC (0.757, 95% CI: 0.661–0.853 vs 0.707, 95% CI: 0.607–0.808) and greater predictive sensitivity (75.0% vs 67%) and specificity (68.1% vs 63.8%), indicating a stronger ability to predict 28-day mortality (Table 4).

Table 4 Receiver Operating Characteristic Curve for the Different Indicators in Predicting the 28d-Mortality Rates in Septic Patients

Table 5 Determine Points Associated with Each of the Categories of the Risk Factors

Table 6 Risk Stratification Using a New Scoring System

Figure 2 AUC for 28-day mortality in the derivation and validation samples. The AUC of New score vs SOFA score in the derivation sample (A), and the validation sample (B). Solid line = New score; dotted line = SOFA score.

Validation Study

Another 100 patients made up the validation study, with a median age of 56 years (range 19–80 years), a male gender ratio of 59% (59/100), and a mortality rate of 43% (43/100). The 28-day mortality rates between the derivation and validation groups did not change significantly (P = 0.528). The illness source distribution resembles the derivation data in terms of distribution (Figure 3). Similar mortality rate and predictive power were found in the derivation and validation study based on the risk stratification of the New scoring system (Table 6).

Figure 3 The distribution of the source of sepsis in the derivation and validation samples. Abbreviations: MDS, myelodysplastic syndrome; MM, multiple myeloma.

There was no difference between the expected and observed results, according to the Hosmer–Lemeshow X² value of 4.73 (P = 0.45), which indicated that the model fit the data well. We evaluated the capacity of the new scoring system in predicting mortality in comparison to the SOFA score. The results are shown in Table 4: the ROC of the new scoring system had a higher AUROC than that of the SOFA score (0.792, 95% CI 0.70–0.88 vs 0.733, 95% CI 0.63–0.84). The new scoring system has a higher sensitivity (0.91 vs 0.842) than that of SOFA score. However, the specificity is no higher than SOFA (0.58 vs 0.58). Additionally, Kaplan–Meier survival analysis revealed that survival fell as the risk level of the new scoring system rose (p < 0.001) (Figure 4). The findings validated the proposed new scoring system as a useful tool for identifying severe sepsis among critically ill patients with MHs.

Figure 4 Kaplan–Meier survival curves of patients with HMs and sepsis with different levels of risk in the new scoring system.

Discussion

In this study, we have derived and validated a new scoring system using three independent variables to identify septic patients with HMs who are at a risk of early death. Our analysis has identified age 60 years or older, PT 27s or more and SOFA Score of 4.5 points or more were the three independent predictors. We found that the new scoring system performed better than SOFA.

Sepsis is a life-threatening syndrome, the total mortality rate is above 22.5%.¹⁸ Patients with HMs and sepsis tend to a comparatively higher mortality rate, and this is frequently attributed to immune dysfunction caused by the malignancies and cytotoxic chemotherapy treatments. The SOFA score was recommended to identify sepsis and evaluate organ function damage in HMs, a higher SOFA score is typically associated with increased mortality.¹⁹ According to Cornet et al retrospective study of the ICU stays of 58 patients with HMs and sepsis, nonsurvivors on the first four days of ICU admission had significantly higher SOFA scores than survivors.²⁰ Furthermore, Price et al found in a study of acute leukemia patients requiring invasive ventilation that a 1 point increase in SOFA score at the time of intubation was associated with a 17% increase in mortality.²¹ However, there are shortcomings in the clinical application of SOFA scores in HMs. The reasons are as follows: chemotherapy-related or tumor-related thrombocytopenia, PLT count cannot be used; Chemotherapy-related elevations in bilirubin and creatinine may have an impact on how the SOFA score is calculated; Assessment is sometimes also hampered by the fact that mental status can alter independently of sepsis.⁵ The aforementioned reasons may cause the SOFA score to be overestimated in patients with HMs and sepsis. In addition, APACHE II, the most widely used scoring system for the prognosis of severe patients, can accurately assess the prognosis of sepsis.²² However, its intricate evaluation metrics and calculation do not support the quick implementation of emergency bedside. To the best of our knowledge, there is the first scoring system that specifically predicts mortality and risk stratification for patients with HMs and sepsis. Our research highlights the importance of establishing a new scoring system that minimizes the shortcomings of SOFA scores in HM application.

In this retrospective study of 199 consecutive patients admitted to the HICU, we identified SOFA score, age and PT as independent predictors for 28-day mortality in patients with HMs and sepsis. Numerous studies have revealed that age factor, in particular, age greater than 65, is significant independent variable.^23,24 However, in this study, we defined age greater than 60 to be the critical cut-off value. The coagulation system and the inflammatory response are both activated by infection and engage in dynamic interaction. It has been shown that the activation of the coagulation system and inflammation aid the host in eliminating pathogens, but if the infection is not promptly controlled, the sepsis-induced coagulopathy (SIC) and SIRS will eventually become out of control, posing a life-threatening threat to multiple organ function. Recent advances in sepsis offered light on the role of SIC in the progression of sepsis, and SIC is closely associated with multiple organ failure and sepsis mortality rate.¹¹ Studies have shown that the SOFA score and coagulation indicator PT are excellent indicators of sepsis prognosis.²⁵ There are also research reports, high mortality in sepsis, when INR >1.5.^26,27 Our finding of an PT ≥16s associated with high mortality in patients with HMs and sepsis has not been reported previously. The current scoring systems for coagulopathy provide advantages for its specific applications. Using the diagnostic criteria for SIC that Iba proposed in 2017. Scores on INR, PLT, and SOFA are included in this criterion. The SIC criteria are suitable for detecting sepsis cases with a greater mortality risk.²⁸ However, it is not feasible to apply it in patients with HMs and sepsis. The Japanese Ministry of Health and Welfare’s (JMHW) DIC diagnostic scoring system included PLT, PT, fibrinogen degradation products (FDP) and FIB. The criteria say that secondary DIC in hematological disorders can be detected with ≥4 points and that PLT are not scored in the diagnosis of hematological diseases. However, its sensitivity for the diagnosis of SIC is poor.¹ The International Society of Thrombosis and Haemostasis (ISTH) and Japanese Association for Acute Medicine (JAAM) criteria for SIC demonstrate a relatively high diagnostic efficacy. PLT count, PT, FDP and FIB were used in both scoring systems which can be used to judge the prognosis of patients with trauma or sepsis. However, the sensitivity of both scoring systems is low, making it difficult to diagnose DIC in the early stage, although the effect is significant for the diagnosis of typical DIC.²⁹

In our study, we demonstrated that the proposed simple new scoring system performed better in identifying critically ill septic patients with HMs than SOFA. An earlier diagnosis of critically ill patients allows for the faster start of rational and effective care. Compared with the current researches, we did not identify PLT, FDP and FIB as independent predictors of the mortality. Yet, we demonstrated that PT was trending towards being a significant independent predictor of mortality. In addition, our study revealed that when the SOFA score paired with PT and age, they complement each other, and the new scoring system of sensitivity improves to over 75%, and the discrimination of new scoring system was much better than the SOFA score. However, the specificity of the scoring system was not materially improved by the limited predictors, so it should be used to identify critically ill patients with sepsis and HMs in the general hematology ward and receive timely intensive care. In the new scoring system, a score of 0 indicates a mortality rate that is comparatively low, whereas a score greater than 0 indicates a larger mortality rate. The mortality rate increases to over 50% when the new scoring system receives a score of 4 or more and prompt admission to intensive care may be considered. Our study contributes to the existing data by providing the new scoring system for predicting mortality among patients with HMs and sepsis. This new scoring system may serve as a useful guide for selecting variables for the creation of sepsis definition specifically for HMs in the future.

However, it may not be a perfect scoring system of mortality on its own. First of all, the current study’s single-centered retrospective study design was biased; Secondly, the new scoring system’s ability to detect severe sepsis has been successfully validated using an independent validation cohort; however, internal validation strategies were unable to assure the generalizability of the results to other patient populations whose fundamental characteristics differed from the current cohorts; Thirdly, all the patient population were critically ill patients in HICU. These findings may not apply to patients who are not in life-threatening condition in the general ward; finally, it is necessary to conduct further researches to determine whether changes in prognostic scores in the days after HICU admission are significant in determining the prognosis rather than at admission to HICU alone. Due to the limited sample size of this study, future multicentered researches of scoring system validation also need in broader and more diverse patient populations.

Conclusion

Our study provided a new scoring system that is more accurate in predicting mortality among patients with HMs and sepsis than the SOFA score. By combining the SOFA score with age and PT, two independent predictors, the new scoring system may be able to overcome its limitations in the application of HMs.