Comparison Between Physiological Scores SIPF, CURB-65, and APACHE II as Predictors of Prognosis and Mortality in Hospitalized Patients with COVID-19 Pneumonia: A Multicenter Study, Saudi Arabia

Introduction

The pandemic of coronavirus disease 2019 (COVID-19), caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), has overwhelmed many health systems and economies in many countries and is associated with high morbidity and mortality.¹ As of October 2022, more than 619 million cases of COVID-19 have been confirmed worldwide, including more than 6.5 million deaths. As of October 2022, 818 thousand cases and 9369 deaths have been confirmed in Saudi Arabia, and more than 12.7 billion doses of vaccine have been administered, resulting in a reduction in disease severity.²

Symptomatic COVID −19 infections can be classified as mild (40%), moderate (40%), severe (15%) requiring oxygen support, critical illness with complications (5%) such as respiratory failure, acute respiratory distress syndrome (ARDS), sepsis and septic shock, thromboembolism, and/or multiple organ failure.³

The prognosis of COVID-19 depends largely on several factors, including the age of the patient, the severity of the disease at the time of diagnosis, preexisting conditions (obesity, diabetes mellitus, chronic lung disease, cardiovascular disease, chronic kidney disease, chronic liver disease, and neoplastic disease), the quality of the health care system, and the response to treatment. Higher levels of inflammatory markers such as C-reactive protein (CRP) and D-dimer, as well as ferritin, have been associated with higher mortality in hospitalized COVID −19 pneumonia. The most common complication of severe COVID-19 disease is acute respiratory failure and ARDS and/or multiorgan failure leading to death.⁴

Several studies have evaluated common critical care severity scores to assess pneumonia severity and mortality COVID −19, including Acute Physiology and Chronic Health Evaluation (APACHE) II, CURB −65 (confusion, urea, respiratory rate, blood pressure, age ≥65), Pneumonia Severity Index (PSI) and Quick Sequential Organ Failure Assessment (qSOFA), and the results have been controversial.⁵

Artero et al showed that PSI and CURB-65 were better than qSOFA in predicting mortality in patients with COVID-19 pneumonia,⁶ whereas others reported inconsistent results.^7,8

The physiologic shock index (SIPF) (shock index and hypoxemia), CURB −65, and PSI have been evaluated in many studies of community-acquired pneumonia to predict ICU admission and mortality, and it has been reported that the SIPF score is simple and can be a useful tool to predict mortality in community-acquired pneumonia (CAP).^9,10

In this multicenter, retrospective study, we investigated the accuracy of SIPF, CURB −65, and APACHE II in predicting prognosis and in-hospital mortality in patients with COVID −19 pneumonia.

Subjects and Methods

This is a retrospective, multicenter study of 1131 adult patients (18 years of age or older) admitted for confirmed COVID −19 pneumonia either in the ward or intensive care unit in 3 major hospitals in Saudi Arabia, 2 in the eastern region and one in the northern region. This study was conducted from May 2020 to May 2021 (More than 75% of included patients did not receive the first dose of covid 19 vaccine).

Inclusion Criteria

Adult patients with pneumonia due to COVID-19 (positive SARS COV 2 RT-PCR nasopharyngeal smear or respiratory secretions) and indicated for hospital admission according to Saudi Ministry of Health protocol were included in our study. Indications for hospital admission include hypoxemia (oxygen saturation < 92% on room air), COVID −19 Patients with pneumonia (without hypoxia) and with risk factors for poor outcomes such as age > 65 years, underlying comorbidities such as chronic cardiovascular disease, chronic lung disease, poorly controlled diabetes, chronic renal failure with dialysis, active cancer with chemotherapy, immunosuppression, and post-transplantation.^11,12

All patients with mild to moderate pneumonia were housed in a negative-pressure isolation room or equipped with a HEPA filter. Patients with severe pneumonia and critically ill patients were treated in the intensive care unit. Criteria for severe pneumonia included: a respiratory rate of 30/min or more in adults, a PaO2/FIO2 ratio < 300, and infiltrates affecting more than 50% of the lungs. Criteria for critical illness included ARDS, altered state of consciousness, and multiple organ failure. All cases were treated according to the Saudi Ministry of Health protocol for patients with confirmed COVID −19.^11,12

Exclusion Criteria

Patients admitted to the ward or intensive care unit for reasons other than COVID-19 pneumonia or who had COVID-19 pneumonia without requiring hospitalization and pediatric COVID-19 patients were excluded.

Data Collections and Outcomes

The primary endpoint was mortality in the ward, ICU, or at discharge. To determine this outcome, all patients were followed up and the relevant information was recorded. Other data, such as length of stay in the ward or ICU, need for different types of oxygen therapy and ventilatory support, or involvement of other organs, were considered as factors that may affect patient outcomes.

Age, gender, BMI, comorbidities (hypertension, asthma, type 2 diabetes, and other comorbidities), and the patient’s symptoms were all gathered (fever, cough, dyspnea, anorexia, diarrhea, and other symptoms). Complete medical history and examination, including vital signs (Glasgow coma scale, temperature, respiration rate, pulse, blood pressure, and mean blood pressure) with local chest examination were taken.

The following investigation’s results were taken from the electronic medical record: SARS-2 virus identification from nasopharyngeal swab using polymerase chain reaction, chest x-ray (portable), oxygen saturation by a pulse oximeter, arterial blood gases, complete blood count (CBC) with differential, complete metabolic panel (serum sodium, potassium, and magnesium), C-reactive protein (CRP), serum ferritin, D-dimer, lactate dehydrogenase, renal function test (urea and creatinine), liver function test (liver enzymes, serum albumin, and prothrombin time (PT)/partial thromboplastin time), procalcitonin, troponin, and electrocardiogram (ECG).

Within 24 hours of admission, the information needed to calculate SIPF, CURB-65, and APACHE IV scores were obtained from medical records. The first values of vital signs and laboratory parameters were used to calculate the scores.

Scores

APACHE II is used to evaluate the condition of patients in the intensive care unit using 12 parameters, as it can detect the severity of the disease. It is used within 24 hours to calculate an integer value from 0 to 71, where a higher value means the condition is more serious.^13,14

CURB-65 score (from 0 to 5) is a clinical prediction rule that has been validated for predicting mortality from community-acquired pneumonia and infection at any site. The score is an acronym for each of the measured risk factors. Each risk factor is assigned a point, so the maximum score is 5. New-onset confusion, a respiratory rate ≥ 30 breaths per minute, a blood urea nitrogen level greater than 7 mmol/L, a systolic blood pressure < 90 mmHg or a diastolic blood pressure ≤ 60 mmHg, and age ≥ years. Score 0–1: low risk, score 2: medium risk, and score ≥ 3: high risk.¹⁵

Shock Index Physiological score SIPF (shock index and hypoxemia) score: a combination of shock index (heart rate/systolic blood pressure) > 0.7 (=1 point) and PaO2/FiO2 250 (=1 point), with a range of 0–2 points. Rather than static criteria, the shock index emphasizes current physiologic dynamics.^9,10

Statistical Analysis

It was carried out using the SPSS version 25.0 (IBM SPSS Statistics for Windows, Armonk, NY: IBM Corp., USA). For descriptive statistics, mean ± SD, median, and interquartile range (IQR) were used for quantitative variables, whereas frequency and percentage were used for qualitative variables. Fisher exact test was used to assess differences in frequencies of qualitative variables. Independent samples t-test was used to evaluate the differences in means of quantitative variables, while the Mann–Whitney test was used for nonparametric statistics. To determine the ability of APACHE II, CURB-65, and SIPF scores to predict ICU admission and mortality in patients with COVID-19 pneumonia, a receiver-operating characteristic curve (ROC) with the associated area under the curve (AUC) was constructed, selecting the most appropriate cut-off point with the best sensitivity and specificity. Logistic regression analysis with odds ratios (OR) and 95% confidence intervals (CI) was used to predict factors associated with COVID-19 mortality. Only significant independent variables from the univariate analysis were included in the logistic analysis. Statistical methods were reviewed, assuming a significance level of p<0.05.

Ethical Consent

The study was conducted in accordance with the Declaration of Helsinki. Ethical approval was obtained from the Institutional Review Board, Almoosa Specialist Hospital (IRB protocol number: ARC −21.03.3), King Khaled Hospital-Hail (Saudi Health Affairs Hail Region: IRB, KACS, KSA: H-08-L-074), Obeid General Hospital (approval by Obeid Hospital Medical Committee).This study was retrospective study. Patient consent to review their medical records was not required by the three Institutional Review Boards however, privacy and confidentiality of patients data were maintained throughout the study process.

Results

The study included 1131 COVID-19 patients who met the inclusion criteria. They were divided into two groups: (A) the ICU group (n=340; 30.1%) and (B) the ward group (n=791; 69.9%). Their distribution and outcomes were illustrated in Figure 1. The most common concomitant diseases of patients at initial ICU admission were hypertension (71.5%) and diabetes (62.4%), and most of them were men (63.8%). They also had higher mean RR (27.7±4.2), higher median D-dimer (2.9 [2.5–3.4]), WBC (11.3 [8.2–15.4]), and LDH (652 [452–895]), lower median PaO2 (59 [50.3–65]) and lower mean hemoglobin (11.4±2.2) and were associated with a longer median hospital stay (11 [4–18] days). Of the group treated in the ward, 9 patients (1.1%) were transferred to other hospitals without a known outcome. The overall mortality was 210 patients out of 1122 (18.7%) (excluding patients transferred to other hospitals). The mortality rate in the group initially treated in the ICU was 39.4% (134 out of 340) while the mortality rate in the ward group was 9.6% (76 out of 791) (Table 1).

Table 1 General, Clinical, Laboratory and Outcome Characteristics of the Studied Groups

Figure 1 Flow diagram for total patients included within the study.

The patients initially admitted to the intensive care unit were significantly the largest group requiring high-flow nasal cannula oxygen therapy or invasive technical ventilation and systemic steroid therapy (Table 2).

Table 2 Type of O2 Therapy and Steroid Therapy Among the Studied Groups

The mean APACHE II, CURB-65, and SIPF scores were significantly higher in patients initially admitted to ICU compared to the ward group (Table 3).

Table 3 Comparing Different Scores Among the Studied Groups

The three scores differed in their ability to predict ICU admission and mortality in patients with COVID-19 pneumonia. The APACHE II score with a cutoff value of > 13 had a sensitivity of 73% and a specificity of 85% (AUC 0.87; p < 0.001) for predicting ICU admission and a sensitivity of 55% and a specificity of 61% (AUC 0.65; p < 0.001) for predicting mortality. The CURB-65 score at a cutoff of > 2 had a sensitivity of 70% and a specificity of 86% (AUC 0.84; p < 0.001) for predicting ICU admission and a sensitivity of 70% and a specificity of 76% (AUC 0.80; p < 0.001) for predicting mortality. The SIPF score had a sensitivity of 87% and a specificity of 88% (AUC 0.89; p < 0.001) for predicting ICU admission and a sensitivity of 86% and a specificity of 90% (AUC 0.90; p < 0.001) for predicting mortality at a cutoff value of =2. Accordingly, the SIPF score showed a significantly higher ability to predict both ICU admission and mortality in patients with COVID-19 pneumonia compared to APACHE II and CURB-65; (AUC 0.89 vs 0.87; p < 0.001) and (AUC 0.89 vs 0.84; p < 0.001) for ICU admission and (AUC 0.90 vs 0.65; p < 0.001) and (AUC 0.90 vs 0.80; p < 0.001) for mortality, respectively (Table 4, Figures 2 and 3).

Table 4 Different Prediction Scores for ICU Admission and Mortality in Patient with COVID-19 Pneumonia

Figure 2 ROC curves analysis of different prediction scores for ICU admission in patients with COVID-19 pneumonia.

Figure 3 ROC curves analysis of different prediction scores for mortality in patients with COVID-19 pneumonia.

Patients who died (non-survivors) were significantly older, male, had several comorbidities (hypertension, diabetes), had higher HR, RR, CRP, ferritin, LDH, and D-dimer levels, leukocyte counts, and higher creatinine and BUN levels than the survivors, and they required mechanical ventilation. In contrast, they had lower BMI, GCS, PaO2, and HB values. These variables, which showed a significant association with non-survivors, were further analyzed using multinomial logistic regression to predict the independent variables contributing to COVID-19 mortality. Significant predictors of COVID-19 mortality were male sex (OR = 1.06; CI: 0.95–1.17), higher RR (OR = 1.03; CI: 1.00–1.06), need for invasive MV (OR = 1.10; CI: 0.96–1.42), lower PaO2 (OR = 1.08; CI: 0.99–1.15), increased creatinine (OR = 1.21; CI: 1.17–1.28), and increased D-dimer (OR = 1.04; CI: 0.98–1.10) (Table 5).

Table 5 Comparing Different Variables Among Survivors and Non-Survivors with Regression Analysis of Factors Predicting COVID-19 Mortality

Discussion

This study was conducted in three large hospitals in different areas of Saudi Arabia, two large private hospitals and one government hospital. Saudi Arabia has a comprehensive public health system in which all residents receive free medical treatment and vaccinations at public and private hospitals during the outbreak of COVID-19. The mortality rate in patients with COVID-19 pneumonia requiring treatment in the intensive care unit varied from country to country and depended on many factors, most importantly on the quality of the health care system in each country.

In this study, the overall mortality was 18.7%. Our results regarding in-hospital mortality are consistent with many studies. In a study from Saudi Arabia (564 inpatients with COVID-19 from 2 hospitals), the overall in-hospital mortality was 20%.¹⁶ In an Egyptian study (175 patients hospitalized with COVID-19 pneumonia), in-hospital mortality occurred in 34 (19.4%) of cases.¹⁷ Dessie et al performed a systematic review and meta-analysis of 42 studies and 423,117 patients. The pooled prevalence of mortality in hospitalized patients with COVID-19 was 17.62%.¹⁸ The in-hospital mortality rate due to COVID-19 ranged from 17 to 38% worldwide.¹⁹ A study of 503409 patients in US acute care hospitals found that the mortality rate was 19.7% in April 2020 and decreased to 9.3% in November 2020.²⁰

Regarding ICU mortality, our results are consistent with a multicenter study in Saudi Arabia in which 229 patients with COVID-19 pneumonia were treated in the ICU; the mortality rate was 37%.²¹ Our results were lower than those of other studies. In a meta-analysis of 24 studies published in PubMed, MEDLINE, Cochrane, and EMBASE databases by May 31, 2020 (10,150 patients), the combined ICU mortality rate was 41.6%.²² In a prospective multicenter study involving 64 hospitals in 10 African countries, including Egypt and Libya, the mortality rate among adults admitted to an intensive care unit with COVID-19 pneumonia was 48.2%.²³ The highest values for mortality rates in critically ill COVID-19 pneumonia patients were reported in 2 studies, one from Libya with 60.4%²⁴ and the other from Iran with 55.9%.²⁵ Our results were higher than those of Auld et al, who studied 1686 patients with coronavirus disease admitted to an intensive care unit and the overall mortality was 29.7%.²⁶ This could be explained by the different expressions of COVID-19 in the patients in the studies investigated and the quality of the healthcare system.

In this study, the non-survivors were significantly older, male, hypertensive, diabetic, with increased HR, RR, and need for MV, with higher CRP, ferritin, LDH, and D-dimer levels, with leukocytosis and lower PaO2. This is consistent with the findings of the Centers for Disease Control and Prevention (CDC)²⁷ and WHO⁴ and with many studies.^28,29

In this study, the mean age of ICU patients was 66.7 ± 13.4 years and that of non-survivors was 64.4 ± 13.9 years, and most of them were men (OR = 1.06; CI: 0.95–1.17), which is consistent with many studies.^30,31 Respiratory rate is one of the components of many severity assessment systems (CURB65 score, APACHE II). In this study, increased respiratory rate (> 27.7 /min) was an independent risk factor for ICU admission and mortality (OR = 1.03; CI: 1.00–1.06), consistent with many studies.^32,33 In this study, elevated D-dimers (OR = 1.04; CI: 0.98–1.10) were a significant predictor of COVID-19 mortality. This is consistent with many studies that reported that critically ill patients with COVID-19 pneumonia have significantly higher D-dimer levels, which can be used as a prognostic marker for in-hospital mortality.^2,21,34

Many scores such as PSI, CURB −65, and APACHE-II have been studied and are valid to assess the severity and outcomes of pneumonia in the period before COVID −19.

In this study, the APACHE II score is a good predictor of ICU admission with a cutoff value of > 13 (sensitivity 73%, specificity 85%, and AUC 0.87; p < 0.001) and a poor predictor of mortality (sensitivity 55%, specificity 61% and AUC 0.65; p < 0.001). In critically ill patients APACHE II - a score of ≥ 25 can predict 50% mortality and a score of ≥ 35 can predict 80% pneumonia mortality.³⁵ This is consistent with many studies reporting that the APACHE II score underestimates the risk of death in COVID-19 patients and the severity of the disease.^25,36 Tian et al also reported that the day 1 score APACHE II was found to be poorly calibrated to predict in-hospital mortality. However, the day-3 score APACHE II best cutoff value 17 is an optimal biomarker for predicting outcomes in ICU patients.³⁷ In contrast to other studies, they reported that the APACHE II -score with the highest AUC of all scores (0.772) prevailed as the most accurate.^38,39 This also contrasts with the study by Zou, which found that APACHE II had higher discriminatory power compared with SOFA and CURB-65, with APACHE II (mean 15.05 ± 7.71) and ROC AUC showing excellent discriminatory power in predicting severity and mortality in COVID-19 pneumonia.⁴⁰ This is in contrast also to many studies reporting that lower levels of APACHE II are a predictor of ICU admission and mortality in COVID-19 patients.^29,32

In our study, the CURB-65 score at a cutoff of > 2 is a good predictor of ICU admission (sensitivity 70%, specificity, 86%, and AUC 0.84; p < 0.001) and a good predictor of mortality (sensitivity 70%, specificity 76%, and AUC 0.80; p < 0.001). This is consistent with many studies reporting that CURB-65 is a useful tool for predicting ICU admission and mortality in COVID-19 patients.^7,41,42 Elmoheen et al reported that CURB-65 had better calibration metrics and overall accuracy (AUC 0.78 (95% CI 0.746–0.814) compared to PSI 0.70 (95% CI 0.665–0.738).⁴³ Ateşer et al⁴⁴ and Fan et al⁷ reported that CURB-65 ≥1 was a good predictor. In one study, the CURB-65 score of ≥2 (AUC: 0.79) had a fair discriminatory power to predict 30-day mortality. PSI performed better in predicting mortality than CURB-65.⁸ Bradley et al showed that patients with a high CURB-65 score (the AUC was 0.79) had an increased risk of death, but the CURB-65 score is not a guide to discharge.⁴⁵

This study is the first to compare the simplified shock index physiological factor with the well-known two scores CURB-65 and APACHE II. The SIPF assesses two components: dynamic hemodynamics by the shock index and oxygenation by the FIO2 /FIO2 ratio.

In this study, the SIPF score showed a significantly higher ability to predict both ICU admission and mortality in patients with COVID-19 pneumonia compared to APACHE II and CURB-65; (AUC 0.89 vs 0.87; p < 0.001) and (AUC 0.89 vs 0.84; p < 0.001) for ICU admission and (AUC 0.90 vs 0.65; p < 0.001) and (AUC 0.90 vs 0.80; p < 0.001) for mortality, respectively. This is consistent with many studies in which the SIPF score showed a significantly higher ability to predict both ICU admission and mortality in patients with CAP compared with PSI and CURB-65 in the period before COVID-19.^9,10 Hypoxemia is the main pathophysiological mechanism of COVID-19 pneumonia and the cornerstone in assessing the severity of the disease and mortality.³⁴ Oxygen saturation of less than 90% on admission is a strong predictor of in-hospital mortality in patients with COVID-19 in one study⁴⁶ and SO2 of less than 85% in other studies.^47,48 Moderate to severe PaO2/FiO2 impairment was independently associated with a threefold increased risk of in-hospital mortality.⁴⁹ Xie et al showed that patients with a SpO2 <of 90% were 47 times more likely to die compared with patients with a SpO2 >of 90%.⁵⁰

Study Strengths

Our study represents one of the largest multicenter retrospective cohorts in Saudi Arabia of patients hospitalized with COVID-19 pneumonia. In our study, multiple variants were evaluated to assess predictors of ICU admission and mortality, such as vital signs, comorbidities, inflammatory markers, and physiological scores.

Study Limitations

Our study is a retrospective study, which is the main limitation with possible missing data. However, the overall missing variables were less than 5% and the mean (or median) was used to fill in any missing ones. Also, unmeasured covariates may cause residual confounding effects on our results. In addition, some variables that have the potential to impact the outcome may not have been recorded as the effect of different COVID-19 stains causing the disease. The absence of further predictive analysis, other than the ROC curve, to evaluate the prediction model is another limitation. Finally, we did not consider multivariable analysis with confounding adjustment so our results should be interpreted with caution. We recommend conducting prospective multicenter studies to validate our study results.

Conclusion

The ability of the SIPF score to predict ICU admission and mortality in COVID-19 pneumonia is higher than that of APACHE II and CURB-65. In-hospital mortality was 18.7%. The main risk factors associated with in-hospital mortality were male sex, comorbidity (type 2 diabetes and hypertension), tachypnea (RR≥ 27), hypoxemia (PaO2 <59), and elevated D-dimers. The ability of the SIPF score to predict ICU admission in COVID-19 pneumonia higher than that of CURB-65 and PSI. The simple SIPF score could be a useful tool to predict ICU admission and mortality in COVID-19 pneumonia.