Comparison of Different Scoring Systems for Prediction of Mortality and ICU Admission in Elderly CAP Population

Introduction

Community-acquired pneumonia (CAP) is the leading cause of mortality and morbidity with substantial clinical and economic impact.¹ The incidence of pneumonia increases with age,² with a 10 times higher hospitalization rate in patients aged 65 years and older (about 2000 per 100,000 per year) than in the younger population.³ Multiple factors are related to the higher incidence of CAP in the older population, such as comorbidities, nutritional status, weakened immune system and swallowing dysfunction.^4,5 Increased mortality of aged CAP patients could be reduced with advanced life support such as invasive mechanical ventilation if early detection were available clinically.^6,7 Several pneumonia severity scores are available to guide the assessment of patients, such as quick Sequential Organ Function Assessment (qSOFA), Combination of Confusion, Urea, Respiratory Rate, Blood Pressure, and Age ≥65 (CURB-65), Modified Early Warning Score (MEWS), Acute Physiology and Chronic Health Evaluation II (APACHE II), Pneumonia Severity Index (PSI) and National Early Warning Score (NEWS),^8–12 which have been proven to predict the prognosis of patient’s with pneumonia in different aspects. As APACHE II and PSI contained acute physiology and chronic health evaluation, timely identification of high-risk patients is difficult,^13,14 whereas other scoring systems, which would be relatively easier-to-use, have more potential in early detection and assessment of high-risk patients.^15–18

Clinically, CURB-65 and qSOFA have been widely used for severity assessment of pneumonia. The CURB-65 score consists of five projects and qSOFA is similar to the CURB-65 score in variables, incorporating hypotension, altered mental status and tachypnea. Previous studies have already supported that the CURB-65 and qSOFA score are highly related to poor prognosis of CAP and prediction of the hospital mortality.^19–21 On the other hand, NEWS and MEWS share several essential parameters, including systolic blood pressure, pulse rate, respiratory rate, temperature, and level of consciousness.¹⁸ In emergency departments, they are most commonly used in risk stratification systems,^22,23 and have recently been introduced in predicting CAP, especially for the elderly patients. So far, only a few studies have verified the predictive value of NEWS and MEWS for the prognosis of patients with CAP.^12,24–26 Hence, this study aims to investigate the feasibility of the utilization of NEWS and MEWS in assessing and predicting the mortality and ICU admission for hospitalized older patients with CAP. With the comparison to CURB-65 and qSOFA, we hope to identify the most suitable score system for better hospitalization. As far as we know, this is the first study comparing the four scoring systems in predicting the hospital mortality and ICU admission in elderly patients with CAP.

Methods

Study Population

This is a retrospective cohort study conducted in the Minhang Hospital, Fudan University in Shanghai from 1 January 2018 to 1 January 2020. It was approved by the Ethics Committee of the Minhang Hospital, Fudan University in Shanghai, and the Lot No: Medical Ethics Committee (2017) No. 42. Regarding the patient informed consent statement, we notified patients and/or their legal guardians by telephone and asked for consent, and for those who did not, we waived the use of their data. Written informed consents were sent to patients and/or their legal guardians who agreed to participate in the study for signature. Signatures of study population were obtained and all procedures are in accordance with the Declaration of Helsinki.

The data were analysed anonymously in this study. Inclusion criteria were as follows: (1) Age ≥65 years; and (2) Diagnosed with CAP (community onset and new infiltrates on chest X-ray or CT scan together with ≥ 2 of the following clinical criteria: cough, sputum production, shortness of breath, pleuritic chest pain, fever or hypothermia, dyspnea; confusion, leukocytosis or leukopenia). Exclusion criteria: (1) immunosuppression, such as in the course of corticosteroids (>14 days), HIV-positive, chemotherapy or radiotherapy within 90 days and transplant recipients; and (2) serious heart failure (New York Heart Association classes 3 or 4); (3) Living in Nursing home.

Data Collection

We recorded the data from the electronic medical records in Minhang Hospital, Fudan University. We collected the information including age, gender, comorbidities, hospitalization days, consciousness state, vital signs, and laboratory variables within 24 hours of admission. Aforementioned variables were used to determine qSOFA, CURB-65, MEWS and NEWS scores were calculated according to the physiological and laboratory variables.

In detail, CURB-65 score: confusion, urea >7mmol/L, respiratory rate≥30/min, blood pressure (systolic blood pressure <90mmHg or diastolic blood pressure ≤ 60mmHg) and age ≥65years; qSOFA score: systolic blood pressure ≤100 mmHg, respiratory rate ≥ 22/min, and altered cognitive state; MEWS score: systolic blood pressure (mm Hg), heart rate (/min), respiratory rate (/min), temperature (°C) and consciousness; NEWS score: respiratory rate (/min), oxygen saturation, use of supplemental oxygen, heart rate (/min), systolic blood pressure (mm Hg), temperature (°C) and consciousness.

Statistical Analysis

Statistical analysis was executed with R (version 3.5.3). All the hypothesis tests were two-sided with a significance level of 0.05. We used Student’s t-test to compare continuous variables, and the chi-square test to compare distributed variables. Numerical parametric data were described as mean (SD), other continuous non-parametric data were described as median (inter-quartile range) and classification variables were reported as percentages. Multivariate analysis using stepwise logistic regression analysis contained all parameters with P-value <0.05 in the univariate analyses. To assess the predictive prognostic efficacy of the predictors, we performed receiver operating characteristic (ROC) curves and calculate their area under the curves (AUC).²⁷

Results

Study Population and Outcomes

In total, 1044 patients were selected for this study. The basic information and clinical characteristics of patient population are listed in (Table 1). Analysis of the comorbidities identified hypertension as the most prevalent comorbidity (40.8%, 426 cases), followed by cerebrovascular disease (24.9%, 260cases), diabetes (21.3%, 222 cases), electrolyte disturbance (20.5%, 214 cases), coronary heart disease (19.2%, 200 cases), congestive heart failure (14.9%, 155 cases), chronic kidney disease (8.2%, 85 cases) and cancer (6%, 63 cases).

Table 1 General Characteristics of the Patients

According to their prognoses, patients were divided into two groups, namely survivor groups (902 cases) and non-survivor groups (142 cases) (Figure 1). The total hospital mortality rate is 13.6% (142/1044). We found that there were significant differences in the leukocyte count, neutrophils count, lymphocyte count, NLR (neutrophil–lymphocyte ratio), c-reactive protein (CRP), procalcitonin (pct), albumin, platelet count, urea nitrogen (BUN), D-dimer, CURB-65, qSOFA, MEWS, NEWS between survivor groups and non-survivor groups (p < 0.05).

Figure 1 Bar charts of non-survivor and survivor counts in older adults with CAP by different scores among four scoring systems. Each chart shows the proportion in CURB-65, MEWS, qSOFA and NEWS, respectively.

According to ICU admission, enrolled patients were divided into ICU admission groups (n = 102) and non-ICU admission groups (n = 942) (Figure 2). The overall ICU admission rate is 9.8% (102/1044). Significant differences were shown between ICU admission groups and non-ICU groups in the leukocyte count, neutrophils count, lymphocyte count, prealbumin, low-density lipoprotein, NLR, CRP, pct, albumin, platelet count, BUN, D-dimer, CURB-65, qSOFA, MEWS, and NEWS (p < 0.05).

Figure 2 Bar charts of non-ICU and ICU counts in older adults with CAP by different scores among four scoring systems. Each chart shows the proportion in CURB-65, MEWS, qSOFA and NEWS, respectively.

The Multivariable Logistic Model

Multivariable logistic model was used to investigate the impacts of each parameter on predicting survival and ICU admission. In the multivariable logistic model, we observed CURB-65 (p < 0.001, z = 3.667) as a risk factor supported by previous studies. Surprisingly, MEWS (p < 0.001, z = 4.496) and pct (p < 0.05, z = −2.482) were also found to be the risk factors for the occurrences of mortality, while the albumin (p < 0.001, z = −3.327) was as a protective factor (Table 2). Meanwhile, we confirmed that the NEWS (p < 0.001, z = 7.758) is the risk factor of ICU admission, while the albumin (p = 0.001, z = −3.274) and platelet count (p < 0.05, z = −2.026) are the protective factors (Table 3).

Table 2 The Multivariable Logistic Model for Mortality

Table 3 The Multivariable Logistic Model for ICU Admission

The ROC Curves Predict the Mortality and ICU Admission

To better explore the predictive function among these four-scoring system, the ROC curves were used (Figures 3 and 4). Regarding the mortality, the AUC values of the qSOFA, CURB-65, MEWS and NEWS scores were 0.844 (p < 0.001,95% CI: 0.821–0.866), 0.868 (p < 0.001, 95% CI: 0.846–0.888), 0.927 (p < 0.001, 0.909–0.942), 0.892 (p < 0.001, 95% CI: 0.872–0.911), respectively (Table 4) (Figure 3).

Table 4 The AUC of Predictive Factors for Mortality

Figure 3 Receiver operating characteristic (ROC) curves and corresponding area under the curve (AUC) statistics for the risk of the mortality in older CAP by CURB-65, qSOFA, MEWS and NEWS scoring systems. The cutoff value, sensitivity and specificity of CURB-65, qSOFA, MEWS and NEWS were (362.68%, 92.45%),(180.99%, 84.02%), (4, 80.28%, 87.36%), (5, 83.1%, 80.49%), respectively.

Figure 4 Receiver operating characteristic (ROC) curves and corresponding area under the curve (AUC) statistics for the risk of the Intensive Care Unit (ICU) admission in older CAP by CURB-65, qSOFA, MEWS and NEWS scoring systems. The cutoff value, sensitivity and specificity of CURB-65, qSOFA, MEWS and NEWS were (2, 96.08%, 56.54%), (1, 86.27%, 81.83%), (4, 90.2%, 85.56%), (7, 96.08%, 95.52%), respectively.

The optimal cutoff values and Youden index for CURB-65, MEWS, NEWS and qSOFA are shown in Table 5. The Youden index for MEWS was maximum, and cutoff value was 4, sensitivity and specificity were 80.28 and 87.36, respectively (Table 5).

Table 5 The Cutoff Value of Predictive Factors for Mortality

The AUC values of the qSOFA, CURB-65, MEWS and NEWS scores for ICU-admission among enrolled patients are 0.866 (p < 0.001, 95% CI: 0.844–0.886), 0.854 (p < 0.001, 95% CI: 0.831–0.875), 0.922 (p < 0.001, 95% CI: 0.904–0.937), 0.976 (p < 0.001, 95% CI: 0.964–0.984), respectively (Figure 4) (Table 6).

Table 6 The AUC of Predictive Factors for ICU Admission

The optimal cutoff values and Youden index for CURB-65, MEWS, NEWS and qSOFA are shown in Table 7. The Youden index for NEWS is maximum, and cutoff value is 7, sensitivity and specificity are 96.08 and 95.52, respectively (Table 7).

Table 7 The Cutoff Value of Predictive Factors for ICU Admission

The Comparison of Different ROC and AUC

In the ROC curves of predictive effects of the mortality, we compared the ROC of qSOFA, CURB-65, MEWS and NEWS scores. There are clear differences in MEWS vs CURB-65 (p < 0.0001, z = 4.146), MEWS vs NEWS (p < 0.001, z = 3.376), MEWS vs qSOFA (p < 0.0001, z = 5.044) (Figure 5; Table 8).

Table 8 Comparison of ROC for Mortality

Figure 5 Comparison of ROC and AUC for selection of better scoring system to predict mortality in older CAP.

In the ROC curves of predictive effects of ICU admission, we compared the ROC of qSOFA, CURB-65, MEWS and NEWS scores. The result shows that there are significant differences in NEWS vs CURB-65 (p < 0.0001, z = 7.257), NEWS vs MEWS (p < 0.001, z = 3.929), NEWS vs Qsofa (p < 0.0001, z = 6.085) (Figure 6; Table 9).

Table 9 Comparison of ROC for ICU Admission

Figure 6 Comparison of ROC and AUC for selection of better scoring system to predict ICU admission in older CAP.

Kaplan–Meier Survival Curves

To further study the survival rate, we separated patients into high score and low score populations with corresponding optimal cutoff values, 1, 5, 4 and 3, respectively.

We compared the 28-day survival between high score and low score groups. Kaplan–Meier survival curves are plotted in Figure 7. Results indicated that these four different scoring systems could be highly related to survival. Specifically, low score cohorts presented a significantly prolonged survival time.

Figure 7 Kaplan–Meier estimates of 28-day overall survival. Panel (A), analysis with CURB65 system, cutoff = 3; Panel (B), analysis with MEWS system, cutoff = 4; Panel (C), analysis with NEWS system, cutoff = 5; Panel (D), analysis with qSOFA system, cutoff = 1.

Discussion

Aging population is a worldwide phenomenon, along with the gradually accelerating process of population aging.²⁸ In 2019, about 11.5% of the total population were above 65 years old in China, and it is expected to reach 16.9% by 2030.²⁹ The incidence of hospitalization and mortality for pneumonia is higher in the elderly population than in the younger group.³ Hence, quick stratification of the high-risk group for the older population with CAP are necessary. The four severity scoring systems mentioned above can provide a rapid assessment for clinicians within minutes and help clinicians to apply therapeutic strategies to high-risk patients in time.

In our study, the analysis from the multivariable logistic model showed that CURB-65, MEWS and pct were the risk factors for the occurrences of mortality, while the albumin was protective factor. Meanwhile, the NEWS was the risk factor of ICU admission, while the albumin and platelet count were the protective factors. All these results were similar to previous studies.^30–34 Previous studies reported NLR has been proven to be a promising candidate predictor of unfavorable outcomes in CAP patients.^35,36 However, unfortunately, NLR was not statistically significant in the multivariable logistic model. In a recent study, Lee et al³⁷ found a significant association between the NLR value on day 4 with 30-day mortality and ICU admission whilst this value on day 1 was not associated significantly. NLR in our study was collected within 24 hours of admission. There was no serial NLR measurement, which may result in the negative outcome. In summary, more studies are needed to prove the role of NLR in the CAP.

In this retrospective cohort study, we compared the ROC of the qSOFA, CURB-65, MEWS and NEWS to predict the inpatient mortality and ICU admission in the patient cohort. We identified that the prognostic accuracy of MEWS in mortality is much better than the qSOFA, CURB-65 and NEWS (p < 0.001), whereas in ICU admission, the NEWS score is better than the qSOFA, CURB-65 and MEWS (p < 0.001). We thus conclude that MEWS is an effective prognostic tool for predicting hospital mortality and the NEWS is a good predictor for ICU admission in the 65-years and older patients with community-acquired pneumonia, so the exploitation of MEWS and NEWS may be encouraged in this age population. Meanwhile, we also found that patients with either qSOFA <1, or with NEWS <5, or with MEWS <4, or with CURB-65 <3 at admission have a significantly increased survival time than the rest of studied cohort (p < 0.001).

Since the MEWS was first introduced in 2001 by Subbe and colleagues,³⁸ who modified Early Warning Score to MEWS, it has been widely used in the clinic. MEWS consists of 5 parameters, which include pulse rate, respiratory rate, systolic blood pressure level of consciousness and temperature. It has been proven to be a crucial tool for early identification of patients with high risk of poor outcome.^24,39,40 In elderly COVID-19 patients, Wang et al⁴¹ identified MEWS as an efficient tool for rapid assessment and the ROC analysis (AUC = 0.913) suggested that the MEWS was capable of predicting in-hospital mortality. Although MEWS has been found to be useful in elderly COVID-19 patients,^40,41 it has rarely been used in older adults with community-acquired pneumonia (CAP). In our research, we found that the AUC of MEWS for in-hospital mortality is 0.927, which is higher than the other scores (p < 0.01). For the ICU admission, the AUC of MEWS was 0.922 (>0.9) but was lower than the NEWS (AUC = 0.976, p < 0.01). We concluded that the MEWS could predict ICU admission to some extent, but with a higher potential power for prediction of the mortality older CAP patients. CAP patients with admission MEWS score <4 had a significantly prolonged survival time (p < 0.001), which was similar to the results of previous studies.^42,43

NEWS was derived in the United Kingdom in 2012, consisting of six physiological parameters arranging patients into three risk categories.⁴⁴ It may be the best evaluation system in the risk stratification systems and is the most widely used scoring system in emergency departments, since it is considered as superior over other risk stratification tools in predicting mortality and ICU admission.^45–47 The NEWS was also found to be an independent predictor of ICU admission and in-hospital death in COVID-19 patients.⁴⁸ The few studies of the NEWS for the predictive outcome of CAP, however, are controversial. Sbiti-Rohr et al²⁵ demonstrated that NEWS was inferior compared to CURB-65 for predicting 30-day mortality, while superior to CURB-65 for ICU admission. While Brabrand et al¹² and Grudzinska et al²⁶ showed no statistical difference between NEWS and other scores at predicting the prognosis of CAP patients, Zhou et al⁴⁹ recently compared the outcome prediction value of NEWS, CURB65, PSI, qSOFA in CAP patients and found that NEWS achieved the highest AUC in predicting 28-day mortality (AUC = 0.861) and ICU admission (AUC = 0.895). In our study, we found that NEWS achieved the highest AUC in predicting ICU admission, and the second highest AUC in predicting the mortality, which both higher than previous studies.^25,49 Therefore, NEWS showed superiority over the other scoring system in predicting ICU admission (AUC = 0.976), and inferior to the MEWS in predicting the mortality in older patients with CAP.

In clinic, patients with severe CAP patients usually had hypothermia, hyperthermia, hypoxemia and required supplemental oxygen,⁷ of whom showed bad prognosis-like ICU admission or death. The differences we observed in elderly CAP prediction could be due to varied factors incorporated in different scoring systems. NEWS includes the oxygen saturation, use of supplemental oxygen and temperature (°C) and MEWS temperature (°C), while CURB-65 and qSOFA do not. Moreover, clinical practice suggests patients with too slow respiratory rate commonly indicative of end of life or invasive mechanical ventilation. In these four scoring systems, MEWS and NEWS measured respiratory rate not only >14/min or >20/min but also <9/min, but CURB-65 and qSOFA failed in slow respiratory rate. In general, with different factors involved, MEWS and NEWS showed superiority than CURB-65 and qSOFA in predicting hospital mortality and ICU admission in older patients with CAP.

Early identification of patients at high risk of mortality can enable the appropriate allocation of scarce resources, such as intensive care beds. The MEWS and NEWS are relatively new scoring systems capable of predicting the prognosis of patients with CAP. The advantage is that they could be the easier tools with few parameters, facilitating assessment procedures. There are few studies employing the MEWS or NEWS as predictors, especially in older patients. Based on our research, the MEWS and NEWS are more useful tools than the previously used score system such as CURB-65 in predicting the prognosis of the CAP patients.

Limitations

This study has certain limitations. Firstly, this was the retrospective single-center study design, and some data are missing for individual patients, such as LDL, D-dimer and mental state. Furthermore, the four scores were calculated retrospectively and vital signs were not measured more frequently than standard clinical practice. Prospective multicentre studies that can ensure collection of complete data sets and ensure generalizability are needed. Secondly, we did not take into account functional decline, dysphagia ability or frailty, which could influence the prognosis of patients with CAP.^50–52 Moreover, it is well known that mortality in patients with aspiration is significantly higher in non-aspiration pneumonia,⁵³ while in our study CAP was not differentiated from aspiration pneumonia as the population was small which is corroborated in past studies estimating 5%–15% cases of community-acquired pneumonia.^54,55 Such studies with large population need to be designed in the future. Thirdly, recruitment of patients with severe CAP who met criteria for ICU admission was limited due to the possible reason that patients refused to ICU in the end due to dignity or hospitalization costs. Fourthly, many elderly patients had comorbidities such as chronic kidney disease, electrolyte disturbance and so on, which needed advanced life support such as renal replacement therapy, leading to the decrease of the ICU admission. This would have impacts on accuracy of these scores to predict ICU admission. Fifthly, we did not address the antibiotic regimen and other adjuvant therapies such as oxygen support, which may influence the prognosis and lead to misclassification of patients. Lastly, we used the 28-day mortality in Kaplan–Meier survival curves, whereas long-term outcomes were not recorded.

Conclusion

We explored the prediction values of CURB65, qSOFA, MEWS and NEWS for patients aged 65-years and older with community-acquired pneumonia. We found that MEWS showed superiority over the other scores in predicting hospital mortality, and NEWS showed superiority over the other scores in predicting ICU admission. Patients with qSOFA score <1, or with NEWS score <5, or with MEWS score <4, or with CURB-65 score <3 at admission have a significantly longer survival time. We hope that our analysis will guide decision in the field for measurement of elder CAP. Also, more multi-center studies are needed to validate our conclusion in the future.