The Available Criteria for Different Sepsis Scoring Systems in the Emergency Department—A Retrospective Assessment

Introduction

Sepsis, defined as infection with associated acute organ system dysfunction, is one of the leading causes of hospital admissions in the United States.¹ Because sepsis is rapidly progressive, delayed recognition can have dire consequences. Recent literature has highlighted the importance of early identification and treatment of sepsis.^1,2 Since 67–79% of septic patients present to the hospital with severe sepsis^3,4 and patients with severe sepsis have the highest rate of mortality,^1,5 recognizing these cases and initiating appropriate treatment is of utmost importance. Efficient scoring methods for diagnosing sepsis are critical, as time can be the difference between life and death.

In order to treat sepsis quickly, providers use sepsis scoring systems to diagnose and quantify the risk of mortality in acutely ill patients.^3,6 Mixed models addressing alerts for sepsis, integrating organ failure assessment scores and general severity scores, have been published but have not gained widespread acceptance.^4,7 Existing scores have marked variation in their ability to provide prognostic support for clinical decision-making.⁸ There is often a lack of context for the identification of sepsis, which can lead to inconsistencies across providers, thus leading to increased mortality in acutely ill patients.⁶ However, a patient’s initial presentation can provide important data that can be used to identify and decrease the possibility of mortality.

Sepsis remains an illness difficult to identify and a gold-standard test for diagnosis does not currently exist. Multiple sepsis screening tools have been developed and evaluated for both pre-hospital and in-hospital systems. Diagnostic scoring systems, which include the systemic inflammatory response syndrome (SIRS) criteria,³ national early warning score (NEWS),⁵ and prehospital early sepsis detection (PRESEP) score,⁹ tend to have fewer criteria, as the goal of these scores is to rule in patients with potential sepsis. Scoring systems that include the sequential organ failure assessment (SOFA),¹⁰ quick sequential organ failure assessment (qSOFA),¹¹ sepsis patient evaluation in the emergency department (SPEED),¹² mortality in emergency department sepsis (MEDS),¹³ and predisposition, infection/insult, response and organ dysfunction (PIRO)¹⁴ are prognostic and primarily focus on identifying patients with increased mortality.

When components required to compute the score are available, there is an increased possibility for corrective action in order to reduce the patient’s overall risk of mortality.¹⁵ However, sepsis scoring criteria are not always available in a busy emergency department (ED), hindering the predictive accuracy of scoring systems and indirectly contributing to patient deterioration. The question arises of how to address missing scoring criteria, as the patient’s outcome relies on the availability of all scoring criteria. Can a patient be given an accurate diagnosis through these sepsis scoring protocols if aspects of the criteria are still missing? This study aims to assess the real-time availability of the criteria for eight commonly used sepsis scoring systems within 6 hours of triage in the ED.

Methods

Study Design and Setting

A retrospective chart review was performed to compare the availability of criteria for eight commonly used sepsis scoring methods (SIRS, NEWS, PRESEP, SOFA, qSOFA, SPEED, MEDS, and PIRO) within the first 6 hours of a patient’s arrival to the MedStar Washington Hospital Center (MWHC) ED. MWHC is a 912-bed, urban, academic hospital with 86,771 annual visits and the largest and busiest ED in the District of Columbia. Data were extracted from the electronic health record (EHR) for visits between June 1, 2017 and May 31, 2018 and captured into a standardized Research Electronic Data Capture (REDCap) abstraction (SR, BF, CB). Discrepancies during data extraction were resolved by discussion and consensus.

Selection of Participants

Adult patients (≥18 years old) with an ICD-10 admitting diagnosis of severe sepsis without or with shock (ICD-10 codes R65.20 and R65.21, respectively) were eligible for inclusion in the study. Subjects were excluded if they were dead on arrival to the ED.

Sepsis Scoring Systems

A side-by-side comparison of each criterion of the eight commonly used sepsis scoring tools is included (Appendix Table 1). SIRS, NEWS, and PRESEP are diagnostic while SOFA, qSOFA, SPEED, MEDS, and PIRO are prognostic. qSOFA has the fewest criteria (3), while PIRO has the most (14). All scoring methods, besides SIRS, allocate points for each criterion. The inclusion of vital signs is common amongst all scoring systems. The diagnostic scoring methods are largely vital signs dependent, with PRESEP being based entirely on vital signs. qSOFA is the only prognostic scoring system with the majority (2/3, 66.7%) of its criteria as vital signs. Four (50%) scoring methods (NEWS, SOFA, qSOFA, and MEDS) account for altered mental status (AMS), with SOFA and qSOFA relying specifically on GCS.

Five (62.5%) scoring systems (SIRS, SOFA, SPEED, MEDS, and PIRO) include laboratory criteria. PIRO and SOFA both include the most lab dependent criteria (4/14, 28.6% and 4/6, 66.7%, respectively) with SOFA having the highest percentage of lab dependent criteria. SPEED, MEDS and PIRO each have an equal number of vital signs as laboratory criteria.

Outcomes

In addition to collecting data relevant to each of the eight scoring tools, patient demographics and outcomes were recorded, including in-hospital mortality, code blue (cardiopulmonary arrest code) activation, rapid response activation, respiratory failure, and progression to septic shock. Vital signs and laboratory values were also recorded at times 0 (T0), 3 (T1), and 6 (T2) hours after triage to parallel sepsis bundles currently implemented at our institution. NEWS uses the Alert, Verbal, Pain, Unresponsive (AVPU) scale in its scoring criteria which is not a standard documented value in the ED at MWHC, so a previously validated Glasgow Coma Scale (GCS) to AVPU conversion was used.¹⁶

Analysis

All data, 181 variables, were cataloged and analyzed in REDCap. Categorical data were reported as frequencies and continuous variables were reported as mean and standard deviation (SD). The MedStar Health Institutional Review Board approved this study and approved a full waiver of Health Insurance Portability and Accountability Act (HIPAA) authorization for research purposes prior to conducting research practices. Human subjects consent was not obtained prior to study commencement due to the impracticality of obtaining consent before subject identification. As part of the study, we analyzed the availability of new vital sign and laboratory criteria at particular times (T0, T1 and T2); however, for the purposes of calculating the percent of criteria available per scoring system, vital signs and laboratory values available from earlier in the ED visit were considered available at later time points as they could still be used clinically.

Results

Characteristics of Study Subjects

The data retrieval produced 631 records with an ICD-10 code for severe sepsis. A random sample of 128 charts were reviewed to allow for the collection of all 181 variables from each eligible record reviewed. Seventy-eight records were excluded given that severe sepsis was not the admitting diagnosis but rather diagnosed later in the hospital course. A total of 50 subjects were included in the final analysis (Table 1). A PRISMA flow diagram depicting case selection is included in Figure 1.

Table 1 Patient Demographics (n=50, with One Patient Identifying as More Than One Ethnicity)

Figure 1 PRISMA flow diagram for case selection.

Main Results

Vital signs were readily obtained (>90% availability) at T0 except for temperature, which was only documented in 39 (78%) patient charts (Table 2). Temperature continued to be the least documented vital sign at T1 and T2 with 30 (60%) and 34 (64%) patients with documented temperatures, respectively. Respiratory rate was documented in 46 (92%) charts at T0, but in only 33 (66%) and 37 (74%) charts at T1 and T2, respectively. Most laboratory values were not available at T0; white blood cell count and hematocrit were the highest reported (14%). By T1 and T2, most laboratory values were available (>90%) except for bilirubin and creatinine (Table 2). GCS was documented for all 50 (100%) patients at T0, with an average score at triage of 13.14 (3.18). Twenty-four (48%) patients were documented as having altered mental status (AMS) either upon arrival or during their ED course. Clinical outcomes included a mean hospital stay of 9.12 (9.05) days, 8 rapid response activations, 3 code blue alerts, 28 (56%) patients progressed to septic shock, 20 (40%) patients developed respiratory failure, and 18 (36%) patients died in the hospital.

Table 2 Vital Signs and Laboratory Value Availability

Most scoring systems had the majority (>50%) of their criteria available at the time of triage, except SOFA only had 38% (13) of its criteria available on average (Table 3). NEWS, PRESEP and qSOFA had the highest percentage of available criteria, with greater than 90% available at all time points (T0, T1, and T2). By 3 hours in the ED, all scores had greater than 75% of their criteria available, with SOFA and SIRS having the least percent of available criteria at 77% (17) and 78% (12), respectively. SOFA and SIRS continued to have the least percent of available criteria at 6 hours in the ED, both having only 79% (SD 16 and 10, respectively) of their criteria available.

Table 3 Scoring Tool Criteria Availability

Discussion

Sepsis can be difficult for providers to define, identify and treat. A patient’s presenting symptoms and the allocation of points towards sepsis scores are closely tied to the availability of patient data at 0 (triage), 3, and 6 hours into the patient’s visit. Knowing which criteria tend to be available at a given time has the potential to improve sepsis detection and monitoring throughout the patient’s ED visit. The goal of this study was to provide a crucial analysis of the availability of criteria in the ED of the most widely used sepsis scoring systems. In addition to diagnostic criteria, we included variables from prognostic scoring criteria not typically used in the ED to evaluate whether there were enough available data to potentially utilize them within the first 6 hours of a patient’s ED course. To our knowledge, this has not been previously completed at the institutional level and no other study has compared the criteria for these eight commonly used sepsis scoring tools in a single table (Appendix Table 1). Since protocols and data availability differ by institution, the traditional ED sepsis diagnostic scores may not be useful for some EDs, as was the case at our ED where SIRS criteria were less readily available. Our results suggest that multiple diagnostic and prognostic scoring systems have enough available criteria to be potentially utilized at different time points in the ED. Heavily vital signs dependent scores (qSOFA, PRESEP and NEWS) have more of their criteria fulfilled closer to the time of triage when quick and accurate medical decision-making is especially crucial while heavily laboratory dependent scores (SOFA, SPEED, MEDS and PIRO) could become more useful as the patient’s time in the ED increases. Of note, SIRS had the third-lowest percent of available criteria (57%) at triage despite being heavily vital signs dependent and diagnostic, while qSOFA had the highest percent of available criteria (97%) despite it being prognostic.

Some sepsis scoring systems (NEWS, SOFA, qSOFA, and MEDS) require the assessment of mental status, as sepsis-associated encephalopathy is indicative of increased mortality risk.¹⁷ GCS and the AVPU scale are two frequently used scales that assess neurologic status.⁵ All patients in this study had GCS assessed at triage. Although these scales can assess an aspect of orientation, they are used to calculate a patient’s level of consciousness and do not account for a change in baseline mental status. While NEWS, SOFA and qSOFA include GCS or the AVPU scale, MEDS relies on a non-numerical classification of AMS by history or physical exam. Although MEDS is currently considered to be a prognostic tool, the inclusion of clinical judgment in assessing AMS could have early diagnostic value in detecting sepsis.

MWHC has a focus on organizational quality improvement including significant improvements in health information technology (IT) for a single, standardized bundle compliance dashboard and automated alerts to ensure timely initiation of sepsis treatment bundles. Of course, there are more general standards for vital sign measurement frequency in the ED regardless of the automated measures. It is unclear why temperature was the least documented vital sign at all measured time points (temperature may have been communicated to providers without being documented or may have been less likely to be documented if patients were afebrile). Of note, time 0 was the first 15 minutes of patient presentation to the ED, so vitals may have been taken and even communicated between staff but not actually documented in the EHR by that time. It is important that future automated sepsis screening tools account for the emergency department workflow.

This study was limited by a small sample size from a single institution and only included patients with an ICD-10 diagnosis of severe sepsis with or without shock. Clinically, severe sepsis involves organ dysfunction; thus, the presentation and symptomatology of our patients may differ from patients with an ICD-10 diagnosis of sepsis alone. However, the term severe sepsis is no longer recommended and the most up-to-date definition of sepsis includes organ dysfunction, but ICD-10 codes have not yet reflected this.¹ Future prospective studies are still needed to assess which sepsis scores are best for early diagnosis in the ED. Additionally, because the study looks at criteria availability of both diagnostic and prognostic sepsis scoring methods, although patients did not have a diagnosis on arrival, an unintended bias was potentially introduced by studying patients who received an admitting diagnosis of severe sepsis since the same variables analyzed in this study are also used to make the diagnosis of sepsis. The diagnostic scoring methods (SIRS, NEWS and PRESEP) tend to have fewer criteria and are more vital signs dependent which demonstrated greater availability within the first 6 hours of patients’ presentations to the ED with exception of SIRS. However, qSOFA, which is prognostic and has the fewest criteria, was in the top three scores with the greatest criteria availability.

In summary, it is important that providers sift through the necessary sepsis criteria in a timely manner and that institutions consider criteria availability when implementing sepsis alert tools in the ED. Our data demonstrate that vital signs were readily obtained (>90% availability) at the time of triage, except for temperature, which continued to be the least documented vital sign. Most laboratory values were available (>90% availability) by T1 and T2, except bilirubin and creatinine. The results suggest that qSOFA, PRESEP and NEWS, which are very vital signs dependent can be utilized close to the time of triage and SOFA, SPEED, MEDS, and PIRO, which are fairly laboratory based can be utilized more as the patient’s time in the ED increases. Although SIRS is largely vital signs dependent and is widely utilized in sepsis detection, it was one of the scoring methods with the lowest percent of available criteria. The availability of patient data at different time points in a patient’s ED visit warrants future research in determining if different scoring methods, including traditionally prognostic ones, could be utilized at different time points in the ED to accurately diagnose sepsis.