The Available Criteria for Different Sepsis Scoring Systems in the Emergency Department—A Retrospective Assessment
Received 5 September 2020
Accepted for publication 14 December 2020
Published 2 March 2021 Volume 2021:13 Pages 91—96
Checked for plagiarism Yes
Review by Single anonymous peer review
Peer reviewer comments 2
Editor who approved publication: Dr Hans-Christoph Pape
Sanjhai Ramdeen,1 Brandon Ferrell,1 Christopher Bonk,2 Laura Schubel,2 Robin Littlejohn,2 Muge Capan,3 Ryan Arnold,4 Kristen Miller1,2
1Georgetown University School of Medicine, Washington, DC, USA; 2National Center for Human Factors in Healthcare, MedStar Health Research Institute, Washington, DC, USA; 3Decision Sciences & MIS Department, LeBow College of Business, Drexel University, Philadelphia, PA, USA; 4Department of Emergency Medicine, Drexel University School of Medicine, Philadelphia, PA, USA
Correspondence: Kristen Miller
National Center for Human Factors in Healthcare, MedStar Health Research Institute, 3007 Tilden Street NW, Suite 6N, Washington, DC, 20008, USA
Tel +1 2022433873
Fax +1 8553449800
Email [email protected]
Objective: The goal of the study was to assess the criteria availability of eight sepsis scoring methods within 6 hours of triage in the emergency department (ED).
Design: Retrospective data analysis study.
Setting: ED of MedStar Washington Hospital Center (MWHC), a 912-bed urban, tertiary hospital.
Patients: Adult (age ≥ 18 years) patients presenting to the MWHC ED between June 1, 2017 and May 31, 2018 and admitted with a diagnosis of severe sepsis with or without shock.
Main Outcomes Measured: Availability of sepsis scoring criteria of eight different sepsis scoring methods at three time points— 0 Hours (T0), 3 Hours (T1) and 6 Hours (T2) after arrival to the ED.
Results: A total of 50 charts were reviewed, which included 23 (46%) males and 27 (54%) females. Forty-eight patients (96%) were Black or African American. Glasgow Coma Scale was available for all 50 patients at T0. Vital signs, except for temperature, were readily available (> 90%) at T0. The majority of laboratory values relevant for sepsis scoring criteria were available (> 90%) at T1, with exception to bilirubin (66%) and creatinine (80%). NEWS, PRESEP and qSOFA had greater than 90% criteria availability at triage. SOFA and SIRS consistently had the least percent of available criteria at all time points in the ED.
Conclusion: The availability of patient data at different time points in a patient’s ED visit suggests that different scoring methods could be utilized to assess for sepsis as more patient information becomes available.
Keywords: sepsis, scoring systems, available criteria, emergency department, patient safety, quality improvement
Sepsis, defined as infection with associated acute organ system dysfunction, is one of the leading causes of hospital admissions in the United States.1 Because sepsis is rapidly progressive, delayed recognition can have dire consequences. Recent literature has highlighted the importance of early identification and treatment of sepsis.1,2 Since 67–79% of septic patients present to the hospital with severe sepsis3,4 and patients with severe sepsis have the highest rate of mortality,1,5 recognizing these cases and initiating appropriate treatment is of utmost importance. Efficient scoring methods for diagnosing sepsis are critical, as time can be the difference between life and death.
In order to treat sepsis quickly, providers use sepsis scoring systems to diagnose and quantify the risk of mortality in acutely ill patients.3,6 Mixed models addressing alerts for sepsis, integrating organ failure assessment scores and general severity scores, have been published but have not gained widespread acceptance.4,7 Existing scores have marked variation in their ability to provide prognostic support for clinical decision-making.8 There is often a lack of context for the identification of sepsis, which can lead to inconsistencies across providers, thus leading to increased mortality in acutely ill patients.6 However, a patient’s initial presentation can provide important data that can be used to identify and decrease the possibility of mortality.
Sepsis remains an illness difficult to identify and a gold-standard test for diagnosis does not currently exist. Multiple sepsis screening tools have been developed and evaluated for both pre-hospital and in-hospital systems. Diagnostic scoring systems, which include the systemic inflammatory response syndrome (SIRS) criteria,3 national early warning score (NEWS),5 and prehospital early sepsis detection (PRESEP) score,9 tend to have fewer criteria, as the goal of these scores is to rule in patients with potential sepsis. Scoring systems that include the sequential organ failure assessment (SOFA),10 quick sequential organ failure assessment (qSOFA),11 sepsis patient evaluation in the emergency department (SPEED),12 mortality in emergency department sepsis (MEDS),13 and predisposition, infection/insult, response and organ dysfunction (PIRO)14 are prognostic and primarily focus on identifying patients with increased mortality.
When components required to compute the score are available, there is an increased possibility for corrective action in order to reduce the patient’s overall risk of mortality.15 However, sepsis scoring criteria are not always available in a busy emergency department (ED), hindering the predictive accuracy of scoring systems and indirectly contributing to patient deterioration. The question arises of how to address missing scoring criteria, as the patient’s outcome relies on the availability of all scoring criteria. Can a patient be given an accurate diagnosis through these sepsis scoring protocols if aspects of the criteria are still missing? This study aims to assess the real-time availability of the criteria for eight commonly used sepsis scoring systems within 6 hours of triage in the ED.
Study Design and Setting
A retrospective chart review was performed to compare the availability of criteria for eight commonly used sepsis scoring methods (SIRS, NEWS, PRESEP, SOFA, qSOFA, SPEED, MEDS, and PIRO) within the first 6 hours of a patient’s arrival to the MedStar Washington Hospital Center (MWHC) ED. MWHC is a 912-bed, urban, academic hospital with 86,771 annual visits and the largest and busiest ED in the District of Columbia. Data were extracted from the electronic health record (EHR) for visits between June 1, 2017 and May 31, 2018 and captured into a standardized Research Electronic Data Capture (REDCap) abstraction (SR, BF, CB). Discrepancies during data extraction were resolved by discussion and consensus.
Selection of Participants
Adult patients (≥18 years old) with an ICD-10 admitting diagnosis of severe sepsis without or with shock (ICD-10 codes R65.20 and R65.21, respectively) were eligible for inclusion in the study. Subjects were excluded if they were dead on arrival to the ED.
Sepsis Scoring Systems
A side-by-side comparison of each criterion of the eight commonly used sepsis scoring tools is included (Appendix Table 1). SIRS, NEWS, and PRESEP are diagnostic while SOFA, qSOFA, SPEED, MEDS, and PIRO are prognostic. qSOFA has the fewest criteria (3), while PIRO has the most (14). All scoring methods, besides SIRS, allocate points for each criterion. The inclusion of vital signs is common amongst all scoring systems. The diagnostic scoring methods are largely vital signs dependent, with PRESEP being based entirely on vital signs. qSOFA is the only prognostic scoring system with the majority (2/3, 66.7%) of its criteria as vital signs. Four (50%) scoring methods (NEWS, SOFA, qSOFA, and MEDS) account for altered mental status (AMS), with SOFA and qSOFA relying specifically on GCS.
Five (62.5%) scoring systems (SIRS, SOFA, SPEED, MEDS, and PIRO) include laboratory criteria. PIRO and SOFA both include the most lab dependent criteria (4/14, 28.6% and 4/6, 66.7%, respectively) with SOFA having the highest percentage of lab dependent criteria. SPEED, MEDS and PIRO each have an equal number of vital signs as laboratory criteria.
In addition to collecting data relevant to each of the eight scoring tools, patient demographics and outcomes were recorded, including in-hospital mortality, code blue (cardiopulmonary arrest code) activation, rapid response activation, respiratory failure, and progression to septic shock. Vital signs and laboratory values were also recorded at times 0 (T0), 3 (T1), and 6 (T2) hours after triage to parallel sepsis bundles currently implemented at our institution. NEWS uses the Alert, Verbal, Pain, Unresponsive (AVPU) scale in its scoring criteria which is not a standard documented value in the ED at MWHC, so a previously validated Glasgow Coma Scale (GCS) to AVPU conversion was used.16
All data, 181 variables, were cataloged and analyzed in REDCap. Categorical data were reported as frequencies and continuous variables were reported as mean and standard deviation (SD). The MedStar Health Institutional Review Board approved this study and approved a full waiver of Health Insurance Portability and Accountability Act (HIPAA) authorization for research purposes prior to conducting research practices. Human subjects consent was not obtained prior to study commencement due to the impracticality of obtaining consent before subject identification. As part of the study, we analyzed the availability of new vital sign and laboratory criteria at particular times (T0, T1 and T2); however, for the purposes of calculating the percent of criteria available per scoring system, vital signs and laboratory values available from earlier in the ED visit were considered available at later time points as they could still be used clinically.
Characteristics of Study Subjects
The data retrieval produced 631 records with an ICD-10 code for severe sepsis. A random sample of 128 charts were reviewed to allow for the collection of all 181 variables from each eligible record reviewed. Seventy-eight records were excluded given that severe sepsis was not the admitting diagnosis but rather diagnosed later in the hospital course. A total of 50 subjects were included in the final analysis (Table 1). A PRISMA flow diagram depicting case selection is included in Figure 1.
Table 1 Patient Demographics (n=50, with One Patient Identifying as More Than One Ethnicity)
Figure 1 PRISMA flow diagram for case selection.
Vital signs were readily obtained (>90% availability) at T0 except for temperature, which was only documented in 39 (78%) patient charts (Table 2). Temperature continued to be the least documented vital sign at T1 and T2 with 30 (60%) and 34 (64%) patients with documented temperatures, respectively. Respiratory rate was documented in 46 (92%) charts at T0, but in only 33 (66%) and 37 (74%) charts at T1 and T2, respectively. Most laboratory values were not available at T0; white blood cell count and hematocrit were the highest reported (14%). By T1 and T2, most laboratory values were available (>90%) except for bilirubin and creatinine (Table 2). GCS was documented for all 50 (100%) patients at T0, with an average score at triage of 13.14 (3.18). Twenty-four (48%) patients were documented as having altered mental status (AMS) either upon arrival or during their ED course. Clinical outcomes included a mean hospital stay of 9.12 (9.05) days, 8 rapid response activations, 3 code blue alerts, 28 (56%) patients progressed to septic shock, 20 (40%) patients developed respiratory failure, and 18 (36%) patients died in the hospital.
Table 2 Vital Signs and Laboratory Value Availability
Most scoring systems had the majority (>50%) of their criteria available at the time of triage, except SOFA only had 38% (13) of its criteria available on average (Table 3). NEWS, PRESEP and qSOFA had the highest percentage of available criteria, with greater than 90% available at all time points (T0, T1, and T2). By 3 hours in the ED, all scores had greater than 75% of their criteria available, with SOFA and SIRS having the least percent of available criteria at 77% (17) and 78% (12), respectively. SOFA and SIRS continued to have the least percent of available criteria at 6 hours in the ED, both having only 79% (SD 16 and 10, respectively) of their criteria available.
Table 3 Scoring Tool Criteria Availability
Sepsis can be difficult for providers to define, identify and treat. A patient’s presenting symptoms and the allocation of points towards sepsis scores are closely tied to the availability of patient data at 0 (triage), 3, and 6 hours into the patient’s visit. Knowing which criteria tend to be available at a given time has the potential to improve sepsis detection and monitoring throughout the patient’s ED visit. The goal of this study was to provide a crucial analysis of the availability of criteria in the ED of the most widely used sepsis scoring systems. In addition to diagnostic criteria, we included variables from prognostic scoring criteria not typically used in the ED to evaluate whether there were enough available data to potentially utilize them within the first 6 hours of a patient’s ED course. To our knowledge, this has not been previously completed at the institutional level and no other study has compared the criteria for these eight commonly used sepsis scoring tools in a single table (Appendix Table 1). Since protocols and data availability differ by institution, the traditional ED sepsis diagnostic scores may not be useful for some EDs, as was the case at our ED where SIRS criteria were less readily available. Our results suggest that multiple diagnostic and prognostic scoring systems have enough available criteria to be potentially utilized at different time points in the ED. Heavily vital signs dependent scores (qSOFA, PRESEP and NEWS) have more of their criteria fulfilled closer to the time of triage when quick and accurate medical decision-making is especially crucial while heavily laboratory dependent scores (SOFA, SPEED, MEDS and PIRO) could become more useful as the patient’s time in the ED increases. Of note, SIRS had the third-lowest percent of available criteria (57%) at triage despite being heavily vital signs dependent and diagnostic, while qSOFA had the highest percent of available criteria (97%) despite it being prognostic.
Some sepsis scoring systems (NEWS, SOFA, qSOFA, and MEDS) require the assessment of mental status, as sepsis-associated encephalopathy is indicative of increased mortality risk.17 GCS and the AVPU scale are two frequently used scales that assess neurologic status.5 All patients in this study had GCS assessed at triage. Although these scales can assess an aspect of orientation, they are used to calculate a patient’s level of consciousness and do not account for a change in baseline mental status. While NEWS, SOFA and qSOFA include GCS or the AVPU scale, MEDS relies on a non-numerical classification of AMS by history or physical exam. Although MEDS is currently considered to be a prognostic tool, the inclusion of clinical judgment in assessing AMS could have early diagnostic value in detecting sepsis.
MWHC has a focus on organizational quality improvement including significant improvements in health information technology (IT) for a single, standardized bundle compliance dashboard and automated alerts to ensure timely initiation of sepsis treatment bundles. Of course, there are more general standards for vital sign measurement frequency in the ED regardless of the automated measures. It is unclear why temperature was the least documented vital sign at all measured time points (temperature may have been communicated to providers without being documented or may have been less likely to be documented if patients were afebrile). Of note, time 0 was the first 15 minutes of patient presentation to the ED, so vitals may have been taken and even communicated between staff but not actually documented in the EHR by that time. It is important that future automated sepsis screening tools account for the emergency department workflow.
This study was limited by a small sample size from a single institution and only included patients with an ICD-10 diagnosis of severe sepsis with or without shock. Clinically, severe sepsis involves organ dysfunction; thus, the presentation and symptomatology of our patients may differ from patients with an ICD-10 diagnosis of sepsis alone. However, the term severe sepsis is no longer recommended and the most up-to-date definition of sepsis includes organ dysfunction, but ICD-10 codes have not yet reflected this.1 Future prospective studies are still needed to assess which sepsis scores are best for early diagnosis in the ED. Additionally, because the study looks at criteria availability of both diagnostic and prognostic sepsis scoring methods, although patients did not have a diagnosis on arrival, an unintended bias was potentially introduced by studying patients who received an admitting diagnosis of severe sepsis since the same variables analyzed in this study are also used to make the diagnosis of sepsis. The diagnostic scoring methods (SIRS, NEWS and PRESEP) tend to have fewer criteria and are more vital signs dependent which demonstrated greater availability within the first 6 hours of patients’ presentations to the ED with exception of SIRS. However, qSOFA, which is prognostic and has the fewest criteria, was in the top three scores with the greatest criteria availability.
In summary, it is important that providers sift through the necessary sepsis criteria in a timely manner and that institutions consider criteria availability when implementing sepsis alert tools in the ED. Our data demonstrate that vital signs were readily obtained (>90% availability) at the time of triage, except for temperature, which continued to be the least documented vital sign. Most laboratory values were available (>90% availability) by T1 and T2, except bilirubin and creatinine. The results suggest that qSOFA, PRESEP and NEWS, which are very vital signs dependent can be utilized close to the time of triage and SOFA, SPEED, MEDS, and PIRO, which are fairly laboratory based can be utilized more as the patient’s time in the ED increases. Although SIRS is largely vital signs dependent and is widely utilized in sepsis detection, it was one of the scoring methods with the lowest percent of available criteria. The availability of patient data at different time points in a patient’s ED visit warrants future research in determining if different scoring methods, including traditionally prognostic ones, could be utilized at different time points in the ED to accurately diagnose sepsis.
KM conceived the study and its design. KM takes responsibility for the paper as a whole. SR, BF, CB undertook the creation of the database, data collection, statistical analysis, and data analysis. SR, BF, CB drafted the manuscript, and all authors contributed substantially to its revision. All authors contributed to data analysis, drafting or revising the article, have agreed on the journal to which the article will be submitted, gave final approval of the version to be published, and agree to be accountable for all aspects of the work.
Ms Robin Littlejohn reports grants from the National Library of Medicine, during the conduct of the study. Dr Ryan Arnold reports grants from NIH/NIGMS, during the conduct of the study. Dr Kristen Miller reports grants from the National Library of Medicine, during the conduct of the study. The authors report no other conflicts of interest in this work.
1. Singer M, Deutschman CS, Seymour CW, et al. The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA. 2016;315(8):801. doi:10.1001/jama.2016.0287
2. Morr M, Lukasz A, Rübig E, et al. Sepsis recognition in the emergency department - impact on quality of care and outcome? BMC Emerg Med. 2016;17(1):11. doi:10.1186/s12873-017-0122-9
3. Bone RC, Balk RA, Cerra FB, et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. Chest. 1992;101(6):1644–1655. doi:10.1378/chest.101.6.1644
4. Levy MM, Fink MP, Marshall JC, et al. 2001 SCCM/ESICM/ACCP/ATS/SIS international sepsis definitions conference. Crit Care Med. 2003;31(4):1250–1256. doi:10.1097/01.CCM.0000050454.01978.3B
5. Smith GB, Prytherch DR, Meredith P, et al. The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death. Resuscitation. 2013;84(4):465–470. doi:10.1016/j.resuscitation.2012.12.016
6. Amland RC, Hahn-Cover KE. Clinical decision support for early recognition of sepsis. Am J Med Qual. 2016;31(2):103–110. doi:10.1177/1062860614557636
7. Timsit J, Fosse J, Troché G, et al. Accuracy of a composite score using daily SAPS II and LOD scores for predicting hospital mortality in ICU patients hospitalized for more than 72 h. Intensive Care Med. 2001;27(6):1012–1021. doi:10.1007/s001340100961
8. Kovach CP, Fletcher GS, Rudd KE, et al. Comparative prognostic accuracy of sepsis scores for hospital mortality in adults with suspected infection in non-ICU and ICU at an academic public hospital. Moreira J, ed. PLoS One. 2019;14(9):e0222563. doi:10.1371/journal.pone.0222563
9. Bayer O, Schwarzkopf D, Stumme C, et al. An early warning scoring system to identify septic patients in the prehospital setting: the PRESEP score. Acad Emerg Med. 2015;22(7):868–871. doi:10.1111/acem.12707
10. Vincent JL, Moreno R, Takala J, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the working group on sepsis-related problems of the European Society of intensive care medicine. Intensive Care Med. 1996;22(7):707–710. doi:10.1007/BF01709751
11. Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA. 2016;315(8):762–774. doi:10.1001/jama.2016.0288
12. Bewersdorf JP, Hautmann O, Kofink D, et al. The SPEED (sepsis patient evaluation in the emergency department) score: a risk stratification and outcome prediction tool. Eur J Emerg Med. 2017;24(3):170–175. doi:10.1097/MEJ.0000000000000344
13. Shapiro NI, Wolfe RE, Moore RB, et al. Mortality in Emergency Department Sepsis (MEDS) score: a prospectively derived and validated clinical prediction rule. Crit Care Med. 2003;31(3):670–675. doi:10.1097/01.CCM.0000054867.01688.D1
14. Howell MD, Talmor D, Schuetz P, et al. Proof of principle: the predisposition, infection, response, organ failure sepsis staging system. Crit Care Med. 2011;39(2):322–327. doi:10.1097/CCM.0b013e3182037a8e
15. Rivers E, MacIntyre L, Morro D, et al. Early and innovative interventions for severe sepsis and septic shock: taking advantage of a window of opportunity. Can Med Assoc J. 2005;173(9):1054–1065. doi:10.1503/cmaj.050632
16. McNarry AF, Goldhill DR. Simple bedside assessment of level of consciousness: comparison of two simple assessment scales with the Glasgow Coma scale. Anaesthesia. 2004;59(1):34–37. doi:10.1111/j.1365-2044.2004.03526.x
17. Sprung CL, Peduzzi PN, Shatney CH, et al. Impact of encephalopathy on mortality in the sepsis syndrome. Crit Care Med. 1990;18(8):801–806. doi:10.1097/00003246-199008000-00001
This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.Download Article [PDF]