Predictive Model Development and Validation of the Emergency Hester Davis Scale© for Fall Risk Screening

Introduction

Background and Objectives

Falls remain one of the most common adverse events in healthcare. In 2017, falls were the number one cause of fatal and nonfatal injuries in adults 65 years and older.^1,2 Although adults in this age group are at higher risk for falling, any patient experiencing physiological changes associated with medical conditions, medications, behavioral or other factors may be at risk for falls. Thirty to fifty percent of falls in the US result in injury and had estimated costs in 2015 of nearly $50 billion in persons ≥65 years.^3,4

Falls in healthcare, including emergency departments (ED), continue to be of crucial concern.^3,5-7 Inpatient fall rates range from 1.7 to 25 falls per 1000 patient days with an overall risk of falling ranging from 1.9% to 3%. Lower rates have been reported in the ED and range from 0.15 to 0.288 falls per 1000 patient visits.^5,8–10

Because falls are such a prevalent issue and are considered a nurse-sensitive metric of performance by the American Nurses Credentialing Center Magnet Recognition Program, the Joint Commission has emphasized the need to reduce fall-related injuries.^7,11–14 In addition, since 2008, fall-related injuries occurring during care have been classified as hospital-acquired conditions, and as such, the Centers for Medicare and Medicaid no longer reimburses providers or hospitals for treating them.^12,13

Three types of falls well documented in the literature include anticipated physiologic, accidental, and unanticipated physiologic falls. Anticipated physiologic falls are predictable, include those caused by morbidities such as gait disturbances, and constitute most falls. Accidental falls, due to slips or trips, and unanticipated physiologic falls from syncope or seizure, are unpredictable.^13–15 In 2015, “intentional falls” was added to this list by the National Database for Nursing Quality Indicators. By definition, these events are not truly falls nor are they considered predictable, but still need to be accounted for.¹⁶

Fall injuries can range from minor bruising and other soft tissue injuries to death.⁵ Brain injuries and injuries to the hips, legs and feet are among the most fatal and costly of fall injuries. These injury types account for 78% of fatalities and 79% of fall-related costs. Occurring in 33% of nonfatal falls, fractures account for 61% of nonfatal fall-related costs.¹⁷ Other consequences of falls include loss of mobility, functional decline, loss of independence, psychological consequences, social impact, and institutionalization.^18–22

While we know falls can result in serious injuries and even death, little research has focused on fall prediction in an ED setting.^10,23–25 Terrell and colleagues retrospectively tested the inpatient Hendrich II model in an ED.¹⁰ In their study, the sensitivity of the Hendrich II model was 37.5%. Specificity was not reported. Alexander, Kinsley and Waszinski implemented a fall prevention program using a prediction model they designed for the ED called the KINDER 1 fall risk assessment tool.²³ The KINDER 1 correctly identified 73% of fallers during the implementation program. Specificity of the tool was not reported. In a study of the MEDFRAT fall prediction tool developed by Flarity, Pate and Finch, researchers sought to determine the validity and reliability of this tool. Interrater reliability was 0.70, reflecting the moderate agreement of scores among clinicians. Due to technical difficulties, they were unable to obtain further validity and reliability from the first study.^24,26

Having the ability to predict and prevent falls is of fundamental clinical relevance. Accurate prediction of risk is the first step in prevention.²⁷ Without accurate risk prediction, implementation of fall prevention interventions is subjective at best, which jeopardizes any standard care approach for preventing falls and related injuries. With identified predictive risk characteristics, nurses could quickly implement prevention interventions.²⁷ Increased use of such strategies could translate into decreased falls and fall-related injuries in the emergency department.

The Hester Davis Fall Risk Assessment Scale (HDS) has been validated to predict fall risk in the hospital.¹³ Although the original HDS has been used successfully in the ED, the characteristics of ED fallers may differ from those in other hospital units.^10,23,24 In addition, the HDS is a comprehensive risk assessment tool that drives care planning based on identified risk factors. In the ED setting, a screening tool can provide more efficient identification of risk that allows for fall prevention safety bundles to be utilized. Therefore, the purpose of this study was to validate a derivative of the HDS, the emergency Hester Davis Scale (eHDS), which could potentially provide a more efficient tool for predicting anticipated physiologic falls occurring in the ED.

Methods

Source of Data, Participants and Sampling

This study included retrospective development and prospective validation of the eHDS. The setting was an academic medical center in the south-central US and a large metropolitan hospital system in southern California. Both sites include Level 1 trauma centers. The retrospective analysis sample was adult ED patients aged 18 and older treated at the study sites who had a documented fall while being treated in the ED between August 2014 and September 2015. The prospective sample included all patients treated in the ED at both the south-central site and a large, urban Level 1 Trauma Center in Los Angeles during the 10 months from April 2016 to January 2017. All fallers and a random sample of non-fallers were included. Randomization was performed by using every third admission to the ED based on admitting records for the study sites.

A secondary data analysis strategy was used to measure sensitivity and develop the model by retrospective analysis of adult falls in the ED. Once the model was retrospectively validated, we prospectively validated it at the two study sites described above. All data used for analysis were aggregated and deidentified, so no personal health information was utilized. Approval was obtained from the Kaiser Permanente Southern California institutional review board (protocol #10986) before beginning the study. In addition, an academic conflict of interest committee based at the University of Arkansas for Medical Sciences monitored the study under a management plan to protect against commercial bias in the conduction and reporting of the study.

Lists of patients who had fallen were retrieved. All adults who fell during their ED visit were considered for inclusion. Patients falling more than once were referred to as “repeat fallers.” Only the first fall of each faller was used for analysis in order to maintain the independence of the observations. This is consistent with previous studies evaluating risk factors associated with predicting falls in the hospital.^14,28,29

Outcome and Predictor Variables

The outcome variable, falls, was operationally defined as an unplanned descent to the floor with or without injury.³⁰ To ensure data accuracy regarding type of fall, the researchers examined all incident reports (reports of falls occurring in the ED) and medical records. Once all falls were confirmed, the researchers performed a review of each event to determine the type of fall each patient experienced. Only anticipated physiologic falls, considered predictable, were included for modeling. Specific patient variables of interest for inclusion in the retrospective model validation included: age, fall history, mobility, medications, cognitive impairment, toileting needs, volume/electrolyte issues, communication/sensory issues, and behavioral issues. The content and construct validity of these variables had already been established through psychometric validation of the HDS.¹³ All these variables were included at the request of frontline ED nursing staff who were familiar with the original HDS model.

Missing Data and Statistical Analysis Methods

For the prospective validation, emergency nurses were trained to use the instrument during the triage process before it was deployed into patient care via the Epic electronic medical record (2019 Epic Systems Corporation, Verona, WI). Compliance in completing the tool accurately was monitored during the study period at both participating institutions. All patients receiving triage at the study sites were screened for falls using the new tool. Standard reporting tools from Epic were utilized to collect data from the medical records. Cases with missing data were excluded from the prospective analysis. Sensitivity and specificity were measured, and the ROC procedure was used to measure the Area Under the Curve (AUC) for the eHDS. All statistical analyses were completed using SPSS v25 (SPSS Predictive Analytics, 2017, IBM Corporation, Armonk, NY).

Results

Data from 152 first time fallers in the study sites from August 2014 to September 2015 were included in a secondary data analysis to begin the development of the tool with an aim of reducing the pool of variables to only those necessary for accurate prediction. Our goal was to develop a screening tool for initial use in the triage setting that would provide quick, easy, and accurate prediction of fall risk. Data were obtained from both the study site’s variance reporting systems. The data source allowed for the collection of the demographic age but did not include information on patient gender. Ages of the 152 first time fallers ranged from 19 years to 98 years with a mean age of subjects being 65.5 years.

The presence of the nine predictor variables described above was determined for each of the fallers. Age was transformed into a categorical variable for analysis. Sensitivity of each variable was evaluated using frequency analysis. Findings from the frequency analysis of predictor variables for the retrospective model validation are presented in Table 1. Combinations of variables were manually evaluated to determine which combination of the fewest number of variables would yield the highest percent sensitivity. This analysis revealed three variables of significance in predicting anticipated physiologic falls in the ED setting. These variables included mobility, medications, and behavior. Using only these three variables to predict fall risk in the 152 first time fallers yielded a sensitivity of 93.4%. Because including any of the other variables (age, fall history, cognitive impairment, toileting needs, volume/electrolyte issues and communication/sensory issues) did not improve the sensitivity, these variables were excluded from the model implemented in the prospective evaluation. The remaining three variables were operationalized into a fall risk screening tool for use by nurses in the ED within the electronic health record as illustrated in Figure 1. Nurses screened the patient as a positive fall risk if they identified any issues with either mobility, medications, or behavior during the triage process.

Table 1 Frequency Analysis of Predictor Variables for the Retrospective Model Validation

Figure 1 Fall risk screening tool used for prospective validation.

ED visits to the study sites during the ten-month prospective study included 110,445 combined visits. A total of 19 patients (1.6%) experienced falls during the study period. Of these, 13 patients had an anticipated physiologic fall. Figure 2 presents the flow diagram of the categorization of patient falls for inclusion in the final analysis. Seven patients were female (54%) and six were male (46%). The average age of fallers was 52.1 years (range 19–100 years). This is consistent with the average age of fallers in other studies.³¹ The fall rate during the study period was 0.118 falls per 1000 patient visits. There was no missing data for fallers. Twenty non-faller cases were excluded for missing data.

Figure 2 Flow diagram of categorization of patient falls for inclusion in the final analysis.

Prospective Model Performance and Updating

Initial sensitivity of the eHDS that included all three variables derived from the secondary analysis was 100%. Initial specificity, obtained from a random sample of 216 non-fallers, was 41%. In a review of the factors, it was determined that medications were always scored together with either mobility or behavior, but never as a single predictor. Therefore, the category of medications was removed from the model leaving two predictor variables as illustrated in Figure 3. Removing medications from the model did not affect sensitivity and increased specificity to 68%. Interrater reliability was established by having the tool developer present three case studies to 20 staff nurses. Two cases were patients with fall risk, and one case study was a patient without fall risk. The interrater reliability of the final model was 90%. The ROC analysis of the final model yielded an AUC of 0.84. Figure 4 presents the output of this analysis. AUC scores range from 0.5 to 1, with 0.5 representing no discriminate ability and 1 representing a perfect test.¹³ The output in Figure 4 deviates from a typical curve as the analysis was performed on a screening tool with a positive/negative result versus a range of possible cut scores and is considered a convex curve with good separation between classes of fallers and non-fallers.³²

Figure 3 Final fall risk tool after prospective data evaluation.

Figure 4 Receiver operating characteristic curve for the emergency Hester Davis Scale.

Discussion

Interpretation

The eHDS correctly identified risk status in all 13 patients who fell during the study period (sensitivity 100%). In a random sample of 216 non-fallers, 69 were identified as at risk (specificity 68%). The sensitivity compares favorably with studies completed by both Terrell and colleagues and Alexander and colleagues (Table 2).^10,23 Neither of these studies reported specificity and Scott and colleagues reported neither sensitivity nor specificity.^10,23,24 An interrater reliability of 0.90, a high level of agreement among raters, and an AUC of 0.84 indicate the eHDS is an accurate fall prediction tool for adult patients in the ED.^33,34

Table 2 Comparison of Sensitivity and Specificity Among Hendrich II, Kinder 1, and Emergency Hester Davis Scale

To purely test an instrument, we would make a prediction using the instrument and simply follow the patient’s outcome. However, this is not viewed as an ethical approach when we know that there are ways to prevent what we predict. This is a known treatment paradox of falls research.¹³ Therefore, we implemented a fall prevention bundle for any patient screened to be at risk by the eHDS. It is also important to keep in context that each fall prediction tool is simply a compass to guide the nurse to know which patients are at risk. Actual fall prevention is achieved through care planning and implementation of fall prevention interventions. However, the low rate of falls in the study site EDs (0.118 per 1000 patient visits) in this study, compared with 0.228 to 0.57 per 1000 patient visits in the EDs in other studies, suggests the fall prevention interventions implemented may have prevented falls that would otherwise have occurred, and thus the tool may have a higher specificity than findings would indicate.^10,23,24 It is also possible that this new process changed usual practice in a way that heightened nurses’ awareness of risk and subsequent implementation of fall prevention bundles that were not previously happening in any reliable way. In addition, there may have been environmental design or other factors that contributed to this performance.

This screening was an additional workflow for the study sites during the triage process. We are currently investigating whether surrogate markers from other assessments could be used. The only barrier to implementation we experienced was getting sequenced into the existing build timelines of the local IT departments, which took some time. Nurses must be careful to interpret our results here to avoid overconfidence in the tool as ongoing evaluation is recommended.

Limitations

The study was conducted in a large urban and an academic university hospital, both Level 1 trauma centers, so findings may not be generalizable to smaller, rural hospital EDs. Ongoing study of the predictive accuracy of the eHDS should include smaller hospitals and neutral third-party testing as the content developed during this study is now proprietary and used by a commercial entity. Instrument evaluation is an ongoing process. In addition, we did not match cases between fallers and non-fallers. Blinding was not used in this study and is not typical in this type of falls research as it does not contribute to the research design.¹³ It is very possible that falls occurred during the study timeframe that were not reported by nursing staff. Although we acknowledge that the sensitivity may not remain perfect over time, evidence from the two years since initial validation and subsequent use in 65 EDs across the United States indicates the eHDS will perform well in EDs of various sizes and locations. Further, independent evaluation in these sites is encouraged.

Implications for Nurses

The ED is a busy, fast-paced environment and emergency nurses need a precise, short, reliable, and accurate screening tool with which to perform fall risk determination. The eHDS can be quickly completed and accurately identifies adult patients at risk for falls in the ED. Accurate prediction is the primary step in decreasing falls.

Conclusions

Findings from this study provide strong evidence that the eHDS is accurate in identifying patients in the ED who are at risk for falls. The specificity of the eHDS should be interpreted with caution, as interventions to prevent falls were in place during the study period which can falsely lower results reported here. The eHDS is the first known ED fall risk screening tool to undergo both retrospective validation and full prospective psychometric evaluation in a multisite study. It is a valid and reliable tool suitable for use in clinical practice.^33,34 An accurate tool that is easy to use in the clinical setting is vital to providing safe and effective care and decreasing the incidence of patient falls in the ED.