Development and Validation of a Nomogram for Predicting Risk of Emergency Department Revisits in Chinese Older Patients

Introduction

Geriatric emergency department (ED) visits and repeat visits place a substantial burden on healthcare systems worldwide. Older people represent disproportionately frequent users of hospital ED care. They accounted for up to 30% of all ED presentations globally,¹ and this proportion is continuously increasing in the context of unprecedented population ageing. Moreover, older people presenting to EDs often exhibit unique and complex clinical characteristics, and they are often accompanied by geriatric conditions such as multimorbidity, frailty and polypharmacy. As a result, this could increase the difficulty of acute diagnosis or treatment and make older patients susceptible to adverse health outcomes after discharge.^2,3 The hospital EDs, however at present, are still not specifically equipped to deal with the unique acute care needs from geriatric patients. Emergency settings that focus on prompt assessment and rapid decision-making are usually stressful for healthcare professionals and patients,⁴ and the quality and continuity of geriatric ED services are sometimes defective owing to failure to recognize and appropriately manage geriatric syndromes.² Older population, compared with their younger counterparts, are therefore more likely to subsequently experience health adversities and make repeated visits to EDs.²

ED revisits are a phenomenon attracting increasing clinical and research attention. Repeated ED attendances are considered indicative of ongoing patient needs that are not being adequately addressed by initial ED treatment, transitional care arrangements following discharge, or subsequent local community-based services.^5,6 The risk of unscheduled ED revisits by older adults is approximately two or three times as high as that for younger adults.⁷ Previous literature reported that up to 22% of older patients had a ED return visit within 1 month after discharge,¹ and that 25%, 38%, 49%, and 60% of patients over 75 years were readmitted to EDs at 1, 3, 6, and 12 months, respectively.⁸ Unplanned re-attendances to hospital EDs are likely to result in adverse events for older patients such as functional decline, mortality and increased healthcare costs,⁹ and may also aggravate the global challenge of ED overcrowding that is further linked to diminished efficiency, quality and safety of emergency care.¹ Repeated ED visits by older people are indeed often avoidable or preventable, if better care management and appropriate geriatric interventions are readily in place;^2,8,10 the identification of older patients at risk of ED revisits has thus become a priority, which can thereafter help to prevent undesirable outcomes.

Older adults with repeated attendances to EDs represent a high-risk population with specific characteristics, and risk assessment tools for their ED revisits can highlight the opportunity to predict future emergency resource utilization based on past available information as well as allow patient triage to facilitate personalized health interventions. Although extensive literature has well documented various risk factors for ED returns in older adults including sociodemographic characteristics, health behaviors, chronic conditions, clinical features, body mass index (BMI) and physical performance measures,^9,11–14 studies attempting to integrate multiple factors and construct clinical prediction tools to individualize ED revisit risk still remain scarce. To date, only a few studies have reported on ED-revisit prediction models in older persons,^5,7,15–19 among which the Identification of Seniors At Risk (ISAR) and the Triage Risk Screening Tool (TRST) are the two most frequently validated tools. However, the existing tools were rarely conducted in Asian countries including China, and were sometimes limited by focusing on specific-disease related emergency revisits instead of all-cause revisits, composite outcomes inclusive of emergency revisits, patients with particular disease or condition, only dichotomizing patients into high/low risks rather than quantifying the estimated risks, or a short time frame immediately following discharge. Nomogram, a visualized prediction tool for quantifying the probabilities of each individual experiencing specific clinical events of interest based on statistical regression equations, has been recently widely applied in clinical settings due to its visual and mathematical advantages but has not yet been adequately explored in the area of ED-revisit estimation.^20,21 The present study thus aims to identify risk factors influencing 1-year ED revisits, and to develop and validate a nomogram as a clinical prediction tool for estimating ED revisits probability in older patients.

Materials and Methods

Study Design and Population

This 1-year prospective cohort study was conducted and reported in accordance with the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) Statement.²² Subjects were selected using convenience sampling from the EDs at two tertiary hospitals in Guangzhou and Nanjing of China during the period from August 2018 to February 2019, and were interviewed through a structured face-to-face questionnaire by trained investigators. Patients with the following characteristics were included: 1) aged 60 years or above; 2) attended the hospital ED for treatment; 3) be accompanied by a knowledgeable caregiver. The exclusion criteria were patients with end-stage illnesses, inability to complete the questionnaire or refusal to participate. We recruited 745 patients from the baseline survey, and followed them for one consecutive year. After excluding subjects reporting missing baseline data and who were dead or lost to follow up, a total of 553 older patients remained and were eligible for analysis. The eligible patients were then randomly assigned to either the training set (n=369) or validation set (n=184) at a ratio of 2:1. A multivariable prediction model was developed based on the training cohort and further validated in the validation cohort. The study protocol was reviewed and approved by the institutional ethics review committee, and informed consent was obtained from each patient or their legal representative.

Measures

Primary Outcome

The primary outcome was any revisits to hospital emergency department within the one-year period of discharge, which was collected through follow-up phone calls. If the patients discharged from ED had at least one ED return visit within 1 year, they were classified as patients with one-year ED revisit.

Candidate Predictors

The presumptive nomogram predictors were potential factors associated with ED revisit in older patients selected based on previous literature and in consideration of accessibility in acute care setting. Candidate predictors covered the socio-demographics, lifestyle, functional status, geriatric parameters and illness conditions of patients, including age, gender, residence, marital status, educational attainment, household income per capita, health insurance, smoking, drinking, nutrition, fruit consumption, vegetable consumption, milk consumption, living arrangement, social isolation, sleep problem, BMI, life satisfaction, frailty status, polypharmacy, comorbidity, hypertension, diabetes, stroke, activities of daily living (ADL) limitation. A detailed description of candidate predictors can be found in Supplementary Table S1.

Statistical Analysis

Descriptive statistics were used to summarize baseline characteristics, where categorical variables were presented as numbers (percentages) and continuous variables were expressed as median (25–75% interquartile). Chi-square test or Mann–Whitney test was used to assess differences in the distribution of variables between training and validation cohorts. Univariable logistic analysis was performed in the training set to identify significant variables (p<0.1) for further multivariable analysis. The final multivariable regression model was built from a set of candidate variables using backward stepwise selection procedure, and the results were expressed as odds ratios (ORs) with 95% confidence intervals (CIs). A nomogram for predicting the probability of 1-year ED revisit was then elaborated based on the independent risk factors identified in the final multivariable model.

Nomogram performance was assessed in both the training and validation cohorts by the area under the receiver operating characteristic curve (AUC) or Harrell’s C-statistic as a measure of discriminative ability, a calibration plot as an indicator of internal calibration and the Hosmer-Lemeshow test to evaluate goodness of fit. Decision curve analysis (DCA) was further applied to determine the clinical utility of the nomogram by quantifying the net benefits at different threshold probabilities in both datasets. Data analyses were performed with Stata software (version 16.0), and differences were considered significant at two-tailed p<0.05.

Results

Participant Characteristics

Of the 553 enrolled participants, the mean (sd) age was 77.2 (8.5) years and 47.6% were female. Revisits to hospital EDs within one year following initial discharge were recorded in 268 (48.5%) older patients. Table 1 displays the baseline characteristics in the training and validation cohorts. The distribution of characteristics was generally similar between the 369 participants in the training set and the 184 participants in the validation set (all p>0.05).

Table 1 Baseline Characteristics of Patients in the Training and Validation Sets

Predictor Variables

Before constructing the nomogram, candidate risk factors were evaluated using univariable and multivariable analyses in the training cohort (Table 2). In univariable analysis, nine variables were identified as factors significantly associated with ED revisits (p<0.1), including age, gender, educational attainment, health insurance, social isolation, BMI, life satisfaction, frailty and polypharmacy, which were selected for further multivariable analysis. Then, following the backward stepwise selection procedure, four significant predictors including age, BMI, frailty index and polypharmacy remained and were included in the final multivariable model for construction of the risk prediction model (p<0.05). Results showed that older patients having revisits to EDs within 1-year period were less likely to be older than 90 years (OR=0.21, 95% CI=0.08–0.56; p=0.002) and were more likely to have a BMI<18.5 kg/m² (OR=2.45, 95% CI=1.12–5.34; p=0.025), have polypharmacy (OR=1.69, 95% CI=1.00–2.83; p=0.049) or be frail (OR=1.17, 95% CI=1.02–1.35; p=0.031) (Table 2).

Table 2 Univariable and Stepwise Multivariable Analyses of Risk Factors for 1-Year Emergency Department Revisits in the Training Cohort

Model Development

The nomogram established based on the final multivariable model is presented in Figure 1. In the nomogram, a score was assigned on the point scale for each subtype of these variables (namely, age, BMI, frailty index and polypharmacy) included in the nomogram, and the total score was calculated as the sum of the score of each variable. The probability of 1-year ED revisit for each individual can then be estimated by drawing a vertical line from the total score up to the risk line, where higher total points represented a greater probability of ED revisit.

Figure 1 Nomogram to estimate risk of emergency department revisits in older patients. Abbreviation: ED, emergency department. Notes: The nomogram incorporates age, BMI, frailty index and polypharmacy to estimate the probability of 1-year ED revisit for a given patient. To use the nomogram, drawing a vertical line from the scale of each variable downward to the Points axis and then recording the corresponding point associated with that scale. Summing up the point of each variable and locating this sum on the Total points axis at the bottom of nomogram. Then drawing a vertical line up to the Probability axis to read off the estimated probability of 1-year ED revisit.

Model Performance

Figure 2 depicts the results of nomogram performance in terms of its discrimination and calibration ability in both the training and validation sets. As shown in Figure 2A and C, ROC curves were constructed and the AUC values of the nomogram were 0.661 (95% CI=0.605–0.716) and 0.651 (95% CI=0.572–0.730) for the training and validation cohorts, respectively, indicating moderate discriminative ability. The calibration plots (Figure 2B and D) showed good agreement between prediction and observation in both sets (close to the 45-degree line), with mean predicted probabilities based on the proposed nomogram close to observed actual probabilities of 1-year ED revisits. The calibration ability was further supported by the Hosmer-Lemeshow test with non-significant p values (training cohort: χ²=5.43, p=0.861; validation cohort: χ²=8.51, p=0.579), suggesting that the nomogram was well-calibrated and provided a good fit of the data.

Figure 2 Receiver operating characteristic curves and calibration plots for the nomogram. Abbreviations: ROC, receiver operating characteristic; AUC, area under the receiver operating characteristic curve. Notes: ROC curves and calibration plots for the nomogram in the training (A and B, respectively) and validation cohorts (C and D, respectively) are presented. In the calibration plot, the horizontal axis was the ED revisit rate predicted by the nomogram, and the vertical axis was the actual or observed ED revisit rate; the dashed line indicates that the predicted probability completely fits the actual probability, and the black solid line shows the actual performance of the nomogram; if the calibration curve of the nomogram is close to the 45-degree line, it shows good agreement between nomogram-based prediction and actual observation.

Furthermore, DCA curves were drawn to illustrate the clinical applicability of the nomogram (Figure 3). The decision curves revealed the nomogram as possessing superior net benefits at a wide range of threshold probabilities, suggesting its clinical usefulness. When the threshold probability of an individual was approximately between 0.25 and 0.80 in both cohorts, using the proposed nomogram model to predict 1-year ED revisit risk added more benefit than either the treat-all tactics or the treat-none tactics.

Figure 3 Decision curves analysis for the nomogram in training and validation cohorts. Abbreviation: DCA, decision curves analysis. Notes: Decision curves analyses for the nomogram in the (A) training set and (B) validation set are presented. The net benefit (y-axis) was plotted versus the threshold probability (x-axis). The dark solid line depicts the net benefit of the proposed nomogram; the grey solid line and dark dash line represent the net benefit of strategies assuming that all patients (treat-all) and no patient had ED revisit (treat-none), respectively.

Discussion

To the best of our knowledge, this is the first attempt to establish a risk prediction model to individualize the probability of 1-year ED revisits for older adults in China. In our study, the enrolled older patients experienced ED revisits at a high rate of 48.5% over the course of 1 year after discharge, suggesting that geriatric ED returns are frequent and common. Previous literature reported similar rates of 1-year ED revisits by older patients as in our study, documenting that the rate exceeded 40% at 1 year in some settings.²³ Other studies reported even higher rates possibly owing to their different focuses on populations with older age or affected by particular diseases, for instance, the rate was recorded as 60% among patients over 75 years⁸ or was two-thirds (66.1%) among community-dwelling older adults living with dementia.²⁴ The exact reasons for high ED-return rates in older patients are uncertain, but may depend on potential factors such as poor quality of care or medical errors at the initial ED visit, patient’s chronic medical conditions with high relapse possibilities, and inadequate primary care or continuity of care.

Our study developed and internally validated a ED risk nomogram integrating four predictors including age, BMI, frailty and polypharmacy. Results based on both training and validation cohorts indicated that our nomogram was effective with regard to its moderate discrimination, satisfactory calibration, and good clinical applicability. The risk prediction models for ED revisits in older adults were also reported in prior studies, although many of them evaluated ED-return risk within a shorter period after discharge and did not specifically calculate the probability. For example, ISAR (consisting of six assessment items: presence of home help, increased dependency, recent hospitalization, impaired vision, impaired memory, and polypharmacy)^8,19,25,26 and TRST (including six items: cognitive impairment, living alone or no caregiver, difficulties with walking or transfer, recent ED admission or hospitalization, polypharmacy, and professional recommendation)^8,15,16 were the two most studied tools to detect high-risk patients for unplanned ED revisits, both of which produced similar discriminative ability as in our nomogram. These two tools, however, were disadvantaged because they can only dichotomize patients into high/low risks but cannot calculate an estimated percentage chance of revisit. We identified only two available prediction tools capable of providing risk probability: one was the LACE index developed among older UK patients, which unfortunately was a poor predictor of 30-day ED readmission with c-statistics of 0.55;¹⁸ the other was a nomogram established among Australian older patients with acceptable c-statistics of 0.65, which incorporated age, polypharmacy, cognitive impairment and depression to predict 28-day ED revisit.^5,17 Nevertheless, the aforementioned tools were all limited to western countries and relatively few were conducted in Asian countries including China. We identified only one relevant study by Chen et al that established a prediction model for 72-hour unscheduled ED return visits in Taiwan, showing that economic status, chronic illness, and ED length of stay were the top three variables influencing revisits.⁷

We identified age, BMI, frailty and polypharmacy as key factors contributing to the risk of 1-year ED repeat visits among older adults, and these variables predicting geriatric ED revisits have also been reported in previous literature. Our results showed that older patients with polypharmacy (≥5 drugs) or frailty were more likely to have repeated ED visits, which is generally consistent with prior studies.^5,9,27–29 Plausible explanations for such findings included that: 1) polypharmacy often indicates the presence of underlying multiple health problems, which may complicate or interrupt the conventional ED treatment and hence increase compensated healthcare needs; 2) frailty represents a comprehensive geriatric syndrome and is known to be associated with increased risks of subsequent adverse outcomes such as disability, falls and fractures, pain, and hospitalization,^30–32 all of which may consequently result in excessive reliance on EDs. Meanwhile, our study found that age ≥90 was associated with a reduced risk of 1-year ED revisits. Although previous evidence reached consensus that older adults encountered more ED returns when compared with younger adults, the increasing age, within the spectrum of the older population, has yet demonstrated contradictory results. Some studies coincided with our finding that the oldest old were less prone to re-visit,^33–35 whereas others concluded that the likelihood of returning to EDs increased concurrently with advancing age.^5,11,13,28 Such inconsistencies in the age effect highlight the importance of considering factors that can more accurately reflect biological or functional age (eg, frailty) rather than simply chronological age in the geriatric population evaluation. Exact reasons behind this counterintuitive result of advanced age are uncertain, but possible explanations are that oldest-old patients may be less likely to receive intensive or invasive types of acute treatment during initial ED visit, and that such patients discharged from EDs may be transferred to a nursing home with provision of basic care or have died within a short period, altogether contributing to reduced ED revisits. Besides, underweight BMI of <18.5 kg/m² compared to normal BMI was demonstrated as a risk factor for geriatric ED revisits in our study. BMI was less often evaluated previously, and we identified only one prior article documenting that obese BMI led to significantly increased ED risk.¹⁴ Our finding on BMI is not unexpected because underweight or unexpected weight loss in older adults are often reported as associated with declined health and poor recovery from treatment.^36,37

The nomogram developed in our study has potential clinical use and provides important implications for future geriatric care and research. On the one hand, the identification of ED-revisit predictors (age, BMI, frailty and polypharmacy) may remind clinical or public health efforts to concertedly work on minimizing the influence of such risk factors. Taking frailty as an example, the current disease-oriented and episodic models of emergency care are incapable of adequately responding to the complex care needs of frail older patients, it is suggested based on our results to move towards a better identification and management of frailty in community-dwelling older adults as well as to integrate frailty into acute care assessment among older patients to aid clinical decision-making and develop appropriate treatment plans. On the other hand, our prediction tool may facilitate the individualized prediction of ED repeated visits, which allows clinicians to identify a subset of older patients at high risk of emergency resource reliance or at greatest need for targeted interventions and re-think the most appropriate care for this particular population group. This may empower ED professionals and other healthcare providers to deliver evidence-based stratified and personalized acute care, design a coordinated and integrated plan for intensive long-term follow up post discharge, and develop a support system with better-targeted and patient-centered community services, thereby improving older patients’ clinical outcomes and reducing potentially avoidable ED representations.

Limitations

Several limitations should be acknowledged in our study. First, although our nomogram incorporated an extensive range of patient-related predictors, physician-related and medical institution-related risk factors for ED revisits are currently unavailable in the study. Second, training and validation samples were drawn from the same cohort, thus our nomogram should continue to be externally validated and updated as additional data become available. Third, the outcome was any revisit to EDs within 1-year of discharge, but we did not address the appropriateness or features of these re-attendances. Fourth, our work was conducted in two hospital sites with relatively small sample size, which may constrain the generalizability of research findings to a broader population. Last, the nomogram we developed to predict ED revisits has not been subjected to a formal trial of its capacity to decrease ED length of stay or improve health outcomes.

Conclusion

In conclusion, a nomogram incorporating four obtainable variables was constructed to individualize ED revisits risk among older patients in China. Further studies are still warranted to validate this nomogram among larger samples or culturally diverse populations and to evaluate whether nomogram-based personalized interventions can improve treatment efficacy or reduce adverse outcomes. Our nomogram may serve as a clinical decision aid to identify older population with higher risk of ED re-attendances, and when possible, to facilitate implementation of better-targeted acute care during initial ED visits or comprehensive community-based services including primary care and home care.