Prediction of critical illness in elderly outpatients using elder risk assessment: a population-based study

Introduction

Frequent hospitalizations and burdensome care transitions are common in the last 3 months of a frail elderly patient’s life.^1,2 In a large study of Medicare beneficiaries, intensive care unit (ICU) use in the last month of life was 26.3%, a significant increase between 2000 and 2009.² There are increasing calls for advanced care planning to help patients and families facing advanced life-limiting illness.^3–5 The American Geriatrics Society has called for a dramatic change in the approach to medical care for the frail elderly patients with multimorbidity, emphasizing that a discussion of goals of care and individualized prognosis should ideally precede that of treatment option.⁶ Patients who have goals of care discussions with their physicians indicate a better understanding and comfort level afterward, and families report less bereavement-related stress.⁷ Turning points in care such as changes in symptoms, new comorbidities, and indications for new interventions open opportunities for goals of care discussions.⁸ However, many elderly patients are doing well until an acute unexpected illness precipitates sudden decline. Being able to predict such risk can be helpful to encourage discussions on goals of care.

The elder risk assessment (ERA) has been shown to predict hospital stay and emergency room visits⁹ as well as mortality and nursing home (NH) placement.¹⁰ An equally important outcome is critical illness defined by Seymour et al¹¹ as the delivery of mechanical ventilation (MV), severe sepsis, or death. It remains unclear how well the ERA predicts critical illness. To answer this question, we performed a retrospective cohort analysis of a population of people using ERA and evaluated the subsequent admission to the ICU for MV or sepsis or death within 2 years of the initial outpatient visit. Our aim was to determine the relationship between high-risk people (as determined by ERA) and critical illness.

Methods

Study design

This was a population-based historical cohort study conducted in elderly patients (age >65 years) identified at the time of primary care visit in Rochester, MN, USA. The study was approved by the Mayo Clinic Institutional Review Board.

Setting

The study was conducted at an academic medical center in Rochester, MN, USA. All individuals were impaneled within the primary care internal medicine practice and had a primary care physician. The initial cohort used to develop the ERA score was designed for January 1, 2005, and had incident outcomes for 2 years ending in December 31, 2006. Details of this study have been previously published.⁹

Participants

All participants in the study were at the age of 65 years or older. Those individuals who did not give consent for medical record review were excluded from the study, according to Minnesota state statue with regard to medical record use and privacy (Minnesota Department of Educations, 2003).

Demographic variables were electronically abstracted from the electronic medical record (EMR) and the administrative databases of the health records system of Mayo Clinic. Those included date of birth, sex, race, marital status, prior hospital admissions, number of days hospitalized in the prior 2 years, and NH residence.

Predictors

The primary predictor variable was the ERA index, an administrative risk index that uses data from the EMR to identify those patients at the highest risk of adverse outcomes. This risk stratification is made available at the point of care, which overcomes the limitations of other prognostic indices, many of which require data collection that is not part of the routine clinical visit.¹² The ERA uses administrative data to help calculate a score on all individuals over the age of 60 years. The ERA scoring mechanism weighs age, previous hospital stay, and comorbid illnesses within the mechanism. The scores range from −1 to 43 with scores >16 determining high risk for hospitalization. The ERA score has also been shown to predict mortality and NH placement.¹⁰ The demographic predictor variables collected included date of birth, sex, marital status, race, and the number of hospital admission days in the prior 2 years (January 1, 2003, to December 31, 2004). Age was stratified into categories of 65–74 years, 75–84 years, and >85 years. Comorbid medical illnesses included the presence or history of diabetes mellitus, coronary artery disease, congestive heart failure, stroke, chronic obstructive pulmonary disease, history of cancer, history of hip fracture, and dementia. History of cancer excluded nonmelanoma skin cancers. Diagnoses were identified using International Classification of Diseases Ninth Revision (ICD-9) billing codes entered by physicians during both inpatient and outpatient encounters. The Charlson Comorbidity Index (CCI) in this study used data based upon ICD-9 codes taken from the billing records. Administrative CCI methodologies have been proven to predict 1-year mortality.¹³

Outcomes

The primary outcome was critical illness defined as MV use, death, or sepsis as previously done by Seymour et al¹¹ within 2 years of initial visit. MV use and sepsis were derived from billing information from the ICUs at St Mary’s Hospital and Rochester Methodist Hospital, the two primary hospitals of Mayo Clinic, Rochester, MN, USA. Mortality information was collected from the EMR and is updated within the EMR when death occurs within the hospital or health care system (NH). The record is also updated with local obituaries and death announcements.

Statistical analysis

The comparisons of the descriptive variables used univariate logistic regression comparing participants who did and did not have ICU outcomes. Because the ERA takes into account age, comorbid status, and utilization information, we did not adjust for other significant variables in this analysis. Dividing the ERA and the CCI into groups allowed the comparison of the lowest score as the baseline compared to higher scores. This comparison allowed the creation of odds ratios (ORs) for the higher scoring groups compared to the lowest scoring group. Confidence intervals (CIs) were set at 95%, and P-values <0.05 were considered significant. All analyses were completed using SAS 9.01 (SAS Institute Inc., Cary, NC, USA). The discrimination of the model was assessed using area under the receiver operating characteristic curve (AUC).

Results

In the initial cohort, there were 13,457 patients who were eligible. Of those, 12,650 had given consent for medical record review.⁹ Within this cohort, claims and full information on ICU utilization were available for 9,872 participants. The demographics of the cohort show an older cohort with a mean age of 75.8 years (SD ±7.6 years), and 58% were female, 94% were Caucasian, 62% were married, and 13% were living in NHs. The mean CCI was 3.34 (±3.29). Eleven percent of the population had ERA score ≥16. Demographics of the cohort are presented in Table 1. In general, age, previous hospital utilization, and comorbid health issues were significant predictors for all three outcomes.

Table 1 Overall demographics and critical illness status Note: Data presented as n (%) or mean ± standard deviation. Abbreviations: MV, mechanical ventilation; ERA, elder risk assessment; CCI, Charlson Comorbidity Index; MI, myocardial infarction; CHF, congestive heart failure; PVD, peripheral vascular disease; CVD, cardiovascular disease.

Critical illness

The ERA and the Charlson index and NH use were predictive for critical illness. The ORs for critical illness outcomes are listed in Table 2. Of the participants admitted to the ICU with critical illness, 290 (33%) had an ERA score >16 as compared to only 13 (1%) with a score of ≤1. Patients with an ERA score of 16 or higher had an OR of 6.35 (95% CI 3.51–11.48) as compared to the lowest scoring group. NH use at any point in the previous 2 years was also a predictor of critical illness, OR 2.02 (95% CI 1.70–2.40), P<0.0001. The CCI performed better than the ERA score with an OR of 6.17 (95% CI 3.93–9.70) for a score between 5 and 8 and 10.74 (95% CI 6.78–17.03) for a score of >9.

Table 2 Univariate logistic regression Abbreviations: MV, mechanical ventilation; CI, confidence interval; ERA, elder risk assessment; CCI, Charlson Comorbidity Index; NH, nursing home.

The receiver operating characteristic curve for the ERA score associated with the main combined outcome is shown in Figure 1. The AUC for ERA score predicting the primary outcome of critical illness was 0.75. For MV use only, the AUC was 0.70. For mortality only, the AUC was 0.77.

Figure 1 Discrimination of the ERA score in predicting critical illness – receiver operating characteristic curve. Note: The ERA score showed good discrimination for predicting critical illness with an area under the curve of 0.75. Abbreviation: ERA, elder risk assessment.

MV use

We also looked separately at MV use as an important discussion point included in advance care planning. Thirty-two percent of the participants using MV had ERA scores >16 compared to only 3% with scores ≤1. The ERA score significantly predicted ventilator use with an OR of 2.49 (95% CI 1.09–5.69), P=0.03, comparing the highest to the lowest group. The CCI also predicted ventilator use with an OR of 22.35 (95% CI 7.79–64.12) comparing the highest to the lowest scoring group.

Discussion

In this retrospective cohort study, we found that higher ERA scores resulted in higher odds of critical illness. The ERA has previously been shown to predict risk for hospital admission or emergency room visits⁹ as well as mortality and NH admission.¹⁰ Thus, the findings that higher ERA scores also predict critical illness, defined as death, MV, or sepsis, are consistent with the general premise that this simple administrative index can help predict patient-important outcomes. The CCI performed better than the ERA score in this study. This is in line with the previous attempts at trying to improve upon this important research tool. However, the CCI is not very practical to use at the bedside with patient because of the many variables included in the score. The ERA score achieves good discrimination with a much simpler index. At our institution, we have already made the ERA score available electronically at the point of care to facilitate efficient enrollment of patients into our clinical care transitions and palliative homebound programs aimed at the prevention of critical illness and rehospitalizations in our high-risk patients.^14,15

Seymour et al¹¹ studied the prediction of critical illness, defined as the delivery of MV, severe sepsis, or death in nontrauma adult patients at the emergency care setting. Older age, lower systolic blood pressure, abnormal respiratory rate, lower Glasgow Coma Score, lower pulse oximetry, and NH residency were predictors of critical illness. Similar to our study, out-of-hospital variables predicted patient-important outcomes and might improve triage decisions in the emergency department. Our study is designed to identify patients in the primary care setting prior to the onset of acute illness to allow ample time for discussions of goals of care. Another difference between the Seymour prediction score and ours is that it is dependent on the collection of clinical data points and the manual scoring at the point of care, while in our system, the ERA score automatically pulls administrative data and makes a risk score available at the point of care. This minimizes the burden on the clinician and should increase its use.

Olmsted County has a relatively low rate of ICU utilization during a terminal hospital admission¹⁶ when compared to national rates.¹⁷ Still a significant proportion of the total within-age group ICU days are used by elderly decedents in the last year of life ranging from 27.2% in the 65–74-year olds to 68.1% in the >85-year olds. This amounts to one in five ICU days used by elderly decedents in their last year of life.¹⁶ Sepsis and other critical illness states could possibly be prevented by targeting the high-risk elderly patients for priority triage and early intervention for ambulatory care-sensitive conditions. Identifying elderly individuals as having high risk should also serve to prompt discussions of goals of care. The current moral and technological imperatives to treat against all odds are powerful and overshadow patient’s choice.¹⁸ We have argued that this is, to a large extent, based on a narrow interpretation of beneficence as life extension as opposed to a broader view of patient-defined goals and values.¹⁹ Many elderly patients value independence over life extension.²⁰ Thus, clarifying goals of care before the onset of acute illness may avoid burdensome ICU care near the end of life where it may not be consistent with the patient’s values and preferences. Palliative care should be presented as options to all patients with poor prognosis as this has been shown to increase quality of care, lengthen life,²¹ and save money²² – a rare win–win situation.

This study has strengths and limitations that are inherent with a retrospective cohort analysis. The strength lies in the strong population-based data in Olmsted County. The patients were required to be impaneled within the primary care groups at Mayo Clinic. This excludes the large referral population, which otherwise could have biased the data. The population of Olmsted County has similar age- and sex-specific mortality as the entire US population; however, it is less ethnically diverse, wealthier, and more educated.²³ These differences need to be taken into account when evaluating the generalizability of our data. The limitations include obtaining patient information from administrative databases that was recorded for purposes other than the pursuit of our hypothesis. The predictors and outcomes were determined from billing data, which may underestimate secondary diagnoses. However, administrative data such as ICD-9 codes have been found to correlate well with chart diagnoses.²⁴ Second, this is a retrospective cohort analysis, which also risks underreported risk factors and outcomes; however, both the ERA score and the outcomes are based on hospitalization data that are well captured by Mayo administrative data. It is also possible that patients in the cohort could have hospitalizations or chronic diagnoses elsewhere and of which our EMR is therefore unaware. However, most impaneled patients receive both their acute and chronic care from Mayo Clinic. By design, the model only collected retrospective collection of risk factor variables and did not include functional status measures or other patient-provided information, which has been shown to predict outcomes. Functional status data are dependent on patient-provided information or clinician-administered performance testing, which is neither routinely collected nor easily extractible from administrative data.^25–27 The use of a combined end point for our outcome of critical illness can be contested. Few will disagree that sepsis is a critical illness and that patient requiring MV is usually although not invariably critically ill. However, death, while an important outcome, is not always preceded by critical illness, and this definition may miss other critical disease states that were not included in the definition. Because of the inherent limitations of working with secondary data, we chose to go with a definition previously used and published.¹¹ Finally, we did not estimate calibration of prediction tool.

Conclusion

A simple model based on easily obtainable administrative data predicts death and critical illness in the next 2 years in elderly outpatients. This model has already proven valuable at Mayo Clinic to facilitate efficient enrollment of patients into studies aiming to prevent critical illness and increase the rate of advanced care planning.^14,28 External validation is required prior to clinical use at other institutions.

Disclosure

The authors report no conflicts of interest in this work.

---

References

1.		Gozalo P, Teno JM, Mitchell SL, et al. End-of-life transitions among nursing home residents with cognitive issues. N Engl J Med. 2011;365(13):1212–1221.
2.		Teno JM, Gozalo PL, Bynum JP, et al. Change in end-of-life care for medicare beneficiaries: site of death, place of care, and health care transitions in 2000, 2005, and 2009. JAMA. 2013;309(5):470–477.
3.		Kettl P. A piece of my mind. One vote for death panels. JAMA. 2010;303(13):1234–1235.
4.		Vierra M. Death panels. Ann Intern Med. 2012;156(5):394–395.
5.		Kirkpatrick JN, Hauptman PJ, Goodlin SJ. Bundling informed consent and advance care planning in chronic cardiovascular disease: we need to talk. JAMA Intern Med. 2015;175(1):5–6.
6.		American Geriatrics Society Expert Panel on the Care of Older Adults with Multimorbidity. Patient-centered care for older adults with multiple chronic conditions: a stepwise approach from the American Geriatrics Society. J Am Geriatr Soc. 2012;60(10):1957–1968.
7.		Wright AA, Zhang B, Ray A, et al. Associations between end-of-life discussions, patient mental health, medical care near death, and caregiver bereavement adjustment. JAMA. 2008;300(14):1665–1673.
8.		Goodlin SJ, Hauptman PJ, Arnold R, et al. Consensus statement: palliative and supportive care in advanced heart failure. J Card Fail. 2004;10(3):200–209.
9.		Crane SJ, Tung EE, Hanson GJ, Cha S, Chaudhry R, Takahashi PY. Use of an electronic administrative database to identify older community dwelling adults at high-risk for hospitalization or emergency department visits: the elders risk assessment index. BMC Health Serv Res. 2010;10:338.
10.		Takahashi PY, Tung EE, Crane SJ, Chaudhry R, Cha S, Hanson GJ. Use of the elderly risk assessment (ERA) index to predict 2-year mortality and nursing home placement among community dwelling older adults. Arch Gerontol Geriatr. 2012;54(1):34–38.
11.		Seymour CW, Kahn JM, Cooke CR, Watkins TR, Heckbert SR, Rea TD. Prediction of critical illness during out-of-hospital emergency care. JAMA. 2010;304(7):747–754.
12.		Yourman LC, Lee SJ, Schonberg MA, Widera EW, Smith AK. Prognostic indices for older adults: a systematic review. JAMA. 2012;307(2):182–192.
13.		Chaudhry S, Jin L, Meltzer D. Use of a self-report-generated Charlson Comorbidity Index for predicting mortality. Med Care. 2005;43(6):607–615.
14.		Chen CY, Thorsteinsdottir B, Cha SS, et al. Health care outcomes and advance care planning in older adults who receive home-based palliative care: a pilot cohort study. J Palliat Med. 2014;18(1):38–44.
15.		Takahashi PY, Haas LR, Quigg SM, et al. 30-day hospital readmission of older adults using care transitions after hospitalization: a pilot prospective cohort study. Clin Interv Aging. 2013;8:729–736.
16.		Seferian EG, Afessa B. Adult intensive care unit use at the end of life: a population-based study. Mayo Clin Proc. 2006;81(7):896–901.
17.		Angus DC, Barnato AE, Linde-Zwirble WT, et al; Robert Wood Johnson Foundation ICU End-Of-Life Peer Group. Use of intensive care at the end of life in the United States: an epidemiologic study. Crit Care Med. 2004;32(3):638–643.
18.		Kaufman SR, Shim JK, Russ AJ. Old age, life extension, and the character of medical choice. J Gerontol B Psychol Sci Soc Sci. 2006;61(4):S175–S184.
19.		Thorsteinsdottir B, Swetz KM, Tilburt JC. Dialysis in the frail elderly – a current ethical problem, an impending ethical crisis. J Gen Intern Med. 2013;28(11):1511–1516.
20.		Haberle TH, Shinkunas LA, Erekson ZD, Kaldjian LC. Goals of care among hospitalized patients: a validation study. Am J Hosp Palliat Care. 2011;28(5):335–341.
21.		Temel JS, Greer JA, Muzikansky A, et al. Early palliative care for patients with metastatic non-small-cell lung cancer. N Engl J Med. 2010;363(8):733–742.
22.		Taylor DH Jr, Ostermann J, Van Houtven CH, Tulsky JA, Steinhauser K. What length of hospice use maximizes reduction in medical expenditures near death in the US Medicare program? Soc Sci Med. 2007;65(7):1466–1478.
23.		St Sauver JL, Grossardt BR, Leibson CL, Yawn BP, Melton LJ 3rd, Rocca WA. Generalizability of epidemiological findings and public health decisions: an illustration from the Rochester Epidemiology Project. Mayo Clin Proc. 2012;87(2):151–160.
24.		Quan H, Parsons GA, Ghali WA. Validity of information on comorbidity derived rom ICD-9-CCM administrative data. Med Care. 2002;40(8):675–685.
25.		Bogardus ST Jr, Towle V, Williams CS, Desai MM, Inouye SK. What does the medical record reveal about functional status? A comparison of medical record and interview data. J Gen Intern Med. 2001;16(11):728–736.
26.		Fried LP, Guralnik JM. Disability in older adults: evidence regarding significance, etiology, and risk. J Am Geriatr Soc. 1997;45(1):92–100.
27.		Fried LP, Kronmal RA, Newman AB, et al. Risk factors for 5-year mortality in older adults: the Cardiovascular Health Study. JAMA. 1998;279(8):585–592.
28.		Takahashi PY, Hanson GJ, Thorsteinsdottir B, et al. The impact of telemonitoring upon hospice referral in the community: a randomized controlled trial. Clin Interv Aging. 2012;7:445–451.