Comparison of nine coronary risk scores in evaluating patients presenting to hospital with undifferentiated chest pain

Plain language summary

Chest pain accounts for 5%–10% of presentations to emergency departments (EDs). After excluding the small percentage with clear-cut evidence of myocardial infarction or other acute noncardiac pathologies on presentation, identifying the 10%–20% of remaining patients with undifferentiated chest pain who have ischemic chest pain due to coronary heart disease (CHD) constitutes a clinical challenge. Risk factors, clinical findings, initial electrocardiographs (ECGs), and clinical judgment are insufficiently sensitive to exclude acute myocardial ischemia with the level of confidence most clinicians desire. Tools are needed that accurately identify patients at intermediate to high risk of CHD who warrant further in-patient evaluation, and those at low risk who can be quickly and safely discharged. To date, more than 12 risk stratification scores for undifferentiated chest pain exist, but which performs best in predicting risk of CHD remains unclear. Dr Wamala’s team for the first time compared the performance of nine different risk scores within the same population of patients presenting with undifferentiated chest pain to the ED of a tertiary hospital. They found that, in distinguishing low-risk patients from high-risk patients, the North American Chest Pain Rule performed best followed closely by the History, ECG, Age, Risk Factors, and Troponin score and Thrombolysis in Myocardial Infarction score, with the Accelerated Diagnostic Protocol to Assess Patients with Chest Pain Symptoms Using Contemporary Troponins score performing worst. The remaining five scores demonstrated intermediate performance. These results have practical implications for clinicians involved in caring for patients presenting to ED with undifferentiated chest pain.

Introduction

In Australia 500,000 patients present with chest pain to emergency departments (EDs) annually, comprising 5%–10% of ED presentations.¹ While more than 80% of cases do not have acute coronary syndrome (ACS),¹ missing this diagnosis has major morbidity and mortality implications. Estimates of ACS risk based on risk factors, clinical findings, initial electrocardiographs (ECGs), and clinical judgment are insufficiently sensitive to exclude ACS^2,3 with the level of confidence most ED physicians desire.⁴

Accordingly, guidelines recommend risk stratification and investigations for ruling out ACS, but these can incur lengthy ED stays, in-patient admissions, and downstream testing with low diagnostic yield,⁵ all at substantial cost.¹ Tools are needed that accurately identify patients at intermediate to high risk of CHD requiring further in-patient evaluation, and those at low risk who can be quickly and safely discharged.

To date, more than 12 risk stratification scores for chest pain exist,⁶ but which performs best in predicting risk of CHD among patients in whom clear-cut noncardiac diagnoses have been excluded remains unclear. Scores derived from high-risk populations with probable or definite myocardial infarction (MI) or ACS at ED presentation have limited generalizability to undifferentiated patients at lower risk.⁷ Differences between studies that assess single scores relating to definitions of clinical variables, ECG changes, and tests, patient populations, and reference outcomes render assessment of relative performance difficult. Systematic reviews comparing different scores rely on data from reports that usually focus on a single score.⁸ Studies comparing performance of different scores in the same population are few,^9–13 and, to our knowledge, none have compared more than three scores within one patient population.

In this study, we compared the performance of nine risks scores within the same population of patients presenting with undifferentiated chest pain to the ED of a tertiary hospital in whom clear-cut noncardiac diagnoses were absent.

Methods

Risk scores

The nine risk scores, with full acronyms listed in Box 1, comprised History, ECG, Age, Risk Factors, and Troponin (HEART),¹⁴ Accelerated Diagnostic Protocol to Assess Patients with Chest Pain Symptoms Using Contemporary Troponins (ADAPT),¹⁵ Emergency Department Assessment of Chest Pain Score (EDACS),¹⁶ The North American Chest Pain Rule (NACPR),¹⁷ Thrombolysis in Myocardial Infarction (TIMI),¹⁸ modified TIMI (m.TIMI),¹⁹ Global Registry of Acute Coronary Events (GRACE),²⁰ Platelet Glycoprotein IIb/IIIa in Unstable Angina: Receptor Suppression Using Integrilin Therapy (PURSUIT),²¹ and Florence Prediction Rule (FPR).²² Supplementary material S1 describes each score in detail based on original studies, and we used the cut-points to distinguish low-risk patients from high-risk patients, where either intermediate or high risk in the original studies was classified here as high risk. Each score was retrospectively applied to each patient in our study. The scores were chosen on the basis of common usage and citation, derivation, and validation in contemporary cohort studies or randomized trials, and ease of application.

Box 1 Risk stratification scores

Assessing score performance

The reference outcome for assessing score performance was the final discharge diagnosis made by the treating clinician following in-patient observation and investigations. These diagnoses were dichotomized as noncardiac chest pain (NCCP) or ischaemic chest pain (ICP), the latter comprising angina syndromes, MI, or ACS. Where investigations were performed, diagnostic test criteria used to confirm final diagnosis of ICP are listed in Supplementary material S2.

Participants and setting

This was a retrospective study of consecutive adult (≥18 years of age) patients presenting with chest pain to the ED at Princess Alexandra Hospital (PAH) in Brisbane, Australia, between June 1 and December 31, 2015 and admitted to the Medical Assessment and Planning Unit for evaluation for CHD. PAH is a 640-bed tertiary hospital with an ED that annually sees 5,000 patients with chest pain of whom around 50% are discharged from ED with noncardiac diagnoses, and the remainder admitted for observation and further investigation as deemed necessary.

Potentially eligible patients were ascertained from clinical coding lists from the hospital Health Information Management Systems with the following International Classification of Diseases – 10th revision – Australian Modification (ICD-10-AM) codes: stable angina (code 120.8), unstable angina (120.0), ST elevation myocardial infarction (STEMI) (121.3), non-STEMI (121.4), ACS (120–125), pericarditis (130–131), precordial pain (R07.2), and chest pain (R07.3, R07.4). Hospital records were retrieved and patients presenting with chest pain identified. Those with ECG changes and/or troponin assay results diagnostic of ICP at ED presentation were excluded. Patients whose records contained sufficient clinical information to indicate clear-cut alternative diagnoses of noncardiac causes of pain (such as pulmonary thromboembolism, gastroesophageal reflux disease, costochondritis, pleurisy, or musculoskeletal chest pain), as adjudicated by consensus of two authors (HW, LA), were also excluded.

Data collection

Patient data were abstracted from electronic medical records for 1) all variables relevant to each risk score (Supplementary material S1); 2) patient characteristics and vascular risk factors; and 3) final discharge diagnosis, categorized as NCCP or ICP.

Statistical analysis

Means (SD) and proportions were compared using t-tests and chi-squared tests, respectively, with P<0.05 denoting statistical significance. Score accuracy in identifying patients at low risk (ie, discharge diagnosis of NCCP) was expressed as odds ratio (OR) with 95% CI in reference to patients diagnosed at high risk (ie, discharge diagnosis of ICP). A lower OR for ICP equated to a greater likelihood of low risk. Ability to discriminate between low and high risk for ICP was assessed using area under the receiver–operator characteristic (ROC) curves (AUCs) calculated for each score. Sensitivity, specificity, positive and negative likelihood ratios (LR + and LR–, respectively), and positive and negative predictive values (PPV and NPV, respectively) for ICP were calculated for each score. Analyses used SPSS version 22 (IBM SPSS, Chicago, IL, USA). As this was a retrospective analysis of deidentified routinely collected data on completed episodes of care with no need for patient contact or consent, and as the study was conducted as part of a quality improvement program for chest pain presentations, the need for ethical approval was waivered by the Director of Clinical Governance as per Metro South Clinical Governance Policy PL 2015/41, V1. The study was carried out in accordance with the principles of the Declaration of Helsinki. There was no funding source for the study.

Results

Patient characteristics

The final cohort comprised 401 patients of mean (±SD) age 61.5 (±14.3) years: 60.3% were men, 41.9% had pre-existing CHD (Table 1), and most (84.5%) were admitted under cardiologists. During admission, 99 patients (24.7%) underwent investigations for myocardial ischemia, of which 21 investigations (5.2% of total cohort) were positive for CHD. At discharge, 278 (69.3 %) had a final diagnosis of NCCP and 123 (30.7%) had ICP, including 23 (5.7%) with troponin-positive ACS. Three (0.7%) patients died in hospital, all from noncardiac causes (two from aspiration pneumonia and one from urosepsis). Compared to patients with NCCP, patients with ICP were older and had higher prevalence of hypertension, dyslipidaemia, and chronic kidney disease. Patients with ICP were also more likely to undergo invasive coronary angiography but less likely to undergo noninvasive tests for CHD compared to patients with NCCP.

Table 1 Patient characteristics Notes: aComparing NCCP and ICP groups. bNumber (%) of patients undergoing investigation – number of patients with positive result for CHD. Abbreviations: CHD, coronary heart disease; CTCA, computerized tomography of coronary arteries; ECG, electrocardiograph; ICA, invasive coronary angiography; ICP, ischemic chest pain; MPS, myocardial perfusion scan; NCCP, noncardiac chest pain; ND, not done; PVD, peripheral vascular disease.

Performance of risk scores

Values for each risk score among patients with NCCP and ICP based on final discharge diagnosis, and the proportions of patients classified as low or high risk according to the cut-points for each score are listed in Table 2. The proportion of patients classified as low risk was highest for GRACE (47.4%) and lowest for ADAPT (1.7%).

Table 2 Values of risk scores according to high or low risk as defined by final discharge diagnosis or by risk score cut-point Notes: aLow risk defined as final discharge diagnosis of NCCP (n=124); high risk defined as final discharge diagnosis of ICP (n=277). bCut-points for each score used to categorize low- vs high-risk patients are listed in Supplementary material S1. Abbreviations: ADAPT, Accelerated Diagnostic Protocol to Assess Patients with Chest Pain Symptoms Using Contemporary Troponins; EDACS, Emergency Department Assessment of Chest Pain Score; HEART, History, ECG, Age, Risk Factors, and Troponin; FPR, Florence Prediction Rule; GRACE, Global Registry of Acute Coronary Events; m.TIMI, modified TIMI; NACPR, The North American Chest Pain Rule; NCCP, noncardiac chest pain; PURSUIT, Platelet Glycoprotein IIb/IIIa in Unstable Angina: Receptor Suppression Using Integrilin Therapy; TIMI, Thrombolysis in Myocardial Infarction.

Scores with lowest ORs for ICP, indicating low risk (Table 3) were NACPR (OR=0.35; 95% CI=0.27–0.46), HEART (OR=0.43; 95% CI=0.35–0.52), and TIMI (OR=0.49; 95% CI=0.41–0.58). Scores with highest ORs, indicating high risk, were PURSUIT (OR=0.80; 95% CI=0.74–0.85), EDACS (OR=0.87; 95% CI=0.84–0.91), and GRACE (OR=0.98; 95% CI=0.97–0.98). Ability to discriminate between low and high risk across all values for each risk score, as measured by AUC, is demonstrated in Figure 1 and Table 4. AUC was highest for HEART (0.82; 95% CI=0.78–0.86) and lowest for ADAPT (0.63; 95% CI=0.58–0.69).

Table 3 Predictive accuracy for identifying low-risk vs high-risk patients Note: A lower OR means a lower likelihood of ICP as discharge diagnosis and therefore higher likelihood of NCCP, which classifies patients as low risk. Abbreviations: ADAPT, Accelerated Diagnostic Protocol to Assess Patients with Chest Pain Symptoms Using Contemporary Troponins; EDACS, Emergency Department Assessment of Chest Pain Score; HEART, History, ECG, Age, Risk Factors, and Troponin; FPR, Florence Prediction Rule; GRACE, Global Registry of Acute Coronary Events; ICP, ischemic chest pain; m.TIMI, modified TIMI; NACPR, The North American Chest Pain Rule; NCCP, noncardiac chest pain; OR, odds ratio; PURSUIT, Platelet Glycoprotein IIb/IIIa in Unstable Angina: Receptor Suppression Using Integrilin Therapy; TIMI, Thrombolysis in Myocardial Infarction.

Figure 1 ROC curves for risk scores. Abbreviations: ADAPT, Accelerated Diagnostic Protocol to Assess Patients with Chest Pain Symptoms Using Contemporary Troponins; EDACS, Emergency Department Assessment of Chest Pain Score; HEART, History, ECG, Age, Risk Factors, and Troponin; FPR, Florence Prediction Rule; GRACE, Global Registry of Acute Coronary Events; m.TIMI, modified TIMI; NACPR, The North American Chest Pain Rule; PURSUIT, Platelet Glycoprotein IIb/IIIa in Unstable Angina: Receptor Suppression Using Integrilin Therapy; ROC, receiver–operator characteristic; TIMI, Thrombolysis in Myocardial Infarction.

Table 4 AUC for different risk scores Abbreviations: ADAPT, Accelerated Diagnostic Protocol to Assess Patients with Chest Pain Symptoms Using Contemporary Troponins; AUC, area under the curve; EDACS, Emergency Department Assessment of Chest Pain Score; HEART, History, ECG, Age, Risk Factors, and Troponin; FPR, Florence Prediction Rule; GRACE, Global Registry of Acute Coronary Events; m.TIMI, modified TIMI; NACPR, The North American Chest Pain Rule; PURSUIT, Platelet Glycoprotein IIb/IIIa in Unstable Angina: Receptor Suppression Using Integrilin Therapy; TIMI, Thrombolysis in Myocardial Infarction.

The ADAPT score had 100% sensitivity for identifying patients with ICP (Table 5), with LR– =0.00 and NPV=0.0% (miss rate for ICP =0%), but classified only 1.7% of patients as being low risk (Table 2). NACPR had LR– =0.05, with NPV=98%, equal to 2% miss rate for ICP, and classified 10.2% of patients as low risk. HEART had LR– =0.14, with NPV=94% equal to miss rate of 6%, and classified 32.4% as being low risk. Corresponding values for LR–, NPV, miss rate, and proportions of patients classified as low risk were as follows: GRACE: 0.56, 79%, 21%, 47.4%; PURSUIT: 0.51, 80%, 20%, 44.6%; and EDACS: 0.47, 82%, 18%, 43.6%. Highest LR+ and PPV for ICP were 2.16 and 51% for TIMI, 2.06, and 50% for m.TIMI, and 1.80 and 46% for HEART.

Table 5 Sensitivity, specificity, likelihood ratios, and predictive values for different risk scores Note: All stated as percentages with 95% CI in parentheses. Abbreviations: ADAPT, Accelerated Diagnostic Protocol to Assess Patients with Chest Pain Symptoms Using Contemporary Troponins; EDACS, Emergency Department Assessment of Chest Pain Score; HEART, History, ECG, Age, Risk Factors, and Troponin; FPR, Florence Prediction Rule; GRACE, Global Registry of Acute Coronary Events; LR+, positive likelihood ratio; LR−, negative likelihood ratio; m.TIMI, modified TIMI; NACPR, The North American Chest Pain Rule; NPV, negative predictive value; PPV, positive predictive value; PURSUIT, Platelet Glycoprotein IIb/IIIa in Unstable Angina: Receptor Suppression Using Integrilin Therapy; TIMI, Thrombolysis in Myocardial Infarction.

Discussion

To our knowledge, ours is the first study to compare nine risk scores in the same patient population presenting to ED with undifferentiated chest pain and requiring further evaluation for CHD after clear-cut noncardiac diagnoses had been excluded. Almost 70% had discharge diagnosis of NCCP, consistent with other studies.¹ The HEART score classified the highest proportion of patients as being low risk while demonstrating good discriminative ability (AUC >0.80), but with a miss rate for ICP of 6%. In contrast, the NACPR classified 10.2% of patients as low risk with a 2% miss rate, with second best discriminative ability. Scores least able to identify low-risk patients were ADAPT, FPR, and GRACE, with EDACS, TIMI, m.TIMI, and PURSUIT showing intermediate performance. Therefore, the NACPR might be the risk score of choice balancing yield of low-risk patients with acceptable miss rate for ICP.

Study strengths and limitations

Study strengths were evaluation of commonly used risk scores subject to varying levels of validation and using reference outcome of discharge diagnosis of NCCP vs ICP, an outcome, which is most salient to clinicians in distinguishing low-risk patients from high-risk patients in whom clear-cut noncardiac diagnoses have been excluded.

Study limitations comprised exclusion of certain risk tools more than a decade old, with limited validation, or relatively cumbersome to use, such as the 2006 Heart Foundation of Australia/Cardiac Society of Australia and New Zealand (NHFA/CSANZ) guidelines,²³ Goldman score,²⁴ and Vancouver chest pain rule.²⁵ The patient sample was relatively small, resulting in wide CIs for some measures. Clinical assessments of subjective variables such as chest pain characteristics, as recorded in medical charts, may not be accurate, thus limiting reliability of calculations of risks scores such as HEART that incorporate such variables. As not all patients underwent formal provocative testing or coronary artery imaging and postdischarge follow-up for cardiac events was not possible, the accuracy of clinician-mediated diagnoses of NCCP or ICP at discharge may be challenged, although the vast majority of diagnoses were made by experienced cardiologists. Moreover, only one in four patients underwent testing, with equal proportions in NCCP and ICP groups (25% vs 23%), with very low positive yield (5 of 71; 7.0%) among NCCP patients, but much higher yield (16 of 28; 57.1%, mostly coronary angiography, 13 of 19) among ICP patients. We suspect that tests in both groups were not done for diagnostic reasons, rather in the former group to reassure clinicians and patients and, in the latter group, to quantify ischemic burden and assess eligibility for coronary revascularization. Ours was not a prospective study of consecutive patients presenting to ED with chest pain, but instead a retrospective study of discharge-coded patients in whom we excluded those with noncardiac diagnoses clearly evident at ED presentation. As we dichotomized patients as either low (NCCP) or high (ICP) risk based on single cut-points, low-, intermediate-, or high-risk strata based on multilevel score cut-points were not investigated. Finally, as a single tertiary site study, our results may not generalize to other hospital settings.

Comparisons with other studies

While our study suggests the HEART score is superior to other scores in identifying the highest proportion of low-risk patients, the miss rate for a discharge diagnosis of ICP was 6%. We could not find any study that compared miss rates for risk scores using discharge diagnosis of ICP as the outcome measure. Hence comparisons were limited to studies that used adverse cardiac events following discharge, in which miss rates overall were much lower. In a prospective analysis of 640 patients of whom 40% had low-risk HEART scores ≤3, miss rate for ACS at 6 weeks was 1.6%.¹¹ In a retrospective analysis of 604 patients, the miss rate for 30-day major adverse cardiac events (MACE) among patients with HEART score ≤3 was 2%.¹² In a prospective evaluation of 12,128 patients involved in a quality improvement program across 14 hospitals within an integrated US health system, the miss rate for HEART score of death or MI at 30 days among low-risk patients (57% of the cohort) was reported as 0.09%.¹³ These lower miss rates compared to the current study probably reflect selection of patients with different risk levels (all our patients were admitted for further evaluation reflecting clinician concern of higher risk), different intensity of testing for ICP, and different outcome measures (higher incidence of discharge diagnosis of ICP in our study, which included all patients with equivocally positive tests for ischemia vs lower incidence of confirmed cardiac events following discharge in other studies).

The ideal miss rate of MACE at 30 days for chest pain risk scores is controversial. Surveys of emergency physicians suggest most want it to be less than 1%,⁴ while formal studies of evaluation protocols that use different testing thresholds suggest a range of 0.5%–3.0%, with an average estimate of 2.0%, which balances miss rates with rates of false-positive testing.²⁶

In studies reporting comparisons of multiple risk scores, a prospective study using HEART, GRACE, and TIMI categorized 986 patients as low risk of 30-day MACE (HEART ≤3, GRACE ≤108, and TIMI ≤1) and found all tools scored most patients as low risk, more so with HEART (85%) than GRACE (81%) and TIMI (80%).²⁷ A retrospective study calculated HEART, TIMI, and GRACE scores on 604 patients in a single site and reported c-statistics (equivalent to AUC) for 30-day MACE rates of 0.78 (95% CI=0.74–0.81), 0.65 (95% CI=0.60–0.69), and 0.62 (95%=CI 0.58–0.67), respectively.¹² These were all somewhat lower than in our study, possibly due to selection of only those patients in the most acute ED triage category with likely higher risk of CHD. Miss rates for 30-day MACE were 2% for HEART score ≤3, compared to 9% for TIMI score=0, and 15% for GRACE score ≤75. Another study compared HEART, TIMI, and GRACE among 1,748 patients presenting to nine Dutch hospitals, with primary outcome of MACE at 6 weeks.¹⁴ The AUCs were 0.86 (95% CI=0.84%–0.88%) for HEART, 0.80 (95% CI=0.78%–0.83%) for TIMI, and 0.73 (95% CI=0.70%–0.76%) for GRACE. Applying a sensitivity between 95% and 98%, HEART score identified 21.8% patients as being low risk compared to none with TIMI and 13.2% with GRACE.

Systematic reviews comparing different risk scores using data from single risk score studies conducted in different sites show similar results. In one review of prospective studies, the HEART score compared to TIMI, GRACE, and NHFA/CSANZ guidelines had highest predictive accuracy for diagnosis of ACS with LR of 13 (95% CI=7.0–24) for high-risk scores (7–10) and LR of 0.20 (95% CI=0.13–0.30) for low-risk scores (0–3).⁹ Another review comparing HEART, EDACS, NACPR, TIMI, and GRACE involving patients presenting to rural EDs suggested EDACS performed best,²⁸ although this finding was heavily influenced by results of a single study.¹⁵

In comparing risk scores developed from populations with chest pain irrespective of final diagnosis (HEART, NACPR, ADAPT, FPR, EDACS) with those developed in populations with confirmed ACS at discharge (GRACE, TIMI, m.TIMI, PURSUIT), our results confirm that the former, in particular HEART and NACPR, demonstrated superior discriminative performance. Other studies also rate TIMI,²⁴ modified TIMI,^10,25 and GRACE^10,28 as inferior in stratifying risk.

Overall, our study and others suggest HEART outperforms GRACE and TIMI in identifying larger numbers of lower risk patients. While the miss rate for HEART of a discharge diagnosis of ICP of 6% seen in our study is potentially unacceptable to clinicians, the much lower incidence of discrete ACS or MACE events reported in other studies is reassuring. In any event, no risk score can accommodate all factors potentially relevant to decision making in individual patients. They aim to assist in risk stratification and provide pretest probabilities in regards to further investigations. Their results are intended to guide, not usurp, clinical judgment at the time patients with undifferentiated chest pain first present to ED.

Conclusion

Our evaluation of nine different risk scores among patients presenting to hospital with undifferentiated chest pain indicates the NACPR rates first and the HEART score second in identifying sizeable proportions of patients at low risk of ICP with miss rates of 6% or less. Future research should evaluate effectiveness of assessment pathways that incorporate better performing risk scores and further refine the scores with the advent of high-sensitivity troponin assays.

Acknowledgments

HW is currently affiliated with the Friendly Society Private Hospital, Bundaberg, QLD, Australia. LA is currently affiliated with Box Hill Hospital, Box Hill, VIC, Australia.

Disclosure

The authors report no conflicts of interest in this work.

References

1.		Cullen L, Greenslade J, Merollini K, et al. Cost and outcomes of assessing patients with chest pain in an Australian emergency department. Med J Aust. 2015;202(8):427–432.
2.		Mitchell AM, Garvey JL, Chandra A, Diercks D, Pollack CV, Kline JA. Prospective multicenter study of quantitative pretest probability assessment to exclude acute coronary syndrome for patients evaluated in emergency department chest pain units. Ann Emerg Med. 2006;47(5):447.
3.		Høilund-Carlsen PF, Johansen A, Vach W, Christensen HW, Møldrup M, Haghfelt T. High probability of disease in angina pectoris patients: is clinical estimation reliable? Can J Cardiol. 2007;23(8):641–647.
4.		Than M, Herbert M, Flaws D, et al. What is an acceptable risk of major adverse cardiac event in chest pain patients soon after discharge from the Emergency Department? A clinical survey. Int J Cardiol. 2013;166(3):752–754.
5.		Hermann LK, Newman DH, Pleasant WA, et al. Yield of routine provocative cardiac testing among patients in an emergency department-based chest pain unit. JAMA Intern Med. 2013;173(12):1128–1133.
6.		Alley W, Mahler SA. Clinical decision aids for chest pain in the emergency department: identifying low-risk patients. Open Access Emerg Med. 2015;7:85–92.
7.		Christenson J, Innes G, Mcknight D, et al. Safety and efficiency of emergency department assessment of chest discomfort. CMAJ. 2004;170(12):1803–1807.
8.		Fanaroff AC, Rymer JA, Goldstein SA, Simel DL, Newby LK. Does this patient with chest pain have acute coronary syndrome? The rational clinical examination systematic review. JAMA. 2015;314(18):1955–1965.
9.		Cullen L, Greenslade J, Hammett CJ, et al. Comparison of three risk stratification rules for predicting patients with acute coronary syndrome presenting to an Australian emergency department. Heart Lung Circ. 2013;22(10):844–851.
10.		Nieuwets A, Poldervaart JM, Reitsma JB, et al. Medical consumption compared for TIMI and HEART score in chest pain patients at the emergency department: a retrospective cost analysis. BMJ Open. 2016;6(6):e010694.
11.		Poldervaart JM, Langedijk M, Backus BE, et al. Comparison of the GRACE, HEART and TIMI score to predict major adverse cardiac events in chest pain patients at the emergency department. Int J Cardiol. 2017;227:656–661.
12.		Sakamoto JT, Liu N, Koh ZX, et al. Comparing HEART, TIMI, and GRACE scores for prediction of 30-day major adverse cardiac events in high acuity chest pain patients in the emergency department. Int J Cardiol. 2016;221:759–764.
13.		Stark AL, Broder B, Sun BC. Improving emergency department care for low-risk chest pain. NEJM Catal 2018; 2018pii. Available from: https://catalyst.nejm.org/ed-acute-coronary-syndrome-heart-score/. Accessed April 18, 2018.
14.		Six AJ, Backus BE, Kelder JC. Chest pain in the emergency room: value of the HEART score. Neth Heart J. 2008;16(6):191–196.
15.		Than M, Cullen L, Aldous S, et al. 2-Hour accelerated diagnostic protocol to assess patients with chest pain symptoms using contemporary troponins as the only biomarker: the ADAPT trial. J Am Coll Cardiol. 2012;59(23):2091–2098.
16.		Than M, Flaws D, Sanders S, et al. Development and validation of the emergency department assessment of chest pain score and 2 h accelerated diagnostic protocol. Emerg Med Australas. 2014;26(1):34–44.
17.		Hess EP, Brison RJ, Perry JJ, et al. Development of a clinical prediction rule for 30-day cardiac events in emergency department patients with chest pain and possible acute coronary syndrome. Ann Emerg Med. 2012;59(2):115–125.
18.		Pollack CV, Sites FD, Shofer FS, Sease KL, Hollander JE. Application of the TIMI risk score for unstable angina and non-ST elevation acute coronary syndrome to an unselected emergency department chest pain population. Acad Emerg Med. 2006;13(1):13–18.
19.		Jaffery Z, Hudson MP, Jacobsen G, Nowak R, Mccord J. Modified thrombolysis in myocardial infarction (TIMI) risk score to risk stratify patients in the emergency department with possible acute coronary syndrome. J Thromb Thrombolysis. 2007;24(2):137–144.
20.		Granger CB, Goldberg RJ, Dabbous OH, et al.; for the Global Registry of Acute Coronary Events Investigators. Predictors of hospital mortality in the global registry of acute coronary events. Arch Intern Med. 2003;163:2345–2353.
21.		Boersma E, Pieper KS, Steyerberg EW, et al.; for the PURSUIT investigators. Predictors of outcome in patients with acute coronary syndromes without persistent ST-segment elevation. Results from an international trial of 9461 patients. Circulation. 2000;101:2557–2567.
22.		Conti A, Poggioni C, Viviani G, et al. Risk scores prognostic implementation in patients with chest pain and nondiagnostic electrocardiograms. Am J Emerg Med. 2012;30(9):1719–1728.
23.		Aroney C, Aylward P, Kelly AM. National Heart Foundation of Australia and Cardiac Society of Australia and New Zealand Guidelines for the Management of Acute Coronary Syndromes. Med J Aust. 2006;184:S1–S32.
24.		Goldman L, Cook EF, Johnson PA, et al. Prediction of the need for intensive care in patients who come to emergency departments with acute chest pain. N Engl J Med. 1996;334(23):1498–1504.
25.		Scheuermeyer FX, Wong H, Yu E, et al. Development and validation of a prediction rule for early discharge of low-risk emergency department patients with potential ischemic chest pain. CJEM. 2014;16(2):106–119.
26.		Kline JA, Johnson CL, Pollack CV, et al. Pretest probability assessment derived from attribute matching. BMC Med Inform Decis Mak. 2005;5:26.
27.		Wang H, Watson K, Robinson RD, et al. Chest pain risk scores can reduce emergent cardiac imaging test needs with low major adverse cardiac events occurrence in an emergency department observation unit. Crit Pathw Cardiol. 2016;15(4):145–151.
28.		Roche T, Jennings N, Clifford S, et al. Review article: Diagnostic accuracy of risk stratification tools for patients with chest pain in the rural emergency department: A systematic review. Emerg Med Australas. 2016;28(5):511–524.
29.		Holly J, Fuller M, Hamilton D, et al. Prospective evaluation of the use of the thrombolysis in myocardial infarction score as a risk stratification tool for chest pain patients admitted to an ED observation unit. Am J Emerg Med. 2013;31(1):185–189.
30.		Macdonald SP, Nagree Y, Fatovich DM, Brown SG. Modified TIMI risk score cannot be used to identify low-risk chest pain in the emergency department: a multicentre validation study. Emerg Med J. 2014;31(4):281–285.
31.		Lyon R, Morris AC, Caesar D, Gray S, Gray A. Chest pain presenting to the emergency department – to stratify risk with GRACE or TIMI? Resuscitation. 2007;74(1):90–93.