The Summit Score Stratifies Mortality and Morbidity in Chronic Obstructive Pulmonary Disease

Plain Language Summary

The Summit Score was derived and internally validated to predict mortality in the SUMMIT trial population and externally validated at Intermountain Healthcare. The Summit Score also predicted COPD exacerbations, cardiovascular events, and changes in spirometry findings.

Introduction

Chronic obstructive pulmonary disease (COPD) is a chronic, progressive lung disease often caused by tobacco use, a shared risk factor with cardiovascular disease that leads to a high rate of cardiovascular events and mortality.^1,2 Cardiovascular disease is the second most common cause of death in COPD patients,³ and the most common cause of death in patients with moderate COPD.^4,5 In non-COPD populations, measures of COPD severity such as spirometry parameters are independently predictive of cardiovascular outcomes.^6,7 In COPD patients, mortality and other respiratory outcomes are also predicted by cardiovascular risk factors, including C-reactive protein, troponin, hypertension, and smoking.^1,2,8 Due to its complexity, COPD may require use of risk predictors related to pulmonary and cardiovascular diseases and to systemic health that existing risk stratification tools do not employ.

A pragmatic parsimonious prediction tool, the Intermountain Mortality Risk Score (IMRS) was created using a broad spectrum of physiologic variables [ie, complete blood count (CBC) and basic metabolic profile (BMP) parameters] to predict all-cause mortality in hospitalized patients.⁹ It has been validated in various populations,^9–13 including pulmonary patients¹⁰ and external populations.^9,14,15 Because CBC and BMP parameters can improve and deteriorate over time, IMRS can add unique risk information each time it is calculated.¹¹ It also predicts common events and diagnoses,¹⁶ is used clinically,¹⁷ and modified versions such as the pulmonary IMRS (pIMRS) for lower-risk COPD exist.^10,17-19

Unfortunately, the unique needs of patients with moderate severity COPD and cardiovascular risk factors may not be addressed by IMRS, pIMRS, or COPD scores. Subjects in the Study to Understand Mortality and Morbidity in COPD (SUMMIT) trial were included in the original trial based on the presence of both COPD and cardiovascular risk factors.^5,20 Utilizing SUMMIT trial subjects and Intermountain patients, this study’s aim was to develop and validate a risk score that predicts mortality and other COPD patient outcomes and that can be used to guide treatment decisions in COPD patients.

Methods

Objectives and Populations

The primary objective of this study was to derive and validate a risk score, the Summit Score, for all-cause mortality among individuals with moderate COPD. A secondary objective was to risk stratify SUMMIT trial subjects by the Summit Score to identify one or more subsets in which randomly assigned trial medical therapy was more and less effective for preventing mortality. Other secondary objectives (Supplement) were to evaluate the association of the Summit Score with non-fatal outcomes, to derive and validate the Summit Lab Score, and to validate the associations of IMRS and pIMRS with all-cause mortality in the SUMMIT trial population. The Intermountain Healthcare Institutional Review Board approved this post hoc evaluation of SUMMIT trial data as a minimal risk study,⁵ and approved evaluation of Intermountain data with a waiver of consent as a minimal risk study.

Populations studied here included subjects from the SUMMIT trial (N=16,485) who were individuals ages 40–80 years with COPD who had moderate airflow limitation and a history of or elevated risk of cardiovascular disease.⁵ Moderate COPD in SUMMIT was defined as forced expiratory volume in 1 s (FEV₁) of 50–70% of predicted, FEV₁/forced vital capacity (FVC) ratio ≤0.70, history of smoking ≥10 pack-years, and modified Medical Research Council dyspnea scale ≥2.⁵ Enrollees were randomly assigned 1:1:1:1 to one of four once-daily treatments: an inhaled placebo, vilanterol (VI) 25 μg, fluticasone furoate (FF) 100 μg, or the combination of FF 100 μg/VI 25 μg.⁵ Additional details about the SUMMIT trial were previously published.^5,20 Herein, SUMMIT subjects were randomly divided 50%/50% into two populations using a binomial random number generator powered by a long-period Mersenne Twister. The first half of SUMMIT subjects constituted the derivation population in which the Summit Score was created and the other half was evaluated for internal validation.

Three Intermountain populations were evaluated as external validations of the Summit Score, with the primary validation being among outpatients with diagnosed COPD and elevated cardiovascular risks. Intermountain outpatients with elevated cardiovascular risks (N=9251) were ages 40–80 years with a history of CAD, MI, stroke, peripheral arterial disease, or diabetes, or ages 60–80 years who were receiving treatments for one or more of the following: hypercholesterolemia, hypertension, or vascular disease, approximating the cardiovascular risk definition from SUMMIT.⁵ Outpatients with COPD but no cardiovascular risks (as defined in SUMMIT,[5] which did not include heart failure) and COPD inpatients were also studied (Supplement). All Intermountain outpatients were managed in an outpatient setting at their first COPD encounter between December 2008 and December 2015. The primary ICD-9 diagnostic codes were 490, 491, 492, 493, 494, 495, or 496. ICD-10 was not used since cohort inclusion ended in 2015. For patients with a primary code of asthma, COPD was required as a secondary ICD-9 code. Outpatients were followed through March 2018. Mortality was determined from electronic health records, Utah death certificates, and the Social Security death master file. COPD exacerbation (defined as moderate when treated with antibiotics or systemic corticosteroids or as severe when requiring hospitalization) and cardiovascular endpoints (ie, myocardial infarction, unstable angina, stroke, and mortality) were determined from electronic health record data of Intermountain’s 22 hospitals and >180 clinics that provide health services to about two thirds of the population of Utah and southeast Idaho.

Development of the Summit Risk Scores

Derivation of the Summit Score was performed in half of SUMMIT subjects (n=8181) using multivariable Cox regression analysis evaluating all-cause mortality, the primary SUMMIT trial outcome.^5,20 Variables evaluated in the Cox regression modeling were: age, sex, race, ethnicity, baseline weight, baseline body mass index (BMI), previous treatment for a COPD exacerbation, number of prior hospitalizations for a COPD exacerbation (categorized as 0, 1, or ≥2), number of prior treated or hospitalized COPD exacerbations (categorized as 0, 1, or ≥2), baseline clinical measures [heart rate (HR), systolic blood pressure (SBP), diastolic blood pressure, FEV₁, FVC, FEV₁/FVC ratio], history of smoking, number of pack-years, modified Medical Research Council dyspnea scale score, history of hypertension, history of hypercholesterolemia, diagnosis of diabetes mellitus, previous hospitalization for MI, prior HF diagnosis, previous hospitalization for stroke, history of coronary artery disease (CAD), history of pneumonia, and baseline medications: anti-arrhythmic agents, anti-hypertensives, anti-thrombotics, beta-blockers, calcium channel blockers, diuretics, inhaled corticosteroids, long-acting anticholinergics, long-acting β2 agonists, lipid lowering agents, mucolytics, renin-angiotensin-aldosterone system inhibitors, short-acting anticholinergics, and xanthines. Quantitative variables were categorized into quintiles of similar sample size and age was categorized by decade with ages 40–49 years used as the referent category. Comorbidities and medication variables were categorized as present or absent. Randomized SUMMIT treatments were not considered in the risk score derivation.

Initial analyses used univariable Cox regression to examine each variable above for association with all-cause mortality. Multivariable Cox models used both backward and forward stepwise selection of study covariables. Additional models utilized forced variable entry for those with p<0.10 or confounders (ie, variables causing >10% change in another variable’s β-coefficient). The final risk model entered predictors with p≤0.05 and variables determined in the model building process to be important contributors to risk stratification. Risk score weightings were assigned based on variables’ β-coefficients from the final model. The risk score was composed as the sum of scalar weightings for each category of each contributing variable based on a modification of the procedure used to compute IMRS.⁹ The scalar weighting for each variable was calculated by subtracting 0.15 from the β-coefficient, dividing the result by 0.15, and rounding that value to the nearest integer. Referent categories (e.g., a comorbidity was not present, a medication was not used, or ages 40–49 years) were assigned a scalar value of zero. Baseline characteristics were evaluated between the SUMMIT derivation and validation populations using the chi-square test for discrete data or the T-test for continuous variables. Life table analysis was used to examine mortality rates in each of the derivation and validation populations. C-statistics, sensitivity, specificity, positive predictive value, negative predictive value, and accuracy were also computed for the Summit Score in the derivation and validation populations. The age, dyspnea, airflow obstruction (ADO) index was calculated using standard methodology and c-statistics were calculated.²¹ Analyses were performed in SPSS v.23.0 (IBM SPSS Statistics, Armonk, NY).

Validating the Summit Risk Scores

To validate the Summit Score, the derived scoring scheme was applied to the SUMMIT validation subjects and the Intermountain validations to predict all-cause mortality. Kaplan–Meier and Cox regression methods were used. Among Intermountain COPD patients, as is common among clinical datasets, most did not have spirometry available [only about 10% did, as noted previously²²] and the data were contained in separate clinical machines (in part, this may be due to spirometry being performed in primary care offices or other locations). Further, although tobacco history is collected routinely, data before 2015 did not include electronic capture and thus pack-years were assigned a null value in the Intermountain validations. Quartiles of each score were created separately for each population. A p-value of p≤0.05 was required in each independent population for validation.

Stratification by the Summit Score for the Evaluation of Trial Treatments

Once validated, the Summit Score was utilized to identify a subset of SUMMIT trial subjects among whom the active treatment of a combination FF 100 μg/VI 25 μg provided a protective mortality benefit compared with placebo, separating those individuals from one or more subsets in whom the benefit was not evident. A minimum of one third of the population with contiguous risk score values was required as a clinically meaningful subset of individuals in whom a benefit may exist. The maximum number of subjects allowable in the subset was not defined because, if it existed, it was necessarily less than the full trial population [the original analysis (N=16,485) did not find an overall mortality benefit: HR=0.88, CI=0.74, 1.04, p=0.137⁵].

To do so, a bar graph was used to assess where a benefit of the FF 100 μg/VI 25 μg combination medication resided and to define the Summit score boundaries of that subset. A search pattern was defined in which no two contiguous Summit Score values could have greater mortality in those receiving FF 100 μg/VI 25 μg than placebo, and at least two thirds of the scores in the subset had to have lower mortality in the FF 100 μg/VI 25 μg arm than placebo. Hazard curves and Cox regression were used to evaluate the trial medications compared with placebo in the subset with benefit and secondarily in the lack of benefit subsets. Secondary endpoints of cardiovascular events, moderate to severe COPD exacerbations, and change in FEV₁, FVC, and the FEV₁/FVC ratio were also evaluated using survival methods or analysis of variance, as appropriate. Other methods are provided in the on-line Supplement.

Results

Baseline demographics, clinical, risk factor, medication, and other variables for the SUMMIT derivation population and the validation populations are provided in Table 1. The randomization of SUMMIT subjects into derivation and validation groups resulted in the majority of variables being distributed similarly between the two subpopulations, with just two of the 38 variables listed in Table 1 (ie, RDW and short-acting anticholinergic medications) having statistically significant (but not clinically meaningful) differences.

Table 1 Baseline Characteristics of the SUMMIT Derivation Population and the Four Risk Score Validation Populations

The derivation of the Summit Score discovered that significant predictors of all-cause mortality were: BMI, number of pack years, number of prior COPD hospitalizations, FEV₁, HR, SBP, prior MI, prior HF diagnosis, prior diabetes diagnosis, age, and prescription of anti-thrombotics, anti-arrhythmics, and xanthines. The c-statistic for the Summit Score in the derivation population was c=0.668, which was comparable to the SUMMIT validation population (c=0.662) and lower than the Intermountain cardiovascular outpatients (c=0.736). In comparison, the ADO index had c-statistics of c=0.592 (95% CI: 0.567, 0.617) in the SUMMIT derivation and c=0.618 (95% CI: 0.593, 0.644) in the SUMMIT validation populations.

Lengths of longitudinal follow-up and annual mortality rates differed between the populations (Table 1), but the Summit Score was validated as a mortality risk stratification tool in the SUMMIT validation population and Intermountain outpatient validation population (Figure 1, Supplemental Figure S1), with risk differences of 4.4- to 7.6-fold between the lowest and highest risk score quartiles (Table 2, Supplemental Table S1). Examining associations of the Summit Score with mortality among Intermountain populations using quartile thresholds from the SUMMIT validation population resulted in similar findings (Supplemental Table S2 and Supplemental Figure S2). Predictive values and accuracy of the Summit Score are shown in Supplemental Table S3. Associations were also found of the Summit Score with first COPD exacerbation or cardiovascular event (Supplemental Table S4) and with changes in spirometry findings (Supplemental Table S5). Associations are shown for IMRS and pIMRS with mortality in Supplemental Figure S4 and for the Summit Lab Score with mortality in Supplemental Table S6 and Supplemental Figure S5.

Table 2 The Association of Quartiles of the Summit Risk Score with All-Cause Mortality in the Internal SUMMIT Validation Population Subset and in the Intermountain Outpatients with COPD and Cardiovascular Risks. As per Table 1, Length of Follow-Up for Mortality Differed Between Populations

Figure 1 Kaplan–Meier survival curves displaying the association of the Summit Risk Score with all-cause mortality among: (A) the validation half of the SUMMIT trial population, N=8304 (for quartiles 4, 3, and 2 compared with quartile 1: Log rank p<0.001, p<0.001, and p=0.002, respectively; c-statistic: c=0.662), (B) an Intermountain Healthcare validation population of outpatients with COPD and cardiovascular risks, N=9251 (for quartiles 4, 3, and 2 compared with quartile 1: Log rank p<0.001, p<0.001, and p<0.001, respectively; c-statistic: c=0.736). Time intervals on the x-axes were designated in 365-day intervals for SUMMIT populations and years for Intermountain populations.

Analyses of the four SUMMIT treatments were performed in risk-based substrata of the combined SUMMIT derivation and validation populations (N=16,485), with the substrata defined by levels of the Summit Score. Mortality in each Summit Score value (Supplemental Figure S3) identified risk score thresholds that differentiated mortality benefit between subjects taking the combination of FF 100 μg/VI 25 μg. Among SUMMIT subjects with a Summit Score of 14–19 (n=9243), Figure 2 reveals that those with the combined FF 100 μg/VI 25 μg treatment had fewer events compared with placebo (HR=0.76, CI=0.61, 0.95, p=0.0158). No effect modification was found when an interaction variable of study treatment with SUMMIT derivation/validation was evaluated (for FF 100 μg/VI 25 μg vs placebo: HR=0.77 in derivation, HR=0.75 in validation; p-interaction= 0.87). In the Summit Score 14–19 subgroup, FF 100 μg/VI 25 μg also was associated with lower risk of moderate to severe COPD exacerbation (HR=0.86, CI=0.77, 0.96, p=0.005) and improved changes in FEV₁, FVC, and FEV₁/FVC ratio (all p<0.001) compared with placebo, but not with cardiovascular events (p=0.12, HR=0.81, CI=0.62, 1.06). The impact of the medications on mortality in subjects with a Summit Score ≤13 (n=5610) and ≥20 (n=1632) are shown in Supplemental Figures S2B and S2C, and results for secondary outcomes in these subsets are also provided in the Supplement. No difference was found between those assigned to FF/VI or placebo based on Summit Score continuous values (p=0.36) or categorizations (p=0.22; 25%, 25%, and 24% of subjects received placebo and 24%, 25%, 26% received FF 100 μg/VI 25 μg in Summit Scores ≤13, 14–19, or ≥20, respectfully).

Figure 2 Hazard curves for the effects of the four SUMMIT trial randomized treatments on all-cause mortality among all SUMMIT trial subjects (ie, both the Summit Score derivation and validation groups). Trial randomizations included: once daily inhaled placebo, vilanterol (VI) 25 μg, fluticasone furoate (FF) 100 μg, or the combination of FF 100 μg and VI 25 μg. Here the combination of FF 100 μg/VI 25 μg (n=2347) had a statistically significantly lower mortality compared with placebo (n=2305) (p=0.0158, HR= 0.76, CI=0.61, 0.95) among subjects with a Summit Score ≥14 and ≤19 (n=9243, which is 56.1% of the SUMMIT trial subjects). The hazard curves for subject with Summit Score ≤13 and ≥20 are provided in Supplemental Figures S2B and S2C.

Discussion

Summary of Findings

The Summit Score predicted all-cause mortality among a derivation population from the SUMMIT trial and was validated in the other half of the SUMMIT trial subjects and in Intermountain patients who had moderate COPD and cardiovascular risk factors. The Summit Score also predicted mortality in lower-risk Intermountain outpatients and higher-risk inpatients. Among a substratum of SUMMIT trial subjects with intermediate Summit Scores (ie, scores between 14–19, which constituted 56% of subjects), SUMMIT treatment with FF 100 μg/VI 25 μg was associated with a lower risk of all-cause mortality; in contrast, no difference in mortality based on trial medical therapy was found in subjects with Summit Scores ≤13 or ≥20. Finally, the Summit Score predicted COPD exacerbations, cardiovascular events (eg, unstable angina, MI, stroke), changes in spirometric measures, incident heart failure, incident cardiac arrhythmias, and coronary revascularization.

COPD, Cardiovascular Disease, and Decision Tools

COPD is a common complex disease that is challenging to diagnose and requires a thorough and encompassing management approach.¹ Patients with COPD frequently have other respiratory and cardiovascular comorbidities, some due to the shared risk factor of tobacco use.^1,2,23-25 COPD severity and prognosis may be stratified by composite risk tools,¹ such as the BODE Index,²⁶ ADO Index,²¹ and DOSE Index,²⁷ but these scores use a very limited number of predictors and focus on pulmonary factors. Commonly utilized in COPD populations, these validated scores are the mainstay of risk prediction in pulmonary disease populations.^1,21,26,27 Analysis of the ADO Index in the derivation and SUMMIT validation populations herein found c-statistics of 0.592–0.618, which were 0.05–0.07 lower than the Summit Score.

Those risk indices were also derived in small populations (fewer than a thousand) with the intended application to all COPD patients. Their use among severely diseased patients or those with complex mixes of morbidities may not fit the originally derivation.²¹ Similarly, while IMRS and pIMRS were predictive of mortality in SUMMIT trial subjects herein, those associations were weaker than in prior evaluations of lower-risk patients.¹⁰ Given the challenges of applying a risk score to patients with moderate COPD and cardiovascular comorbidities, the Summit Score was developed to attune prediction models to that population.

To predict mortality, the Summit Score employed a myriad of variables spanning the spectrum of pulmonary disease, cardiovascular disease, and general health. These included clinical exam findings, COPD-specific measurements, cardiovascular parameters, prescription data, and other variables that are not disease-specific. This score utilizes both static and dynamic factors, meaning that some component variables can respond to both positive and negative changes in health (eg, FEV₁, SBP, and RDW). Static factors are strong predictors (eg, experiencing an MI) but will not return to being absent once they occur. While the scores provided moderate risk discrimination in the trial population and were useful in identifying subjects who may receive benefit from FF 100 μg/VI 25 μg treatment, among real-world Intermountain patients the risk differences between Summit Score categories were greater. These findings require further study including whether the Summit Score can identify patients for whom FF 100 μg/VI 25 μg is effective in actual practice, and evaluation in implementation science.

Implementation of risk scores in clinical medicine has the goal of achieving best care, whether scores guide a patient care plan or are used to tailor a medication prescription to patient needs. Often today, clinical decision tools such as the Summit Score are used to guide medication prescriptions because the score reveals therapeutic need based on a favorable benefit to risk profile, while indicating which patients do not have a favorable risk-benefit. For example, among atrial fibrillation patients, the CHADS₂ and CHA₂DS₂-VASc scores are used to identify which patients should receive oral anticoagulation and which excessive bleeding risk.²⁸ Among post-stent patients, the DAPT Score is used in determining which patients should receive long-term (ie, >12 months) dual antiplatelet therapy after percutaneous coronary intervention.²⁹ Instead of guiding a specific medical therapy, IMRS and other scores are used as attention metrics to point clinicians to higher-risk patients and to inform them of which patients require less of their time because they will have the same outcomes from standard care as from intense evaluation and treatment.¹⁷ This approach is associated with lower 30-day mortality and 30-day readmission in high-risk HF patients, with no worsening of outcomes for lower-risk patients.¹⁷

The Summit Score has the potential to indicate both therapeutic effect of a medical therapy and to be used as an attention metric for aggressive prevention (see the Supplemental Use Cases). This study suggests that FF 100 μg/VI 25 μg may have varying effectiveness across the spectrum of COPD patients, perhaps due to heterogeneity of disease severity or complexity. A biological rationale exists for the benefit of FF+VI and a reduction in cardiovascular benefits as this combination has been associated with a broad spectrum of anti-inflammatory effects in patients with COPD.³⁰ Study limitations are described in the Supplemental Discussion.

Limitations

This study was challenged by a number of issues in the process of developing a risk score since the predictive performance is critically determined by the available data and the nature of the populations studied. One was that the primary study population consisted of individuals who were selected for inclusion in a large prospective randomized clinical trial based on stringent criteria and are unlikely to be broadly representative of clinical practice.^5,20 Actual medical practice includes a broad range of patients rather than those with a tightly defined COPD diagnosis as in a trial, which may in part account for the greater ability for the Summit Score to stratify risk in the three Intermountain populations given that some parameters were not necessarily pulmonary-related. Further, the Intermountain clinical population was limited because a few Summit Score parameters were not available (i.e., FEV₁–which was unavailable for ≈90% of patients and for the few who did have spirometry data it was housed in a proprietary vendor database not accessible to study personnel, and smoking status which was unavailable outside of free text reports).

Secondly, this study derived the risk scores among subjects in whom three quarters were assigned an active medication in the SUMMIT trial that may have some effect on mortality (the other quarter received placebo). The derivation of the score, though, did not account for the treatment variable so that subsequent analysis could examine using the score for therapeutic risk stratification. Analysis showed that the percentage of subjects receiving each medication assignment did not differ across levels of the risk score, indicating that the score did not include the treatment effects. If any bias existed due to the treatment effects, thus, it was toward the null hypothesis. Finally, despite that the c-statistics for mortality in the SUMMIT population were of only moderate strength, the risk was strongly stratified by the Summit Score. Further evaluation of the score is needed in additional populations and in studies using the risk score to identify people who will benefit from combined FF 100 μg/VI 25 μg treatment. Because the SUMMIT Trial did not use ICD codes and the Intermountain cohort inclusions ended in 2015, ICD-10 coding was not evaluated herein and could provide improved resolution.

Conclusions

The Summit Score was developed and validated in geographically and constituently distinct populations as a predictor of all-cause mortality. It also predicted COPD and cardiovascular endpoints. When modified to include common laboratory panels (ie, CBC and BMP), the resulting Summit Lab Score also predicted those outcomes (see Supplement). Uniquely, the Summit Score identified a subset of individuals with similar risk levels who may receive therapeutic benefit from a combination of FF 100 μg/VI 25 μg, which requires additional evaluation and validation. Further study of these therapeutic risk scores in research and clinical implementation environments is indicated.