Performance of Different Comorbidity Indices in Predicting Mortality in Danish Pancreatic Cancer Patients

Introduction

Pancreatic cancer is an aggressive malignancy with a 5-year survival of ~10%.¹ It is often diagnosed at an advanced stage due to its asymptomatic progression in the early stages, making effective treatment options limited and prognosis poor.² Surgery remains the best chance for long-term survival,³ but as few as 20% of patients present with localized disease and are eligible for surgical treatment.² The disease is characterized by rapid metastatic spread and resistance to conventional therapies.⁴ As a result, improving the understanding of factors that influence outcomes is important to identify strategies that can improve survival.

Comorbidity indices are commonly used in epidemiological research to adjust for the impact of comorbid conditions in prognostic studies.⁵ These indices quantify the burden of comorbidities a patient may have, providing a summary measure that can be used in multivariable regression models. Several comorbidity indices have been developed such as the Charlson Comorbidity Index, the Elixhauser Comorbidity Index, the van Walraven Index, the Gagne Comorbidity Index, and the Nordic Multimorbidity Index.^6–10

However, the performance of such indices may vary depending on the clinical context, making it essential to assess their utility in specific patient populations. In the present study, we compared the indices’ discrimination of risk of death in Danish pancreatic cancer patients, considering discrimination a measure of the indices’ ability to adjust for confounding.

Materials and Methods

Setting and Data Sources

We linked data from nationwide healthcare registries in Denmark to identify patients diagnosed with pancreatic cancer during 2004–2022. We used information from the Danish Cancer Registry,¹¹ Danish National Patient Registry,¹² Civil Registration System,¹³ the Danish Pathology Registry,¹⁴ and the Danish National Prescription Registry.¹⁵

The Danish Cancer Registry was established in 1943 and includes information on all cancers diagnosed in Denmark. This registry contains information on date of diagnosis, cancer site, histology, dissemination, and other variables.

The Danish National Patient Registry was established in 1977 and contains information on all inpatient hospitalizations to Danish public hospitals. Outpatient and emergency room visits have been included since 1995. Patients are registered with diagnoses according to the International Classification of Diseases (ICD) 8^th revision (ICD-8) from 1977 through 1993 and ICD 10^th revision (ICD-10) hereafter.

The Civil Registration System, which was established in 1968, is an administrative registry containing data on variables like birth date, sex, sequential dates of migration, and vital status for every resident in Denmark. The Civil Registration System is updated daily and virtually complete.

The Danish National Pathology Registry contains information on all tissue examinations and pathology specimens examined at Danish hospitals since 1997. This registry includes information on resection margins, histology, and other potential predictors of cancer prognosis.

The Danish National Prescription Registry has collected detailed information on all prescriptions filled at Danish community-based pharmacies since 1995. This registry contains Anatomical Therapeutic Chemical (ATC) classification codes for drugs, purchase dates, and dose information, among other variables.

Study Design, Population, and Outcomes

We conducted a cohort study, where we first identified all patients diagnosed with pancreatic cancer in the Danish Cancer Registry from 1 January 2004 to 31 December 2022 and excluded patients with unknown marital status or area of residence, patients diagnosed at death or autopsy, patients aged less than 18 years at diagnosis, patients with less than 10 years continuous residency in Denmark before diagnosis, patients with unknown or non-adenocarcinoma histology patients with unknown tumour stage, and patients emigrating from Denmark within 2 years after diagnosis (Supplementary Figure 1). The outcome was mortality assessed at one (primary analysis) and two years (sensitivity analysis).

Comorbidity Indices

For each patient in the study population, we assessed the presence of each comorbidity from each of the following indices. We then computed the index scores, which were used in analyses.

Charlson Comorbidity Index

The Charlson index was developed in 1987 on 604 patients from 1984 and designed to predict 1-year mortality.⁶ It initially included 19 medical conditions, defined by ICD codes, which are each assigned a weight depending on their association with one-year mortality. It was later revised to include 17 conditions, as leukaemia and lymphoma were combined with non-metastatic solid tumours. In the original publication, no lookback period for comorbidity assessments was reported.⁶ In the present study, we used lookback periods of 1, 5, and 10 years.

Elixhauser Comorbidity Index

The Elixhauser index was developed in 1998 on nearly 2 million patients from California, US, and designed to predict in-hospital mortality, length of stay, and hospital costs.⁷ It included 30 medical conditions, defined by ICD codes, all of which were equally weighed. In 2009, the index was revised and different weights were assigned to 21 of the included conditions (the van Walraven weights).⁸ Similar to the Charlson index, no lookback period for the Elixhauser index was reported in the original publication,⁷ whereas we used lookback periods of 1, 5, and 10 years.

Gagne Comorbidity Index

The Gagne index was developed in 2011 on more than 120,000 Medicare beneficiaries from Pennsylvania, US, and was designed to predict 1-year mortality.⁹ The Gagne index combined the diagnoses from the Charlson index and the Elixhauser index using the van Walraven weights.

Nordic Multimorbidity Index

The Nordic Multimorbidity Index was recently developed in Denmark on a random sample of the general population aged 40 years or older, and was designed to predict 5-year all-cause mortality.¹⁰ This index was specifically developed and validated on a Danish population, and it includes 50 predictors combining diagnoses defined by ICD-codes and prescription drugs identified by Anatomical Therapeutic Classification (ATC) codes. The lookback period for ICD codes is five years and for prescription drug use six months.

Statistical Analyses

We present descriptive characteristics for the study population reporting counts and percentages for categorical variables, and median with interquartile interval (IQI) for age. For each comorbidity index, we calculated the 1^st, 10^th, 25^th, 50^th, 75^th, 90^th, and 99^th percentile to present distributions of the scores. We calculated the correlation between the different index scores using the Spearman correlation coefficient. Using a logistic regression, we first fitted three basis models with one-year mortality as the dependent variables and the following independent variables: (1) age, sex, and year of diagnosis, (2) age, sex, year of diagnosis, and marital status and area of residence, and (3) age, sex, year of diagnosis, and marital status and area of residence, and tumour stage. For each comorbidity index, we assessed its performance by including the index score as an additional independent variable to the third model. Age, year of diagnosis and Nordic Multimorbidity score was modelled using restricted cubic splines with three knots, whereas the Charlson, Elixhauser, van Walraven and Gagne scores were all included as linear covariates. Performance was measured by c-statistics, specifically, by the area under the receiver operator characteristic curve (AUC). All estimates are presented with corresponding 95% confidence intervals (CIs). Statistical analyses were conducted using Stata 19 (StataCorp LP, College Station, TX, USA).

Sensitivity Analyses

In addition to the main analysis, we conducted three sensitivity analyses. First, we excluded all malignant diagnoses from the comorbidity indices. Second, we restricted to patients undergoing curative-intent surgery. In this analysis, we used two-year survival instead of one-year survival as the outcome and set the index date to the date of surgery. Third, to explore if the performance of the comorbidity indices differed across different levels of tumour stage, we conducted an analysis stratifying by AJCC stage.

Results

Descriptive Characteristics

We initially identified 19,195 patients, of which 8782 were excluded (Supplementary Figure 1), leaving a study population of 10,413 patients. The median age was 69 years (IQI: 62–75 years), and 5492 (52.7%) were men (Table 1). Distribution of comorbidity scores are presented in Table 2. One-, and two-year survival for the entire cohort was 30.4% (95% CI: 29.5–31.3%) and 15.1% (95% CI: 14.3–15.8%), respectively.

Table 1 Descriptive Characteristics of 10,413 Danish Patients Diagnosed with Pancreatic Adenocarcinoma During 2004–2022

Table 2 Distribution and Performance of Comorbidity Scores

Correlation Between Comorbidity Index Scores

There was a moderate to strong correlation between the different comorbidity index scores, measured by the Spearman correlation (Figure 1). The correlation between the Charlson and Elixhauser scores were high across lookback periods, whereas the correlation between these two scores and the Nordic Multimorbidiy and Gagne scores were more modest (Figure 1).

Figure 1 (A) Correlation of the comorbidity scores (1-year lookback period for Charlson, Elixhauser, and van Walraven indices). (B) Correlation of the comorbidity scores (5-year lookback period for Charlson, Elixhauser, and van Walraven indices). (C) Correlation of the comorbidity scores (10-year lookback period for Charlson, Elixhauser, and van Walraven indices).

Performance of Comorbidity Indices

All Patients

The basis model using age, sex, and calendar year of diagnosis had an AUC of 0.64 (95% CI: 0.63–0.65). Adding information on marital status and area of residence provided little additional value (AUC: 0.65; 95% CI: 0.64–0.66). The AUC increased to 0.82 (95% CI: 0.81–0.83) after adding tumour stage to the model (Table 2). Adding comorbidity index scores to this model provided no additional value, as all models had an AUC of 0.82–0.83. There were no differences in the AUC’s regardless of the lookback period used (1-, 5- or 10-year period) for neither the Charlson, Elixhauser or van Walraven scores (Table 2).

Sensitivity Analyses

Removing malignant diagnoses from the comorbidity indices had no impact on the estimates (Supplementary Table 1). Descriptive characteristics for the population undergoing curative-intent surgery are listed in Supplementary Table 2. Survival after surgery was 76.4% (95% CI: 74.4–78.2%) at one year and 52.8% (95% CI: 49.6–53.9%) at two years. Looking at two-year mortality in this population, the AUC of the basis models were lower, ranging from 0.57 without tumour stage to 0.63 with tumour stage (Table 3). Adding comorbidity index scores to the model provided little additional value, as all models had an AUC of 0.63–0.65 (Table 3). When we stratified by tumour stage, the comorbidity indices generally had some predictive value beyond the basis model with the highest increase in the AUCs seen for the Nordic Multimorbidity Index (Supplementary Tables 3–6). However, the estimated AUCs were imprecise. The discriminative ability was low to intermediate for all indices in all strata with a maximum of 0.76 (95% CI: 0.72–0.81) when adding NMI to the basis model in stage I. For stages II–IV all estimated AUCs were below 0.7.

Table 3 Distribution and Performance of Comorbidity Scores, Restricting to Patients Undergoing Curative-Intent Surgery

Discussion

In this nationwide cohort study, we assessed the performance of some commonly used comorbidity indices in predicting mortality among Danish patients diagnosed with pancreatic cancer. Our findings indicate that while these indices overall correlates well with each other, their addition to predictive models offers minimal incremental prognostic value beyond well-established clinical factors, most importantly tumour stage. After stratification for tumour stage, the discriminative value of the Nordic Multimorbidity Index may be slightly higher than the other indices, but confidence intervals were overlapping.

Accurate adjustment for comorbid conditions is critical in observational studies of determinants of survival in most cancers. To improve statistical efficiency, comorbidity indices are widely used for this purpose, but their utility may vary by clinical context. None of the indices examined in this study improved the discrimination of one-year mortality beyond a model incorporating age, sex, calendar year, marital status, area of residence, and - most importantly – tumour stage. The tumour stage emerged as the dominant predictor of survival, increasing the AUC from 0.65 to 0.82. The limited added predictive value of comorbidity indices may be attributable to the overwhelming prognostic influence of tumour stage in a highly aggressive malignancy, such as pancreatic cancer.² However, comorbidity indices may perform better in other less aggressive malignancies, such as colorectal or breast cancer.

Similarly, among the subset of patients undergoing curative-intent surgery, comorbidity indices provided minimal incremental benefit in predicting two-year survival. This finding may reflect the inherent selection of patients undergoing surgery, as these are fitter with fewer severe comorbidities. Moreover, excluding malignant diagnoses from the indices did not alter the model performance, suggesting that comorbidities are of limited discriminative relevance in pancreatic cancer patients.

The study benefits from the use of population-based Danish national registries, ensuring virtually complete capture of diagnoses, treatments, and vital status.^12,13 However, the utility of comorbidity indices may differ in other healthcare systems, limiting generalizability. This could be addressed in future similar studies conducted in other healthcare settings. In the era of personalized medicine, information on genetic alterations in genes such as KRAS, P53, CDKN2A, SMAD4, and BRCA may provide additional value in the prediction of mortality in pancreatic cancer patients. Unfortunately, such data was not available in our study.

Conclusions

While comorbidity indices correlate moderately to strongly and are valuable tools for summarizing patient health status, their utility in improving mortality discrimination for pancreatic cancer patients in Denmark is limited, highlighting the need for adjustment of clinically important prognostic factors rather than a generic index score in epidemiological studies.