Treatment selection bias for chemotherapy persists in colorectal cancer patient cohort studies even in comprehensive propensity score analyses

Background

Colorectal cancer (CRC) is mostly diagnosed at older age, when comorbidities are common.¹ In many countries, the standard treatment for high-risk stage II and stage III colon cancer (CC) is surgery followed by adjuvant chemotherapy^2,3 and for stage II-III rectal cancer (RC) is preoperative chemoradiotherapy and surgery followed by postoperative chemo(radio)therapy.^3,4 Multiple studies have suggested underuse of chemotherapy in comorbid CC patients.^5–14 However, previous studies mostly focused on CC patients, and evidence on comorbidity and neoadjuvant chemoradiotherapy in RC patients is sparse. Furthermore, many of the prior studies referred to patients diagnosed in 1995–2003,^5–9,14 but chemotheraupeutic compounds have evolved¹⁵ and comorbidity profiles might have changed since then, asking for timely assessment of the extent to which comorbidities influence the current regimens of CRC care.

Data from randomized clinical trials (RCTs)¹⁶ have shown that chemotherapy is associated with 24% reduction in all-cause mortality in stage III CC patients. However, patients enrolled in RCTs are often younger, have fewer comorbidities and are more functionally fit than the general population. Multiple population-based studies^17–22 have demonstrated substantial benefits of chemotherapy in stage III CC patients, with estimates for reduction in all-cause mortality ranging from 24% to 52%.^17–19,22 Most of these studies, however, lacked detailed information on comorbidities and had no information on functional status (FS). Hence, to what extent imperfectly ascertained comorbidity status or unmeasured FS impacts the benefit of chemotherapy in routine practice is unknown.

Propensity score (PS) methods are increasingly used to address confounding related to treatment selection bias,²³ but their potential to correct for selection bias strongly depends on the comprehensiveness of the information used for matching/weighting. We hypothesized that if PS analyses completely account for treatment selection, then the benefit of chemotherapy for mortality from causes other than CC should be lower than for CC mortality, as there is no evidence to date that chemotherapy prevents such incidents.

In this population-based study, we aimed to investigate (i) the association of comorbidity (separately and jointly with FS) with treatment patterns in stage II-III CC and RC patients, (ii) to what extent lower comorbidity and higher FS explain the benefit of chemotherapy in stage III CC patients, and (iii) whether selection bias persists in treatment effects’ estimates from PS analyses.

Methods

Study population

This study used data on 1102 CRC patients diagnosed in 2003–2014 and recruited into the DACHS (Darmkrebs: Chancen der Verhütung durch Screening) study, a population-based case-control study with additional regular follow-up of cases conducted in the Rhine-Neckar region of Germany. Patients with first time diagnosis of CRC (International Classification of Diseases, 10th Revision (ICD-10), codes C18–C20) and aged ≥30 years were eligible. Patients were recruited from all 22 hospitals providing first-line treatment of CRC in the study region. Further details of the DACHS study have been described elsewhere.^24,25 The DACHS study was approved by the ethics committees of the Medical Faculty of Heidelberg University and the state medical boards of Baden-Wuerttemberg and Rhineland-Palatinate. All participants gave written informed consent.

At baseline, trained interviewers conducted interviews with the participants to collect information on lifestyle factors and medical history, including comorbidities, using a standardized questionnaire. Around three years after diagnosis, detailed information on CRC treatments was collected from medical reports and from a physician administered questionnaire (physicians extracted information on patients’ treatments saved on computer systems). Follow-up started from CRC diagnosis and vital status and cause of death were ascertained from population registries.

Ascertainment of comorbidities and functional status

Comorbidities that were diagnosed either before or at the time of CRC diagnosis were ascertained from medical reports, using ICD-10 codes (Table S1). To ensure comparability with previous studies, we used the Charlson comorbidity index (CCI)²⁶ to compute an overall comorbidity score. We grouped patients into four groups, namely CCI score 0 (no comorbidity), 1, 2 and 3+ (severe comorbidity). Perioperative assessment of FS varied between hospitals and within hospitals over time. The most commonly employed scales were American Society of Anesthesiologists’ (ASA) grade,²⁷ Eastern Cooperative Oncology Group (ECOG) score²⁸ and Karnofsky Performance Score (KPS).²⁹ ASA is a 5-grade (I-V) scale, with higher grades indicating poor FS.²⁷ ECOG assesses FS on a 6-point scale (0–5), with higher scores indicating worse FS.²⁸ KPS rates FS on a scale of 0–100; a score of 100 indicates optimal health.²⁹ We adapted the ESMO criteria for comparing ECOG and KPS scores and used the distribution of ASA grades to classify patients into three groups – excellent (ASA=I, ECOG=0 or KPS=90–100), fair (ASA=II, ECOG=1 or KPS=70–80) and poor FS (ASA=III-IV, ECOG=2–4 or KPS=10–60).^28,30

Inclusion criteria

Our analytic sample comprised all radically resected high-risk stage II and stage III CC and stage II-III RC patients who were recruited in hospitals that reported FS data for at least 75% of the patients in a year (Figure S1). We defined high-risk stage II CC as the presence of T4 stage (all years of diagnosis) and with <12 examined lymph nodes (patients diagnosed in ≥2008). Further criteria such as presence of bowel obstruction or perforation could not be considered. The survival analyses were, however, restricted to stage III CC patients only.

Statistical analysis

The distribution of patients with CCI=1+ according to baseline characteristics was assessed in the whole sample. In CC patients, we investigated the association of comorbidity with adjuvant chemotherapy administration (yes/no) using multivariable logistic regression. In RC patients, we examined the association of comorbidity with neoadjuvant chemoradiotherapy and adjuvant chemotherapy (yes/no). The joint association of comorbidity, FS and age with treatment patterns was moreover investigated, and potential interaction of comorbidity with age or FS was examined, by adding interaction terms to the models.

Lastly, we estimated to what extent lower comorbidity and higher FS explain the apparent benefit of chemotherapy and whether treatment selection bias persists in estimates for treatment effects in stage III CC patients. Here, the association of chemotherapy with overall (OS; mortality from any cause), disease-specific (DSS; mortality from CC) and non-disease-specific survival (nDSS; mortality from causes other than CC) were estimated using Cox proportional-hazards regression. Time was calculated from CC surgery to the respective endpoints or end of follow-up, whichever occurred first. We assessed the proportional hazards assumption by adding time-dependent interaction terms to the model and checking whether their effects were significant. Time-dependent interaction terms (with age, body mass index [BMI] and CCI scores) were retained in the model in case of violation of the assumption.

To address confounding related to selection bias, we calculated PS²³ for chemotherapy from three separate logistic regression models: (i) sex, age, year of diagnosis, education level, smoking status, BMI, lifetime physical activity and alcohol consumption, T-stage, N-stage, grade, number of examined lymph nodes, surgical technique and surgical volume; (ii) additional inclusion of CCI scores (here, patient-reported comorbidities that were not captured in the medical reports were included); and (iii) further inclusion of FS. The degree of overlap of the distributions of PS between the treatment groups was assessed. Because of our relatively small sample size, we used the inverse probability of treatment weighting to balance PS in our analyses, as appropriate.³¹

Sensitivity analysis

To assess the robustness of our estimates for treatment effects, we truncated weights at the 99th and 95th percentiles, by setting them as the threshold.³² Results from the truncated models were then compared to those from the main and PS adjusted models. We also used the Mantel-Byar method³³ to account for immortal time bias, by using time-dependent exposure definition whereby the time from surgery to chemotherapy initiation was defined as “no chemotherapy” in the Cox models. Moreover, we used the Fine and Gray model³⁴ to account for non-CC mortality (a competing event for DSS) and CC mortality (a competing event for nDSS). All analyses were conducted with SAS software, version 9.4 (SAS Institute, Cary, NC), with α=0.05.

Results

Of 4916 CRC patients diagnosed in 2003–2014, those recruited in hospitals that reported <75% FS data for participants and those with stage I and IV tumors, non-R0 resection and with missing data on any covariate of interest were excluded (Figure S1). A total of 523 stage II (high-risk) and III CC and 579 stage II-III RC patients were included in the treatment pattern analyses.

The median age was 70 years. Nearly half of the patients had CCI=1+ (47%; Table S1). The two most common comorbidities were various cardiovascular diseases (overall=20%) and diabetes (19%). Comorbidities were particularly common in men, older, less educated, underweight and overweight/obese patients, former smokers, patients with functional impairment and those recruited in low surgical volume hospitals (Table 1). Adjuvant chemotherapy was administered to 68% of the CC patients. Among the RC patients, 36% received neoadjuvant chemoradiotherapy and 58% received adjuvant chemo(radio)therapy. A total of 430 stage III CC patients were included in the survival analysis. During a median follow-up of 4.7 years, 158 (37%) deaths occurred, of which 89 (56%) were due to CC.

Table 1 Distribution of frequency of patients with at least one comorbid condition

Comorbidity and treatment patterns

In CC patients (Table 2), chemotherapy use was 83% in non-comorbid patients and 43% in severely comorbid patients. After adjusting for multiple covariates including FS, CCI scores 1, 2 and 3+ were associated with 57%, 64% and 70% lower odds of chemotherapy receipt. In RC patients, neoadjuvant chemoradiotherapy use was 42% in non-comorbid patients and 26% in patients with CCI=3+. Administration of adjuvant chemotherapy was also lower in severely comorbid (34%) than in non-comorbid (65%) patients (odds ratio [OR]: 0.51; 95% confidence interval [CI]: 0.26–0.98). No significant interaction with tumor stage was observed (Table S2). In CC patients, heart failure, peripheral vascular disease, stroke, chronic obstructive pulmonary disease, renal disease, diabetes and other cancers were significantly associated with less frequent chemotherapy administration (Table 3). Among RC patients, peripheral vascular disease and other cancers were associated with not receiving neoadjuvant chemoradiotherapy, and heart failure with not receiving adjuvant chemotherapy.

Table 2 Association of comorbidity scores with administration of neoadjuvant and adjuvant treatments in colon and rectal cancer patients

Table 3 Association of individual comorbidities with treatment patterns

Joint effect of comorbidity and functional status

In CC patients without comorbidities and functional impairment, chemotherapy use was 94% in younger (<70) and 81% in older patients (70+ years; Table 4). The presence of comorbidities and functional impairment reduced chemotherapy use to 75% in younger and 35% in older patients. In younger and older patients with functional impairment, comorbidity was associated with 86% and 97% lower odds of receiving chemotherapy, respectively. In RC patients with functional impairment, comorbidity was not associated with treatments in younger patients, but was associated with 62% and 83% decreased odds of neoadjuvant chemoradiotherapy and adjuvant chemotherapy use in older patients, respectively. Significant interactions between comorbidity and age (p_interaction=0.027) and FS (p_interaction=0.001) for adjuvant treatments were observed in RC but not in CC.

Table 4 Association of comorbidity with treatment patterns according to age and functional status

Association of chemotherapy with survival outcomes

Figure 1 and Table 5 show adjusted survival curves and hazard ratios (HRs) for chemotherapy and survival outcomes in stage III CC patients, respectively. Without comorbidity and FS in the PS models, chemotherapy was associated with 47% and 38% improved OS and DSS, respectively (Table 5). Approximately 38% and 24% of the benefits of chemotherapy for OS and DSS, respectively, were explained by lower comorbidity and higher FS. In the comprehensive models, chemotherapy was associated with improved OS (HR=0.73; 95% CI=0.58–0.92) and non-significantly improved DSS (HR=0.77; 95% CI=0.56–1.06). Selection bias was observed, as chemotherapy was associated with substantially improved nDSS (HR=0.66; 95% CI=0.46–0.92) than DSS, especially in patients <75 years (HR=0.33; 95% CI=0.17–0.63; p_interaction=0.004). The benefits of chemotherapy were substantially overestimated in all the conventional covariate adjustment models. No significant interaction between chemotherapy and BMI was observed for OS and DSS (Table S3).

Table 5 Association of chemotherapy with survival outcomes in stage III colon cancer patients (n=430)

Figure 1 Adjusted survival curves for chemotherapy use and survival outcomes in stage III colon cancer patients.

In sensitivity analyses where the weights of chemotherapy non-recipients were truncated at the 99th percentile, the associations became slightly stronger and were similar to those from PS adjusted models (Table S4). The associations also persisted after accounting for immortal time bias (Table S5). Upon accounting for competing events, the associations for DSS (HR=0.83) and nDSS (HR=0.73) were attenuated. The most common cause of death other than CC was due to cardiovascular incidents (39%), which were more frequent in non-recipients than in recipients of chemotherapy (Table S6).

Discussion

Older CC patients, in whom comorbidities and functional impairment are common, constitute a large proportion of CC patients who are often underrepresented in RCTs. Hence, estimates for chemotherapy effects from RCTs might be less generalizable to unselected patient populations. Results from observational studies might address this limitation, but these studies often lacked information on pertinent comorbidities and FS overall.^17–22 Furthermore, treatment selection bias could impact estimates for treatment effects, but it is unclear whether adjustment for comorbidities and FS is sufficient to address this bias. In this population-based study, comorbidities were associated with less frequent administration of chemo/radiotherapy in both CC and RC patients, especially in those with functional impairment and older age. After accounting for comorbidity and FS in our comprehensive PS analyses in stage III CC patients, treatment selection bias for chemotherapy persisted, as chemotherapy improved nDSS more strongly than DSS, especially in patients <75 years.

In agreement with findings from previous studies,^5–13 we found substantially lower chemotherapy use in comorbid CC patients. Preoperative chemoradiotherapy in non-metastatic RC patients reduces local recurrence and improves sphincter preservation.³⁵ Comorbidity was inversely associated with administration of neoadjuvant chemoradiotherapy and adjuvant chemotherapy in RC patients in our study. Only two previous studies^13,36 have addressed the association of comorbidity with neoadjuvant and adjuvant chemoradiotherapy, but they did not distinguish between these types of treatments in their analysis.

No previous study has investigated the joint effect of comorbidity and FS on CRC treatments. We observed stronger effects of comorbidity in older and functionally impaired patients. Our results suggest that in younger patients, comorbidities alone might have a minor role in therapeutic decisions. In combination with functional impairment, comorbidity however strongly impacted treatment decisions, especially in older patients. This indicates that enhanced management of geriatric syndromes might be most useful. Our results also highlight the need for studies estimating treatment effects to account for these characteristics in order to obtain unbiased estimates.

Population-based studies have shown substantial benefit of chemotherapy in stage III CC patients.^17–22 These studies, however, mostly ascertained comorbidities from administrative records, which are known to underestimate comorbidities, and also lacked information on FS. Given the important roles of comorbidities and FS in treatment decisions and in CC prognosis,³⁷ we estimated the extent to which these characteristics explain chemotherapy benefits in stage III CC patients. We found that lower comorbidity and higher FS of chemotherapy recipients explained approximately 38% of the improved survival in chemotherapy recipients. In the comprehensive PS models, we observed a 27% significant reduction in all-cause mortality in chemotherapy recipients, which is comparable to results from the US-SEER²² and NYSCR-Medicare data,¹⁹ but is lower than those reported in many previous studies (HR_range 0.42 to 0.63).^17–19 Similarly, our estimate for DSS (HR=0.77) was much lower than those previously reported (HR_range 0.36 to 0.65).^17,20,21 Our 5-year OS estimates for chemotherapy recipients and non-recipients (73.6% vs 65.6%; HR=0.73) were, however, very similar to those from RCTs (71% vs 64%; HR=0.76).¹⁶

A possible explanation for the much smaller benefits of chemotherapy in our study than those reported in previous observational studies is that many previous studies did not include important factors such as BMI, alcohol consumption, smoking and physical activity in their PS models,^17,19,20 and the study by Boland et al²¹ in particular lacked information on comorbidities. Our results suggest that treatment effects were most likely overestimated in previous observational studies due to residual confounding from either imperfectly ascertained comorbidity status or lack of information on FS. For instance, when we excluded comorbidity and FS from the models, chemotherapy benefits increased from 27% to 47% for OS and from 23% to 38% for DSS and were comparable to those from previous studies. It is thus important that future studies evaluating treatment effects discuss the possibility of strong overestimation of chemotherapy effects when lacking a comprehensive adjustment for FS and comorbidities.

Adjustment for FS and comorbidity in addition to other known treatment-related and prognostic factors is necessary to obtain unbiased estimates for treatment effects, but residual confounding could still persist. We hypothesized that if the benefit of chemotherapy is independent of selection bias, then it should be much more pronounced for DSS than for nDSS. We investigated this by evaluating the association of chemotherapy with non-CC mortality. Our data showed a stronger benefit of chemotherapy for nDSS (HR=0.66; p=0.021) than for DSS (HR=0.77; p=0.105), especially in younger patients (HR=0.33; p=0.001). Despite the strong overlap of the distribution of PS observed in the treatment groups and the estimates from the PS models being substantially lower than those from the covariate adjustment models, selection bias was still observed and affected the treatment effect for OS. Among the chemotherapy non-recipients, the majority died from causes other than CC (58%). Upon accounting for competing events, the association for nDSS was attenuated and lost statistical significance (HR=0.73) but was still stronger than for DSS (HR=0.83). Clinicians and researchers should thus be aware that even after comprehensive account for confounding with PS methods, substantial treatment selection bias can be observed in observational studies leading to an overestimation of treatment effects. Treatment selection bias or confounding by indication might therefore not be possible to be resolved by PS analysis.³⁸

For the first time, we have investigated the extent to which comorbidities and FS might jointly impact treatment decisions in both younger and older CC and RC patients. Information on treatment was collected prospectively, and comorbidities were assessed around the time of CRC diagnosis, thereby minimizing potential reverse causality issues. Likewise for the first time, we have estimated the extent to which lower comorbidity and higher FS might explain the benefit of chemotherapy in stage III CC patients, using robust PS methods. Our study also has limitations. Although we made extensive efforts to include patients of all ages, the overall recruitment rate of patients in the study region was approximately 50%, with higher rates in younger patients. However, unlike many previous studies, our study had no upper age limit, which has merits when evaluating the impact of comorbidity on treatment patterns. We restricted our analytic sample to patients recruited in hospitals that reported FS data for at least 75% of the patients. However, it is worth mentioning that only 23 (5.1%) patients were excluded, and they were not different from the included sample in terms of mean age (72.3 vs 69.9; p=0.287) or proportion with comorbidities (47.8% vs 46.1%; p=0.868).

Conclusion

Lower comorbidity and higher FS explain approximately 40% of the benefit of chemotherapy in stage III CC patients. Previous observational studies most likely overestimated the benefit of chemotherapy, possibly due to less comprehensive ascertainment of comorbidities or lack of information on FS. Regardless of how robust PS analyses might be in observational studies, treatment selection bias might persist and affect estimates of treatment effects.