Identifying Recurrences Among Non-Metastatic Colorectal Cancer Patients Using National Health Data Registries: Validation and Optimization of a Registry-Based Algorithm in a Modern Danish Cohort

Introduction

Colorectal cancer (CRC) represents a significant health burden worldwide. At the time of diagnosis, about 75% of the patients present with non-distant metastatic disease (UICC TNM stage I–III) and are eligible for surgery with curative intent.¹ Still, an estimated ~25% of the patients experience recurrence. Between 70% and 90% of the recurrences are detected within the first 3 years after surgery.^2–9

In Denmark and many other countries, CRC recurrence is not recorded routinely in clinical databases or nationwide healthcare registries. Therefore, Lash et al developed and validated an algorithm combining data from different Danish registries to identify CRC recurrences for the period 2001–2011.¹⁰ The algorithm detected recurrence with a sensitivity of 95%, specificity of 97%, positive-predictive value of 86%, and negative predictive value of 99%.¹⁰ However, the predictive performance of the algorithm may have changed since 2011 because of improvements in care and, consequently, changes in recurrence risk. Changes in care include, eg, the implementation of central vascular ligation, complete mesocolic excision, increased lymph node yield,¹¹ implementation of population-based CRC screening,¹² and standardization of diagnostic CT-imaging.¹³ In addition, the algorithm has not previously been validated with regard to time to recurrence (TTR). TTR is important when using the algorithm for time-to-event analysis. Finally, because the algorithm includes codes for chemotherapy that are also used outside the oncology setting (for example, methotrexate treatment for rheumatoid arthritis), further optimization of the algorithm might be possible.

Therefore, the primary aim of this study was to estimate recurrence rates in a modern cohort of UICC TNM stage I–III CRC patients and to validate the CRC recurrence algorithm using medical chart review as a reference standard on a subset of this cohort. The secondary aims were to evaluate the accuracy of the algorithm for estimating TTR and to evaluate the potential of improving the performance of the algorithm.

Materials and Methods

Data Sources and Data Collection

Data from the period January 2012 to December 2018 were obtained from the following Danish health and medical registries,¹⁴ the Danish Colorectal Cancer Group (DCCG) database,^15,16 the Danish Cancer Registry (DCR),¹⁷ the Danish National Registry of Patients (DNPR),¹⁸ and the Danish Pathology Registry (DPR).¹⁹ Individual-level records were linked using the unique 10-digit civil registration number issued to each Danish resident by the Danish Civil Registration System (CRS).^20,21 Data from 2019 and onwards were not available to us due to implementation of a temporary new version of DNPR (LPR3) in 2019.

The DCCG database provided information on the date of incident CRC diagnosis, date of surgery, and patient characteristics. The DNPR contains administrative and clinical data and provided information on chemotherapeutic treatment and diagnoses of metastasis. The DCR contains incidences of malignant neoplasms and provided information on previous diagnoses of cancer.

Cohort

The study cohort was defined as individuals who consented to participate in the CRC biobank at the Department of Molecular Medicine, Aarhus University Hospital, Denmark, with available data in the DCCG database (Figure 1). All patients had to be ≥18 years of age with DCCG-registered date of surgery for UICC TNM stage I–III CRC between January 2012 and December 2017 at a Danish hospital. Patients with a diagnosis of CRC or metastases in the DNPR or the DCR more than 180 days before the date of debut of CRC were excluded. Patients with a diagnosis of cancer different from CRC (except non-melanoma skin cancer, NMSC), or metastases with unknown origin at any time before the date of CRC debut were also excluded.

Figure 1 Identification of study cohort in a flowchart. All patients included in the biobank at the Department of Molecular Medicine, Aarhus University Hospital, Denmark, between January 2012 and December 2018, were identified and included. The cohort was linked by unique 10-digit civil registration number with data from Danish health and medical registries (Danish Colorectal Cancer Group database, the Danish Cancer Registry, the Danish National Registry of Patients, and the Danish Pathology Registry) and used by the algorithm to identify CRC recurrences. A validation cohort was selected for medical chart review to identify recurrences. BIOBANK = CRC biobank at the Department of Molecular Medicine, Aarhus University Hospital, Denmark. Abbreviations: DCCG, Danish Colorectal Cancer Group; CRC, colorectal cancer.

A subcohort was selected for validation of the algorithm performance following two steps: 1) an existing cohort of patients with follow-up from prospective clinical studies^22,23 (n=322) and 2) an additional random sample of 200 UICC stage I–II patients. One investigator (JN) did the final medical chart review of all patients in the validation cohort (n=522) to ensure consistent definition of recurrence.

Recurrence Detection by Algorithm

The algorithm used to identify recurrences has previously been described in detail.¹⁰ We defined the date of recurrence as the earliest date of a code/combination leading to fulfillment of at least one of the following four criteria (Figure S1):

1. Metastases: DNPR-registered or DCR-registered metastases codes (ICD-10 DC76-DC80) 180 or more days after CRC surgery without a new primary cancer diagnosis other than CRC (except NMSC) between time of CRC surgery and time of metastases.
2. Chemotherapy: DNPR-registered treatment codes (BWHA1-2, BOHJ17 or BOHJ19B1) at least 180 days after CRC surgery and 60 or more days after last chemotherapy treatment code without a new primary cancer diagnosis other than CRC (except NMSC).
3. Pathological assessment of tissue: DPR-registered SNOMED combinations recorded at least 180 days after CRC surgery and without a new primary cancer diagnosis other than CRC (except NMSC).

 Combinations included

  3.1 T-code in the colon or rectum (T6491, T65900, T65901. T65925, T65926, T660, T67 or T68) with morphology codes M8 or M9 with three or more in the fifth position (M8XXX3).

  3.2 Any T-code with morphology codes M8 or M9 with the digits 4, 6 or 7 in the fifth position.

  3.3 Liver adenocarcinoma: T56 in combination with M81403, which we interpreted as representation of a liver metastasis.

4. Diagnosis of local recurrence: DNPR-registered CRC local recurrence codes (DC189X or DC209X).

Medical Chart Review-Based Recurrence Detection

CRC recurrences were defined as (1) tumor growth at or near the site of the original tumor and in the same organ (colon or rectum) or (2) metastases to tissue adjacent to the original tumor site or to a distant organ. The recurrence date identified through clinical follow-up by electronic medical chart review was defined as the date of the first determinate statement of CRC recurrence in either 1) the patient record, 2) radiological imaging descriptions, 3) pathological assessments of tissue, or 4) a multidisciplinary team conference decision.²⁴ Patients were followed up for recurrence until December 31, 2018, in both registries and medical records.

Statistical Analysis

The frequency and proportion of patients in the study and validation cohorts were calculated and presented within categories of gender, age, comorbidity, year of surgery and disease stage. Categorical variables were given as number of total (n/N). Continuous variables were reported as median with inter quartile range (IQR).

The cumulative incidence curves of recurrence, based on the algorithm, were constructed for the study cohort stratified by stage to evaluate the external validity of the CRC biobank.

The agreement between the algorithm and our medical chart review for the validation cohort were computed with kappa statistics, and the sensitivity, specificity, positive predictive value, and negative predictive value for the registry-based algorithm were calculated from a contingency table. Patients with inconsistent recurrence status between the algorithm and medical chart review were investigated further and reported in a descriptive manner.

The TTR was calculated as days between the date of CRC surgery and date of recurrence. The correlation coefficient between TTR_{Medical_chart} and TTR_algorithm was calculated, and the difference was reported as the median and IQR (TTR_{Medical_chart} - TTR_algorithm). Recurrence rates were calculated and reported as 1- and 3-year cumulative incidence proportions within each UICC stage treating death as competing risk.

Statistical analyses were carried out using R version 4.1.3 (https://www.r-project.org/) and RStudio v.2021.9.1.372 (RStudio, PBC, Boston, MA, http://www.rstudio.com/). Sensitivity, specificity, and predictive values of the algorithm were calculated using the R package “epiR” version 2.0.39 (https://CRAN.R-project.org/package=epiR) with recurrence status from medical chart review as reference standard.

This study was reported according to The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) Statement.²⁵

Ethics

Informed consent to participate in the biobank had previously been obtained from all participants. The study was approved by the National Committee on Health Research Ethics (J. no. 1704081). The CRC biobank has been approved by the Danish Data Protection Agency (j. no. 1-16-02-27-10).

Results

In the period 2012–2017, the merging between the biobank and the DCCG database identified 1468 UICC TNM stage I–III CRC patients (Figure 1). After exclusions, the final study cohort included 1129 UICC TNM stage I–III patients. An overview of the clinical characteristics of the study cohort is presented in Table 1.

Table 1 Descriptive Characteristics of the Study Cohort and the Validation Cohort

In the full study cohort, the algorithm identified 222 recurrences (Cohort prevalence = 20%, 95% CI: 17–22%). The cumulative incidence of CRC recurrence at 1-year was 7.9% (95% CI: 6.3–9.5%) and 20% (95% CI: 17–22%) after 3 years (Figure 2A).

Figure 2 Cumulative incidence curves for CRC recurrence and time to recurrence visualization. (A) Cumulative incidence curves for CRC recurrence within the study cohort. Curves are visualized within each UICC TNM stage treating death as a competing risk. Dashed lines represent lower and upper 95% confidence interval limits. Censoring is illustrated with “|”. No recurrences can be identified by the algorithm within 180 days from CRC diagnosis. No patients were censored within 180 days due to exclusion prior to analysis. (B) Cumulative incidence curves for CRC recurrence within the validation cohort visualized by recurrence identification method. (C) Time to recurrence (TTR) by algorithm vs medical chart review. Datapoints represent the 75 patients with recurrence detected both by the algorithm and through medical chart review. Colored by difference in TTR. The two standard-of-care surveillance CT scans for Danish CRC patients at 12 and 36 months postoperative are annotated. The association was assessed using Spearman correlation. (D) Difference in TTR by algorithm vs medical chart review around postoperative day 180, which is the earliest possible TTR by the algorithm. (E) Difference in TTR by algorithm vs medical chart review at 12 months postoperative. Long dashed and dashed line shows TTR by medical chart review +2 and +4 weeks, respectively. Three recurrences are not registered by the algorithm before +6 weeks from date of recurrence in medical chart (dotted line). (F) Difference in TTR by algorithm vs medical chart review at 36 months postoperative. Abbreviations: UICC, Union for International Cancer Control; TNM, tumor node and metastasis; TTR, time to recurrence; SNOMED, Systematized Nomenclature of Medicine.

The clinical characteristics of the validation cohort are presented in Table 1. The medical chart review identified 80 CRC patients with recurrence (cohort prevalence = 15%, 95% CI: 12–19%). The algorithm identified 86 patients with recurrence of whom 75 were true positives resulting in a sensitivity of 94% (95% CI: 86–98%) and a specificity of 98% (95% CI: 96–99%) (Table 2). The positive and negative predictive values of the algorithm were 87% (95% CI: 78–93%) and 99% (95% CI: 97–100%), respectively. Kappa statistics were strong with 0.89.

Table 2 Concordance of Recurrences Identified by a Registry-Based Algorithm and Recurrences Identified by Medical Chart Review

Characteristics of False Positives and False Negatives

The algorithm captured false recurrences in 11 patients. Two cases were new primary cancers falsely interpreted as metastases from CRC. A third case was captured because of an incorrect date of surgery in DCCG leading to interpretation of the date of primary surgery as the date of recurrence. A fourth case was captured because of a misclassified liver metastases code in the DNPR. The remaining seven cases were captured due to the algorithm not distinguishing between chemotherapy codes for treatment of medical conditions and recurrence. In all seven cases, the capture was due to methotrexate or cyclophosphamide treatment for rheumatoid arthritis or essential thrombocythemia and not due to CRC recurrence. Importantly, these treatment codes were all registered at a medical department rather than an oncology department.

The algorithm failed to capture recurrence in five patients. Two of these patients were diagnosed with recurrence based on radiological imaging in December 2018; one patient started chemotherapy in January 2019 and the other patient had a biopsy in January 2019 with a SNOMED code of lung metastasis. Due to the end of follow-up of our study on December 31, 2018, these patients appeared as false negatives in our analysis but would have been captured with longer follow-up. A third patient was diagnosed with CRC recurrence less than 180 days after primary surgery and died shortly after; the patient did not receive palliative chemotherapy in the timeframe from day 180 until death, explaining why the algorithm was unable to capture the recurrence. One patient was diagnosed with disseminated disease at the 12 months postoperative CT-scan but did not receive any further diagnostics or treatment before death. The fifth patient was diagnosed with a biopsy-verified retroperitoneal metastasis from CRC (DNPR: DC786), which was not detected due to a DNPR-registered code of Pseudomyxoma peritonei the day before biopsy-verification (counting as a new primary cancer).

Time to Recurrence (TTR)

The cumulative incidence curves generated using the algorithm and medical chart recurrences were similar (Figure 2B). Correspondingly, comparison of the TTR_algorithm and TTR_{Medical_chart} showed that they were highly correlated, R²=0.993 (Figure 2C), with the largest differences in TTR found for recurrences diagnosed clinically before day 180 (Figure 2D). The median difference between TTR_algorithm and TTR_{Medical_chart} was −8 days (IQR: −21 to +3 days). The incidence of CRC recurrence accumulated at the timepoints of standard-of-care postoperative surveillance CT-scans at 12 and 36 months. This observation, combined with 1- and 3-year cumulative recurrence rates being commonly used endpoints, prompted us to explore the correlation between TTR_algorithm and TTR_{Medical_chart} particularly at 12 and 36 months. This revealed a general but minor delay in TTR_algorithm (Figure 2E and F).

Optimization of Algorithm

In the validation cohort, a total of 28 recurrences were captured by incorporating a chemotherapy code; seven of these were false positives. One of these patients was later diagnosed with breast cancer and treated with chemotherapy. The six other captures were due to non-colorectal-cancer related chemotherapy codes originating from non-oncology departments, eg, methotrexate codes from departments of hematology. When restricting the algorithm to chemotherapy codes from oncology departments, the PPV of the algorithm increased to 94% (95% CI 86–98%) from the original 87% (95% CI 78–93%) (Table 2).

Discussion

In this study, we found a 3-year cumulative recurrence rate of 20% (95% CI: 17–22) in the total cohort. Medical chart review of our validation subgroup confirmed that the algorithm has high validity – with 94% sensitivity, 98% specificity, 87% positive-predictive value and 99% negative-predictive value. These validation parameters are compared to those reported for the algorithm on a cohort of CRC patients diagnosed 2001–2011.¹⁰ As such, the algorithm performs equally in a modern cohort.

The algorithm has previously been used to evaluate CRC recurrence in unselected populations. Holmes et al investigated CRC recurrence in 21,152 Danish patients diagnosed between 2001 and 2011 and found increasing risk of recurrence with increasing disease stage.²⁶ Our results are in line with these findings with recurrence rates increasing from 4.7% to 16% and 32% in UICC TNM stage I, II and III diseases, respectively. Another study used the algorithm to identify CRC recurrences in 4369 unselected patients operated between 2010 and 2013 and found a 5-year cumulative rate of recurrence between 22.5% and 24.3%, which is comparable to our observation.²⁷

Adjuvant treatment as a combination of fluorouracil and oxaliplatin for a duration of 6 months is recommended in the Danish guidelines for UICC TNM stage III colon cancer patients. A 180-day period was implemented in the algorithm to ensure that diagnostic work-up and adjuvant treatment was complete. The same principle has been applied in other studies reporting recurrences before postoperative day 180 as synchronous metastases.^28–30 Consequently, we observed a difference in TTR for algorithmic and medical chart recurrences for clinical recurrences before day 180, although we showed a very high correlation overall. We also demonstrated that there is a delay from clinical recurrence to capture of recurrence in health care registries. As 1- and 3-year cumulative recurrence rates are common endpoints in the literature,^31–33 this should be noted to prevent underestimation of recurrences by the algorithm. We suggest a 4-week interval around the 1- and 3-year algorithmic recurrence rates to mitigate the effects of the delay.

Our analysis of the false-positive cases revealed the CRC recurrence algorithm was limited by its inability to distinguish chemotherapeutic treatment of medical conditions (for example, methotrexate for the treatment of rheumatoid arthritis) from that of cancer recurrence. This is because the ICD-10 code “BWHA1” (Basic cytostatic treatment) is nonspecific and could potentially be used when administering prescription medication such as methotrexate. However, methotrexate is never used in the management of CRC. In Denmark, hospital departments are classified according to a standardized code system (The National Catalogue of Health Organizations).³⁴ By medical chart review, we extracted the departmental codes for all the recurrences identified from chemotherapy codes in the validation cohort. When we restricted the algorithm to only capture recurrences when the chemotherapy code originated from oncology departments, this improved the positive predictive value of the algorithm from 87% to 94%. The same approach is being used by the DCCG database for identifying the patients receiving adjuvant chemotherapy, and the DCCG report that the approach has an accuracy of 94%.³⁵

Limitations of the present study include the relatively short follow-up of our validation subgroup. Three years of complete follow-up would be preferable as 90% of all recurrences occur within this timeframe.^2–9

Conclusion

In conclusion, the algorithm detected recurrence and TTR with high precision in this contemporary cohort of patients diagnosed 2012–2017 as in the original 2001–2011 population. Restriction to chemotherapy codes from oncology departments using department classifications improves the algorithm. The algorithm is suitable for use in future observational studies.