A validated algorithm to identify recurrence of bladder cancer: a register-based study in Denmark

Introduction

The number of cancer survivors is rising. This development is due to an aging population and improved diagnostics and treatments.^1,2 In 2012, the age-standardized incidence rate of bladder cancer (BC) in Denmark was 35.2 per 100,000 for males and 10.7 per 100,000 for females (European standard).³ In 2017, the annual incidence of BC in Denmark was 2,155; 1,048 (49%) of these were diagnosed with invasive tumor stage > Ta.⁴ BC recurrence rates in the range of 20–60% have been reported; these depend on age, comorbidity, and cancer stage.^5,6

Insight into the patient pathway in the period before cancer recurrence is essential when caring for BC survivors.^7–9 Nevertheless, the research in this field is sparse. It has proved challenging to identify patients with cancer recurrence from health registers as cancer recurrence is not routinely registered in most patient registries. No studies have reported results on the identification of BC recurrence, but some studies have reported on identification of recurrence of other types of cancer.^10–17 These studies were based on Medicare claims or other administrative data, and primary indicators of recurrence were new cancer treatment and malignant diagnosis. Gold standards included manual chart abstractions,^12,13,16 clinical trial data,¹¹ and other administrative data.^14,15,17 Several limitations have been raised regarding these studies, such as moderate performance, infrequent recurrences, and small sample sizes originating from single institutions and academic centers, where the coding was thought to be more complete and accurate.⁹ Moreover, unqualified validation inferred a risk of selection bias.^10,15

A recent Danish study validated an algorithm to identify patients with recurrence of colorectal cancer in the Danish national health registers, and this algorithm displayed high sensitivity (SEN) and specificity (SPE).¹⁸ Danish health registers may thus provide complete and valid data to identify patients diagnosed with cancer recurrence. Yet, similar studies are warranted for other cancer types.

This study aimed to develop and validate a register-based algorithm to identify patients diagnosed with recurrence of BC in a population of curatively treated invasive BC patients in Denmark. Furthermore, the study aimed to analyze the accuracy of the date of the cancer recurrence diagnosis as derived from the algorithm.

Materials and methods

This study was conducted as a cohort study based on the data from Danish national health registers. Data were linked at a personal level through the unique personal registration number (Danish civil registration number, CPR), which is assigned to all Danish citizens at birth or immigration.¹⁹

Data sources

The following five Danish national registers were used to extract data: 1) the Danish Civil Registration System (CRS) holds information on vital status and migration, and the CRS is updated daily.¹⁹ 2) The Danish Bladder Cancer Database (DaBlaCa-data) holds information on all patients diagnosed with invasive BC in Denmark since 2012.²⁰ When a patient recorded in the database dies, hospital charts, pathology results, and radiology descriptions are manually reviewed to determine whether the patient had BC recurrence at any time. 3) The Danish Cancer Register (CAR) holds data on the date of diagnosis and tumor stage for all incident cancer diagnoses in Denmark since 1943.²¹ Diagnoses and diagnosis dates in the DaBlaCa-data and the CAR are defined by algorithms based on diagnosis codes and dates from the Danish National Patient Register (DNPR) and relevant pathology test results in the Danish National Pathology Register (NPR). The tumor stage variable in the CAR is automatically imported from the DNPR. 4) The DNPR holds information on all information on all somatic inhospital contacts since 1977 and all emergency department and outpatient specialty contacts since 1995.²² The DNPR includes data on tumor stage, procedures performed, and diagnosis codes for all cancer-related contacts. 5) The Danish NPR holds information on all pathology specimens analyzed in Denmark since 1997.²³ The NPR uses Systematized Nomenclature of Medicine (SNOMED) classification,²⁴ which allows for the identification of malignant morphology (codes: M8 and M9). The fifth digit of the morphology code denotes the specimen, eg, malignant, direct spread to surrounding tissue – 4, malignant, metastasis – 6, or malignant, recurrent – 7.

Time of data extraction and end of follow-up was in October 2016.

Study population

The study base was patients registered with a BC diagnosis according to the International Classification of Diseases, 10^th revision (ICD-10) (code: C67*) in the DaBlaCa-data during 2012–2014 with tumor stage >Ta and curatively intended radical treatment. We included patients deceased by the time of data extraction in October 2016, as we only had information on recurrence status on the deceased. Patients were eligible if registered in the DNPR with a cystectomy procedure code or more than 15 fractions of radiotherapy. Patients were excluded if registered in the CAR with a previous cancer (except for nonmelanoma skin cancer) or registered in the CAR or the DNPR with distant metastasis within 90 days of cystectomy or the first day of radiotherapy (ICD-10 codes: C78, C79, CxxxM, or distant tumor stage based on the TNM classification).²⁵ Furthermore, patients were excluded if tumor stage data were missing, except for patients who received cystectomy as primary cancer treatment. Cystectomy is generally performed with curative intent, ie, in patients with no distant metastases. Finally, patients below the age of 18 years and patients who emigrated or died within 90 days of the first day of cancer treatment were excluded.

Gold standard

Status and date of recurrence diagnosis as recorded in the DaBlaCa-data were considered the gold standards of cancer recurrence.

Algorithm

A patient’s date of the end of treatment was defined as the date of cystectomy or the date of the last fraction of radiotherapy (Figure 1). To ensure that patients were in remission, no ongoing disease was allowed during the 90 days after cystectomy and the 180 days after radiotherapy (the disease-free period). Indicators of ongoing disease were: 1) a SNOMED registration with M8–M9 morphology, 2) a radiotherapy or chemotherapy procedure code with malignant indication (ICD-10: C00-C96 and D37-D48), 3) a cystectomy procedure code, 4) a distant tumor stage based on the TNM registration, or 5) a malignant diagnosis code (ICD-10: C00-C96 and D37-D48). Exemptions were diagnosis codes of BC and nonmelanoma skin cancer. We allowed for malignant pathology in the prostate and diagnosis codes of prostate cancer; this approach was taken because many male patients with BC are diagnosed with prostate cancer synchronously with BC with no influence on the BC treatment or the patient prognosis.²⁶

Figure 1 Schematic overview of the algorithm. Notes: aCystectomy or >15 fractions of radiotherapy. bNinety days after cystectomy and 180 days after the last fraction of radiotherapy.

After the disease-free period, the cancer recurrence surveillance period commenced. Indicators of cancer recurrence in the algorithm were identified through diagnostic and procedural codes in the DNPR and test results from the NPR. To achieve the best performing algorithm, different versions with varying numbers of indicators were tested. The best performing algorithm was as follows: a patient was considered to have a diagnosis of cancer recurrence if one of the following indicators was present: 1) registration in the DNPR of an ICD-10 diagnosis code of BC recurrence (C679X) or local recurrence from cancer in the urinary system (C689X), 2) registration in the DNPR of an ICD-10 diagnosis code of a distant metastasis (C76-C79 and CxxxM), 3) registration in the DNPR of radiotherapy or chemotherapy procedures with a diagnosis code of BC (C67*), metastases (C76-C79 and CxxxM), or cancer recurrence (C679X/C689X), 4) registration in the NPR of SNOMED morphology codes M8 or M9 with 4, 6, or 7 in the fifth digit and a morphology similar to a morphology code registered in the NPR within 90 days of the primary BC diagnosis or the first day of primary BC treatment, or 5) registration in the NPR of SNOMED codes M8 or M9 with 7 in the fifth digit, regardless of the type of histology.

If a second primary cancer diagnosis was present in the CAR or the DNPR before or within 30 days of a metastasis diagnosis code, such metastasis diagnosis code was disregarded as it could be related to the second primary cancer. If more than one indicator of cancer recurrence was present in the same patient, the first date of an indicator of recurrence was listed as the date of cancer recurrence. Results from this algorithm are presented in the “Results” section.

Statistical analyses

The concordant and discordant frequencies between recurrences identified by the algorithm and by the gold standard were listed to compute the SEN, SPE, positive predictive value (PPV), and negative predictive value (NPV) with 95% CI. Cohen’s k coefficient test for agreement was calculated.

The strength of the agreement between the date of recurrence identified by the algorithm and the gold standard was measured by Lin’s²⁷ concordance correlation coefficient (CCC). In line with the literature, the agreement was considered “poor” for the CCC of <0.90, “moderate” for the CCC of 0.90–0.95, “substantial” for the CCC >0.95, and almost perfect for the CCC of >0.99.²⁸

All analyses were performed for the total population and stratified by primary cancer treatment regime, cystectomy, and radiotherapy. Data were analyzed using the statistical software Stata 15.0 (StataCorp LP, College Station, TX, USA).

Results

At the time of data extraction in October 2016, 286 patients fulfilled the inclusion criteria (Figure 2). We excluded 57 patients registered with a new cancer event before the first date of the cancer recurrence surveillance period. Furthermore, 42 patients who died or were registered with cancer-related treatment in the same period were excluded. This left 187 patients in the final study population; 128 (68%) of these were registered with cancer recurrence in the DaBlaCa-data. The characteristics of the study population stratified on cancer recurrence status in the DaBlaCa-data are presented in Table 1.

Figure 2 Flowchart of study population. Abbreviations: BC, bladder cancer; DaBlaCa-data, Danish Bladder Cancer Database.

Table 1 Characteristics of included patients stratified on cancer recurrence status in the DaBlaCa-data (n=187) Notes: aNumbers are n (%) if nothing else is stated. bData are masked for data privacy for less than five observations. Abbreviations: CCI, Charlson’s comorbidity index; DaBlaCa-data, Danish Bladder Cancer Database; IQR, interquartile range.

The algorithm identified 109 of the 128 patients registered with a diagnosis of recurrence according to the gold standard and additionally six false-positive (FP) diagnoses (Table 2). The algorithm yielded an SEN of 85% (95% CI: 78–91), an SPE of 90% (95% CI: 79–96), a PPV of 95% (95% CI: 89–98), and an NPV of 74% (95% CI: 62–83) (Table 3). The algorithm demonstrated superior performance in the cystectomy population (results are not displayed due to data protection in the case of less than five observations). The performance varied across the individual indicators of recurrence. The pathology indicators were superior and reached an SEN of 48% and a PPV of 98% and in patients treated with cystectomy, an SEN of 57% and a PPV of 100%.

Table 2 Concordance of bladder cancer recurrence identified by the gold standard and the algorithm

Table 3 Performance of the algorithm for bladder cancer recurrencea Note: aNumbers are percentages (95% CI) if nothing else is stated. Abbreviation: CI, confidence interval

Seven of the patients registered with recurrence in DaBlaCa-data had no record of the recurrence date. This left 102 patients for the analysis of concordance between recurrence date in the DaBlaCa-data and recurrence date generated by the algorithm. The CCC was 0.96 (95% CI: 0.95–0.98), which indicates substantial concordance (Figure S1 and Table S1). The recurrence date was estimated within 30 days of the gold standard date in 64% of cases and within 90 days in 90% of cases.

Discussion

Main findings

We developed and validated a register-based algorithm to identify patients with recurrence of invasive BC in Denmark. To the best of our knowledge, this is the first study to develop a method to identify patients with recurrence of BC. The algorithm demonstrated an SEN of 85% and a PPV of 95%. The algorithm performance was substantially superior in patients undergoing cystectomy compared to patients undergoing radiotherapy as primary cancer treatment. The analyses indicated substantial agreement between recurrence dates; the estimated date was set within 90 days of the gold standard date for 90% of patients.

Strengths and limitations

This study has several important strengths. First, the algorithm is based on Danish national registers with complete high-quality records, which ensure high data validity.^19,21–23 Second, the tax-funded public health care system in Denmark provides free and equal access for all citizens. Both the high validity and the free access to health care reduce the risk of selection bias.

Another strength is that the gold standard originated from a population-based database. This ensures higher generalizability to the entire population of BC patients compared to using a gold standard population identified from a randomized controlled trial, which may induce the risk of a restricted (younger and healthier) population with more uniform pathways compared to the background population.²⁹

The use of malignant pathology test results and a specific diagnosis code of BC recurrence improved the performance of the algorithm, and these two indicators reached a PPV of 100% in patients treated with cystectomy as primary cancer treatment. Using SNOMED (pathology) codes as an indicator of recurrence is known to increase the performance in similar algorithms.¹⁸ In addition, we found that the most accurate indicator to estimate the recurrence date was pathology results.

The primary limitation of the study is the risk of misclassification of recurrence and recurrence dates, which could result from missing or incorrect registrations.

Missing data are of less concern, as patients with recurrence of BC are likely to be in contact with a hospital. Cystectomy is a major surgical procedure with high risk of postoperative complications and death among older and comorbid patients.^5,29,30 Patients initially treated with radiotherapy are suspected to be older, to have more comorbidity, and to be more fragile at initial treatment. At the time of recurrence, their performance status may have deteriorated further and thus contraindicate biopsy and cancer treatment, which may explain a poorer performance of the algorithm in patients initially treated with radiotherapy. Eleven of the 19 false negatives were patients initially treated with radiotherapy, corresponding to 49% of all patients undergoing radiotherapy with cancer recurrence according to the gold standard. In the cystectomy patients, only 8% of recurrences were missed by the algorithm. The median age by the time of cancer recurrence was 76.0 years in the false negatives, 7.3 years older than the median age in the true positives. This infers a risk of biased results in studies using the algorithm to define the study population.

To enable us to identify patients in remission after initial therapy, no indications of ongoing BC disease were allowed for a period of 90 and 180 days in patients treated with primary cystectomy and radiotherapy treatment, respectively. These cutoffs may cause an early recurrence of cancer to be misclassified as ongoing disease, which leads to the exclusion of the patients. Furthermore, a patient who did not receive successful cancer treatment for primary BC and had, eg, malignant pathology test results after a period of 90/180 days with no indication of ongoing disease would be misclassified as a patient with recurrence by the algorithm, but this scenario is considered to be rare. Finally, a new primary cancer may be misclassified as BC recurrence as certain distinctions between cancer recurrence and new primary cancer may require analysis of tumor molecular features.

Most of the six FP patients had a metastasis diagnosis as the indicator of recurrence. When omitting metastasis diagnoses from the algorithm, the SEN declined and the PPV increased (results not reported due to data protection regulations). Overall, one could question if the six FP cases of recurrence identified by the algorithm were BC recurrences, secondary primary diseases, or true FPs. None of the FPs had registrations of a second primary cancer in the CAR or the DNPR before or within 30 days of the recurrence indicator. However, most FPs had two or more indicators of recurrence, including metastasis diagnosis codes, multiple radiotherapy registrations, malignant biopsy test results, and (besides the indicators of recurrence) frequent BC diagnosis codes in the DNPR and concurrent registration of palliative care with BC as co-diagnosis, which indicates that these patients may have recurrence missed by the gold standard.

We included patients undergoing cystectomy or radiotherapy in the study to ensure a population of patients undergoing intended curative cancer treatment for primary BC. Some patients are curatively treated by one or several consecutive transurethral resections of the bladder (TUR-B). The registration of the TUR-B procedure does not provide information about when a patient is considered to be curatively treated or whether a TUR-B is part of the initial therapy or of recurrence therapy. The algorithm did not perform well in these patients, and we chose not to include patients initially treated by a TUR-B in the study although this implies that this specific group of patients will be excluded from future studies based on populations generated by the algorithm.

Comparison with other studies

Direct comparison with other studies of BC recurrence detected through health registers is not possible as we have not been able to identify such studies. Yet, studies report on register-based identification of patients with recurrence after other types of cancer. In a recent Danish study, Lash et al¹⁸ identified colorectal cancer recurrence in patients undergoing surgery for primary cancer. Their algorithm was based on the same health registers as the present study and reached an SEN of 95% (95% CI: 87–99), an SPE of 97% (95% CI: 94–98), and a PPV of 86% (95% CI: 75–93). This performance was similar to the performance of the algorithm in the present study for patients undergoing surgery, although our PPV was higher. In the previous study,¹⁸ the gold standard comprised data from a cohort study based on a single-center hospital and a randomized controlled trial with age- and comorbidity-related exclusion criteria.^31,32 This infers a risk of a uniform population of younger and healthier patients with more complete registrations, and the results may not be comparable to those in a population-based cohort. In the present study, the gold standard was adopted from a population-based database with no restrictions for age or comorbidity.⁴ Despite different gold standard populations in the two Danish studies, the two algorithms yielded comparable performance. Hence, the bias that could be introduced by a selected gold standard population may not be of great concern when the algorithm is based on complete and valid registers and a population with free and equal access to health care services.

In studies from USA,^10–17 routine registration and platforms may well be different from a Danish setting, which complicates direct comparison. Hassett et al¹⁴ reported that the recurrence date produced by their algorithm was within 30 days of the gold standard date in 20–36% of the cases and within 6 months in 80% of the cases of breast and colorectal cancer recurrence. We managed to identify the recurrence date more accurately; 64% of cases were within 30 days of the gold standard recurrence date, and 84% were within 60 days. Including pathology as an indicator of recurrence may explain why our algorithm was more accurate in estimating the recurrence date; the date of pathology test results estimated the recurrence date most accurately. Despite a more precise estimation of the recurrence date in the present study, future research based on a study population identified by the algorithm should consider potential bias caused by the estimation of the recurrence date.

Previous studies identifying patients with cancer recurrence from register-based algorithms have included only patients known from clinical trial information to have completed definitive locoregional therapy for the incident cancer disease.^11,14,16 The algorithm is not applicable to the background population if such information is not available in routine data. Other studies have included patients treated with surgery.^10,15,18 Compared to a course of radiotherapy treatment, a single surgical procedure may be easier to identify in the registers and to distinguish from a palliative treatment regimen. Hence, the inferior performance of the algorithm in patients treated with radiotherapy may reflect difficulties with identifying a population of curatively treated patients and the date of completed therapy.

Implications

There is an increasing need for research in populations with cancer recurrence.^7–9 Worldwide, researchers are faced with challenges in identifying patients with cancer recurrence for population-based research. The algorithm developed in this study is a valuable instrument that is directly reusable in future research and will enable nationwide epidemiological register-based research on BC recurrence in Denmark. The major advantage of linking data in Danish health registers at the personal level is that it enables enrichment of populations sampled by cancer registry data with registrations from hospital admissions, and this provides great prospects for future epidemiological research in BC recurrence. Furthermore, the algorithm allows identification of patients who have recently been diagnosed with BC recurrence, which holds new opportunities for gaining knowledge on patient experiences throughout the entire trajectory from primary BC disease to BC recurrence.

Conclusion

We developed an algorithm to identify BC recurrence through routinely collected data in Danish health registers. The algorithm showed excellent performance, including high SEN and SPE, especially in patients treated with cystectomy for primary BC. Moreover, the algorithm accurately estimated the recurrence date. The algorithm has the potential to serve as a high resource for future research in the field of BC recurrence.

Abbreviations

BC, bladder cancer; CaP: the Research Centre for Cancer Diagnosis in Primary Care; CAR: Danish Cancer Register; CCC: concordance correlation coefficient; CPR: Danish civil registration number; CRS: civil registration system; DaBlaCa-data: Danish Bladder Cancer Database; DNPR: Danish National Patient Register; FP: false positive; ICD-10: International Classification of Disease, 10^th revision; NPR: National Pathology Register; NPV: negative predictive value; PPV: positive predictive value; SEN: sensitivity; SPE: specificity; TUR-B: transurethral resection of the bladder.

Ethics approval and informed consent

The patient pathway for cancer recurrence research project (Study 1, id 119) has been approved and is registered in the Record of Processing Activities at the Research Unit of General Practice in Aarhus in accordance with the provisions of the General Data Protection Regulation (GDPR). This type of study did not require approval from the Regional Committee on Health Research Ethics as the study did not involve human biological material.

Data availability

The data that support the findings of this study are stored and maintained electronically at Statistics Denmark. Data can only be accessed by approved collaborative partners via a secured virtual private network (VPN). The data are not publicly available due to the Danish data protection legislation as the data contain information that could compromise the privacy of the research participants.

Acknowledgments

The DaBlaCa-data provided data for the study. This study was supported by research grants from the Danish Cancer Society and Aarhus University. The funders did not have any influence on any aspects of the study (from design to submission of the paper for publication).

Author contributions

LAR contributed to the conception and design of the study, performed the data acquisition and the statistical analyses, contributed to data interpretation, and drafted the manuscript. LFV, HJ, JBJ, and PV contributed to the conception and design of the study, data interpretation, and provided critical revision of the intellectual contents of the manuscript. All authors contributed toward data analysis, drafting and revising the paper and agree to be accountable for all aspects of the work.

Disclosure

The authors report no conflicts of interest in this work.

References

1.		Hovaldt HB, Suppli NP, Olsen MH, et al. Who are the cancer survivors? A nationwide study in Denmark, 1943-2010. Br J Cancer. 2015;112(9):1549–1553.
2.		The National Board of Health [webpage on the Internet]. Nye kræfttilfælde i Danmark 2016 [Cancer incidence in Denmark 2016]; 2017. Available from: https://sundhedsdatastyrelsen.dk/da/tal-og-analyser/analyser-og-rapporter/sygdomme/cancerregisteret. Accessed June 4, 2018.
3.		Ferlay J, Steliarova-Foucher E, Lortet-Tieulent J, et al. Cancer incidence and mortality patterns in Europe: estimates for 40 countries in 2012. Eur J Cancer. 2013;49(6):1374–1403.
4.		The Danish Bladder Cancer Group. Årsrapport 2017 [Annual report 2017]; 2018. Available from: https://www.sundhed.dk/content/cms/86/15686_dablacadata_-aarsrapport_2017_v2_endelig.pdf. Accessed June 4, 2018.
5.		Guancial EA, Roussel B, Bergsma DP, et al. Bladder cancer in the elderly patient: challenges and solutions. Clin Interv Aging. 2015;10:939–949.
6.		Stanik M, Poprach A, Macik D, et al. Clinically node-positive bladder cancer: oncological results of induction chemotherapy and consolidative surgery. Neoplasma. 2018;65(2):287–291.
7.		Rubin G, Berendsen A, Crawford SM, et al. The expanding role of primary care in cancer control. Lancet Oncol. 2015;16(12):1231–1272.
8.		Soerjomataram I, Coebergh JW. Epidemiology of multiple primary cancers. Methods Mol Biol. 2009;471:85–105.
9.		Warren JL, Yabroff KR. Challenges and opportunities in measuring cancer recurrence in the United States. J Natl Cancer Inst. 2015;107(8).
10.		Cummings KC, Xu F, Cummings LC, Cooper GS. A comparison of epidural analgesia and traditional pain management effects on survival and cancer recurrence after colectomy: a population-based study. Anesthesiology. 2012;116(4):797–806.
11.		Lamont EB, Herndon JE, Weeks JC, et al. Measuring disease-free survival and cancer relapse using Medicare claims from CALGB breast cancer trial participants (companion to 9344). J Natl Cancer Inst. 2006;98(18):1335–1338.
12.		Mcclish D, Penberthy L, Pugh A. Using Medicare claims to identify second primary cancers and recurrences in order to supplement a cancer registry. J Clin Epidemiol. 2003;56(8):760–767.
13.		Earle CC, Nattinger AB, Potosky AL, et al. Identifying cancer relapse using SEER-Medicare data. Med Care. 2002;40(8 Suppl):IV-75–81.
14.		Hassett MJ, Uno H, Cronin AM, Carroll NM, Hornbrook MC, Ritzwoller D. Detecting lung and colorectal cancer recurrence using structured clinical/administrative data to enable outcomes research and population health management. Med Care. 2017;55(12):e88–e98.
15.		Warren JL, Mariotto A, Melbert D, et al. Sensitivity of Medicare claims to identify cancer recurrence in elderly colorectal and breast cancer patients. Med Care. 2016;54(8):e47–e54.
16.		Deshpande AD, Schootman M, Mayer A. Development of a claims-based algorithm to identify colorectal cancer recurrence. Ann Epidemiol. 2015;25(4):297–300.
17.		Chawla N, Yabroff KR, Mariotto A, Mcneel TS, Schrag D, Warren JL. Limited validity of diagnosis codes in Medicare claims for identifying cancer metastases and inferring stage. Ann Epidemiol. 2014;24(9):e1-2:666–672.
18.		Lash TL, Riis AH, Ostenfeld EB, Erichsen R, Vyberg M, Thorlacius-Ussing O. A validated algorithm to ascertain colorectal cancer recurrence using registry resources in Denmark. Int J Cancer. 2015;136(9):2210–2215.
19.		Schmidt M, Pedersen L, Sørensen HT. The Danish Civil Registration System as a tool in epidemiology. Eur J Epidemiol. 2014;29(8):541–549.
20.		Hansen E, Larsson H, Nørgaard M, Thind P, Jensen JB. The Danish Bladder Cancer Database. Clin Epidemiol. 2016;8:439–443.
21.		Gjerstorff ML. The Danish Cancer Registry. Scand J Public Health. 2011;39(7 Suppl):42–45.
22.		Schmidt M, Schmidt SA, Sandegaard JL, Ehrenstein V, Pedersen L, Sørensen HT. The Danish National Patient Registry: a review of content, data quality, and research potential. Clin Epidemiol. 2015;7:449–490.
23.		Erichsen R, Lash TL, Hamilton-Dutoit SJ, Bjerregaard B, Vyberg M, Pedersen L. Existing data sources for clinical epidemiology: the Danish National Pathology Registry and Data Bank. Clin Epidemiol. 2010;2:51–56.
24.		SNOMED International [homepage on the Internet]. SNOMED International. Available from: https://www.snomed.org/. Accessed May 4, 2018.
25.		Holland-Bill L, Frøslev T, Friis S, et al. Completeness of bladder cancer staging in the Danish Cancer Registry, 2004-2009. Clin Epidemiol. 2012;4(Suppl 2):25–31.
26.		Mazzucchelli R, Barbisan F, Scarpelli M, et al. Is incidentally detected prostate cancer in patients undergoing radical cystoprostatectomy clinically significant? Am J Clin Pathol. 2009;131(2):279–283.
27.		Lin LI. A concordance correlation coefficient to evaluate reproducibility. Biometrics. 1989;45(1):255–268.
28.		McBride GB. A Proposal for Strength of Agreement Criteria for Lin’s Concordance Correlation Coefficient. Hamilton, New Zealand: National Institute of Water & Atmospheric Research; 2005.
29.		Galsky MD. How I treat bladder cancer in elderly patients. J Geriatr Oncol. 2015;6(1):1–7.
30.		Jensen BT, Petersen AK, Jensen JB, Laustsen S, Borre M. Efficacy of a multiprofessional rehabilitation programme in radical cystectomy pathways: a prospective randomized controlled trial. Scand J Urol. 2015;49(2):133–141.
31.		Hansdotter Andersson P, Wille-Jørgensen P, Horváth-Puhó E, et al. The COLOFOL trial: study design and comparison of the study population with the source cancer population. Clin Epidemiol. 2016;8:15–21.
32.		Stender MT, Frøkjaer JB, Larsen TB, Lundbye-Christensen S, Thorlacius-Ussing O. Preoperative plasma D-dimer is a predictor of postoperative deep venous thrombosis in colorectal cancer patients: a clinical, prospective cohort study with one-year follow-up. Dis Colon Rectum. 2009;52(3):446–451.

Supplementary materials

Table S1 Agreement between the date of recurrence of bladder cancer identified by the algorithm and the gold standard Abbreviation: CI, confidence interval.

Figure S1 Discordance between gold standard and algorithm on recurrence date.