Back to Journals » Clinical Epidemiology » Volume 13

A Validated Register-Based Algorithm to Identify Patients Diagnosed with Recurrence of Malignant Melanoma in Denmark

Authors Rasmussen LA, Jensen H, Virgilsen LF, Hölmich LR, Vedsted P

Received 4 December 2020

Accepted for publication 18 February 2021

Published 15 March 2021 Volume 2021:13 Pages 207—214

DOI https://doi.org/10.2147/CLEP.S295844

Checked for plagiarism Yes

Review by Single anonymous peer review

Peer reviewer comments 2

Editor who approved publication: Professor Irene Petersen


Linda Aagaard Rasmussen,1 Henry Jensen,1 Line Flytkjaer Virgilsen,1 Lisbet Rosenkrantz Hölmich,2,3 Peter Vedsted1

1Research Centre for Cancer Diagnosis in Primary Care (CaP), Research Unit for General Practice, Aarhus, Denmark; 2Department of Plastic Surgery, Herlev and Gentofte Hospital, Herlev, Denmark; 3Department of Clinical Medicine, Copenhagen University, Copenhagen, Denmark

Correspondence: Linda Aagaard Rasmussen
Research Unit for General Practice, Aarhus, Denmark
Tel +45 8716 8369
Fax +45 8612 4788
Email [email protected]

Purpose: Information on cancer recurrence is rarely available outside clinical trials. Wide exclusion criteria used in clinical trials tend to limit the generalizability of findings to the entire population of people living beyond a cancer disease. Therefore, population-level evidence is needed. The aim of this study was to develop and validate a register-based algorithm to identify patients diagnosed with recurrence after curative treatment of malignant melanoma.
Patients and Methods: Indicators of recurrence were diagnosis and procedure codes recorded in the Danish National Patient Register and pathology results recorded in the Danish National Pathology Register. Medical records on recurrence status and recurrence date in the Danish Melanoma Database served as the gold standard to assess the accuracy of the algorithm.
Results: The study included 1747 patients diagnosed with malignant melanoma; 95 (5.4%) were diagnosed with recurrence of malignant melanoma according to the gold standard. The algorithm reached a sensitivity of 93.7% (95% confidence interval (CI) 86.8– 97.6), a specificity of 99.2% (95% CI: 98.6– 99.5), a positive predictive value of 86.4% (95% CI: 78.2– 92.4), and negative predictive value of 99.6% (95% CI: 99.2– 99.9). Lin’s concordance correlation coefficient was 0.992 (95% CI: 0.989– 0.996) for the agreement between the recurrence dates generated by the algorithm and by the gold standard.
Conclusion: The algorithm can be used to identify patients diagnosed with recurrence of malignant melanoma and to establish the timing of recurrence. This can generate population-level evidence on disease-free survival and diagnostic pathways for recurrence of malignant melanoma.

Keywords: melanoma, recurrence, algorithms, validation study, registries, Denmark

Introduction

The number of people living beyond a diagnosis of malignant melanoma has increased annually by 5–7% in women and by 6–9% in men in Denmark in recent years. The number amounted to 33,176 people in 2018, corresponding to almost 10% of all people being alive after a cancer diagnosis in Denmark.1 The growing population of cancer survivors has emerged from advances in diagnostic technologies and cancer treatments,2 and the outcomes of interest after cancer treatment are changing from overall survival towards disease-free survival and cancer recurrence.3

General practice is increasingly engaged in cancer follow-up and recurrence surveillance in many healthcare systems.4 Nevertheless, little evidence exists to inform this shift. Information on disease-free survival and recurrence is rarely available outside clinical trials, and clinical trials often use wide exclusion criteria, which limit the generalizability of findings to the entire population of cancer survivors.5 Thus, population-level evidence is warranted in this field.3

Internationally, algorithms based on routinely collected data like medical claims and cancer-related register data have been developed to identify patients diagnosed with cancer recurrence, mainly from breast and colorectal cancer; the validation of these algorithms has shown mixed results.6–12 Recent studies have shown that cancer-specific algorithms based on Danish national health registries can accurately identify patients diagnosed with recurrence of colorectal cancer,13 breast cancer,14–16 bladder cancer,17 or endometrial cancer,18 with little selection bias. However, no similar approach exists to identify patients diagnosed with recurrence of malignant melanoma.

We aimed to develop and validate a register-based algorithm to identify patients with recurrence of malignant melanoma at population level in Denmark, and to estimate the accuracy of the cancer recurrence diagnosis date derived from the algorithm.

Methods

Design and Setting

We conducted a register-based study in Denmark, where tax-funded (free of charge) healthcare is available to all citizens. The data were linked at the individual level through the unique personal registration number assigned to all Danish citizens at birth or immigration.19

Data Sources

Data were obtained from five national population-based registers. The Danish Melanoma Database (DMD)20 comprises clinical information on the treatment and survival of incident cases of cutaneous melanoma, including follow-up on recurrence, since 1985. The Danish Cancer Register21 holds data on diagnosis codes, diagnosis dates, and tumor stage on all incident cancers diagnosed in Denmark since 1943. The Danish National Patient Register22 contains data on somatic in-hospital contacts since 1977 and on emergency and outpatient specialty care since 1995, including diagnosis codes, procedure codes, and tumor stage for cancer-related contacts. The Danish National Pathology Register23 holds information on all pathology specimens analyzed in Denmark since 1997. The registrations are classified according to the Danish Systematized Nomenclature of Medicine (SNOMED) system, which allows for identification of malignant morphology (codes M8 and M9). The fifth digit of the morphology code denotes the specimen (eg “4: malignant, direct spread to surrounding tissue”, “6: malignant, metastasis”, and “7: malignant, recurrent”). The Danish Civil Registration System24 holds (daily updated) information on migration and vital status on all Danish residents.

Gold Standard

The gold standard of recurrence from malignant melanoma was defined as recurrence status and recurrence date recorded in the DMD for patients treated for malignant melanoma at Herlev and Gentofte Hospital in 2008–2014. The hospital is located in the Capital Region of Denmark, and approximately 20% of all malignant melanomas in Denmark are treated here. In 2018–2019, recurrence registrations recorded in the DMD were updated for all patients treated at Herlev and Gentofte Hospital from 2008 and onwards. This was done by identification of metastases or recurrences from malignant melanoma in the Danish National Pathology Register.

Algorithm

The algorithm was constructed similarly to the algorithms for bladder, breast, and endometrial cancer recurrence by Rasmussen et al15,17,18 (Figure 1). The date of surgery for malignant melanoma was defined as the first biopsy date recorded in the DMD. If a re-resection date was registered within 60 days of the first biopsy date, the re-resection date was defined as the date of surgery. To prevent inclusion of patients with residual cancer after surgery, we excluded patients with malignant disease recorded in the registries within 90 days after the date of surgery. Indicators of residual disease were: 1) distant tumor stage based on the Union for International Cancer Control (UICC) classification of malignant tumours,25 2) new malignant diagnosis code based on the International Classification of Diseases, 10th revision (ICD-10): C00-C96 and D37-D48, excluding C44 (non-melanoma skin cancer) and C43 (malignant melanoma), 3) SNOMED registration of M8–9 morphology, and 4) procedure code for excision of skin or lymph nodes, amputation, radiotherapy, or chemotherapy combined with a malignant diagnosis code (ICD-10: C00-C96 and D37-D48). This 90-day cut-off period running from primary cancer treatment until recurrence surveillance is in line with other Danish15,17 and international7,8 cancer recurrence algorithms.

Figure 1 Temporal overview of the algorithm to identify patients with recurrence of malignant melanoma. Reproduced from Cancer Epidemiol, 59, Rasmussen LA, Jensen H, Flytkjær Virgilsen L, Jellesmark Thorsen LB, Offersen BV, Vedsted P. A validated algorithm for register-based identification of patients with recurrence of breast cancer—Based on Danish Breast Cancer Group (DBCG) data, 129–134, Copyright (2019), with permission from Elsevier.15

After the 90-day disease-free period, the algorithm searched for indicators of cancer recurrence (Figure 1). These indicators were identified through diagnosis codes and procedure codes in the Danish National Patient Register and through test results in the Danish National Pathology Register. A patient was defined as being diagnosed with melanoma recurrence if one of the following indicators was present:

  • ICD-10: C439X (melanoma recurrence).
  • ICD-10: C76*-C79* or C43xM (metastasis).
  • Procedure codes: BWG* (radiotherapy), except BWGA* (scheduling), BWHA* (chemotherapy), or BOHJ* (immunotherapy) combined with one of the following ICD-10 codes: C43* (malignant melanoma), or C76*-C79* or C43xM (metastasis).
  • Procedure codes: KCBB30/50/99, KDAB00/10, KDHB00/05/10, KEAA10/20/30/99, KEKB*, KQAE00/10, KQBE00/10, KQCE00/10, KQDE00/10 (surgical excision of skin), KPDJ* (excision of lymph nodes), or KNHQ*, KNDQ* (amputation), combined with one of the following ICD-10 codes: C439X (malignant melanoma recurrence), or C76*-C79* or C43xM (metastasis).
  • SNOMED morphology codes: M8-M9 with 4, 6, or 7 in the fifth digit, and a morphology code similar to a morphology code registered within 90 days of the date of primary melanoma surgery.

A metastasis diagnosis code was disregarded as an indicator of recurrence if a second primary cancer diagnosis was recorded in the Danish Cancer Register or Danish National Patient Register either before the metastasis diagnosis date or within 30 days after this date. This was done to avoid confusion between a metastasis from a second primary cancer and a metastasis from malignant melanoma. If a patient had more than one indicator of recurrence, the date of the first occurring indicator was entered as the date of cancer recurrence.

Study Population

Patients from the gold standard population were eligible for inclusion if they were registered in the Danish Cancer Register with a diagnosis code of malignant melanoma (ICD-10: C43), with no prior cancer diagnosis (except for non-melanoma skin cancer, ICD-10: C44), and aged ≥18 years at the time of diagnosis. Recurrence status in the DMD was not validated for 115 patients with a positive sentinel node identified at the time of the surgery for the primary cancer, and these 115 patients were excluded. Patients who emigrated or died within 90 days of their cancer surgery were excluded. Finally, patients recorded with residual malignant disease within 90 days of cancer surgery were excluded, as described in the sub-section “Algorithm”.

Analyses

The sensitivity, specificity, and positive and negative predictive values of the algorithm with 95% confidence interval (CI) were estimated from the concordant and discordant frequencies between the recurrences identified by the algorithm and the gold standard. To investigate the influence of each of the indicators, the sensitivity, specificity, and positive and negative predictive values were analyzed separately for recurrences identified by a diagnosis code, a procedure code, and a pathology test result.

The median follow-up time with interquartile interval (IQI) was estimated from the surgery date until the date of the first of one of the following events: cancer recurrence, death, emigration, last date of follow-up in the gold standard, or end of study (10 March 2018).

The agreement between the date of recurrence identified by the algorithm and the date identified by the gold standard was measured by Lin’s concordance correlation coefficient (CCC).26 Values <0.90 are considered “poor”, 0.90–0.95 “moderate”, >0.95 “substantial”, and >0.99 “almost perfect”.27 We estimated the proportions of recurrence dates estimated on the same date as the gold standard date and within an interval of 7, 14, 30, 60, and 90 days. Furthermore, we estimated the median number of days between the recurrence dates of the gold standard and the recurrence dates identified by the algorithm.

A sub-analysis was performed in a study population identified through the diagnosis date in the Danish Cancer Register and procedure codes for primary melanoma surgery in the Danish National Patient Register. This was done to assess the performance of an algorithm with no information from the clinical database (DMD).

Results

The final study population consisted of 1747 patients, hereof 95 (5.4%) had recurrence from malignant melanoma according to the gold standard (Figure 2). The median time from the start of the surveillance period until identification of recurrence was 18 months (IQI: 9–34 months) according to the gold standard. Study population characteristics are shown in Table 1.

Table 1 Characteristics of Included Patients Stratified on Cancer Recurrence Status in the Gold Standarda

Figure 2 Flowchart of the study population.

The algorithm identified 89 of the 95 recurrences and additional 14 false positives (Table 2), which demonstrated a sensitivity of 93.7% (95% CI: 86.8–97.6) and a specificity of 99.2% (95% CI: 98.6–99.5) (Table 3). The stratified analysis revealed that the highest sensitivity was seen for indicators of diagnosis codes, whereas the highest positive predictive value was obtained for indicators of pathology codes (Table 3).

Table 2 Concordance of Recurrence of Malignant Melanoma Identified by the Gold Standard and the Algorithm

Table 3 Performance of the Algorithm to Identify Recurrence from Malignant Melanoma (n=1747)

The CCC for the agreement between recurrence dates generated by the algorithm and the dates generated by the gold standard was estimated to be almost perfect, as it showed a CCC of 0.992 (95% CI: 0.989–0.996). The date was estimated within 7 days of the gold standard date in 42% and within 60 days in 84% (Table 4). The median number of days between the recurrence date identified by the algorithm and the date identified by the gold standard was 12 days (IQI: 1–35 days).

Table 4 Accuracy of Cancer Recurrence Date as Estimated from the Algorithm Compared with the Gold Standard

In the sub-analysis, we assessed the performance of an algorithm with no information from the DMD and the Danish National Patient Register served to identify the primary melanoma surgery date. We failed to identify procedure codes indicating primary melanoma surgery in 127 patients in the Danish National Patient Register, and these patients were excluded. The final study population consisted of 1620 patients, hereof 75 had recurrence according to the gold standard. The performance of the algorithm in this population was similar to its performance in the main analyses (Supplementary Tables 1–3).

Discussion

We developed and validated a register-based algorithm to identify patients diagnosed with recurrence of malignant melanoma in Denmark. The algorithm correctly identified 94% of patients with recurrence and 99% of patients without recurrence according to the gold standard. With a recurrence prevalence of 5%, the positive predictive value was 86% and the negative predictive value over 99%. In 84% of cases, the algorithm estimated the date of recurrence within 60 days of the date estimated by the gold standard.

Strengths and Limitations

The study has several strengths. The algorithm is based on Danish national registries with near-complete high-quality records ensuring data of high validity.19,21–24 Tax-funded, free, and equal access to healthcare in Denmark limits the risk of selection bias.

Another important strength was that the gold standard included information on all patients treated for malignant melanoma at Herlev and Gentofte Hospital. The absence of exclusion criteria for patients registered in the DMD ensures high generalizability of the results to the entire Danish population diagnosed with malignant melanoma.5 Furthermore, treatment and follow-up protocols for malignant melanomas in Denmark are described by the Danish Melanoma Group.28 These uniform national strategies further increase the generalizability of the gold standard population to the entire population of patients treated for malignant melanoma in Denmark. Finally, the access to a large gold standard population was important as we investigated a disease with a relatively low recurrence rate. A second validation of the algorithm in a different population with information on melanoma recurrence status would further strengthen the validation of the algorithm. Unfortunately, such a population was not available.

All indicators of recurrence were based on electronic registrations in hospital records. Consequently, contact to a hospital was required for the case to be identified by the algorithm. The current Danish guidelines for recurrence surveillance after treatment for malignant melanoma recommend self-monitoring and patient-initiated follow-up of patients with low risk of recurrence (melanoma in situ and stage IA), five-year clinical and hospital-based follow-up of patients with intermediate risk of recurrence (stage IB and IIA), and specialized clinical and imaging follow-up of patients with high risk of recurrence (stage IIB, IIC and III) followed by five-year follow-up in general practice. However, in case of suspicion of recurrence, all patients are seen in a specialized hospital setting, which increases the likelihood of the algorithm to identify recurrence.

In the sub-analysis, the algorithm failed to identify procedure codes for primary cancer surgery in the Danish National Patient Register in 7% of the gold standard population, and these patients were excluded. This increased the risk of underestimating the absolute number of recurrences. However, the performance of the algorithm in terms of sensitivity, specificity, and predictive values was similar to the performance in the main analyses.

The algorithm identified 89 of 95 recurrences in the gold standard and additional 14 false positives. Half of the false positives had more than one and up to four different indicators of recurrence, and the algorithm may thus confuse second primary melanomas with melanoma recurrence. Conversely, in the validation study of recurrences at Herlev and Gentofte Hospital, the DMD may have missed a recurrence if registered as a primary tumor (not metastasis or recurrence) in the Danish National Pathology Register.

A total of 115 patients were excluded, as their recurrence status was not validated in the DMD. Most of these 115 patients were diagnosed at stage III. This is reflected in the stage distribution; less than 1% are stage III patients compared to the 10% stated in the annual report 2014 of the DMD.29 After a diagnosis of malignant melanoma in stages I–III, the recurrence rates are reported to be up to 15%.30 When simulating the algorithm in a population with a recurrence rate of 15%, the positive predictive value increases to 95.1%, and the negative predictive value remains almost unchanged at 98.9%. Thus, the algorithm is expected to perform superior in a non-selected population compared to the performance in the present study.

Comparison with Literature

To our knowledge, no previous studies have developed register-based algorithms to identify patients diagnosed with recurrence of malignant melanoma. The algorithm performed similarly to previously developed algorithms that used the same data sources to identify patients diagnosed with recurrence of bladder, breast and colorectal cancer.13–15,17,18 Studies from North America have validated algorithms to identify patients diagnosed with recurrence of breast, colorectal, and lung cancer.6–10,12 In these studies, indicators of recurrence were based on procedure codes, diagnosis codes, and hospice and oncology visits; the sensitivity ranged from 70% to 94%, and the specificity ranged from 70% to 98%. Only two studies reached a sensitivity above 90%.6,9 The accurate performance of the algorithms described in Danish studies seems to result from the inclusion of pathology data as indicators of recurrence.13–15,17 Restrictions to the pathology coding prevented false positives, and the pathology data reached high positive predictive values, ranging from 91% to 100% across the studies by Rasmussen et al.15,17,18

Studies from the United States based their algorithms on specific medical claims combined with cancer-related register data and other administrative data restricted to specific populations, geographic areas, or insurance groups.6–8,10 This limits their generalizability to entire populations, whereas algorithms based on the population-based nationwide Danish health registers allow for national applicability.

Three studies from outside Denmark assessed the accuracy of the estimated recurrence diagnosis date; they identified 14–36% of the recurrences within 30 days of the gold standard recurrence diagnosis date6,10 and with an average prediction error of 4.5 months.11 Recurrence dates were identified more accurately in the present study; 73% of recurrences were identified within 30 days of the gold standard recurrence diagnosis date. These results were in line with previous Danish studies in bladder, breast, and endometrial cancer.15,17,18 The validity of research in disease-free survival and recurrence pathways relies highly on the accuracy of the identified recurrence diagnosis date, and potential bias caused by the estimated date should always be considered. In the present study, deviations of recurrence dates were equally distributed before and after the gold standard date (data not shown), which limits potential bias.

Existing studies and databases have used different cut-offs to distinguish between persistent primary disease and recurrence. We used a cut-off of 90 days after primary melanoma surgery, whereas others have defined a recurrence from malignant melanoma as the return of disease as early as 30 days after the primary melanoma surgery.31 Studies using the algorithm to identify a population with recurrence from malignant melanoma should be highly aware of the definitions of remission, persistent disease, and recurrence in the algorithm.

Implications

The algorithm facilitates future large-scale research in recurrence from malignant melanoma, which can inform cancer follow-up strategies and improve the quality of cancer survivor care. Cancer follow-up is being reorganized in many healthcare systems. General practice is planned to play a greater and more formal role in the future cancer follow-up in Denmark and in other countries with similar healthcare systems.4 The recent changes in the organization of follow-up after treatment for malignant melanoma in Denmark were primarily based on expert opinions. However, the proposed algorithm has the potential to evaluate the ability of cancer follow-up programs to detect recurrences in a timely manner.

In 2018, 99% of patients diagnosed with malignant melanoma in Denmark were candidates for curative intent surgery.32 A high rate of successfully treated patients increases the relevance of using disease-free survival as the endpoint. Systemic treatment for malignant melanoma is gaining ground these years, and measurement of recurrence at population level provides an opportunity to evaluate new treatment strategies. Furthermore, the population-based design of the algorithm facilitates epidemiological research on potential predictors and causation of recurrence of malignant melanoma.

Conclusion

We developed an algorithm to identify recurrence of malignant melanoma using routinely collected data in Danish health registries. The high accuracy of the algorithm to identify cases of recurrence and to estimate the date of recurrence facilitates population-based research in the field of recurrence of malignant melanoma. This can extend our knowledge on recurrence of malignant melanoma and guide us towards the ultimate goal of improving the prognosis for the patients.

Data Sharing Statement

The data supporting the findings of this study are stored and maintained electronically at Statistics Denmark. The data is only accessible via a secured virtual private network (VPN) and only by approved collaborative partners. The data is not publicly available due to the Danish data protection legislation as the data contains information that could compromise the privacy of the research participants.

Ethics

The study is registered as “The patient pathway for cancer recurrence” (study 1, id 119) in the Record of Processing Activities at the Research Unit for General Practice in Aarhus in accordance with the provisions of the General Data Protection Regulation (GDPR). According to Danish law, the study required no approval from the Committee on Health Research Ethics in the Central Denmark Region.

Acknowledgments

We wish to thank the Danish Melanoma Database for providing the data for the study. We also thank data manager Aleksandar Jovanovic for his assistance in data retrieval and introduction to the DMD data.

Disclosure

The authors report no conflicts of interest in this work.

References

1. The National Board of Health. Nye kræfttilfælde i Danmark. 2018 [Cancer Incidence in Denmark 2018]. Copenhagen: National Board of health; 2019.

2. Hewitt ME, Greenfield S, Stovall E. From Cancer Patient to Cancer Survivor: Lost in Transition. Washington DC: The National Academies Press; 2006.

3. Warren JL, Yabroff KR. Challenges and opportunities in measuring cancer recurrence in the United States. J Natl Cancer Inst. 2015;107(8):djv134–djv134. doi:10.1093/jnci/djv134

4. Rubin G, Berendsen A, Crawford SM, et al. The expanding role of primary care in cancer control. Lancet Oncol. 2015;16(12):1231–1272.

5. Cohen AT, Goto S, Schreiber K, Torp-Pedersen C. Why do we need observational studies of everyday patients in the real-life setting? Eur Hear J. 2015;17(Suppl D):D2–D8. doi:10.1093/eurheartj/suv035

6. Hassett MJ, Uno H, Cronin AM, Carroll NM, Hornbrook MC, Ritzwoller D. Detecting lung and colorectal cancer recurrence using structured clinical/administrative data to enable outcomes research and population health management. Med Care. 2017;55(12):e88–e98. doi:10.1097/MLR.0000000000000404

7. Deshpande AD, Schootman M, Mayer A. Development of a claims-based algorithm to identify colorectal cancer recurrence. Ann Epidemiol. 2015;25(4):297–300. doi:10.1016/j.annepidem.2015.01.005

8. Warren JL, Mariotto A, Melbert D, et al. Sensitivity of medicare claims to identify cancer recurrence in elderly colorectal and breast cancer patients. Med Care. 2016;54(8):e47–e54. doi:10.1097/MLR.0000000000000058

9. Xu Y, Kong S, Cheung WY, et al. Development and validation of case-finding algorithms for recurrence of breast cancer using routinely collected administrative data. BMC Cancer. 2019;19(1):210. doi:10.1186/s12885-019-5432-8

10. Ritzwoller DP, Hassett MJ, Uno H, et al. Development, validation, and dissemination of a breast cancer recurrence detection and timing informatics algorithm. J Natl Cancer Inst. 2018;110(3):273–281. doi:10.1093/jnci/djx200

11. Uno H, Ritzwoller DP, Cronin AM, Carroll NM, Hornbrook MC, Hassett MJ. Determining the time of cancer recurrence using claims or electronic medical record data. JCO Clin Cancer Inform. 2018;2(2):1–10.

12. Izci H, Tambuyzer T, Tuand K, et al. A systematic review of estimating breast cancer recurrence at the population level with administrative data. J Natl Cancer Inst. 2020;112(10):djaa050. doi:10.1093/jnci/djaa050

13. Lash TL, Riis AH, Ostenfeld EB, Erichsen R, Vyberg M, Thorlacius-Ussing O. A validated algorithm to ascertain colorectal cancer recurrence using registry resources in Denmark. Int J Cancer. 2015;136(9):2210–2215. doi:10.1002/ijc.29267

14. Cronin-Fenton D, Kjærsgaard A, Nørgaard M, et al. Breast cancer recurrence, bone metastases, and visceral metastases in women with stage II and III breast cancer in Denmark. Breast Cancer Res Treat. 2018;167(2):517–528. doi:10.1007/s10549-017-4510-3

15. Rasmussen LA, Jensen H, Flytkjær Virgilsen L, Jellesmark Thorsen LB, Offersen BV, Vedsted P. A validated algorithm for register-based identification of patients with recurrence of breast cancer—Based on Danish Breast Cancer Group (DBCG) data. Cancer Epidemiol. 2019;59:129–134. doi:10.1016/j.canep.2019.01.016

16. Pedersen RN, Öztürk B, Mellemkjær L, et al. Validation of an algorithm to ascertain late breast cancer recurrence using Danish medical registries. Clin Epidemiol. 2020;12:1083–1093. doi:10.2147/CLEP.S269962

17. Rasmussen LA, Jensen H, Virgilsen LF, Jensen JB, Vedsted P. A validated algorithm to identify recurrence of bladder cancer: a register-based study in Denmark. Clin Epidemiol. 2018;10:1755–1763. doi:10.2147/CLEP.S177305

18. Rasmussen LA, Jensen H, Virgilsen LF, et al. Identification of endometrial cancer recurrence – a validated algorithm based on nationwide Danish registries. Acta Oncol. 2020. doi:10.1080/0284186X.2020.1859133

19. Schmidt M, Pedersen L, Sørensen HT. The Danish civil registration system as a tool in epidemiology. Eur J Epidemiol. 2014;29(8):541–549. doi:10.1007/s10654-014-9930-3

20. Rosenkrantz Hölmich L, Klausen S, Spaun E, et al. The Danish melanoma database. Clin Epidemiol. 2016;8:543–548. doi:10.2147/CLEP.S99484

21. Gjerstorff ML. The Danish cancer registry. Scand J Public Health. 2011;39(Suppl 7):42–45. doi:10.1177/1403494810393562

22. Schmidt M, Schmidt SAJ, Sandegaard JL, Ehrenstein V, Pedersen L, Sørensen HT. The Danish national patient registry: a review of content, data quality, and research potential. Clin Epidemiol. 2015;7:449–490. doi:10.2147/CLEP.S91125

23. Erichsen R, Lash TL, Hamilton-Dutoit SJ, Bjerregaard B, Vyberg M, Pedersen L. Existing data sources for clinical epidemiology: the Danish national pathology registry and data bank. Clin Epidemiol. 2010;2:51–56. doi:10.2147/CLEP.S9908

24. Pedersen CB. The Danish civil registration system. Scand J Public Health. 2011;39(Suppl 7):22–25. doi:10.1177/1403494810387965

25. Union for International Cancer Control. TNM classification of malignant tumours. Available from: https://www.uicc.org/resources/tnm. Accessed December 24, 2020.

26. Lin LI. A concordance correlation coefficient to evaluate reproducibility. Biometrics. 1989;45(1):255–268. doi:10.2307/2532051

27. McBride GB. A Proposal for Strength of Agreement Criteria for Lin’s Concordance Correlation Coefficient. New Zealand: National Institute of Water & Atmospheric Reseach; 2005.

28. Danish Melanoma Group. Available from: https://www.melanoma.dk/index.html#header1-1. Accessed December 4, 2020.

29. The Danish Clinical Quality Program and Clinical Registries (RKKP) and Danish Melanoma Database. Dansk melanom databases national årsrapport 2014 [Danish melanoma database annual report 2014]. Aarhus, Denmark. 2015. Available from: https://www.melanoma.dk/assets/files/DMG_Aarsrapport_2014.pdf. Accessed December 4, 2020.

30. Leiter U, Buettner PG, Eigentler TK, et al. Hazard rates for recurrent and secondary cutaneous melanoma: an analysis of 33,384 patients in the German central malignant melanoma registry. J Am Acad Dermatol. 2012;66(1):37–45. doi:10.1016/j.jaad.2010.09.772

31. Tas F, Erturk K. Early and late relapses of cutaneous melanoma patients. Postgrad Med. 2019;131(3):207–211. doi:10.1080/00325481.2019.1569354

32. The Danish Clinical Quality Program and Clinical Registries (RKKP) and Danish Melanoma Database. Dansk melanom database national årsrapport 2017 [Danish melanoma database annual report 2017]. Aarhus, Denmark. 2018. Available from: https://www.melanoma.dk/assets/files/DMG_Aarsrapport_2018.pdf. Accessed December 4, 2020.

Creative Commons License This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.

Download Article [PDF]