The Danish Heart Failure Registry: A Validation Study of Content

Introduction

Despite advances in treatment, the syndrome of heart failure (HF) remains a major cause of mortality, hospitalization, and poor quality of life worldwide.¹ The prevalence is approximately 1–2% of the adult population in developed countries, and with a marked rise to >10% for people aged ≥70 years.² The prevalence of patients with HF is expected to rise in the future owing to better treatment, higher life expectancy in the general population as well as improved survival rates after acute myocardial infarction (AMI).¹

The Danish Heart Failure Registry (DHFR) was established in 2003 with the intention to monitor and improve the quality of care for patients with incident HF.³ Since 2005 the DHFR has had nationwide coverage. Reporting to the registry is mandatory for all hospital departments treating patients with incident HF.

Knowledge of data validity is a prerequisite for meaningful use of the collected data. However, little is known about the validity of the DHFR. Data from this register is used in research and decision-making regarding the healthcare system, so it is essential to have valid, complete, and unbiased data. Therefore, we examined the validity of the content recorded in the DHFR.

Methods

Setting

In Denmark, the healthcare system is tax-funded and provides universal healthcare for all 5.8 million Danish residents. The Danish National Health Service provides tax-supported free access to general practitioners and hospital care, outpatient specialty clinics, and partial reimbursement of prescribed medications.⁴ All Danish citizens have an assigned unique civil registration number which enables unambiguous individual-level linkage between registers.

Data Registration

The DHFR is an internet-based online database. DHFR includes both inpatients and outpatients with incident HF. To optimize the data quality, data are entered by the local staff of the cardiology department, at which the treatment is performed, using the unique civil registration number. The criteria used for a HF diagnosis in the DHFR is defined by the European Society of Cardiology and validated before entering the database by a local cardiologist.⁵ The following primary diagnoses (International Classification of Diseases version 10 codes) are screened for inclusion in the DHFR: DI11.0, DI13.0, DI13.2, DI42.0, DI42.6, DI42.7, DI42.9, DI50.0, DI51.0, and DI50.9. Additional inclusion criteria to the DHFR are age of 18 and older, a first-time hospital contact with HF as the primary diagnosis and symptoms of HF, and/or objective signs of HF, and/or clinical response to HF treatment. Exclusion criteria are isolated right-sided HF, HF secondary to valvular heart diseases, non-correctable structural heart diseases, and tachycardia-induced HF. Furthermore, patients discharged with a primary diagnosis of AMI and concomitant HF are excluded. However, these patients will be included if they are later admitted with HF or are referred to an outpatient cardiology clinic for the treatment of HF. The performance measures recorded in the DHFR are related to diagnostic tests performed, pharmaceutical and non-pharmaceutical treatment, readmission within four weeks, and mortality within one year.³ Furthermore, patient characteristics are registered as well. The full list is included in Table 1.

Table 1 Patient Characteristics

Study Population

We aimed to retrieve and review medical records for a random computer-generated sample of 10 patient records per year per hospital department reporting to the DHFR between the 1st of January 2016 to the 31st of December 2018. All 29 cardiological departments reporting to the DHFR were invited to participate in the study. Eighteen departments accepted the invitation. The reason for no participation was either the legal restrictions due to data protection or lack of response to the invitation. We were unable to retrieve medical records for seven of the patients (1.5%) as they were deleted from the record achieves at the time of the review. In total, 453 patients were eligible for inclusion in the study population (Figure 1).

Figure 1 Consort for the patient selection.

Included patients covered hospitals in four of the five Danish regions; Central Denmark, Southern Denmark, North Denmark, and Region Zealand, and included both university hospitals and regional hospitals (Figure 2).

Figure 2 Distribution of Danish heart failure departments included in the validation (green dot = participating; black dot = not participating).

Statistical Analysis

We considered and reviewed the performance measures mentioned in Table 1. We presented categorical variables using frequencies and proportions, and continuous variables using medians and interquartile ranges.

For validation, we assessed the validity of all performance measures in DHFR through review of hospital medical records considered as the golden standard. We included both patients from university and regional hospital cardiac centers. Within each cardiac center, patients were selected randomly regardless of whether data were missing on one or more variables.

Sensitivity and specificity were computed for all variables. We used Wilson´s score methods to calculate 95% confidence intervals (CIs). In addition, we constructed 2×2 tables and computed positive predictive values (PPVs) and negative predictive values (NPVs) for each variable. PPV was defined as the number of true positives/(number of true positives + number of false positives). NPV was defined as the number of true negatives/(number of true negatives + number of false negatives).

All medical records were reviewed blinded, with only access to the civil registration number of the patient and the date of enrollment in the DHFR. If data was not obtainable from the medical records, the data point would be marked as unknown and would be excluded from the analysis. Medical records with doubtful conclusion regarding patient characteristics and/or performance measures, were discussed within the study group together with a specialist in cardiology and a clinical nurse specialist with in-depth knowledge of the database. Before the data collection started, we tested the interobserver reproducibility by two independent examinations of 720 variables from 30 random medical records. The test showed a 98.0% correlation between the two observers. The main discrepancies between the two observers were the exact New York Heart Association (NYHA) classification and smoking status.

All analyses were performed with STATA (version 17.0). The randomization of 10 patients within each combination of department and year was performed using SAS´ RANUNI function. The present study was approved by the Danish Data Protection Agency (1–16-02-273-20). Every single participating department management/hospital management gave permission for the dissemination of data from the medical records.

Results

Baseline Characteristics

The demographics and clinical characteristics of the 453 patients included from the DHFR are listed in Table 2. Sixty-five percent (n=295) were males, and the median age was 70.9 years at the time of registration in the DHFR.

Table 2 Baseline Characteristics for Patients Included (N= 453)

Validity

Tables 3–5 shows the sensitivity, specificity, and predictive values for the prognostic factors, and performance measures registered in the DHFR compared with the information retrieved from the medical records.

Table 3 Sensitivity, Specificity and Predictive Values for Comorbidities

Table 4 Sensitivity, Specificity and Predictive Values for Smoking Status and Alcohol Consumption

Table 5 Sensitivity, Specificity and Predictive Values for Echocardiography, NYHA Classification, Medical Treatment, Patient Education, Physical Exercise and Readmission

The prognostic variables (AMI, stroke, chronic obstructive pulmonary disease, diabetes, hypertension, and s-creatinine) showed the sensitivity to range from 81.0% to 95.2% and specificity ranged from 79.8% to 99.5%. Furthermore, we observed s high PPV between 93.0% and 99.5% for all variables (Table 3). The highest PPV was seen for hypertension (99.5%; 95% CI:96.6–99.9) and lowest for s-creatinine>150 umol/L (93.8%; 95% CI: 82.9–97.9). The NPVs ranged from 81.4% to 99.0%.

Looking at smoking and drinking habits, especially drinking (>7/14 alcohol units/week) showed low sensitivity (47.2%; 95% CI: 30.4 to 64.5). Sensitivity for both these prognostic factors ranged from 47.2% to 92.8%, specificity ranged from 39.6% to 98.8%. The PPV for smoking status were between 76.0% and 96.2%. The PPV for alcohol consumption (>7/24 alcohol units/week) were low (63.0%; 95% CI: 42.4 to 80.6) (Table 4). The NPV for both smoking status and alcohol consumption ranged between 62.7% to 95.5%.

As shown in Table 5, the performance measures generally showed high PPVs, with 9 out of 10 variables having PPVs above 80%. The diagnostic variables of echocardiography and NYHA assessment within 12 weeks showed high sensitivity (99.8% for both) and high PPVs (99.6% and 92.4%, respectively). However, because of a few observations not having these diagnostic variables assessed, the specificity was low for both echocardiography and NYHA assessment (33.3% and 34.0%), as well as the NPV for echocardiography (50.0%), whereas the NPV for NYHA assessment was 94.4%. The pharmacological treatment variables also showed high agreement regarding sensitivity, ranging from 77.4% to 98.6%, and PPVs, ranging from 84.0% to 94.0%. Furthermore, the specificity ranged from 66.7% to 98.0%, and the NPVs ranged from 90.1% to 96.5%. For the non-pharmacological treatment variables, patient education showed high sensitivity (98.0%; 95% CI: 96.1–99.1) and PPV (94.9%; 95% CI: 93.0–96.3), whereas referral to physical exercise within 12 weeks was a little lower, with a sensitivity of 77.8% (95% CI: 71.6–83.1) and a PPV of 74.5% (95% CI: 69.6–78.6).

Discussion

This is the first validation of the content of the DHFR. We demonstrated a high level of agreement between data on processes of care recorded in the DHFR performed in a random computer-generated sample of patients registered in the DHFR between 1st of January 2016 to the 31st of December 2018. The medical records were used as golden standard. The validity was assessed as sensitivity, specificity and PPV/NPPs.

Despite the generally high levels of validity of data recorded in the DHFR, differences were observed between the various types of processes. For the variables regarding prognostic comorbidities and therapeutic pharmacological treatment, moderate to very high sensitivities, specificities, PPVs, and NPVs were found for all variables. The variables of the prognostic factors regarding smoking habits also showed high to very high sensitivities, specificities, PPVs, and NPVs, whereas variables regarding alcohol consumption, referral to physical exercise, and readmission showed moderate low sensitivities, specificities and PPV/NPVs.

As mentioned above, there were notable differences in validity of especially the patient characteristics. The primary reason for the variables yielding low sensitivities, specificities, PPVs, and NPVs, was the incompleteness of documentation on the variable in the medical records. This was, for example, a problem with the variable on alcohol consumption, as information on this variable were missing in 72 patient records at the time of data collection. This could be caused by the lack of record keeping but more likely because the data were inaccessible for the data collector, which consequently could cause the medical records to be a problematic gold standard.

Other HF registries are used to monitor the quality of care in HF. For example, European Society of Cardiology started in 2010 to collect data in patients with either chronic or acute HF to describe the clinical epidemiology and the diagnostic and therapeutic approaches applied to these patients.^6–8 The Swedish Heart Failure Registry (SwedeHF) was established in 2001, and since 2003 it has registered patients with chronic HF in Sweden.⁹ The strengths of the DHFR are that it is a national, multicenter registry with detailed information on patient characteristics, medical treatment, and outcomes. The diagnosis is validated by clinicians according to guidelines, entered electronically, and further individual-level data linkage (using the civil registration number) to the Danish Civil Registration System enables complete patient follow-up with accurate accounting for censoring due to emigration or death. Moreover, linkage to other registries provides valuable information on comorbidity, socioeconomic data, and subsequent diagnostic and therapeutic resource utilization, clinical events, and mortality.^4,10–13 Finally, DHFR can be used on a patient level to support the best medical treatment on an individual basis, and by now, a validated registry. Although our study supports the use of DHFR for research and as a monitoring tool for quality of care in patients with incident HF, there are some limitations to be considered. Limitations of this study include using historic medical records as the gold standard. A concrete problem we faced was that the electronic patient journal system in Region Zealand, from May 2016 to November 2017, transitioned from OPUS to Sundhedsplatformen. This caused that the medical records from this period and before were archived and not accessible in their original form. This could have caused loss of information. Furthermore, selected variables may have changed during the years. These changes in reporting standards may partly explain some of the missing values over time. It is possible that the accuracy of the recorded data may vary across reporting hospitals or subtypes of patients; however, this study was not aimed at examining causality and therefore the role of classical confounding was not relevant to consider.

A crucial issue is how to achieve higher accuracy of data in the future. We believe, it is of utmost importance to use the continuous development of automatic digital capture, making it possible to automatically “catch” data directly from the electronic medical records. This would make it possible to link data across data sources and avoid error-prone manual data reporting. This generally high level of accuracy in DHFR will hopefully further increase and impact studies based on these data. Also, as part of a clinical quality database, these valid data can continue to form the basis for evaluations of clinical outcomes and guide clinicians, health authorities and political decisions in Denmark.

However, it is also important to be aware that data validity may change over time. Therefore, validation of a clinical quality database like the DHFR is a continuous process – which should be repeated later.

Conclusion

The DHFR provides ongoing prospective registration of incident HF patients in Denmark. Overall, the registry data have high degree of completeness and validity, making it a valuable tool for monitoring quality of care and clinical epidemiological research in HF.