Validity of Algorithm for Classification of In- and Outpatient Hospital Contacts in the Danish National Patient Registry

Introduction

The Danish National Patient Registry (DNPR) was established in 1977 and has since been used extensively to conduct epidemiological studies on incidence, risk, and prognosis of diseases.^1–3 The registry comprises standardized data collection of each hospital contact from computerized patient administrative systems at all Danish hospitals.

A major revision of the DNPR commonly referred to as “DNPR-3”, ie, 3rd version of DNPR, was implemented in February and March 2019 to improve documentation of related hospital contacts for administrative purposes for the Danish Board of Health.⁴ The new data model made it difficult to differentiate between in- and outpatient contacts as well as visits to the emergency room (ER). This change may have increased the risk of misclassification in registry-based studies since ER visits were previously often excluded to increase the positive predictive values (PPVs) of diagnosis codes.¹ In addition, it may have disrupted longitudinal studies across different versions of the DNPR. In early 2020, an immediate demand for crucial information on the impact of Coronavirus disease 2019 (COVID-19) on a population-level resulted in the adoption of a provisional 12-hour rule-of-thumb to join and characterize department contacts as overnight inpatient hospitalizations or not.^5–10 However, the accuracy of this proxy for inpatient hospitalizations has never been validated and it cannot be used to separate ER visits from outpatient contacts.

This study examined the PPV of an algorithm to categorize hospital departments and their corresponding healthcare contacts as inpatient, outpatient, or ER in the DNPR in North Denmark Region. Second, the study explored various time intervals for joining healthcare contacts and derived proportions of overnight hospital stays among inpatients, outpatients, and ER visits.

Methods

Study Design and Setting

A population-based cross-sectional study of all patients hospitalized at somatic hospitals in North Denmark Region from January 1, 2018, through March 31, 2020 (Figure 1). The population of North Denmark Region was 588,888 on July 1st, 2018, and 590,403 on July 1st, 2021.¹¹ Healthcare in Denmark is tax-financed and free of charge at the point of delivery for all residents. A personal identification number is assigned to all Danish residents at birth or immigration and can be used for individual linkage of all contacts with healthcare as well as other government bodies and authorities.¹²

Figure 1 Organization of somatic and psychiatric hospitals in North Denmark Region. Red dots indicate psychiatric hospital units and green dots indicate somatic hospital units.

Data Sources

The DNPR holds detailed longitudinal information on all patients discharged from somatic hospitals in Denmark and was extended to include psychiatric inpatients as well as ER and outpatient contacts since 1995 (DNPR-2).^1,3,13 A primary and an optional number of secondary diagnosis codes were assigned to each hospital contact by the physician in charge using the World Health Organisation International Classification of Diseases (ICD) version 8 until 1993 and ICD-10 thereafter. In addition, registrations include department and hospital of each contact, dates of admission and discharge, certain diagnostic examinations and in-hospital medical treatments (eg, chemotherapy and biological treatments), and surgical procedures. Information on whether the contact is acute or elective is available in both DNPR-2 and DNPR-3.

In DNPR-2, each hospital contact was defined by the unit primarily responsible for managing the patient during hospitalization (Figure 2). The contact was grouped as inpatient, outpatient, or ER visits by the variable “patient type” until 2014 when ER visits were omitted and subsequently categorized as outpatient. Since then, ER contacts were often identified by a combination of data entries as “outpatient” and “acute hospitalization”. In DNPR-2, details on duration of inpatient and acute outpatient contacts are given in terms of the timestamp of start and end of the contact, whereas for elective outpatient contacts only the date of the first visit is recorded. Further visits within the same outpatient sequence are summarized by the number of visits and the date of discharge from the outpatient clinic or the date of transfer to DNPR-3.

Figure 2 Illustration of patient flow including two inpatient department contacts and subsequent outpatient contact sequence in the Danish National Patient Registry versions 2 (DNPR-2) and DNPR-3, respectively. Gray boxes in DNPR-2 represent contacts that are not recorded.

DNPR-3 was implemented at Danish hospitals from February 1 to March 1, 2019.⁴ The focus of this change was an improved documentation of related diagnostics and treatments for each hospital contact across different departments and time. Within this data model, the index hospitalization may initiate a sequence of contacts related to that specific clinical problem. Additional and separate hospital contact sequences may also be established if other diseases requiring follow-up are identified. In DNPR-3, all contacts are recorded with the same level of detail including timestamps of start and end of each physical or virtual contact. Importantly, the variable “patient type” (inpatient vs outpatient vs ER) is not present in DNPR-3, which makes it difficult to determine if patients were managed in the ER (and sent home), admitted at hospital, or treated in an outpatient setting. This study was based on data from a temporary data model of DNPR-3, called LPR3_SB. A permanent research-oriented version, LPR3_F, was made available during 2022 by the Danish Health Data Agency and will replace LPR3_SB. The information and structures used for this study based on LPR3_SB will also be present in LPR3_F.

All Danish hospitals and departments have been organized by individual SHAK codes (“Sygehus-afdelingsklassifikation”) in the DNPR. In 2020, this characterization was replaced by SOR codes (“Sundhedsvæsenets Organisationsregister”) and expanded with information on type of clinical unit and healthcare category based on Systematized Nomenclature of Medicine Clinical Terms (SNOMED CT) terminology (eg, radiology, occupational therapy, palliative care, or ER). The use of SOR also entailed an administrative division of emergency departments (EDs) into ERs, ie, healthcare conditions managed and sent home on the same day without requirement for inpatient hospital admission, and emergency wards (EW), ie, healthcare conditions with requirement of inpatient hospital admission for up to 48 hours before transfer to another department or discharge.

Study Periods and Study Variables

This study included information on all somatic hospitalizations in North Denmark Region within one year before and after the implementation of DNPR-3:

1. DNPR-2: January 1 through December 31, 2018
2. DNPR-3: April 1, 2019 through March 31, 2020

The time period of Implementation of DNPR-3 from January 1 through March 31, 2019 was censored. Hospital contacts were grouped according to the date of each contact. To align with DNPR-2 which only included physical contacts, all non-physical contacts such as virtual consultations or contacts outside hospital were excluded from DNPR-3 for this analysis.

Algorithm

Based on individual contact data including start and end timestamp, the algorithm combined a characterization of each clinical unit by the duration of healthcare contacts, ie, extending over night or not, and proportion of acute contacts (Table 1). The hypothesis was that inpatient departments had ≥50% of contacts extending overnight (regardless of acute or elective hospitalizations), and outpatient clinics had ≥50% of same-day contacts combined with ≥50% of elective hospitalizations. In contrast, ERs would have ≥50% of same-day contacts with <50% registered as elective hospitalizations. An outline of the programming steps is provided in the Supplementary Material.

Table 1 Algorithm for Characterizing Hospital Departments and Clinics as Inpatient, Outpatient, or Emergency Departments

Reference

All hospital departments and clinics in North Denmark Region were manually characterized as inpatient, outpatient, ER, or anaesthesia/intensive care unit by two senior consultants with 12 (JB) and 25 (HN) years of clinical experience, respectively, in North Denmark Region as reference. Individual departments were contacted in all cases of doubt. Next, anaesthesia/intensive care units were re-classified as outpatient since they did not have independent administrative responsibility of patients.

Statistical Analyses

First, a kappa correlation score on level of agreement between manual categorization of hospital units was computed. Next, using manual categorization of departments and clinics as reference, PPVs of the algorithm were computed for all hospital departments and their related healthcare contacts in DNPR-2 and DNPR-3. In addition, distributions of overnight in-hospital stays and types of healthcare contacts (ie, inpatient, outpatient, and ER visits) were explored by allowing time intervals of 0, 3, 6, 12, or 18 hours within which hospital contacts were joined to represent uninterrupted hospitalizations. Further details on joining of hospital contacts by time intervals are provided in the Supplementary Material. SAS software version 9.4 (SAS Institute Inc., Cary, NC, USA) was used for data management and analyses.

Ethical Considerations

This study was approved by the hospital administrations of Aalborg University Hospital and Regionshospitalet Nordjylland as a quality-of-care project and patient consent or permission from an ethical committee was therefore not required for this type of study in Denmark. Handling of data complied with relevant data protection and privacy regulations and was conducted in accordance with the Declaration of Helsinki.

Results

The two study periods (365 days each) comprised a total 610,471 individual hospital contacts among 161 departments in North Denmark Region in DNPR-2 and 1,279,337 hospital contacts among 322 departments in DNPR-3. During the entire observation period, the number of inpatient contacts and ER visits (≈100,000 each) increased by 11% and 3%, respectively, whereas the number of outpatient contacts increased by 243%. The distribution of subsequent hospital contacts within 36 hours according to index hospital contact in DNPR-3 is shown in Figure 3. The kappa correlation score reflected a high level of agreement at 0.93 between manual categorizations of hospital units.

Figure 3 Distribution of subsequent hospitalizations within 36 hours following index hospital contacts in the Danish National Patient Registry version 3 (DNPR-3). Not all patients had repeated hospitalizations within the specified time interval and each patient may have had multiple contacts initiated.

The algorithm yielded PPVs of 91% for units classified as inpatient in DNPR-2 and 89% for related contacts compared with 100% on both unit and contact level in DNPR-3 (Table 2, Figure 4). For units categorized as outpatient, the PPVs were 99% in both DNPR-2 and DNPR-3, whereas the corresponding PPVs were 99.6% and 99% on the contact level. Although the algorithm correctly identified all ERs in both study periods, the PPV for ER was 33% in DNPR-2 and 56% in DNPR-3, primarily due to misclassification of outpatient clinics. Still, the proportion of correctly categorized ER visits represented more than 87% of all ER hospital contacts in DNPR-2 and 85% in DNPR-3.

Table 2 Distribution of Hospital Departments and Associated Healthcare Contacts According to Algorithm Using Manual Categorization as Reference

Figure 4 Distributions of hospital unit classifications in the Danish National Patient Registry versions 2 (DNPR-2) and 3 in North Denmark Region.

SOR registrations of ERs had PPVs of 100% in DNPR-2 and DNPR-3 on both unit and contact level (Supplementary Table 1). However, the PPVs of SOR registrations for EWs and associated hospital contacts were 17% and 59% in DNPR-2, which were consistent with the corresponding 33% and 54% in DNPR-3.

Using time intervals from 0 to 12 hours to join unit contacts showed that overnight hospitalizations comprised inpatient contacts in 97–99% in DNPR-2 and DNPR-3 (Table 3). The proportions decreased to 89% in DNPR-2 and 94% in DNPR-3 if 18 hours was allowed between unit contacts. For same-day contacts, inpatient hospitalizations accounted for 5–7% of contacts in DNPR-2 and 2% in DNPR-3, whereas ER contacts reduced from 10–17% to 5–7%, respectively.

Table 3 Distribution of Inpatient, Outpatient, and Emergency Department Contacts by Same-Day or Overnight Hospitalizations According to Different Time Intervals for Joining Department Contacts in North Denmark Region

Allowing 3- to 12-hour intervals to join department contacts (and thereby define overnight hospital stays) yielded a sensitivity of 76–78% for all inpatient hospitalizations in DNPR-2 and DNPR-3 (Table 4). The proportion of overnight stays increased from 69% using 0-hour to 77% using 3-hour intervals among inpatients in DNPR-2. In contrast, the corresponding estimates did not change much in DNPR-3. For outpatients, the relative proportion of overnight contacts was consistently low and ranged from 0.1% to 2% with increasing time intervals. Although the total number of ER contacts decreased substantially with increasing time intervals, ≥99% remained categorized as same-day contacts in both DNPR-2 and DNPR-3.

Table 4 Distribution of Same-Day and Overnight Hospitalizations by Time Interval Between Joined Department Contacts and According to Inpatient, Outpatient, or Emergency Room in North Denmark Region

Discussion

This study showed that an algorithm based on proportions of overnight and elective admissions can accurately categorize inpatient, outpatient, and ER units as well as hospital contacts in DNPR-2 and DNPR-3. In addition, 3- to 12-hour time intervals to join department contacts and subsequent restriction to overnight hospitalizations may be used as a proxy for inpatient contacts but the sensitivity was moderate as a substantial proportion of inpatients were discharged on the day of admission.

Data in the DNPR is often uniquely linked across numerous other registries at the individual level to provide nationwide and population-based trajectories for health-related conditions treated at hospital with virtually complete follow-up.^1–3,14 Yet, gradual changes in content and categorization of information implemented throughout the decades may pose a challenge to the internal validity and use as a research tool.¹ The first version of DNPR was evaluated by the Danish Health and Medicines Authority using medical record review (n = 1000) as reference at Hillerød Hospital in 1980.¹⁵ PPVs of admission data and diagnosis codes were considered unsuitable for research purposes at the time. Two subsequent reports from several hospitals in 1990 and 2002 observed improved quality of administrative and discharge data with overall PPVs of 81% at the three-digit level and 88% when alternative and secondary diagnosis codes were accepted.^16,17 Still, PPVs of diagnosis codes differed considerably at the five-digit level and between disease groups.

Numerous studies have examined the reliability of specific diagnosis codes for various medical conditions in the DNPR since then and consistently found PPVs to be higher after exclusion of ER contacts.¹ However, the omission of the variable “patient type” (ie, inpatient, outpatient, or ER visit) in DNPR-3 has thus far precluded such exclusions in a coherent and reliable manner in different versions of the DNPR. Of note, expanded SOR registrations with a homogenous unit classification across hospitals in DNPR-3 may prove helpful for this purpose in the future, but currently it is only partially implemented in North Denmark Region and thereby of limited use for registry-based studies.

The results of the current study are consistent with a recent paper examining the stability of diagnosis coding of psychiatric outpatient visits in DNPR-2 and DNPR-3.¹⁸ The transition from DNPR-2 to DNPR-3 also substantially increased the overall number of individual hospital contacts from 610,471 in 2018–2019 (DNPR-2) to 1,279,337 in 2019–2020 (DNPR-3). This may affect longitudinal studies in an unpredictable way and is likely due to the new data model in DNPR-3 combined with an increased number of administrative hospital units in the SOR registry.

A provisional 12-hour interval to join department contacts and thereby define inpatient hospitalizations in DNPR-3 was implemented during the COVID-19 pandemic. The development involved elements from a Delphi process such as informal discussions among Danish experts in epidemiology, biostatistics, and data management.⁴ Although this pragmatic approach has been used in numerous studies since then and was guided by significant collective experience with Danish registries, the impact of different time intervals has not been validated until now.^5–7,9,10 In this study, time intervals ranging 0 to 12 hours showed consistently high proportions of inpatient contacts among those categorized as overnight hospitalizations in both DNPR-2 and DNPR-3. However, the sensitivity for capturing all inpatient contacts was moderate (77–78%) and the ability to differentiate outpatient contacts from ER visits remains unsolved. In contrast, the suggested algorithm in the current study may also allow accurate identification of ER contacts.

This study has limitations. Characterizations of hospital departments are not always straightforward and may vary from setting to setting, especially for ED. The reference for computation of PPVs in this study was based on two experienced clinicians, and misclassification of hospital departments cannot be ruled out. However, each department or clinic was contacted in cases of doubt and definitions of ED, ER, and EW were provided to improve clarity. In addition, the level of agreement of hospital unit categorization between assessors was high (Kappa 0.93). Assignment of time stamps by secretaries in the DNPR-3 should represent “time of action”, but procedures and accuracy may vary between hospital departments. Any misclassification is likely non-differential and should not substantially influence associations between time intervals for joining hospital contacts and categorizations as inpatient, outpatient, or ER. The generalizability to other Danish regions is likely high since this was a population-based study of all hospital contacts in North Denmark Region. We are unaware of similar changes in healthcare databases in other countries. However, it is likely that fundamental differences between registries require the development and testing of specific algorithms in each setting.

To summarize, scientists engaged in registry-based studies in Denmark need to consider important changes introduced with DNPR-3, which has disrupted crucial aspects of the data infrastructure such as type of hospitalization (inpatient, outpatient, or ER visits) as well as number and linkage of contacts. The current study provides researchers with a simple and validated algorithm to uniformly characterize hospitalizations as inpatient, outpatient, or ER contacts in both DNPR-2 and DNPR-3.