The positive predictive value of spontaneous subarachnoid hemorrhage diagnoses in the Danish National Patient Register

Introduction

Spontaneous (non-traumatic) subarachnoid hemorrhage (ntSAH) is often suspected in the emergency department but not commonly diagnosed. Many aspects regarding recognition, treatment and outcome for ntSAH are still investigated and registers can be an effective data source, provided sufficient accuracy. This depends on data quality and the degree with which all cases are included.¹ Awareness of this is key if existing registers are used for research.

The Danish National Patient Register (DNPR) is an often-used data source for epidemiological research in Denmark. It is mandatory for private and public hospitals to report all health care contacts and 99.4% of admissions and outpatient contacts are registered.² The overall categories of variables are admission and discharge diagnoses, examinations, interventions and length of stay.^1,3

However, diagnostic accuracy in the DNPR may vary. This can be due to several factors, eg, the frequency with which diagnoses occur. As an example, the positive predictive value (PPV) of cardiac diagnoses that are often seen in clinical practice have been found to be higher than for more rare diagnoses.⁴ Also, the PPV of registrations in the DNPR performed by specialized departments tend to be higher than those performed by non-specialized departments.⁵

If registrations of ntSAH in the DNPR is to be used for research purposes or development of clinical practice, data quality must first be established.

The main objective of this study was to determine the PPV of the ntSAH diagnosis in the DNPR.

The secondary aims were to describe the patients registered with a diagnosis of ntSAH, but who were found to not have had a ntSAH, and to estimate the incidence rate of ntSAH.

Methods

Ethical aspects and approvals

Data were extracted by local department staff with prior approval from the Danish Data Protection Agency, the Danish Health and Medicines Authority and the chairmen of the individual departments.

Design

In this cohort study diagnoses in the DNPR were validated against medical records by two independent blinded physicians.

Defining the cohort

From the DNPR we extracted data on patients ≥18 years of age, admitted between January 1, 2008 and December 31, 2014 to any hospital in the Capital Region of Denmark, who were discharged with a primary diagnosis of ntSAH. This was defined as International Classification of Diseases version 10 (ICD-10) codes I60.0 through I60.9 (subarachnoid hemorrhage from carotid siphon and bifurcation; subarachnoid hemorrhage from middle cerebral artery; subarachnoid hemorrhage from anterior communicating artery; subarachnoid hemorrhage from posterior communicating artery; subarachnoid hemorrhage from basilar artery; subarachnoid hemorrhage from vertebral artery; subarachnoid hemorrhage from other intracranial arteries; subarachnoid hemorrhage from intracranial artery, unspecified; other subarachnoid hemorrhage and subarachnoid hemorrhage, unspecified). Data extraction from the DNPR was restricted to inpatient contacts but was not restricted to neither acute nor non-acute admissions. In cases of spontaneous subarachnoid hemorrhage as a complication to medical or surgical treatment patients were included as these should rightfully be registered as ntSAH in the DNPR. No secondary diagnoses were extracted as we wished to focus on patients were ntSAH was the primary cause of admission. We excluded patients transferred from hospitals outside the region and patients with onset while abroad to ensure full access to the medical records as well as to focus on the incidence rate within the geographical entity of the Capital Region of Denmark. Patients were also excluded if they had a DNPR-registration within the study period but admission took place prior to this.

A unique identification number is assigned to all Danish citizens at birth or immigration; a Central Person Register number (CPR-number).⁶ The CPR-number is used in all aspects of public administration, health services and social services. In the DNPR patients are identified by the CPR-number.^1–3

Data sources and collection

CPR-numbers and hospital identification numbers were extracted from the DNPR for patients registered with ntSAH. These were then used to identify the medical records for each patient. Three electronic medical record systems were used in the region (Opus, Epic and GS Open). Also, two archives with paper versions of medical records and an electronic database (Onbase) with scanned versions of medical information were accessed.

The data extraction process was two-fold. First, we assessed if the ntSAH-diagnosis could be confirmed. For this we used specialist assessments, radiological descriptions, discharge letters, medical summaries, ambulance reports, and the medical record. Radiological images and laboratory results were not reinterpreted. A confirmed diagnosis required a conclusion of ntSAH in the medical record supported by a computer-tomography scan with blood in the subarachnoid space or xanthochromia on spinal fluid analysis. Secondly, we described the patients without confirmed ntSAH using a case report form (CRF) developed on the basis of the first 100 medical records (Figure S1). The main categories of this was traumatic subarachnoid hemorrhage (tSAH); parenchymal hemorrhage; scheduled outpatient follow-up; suspected ntSAH not confirmed and ntSAH never suspected. No restrictions were made on which time periods of the medical record was accessed to gather data. Data were stored in Microsoft Excel and statistical analysis performed in SAS Enterprise Guide statistical software package version 7.1.

Blinding

Investigator 1 (AS) went through the medical records of all patients and data were saved in an Excel spreadsheet on a secure hard drive. Investigator 2 (JBA) then repeated the process blinded to the findings of investigator 1, by not having access to the secure hard drive. Data from investigator 2 were saved in a separate spreadsheet on a separate secure hard drive. Finally, we compared the two spreadsheets and in the case of disagreement the investigators discussed the case until agreement was reached.

Statistical analysis

In order to report the PPV of ntSAH in the DNPR, we defined the proportion of patients with a confirmed ntSAH as the true positive number of cases and all eligible patients as the total number of positive cases. PPV, proportions and incidence rates were reported with 95% confidence intervals and continuous data as medians with interquartile range.

To calculate the incidence rate, we retrieved the size of the background population of people 18 years of age or older from national statistics at Statistics Denmark. As a substantial number of people commuted in and out of the Capital Region of Denmark everyday, we defined the population at risk by averaging day- and nighttime populations over the years 2008–2014. The daytime population was defined as the number of people with awork address in the region plus people outside the workforce with a registered address in the region. We defined the nighttime population as the number of people with an address in the region.

Results

We identified 1,101 individuals in the DNPR, over the age of 18 years on admission, who had been admitted to a hospital in the Capital Region of Denmark between January 1, 2008 and December 31, 2014 and were discharged with a primary diagnosis of ntSAH (Figure 1).

Figure 1 Flowchart for study of patients registered in the Danish National Patient Register (DNPR) with a diagnosis of non-traumatic subarachnoid hemorrhage (ntSAH).Abbreviations: ntSAH, non-traumatic subarachnoid hemorrhage; n, number; SAH, subarachnoid hemorrhage.

Data extraction from medical records was conducted between spring 2015 and summer 2018 locally at the departments where the patients had been admitted to. We visited a total of 26 departments at 9 different hospitals across the Capital Region of Denmark. Furthermore, paper records from two archives were retrieved.

Medical records could not be retrieved for 11 (1.0%) patients. In 21 (1.9%) patients only one investigator could assess the medical records because of changes in IT-systems. Of 1069 (97.1%) records reviewed twice, 227 (21.2%) were excluded. Of these, 210 (92.5%) were patients transferred from other geographical regions, 16 (7%) were abroad during onset of their hemorrhage and one (0.44%) was admitted prior to the study period. The remaining 842 (78.8%) were included for analysis.

A non-traumatic bleed into the subarachnoid space was confirmed in 537 of 842 cases, giving a PPV of 63.8% (95% CI: 60.5–67.0%). This includes ntSAH from vascular malformations, perimesencephalic hemorrhages and complications to surgical or medical treatment. Including only suspected or confirmed hemorrhages from aneurysms (n=493) the PPV was 58.5% (95% CI: 55.2–61.9%).

Recurrent ntSAH’s comprised 8 of the 537 cases (1.5%). Twelve (2.2%) occurred while the patient was in a hospital; either as a relative, an employee or during an unrelated admission. Another nine (1.7%) developed ntSAH as a complication to treatment (surgical, neurosurgical or medical). Twelve (2.2%) patients had a prehospital cardiac arrest and a confirmed ntSAH (Table 1).

Table 1 Sub-group characteristics of selected groups with confirmed non-traumatic subarachnoid hemorrhage registered in the Danish National Patient Register

In 305 of 842 patients (36.2% (95% CI: 33.0–39.5)) ntSAH was not confirmed. The majority had a tSAH (40%); 18.9% had been suspected of ntSAH which was later not substantiated; 14.8% did not have a ntSAH in the study period but had a scheduled follow-up visit in a neurosurgical department after a previous SAH or because they had a known aneurysm; 14.5% had experienced a parenchymal hemorrhage and 0.3% (n=1) had developed symptoms of ntSAH during admission, which was later not substantiated. In 11.5% (n=35) ntSAH had never been suspected (Table 2). Of the latter, 18 had been admitted to general medical departments, 6 to neurosurgical departments, 6 to neurologic departments, 2 to departments of respiratory medicine, 2 to departments of cardiology and 1 to an emergency department.

Table 2 Characteristics of patients found not to have had non-traumatic subarachnoid hemorrhage despite being registered with this in the Danish National Patient Register

Of the patients with confirmed ntSAH (n=537) 62.9% (95% CI: 58.9–67.0) (n=338) were women. The median age was 59.0 years (IQR: 48–70 years) among women and 57.0 years (IQR: 47–66 years) (n=199) among men. Including only those with confirmed or suspected aneurysmal hemorrhage (n=493) 65.7% (95% CI 61.5–69.9) (n=324) were women and their median age was 59.0 years (IQR: 48.0–70.0 years) while men (n=169) had a median age of 57.0 years (IQR: 48.0–66.0 years).

Of the 32 patients with missing medical records the majority were women, and all had been admitted between 2008 and 2010 (Table 3).

Table 3 Characteristics of patients registered with non-traumatic subarachnoid hemorrhage in the Danish National Patient Register that were not available for double review

Incidence rate

The population at risk in the Capital Region of Denmark was between 1,312,590 (year 2008) and 1,406,830 (year 2014) at night and between 1,390,647 (year 2008) and 1,478,722 (year 2014) during daytime, averaging 1,393,359 persons. Having confirmed 537 diagnoses of ntSAH, this corresponds to an incidence rate of 5.5 per 100,000 persons per year (95% CI: 4.9–6.1). Including only those of confirmed or suspected aneurysmal origin the incidence rate is 5.1 per 100,000 (95% CI: 4.5–5.6) persons per year.

Discussion

We were able to confirm the diagnosis of ntSAH in 63.8% of cases. The main categories of non-ntSAH cases were tSAH, suspected ntSAH that could not be substantiated during admission, parenchymal hemorrhage and scheduled examinations and procedures. Based on our findings, the incidence rate of ntSAH in the Capital Region of Denmark is 5.5 per 100.000 persons per year (95% CI: 4.9–6.1).

The major strength of this study is the large sample size, which is unlike any other validation of ntSAH in the DNPR we are aware of. Also, medical records were reviewed twice by independent investigators, which further increases the validity. Finally, our study includes all adult patients registered with ntSAH from any department in the region.

However, this study is subject to limitations too. Firstly, medical records did not always state a definitive cause of hemorrhage but the attending specialists´ “best estimate”. Also, an ICD-10 diagnosis may indicate a specific hemorrhage site, but we chose not to assess if these were correct as it would have required reinterpretation of radiological imaging, which was outside the scope of this study. Using ICD-10 discharge codes I60.0-I60.9 our results are of any non-traumatic bleeding in the subarachnoid space. Hence, we do not distinguish between ntSAH caused by ruptured aneurysms, perimesencephalic hemorrhages, vascular malformations, tumors or due to treatment complications.

Only those alive to admission are included, meaning that the incidence rate does not include the patients with ntSAH who died in the prehospital phase. It also does not include those patients with ntSAH erroneously registered in DNPR under other diagnoses. Further, the reported incidence rate is based on the estimated population at risk as the exact number of people at risk within the region at any given time cannot be determined.

Also, the two investigators initially disagreed on a substantial number of patients. A measure of interrater-variability would have strengthened this methodological limitation. Unfortunately, our study was not designed to accommodate this.

Another potential bias is the several changes in electronic medical records systems used during our study period. This posed severe challenges to data access. We experienced that data on deceased patients were not consistently transferred to the new systems. Great effort was done to overcome these barriers, but we were unsuccessful in 32 cases. This could bias our results in the direction of not having included patients that died. Had both investigators accessed all medical records at the same time, this bias might have been eliminated. For practical reasons this was not possible, however.

The true number of patients with ntSAH is not known as a diagnosis-specific register for ntSAH was not established at the time of our study. Therefore, the sensitivity of the DNPR for ntSAH cannot be calculated.

The only other larger validation of ntSAH in DNPR in recent years was a substudy of the large Danish “Diet, Cancer and Health”-study.⁷ In this, ntSAH was confirmed in 60.6% (95% CI: 53.3–67.7) of patients when DNPR-diagnoses were validated against medical records. The study was limited, however, to patients aged 50–64 years at enrolment in 1993–1997⁷ and as such cannot automatically be presumed representative of all ages or present-day.

The PPV of ntSAH seems to depend on the level of specialization of the department from which the diagnosis is registered. It was 93% (95% CI: 85–98%) from neurosurgical departments but as low as 47% (36–59%) from other nonspecialized departments in a previous Danish study.⁸ A PPV of 63.8% in the present study on a population of patients from a variety of departments is in line with both of these studies. However, compared to a validation of the Finish hospital discharge register the Danish validity is rather low, as the Finish was found to be 87% (83–91%).⁹

The largest group of incorrect registrations in our study were tSAH. This could be a result of inaccuracy when choosing the ICD-10-code, which is supported by the observation that ntSAH had never been suspected in 35 of the patients. In another 45 patients, medical records revealed scheduled outpatient follow-ups or surgery which could also be a result of inaccuracy in registration or a lack of more appropriate options when registering. These appeared as new cases in the DNPR and could introduce a bias if unvalidated DNPR-data are used.

Generally, the etiology of ntSAH is reported as 80–85% aneurysmal, 10% perimesencephalic and 5–10% being a mix of vascular malformations, tumors etc.^10,11 Apart from a somewhat smaller proportion of vascular malformations and so on, our distribution is similar to that reported by others. The proportion of patients that presented with out-of-hospital cardiac arrest (OHCA) in our study was also comparable to the 3% reported by van Gijn¹¹ but substantially lower that the 11% reported in a 2011 Japanese study¹² indicating the possibility of a geographical difference.

With a median age of 59.0 years for women and 57.0 years for men our cohort is older than in most other studies, but with 62.9% women the gender distribution is rather similar.^8,13–19 There is, however, great discrepancies in patients’ age between studies: a Canadian study of 130 patients with ntSAH found a mean age of 43.3 years (range 16–93);²⁰ a recent British study of 3,341 patients with aneurysmal SAH had a median age of 55 (IQR 46–64)²¹ and a Danish study of 1,076 patients with ntSAH, published in 1987, had a median age of 49 years (range 14–77) for women and 48 years (range 12–76) for men.²² In contrast Yamada reported a mean age of 65.3 years (SD 14.8) in a study of 494 patients²³ and a systematic review of 15 papers reported a mean age of 59 years for women and 55 for men in an analysis of 2,155 patients.²⁴ As the prognosis after ntSAH has been found to be worse with advanced age,²¹ studies based on unvalidated data may potentially be reporting a worse overall outcome than if based on validated data on a younger population.

Geographical differences in the incidence rate of ntSAH is well-known. In Western populations 6–10:100,000 per year is generally reported.^10,11 Danish incidence rates have been reported as 9.25:100,000 person-years based on medical record reviews⁸ and as high as 12.3 (adjusted to European Standard Population) based on unvalidated diagnoses of the DNPR and the Cause-of-Death-Register.²⁵ In line with this, a German study reported an incidence rate of 11.3:100,000 based on unvalidated diagnoses from a national register.¹⁸ It is likely that the incidence rate of ntSAH is overestimated when non-validated databases are used as data sources.

Deaths before contact with health care professionals or admission are often not included when reporting incidence rates. A systematic review from 2002 found that overall 12.4% of patients experiencing ntSAH died before admission.²⁶ This proportion could probably be added to our cohort as well, to approach amore accurate incidence rate, but this is speculative.

Large diagnosis registers such as the DNPR can be important sources of data for studies on rare diagnoses. High data quality and completeness is essential to ensure that correct assumptions can be made about the cohort. This is why the DNPR is constantly evaluated. Most often the data quality is monitored by estimating the PPV´s using medical record review as the reference standard and data completeness by comparison to other data sources.¹ The PPV of stroke diagnoses (ntSAH was not specifically evaluated) has been validated against the Danish Stroke Register and medical records and ranged from 62% to 88% which led the investigators to conclude that precautions should be taken if DNPR data were to be used for research purposes.² Our present findings support this and, assuming that not many cases are missed due to an imperfect sensitivity of the DNPR for ntSAH, the low PPV of 63.8% can have important implications on studies. The high proportion of patients without ntSAH could lead to a gross overestimation of the incidence rate of ntSAH; the results of potential improvements in treatment could be masked and studies on longterm outcomes could give the wrong image of the prospects for these patients.

Conclusion

We found a positive predictive value (PPV) of 63.8% for the diagnosis of non-traumatic subarachnoid hemorrhage (ntSAH) in the Danish National Patient Register. Of the remaining patients, the majority had had a traumatic SAH or a parenchymal hemorrhage. The incidence rate of ntSAH was 5.5 per 100,000 persons per year. The low PPV suggests that care should be taken if unvalidated data are used for research and that results of previous studies should be interpreted with caution.