Registration of acute stroke: validity in the Danish Stroke Registry and the Danish National Registry of Patients

Introduction

Stroke medicine has developed dramatically in recent years with the emergence of new and effective treatment methods and increasing attention paid to implementation of clinical guidelines for treatment and rehabilitation of stroke patients.^1,2 This development has led to the establishment of several stroke-specific registries, which are used for monitoring and improving the quality of stroke care and for research in a wide range of countries, eg, Sweden, UK, Germany, Poland, Austria, US, Australia, China, and Denmark.^3–11 Although the value of such registries is entirely dependent on the validity and completeness of data,¹² few validation studies of stroke-specific registries have been conducted.¹³ In addition, studies that have directly compared the validity of the registration of patients in stroke-specific and hospital discharge registries are largely missing. The aim of this study was to examine and compare the validity of the registration of acute stroke events leading to hospitalization in the Danish Stroke Registry (DSR) and the Danish National Registry of Patients (DNRP).

Materials and methods

Danish Stroke Registry

The DSR was established in 2003 with the aim of documenting and improving the early treatment and care of acute stroke.^11,14 It is mandatory for all 28 Danish hospitals treating patients with acute stroke to report data on selected evidence-based performance measures, which are monitored by the DSR.

All patients aged 18 years or older who are admitted with an acute stroke to a Danish hospital should be registered in the DSR. Data are reported to a central database after discharge of the patient, and detailed instructions on the reporting of data have been provided by the DSR in order to ensure the data validity and completeness (case ascertainment) of the database. Once a year, a structured audit is conducted where the completeness of the database is assessed (ie, the number of persons in the database with stroke/the number of persons with stroke admitted to hospital). Acute stroke should be the primary condition in case the patient is admitted with multiple diagnoses. Data are prospectively collected using a standardized registration form in a multidisciplinary effort involving the entire staff of health professionals involved in the patients’ care, including physicians, nurses, physiotherapists, and occupational therapists. The data are subsequently entered into a web-based database, usually by a nurse or a medical secretary.

Danish National Registry of Patients

The DNRP was established in 1977 as a national administrative registry containing data on all discharges from Danish non-psychiatric hospitals. Reporting to the registry is mandatory and 99.4% of all discharges are registered therein.¹⁵ Data include hospital and department codes, dates of admission and discharge, surgical procedures performed during the hospital stay, and up to 20 diagnosis codes at discharge, defined according to the International Classification of Diseases, 10th edition (ICD-10) since 1994.¹⁶ Diagnosis codes are assigned at discharge by a physician involved in the care of the patient and reported to the registry by a medical secretary. In both the DSR and the DNRP, patients are identified by a unique Civil Registration Number, which is assigned to all Danish citizens at birth or at the time of immigration. The Civil Registration Number enables unambiguous identification and linkage of patients from different registries.

Study design and population

Our study included two approaches. The aim of the first approach was to examine the validity of the registration of patients on a national level based on the assumption that all hospitalized patients diagnosed with an acute stroke in 2009 would most likely be registered in the DSR, DNRP, or both. For this purpose, we identified 25 patients who were only registered in the DSR, 25 patients who were only registered in the DNRP, and 25 patients who were registered in both data sources. The sample sizes were not based on a formal power analysis, but pragmatically determined based on a priori estimation of the expected precision associated with different sample sizes. The patients were randomly identified in the data sources using a computer algorithm which specified that patients should be at least 18 years old, inpatients diagnosed at a Danish hospital with an acute stroke as the primary diagnosis (ICD-10 codes I61, intracerebral hemorrhage; I63, cerebral infarction; and I64, unspecified stroke), and discharged in the period January 1, 2009 to December 31, 2009.

The aim of the second approach was to examine the validity of the registration of patients on a local level, based on four major neurologic wards that had agreed to participate in a more comprehensive study on the feasibility of extending the use of administrative data for quality improvement activities. These four wards were all characterized by a high volume of patients with stroke (approximately 350–950 patients with stroke/year) and extensive experience with diagnostics and treatment of stroke. We reviewed the medical summaries for all patients discharged from the four participating neurologic wards in the period January 4, 2010 to January 10, 2010, focusing on patients with a potential stroke diagnosis, including patients registered with discharge diagnoses of cerebrovascular disease (I60–I69), inflammatory diseases of the central nervous system (G00–G09), episodic and paroxysmal disorders (G40–G47), hemiplegia/paraplegia and tetraplegia/other paralytic syndromes (G81–G83), symptoms and signs involving the nervous and musculoskeletal systems (R25–R29), head injuries (S062–S069), and care involving use of rehabilitation procedures (Z501). This initial sorting of medical summaries was performed in order to focus efforts most effectively on the potentially stroke-related medical records. If the medical summary indicated that the patient had potentially been admitted to hospital due to stroke, we obtained the medical record in order to verify the stroke diagnosis. In all, we retrieved 160 medical summaries.

Medical record review

In the first approach, the medical records, including descriptions of diagnostic imaging procedures (magnetic resonance imaging [MRI] and/or computed tomography [CT] scans), were retrieved. All information was reviewed by a physician (SPJ), and in the event of an uncertain diagnosis, the case was assessed independently by two consultants in neurology (HKI and KV), who made a final decision by consensus. We defined stroke in accordance with the criteria proposed by the World Health Organization, ie, a condition with a presumed vascular etiology leading to a rapid progression of symptoms of loss of neurologic function lasting >24 hours or causing death.¹⁷ Although fulfilling the World Health Organization criteria, we excluded patients with subarachnoid hemorrhage because these patients are not included in the DSR. Patients with subdural hematoma or epidural hemorrhage, retinal infarct or infarct caused by trauma, infection, or an intracranial malignant process, as well as asymptomatic patients with infarct detected only by CT or MRI scan were not confirmed as stroke cases in our study. Similarly, patients with purely diffuse symptoms (eg, isolated vertigo or headache) did not fulfill the diagnostic criteria. A diagnosis of intracerebral hemorrhage was considered confirmed in the presence of clinical symptoms of stroke combined with verification of hemorrhage by CT or MRI scan and the absence of evidence of a previous stroke in the same area. A diagnosis of ischemic infarction was confirmed in the presence of clinical symptoms of stroke with no visualization of hemorrhage by CT or MRI scan. Brain imaging is not required according to the World Health Organization definition; however, all potential patients in our study did have a CT and/or MRI scan. The original MRI and/or CT scans were not reassessed. We excluded one record (1.3%) due to a missing Civil Registration Number (DSR sample) and two records (2.7%) because the available data did not allow a meaningful validation (joint sample), leaving 72 medical records available for data analysis.

In the second approach, medical summaries for patients discharged from the participating neurologic wards in the study period were retrieved and reviewed (by CW and SPJ). On suspected diagnosis of stroke or insufficient information in the summary, we retrieved the medical record and descriptions of diagnostic imaging procedures, and reviewed the material in accordance with the same diagnostic criteria used in the first part of the study (SPJ). Any uncertain diagnoses were settled by the two consultants in neurology. For two patients (1.3%) there were no medical records available to confirm a suspected diagnosis of stroke, and two patients (1.3%) did not have a Civil Registration Number, leaving 156 medical summaries for data analysis.

Permission to conduct the study was obtained from The Danish Data Protection Agency (journal number 2007-58-0016) and The National Board of Health (journal number 7-604-04-67/1/EHE).

Statistical analysis

In the first approach of the study, we assessed the sensitivity and positive predictive value of the registered diagnosis of stroke in the DSR and the DNRP, respectively, using the medical record as the gold standard. We calculated the positive predictive value of stroke in each sample and then, referring to the total number of patients registered with a diagnosis of stroke in the DSR, the DNRP, or in both data sources in 2009, we extrapolated the positive predictive value to the number of patients with stroke on a national level in 2009 in each registry. Using this approach, we were able to estimate the number of verified patients with stroke, and the total number of patients with stroke by gold standard and registry. The sensitivity of the diagnosis of stroke was calculated as the number of patients with a verified diagnosis registered in either the DSR or DNRP divided by the total number of patients with a verified diagnosis of stroke. The positive predictive value was calculated as the number of patients with a verified diagnosis registered in either the DSR or DNRP divided by the total number of patients registered with a diagnosis of stroke in either the DSR or DNRP. Given that the basis of this approach was the assumption that all relevant patients would be registered in either or both of the registries, we were not able to estimate the specificity or the negative predictive value. Since it is complex and debatable how best to estimate the 95% confidence intervals (CIs) accurately when extrapolating to the total stroke population, we did not estimate 95% CIs when extrapolating. Instead, we examined the robustness of our findings by performing additional analyses (“best” and “worst” case scenarios) using the 95% CIs from the positive predictive values of the samples presented above. The “best case scenario” reflected the highest possible extrapolated sensitivity and positive predictive value, whereas the “worst case scenario” represented the opposite. The data analyses in the first approach are described in further detail in the Appendix.

In the second approach, we assessed the sensitivity, specificity, positive predictive value, and negative predictive value of the registered diagnosis of stroke in the DSR and the DNRP, using the medical record as the gold standard. According to the guidelines of the DSR, a patient should not be registered as discharged if he or she is transferred to another hospital ward for further treatment. Because patients are often transferred to other wards from the neurologic ward, the registered date of discharge in the DSR is not necessarily in agreement with the date of discharge from the neurologic ward. We therefore identified patients registered in the DSR with a date of discharge that was within ±30 days of our study period. Given that we included patients according to whether they were discharged in the study period, both long-term and shorter-term hospitalizations were represented. The sensitivity and positive predictive value were calculated as described for the first approach. The specificity was calculated as the number of patients verified not to have had a stroke divided by the total number of patients without a stroke according to their medical record. The negative predictive value was calculated as the number of patients verified not to have had a stroke divided by the total number of patients not registered in the DSR or DNRP with a diagnosis of stroke. The data were analyzed using SAS software version 9.2 (SAS Institute, Cary, NC, USA).

Results

In our sample of 72 patients registered in the DSR, DNRP, or both registries with a diagnosis of stroke in 2009, we were able to verify the diagnosis for 47 patients. Twenty-five patients had not had a stroke as defined by the inclusion criteria for the DSR, of whom the majority were patients with old non-acute strokes and patients with transient ischemic attacks, whereas the rest were admitted with non-stroke related conditions (including unspecified dizziness, migraine, and meningitis); 20 of these unconfirmed patients originated from the DNRP sample, three from the DSR sample, and two from the joint sample. On the basis of these three samples, the positive predictive value in the DSR was calculated as 21/24=88% (95% CI 69%-96%), in the DNRP as 5/25=20% (95% CI 9%-39%), and in both registries as 21/23=91% (95% CI 73%-98%). When extrapolating the positive predictive value to the number of patients with stroke on a national level in 2009, we referred to the total number of patients registered with stroke in one or both of the registries in 2009 (Figure 1 and Appendix). We thus estimated the number of patients with true stroke to be 0.88 × 2,429=2,138 (95% CI 1,676–2,332) in the DSR, 0.20 × 1,722=344 (95% CI 155–672) in the DNRP, and 0.91 × 8,278=7,533 (95% CI 6,043–8,112) in both registries (Figure 2).

Figure 1 Number of patients registered with stroke in the Danish Stroke Registry, the Danish National Registry of Patients, and in both data sources in 2009.

Figure 2 Number of patients with true stroke in the Danish Stroke Registry, the Danish National Registry of Patients, and in both data sources in 2009, extrapolated from review of 75 medical records.

Table 1 shows that based on these samples, the validity of the registration of patients was higher in the DSR than in the DNRP. The sensitivity of the DSR was 97% in contrast to a sensitivity of 79% for the DNRP when using the entire population included in the DSR and DNRP. The positive predictive value of the DSR was 90% compared with 79% in the DNRP. Based on the 95% CIs obtained from the samples of patients included in the medical record review, worst and best case scenarios were assessed. These supplementary analyses showed that the sensitivity of the DSR could in fact be as low as 92% and as high as 99%, whereas the positive predictive value could be as low as 72% and as high as 98%. For the DNRP, the sensitivity could be as low as 73% and as high as 84%, whereas the positive predictive value could vary from 62% to 88%.

Table 1 Sensitivity and positive predictive value for acute stroke in the Danish Stroke Registry and the Danish National Registry of Patients in Denmark, 2009 Notes: *Reflects sensitivity analyses. Abbreviation: PPV, positive predictive value.

Of the 156 medical summaries that were reviewed using the second approach, we identified 74 cases of acute stroke. Table 2 shows that based on data from the second approach, the difference in sensitivity between the two registries was even greater, ie, 91% (95% CI 81%-96%) in the DSR and 58% (95% CI 46%-69%) in the DNRP. The negative predictive value was 91% (95% CI 83%-96%) in the DSR compared with 72% (95% CI 62%-80%) in the DNRP. However, the specificity and positive predictive value did not appear to differ between the registries.

Table 2 Sensitivity, specificity, positive predictive value, and negative predictive value for acute stroke in the Danish Stroke Registry and the Danish National Registry of Patients, based on patients hospitalized at four major departments of neurology in Denmark, January 4–10, 2010 Abbreviations: CI, confidence interval; PPV, positive predictive value; NPV, negative predictive value.

Discussion

Using two different approaches in this study, we found that the sensitivity appeared to be higher in the DSR than in the DNRP, while the positive predictive value was comparable in the two registries.

The study has some important methodologic strengths and limitations. Using two alternative approaches to assess the validity of the registration of hospitalized patients, we aimed to both ensure results that were representative for the entire country (approach 1) and findings that were based on a thorough assessment of all of the hospitalized potential stroke cases, including patients who had not been captured by any of the registries (approach 2). The study design allowed for an estimation of the negative predictive value as well as sensitivity and specificity, which is otherwise rarely assessed in validation studies due to lack of access to an independent reference data source.¹⁸ Further, the medical records were reviewed by a physician, with independent reviews of all difficult cases by two consultants in neurology. Only a negligible number of medical records could not be found or were excluded due to insufficient information. It appeared from our review of medical records in approach 2 that 82 of 156 admissions to the four neurologic wards were not due to an acute stroke. Other diagnoses included transient ischemic attack, sequelae of stroke, subdural hematoma, and intracerebral tumors, implying that patients admitted to these wards did have neurologic symptoms potentially indicating a stroke, which warranted their admission to a neurologic ward.

The limitations of the approaches used in our study should, however, be noted. In approach 1, we relied on the assumption that all hospitalized patients with acute stroke would be registered in the DSR and/or the DNRP, although it is evident that there will be exemptions. Hospitalized patients not registered with an acute stroke may include surgical patients with minor stroke or patients admitted with other serious underlying conditions such as cancer or hematologic illness, or they may be inadequately diagnosed and thus misclassified by the hospital ward where they were initially admitted. Further, the number of patients/records involved in approach 1 was relatively small, precluding us from making otherwise relevant subanalyses (eg, according to type of stroke or patient age). In approach 2, we only included information from four neurologic wards. These wards were characterized by a high patient volume and had agreed to take active part in a larger project. Therefore, they were not necessarily representative of Danish neurologic wards in general. However, the overall high agreement between findings from the two approaches indicates that these limitations did not have any major impact on the study findings. Finally, only one non-stroke specialist read all the medical records.

Our data indicated a higher sensitivity in the DSR than in the DNRP, in addition to a slightly higher positive predictive value in the DSR. This may, in part, be attributed to different ways of collecting and reporting data on acute stroke events to the two registries; for the DSR, data collection and reporting relies on a standardized registration form prospectively filled out by the health professionals responsible for the treatment and care of the patient. The collection of data is a multidisciplinary effort with multiple staff members, including physicians, nurses, physiotherapists, and occupational therapists, being involved with each patient and a detailed, large dataset is recorded for each patient. In addition, yearly audit reports are published by the DSR based on the performance of the individual wards, and these may enhance the commitment of the staff involved to ensure the validity of data reported to the DSR. In contrast, for the DNRP, only a physician is involved in coding of the diagnosis. The physician has typically been involved in the treatment of the patient, but does not necessarily have detailed knowledge about the entire admission or disease history. Further, detailed instructions on the coding of diagnoses in the DNRP are not available, and the coding of diagnosis may potentially be influenced by economic incentives due to use of the Diagnosis-Related Group system. Finally, the staff receive no direct feedback on the reported data, which may negatively influence the interest and sense of ownership of the data.

Generally, previous studies are characterized by use of different gold standards in the assessment of the validity of registries, different definitions of relevant diagnosis codes to validate as well as varying use of diagnostic procedures (such as CT or MRI scans), all of which complicate direct comparisons of existing results. However, several studies have reported a high positive predictive value of stroke in hospital discharge registries,^18–28 and a few studies have reported moderate positive predictive value.^13,29–32 Fewer studies have examined the sensitivity of discharge registries; a sensitivity of 76%–82% was reported from an Italian hospital discharge register,²⁵ whereas a sensitivity of 88%–93% was found in Swedish and Norwegian hospital discharge registers,^13,28,31,33 85%–97% in a Finnish hospital discharge register,^26,27 and 76% in an American hospital discharge register.³² To our knowledge, only one study has investigated the validity of patient registration in a stroke-specific registry; using an estimated “true” number of stroke cases in the population as the gold standard, Stegmayr and Asplund reported a sensitivity of 96% and a positive predictive value of 100% in the Northern Sweden MONICA (Multinational MONItoring of trends and determinants in CArdiovascular disease) registry compared with values of 93% and 68%, respectively, in hospital discharge registers.¹³ These results are in accordance with our findings, although the positive predictive value of the MONICA registry was slightly higher than that of the DSR; this may be partly due to exclusion of patients over 74 years of age in the Swedish study, for whom diagnosing of stroke may be less accurate. This suggests that a greater validity of stroke-specific registries may also apply to registries in other countries. However, further studies are needed to confirm this assumption.

We recommend that further efforts are needed if DNRP is to be used for monitoring of stroke epidemiology and stroke care. Such efforts should probably both involve further refinement of the algorithms used for identifying stroke events in the DNRP (eg, in line with recent experiences from Sweden)²⁸ and improvements in the registration practice with the registration of data becoming a more integrated part of the work flow during clinical work combined with more emphasis on training and instructions to staff involved in the coding of diagnoses to the DNRP. The DSR currently appears to have better data validity than the DNRP and appears to be a useful data source, but continued attention is required in order to improve the positive predictive value further.

Conclusion

Registration of acute stroke in the DSR registry overall appeared to have high validity. However, further efforts are warranted to improve the validity of the DNRP and precautions should be taken if using the registry for administrative, quality improvement, and research purposes.

Acknowledgments

The authors wish to thank Søren Vingtoft, Center for Health Informatics and Quality of Care (East), for contributing to the collection of data, and Steffen Høgskilde, Iq-Care Systems, for contributing data support and management. The study was supported by The Danish Public Welfare Technology Foundation.

Disclosure

The authors report no conflicts of interest in this work.

---

References

1.	Jauch EC, Saver JL, Adams HP Jr, et al. Guidelines for the early management of patients with acute ischemic stroke: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke. 2013;44(3):870–947.
2.	Ringleb PA, Bousser M-G, Ford G, et al. Guidelines for management of ischaemic stroke and transient ischaemic attack, 2008. Available from: http://www.eso-stroke.org/pdf/ESO08_Guidelines_Original_english.pdf. Accessed June 24, 2013.
3.	Asplund K, Hulter Asberg K, Appelros P, et al. The Riks-Stroke story: building a sustainable national register for quality assessment of stroke care. Int J Stroke. 2011;6(2):99–108.
4.	Grau AJ, Eicke M, Biegler MK, et al. Quality monitoring of acute stroke care in Rhineland-Palatinate, Germany, 2001–2006. Stroke. 2010;41(7):1495–1500.
5.	Heuschmann PU, Wiedmann S, Wellwood I, et al. Three-month stroke outcome: the European Registers of Stroke (EROS) investigators. Neurology. 2011;76(2):159–165.
6.	Rudd AG, Hoffman A, Irwin P, Lowe D, Pearson MG. Stroke unit care and outcome: results from the 2001 National Sentinel Audit of Stroke (England, Wales, and Northern Ireland). Stroke. 2005;36(1):103–106.
7.	Schwamm LH, Fonarow GC, Reeves MJ, et al. Get with the guidelines – stroke is associated with sustained improvement in care for patients hospitalized with acute stroke or transient ischemic attack. Circulation. 2009;119(1):107–115.
8.	Steiner MM, Brainin M; Austrian Stroke Registry for Acute Stroke Units. The quality of acute stroke units on a nation-wide level: the Austrian Stroke Registry for acute stroke units. Eur J Neurol. 2003;10(4):353–360.
9.	Cadilhac DA, Lannin NA, Anderson CS, et al. Protocol and pilot data for establishing the Australian Stroke Clinical Registry. Int J Stroke. 2010;5(3):217–226.
10.	Wang Y, Cui L, Ji X, et al. The China National Stroke Registry for patients with acute cerebrovascular events: design, rationale, and baseline patient characteristics. Int J Stroke. 2011;6(4):355–361.
11.	Mainz J, Krog BR, Bjornshave B, Bartels P. Nationwide continuous quality improvement using clinical indicators: The Danish National Indicator Project. Int J Qual Health Care. 2004;16 Suppl 1: i45–i50.
12.	Sorensen HT, Sabroe S, Olsen J. A framework for evaluation of secondary data sources for epidemiological research. Int J Epidemiol. 1996;25(2):435–442.
13.	Stegmayr B, Asplund K. Measuring stroke in the population: quality of routine statistics in comparison with a population-based stroke registry. Neuroepidemiology. 1992;11(4–6):204–213.
14.	Ingeman A, Pedersen L, Hundborg HH, et al. Quality of care and mortality among patients with stroke: a nationwide follow-up study. Med Care. 2008;46(1):63–69.
15.	Andersen TF, Madsen M, Jorgensen J, Mellemkjoer L, Olsen JH. The Danish National Hospital Register. A valuable source of data for modern health sciences. Dan Med Bull. 1999;46(3):263–268.
16.	World Health Organization. ICD-10 version; 2010. Available from: http://apps.who.int/classifications/icd10/browse/2010/en. Accessed August 9, 2013.
17.	Cerebrovascular diseases: Prevention, treatment, and rehabilitation. Report of a WHO meeting. World Health Organ Tech Rep Ser. 1971;469:1–57.
18.	Johnsen SP, Overvad K, Sorensen HT, Tjonneland A, Husted SE. Predictive value of stroke and transient ischemic attack discharge diagnoses in The Danish National Registry of Patients. J Clin Epidemiol. 2002;55(6):602–607.
19.	Spolaore P, Brocco S, Fedeli U, et al. Measuring accuracy of discharge diagnoses for a region-wide surveillance of hospitalized strokes. Stroke. 2005;36(5):1031–1034.
20.	Gaist D, Vaeth M, Tsiropoulos I, et al. Risk of subarachnoid haemorrhage in first degree relatives of patients with subarachnoid haemorrhage: follow up study based on national registries in Denmark. BMJ. 2000;320(7228):141–145.
21.	Krarup LH, Boysen G, Janjua H, Prescott E, Truelsen T. Validity of stroke diagnoses in a national register of patients. Neuroepidemiology. 2007;28(3):150–154.
22.	Leppala JM, Virtamo J, Heinonen OP. Validation of stroke diagnosis in the National Hospital Discharge Register and the Register of Causes of Death in Finland. Eur J Epidemiol. 1999;15(2):155–160.
23.	Lindblad U, Rastam L, Ranstam J, Peterson M. Validity of register data on acute myocardial infarction and acute stroke: the Skaraborg Hypertension Project. Scand J Soc Med. 1993;21(1):3–9.
24.	Liu L, Reeder B, Shuaib A, Mazagri R. Validity of stroke diagnosis on hospital discharge records in Saskatchewan, Canada: implications for stroke surveillance. Cerebrovasc Dis. 1999;9(4):224–230.
25.	Rinaldi R, Vignatelli L, Galeotti M, Azzimondi G, de Carolis P. Accuracy of ICD-9 codes in identifying ischemic stroke in the General Hospital of Lugo di Romagna (Italy). Neurol Sci. 2003;24(2):65–69.
26.	Mahonen M, Salomaa V, Keskimaki I, et al. The feasibility of combining data from routine hospital discharge and causes-of-death registers for epidemiological studies on stroke. Eur J Epidemiol. 2000;16(9):815–817.
27.	Tolonen H, Salomaa V, Torppa J, Sivenius J, Immonen-Raiha P, Lehtonen A. The validation of the Finnish Hospital Discharge Register and Causes of Death Register data on stroke diagnoses. Eur J Cardiovasc Prev Rehabil. 2007;14(3):380–385.
28.	Koster M, Asplund K, Johansson A, Stegmayr B. Refinement of Swedish administrative registers to monitor stroke events on the national level. Neuroepidemiology. 2013;40(4):240–246.
29.	Hasan M, Meara RJ, Bhowmick BK. The quality of diagnostic coding in cerebrovascular disease. Int J Qual Health Care. 1995;7(4):407–410.
30.	Goldstein LB. Accuracy of ICD-9-CM coding for the identification of patients with acute ischemic stroke: effect of modifier codes. Stroke. 1998;29(8):1602–1604.
31.	Ellekjaer H, Holmen J, Kruger O, Terent A. Identification of incident stroke in Norway: Hospital discharge data compared with a population-based stroke register. Stroke. 1999;30(1):56–60.
32.	Leibson CL, Naessens JM, Brown RD, Whisnant JP. Accuracy of hospital discharge abstracts for identifying stroke. Stroke. 1994;25(12):2348–2355.
33.	Appelros P, Terent A. Validation of the Swedish inpatient and cause-of-death registers in the context of stroke. Acta Neurol Scand. 2011;123(4):289–293.

---

Appendix

Estimating the sensitivity and positive predictive value (PPV) of acute stroke diagnoses in the Danish Stroke Registry (DSR) and the Danish National Registry of Patients (DNRP) using approach 1

In approach 1, we sampled the patients/records to be reviewed not simply randomly among all patients in the registries; rather, we randomly selected 25 patients who were registered in the DSR alone, 25 patients registered in the DNRP alone, and 25 patients registered in both data sources. Table 1 presents the distribution of the random samples according to whether the acute stroke diagnosis could be confirmed by medical record review.

Table 1 Distribution of random samples of acute stroke diagnoses according to data source and medical record review Notes: aOne patient was excluded due to lack of a Civil Registration Number; btwo patients were excluded due to lack of diagnostic information in the medical records.

Based on these data, the PPV of an acute stroke diagnosis could be computed in the DSR only, the DNRP only, and the DSR + DNRP, respectively.

PPV of acute stroke diagnosis

PPV among patients only registered in the DSR: 21/24=0.88 (95% CI 0.69–0.96).

PPV among patients only registered in the DNRP: 5/25=0.20 (95% CI 0.09–0.39).

PPV among patients registered in both the DSR and DNRP: 21/23=0.91 (95% CI 0.73–0.98).

Extrapolations based on the PPVs were made using the total number of patients registered with acute stroke in the different data sources in 2009:

2,429 patients were only registered in the DSR.

1,722 patients were only registered in the DNRP.

8,278 patients were registered in both the DSR and the DNRP.

A total of 12,429 patients were registered with acute stroke in 2009.

It is complex to estimate the 95% CIs accurately when extrapolating to the total stroke population; however, it is possible to do sensitivity analyses using the 95% CIs from the PPVs of the samples presented above. “Best case scenario” reflects the highest possible extrapolated sensitivity and PPV, whereas “worst case scenario” represents the opposite. An example: Keeping in mind that in approach 1, we assume that all stroke patients are registered in the DSR and/or the DNRP, the best case scenario for DSR will be a situation where the true PPVs from the random samples were equal to the upper limit of the 95% CIs for registration only in the DSR, and for registration both in the DSR and the DNRP, whereas the PPV for registration only in the DNRP was equal to the lower limit of the estimated 95% CI.

Extrapolation to the number of patients registered with acute stroke in 2009

True acute stroke among patients only registered in the DSR: 0.88*2,429=2,138.

Worst case scenario: 0.69*2,429=1,676.

Best case scenario: 0.96*2,429=2,332.

True acute stroke among patients only registered in the DNRP: 0.20*1,722=344.

Worst case scenario: 0.09*1,722=155.

Best case scenario: 0.39*1,722=672.

True acute stroke among patients registered in both the DSR and DNRP: 0.91*8,278=7,533.

Worst case scenario: 0.73*8,278=6,043.

Best case scenario: 0.98*8,278=8,112.

On this background, the validity of the acute stroke diagnosis in the DSR (Table 2) and the DNRP (Table 3) were estimated.

Table 2 Distribution of patients registered with acute stroke according to the Danish Stroke Registry and medical record review Notes: Sensitivity: 9,671/10,015=96.6%; Worst case scenario: 7,719/8,391=92.0%; Best case scenario: 10,444/10,599=98.5%. Positive predictive value: 9,671/10,707= 90.3%; Worst case scenario: 7,719/10,707=72.1%; Best case scenario: 10,444/ 10,707=97.5%.

Table 3 Distribution of patients registered with acute stroke according to the Danish National Registry of Patients and medical record review Notes: Sensitivity: 7,877/10,015=78.7%; Worst case scenario: 6,198/8,530=72.7%; Best case scenario: 8,784/10,460=84.0%. Positive predictive value: 7,877/10,000=78.8%; Worst case scenario: 6,198/10,000=62.0%; Best case scenario: 8,784/10,000=87.8%.