Reliability of recurrent pregnancy loss diagnosis coding in the Swedish National Patient Register: a validation study

Introduction

Miscarriage is one of the most common early pregnancy complications. When it happens repeated times it is not only bad luck but is considered as a disease defined as recurrent pregnancy loss (RPL).^1–3 RPL is traditionally defined as three or more consecutive miscarriages before the fetus reaches viability, ie, before 22 gestational weeks.⁴ Recently the definition has been debated and changed by the European Society of Human Reproduction and Embryology (ESHRE) and the Practice Committee of the American Society for Reproductive (ASRM) to as few as two miscarriages.^2,5–7

The prevalence of RPL is stated to be 0.5–2.3%,^1,2,8–10 and the incidence has been estimated at 650/100,000 pregnant women based on the results from a retrospective register study in Sweden.¹¹ Estimating the incidence of RPL is somewhat difficult because of a lack of consensus regarding the definition of RPL but also due to uncertainty about whether to include biochemical miscarriages (positive urinary human chorionic gonadotropin (hCG), a raised serum hCG) or only clinical miscarriages verified with ultrasound or histology.^2,3,12,13 The above incidence of RPL in Swedish women is to our knowledge the first study on the RPL incidence^11,13 as others report an occurrence or a prevalence of PRL.^1,9 The incidence based on registers will also depend on if women with miscarriage get in contact with a physician in the health care system.¹³ This often happens if bleeding occurs in early pregnancy or a routine scan is performed.

Women and/or couples with RPL are often in distress. However, a possible explanation for why they miscarry is only found in 40–50% of the cases. Parental chromosomal translocations, congenital and acquired uterine abnormalities, endocrine imbalances, autoimmune factors including the antiphospholipid syndrome, as well as infections and thrombophilia, are all possible causes.^10,12,14,15

Many epidemiological studies on RPL are observational and based on data from registers that are available for research purposes. Such studies are also important to improve the quality of health care, and to manage health care services.^16,17

One such register is the Swedish National Patient Register (NPR), which was used in the above incidence study on RPL.^11,17 Clearly, it is important that the data in the registers are valid and representative. A recent review stated that the accuracy for many diseases in the NPR register ranged from 85% to 95%.¹⁶

To our knowledge, there is no evidence regarding the precision of RPL in the NPR. Such evidence is essential for the use and interpretation of RPL based on NPR data. The aim of this study was to validate the ICD-10 diagnoses codes registered in the NPR for RPL (N96.9, O26.2). A secondary aim was to analyze whether only miscarriages verified by ultrasound or also pregnancy losses verified only by urine or serum hCG are registered in the NPR and used for the RPL diagnosis.

Material and methods

Study design

A cross-sectional observational study designed was used to validate the RPL diagnosis registered in the NPR compared with the information obtained in the medical records. The medical records were considered as “reference standard”. In order to assess whether a diagnosis of RPL was defined as three or more consecutive miscarriages, the medical records were reviewed.

The main aim was to evaluate the positive predictive value (PPV), defined as the proportion of women registered as RPL in the NPR with a diagnosis verified by information from medicals records. Sensitivity was assessed and defined as the proportion of women suffering RPL and registered with the diagnosis in the NPR.

Population

In our previous study, a cohort of 6,852 women was diagnosed with RPL in the NPR over a 10-year period in Sweden.¹¹ We assumed that only one-third of the medical records requested from hospitals would be available and/or would provide the necessary information. A sample size of 711 women, registered with the ICD-10 diagnoses codes N96.9 and/or O26.2 were randomly selected by the National Board of Health and Welfare (NBHW), who are responsible for the NPR (Figure 1).

Figure 1 Flowchart of women diagnosed with recurrent pregnancy loss in the national patient register and validated with medical records.

Data sources

The NPR includes information about hospitalization, recorded at discharge. Each record states admission and discharge dates, hospital department, and the primary and up to seven contributory diagnoses coded according to the Swedish versions of the current edition (at the time of diagnosis) of the International Classification of Diseases (ICD).¹⁷

NBHW is the Swedish agency that administrates and is responsible for the NPR. Reporting information to the NPR is mandatory and is delivered to the NBHW once a month from each of the 21 county councils in Sweden. Since 1987, the NPR has included all inpatient events in Sweden. Since 2001, the register has also covered outpatient visits, including both private and public caregivers. The coverage of in-patients is currently almost 100%. For hospital-based outpatients, data from some private caregivers are missing, so the total cover for outpatients is considerably lower (approximately 80%).^16,18

The samples selected by NBWH were received encrypted with the 10-digit personal identification number assigned to all Swedish residents, as well as a hospital and clinic code. The selected hospitals provided a variation in demographic characteristics from larger university hospitals to smaller regional hospitals with geographic representation from all regions. All relevant available clinics were contacted to obtain the medical records. Consent was secured from the head of each clinic. The medical secretary or archive staff at the hospitals provided the medical records by registered mail. The first author reviewed the medical records and the RPL diagnoses (N96.9 and O26.2) were compared and validated against data in the NPR.

In our previous incidence study, an estimation was made of how many women, registered with three or more consecutive miscarriages, ie, fulfilling the diagnostic criteria of RPL, lacked the registered diagnosis codes of RPL (N96.9, O26.2) in the NPR.¹¹ This group comprised women, without an RPL diagnosis in the register, with three or more of the following ICD codes for miscarriages (main or bi-diagnosis) without interspersed live births: O02.1, O03 (including sub-diagnosis O03.0–O03.9) during 2003 and 2012.¹¹ Since the same diagnosis can be used several times on different occasions when contacting the healthcare system about the same pregnancy, a 90-day interval was required between two registered diagnoses of miscarriage to count as a new miscarriage. This group of women, with three or more consecutive miscarriages fulfilling the diagnostic criteria for RPL, but without registration in the NPR as RPL, are considered “false negative” and are used for calculating the sensitivity of RPL in the NPR.

Variables used

The method of diagnosing miscarriage: ultrasound examination or diagnosis based on a positive hCG-test, as well as method of conception: spontaneous or in vitro-fertilization pregnancy, were evaluated. Furthermore, the distribution regarding primary or secondary RPL and known risk factors for RPL were also evaluated.

Table 1 Validation of RPL diagnosis in medical records, randomly selected from women diagnosed with RPL in the NPR

Table 2 Women with correct diagnosis of recurrent pregnancy loss and pregnancy variables

Statistical analysis

The accuracy of positive register diagnoses of RPL was determined by PPV, ie, the probability of positive registered cases being confirmed by the medical records (reference standard). The PPV was calculated as: [true positive register cases] divided by [true positive in reference standard] plus [false-positive register cases] with 2×2 cross tables and a χ-square test with 95% confidence intervals (95% CI).

The ability of the registers to detect women with three or more consecutive miscarriages (“true positive”) was determined by sensitivity, ie, the probability of positive cases ever appearing in the register with a diagnosis of RPL. Sensitivity was calculated as: [true positive register cases] divided by [true positive register cases] plus [false negative register cases] with 2×2 cross tables and a χ-square test with a 95% CI.

A power analysis, assuming that PPV would be 85% and the accepted width of a 95% CI was 10 percentage units (eg, 80–90%), calculated that a sample size of 228 medical records were needed to achieve a power of 90%.

All data were analyzed using SPSS version 24 (Armonk, NY: IBM Corp.)

Ethics

The Regional Ethical Board at Lund University (Lund, Sweden) approved the national study (Dnr 2016/478).

Results

In the randomly selected cohort of 711 women, registered with RPL in the NPR, 485 medical records were requested from 60 different hospitals and private caregivers. Out of the 60 hospitals, 73% of the university hospitals, 60% of the regional hospitals and 50% of the private caregivers responded. In total, 254 medical records were available for review from 38 hospitals. In 16 cases, the medical records were not conclusive regarding the correctness of the diagnoses as information was missing (Figure 1). Correct diagnosis was made in 202 out of the 238 medical records, resulting in PPV (202/238)=85% (95% CI 78–89%) (Table 1). Out of the remaining 36 women with incorrect diagnosis, 27 cases (16+4+1 (Table 1)) could have been included as correct if the definition of two miscarriages had been used to qualify as RPL.

In our previous study about RPL incidence,¹¹ a cohort consisting of 990 women had had three consecutive miscarriages during 2003–2012 but with missing RPL diagnosis in the NPR. These women were defined as “false negative“ as they fulfilled the criteria for the diagnoses of RPL but were not registered as such in the NPR. In the same study,¹¹ a cohort of 6,852 women was diagnosed with RPL in the NPR during the 10-year period. With a PPV of 85%, estimated in this current study, the number of “true positives” in the cohort registered with RPL was calculated to be 5,824 women (0.85*6,852=5,824). For calculating sensitivity, “true positive” was divided by “true positive” plus “false negative” which gave 5,824/(5,824+990)=0.85, ie, a sensitivity of 85%.

Pregnancy variables were evaluated among the women who were correctly diagnosed (n=202) (Table 2). One hundred and twenty (59%) women had their miscarriage verified with ultrasound, whereas for 70 (35%) it was verified only by urine/serum hCG. There were 100 women (50%) without known risk factors for RPL. One hundred and eleven (55%) and 88 (44%) women had primary and secondary RPL, respectively.

Discussion

This validation study for RPL diagnosis registered in the Swedish NPR showed 85% PPV and a sensitivity of 85%. The findings are important for future research using data from the NPR in general and on RPL specifically. The data confirm our previous incidence study.¹¹

To our knowledge, no other studies have evaluated the data quality of registered RPL in national health registers. To avoid selection bias and confounding factors, our validation study was conducted with data from a large number of hospitals from different regions in Sweden. The responding rate for university hospitals (73%) and regional hospitals (60%) was pretty similar. There was a good geographic and demographic distribution of the participating hospitals so the material of the responded hospital can be considered representative and one of the strengths of the study.

Some medical records (n=16) were not conclusive regarding whether the RPL criteria were met or not. Furthermore, there were a substantial number of medical records (n=231) that could not be obtained. As high as 37% of the hospitals did not respond to the request to participate, a factor that might skew the results if there is a difference in the accuracy of diagnosing RPL in those who responded and those who did not. There are no obvious defined characteristics in the hospitals that did not respond compared to those that did respond that could suspect an influence in the results.

Despite these minor weaknesses, the power estimation of 228 medical records was reached in the study. Only one physician reviewed the medical records and consequently, no cross-reading of the medical records was possible. Such a cross-reading could have strengthened the study further and could also have enabled an inter-observational sub-analysis of the validated diagnoses.

The proportion of valid diagnoses, ie, PPV, in the NPR is probably higher in patients with severe as opposed to mild diseases.¹⁶ RPL is considered as a benign diagnosis, and taking this into consideration, the estimated PPV of RPL in the current study is fully acceptable and in line with the overall accuracy for several other diseases registered in the NPR.¹⁶

Sensitivity concerns how complete a register is and how good the register is at capturing cases with a disease.¹⁹ To assess the sensitivity a random population should be studied for the disease and then there should be a crosscheck to see whether individuals with the disease are recorded in the register. Many women suffer miscarriages without contacting a hospital or another caregiver and therefore do not exist in medical records.²⁰ It was not possible to study a random population for “false negative” cases; instead, we used data from our previous study describing the incidence of RPL.¹¹ This might be considered less accurate than the first mentioned sensitivity assessment, as it was dependent on coding of miscarriages in the NPR.

Whether a woman with RPL is reported in the NPR depends on several factors, including the care-seeking behavior of an individual, access to health care, and the physician’s awareness of the disease. Hospital fees can be an issue in some countries but since they are relatively low in Sweden this factor is of less importance. The NPR coding is also used as the basis for management and financing, and some ICD codes give higher financial compensation than others. The effects of financial incitements on ICD coding have probably been underestimated.¹⁶ To our knowledge, there have not been any significant changes in financial compensation for miscarriage or RPL codes over the last few decades.

Coding in registers is also associated with errors such as not having a better code to use and typing mistakes when documenting the code.²¹ Ludvigsson et al presented three types of diagnostic coding error in medical records; I) diagnostic errors, ie, the patient received an incorrect diagnosis, II) translation errors, ie, the ICD code in the NPR is different from the code actually listed in the patient chart, and III) coding errors, ie, a faulty ICD code accompanies an otherwise correct diagnosis.¹⁶

If two miscarriages had been used to define RPL as suggested by the ESHRE and the ASRM Practice Committee,^6,7 the PPV would have been higher. However, the definition of RPL used in Sweden during this period was three or more miscarriages. Our search in the NPR included diagnoses from both in- and outpatient records, although most women with a miscarriage or RPL are treated as outpatients. The NPR started registering outpatients in 2001 and according to the NBHW, the register is reliable for the period 2003–2005. It would, therefore, have been interesting to stratify the diagnoses by year and analyze whether any differences in the validation parameters for RPL changed over the years. However, this information was not possible to obtain and a stratification could therefore not be performed. The negative predictive value and specificity could not be estimated in this study, as the numbers of “true negatives” were unknown.

Only 59% of the pregnancies were verified with ultrasound, which is interesting considering that some scientific societies require a clinical pregnancy (verified by ultrasound or histopathology) as part of the definition.⁷ Thirty-five percent of the women with RPL had at least one of the pregnancies confirmed by hCG only and many of these would not have been given the RPL diagnosis in the opinion of the ASRM Practice Committee.⁷ This mode of confirmation is, however, acceptable according to ESHRE’s RPL 2018 guideline for pregnancies past six gestational weeks with heavy bleeding.⁶ According to this definition, the verified pregnancies used in this study are therefore correctly defined.

In half of the women who were verified with a RPL diagnosis in the medical record, no possible risk factor for RPL was found. This is in line with other studies reporting on the RPL etiology.^15,22 Of the women, 55% and 44% had experienced primary and secondary RPL, respectively, which is also in line with previously reported data.¹¹

Conclusion

The data indicate that the obtained PPV and sensitivity for the RPL diagnosis in the Swedish NPR have high quality and are therefore useful to use for future RPL research.