Positive Predictive Values of ICD-10-CM Diagnostic Codes in Identifying Diabetic Macular Edema in Claims Data: A Multi-Institutional Study in Taiwan

Introduction

Diabetic macular edema (DME) is a major complication in patients with diabetes mellitus, leading to blindness in those affected. It has been estimated that 28.61 million people will be affected by 2045.¹ Intravitreal anti-vascular endothelial growth factor (anti-VEGF) drugs are recommended as the first-line treatment for DME due to their pronounced effect in mitigating the swelling of the macula and improving visual acuity.^2,3 Previous studies based on secondary data sources, including electronic medical records and claims data from health insurance, have reported the clinical characteristics, risk factors, and treatment results of DME.^4,5 For example, Lundeen et al analyzed health insurance data in the United States and observed a 1.5-fold increase in the annual prevalence of DME from 2009 to 2018.⁴ Analyzing Taiwanese claims data, Lee WA et al found that, compared to ranibizumab, aflibercept was associated with a greater risk of adverse renal events in patients with DME.⁵ Since these studies relied on diagnostic codes to identify patients with DME, the validity of the findings may be subject to misclassification bias if the accuracy of these codes in capturing true DME cases is uncertain.⁶

To date, only one validation study in the United States has investigated the accuracy of International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) DME diagnosis codes, and it reported a sensitivity of 0.88 and a specificity of 0.96 in identifying DME for a series of ICD-9-CM codes 250.xx and 362.53.⁷ Although the accuracy of ICD-9-CM codes in identifying DME has previously been established, validation studies of ICD-10-CM codes for this purpose have yet to be conducted. The United States and many other countries transitioned from ICD-9-CM to ICD-10-CM on October 1^st, 2015,⁸ introducing substantial changes in code structure, granularity, and classification logic. Unlike ICD-9-CM systems, ICD-10-CM systems include laterality and more detailed clinical descriptors,⁹ which may affect how coders assign diagnostic codes in real-world settings.

Given the structural and coding differences between ICD-9-CM and ICD-10-CM systems, the diagnostic accuracy of ICD-10-CM codes cannot be assumed from prior ICD-9-CM validations. As most secondary data sources now use ICD-10-CM systems, validation of their accuracy in identifying true DME cases is essential to ensure study validity. Since no study to date has evaluated the validity of ICD-10-CM codes for DME, this study aimed to assess the accuracy of ICD-10-CM diagnosis codes in identifying true DME cases, using claims data from Taiwan.

Methods

Design and Setting

This retrospective study analyzed electronic medical records data from four regional hospitals (Taipei, Keelung, Linkou and Taoyuan) within the Chang Gung Medical Foundation (CGMF) in Taiwan.¹⁰ CGMF represents the largest multi-institutional healthcare system in Taiwan, and its electronic medical records data has provided abundant real-world evidence for research in various medical fields,^11–13 and especially in ophthalmology.^14–16 This study was approved by the Institutional Review Board of CGMF (IRB No: 202200229B0) and was conducted in accordance with the principles outlined in the Declaration of Helsinki. The requirement for informed consent was waived due to its retrospective design.

Data Sources

In Taiwan, the ICD-10-CM coding system was introduced in 2016.¹⁷ This study retrieved claims data reported by the four study hospitals to Taiwan’s National Health Insurance Administration, and included patients newly assigned the ICD-10-CM DME diagnostic codes E08-13.311, E08-13.321, E08-13.331, E08-13.341 or E08-13.351 in outpatient settings between 2017 and 2022.⁴ The details of ICD-10-CM DME diagnostic codes used in this study were presented in Supplementary Table 1. The study index date was defined as the first assignment date of an ICD-10-CM diagnostic code for DME. We randomly selected 15% of patients from the original cohort using simple random sampling for clinical ascertainment of DME based on electronic medical records. Similar validation steps have been followed by several other published studies, using electronic medical records data from CGMF, to validate diagnostic codes for cerebral venous sinus thrombosis, myocarditis, Guillain-Barré syndrome, anaphylaxis, carbon monoxide poisoning and glaucoma.^18–23

Ascertainment of DME

Two ophthalmologists (YCH and LY) independently reviewed the electronic medical records of the cases randomly selected from the original cohort to identify true DME cases based on examination reports (fundus photos, fluorescein angiography (FA), and optical coherence tomography (OCT)), laboratory data (hemoglobin A1c levels), medical history (diabetes), and medication history (use of anti-hyperglycemic drugs). Any discrepancies in judgment between the reviewers were resolved through discussion. According to the International Council of Ophthalmology Guidelines for Diabetic Eye Care,²⁴ DME was traditionally defined as retinal thickening proximal to the fovea. Due to advances in OCT, however, DME is now more commonly classified as center-involved or non-center-involved DME, respectively defined as retinal thickening in the macula with or without involvement of the central subfield zone (1 mm in diameter), Center-involved DME is defined if the retinal thickness ≥ 300 µm assessed by OCT. Overall, the kappa statistics were 0.80, indicating substantial agreement and consistent adjudication of DME diagnoses between two ophthalmologists.

Data Analyses

We calculated the positive predictive value (PPV) of the ICD-10-DME codes as the proportion of diagnostically ascertained DME cases among patients identified by ICD-10-CM DME diagnosis codes. We also determined the PPVs to identify true DME cases for (1) cases with ICD-10-CM coding for DME in the primary diagnosis position only, and (2) combinations of ICD-10-CM coding for DME with first-line treatment (ie., intravitreal anti-VEGF drugs covered by Taiwan’s National Health Insurance (NHI) within 3 months of diagnosis). We used the Clopper–Pearson exact method to calculate 95% confidence intervals (CI) for the PPVs. Data analyses were performed using SPSS (version 25.0; IBM Corp., Armonk, NY, USA).

Results

We identified a total of 4,962 patients newly assigned ICD-10-CM diagnosis codes for DME from 4 study hospitals between 2017 and 2022, of which 745 patients (15% of the total) were randomly sampled for detailed review of their electronic medical records (Figure 1). Within this sample cohort, 197 cases (26.4%) were confirmed as true-positive DME cases, while 548 cases (73.6%) were classified as false-positives. Among the false-positive DME cases, 94.2% (516/548) were considered tentative diagnoses, 2.2% (12/548) were actually other retinal diseases, and 3.6% (20/548) were remote DME.

Figure 1 Validation study flow chart. Abbreviations: ICD-10-CM, International Classification of Diseases, 10th Revision; DME, diabetic macular edema.

PPVs of ICD-10-CM DME Diagnostic Codes

The demographics and general clinical characteristics of the 197 true DME cases are summarized in Table 1. The mean (SD) age was 61.2 (11.5) years, and 82 (41.6%) of the true DME patients were female. Among the 197 true DME cases, 196 (99.5%) were diagnosed with type 2 diabetes, with one case of type 1 diabetes. The detailed PPVs for each ICD-10-CM DME code are presented in Table 2. Using the ICD-10-CM codes for DME in any of the three positions of the diagnosis yielded a PPV of 26.4% (95% CI: 0.23–0.30), Among the 402 cases where all ICD-10-CM DME diagnostic codes were present only in the primary diagnosis position, the PPV of ICD-10-CM codes for true DME cases remained similar (31.3% or 126/402, 95% CI: 0.27–0.36). Of the various coding patterns, the ICD-10-CM DME code E08–E13.311 (Unspecified diabetic retinopathy with macular edema) in any of the three positions of the diagnosis accounted for 108 true-positive cases, yielding a PPV of 21.6% (95% CI: 0.18–0.25). When the analysis was restricted to the primary diagnosis position only, the PPV improved slightly to 24.0% (95% CI: 0.19–0.30), but at the expense of a lower number of true DME patients identified.

Table 1 Clinical Characteristics of 197 Ascertained DME Cases

Table 2 Accuracy of ICD-10-CM Codes to Identify DME

PPVs of ICD-10-CM DME Diagnostic Codes Combined with Intravitreal Anti-VEGF Drug Uses

However, when ICD-10-CM DME coding was combined with NHI-covered intravitreal anti-VEGF drugs administered within 3 months, all 55 identified patients were confirmed as true-positive cases of DME, all with center-involved DME, resulting in a high PPV (100%, 95% CI: 0.94–1.00) (Table 3).

Table 3 Accuracy of ICD-10-CM Codes Plus Intravitreal Anti-VEGF Drug Use to Identify DME

Discussion

In this validation study using electronic medical records from four hospitals in Taiwan, the presence of ICD-10-CM codes for DME in any position of the outpatient diagnosis carried a PPV of 26.4% (95% CI: 0.23-0.30). The PPV did not significantly improve when the code position was restricted to the primary diagnosis column. However, combining ICD-10-CM diagnosis codes with the use of intravitreal anti-VEGF drugs for DME achieved a perfect PPV, with all identified cases classified as center-involved DME.

Comparison with the Literature

In the present study, the PPV of ICD-10-CM codes for identifying DME was substantially lower than that reported by Bearelly et al for ICD-9-CM codes in U.S. claims data (26.4% vs. 92.1%).⁷ Several factors may explain this discrepancy, including differences in coding practices and sample selection. First, our review of electronic medical records revealed that a large proportion of the false positives consisted of tentative diagnoses (94.2%, or 516/548). This may reflect a common practice in Taiwan, where ophthalmologists provisionally assign DME codes to patients with diabetic retinopathy who present symptoms such as blurred vision, metamorphopsia, or central scotoma. To improve coding accuracy, ophthalmologists could consider reserving ICD-10 CM DME diagnostic codes for cases with objective evidence of macular edema, preferably OCT-confirmed (and/or FA-supported when indicated), and use alternative codes (eg., diabetic retinopathy without macular edema or symptom-only codes) when DME remains only a tentative diagnosis pending imaging confirmation. Second, Bearelly et al used a convenience sample drawn from ophthalmologists who voluntarily submitted records, with only 22 of 43 agreeing to participate, potentially introducing selection bias due to better documentation practices. In contrast, our study was based on a random sample of patients with ICD-10-CM DME codes across multiple hospital levels, reducing the potential for selection bias related to physician participation. These findings underscore the critical need to validate the accuracy of ICD-10-CM codes before using secondary databases for DME research.

Implications for Future Researchers

Our results indicated that only about one out of every four patients assigned an ICD-10-CM diagnosis code for DME was actually an ascertainable DME case. Since the accuracy of ICD-10-CM codes to define DME is therefore suboptimal, future researchers relying on these codes face significant risks of misclassification. For example, when defining study populations, the use of inaccurate ICD-10-CM codes may lead to the inclusion of patients who do not truly have the disease (false positives) or the exclusion of true cases (false negatives), leading to selection bias and reduced generalizability. When defining covariates, misclassified comorbidities can result in inadequate confounding adjustment. When defining outcomes, low ICD-10-CM coding accuracy may lead to misclassification of clinical events, biasing effect estimates and threatening internal validity. Hence, it is imperative that researchers use validated ICD-10-CM coding algorithms with high PPVs, whereby we suggest combining diagnostic codes with the use of related medications, such as combining ICD-10-CM DME diagnosis codes with intravitreal anti-VEGF drugs to improve the accuracy of DME identification using Taiwan’s NHI data or other secondary healthcare data sources.

Strengths and Limitations

A major strength of this study is its use of a random sample from four hospitals of varying levels in Taiwan, enhancing the representativeness of the patient population. Among the 745 patients examined, 197 true DME cases were identified, all of whom underwent ophthalmological examinations (fundus photography or OCT). Moreover, all 55 patients who received NHI-covered intravitreal anti-VEGF drugs had an ascertained diagnosis of center-involved DME, consistent with current clinical guidelines, thereby supporting the internal validity of our study.

Some limitations of our study must also be acknowledged. First, because the target population was first identified from the electronic medical records database in our validation study step, we were unable to evaluate the sensitivity, specificity, or negative predictive value of the ICD-10-CM codes for DME. Second, the generalizability of our findings to other countries should be approached with caution, as Taiwan operates under a single-payer NHI system. For instance, intravitreal anti-VEGF drugs for DME are reimbursed under specific criteria in Taiwan, including that best-corrected visual acuity must be between 0.05 and 0.5, with central retinal thickness ≥ 300 μm. Moreover, HbA1c level within the three months prior to initiation of intravitreal anti-VEGF drugs must be under 10%. This HbA1c requirement may lead to the exclusion of patients with poorly controlled diabetes and, consequently, may disproportionately exclude those with more severe diabetic retinopathy. However, as these criteria largely reflect the inclusion criteria of Phase 3 clinical trials for these drugs,^25–27 variability in the prescribing patterns of intravitreal anti-VEGF drugs for DME among ophthalmologists worldwide is likely to be minimal.

Conclusion

The accuracy of ICD-10-CM codes alone for identifying DME was found to be suboptimal in Taiwan. However, when combined with records of intravitreal anti-VEGF drug use, the ICD-10-CM codes reliably identified DME cases, particularly those with center-involved DME, in routine clinical practice in Taiwan. These findings provide a valuable reference for future claims-based research requiring the identification of DME cases in Taiwan.