Development and Validation of an Algorithm for Thrombosis with Thrombocytopenia Syndrome (TTS) at Unusual Sites

Introduction

Passive safety surveillance, case series, and observational studies reported the occurrence of thrombosis with thrombocytopenia syndrome (TTS), including thromboses that occurred at unusual sites (ie intracranial or cerebral,¹ and intraabdominal² thromboses), after receipt of adenoviral vector-based COVID-19 vaccines, noting that no such association with COVID-19 mRNA vaccines has been established. While a standard case definition for TTS does not yet exist, an interim definition from the Brighton Collaboration defines TTS as co-occurring thrombosis and new onset thrombocytopenia.³

The performance of clinical coding to identify unusual site TTS is unknown, with no published validated algorithms. In this study, the two main objectives were to: 1) develop an International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM)-based algorithm to identify unusual site TTS, and 2) validate the algorithm using laboratory results, pathology reports, and imaging reports in an electronic health record (EHR) database from an academic health network that serves an average of 1.5 million patients annually in the Mid-Atlantic region of the United States. This unusual site TTS algorithm validation study will help estimate its background incidence rate and facilitates active surveillance of this rare outcome in administrative claims and EHR databases.

Methods

Data Source and Study Population

An anonymous academic health network EHR database, serving an average of 1.5 million patients annually in the Mid-Atlantic region of the United States, was used for this study. The study population included care-seeking patients with at least one thrombosis diagnosis with co-occurring thrombocytopenia diagnosis within 14 days of the thrombosis diagnosis between January 1, 2017 and December 10, 2020, when the United States Food and Drug Administration (FDA) issued the first Emergency Use Approval (EUA) for a COVID-19 vaccine. The start date was chosen as the earliest date data was available from the academic health network EHR database, while the date of the first EUA for a COVID-19 vaccine was used as the study end date to avoid any potential source of bias introduced by vaccine-associated TTS.

The study was conducted according to the guidelines of the Declaration of Helsinki; ethical review and approval were waived for this study due to the nature of this study as a secondary data analysis involving no personal health information.

Algorithm Development

Unusual site TTS was defined as co-occurring diagnoses of intracranial or intraabdominal venous thrombosis and thrombocytopenia up to 14 days apart at maximum. All ICD-10-CM diagnosis codes included in the algorithm were based on literature review (summarized in Appendix A, Supplemental Tables A1, A2 and A3) and were reviewed by clinicians (J Beers, A Srikrishnan, A Hettinger, A Shenoy, T Burrell) on the study team. The algorithm was operationalized according to the two algorithm components outlined below, with further details on ICD-10-CM codes included in Appendix B.

• Algorithm component 1 (unusual site thrombosis): ≥1 ICD-10-CM diagnosis code indicating unusual site thrombosis (intracranial venous thrombosis OR intraabdominal venous thrombosis). ICD-10-CM diagnosis codes are found in Supplemental Table B1.
• Algorithm component 2 (thrombocytopenia): ≥1 ICD-10-CM diagnosis code indicating thrombocytopenia (ICD-10-CM diagnosis codes are found in Supplemental Table B2) within 14 days of an unusual site thrombosis diagnosis code (algorithm component 1).

Patients with unusual site TTS were identified based on their first recorded ICD-10-CM diagnosis in the Academic Health System EHR database indicating an unusual site thrombosis (algorithm component 1). Care settings for algorithm component 1 were limited to the inpatient (IP) and hospital emergency department (ED), where confirmatory imaging procedures for a venous thrombosis diagnosis are most likely performed. Patients were also required to have an ICD-10-CM diagnosis code indicating thrombocytopenia (algorithm component 2) occurring up to 14 days from an unusual site thrombosis diagnosis. Care settings for algorithm component 2 were included from the IP, ED, or outpatient and ambulatory care (OP) settings since thrombocytopenia can be confirmed via platelet count laboratory tests in any of these settings. All ICD-10-CM diagnosis codes were required to be labeled as final or discharge diagnoses on the patient’s EHR diagnosis record for inclusion in the algorithm.

Algorithm Validation

For each case selected for validation, all available records were extracted, including unstructured narrative notes and reports. The extracted data was converted to Health Level 7 (HL7) Fast Healthcare for Interoperability Resources (FHIR) Draft Standard for Trial Use (DSTU2) format and uploaded to a semi-automated chart review tool for clinician adjudication. All information in the chart review tool was searchable with the ability to adjudicate cases using an embedded case definition and guided questionnaire based on the Brighton Collaboration’s interim case definition for TTS. Each case was adjudicated by two clinicians per the Brighton Collaboration’s interim case definition for TTS as the reference method.³ A third clinician was available to adjudicate discordant case status. Case adjudication based on an interim Brighton Collaboration case definition included:

• Confirmation of intracranial or intraabdominal thrombosis diagnoses via confirmatory imaging reports or pathology reports within 14 days of the unusual site thrombosis diagnosis. AND
• Confirmation of thrombocytopenia diagnoses via at least one low (<150,000/μL) platelet count lab within 14 days of the venous thrombosis diagnosis.

Positive predictive value (PPV) and corresponding 95% Agresti Coull confidence interval (CI) was calculated for the unusual site TTS algorithm (co-occurrence of algorithm components 1 and 2). Subgroup analyses were conducted, stratified by specific unusual thrombosis site (in these analyses, it was necessary to select a random sample of portal vein thrombosis cases for the validation cohort, due to the larger number of cases associated with this location). Secondary analyses were also performed to calculate PPV of the unusual site thrombosis (algorithm component 1) and the co-occurring thrombocytopenia diagnosis event (algorithm component 2) as independent events. PPV was defined as the proportion of algorithm-identified cases in the validation sample (with sufficient data for adjudication) that was confirmed by adjudicators using clinical evidence in the EHR. Thrombosis diagnoses were confirmed by imaging report or pathology report. Qualifying imaging reports within 14 days of the incident thrombosis diagnosis included narrative reports accompanying ultrasound–Doppler, computed tomography (CT scan), magnetic resonance imaging (MRI) including magnetic resonance venography (MRV), or perfusion V/Q scan.

Results

The algorithm identified 278 unusual site TTS cases based on the co-occurrence of algorithm components 1 and 2 (each case representing a unique patient) between January 1, 2017, and December 10, 2020.¹ Of the 278 unusual site TTS cases identified (algorithm-identified cohort); 267 had an intraabdominal thrombosis site and 11 had an intracranial thrombosis site. Of the 267 abdominal site TTS cases, thromboses were identified in 211 (79.0%), 39 (14.6%), 10 (3.7%) and 7 (2.6%) using ICD-10-CM codes I81 (portal vein thrombosis), I82.890 (acute embolism and thrombosis of other specified veins), I82.0 (Budd-Chiari Syndrome), and I82.3 (embolism and thrombosis of renal vein) codes, respectively. Of the 11 unusual site TTS cases with an intracranial thrombosis site, thromboses were identified by ICD-10-CM code G08 (intracranial and intraspinal phlebitis and thrombophlebitis) code in 9 (81.8%) cases, and by I67.6 (nonpyogenic thrombosis of intracranial venous system) code in 2 (18.2%) cases.

From the algorithm-identified cohort, 50 of the 211 cases associated with portal vein thrombosis (ICD-10-CM code I81), were randomly selected for validation. The remaining 67 unusual site TTS cases were all selected for validation, yielding a total of 117 unusual site TTS cases selected for validation (validation sample cohort).

In both the algorithm-identified and validation cohorts, over 60% of patients were 56 years or older, and over 52% were male (Table 1).

Table 1 Patient Demographics and Thrombosis Site for Algorithm-Identified and Validation Cohorts

There were 113 cases (of 117 cases selected for validation) with sufficient information to validate diagnoses. Two patients who did not have imaging reports available and two patients who did not have platelet counts were not included in the denominator for the PPV due to lack of sufficient data for adjudication. Authors assume that patients without reports do not differ significantly from those with reports, and that the small number of exclusions will not meaningfully impact estimates.

Among these 113 cases, the unusual site TTS algorithm (algorithm components 1 and 2) had a PPV of 76.1% (95% CI 67.2–83.2%) (Table 2). PPV was 80% or higher for each of the thrombosis diagnosis codes within the TTS algorithm, except for the I82.890 code (PPV: 39.5%, 95% CI 25.3–55.6%). Appendix C, Supplementary Table C1 reports the validation of the thrombosis and thrombocytopenia components of the unusual site TTS diagnosis as independent events.

Table 2 Algorithm Validation Results, Stratified by ICD-10-CM Thrombosis Diagnosis

Discussion

This study represents the first validation of an ICD-10-CM-based coding algorithm for unusual site TTS. The algorithm demonstrated an intermediate-to-high positive predictive value (PPV) when compared to full electronic health record (EHR) data as the reference method. This validated algorithm shows potential for use in observational studies, including active surveillance of COVID-19 vaccines and other medical products.

Previous validation studies evaluated intracranial thrombosis algorithms regardless of co-occurrence with thrombocytopenia. These studies reported performance metrics for a sub-component of unusual site thrombosis (independent of co-occurrence with thrombocytopenia), including a PPV of 75.7% (95% CI 66.9–82.8%)⁴ and 92.3% (95% CI 82.8–97.1%).⁵ These results are comparable with our PPV estimate of 100.0% (95% CI 69.9–100.0%) for intracranial thrombosis with co-occurring thrombocytopenia (intracranial TTS). As Handley reported, of 65 intracranial thrombosis cases identified in UK EHRs, 91% were coded with G08, compared to 1.5% coded with I63.6 and 1.5% coded with I67.6.⁵ In contrast, as Liao reported, of 166 cerebral venous sinus thrombosis (CVST) cases identified in Taiwan EHRs, I67.6 was more prevalent (54.2%) compared to G08 (41.0%).⁶ Liao also reported the PPV of the ICD-10-CM codes of G08 and I67.6 was 88.2% and 91.3%, respectively.⁶

Consistent with Handley and Emsley⁵ (but contrary to the findings reported by Liao⁶), the less specific ICD-10-CM code for intracranial thrombosis (G08), rather than the more specific code of I67.6, identified the majority of intracranial TTS in our study.⁵ All of the individual codes in our unusual site TTS algorithm had a PPV point estimate of 80.0% or greater except for the I82.890 code that was non-specific to intraabdominal venous thrombosis. If the 182.890 code were excluded from the algorithm, the PPV point estimate for the unusual site TTS algorithm would increase from 76.1% to 94.6%.

Our study has a few limitations. First, adjudicating clinicians did not have access to source imaging files used by the diagnosing clinician(s) and instead relied on the diagnosing clinician’s determination in imaging and pathology reports. Second, the PPV point estimates stratified by thrombosis diagnosis code were based on ten cases or fewer for four of the six stratifications, resulting in wide confidence intervals. Third, the algorithm performance was assessed in a single US healthcare network, which may not be generalizable to other healthcare networks. Lastly, limited observability of healthcare encounters outside of the Academic Health Network may also contribute to potential selection and misclassification biases. The algorithm-identified cohort was limited to patients seeking care within the Academic Health System EHR network who had both the thrombosis and thrombocytopenia components of the TTS diagnosis recorded in the database within 14 days of one another. If either the thrombosis or thrombocytopenia diagnoses were undiagnosed, or otherwise not observed in the EHR network, the population may not be representative of the population with TTS in the Academic Health Network patient population, and result in selection bias. Even among the selected cohort, TTS confirmation status may be misclassified if a patient had unobserved confirmatory laboratory and pathology results recorded outside of the network.

A major strength of this study is validating a composite unusual site TTS algorithm, encompassing both unusual site thrombosis and thrombocytopenia, which have not been validated yet in the literature. A second strength of this study is that the algorithm was validated using full EHR records and semi-automated chart review, including unstructured narrative reports and notes, neither of which have been applied to validation of algorithms to identify TTS in other studies. Additionally, this ICD-10-CM-based algorithm can be operationalized in many observational studies that require identification of patients with unusual site TTS cases.

In conclusion, this study developed and validated an algorithm to identify unusual site TTS in a healthcare database, and the algorithm performed at an intermediate-to-high PPV. To date, this algorithm has been used to estimate the background rates as historical comparators and to identify unusual site TTS cases in near real-time surveillance of COVID-19 vaccines in administrative claims databases in the FDA active surveillance program called the Biologics Effectiveness and Safety (BEST) Initiative.⁷.