Performance of Novel Inflammatory Bowel Disease-Specific Models for Risk Adjustment

Introduction

Crohn’s disease (CD) and ulcerative colitis (UC) are debilitating chronic and progressive inflammatory bowel diseases (IBDs) that are estimated to affect more than 2 million people in the USA.^1,2 As a result, the economic burden of IBD is high. Although CD and UC typically follow a relapsing and remitting disease course,³ a subset of patients present with continuous active inflammation and a high number of complications, requiring numerous costly diagnostic tests, medications, and surgeries.⁴ In two US studies published in 2020, mean total direct costs were estimated to be approximately $25,000 per patient per year for those with CD,⁵ and more than $18,000 per patient per year for those with UC;⁶ however, these figures provide only a partial picture of the situation, because a disproportionately small number of patients account for a large proportion of the medical expenditure.^7,8

Diagnosis-based clinical grouper models, also known as risk-adjustment models, provide a simple way of predicting health care costs based on diagnosis codes that are routinely captured in administrative claims data.^9,10 Although risk-adjustment models have similarities to clinical prediction models, the developmental approaches and their applications may differ. For example, the Diagnostic Cost Group Hierarchical Condition Category (DCG-HCC) models were developed and adapted for the Centers for Medicare and Medicaid Services, and are commonly used in the insurance industry to risk-adjust capitated payments and to set premiums for health care plans that reflect expected costs.^9,10 In the DCG-HCC model, diagnostic codes are assigned to risk groups of increasing severity based on their associated medical costs. Risk scores are then calculated for individuals based on how their expected costs differ from others according to their observed diagnoses and other relevant predictive variables.^9,11

Most published actuarial risk-adjustment models are not disease specific because they aim to categorize the full population broadly within a managed care plan. Although clinical prediction models exist for IBD, they are predominantly designed to predict clinical outcomes.^12,13 There is increasing interest in using machine learning-based methods for these models, most commonly tree-based algorithms, rather than conventional risk prediction models that rely on statistical methods.^12,13 While there is evidence to suggest these complex models can outperform conventional risk prediction models, they require large datasets and have a high risk of bias.¹² In contrast, few models are designed to predict health care costs.¹⁴ To the best of our knowledge, there are no clinical-risk grouper models for IBD that utilize actuarial approaches that are familiar to payers. A simple risk-adjustment model that can predict IBD-related medical costs using readily available claims data has broad application. In clinical practice, it could offer a quick method for identifying patients who are likely to incur high costs and who may benefit from disease management programs.¹⁵ In clinical research, it could enable the risk adjustment of cost-based outcomes in IBD studies, and, in the managed care setting, it could enable capitated arrangements with provider groups. Herein, we describe the development and performance of novel IBD-specific risk-adjustment models designed to risk-normalize concurrent and predict prospective per patient per month (PPPM) medical costs in patients with IBD using diagnosis codes and other predictors.

Methods

Study Design and Objectives

This retrospective, cross-sectional study was conducted using administrative claims data from commercially insured and Medicare Advantage-insured patients with CD and/or UC between January 1, 2020, and December 31, 2022. An actuarial approach was employed with three objectives. First, create an IBD-specific diagnostic classification system (clinical-risk grouper) based on IBD-related medical expenditures and assign patients to severity groups based on their IBD diagnosis codes. Second, develop models using these diagnosis-based severity groups (and additional predictors) to risk-normalize concurrent IBD-related PPPM medical costs and predict prospective IBD-related PPPM medical costs. Third, evaluate and describe the performance of the models.

Data Source and Study Population

Claims data were obtained from Optum’s proprietary de-identified Normative Health Information database: a large, longitudinal claims database comprising data from 42 million individuals per year. The database is representative of commercially insured and Medicare Advantage-insured patients in the US covering diverse geographic regions and healthcare settings. No adjustments were made in this study for missing data. The study population included patients with at least one claim containing a primary International Classification of Diseases, 10th Revision, Clinical Modification (ICD-10-CM) diagnosis code for CD or UC during the 3-year study period. For the concurrent risk-adjustment models, patients were required to be eligible for and to have medical and pharmacy coverage for at least 1 month during the study year. For the prospective risk-adjustment models, patients were required to be eligible for and to have medical and pharmacy coverage for at least 1 month and to have at least one CD or UC claim in 2 consecutive study years.

Data Analysis

Descriptive statistics were used to summarize the demographics and the clinical characteristics of the study population for each study year.

Development and Application of the Clinical-Risk Grouper

The clinical-risk grouper was developed by applying a similar approach and the principles that were used to develop the DCG-HCC models.^9,16 The principles are intended to guide model development rather than impose strict rules, and competing principles should be prioritized according to needs. The degree to which each principle was followed was determined by author consensus through multiple model iterations. The referenced principles and how they were applied are outlined below and described fully in the Supplementary Table 1.

Model principles state that because each diagnostic code contains potentially relevant clinical information, models should incorporate all International Classification of Diseases (ICD) codes (principle 9; Supplementary Table 1).^9,16 There are 77 IBD ICD-10-CM diagnosis codes (28 CD codes and 49 UC codes); all were included in our clinical-risk grouper. The CD and UC diagnosis codes describe disease location (eg, small intestine) and complications (eg, presence of fistula or abscess). To assign the CD and UC codes to severity groups, the mean IBD-related PPPM medical costs were calculated for patients with a primary diagnosis for each code.

DCG-HCC model principles state that diagnoses in the same severity group are reasonably homogenous with respect to their effect on incurred medical costs (principle 2; Supplementary Table 1).^9,16 To achieve this, rules were developed to group ICD-10-CM codes together with similar mean IBD-related medical costs per month using cluster analysis. For CD diagnosis codes, mean PPPM costs ranged from 84% to 155% of the mean CD-associated PPPM costs. The distribution of costs was greater across UC diagnosis codes, ranging from 26% to 182% of the mean; therefore, different rules were applied to assign codes to CD or UC severity groups (Supplementary Table 2). Each code was assigned to one severity group only, and all severity groups were assessed for separation by cost, sample size, and clinical meaning before finalization, in line with DCG-HCC model principles.^9,16 In the final groupings (Table 1), the diagnosis codes in CD severity group 1 (lowest risk) predominantly comprised codes for CD without complications, with unspecified complications, or with rectal bleeding. CD severity group 3 (highest risk) included codes indicating extensive disease (in the small and large intestines) and CD with fistula or abscess. UC severity group 1 predominantly included codes related to inflammatory polyps, and severity group 3 included codes associated with more extensive disease (eg, pancolitis). The relative PPPM IBD costs for CD severity groups 1, 2, and 3 were 88%, 114%, and 141% of the overall mean CD costs, respectively; for UC severity groups 1, 2, and 3, these costs were 48%, 103%, and 150% of the overall mean UC costs, respectively (Table 1). The mean and relative costs for all IBD-related codes are shown in Supplementary Table 3.

Table 1 Diagnosis Codes and Costs by Severity Group in the Proposed Clinical-Risk Grouper System for CD and UC

To apply the clinical-risk grouper, patients were hierarchically assigned to the highest severity group based on the CD and/or UC diagnosis codes recorded for their claims data during the study year (see Supplementary Table 4 for examples). This reflects a key principle of HCC-DCG models: more severe manifestations of a condition override the effect of less serious ones (principle 4; Supplementary Table 1). The diagnostic classification should not reward coding proliferation;^9,16 therefore, the number of claims submitted for each code did not affect group designation (principle 6; Supplementary Table 1).

Risk-Adjustment Models

The relationship between clinical-risk groupers and concurrent or prospective IBD-related PPPM costs, with and without additional predictors, was evaluated using ordinary least-squares regression analyses. This is the preferred method for risk-adjustment models, and as such the approach is well established.¹⁷ Ordinary least-squares regression models perform well in this setting because medical expenditure data are often skewed, and more complex log-transformed models may produce biased cost estimates under heteroskedasticity.^9,18–20 This is the primary approach that the Centers for Medicare & Medicaid Services uses for their risk-adjusted capitated payment programs for Medicare Hierarchical Condition Categories and Medicare Part D Prescription Drug Hierarchical Condition Categories, as well as being commonly used in secondary capitated payment arrangements.^9,21–23

For the prospective risk-adjustment model, patient characteristics were measured in the first study year and IBD-related medical costs were captured in the second year (Supplementary Figure 1). Our primary aim was to build and validate the clinical-risk grouper, which was therefore included in the model as a prespecified variable. Regression models were fitted without intercept to allow the β-coefficients to represent US dollar values and to ensure overall predicted costs were positive. The presence of both CD and UC diagnoses for a given patient within a study year was additive because both diagnoses are independently associated with higher health care costs.⁸ Interactions between CD and UC were not tested. Patients with both a CD diagnosis and a UC diagnosis were classified as having indeterminate colitis.

To account for medical expenditure differences between patient subgroups, separate models were developed for commercially and Medicare Advantage-insured patients, and for patients who were users and nonusers of targeted IBD therapies (biologics, including biosimilars; and small synthetic molecules, including Janus kinase inhibitors and sphingosine-1-phosphate pathway modulators). The performance of each model was evaluated using R², which indicates the proportion of variance in the outcome (PPPM IBD-related medical costs) that is accounted for by the clinical-risk grouper and model predictors. Previously published concurrent risk-adjustment models typically report R² values between 25% and 50%;²⁴ in prospective models, R² typically ranges from 2% to 16%.²⁵ To create a parsimonious model and maximize R², forward selection was used to determine additional predictors for inclusion, retaining variables that improved R² only. The following predictors were tested: age, gender, insurance type, treatment type, comorbidities that may influence IBD outcomes, IBD-related hospitalizations and associated duration of stay, IBD-related emergency room (ER) visits, IBD flares, gastrointestinal surgeries/procedures, and the presence of extraintestinal manifestations.

Results

Study Population

Across all 3 study years, 158,649 patients with IBD were identified, including 64,714 patients with CD, 88,507 patients with UC, and 5428 patients with indeterminate colitis (Table 2). The mean age across all study years was 51.3 years in patients with CD, 57.3 years in patients with UC, and 49.9 years in patients with indeterminate colitis. Targeted IBD therapy use was highest in those with indeterminate colitis, and lowest in those with UC (CD, 41.3%; UC, 16.5%; indeterminate colitis, 53.2%). The proportion of patients with one or more IBD-related hospitalization was also highest in those with indeterminate colitis and lowest in those with UC (CD, 6.8%; UC, 3.7%; indeterminate colitis, 16.5%); this trend was reflected in the mean concurrent PPPM IBD-related costs (CD, $619; UC, $304; indeterminate colitis, $1217) and the proportion of patients who were commercially insured (CD, 62.1%; UC, 53.1%; indeterminate colitis, 65.5%).

Table 2 Patient Demographics, Clinical Characteristics, and IBD-Related Medical Costs in Patients with CD, UC, or Indeterminate Colitis During the 3-Year Study Period

Model Performance in Commercially Insured Patients

In the concurrent models, the clinical-risk grouper explained 20.99% of the variance in costs of patients who were targeted IBD therapy users and 2.07% of the variance for nonusers (Table 3). Number of IBD-related hospitalizations (range, 0–5) and duration of IBD-related hospitalizations (range, 0–119 days) increased explanatory power of the concurrent models to 41.56% in patients who were users of targeted IBD therapies and to 29.20% in nonusers, while still preserving the clinical-risk grouper severity rankings.

Table 3 Performance of Multivariable Concurrent and Prospective Linear Regression Models

In the prospective models, the clinical-risk grouper explained 12.04% of the variance in costs of patients who were users of targeted IBD therapies and 6.73% of the variance of nonusers. None of the tested predictors improved model explanatory power beyond the variance explained by the clinical-risk grouper.

Model Performance in Medicare Advantage-Insured Patients

In the concurrent models, the clinical-risk grouper explained 20.72% and 2.01% of the variance in costs of patients who were users and nonusers of targeted IBD therapies, respectively (Table 3). Model explanatory power was increased with the inclusion of IBD-related hospitalizations (range, 0–7) and duration of IBD-related hospitalizations (range, 0–84 days) as additional predictors, giving final R² values of 22.71% and 16.56% in users and nonusers of targeted IBD therapies, respectively. The addition of predictors preserved the clinical-risk grouper rankings in the patients who were users of targeted IBD therapies. In nonusers, expected costs in each severity group were similar and the risk distinction was lost.

In the prospective models, the clinical-risk grouper explained 31.68% of the variance in the outcome in patients who were users of targeted IBD therapies versus 4.65% in nonusers. None of the evaluated predictors improved model performance.

The Influence of Multiple Predictors on R² in Concurrent Models

As previously described, only IBD-related hospitalizations (number and duration of stays) improved the explanatory power of the concurrent models beyond that of the clinical-risk grouper. Predictors routinely captured in claims databases and often used in risk-adjustment models, such as age, gender, number of IBD flares, IBD-related ER visits, or comorbidities, resulted only in minimal incremental increases in R² (Figure 1).

Figure 1 The impact of additional predictors on explanatory power (R2) in concurrent models for (a) commercially insured targeted IBD therapy users and (b) nonusers and for (c) Medicare Advantage-insured targeted IBD therapy users and (d) nonusers. The R2 value reported for the clinical-risk grouper predictor is the value for UC severity group 3. The sequence of predictors is based on contribution to R2. Abbreviations: ER, emergency room; GI, gastrointestinal; IBD, inflammatory bowel disease; UC, ulcerative colitis.

Hypothetical Examples of Cost Prediction

In each model, the β-coefficients for each variable represent the associated PPPM IBD medical costs. These values can be converted into relative weights using the models’ mean overall PPPM cost. The relative weights can then be used to generate risk scores.

To illustrate how the risk scores can be used to risk-adjust costs, we consider two examples (Figure 2). Patient A is a commercially insured patient with a recent claim for CD of the large intestine with rectal bleeding. Patient A is prescribed a targeted IBD therapy and has never had IBD-related surgery. Patient A is assigned to CD severity group 2 based on their diagnosis, the weighting for which is 0.76. The absence of any other relevant risk factors means patient A’s risk-adjusted expected PPPM cost is $990.33, which is 0.76 times the mean overall PPPM costs of $1303.06.

Figure 2 Hypothetical examples of cost prediction using the concurrent model for commercially insured targeted IBD therapy users with (a) instructions on how to select the appropriate model, determine the severity group, and calculate predicted costs, and a cost prediction example in (b) a patient with CD and (c) a patient with UC. Abbreviations: CD, Crohn’s disease; IBD, inflammatory bowel disease; ICD-10-CM, International Classification of Diseases, 10th Revision, Clinical Modification; PPPM, per patient per month; UC, ulcerative colitis. Notes: Bold and color formatting are used for emphasis and legibility, respectively. Formatting differences do not indicate significance.

Patient B has UC and was recently hospitalized for 5 days for ulcerative rectosigmoiditis with rectal bleeding (UC severity group 3, weight 0.85). IBD-related hospitalization carries a weight of 0.60, and a weight of 0.32 is applied for each day spent in hospital. This gives a total risk score of 3.05, equating to expected risk-adjusted costs of $3974.33 (3.05 times the mean of $1303.06).

In the managed care setting, these model weights can be used to determine a patient’s expected costs using only the mean PPPM costs of the plan; no additional modeling is required.

Discussion

Herein, we describe the development of the first concurrent and prospective IBD-specific risk-adjustment models for commercially insured and Medicare Advantage-insured patients with CD and/or UC. Our models, which were developed and tested using the DCG-HCC methods and principles for risk-adjusted capitated payment systems,^9–11 explained up to 41.56% of the variance in PPPM IBD-related medical costs, which demonstrates that financial risk can be explained using a simple diagnostic risk grouper. In comparison, the typical explanatory power of non-specific concurrent models ranges from 1.02% to 54.74%.²⁶ Our models performed better in patients who were users of targeted IBD therapies than in nonusers; R² values ranged from 2.01% to 6.73% for models with clinical-risk groupers alone in nonusers of targeted IBD therapies, suggesting limited predictive value in this population. The reason for this is unclear; however, it is possible that medical costs are more predictable in this population because the disease is controlled by targeted therapies. There may also be differences in healthcare utilization, clinical coding, or care patterns between users and nonusers.

In risk-adjustment models, greater explanatory power is typically observed with concurrent models than with prospective models.^10,25 This is generally because major costly medical events are often unpredictable; indeed, prospective risk-adjustment models typically account for only 2–16% of the variance in prospective medical costs.²⁵ However, in the Medicare Advantage-insured population, the clinical-risk grouper alone explained 31.68% of prospective costs in patients using targeted IBD therapies, implying that future medical costs are inherently more predictable in this population. None of the tested predictors improved the explanatory power of prospective models to a greater extent than the clinical-risk grouper alone. Although the incorporation of other clinical variables (eg, from electronic health records) may have further improved predictive power, our goal was to maximize real-world utility by creating parsimonious models using readily available administrative claims data and the fewest possible predictors.

In concurrent models, only the incidence and duration of IBD-related hospitalizations improved model predictive power beyond that of the clinical-risk grouper. The influence of hospitalizations on concurrent costs is to be expected given that the occurrence is a known costly event in the study year. These data suggest that the location and extent of disease that predominantly separate CD- and UC-specific ICD codes is an important determinant of concurrent medical expenditure in people with IBD. Consistent with previous studies, predicted IBD-related medical costs were higher in patients with CD than in patients with UC in all models;⁸ greater targeted IBD therapy and corticosteroid use, and more commonly experiencing disease flares, IBD-related hospitalization, and undergoing GI surgery, are likely to be contributing factors in the cost difference between patients with CD and UC.

The predictive power of the final concurrent models (ie, clinical-risk grouper plus predictors) were lower for Medicare Advantage-insured patients than for commercially insured patients. We speculate that the lower performance reflects differences in the patient populations (eg, demographics, disease characteristics), patterns of care, benefit design, and potentially coding practices. There also could have been more variability in the Medical Advantage-insured group, which could make prediction more difficult.

To the best of our knowledge, we present the first concurrent and prospective risk-adjustment models specifically developed for patients with IBD. We hope that these models will provide potential opportunities to improve patient care and better allocate health care resources to those in need, for example, by identifying patients who are likely to incur high future medical costs and who may benefit from case management intervention. As the global costs of treating IBD continue to rise, there is increasing interest in determining drivers of cost and ways to deliver affordable care.²⁷ Our models may support these efforts by allowing risk adjustment of cost-based outcomes in real-world IBD studies, and by enabling capitated arrangements to ensure that providers are incentivized to prioritize preventive care.

Limitations

Several limitations should be considered when interpreting the results of this study. First, the study population is limited to commercially and Medicare Advantage-insured patients; the exclusion of other insurance types may affect the generalizability of these findings. As such, we do not advise using this model for other insurance types. We also recommend recalibration before implementation as predictive ability can be influenced by differences across systems. Second, the model is cross-sectional and does not incorporate longitudinal changes in disease severity or time-dependent variables, which could have improved the predictive accuracy of the model. Third, the concurrent models are sensitive to acute care costs and may underestimate the influence of other variables. Fourth, the use of de-identified data from a claims database means that not all relevant clinical information is routinely captured (eg, laboratory test results). Fifth, claims data are primarily collected for billing and administrative purposes, therefore, the accuracy and completeness of clinical information may vary, the allocation of primary and secondary ICD-diagnosis codes may be subject to bias from reimbursement policies, and medical conditions or procedures may be underreported or misclassified. Lastly, the robustness of the model has not been validated in an external data set.

Conclusions

Collectively, these results indicate that diagnosis-based clinical-risk groupers may provide a simple and effective means for estimating concurrent and prospective real-world IBD-related medical costs. Our model was developed using data from commercially insured and Medicare Advantage-insured patients, and demonstrated greater predictive power in users of targeted IBD therapy than in nonusers. The model has potential future utility for risk adjustment of IBD-related cost outcomes in real-world evidence studies, informing patient selection for disease state management programs, and enabling capitated arrangements with provider groups.