Real-World Comparability of Antiresorptive Osteoporosis Treatment Groups Among Treatment-Naïve and Treatment-Experienced Women Ages 55 and Older in the United States

Introduction

Osteoporosis (OP) is a systemic skeletal disorder characterized by low bone mass and compromised bone strength, predisposing individuals, particularly older women, to an increased risk of fracture.¹ Management of OP includes reducing modifiable risk factors through dietary and lifestyles changes (eg, quitting smoking, reducing alcohol consumption, increasing calcium and vitamin D intake, exercise) and treatment with pharmacologic therapy.² Pharmacologic treatments available for OP include antiresorptives such as oral bisphosphonates (BPs) (alendronate, risedronate, and ibandronate), intravenous BPs such as zoledronic acid (ZA), and RANKL inhibitors such as denosumab.

Real-world comparative effectiveness studies of antiresorptive medications may be confounded by disease severity, as selection of OP treatment is likely affected by measured and unmeasured factors related to patients’ fracture risk. Previous research suggests that treatment-naïve initiators of oral BPs are comparable to each other (eg, alendronate versus risedronate).³ This would be expected because oral BPs are well established in clinical practice, have the same labeling/indications and route of administration, and are commonly used as first line therapies. However, treatment-naïve initiators of denosumab were not comparable to treatment-naïve initiators of oral BPs,³ likely due to increased relative ease of use of denosumab compared to oral BPs (ie, single administration every six months vs daily or weekly administration, respectively) and reimbursement constraints that restrict access to denosumab to patients at higher risk of fracture or among patients that have failed on various classes of therapy.

It is commonly accepted that all studies using real-world data have some amount of residual confounding. Prior to conducting a comparative effectiveness study between antiresorptives, evidence of residual confounding using negative control outcomes (NCOs) needs to be evaluated.^4,5 For an NCO to be valid it must: 1) be causally unrelated to the treatments under study; and 2) share common bias structures with the treatments and outcomes of interest.^4,6 A lack of an association between the treatment and NCO provides evidence of comparability of treatment groups, supporting further comparative analyses using relevant disease endpoints and mitigating concerns about residual confounding. NCOs have been used to assess potential bias in several recent studies, including assessments of methylphenidate treatment and seizures,⁷ antidiabetics and COVID-19 diagnosis or death,⁸ and the effectiveness of mammography screening.⁹ The use of NCOs is also being adopted by the FDA for real-world evidence as part of the Prescription Drug User Fee Act VII Commitment.¹⁰

Using 12 NCOs, we sought to assess the comparability of antiresorptive OP treatment groups among both treatment-naïve and treatment-experienced women in a large, national commercial insurance claims database. Collectively, the findings from this study inform the choice of valid treatment contrasts for subsequent comparative effectiveness and safety studies of OP medications and fracture outcomes.

Materials and Methods

Study Population

The study population included women aged ≥55 years in the Optum^© Clinformatics^® Data Mart database, a de-identified adjudicated administrative health claims database that includes 15–18 million individuals covered annually by commercial insurance in all 50 US states, including privately insured individuals (<65 years) and those with employer-sponsored Medicare Advantage plans (≥65 years). The data capture reimbursement records for inpatient and outpatient diagnoses, procedures, and medications.

We used data from October 2010 through June 2019, with patients entering the cohort from February 2012 through December 2018. Women were included if they received denosumab, zoledronic acid, risedronate, alendronate, or oral ibandronate, and had at least 15-months of continuous enrollment preceding treatment initiation (index date). We excluded women with a history of Paget’s disease, cancer (excluding non-melanoma skin cancer), cancer treatment (chemotherapy, hormonal treatment, radiation therapy), or dementia, as well as those prescribed more than one study treatment at index. For the analysis of early hip fracture, we additionally excluded women who had a documented hip fracture within 90 days prior to index.

Treatments, Outcomes, and Follow-Up

Exposures of interest included new use of the following four OP treatments: alendronate alone, oral BPs (alendronate, risedronate, and ibandronate), ZA, and denosumab. Alendronate was an individual exposure of interest because it is the most commonly used first line OP treatment among oral BPs. Medication-specific dosing intervals were as follows: oral BPs, 30 days; ZA, 365 days; denosumab, 182 days. Women who received a different study medication within 455 days prior to index were considered treatment-experienced and those with no prior OP study medication use (based on all available patient history) were categorized as treatment-naïve. Teriparatide and abaloparatide were not included because they are used infrequently and romosozumab was not included because of its recent approval. Within the treatment-experienced group, oral BP users were required to switch from an alternate oral BP, while denosumab or ZA initiators could have switched from any other study drug during the 455 days prior to index. Denosumab and ZA were identified using Healthcare Common Procedure Coding System (HCPCS) and National Drug Code (NDC) codes. The HCPCS codes for these drugs were not specific to the indication for postmenopausal osteoporosis; however, we excluded patients with any evidence of cancer (ie, diagnoses, procedures, treatments) to remove other possible indications (eg, bone metastases). Oral BPs were identified using NDCs. The exposure contrasts of interest were denosumab versus ZA, denosumab versus oral BPs, denosumab versus alendronate, and ZA versus oral BPs.

The 12 NCOs were selected a priori by an independent panel of subject matter experts and identified in the data using diagnosis, procedure, and drug codes. The selected conditions had to be identifiable in the data, expected to have no causal association with the OP medications of interest, and were thought to have some association with a domain of confounding, such disease severity, frailty, access to care, and health-seeking behavior.¹¹ The pre-specified outcomes included: decubitus ulcer, dementia, transfusions, accidents, annual wellness visit, pelvic exams for cancer screening, influenza vaccine use, Mohs surgery, herpes zoster vaccine, colon cancer screening, visual field tests, and early hip fracture. Supplementary Table S1 summarizes the domain(s) of confounding addressed by each of the selected NCOs.

Women were followed from treatment initiation until experiencing the first of the following: NCO of interest, death, treatment discontinuation (defined as ≥30-day gap), disenrollment, diagnosis or treatment related to cancer or Paget’s disease, end of risk period (90 days for early hip fracture; 365 days for all other outcomes), or end of study data on December 31, 2019.

Covariates

A total of 53 covariates that may influence the provision of OP treatment or which are known or hypothesized risk factors for the outcome were included to control for confounding. Covariates were measured during the 15-month lookback period (inclusive of the index date) and included: demographics (age, geographic region, calendar year of index), healthcare utilization (number of physician office and emergency room visits and number of hospital visits), and the Charlson comorbidity index score, as well as the individual chronic diseases and medications specified in Supplementary Table S2. Each covariate was identified using a case ascertainment algorithm, provided in Supplementary Table S3, that utilized individual or combinations of inpatient and outpatient ICD-9-CM or ICD-10-CM diagnosis or procedure codes, CPT-4 or HCPCS procedure codes, and/or NDCs.

Statistical Analysis

We estimated the 1-year cumulative risk of each NCO (apart from early hip fracture, which was evaluated at 90 days) using an augmented inverse-probability of treatment and censoring weighted (AIPW) estimation function, with death treated as a competing risk.^12–14 This approach, as described by Ozenne et al (2020), uses models for both treatment/censoring to address confounding in initial treatment assignment and potentially informative censoring by loss to follow-up, augmented with an assumed model for the outcome. The AIPW estimator is doubly robust: if either the treatment/censoring model or the outcome model is properly specified, then the estimator will be consistent. We employed the AIPW estimator to mitigate bias due to model misspecification given that NCO analyses may indicate lack of comparability due to residual confounding or model misspecification. The propensity score for each exposure contrast was estimated using a multinomial logistic regression model that included all baseline covariates and history of the NCOs, except binary covariates with a prevalence of less than 0.1% within any index drug group. Covariate balance for each treatment comparison was assessed in the corresponding inverse-probability of treatment weighted population using the standardized mean difference (SMD). An SMD >0.1 suggested residual imbalance for a given covariate. For treatment contrasts with residual imbalances, we collapsed categories where appropriate. Inverse probability of censoring weights was estimated by modeling the composite risk of censoring (cancer or Paget’s disease, treatment discontinuation, disenrollment) for each outcome using a Cox proportional hazards model with age as a predictor. The outcome model was also fit using Cox proportional hazards regression and contained the same covariates as the propensity score model. We note that both Cox regression models evaluated time in intervals according to the observed occurrence of events in the data (as opposed to a prespecified coarsening of time, eg, 30-day intervals). For the subgroup analyses, we used a minimal set of covariates for the outcome model to avoid issues with model convergence because of the smaller sample size. All model specifications are available in Supplementary Table S2.

We conducted subgroup analyses among: 1) patients older than age 75 years; 2) patients with a recent history of fracture (within 12 months prior to index); and 3) patient results stratified by calendar time (before and after switching from ICD-9 to ICD-10 coding system on October 1, 2015). We also conducted a sensitivity analysis that emulated an ITT design, where we did not censor for treatment discontinuation, treatment switches, or for patients starting another treatment while still on their index treatment. We did not adjust for multiple comparisons when reviewing outcome estimates, because the purpose of our study was to identify possible bias rather than detect true associations.¹⁵

We additionally implemented a Bayesian framework that aggregates multiple NCO estimates and their associated variances into a new form characterized by two summary parameters: magnitude and probability of residual bias. As conclusions from an NCO study are strengthened by evaluating multiple negative controls, the results are typically a collection of estimates and their corresponding variance estimates. However, when these effect measures range in both size and precision, interpreting the totality of results is not straightforward.

A complete description of the Bayesian approach is provided in the Supplemental Materials (Supplementary Methods 1). Briefly, we constructed the posterior predictive distribution of the expected RD for the collection of 12 NCOs (as identified a priori by subject matter experts) and reported the 80%, 90%, and 95% quantiles of the absolute value of the distribution. The modeling approach specified an underlying distribution with modes on either side of the null value of 0, reflecting the possibility that residual bias may manifest itself positively or negatively away from the null. The resulting posterior distributions may then be used to consider whether the magnitude and probability of bias is acceptably low enough to perform a comparative safety or effectiveness study under the same conditions. For example, a Bayesian summary RD close to the null value of 0 (ie, the interval between the 2.5 and 97.5% quantiles of the posterior predictive distribution contains the null) may indicate adequate comparability between treatment groups. It can also be used to contextualize existing comparative results. Posterior predictive distributions were compiled for each combination of exposure contrast of interest, patient treatment history type (ie, naïve vs experienced), and population sub-group and/or sensitivity analysis described above for a total of 66 scenarios.

This study was a retrospective analysis of deidentified, secondary data and determined to be exempt from oversight by an Institutional Review Board (IRB) under 45 CFR 46.101(b)(4) of the US Department of Health and Human Services regulations (Advarra IRB, Columbia, Maryland, waived ethical approval for this work). All statistical analyses were performed using R software, version 3.5.2.

Results

Baseline Characteristics

There were 199,335 eligible treatment-naïve women (67% alendronate alone, 81% oral BPs, 6% ZA, 13% denosumab) and 33,296 treatment-experienced women (26% alendronate alone, 51% oral BPs, 12% ZA, 37% denosumab) (Figure 1). In both cohorts, women initiated treatment at a mean age of 71.8 years (SD = 8.2). The denosumab and ZA groups were generally similar with regard to baseline characteristics but were older and had a higher prevalence of comorbidities compared to oral BP initiators (both naïve and experienced patients) (Table 1). Before weighting, denosumab and ZA initiators had a higher prevalence of prior preventive measures (eg, influenza vaccine, colon screening) than oral BP initiators in both cohorts (Table 2). In addition, accidents were more common in ZA and denosumab treated subjects compared to oral BP treated subjects. After weighting, the measured covariates were balanced for all treatment comparisons (Supplementary Table S4).

Table 1 Baseline Characteristics (%) Assessed During the 15 Months Prior to Treatment Initiation Among Treatment-Naïve and Treatment-Experienced Women

Table 2 Baseline History of Negative Control Outcomes (%) Assessed During the 15 Months Prior to Treatment Initiation Among Treatment-Naïve and Treatment-Experienced Women

Figure 1 Study attrition diagram.

Treatment-Naïve Comparisons

Women initiating denosumab had similar 1-year risks of most NCOs compared with initiators of ZA (Figure 2). However, when comparing ZA or denosumab with oral BPs, we observed significant differences on the risk difference scale for 7 out of 12 NCOs, suggesting that residual bias exists between ZA or denosumab and oral BPs. Therefore, in a treatment-naïve population, comparative studies between ZA or denosumab and oral BPs are not justified. For example, compared with oral BPs, patients on denosumab had a higher risk of having an annual wellness exam (RD = 4.7%, 95% CI = 3.5, 5.8%). Overall, patterns were similar on the risk ratio scale (Supplementary Figure S1).

Figure 2 1-year cumulative risk differences of negative control outcomes for treatment-naïve women initiating an OP treatment.

Treatment-Experienced Comparisons

Among treatment-experienced women, all NCOs when comparing denosumab to alendronate alone indicated similar risks (Figure 3). Significant differences in risk were observed for only 2 of 12 NCOs (dementia and wellness) when comparing denosumab to oral BPs, and only 1 NCO (influenza vaccine) was associated with treatment when comparing ZA to oral BPs. Estimates on the risk ratio scale were similar and are available in Supplementary Figure S2.

Figure 3 1-year cumulative risk differences of negative control outcomes for treatment-experienced women switching to a new OP treatment.

Subgroup Analyses

Figures 4 and 5 present RDs for the 12 NCOs for subgroups within treatment-naïve and treatment-experienced women, respectively. Across all subgroup analyses in treatment-naïve women, comparisons between denosumab and ZA were more consistent than comparisons between denosumab and oral BPs. Like the main analysis, there was more residual confounding in the comparisons against oral BP. Treatment-naïve women aged >75 years experienced similar patterns of risk differences for all NCOs as the main cohort. However, among treatment-naïve patients with a recent fracture, the RD between treatment and NCOs was attenuated, suggesting decreased residual bias in this subgroup compared to the overall cohort. For example, among treatment-naïve patients with a recent fracture, the RD for wellness exams when comparing denosumab or ZA with oral BPs moved closer to the null. Similarly, among treatment-experienced patients initiating denosomab (as compared to oral BPs) RDs for the wellness exam and dementia NCOs were also attenuated. When stratifying by calendar period, we observed stronger associations before October 2015 than after among treatment-experienced women (eg, RD for influenza vaccine for denosumab versus oral BPs, before 2015: 3.9% [95% CI: 1.7–6.2%] versus after 2015: 2.6% [95% CI: −0.1–5.4%]). However, risks were similar in both time periods for the treatment-naïve cohort. RDs for the 12 NCOs in the ITT sensitivity analysis are provided in Supplementary Figure S3 (treatment-naïve) and Supplementary Figure S4 (treatment-experienced). Overall, associations with individual NCOs in the sensitivity analysis were consistent with the results in the overall cohort.

Figure 4 1-year cumulative risk differences of negative control outcomes for treatment-naïve women initiating an OP treatment, by subgroup.

Figure 5 1-year cumulative risk differences of negative control outcomes for treatment-experienced women switching to a new OP treatment, by subgroup.

Bayesian NCO Bias Characterization

The 80%, 90%, and 95% quantiles of the absolute value of the posterior distributions estimated for all treatment contrasts on the risk difference scale are provided in Supplementary Table S5. Overall, the results of the Bayesian analysis aligned with our qualitative interpretation. For example, for the denosumab vs ZA contrasts in the treatment-naïve cohort, only one RD (dementia) had a 95% CI that excluded the null. Findings from the Bayesian analysis were consistent with the low levels of residual bias indicated by the individual NCO results; the upper bound of the 95% quantile-based interval of the absolute RD was 5 events in 1000. In scenarios with higher residual bias, the results from the Bayesian analysis were also consistent. When considering the denosumab vs alendronate contrasts in the treatment-naïve cohort, more than one-half of the 95% CIs excluded the null, suggestive of greater residual bias. The Bayesian analysis produced a 95% quantile-based interval of the absolute RD with an upper bound of 37 events in 1000.

Discussion

Among a population of 199,335 treatment-naïve and 33,296 treatment-experienced women, we estimated residual bias of different OP treatments using 12 NCOs to assess the comparability of the treatment groups for comparative effectiveness and safety studies. After adjusting for over 50 possible measured confounders using a doubly robust estimator, we found that denosumab and ZA treatment-naïve users had similar risks of NCOs, suggesting that these two treatments could be reasonably compared with respect to safety and effectiveness outcomes of interest. Based on the NCO analyses, no other treatment groups were considered comparable among treatment-naïve women overall. After aggregating the results across all NCOs, the absolute magnitude of residual bias for comparisons of treatment-naïve users of ZA and oral BP was 2.9% (as indicated by the 95% credible interval), an amount that was considered to be a high level of bias compared to any hypothesized treatment effect for a potential comparative effectiveness follow-up study. However, restriction to patients with severe disease, indicated by a recent fracture, improved comparability for treatment-naïve women initiating ZA as compared to oral BPs. Among treatment-experienced women, it appeared that all women initiating denosumab, ZA, and oral BPs (including alendronate alone), were comparable (95% credible intervals of the absolute risk differences having between as many as 1.1% and 2.5%, depending on the contrast). Restriction by age showed no improvement in residual confounding among treatment-naïve or treatment-experienced women. However, restriction by calendar time suggested better comparability in the later period among treatment-experienced women; however, there was little impact in the treatment-naïve population.

Our results in the treatment-naïve population were similar to results previously reported among women enrolled in MarketScan commercial or Medicare Advantage plans.³ Women initiating denosumab were comparable to those initiating ZA, but residual bias was likely present in the comparison with oral BPs. While we found residual confounding to be associated with women taking denosumab vs oral BPs, the estimates were generally in the same direction. For example, the RD estimate for wellness exam in our study was 4.7% (95% CI: 3.5–5.8%) and was 1.15% (95% CI: 0.41–1.89%) in the MarketScan population.³ It is encouraging that we found similar results in two different data sources, as our study population was generally older (mean age = 71 years vs 67 years in MarketScan) and had more comorbidities (15% vs 10% in MarketScan had a history of fracture).³

It is likely that women initiating an injectable medication as their first observed treatment for OP are meaningfully different from women initiating an oral BP. Indeed, we observed that, compared to women initiating an oral BP, women starting denosumab or ZA had increased healthcare utilization (eg, outpatient visits and hospitalizations) and appeared to be in worse health (eg, higher prevalence of comorbidities). Elevated differences in the risk of NCOs indicated that unmeasured confounding remained after adjustment. This is further evidence that using claims data alone to estimate comparative effectiveness or safety of first line injectable medications as compared to an oral BP is likely to result in a biased comparison among treatment-naïve, commercially insured women in the United States.

Among women starting second line treatment, the estimates were closer to the null, suggesting less evidence of residual confounding. Even when comparing women switching to denosumab with women switching to an oral BP, the risk of most NCOs was similar. Patients can switch medications for a variety of reasons, including more advanced disease (such as fractures while on treatment), the emergence of side effects, and physician or patient preference for specific dosing schedules.¹⁶ This could implicitly make the treatment groups more comparable even in the absence of clinical measurements, such as low bone mineral density, which is an important predictor of fracture.¹⁷

Restricting the cohort to the recent fracture subgroup appeared to make the treatment groups more comparable. For example, among treatment-naïve women, there were marginal differences in the risk for wellness visits when comparing denosumab to oral BP users in the full cohort (4.7% RD) which were further reduced in the recent fracture subgroup (2.6% RD). There were fewer NCOs that were associated in the high-risk subgroup for both treatment-naïve and treatment-experienced patients. It has been shown that restriction can often reduce unmeasured confounding where analytic methods for adjustment fall short.^18,19 Restricting to women aged >75 years did not appear to reduce residual confounding, as estimates were similar between the subgroup and the full cohort, likely attributable to the median age at treatment initiation being 71 years. This could indicate that recent fracture is a better indicator of severe/advanced disease in this population, which is supported by research showing that prior fractures are predictive of future fractures.²⁰

We observed a difference in the amount of possible residual confounding among the treatment-experienced group stratified by calendar time—there were more NCOs that were associated with treatment (for all treatment groups) before October 2015 than after October 2015. This could indicate channeling bias that may have been more apparent in the years soon after denosumab was approved for post-menopausal osteoporosis in 2010.^21,22 Physicians may preferentially administer denosumab to sicker patients with more advanced OP, resulting in unmeasured confounding. However, we did not see similar trends suggestive of channeling bias among the treatment-naïve cohort. Similarly, oral bisphosphonates represent a heterogeneous group of medications with numerous generic formulations whereas no generic versions of denosumab were available during the time period of the current study (the first interchangeable biosimilar to denosumab was approved by the FDA in March 2024).²³ Although this heterogeneity may limit comparability when grouping oral bisphosphonates together, our findings did not change substantially when we restricted comparisons to alendronate alone.

Our study had some limitations. The 12 NCOs were identified by clinical subject matter experts with specific consideration of various hypothetical confounding mechanisms, increasing our confidence that the observed null associations indicate the relevant treatment groups are comparable. However, as in all real-world studies, residual confounding may remain due to undefined confounding domains or unmeasured variables. For example, the data did not contain clinical measures of disease severity, such as bone mineral density or body mass index. Bone mineral density and fracture history while on therapy are important determinants of whether clinicians initiate or escalate to denosumab or zoledronate, particularly among patients with more severe osteoporosis or fractures occurring despite oral bisphosphonate treatment. Prescribers may also avoid oral bisphosphonates in patients using proton pump inhibitors (PPIs) due to concerns about gastrointestinal absorption. Additionally, PPIs themselves have been associated with increased fracture risk—which may result in channeling higher-risk patients toward IV or injectable agents.^24,25 Additionally, the confounders we included were measured at baseline and thus our assessment did not capture information on potential time-varying confounding mechanisms. While we required patients in our study sample to be continuously enrolled for at least 15 months prior to index and used all available look-back to distinguish treatment naïve patients from experienced users, some patients may have been misclassified if their prior OP medication use was not captured in the available data. Prior research suggests that BP users may have long gaps in treatment due to “treatment holidays”,^26,27 and that denosumab and ZA users may have extended intervals between sequential treatments. Results from the Bayesian analysis could benefit from a formal “gating” approach where an a priori rule determining whether a particular contrast is comparable is defined based on some maximum allowed observed absolute risk difference relative to a given level of probability. The level of the allowed absolute risk difference could be determined relative to a hypothesized risk difference for a comparative effectiveness study or a required level of confidence for a safety study. Lastly, our cohort was comprised of women with commercial or Medicare Advantage insurance; therefore, our findings may not generalize to other populations, such as the general Medicare population or those residing outside the United States.

Conclusion

Overall, our findings support the conclusion that comparative effectiveness and safety studies of fracture outcomes and denosumab or ZA versus oral BPs among treatment-experienced, commercially insured women in the United States are likely valid with respect to the comparability of treatment groups, particularly for high-risk subgroups and in later calendar periods. However, our results do not support conducting observational studies examining these treatment contrasts among treatment-naïve patients in this commercially insured study population.