Considerations for the Utility of Real-World Evidence Beyond Trial Data in Advanced NSCLC: The Case of Frontline Tyrosine Kinase Inhibitors

Introduction

Identification of new oncological targets has led to a sharp increase in the development and approval of novel cancer therapies.¹ In order to accelerate access to innovative cancer medications, regulatory approvals are often based on non-OS endpoints. Between 1992 and 2017, accelerated approvals in the US were granted in 93 cancer indications, of which 81 (87%) were based on response rates, eight (9%) on PFS or time to progression, and four (4%) on disease-free survival.² During 2006–2017, in the US, >70% of cancer treatment approvals for later lines of therapy were granted based on (non-OS endpoints) PFS or relapse-free survival.³ In Europe, between 2014 and 2017, 39% (34 of 88 marketing authorization applications) of cancer treatments were approved based on non-OS endpoints because OS was immature at the time of the drug application.⁴

In situations where trial OS data are immature, the evidence generated based on real-world data (RWD) can be helpful in addressing the uncertainty payers face related to the effectiveness of a therapy beyond the trial data.⁵ Given the widespread availability of RWD, there is substantial interest in validating and comparing clinical effectiveness using RWD observed in clinical practice settings and clinical efficacy generated from randomized clinical trials (RCTs).^6,7 Real-world evidence (RWE) can demonstrate the short- and long-term clinical value of a drug in a clinical practice,⁸ as well as help payers predict meaningful health benefit for patients underrepresented in RCTs.^9,10

Patients with aggressive cancers, such as advanced (ie, locally advanced or metastatic) non–small cell lung cancer (aNSCLC),¹¹ can benefit from innovation in accelerating the approval and reimbursement of potent novel therapies. In an effort to extend the discussion on the usefulness of RWE to support payer decision-making, the main objective of this study was to compare real-world effectiveness (using electronic health record [EHR] data from US oncology practices) and efficacy observed in aNSCLC clinical trials. To do this, we chose three RCTs on tyrosine kinase inhibitors (TKIs) given as frontline treatment in aNSCLC patients with EGFR or ALK mutations versus platinum-based chemotherapy (PBC): (1) erlotinib vs PBC in EGFR+ patients from the EURTAC trial,¹² (2) afatinib vs PBC in EGFR+ patients from the LUX-Lung 3 trials,¹³ and (3) crizotinib vs PBC in ALK+ patients from the PROFILE 1014 trial.¹⁴ At the time of initial reimbursement decisions by payers, all three TKI trials had demonstrated substantial benefit in PFS, while OS data were still immature and regulatory approvals were achieved based on PFS benefit. The focus on these therapies stems from the fact that they represent approved innovative therapies in aNSCLC with considerable long-term use and follow-up in RWD.

Methods

Summary of Included RCTs

A detailed summary of the three RCTs we attempted to mimic using RWD is shown in Table 1. NSCLC patients from the three RCTs were included if they were adults, had a histological diagnosis of stage IIIB or stage IV, had received no prior chemotherapy for advanced NSCLC, and largely had an Eastern Cooperative Oncology Group (ECOG) performance status <2. The primary endpoint was PFS, defined as the time from the date of randomization to the date when disease progression was first observed or death occurred. Treatment switching was allowed post progression.

Table 1 Summary of baseline characteristics of included RCTs

RWE Study Design

This RWE analysis was a retrospective cohort study, implemented using Flatiron Health’s longitudinal EHR database. Flatiron Health contains aggregated, normalized, and harmonized de-identified patient-level data curated from structured and unstructured data via technology-enabled chart abstraction.¹⁵ This database represents over 265 US cancer clinics, including >2 million patients with cancer overall and 120,000 patients with a structured International Classification of Diseases code for lung cancer.

The Flatiron Health’s EHR data used for this analysis were a secondary database that included de-identified patient data. The secondary use of de-identified patient data is explicitly exempted from ethics review and informed consent as per the Department of Health and Human Services regulation 45 CFR 46.104(d)(4). The study authors received permission to use this EHR database from Flatiron Health through a third-party agreement.

Patients aged ≥18 years with advanced stage IIIB/IV NSCLC on or after January 1, 2011 and followed up until August 30, 2020 were included in the study as long as they had had at least 2 clinical encounters and had recorded treatment within 30 days before or 90 days after their aNSCLC diagnosis. We restricted analysis to patients with a positive EGFR or ALK mutation status. The index date was defined as the date of initiation of first line of therapy (1LOT), and patients were followed longitudinally until death or end of follow-up. Patients initiating 1LOT with afatinib, crizotinib, and erlotinib were allowed to initiate the therapy as monotherapy or combination therapy. Patients initiating 1LOT with PBC were categorized based on the use of carboplatin/cisplatin in combination with any of the regimens used in the three pivotal trials (pemetrexed, docetaxel, or gemcitabine). While the selection of chemotherapy regimens may not have reflected the chemotherapy arm of each individual trial, this decision was made in order to achieve a large enough sample size for the RWE chemotherapy comparator arms. Additionally, the literature suggests that cisplatin is equivalent to carboplatin and that pemetrexed is more efficacious than docetaxel/gemcitabine.¹⁶ Therefore, adding pemetrexed to the RWE comparator arm was considered a conservative assumption with respect to mimicking the chemotherapy arm of the EURTAC RCT,¹⁶ which consisted of cisplatin with docetaxel/gemcitabine with optional substitution with carboplatin for cisplatin. In contrast, docetaxel or gemcitabine combinations were not added to the RWE chemotherapy arm for the ALK+ cohort, as that could have made the RWE comparator arm less effective on average (the chemotherapy regimen in PROFILE 1014¹⁴ consisted of cisplatin/carboplatin plus pemetrexed).

RWE Data Analysis

The following patient and clinical characteristics were assessed during the baseline period as presented in the source trial publications for the three RWE TKI vs PBC cohorts of interest: age, sex, year of diagnosis, race/ethnicity, community or academic treatment setting, geographic region, smoking status, time from initial diagnosis to index date, stage at diagnosis, ECOG performance status, histology, mutation variants, and brain and bone metastases. Each of the three TKI analytic cohorts was matched to the respective PBC arms to achieve balance in these important baseline characteristics. Specifically, an exact matching algorithm allowing for variable ratio between the number of matched treated and control patients was used in order to optimize retention of sample size with balancing baseline characteristics.^17,18 The variables matched on included age, sex, race, smoking status, BMI, ECOG performance status, histology, mutation type, stage, region, physician practice setting, time to treatment initiation, year of diagnosis, Charlson Comorbidity Index, and presence of bone and brain metastases. The erlotinib and crizotinib cohorts resulted in a 1:1 match and the afatinib cohort a 1:2 match.

While progression data could be available in Flatiron for select cohorts, there were several reasons for choosing time to treatment discontinuation (TTD) in our analysis. Clinical trials define progression using the RECIST criteria, while real-world PFS obtained via EHRs is defined using clinician notes, chart documents, and other evidence, such as pathologic and radiology reports.¹⁹ Irrespective of the richness of the EHR data, PFS by RECIST criteria cannot be adequately obtained via EHR-derived data.¹⁹ Additionally, at the time of this study, we did not have information on the concordance between real-world PFS and conventional PFS for TKI agents for aNSCLC. Finally, as an endpoint readily available in RWD sources, real-world TTD could be a useful measure indicative of short- and long-term benefit in aNSCLC and can be employed during reimbursement decision-making discussions, given that OS data interpretation may be limited, even with longer follow-up.⁵ Previous patient-level pooled analyses of eight trials showed an overall strong correlation (r=0.91) between TTD and PFS when studying patients with aNSCLC treated within the drug categories of EGFR-TKI, ALK-TKI, immune checkpoint inhibitors, or chemotherapy.²⁰ Therefore, TTD was used to evaluate real-world effectiveness in the three analytic cohorts as a proxy for PFS, which was the primary endpoint in the three selected RCTs. TTD was defined as the time difference between the first and last date of a 1LOT regimen unless there were <120 days available follow-up after the end of 1LOT. In those cases, alive patients were censored for TTD, based on the last confirmed activity (last clinical visit or drug administration). The choice of the 120-day criterion was made based on the refill timing distribution observed in the data set. The choice of using TTD was justified based on recent literature in aNSCLC highlighting its use as a potential real-world endpoint that could be used to help determine a therapy’s efficacy in the postmarketing setting;²¹ TTD has been found to correlate with both PFS (r=0.87) and OS (r=0.68) among 8,947 patients with aNSCLC from clinical trials.²²

Real-world TTD and OS were measured in months and are presented using Kaplan–Meier curves. HRs were calculated via a Cox proportional hazard model while the proportional hazard assumption was tested using Schoenfeld residuals. Similarly, reported PFS and OS curves from the RCTs were digitized and plotted alongside the RWE curves. Then, measures of relative efficacy (ie, HRs) were naïvely compared between the two sets of data for each of the three individual cohorts of interest. No statistical test was conducted to compare the results from the RCT and Flatiron data. To examine potential correlations between TTD and OS in this exploratory analysis, the difference between the area under the curve (in 6-month intervals) for real-world TTD and OS of TKIs vs PBC was calculated and correlated using the Spearman’s rank-order metric. Additionally, switching patterns and subsequent therapies were also described. Data management and analyses were conducted using SAS version 9.4 (Cary, NC, USA) and R software (R Foundation for Statistical Computing, Vienna, Austria).

Results

Applying inclusion and exclusion criteria overall resulted in 1,907 aNSCLC patients (Figure 1), and after cohort matching this decreased to 1,118 patients, distributed across the following groups: erlotinib (n=487) vs PBC (n=173) in the first EGFR+ cohort; afatinib (n=178) vs PBC (n=107) in the second EGFR+ cohort; and crizotinib (n=126 vs n=47) in ALK+ patients. The size of the RWE patient population for the first EGFR+ cohort was close to four times that of the EURTAC trial. The second EGFR+ cohort was a similar size to the LUX-Lung 3 trial, while the ALK+ cohort was about half that of PROFILE 1014.

Figure 1 Patient attrition flow in the Flatiron Health database.

The erlotinib- and afatinib-treated EGFR+ and the crizotinib-treated ALK+ patients were well balanced (Table 2) compared to their PBC comparator group with respect to demographics, region, practice setting, and clinical characteristics (ie, history of smoking, ECOG performance status, stage, histology, bone and brain metastasis) and were overall deemed similar to the patient population examined in clinical trials. Specific to the EGFR+ cohorts, exon 19 deletion mutations and L858R mutation in exon 21 were also equally distributed between the TKI vs PBC groups. The main difference between patients on TKI vs PBC was the timing of the EGFR/ALK test relative to 1LOT initiation: the majority of TKI patients (85%–94%) had had their mutation status established before the date of 1LOT receipt, compared to 30% or less among patients on PBC.

Compared to the patient population from the RCTs (Table 1), more patients in the matched RWE cohorts (Table 2) had a history of smoking, were older (especially for the ALK+ cohort), and white. Specific to the RWE EGFR+ cohorts, slightly more patients had stage IV NSCLC and fewer had exon 19 deletion mutations than patients in the RCTs.

Table 2 Baseline characteristics in matched cohorts

Frontline TKI regimens had statistically significantly better real-world TTD than their matched PBC comparator group (HR 0.37, 95% CI 0.30–0.44 for erlotinib; HR 0.42, 95% CI 0.32–0.55 for afatinib; HR 0.37, 95% CI 0.26–0.53 for crizotinib) (Figure 2). These estimates of relative benefit were similar to those measured for PFS in registrational trials (HR 0.37, 95% CI 0.25–0.54 for erlotinib in EURTAC;¹² HR 0.49, 95% CI 0.37–0.65 for afatinib in LUX-Lung 3;¹³ HR 0.45, 95% CI 0.35–0.60 for crizotinib in PROFILE 1014).¹⁴ Benefit in real-world OS was not different between first-line TKIs initiators and PBC, similar to what was reported in clinical trials (Figure 2), with 95% CIs for HRs for OS overlapping across the RWE and RCT paired comparisons.

Figure 2 Real-world TTD, OS, and PFS from the randomized clinical trials for the erlotinib, afatinib, and crizotinib cohorts. Notes: For erlotinib and PBC data from the RCT, OS data from the OPTIMAL, CTONG-080240,41 are plotted, as EURTAC did not report a readily available Kaplan–Meier curve for OS.

Kaplan–Meier curves for long-term real-world TTD and OS curves were similar or lower than the shorter-term curves for PFS and OS observed in the RCTs. The starkest contrast was that for the ALK+ cohort, where RWE patients were much older than those examined in the RCT (median age 62 vs 53 years) and expectedly had lower probabilities of treatment continuation and survival. The benefit of better real-world TTD did not correlate with benefit of better real-world OS for first-line TKI initiators in the matched cohort over PBC (r=–0.071, P=0.79 for erlotinib; r=–0.25, P=0.79 for afatinib; r=0.062, P=0.82 for crizotinib; Figure 2). This was attributed to a high proportion of early switching among PBC patients to subsequent lines of TKI and programmed death-1 (PD-1)/programmed death-ligand 1 (PD-L1) regimens, likely post progression (Table 3). Proportionally, more PBC patients were exposed to TKI therapy across all subsequent lines of therapy than first-line TKI initiators in the EGFR+ cohort (47.6% vs 61.7% for erlotinib vs PBC; 52.4% vs 63.7% for afatinib vs PBC; 67.1%) and more PD-1/PD-L1 therapy exposure in the ALK+ cohort (6.0% vs 17.9% for crizotinib vs PBC).

Table 3 Patterns of subsequent lines of therapy after treatment discontinuation of first line

Discussion

Mirroring the populations from three sets of RCTs, our RWE study showed that the difference in the longer-term TTD was consistent with the difference in PFS benefit observed in pivotal trials among aNSCLC EGFR/ALK+ patients. Our study is insightful, as it shows that it is possible to replicate the comparative assessment from RCTs in RWE settings, as well as RWD being an important data source to investigate longer-term clinical benefits from RCTs. Last but not least, the results demonstrate that TTD could be an important proxy for non-OS outcomes, such as PFS. Our findings are even more relevant in the context of the increasing use of new techniques used to detect genetic factors associated with advanced NSCLC, such as cell searching for circulating tumor cells.^23,24

In terms of endpoints, PFS adequately demonstrates the clinical benefit of drug therapy, has an impact on health-related quality of life, and shows that a sizeable benefit generates incremental social value.^25–27,41 Given that our results demonstrate that relative treatment effect assessed via TTD is a good proxy for PFS in aNSCLC when comparing the HRs, future research could investigate the correlation between RCT-assessed PFS and RWD-assessed TTD quantitatively, as the latter is a more readily available measure in observational databases.²⁸ Additionally, such research conducted on tumors beyond aNSCLC will support the validity of TTD in conveying the clinical benefit of therapies. As mentioned in Blumenthal et al, comparing absolute TTD with absolute PFS is not recommended for finite regimens. However, based on the results presented in Blumenthal et al, Griffith et al, and our study, TTD HR adequately aligns with PFS HR; therefore, it is a good proxy to demonstrate clinical benefit. Through TTD, we assess duration of treatment, which is a helpful construct for managed entry agreements (ie, innovative value strategies) structured as outcome-based contracts with payers, especially for continuous treatments. More than half the health plans and pharmacy benefit managers in the US use some type of risk-sharing agreement with drug and device manufacturers.²⁹ Therefore, during negotiations, if health plans and pharmacy benefit managers accept TTD/duration of treatment as an endpoint that adequately conveys clinical benefit as well, this could simplify value-based pricing frameworks.³⁰

It is worth noting that neither the TTD benefit of initial TKI treatment seen in our RWE study nor the PFS benefit observed in clinical trials translated to OS gains. This is likely due to the subsequent switching to TKIs and other later treatment lines as salvage therapy post progression, as was observed in the RCTs.³¹ While methods for treatment-switching adjustment, such as rank-preserving structural failure time (RPSFT), two-stage methods, and inverse probability-of-censoring weights do exist, their adjustment is often limited, as each one’s performance depends on strong assumptions that are often not met, given the data at hand. For example, while RPSFT could have been an appropriate methodology to adjust for treatment switching in PROFILE 1014,³² this was the case because the conditions for crossover from PBC to crizotinib and from crizotinib to PBC were substantially met. In other trials, switching patterns are more complicated,³³ just like in our RWE study, and not one single method guarantees that OS adjustment can be attributed to the therapies patients were initiated on. While we attempted to conduct adjustment for treatment switching on OS in the three study cohorts using RPSFT, the results (not shown) were not informative, as the common treatment-effect assumption of the method was violated (ie, the assumption that switching to another TKI or other subsequent line of therapy post progression will show the same therapeutic benefit as before the switch was violated). While inverse probability-of-censoring weights is another method to adjust for treatment switching, its implementation may also be limited, especially in small samples, if the proportion of patients who switched is very high and if there are unmeasured confounders.³⁴

In recent years, there has been faster development of novel cancer agents, which usually target later lines of treatment for patients with advanced disease as their first indication. This has led to an increased use of novel agents as subsequent therapies, resulting in methodological challenges to discern the effect of initial treatments from that of subsequent therapies.³⁵ The aforementioned points thus contribute to the limitations of OS as an endpoint to demonstrate long-term treatment benefit.⁵ While OS is an important endpoint in measuring the value of therapies and what matters to patients, with more therapies and multiple lines of therapy, it is often difficult to use OS as a clear measure of treatment effect. Therefore, further consideration should be given to other clinical endpoints that are important to health-care workers, prescribers, and patients.^36,37

Discussions on the use of pragmatic and measurable endpoints before treatment switching will be further strengthened by the development of health-policy frameworks related to non-OS endpoints and their facilitated integration into the regulatory and payer decision-making process.²⁷ With regulatory bodies and reimbursement agencies shifting to make RWE a more central factor in decision-making,^38,39 the need for more readily available RWE measures of efficacy is paramount.¹¹

Our RWE study is not without limitations. Only about a third of PBC patients had their mutation status measured before treatment initiation, compared to three times as many among TKI patients, which may confound results on treatment duration and likelihood of switching among PBC patients. Also, treatment provided in hospitals or any other setting outside the cancer clinic in the Flatiron database could not be observed and may have resulted in misclassification of treatment and outcomes. Information on surgery (including thoracic surgery) and radiation therapy was not available in the data, nor were there data on adverse events. Similarly, comorbidities that are treated outside the cancer setting are largely underreported in the Flatiron database, preventing adequate adjustment for such factors in the analyses. Last but not least, more exact matching of the EGFR cohorts on year of diagnosis could have reduced any unobserved bias in treatment assignment, although if we had done so, this would have substantially reduced the sample size of the study. Given the rapidly progressing nature of aNCLSC, however, these potential sources of bias are a relatively minor concern in affecting the observed treatment benefit in our study.

Conclusion

Using TKI treatments for aNSCLC as a case study, we conclude that relative TTD evaluated in RWE conveys a benefit similar to that observed via PFS in the RCTs. This implies that the benefits seen in accelerated approvals based on PFS may hold true in real life and that differences in TTD may be a good RWE proxy for PFS. However, similarly to clinical trials, long-term OS gains in RWE were not observed for frontline TKI initiations compared to chemotherapy, due to a great deal of treatment switching to subsequent therapy.