Challenges with Estimating Long-Term Overall Survival in Extensive Stage Small-Cell Lung Cancer: A Validation-Based Case Study

Introduction

Treatment strategies presented to healthcare payers can be assessed in terms of their long-term costs and benefits. This assessment is achieved through decision analytical modeling, which requires projecting outcomes after the follow-up period of the clinical trial, often over a lifetime horizon. In oncology, cost-effectiveness analyses are usually based on partitioned survival models or semi-Markov models, details of which have been described in the literature.^1,2 Assumptions are often needed in the models to extrapolate long-term survival data from clinical trials when treatments are evaluated beyond clinical trial horizons.

Statistical models, which often rely on clinical assumptions, are used to extrapolate survival data. Fitting parametric models to clinical data is the standard approach taken to estimate long-term survival data. However, standard parametric extrapolation models present some limitations. They generally estimate the survival of the entire population and may not adequately capture heterogeneity within trial populations such as differences across response status at different time points, long-term response, and whether patients can be regarded as long-term benefiters of the therapy.^1,2 On the other hand, semi-Markov models capture health outcomes using health states and use transition probabilities to estimate the movement from one health state to another.³ These models are similarly limited and may not adequately account for competing risks in certain decision problems.

Recently, the National Institute for Health and Care Excellence Decision Support Unit (NICE DSU) published guidelines to support researchers in the selection of the most appropriate extrapolation method (NICE DSU 14 and NICE DSU 21).^1,2 These guidelines suggest that researchers should consider all relevant approaches and compare the generated extrapolations of long-term survival among different approaches and the literature, especially in the disease area of oncology.

These recommendations are particularly relevant in the context of immunotherapies, where the usual parametric distributions present additional limitations as they may not reflect their mechanism of action.^1,2 In this regard, fitting models to trial data is challenging since long-term responders to treatment can drive the tails of survival curves to a “plateau” – a situation where some patients do not experience the event (not progressing nor dying) over time.⁴ Such plateaus may appear when immunotherapies are used to treat lung cancer, yet they may not be reflected in the functional forms of standard parametric distributions.⁵ In addition, long-term extrapolations may be too optimistic (with the plateau occurring earlier) or pessimistic (with the plateau occurring later), and different hazard functions shapes may appear when using immunotherapies (especially increasing followed by decreasing shapes).

Consequently, limitations with current standard approaches have led to the consideration of more flexible, and in many cases more complex models such as spline models, cure models, and landmark models that can model complex hazard functions and changes in the slopes of survival curves, which can be applied in the context of immunotherapies. However, there is still uncertainty on how to define the best approach to modelling long-term survival, and whether adding complexity to these approaches improves long-term survival predictions. Therefore, the objective of this study was to highlight the challenges that are present when modeling survival beyond the clinical trial by assessing the performance of different approaches to estimating long-term survival rates of patients with extensive stage small-cell lung cancer (ES-SCLC) treated with immunotherapies. For this purpose, we used individual patient-level data from CASPIAN (NCT03043872), a previously published clinical trial of PD-L1 inhibitor durvalumab added to first-line standard platinum-based chemotherapy in patients with previously untreated ES-SCLC.

Small-cell lung cancer (SCLC) is characterized by aggressive disease progression and accounts for 15% of lung cancer cases.⁶ Patients with ES-SCLC have traditionally been treated with the chemotherapeutic agents etoposide and platinum (EP) as first-line therapy. While patients with ES-SCLC initially respond to first-line EP therapy, patients often experience relapse; thus, identifying effective treatment options that induce durable disease control has been a challenge.⁷ However, recent clinical trials adding immunotherapy (eg, atezolizumab, durvalumab, pembrolizumab, or nivolumab) to EP have shown promising results in prolonging overall survival (OS), which has resulted in immunotherapy plus EP becoming the standard of care in many countries, including the United States and Europe.^8–12 CASPIAN is a phase 3, randomized clinical trial comparing durvalumab with or without tremelimumab plus EP to EP alone for first-line treatment of patients with ES-SCLC. The trial had 2-year and 3-year data cuts, occurring in January 2020 (referred hereafter as 2-year data, with a median follow-up of 25.1 months) and March 2021 (referred hereafter as 3-year data, with a median follow-up of 39.4 months), respectively.^11,13 In the 2-year data, patients treated with durvalumab plus EP showed significant improvement in OS compared to those treated with EP.¹³ At 24 months, the OS rate was 22.6% in the durvalumab plus EP arm, compared to 13.5% in the EP arm alone. In the 3-year data, durvalumab plus EP continued to show superior effect compared to EP alone, with a 36-month OS rate of 17.6% for durvalumab plus EP compared to 5.8% for EP alone.¹¹ The primary endpoint of OS for durvalumab plus EP vs EP alone was met in CASPIAN, while durvalumab plus tremelimumab plus EP continued to numerically improve OS vs EP alone.¹¹

As the treatment landscape for ES-SCLC is rapidly evolving, there is a need to understand – from both clinical and payer/reimbursement perspectives – the extrapolation method that is likely to be the most accurate in estimating long-term survival rates of different treatments, especially immunotherapies. Since CASPIAN is the only phase 3 study to publish 2- and 3-year follow-up data in this setting, we used the 2-year data from CASPIAN¹⁰ to compare different approaches to estimating long-term survival rates of patients with ES-SCLC in durvalumab plus EP and EP alone treatment arms, and compared our findings to the 3-year data from the same trial¹¹ to identify the method that most accurately estimates short-term follow-up survival while also considering the clinical plausibility of the longer-term predictions.

Methods

Modelling Approaches

The following approaches were used to model the long-term OS using the 2-year data: (1) standard parametric survival models; (2) spline models;² (3) cure models;^14,15 (4) landmark models;¹⁶ and (5) Markov models,¹⁷ with all of them having been described previously. The approaches were chosen based on clinical opinion, the NICE DSU 21 guidelines,² and the recent algorithm for selecting flexible survival models for cancer immunotherapies by Palmer et al.¹⁵ Each parametric distribution is associated with underlying hazard and survival function assumptions, which were carefully reviewed for their clinical validity.² A limitation of standard parametric distributions is that heterogeneity in the modeled population is not directly considered. Spline, cure, and landmark models have additional assumptions that allow more flexibility in modeling complex trends. A semi-Markov model approach obtains OS indirectly based on probabilities calculated from other states. Clinical assumptions, strengths, and limitations of all approaches used are summarized in Table 1.

Table 1 Comparison of Selected Extrapolation Methods

Analyses

In order to derive the life expectancy associated with the selected parametric, spline, cure, and landmark survival models, the mean area under the survival curves was estimated. Following the NICE DSU guidelines,¹ the parametric distributions considered were as follows: Exponential, Weibull, Gamma, Gompertz, log-logistic, log normal, and Generalised Gamma. For the spline approach, the following models were considered: spline odds, spline inverse normal, and spine hazard, with a maximum of three knots considered.

Mixture cure and non-mixture cure models were estimated by adding the covariates collected in the trial¹⁸ (age, gender, race, ethnic group, body mass index, smoking status, treatment with platinum-based chemotherapy, performance status, lactate dehydrogenase levels, presence of brain metastases, and presence of liver metastases) and the general population’s mortality to better estimate the cured fraction and long-term survival. Survival of the general population for the mixture cure model was available in country life tables (United Kingdom [UK] for the current analyses). CASPIAN was a multinational trial, and the general population’s mortality likely is different across countries in the trial. To reflect this, we included sensitivity analyses with general population survival estimates for countries in the CASPIAN trial with the worst (Ukraine) and the best (Japan) life expectancies; they did not significantly affect the results (not reported).

For the landmark analysis, we assessed tumor response at the 12-week landmark point, which coincided with one of the time points of response measurement and the end of the induction period in the CASPIAN trial (12 weeks).¹⁹ We followed the Response Evaluation Criteria In Solid Tumors (RECIST) criteria, where any complete response (CR) or partial response (PR) was considered to be a response, while stable disease (SD) or progressive disease (PD) were considered to be non-responders.²⁰

OS curves were derived for the following groups: durvalumab plus EP responders, EP responders, durvalumab plus EP non-responders, and EP non-responders. Before the response assessment, the risk of death of all patients (responders and non-responders) was estimated based on the observed OS for CASPIAN’s intention-to-treat (ITT) population. A weighted extrapolation for survival was performed based on each response group for both treatment arms at 12 weeks using the formula:

Visual inspection was conducted to ensure that the selected responder and non-responder curves did not cross each other.

For each method, the best fitting estimated survival curve was chosen based on the Akaike information criterion (AIC) and the Bayesian information criterion (BIC). The chosen models were also evaluated based on visual inspection (comparisons of the extrapolations and the Kaplan–Meier [KM] curves), and external information (literature). Results from all methods were compared to the 3-year data presented in the CASPIAN trial¹¹ to determine the approach whose results aligned with the observed data in the trial. Further validation of longer-term data was conducted using data from real-world evidence (RWE) estimates. Mean OS – over a 30-year time horizon – estimated by each method was also included. The selected lifetime horizon reflects the conventional approach used in cost-effectiveness analyses. Additionally, the selected time horizon was a sufficient length to account for methods that yield optimistic long-term results.

Sensitivity Analyses

Sensitivity analysis was performed for each of the approaches, including distributions for all the different standard parametric curves, spline models, and cure models, as well as complementary results for the landmark analysis using the 20-week timepoint. As the cure models consider the general population mortality in its prediction, different countries’ mortalities were tested. As noted above, the countries with the best and worst life expectancies (Japan and Ukraine, respectively, compared to the UK in the base case) from the CASPIAN trial were tested and included as a sensitivity analysis. The 12-week timepoint represented the completion of induction chemotherapy in CASPIAN, whilst providing enough follow-up data to derive survival beyond the landmark point, whereas, the 20-week timepoint will have fewer patients.

Results

Overall, all the proposed approaches had similar hazard shapes, as expected for immunotherapies, as shown in Figure 1. The hazard curves showed an increase until 12–18 months, followed by a decrease in the hazard function. For the cure models in particular, the hazard functions started to increase again after five years due to the inclusion of the general population’s mortality.

Figure 1 Hazard plots for proposed approaches. (A) Durvalumab plus etoposide and platinum; (B) etoposide and platinum. Abbreviations: Durva, Durvalumab; EP, Etoposide and platinum.

We selected the following distributions for each approach: log-logistic for both treatment arms in the standard parametric approach; spline-odds 3 knots for the splines approach; log-normal and Weibull for the responders and non-responders, respectively, for the durvalumab plus EP landmark model, while the EP arm had the generalized gamma and exponential distributions; Gompertz and Weibull for the durvalumab plus EP arm and EP arm, respectively, in the mixture cure model approach; and Gompertz and log-logistic for the durvalumab plus EP arm and EP arm, respectively, in the non-mixture cure model approach. Detailed justifications for the selection of the best distribution for OS extrapolation of the 2-year data for each model per treatment arm are provided in Table 2.

Table 2 Distribution Selection and Justification for All Approaches

Modelling Long-Term OS in the Durvalumab Plus EP Arm

The 3- and 5-year survival estimates obtained through each modeling approach are presented in Table 3 and visual fits for all approaches are presented in Figure 2, including the KM OS survival data from the CASPIAN trial.

Table 3 Overall Survival Extrapolation Estimates Compared to 3-Year Data from the CASPIAN Trial

Figure 2 Best distribution for each approach. (A) Durvalumab plus etoposide and platinum overall survival and (B) etoposide and platinum overall survival. Abbreviations: KM, Kaplan–Meier; OS, Overall Survival.

The 3- and 5-year OS estimates for the durvalumab plus EP arm for the standard parametric distribution (log-logistic) are 14.9% and 7.0%, respectively. For the Markov model in the same treatment arm, 3- and 5-year OS estimates were 15.9% and 9.6%, respectively.

Among the more flexible approaches, there were general trends for the durvalumab plus EP arm, although the models underestimated survival at three years, which was 17.6% in the CASPIAN trial.¹¹ For the landmark analysis at 12 weeks, the 3-year estimate was 10.6%, while the spline model (spline odds with 3-knots) 3-year estimate was 13.3%.

On the other hand, the mixture and non-mixture cure models returned 3-year estimates for the durvalumab plus EP arm of 19.1% and 18.5%, respectively. At 5-years, those models return estimates of 18.6% and 18.0% for durvalumab plus EP. In this case, the cure models reflected the real data from the CASPIAN trial better because these models account for the plateau in survival at the tail of the distribution.

Finally, the mean OS for durvalumab plus EP ranged between 1.41 years (landmark 12-week analysis) and 4.81 years (mixture cure model) (Table 4). Restricted mean overall survival at 2 and 3 years are provided in Supplementary Table 1.

Table 4 Mean Overall Survival Calculated Over a 30-Year Time Horizon for Durvalumab Plus Etoposide and Platinum and Etoposide and Platinum Alone

Modelling Long-Term OS in the EP Alone Arm

A comparison of the modeling approaches for the EP arm was also conducted (Table 3 and Figure 2). Of the common approaches, the 3- and 5-year survival estimates were 7.9% and 3.1% for the standard parametric distribution (log-logistic), respectively. Similarly, the Markov model showed survival estimates of 6.5% and 3.0% at 3- and 5-years, respectively.

When using the more flexible approaches, the 3-year survival estimate for the EP arm from the mixture model (6.1%) is the closest to the observed 3-year OS in the CASPIAN trial (5.8%), followed by the spline (6.7%) and Markov (6.5%) analyses.

Finally, the mean OS estimates over a 30-year time horizon for EP ranged between 1.01 (landmark 12-week analysis) and 2.00 (mixture cure model) – Table 4. The values did not vary as widely as the mean OS estimates of the durvalumab plus EP arm. However, those values vary depending on the distribution chosen. Table 4 presents the mean OS for different distributions tested in sensitivity analyses.

Sensitivity Analyses

Three-year and five-year survival rates for all the sensitivity analyses explored are presented in Supplementary Table 2. Those results show that the survival rates are highly sensitive to the distribution chosen in the cure model settings. Some distributions do not estimate a plateau at the end of the curve. For example, the mixture cure model with the exponential distribution estimates a 3.4% survival at 5 years for patients receiving durvalumab plus EP compared with 19.1% in the mixture cure base case (Gompertz). For traditional distributions, the sensitivity analyses do not show a high variability of the survival rates.

Mean OS for the sensitivity analyses is presented in Table 4. For traditional extrapolations, the estimated mean OS is not highly sensitive to the distribution chosen (from 1.34 years to 1.98 years for durvalumab plus EP and from 1.08 to 1.36 for EP alone). However, for more complex methods such as cure models, the distribution had an important impact on the results (mean OS from 1.78 years to 4.81 for durvalumab plus EP and from 1.27 to 2.00 for EP alone). In contrast, the baseline country used in the general population mortality parameter required in this model did not considerably impact the expected mean OS. The other methods tested also showed high variability in the mean OS depending on the assumption. For example, the landmark method estimated a lower mean OS for both durvalumab plus EP and EP alone when a landmark point of 20 weeks was considered (instead of the 12 weeks that was considered in the base case).

Discussion

This study aims to highlight the challenges that are present when modeling survival beyond clinical trial horizons. Given the influence of OS (a long-term outcome with uncertainty) on the estimation of quality-adjusted life years (QALYs) and incremental cost-effectiveness ratios (ICERs), it is important to not only ensure that OS predictions made by health economic models are as close as possible to future observed data but that the model selected is also clinically plausible. Furthermore, accurate OS predictions are beneficial for understanding an intervention’s impact on long-term clinical outcomes in general. In line with previous studies, our work highlights the importance of comparing multiple extrapolation methods and cross-checking the survival estimates obtained through these methods with external and observed data as well as clinical opinion.^17,21,22

A study was found, Khakwani et al,²³ which examined survival trends in patients with SCLC treated with chemotherapy in England, reported a survival rate of about 5% at 5 years. For the EP arm, our results from the spline, landmark, and cure models seemed to align with the estimates projected by the literature (1–3%), as our analyses estimated 5-year OS to be between 1.0% and 4.9% depending on the model used.

When assessing fit to the trial data, all models used provided a reasonable fit to the observed 3-year data from CASPIAN. Additionally, the sensitivity analyses for the different approaches found that most curves fitted adequately to the KM curves for OS. However, upon considering the clinical plausibility of the models, each model had its strength and limitations regarding the assumptions made.

The cure models (both mixture and non-mixture) had the best predictive ability when compared to the 3-year data. They also provide the largest accumulated life-year gains over 15 years compared to other approaches because estimates for the mean survival for durvalumab plus EP in this approach were consistently high. The relative better fit observed for these models compared to other approaches can be attributed to the fact that cure models account for patient heterogeneity within the clinical trial, though other methods such as the landmark model also consider heterogeneity in survival experience (responders and non-responders). Yet, not all the distributions estimated a long-term plateau within the cure framework, and long-term survival was overestimated in some cases despite the addition of general population mortality to the model. While the cure models demonstrated the best predictive ability at 3 years, when considering the plausibility of the model, the assumptions made need to be clinically valid, including clinical evidence to show that long-term survival is heterogeneous. An inaccurate estimation of the cure fraction may result in long-term survival predictions that are not clinically plausible and as ES-SCLC is a very aggressive disease, it is not possible to know if the sustained survival estimated by the cure models will be seen in real-world populations, especially without availability of more mature clinical trial data for comparison.

Following the cure models, the spline models and the Markov approach were the second-best models in terms of predictive ability compared to the 3-year data. Although the spline models can apply “knots” that can model more complex trends seen in the survival data, the added complexity makes clinical interpretation of the spline model challenging. Conversely, the Markov approach models PFS and OS such that they are explicitly related. However, the model necessitates the use of more mature data for the PFS (with a clear plateau), which directly impacts the OS (a plateau in the OS estimation), resulting in an estimate that is close to the observed 3-year data. In the context of clinical plausibility, the Markov approach may align more with clinical expectations despite difficulties with modelling post-progression survival data from clinical trials, such as the non-randomised nature of patients once they progress.

The landmark models were less accurate when estimating long-term survival as they did not anticipate the plateau observed in the CASPIAN trial at three years. The results obtained with this approach were driven by the extrapolation of OS estimates for non-responders because of the higher number of patients in this group. Consequently, the model provides more pessimistic survival results compared to the other approaches and the observed 3-year data from the trial, which is expected when working with short-term data. When considering clinical validity, assumptions made by the landmark model result in short-term health status driving long-term survival predictions as survival is dependent on patients’ response status at a landmark point. In this case, responders do not experience the event as early as the non-responders, and the timing of the assessment (12 weeks) may be too early to properly discriminate between prolonged response to treatment resulting in extended survival versus little to no response. This short time frame coupled with the limited event data for responders may have affected the ability of the landmark model to return more nuanced extrapolations of long-term survival. In addition, the model may not properly reflect the long-term benefits of treatments.

It is acknowledged that this study has several limitations. Firstly, our analyses did not consider the potential impact of subsequent treatments on OS. Nevertheless, given the aggressiveness of the disease and the limited efficacy of second-line treatments for patients who progress to first-line therapy, the data reported in the CASPIAN trial are unlikely to be strongly affected by potential subsequent treatments. In addition, though the cure models show promising predictive ability for short-term follow-up survival when compared to the observed 3-year data from CASPIAN and the other modelled approaches, conclusions are subject to further scrutiny once more mature data are available. Another limitation of this study is associated with the utilization of complex methods, which rely on underlying clinical assumptions that should be considered carefully. For instance, cure models presuppose the possibility of patients being cured from the disease. Additionally, the Markov approach implies an automatic correlation between the gain in PFS and OS in patients. Finally, results cannot be generalized to other indications.

Extrapolations are necessary to support HTA decision-making, especially where trial data with short follow-up period for novel therapies such as immunotherapies may not reflect the long-term benefit of treatment. Complex models may result in better extrapolations; however, beyond the statistical goodness-of-fit, the clinical plausibility of the predictions should be considered. Overall, our results demonstrate the importance of adopting a multi-faceted approach including clinical perspective when selecting the most appropriate and plausible model. In this study, cure models demonstrated the most accurate estimate of the CASPIAN 3-year OS data; however, there are other models that may be more appropriate and clinically more plausible longer term. Echoing similar remarks from the recent NICE DSU guidelines: health economic modelers should consider all relevant approaches and compare generated long-term survival extrapolations to the literature in order to draw conclusions.² In addition to providing researchers with insights into advanced techniques for survival extrapolation, this study also encourages consideration of the plausibility and limitations of employing each technique.

Conclusion

While it is tempting to think that increasing model complexity leads to a higher-quality model with better predictive power, its applicability in the context of modelling OS in an aggressive cancer like ES-SCLC remains uncertain. The CASPIAN study showed a flattening of the OS curve from the two-year to the three-year data-cut, and with the exception of the cure models, all models fitted to the two-year data underestimated three-year OS. There were also substantial differences in the long-term OS extrapolations of the models assessed, which will have an impact when assessing long-term costs and benefits of ES-SCLC treatments for economic evaluations. The more complex models such as cure models attempt to better represent the underlying method of action of immunotherapy treatment and yielded the best predictions of OS at three years; however, it is unclear whether longer term survival predictions estimated by these models will be more accurate than those obtained using simpler or standard models. Hence, it is vitally important to assess if there is the clinical basis and evidence to justify the use of more complex approaches to model survival, which will vary across different disease areas and stages, as well depending on the data that are available. Additionally, limitations around the generalizability of the model and the impact of subsequent treatments received after progression should be considered with each approach. Further research, including the use of more mature OS data for IO treatments in ES-SCLC, is still needed.