Value Insider Season 1 Episode 4: How are Costs Measured, and How are CEAs Constructed and Used? (CEA) [Podcast]

Host: Mr Michael Chambers, Independent expert (MC)

Guest: Prof. Maureen Rutten-van Mölken, Independent expert (MRM)

This podcast is brought to you by Boehringer Ingelheim.

Chapter 1: General Introduction [00.00]

MC:Welcome to the Value Insider podcast series. In this series, with the help of experts in the field, we will be exploring the fundamentals of assessing value in healthcare, especially when looking at the value of new healthcare interventions. My name is Mike Chambers, I am founder and director of MC Healthcare Evaluation, and I have spent the last twenty-five years working in health economics and health technology assessment for the pharmaceutical and medical diagnostics industries, and more recently as an independent advisor. I am also a member of the Technology Appraisal Committee at NICE: the National Institute for Health and Care Excellence in the UK. It is my great pleasure to be your moderating host for this season of the Value Insider podcast series.

Chapter 2: Episode Introduction and Welcome [00.58]

MC:Thank you for joining us today, be sure to subscribe and follow this series to ensure that you do not miss any of the informative podcasts in the series. Today, we will be discussing economic evaluations in healthcare and how value is considered in these evaluations. Our guest speaker is Professor Maureen Rutten-van Mölken, Professor of Economic Evaluation of Innovations for Health at the Erasmus School of Health Policy & Management, Erasmus University Rotterdam, in the Netherlands. She is head of the HTA department there. Maureen is also the Scientific Director of the Institute for Medical Technology Assessment, iMTA. Welcome, Maureen.

MRM:Thank you, Mike, it’s nice to talk to you again.

Chapter 3: Economic Evaluation [01.43]

MC:So let us start with this question: What is the purpose of an economic evaluation of a healthcare intervention?

MRM:Yes, as you know, in many countries there is a shortage of staff and a finite budget for the healthcare system. And especially during the corona pandemic, people became much more aware of the scarcity of resources. A Euro can only be spent once, and we cannot afford every possible innovation. So we have to make choices. And an economic evaluation is done to inform reimbursement decision makers on how to spend that budget in such a way that the health gains are maximized. So it helps them to set priorities.^1,2

MC:So an economic evaluation is a comparative assessment?

MRM:Yes, indeed. It’s always comparative. It compares both the cost and the benefits, or the health outcomes, of a new intervention or technology with one or more alternatives. And these alternatives usually include the current standard of care or the best available treatment option.

Chapter 4: Economic Evaluation versus Health Technology Assessment (HTA) [02.50]

MC:In this series, we have heard a lot about health technology assessment. What’s the difference between an economic evaluation and a health technology assessment?

MRM:Yes, health technology assessment is much broader.³ It does not only look at comparative effectiveness and cost-effectiveness as in an economic evaluation, but it also addresses organizational, social, legal and ethical aspects that may be associated with the introduction of a new technology. And reimbursement decision makers have to take these aspects into account as well in their decision-making.

MC:And do the requirements of all HTA organizations include an economic evaluation?

MRM:Well for most, it is a mandatory requirement in applications for public reimbursement of new drugs. And this started originally in Australia, Canada, United Kingdom, but quickly spread to the Netherlands as well as other European countries, and more recently to other regions in the world, such as the Asia Pacific. And for some countries and HTA organizations, the requirement for economic evaluation is more selective, for example, in Germany, where HTA focuses much more on the clinical benefit in head-to-head comparisons of treatment alternatives.⁴

Chapter 5: Types of Evaluations [04.14]

MC:So Maureen, are there many different types of economic evaluation? And which ones are used most frequently in health technology assessment?

MRM:Yes indeed, Mike, there are several types of economic evaluations, and they differ in how the benefits are expressed.¹ So, the most frequently used are cost-effectiveness analysis⁵ and cost-utility analysis, but there are additional ones as well. Cost-benefit analysis, in which the benefits are expressed in monetary terms. Cost-minimization analysis, in which we focus on a cost comparison only, assuming that the benefits are the same, in the intervention as well as the comparator group. And we have cost-consequence analysis, in which we carefully describe both costs and several types of consequences. But like I said, the most frequently used ones are cost-effectiveness analysis and cost-utility analysis. Well, the difference is that in a cost-effectiveness analysis the benefits are measured in a natural unit, like for example life years gains or events avoided, or something like the proportion of patients with a certain improvement on a disease-specific quality-of- life questionnaire. So we calculate for example the costs per event avoided, or the costs per life year gained. And in a cost-utility analysis, the benefits are expressed in quality-adjusted life years. And as you know, QALYs combine life years with the quality of life during these years. So in a cost-utility analysis, we calculate the costs per QALY … gained. The advantage of using such a generic health outcome as a QALY, is that it enables … the comparison of the cost-effectiveness of many different types of interventions across different disease areas. And that is not possible in a cost-effectiveness analysis, when a disease-specific outcome is used in that analysis. What is a bit confusing to the lay audience, is that the term cost-effectiveness analysis is sometimes used a bit more loosely to refer to all types of economic evaluations.

MC:So in particular we may find cost-utility analyses, where QALYs are the measure of outcome, referred to in the literature in certain places, as cost-effectiveness analysis, which is used a broader term.

MRM:Indeed.

MC:And those who are interested to find out more about QALYs can listen to podcast 2 of this series, where Professor Nancy Devlin went into detail explaining how QALYs are derived, and what they are.

Chapter 6: Models and Their Robustness [07.05]

MC:Many cost-effectiveness analyses are based on economic models. What are these things called economic models?

MRM:[Laughter] Good question. A model is essentially a set of mathematical equations, that simulate the disease progression and the care pathway of patients in the intervention group as well as in the comparator groups, and link then costs and benefits to these pathways. They either simulate groups of patients, we call that then cohort models, or individual patients, so-called micro-simulation models. And these models are filled with many different types of data, for example, on patient- and disease characteristics, on epidemiology, on costs, on health outcomes, and treatment effect of course. So lots of different data are brought together and synthesized through a health economic model. They are usually programmed in specific software or simply in Excel.

MC:And why do we need these models?

MRM:Yeah, for several reasons. For example, because we want to combine data from multiple different sources and synthesize them in one model. Or we want to extrapolate the effect that was observed in a clinical trial with a limited time horizon to a longer time horizon. Or we want to include more treatment options that were not compared in a clinical trial head-to-head. Or to include more countries that were not even participating in a clinical trial.

MC:Models are based on a lot of inputs. Where do these inputs come from?

MRM:Trials are important sources of data, especially on the efficacy of the new interventions that we want to model. But there are also some limitations as to what can be collected in clinical trials. And with respect to the generalizability of clinical trial results for the real world. So relevant input data often also come from observational studies, like cohort studies, who follow a cohort of patients over a certain period of time.

MC:I think we have seen models predicting some rather unusual things in the era of Covid. And those predictions have been sometimes not very consistent with what actually happened. How do we trust these model outcomes?

MRM:Yeah, we as modelers, put a lot of effort into the validation of the models. And for example, we need to check the face validity of the models, and what we do is then talk to clinical experts, to see whether the models have captured all important aspects of the disease, and the disease progression, and the treatment pathways. We also check internal validity, which means that we check if we get the expected results when we change certain input parameters, or that we can replicate the results of a clinical trial if you set the model inputs to mirror the trial data. And, of course very important is to check the external validity of a model, to see whether it predicts the health outcomes and the costs consistent with other models or with what was observed in real world observational data. And of course, the latter is a challenge, because you need good longitudinal cohort data to represent the long-term progression of the disease.¹

MC:But nevertheless, these validation processes, face validity, or checking against external sources of data, are very important elements of building models, that are credible, and can be used for decision-making.

MRM:Absolutely, yeah. We need to demonstrate that results are robust.

Chapter 7: Costs [11.07]

MC:One of the ingredients, or inputs into cost-effectiveness analyses, and economic models to support those analyses, is costs. Which costs should be considered for inclusion in these analyses and how are they measured?

MRM:Yeah, there are many different types of costs, but we generally make a distinction in four different categories. So the most important, are obviously the direct healthcare costs. So direct costs are the costs directly associated with the disease and the intervention. So for example the cost of the intervention or the technology, or cost of diagnosis or monitoring. Also relevant are the direct costs of managing events in the future, such as hospital admissions or subsequent lines of treatment if a disease progresses. So especially when such costs may be delayed or prevented by the intervention, then it’s important to include these as well. So that’s the first category, direct healthcare costs. There are also direct costs outside of the healthcare sector, and this mostly refers to the cost of informal caregiving. A third category of costs, are the indirect costs outside of the healthcare sector, which usually refers to the productivity cost, which are the costs of reduced ability to work of patients and maybe also caregivers. And the last category of costs are the indirect costs within the healthcare sector, and that mostly refers to the costs of healthcare not related to the disease in which we are interested, during the life years that are gained due to the new technology or the new intervention. And which of these cost categories to include in a cost-effectiveness analysis really depends on the perspective from which the analysis is done. When the analysis is done from a healthcare perspective, we usually include only the direct costs of the healthcare sector. But when a adopt broader societal perspective, we can include the other categories of costs as well. Mostly the productivity costs, and cost of informal caregiving.^1,6 And the data to estimate these costs are sources of cost data are very often a combination of many different sources. They can come from clinical trials, or from observational studies. And generally, we measure then the units of resource use, such as the number of GP visits, or emergency room visits, or bed days, or number of diagnostic and type of diagnostic tests, etcetera. And then we combine that with the price or the unit cost of these resources. And many countries actually have standard schedules for these unit costs.

Chapter 8: Perspectives [14.11]

MC:I think one of the problems of trials is that the treatment patients receive may not be representative of what they would receive in routine practice, is that right?

MRM:Yeah, that’s indeed right, and that’s why we often bring in claims data, to better represent the real world.

MC:And in episode 3 of this series, we heard Sean Sullivan speaking quite a lot about perspective, particularly from the US. Just thinking more about Europe, do all health technology assessment bodies use the same perspective when they are thinking about costs?

MRM:No, some countries, like the UK, primarily adopt a healthcare perspective. Other countries like for example the Netherlands and Sweden adopt the broader societal perspective and request for example the inclusion of productivity costs. And the Dutch guidelines even recommend the inclusion of all sources, or resources, no matter who bears the costs. So we even include healthcare costs during the life years gained, irrespective of whether they are related to the disease or not.⁷

MC:And, in episode 5 to come in this series, we will hear from Professor Lou Garrison about the societal perspective. And perhaps cost elements that may not be included currently in health technology assessments.

Chapter 9: Incremental Cost-Effectiveness Ratio (ICER) [15.36]

MC:The result of a cost-effectiveness analysis, is an “Incremental Cost Effectiveness Ratio” or ICER for short. Can you explain what this means?

MRM:ICER is incremental cost-effectiveness ratio, and that is a ratio that represents the additional costs per unit of health outcome gained, like the costs per QALY gained. And it’s simply calculated as the difference in costs between the new intervention and the comparator, divided by the difference in QALYs.⁸ And we calculate an ICER when the new intervention is more costly than the alternative but also more effective, so it improves the health outcome. And then the lower the ICER, the better the cost-effectiveness of the intervention. In cases where the new intervention is both more effective and less costly than the alternative, we do not calculate an ICER. We simply say that the new intervention dominates the alternative, or comparator. The other way around, if the new intervention is less effective and more costly, then we say it is dominated by the alternative.

Chapter 10: ICER Thresholds [16.51]

MC:How do we judge whether the result of a cost-effectiveness analysis, this ICER value that drops out of our model, is favorable or not?

MRM:We do this by comparing the ICER with an external pre-determined threshold value, to decide whether the new intervention is efficient or not. And this threshold is a value or a range of values that often reflects the maximum willingness-to-pay for an additional quality-adjusted life year. And then we say: if the ICER is below the threshold value, the new intervention is more efficient in creating health than interventions that could be displaced if the new intervention was introduced into the system. Assuming that our healthcare spending is constrained by a limited healthcare budget. And if the ICER is above the threshold, the new intervention is less efficient in creating health or we can say it’s not cost-effective.

MC:So this threshold value is not the result of a cost-effectiveness analysis.

MRM:No, indeed, it is not. It is independent from the cost-effectiveness analysis and set by public decision-makers. And the threshold varies between healthcare systems.⁸

MC:Where does this threshold value come from?

MRM:It can come from willingness-to-pay studies, but it can also come from investigating what the value of a QALY in the current system as it currently is.

MC:And can you give some examples of these “willingness to pay” thresholds?

MRM:In England, NICE⁹ uses a threshold between twenty and thirty-thousand pounds per QALY, rising to fifty-thousand and above for some life-threatening or rare diseases. In the Netherlands,¹⁰ the maximum “willingness to pay” threshold is eighty-thousand per QALYs gained. A cost per QALY gained, so the result of the cost-effectiveness analysis, above these thresholds would not be considered cost-effective.

Chapter 11: ICER Uncertainty [19.07]

MC:How sure can we be in the result of a cost-effectiveness analysis? Is there uncertainty?

MRM:Indeed, there is uncertainty around the ICER.¹¹

MC:So how do you assess this uncertainty in economic models?

MRM:Yeah, well as modelers we do sensitivity analyses and scenario analyses to get a better feeling and understanding of uncertainty. So in a sensitivity analysis, we adjust the values of the key input parameters and then assess what the impact of that adjustment is on the ICER. And we can vary the input parameters one-by-one, or all at the same time. And when we really want to assess the entire joint uncertainty in the parameters in a model, then we do what we call a probabilistic sensitivity analysis. And that means that we sample parameter values from their distributions, and then re-run the model for every new set of parameter values.

MC:And how about scenario analyses, what are they?

MRM:Well, in scenario analysis, we do not vary the key input parameters, but we investigate what the impact of changing assumptions and model design choices are, on the ICER, to assess the structural uncertainty. But in these sensitivity and scenario analyses, the most important thing is to assess whether our overall conclusion on whether the intervention is cost-effective or not, changes or not, if we change the inputs, or the assumptions. So how confident are we that the overall conclusion about cost-effectiveness holds? And if that conclusion is rather uncertain, then we may invest in further data collection, further research, to reduce that uncertainty that is associated with relevant input parameters or assumptions.

MC:A concern expressed about the use of economic evaluations in health technology assessment is that they focus on maximizing health outcomes, but can they take inequalities into account?

MRM:Yes, you are right, most cost-effectiveness analyses either ignore health inequality impacts or just describe potential impacts, without really assessing them.¹² And we know of course that policy-makers and society as a whole does care about the distributions of the health gains, and not only about maximizing health gains. For example, there is empirical evidence that people value health gains more when patients are more severely ill, than in patients who are less severely ill. We can support that for example by weighing QALYs differently, or by adopting more flexible threshold values. We see in the Netherlands, where we use three threshold values depending on the severity of the disease. But also in other countries, where higher ICERs are allowed for, for example, for end-of-life treatments or for very rare diseases. And that is based on the public’s preference for funding interventions in these areas.

Chapter 12: New Challenges [22.39]

MC:And a final question, Maureen. How is cost-effectiveness analysis, and economic modeling adapting to the new challenges faced by healthcare systems, particularly the challenges of personalized or targeted medicines?

MRM:Personalized medicine stratifies the patients in smaller subgroups. And this may add uncertainty to the evidence we have for the separate subgroups because we may not have clinical trial evidence for all the subgroups that we have to include in our model. In the end, they may become more complex, and include more elements of uncertainty. But that reflects real-world uncertainty as well.

MC:So there’s quite a lot of work ahead for those working on economic models, and cost-effectiveness analyses, to adapt these to the new situations and the new interventions that we are looking at, for potential reimbursement in the healthcare system.

MRM:Agreed, yes.

Chapter 13: Conclusion [23.44]

MC:Well, thank you very much for taking us through this grand tour of economic evaluation and economic modeling. And thank you everybody, for joining us today on this podcast. I would like to thank my guest, Professor Maureen Rutten-van Mölken, for a very insightful conversation and overview of economic evaluations, particularly as they are used in health technology assessment.

MRM:Thank you Mike, it was a pleasure to be there.

MC:I hope you can join us for the fifth podcast in this series where we will focus on the societal perspective in value assessment with Professor Lou Garrison, at the University of Washington in the United States. If you have enjoyed this podcast, please subscribe to our series, and thank you for listening.