Oncology drug health technology assessment recommendations: Canadian versus UK experiences

Background

Health spending has continuously increased in Canada over the past few decades. Canada is among the top five health spenders in the Organisation for Economic Co-operation and Development (OECD) countries, after the United States, the Netherlands, France, and Germany.¹ In the last 3 years, the rate of growth has been below inflation and population growth, reflecting government efforts to balance budget. Health spending represents 11.2% of the Canada’s gross domestic product, a share that has fallen from a nonsustainable peak of 11.6% in 2009. Weak prospects for economic growth combined with fiscal deficits continue to create extreme pressure to contain health spending. As the percentage of the population aged 80 years and older increases, decision-makers will continue to be challenged to innovate and reform how health care is provided, including ensuring affordability of drugs.¹

Canada has been a leader in the use of health technology assessment (HTA) to assist in the allocation of resources within a drug budget. Since its first use in the province of Ontario in 1991, HTA has played an increasing role in informing public payers’ coverage and policy decisions in Canada.² Over the years, many other jurisdictions around the world have adopted HTA frameworks to inform drug funding decision-making by considering both clinical and cost-effectiveness evidence.

Conventional HTA criteria pose particular challenges for reimbursement decision-making with respect to the treatment of advanced cancers.³ These drugs often have only surrogate outcomes, are life-extending rather than curative, and are expensive, thus increasing the cost-effectiveness ratio. Further, there are equity issues around the weight that should be given to the extension of life in patients with terminal cancer disease relative to other patients’ needs. The associated emotional and political context makes these decisions more likely to be difficult and controversial. To manage this challenge, at least one HTA agency – the National Institute for Health and Care Excellence (NICE) in the United Kingdom – has developed end-of-life–specific HTA criteria.⁴

In Canada, two HTA agencies assess cancer drugs. The Institut National d’Excellence en Santé et Services Sociaux (INESSS) in the province of Québec assesses both oncology and nononcology drugs, although a specific subcommittee focuses on oncology drugs. The pan-Canadian Oncology Drug Review (pCODR) assesses cancer drugs while the Common Drug Review (CDR) is responsible for nononcology drugs in the rest of the country. NICE, which operates in England and Wales, has been often used as a reference for Canadian decision-makers, at the levels of both process and results.^5–7

The objective of this research was to review and compare the oncology-specific reimbursement recommendations that have been made in Canada alongside those made in the United Kingdom by NICE with respect to their recommendations records and the influence of clinical and cost-effectiveness evidence on the recommendations.

Methods

Setting

Recommendations were reviewed from three agencies: the provincial (INESSS) and national (pCODR) oncology HTA agencies in Canada, alongside an external reference (NICE) that is considered the closest parallel agency internationally.

Canada’s national oncology review panel has evolved over time. Prior to 2007, the CDR was responsible for making funding recommendations regarding oral and subcutaneous oncology drugs, while provincial authorities made the funding decisions. Intravenous oncology drugs were both reviewed and funded by individual or regional hospital committees or provincial cancer boards, outside of the CDR process. Starting in March 2007, the interim Joint Oncology Drug Review (JODR) took responsibility for the national review process, inclusive of both oral and intravenous oncology drugs. JODR recommendations were passed to the provinces for funding decisions. pCODR replaced JODR in July 2011, again making funding recommendations for both oral and intravenous drugs. Unlike JODR, pCODR maintains a website with public posting of all reviewed products. Therefore, the national process historically consisted of a sequence of three agencies (CDR to JODR to pCODR).

INESSS was established by legislation in 2011, succeeding the Conseil du Médicament and the Agence d’évaluation des Technologies et Modes d’Interventions en santé in the province of Québec. The mission of INESSS is to prepare drug funding recommendations for the Minister of Health. Previous to INESSS, the Conseil performed this function; the legislation that established INESSS formalized this process by law within the institution. Recommendations from both agencies have been stored in the same database.

In order to manage the scope of the analysis, and to facilitate the ability to observe and draw inferences among common issues, the analysis focused on disease sites that had the most activity with respect to HTA assessments. This comprised five disease sites: the most prevalent cancers (lung, breast, and colorectal) as well as cancers of the kidney and blood/plasma cells (for which many new interventions have been introduced in recent years). Furthermore, the analysis was limited to drugs for advanced/metastatic stages, or patients with poor prognosis, in order to focus on the particular challenge with noncurative, end-of-life drugs.

The observation period was January 1, 2002 to June 1, 2013. All drugs for which final recommendations were issued, and which met the criteria above, were included in the analysis.

Data abstraction

The agency websites were used as the primary data source for the analysis. Consistent with the HTA agency’s individual commitments to transparency, only data that were formally disclosed in the public domain and posted on the agencies’ websites were included in the analysis. (Note: there was one exception – the review of bevacizumab for metastatic colorectal cancer in Québec was never published on the INESSS website but was included in the analysis because the information was in the public domain). Specifically, the documents retrieved for the analysis were:

• INESSS: Avis au Ministre⁸
• Rest of Canada: the Canadian Expert Drug Advisory Committee Final Recommendation from the Common Drug Review (2002–2007),⁹ the Executive Officer Decision from the interim interprovincial Joint Oncology Drug Review (2007–2011)^10,11 and the pCODR Expert Review Committee Final Recommendation (2011–2013)¹²
• NICE: information from the Appraisal Committee only (not information from the Manufacturer’s Submission).¹³

Some drugs were reviewed for multiple indications. If so, then all distinct indications were included in the analysis. Some drugs were reviewed multiple times for the same indication. If so, then only the most recent recommendation was included in the analysis. The following data were extracted for each recommendation:

• efficacy: progression-free survival (PFS) gain
• efficacy: overall survival (OS) gain
• incremental cost-effectiveness ratio (ICER)
• recommendation
• reason for negative recommendation (if applicable)
• arrangements between payers and manufacturers (if applicable: risk-sharing, price negotiation, access scheme). Note: NICE uses the terminology “pricing scheme” to denote any type of manufacturer agreement, including product discounts and rebates.

It is recognized that clinical benefit goes well beyond PFS or OS, including quality of life and safety endpoints, among others. While these are extremely relevant endpoints in the decision-making context of end-of-life cancer treatments, they were not included in this analysis. Clinical benefit was narrowly defined using efficacy (PFS/OS) under the prior assumptions that 1) efficacy was the paramount aspect of clinical benefit and 2) the use of PFS instead of OS has been controversial among Canadian HTA experts.

It is worth noting that all three agencies required cost-effectiveness evidence as a submission requirement, but there was an important difference for INESSS. At INESSS, the five factors defined by legislation that must be considered when preparing drug funding recommendations for the Minister of Health are: therapeutic value, reasonableness of price, cost effectiveness, the advisability of entering the drug on the list, and its impact on the health and social services system. The drug’s therapeutic value is a prerequisite to consideration of the other four criteria. INESSS does not conduct the economic assessment when the clinical effectiveness is not satisfactorily established. In the rest of Canada and at NICE, cost-effectiveness is always assessed, regardless of clinical benefit. Nevertheless, since clinical uncertainty leads directly to economic uncertainty, it is implicit at other agencies that in the absence of clinical benefit, cost-effectiveness is unlikely to be favorable.

Data analysis

Every unique recommendation was considered a separate observation. Once all recommendations were identified and data were abstracted, descriptive statistics were performed. Factors related to recommendations were counted both within each agency and compared across agencies.

Results

A total of 90 recommendations were included in the analysis: 34 from INESSS, 27 from CDR/JODR/pCODR, and 29 from NICE. The distribution by disease site and by HTA agency is portrayed in Table 1. INESSS conducted the highest number of reviews; INESSS has provided funding recommendations continuously throughout the time period, and it was charged with reviewing all drugs subject to funding by the Régime Général d’Assurance Médicament. For the rest of Canada, there were gaps for two reasons. There was an absence of intravenous drug reviews before 2007 (during the CDR era, when only oral drugs were reviewed). Further, during the JODR era, reviews were conducted but not necessarily posted (as there was no formal JODR website). At NICE, there was no formal requirement to review all newly licensed drugs/indications. In four cases, reviews had been initiated but were terminated before completion, because the manufacturer did not submit evidence.

Table 1 Eligible drugs and indications Notes: In attempting to conduct this research, there were some important barriers with respect to comprehensiveness of drug assessment. NICE is not required to review all drugs, and several reviews were terminated for lack of manufacturer-provided evidence. Prior to the establishment of a common national process exclusive to cancer drugs (March 2007), centralized Canadian reviews only occurred for oral agents. During the JODR era, all cancer drugs were reviewed consistently, but recommendations were not all posted; the lack of publication of recommendations posed another challenge. pCODR intentionally corrected these issues, with a mandate to be inclusive of all new cancer drugs marketed in Canada (irrespective of route of administration) and to establish publicity and transparency as a core value. Abbreviations: ALK, anaplastic lymphoma kinase; ASCT, autologous stem cell transplant; CDR, Common Drug Review; CML, chronic myeloid leukemia; EGFR, epidermal growth factor receptor; HR, hormone receptor; HER2, human epidermal growth factor receptor 2; INESSS, Institut National d’Excellence en Santé et Services Sociaux; JODR, Joint Oncology Drug Review; KRAS, Kirsten rat sarcoma viral oncogene homolog; mBC, metastatic breast cancer; mCRC, metastatic colorectal cancer; MM, multiple myeloma; mNSCLC, metastatic non-small-cell lung cancer; mRCC, metastatic renal cell cancer; NHL, non-Hodgkins’s lymphoma; NICE, National Institute for Health and Care Excellence; NR, not reviewed, NP, reviewed but not posted; pCODR, pan-Canadian Oncology Drug Review; UR, under review.

The number of positive versus negative recommendations varied widely between agencies. Although the national process reviewed 27 drugs, recommendations were posted for only 20 drugs, of which 19 (95%) received positive recommendations (Figure 1). The missing recommendations were all from the JODR era. During the most recent pCODR era, six of the seven recommendations were positive (86%). INESSS had the next highest number of positive recommendations, at 77% (Figure 2). NICE had the lowest percentage of positive recommendations at 48% (Figure 3).

Figure 1 Types of recommendations – CDR/JODR/pCODR (n=20). Note: Seven recommendations from JODR were not posted and could not be included in the analysis. Abbreviations: CDR, Common Drug Review; JODR, Joint Oncology Drug Review; pCODR, pan-Canadian Oncology Drug Review.

Figure 2 Types of recommendations – INESSS (n=34). Abbreviation: INESSS, Institut National d’Excellence en Santé et Services Sociaux.

Figure 3 Types of recommendations – NICE (n=29). Abbreviation: NICE, National Institute for Health and Care Excellence.

Additional details are displayed in Figures 1–3. Canada’s national process (CDR/JODR/pCODR) yielded the most positive recommendations, but these were largely circumscribed by a request for a pricing arrangement (45%) or restricted criteria (20%). The pricing arrangements almost always were intended to address issues with cost-effectiveness. For example, during the JODR era, typically the recommendation would be negative on the basis of an economic value gap, while the funding decision would be positive, on the basis of a negotiated agreement between the province of Ontario and the manufacturer. pCODR created a special category of recommendations that would specify that funding is conditional to a negotiated agreement to improve “cost-effectiveness” – in fact, this recommendation was made for most positive recommendations at pCODR. With these options available, there were no negative recommendations based on cost-effectiveness. In contrast, at NICE, while pricing schemes were sometimes used to address cost-effectiveness (31%), a rejection based on cost-effectiveness was more common (48%). At INESSS, the majority of recommendations had no restrictions or arrangements (59%), with the remaining positive recommendations under a risk-sharing agreement (18%).

Table 2 shows the percentage of positive recommendations by disease site. There was no observable consistent pattern to the variability observed between disease sites.

Table 2 Percent negative recommendations by disease site Abbreviations: CRC, colorectal cancer; INESSS, Institut National d’Excellence en Santé et Services Sociaux; NICE, National Institute for Health and Care Excellence; pCODR, pan-Canadian Oncology Drug Review.

In the metastatic treatment setting, either PFS or OS could be used as a primary or secondary outcome in drug trials. The relationship between OS gain and a positive recommendation was explored (Table 3). Interestingly, the majority of positive recommendations across all agencies did not include evidence for OS. Moreover, several drugs with OS benefit received negative recommendations. The positive recommendations were largely based on surrogate endpoints, specifically PFS in most cases. HTA agencies recognized that crossover limited the opportunity to obtain OS. Overall, across all agencies, only a small minority of drugs were rejected for lack of clinical evidence: 13% at INESSS (four drugs), 5% by CDR/JODR/pCODR (one drug) and 3% at NICE (one drug).

Table 3 Recommendations with positive overall survival benefit Note: *The seven JODR recommendations that were not posted were excluded. Abbreviations: CDR, Common Drug Review; INESSS, Institut National d’Excellence en Santé et Services Sociaux; JODR, Joint Oncology Drug Review; NICE, National Institute for Health and Care Excellence; pCODR, pan-Canadian Oncology Drug Review.

Interagency agreement was assessed by comparing the recommendations for drugs/indications combinations that were common to all three agencies (Table 4). Of the 14 pairings evaluated by the three bodies, only six (43%) had convergent (positive) recommendations, while divergence was observed in eight cases. Divergence was found for: pemetrexed, sorafenib, temsirolimus, axitinib, eribulin, cetuximab, panitumumab, and dasatinib. Convergence was more common when NICE was excluded. Of the 18 drug/indication combinations reviewed by both INESSS and the national process (details not reported), agreement on a positive recommendation was made for 16 pairs (89%); the remaining two drug/indication combinations were rejected by INESSS for cost-effectiveness issues while they were accepted by CDR/JODR/pCODR with no evidence of pricing negotiation.

Table 4 Interagency agreement on recommendations Notes: *Based on Cancer Care Ontario guidelines; funded guidelines assumed even if no HTA review. Abbreviations: CDR, Common Drug Review; CML, chronic myeloid leukemia; EGFR, epidermal growth factor receptor; HTA, health technology assessment; INESSS, Institut National d’Excellence en Santé et Services Sociaux; JODR, Joint Oncology Drug Review; KRAS, Kirsten rat sarcoma viral oncogene homolog; mBC, metastatic breast cancer; mCRC, metastatic colorectal cancer; mNSCLC, metastatic non-small-cell lung cancer; mRCC, metastatic renal cell cancer; NICE, National Institute for Health and Care Excellence; pCODR, pan-Canadian Oncology Drug Review.

Discussion

Funding recommendations and decisions for pharmaceuticals have been a focus for attention and even dissent over the past few decades, but advanced cancer treatment decisions have been the most contentious and troubling, with frequent media attention to funding decisions. The three jurisdictions under examination in this analysis have each evolved a process or framework for the evaluation of cancer drugs that differs from other products. In most of Canada, there is a separate HTA assessment process for the evaluation of cancer versus noncancer drugs; in Québec, the process is the same, but there is a separate advisory committee that focuses on cancer drugs; and in the United Kingdom, there is a separate set of criteria and cost-effectiveness threshold for end-of-life (cancer) drugs.

When initiating this research, one of the questions was the acceptability of surrogate outcomes (specifically PFS) in the metastatic/advanced cancer setting. A working hypothesis was that despite approval on the basis of surrogate endpoints such as PFS, those endpoints would be seldom recognized by HTA bodies. When examining the different jurisdictions, we found that this hypothesis was rapidly discarded, as very few drugs were rejected for clinical reasons. This occurred even though the majority of cancer drugs did not have evidence of OS benefit. There are many possible reasons for the acceptability of PFS as a surrogate outcome. HTA reviewers could be willing to accept the unavoidable clinical uncertainty associated with surrogate endpoints given the limitations of clinical trial design. In other cases, clinical benefit may have been observed for nonefficacy reasons (such as a safety advantage of one treatment over the standard of care, which could be of much more value than an impact on OS to patients with advanced cancer). The current incorporation of patient input into HTA reviews suggests that the patient experience of safety and quality-of-life issues may predominate in some advanced cancers.

Given that most drugs were considered to provide acceptable clinical benefit, cost-effectiveness was left as the most-common criterion for a negative recommendation. This was not expected, as a recent Canadian paper reported that quality flaws in economic analyses in oncology drug reimbursement submissions rendered them uninformative.¹⁴ Also, previous research on CDR and NICE recommendations (not limited to oncology drugs) has found that clinical concerns were the most compelling aspect of a submission.⁷ Once the clinical hurdle had been passed, the agencies behaved differently. In Canada, cost-effectiveness did not influence the recommendation once clinical factors had been taken into account; rather, price remained as one of four statistically significant predictors of a recommendation.¹⁵ In the United Kingdom, Devlin et al¹⁶ found that cost-effectiveness explained 85% of NICE decisions. INESSS recommendations have not been studied in previously published research.

The analysis demonstrated that cost-effectiveness had a larger-than-expected role in oncology recommendations. This led to a close examination of the optional arrangements between payers and manufacturers to address an unfavorable cost-effectiveness ratio.

In Québec, 12% of drugs were rejected for lack of cost-effectiveness and 18% were recommended with some risk-sharing arrangements. Risk-sharing agreements were introduced in November 2011 to address concerns about the rejection of promising cancer drugs.¹⁷ For those “drugs deemed promising,” a risk-sharing agreement recommendation could substitute a “do not list” recommendation from INESSS when the cost-effectiveness criterion was not met or in case of clinical uncertainty. As of October 28, 2013, 17 cancer drugs were deemed promising by INESSS, with recommendation of coverage with evidence development in one case and financial arrangements in 16 other cases.¹⁸ Of those 17 drugs, six were part of the current study. None of the risk-sharing agreements recommended by INESSS were accepted by the Minister of Health, who instead decided to fund all drugs at their list price under exception status, without further patient access delays. Legal and equity issues versus the private sector were brought forward to justify the status quo by the Minister. There was a clear willingness to fund drugs once they met the clinical value criterion, despite concerns that may have existed regarding cost-effectiveness. The political environment was and remains strongly supportive of transparent and uniform pricing across all payers – mitigating against pricing schemes. Although substantial savings could have been achieved through the arrangements proposed by INESSS, they were still not introduced by the government. Although it has a leading position in providing drug accessibility, Québec is the only province in Canada that does not leverage risk-sharing agreements to provide drug access to cancer patients at the end of life.

At Canada’s national HTA process, the postrecommendation jurisdictional funding decision could include confidential pricing arrangements. The majority of drugs were recommended within the framework of a proposed pricing arrangement (45%) or restricted criteria (20%), and none were rejected for lack of cost-effectiveness – presumably because payers have the opportunity to negotiate pricing arrangements to improve the cost-effectiveness ratio. Since July 2010, Canadian jurisdictions have been cooperating on price negotiations through the creation of the pan-Canadian Pricing Alliance (pCPA).¹⁹ A “do not list” recommendation from pCODR is considered a “do not negotiate” listing agreement by the pCPA process. Patient access is highly unlikely to occur in this context unless under an exceptional case-by-case basis, as adjudicated by the various jurisdictions. Of the 21 notices of implementation (as of October 29, 2013) issued by pCODR since its inception in July 2011, no drug has been rejected on the basis of lack of cost-effectiveness; all drugs that did not meet the cost-effectiveness criteria were recommended with funding conditional on improved cost-effectiveness. In the absence of a formal product listing agreement process, these drugs could be amenable to pCPA for an interprovincial negotiation or to any other informal negotiation structure that could represent a unique province or a group of provinces. It is too early to provide statistics on the number of agreements that were successfully negotiated and that made the drug actually accessible to patients in the various provinces. With the inception of pCODR, all jurisdictions intend to align with pCODR recommendations, although the provinces retain the authority to make independent decisions.

For most of the CDR’s tenure to review oncology medicines, price negotiations were not available or considered within recommendations, while both the JODR and pCODR eras featured the opportunity for product-listing agreements. There were important differences that could be responsible for changes over time seen within the body of national recommendations – differences in mandate, processes, legislative framework, expert panel composition, etc. However, it was the introduction of price negotiations that truly defined the different nature of JODR/pCODR versus CDR recommendations, and that allowed for such a high percentage of positive recommendations conditional on improvement of the cost-effectiveness ratio through negotiation with the manufacturers.

In contrast, at NICE, almost half of drugs were rejected on the grounds of cost-effectiveness, despite the option to negotiate access schemes within the HTA process. Whereas no explicit or implicit cost-effectiveness thresholds have been introduced in Canada, NICE has a stated cost-effectiveness threshold range that drugs are expected to meet. If a patient access scheme could not be devised to improve the cost-effectiveness ratio, then there was an unwillingness to make a positive recommendation. In January 2009, NICE introduced supplementary advice to accommodate a higher cost-effectiveness threshold for oncology drugs that may be life-extending for patients with short life expectancy. The additional advice applies when the cost-effectiveness ratio exceeds the NICE upper end of £30,000 per quality-adjusted life-year (QALY) (£35,000 to £45,000), provided that the treatment extends life by at least 3 months compared with available alternatives. The rationale for this adaptation was the assumption that society valued QALYs obtained by patients at the end of life more than QALYs obtained by other patients. NICE accommodated a higher threshold for end-of-life drugs, but this higher threshold must still be met for a positive recommendation.⁴

More recent rejections by NICE of various oncology products highlighted the problems facing the launch of new cancer medicines in the United Kingdom. These negative decisions were noted in many other countries around the world, promoting the perception that UK patients were being denied access to oncology medicines. What is less well-known internationally is that the UK National Health Service (NHS) offered an alternative route to fund cancer medicines: the national Cancer Drugs Fund (CDF). In contrast to their well-publicized rejections by NICE, many drugs were fully funded for use via the CDF, introduced in England in April 2011. This fund was established in order to enable patients to access the cancer drugs their doctors think will help them, even if funding is not recommended by NICE. Its establishment was confirmed by the UK government’s coalition agreement in 2010.²¹ The objectives of the CDF, as set out by the UK government are:

• to provide maximum support to NHS patients;
• to put clinicians and cancer specialists at the heart of decision-making, consistent with the government’s wider policy of empowering health professionals and enabling them to use their professional judgment about what is right for patients; and
• to act as an effective bridge to the government’s aim of introducing a value-based pricing system for branded drugs in 2014.²²

The funding system was initially set up to run for a period of 3 years, from April 2011 to March 2014, but the government has announced an extension for another 2 years to make the fund available until March 2016. The CDF is allocated £200 million per year.

The scope of the funding includes: 1) drug/indication combinations appraised by NICE and not recommended on the basis of cost effectiveness, or where the recommendations restrict access to a smaller group of patients than the specifications set out in the marketing authorization; and 2) drug/indication combinations for which NICE has not, or not yet, issued guidance.²² The establishment of the CDF was a political response to increasing public and healthcare professional pressures about access to new cancer treatments. Setting aside £200 million per year out of the healthcare budget highlighted the government’s additional willingness to pay for cancer therapies above and beyond the NICE threshold and made predictable budgeting an acceptable alternative to the cost/QALY approach. This may not be a viable solution in the long term for several reasons: the budget may end up being insufficient as other treatments are approved; it creates inequities in access by being available only to patients in England (not Wales); and it seems to value the QALYs obtained for cancer patients more than those obtained for other patient populations who have started advocating for special funds (eg, patients with rare disease). Overall, a NICE positive recommendation still remains the optimal way to ensure all eligible patients in England and Wales have routine access to therapy.

In summary, the lack of rejection based on clinical grounds suggested that the evidence packages submitted to HTA agencies were sufficient to address questions of therapeutic value in advanced cancers, across all jurisdictions. In contrast, disparate findings were observed for the role of cost-effectiveness. Lambda, the cost-effectiveness threshold, was found by Canadians involved in oncology reimbursement to be only one of many factors that should inform funding decisions – alongside consideration of broader factors, including formal guidance on weighting both evidence and values.³ Despite this broad vision expressed by a variety of oncology experts, Canadian HTA agencies are narrowly focusing on price as a primary mechanism to improve cost-effectiveness and yield more positive funding recommendations.

The UK is also exploring alternative ways to widen the range of factors considered in decision-making to improve NHS patient access to effective and innovative drugs while ensuring they are available at a price that reflects the value they bring. The Department of Health proposed to introduce a value-based pricing (VBP) approach that would have succeeded the Pharmaceutical Price Regulation Scheme (PPRS) expiring at the end of 2013.²³ Abolishment of the CDF was one of the objectives of this pricing reform. However, the UK government moved away from the proposed VBP approach by reaching an agreement on a new PPRS with the Association of the British Pharmaceutical Industry. By making the “P” in VBP no longer applicable, the 2014 PPRS led to the recommendation by NICE of a value-based assessment (VBA) approach that adds two factors to their appraisal methods: the burden of illness and the wider societal benefit.²⁴ How these proposed changes will eventually impact decisions in the United Kingdom remains to be seen.

Conclusion

This Canada–UK comparison revealed the lack of interagency consistency in recommendations, relative importance of cost-effectiveness to oncology recommendations, and the diversity of approaches to address unfavorable cost-effectiveness: an exclusion of price negotiation from both the drug review and funding process (INESSS), a reliance on confidential, postreview price negotiation to achieve cost-effectiveness (pCODR), a fixed cost-effectiveness threshold that requires either a pricing scheme during the review process or a high level of rejection based on failure to meet the threshold (pre-2014 NICE), or the explicit incorporation of nonmonetary factors to broaden the cost-effectiveness threshold (value-based assessment proposed by NICE in 2014).

One reason for the findings is that each jurisdiction has different HTA objectives and processes. The low rate of convergence for positive outcomes on the same drug/indication (43%) supported the fact that the three agencies had very different strategies to deal with clinical uncertainty or unfavorable cost-effectiveness. There was an apparent relationship between the pressure of negotiating solutions to improve ICERs and the outcome of the HTA recommendation. The Canadian processes put most of the pressure on the payers (in Québec) and manufacturers (in the rest of Canada) to reach a financial arrangement. On the other end, the NICE approach was more in favor of risk-sharing agreement and price negotiation during the HTA review to meet an agency-defined ICER threshold, and thus NICE itself bore most of the pressure.

There were many more differences between jurisdictions than simply their perspectives on cost-effectiveness within the HTA assessment process. There were important and substantial differences with respect to mandate, process, legislative setting, etc. Even the evidence base presented to HTA agencies may differ between jurisdictions.²⁵ These obviously all contributed to the observed findings. The future for the cost-effectiveness criterion within an HTA assessment, and within a funding decision-making framework, will depend on the jurisdiction under consideration, its current mandates and processes, and the tools available to achieve its goals. There are multiple levers, outside of HTA evaluation, that can be used to address high ICERs: coverage with evidence programs, value-based pricing, adaptive licensing, etc, in addition to the price negotiations that were observed in this research. In all jurisdictions, the review process and funding decisions for pharmaceuticals should have governance structures that are fair, objective, transparent, and accountable to patients, payers, and the public.

In this analysis, the primary reason for the observed divergence between agency recommendations appeared to be the availability of mechanisms in each jurisdiction to address cost-effectiveness subsequent to the HTA assessment process. Furthermore, caution is needed when interpreting cross-agency comparisons between HTA agencies, as recommendations may not correspond directly to subsequent funding decisions and actual patient access. This may be a concern, given the high profile internationally of HTA assessments conducted by the reviewed agencies.

Author contributions

All authors made substantial contributions to data generation and analysis, drafting or critical revision of the manuscript, and approval for the final version to be published.

Disclosure

This research was funded by Canada’s Research-Based Pharmaceutical Companies. This funding was not associated with any role in the design and conduct of the study, in the collection, management, analysis, and interpretation of the data, or in the preparation of the manuscript and decision to publish it.

Isabelle Chabot was a Senior Manager in Health and Economic Policy at Pfizer Canada when she collaborated in this research and in the development of the manuscript. Angela Rocchi (Athena Research) has been retained as a consultant on matters related to health economics and pharmaceutical policy in Canada by Canada’s Research-Based Pharmaceutical Companies and various member companies. The authors report no other conflicts of interest in this work.

---

References

1.	Canadian Institute for Health Information. National Health Expenditure Trends, 1975 to 2013. Ottawa, ON: CIHI; 2013. Available from: https://secure.cihi.ca/free_products/NHEXTrendsReport_EN.pdf. Accessed May 8, 2014.
2.	Detsky AS. Guidelines for economic analysis of pharmaceutical products: a draft document for Ontario and Canada. Pharmacoeconomics. 1993;3(5):354–361.
3.	Rocchi A, Menon D, Verma S, Miller E. The role of economic evidence in Canadian oncology reimbursement decision-making: to lambda and beyond. Value Health. 2008;11(4):771–783.
4.	National Institute for Health and Clinical Excellence. NICE announces measures on end of life medicines [press release]. London, UK: NICE; 2008 [November 5]. Available from: http://www.nice.org.uk/media/6C4/6C/2008066MeasuresOnLifeMedicines.pdf. Accessed June 17, 2014.
5.	Morgan SG, McMahon M, Mitton C, et al. Centralized drug review processes in Australia, Canada, New Zealand, and the United kingdom. Health Aff (Millwood). 2006;25(2):337–347.
6.	McMahon M, Morgan S, Mitton C. The Common Drug Review: a NICE start for Canada? Health Policy. 2006;77(3):339–351.
7.	Clement FM, Harris A, Li JJ, Yong K, Lee KM, Manns BJ. Using effectiveness and cost-effectiveness to make drug coverage decisions: a comparison of Britain, Australia, and Canada. JAMA. 2009;302(13):1437–1443.
8.	Institut National d’Excellence en Santé et Services Sociaux. Évaluation des médicaments [webpage on the Internet]. Québec, QC: Gouvernement du Québec; 2014. Available from: http://www.inesss.qc.ca/index.php?id=42. Accessed October 28, 2013. French.
9.	Canadian Agency for Drugs and Technologies in Health. Search CDR drug database [webpage on the Internet]. Ottawa, ON: Canadian Agency for Drugs and Technologies in Health; 2014. Available from: http://www.cadth.ca/en/products/cdr/search. Accessed October 28, 2013.
10.	Ontario Public Drug Program. Drugs Reviewed by the Interim Joint Oncology Drug Review (JODR) Process: March 2007 to Present. Toronto, ON: Ministry of Health and Long-Term Care; 2011. Available from: http://www.health.gov.on.ca/en/pro/programs/drugs/drug_submissions/pdf/drug_products_reviewed2.pdf. Accessed October 28, 2013.
11.	Ontario Public Drug Programs. EO decisions and CED recommendations [webpage on the Internet]. Toronto, ON: Ministry of Health and Long-Term Care; 2014 [updated January 21, 2014]. Available from: http://www.health.gov.on.ca/en/pro/programs/drugs/ced_rec_table.aspx. Accessed October 28, 2013.
12.	Pan-Canadian Oncology Drug Review. Find a review [webpage on the Internet]. Toronto, ON: Pan-Canadian Oncology Drug Review; 2014 [updated June 17, 2014]. Available from: http://www.pcodr.ca/wcpc/portal/Home/FindaReview?_afrLoop=2226078236285000&lang= en&_afrWindowMode=0&_adf.ctrl-state=1bfprl5yfn_47. Accessed October 28, 2013.
13.	National Institute for Health and Care Excellence. Guidance list [webpage on the Internet]. London, UK: NICE; 2014. Available from: http://www.nice.org.uk/guidance/published?type=TA. Accessed October 28, 2013.
14.	Yong JH, Beca J, Hoch JS. The evaluation and use of economic evidence to inform cancer drug reimbursement decisions in Canada. Pharmacoeconomics. 2013;31(3):229–236.
15.	Rocchi A, Miller E, Hopkins RB, Goeree R. Common Drug Review recommendations: an evidence base for expectations? Pharmacoeconomics. 2012;30(3):229–246.
16.	Devlin N, Dakin H, Rice N, Parkin D, O’Neill P. NICE’s cost-effectiveness threshold revisited: new evidence on the influence of cost-effectiveness and other factors on NICE decisions. Value Health. 2010;13(7):A246.
17.	Institut National d’Excellence en Santé et Services Sociaux. Access to Cancer Drugs Deemed Promising: Situational Overview and Pilot Project Assessment. Summary. Québec, QC: INESSS; 2012. Available from: http://www.inesss.qc.ca/fileadmin/doc/INESSS/Rapports/Medicaments/synthese_ONCO_mini_EN_21112012.pdf. Accessed June 17, 2014.
18.	Institut National d’Excellence en Santé et Services Sociaux. A view from Québec: updates from INESS. Presentation at Market Access Summit, EyeforPharma. Proceedings of the 5th Annual Market Access Canada; October 28–29, 2013; Toronto, ON.
19.	Morgan SG, Thomson PA, Daw JR, Friesen MK. Inter-jurisdictional cooperation on pharmaceutical product listing agreements: views from Canadian provinces. BMC Health Serv Res. 2013;13:34.
20.	Collins M, Latimer N. NICE’s end of life decision making scheme: impact on population health. BMJ. 2013;346:f1363.
21.	HM Government. The Coalition: Our Programme for Government. London, UK: Cabinet Office; 2010. Available from: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/78977/ coalition_programme_for_government.pdf. Accessed October 28, 2013.
22.	Department of Health. The Cancer Drugs Fund: Guidance to Support Operation of the Cancer Drugs Fund in 2012–13. London, UK: Crown Copyright; 2012. Available from: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/216574/dh_133683.pdf. Accessed May 8, 2014.
23.	Department of Health. A New Value-Based Approach to the Pricing of Branded Medicines: A Consultation. London, UK: Crown Copyright; 2010. Available from: http://webarchive.nationalarchives.gov.uk/20130107105354/http:/www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/@dh/@en/documents/digitalasset/dh_122793.pdf. Accessed May 8, 2014.
24.	National Institute for Health and Care Excellence. Public Board Meeting Held on 20th November 2013 at 1.45 pm in Conference Rooms 1 and 2, Riddel Hall, Queens University Belfast, 185 Stranmillis Road, Belfast, BT9 5EE. Item 4: Value Based Assessment of Health Technologies. London, UK: NICE; 2014. Available from: http://www.nice.org.uk/media/B00/0E/January2014PublicBoardMeetingAgendaAndPapers.pdf. Accessed May 8, 2014.
25.	Spinner DS, Birt J, Walter JW, et al. Do different clinical evidence bases lead to discordant health-technology assessment decisions? An in-depth case series across three jurisdictions. Clinicoecon Outcomes Res. 2013;5:69–85.