A Systematic Review of Sources of Outcomes and Cost Data Utilized in Economic Evaluation Research Conducted in the Gulf Cooperation Council

Introduction

Scarcity of resources makes cost-effective choices increasingly crucial. Economic evaluations aid decision-makers in effective resources allocation through the identification, measurement, valuing, and comparing the cost and outcomes of alternative interventions or services.¹ Although the use of economic evaluation research is growing, methodological flaws and poor-quality research impact the validity of the results and limit their usefulness for health-care decisions.^2–4 Therefore, quality assessment of economic evaluation studies is highly warranted prior to the application of the results. Additionally, clear and sufficient reporting of cost and outcomes data will allow for reproducibility of data in the future.

The Gulf Cooperation Council (GCC) is a political and economic alliance between six Middle Eastern countries: the Kingdom of Saudi Arabia (KSA), Kuwait, the United Arab Emirates (UAE), Bahrain, Qatar, and Oman. GCC members have similar political and cultural identities and are classified as high-income countries according to the World Bank reports.⁵ Regarding economic evaluation research, despite the increase in the number of studies globally, previous studies, conducted exclusively in the GCC or in a broader or narrower geographical region, reported limited quantity and low quality of published economic evaluation studies.^6–8 Alefan et al⁷ searched PubMed, MEDLINE, and Google Scholar and identified 17 studies published in the World Health Organization Eastern Mediterranean Countries (WHO EMCs), which include GCC countries. Although the researchers utilized multiple databases, the study has several methodological limitations, including the exclusion of multiple-country studies, failure to report the origin country for the studies in the results, and covering only studies published up to 2013. In the second review, by Eljilany et al, PubMed was the only database used to identify the studies and was searched up to the end of 2017.⁶ The researchers identified 49 studies, of which nine were full economic studies. The quality assessment revealed that 47% were poor and extremely poor quality based on the Quality of Health Economic Studies (QHES) instrument.⁹ However, disaggregated quality assessments for each study were not reported, which in turn makes identifying the major methodological flaws impossible. Neither review examined the methods used to identify, measure, and value cost and outcomes data in the included economic evaluations.^6,7 Details on quality criteria assessment and methods used to identify, measure, and value cost and outcomes data will guide future researchers in the region in conducting robust economic evaluations that will aid decision-makers to allocate resources efficiently. Therefore, the current review aimed to 1) assess the reporting quality of the studies using the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist² and 2) determine the methods used to identify, measure, and value costs and outcomes data in full economic evaluation studies published in the GCC area.

Methods

The systematic review was performed in accordance with the Centre for Reviews and Dissemination’s guidance for undertaking reviews in health care and reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement.^10,11

Data Source and Search Strategy

A systematic search of electronic literature was conducted in four databases (Medline via Ovid, Scopus, EMBASE via Ovid, and NHS EED) to identify relevant English-language publications related to economic evaluation studies in the GCC area from 2009 to 2019. For the electronic search of economic keywords, McMaster University’s Hedges project filters for MEDLINE and EMBASE (economic category) were used to ensure comprehensive and precise coverage of the economic terms.^12,13 For the SCOPUS database, the keywords were adopted from a systematic review on economic evaluation studies.¹⁴ NHS EED is a specialized database for economic evaluation studies. Therefore, no special economic filters were applied. The keywords “Saudi Arabia,” “KSA,” “Kuwait,” “Bahrain,” “United Arab Emirates,” “UAE,” “Qatar,” “Oman,” “Gulf Cooperation Council,” and “GCC” were used to search for the countries. The detailed search strategy for each database is presented in Supplementary File 1. The last search was performed on October 14, 2019. Furthermore, a hand search was performed in three relevant systematic reviews to identify other related publications.^6–8

Eligibility Criteria

For a study to be included, all the following inclusion criteria had to be met:

• Performed in at least one GCC country with or without other international countries.
• Full economic evaluation study employing cost-effectiveness analysis (CEA), cost-utility analysis (CUA), cost–benefit analysis (CBA) and cost-minimization analysis (CMA). These evaluations can be based on trials or synthesis data from different sources using decision analysis models.
• Economic evaluation of a health-related topic.

Studies were excluded if they were:

• With no economic evaluation component.
• Partial economic evaluations that described costs and outcomes but did not involve a comparison between alternative interventions, did not relate costs to benefits, or focused solely on costs (cost analysis) or outcomes (efficacy or effectiveness evaluation).
• Not a primary research paper (eg, narrative review or letter)
• Not published in a peer-reviewed journal (eg, thesis or conference abstract).

Study Selection and Data Extraction

After removing duplicate studies in EndNote, following Bramer et al’s methodology,¹⁵ one author (SSA) reviewed the search results and screened retrieved studies through titles and abstracts. Full-text copies were obtained if a study appeared to meet the inclusion criteria or it was unclear whether it would meet the criteria. Double screening of full texts was conducted by two authors (SAA and SHA). Information was extracted from the full-text articles by a research assistant and reviewed by one of the authors (SSA) using a standardized form that includes study ID, country, setting, study design, disease/condition, perspective, cost year, currency, source of cost data (healthcare resource use (HRU)), unit cost source, primary outcomes used, and source of outcomes.

Study Quality Assessment

To assess the reporting quality of the economic evaluations, the CHEERS checklist was used by the three authors independently.² The CHEERS checklist has been supported by the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) organization² and endorsed by the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) network.¹⁶ The checklist includes six areas, with 24 items: 1) title and abstract (2 items), 2) introduction (1 item), 3) methods (14 items), 4) results (4 items), 5) discussion (1 item), and 6) other, which is related to funding and conflicts of interest (2 items). The aim of the checklist is to improve the reporting of key items in economic evaluation studies; however, the quality of published studies can be evaluated indirectly, via assessment, using the checklist.²

Results

Study Selection

Figure 1 illustrates the PRISMA flowchart. The search retrieved 1857 papers, of which 428 were from Medline, 550 were from Scopus, 872 were from EMBASE, and 4 were from NHS EED. After removing 378 duplicates, 1479 abstracts and titles were screened, from which 1313 were excluded. The remaining 166 studies were retrieved and assessed in full text: 152 studies were excluded, and 14 studies were considered for the systematic review. The reasons for exclusion are reported in Figure 1. No additional references were identified by searching the bibliographies of three key papers.^6–8

Figure 1 PRISMA flowchart for the process of selecting the included economic evaluation studies.Notes: PRISMA figure adapted from Liberati A, Altman D, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Journal of clinical epidemiology. 2009;62(10). Creative Commons.10

The search identified 51 conference abstracts. These were excluded from the current review based on our inclusion criteria. Supplementary File 2 presents a summary of these excluded abstracts after the removal of two identified duplicates.

Study Characteristics

A description of the main characteristics for each of the studies included is provided in Table 1. All were published from 2014 to 2019 and conducted in Saudi Arabia (n = 11),^17–27 the UAE (n = 1),²⁸ Qatar (n = 1),²⁹ and Oman (n = 1).³⁰ Out of the 11 studies undertaken in Saudi Arabia, four studies were multinational.^17–19,27 Of the 14 reviewed studies, eight were based on the Markov modelling technique, three on decision trees, two on RCTs, and one on a retrospective observational study. Two studies did not report the perspective of the analysis.^18,19 The majority of the studies evaluated costs from the health-care system perspective, which refers to a variety of bodies, including hospitals (n = 8)^{17,20,23,24,26,27,29,30} or public payers (n = 1),²⁸ with one additionally considering the societal perspective.³⁰ Societal (n = 2)^22,25 and patient (n = 1)²¹ perspectives were also adopted.

Table 1 Characteristics of Included Economic Evaluation Studies

All studies were presented as CEA, except one study, in which the analysis was cost-minimization.²⁰ One study intended to perform CEA but performed CMA due to the insignificant difference in effectiveness data.¹⁷ Among the 13 cost-effectiveness studies, quality-adjusted life years (QALYs) was the main outcome in 10 studies. Therefore, we can consider that CUA was applied in these studies. The remaining studies used intermediate disease-specific outcomes, clinical cure/success rate, and adverse drug event rate.

The costing year was explicitly reported in 10 studies; the remaining studies reported the year for the conversion rate.^18,19,24,25 A variety of currency data was reported: Five studies used both the local currency and international dollars,^{18,19,23,29,30} five studies used international dollars,^{17,21,25–27} and the remaining four studies used the local currency.^20,22,24,28

Quality of Economic Evaluations

A visual quality assessment of the studies using CHEERS criteria is presented in Figure 2. Most of the included studies met the majority of CHEERS quality criteria. However, none met all of the criteria. In the title and abstract, the majority of the studies partially reported the main required elements, such as the compared interventions in the title and the perspective or the base case and uncertainty analysis results in the abstract. In the analytical methods, the basic elements required to sufficiently report the criteria according to CHEERS criteria were methods for dealing with skewed and missing data, extrapolation methods, approaches to validate or make adjustments to a model, and methods to handle population heterogeneity and uncertainty. Of these elements, uncertainty analysis was reported in all studies. The impact of uncertainty was examined using probabilistic sensitivity analysis (n=9) and deterministic sensitivity analysis (n=4), and a method to handle missing data was reported in two studies.^17,27 Model validation was conducted in two studies only.^25,26

Figure 2 Results of methodological quality assessment of the studies using CHEERS criteria.Notes:  The figure is based on Husereau D, Drummond M, Petrou S, et al. Consolidated health economic evaluation reporting standards (CHEERS) statement. BMJ. 2013;346(mar25 1):f1049. Creative Commons.2

The majority of the studies failed to report the inputs ranges explicitly. Furthermore, studies undertaken in multinational countries,^18,19,27 country-specific inputs, or disaggregated data were not reported thoroughly. When estimating the total costs of interventions or services, HRU data and unit costs are essential. Some studies reported the total cost without the primary cost elements,^19,23,24 or they reported unit and total costs without HRU data.²⁵

When characterizing heterogeneity, all studies except two^17,27 failed to report any difference in costs, effectiveness, or cost-effectiveness results when varying the baseline characteristics of subgroup of patients. The final criterion that was not reported efficiently in the studies was the “incremental cost and outcomes.” Failing to report one or more elements of this criterion was highly observed in the studies,^{18,20–26,29,30} such as main costs and outcomes of comparators, the incremental differences in costs and outcomes, and ICER.

Data and Sources of Outcome, Healthcare Resource Use, and Costs

Outcome Data and Sources

For utility assessment (Table 2), the majority of studies used EQ-5D (n = 6), and they valued using the UK value set tariff^26,27 or did not report methods of valuation.^18,19,21,30 Other studies used the Health Utilities Index (HUI2),³⁰ standard gamble,²³ or did not report the instrument for utility valuation.^22,25,28

Table 2 Data and Sources of Outcomes, Healthcare Resource Use, and Costs of Included Economic Evaluation Studies

In terms of outcome data sources, three studies utilized local sources such as electronic medical records, physicians’ notes, and hospital data;^20,24,29 four studies utilized a preceding published multinational RCT or observational study;^17–19,27 and the remaining studies obtained data from published international literature.^{21–23,25,26,28,30}

Healthcare Resource Use, Costs, and Sources

When examining the type of measured healthcare resource use data (Table 2), medications and other medical costs such as physician visits, hospital admission, laboratory, and imaging costs were the most collected data in the studies. These types of HRU are direct medical cost and were relevant to the health-care system perspective of the studies.^{17,20,23,24,26–30} Cost incurred by patients for medications, physician visits and adverse event management were collected in one study²¹ and was relevant to the perspective. Direct non-medical cost (n=1)³⁰ and productivity cost (n=2)^22,25 were collected in studies in which the perspective was societal.

Resources utilization data were derived from several sources, including local sources such as hospital-based data (electronic medical records, with or without physicians’ notes) (n=4),^20,23,24,29 original studies including a preceding published multinational RCT or observational study performed in the same country (n=3),^17–19,27 literature based in the same country of the evaluation (n=1)³⁰ or other countries (n=1),²⁶ assumptions (n=2),^21,30 and expert opinion (n=3).^22,25,28 Al-Aidaroos et al performed two rounds of the Delphi method with a panel of experts to estimate HRU data;²² consultant panels were used to validate local data in Al-Senani et al²⁵; and Nuhoho et al used both literature and assumptions, which were then validated by clinical experts in the UAE setting.²⁸

Regarding unit cost, various sources were utilized in most evaluations, including local sources such as hospital-specific costs (n=7),^{17,20,21,23,24,27,29} local/national data (n=7),^{18,20,21,23,25,26,28} panel of local experts (n=2),^22,25 assumptions (n=3),^22,27,30 costs imputed from other country settings (n=10),²⁷ or published data from previous national and international literature (n=4).^18,26,28,30 One study did not report the method of valuing the unit cost.¹⁹ When the unit cost was identified through local/national data, a variety of sources were used in the evaluations, including a local pharmaceutical company utilized in one study,¹⁸ national databases including the Saudi Food and Drug Authority (SFDA) to derive medications cost in three studies undertaken in Saudi Arabia,^20,23,26 and the Ministry of Health databases to derive treatment costs such as diagnostics, devices, drugs, and staff in two studies in the Saudi Arabia^21,25 and one study in the UAE.²⁸

Of the three studies that considered the societal perspective and estimated productivity cost, or indirect cost, one was based on the human capital approach,²⁵ one was based on a Delphi panel of experts to estimate productivity changes,²² and the last was based on expert opinion.³⁰

Discussion

This systematic review provides the current state of health economic research in the GCC region. The findings show that full economic evaluations are still limited in number in the region. Only 14 studies were published over a 10-year period. The majority were conducted in Saudi Arabia, whereas no studies were undertaken in Kuwait or Bahrain. One possible explanation for this paucity in health economics research could be that economic evidence is not a mandatory regulatory requirement for licensing, pricing, and formulary addition in the GCC countries. Excluding economic analysis from the decision-making process has been reported as a major barrier in conducting economic evaluations.³²

In addition to being limited in number, none of the reviewed studies met all the CHEERS quality criteria. The low quality of published economic evaluations has been observed in previous research.³³ The differences in methodological quality of economic evaluations across low-income, middle-income, and high-income countries could be attributed to many challenges, including contextual differences; scarcity, quality, and accessibility of data because of the absence of routine cost accounting systems and limited patient-information systems; limited health economics research capacity; and differing expectations and research environments.^34,35

We identified four multinational studies. This is a promising endeavor for the region. However, it is recommended that guidelines be developed to aid researchers in this area based on an understanding of the challenges associated with multinational trials to ensure that results will aid decision-makers in their individual countries.³⁶

Regarding sources of outcomes, HRU, and costs, a number of the reviewed studies obtained data on outcomes and costs from published literature from foreign countries. A plausible reason for using international data is the lack of quality local data, such as randomized trials and observational research, in the region. In Saudi Arabia, which had the highest number of economic evaluations in our review, a study found that only 61 randomized trials were published between 1987 and 2018.³⁷ Unavailability of data has been reported as a major barrier for researchers to conduct health economics research,^32,38 which reflects the reason behind the limited number of studies in this review.

Implications for Practice and Research

This review highlights several implications. First, there is a need to increase the quantity and quality of economic evaluations in the GCC. To enhance the reliability of economic evaluation results, researchers should employ rigorous methodologies and analytical methods and report them thoroughly. Several guidelines for the conduct and reporting of economic evaluations and validation of decision models are available.^2,39–41 The development of local guidelines for GCC countries should be considered to strengthen the quality of economic evaluation research. Researcher in the GCC countries should pay special attention for estimating and reporting HRU quantities and unit cost. These data should be reported explicitly, including values, ranges, and references for all study inputs and taking into consideration any inflation or discounting in the parameters.² A publication by Frankin et al provides educational material on methods to identify, measure, and value costs in economic evaluations.⁴² Moreover, research investigating the challenges facing researchers in the GCC region for conducting economic evaluations and proposing possible solutions to overcome these challenges are highly warranted.

Second, looking at sources of cost and outcomes data in the reviewed studies, there is a need for publicly available national data on costs and outcomes to diminish the need to use international data. There is a need to establish national-level databases in the GCC countries and make it accessible to researchers. The number of economic evaluations using these data, called real-world data (RWD), has been rising in recent years.⁴³ RWD provide data that are more reflective to the daily clinical practice and overcome the limitation of short-term data in randomized trials through longitudinal follow-up of patients.⁴⁴ Furthermore, available and accessible RWD for researchers could lower the use of international literature to estimate costs and outcomes in the future.

Finally, when estimating QALYs for cost-utility analysis, the EQ-5D is a widely used preference-based outcome measure. To make the EQ-5D suitable for use in economic evaluation, the produced health states need to be assigned an index score by applying scores from preference weights (tariffs). However, a tariff for any GCC country is not established yet.⁴⁵ The lack of tariffs can explain the low use of EQ-5D local utility values in the reviewed studies to estimate QALYs and the use of published QALYs from international studies instead. Future research is required to estimate utility values to summarize the importance of different aspects of health by the general public in the GCC countries.

Strengths and Limitations

This review followed the robust methodology outlined by the Centre for Reviews and Dissemination’s guidance for undertaking reviews in health care.¹¹ We identified 10 studies that were not included in the work of Eljilany et al.⁶ Six of them were published after 2017. In addition, a robust methodology was used to search four databases, including a specialized health economics database. Moreover, the quality assessment for each study was reported in a disaggregate form, allowing future researchers to identify the major methodological flaws in previous economic evaluations. A final strength is that the current review reported details on the identification, measurements, and valuations of cost and outcomes data utilized in the studies that can benefit future researchers.

An acknowledged limitation of this review is the exclusion of scientific abstracts. However, a summary of excluded scientific abstracts that include information on different sources of cost data in the GCC area is available in Supplementary File 2. A second limitation is that the retrieved studies were limited to those indexed in Medline, Scopus, EMBASE, and NHS EED databases. An additional hand search in three relative publications was used to broaden the search. The final limitation is the restriction to English publications.

Conclusion

Economic evaluation studies in the GCC area are scarce, and none met all the CHEERs quality criteria. Variable sources of data were used in the studies, with the majority of outcomes data obtained from foreign-country publications, which might affect the conclusions of the studies. Future economic evaluations in the GCC should adhere to current health economics guidelines. Furthermore, the data for economic evaluations could be further improved to country-specific data. Robust economic evaluations will support decision-makers to allocate resources efficiently.