Systematic Review of Studies Using Conjoint Analysis Techniques to Investigate Patients’ Preferences Regarding Osteoarthritis Treatment

Background

Osteoarthritis (OA) is the most common form of arthritis.¹ It is a long-term chronic disabling degenerative joint disease that causes pain and limitation of movement.^2,3 Pain associated with OA substantially reduces the patient’s mobility and quality of life.⁴ Treatments primarily target joint pain to maintain and improve joint mobility.⁵ Options include surgery, pharmacological and non-pharmacological treatments.^6,7 However, alternative treatments differ in terms of the risks and benefits offered. Preferences for alternative treatments vary across individuals and depend on how they value the benefits relative to the associated risks.^8,9

It has increasingly become the goal of healthcare systems to promote patient involvement,¹⁰ especially that the discordant patient and healthcare provider preferences for different attributes of healthcare interventions are common.¹¹ In the United Kingdom (UK), the Health and Social Care Act 2012 made clear the duties of the national health service (NHS) to involve patients in the decisions about their treatment.¹² The use of stated preference techniques to elicit and understand patients’ preferences and values for health services and treatments to then inform treatment decisions is an accepted method of promoting patient-centred care^13–15 and its use has grown dramatically.^16–18 Specifically, identifying patients’ preferences for OA treatment offers a potential method for tailoring treatments, enhancing compliance, and improving patients’ satisfaction.¹⁹

One of the commonly used stated preference methods is conjoint analysis (CA)^20,21 which is a popular analytical technique for eliciting preferences.²² The idea behind CA is that it closely resembles the decisions that individuals make daily when choosing between multi-attribute alternatives.²³ The popularity of CA in health care is growing and it has gained increasing attention in health services research.^24,25 It is used as a method to measure patient preferences for health care and medicine, and as a means to identify and evaluate the relative importance of aspects of health outcomes and healthcare services.^26,27 CA methods and particularly discrete-choice experiments (DCEs) have become the most frequently applied approach in health care in recent years.²⁸ A review of published studies using DCEs to quantify preferences in healthcare reported that their use increased from fewer than 20 per year on average in the 1990s to over 60 published per year between January 2013 and December 2017.²⁹ Whilst DCEs are not the only conjoint analysis method, they make up the majority of published stated preference studies in healthcare.²⁹ Other CA techniques include traditional choice based conjoint (CBC), best-worst scaling (BWS), adaptive conjoint analysis (ACA) and adaptive choice-based conjoint (ACBC). All techniques require participants to compare and make trade-offs between a set of attributes and levels that define the health service or treatment under evaluation, and the trade-offs that participants make between these.³⁰

Alongside the increasing use of CA techniques, increased attention has been paid to their methodological quality. In 2011 and prior to the Health and Social Care Act of (2012), the International Society of Pharmacoeconomics and Outcomes Research (ISPOR) published a checklist for good research practices for CA studies, highlighting the items to be considered for best practice for CA applications in healthcare studies.²⁶

This systematic review aims to identify, summarise, and assess the methodological quality of the evidence that used CA techniques to quantify patient preferences for OA treatments and identify common approaches and methods employed and attributes considered important in eliciting patients’ preferences regarding OA treatment.

Methods

Search Strategy

A comprehensive search strategy was developed by the lead author. The Cochrane Library, PubMed (MEDLINE), CINHAL, EMBASE, and web of science were electronically searched from their inception until 10th June 2019. Medical Subject Headings (MeSH) and search terms were used to interrogate the databases. OA and CA related terms were used to conduct the search. No restrictions on publication language were used in the search strategy (appendix 1 shows an example of a MEDLINE search). In addition, electronic searching of Google, hand searching through an examination of the reference list of the published articles and contact with experts were also used to identify additional publications.

Three authors reviewed the titles and abstracts and evaluated all records against the inclusion/exclusion criteria.

Inclusion Criteria

Studies included in the review fulfilled the following criteria: 1) used any conjoint analysis methodology to elicit patient preferences including Conjoint Value Analysis (CVA), Choice-Based Conjoint (CBC), Discrete Choice Experiments (DCE), Best-Worst Scale (BWS), Adaptive Conjoint Analysis (ACA) and Adaptive Choice-Based Conjoint (ACBC); 2) focussed on patients diagnosed with OA irrespective of their age, gender, illness severity or joint of the body affected; 3) considered any form of OA intervention treatment.

Exclusion Criteria

Studies were excluded from the review if 1) participants were clinicians or healthcare workers (ie, not patients); 2) the focus was on the economic evaluation or willingness to pay (WTP) of a service or intervention; 3) the evaluation was restricted to quality rather than effectiveness or patient preference; 4) the focus was on the priority of treatment allocation, such as prioritising patients on the waiting list.

Quality Assessment and Data Extraction

The included papers were quality assessed and the data were extracted by the three authors. The ISPOR checklist for CA²⁶ was adopted to review and assess the methodological quality of studies included in this review. In the absence of a validated tool for quality assessment of CA studies, we considered the use of ISPOR checklist to guide this process. The checklist contains 10 main questions, each has 3 sub-questions, which adds up to 30 items in total.²⁶ Studies were assigned a score of “1” for each item of the ISPOR checklist if they were considered to meet at least one aspect of this item and “0” if not. A total score for each study was calculated by summing the item scores. The maximum possible final score was 30.

A data extraction form was developed by two authors. Key data elements included: study aims, population characteristics (country, number, age, and gender), sampling method, response rate, CA method, inclusion criteria, treatment, attributes, levels, and scenarios, statistical analysis, main results, and authors’ conclusion.

The included papers were independently assessed and scored by at least two of the three authors. Where there was a conflict of interest or potential reviewer bias, the reviewer in question was not involved in the assessment of scoring or the data extraction. Disagreements were resolved by discussion and consensus between all authors. A narrative data synthesis approach was used to analyse and report the results from the studies reviewed.

Results

Studies Identified

The search identified 534 records. Three hundred and sixteen records remained after removing duplicates. Based on the titles/abstracts review, a total of 297 records were deemed irrelevant and excluded as they did not meet one or more of the inclusion criteria. A further three records were excluded as they were published as conferences proceeding abstracts and the full reports were not published and not available from the authors. The remaining sixteen records were selected for full-text review and quality assessment. The PRISMA flowchart illustrating this process (see Figure 1).

Figure 1 The PRISMA flowchart. Notes: Adapted from Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med. 2009;6(7):e1000097. © 2009 Moher et al. Creative Commons.54

Quality Assessment

Sixteen studies were included in the review. The quality assessment scores of studies ranged between 19/30 and 29/30. This indicates that these studies fulfil at least 19 of the 30 best practice criteria in the ISPOR checklist. Across the 16 studies, there was low variation in total and individual item scores. Furthermore, the checklist did not provide emphasis to the themes that may have not been considered in the studies, which resulted in a high level of subjectivity in relation to the judgments made regarding the individual and total scores. Therefore, we are unable to make judgments on the quality of the studies or discriminate based these scores.

Study Population, Sample Size and Recruitment

All the included studies expressed a clearly defined research aim and conducted original research to examine patients’ preferences comparing OA treatments (exercise, drug, or surgery), and presented testable hypotheses (see Table 1).

Table 1 Type of OA Treatment, Aims, and Findings for All Reviewed Studies

Fifteen studies were conducted in a single country site – one in Australia, five in the UK, and nine in the United States of America (USA). One study was conducted across multiple countries – Australia, Canada, the UK, and the USA. Sample sizes for the studies ranged from 11 in the pilot study³¹ to 3895 the multi-site study.³² Justifications for the sample sizes were based on the study type (eg, whether it was a pilot study or part of a larger trial) and the sampling strategies employed. Most studies recruited patient participants from clinical lists directly using letters, telephone interviews or face-to-face methods. Four studies sampled members of the general population via emails through market research databases to recruit participants who self-identified as living with OA. One study recruited participants from both clinical lists – the patient sample; and a random public sample (identified through random-digit telephone dialling).²³ One study recruited participants from a clinical trial as part of the evaluation³³ (see Table 2).

Table 2 Sampling for All Reviewed Studies

All studies included participants with OA, mean age 55 years or more, and reported higher numbers of females to males. One study included a public sample of people age 20 and over.²³ One study did not report the gender of their population.¹⁹ The response rates (RR) reported varying from 7.6%³² to 100%^9,31,34,35 in the included studies, population and sampling features are presented in Table 2. The methods of data collection used in the studies also vary, reporting mostly either computer-based questionnaire,^9,31,34–40 or online web-based questionnaires^32,40,41 (see Table 2).

Conjoint Analysis Method

A range of CA methods was used in the included studies. One study used Conjoint Value Analysis (CVA), three studies used Choice-Based Conjoint (CBC), three studies used Discrete Choice Experiments (DCE), three studies used Adaptive Choice-Based Conjoint (ACBC), and six studies used Adaptive Conjoint Analysis (ACA) (see Table 3). The number of attributes and levels identified in the studies ranged from 4 attributes with 12 levels³⁵ to 9 attributes with 29 levels⁴¹ (see Table 3). The attributes tended to define the features of the OA symptoms, OA treatment such as the benefits and the risks, and cost of treatment (for all attributes and levels of the included studies see appendix 2).

Table 3 The CA Methods’ Characteristics for All Reviewed Studies

Statistical Analysis

In all types of CA, regression analysis techniques are generally used to study the patient’s preference. The choice of regression analysis type in CA depends on the type of the main outcome under study (eg, binary outcome, continuous outcome, etc.). More recent studies have adopted Hierarchical Bayesian (HB) models to investigate participants’ preferences at both the group “average” level as well as at the individual level^31,35,41 (see Table 3).

Treatment Preferences

The review included studies investigating pharmaceutical, non-pharmaceutical, and surgical treatment for OA (see Table 1).

NSAID and Other Medication Treatment

The majority of studies investigated the side effects and other features of nonsteroidal anti-inflammatory drugs (NSAIDs) and other medications such as disease-modifying drugs and supplements (glucosamine) on patients’ preferences for treatment of OA.^{9,32,33,35–39,41–43}

The relative importance of the risks of side effects; both rare and common were rated more important than the benefits associated with the treatment, time to benefit, out-of-pocket monthly cost, route of administration, and the product label.^36–38 One study found that relatively the most important attribute was the route of administration (cream, pills, injections into the knee and exercise) (relative importance of 24%), followed by the risk of dyspepsia and risk of bleeding ulcer, with the least important being decrease in pain and improved strength (relative importance of approximately 14%).⁴² Similarly, a study investigating the long-term evaluation of glucosamine sulphate, found that relatively the most important attributes were the side effects of high blood pressure, heart/liver/kidney problems followed by cost.³³ The authors concluded that in their study, preferences to continue with OA treatments were influenced by side effects first and foremost and treatment efficacy did not significantly influence patient choice.³³ Again, a study³¹ investigating 8 medication attributes, found that relatively the risks of side effects were the most important (combined their relative importance accounted for 66% of the treatment decision) and effectiveness of the medication only accounted for 8% of the treatment decision.

Exercise Treatment

One study examined patients’ preferences for exercise in the context of other available treatment options (excluding surgery).⁴² The authors found that patients prefer exercise over pharmacological treatment for; risk of dyspepsia and bleeding ulcer combined accounted for the relative importance of 41.3% compared to 28.9% relative importance for both decrease pain and improve strength attributes.⁴² Another study investigated individual preferences for physical activity attributes (with no comparison to other types of OA treatment).⁴⁰ This study found that “health benefits” (26%) and “enjoyment” (24%) attributes were considered by patients to be relatively the most important.

Surgical Treatment

Three studies investigated patients’ preferences for surgical treatment of OA. One study investigated the relative preferences for 9 different surgical related procedure attributes and simulated how patients may have responded to real-world knee OA procedures based on their preferences.⁴¹ They found that patient preferences for surgical interventions were influenced by “the amount of cutting and removal of existing bone required” (relative importance of 18.7%), followed by “chance of additional surgery” (relative importance of 14.1), “amount of pain relief” (relative importance of 12.7%), with the least important attributes being “limits or complicates any future treatment need on the knee” and” length of hospital stay” with a relative importance of 7.3% each.⁴¹

Similarly, in the study comparing patient preferences for surgery for patients with a hand OA diagnosis,⁴⁴ the authors found that “the need for future surgery” (relative importance=19%) and “recovery time” (relative importance=3%) were the least important factors influencing surgical preferences, while “joint stiffness” (relative importance=32%) and “grip strength” (relative importance=29%) were the most important. This supports the results from the earlier study that explored preferences for surgery versus medical treatment of knee OA,²³ which found that the severity of OA symptoms, directly and indirectly, influenced the patients’ choice of OA treatment, even in the presence of cultural differences in attitudes towards particular treatments.

Discussion

To the best of our knowledge, this is the first review to investigate and summarise the use of CA techniques to value patients’ preferences for OA treatment. In addition, the search strategy was comprehensive, including the search of many databases, contacting authors and experts in the field, and searching the reference lists of published studies.

One of the limitations of this review is the lack of a validated quality assessment tool for CA studies. The use of the ISPOR checklist to score studies may be subjective to the examiner’s opinion. We tried to assess the methodological quality of these studies using the ISPOR Conjoint Analysis Experimental Design Good Research Practices Checklist. We were unable to make an objective decision regarding the minimum acceptable evidence required to award the scores. For example, question 2 “was the choice of attributes and levels supported by evidence?” we were unable to determine the quality and quantity of evidence required. This caused lengthy subjective disputes between the reviewers. Furthermore, the total scores for the studies indicated that CA studies published post the publication of the ISPOR checklist scored higher than those published pre-2011. This would be expected as most of these studies referenced ISPOR in their papers, meaning that we are assessing their quality against the same or similar criteria they used to design their studies, which was not available for studies published before 2011. It is not clear if this improvement in the scores is correlated with the publication of the ISPOR checklist or is simply reflecting an improvement in reporting. We agree with Webb and colleagues that the ISPOR checklist should not be used as a quality assessment tool for conjoint studies in its current format, as it was not originally developed for this purpose.⁴⁵

The studies have a high degree of heterogeneity in study design, study population, and treatment choice. The included papers incorporated studies using both rating/ranking and choice-based methods to investigate different options of treatment for OA (exercise, medication, and surgery) in the UK, Australia, Canada, and the USA. All included studies had homogeneous samples in terms of suffering from OA. Thus, the studies sample may represent the OA population. However, the healthcare systems differ between the countries within which the studies were conducted; therefore, the generalisability of the results could be limited.

Variations in the sample sizes between included studies (n = 11 to 3895) may indicate that there is still no consensus on the appropriate or agreed sample size calculation method for CA studies, as it depends on many factors such as the number of questions and scenarios in the conjoint task. It has been suggested that the sample size for a CA study should be at least 300 in one sample group.⁴⁶ However, the traditional calculations for sample size determination cannot readily be applied to CA⁴³ and are rarely applied for practical reasons.⁴⁷ Furthermore, it has been argued that collecting more data from each respondent by designing high-quality conjoint tasks may reduce sampling and measurement error.⁴⁶ Using similar CA methods to those in the review^36–38,42 in a study of patient preferences for acute pain treatment researchers attempted to reduce the limitation of a small sample (50 participants) by interviewing their respondents 4 times at 4 different stages of pain treatment.⁴⁸ Limitations around sample size in CA studies may be overcome in the design of the conjoint task and data collection.

The variation in the RR (7.6% to 100%) in the studies is potentially a reflection of the robustness of the methods of recruitment and methods of data collection. The included studies used a variety of methods of data collection. Methods reporting face-to-face interviewing or questionnaires targeted a specific population of interest tended to have higher response rates. Studies using telephone interviewing or emails, predominantly in a general population, had a lower response rate. These studies with low RR recognised the limitations of using an untargeted strategy and suggested response rates could be improved in future research by pre-screening participants in order to target the full survey to those who report a diagnosis or other study characteristic of interest.³²

All included studies recognised the value in utilising CA method to investigate patients’ preferences for OA treatment, but there was no consensus on which CA approach is the most appropriate. Both rating/ranking and choice-based methods were used to examine patients’ preferences for the treatment of OA. Recent academic and practical research applications have tended to favour choice-based approaches as opposed to rating/ranking.⁴⁹ However, the rating/ranking approach has also been used and recommended by many researchers to study patients’ preferences for OA treatment^36–38,42 as well as treatment preferences in rheumatoid arthritis (RA),⁵⁰ chronic pain,⁵¹ and abdominal surgery⁴⁸ because it allows the inclusion of a large number of attributes and levels, which reflect the outcomes/concerns of patients with OA. The main advantage of ACA is that it is adaptive and therefore allows a large number of characteristics to be evaluated without resulting in information overload or respondent fatigue, and minimises interviewer, product, and brand bias. Nevertheless, there are still practical limitations associated with ACA, with researchers reporting that not all treatment characteristics could be included in an ACA task.^36–38

In this review, studies that used the choice-based approach reported that the use of the discrete choice method allowed them to identify attributes significantly influencing patients’ preferences for OA treatment.⁴³ Furthermore, a very low number of inconsistent responses were found, and participants reported that the questions were easy or very easy to answer.²³ Those studies that used ACBC^9,31,34 argued that the approach can capture more individual-level data and precise estimates than through a traditional CBC approach and that it can yield similar group-level standard errors using up to 38% fewer participants.^39,40 Furthermore, it has been reported that the ACBC method is more user friendly and engaging than alternative CA methods^31,34,52,53 and it can be used to elicit individual patients’ preferences.⁹

Overwhelmingly the results of the studies in this review indicated that patient preferences for OA medications were driven by the desire to avoid both common and rare side effects, especially those with more serious drug-related toxic effects and that the effectiveness of the OA medication had very little impact on patients’ preferences. However, where investigated, studies suggested that preferences for side effects were affected by patient characteristics such as age and symptoms severity. Older respondents were more willing than younger respondents to trade-off an increased risk in the side effects^36–38,43 for an improvement in the symptoms of OA. The side effects associated with NSAIDs had a greater negative influence on the preferences of patients with milder OA than those in more severe OA states.³² Even when exercise was compared to OA medications, patients were still more concerned about the side effects of the treatment than the benefits.⁴² However, patients with more knee pain were more reluctant to choose exercise.

Patients generally attached greater importance to reducing or eliminating adverse events than reducing pain, but one study investigated the level of treatment-related risks patients were willing to accept in exchange for various improvements in pain.³⁹ The investigators found that participants’ “risk tolerance” varied according to their pain level at baseline and type of symptom relief – participants were willing to accept greater risks for improvements in ambulatory pain than in resting pain.³⁹ Similarly, a study of treatment options for disease-modifying drugs found that sub-groups of participants were willing to trade-off the risks of side-effects for improvements in a benefit.³⁵ In relation to surgical treatment for OA, it was reported that younger patients and those who reported the highest pain thresholds, and the greatest functional limitations were more likely to opt for surgical intervention.⁴¹ Furthermore, the severity of the patients underlying symptoms proved to be the main driver influencing their preferences for surgery.⁴⁴

Where the severity of OA symptoms was measured alongside the conjoint task, all included studies suggested that the severity of symptoms influenced the patients’ preference of treatment, and consequently the relative importance of treatment characteristics. However, it is not clear whether these differences are a result of symptom severity or artefacts of the CA methods, attributes used, or treatments being assessed.

Conclusion

The severity of OA symptoms and the side effects of treatment have a significant influence on patients’ preferences for OA treatment. Both rating/ranking and choice-based CA methods are recommended in investigating patients’ preferences for OA treatment, but there is no consensus on which CA approach is the most appropriate.