Back to Journals » Patient Preference and Adherence » Volume 16

Does It Matter How You Ask? Assessing the Impact of Failure or Effectiveness Framing on Preferences for Antibiotic Treatments in a Discrete Choice Experiment

Authors Smith IP , Ancillotti M , de Bekker-Grob EW , Veldwijk J 

Received 15 March 2022

Accepted for publication 1 July 2022

Published 27 October 2022 Volume 2022:16 Pages 2921—2936


Checked for plagiarism Yes

Review by Single anonymous peer review

Peer reviewer comments 2

Editor who approved publication: Dr Johnny Chen

Ian P Smith,1,* Mirko Ancillotti,2,* Esther W de Bekker-Grob,3,4 Jorien Veldwijk2– 4

1Julius Center for Health and Primary Care, University Medical Center Utrecht, Utrecht, the Netherlands; 2Centre for Research Ethics and Bioethics, Department of Public Health and Caring Sciences, Uppsala University, Uppsala, Sweden; 3Erasmus Choice Modelling Centre, Erasmus University Rotterdam, Rotterdam, the Netherlands; 4Erasmus School of Health Policy & Management, Erasmus University Rotterdam, Rotterdam, the Netherlands

*These authors contributed equally to this work

Correspondence: Ian P Smith, Julius Center for Health and Primary Care, University Medical Center Utrecht, Str. 6.131, P.O. Box 85500 3508 GA, Utrecht, the Netherlands, Tel +31 88 75 69 616, Fax +31 88 75 554 84, Email [email protected]

Purpose: Studies assessing framing effects in discrete choice experiments (DCE) primarily focused on attributes related to mortality/survival information. Little is known about framing effects for other attributes in health-related DCEs. This study aimed to investigate how framing treatment outcome as effective, failure, or a combined frame impacts respondent choices and DCE outcomes.
Patients and Methods: Three Bayesian D-efficient designed DCE surveys measuring preferences for antibiotic treatments were randomly distributed to a representative sample of the Swedish population aged 18– 65 years (n=1119). Antibiotic treatments were described using five attributes. Four attributes were static: Contribution to Antibiotic Resistance, Treatment Duration, Likelihood of Side-Effects, and Costs. A fifth treatment attribute was framed in three ways: Effectiveness, Failure Rate, or both. Mixed logit models were used to analyze attribute level estimates, importance value, and choice predictions.
Results: Significant differences between the frames were found for the parameter estimates of the attributes of Treatment Duration and Likelihood of Side-Effects, but not Treatment Outcome which was the alternatively framed attribute. Contribution to Antibiotic Resistance and Costs were the most important attributes for all participants regardless of framing. Choice predictions for the “best option” antibiotic only slightly differed between the groups based on the frame seen (95.2– 92.4%).
Conclusion: Our study showed that attribute framing can impact preferences regardless of the attribute’s importance value in alternative valuation. However, the practical implication of this effect may be limited. A theoretical discussion is needed to identify how researchers should accommodate and report any potential framing effect in their studies.

Keywords: patient preferences, patient-centered care, DCE


Discrete choice experiments (DCEs) are a commonly used method to elicit stated preferences for health treatment outcomes.1–3 In a DCE, participants are asked to choose between different choices or alternatives, which are described using sets of pre-specified attributes and levels. The foundational theory of DCEs, the Random Utility Theory (RUT), assumes that the utility of an alternative can be derived through the compound valuation of the different attributes and levels describing the alternative.4–6 This total utility is used by the participant to compare alternatives in a choice context and the resulting choice should represent the greatest total utility according to RUT.7

When developing a DCE, researchers are tasked with creating a limited set of attributes to describe the alternatives. This set of attributes should be based on a thorough qualitative process including a literature review, interviews with experts, and discussions with the (potential) consumers of the healthcare product.8 The goal of this process is to create a list of attributes which is thorough enough to contain those attributes which are relevant to the consumers of the healthcare product and stakeholders who will use this information without being so large as to make the study design over complex or burdensome.9–11 A key part of this process is having attributes described in such a way that they can be clearly understood by the participants. In order to ensure that the attributes are understood, qualitative assessment of the attributes by the researchers is recommended by good practice guidelines, but often limited to checks of comprehensibility and plausibility.9,12 Current guidelines for developing an attribute list in patient preference studies do not address the impact that framing can have on interpreting attributes.9,13–15

The impact that framing can have on decision-making is well documented and has been studied for years in psychological fields.16–18 This framing bias has been found to influence a participant’s understanding of the health information commonly used in DCEs, such as probability of negative outcomes or treatment risks.19–22 Specific examples include patients preferring lung cancer treatments framed in terms of survival rather than mortality,23 an effect so great that the framing can move an attribute from being the least important to the most important when eliciting preferences.24 In one study, participants were more likely to choose a healthy lifestyle when the outcomes were framed as an increase in life expectancy vs poor health prevention.25 In another study, participants were more willing to pay for colorectal screening when the tests were framed in regards to percentage of cancers found versus missed.26 Most of these studies are similar in that they assessed the impact of framing for attributes that generally play a large role in decision-making (such as survival or mortality). In the field of healthcare preference assessment, only the study by Howard and Salkeld looked at the impact attribute framing had on an attribute not related to survival or mortality (ie, specificity or sensitivity of colorectal cancer screening).26 None of the studies presented the different frames simultaneously as a means to possibly control for the framing effect which has previously been suggested as a method to control for framing effects.27 There is thus a lack of research specific to the impact of framing in the field of patient preference assessment and the best practices to overcome it.28

This article reports on a methodological study designed specifically to address this gap in knowledge by assessing the impact of framing on preferences for a hypothetical personal medical treatment. Specifically, this study aims to assess whether presenting an attribute not related to survival or mortality in a positive, negative, or combined frame impacts preference outcomes in terms of stated choices, preference estimates, importance value scores, and expected choice predictions.


Case Study and Participants

A DCE was developed to assess preferences for different antibiotic treatments when considering antibiotic resistance as an attribute of this treatment.29 Antibiotics are a commonly prescribed medical treatment for bacterial infection and, like any medical treatment, they are often described in regards to treatment outcome making them an ideal subject for this preference task.30–32 The participant group was recruited via Dynata, a commercial survey panel provider. The sample was drawn from the general Swedish population and was nationally representative in terms of age, gender, education, and geographic region. The inclusion criteria for the study were that the participant was 18–65 years of age, proficient in Swedish language, and self-reported being medically able to take antibiotics. Respondents were excluded if they could not take antibiotics (eg, due to allergies). Participants were recruited until the desired sample size (N=350 in each arm) was reached as this should provide sufficient power for the analysis.33 Informed consent was obtained from all participants before starting and prior to completing the survey.

This study was planned in adherence to Swedish research regulations and was evaluated and approved by the Uppsala Regional Ethical Review Board (Dnr 2018/293) and complies with the Declaration of Helsinki.

Attributes, Levels and Experimental Design

The antibiotic treatments were described using five different attributes: Contribution to Antibiotic Resistance, Duration of Treatment, Likelihood of Side-Effects, Treatment Outcome, and Cost. The attributes were developed using a qualitative process involving literature reviews and four focus groups including ranking exercises with members of the general population (13 women/10 men, mean age = 38 years, age range 20–81 years). The attributes generated were then reduced and refined in discussion with research colleagues in accordance with best practice guidelines.9,14,29 The levels for the attributes Duration of Treatment, Likelihood of Side-Effects, Treatment Outcome, and Cost were derived based on characteristics of the most commonly prescribed antibiotics used in clinical care in Sweden. Costs were included to reflect the co-pay insurance system where patients pay up to a fixed amount for medications each year which is the standard in Sweden. The levels for Contribution to Antibiotic Resistance were derived based on the report from the Swedish public health authority regarding current levels of antibiotic resistance.34,35 The final list of attributes and levels was reviewed in eight stakeholder interviews (N=4 patients, N=1 expert on antibiotic resistance, N=1 nurse, N=2 general practitioners). All attributes were described identically for all participants except for Treatment Outcome.

In order to address the research question, three DCEs with different frames were designed for the attribute of Treatment Outcome. Treatment Outcome was framed as Failure Rate (Arm 1), Effectiveness (Arm 2), or both Effectiveness and Failure Rate combined (Arm 3). The levels for this attribute were logically identical across the three study arms, while being framed in opposite terms. For example, if in Arm 1 the Failure Rate is described as “5%”, then in Arm 2 the Effectiveness is “95%”, and in Arm 3 “95%/5%”. Thus, all treatment options were fundamentally the same even if framed in different ways. The attributes included can be found in Table 1 along with the full description presented to the participant and the levels for each attributed.

Table 1 Attributes and Levels for Each Attribute Used in the Choice Tasks Along with Explanatory Text as Presented to the Participants

For each Arm frame, the DCE design was optimized using a Bayesian D-efficient design created with Ngene 1.0 (ChoiceMetrics, 2011). The initial prior preference information was generated using best guess estimates based on literature review and expert opinions (N=1 pharmacist, N=1 nurse, N=2 general practitioners). The choice tasks consisted of a forced-choice in which a participant had to choose between two unlabeled alternatives (“Antibiotic A” or “Antibiotic B”). The decision to use a forced choice instead of an unforced choice with an opt-out alternative was based on the methodological needs of this study and desire to increase the efficiency of the design.36 All participants were asked to imagine that they were planning to take antibiotics to treat a bacterial infection. The two alternatives were presented as choices the doctor gave them to treat the infection. An example choice task can be found in Figure 1.

Figure 1 Example choice task using a treatment failure framing with pop-up window.

A pilot test of the complete survey was conducted in February 2019. The pilot test consisted of 129 respondents evenly split between the different arms and recruited using the same methods and research population as the final survey. Coefficients to be used as Bayesian priors for the experimental design of the final DCE were derived from the output of multinomial logit (MNL) models fitted to the pilot data. The final Bayesian D-efficient design consisted of 48 unique choice tasks divided over 3 blocks of 16 choice tasks for each arm (respondents were randomly assigned to these blocks).


The full questionnaire consisted of three different sections. The first section assessed demographic information such as age, gender, highest attained educational level, and occupation. In addition to these, two validated sets of questions were used to determine respondent’s health literacy (the Communicative and Critical Health Literacy Scale – Swedish Version; S-CCHL)37 and numeracy (the 3-Item Version of the Subjective Numeracy Scale; SNS-3).38 Health literacy and numeracy reflect the participants’ ability to utilize health information presented in text form or numeric form (respectively). The S-CCHL assesses health literacy using five items on a five-point Likert scale (“never (1)” to “always (5)”. The SNS-3 consists of three items on a six-point Likert scale from “not good at all/never (1)” to “extremely good/very often (6)”. Overall scores for each scale were generated by averaging their responses. Participants were then categorized as follows: scores 1/2 were classed as “inadequate”; those who had at least one score of 3 in the S-CCHL and 3/4 in the SNS-3 were classed as “problematic”; and those who consistently scored 4/5 in the S-CCHL and 5/6 in the SNS-3 were classed as “sufficient”. These scores were used to ensure the comparability of the samples in regard to their ability to comprehend written and numerical health information. Finally, six questions were used to assess experience with antibiotics, knowledge about antibiotic resistance, and self-reported health status.

The second section of the survey consisted of the DCE. Participants were instructed to imagine that they had a non-life-threatening bacterial infection and their doctor wants to prescribe antibiotics to treat the infection and avoid complications. Participants were randomized to receive a survey with Treatment Outcome framed as either treatment effectiveness, treatment failure, or a combination of both. Prior to answering the choice tasks, respondents received descriptions of bacterial infections, antibiotic treatments and antibiotic resistance. Additional information explaining the attributes and levels was available in a pop-up window which the participant could view during the task by hovering over the attribute. The attributes of Likelihood of Side-Effects and Treatment Outcome were presented as a percentage in the choice task and as a text and an icon array of these percentages in a pop-up box for further explanation to ensure participant comprehension.19 See Figure 1 for an example choice task with a pop-up window.

The final section consisted of short questions that assessed the participant’s evaluation of the questionnaire in regard to length and difficulty on a 5-point Likert scale along with an optional comments field. The questionnaire was developed using Light House Studio 9.6.1 software. The entire survey was pretested test to assess comprehensibility by eight stakeholders. This pretest resulted in two changes: rewording a knowledge related question, and the addition of an exclusion criterion regarding ability to use antibiotics.

Statistical Analysis

All analyses were conducted using STATA 14 clogit, and the mixlogit packages. A significance level of 0.05 was used for all analyses. The reference case used for all analyses was an antibiotic treatment with low contribution to antibiotic resistance, a 3-day treatment duration, 1% likelihood of side-effects, high effectiveness/low failure rate/ (95%/5%), and cost of €10.

A straight-line test was done to see if participants only chose either the left or right alternative. Potential scale parameter differences between data of the three arms were assessed using a Swait and Louviere test.39 Three tests were done comparing the three model pairs (Failure-Effectiveness, Failure-Combined, Effectiveness-Combined) following the same procedure: the log likelihood of the MNL model was fitted separately in all three frames and those were tested against the log likelihood of the MNL model for the different pooled data sets, which accounted for potential scale parameter differences. The hypothesis of equal attribute-level estimates was rejected in all comparisons by means of the chi-squared test.

Parameter coefficient estimates and importance value scores for the different arms were then derived using a two-step process. First, a logit model was used to identify significant main effects, interaction effects, linearity of parameter estimates across attribute levels, and left-right bias.40 No attributes were found to be linear across all three arms; thus, each attribute level was added to the model as a single parameter using dummy coding to allow for easy interpretation of results and willingness-to-pay estimates were not possible to calculate.41 All possible attribute level/framing-arm interaction terms were checked in an initial logit model (eg, Effectiveness* Duration: 7 days; Failure* Duration: 7 days; Combination* Duration: 7 days). Interaction terms that did not significantly add to the model were removed using backwards elimination. A mixed logit model (MIXL) was used for this analysis as it allows for the inclusion of random effects parameters accounting for naturally occurring heterogeneity of preferences in samples and results in more accurate model estimations.42 A MIXL model was built for each individual arm as well as the entire population in order to show the preference differences and test for interaction effects. All attributes were found to have significant heterogeneity at some level and were thus all included as random parameters. Each model was built using 14,000 Halton draws to ensure robust results.43 The final MIXL model can be found in Equation 1.

In this model, the value (U) of alternative a or b in a specific decision context (c) for an individual (i) is derived as the sum of the attribute-level estimates indicating the importance value of each attribute level (), plus the specific parameter interaction term coefficients for the arm (a) of that individual (). The stochastic factors for this alternative are included in the utility function as a random error term ε. Importance value scores (IVS) and choice predictions were derived from the results of the MIXL models. The IVS were generated by standardizing the score of the attribute with the greatest absolute utility difference between the least preferred level and most preferred level to 1.44 The attribute with the greatest absolute utility for the Failure and the Effectiveness arms was Contribution to Antibiotic Resistance and for the combination arm it was Costs.

In order to compare the differences as they may be used in an applied setting, hypothetical choice predictions were calculated as proportion of exponentiated utility attributed to an alternative from the total exponentiated utility present in a choice scenario (see Equation 2). In this equation, the uptake probability is calculated as the mean uptake of alternative (V) for an individual when asked to choose between this alternative or another (W). The choice predictions presented show the likely uptake if a participant was presented with either the most desirable antibiotic for their specific frame, the least desirable antibiotic for their specific frame, or an amoxicillin proxy. Amoxicillin was chosen as it is one of the most commonly prescribed antibiotics for outpatient usage.45 For the purposes of this calculation, amoxicillin was described based on descriptions found in published literature. This description is having a high contribution to antibiotic resistance,46 a treatment duration of 7 days, 10% of the users experiencing a side effect,47 an 85%/15% effectiveness/failure rate,48 with low costs.49 In addition to this, individual probabilities were generated per person based on the individual parameter estimates to see what the likely individual choice predictions would be.


Study Population

In total, 1124 completed responses were received. A straight-line test resulted in five responses being excluded due to selecting all right or all left alternatives; thus, the final analysis included 1119 responses. A summary of respondent demographic information can be found in Table 2. Participants generally found the survey acceptable in regards to length (M=2.72, SD=5.94) and not difficult (M= 3.42, SD=0.859) with no significant differences found between the groups (F(2)= 0.145, p=0.865) and F(2)=0.71, p = 0.932, respectively. The percentage of people who showed dominant decision-making, in which their choices were always associated with the best level of one specific attribute, did not significantly differ across arm and was 19%, 22% and 23% for the effectiveness, failure rate and the combined arm, respectively. In all arms, these lexicographical preferences were mostly registered for the resistance attribute and the cost attribute.

Table 2 Respondent Demographic Information

Preference Estimates

The preference estimates resulting from the MIXLs for each individual arm can be found in Table 3. All attributes were found to have significant parameter estimates for at least one level of each arm. Significant heterogeneity was found for at least one level of each parameter in the MIXL model in each arm indicating that the sample participants were highly varied in how they valued the different attributes when assessing antibiotics. The exception to this was the attribute of Treatment Outcome in the Effectiveness arm.

Table 3 MIXL Model Results for the Three Different Arms

In the pooled sample, no significant interaction effect was found for the attribute of Treatment Outcome despite this being the attribute which was framed differently between the arms. Significant interaction effects were found between the way that Treatment Effectiveness was framed and the attributes of Likelihood of Side-Effects and Treatment Duration. The Likelihood of Side-Effects was significantly more important to participants who saw the effectiveness framing compared to respondents who saw the Failure or combined framing. For Treatment Duration, those who saw a Failure Rate frame used this attribute to a greater extent than Effectiveness or Combined Frame. Further, they viewed a treatment lasting 7 days as somewhat better than a duration of 3 or 14 days. The preference estimates resulting from the MIXLs for all arms together with the significant framing interaction terms can be found in Table 4.

Table 4 Attribute-Level Estimates for the MIXL Model of the Pooled Results from All Arms

The different arms had comparable IVS patterns when standardizing on the most important attribute per arm. Costs and Contribution to Antibiotic Resistance were found to have the highest IVS with individual effectiveness or failure frames valuing Contribution to Antibiotic Resistance slightly higher and those who saw the combination frame valuing costs slightly higher. After these, all three arms had similar attribute valuations with Likelihood of Side-Effects, Treatment Outcome, and Treatment Duration being the third, fourth, and fifth most important attributes. The primary difference was that the failure frame had very similar IVS for Likelihood of Side-Effects and Treatment Outcome while the combination and effectiveness frame had more pronounced differences between the attribute valuations. When the IVS were standardized on the attribute of Treatment Outcome, the differences in scale parameter become more visible as the scale of the Failure framing is significantly smaller than that of Effectiveness of Combined frames. The IVS are visualized in Figure 2.

Figure 2 Attribute IVS when standardized on most important attributed (left) or on Tx outcome (right).

Abbreviations: IVS, importance value scores; Tx, treatment.

Notes: Key: Light grey: failure framing; striped: effectiveness framing; dark grey: combined frames.

Choice Predictions

Choice predictions based on individual parameter estimates were similar between arms. The most preferred antibiotic was predicted to be chosen by 92.3–95.2% of the participants. The least preferred antibiotic option was predicted to be chosen by 0.3–0.5% of the participants when compared to the most preferred antibiotic. The Amoxicillin was predicted to be chosen by 4.2–7.3% of the participants when compared to the most preferred antibiotic. See Table 5 for predicted choice predictions by frame.

Table 5 Choice Predictions by Arm (Individual Uptake Probability %)


This study was the first that we know of that used a randomized design to compare the impact of framing when an attribute in a health-oriented DCE is framed as positive, negative, or a combination of different frames. This study showed that the impact of framing should not be evaluated simply in regards to the attribute in question. Rather, the way that an attribute is framed impacts the concurrent valuation of other attributes in a DCE, altering the utility of the alternative as a whole which is similar to previous findings.24 These findings reflect the foundational theory underlying DCEs that the coefficients represent not only the valuation of the individual attributes but the valuation of these attributes in relation to one another and the decision context in which they are being valued.7

In our study, those who saw the treatment outcome framed as effectiveness were more concerned with the likelihood of side-effects in their decision-making than those who saw treatment outcome framed in terms of failure. One possible explanation for this is a negativity bias where participants’ attention is drawn away from positively framed attributes to more negative ones.50,51 The potential presence of this bias is reflected in the IVS of the different frames. We found that participants who saw a positive frame (effectiveness) had a greater importance value for the negative attribute of likelihood of side-effects compared to those who saw a negative frame. For the negative framed participants, the importance value of these two negative attributes was essentially the same. A similar explanation could also be that participants were more averse to risk in a positive treatment context than in a negative treatment context. Thus, participants were more concerned with the risk of side-effects when they believed a treatment to be effective than when a treatment was believed to not be effective. This type of risk aversion also reflects previous psychological research.18 Interestingly, these significant differences occurred without a statistically significant difference in the valuation of the treatment effectiveness attribute itself.

This study builds upon existing research looking at how attribute framing impacts the preferences of participants when elicited in a DCE. The framing effects that we found were smaller than those found in other preference studies.23–26 The different preference estimates did not translate into largely different uptake rate projections. These comparable choice predictions would likely not lead to drastically different decisions being made by stakeholders who use this type of information to guide decision-making. Thus, the practical impact of this framing effect may be limited.

The question then is should anything be done to account for this effect. Druckman previously argued that presenting an attribute with both frames would serve as a type of baseline, eliminating the effect of presenting only one attribute.27 This was not supported in our study where the two most important attributes were reversed for the combination frame compared to the individually framed alternatives. Additionally, the combination framing did not seem to present a “middle of the road” solution as the attribute level estimates framing did not fall between the positive and negative frames for any attribute across all levels. This seems to imply that the combination frame is a new frame in and of itself rather than a way to compensate for differences in opposing frames and thus cannot be used to account for a framing effect. Howard and Salkeld proposed reviewing attributes for potential framing issues during the development of the DCE and adjusting for a framing effect in the study design by presenting both frames or randomizing to different presentations of attributes.26 Our study highlights issues with these recommendations as the presentation of multiple frames resulted in a new framing bias which did not necessarily reflect a middle point between the two frames across all attributes. This was especially evident for the importance value of the two most important attributes. Further, randomization to different frames between arms does not provide the relevant information needed to account for framing biases without increasing the number of responses needed for comparison. Randomization within a survey (presenting alternating frames between choice tasks) may be a way to see how large this framing effect is, but could not account for it and would require an increase in the number of choice scenarios needed to derive the requisite parameters that the participant would essentially need to complete two DCEs.

All of these suggestions treat framing effects as something to control for. It may be that the way that an attribute is framed fundamentally changes the decision context in a way that cannot and should not be corrected for. The outcomes resulting from stated preferences in these different contexts represent the “true” preferences of the participant in that context that they are presented with. Best practice guidelines for conducting a DCE say that attributes and levels should be developed through a qualitative process including contributions from representatives of all stakeholders who participate in or use the outcomes from preference studies.8,13,14 As this study has shown, an important aspect of this process should also be to identify attributes which may be sensitive to framing effects and see how the different frames impact their relevance to these stakeholders. These do not necessarily need to be used to correct for this effect, but rather should be used to understand the preferences in varied contexts which reflect the variance seen in consultation rooms and information presented to patients. A potential area for further research would be a qualitative study to assess why the different frames had the impact that they did. Understanding the working mechanism would give a better understanding of how the different frames impact valuation. This would also help to determine what actions, if any, should be taken in situations where a framing effect may occur. Without this qualitative information, we can only make conjectures about why different valuations were found in the different frames.

A strength of this study is that the randomization process resulted in three groups that were comparable to each other and the general Swedish population in regard to demographic and psychosocial aspects. This limits the chance that differences between the groups impacted the outcomes that were found and supports the generalizability of these findings to the general population. Further, the preference scenario used a medication which many people were already familiar with supporting the validity of the findings as participants had personal experiences to draw from. One limitation, however, is that relative lack of information on participants with a lower education levels or lower health literacy. While the demographic makeup was similar to the general Swedish population, the framing effect in these groups may be more pronounced. Another limitation is that the choice scenario was not necessarily realistic as patients are not often given different antibiotic treatment options by their treating physicians. Thus, the decision scenario was almost entirely hypothetical. Different framing effect outcomes may be found when samples consist of patients currently in treatment with decision contexts relevant to this care. One aspect that was not checked between the different arms was whether participants reinterpreted the outcomes to view them in different frames. That is, did a respondent who saw a treatment profile with 90% effectiveness see this in terms of the 90% effectiveness or as a 10% failure. While this is hypothetically possible, previous research has not shown that patients reinterpret frames in this way and we would have likely seen higher concordance between the combination arm and a single framed arm if this were the case. Future studies should consider qualitative research to see whether patients reinterpret these frames. Similarly, as this was a hypothetical situation it is not possible to know whether the participants would truly make the same choice if presented with these options in their actual care.


This study reaffirmed the impact that framing can have on preference outcomes. In a first of its kind design, three frames (including a combined frame) were used to evaluate the impact of framing on an attribute describing treatment effectiveness. While framing effects were found, the practical implications and interpretation of preferences outcomes was not drastically changed because of this bias. A theoretical discussion is needed to address how to move forward in light of these results. Specifically, addressing the question of whether framing effects should be controlled for, or simply better understood through improved qualitative research.


The authors would like to acknowledge Dr G. A. de Wit for her contributions in the manuscript writing and input on interpretation of outcomes. Ian P Smith and Mirko Ancillotti are co-first authors for this study. The abstract of this paper was presented at the ISPOR Europe 2020 as a poster presentation with interim findings. The poster’s abstract was published in “Poster Abstracts” in Value in Health:


There is no specific funding or support to report for this study.


The authors have no conflicts of interest to declare in regards to the content of this article.


1. De Bekker-Grob EW, Ryan M, Gerard K. Discrete choice experiments in health economics: a review of the literature. Health Econ. 2012;21(2):145–172. doi:10.1002/hec.1697

2. Clark MD, Determann D, Petrou S, Moro D, De Bekker-Grob EW. Discrete choice experiments in health economics: a review of the literature. PharmacoEconomics. 2014;32(9):883–902. doi:10.1007/s40273-014-0170-x

3. Soekhai V, De Bekker-Grob EW, Ellis AR, Vass CM. Discrete choice experiments in health economics: past, present and future. PharmacoEconomics. 2019;37(2):201–226. doi:10.1007/s40273-018-0734-2

4. Lancaster KJ, New A. Approach to Consumer Theory. J Polit Econ. 1966;74(2):132–157. doi:10.1086/259131

5. Manski CF. The structure of random utility models. Theory Decis. 1977;8(3):229–254. doi:10.1007/BF00133443

6. McFadden D. Econometric models of probabilistic choice. In: Structural Analysis of Discrete Data with Econometric Applications. MIT Press; 1981:198272.

7. Ryan M, Amaya-Amaya M. ‘Threats’ to and hopes for estimating benefits. Health Econ. 2005;14(6):609–619. doi:10.1002/hec.949

8. Hollin IL, Craig BM, Coast J, et al. Reporting formative qualitative research to support the development of quantitative preference study protocols and corresponding survey instruments: guidelines for authors and reviewers. The Patient. 2020;13(1):121–136. doi:10.1007/s40271-019-00401-x

9. Bridges JFP, Hauber AB, Marshall D, et al. Conjoint analysis applications in health—a checklist: a report of the ISPOR good research practices for conjoint analysis task force. Value Health. 2011;14(4):403–413. doi:10.1016/j.jval.2010.11.013

10. Lancsar E, Louviere J. Conducting discrete choice experiments to inform healthcare decision making. PharmacoEconomics. 2008;26(8):661–677. doi:10.2165/00019053-200826080-00004

11. Helter TM, Boehler CEH. Developing attributes for discrete choice experiments in health: a systematic literature review and case study of alcohol misuse interventions. J Subst Use. 2016;21(6):662–668. doi:10.3109/14659891.2015.1118563

12. Coast J, Al-Janabi H, Sutton EJ, et al. Using qualitative methods for attribute development for discrete choice experiments: issues and recommendations. Health Econ. 2012;21(6):730–741. doi:10.1002/hec.1739

13. Coast J, Horrocks S. Developing attributes and levels for discrete choice experiments using qualitative methods. J Health Serv Res Policy. 2007;12(1):25–30. doi:10.1258/135581907779497602

14. Kløjgaard ME, Bech M, Søgaard R. Designing a stated choice experiment: the value of a qualitative process. J Choice Model. 2012;5(2):1–18. doi:10.1016/S1755-5345(13)70050-2

15. U.S. Food and Drug Administration. Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims.Silver Spring, MD; 2009.

16. Lichtenstein S, Slovic P. The Construction of Preference. Cambridge University Press; 2006.

17. Slovic P. The construction of preference. Am Psychol. 1995;50(5):364. doi:10.1037/0003-066X.50.5.364

18. Tversky A, Kahneman D. The framing of decisions and the psychology of choice. Science. 1981;211(4481):453–458. doi:10.1126/science.7455683

19. Adrian Edwards GEJ. Presenting risk information a review of the effects of framing and other manipulations on patient outcomes. J Health Commun. 2001;6(1):61–82. doi:10.1080/10810730150501413

20. Visschers VHM, Meertens RM, Passchier WWF, De Vries NNK. Probability information in risk communication: a review of the research literature. Risk Anal. 2009;29(2):267–287. doi:10.1111/j.1539-6924.2008.01137.x

21. Gong J, Zhang Y, Yang Z, Huang Y, Feng J, Zhang W. The framing effect in medical decision-making: a review of the literature. Psychol Health Med. 2013;18(6):645–653. doi:10.1080/13548506.2013.766352

22. Peng J, Jiang Y, Miao D, Li R, Xiao W. Framing effects in medical situations: distinctions of attribute, goal and risky choice frames. J Int Med Res. 2013;41(3):771–776. doi:10.1177/0300060513476593

23. McNeil BJ, Pauker SG, Sox HC, Tversky A. On the elicitation of preferences for alternative therapies. N Engl J Med. 1982;306(21):1259–1262. doi:10.1056/NEJM198205273062103

24. Veldwijk J, Essers BAB, Lambooij MS, Dirksen CD, Smit HA, de Wit GA. Survival or mortality: does risk attribute framing influence decision-making behavior in a discrete choice experiment? Value Health. 2016;19(2):202–209. doi:10.1016/j.jval.2015.11.004

25. Grisolía JM, Longo A, Hutchinson G, Kee F. Comparing mortality risk reduction, life expectancy gains, and probability of achieving full life span, as alternatives for presenting CVD mortality risk reduction: a discrete choice study of framing risk and health behaviour change. Soc Sci Med. 2018;211:164–174. doi:10.1016/j.socscimed.2018.06.011

26. Howard K, Salkeld G. Does attribute framing in discrete choice experiments influence willingness to pay? Results from a discrete choice experiment in screening for colorectal cancer. Value Health. 2009;12(2):354–363. doi:10.1111/j.1524-4733.2008.00417.x

27. Druckman JN. Evaluating framing effects. J Econ Psychol. 2001;22(1):91–101. doi:10.1016/S0167-4870(00)00032-5

28. Armstrong K, Schwartz JS, Fitzgerald G, Putt M, Ubel PA. Effect of framing as gain versus loss on understanding and hypothetical treatment choices: survival and mortality curves. Med Decis Making. 2002;22(1):76–83. doi:10.1177/0272989X0202200108

29. Ancillotti M, Stefan Eriksson S, Andersson DI, Godskesen T, Nihlén Fahlquist J, Veldwijk J. Preferences regarding antibiotic treatment and the role of antibiotic resistance: a discrete choice experiment. Int J Antimicrob Agents. 2020;56:106198. doi:10.1016/j.ijantimicag.2020.106198

30. Shallcross L, Beckley N, Rait G, Hayward A, Petersen I. Antibiotic prescribing frequency amongst patients in primary care: a cohort study using electronic health records. J Antimicrob Chemother. 2017;72(6):1818–1824. doi:10.1093/jac/dkx048

31. Poole NM, Shapiro DJ, Fleming-Dutra KE, Hicks LA, Hersh AL, Kronman MP. Antibiotic prescribing for children in United States emergency departments: 2009–2014. Pediatrics. 2019;143(2):e20181056. doi:10.1542/peds.2018-1056

32. Fleming-Dutra KE, Hersh AL, Shapiro DJ, et al. Prevalence of inappropriate antibiotic prescriptions among US ambulatory care visits, 2010-2011. JAMA. 2016;315(17):1864. doi:10.1001/jama.2016.4151

33. de Bekker-Grob EW, Donkers B, Jonker MF, Stolk EA. Sample size requirements for discrete-choice experiments in healthcare: a practical guide. Patient. 2015;8(5):373–384. doi:10.1007/s40271-015-0118-z

34. Sweden TPHAo. Future costs of antibiotic resistance: final reporting of Government commission on direct and indirect costs and consequences of antibiotic resistance in Swedish health care; 2018.

35. Swedres-Svarm. Consumption of Antibiotics and Occurrence of Antibiotic Resistance in Sweden. The Public Health Agency of Sweden, National Veterinary Institute; 2018.

36. Veldwijk J, Lambooij MS, De Bekker-Grob EW, Smit HA, De Wit GA. The effect of including an opt-out option in discrete choice experiments. PLoS One. 2014;9(11):e111805. doi:10.1371/journal.pone.0111805

37. Wångdahl JM, Mårtensson LI, Communicative T. Critical health literacy scale – Swedish version. Scand J Public Health. 2013;42(1):25–31. doi:10.1177/1403494813500592

38. McNaughton CD, Cavanaugh KL, Kripalani S, Rothman RL, Wallston KA. Validation of a short, 3-item version of the subjective numeracy scale. Med Decis Making. 2015;35(8):932–936. doi:10.1177/0272989X15581800

39. Swait J, Louviere J. The role of the scale parameter in the estimation and comparison of multinomial logit models. J Mark Res. 1993;30(3):305–314. doi:10.1177/002224379303000303

40. Bansback N, Hole AR, Mulhern B, Tsuchiya A. Testing a discrete choice experiment including duration to value health States for large descriptive systems: addressing design and sampling issues. Soc Sci Med. 2014;114:38–48. doi:10.1016/j.socscimed.2014.05.026

41. Hauber AB, González JM, Groothuis-Oudshoorn CGM, et al. Statistical methods for the analysis of discrete choice experiments: a report of the ISPOR conjoint analysis good research practices task force. Value Health. 2016;19(4):300–315. doi:10.1016/j.jval.2016.04.004

42. Hensher DA, Greene WH. The mixed logit model: the state of practice. Transportation. 2003;30(2):133–176.

43. Train KE. Drawing from Densities. In: Train KE, editor. Discrete Choice Methods with Simulation. 2nd ed. Cambridge: Cambridge University Press; 2009:205–236.

44. Gonzalez JM. A guide to measuring and interpreting attribute importance. Patient. 2019;12(3):287–295. doi:10.1007/s40271-019-00360-3

45. Centers for Disease Control and Prevention, . Outpatient antibiotic prescriptions — United States, 2014 2014.

46. de Greeff S, Mouton J, Schoffelen A, Melles D, Mevius D, Natsch S. Consumption of Antimicrobial Agents and Antimicrobial Resistance Among Medically Important Bacteria in the Netherlands in 2015: NethMap 2016. Dutch Foundation of the Working Party on Antibiotic Policy (SWAB); 2016.

47. Gillies M, Ranakusuma A, Hoffmann T, et al. Common harms from amoxicillin: a systematic review and meta-analysis of randomized placebo-controlled trials for any indication. Can Med Assoc J. 2015;187(1):E21–E31. doi:10.1503/cmaj.140848

48. Henry DC, Riffer E, Sokol WN, Chaudry NI, Swanson RN. Randomized double-blind study comparing 3- and 6-day regimens of azithromycin with a 10-day amoxicillin-clavulanate regimen for treatment of acute bacterial sinusitis. Antimicrob Agents Chemother. 2003;47(9):2770–2774. doi:10.1128/AAC.47.9.2770-2774.2003

49. Sjöberg D, Johansson O, Grundström U, Dalin S. International Price Comparison 2019: An Analysis of Swedish Pharmaceutical Prices and Volumes Relative to 19 Other European Countries. Dental and Pharmaceutical Benefits Agency; 2019.

50. Rozin P, Royzman EB. Negativity bias, negativity dominance, and contagion. J Pers Soc Psychol. 2001;5(4):296–320. doi:10.1207/S15327957PSPR0504_2

51. Baumeister RF, Bratslavsky E, Finkenauer C, Vohs KD. Bad is stronger than good. Rev Gen Psychol. 2001;5(4):323–370. doi:10.1037/1089-2680.5.4.323

Creative Commons License © 2022 The Author(s). This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.