Using qualitative research to facilitate the interpretation of quantitative results from a discrete choice experiment: insights from a survey in elderly ophthalmologic patients

Introduction

Age-related macular degeneration (AMD) is the leading cause of visual impairment and blindness in industrialized countries. Approximately 8.7% of people worldwide are affected by the disease. After the age of 75, prevalence increases rapidly up to 35%.¹ While the neovascular form of AMD accounts for only 10% to 15% of all AMD cases, it accounts for around 90% of respective legal blindness. AMD is characterized by choroidal neovascularization, meaning that leaky blood vessels grow into the retina. The progression of disease and loss of central vision can develop within only several months if the disease is left untreated. Therefore, early initiation of treatment is essential.^2,3 Currently, the drugs ranibizumab (Lucentis; Novartis International AG, Basel, Switzerland) and aflibercept (Eylea; Bayer AG, Leverkusen, Germany) are approved for the treatment of AMD in Germany. Additionally, the drug bevacizumab (Avastin; Hoffman-La Roche Ltd., Basel, Switzerland) is not approved for AMD treatment, but is widely used, especially in the ambulatory care setting.⁴ Its use is even expected to increase due to recent changes in regulations favoring its reimbursement. Drug costs in Germany vary between <50 Euro (<55 US dollars) per injection for bevacizumab and about 800–1,000 Euro (about 870–1,100 US dollars) for ranibizumab and aflibercept. All drugs are administered as intravitreal injections targeting VEGF, but differ regarding monitoring and injection schemes. While ranibizumab requires monthly monitoring and injections in case of recurrent disease activity (pro re nata [PRN]), aflibercept is the first drug that is administered bimonthly with bimonthly monitoring visits.^5,6 As approval studies are missing for bevacizumab or were underpowered to detect differences in side effect frequencies for the other two drugs, the comparative safety of treatments continues to be an issue for research and discussion.⁷

Since early initiation of treatment and adherence impacts the AMD treatments’ effectiveness, patients need to be well informed about the treatment alternatives to enable truly informed decision making and acceptance of the selected therapy. Addressing all patient relevant treatment characteristics, including process, as well as outcome-related characteristics is necessary. Understanding patient preferences can support physicians to structure and prioritize the information they provide to their patients before making treatment decisions. Two studies by Droege et al investigated patient preferences for specific characteristics of AMD treatments.^8,9 However, these studies only focused on preferences for monthly versus PRN injection schemes, and the acceptance of monthly monitoring. Owing to the introduction of aflibercept in 2012, preferences for less frequent injection and monitoring schemes should be considered as well. Moreover, additional treatment attributes might be relevant to patients. Therefore, the objective of the present study was to identify patient relevant treatment attributes and to quantify patients’ preferences by means of a discrete choice experiment (DCE). Also, the study highlights specific methodological challenges for conducting a DCE in an elderly and visually impaired population.

Methods

The conduct of this study was based on the steps described by the Conjoint Analysis Task Forces of the International Society for Pharmacoeconomics and Outcomes Research.^10,11

Identification and selection of attributes and levels

Seven potentially patient relevant treatment attributes were identified by a review of clinical trials, approval documents, summaries of product characteristics, and two ophthalmologist consultations. These included “side effects”, “effect on visual function”, “approval status”, “injection frequency”, “monitoring frequency”, “costs for the patient”, and “costs to statutory health insurance”. As seven attributes were considered too many to create cognitively appropriate choice tasks for the specific patient population, finding the balance between comprehensive descriptions of treatments without overwhelming participants was necessary. Five focus group interviews including 21 patients were performed to select and frame the most patient relevant attributes.^12–14 For each attribute patients were first asked to mention their spontaneous associations with it. Subsequently, participants received a detailed explanation on the respective attribute and again were asked to comment on the relevance of each attribute with regard to their personal treatment decisions. The initial open discussion enabled the researchers to identify the wording used by patients, the second phase provided insights into the ranges of levels, which patients considered relevant for each attribute. Each focus group interview was rounded off with a ranking exercise during which participants sorted the seven attributes from most to least important. As patients had difficulties in ranking seven attributes, they were asked to sort out the two least important ones and only give a specific order to the remaining five. This ranking exercise suggested that five attributes were still manageable for participants. Based on the ranking exercise and the qualitative evaluation of attribute importance the following attributes and their respective levels were selected for inclusion in the DCE study: “side effects”, “effect on visual function”, “approval status”, “injection frequency”, and “monitoring frequency” (Table 1). Further details of the focus group study are published.¹⁴

Table 1 Attributes and levels included in the discrete choice experiment Note: *Description used for the medical concept pro re nata. Abbreviation: AMD, age-related macular degeneration.

Development of choice tasks

To minimize cognitive demands of the experiment for our visually impaired patient population, participants had to choose between only two treatment options in each choice task. In the present study, two attributes with two levels, two attributes with three levels, and one attribute with four levels resulted in:

22 * 32 * 41=144

(1)

treatment descriptions. Using all pairs of these 144 treatment descriptions in the experiment would imply more than 20,000 possible choice tasks for patients. To select only a subset of all possible choice tasks, a Bayesian efficient design was developed with the help of Ngene 1.1.1 (www.choice-metrics.com). The design maximizes the precision of parameter estimation for a selected number of choice sets by maximizing the D-efficiency – a summary measure of the variance-covariance matrix of the estimated parameters.¹⁵ Information on patients’ preference structures based on the insights from preceding focus group interviews were used to develop a Bayesian efficient design. Prior assumptions were preferences for approved drugs, for fewer injections and for improvement of visual functioning compared to stabilization. To reflect currently available treatment schedules, the level “injection on demand, following monthly monitoring” only appeared in combination with the level “monthly monitoring” (recommended schedule for ranibizumab). An opt-out option was not offered to patients since untreated neovascular AMD may ultimately (and often rapidly) lead to vision loss. Therefore, “no treatment” did not seem to be a realistic choice, especially for patients already undergoing treatment. Also, a standard care option was not offered since the standard care for AMD is intravitreal injections. Each participant received 12 choice tasks, which seemed a reasonable compromise between trying to minimize the cognitive and time-related demands for patients, and to ensure generation of a sufficient number of observations for data analyses. Additionally, a duplicate of one choice set was added at the end of the choice tasks to test for consistency in patients’ judgments. The generic labels “treatment A” and “treatment B” were used to prevent participants from purely basing their choice on treatment names, especially since the focus group interviews had revealed patients’ preference for familiar treatments. The order of attributes differed between questionnaires to prevent bias in preference estimates due to ordering effects. An example of a choice task is displayed in Figure 1.

Figure 1 Example of a choice task.

Instrument design

Next to selecting patient relevant attributes, focus group interviews were conducted to get familiar with the elderly and visually impaired patient population. Specifically, the questionnaire design and data collection strategy had to be tailored to the specific needs of the target population. These included, eg, using a clear and simple questionnaire layout with large font sizes.¹⁴ The questionnaire consisted of three parts. First, participants received a general introduction to the study and questions on socio-demographic and disease-specific data. Second, a description of the choice situation was provided, followed by the explanation of attributes and levels. These descriptions were whenever possible aligned with the expressions patients used during the focus group interviews. The final part consisted of the choice tasks. The questionnaire was finalized by asking patients to rate the DCE task difficulty on a five-point scale from easy to difficult. Initial pilot tests with two patients revealed that patients were not prepared to make treatment decisions on their own. To get patients in the mind-set of choosing based on their own preferences, the choice task was explained to patients using a real-life example of having to choose between two dishes. This “real-life example” made people feel more at ease with the task they had to perform and encouraged them to think about their personal preference, instead of repeating what others, especially their treating physician, would have said. After including this example, two additional pilot tests did not reveal any further problems in understanding the task or the attribute and level descriptions. The questionnaire was administered in German and is available upon request (see Supplementary material for the translated English version).

Data collection and participants

Data were collected by means of an interviewer-assisted paper–pencil questionnaire. During the pilot tests and ten interviews using the finalized questionnaire, the two interviewers observed and evaluated each other in order to ensure adherence to a standardized interviewing procedure. While answering the questionnaire, patients were encouraged to “think-aloud” and state their considerations during decision making. Interviewers read out the questionnaire to some patients and provided practical support if needed (eg, turning pages), especially to patients whose visual functioning or fine motor skills were insufficient. For patients unable to read who had to memorize treatment combinations and complete choice sets, researchers had to repeatedly read choice tasks out loud. The minimum sample size for analyzing main effects was estimated to be 84, based on Orme’s rule-of-thumb:

(2)

where n= number of participants, t= number of choice tasks, a= number of alternatives per choice task, and c= maximum number of levels within an attribute.¹⁶ Data collection took place between April 2014 and March 2015 at the Centre of Ophthalmology at Cologne University Hospital. Consecutive patients were recruited at the Centre for Ophthalmology. Patients with a diagnosis of neovascular AMD established through optical coherence tomography, a minimum best corrected visual acuity letter score of 5%, treatment experience of at least one intravitreal injection, sufficient command of the German language, good overall health, and those insured by the statutory health insurance (SHI) were included in the study. Patients were excluded if they were also diagnosed with diabetes, diabetic macular edema or retinal vein occlusion. Study nurses invited patients via telephone to participate in the study. Appointments for completing the questionnaire were scheduled preceding patients’ injection appointments. Interviewers took field notes to keep track of patients’ remarks from the think-aloud task. Ethical approval for this study was granted by the institutional review board of the University Hospital Cologne. Written informed consent was not required as the questionnaire was completely anonymous.

Data analyses

The analysis of the DCE is based on the framework of random utility theory. The utility a patient derives from a treatment is assumed to consist of a systematic component based on the levels that describe a treatment and a random component, which cannot be observed. Preferences for each attribute level were modeled with a mixed multinomial logit model (MMNL), which models the utility (V) of a treatment as:

(3)

with β₁ to β₉ representing the coefficients for each attribute level and η₁ to η₉ representing the corresponding individual preference-variation. The residual error is denoted by ε and captures the unmeasured error in choices including, eg, psychological or behavioral determinants of choice. The resulting coefficients represent the mean part-worth utility of treatment characteristics as judged by the respondents. All variables were effects coded for the analysis. The coefficients for the omitted levels were calculated as described by Bech and Gyrd-Hansen.¹⁷ Positive coefficients of parameters indicate the relative preference for the respective level in comparison to the mean of all levels within the attribute. The corresponding confidence intervals were obtained by calculating the variance of a sum of correlated variables,¹⁸ and the corresponding P-values were calculated as proposed by Altman and Bland.¹⁹ The panel nature of data, which results from the dependency between each participant’s choices, was taken into account. The MMNL is not based on the assumption of homogeneous preferences within the population, but allows for preference heterogeneity. In the analysis, all parameters were treated as random and estimated based on 2,000 Halton draws from a normal distribution. McFadden pseudo R² was used to evaluate the model fit. A model including only the constant was estimated to investigate potential left-to-right bias in the participants’ choices. Consistency of participants’ answers was checked with a test–retest question and a separate model, including only participants who passed this question, was estimated. Using the described model specifications,³ exploratory subgroup analyses were performed for subgroups based on sex, age group, and injection experience. Choice data from the DCE were analyzed with Nlogit 5 (Econometric Software, Inc., Plainview, NY, USA). Field notes, containing the data from the think-aloud task, were analyzed independently by the two interviewers. Commonly mentioned reasons that guided patients’ choices were summarized for each attribute. Also, general comments relating to treatment decisions were summarized.

Results

Sample characteristics

Of the 141 patients eligible and invited for participation, 86 completed the DCE questionnaire. Active inquiries were not made regarding reasons for refusal to participate, but commonly communicated reasons included general unwillingness to participate in research, acute health problems or time/logistical constraints. For two participants the DCE task was too demanding and had to be abandoned. Respondent characteristics are displayed in Table 2.

Table 2 Patients’ characteristics (N=86) Abbreviations: N, total number of participants; n, number of participants in subgroups; AMD, age-related macular degeneration.

Preference analysis

The choice tasks were rated as rather easy by 25 (29%), and as rather difficult by 21 (24%) respondents. Forty-five percent of respondents considered the task difficulty as neutral. As all 86 patients completed the 12 choice tasks, 1,032 choice observations were included in the preference analysis. No indication for a left-to-right bias could be detected, as an equal number of choices were made for alternatives A and B, reflected in the model constant being non-significant (data not shown). The results of the MMNL model of preferences are displayed in Table 3. Participants preferred to have an improvement instead of reaching stabilization of their visual function (P<0.05). While patients favored monitoring visits every 2 months (P<0.05), they least preferred to have monitoring visits only every 4 months (P<0.05). Approved drugs were significantly preferred over non-approved drugs (P<0.05). Patients preferred bimonthly and PRN injection schemes over monthly injections, with the PRN injection scheme being most preferred (P<0.05), followed by the bimonthly injection scheme. Patients especially disliked having injections every month. When comparing the types of side effects to each other, mild–moderate, frequent eye-related side effects were considered as the most acceptable type, whereas very severe, very rare systemic side effects were rated as least preferable (P<0.05).

Table 3 Results from panel mixed multinomial logit model Notes: Number of observations =1,032 (86 respondents ×12 choice tasks). Pseudo R2: 0.18; Akaike information criterion: 1.17. Abbreviation: CI, confidence interval.

The range of the estimated coefficients as displayed in Figure 2 indicates that in this particular choice situation patients were most sensitive to changes within the attributes “side effects” (range =1.69) and “injection frequency” (range =1.28), followed by the “approval status” (range =1.12) and “monitoring frequency” (range =0.62). Patients valued the differences between levels within the attribute “effect on visual function” as least important (range =0.44).

Figure 2 Patient preference estimates.

Significant standard deviations, which reflect preference heterogeneity among participants, were estimated for all levels except bimonthly monitoring and rare eye-related side effects (see Table 3). Seventy-eight participants (91%) made consistent treatment choices in the test and retest question. Excluding participants who did not give the same answer to the test–retest question from the analysis, did not alter the results. Separate models for each interviewer did not show significant differences between preference estimates.

Insights from think-aloud exercise

The majority of participants explained their choices in relation to their personal experiences while answering the tasks. Examples of patient statements given during completion of the questionnaire are provided in Table 4. For the attributes “injection” and “monitoring frequency”, patients stressed the importance of having these neither too frequently nor too infrequently. On the one hand, too few monitoring visits were considered a risk of leaving disease deterioration undetected, on the other hand, too many monitoring visits were considered too time-consuming. Similarly, patients considered having many injections (ie, monthly) inconvenient and were afraid that too few injections (ie, every 4 months) would lead to under-treatment. Disfavor for non-approved drugs was explained by being unwilling to participate in clinical trials in general. Patients who participated in clinical trials expressed their indifference regarding the drug’s approval status, as they were aware that non-approved drugs can also be effective and safe. Regarding the effect on visual function, patients explained that both stabilization and improvement represent a valuable treatment effect, and only few patients stressed that nothing but an improvement of visual function would be valuable and relevant for them. Several specific personal experiences seemed to influence intense consideration of one attribute. These were, eg, fear of injections, which led to strong preferences for fewer injections. However, patients in this context also stressed that they considered intravitreal injections an unwanted but urgently needed type of treatment, which they were willing to accept after diagnosis. The wish to avoid the risk of systemic side effects was often explained by having had a stroke or heart attack in the past and being scared of a recurrence. Being asked for personal judgments on treatment options initially was challenging for patients. Nevertheless, they appreciated being informed about various treatment characteristics and took the opportunity to state their personal opinions, as this was not regularly possible during clinical routine in such detail.

Table 4 Examples of patients’ statements

Discussion

The present study elicited patients’ preferences for five treatment attributes and their levels, which characterize intravitreal injections of anti-VEGF drugs for the treatment of neovascular AMD. The analysis revealed that patients prefer treatments which improve visual function, are approved, are administered in a PRN scheme, and are accompanied by bimonthly monitoring visits. Patients least preferred to be at risk of very severe, very rare systemic side effects.

Disfavor regarding monitoring and injection monthly and every 4 months was explained clearly by patients in the think-aloud exercise. While they generally preferred to have fewer appointments, they doubted equal effectiveness of treatments with lower injection frequency and were afraid that current disease activity would be missed with longer examination intervals. While Droege et al^8,9 reported a high acceptance of monthly monitoring visits, our results suggest that patients would prefer monitoring every 2 months, as this represents a compromise between less appointments and regular reassurance.¹⁴ Consistent with Droege et al,⁸ a preference for a PRN scheme compared to monthly injections was found. The strong preference for a PRN scheme might be explained by patients’ familiarity with this scheme, which is the primary injection scheme at the University Hospital Cologne. Prior focus group interviews and the think-aloud exercises revealed the patients’ anxiety regarding undetected disease deterioration and their preference for prompt treatment in case of deterioration, which they assume is best achieved in the PRN treatment scheme. In addition to Droege et al, our study elicited preferences for the recently introduced treatment scheme of bimonthly injections and the additional treatment characteristics: side effects, approval status, and effect on visual function.

The ranges of preference coefficients illustrated that patients were least sensitive to changes within the attribute “effect on visual function”. This seems counterintuitive as patients in previous studies stressed the high importance of their visual function.¹⁴ Nevertheless, patients also explained that either stabilization or improvement would represent a great achievement. Only few patients with very severe visual impairments regarded improvement as the only relevant option. This example illustrates that the relative importance estimated by the coefficient ranges of attributes can only be interpreted in relation to the levels describing a particular choice situation, and does not represent a value for the absolute importance of an attribute. The small difference between these levels might also result from the qualitative, and therefore potentially vague attribute definition. Quantitative definitions such as percentage of visual functioning or number of lines that can be read were tested in preceding focus group interviews. However, these were not meaningful to patients. Patients described their desired treatment goals in maintaining or improving their functioning, eg, being able to read rather than the wish for reaching a specific percentage of visual functioning. Moreover, based on the treatment experiences reported by patients during the preparatory qualitative work, it became clear that the same objectively measured increase in visual acuity does not mean the same to all patients. While some patients subjectively perceived very poor visual functioning with a visual acuity of, eg, 20%, others reported good visual functioning with the same percentage of visual acuity. Therefore, universal quantitative definitions of visual functioning did not appear to be meaningful to patients and would have left even more room for individual interpretation.

This study illustrates the added value of combining quantitative and qualitative preference information. While quantitative preference data only provide information on the strength and direction of preferences, qualitative information helps to interpret and understand the numerical data.²⁰ Even though statistical analyses display results, which can be explained by the researcher, receiving the explanations from patients facilitates the evaluation of the study’s validity at the input and output level.^20,21 At the input level, misunderstandings of the task or of the included attributes and levels can be detected and eliminated. At the output level, researchers can evaluate whether the preference estimates correspond to the preferences expressed by patients in think-aloud or preceding focus group statements. Without the qualitative research conducted in advance and in parallel, some findings would not appear reasonable at first sight. Especially for the attributes “monitoring” and “injection frequency”, a linear relationship with the lowest possible frequency being most preferred over the more frequent intervals could be expected, especially since intravitreal injections are not attractive to patients and both monitoring and injection visits, are time-consuming. In contrast, the think-aloud task in our preference elicitation revealed that while patients generally prefer to have less injection and monitoring visits, the majority of patients are afraid of undetected and untreated disease activity and consider these a disliked but necessary need.

Several limitations of the present study should be addressed. First, generalizability of results might be limited due to exclusively eliciting preferences of treatment experienced patients. Preferences from treatment experienced and treatment-naïve patients might differ, as indicated by preparatory focus group interviews.¹⁴ However, previous qualitative research showed that already, after receiving few injections, the majority of patients get used to their treatment. Recruiting patients only from the ophthalmologic center of the University Hospital Cologne, and only those insured with the SHI might limit the generalizability of the estimated preferences for a wider patient population. Especially, the preference for approved drugs found in this study might be overestimated. As AMD patients at Cologne University Hospital were treated almost exclusively with approved drugs, patients might intentionally have chosen to undergo treatment at this center. However, the majority of patients were not familiar with the concept of drug approval and needed detailed explanations. In turn, providing detailed information might have influenced patients’ judgments and induced an overestimation of the preference for approved drugs. Nevertheless, thorough descriptions of all treatment characteristics are necessary to enable truly informed choices. Generalizability of the results might also be limited as only patients insured with the SHI were included. As costs for AMD treatments are now fully covered by the German SHI, patients do not have to make copayments for these treatments. SHI-insured patients also stressed that they do not consider treatment costs a determinant of treatment choice as long as these are covered by the SHI.¹⁴ Including a cost attribute and estimating willingness-to-pay might have been of interest in international studies or when treatments were not yet covered by the German SHI. Another limitation might result from collecting data with interviewer-administered questionnaires. Efforts to prevent any influence from interviewers included, eg, adherence to an interviewing procedure and mutual evaluation of interviewing practices. Despite its potential for introducing bias, such as evoking socially desirable choice behavior, the interview-based data collection offers the opportunity to survey patients who would otherwise be unable to complete a survey on their own. In our study patients who were unable to read due to their visual impairment, or those with limited fine motor skills and being unable to turn the pages of the questionnaire would have been excluded from the study when choosing other data collection techniques. Using interviewer-assisted questionnaires therefore contributed to increasing representativeness of the overall patient population. Moreover, interviewer assistance provides the opportunity to identify misunderstandings of the task at early stages in questionnaire administration and enables the researcher to gain more detailed insights into the patients’ reasoning behind their preferences. Completing the task with a qualified and neutral interviewer also minimizes the bias that might be introduced if, eg, family members get involved in completing the questionnaire when it is given to patients for self-completion.

Despite reaching the estimated minimum sample size of 84, the low sample size of this study could represent a further limitation. The insignificant differences between several levels within the attributes “monitoring” and “injection frequency” might be an artifact of the small sample size, implying a lack of statistical power. The sample size calculation proposed by de Bekker-Grob et al might be more precise given that prior preference information is taken into account. However, the validity of this calculation largely depends on the accuracy of a priori estimates of coefficients, which might not always be given.²² In our case, the sample size generated with de Bekker-Grob’s approach gave very similar results to Orme’s formula, except for one level, where a much higher (<1,000 participants) sample size would have been needed. Since significant preference estimates could be obtained with the relatively small sample size in this study and interviews were demanding for both patients and researchers, including more patients in the study would most likely not have been efficient.

Despite its limitations, this study could have several implications for clinical practice and research. The study shows that patients with AMD have strong preferences for or against various treatment characteristics. In addition to data from clinical trials, expert experiences, and financial considerations, patient preferences should be integrated into medical decision making.²³

Specifically, the variation in individual preferences found in this study underpins the need to actively elicit and include patient preferences in patient–physician communication and decision making. The study also suggests the importance of giving special care to the target population’s needs when designing preference analyses. Tailored instrument design, data collection techniques, and interviewer assistance can prevent exclusion of participants due to their physical or cognitive functioning.

Conclusion

The study illustrates the value of combining quantitative with qualitative research when eliciting patient preferences. While quantitative data deliver numerical preference estimates, qualitative data enable researchers, decision makers, and physicians to understand the personal experiences and reasons shaping the patients’ preferences.

Acknowledgments

The authors thank Dr Dirk Müller, Dr Philipp Müther, and Fabian Jülich for their input during the study preparation; Sabrina Liebner and Fatjana Bylo for their support during patient recruitment. Furthermore, the authors would like to thank all patients for their participation. This investigator-initiated study was financially supported by Bayer Vital GmbH, Germany (which markets the product aflibercept). The sponsor had no role in the study design, data collection, data analysis or writing and publishing of the report.

Author contributions

All authors made substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data; took part in drafting the article or revising it critically for important intellectual content; gave final approval of the version to be published; and agree to be accountable for all aspects of the work.

Disclosure

The authors report no conflicts of interest in this work. This manuscript is not under consideration for publication in any other journal.

---

References

1.		Wong WL, Su X, Li X, et al. Global prevalence of age-related macular degeneration and disease burden projection for 2020 and 2040: a systematic review and meta-analysis. Lancet Glob Health. 2014;2(2):e106–e116.
2.		Muether PS, Hermann MM, Koch K, Fauser S. Delay between medical indication to anti-VEGF treatment in age-related macular degeneration can result in a loss of visual acuity. Graefes Arch Clin Exp Ophthalmol. 2011;249(5):633–637.
3.		Real JP, Granero GE, De Santis MO, et al. Rate of vision loss in neovascular age-related macular degeneration explored. Graefes Arch Clin Exp Ophthalmol. 2015;253(11):1859–1865.
4.		Wenzel M, Auffarth G, Scharrer A, Schayan K, Reinhard T. Ambulante und stationäre Intraokularchirurgie 2013: Ergebnisse der Umfrage von BDOC, BVA, DGII und DOG [Inpatient and outpatient intraocular surgery 2013: results of a survey among BDOC, BVA, DGII and DOG]. Ophthalmo-Chirurgie. 2014;26:171–182. German.
5.		European Medicines Agency [homepage on the Internet]. Summary of the European public assessment report (EPAR) for Lucentis (ranibizumab). EMA; 2008 [updated November 13, 2015]. Available from: http://www.ema.europa.eu/ema/index.jsp?curl=pages/medicines/human/medicines/000715/human_med_000890.jsp&mid=WC0b01ac058001d124. Accessed March 31, 2016.
6.		European Medicines Agency [homepage on the Internet]. Summary of the European public assessment report (EPAR) for Eylea (aflibercept). EMA; 2012 [updated February 4, 2016]. Available from: http://www.ema.europa.eu/ema/index.jsp?curl=pages/medicines/human/medicines/002392/human_med_001598.jsp&mid=WC0b01ac058001d124. Accessed March 31, 2016.
7.		Moja L, Lucenteforte E, Kwag KH, et al. Systemic safety of bevacizumab versus ranibizumab for neovascular age-related macular degeneration. Cochrane Database Syst Rev. 2014;9:CD011230.
8.		Droege KM, Caramoy A, Kersten A, et al. Patient preference of ranibizumab treatment regimen for neovascular age-related macular degeneration – monthly injections versus pro re nata. Graefes Arch Clin Exp Ophthalmol. 2014;252(1):31–34.
9.		Droege KM, Muether PS, Hermann MM, et al. Adherence to ranibizumab treatment for neovascular age-related macular degeneration in real life. Graefes Arch Clin Exp Ophthalmol. 2012;251(5):1281–1284.
10.		Bridges JF, Hauber AB, Marshall D, et al. Conjoint analysis applications in health – a checklist: a report of the ISPOR Good Research Practices for Conjoint Analysis Task Force. Value Health. 2011;14(4):403–413.
11.		Reed Johnson F, Lancsar E, Marshall D, et al. Constructing experimental designs for discrete-choice experiments: report of the ISPOR Conjoint Analysis Experimental Design Good Research Practices Task Force. Value Health. 2013;16(1):3–13.
12.		Coast J, Al-Janabi H, Sutton EJ, et al. Using qualitative methods for attribute development for discrete choice experiments: issues and recommendations. Health Econ. 2012;21(6):730–741.
13.		Coast J, Horrocks S. Developing attributes and levels for discrete choice experiments using qualitative methods. J Health Serv Res Policy. 2007;12(1):25–30.
14.		Danner M, Vennedy V, Hiligsmann M, Fauser S, Stock S. Focus Groups in Elderly Ophthalmologic Patients: Setting the Stage for Quantitative Preference Elicitation. Patient. 2016;9(1):47–57.
15.		Ryan M, Gerard K, Amaya-Amaya M. Using discrete choice experiments to value health and health care. Vol 11. Dordrecht: Springer Science & Business Media; 2007.
16.		Orme BK. Getting started with conjoint analysis. Madison, WI: Research Publishers LLC; 2006.
17.		Bech M, Gyrd-Hansen D. Effects coding in discrete choice experiments. Health Econ. 2005;14(10):1079–1083.
18.		Feller W. An introduction to probability theory and its applications. Vol 1, 3rd ed. Hoboken, NJ: John Wiley and Sons, Inc.; 1968.
19.		Altman DG, Bland JM. How to obtain the P value from a confidence interval. BMJ. 2011;343:d2304.
20.		Ryan M, Watson V, Entwistle V. Rationalising the ‘irrational’: a think aloud study of discrete choice experiment responses. Health Econ. 2009;18(3):321–336.
21.		Lancsar E, Louviere J. Deleting ‘irrational’ responses from discrete choice experiments: a case of investigating or imposing preferences? Health Econ. 2006;15(8):797–811.
22.		de Bekker-Grob EW, Donkers B, Jonker MF, Stolk EA. Sample Size Requirements for Discrete-Choice Experiments in Healthcare: a Practical Guide. Patient. 2015;8(5):373–384.
23.		Sackett DL, Rosenberg WM, Gray J, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ. 1996;312(7023):71–72.