Patients' preferences in periodontal disease treatment elicited alongside an IQWiG benefit assessment: a feasibility study

Introduction

The transparency of regulatory and reimbursement decisions in health care has increased.^1–4 Many health technology assessment (HTA) organizations focus on cost-effectiveness analyses to inform their decision-making processes whereas in Germany reimbursement decisions are primarily based on comparative effectiveness, epidemiological, and cost data. Health economic evaluations according to the Social Code Book V are only to be done upon request by the Federal Joint Committee (Gemeinsamer Bundesausschuss, G-BA) or the Ministry of Health⁵ but have never been requested in this framework until now. Recently, HTA organizations and decision-making bodies have started to explore options to incorporate patients’ or insurees’ perspectives into their decisions.^6,7 The aim of these initiatives is to take stakeholder – especially patients’ – preferences into account and increase public acceptance of decisions. Several HTA agencies have thus explored methods for eliciting preferences.^8–13 Since HTA needs robust evidence on patient preference to be considered in decisions, the focus is on gaining real-world experience with different methods and developing methodological standards. Besides the Analytic Hierarchy Process (AHP), the German Institute for Quality and Efficiency in Health Care (IQWiG) focused on discrete choice experiment (DCE) as a methodological approach for patient preference elicitation. In contrast to other HTA institutions, the objective was not to use patients’ quantified trade-offs between benefits and risks or patients’ willingness to pay in their assessments as a basis for decisions. Initially, the IQWiG tested the feasibility of AHP and DCE to generate preference weights to derive recommendations for outcome-specific reimbursement prices for later price negotiations.^14,15 The IQWiG conducted two pilot projects for weighing the outcomes of treatments; it used the AHP for preference analysis in major depression, and it used a DCE for preference analysis in chronic hepatitis C.^9,10 Currently, the IQWiG explores how to get insights into the relative importance that patients place on different types of risks and benefits in order to be able to prioritize outcome-specific information for decision making.¹

While the pilot projects demonstrated the general feasibility of the methods, up to now, preference elicitation has not become a regular part of IQWiG’s benefit assessments and the respective methods are still under scrutiny. Since DCE – contrary to the AHP – is rooted in utility theory and well accepted in health economic applications,^16,17 further research in this method was initiated with the current pilot project.

HTA agencies are often legally required to complete their assessments within short time frames. The IQWiG’s benefit assessments for pharmaceuticals and non-drug interventions are to be done within 3 months and within 15 months, respectively.^1,18 Since current methodological guidelines recommend qualitative patient and expert interviews in preparation of quantitative preference studies, these short time frames appear challenging.^19,20 Therefore, the IQWiG initiated the present preference elicitation conducted alongside a benefit assessment of the systematic treatment of periodontal diseases.²¹

Periodontitis is a common chronic condition characterized by inflammation of the gingiva and the supporting tissues around the teeth. The disease is a major cause of tooth loss.^22,23 Patients perceive symptoms such as gingival bleeding, gingival recession, dentine hypersensitivity, and tooth mobility. The systematic treatment of periodontitis is divided into active periodontal therapy (APT) and supportive periodontal therapy (SPT). Both include, inter alia, regular professional plaque removal. APT consists of scaling and root planning (SRP), and surgical or regenerative approaches, if needed. SPT includes periodontal examination, professional plaque removal, and retreatment, if needed. Only the SRP and surgical periodontal therapy itself are currently covered by the German statutory health insurance, whereas patients have to pay out of pocket for professional plaque removal, which is usually a prerequisite for receiving SRP or surgical interventions.²⁴

In this study, we tested the feasibility of a DCE to estimate the relative importance of attributes included in an IQWiG assessment within 3 months. The relative importance of attributes characterizing periodontitis and its treatment was to be estimated.²⁵

Patients and methods

In a DCE, participants are repeatedly asked to indicate their preference for one out of at least two hypothetical treatment situations in choice tasks. The description of hypothetical treatment situations consists of the treatments’ attributes, such as effectiveness and the corresponding levels ranging, for example, from low to high effectiveness. From these choices, preferences for treatment attributes and levels can be estimated using regression modeling. This DCE followed the recommendations of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR).²⁰ The ethics committee of the University Hospital Cologne granted approval for this study (no 15–385). All participants of the preparatory interviews provided written informed consent. Prior to completing the questionnaire, patients received a study information sheet. Since the questionnaire was completely anonymous, returning the questionnaire was considered as consenting to participate in the study.

Identification, selection, and framing of attributes and levels

The identification of treatment outcomes was based on the preliminary report plan for the benefit assessment of systematic treatment of periodontal diseases. This preliminary report plan specifies the methods for the particular benefit assessment and includes a range of potential treatment outcomes to be assessed (ie, tooth loss, tooth mobility, pain, symptomatic gingivitis [bleeding, swelling], adverse events, quality of life, and attachment level).²⁵ In addition, a literature review, interviews with three experts and 17 patients were performed to identify additional treatment and disease-related attributes, which are relevant from the patients’ perspective. Fifteen patients were interviewed in focus groups including three to four patients each. Two patients underwent personal interviews since they were not available on the day of focus group interviews. In these interviews, patients were first asked to state their primary treatment objective. Subsequently, patients named important symptoms that they had either experienced or were afraid of, followed by a discussion of each of the stated symptoms. Additionally, other attributes related to the treatment process were discussed. Besides identifying and selecting attributes, patient interviews were conducted to label and frame attributes and levels in a way which is well understandable for patients. In this context, interviews revealed, for example, that patients were not familiar with the outcome “attachment level” applied as a surrogate for the patient-relevant outcome “tooth loss” in clinical trials and, accordingly, in the IQWiG benefit assessment. Therefore, the patient-relevant outcome “tooth loss” rather than the surrogate “attachment level” was selected for inclusion in the preference analysis. The level ranges for this attribute were defined by prospective cohort studies reporting this outcome.^26–32 Patients and periodontists reported similar common symptoms but used different wording. Gingival recession was a complaint described as “long teeth” by patients. Both highlighted the attribute’s relevance to patients as an esthetic consequence of gingival recession. Furthermore, both periodontists and patients mentioned sensitive tooth necks and gingival bleeding as common and relevant symptoms. Since patients are required to pay out-of-pocket for the professional plaque removal preceding periodontal therapy and for the repeated follow-up treatments, patients considered costs a relevant treatment attribute.²⁴ Also, dentists highlighted that some patients quit periodontal preventive therapy due to its high costs. A summary of attributes and levels included in the study is displayed in Table 1. Further details on the attribute and level selection procedure can be found in Supplementary material 1 (interview guide), Supplementary material 2 (summary of attribute and level development), or in the IQWiG’s Final Report GA15-01 preference measurement for periodontal disease.²¹

Table 1 Attributes and levels Note: aJanuary–February 2016: 1 Euro = 1.09 US dollars.

Experimental design

To minimize the number of choice tasks for each participant but maximize statistical efficiency, a Bayesian efficient design was developed with Ngene (ChoiceMetrics). Directional priors were used in the design development. Losing more teeth, having to pay more, or experiencing symptoms and complaints was assumed to be associated with a decrease in utility. Additional information on the selection of prior information and design syntax is provided in Supplementary material 3. The final design consisted of 12 choice tasks with two options each. The number of choice tasks allowed for complete level balance in the design. An example of these choice tasks is displayed in Figure 1.

Figure 1 Example of a choice task.

Questionnaire development

The questionnaire was structured into three parts (see Supplementary material 4). The first part included a brief explanation of the study and the survey task. This was followed by the choice sets including one additional choice set with a dominant treatment option (question 13 in questionnaire). The latter was used to check for the percentage of respondents’ choosing the dominant option. The questionnaire further collected data on demographic and disease-related characteristics to allow for later subgroup analyses. Questionnaire pretests with three patients resulted in minor editorial changes. The survey was conducted as a self-administered paper-pencil survey.

Sample size, patient recruitment, and data collection

The minimum sample size necessary for calculating the relative importance of attributes based on a main effects model was estimated using the empirically derived formula proposed by Orme.³³ While 84 patients were the required sample size for estimating a main effects model, recruitment was continued until the end of the 3 months project time to facilitate additional analyses. Patients were recruited at the Department of Operative Dentistry and Periodontology of the University Hospital Cologne and at nine ambulatory dentist offices in the federal state of North-Rhine-Westphalia. Most patients filled in the questionnaire at the dentist office, but those who lacked time either completed it at home or received a link to an online version. Questionnaire completion took 10–15 minutes. In appreciation of their participation, patients received several products for dental care.

Data analysis

The analysis of DCEs is based on the assumption that participants choose the option with which they associate a higher utility. Preferences were estimated using a random parameter logit model in Nlogit (Econometric Software, Inc., Plainview, NY, USA). This model takes the interdependence between the judgments provided by one participant into account. Moreover, the model allows assessing the preference heterogeneity among the surveyed individuals. Significant standard deviations (P < 0.05) indicate preference heterogeneity.³⁴ All categorical levels were dummy coded with the following reference levels for the respective attributes: no teeth lost, no symptoms and complaints, and no follow-up visits necessary. Preferences for all categorical levels were assumed to be normally distributed. Cost preferences were coded as a continuous variable and assumed to be log-normally distributed. These distributional assumptions yielded the best model-fit, based on McFadden’s Pseudo-R² and the Akaike information criterion.³⁴ The estimation of the random parameter model (Formula 1) was conducted by using 2000 Halton draws.

(1)

Assuming a normal distribution, the random coefficient (β) for one attribute level corresponds to the expected mean part-worth utility of this specific level aggregated for all individual patients (j) and alternatives (q). The coefficient for cost preferences was transformed into its median, which provides a better measure of central tendency for log-normally distributed variables.³⁵

Importance of attributes

Using the results of the random parameter logit model, the relative importance (I) of each attribute (a) was calculated as described in the ISPOR guidelines

(2)

where r(x_a) equals the range of level coefficients within an attribute.³⁶ These ranges are standardized based on the sum of ranges within all attributes. The highest relative attribute importance corresponds to the attribute in which changes from the least valued level to the highest valued level had the largest impact on patients’ decisions. Next to the DCE, patients were asked directly to state which of the attributes had the largest or smallest impact on their decision and to state the symptoms or complaints they wanted to prevent most.

Additional analyses

While the IQWiG’s primary objective was to generate preference-based outcome weights that were generated based on coefficients, additional analyses were performed. Heterogeneity in preferences was further explored by performing subgroup analyses. Separate models were estimated based on the patient characteristics in the questionnaire (eg, gender, educational background, age groups, and time since diagnosis). The aim was to identify whether specific patient characteristics were associated with differences in preference structures. Moreover, a latent class model was used to analyze preference heterogeneity that could not be traced back to reported patient characteristics.

Results

Within 3 months, all study steps including study planning, preparatory dentist and patient interviews, questionnaire design, data collection and analysis, as well as providing a project report to the IQWiG could be completed. From 15 January 2016 until 1 March 2016, a total of 267 patients were surveyed. This number exceeded the estimated minimum sample size and therefore allowed for additional analyses, which are presented next to the main effects model in this paper.

Sample characteristics

A total of 25% of the 267 surveyed patients were treated at the University Hospital Cologne; the remaining 75% were treated by local dentists (Table 2). Of these patients, 75% were 41–70 years old. More women (58%) than men (42%) participated in the study. Most participants had a treatment experience of 3 or more years (68%). Due to anonymity requirements participation rates could not be recorded systematically, but dental offices reported that around 90% of eligible patients agreed to participate in the study. Language barriers, time constraints, or unwillingness to participate in studies in general were common reasons for refusing participation. About 10 patients were unable to complete the questionnaire due to excessive cognitive demands or time constraints. Out of the 261 participants who answered the additional choice set including a dominant alternative, 257 participants (98%) chose this dominant alternative. All 267 participants were included in the analyses.

Table 2 Patient characteristics Note: aNumber of patients within group (if numbers do equal 267, data are missing).

Patient preferences and relative attribute importance

The results of the preference analysis in the main effects model are displayed in Table 3. The coefficients for levels indicate to what extent the corresponding level influenced whether an alternative was chosen. Patients significantly disliked increasing loss of teeth as reflected in the negative coefficients for losing one tooth (−1.97) or two teeth (−5.00) in comparison with no tooth loss. Patients preferred to pay less for periodontal therapy, as indicated by the negative coefficient for costs (−0.0002 per Euro). Compared to having no complaints, patients disliked most having long teeth due to gum recession (−1.51). Sensitive tooth necks (−0.68) and occasional gum bleeding (−0.05) were also significantly disliked compared to no complaints. No significant preferences for a specified frequency of periodontist visits was found; however, patients expressed the tendency to prefer two visits over none (0.18). Significant preference heterogeneity was found for all levels except losing one tooth, having sensitive tooth necks and attending periodontist visits twice per year.

Table 3 Results of preference elicitation Notes: aExpected values of random coefficients. bLognormal coefficient was transformed into expected value of median by eμ. The standard deviation was calculated using the following formula: . Number of observations, that is, answered choice sets: 3,171; missing answers for choice sets: 33; Pseudo-R2: 0.4462, log likelihood: −1,217.31, Akaike Information Criterion: 2,466.6. Abbreviation: CI, confidence interval.

Based on the results displayed in Table 3, relative attribute importance was calculated. Level differences within the attribute tooth loss within the next 10 years had the largest relative impact on patients’ decisions (0.73), followed by complaints and symptoms (0.22). The differences in frequency of periodontist visits (0.02) and the out-of-pocket expenses were least important for patients (0.03).

Next to the prioritization based on the preference analysis, patients were asked to directly state which attribute they considered most important in their decision (Table 4). The majority of patients stated that tooth loss was most important (82%). A total of 71% of patients considered the frequency of periodontist visits least important, 65% of patients considered “long teeth” due to gum recession the worst symptom, whereas occasional gum bleeding was considered worst by only 7% of patients.

Table 4 Directly elicited attribute importance Note: n, number of patients with the respective answer; percentages summing up to ≠ 100% due to rounding.

Results of additional analyses

Subgroup analyses (Supplementary material 5) did not reveal any significant differences in preferences related to any patient characteristics surveyed in this study. The results of the latent class model consisting of two classes are shown in Table 5. Patients were more likely to belong to class 1 (63.3%) than to class 2. Both classes disliked tooth loss; however, the extent of this negative preference is larger in class 1 than in class 2. Costs of treatment are valued positively in class 1 (coefficient: 0.001, P < 0.05), whereas they are valued negatively in class 2 (coefficient: −0.002, P < 0.05). In both classes “long teeth due to gum recession” are disliked, but unlike class 2, class 1 also values having sensitive teeth negatively (−0.690, P < 0.05). Class 1 does not exhibit any significant preferences for frequencies of periodontist visits, but significant preferences for two periodontist visits were observed in class 2 (coefficient: 0.481, P < 0.05).

Table 5 Latent class model Note: Number of observations, that is, answered choice sets: 3,171; missing answers for choice sets: 33; Pseudo-R2: 0.4517, log likelihood: −1,204.99, Akaike Information Criterion: 2,444.0. Abbreviation: CI, confidence interval.

Discussion

In this study, we tested the feasibility of a DCE to estimate preference weights for periodontal disease and treatment attributes in 3 months. During this time frame, study planning, data collection among 267 patients, data analysis, and provision of a study report to IQWiG could be completed. Since almost all patients chose the dominant alternative in the test choice set, the task seems to be well understood and attentively completed by patients. The study showed that first and foremost patients wanted to avoid tooth loss. The most disliked symptom was having long teeth due to gum recession. These findings are supported by the preceding qualitative research as well as the direct question on the most important attribute and least preferred symptom. Our results may soon be compared to the results from the British Improving the Quality of Dentistry (IQuaD) trial, which currently investigates the effectiveness of different periodontal treatments and surveys patients in a DCE preference analysis.³⁷

The frequency of periodontist visits and costs of treatment seem to play a minor role in patients’ decisions. Also, the latent class model showed that the sample consisted of a majority of patients who considered costs as slightly positive, while a smaller group of patients regarded costs as negative. During the patient interviews, some patients described the costs of treatment as an investment in good health. They considered the treatment necessary for preventing even more costly treatments in the future such as dental prostheses. Since such care is expensive and often not covered by SHIs in Germany, patients know that losing a tooth will cause high out-of-pocket expenses. While patients similarly disliked, for example, losing one tooth, having sensitive tooth necks or occasional gum bleeding, preferences were more heterogeneous especially for losing two teeth or getting long teeth. This heterogeneity is primarily caused by a wide range of negative coefficients within the sample. Such heterogeneity might be caused, for example, by the value individuals associate with esthetics.

Our study has several limitations. The importance of costs might have been underestimated in our study. First, patients treated at the University Hospital of Cologne did not have to pay for their treatment at the time of the survey and might therefore have disregarded this attribute. Second, the exclusive recruitment of patients regularly undergoing dental treatment might have led to a selection bias, since these patients in general place a higher value on dental care than those who do not seek care. Finally, our sample included more than 50% of patients with an academic or comparable educational background, which is usually associated with above average incomes. Therefore, the low importance of costs observed in this study might also result from a higher ability to pay.³⁸ Those who do not regularly visit dentists or discontinued treatment due to out-of-pocket expenses were unlikely to participate in our study and costs could have been much more influential for those patients. This clearly limits transferability of our results to the general periodontal patient population. On the other hand, recruiting patients in dentist offices seemed to be best to identify eligible patients. Preliminary patient and dentist interviews revealed that patients often confuse different dental health problems (eg, caries and periodontal disease),²¹ which might otherwise have led to patient misclassification. Other recruitment strategies such as panel surveys or a population-wide survey involve the risk of including ineligible patients, especially if diagnosis is based on patient self-report. Additionally, a population-based screening for eligible patients was not performed in this study since it would not be possible to perform such screenings if patient preference analysis was a regular step in HTA.

Another potential limitation of our study is that for the attribute “tooth loss” patients might have disregarded the specification “within the next 10 years”, since this is not a common time horizon for patients’ decision making. However, estimates on tooth loss are only available from long-term cohort studies. Adjusting the numbers to shorter time frames, for example, as percentages of teeth lost within 1 year, seemed not intuitive, since teeth are usually lost as complete entities and not fractions.

Despite the general feasibility of a preference analysis within 3 months in this particular indication, several challenges need to be discussed. Due to the time restriction, our prior information for constructing the experimental design was not based on pretest data, but preference tendencies identified during patient interviews. Also, we did not use pretest data to derive sample size estimates as proposed.³⁹ However, our sample size falls within the range identified in several reviews.^39,40

A specific, and maybe the most important, challenge of our study was the limited transferability of outcomes included in the IQWiG benefit assessment to the DCE. This was the case, for example, for the outcome attachment level, which served as a surrogate for “tooth loss” in the benefit assessment but was not an outcome patients were familiar with. In addition, several clinical outcomes were correlated with each other, for example, attachment loss being correlated with both characteristics included in this DCE – tooth loss and long teeth. However, independence of attributes in DCEs is necessary as much as possible. While only tooth loss and symptoms such as pain or gingival bleeding would be considered patient relevant according to the IQWiG or the Cochrane Group (focusing on mortality, morbidity, and quality of life), the outcome “long teeth” would not be considered patient-relevant but rather reflects the esthetic implications of disease.⁵ This outcome is sometimes measured as gingival recession in clinical trials and of importance for patients. However, the IQWiG would only consider this aspect, if it was reflected, for example, in a validated instrument for measuring (oral-) health-related quality of life. In addition, outcomes measured in clinical trials often measure overlapping concepts of disease. For example, disease-specific quality of life questionnaires often include measurement of several symptoms that might be considered relevant to patients and need to be included as non-overlapping attributes in a DCE.^20,34 These examples illustrate the difficulty of translating patient-relevant outcomes as measured in clinical trials or considered in HTA assessments into attributes and levels for a DCE and vice versa. As in this case, such problems will often complicate the inclusion of patient preferences measured with a DCE in benefit assessments and are independent of the time frames allowed for preference analyses. Future studies should therefore address the question on how to combine results from preference analyses with results from HTAs, especially if outcomes are not identical.

While a DCE may be used to calculate exchange rates between attributes from the patients’ perspective and provide information, for example, on how much a patient would be willing to pay for preventing the loss of one additional tooth, this was not the IQWiG’s aim of conducting this feasibility study. The cost attribute was primarily included because the qualitative interviews suggested that cost was the only aspect that might withhold patients from undergoing therapy, since in general no severe health risks or highly uncomfortable treatment processes are associated with periodontal treatment. Preference analyses are especially of added value if patients have to take preference-sensitive decisions, that is, decisions where relevant risks and benefits such as prolonged life and quality of life have to be weighed against each other.

For estimating the relative attribute importance for periodontal disease and its treatment, the methodological guidance issued by the ISPOR Taskforce on Statistical Methods for the Analysis of Discrete Choice Experiment was followed.³⁶ Accordingly, the relative attribute importance was calculated based on the ranges of level coefficients within each attribute. Since choices are primarily based on levels, not attributes, attribute and level importance cannot clearly be separated from each other.^41–43 Therefore, with a DCE the importance of each attribute is relative to all other attributes included in the study, conditional on the range of levels of each attribute. Other methods for estimating relative attribute importance have been discussed, but none of them is clearly preferred in international methodological guidelines.⁴³

The feasibility of this preference analysis needs to be seen in the context of the specific periodontal patient population and research infrastructure. The high periodontal disease and treatment prevalence combined with the spread through all age groups made the population easily accessible and allowed for the time saving survey format of a self-administered questionnaire. In contrast, surveying elderly or impaired patients might require more cost- and time-consuming methods such as interviewer-assisted surveys,^44,45 which might not be feasible with similar resources in the framework of an IQWiG assessment. In this study, especially the cooperating networks of dentists and a highly motivated and easily accessible patient population facilitated timely recruitment.

In HTA, quantitative preference analyses can be of added value, in particular, if patients have to take preference-sensitive decisions, that is, if risks and benefits such as prolonged life and quality of life have to be weighed against each other. Given the clear preference trends identified in our patient interviews, it is also questionable whether quantitative preference analyses should always be required or just requested if controversial preferences for patient-relevant outcomes are identified in preparatory work. Another open question is at which point in time in the process of HTA in general, and in an IQWiG assessment in particular, should patient preferences be elicited and who should be responsible for generating these data. Vass et al discussed a variety of methodological questions for further DCE research, including, for example, the framing of risks or probabilities, the influence of personal characteristics on preferences, and how to deal with heterogeneous preferences, a specification of whose preferences should be surveyed.⁴⁶ Solving all these still unresolved issues is a prerequisite before regular inclusion of preference data into HTA. Vass et al in this respect call for the development of a “reference case” to be able to “safely and confidently use discrete choice experiments” in HTA.⁴⁶

Due to the challenges and open questions related to including outcomes from an IQWiG assessment into a preference elicitation or vice versa, it might be reasonable to assess preferences much earlier. Ideally, patient preferences can already inform the development of new therapies. Thereby also, clinical trials can focus on collecting complete data on patient-relevant outcomes that are considered important from the patient perspective.

Conclusion

A patient preference elicitation performing a DCE can be feasible within a time frame of 3 months. Time constraints should not be a reason for refusing to implement systematic inclusion of patient views into regulatory decision making. However, disease prevalence and severity and the health care and research infrastructure in an indication might considerably influence patient accessibility and the speed and quality of patient recruitment. Many methodological and practical questions on integrating preferences in HTAs are currently still under debate. Patient preference elicitations can provide useful insight to support HTAs and appraisals and can increase the transparency of decisions. However, the precise process of generating and integrating preference data needs to be clearly defined prior to introducing preference analysis as a regular additional component of benefit assessments.

Acknowledgments

We thank the dentist offices for their helpful comments during study planning and support of patient recruitment. Furthermore, we thank all patients for participating in the interviews and completing the questionnaire.

Disclosure

VV, MD, DC, and SS received financial support from the IQWiG for conducting this study. AS, AS, and FS were employed by the IQWiG at the time of study planning and conduct. The other authors declare no conflicts of interests. The abstract of this paper was presented at the 19th Annual European Congress of the ISPOR and at the 16th German Congress for Health Services Research (DKVF) as an abstract presentation with interim findings. The abstract was published in Value in Health (2016) Volume 19, Issue 7, Pages A352–A353; DOI: https://doi.org/10.1016/j.jval.2016.09.036 and in the online program of the 16th German Congress for Health Services Research at https://events.mcon-mannheim.de/frontend/index.php?page_id=1210&v=List&do=15&day=31&ses=1884#.

---

References

1.		Institute for Quality and Efficiency in Health Care. General methods version 5.0. Available from: https://www.iqwig.de/download/IQWiG_General_Methods_Version_%204-2.pdf. Accessed July 10, 2017.
2.		National Institute for Health and Care Excellence [webpage on the Internet]. Guide to the methods of technology appraisal 2013. Available from: https://www.nice.org.uk/process/pmg9/chapter/foreword. Accessed July 25, 2017.
3.		Canadian Agency for Drugs and Technologies in Health [webpage on the Internet]. Methods and guidelines. 2009. Available from: https://www.cadth.ca/about-cadth/how-we-do-it/methods-and-guidelines. Accessed July 25, 2017.
4.		Haute Autorité de santé. A methodological guide-choices in methods for economic evaluation. 2015. Available from: https://www.has-sante.fr/portail/upload/docs/application/pdf/2012-10/choices_in_methods_for_economic_evaluation.pdf. Accessed July 25, 2017.
5.		Bundesministerium der Justiz und für Verbraucherschutz. § 130b Agreements between the central association of health insurance companies and pharmaceutical companies on reimbursement for drugs, prescription authorization. 1988. Available from: http://www.gesetze-im-internet.de/sgb_5/__130b.html. Accessed October 29, 2018. German.
6.		Mühlbacher AC, Juhnke C. Involving patients, the insured and the general public in healthcare decision making. Z Evid Fortbild Qual Gesundhwes. 2016;110–111:36–44. German.
7.		Mühlbacher AC, Juhnke C, Beyer AR, Garner S. Patient-focused benefit-risk analysis to inform regulatory decisions: the European Union perspective. Value Health. 2016;19(6):734–740.
8.		Center for Devices and Radiological Health. Patient preference information – voluntary submission, review in premarket approval applications, humanitarian device exemption applications, and de novo requests, and inclusion in decision summaries and device labeling. Guidance for industry, food and drug administration staff, and other stakeholders. 2016. Available from: http://www.fda.gov/downloads/%20MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/UCM446680.pdf. Accessed August 29, 2016.
9.		Danner M, Hummel JM, Volz F, et al. Integrating patients’ views into health technology assessment: analytic hierarchy process (AHP) as a method to elicit patient preferences. Int J Technol Assess Health Care. 2011;27(4):369–375.
10.		Mühlbacher AC, Bridges JF, Bethge S, et al. Preferences for antiviral therapy of chronic hepatitis C: a discrete choice experiment. Eur J Health Econ. 2017;18(2):155–165.
11.		Postmus D, Mavris M, Hillege HL, et al. Incorporating patient preferences into drug development and regulatory decision making: results from a quantitative pilot study with cancer patients, carers, and regulators. Clin Pharmacol Ther. 2016;99(5):548–554.
12.		Ho MP, Gonzalez JM, Lerner HP, et al. Incorporating patient-preference evidence into regulatory decision making. Surg Endosc. 2015;29(10):2984–2993.
13.		Janssen IM, Gerhardus A, Schröer-Günther MA, Scheibler F. A descriptive review on methods to prioritize outcomes in a health care context. Health Expect. 2015;18(6):1873–1893.
14.		GKV Spitzenverband [webpage on the Internet]. AMNOG – evaluation of new pharmaceutical. Available from: https://www.gkv-spitzenverband.de/english/statutory_health_insurance/amnog_evaluation_of_new_pharmaceutical/amnog_english.jsp. Accessed January 23, 2018.
15.		Institute for Quality and Efficiency in Health Care [webpage on the Internet]. G09-01 Health economic evaluation of venlafaxine, duloxetine, bupropion, and mirtazapine compared to further prescribable pharmaceutical treatments 2013. Available from: https://www.iqwig.de/de/projekte-ergebnisse/projekte/gesundheitsoekonomie/g09-01-kosten-nutzen-bewertung-von-venlafaxin-duloxetin-bupropion-und-mirtazapin-im-vergleich-zu-weiteren-verordnungsfaehigen-medikamentoesen-behandlungen.1256.html. Accessed January 23, 2018.
16.		de Bekker-Grob EW, Ryan M, Gerard K. Discrete choice experiments in health economics: a review of the literature. Health Econ. 2012;21(2):145–172.
17.		Clark MD, Determann D, Petrou S, Moro D, de Bekker-Grob EW. Discrete choice experiments in health economics: a review of the literature. Pharmacoeconomics. 2014;32(9):883–902.
18.		Bundesanzeiger Verlag. Law for the reorganization of the pharmaceutical market in the statutory health insurance (Arzneimittelmarktneuordnungsgesetz – AMNOG). 2010. Available from: http://www.bgbl.de/xaver/bgbl/start.xav?startbk=Bundesanzeiger_BGBl&jumpTo=bgbl110s2262.pdf. Accessed October 29, 2018. German.
19.		Rydén A, Chen S, Flood E, Romero B, Grandy S. Discrete choice experiment attribute selection using a multinational interview study: treatment features important to patients with type 2 diabetes Mellitus. Patient. 2017;10(4):475–487.
20.		Bridges JF, Hauber AB, Marshall D, et al. Conjoint analysis applications in health – a checklist: a report of the ISPOR Good Research Practices for Conjoint Analysis Task Force. Value Health. 2011;14(4):403–413.
21.		Institute for Quality and Efficiency in Health Care. GA15-01 Präferenzmessung bei Parodontopathien. 2016. Available from: https://www.iqwig.de/download/GA15-01_Arbeitspapier_Praeferenzmessung-bei-Parodontopathien_V1-1.pdf. Accessed July 25, 2017.
22.		Savage A, Eaton KA, Moles DR, Needleman I. A systematic review of definitions of periodontitis and methods that have been used to identify this disease. J Clin Periodontol. 2009;36(6):458–467.
23.		Chapple IL, Van der Weijden F, Doerfer C, et al. Primary prevention of periodontitis: managing gingivitis. J Clin Periodontol. 2015;42(Suppl 16):S71–S76.
24.		Kassenzahnärztliche Bundesvereinigung [webpage on the Internet]. Behandlung der Parodontitis. Available from: http://www.kzbv.de/behandlung-der-parodontitis.97.de.html. Accessed September 7, 2017.
25.		Institute for Quality and Efficiency in Health Care [webpage on the Internet]. N15-01 Assessment of the systematic treatment of parodontopathies. 2015. Available from: https://www.iqwig.de/en/projects-results/projects/non-drug-interventions/n15-01-assessment-of-the-systematic-treatment-of-parodontopathies.6629.html. Accessed July 19, 2017.
26.		Lee CT, Huang HY, Sun TC, Karimbux N. Impact of patient compliance on tooth loss during supportive periodontal therapy: a systematic review and meta-analysis. J Dent Res. 2015;94(6):777–786.
27.		Bäumer A, Pretzl B, Cosgarea R, et al. Tooth loss in aggressive periodontitis after active periodontal therapy: patient-related and tooth-related prognostic factors. J Clin Periodontol. 2011;38(7):644–651.
28.		Eickholz P, Kaltschmitt J, Berbig J, Reitmeir P, Pretzl B. Tooth loss after active periodontal therapy. 1: patient-related factors for risk, prognosis, and quality of outcome. J Clin Periodontol. 2008;35(2):165–174.
29.		Costa FO, Lages EJ, Cota LO, Lorentz TC, Soares RV, Cortelli JR. Tooth loss in individuals under periodontal maintenance therapy: 5-year prospective study. J Periodontal Res. 2014;49(1):121–128.
30.		Costa FO, Miranda Cota LO, Pereira Lages EJ, et al. Progression of periodontitis and tooth loss associated with glycemic control in individuals undergoing periodontal maintenance therapy: a 5-year follow-up study. J Periodontol. 2013;84(5):595–605.
31.		Gilbert GH, Shelton BJ, Chavers LS, Bradford EH. Predicting tooth loss during a population-based study: role of attachment level in the presence of other dental conditions. J Periodontol. 2002;73(12):1427–1436.
32.		Hujoel PP, Leroux BG, DeRouen TA, Powell LV, Kiyak HA. Evaluating the validity of probing attachment loss as a surrogate for tooth mortality in a clinical trial on the elderly. J Dent Res. 1997;76(4):858–866.
33.		Orme BK. Getting Started with Conjoint Analysis. Madison, WI: Research Publishers LLC; 2006.
34.		Hensher DA, Rose JM, Greene WH. Applied Choice Analysis: A Primer. Cambridge University Press: Cambridge; 2005.
35.		Daly LE, Bourke GJ. Interpretation and Uses of Medical Statistics. 5th ed. Oxford; Malden, MA: Blackwell Science; 2000.
36.		Hauber AB, González JM, Groothuis-Oudshoorn CG, et al. Statistical methods for the analysis of discrete choice experiments: a report of the ISPOR Conjoint Analysis Good Research Practices Task Force. Value Health. 2016;19(4):300–315.
37.		Clarkson JE, Ramsay CR, Averley P, et al. IQuaD dental trial; improving the quality of dentistry: a multicentre randomised controlled trial comparing oral hygiene advice and periodontal instrumentation for the prevention and management of periodontal disease in dentate adults attending dental primary care. BMC Oral Health. 2013;13:58.
38.		Tan SHX, Vernazza CR, Nair R. Critical review of willingness to pay for clinical oral health interventions. J Dent. 2017;64:1–12.
39.		de Bekker-Grob EW, Donkers B, Jonker MF, Stolk EA. Sample size requirements for discrete-choice experiments in healthcare: a practical guide. Patient. 2015;8(5):373–384.
40.		Bien DR, Danner M, Vennedey V, Civello D, Evers SM, Hiligsmann M. Patients’ preferences for outcome, process and cost attributes in cancer treatment: a systematic review of discrete choice experiments. Patient. 2017;10(5):553–565.
41.		Flynn TN, Louviere JJ, Peters TJ, Coast J. Estimating preferences for a dermatology consultation using best-worst scaling: comparison of various methods of analysis. BMC Med Res Methodol. 2008;8:76.
42.		Flynn TN, Louviere JJ, Peters TJ, Coast J. Best – worst scaling: what it can do for health care research and how to do it. J Health Econ. 2007;26(1):171–189.
43.		Lancsar E, Louviere J, Flynn T. Several methods to investigate relative attribute impact in stated preference experiments. Soc Sci Med. 2007;64(8):1738–1753.
44.		Milte R, Ratcliffe J, Miller M, Whitehead C, Cameron ID, Crotty M. What are frail older people prepared to endure to achieve improved mobility following hip fracture? A discrete choice experiment. J Rehabil Med. 2013;45(1):81–86.
45.		Danner M, Vennedey V, Hiligsmann M, Fauser S, Stock S. Focus groups in elderly ophthalmologic patients: setting the stage for quantitative preference elicitation. Patient. 2016;9(1):47–57.
46.		Vass CM, Payne K. Using discrete choice experiments to inform the benefit-risk assessment of medicines: are we ready yet? Pharmacoeconomics. 2017;35(9):859–866.