Back to Journals » Patient Preference and Adherence » Volume 15

Taking into Account Patient Preferences: A Consensus Study on the Assessment of Psychological Dimensions Within Patient Preference Studies

Authors Russo S , Monzani D , Pinto CA , Vergani L, Marton G, Falahee M , Simons G, Whichello C, Kihlbom U , Pravettoni G 

Received 7 May 2020

Accepted for publication 23 March 2021

Published 18 June 2021 Volume 2021:15 Pages 1331—1345


Checked for plagiarism Yes

Review by Single anonymous peer review

Peer reviewer comments 2

Editor who approved publication: Dr Johnny Chen

Selena Russo,1,2,* Dario Monzani,3,4,* Cathy Anne Pinto,5 Laura Vergani,3,4 Giulia Marton,3,4 Marie Falahee,6 Gwenda Simons,6 Chiara Whichello,7 Ulrik Kihlbom,8,* Gabriella Pravettoni3,4,*

1Department of Medicine and Surgery, University of Milano-Bicocca, Milan, Italy; 2Discipline of Paediatrics, School of Women’s and Children’s Health, The University of New South Wales, Kensington, NSW, Australia; 3Applied Research Division for Cognitive and Psychological Science, IEO, European Institute of Oncology IRCCS, Milan, Italy; 4Department of Oncology and Hemato-oncology, University of Milan, Milan, Italy; 5Department of Pharmacoepidemiology, Merck & Co., Inc., Kenilworth, NJ, USA; 6Rheumatology Research Group, Institute of Inflammation and Ageing (IIA), University of Birmingham, Birmingham, UK; 7Erasmus School of Health Policy & Management (ESHPM) and Erasmus Choice Modelling Centre (ECMC), Erasmus University Rotterdam, Rotterdam, The Netherlands; 8Centre for Research Ethics and Bioethics, Uppsala University, Uppsala, Sweden

*These authors contributed equally to this work

Correspondence: Selena Russo
Department of Medicine and Surgery, University of Milano, Bicocca, via Cadore 48, Monza (MB), 20052, Italy
Email [email protected]

Abstract: Patient preferences are gaining recognition among key stakeholders involved in benefit-risk decision-making along the medical product lifecycle. However, one of the main challenges of integrating patient preferences in benefit-risk decision-making is understanding differences in patient preference, which may be attributable to clinical characteristics (eg age, medical history) or psychosocial factors. Measuring the latter may provide valuable information to decision-makers but there is limited guidance regarding which psychological dimensions may influence patient preferences and which psychological instruments should be considered for inclusion in patient preference studies. This paper aims to provide such guidance by advancing evidence and consensus-based recommendations and considerations. Findings of a recent systematic review on psychological constructs having an impact on patients’ preferences and health-related decisions were expanded with input from an expert group (n = 11). These data were then used as the basis for final recommendations developed through two rounds of formal evaluation via an online Delphi consensus process involving international experts in the field of psychology, medical decision-making, and risk communication (n = 27). Three classes of recommendations emerged. Eleven psychological constructs reached consensus to be recommended for inclusion with the strongest consensus existing for health literacy, numeracy, illness perception and treatment-related beliefs. We also proposed a set of descriptive and checklist criteria to appraise available psychological measures to assist researchers and other stakeholders in including psychological assessment when planning patient preference studies. These recommendations can guide researchers and other stakeholders when designing and interpreting patient preference studies with a potential high impact in clinical practice and medical product benefit-risk decision-making processes.

Keywords: patient preferences, patients reported outcomes, psychological assessment, decision-making

Key Points

  • In the medical products benefit-risk decision-making process and in the research field of preference studies, patient preferences are gaining recognition among key stakeholders. There is a strong need for evidence-based guidance on what psychological dimensions may influence patient preferences and which should be considered for inclusion in patient preferences studies.
  • This study fills an important gap in patient preference methodology. It helps stakeholders selecting psychological constructs and measurements to use for evaluating how patients form, maintain, change and negotiate preferences in medical and healthcare context. In addition, psychological assessments can assist in explaining preference heterogeneity in cases where research participants’ preferences diverge.
  • Evidence- and consensus-based considerations on what psychological constructs to include in patient preference studies are advanced. A criteria checklist to select appropriate psychological measurements to include in patient preference studies are also advanced. Recommendations and criteria proposed will inform stakeholders involved in the medical products benefit-risk decision-making of the patient’s formative abilities and describe preference heterogeneity.


Patient preferences are gaining recognition among key stakeholders involved in benefit-risk decision-making along the medical product lifecycle.1,2 A patient preference (PP) may be defined as

A statement of the relative desirability or acceptability to patients of specified alternatives or choices among outcomes or other attributes that differ among alternative health interventions.1

PPs are specific type of patient perspective. Patient perspective can be defined as any information related to patients’ experiences with a medical condition and its management. Overall, this information allows a better comprehension of the disease and its impact, the effective identification of outcomes that are more relevant for patients, and a more fine-grained understanding of the benefit-risk trade-off for treatment.3 PPs are especially relevant for the evaluation of the benefit-risk profile of medical products in case of preference-sensitive decisions, namely situations where medical equipoise among alternative treatments exists.4 In the clinical setting, the attention given to PP is relevant in different theoretical and practical approaches, such as the shared decision-making model that underlies the relevance of recognising PP to empower patients and personalise treatment and care.5–7 The present paper is a direct result of the tasks carried out within the framework of The Patient Preferences in Benefit-Risk Assessments during the Drug Life Cycle (PREFER) project, a European undertaking supported by the Innovative Medicines Initiative (IMI). The main objective of PREFER is to strengthen patient-centric decision-making throughout the medical product life-cycle by developing evidence-based recommendations to guide industry, regulatory authorities, health technology assessment (HTA)/payer bodies, reimbursement agencies, academia, and health care professionals on how and when patient-preference studies should be performed and the results used to support and inform decision-making throughout the medical products lifecycle and enable the implementation of personalised medicine.8 However, as reported by a recent systematic review by Huls and colleagues,9 one of the main challenges of integrating PP in benefit-risk decision-making is taking into account patient preference heterogeneity and formation. PP heterogeneity can be defined as “differences in preferences among a sample”10 with the existence of subgroups of patients with relevant differences in preferences. Within this framework, the measurement of psychological constructs might be relevant to obtain a psychological profile of subpopulations of patients with relevant differences in preferences. The identification of subgroups of patients with heterogeneous preferences could

Be particularly valuable when there is a suggestion that these differences are important enough to alter the decision whether to approve a product for at least one subgroup of patients.10

The need to better understand sources of PP heterogeneity is also stressed by a qualitative study carried out within the PREFER project investigating stakeholders’ attitudes, expectations and concerns towards measuring and including PP in decision-making along the medical product lifecycle. Its results highlighted that stakeholders were often concerned about whether and how psychological and emotional factors may influence PP. Stakeholders reasoned that these factors and their impact on preferences are often unknown for researchers and, hence, difficult to control for in PP studies.11,12

All this empirical evidence and the practical demands called for a need of identifying, describing and assess the feasibility of using different ways to profile psychological variables that, alongside demographic (eg, age, gender, culture) and clinical (eg, disease stage, comorbidity) characteristics of patients, can affect the construction, elicitation, and interpretation of PP. With this purpose, a systematic review was carried out within the PREFER project to explore the existing instruments which are currently used in PP studies or health-related decision studies to measure the psychological variables that can affect the formation of PP, its elicitation and its heterogeneity.13 This initial step provided an overview of the literature on the psychological constructs measured in PP or health-related decisions studies and the instruments used to measure these constructs. This review resulted in the identification of 18 constructs and 33 psychological instruments indicating that health literacy, numeracy, and locus of control have an impact on health-related preferences and decisions The present work builds upon these results and aims at providing researchers and stakeholders with useful criteria and tools to select psychological constructs to include in PP studies to take into account patient heterogeneity and preferences formation. Specifically, three main tasks were undertaken to achieve this goal: (i) identifying a list of candidate psychological constructs by supplementing the results of the PREFER systematic review13 with input from experts in the field of PP studies, health psychology, and psychological assessments; (ii) proposing recommendations on which psychosocial constructs to include in PP studies reached through a consensus-based process, that is a two-round Delphi method; (iii) advancing a set of criteria and a check list that can assist in describing and selecting psychological instruments to be included in PP studies.

Psychological Constructs Linked to Patient Preferences

To provide researchers and other stakeholders with practical advice on psychological constructs that can influence PP formation and its heterogeneity, we propose a list of psychological constructs empirically and theoretically linked to PP. Specifically, a pool of 19 international experts in the fields of PP studies, health psychology, and psychological assessment were invited to review an existing list of psychological constructs identified during an earlier systematic review conducted within the PREFER project.13 Reviewers were internationally recognised experts both within and external to the PREFER project who were not involved in the original systematic review. The reviewers were invited to contribute via e-mail. Out of 19 experts contacted, 11 provided their feedback. Participants were provided with the list of psychological constructs from the systematic review, and they were asked to supplement the list with: Psychological dimensions that can empirically or theoretically have a bearing on patients’ preferences formation and/or explain PP heterogeneity. Three authors (SR, DM, GP) independently analysed the experts’ contributions to evaluate whether the suggested constructs met the aforementioned inclusion criteria. Disagreements in construct selection were resolved through discussion between the researchers until consensus was reached. The panel of experts indicated 13 constructs which supplemented the 18 constructs identified in the systematic review leading to the identification of a total of 31 psychological constructs which could be relevant for future PP studies (Table 1).

Table 1 List and Definition of Psychological Constructs Identified by the Systematic Literature Review from Russo et al13 and the Panel of Experts. They are Reported in Alphabetic Order

We grouped the psychological constructs considering the available scientific evidence supporting the relevance of the psychological constructs for PP studies. Four researchers (UK, DM, CAP, SR) organised the constructs in three classes basing their grouping on the quality and soundness of the available evidence of the association between psychological constructs and PP within the context of decision-making processes. In particular, to classify these psychological constructs the researchers considered: (i) the number of studies assessing the association between each psychological construct and PP; (ii) the quality of these studies; (iii) the overall agreement and consistency of their finding. For each identified construct emerging from the systematic review an overall rating of the quality of the empirical evidence was considered. In Russo et al13 each study received a score based on its quality ranging from 1 to 3 (1=weak; 2=moderate; 3=strong), then summed to the score of the other studies investigating the same construct and the mathematical average of the resulting value was categorised as follow: from 1 to 1.4 weak; from 1.41 to 1.8 weak to moderate; from 1.81 to 2.2 moderate; from 2.21 to 2.6 moderate to strong; from 2.61 to 3 strong. The constructs suggested by the experts were included in the Group C as no empirical data are yet available on the association between these constructs and PP (see Table 2). Disagreements in grouping between researchers were resolved through discussion until consensus was reached.

  • Group A: psychological constructs for which there are strong and consistent results regarding their role in influencing patients’ preferences and decisions.
  • Group B: psychological constructs that could be theoretically promising in the field of PP studies, but the number of available studies with consistent results is not yet satisfactory.
  • Group C: psychological constructs for which data are not available, or there are inconclusive or inconsistent results on their role in influencing PPs and decisions.

Table 2 List of Identified Psychological Constructs Organised by Strength of Available Empirical Evidence (Alphabetically Ordered Within Each Class)

Development of Recommendations: A Delphi Consensus Process

To further assist researchers and stakeholders with the selection of psychological constructs for preference studies, we developed recommendations based on the empirical evidence and experts’ contributions previously gathered using a consensus-based process. Formal expert review via an online two-round Delphi panel was used to reach consensus.50

Materials and Procedures

Although there is no standardized sample size for the Delphi approach, 8 to 15 panel members may be adequate for a focused Delphi study51 such as the one reported here. Kezar and Maxey52 suggest that a smaller expert panel is appropriate when the panel is heterogeneous and up to 30 or more panel members may be required with a homogenous panel of experts. Accordingly, a minimum sample of 15 experts was targeted for the panel. Researchers and professionals with a diverse range of expertise were invited via email to participate in the online Delphi consensus process. Fields of expertise included health preference research, clinical and health psychology, health economics, public health, risk communication, and decision-making. Panel members were recruited through searching relevant literature and recommendations from the project team. Additionally, five international organisations (European Health Psychology Society, Society for Health Psychology, European Association for Communication in Healthcare, Society for Medical Decision Making, and International Society for Pharmacoeconomics and Outcomes Research) were contacted and asked to forward the Delphi survey invitation to their associates. No remuneration was offered to participants. Participants were sent a web link to the consent form and once informed consent was provided, participants were presented with the study survey.

Participants were asked to provide demographic and professional information and rate their level of agreement on considering and measuring each of the 31 constructs in PP studies (Table 1) on a 5-point Likert scale (“strongly disagree” to “strongly agree”) and also provided open text space for participants to give reasons for their evaluations. For each psychological construct outlined, the proportion of panel members that either agreed or disagreed with the inclusion/usefulness of each corresponding construct in PP studies was calculated. Criteria for the level of consensus were defined a priori based on agreement responses on the Likert scale [“agree” or “strongly agree”], and included the following categories of consensus: “unanimity” of inclusion, “consensus”, “majority” and “discrepancy” when 100%, 80%, ≥70%, and <70% of participants agreed with inclusion. Two drag-and-drop items asked participants to indicate which of the outlined psychological constructs could account for preference heterogeneity and misunderstanding differences in PP, respectively. “Preference heterogeneity” was defined as differences in preferences across sub-populations or classes of people, while “misunderstanding differences” were defined as differences in patient preferences due to patients not understanding or not interpreting the patient preference study questions and information as meant by the researcher. After presenting the two operational definitions, participants were also asked to select one of five figures visually presenting the possible relations between the two concepts (see Figure 1).

Figure 1 Item asking participants’ understanding of the relations between heterogeneity differences and misunderstanding differences in patient preference studies. Note: (A) Hetereogeneity differences as subset of misunderstanding differences; (B) misunderstanding differences as subset of heterogeneity differences; (C) misunderstading and heterogeneity differences completley overlap; (D) misunderstanding and heterogeneity differences are completely distinct; (E) misunderstading and heterogeneity differences partially overlap.

Participants who completed the questionnaire in round one of the Delphi survey were invited via email to complete round two of the survey. For each construct, participants were presented with the rating they provided during round one and the overall mean and standard deviation from the panel and were asked to rate again their level of agreement for inclusion of each construct in PP studies using the same 5-point Likert scale (“strongly disagree” to “strongly agree”).


Delphi Panel Demographic and Professional Features

Twenty-seven out of the 34 experts who participated in round one participated in round two of the Delphi survey. As reported in Table 3, panel members had a mean age of 41.19 years (SD= 9.77) and 59.3% were female. They mainly worked in the European Union (41%), followed by North America (33%), the United Kingdom (15%), with the remainder split among Asia, Australia, and Switzerland. The panel was heterogeneous in terms of both work sector and field of expertise. Most panel members (74%) worked in the academic sector, followed by industry, government agencies, and other sectors (ie, consulting services and hospitals). When considering field of expertise, participants were allowed to provide more than one area of expertise if applicable: 41% of panel members declared to work in the health psychology, 15% in clinical psychology, 22% in risk communication, 30% in medical decision-making, 30% in other types of decision-making, 26% in public health, and 22% in other fields (ie, health preference research, medical education, and sociology). The average years of experience in their field of expertise was 13.48 (SD= 8.40).

Table 3 Delphi Panel Sociodemographic and Professional Characteristics

Preference Heterogeneity and Misunderstanding Differences

In round one of the Delphi survey, experts were asked to report their opinion regarding the relationship between PP heterogeneity and misunderstanding differences in PPs by choosing one among five figures graphically representing different relationship between them (Figure 1). Most experts (44%) reported that PP heterogeneity and misunderstanding differences in PPs are different but overlapping aspects (Figure 1E); 26% that they are different and not overlapping phenomena (Figure 1D); 18% that misunderstanding differences is a specific subtype of PP heterogeneity (Figure 1B); 7% that PP heterogeneity and misunderstanding differences refer to the same concept (Figure 1C), and % reported that PP heterogeneity is a specific subtype of misunderstanding differences (Figure 1A).

Delphi Consensus

Based on the results of round two of the Delphi survey, three classes of consensus-based recommendation on inclusion of the listed psychological constructs in PP were created (Table 4):

  • Class I: psychological constructs for which there is unanimity or consensus regarding their inclusion in PP studies. The evaluation of these constructs is recommended for inclusion in PP studies when deemed relevant to address the research question and describing preference heterogeneity.
  • Class II: psychological constructs for which the majority of experts agreed for their inclusion in PP studies. The evaluation of these constructs could be recommended to understand PP in a healthcare setting. The decision about including or not including a psychological construct should be made by considering: its theoretical link with PP, the previous empirical evidence about its role in influencing PP, the possible cognitive load for patients in including a further instrument to complete and other feasibility criteria (eg timelines, costs).
  • Class III: psychological constructs for which there is discrepancy among experts regarding their inclusion in PP studies. Therefore, their evaluation could not be recommended based on current expert evaluation to understand PP in a healthcare setting.

Table 4 Consensus-Based Recommendations at the Two Rounds of the Delphi Method

As reported in Table 4, the percentages of agreement at the second round of the Delphi ranged between 11% and 96%, with the lowest percentage for the inclusion of mood states and sense of coherence, and the highest percentages for the inclusion of health literacy. Specifically, consensus was reached for the inclusion of eight psychological constructs, namely health literacy, health numeracy, illness perception, treatment-related beliefs, risk propensity, health locus of control, control preference, and patient activation (Class I). Three constructs, namely autonomy preference, decision-making style, and health orientation, reached the majority consensus (Class II). Twenty out of the 31 psychological constructs considered by the Delphi panel did not reach consensus about their inclusion or exclusion when considering PP studies falling into the Class III of recommendation.

As reported in Table 5 amongst the 11 constructs included in Class I and II, health literacy and health numeracy were mainly conceptualized as psychological constructs that can account for understanding differences in patient preferences due to patients not understanding or not interpreting the patient preference study questions and information as meant by the researcher (ie, misunderstanding differences in PPs). On the contrary, the remaining nine constructs were mainly considered as relevant psychological characteristics that could account for PP heterogeneity.

Table 5 Percentages of Selection of the Class I and Class II Psychological Constructs as Relevant Information to Account for PP Heterogeneity or Misunderstanding Differences in PPs

Descriptive Criteria and Checklist for Selecting Measurement Instruments

After deciding which construct(s) to measure depending on evidence- and consensus-based considerations, stakeholders and researchers should appraise available instruments in terms of quality, usability, restrictions, and shortcomings and select the best tool to evaluate these candidate psychological differences. This step is especially relevant when more than one instrument is available for the same construct.

We therefore propose a set of descriptive criteria (see Supplementary Material A) and a criteria checklist (see Supplementary Material B) to assist researchers and other stakeholders respectively in i) assessing the overall quality and usability of each tool, comparing different instruments when more than one instrument is available for the same construct, and selecting the best measure and ii) evaluating the feasibility of measuring candidate psychological differences in the context of a specific PP study. In detail, two researchers (SR, DM) developed a set of descriptive criteria and a checklist to evaluate psychological instruments and to guide researchers to decide which tool best fit the research needs of a specific PP study. The initial lists of descriptive criteria and the initial draft of the checklist were extended and revised based on conversations and feedback from the authors and other members of PREFER consortium with expertise and experience in the fields of PP studies, health psychology, and psychological assessment.

Starting from the final list of descriptive criteria (see Supplementary Material A), here we are proposing a briefer set of questions addressing the main features of psychological instruments that could support stakeholders and researchers in evaluating and selecting available measures. This checklist – reported in Table 6 - may be especially helpful when more than one instrument is available for the same construct. By given answers to each question, stakeholders are assisted in assessing the overall quality and usability of each tool.

Table 6 Brief Checklist for the Evaluation and Selection of Available Measures to Assess Psychological Constructs

Strong knowledge and skills concerning psychometrics, psychological assessment and testing are essential to properly evaluate available psychological measures and select the best measurement tool based on its features and psychometric characteristics. Specifically, the first criterion advanced deals with two essential features of any psychological tool: validity and reliability. They refer respectively to the extent to which any tool actually measures what it is intended to measure53 and the precision of psychological measures in terms of internal consistency or the consistency of observed scores over different administration of the same instrument.54 Since validity and reliability of an instrument should be investigated and demonstrated in a specific context of use, empirical evidence coming from previous validation studies in similar contexts of the PP study should be considered to assess the appropriateness of candidate psychological measures. Stakeholders are recommended to use only those psychological tools with consistently proven reliability and validity in the specific population under investigation in the PP study. Second, psychological instruments are generally developed, constructed, and validated for a specific language and culture. They can be adapted for use in a new country, culture, and/or language through “cross-cultural adaptation”, namely a unique and complex method to reach equivalence between the original and the translated versions of the instrument.55 Caution should be taken when using in a PP study a translated version of an instrument that has not been properly adapted to the new language and culture. Thus, stakeholders are always recommended to choose only those psychological instruments specifically developed or properly adapted to the language and culture of the target population of their PP study. Third, since psychological instruments are often constructed and validated for a specific population of patients, caution should be taken when using the tool to a different population without previous empirical evidence regarding its validity and reliability in the new population. Thus, stakeholders are recommended to prefer those psychological instruments that have been developed specifically for the population being targeted by the PP study over other available tools. Fourth, while some psychological instruments return only raw scores that may be difficult to interpret without knowledge of how one raw score compares to a norm group, other psychological tools adopt a reference group and can provide standardized scores. The latter might help stakeholders to get a clearer picture of the position of each patient in a predefined population for the psychological differences being measured. Thus, if relevant, psychological measures with standardized and norm-referenced scores should be preferred over instruments with only raw scores. Fifth, some psychological tools can also provide meaningful cut-off scores to classify patients into groups based on their scores. These cut-off scores are generally used with screening purposes to differentiate clinical populations from non-clinical ones or differentiate among people with adequate or inadequate abilities. If relevant, stakeholders are recommended to prefer psychological measures providing cut-off scores over tools providing only raw or standardized scores. Sixth, while the noncommercial use of psychological instruments is often allowed without requiring written permission, a license or a fee, sometimes developers require explicit approval to use their instrument.56 Other authors use the copyright status and put limits on use, adaptations, and translations. Finally, other instrument developers can charge fees to use their tools, namely licensing fee, administration fee or fee for obtaining scoring instructions. When budget constraints do not allow the use of psychological instruments with fees, stakeholders might consider alternative measures with equivalent characteristics but not requiring payment. Finally, the protection of people participating in research demands investigators and regulatory bodies to minimize the respondent burden.57 Respondent burden is a multifaceted phenomenon encompassing mainly cognitive burden and time commitment to take part in a study,58 as well as participants’ perceived psychological, physical, and economic discomfort caused by the participation in research.59 Psychological instruments can vary in both their completion time and required cognitive effort for completion. Thus, stakeholders should balance the benefits of including psychological profiling within PP studies with the possible additional respondent burden. Moreover, when two psychological instruments are deemed to be equivalent in their informative values, the less cognitively burdensome and time-consuming tool should be preferred over the more cognitively and time demanding.

We would like to stress that the criteria and checklist advanced here is not intended to be rigidly applied to PP studies but rather to function as an initial guide to support PP researchers and other relevant stakeholders in selecting psychological instruments to be included in PP studies.

Discussion and Limitations

The present paper provides scholars and other stakeholders involved in the field of PP studies with consensus-based recommendations for the inclusion of psychological dimensions in PP studies and a checklist for the appraisal and selection of psychological instruments to assess the candidate psychological constructs. Building upon literature on the relationship between psychological dimensions and PP and on contributions from experts, we identified a list of 31 psychological constructs which could be relevant when planning PP studies. A panel of experts in PP studies and related disciplines enlarged a list of psychological constructs that emerged from a systematic literature on psychological dimensions known to have an impact on patients’ preferences and health-related decisions. To provide the reader with a snapshot of the current scientific data on the relationship of each construct with PP, the authors grouped the final list of psychological constructs in three clusters according to the strength and soundness of the existing empirical evidence.

Furthermore, the psychological dimensions underwent a consensus-based process to identify those psychological constructs to be recommended for inclusion in PP studies. Consensus for inclusion was reached for 11 psychological constructs. A consensus-based classification of the psychological constructs also emerged. Three classes of recommendation have been detected. Class I-constructs, namely control preference, health literacy, health locus of control, health numeracy, illness perception, patient activation, risk propensity, and treatment-related beliefs have been recognised as those with a theoretical basis for forming preferences and predicting preference heterogeneity. These constructs should be a basic consideration for inclusion in preference studies conducted for medical product decision-making. Constructs in Class II (autonomy preference, decision-making style, and health orientation, reached the majority consensus) have been considered as promising constructs to better understand PPs. In Class III are listed psychological constructs for which there was not consensus among experts regarding their inclusion in PP studies. Our expert panel did not recognise the potential contribution of these psychological constructs in shedding light on PP formation or heterogeneity. Because of lack of consensus, their evaluation could not be recommended. Nevertheless it has to be noted that expert consensus on inclusion might fail to be reached due to lack of scientific evidence on the relationship between PP and the psychological dimensions considered. Out of the 11 psychological dimensions reaching expert agreement for inclusion, 10 belong to Group A or B showing association between expert-consensus and the empirical data. Psychological constructs that have an important role in PPs may not yet have been studied systematically. Moreover, it should be underlined that available empirical evidence concerning the impact of psychological constructs on PP is still limited to specific contexts, patient populations, or diseases. Thus, the generalizability of these results to other medical conditions might not be guaranteed. More investigations are needed on the link between PP and psychological dimensions. It is particularly relevant to explore how particular psychological dimensions connect to PP heterogeneity. As the Medical Device Innovation Consortium highlighted10 differences in PP amongst subgroups of patients may be highly relevant to guide decision in medical products and devices approval process. From this perspective, detecting patient preference heterogeneity can be considered as a complement of understanding heterogeneity of treatment effects based on other observable characteristics because it might assure that sponsors and regulators have appropriately identified the specific population for which the medical product should be indicated. Understanding patient preference heterogeneity might be especially relevant if the benefits of a medical product outweigh its risks only for a subgroup of patients but not for others. In this case, it would be meaningful to infer the size of the overall patient population presumably outweighing benefits over risks and identify specific patients’ characteristics that are associated with the likelihood of being in this group.10 The identification of heterogeneity in patient preferences might inform patient subgroup considerations as part of the benefit-risk assessments.1 Specifically, it is relevant to identify any potential population-, condition-, and treatment-variability in patient preferences to obtain a clear picture of preferences of patients from the full spectrum of disease for which the medical product is intended to be used. Working in this direction our study explored how our expert panel members understand the role that psychological constructs can play in explaining differences in PP. In particular, in the context of PP studies, we asked them which psychological constructs could explain both preference heterogeneity and possible differences springing from misunderstanding between experimental requests and patients’ understanding of those.

Amongst the 11 constructs that reached agreement on inclusion, health literacy and health numeracy were the ones considered the more informative when it comes to differences due to misunderstanding. For example, in a PP study aimed to determine maximal acceptable risk levels for a certain treatment in elderly people with a chronic disease, the assessment of health literacy and numeracy may be especially relevant to identify patients that may not able to properly understand complex numerical and/or medical information or the scenario proposed by the researcher.

We furthermore investigated what relationship our experts assumed exist between heterogeneity in PP defined as “differences in preferences across sub-populations or classes of people” and differences in misunderstanding differences defined as “differences in patient preferences due to patients not understanding or not interpreting the patient preference study questions and information as meant by the researcher.” The majority of the expert panel members reported that the two phenomena are similar, sometimes overlapping, but distinct.

Further research is necessary to expand the recommendations advanced here and base them on scientific data to further them towards more comprehensive and evidence-based guidance. Specifically, the classes of recommendation identified here are a first attempt to develop a common framework to further facilitate sharing of information and the accumulation of evidence to demonstrate how a specific psychological construct relates to PPs and which measures should be considered within a given context. It is particularly relevant to explore how a particular psychological dimension connects to PP heterogeneity.

Once the psychological dimensions to be investigated and included in a PP study have been identified, researchers need to select appropriate measurement instruments. We have suggested a checklist and set of descriptive criteria to assist and guide scholars in the selection of relevant psychological instruments to be include in PP studies. It should be stressed that the applicability and usefulness of the checklist and descriptive criteria advanced here are not confined to the assessment of PP. Our checklist and criteria are all grounded in the common need for good study design principles and are intended to support researchers to select or develop an optimal instrument to address specific research questions. Overall, the decisions concerning the inclusion of psychological constructs and related psychological tools for their measurement should be made by a multidisciplinary team with clinical and psychological expertise, and study management experience. Specifically, psychological knowledge is needed to identify relevant psychological differences that can be related to the formation and heterogeneity of patient preferences. Moreover, strong knowledge and skills concerning psychological assessment and measurement are essential to properly evaluate available psychological measures and select the best measurement tool based on its psychometric characteristics, such as overall validity, reliability, and measurement properties.


The list of psychological constructs identified in this study, along with the classes of recommendations, the checklist and descriptive criteria to evaluate psychological measures are valuable tools to assist researchers and stakeholders when designing PP studies in order to obtain results that can inform decision-making along the medical product lifecycle and enable the implementation of personalised medicine and patient-centred care. The classification and criteria advanced here are meant to assist and to be considered in light of existing guidelines on qualities and characteristics of PP information from relevant agencies and authorities such as the FDA’s Center for Devices and Radiological Health guidance on Patient Preference Information.60 The present study serves as a common framework to stimulate further research, and foster reflection and discussion on the formalisation of guidelines to assist stakeholders in whether and how to include psychological dimensions and measurements when designing PP studies.


We would like to thank the PREFER members of the work package 2.5 who contributed providing their precious insight and expertise: Jorien Veldwijk, Rosanne Janssens, Chiara Jongerius, Isabelle Huys, Richard Hermann, Flavia Faccio, Silvia Pizzoli. We furthermore are deeply grateful to the experts who contributed to the study as either advisory experts or Delphi panel members.


This research was supported by The Patient Preferences in Benefit-Risk Assessments during the Drug Life Cycle (PREFER) project. PREFER has received funding from the Innovative Medicines Initiative 2 Joint Undertaking under grant agreement No 115966. This Joint Undertaking receives support from the European Union’s Horizon 2020 research and innovation programme and EFPIA. This text and its contents reflects the author’s and PREFER project’s view and not the view of IMI, the European Union or EFPIA.


The authors report no conflicts of interest in this work.


1. FDA. Patient preference information – voluntary submission, review in premarket approval applications, humanitarian device exemption applications, and de novo requests, and inclusion in decision summaries and device labeling: guidance for industry, food and drug administration staff, and other stakeholders. U.S. Department of Health and Human Services Food and Drug Administration, Center for Devices and Radiological Health and Center for Biologics Evaluation and Research, ed. Available from: Accessed May 25, 2021.

2. EMA. The Patient’s Voice in the Evaluation of Medicines. European Medicines Agency, Stakeholders and Communication Division; 2013. Available from: Accessed April 14, 2021

3. US Department of Health and Human Services Food and Drug Administration, Center for Devices and Radiological Health and Center for Biologics Evaluation and Research. Patient preference information—voluntary submission, review in premarket approval applications, humanitarian device exemption applications, and de novo requests, and inclusion in decision summaries and device labeling: guidance for industry, food and drug administration staff, and other stakeholders. Available from: Accessed May 25, 2021.

4. Elwyn G, Frosch D, Rollnick S. Dual equipoise shared decision making: definitions for decision and behaviour support interventions. Implement Sci. 2009;4(1):1–8. doi:10.1186/1748-5908-4-75

5. Cutica I, Mc Vie G, Pravettoni G. Personalised medicine: the cognitive side of patients. Eur J Intern Med. 2014;25(8):685–688. doi:10.1016/j.ejim.2014.07.002

6. Kondylakis H, Kazantzaki E, Koumakis L, et al. Development of interactive empowerment services in support of personalised medicine. Ecancermedicalscience. 2014;8:8. doi:10.3332/ecancer.2014.400

7. Renzi C, Riva S, Masiero M, Pravettoni G. The choice dilemma in chronic hematological conditions: why choosing is not only a medical issue? A psycho-cognitive perspective. Crit Rev Oncol Hematol. 2016;99:134–140. doi:10.1016/j.critrevonc.2015.12.010

8. de Bekker-grob EW, Berlin C, Levitan B, et al. Giving Patients’ Preferences a Voice in Medical Treatment Life Cycle: The PREFER Public–Private Project. Springer; 2017.

9. Huls SP, Whichello CL, van Exel J, Uyl-de Groot CA, de Bekker-grob EW. What is next for patient preferences in health technology assessment? A systematic review of the challenges. Value Health. 2019;22(11):1318–1328. doi:10.1016/j.jval.2019.04.1930

10. (MDIC) MDIC. Patient centered risk-benefit project report. Available from:

11. Whichello C, Van Overbeeke E, Janssens R, et al. Factors and situations affecting the value of patient preference studies: semi-structured interviews in Europe and the US. Front Pharmacol. 2019;10:1009. doi:10.3389/fphar.2019.01009

12. Janssens R, Russo S, van Overbeeke E, et al. Patient preferences in the medical product life cycle: what do stakeholders think? Semi-structured qualitative interviews in Europe and the USA. Patient. 2019;12(1):1–14. doi:10.1007/s40271-018-0324-6

13. Russo S, Jongerius C, Faccio F, et al. Understanding patients’ preferences: a systematic review of psychological instruments used in patients’ preference and decision studies. Value Health. 2019;22(4):491–501. doi:10.1016/j.jval.2018.12.007

14. Lazarus RS. Progress on a cognitive-motivational-relational theory of emotion. Am Psychol. 1991;46(8):819. doi:10.1037/0003-066X.46.8.819

15. Spielberger CD, Gorsuch RL, Lushene RE. Manual for the state-trait anxiety inventory 1970; 2018. Available from: Accessed September 8, 2018.

16. Lazarus RS. Emotion and Adaptation. Oxford University Press on Demand; 1991.

17. Rakos RF. Assertive Behavior: Theory, Research, and Training. Taylor & Frances/Routledge; 1991.

18. Deci EL, Ryan RM. The support of autonomy and the control of behavior. J Pers Soc Psychol. 1987;53(6):1024. doi:10.1037/0022-3514.53.6.1024

19. Kasser VG, Ryan RM. The relation of psychological needs for autonomy and relatedness to vitality, well‐being, and mortality in a nursing home 1. J Appl Soc Psychol. 1999;29(5):935–954. doi:10.1111/j.1559-1816.1999.tb00133.x

20. Gray J, McNaughton N. The Neuropsychology of Anxiety. Oxford University Press; 1982.

21. Suziedelis A, Lorr M. Conservative attitudes and authoritarian values. J Psychol. 1973;83(2):287–294. doi:10.1080/00223980.1973.9915616

22. Degner LF, Sloan JA, Venkatesh P. The control preferences scale. Can J Nurs Res. 1997;29(3).

23. Compas BE. Coping with stress during childhood and adolescence. Psychol Bull. 1987;101(3):393. doi:10.1037/0033-2909.101.3.393

24. Driver MJ. Individual decision making and creativity. Organ Behav. 1979;59–91.

25. Harren VA. A model of career decision making for college students. J Vocat Behav. 1979;14(2):119–133. doi:10.1016/0001-8791(79)90065-4

26. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 5th ed. Arlington, VA: American Psychiatric Publishing; 2013.

27. Scheier MF, Carver CS. Optimism, coping, and health: assessment and implications of generalized outcome expectancies. Health Psychol. 1985;4(3):219. doi:10.1037/0278-6133.4.3.219

28. Baker DW, Williams MV, Parker RM, Gazmararian JA, Nurss J. Development of a brief test to measure functional health literacy. Patient Educ Couns. 1999;38(1):33–42. doi:10.1016/S0738-3991(98)00116-5

29. Wallston KA, Strudler wallston B, DeVellis R. Development of the multidimensional health locus of control (MHLC) scales. Health Educ Monogr. 1978;6(1):160–170. doi:10.1177/109019817800600107

30. Reyna VF, Nelson WL, Han PK, Dieckmann NF. How numeracy influences risk comprehension and medical decision making. Psychol Bull. 2009;135(6):943. doi:10.1037/a0017327

31. Zikmund-Fisher BJ, Smith DM, Ubel PA, Fagerlin A. Validation of the subjective numeracy scale: effects of low numeracy on comprehension of risk communications and utility elicitations. Med Decis Mak. 2007;27(5):663–671. doi:10.1177/0272989X07303824

32. Dutta M, Bodie G, Basu A. Health disparity and the racial divide among the nation’s youth: internet as a site for change? In: John D, Catherine T, editors. MacArthur Foundation Series on Digital Media and Learning. Cambridge, MA: The MIT Pr; 2008:175–198.

33. Baker K, Jaksic S, Rowley D. The self-regulation model of illness representation applied to stuttering. J Psychosom Res. 1995;60:631–637.

34. Morgan GA, Harmon RJ, Maslin-Cole CA. Mastery motivation: definition and measurement. Early Educ Dev. 1990;1(5):318–339. doi:10.1207/s15566935eed0105_1

35. Kleinstäuber M. Mood. In: Encyclopedia of Behavioral Medicine. New York: Springer; 2013:1259–1261.

36. Kruglanski AW, Webster DM. Motivated closing of the mind: ‘seizing’ and ‘freezing’. Psychol Rev. 1996;103(2):263–283. doi:10.1037/0033-295X.103.2.263

37. Cacioppo JT, Petty RE, Feinstein JA, Jarvis WBG. Dispositional differences in cognitive motivation: the life and times of individuals varying in need for cognition. Psychol Bull. 1996;119(2):197. doi:10.1037/0033-2909.119.2.197

38. Cacioppo JT, Petty RE, Feng Kao C. The efficient assessment of need for cognition. J Pers Assess. 1984;48(3):306–307. doi:10.1207/s15327752jpa4803_13

39. Hibbard JH, Mahoney ER, Stockard J, Tusler M. Development and testing of a short form of the patient activation measure. Health Serv Res. 2005;40(6p1):1918–1930. doi:10.1111/j.1475-6773.2005.00438.x

40. Allport GW. Pattern and Growth in Personality. New York: Holt, Rinehart & Winston; 1961.

41. Ryff CD, Singer BH. Best news yet on the six-factor model of well-being. Soc Sci Res. 2006;35(4):1103–1119. doi:10.1016/j.ssresearch.2006.01.002

42. Pacini R, Epstein S. The relation of rational and experiential information processing styles to personality, basic beliefs, and the ratio-bias phenomenon. J Pers Soc Psychol. 1999;76(6):972. doi:10.1037/0022-3514.76.6.972

43. American Psychological Association. The road to resilience 2014; 2017. Available from: Accessed May 25, 2021.

44. Mellers BA, Cooke AD. The role of task and context in preference measurement. Psychol Sci. 1996;7(2):76–82. doi:10.1111/j.1467-9280.1996.tb00333.x

45. Bandura A. Self-efficacy. In: Ramachandran VS, editor. Encyclopedia of Human Behavior. Vol. 4. San Diego: Academic Press; 1994:71–81.

46. Zuckerman M. Behavioral Expressions and Biosocial Bases of Sensation Seeking. Cambridge university press; 1994.

47. Antonovsky A, Sourani T. Family sense of coherence and family adaptation. J Marriage Fam. 1988;50(1):79–92. doi:10.2307/352429

48. Cohen S. Social relationships and health. Am Psychol. 2004;59(8):676. doi:10.1037/0003-066X.59.8.676

49. Horne R, Weinman J. Patients’ beliefs about prescribed medicines and their role in adherence to treatment in chronic physical illness. J Psychosom Res. 1999;47(6):555–567. doi:10.1016/S0022-3999(99)00057-4

50. Keeney S, McKenna H, Hasson F. The Delphi Technique in Nursing and Health Research. John Wiley & Sons; 2011.

51. Vernon W. The delphi technique: a review. Int J Ther Rehabil. 2009;16(2):69–76. doi:10.12968/ijtr.2009.16.2.38892

52. Kezar A, Maxey D. The delphi technique: an untapped approach of participatory research. Int J Soc Res Methodol. 2016;19(2):143–160.

53. Carmines EG, Woods J. Validity. In: Lewis-Beck M, Liao TF, editors. The Sage Encyclopedia of Social Science Research Methods. 2003.

54. Gushta MM, Rupp AA. Reliability. In: Salkind NJ, editor. Encyclopedia of Research Design. 2010.

55. Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine. 2000;25(24):3186–3191. doi:10.1097/00007632-200012150-00014

56. Hays RD, Weech-Maldonado R, Teresi JA, Wallace SP, Stewart AL. Commentary: copyright restrictions versus open access to survey instruments. Med Care. 2018;56(2):107–110. doi:10.1097/MLR.0000000000000857

57. Lingler JH, Schmidt K, Gentry A, Hu L, Terhorst L. Perceived Research Burden Assessment (PeRBA): instrument development and psychometric evaluation. J Empir Res Hum Res Ethics. 2014;9(4):46. doi:10.1177/1556264614545037

58. Sharp LM, Frankel J. Respondent burden: a test of some common assumptions. Public Opin Q. 1983;47(1):36–53. doi:10.1086/268765

59. Ulrich CM, Wallen GR, Feister A, Grady C. Respondent burden in clinical research: when are we asking too much of subjects? Ethics Hum Res. 2005;27(4):17–20. doi:10.2307/3563957

60. CDRH F. Patient Preference information–voluntary submission, review in premarket approval applications, humanitarian device exemption applications, and de novo requests, and inclusion in decision summaries and device labeling. Food and Drug Administration Staff, and Other Stakeholders. 2016.

Creative Commons License This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.