A Method for Designing a Patient Burden Questionnaire in Dermatology
Received 28 April 2020
Accepted for publication 7 July 2020
Published 6 August 2020 Volume 2020:13 Pages 521—528
Checked for plagiarism Yes
Review by Single anonymous peer review
Peer reviewer comments 3
Editor who approved publication: Dr Jeffrey Weinberg
Khaled Ezzedine,1 Mohammed Bennani,2 Jason Shourick,3 Charles Taieb4,5
1University Hospital Henri-Mondor, Department of Dermatology, Créteil, France; 2Qualees, Paris, France; 3University of Paris-Sud, Paris, France; 4Necker Enfants Malades Hospital, Paris, France; 5European Market Maintenance Assessment, Paris, France
Correspondence: Charles Taieb
European Market Maintenance Assessment, 94120 Fontenay-sous-Bois, Paris, France
Tel +33 771 772 100
Email [email protected]
Introduction: In recent years, the concept of “disease burden” has been given a central role in evaluating patient care, particularly in skin diseases. Measuring patient-reported outcomes (PRO) such as symptoms and disease burden may be useful.
Aim: To present a methodology that facilitates the development and validation of burden questionnaires for patients suffering from skin diseases.
Methodology: Based on past published burden questionnaires, a methodology for designing skin disease burden questionnaires was to be developed.
Results: Based on 16 burden questionnaires developed and published over the last 10 years, the authors propose a standardized methodology for the easy design and validation of disease burden questionnaires for patients with chronic skin diseases. The authors provide detailed guidance for the conception, development and validation of the questionnaires, including reliability, internal consistency, external validity, cognitive debriefing, testing–retesting, translation and cross-cultural adaptation, as well as for statistical analysis.
Conclusion: The proposed methodology enhances the design and validation of disease burden questionnaires in dermatology. Burden questionnaires may be used in clinical research as well as in daily clinical practice.
Keywords: individual burden, dermatology, method, patient-reported outcome, questionnaire
The “disease burden” concept is central for evaluating patient care, particularly so in skin diseases.1 It was introduced in 2010 and has since been proven to be particularly useful for quantifying population health and determining action priorities in the public health field.2 It focuses on the “individual disease burden”, deemed to assess disease “disability” in its broadest sense, including psychological, social, economic and physical features thus differing from Quality of Life issues covering five dimensions: physical wellbeing, material wellbeing, social wellbeing, emotional wellbeing, and development and activity.3,4
Certain chronic diseases require specific care that can onerous, costly and modify daily life.5 Thus, they can exclude the patients from their social and/or family environment. Moreover, other collateral effects, such as reduced working hours, loss of employment in more serious cases, and relationship and parental crises, frequently may not be considered by the patient as impacting their lives. Environmental factors may also have an impact on symptoms, including pain and the occurrence of complications such as photosensitivity, deficiencies, the cancer risk and infections.6–12
Individual disease burden has already been evaluated in several skin diseases using patient questionnaires such as topic dermatitis, vitiligo hand eczema, haemangioma, inherited ichthyosis, sensitive skin and very recently psoriasis.13–17
Even though it has become the main focus in the management of dermatological diseases, no methodology allowing the design of patient-oriented burden questionnaires in dermatology has been proposed so far.
The aim of the present article is to provide guidance regarding the design of disease-related burden questionnaires for patients with skin diseases.
The authors provide a detailed step-by-step methodology for building a patient-reported burden questionnaire, based on the authors’ recommendations for designing self-administered burden questionnaires described in articles published over the last 10 years for atopic dermatitis, inherited ichthyosis, haemangioma, vitiligo, epidermolysis bullosa, albinism, sensitive skin, psoriasis and hand eczema.14–30
Even though this method was developed prior to the publication of the COSMIN risk of bias checklist in 2018, items were verified in posteriori against this checklist.33
Any patient self-completion questionnaire must allow the target patient or care-giver-population to easily understand and answer all of its questions.
Ambiguous wording or complicated phrasing should be avoided and the number of questions should be limited to a reasonable number. The length of questions and their number should correspond to the target population; ie questionnaires and questions might be shorter for children or elderly patients than for adults or care-givers.
To ensure rigour throughout the development process, questionnaires should be designed by a multidisciplinary team, consisting of experts in the drafting and development of questionnaires, health care professionals (specialised physicians, general practitioners, nurses and patient associations), all having expertise in the management of patients suffering from the target pathology.
The design of burden questionnaires follows three steps: conception, development and validation (Figure 1).
Figure 1 Flow chart for building a burden questionnaire.
The conception of a verbatim is based on an extensive literature review and semi-structured face-to-face interviews with health care professionals in a first step, and with patients suffering from the target pathology in a second step.
Once the principal topics to be addressed have been identified, the proposed list should be confirmed by a target patient group.
This target patient group should be homogenous and recruited primarily through specialists, GPs, pharmacists or patient associations.
This confirmation identifies the main concerns raised by the patients, such as impact on work in general, impact on day-to-day work, stress, daily life, every day care, relationships with others, isolation, perception of other people, and the economic impact of treatment (direct and indirect impact-managing consequences).
Based on confirmed topics, a first questionnaire is designed by the working group, allowing the conversion of individual comments or complaints into simple and direct questions. The working group may meet several times in order to simplify the questionnaire and to check for duplications.
Should no consensus be reached on a given question, then both versions should be listed; the most suitable question should be confirmed during development.
During the first part of the development phase, questions are developed based on items identified in the conception step leading to an almost finalised questionnaire content. The questionnaire’s responses are determined using a Likert scale including six responses (“never”, “rarely”, ‘sometimes’, “often”, “very often”, “always”).34 Even though there are other means, a 6-point Likert scale may be used, as this tool is easy to use and, based on our experience, the best adapted. Likert scales are frequently used in self-completion questionnaires and this response method is commonly identified by working groups as being the most pertinent method for the questionnaire in development.20,26,35,36 In order to limit missing data, an additional “not applicable” response should be added to the first six possible responses. Reliability of an evaluation scale increases with the number of response categories. Therefore, Guilford and Sellitz et al suggested standardising responses, thereby reducing the risk of distorted scores.37,38
During the second step, the questionnaire should be tested on a sample of target participants (patients or care givers). Care should be taken to avoid too large population diversity as this may lead to an alteration or weakening of the questionnaire’s sensitivity risks. Results may result in the replacement, deletion or merging of several questions into one single question. This may be due to the format of the question (too close or considered a complementary question). Moreover, items having too much or insufficient data should be rejected. The threshold value determining this rejection is determined based on the results of the analysis of the different items. Items for which the percentage of data is missing or “Not applicable” is too high should be removed; this should also apply for misunderstood or badly perceived items. Moreover, items correlating highly or too low with other items (>0.7 or 0.8 and<0.4, Pearson, Spearman, polychoric coefficient) should be rejected as well. The potential risk from retaining highly correlated items is that they may lead to the creation of dimensions uniquely based on these duplications and not on the real dimensions of the tool’s underlying structure. The risk from retaining items with low correlation is keeping those that do not measure the same underlying concept and would not allow for the factor analysis.
The assessment of the distribution of responses allows for the rejection of all items presenting with limitations or which result in non-adapted responses. The role of the working group during this second step is essential in order to ensure that significant items are not rejected or excluded or that there is a potential discrimination of conceptual aspects. For example, an item that is of interest for daily care may not be pertinent at the time the questionnaire is created. However, it may become pertinent once the treatment or management has been made available. Similarly, an item of interest for professional activities may not be pertinent to senior participants, but be applicable to others.
Factor analysis is undoubtedly the oldest method still used in psychometrics to confirm connections between factors and latent properties. It was introduced at the beginning of the 20th century by Spearman.39
Factor analysis offers two different techniques: principle component analysis and correspondence analysis.
When designing questionnaires, the principle component analysis technique is directed by the decision to use variables that are modelled as continuous with a Likert-type score, ranging from “not at all” to “completely”, with the possibility of intermediate responses and an additional category for “not applicable” to avoid non-responses as a result of hesitation or discomfort.
The principal component factor analysis is to simplify variables obtained from the transcripts of patient interviews by organizing them into groups. Multiple variables may measure the same one-dimensional construct (whether in reality or as a hypothesis suggested by experts). For example, “self-image”, “withdrawal”, or “relationship with others “, or that the same concept has multiple dimensions (eg the economic aspect would include work absenteeism and non-reimbursed medical and non-medical fees).
Thus, the exploratory factor analysis allows different items that measure the same “effect” to be grouped together, providing a method for summarizing these items into a synthetic score. Prior to an exploratory factor analysis, a Kaiser-Meyer-Olkin (KMO) analysis should be performed. This is to ensure that data are appropriate for the factor analysis. Therefore, all questions presenting a KMO of less than 0.5 should be excluded. Following this analysis, a scree plot is to be used in order to determine the number of dimensions (number of groups) in which each variable will be gathered.
Following this, an exploratory factor analysis will be performed. Once construct validity has been completed, reliability should be assessed through internal consistency checking and test–retesting.
To ensure internal reliability, the correlation of items to factors should be of 0.5 or above and item cross-correlations should not exceed 0.2. It is important to ensure that the questionnaire really measures what has been agreed to be measured.
- Internal consistency
The burden questionnaire is designed using multiple questions that are all different, or at least appear to be.
Therefore, analysing each question, the relationship between all questions, as well as the relationship between the questions and the overall questionnaire score, is essential.
This requires testing in order to confirm the consistency of the tool and that each of the components responds coherently to any given answer. For a given individual, the answers to questions such as “I feel lonely”, “I am isolated,” “I have no social relations” should not be radically different, even though they may be nuanced to achieve a level of sensitivity which is capable of recognizing a difference in the experiences of two different subjects.
Thus, internal validation corresponds to a confirmation that the questionnaire is capable of measuring a given construct in a coherent fashion. The Cronbach alpha fulfils this role.40 It provides a synthetic calculation of the average correlation between responses to different questions within a questionnaire, and of the average correlation between responses to questions that have been identified as measuring the same dimension. It provides a score between 0 and 1 with 1 corresponding to perfect consistency between questions. The internal validity of a questionnaire is considered to be adequate if the Cronbach’s alpha is 0.7 or higher. Conversely, a Cronbach’s alpha above 0.95 may be considered as suspect, and, in such a case, questioning the reasons behind this excessive validity may be adequate.41
A confirmatory factor analysis with a higher order factor should be performed. This is to ensure that subscales obtained using the exploratory factor analysis are appropriate and that items can be summed as an overall measure. Three values should be presented: root-mean-square error of approximation (RMSEA), Goodness-of-fit index (GFI) and Benley’s comparative fit index (CFI). GFI and CFI should be superior to 0.9 and at least 0.8, RMSEA should be inferior to 0.5. RMSEA can be highly dependent on the number of items, showing a tendency to be higher when the number of items is low.
An item response theory should be performed to ensure that Likert scales are correctly ordered and that each level of every scale corresponds to its own level of difficulty.
- Cognitive debriefing
Cognitive debriefing (CD) removes potential ambiguity leading to a question that could easily be misunderstood. It ensures that each question is understood in the same way by participants, regardless of their socio-cultural status.
CD should be applied to each newly designed questionnaire. Once CD has been completed, the final questionnaire should be tested on a sample of the native speaking population during a personal CD interview, in order to determine issues related to the wording of the questions and responses, such as ambiguity, misunderstanding, acceptability, etc. This testing should be managed by specialised institutions.
A CD produced by specialised institutions allows us to verify that the participant understands the questionnaire and to identify and resolve potential problems, such as the translation of a question that may be confusing or hard to understand.44 The goal is not to change the sense of the original question, but to express it clearly in the language into which it is being translated. A native examiner with a strong background in cognitive interviewing techniques should conduct each interview. Respondents should be sufficiently representative of those for whom the questionnaire has been designed and should be natives of the questionnaire’s language. This simple-to-use and easy-to-understand questionnaire should be considered the final version to be validated.
Verification should be performed according to the principles of good practice for the translation and cultural adaptation process for patient-reported outcome measures.45
- External validity
External validity measures the degree of certainty that the questionnaire achieves when measuring a given construct.
Several types of validation may be performed to establish the claim of external validity for a questionnaire. Nomological validity should be preferred. Indeed, nomological validity supports the claim of external validity for a questionnaire, and also provides a certain amount of additional, relevant information regarding the context and the environment of the subject.
The external structure’s validity is based on correlations of the tool, with one or more tools that have already been validated and for which pertinence has already been confirmed. The choice of questionnaire should be considered thoroughly. It is important that the questionnaire is validated and that it examines the agreed concepts with the specifically created tool.
External validity of the questionnaire should be studied by analysing its correlation with a specific questionnaire and with non-specific questionnaires such as the Short-Form health questionnaire-12 (SF-12), using the Pearson correlation.46 Correlation should range between 0.6 and 0.9. A correlation exceeding 0.9 may indicate that the questionnaire does not differ from the reference questionnaire.
Another common good practice is to ascertain the correlation of the questionnaire with variables known for their clinical impact on the patient’s burden, such as severity or specific localisation related to the disease’s burden such as the face or the extremities.
Test–retesting should be performed by distributing the same questionnaire to the same participant sample at two different times, to confirm that there are no significant changes between the two measures. The time interval between the original test and the retest should be short.
Reproducibility studies should be conducted to confirm the stability of scores in stable (state of health and treatment) subjects between two administrations. The calculation of the intra-class correlation coefficients (ICC) of responses given by the same individual qualified as stable at two distinct moments evaluates the Test–Retest stability.
Coefficients of at least 0.40 are considered correlated regardless of the chosen method. Results should help to understand how different items come together to form the dimensions. Different rotations (eg Varinax and Pronax) may be performed in order to allow determining factors. During this stage, items that do not belong to any dimension will be removed. Once these items are excluded, exploratory techniques should be applied once again and some items may be removed to produce a stable structure. An intra-class coefficient (ICC) of ≥0.7 may be considered satisfactory.
- Translation and cross-cultural adaptation
The Principles of Good Practice for the Translation and Cultural Adaptation Process for Patient-Reported Outcomes should be followed to generate versions in different languages and to account for cross-cultural validation.45
For each language, linguistic and cross-cultural validation should be conducted. This process aims to refine the translation, considering the nuances of the original version. Several changes can be implemented throughout the validation process, without modifying the content. The aim is to improve the first idiomatic rendering.
Linguistic and cross-cultural validation steps are summarized in Table 1.
Table 1 Linguistic and Cross-Cultural Validation45
Over the last decade, PROs have become more and more important to assess the impact of diseases on the patient’s Quality of Life, the mental health status and self-esteem and on daily and professional life, especially in visible, chronic or hereditary diseases.47–50 This is because chronic or hereditary skin diseases, impact on appearance, add a psychological burden to the physical burden, thus occupying an important part in the patient’s daily life.51–55
Although, until today, no standardized method for designing a patient-reported outcome (PRO) questionnaire for patients with skin diseases has been proposed, several studies have assessed in the past the burden of specific skin diseases in patients.14–29,56
The methodology proposed herewith allows to design PRO questionnaires using a validated and easy-to-follow method. Building such questionnaires requires the involvement of different specialists, especially dermatologists, statisticians, institutions and professionals specialized in the testing of questionnaires and the recruitment of potentially large panels of participants, including patients. Moreover, several conceptual and internal, as well as external, validation steps are mandatory for making a questionnaire become a powerful and recognized tool for assessing PROs.
Furthermore, burden scores should be evaluated for responsiveness and sensitivity to change their minimal clinically important difference (MCID) which can be defined as the smallest difference in score which patients perceive beneficial.57 Ideally, the score should be used in clinical trials or clinical setting and should correlate with improvement after treatment. Although this step is not always performed, it is essential. Another important step is defining the cut-off for severity allowing translating any patient-reported outcome scoring into severity. This can be done using anchorage methods with a patient global severity score and might help for patient selection when conducting a clinical trial.
One major limitation may be the fact that we did not follow COSMIN guidelines and did not use the latest version of the COSMIN checklist to validate our approach. Indeed, the presently proposed method was developed based on PRO questionnaires in dermatology developed and published prior to COSMIN recommendations and the COSMIN checklist. Thus, we were unable to respect guidelines proposed the time of its creation. However, we checked our method using this checklist allowing to validate the proposed methodology.
To our knowledge, this article provides the first validated methodology for designing a patient burden questionnaire in dermatology, based on published questionnaires used to assess the burden of patients suffering from different skin diseases and the respective burden of their care givers.
The authors acknowledge the writing assistance of Karl Patrick Göritz, SMWS, Scientific and Medical Writing Services, France.
There is no funding to report.
The authors have no conflicts of interest to disclose.
1. Chren MM, Weinstock MA. Conceptual issues in measuring the burden of skin diseases. J Investig Dermatol Symp Proc. 2004;9:97–100. doi:10.1111/j.1087-0024.2004.09119.x
2. WHO. About the global burden of disease (GBD) project; 2010. Available from: https://www.who.int/healthinfo/global_burden_disease/about/en/.
3. Felce D, Perry J. Quality of life: its definition and measurement. Res Dev Disabil. 1995;16(1):51–74. doi:10.1016/0891-4222(94)00028-8
5. Iyendo TO, Uwajeh PC, Ikenna ES. The therapeutic impacts of environmental design interventions on wellness in clinical settings: a narrative review. Complement Ther Clin Pract. 2016;24:174–188. doi:10.1016/j.ctcp.2016.06.008
6. Castro TM, Marinho DR, Cavalcante CC. The impact of environmental factors on quality of life and symptoms of children with allergic rhinitis. Braz J Otorhinolaryngol. 2013;79(5):569–574. doi:10.5935/1808-8694.20130102
7. Kim YM, Kim J, Lee JY, et al. Impact of solar ultraviolet radiation on atopic dermatitis symptoms in young children: a longitudinal study. Pediatr Allergy Immunol. 2017;28(6):551–556. doi:10.1111/pai.12746
8. Lu Y, Lin S, Lawrence WR, et al. Evidence from SINPHONIE project: impact of home environmental exposures on respiratory health among school-age children in Romania. Sci Total Environ. 2018;621:75–84. doi:10.1016/j.scitotenv.2017.11.157
9. Tsabouri S, Bleta AG, Nastos PT, Priftis KN. Ambient environmental risk factors for childhood wheezing illness. Front Biosci (Elite Ed). 2015;7:447–468.
10. Goad N, Gawkrodger DJ. Ambient humidity and the skin: the impact of air humidity in healthy and diseased states. J Eur Acad Dermatol Venereol. 2016;30(8):1285–1294. doi:10.1111/jdv.13707
11. Dreno B, Bettoli V, Araviiskaia E, Sanchez Viera M, Bouloc A. The influence of exposome on acne. J Eur Acad Dermatol Venereol. 2018;32(5):812–819. doi:10.1111/jdv.14820
12. Tsabouri S, Gkoutsias A, Lolis CJ, Makis A, Chaliasos N, Bartzokas A. Impact of meteorological factors on the emergence of bronchiolitis in North-western Greece. Allergol Immunopathol (Madr). 2018;46(1):24–30. doi:10.1016/j.aller.2017.01.009
13. Meyer N, Paul C, Feneron D, et al. Psoriasis: an epidemiological evaluation of disease burden in 590 patients. J Eur Acad Dermatol Venereol. 2010;24(9):1075–1082. doi:10.1111/j.1468-3083.2010.03600.x
14. Boccara O, Meni C, Leaute-Labreze C, et al. Haemangioma family burden: creation of a specific questionnaire. Acta Derm Venereol. 2015;95(1):78–82. doi:10.2340/00015555-1847
15. Dufresne H, Hadj-Rabia S, Meni C, Sibaud V, Bodemer C, Taieb C. Family burden in inherited ichthyosis: creation of a specific questionnaire. Orphanet J Rare Dis. 2013;8:28. doi:10.1186/1750-1172-8-28
16. Taieb A, Boralevi F, Seneschal J, et al. Atopic dermatitis burden scale for adults: development and validation of a new assessment tool. Acta Derm Venereol. 2015;95(6):700–705. doi:10.2340/00015555-1945
17. Salzes C, Abadie S, Seneschal J, et al. The vitiligo impact patient scale (VIPs): development and validation of a vitiligo burden assessment tool. J Invest Dermatol. 2016;136(1):52–58. doi:10.1038/JID.2015.398
18. Dufresne H, Hadj-Rabia S, Taieb C, Bodemer C. Development and validation of an epidermolysis bullosa family/parental burden score. Br J Dermatol. 2015;173(6):1405–1410. doi:10.1111/bjd.14072
19. Morice-Picard F, Taieb C, Marti A, et al. Burden of albinism: development and validation of a burden assessment tool. Orphanet J Rare Dis. 2018;13(1):162. doi:10.1186/s13023-018-0894-3
20. Meni C, Bodemer C, Toulon A, et al. Atopic dermatitis burden scale: creation of a specific burden questionnaire for families. J Eur Acad Dermatol Venereol. 2013;27(11):1426–1432. doi:10.1111/jdv.12180
21. Misery L, Jourdan E, Abadie S, et al. Development and validation of a new tool to assess the Burden of Sensitive Skin (BoSS). J Eur Acad Dermatol Venereol. 2018;32(12):2217–2223. doi:10.1111/jdv.15186
22. Maccari F, Ezzedine K, Fougerousse AC, et al. Building a scale for measuring burden of hand eczema: boHEM. J Am Acad Dermatol. 2019;80(6):1784–1785. doi:10.1016/j.jaad.2018.12.036
23. Serra E, Spaeth M, Carbonell J, et al. Development of the fibromyalgia burden assessment: measuring the multifaceted burden of fibromyalgia. Clin Exp Rheumatol. 2010;28(6 Suppl 63):S87–93.
24. Misery L, Seneschal J, Reguiai Z, et al. Patient burden is associated with alterations in quality of life in adult patients with atopic dermatitis: results from the ECLA study. Acta Derm Venereol. 2018;98(7):713–714. doi:10.2340/00015555-2940
25. Audureau E, Guellich A, Guery E, et al. Development and validation of a new tool to assess burden of dietary sodium restriction in patients with chronic heart failure: the BIRD questionnaire. Nutrients. 2018;10(10):1453. doi:10.3390/nu10101453
26. Guex JJ, Rahhali N, Taieb C. The patient’s burden of chronic venous disorders: construction of a questionnaire. Phlebology. 2010;25(6):280–285. doi:10.1258/phleb.2010.010039
27. Hickman G, Bodemer C, Bourrat E, Bennani M, Taieb C. Palmoplantar keratoderma: creating a disease burden questionnaire. J Eur Acad Dermatol Venereol. 2019;33(8):e291–e293. doi:10.1111/jdv.15563
28. Misery L, Seneschal J, Reguiai Z, et al. The impact of atopic dermatitis on sexual health. J Eur Acad Dermatol Venereol. 2019;33(2):428–432. doi:10.1111/jdv.15223
29. Ezzedine K, Ahmed M, Tovar-Garza A, et al. Cross cultural validation of a short-form of the vitiligo impact patient scale (VIPs). J Am Acad Dermatol. 2019;81(5):1107–1114. doi:10.1016/j.jaad.2019.02.008
30. Ezzedine K, Fougerousse AC, Aubert R, et al. Individual Burden of Psoriasis (I-BOP): building and validation of a new scoring tool for patients with psoriasis. Clin Cosmet Investig Dermatol. 2020;13:325–332. doi:10.2147/CCID.S249776
31. Seidenberg M, Haltiner A, Taylor MA, Hermann BB, Wyler A. Development and validation of a multiple ability self-report questionnaire. J Clin Exp Neuropsychol. 1994;16(1):93–104. doi:10.1080/01688639408402620
32. Leidy NK, Revicki DA, Geneste B. Recommendations for evaluating the validity of quality of life claims for labeling and promotion. Value Health. 1999;2(2):113–127. doi:10.1046/j.1524-4733.1999.02210.x
33. Mokkink LB, de Vet HCW, Prinsen CAC, et al. COSMIN risk of bias checklist for systematic reviews of patient-reported outcome measures. Qual Life Res. 2018;27(5):1171–1179. doi:10.1007/s11136-017-1765-4
34. Likert R. A technique for the measurement of attitudes. Arch Psychol. 1932;140:55.
35. Ezzedine K, Grimes PE, Meurant JM, et al. Living with vitiligo: results from a national survey indicate differences between skin phototypes. Br J Dermatol. 2015;173(2):607–609. doi:10.1111/bjd.13839
36. Sibaud V, Dalenc F, Chevreau C, et al. HFS-14, a specific quality of life scale developed for patients suffering from hand-foot syndrome. Oncologist. 2011;16(10):1469–1478. doi:10.1634/theoncologist.2011-0033
37. Holt RW, editor. Research Methods in Social Relations. New York: Holt, Rinehart & Winston; 1976.
38. Guilford JP. Psychometric Methods.
39. Bartholomew DJ. Spearman and the origin and development of factor analysis. Br J Math Stat Psychol. 1995;48(2):211–220. doi:10.1111/j.2044-8317.1995.tb01060.x
40. Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16(3):297–334. doi:10.1007/BF02310555
41. Rocque R, Leanza Y, Harper DM. A systematic review of patients’ experiences in communicating with primary care physicians: intercultural encounters and a balance between vulnerability and integrity. PLoS One. 2015;10(10):e0139577. doi:10.1371/journal.pone.0139577
42. Collins D. Pretesting survey instruments: an overview of cognitive methods. Qual Life Res. 2003;12(3):229–238. doi:10.1023/A:1023254226592
43. McColl E, Meadows K, Barofsky I. Cognitive aspects of survey methodology and quality of life assessment. Qual Life Res. 2003;12(3):217–218.
44. Ploughman M, Austin M, Stefanelli M, Godwin M. Applying cognitive debriefing to pre-test patient-reported outcomes in older people with multiple sclerosis. Qual Life Res. 2010;19(4):483–487. doi:10.1007/s11136-010-9602-z
45. Wild D, Grove A, Martin M, et al. Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures: report of the ISPOR task force for translation and cultural adaptation. Value Health. 2005;8(2):94–104. doi:10.1111/j.1524-4733.2005.04054.x
46. Brazier JE, Roberts J. The estimation of a preference-based measure of health from the SF-12. Med Care. 2004;42(9):851–859. doi:10.1097/01.mlr.0000135827.18610.0d
47. Lifschitz C. The impact of atopic dermatitis on quality of life. Ann Nutr Metab. 2015;66(Suppl 1):34–40. doi:10.1159/000370226
48. Greene A, Meskell P. The impact of lower limb chronic oedema on patients’ quality of life. Int Wound J. 2017;14(3):561–568. doi:10.1111/iwj.12648
49. Yokota RT, Van der Heyden J, Nusselder WJ, et al. Impact of chronic conditions and multimorbidity on the disability burden in the older population in Belgium. J Gerontol a Biol Sci Med Sci. 2016;71(7):903–909. doi:10.1093/gerona/glv234
50. Udompap P, Kim D, Kim WR. Current and future burden of chronic nonmalignant liver disease. Clin Gastroenterol Hepatol. 2015;13(12):2031–2041. doi:10.1016/j.cgh.2015.08.015
51. Armstrong A, Jarvis S, Boehncke WH, et al. Patient perceptions of clear/almost clear skin in moderate-to-severe plaque psoriasis: results of the clear about psoriasis worldwide survey. J Eur Acad Dermatol Venereol. 2018;32(12):2200–2207. doi:10.1111/jdv.15065
52. Pereira MP, Stander S. Assessment of severity and burden of pruritus. Allergol Int. 2017;66(1):3–7. doi:10.1016/j.alit.2016.08.009
53. Pereira MG, Ponte M, Ferreira G, Machado JC. Quality of life in patients with skin tumors: the mediator role of body image and social support. Psychooncology. 2017;26(6):815–821. doi:10.1002/pon.4236
54. Sanclemente G, Burgos C, Nova J, et al. The impact of skin diseases on quality of life: a multicenter study. Actas Dermosifiliogr. 2017;108(3):244–252. doi:10.1016/j.ad.2016.11.008
55. Apfelbacher CJ, Ofenloch RF. The impact of skin conditions on generic health-related quality of life. Br J Dermatol. 2017;176(5):1109–1110. doi:10.1111/bjd.15453
56. Porkert S, Lehner-Baumgartner E, Valencak J, Knobler R, Riedl E, Jonak C. Patients’ illness perception as a tool to improve individual disease management in primary cutaneous lymphomas. Acta Derm Venereol. 2018;98(2):240–245. doi:10.2340/00015555-2819
57. Jaeschke R, Singer J, Guyatt GH. Measurement of health status. Ascertaining the minimal clinically important difference. Control Clin Trials. 1989;10(4):407–415. doi:10.1016/0197-2456(89)90005-6
© 2020 The Author(s). This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.