Back to Journals » Neuropsychiatric Disease and Treatment » Volume 13

Illness denial questionnaire for patients and caregivers

Authors Rossi Ferrario S , Giorgi I, Baiardi P, Giuntoli L, Balestroni G, Cerutti P, Manera M, Gabanelli P, Solara V, Fornara R, Luisetti M, Omarini P, Omarini G, Vidotto G 

Received 24 November 2016

Accepted for publication 23 January 2017

Published 23 March 2017 Volume 2017:13 Pages 909—916

DOI https://doi.org/10.2147/NDT.S128622

Checked for plagiarism Yes

Review by Single anonymous peer review

Peer reviewer comments 2

Editor who approved publication: Dr Roger Pinder



Silvia Rossi Ferrario,1 Ines Giorgi,2 Paola Baiardi,3 Laura Giuntoli,4 Gianluigi Balestroni,1 Paola Cerutti,1 Marina Manera,2 Paola Gabanelli,2 Valentina Solara,5 Roberta Fornara,6 Michela Luisetti,1 Pierangela Omarini,1 Giovanna Omarini,1 Giulio Vidotto4

1Psychology Unit, Istituti Clinici Scientifici Maugeri SpA SB, Veruno, NO, Italy; 2Psychology Unit, 3Scientific Direction, Istituti Clinici Scientifici Maugeri SpA SB, Pavia, Italy; 4Department of General Psychology, University of Padova, Padova, Italy; 5Department of Neurology, ALS Centre, “Maggiore della Carita`” University Hospital, Novara, Italy; 6Psychology Unit, SS Trinità Hospital, Borgomanero, NO, Italy

Purpose: Interest in assessing denial is still present, despite the criticisms concerning its definition and measurement. We tried to develop a questionnaire (Illness Denial Questionnaire, IDQ) assessing patients’ and caregivers’ denial in relation to their illness/disturbance.
Patients and methods:
After a preliminary study, a final version of 24 dichotomous items (true/false) was selected. We hypothesized a theoretical model with three dimensions: denial of negative emotions, resistance to change, and conscious avoidance, the first two composing the actual Denial and the last representing an independent component of the illness denial behavior. The IDQ was administered to 400 subjects (219 patients and 181 caregivers) together with the Anxiety–Depression Questionnaire – Reduced form (AD-R), in order to assess concurrent validity. Confirmatory factor analysis (CFA), internal consistency indices (Cronbach’s α and McDonald’s ω), and test–retest analysis were performed.
Results: CFA and internal consistency indices (Cronbach’s α: 0.87–0.96) indicated a clear and meaningful three-factor structure of IDQ, for both patients and caregivers. Further analyses showed good concurrent validity, with Denial and its subscale negatively associated with anxiety and depression and avoidance positively associated with anxiety and depression. The IDQ also showed a good stability (r from 0.71 to 0.87).
Conclusion: The IDQ demonstrated good psychometric properties. Denial of negative emotions and resistance to change seem to contribute to a real expression of denial, and conscious avoidance seems to constitute a further step in the process of cognitive–affective elaboration of the illness.

Keywords: assessment, denial, avoidance, chronic diseases

Introduction

The onset of an acute or a chronic illness or disturbance can cause psychological distress and trigger the emergence of defense mechanisms to contain it. These mechanisms, introduced by Sigmund Freud and alternatively celebrated, rejected, and discussed over the years,1 are now universally recognized as playing an important role in psychological functioning.2 Contrary to coping strategies that are intentionally used by people to manage their problems, defense mechanisms come into play unintentionally to reduce the anxiety caused by a specific threatening problem.2 One of the most frequent is denial, which preserves a person from something that he/she is not yet ready to face. Like the other defense mechanisms, denial may be positive or negative; on the positive side, it may be an effective method for facing the initial phases of an illness, invasive examinations, or burdensome therapies, but in its more severe and persistent forms, it may lead to maladaptive behaviors and serious psychological disturbances. With regard to illnesses, research interest began to focus on denial in the psychiatric and neurological settings during the mid-1950s/1960s.3,4 Denial was considered substantially as a single dimension, and some questionnaires were produced combining scales or items derived from the Minnesota Multiphasic Personality Inventory (MMPI).5 The limits of these measures were evident, especially concerning the validity.68 In the 1970s, interest in denial also grew in the cardiac and oncological settings.9,10 In the early 1980s, the biopsychosocial approach transformed the physician–patient relationship. It promoted the patient’s active involvement and encouraged the physician to reinforce patient behaviors and emotional reactions that could favor compliance or prevent maladaptation. Typically, in those years, semistructured interviews were proposed to investigate illness denial. Despite difficulties in their use, measurement bias, and the limited samples used (mostly patients affected by coronary disease), these interviews introduced the characteristics of multidimensionality and fluctuation of the denial mechanism.1113 These characteristics were later underlined by Goldbeck14 in 1997, who affirmed the importance of resuming the study of denial in relation to physical illnesses. Moreover, Goldbeck observed that, apart from the patient, denial could also be an issue for family members and health professionals. In the following years, the concept of denial in chronic illness was criticized by some authors who pointed out that some reactions often interpreted as denial (eg, optimism) may well be part of a normal psychological adaptation15 or of the continuous shifting process of perspective that characterizes it.16 They warned health professionals against the use of denial as a foregone label to explain patients’ behavior or attitude in relation to illness, especially their noncompliance to therapies, thus neglecting the need for active listening. Thus, it appears that, along with the widespread use of the concept of denial, the lack of adequate objective tools to measure it caused its misuse. Recently, other authors have discussed denial in the psychiatric, oncologic, and pulmonary settings,1719 evidencing once again that interest in assessing denial is still present, despite the criticisms concerning its definition and measurement. In fact, health professionals need to understand if a patient (or caregiver) is denying one or more aspects of the illness, for example, the diagnosis, its impact on their lifestyle or quality of life, the correlated anxiety and/or depression, or even all these aspects together.13 The aim of our study was to develop a questionnaire, the Illness Denial Questionnaire (IDQ), to assess patients and caregivers denial in relation to their illness, independently from the specific kind of illness or disturbance involved. We decided to define denial as composed of two specific aspects, namely denial of negative emotions and resistance to change, hypothesizing that a third aspect, conscious avoidance, could be resulted as an independent component.

The study was approved by the local ethical committees of the recruitment sites, where the authors work (974 CE, March 10, 2014, Comitato Etico ex Fondazione S Maugeri, Pavia; 566 CE, June 13, 2014, Comitato Etico Interaziendale AOU ‘Maggiore della Carità’ [NO]; 781 CE, September 5, 2014, Comitato Etico Interaziendale AOU ‘Maggiore della Carità’ [NO]), and all participants were recruited on a voluntary basis and signed the informed consent.

Materials and methods

Item generation

We used two reference definitions as starting points: 1) Freud’s explanation of denial, as a way of achieving awareness of what is repressed, without necessarily accepting it; and 2) the concept of “phases” that may represent the interchangeability/fluctuation of one’s perspectives during the course of the illness. We decided to focus our attention on some components of denial that frequently emerge after a diagnosis or during the course of the illness: the denial of negative emotion, the difficulty of changing one’s lifestyle, and the need to take distance from the real condition or prognosis. After preliminary discussion, we selected a pool of items (n=53), which we proposed to patients and caregivers recruited where the authors usually work and organized in focus groups of 5–6 individuals each. A total of 29 patients affected by cardiac, pulmonary, neurological, or renal disease and 29 caregivers, including 10 parents of children affected by attention deficit disorder (learning disabilities/attention-deficit hyperactivity disorder [LD/ADHD]), participated in this initial phase. They discussed, rephrased, or eliminated the items, which resulted in the approval of 43 items. Then, a preliminary dichotomous form (true/false) questionnaire of 43 items was set up.

Item reduction

This preliminary 43-item questionnaire was administered to 112 subjects: 62 patients and 50 caregivers, also recruited where the authors usually work. Exclusion criteria were any conditions or deficits that could make the administration of the questionnaire impossible or unreliable (eg, not Italian mother tongue, sensorial deficits, and severe psychiatric conditions). The participants recruited were 57 males and 55 females, with a mean age of 53.7±13.4 years (range 20–80) and a mean school education of 10.8±3.8 years (range 3–18). Patients were prevalently affected by neurological diseases (n=54 subjects), followed by cardiac (n=24), pulmonary (n=14), oncologic (n=6), and renal (n=5) diseases. Nine participants in the caregiver group were parents of children affected by LD/ADHD. The items were coded in order to obtain a correspondence between high score and high denial (1= false; 2= true) reversing the score of those at the opposite direction.

Through item analysis, the problematic items were identified and discussed by the authors. As a final step, the items that were judged the worst descriptors for the hypothesized dimension (denial of negative emotions, resistance to change, and conscious avoidance) were dropped. The final version of the IDQ consisted of 24 items (Supplementary materials: IDQ Italian and English version for patients and caregivers).

Validation of IDQ

Participants

A total of 400 subjects, 219 patients and 181 caregivers (not necessary in relationship with those patients), were recruited where the authors usually work and signed informed consent. In the patient group (47.9% females), average age was 57.3 (SD =12.8) for females and 58.3 (SD =11.6) for males. In the caregiver group, subjects were prevalently females (71.3%), with a mean age of 51.0 years (SD =13.7) for females and 54.3 years (SD =15.1) for males. More details about patients and caregivers are reported in Table 1. In particular, concerning patients’ illnesses, cardiac diseases included postsurgical valve replacement, coronary artery bypass, left ventricular assistance, transplantation, and angioplasty. Oncological diseases prevalently included breast and abdominal cancers. Neurological diseases included Parkinson’s disease, stroke, amyotrophic lateral sclerosis (ALS), cerebral anoxia, and vegetative or minimally conscious state. Renal diseases included renal failure, mostly in dialysis, and renal transplantation. Lung diseases included chronic obstructive pulmonary disease or respiratory failure.

Table 1 Characteristics of the sample
Abbreviation: LD/ADHD, learning disabilities/attention-deficit hyperactivity disorder.

Measures

Subjects were administered the Anxiety–Depression Questionnaire – Reduced form (AD-R)20 and the final version of the IDQ. The AD-R measures state anxiety and symptoms of depression in the present condition and is composed of the reduced versions of the well-known State-Trait Anxiety Inventory21 and Depression Questionnaire,22 respectively, of 10 and 15 items. In the AD-R, for anxiety, there is a 4-level rating scale (not at all, somewhat, moderately so, and very much so), with a score range between 10 and 40; for symptoms of depression, responses are “Yes” or “No”, with a score range from 0 to 15. Different cutoffs exist for males and females: scores ≥22 and ≥6, respectively, indicate state anxiety and symptoms of depression for males while the corresponding scores for females are ≥25 and ≥8.

Data analysis

All the analyses were conducted through the software R.23 Confirmatory factor analysis (CFA) was performed through the R-package Lavaan.24 To determine the fit of the CFA models, we considered the comparative fit index (CFI), the Tucker–Lewis index (TLI), and the root mean square of approximation (RMSEA). The CFI and TLI compare the current model with an unstructured baseline model. CFI and TLI range from 0 to 1, with 0 indicating poor fit and 1 indicating a perfect fit. Generally, CFI and TLI values >0.90 indicate acceptable fit, although values >0.95 are desirable.25 The RMSEA is a measure of the error of approximation of the specified covariance and mean structures to the covariance and mean structures in the population. Values <0.05 indicate close fit, values between 0.05 and 0.08 indicate acceptable fit, values between 0.08 and 0.10 indicate mediocre fit, and values >0.10 indicate poor fit.26 Since the data were in a dichotomous response mode, we used the Diagonally Weighted Least Squares estimation technique27 that allows us to obtain unbiased estimates in the case of ordinal data and/or data which seriously violate multivariate normality assumptions.

Internal consistency was evaluated through Cronbach’s α28 and McDonald’s ω.29,30 The α coefficients were calculated from the tetrachoric correlation matrices, in order to attenuate possible underestimation due to the use of a dichotomous rating scale.31 Since one-dimensionality is a fundamental assumption for the use of Cronbach’s α and we had in our case a multidimensional construct, we also calculated the McDonald’s ω coefficient, which is based on the estimation of the factor model parameters and can also be used for multidimensional tests. Concurrent validity was assessed calculating the correlations of the total IDQ score with the scores on the AD-R. The test–retest analysis was performed on a sample of 68 subjects (47 patients and 21 caregivers).

Results

Descriptive analysis

Descriptive statistics for the IDQ scales are shown in Table 2. The normal quantile–quantile (QQ) plots of test scores exhibited a violation of normal distribution assumption. Furthermore, skewness and kurtosis were above the recommended range.32 According to the data properties, we used adequate estimation methods.

Table 2 IDQ descriptive statistics
Note: “Denial” includes the subscales “resistance to change” and “denial of negative emotions”.
Abbreviations: IDQ, Illness Denial Questionnaire; min, minimum; max, maximum; SD, standard deviation.

Group differences

The analysis of variance (ANOVA) revealed a significant difference between males and females only on the Avoidance scores for the caregiver sample F(1,179) =3.99, P=0.047. The ANOVA revealed a significant difference between patients and caregivers for both the Denial scores F(1,398) =8.53, P=0.004 and Avoidance scores F(1,398) =19.01, P<0.001. Considering the different pathologies in the patient sample (Table 3), the ANOVA revealed no differences on the Avoidance scale, but showed a significant difference on the Denial scale (F[4,214] =5.66, P<0.001). Post hoc comparisons using Bonferroni tests revealed significant differences between the renal group and both the oncological (P=0.014) and respiratory (P=0.001) groups on the Denial scale. There was also a significant difference between cardiac and respiratory patients (P=0.008).

Table 3 Denial and conscious avoidance across different pathologies
Abbreviations: LD/ADHD, learning disabilities/attention-deficit hyperactivity disorder; SD, standard deviation.

In the caregiver sample, the ANOVA revealed a significant difference in the pathologies on both the Denial scale (F[5,175] =3.79, P<0.01) and Avoidance scale (F[5,175] =2.75, P<0.05). Post hoc comparisons using Bonferroni tests revealed significant differences on the Denial scale between the LD/ADHD group and both the neurological (P=0.008) and oncological (P=0.008) groups. Post hoc comparisons using Bonferroni tests also revealed significant differences between the LD/ADHD group and the oncological group (P=0.005) on the Avoidance scale.

Internal consistency

The ordinal Cronbach’s α was calculated separately for the Denial and the Conscious Avoidance scales (Table 2). McDonald’s ω coefficient for the complete instrument was good for both patients (ω=0.95) and caregivers (ω=0.96).

CFA

In the patient sample, the CFI and TLI were both above their desired levels (CFI =0.948 and TLI =0.943). In addition, the RMSEA fell below the desired upper limit (RMSEA = 0.065). CFA of the caregiver sample showed good CFI, TLI and RMSEA values (CFI =0.939, TLI =0.932, RMSEA =0.065). The results are reported in Table 4. The correlations between the three latent dimensions are reported in Table 5. Correlations between the Denial of negative emotions and Resistance to change were high for both the patient group (r=0.92) and the caregiver group (r=0.81). The correlations between the Denial of negative emotions and Conscious avoidance were low for the patient group (r=−0.17) and medium-high for the caregiver group (r=−0.41). The Resistance to change and Conscious avoidance dimensions demonstrated to be independent on both samples, with correlations of 0.01 in the patient sample and −0.01 in the caregiver sample. These results confirm the three-factor structure of the IDQ. For both the samples, all items loaded >0.45 onto the single factor (except item 15 in the caregiver group =0.372; Table 6).

Table 4 Confirmatory factor analysis
Abbreviations: CFI, comparative fit index; TLI, Tucker–Lewis index; RMSEA, root mean square error of approximation; Conf Int RMSEA, confidence interval of the RMSEA.

Table 5 Correlations between the three latent dimensions

Table 6 Factor loadings for the patients and caregiver samples

Concurrent validity

The concurrent validity for the IDQ was analyzed by the comparison of the IDQ subscales with the AD-R anxiety and depression subscales. The correlations were calculated separately for the patients (Table 7) and the caregivers (Table 8) groups. The Denial total scores correlated moderately and negatively both with anxiety (−0.52 and −0.45) and depression (−0.54 and −0.37). The Avoidance subscale showed positive and low correlations both with anxiety (0.19 and 0.21) and depression (0.09 and 0.26).

Table 7 Correlations between the IDQ subscales and the AD-R subscales in the patient sample
Abbreviations: AD-R, Anxiety–Depression Questionnaire – Reduced form; IDQ, Illness Denial Questionnaire.

Table 8 Correlations between the IDQ subscales and the AD-R subscales in the caregiver sample
Abbreviations: AD-R, Anxiety–Depression Questionnaire – Reduced form; IDQ, Illness Denial Questionnaire.

Stability

The IDQ showed a good temporal stability. The test–retest analysis was carried out on a sample of 68 subjects. In the patient group, the correlation between the first and the second administration (3.89±1.20 days) was high (r=0.87) for the Denial scores and medium-high (r=0.71) for the Avoidance scores. Pearson’s coefficients for the caregiver group, between the first and second administration (3.81±1.21 days) was high (r=0.85) for the Denial scores and medium-high (r=0.75) for the Avoidance scores.

Discussion

The aim of our study was to create a questionnaire for the investigation of some components of denial that frequently emerge after diagnosis or during the course of an illness, such as the denial of negative emotion, resistance to the need for lifestyle change, and distancing oneself from the real condition or prognosis. The 24-item IDQ demonstrated to have good psychometric properties, and its items are distributed across three factors: denial of negative emotions (nine items), resistance to change (seven items), and conscious avoidance (eight items). The first two dimensions seem to contribute to a real expression of Denial (D), while Conscious Avoidance (CA) seems to constitute a further step in the process of cognitive-affective elaboration of the illness.33 In our opinion, this result makes sense if we think of denial as a process where one’s perspective may fluctuate during the course of the illness from an unconscious to preconscious and conscious level, or, in other words, from an experiential to rational system.34 Moreover, if denial is a way of achieving awareness of what is repressed, without necessarily accepting it, according to Freud’s explanation, then being aware of one’s own distancing from the illness may represent the point in the denial process at which acceptance of the illness’s existence begins, in an ongoing process of construing reality, according to Lazarus.35 This kind of awareness may precede the illness’s acceptance in all of its implications or even become a more steady coping reaction. It is not rare, for example, for patients to adhere to the medical treatment or intervention without enquiring about the real consequences or without changing their usual lifestyle. Avoidance, when conscious, consequently should be considered as something different from denial, rather than as an interchangeable term. In fact, this may create confusion about the kind of reactions that are actually being used by the person.35 Attention to this is even more important considering that the scales that measure denial are often composed of items more representative of conscious avoidance, which, in our opinion, refers to a more conscious/rational level. As Livneh36 points out, “avoidance it is typically viewed as a conscious process or effort where the individual a) deliberately avoids stressful material and threatening information and b) is only partially successful in averting emotional distress (relieving anxiety and fear)”. This last statement is also confirmed in our sample, with CA showing positive and low correlations with State Anxiety and Symptoms of Depression, while D, on the other hand, correlates moderately and negatively with them. The IDQ also seems to discriminate very well between patients and caregivers, with patients showing higher scores both in D and CA. Considering the different pathologies involved, it is interesting to note that patients affected by respiratory diseases scored lowest in D, probably because of impacting symptoms and visible supporting devices. On the contrary, renal patients (11/35 on dialysis) showed the highest score, suggesting that these subjects need to protect themselves from a disease that may heavily alter their daily life because of the need for frequent dialysis and/or the uncertainty of a transplant. In this sense, D has a positive function, in agreement with other authors’ observations.36 Caregivers show the same direction as patients in the scores, but here we note that parents of children affected by LD/ADHD presented the highest score in D and the lowest in CA. This may be considered a further demonstration of the discriminative power of IDQ: being parents of a child affected by emotional and behavioral disturbances is a great challenge and, on the other hand, the child’s young age is a source of hope that everything will recover.

Limitations

It is evident that the interpretations of the differences we found between males and females and between the pathologies should be treated with caution, and further studies are needed to confirm/disconfirm our findings, defining better the illness conditions and their duration: the lack of these detailed definitions is the principal limit of our study. For example, we did not find any influence of the duration of our patients’ conditions on D and CA, but in our sample those with less duration were cardiac subjects. ALS or cancer patients/caregivers interviewed after 1 month from diagnosis would probably show a different profile.

Conclusion

Denial has been, since its first introduction, an intriguing concept: apparently simple to define and measure, it has instead produced misunderstandings and doubts because of its complexity.37 In our study, we have proposed a new questionnaire for the assessment of denial and have discussed some facets of denial often encountered by health professionals in relation to patients and caregivers. Further studies are needed to deepen our knowledge about this important defense mechanism, especially in order to improve our ability of caring.

Acknowledgment

The authors thank Rosemary Allpress for her English review.

Author contributions

Silvia Rossi Ferrario and Giulio Vidotto managed the literature searches and designed the study. Paola Baiardi, Giulio Vidotto, and Laura Giuntoli wrote the protocol and managed the statistical analysis. Ines Giorgi, Gianluigi Balestroni, Paola Cerutti, Paola Gabanelli, Roberta Fornara, Michela Luisetti, Marina Manera, Pierangela Omarini, Giovanna Omarini, and Valentina Solara recruited the subjects and collected the data. Silvia Rossi Ferrario, Laura Giuntoli and Ines Giorgi wrote the first drafts of manuscript. All authors contributed toward data analysis, drafting and critically revising the paper and agree to be accountable for all aspects of the work.

Disclosure

The authors report no conflicts of interest in this work.


References

1.

O’Neil MK, Akhtar S. On Freud’s ‘Negation’. London: Karnac Books; 2011.

2.

Cramer P. Defense mechanisms in psychology today. Further processes for adaptation. Am Psychol. 2000;55(6):637–646.

3.

Rosen VH. The role of denial in acute postoperative affective reactions following removal of body parts. Psychosom Med. 1950;12(6):356–361.

4.

Ullman M, Ashenhurst EM, Hurwitz LJ, Gruen A. Motivational and structural factors in the denial of hemiplegia. Arch Neurol. 1969;3:306–318.

5.

Cramer P. The Development of Defense Mechanism – Theory, Research, Assessment. New York: Springer-Verlag; 1991.

6.

Levitt EE. A note on “MMPI scale development methodology”. J Pers Assess. 1978;42(5):503–504.

7.

McGrath RE, O’Malley WB. The assessment of denial and physical complaints: the validity of the Hy scale and associated MMPI signs. J Clin Psychol. 1986;42(5):754–760.

8.

Levitt EE, Gotts EE. The Clinical Application of MMPI Special Scales. 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates; 1995.

9.

Hackett TP, Cassem NH. Development of a quantitative rating scale to assess denial. J Psychosom Res. 1974;18(2):93–100.

10.

Kubler-Ross E. Questions and Answers on Death and Dying. New York: Macmillan; 1974.

11.

Havik OE, Maeland JG. Dimensions of verbal denial in myocardial infarction. Correlates to 3 denial scales. Scand J Psychol. 1986;27(4):326–339.

12.

Levine J, Warrenburg S, Kerns R, et al. The role of denial in recovery from coronary heart disease. Psychosom Med. 1987;49(2):109–117.

13.

Jacobsen BS, Lowery BJ. Further analysis of the psychometric properties of the Levine Denial of Illness Scale. Psychosom Med. 1992;54(3):372–381.

14.

Goldbeck R. Denial in physical illness. J Psychosom Res. 1997;43(6):575–593.

15.

Telford K, Kralik D, Koch T. Acceptance and denial: implications for people adapting to chronic illness: literature review. J Adv Nurs. 2006;55(4):457–464.

16.

Paterson BL. The shifting perspectives model of chronic illness. J Nurs Scholarsh. 2001;33(1):21–26.

17.

Vos MS, Putter H, Leurs A, Rooijmans HG, de Haes HC, van Houwelingen HC. The denial of cancer interview: development and first assessment of psychometric properties in lung cancer patients. Patient Educ Couns. 2007;67(1–2):224–234.

18.

Kruck CL, Flashman LA, Roth RM, Koven NS, McAllister TW, Saykin AJ. Lack of relationship between psychological denial and unawareness of illness in schizophrenia-spectrum disorders. Psychiatry Res. 2009;169(1):33–38.

19.

Boer LM, Daudey L, Peters JB, Molema J, Prins JB, Vercoulen JH. Assessing the stages of the grieving process in chronic obstructive pulmonary disease (COPD): validation of the Acceptance of Disease and Impairments Questionnaire (ADIQ). Int J Behav Med. 2014;21(3):561–570.

20.

Moroni L, Bettinardi O, Vidotto G, et al. Scheda Ansia e Depressione forma ridotta: norme per l’utilizzo in ambito riabilitativo [Anxiety and Depression Short Scale: norms for its use in rehabilitation]. Monaldi Arch Chest Dis. 2006;66(4):255–263.

21.

Spielberger CD, Gorsuch RL, Lushene RE. The State-Trait Anxiety Inventory (STAI) Test Manual for Form X. Palo Alto: Consulting Psychologist Press; 1970. (It. Tr. by Lazzari R, Pancheri P. S.T.A.I. Questionario di autovalutazione dell’ansia di stato e di tratto. Firenze: Organizzazioni Speciali; 1980).

22.

Bertolotti G, Michielin P, Vidotto G, Zotti AM, Sanavio E. Depression Questionnaire (DQ). In: Nezu AM, Ronan GF, Meadows EA, McKlure KS, editors. Practitioner’s Guide to Empirical Based Measures of Depression. Norwell, MA: Kluwer Academic/Plenum Publishers; 2000.

23.

R Development Core Team [homepage on the Internet]. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2015. ISBN 3-900051-07-0. Available from: http://www.R-project.org. Accessed February 17, 2016.

24.

Rosseel Y. Lavaan: an R package for structural equation modeling. J Stat Softw. 2012;48(2):1–36.

25.

Hox JJ, Moerbeek M, van de Schoot R. Multilevel Analysis: Techniques and Applications. New York and Hove: Routledge; 2010.

26.

Browne MW, Cudeck R. Alternative ways of assessing model fit. Sociol Methods Res. 1992;21:230–258.

27.

Jöreskog KG, Sörbom D. LISREL 7 User’s Reference Guide. Chicago: SPSS Publications; 1989.

28.

Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16:297–334.

29.

McDonald RP. Generalizability in factorable domains: “domain validity and generalizability”. Educ Psychol Meas. 1978;38(1):75–79.

30.

McDonald RP. Test Theory: A Unified Treatment. Mahwah, NJ: Erlbaum; 1999.

31.

Gadermann AM, Guhn M, Zumbo D. Estimating ordinal reliability for Likert-type and ordinal item response data: a conceptual, empirical, and practical guide. Pract Assessment Res Evaluation. 2012;17(3):1–13.

32.

Lei M, Lomax RG. The effect of varying degrees of nonnormality in structural equation modeling. Struct Equ Modeling. 2005;12:1–27.

33.

Eder A, Hommel B, De Houwer J, editors. How Distinctive Is Affective Processing? Cognition and Emotion. Hove, England: Psychology Press; 2007.

34.

Epstein S. Integration of the cognitive and the psychodynamic unconscious. Am Psychol. 1994;49(8):709–724.

35.

Lazarus RS. Fifty Years of the Research and Theory of R.S.Lazarus: An Analysis of Historical and Perennial Issues. Routledge Sources in History, Psychology Press, London, 1998.

36.

Livneh H. Denial of chronic illness and disability. Part I. Theoretical, functional, and dynamic perspectives. Rehab Couns Bull. 2009;52(4):225–236.

37.

Livneh H. Denial of chronic illness and disability. Part II. Research findings, measurement considerations, and clinical aspects. Rehabil Couns Bull. 2009;53(1):44–55.

Creative Commons License © 2017 The Author(s). This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.