The Negative Emotions Due to Chronic Illness Screening Test (NECIS): Construct Validity in Patients with Coronary Artery Disease in Mainland China

Introduction

Coronary heart disease (CAD) is one of the most prevalent causes of cardiovascular morbidity and mortality globally.¹ The mortality rates of CAD in China were 113.46～118.74 per 100 thousand people in 2016.² It was predicted that there would have an estimated 64% increase in the number of CAD events among Chinese adults in 2020–2029 compared with 2000–2009.³

As a negative life event, the onset of CAD can generate negative psychological states. Individuals diagnosed with CAD will face the challenge of long-term self-management to control multiple cardiovascular risk factors and prevent the recurrence of adverse cardiac events. Therefore, patients with CAD are susceptible to negative psychological states such as depression, anxiety, anger, hostility, etc.^4–6 Anxiety and depression are the most common negative emotions in patients with CAD.⁷ Nearly 70−80% of patients who suffered an acute heart attack experienced anxiety, and almost 20−25% of whom persisted this unpleasant emotion in the long term.⁸ For those with stable CAD, the prevalence of anxiety ranged from 16% to 42%.⁴

Negative emotions may be harmful to the cardiovascular system through physiological alterations and related to poor outcomes, which will precipitate the progress of CAD and increase the risk of recurrent cardiac events.^9–12 High level of anxiety has negative impacts on the cardiovascular physiology, which can activate the sympathetic nervous system, thus leading to an increase in heart rates and change in vascular tone. If the condition continues, it will damage the cardiovascular functions.^13–16 A large number of studies reported that patients with depression were more likely to have poor medication adherence and self-management behaviours including physical inactivity, unhealthy dietary habits, tobacco and alcohol use.^17–21 A meta-analysis showed that patients suffered from acute myocardial infarction combined with depression would have an increased likelihood of all-cause mortality and recurrence of the acute cardiac events.²² In a prospective study of 1055 men with 32 to 48 years’ follow-up, Chang et al²³ found that young men with high level of anger were susceptible to an increased risk of subsequent premature CAD.

Maladaptive negative emotions are indispensable psychological outcomes in clinical mental health services and psychological interventions for patients with CAD. Therefore, it is necessary to have a suitable tool to measure negative emotions for this vulnerable group. However, to our knowledge, few measurements are available in mainland China to specially assess the negative emotional experiences in the context of CAD. Although several instruments are available to measure negative psychological states (such as Zung’s self-rating depression scale (SDS), Self-rating Anxiety Scale (SAS) and Hospital Anxiety and Depression Scale (HADS)), which are widely used to detect depression and anxiety, they are developed to target depression and anxiety, not measuring general negative emotions (including anger, nervousness, fear etc.), additionally the above measurements fail to assess the negative emotional experience in the long-term management of chronic conditions.

The Negative Emotions due to Chronic Illness Screening Test (NECIS) was developed by Huang et al²⁴ in Taiwan in 2017. It is a simple self-reported, time-saving measurement originally developed to screen a wide range of negative emotional experience among patients with chronic illness in primary clinical settings. It includes 8 items, 5 items for the general negative emotions and 3 items for the subjective judgments referring to disease-control requirements. It only takes a few minutes to complete. The NECIS has been validated among individuals with chronic diseases in primary care settings in Taiwan. Psychometric properties including the internal consistency reliability, test-retest reliability, criterion-related validity and cut-off point have been established.²⁴

Considering the high prevalence of negative emotions among patients with CAD and its adverse influence on disease prognosis, it is of great necessity to timely assess negative emotions and provide psychological support for this population. The NECIS can serve as a suitable tool to measure the negative emotions. Though the NECIS has been validated in patients with chronic illness in Taiwan, it has not been administered especially in patients with CAD, and psychometric properties of a measurement may differ when applied in different population and cultural backgrounds.²⁵ Furthermore, the prior work has not established the construct validity of the NECIS, however, as a newly developed measurement, the construct validity is a powerful evidence to support the psychometric properties. Therefore, the evidence of construct validity is needed to further support the psychometric characteristics of the NECIS. The purpose of this study was to examine the construct validity of the NECIS in patients with CAD in mainland China.

Method

Design and Setting

This cross-sectional study was approved by the Institutional Review Board (IRB) of Wuhan University and was conducted in accordance with the Declaration of Helsinki. The investigation was conducted in general teaching tertiary hospitals affiliated to the university.

Participants

Inpatients with CAD were recruited conveniently from cardiovascular medicine departments of three teaching hospitals in Wuhan, China from October 2017 to April 2018. The inclusion criteria were patients who: 1) had confirmed diagnoses with CAD according to the American Heart Association (AHA) guideline;²⁶ 2) were able to read and understand Mandarin Chinese; and 3) were above 18 years old. The exclusion criterion was patients who 1) had severe mental or cognitive disorders (such as schizophrenia, dementia), which might influence the understanding and completion of the questionnaires. These patients were identified based on the medical records; 2) had severe dysfunction of the main organs or serious medical complications. All eligible patients have received the written description about this research. Participants were asked to complete all instruments after they agreed to participate in this study and signed consent forms. For those with blurred vision and had difficulty in reading, the researcher performed a structured interview through reading the items aloud and recording the responses.

The sample size for the psychometric analysis was determined based on the recommendation that the number of samples should be, at least, five to ten times the number of total variables.²⁷ The NECIS includes 8 items, thus a minimum of 80 subjects is required. Considering that in this study both exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) would be performed to detect the internal structure of the NECIS, plus the loss rate (20%), we estimated that a minimum of 192 subjects would be required.

Measurements

Participants were asked to complete the NECIS, Control Attitudes Scale-Revised (CAS-R), Multidimensional Scale of Perceived Social Support (MSPSS). Additionally, demographic data including the age, gender, education level, marital status, location, etc., were extracted from the medical records.

The Negative Emotions Due to Chronic Illness Screening Test (NECIS)

NECIS is a new and innovative tool developed by Huang et al in Taiwan in 2017, which is used to assess the general emotional disturbance in patients with chronic illness.²⁴ It consists of 8 items referring to different negative emotional experience and troublesome feelings in chronic disease management: (1) nervousness or anxiety, (2) sorrow accompanied by crying, (3) sadness or depression, (4) anger and (5) worry or fear; (6) troublesome feelings in taking medication, (7) troublesome feelings in controlling diet and (8) troublesome feeling in doing exercise regularly. Participants rate how often they experience the overall negative emotions and feel troubled by daily disease control requirements on a 5-point Likert-type scale (1 = never, 5 = always). Summing scores range from 8 to 40, with higher total scores indicating more severe emotional disturbance. This tool has been tested in patients with chronic illness in Taiwan. The Cronbach’s α coefficients were above 0.80. The test–retest reliability was 0.71.

Control Attitudes Scale – Revised (CAS-R)

CAS-R is an 8-item scale developed by Debra K. Moser.²⁸ It is applied to measure one’s perceived control in the context of cardiac diseases. It consists of 8 items, which is rated on an ordinal scale from 1 (totally disagree) to 5 (totally agree). CAS-R is scored by adding all item scores before two items (items 5 and 8) are reversed. Possible scores range from 8 (no perceived control) to 40 (the highest level of perceived control), with higher scores indicating better perceived control toward cardiac diseases. The English version of CAS-R was psychometrically tested in three cardiac samples.²⁸ The Cronbach’s alphas in all samples were greater than 0.70. In Mainland China, Li-li Huang translated it into Chinese and validated in elderly patients with heart failure.²⁹ The Cronbach’s alpha was 0.874 and test-retest validity was 0.825. The Cronbach’s alpha in this study was 0.770.

The Multidimensional Scale of Perceived Social Support (MSPSS)

The MSPSS is a widely used instrument to measure the perceived social support from family, friends, and significant others.³⁰ It includes 12 items and is answered on a 7-point Likert scale (1=very strongly disagree; 7=very strongly agree). The MSPSS is scored through summing all item scores. Total scores range from 12 to 84, higher scores suggesting more perceived social support. The MSPSS has been proven to possess sound psychometric properties in a variety of population, including patients with cardiovascular diseases.^30–34 The traditional Chinese version of MSPSS was first translated by Chou,³⁴ and later the simplified Chinese version was translated by Guan et al³¹ These two studies provided evidence of good psychometric properties of the Chinese version of MSPSS.^31,34 In this study, the Cronbach’s α coefficient was 0.885.

Data Analysis

The demographic data were described through descriptive statistics. Continuous variables were described through mean values and standard deviations (SDs), while categorical/nominal variables were shown through frequencies and percentages. Prior to data analysis, all continuous data were tested for normal distribution by Shapiro-Wilk test, the P value <0.05 was considered a violation of the normality assumption.³⁵

Construct validity of the NECIS was assessed via factor validity, convergent validity and discriminant validity. Additionally, the assumed relationship between negative emotions with other associated variables was tested to provide additional evidence of construct validity.

Firstly, before performing factor analysis, Kaiser-Meyer-Olkin Test (KMO) and Bartlett’s test of sphericity would be performed to assess correlation matrix and sample adequacy, which were recommended to evaluate the suitability for factor analysis. The value of the KMO index >0.7 is considered acceptable and the value of Bartlett’s test of sphericity <0.05 isconsidered significant.³⁶ If the data were suitable to perform factor analysis, we used half of the data to perform exploratory factor analysis (EFA) to gain insight into the internal structure of the NECIS. Considering item responses of the NECIS was ordinal in nature, we chose the robust maximum likelihood estimator (MLR) with geomin (oblique) rotation.³⁷ The number of factors was determined based on flowing criteria: 1) the eigenvalues >1; 2) factor loadings > 0.5; 3) no cross-loading items.²⁷ Secondly, we conducted confirmatory factor analysis (CFA) using the remaining half of the data to verify the factorial structure revealed in EFA. CFA was a modeling technique to confirm whether a proposed factorial structure manifests the correlation or covariance of observable variables.³⁸ To prevent multivariate non-normal distribution, robust maximum likelihood estimation (MLR) was employed.³⁷ Model fit indices, including Chi-square by degrees of freedom ratio (χ²/df), goodness-of-fit index (GFI), comparative fit index (CFI), root-mean-square error of approximation (RMSEA) and standardized root-mean-square residual (SRMR), were used to determine whether the expected model fitted the data. The model was considered acceptable if 1≤χ2/df≤3 (no significance), GFI>0.90, CFI>0.90, RMSEA ≤0.08, SRMR<0.08.³⁹

Thirdly, convergent validity was assessed via the following three indicators. The first was the factor loading of each item. High factor loadings (≥0.5) of items on a specific construct manifest items within that construct have much in common, which shows high convergent validity.⁴⁰ The second indicator was average variance extracted (AVE). AVE refers to the average variance extracted from the item loadings of a specific construct, the value greater than 0.5 indicating the high convergent validity.⁴¹ The third indicator was composite reliability (CR), the value equal to or greater than 0.70 is viewed as acceptable.²⁷

Fourthly, discriminant validity was examined through comparing the AVEs of the constructs with the values of the squared correlations between the constructs (R²).²⁷ It was established when the value of AVE for each factor was larger than the squared correlation between constructs (R²).²⁷

Finally, additional evidence of construct validity was obtained through testing the assumed relationship between the negative emotions with other associated variables. Based on previous empirical evidence, perceived control was negatively correlated with negative emotions such as anxiety and depression,^28,42–44 and individuals with higher perceived social support would have lower negative emotions.^45–47 Therefore, two hypotheses were proposed and tested in this study: 1) Hypothesis 1: Participants with higher scores of CAS-R would report lower scores of NECIS; 2) Hypothesis 2: Participants with higher scores of MSPSS would show lower scores of NECIS. Spearman correlation or Pearson product-moment correlation was used to test the above hypotheses. Evidence of construct validity was supported by confirming these two hypotheses.

The statistical package SPSS version 22.0 (SPSS Inc, Chicago, Illinois) and Mplus v.7 (Los Angeles, CA) were used for the statistical analysis. Significance was set at the 0.05 level (two-tailed test).

Result

Sample Size and Demographic Characteristics

A total of 420 eligible patients were recruited in this study. Of these 420 subjects, 32 patients refused to participate and sign informed consents, and 12 patients did not complete all the instruments. The reasons for those who refused to participate and sign informed consents included lack of interest or lack of time. Therefore, a sum of 376 subjects completed all the instruments. The mean age was (65.32±12.46) years, ranging from 25 to 88 years. The demographic characteristics of the participants are shown in Table 1.

Table 1 Demographic Characteristics

Factorial Validity

The results of KMO and Bartlett’s test suggested this study had a sufficient sample size relative to the number of items, and the correlation matrix was not an identity matrix (KMO=0.830; χ²=1126.486, P <0.001), which indicated that data of this present study were suitable for factor analysis. EFA revealed a two-factor solution with Eigen values >1, which explained 64.6% of the total variance. Factor loading coefficient of each item was sufficient, ranging from 0.519 to 0.852 (Table 2). In this two-factor model, factor 1 reflected the overall negative emotional experience (including items 1 to 5), named “general negative emotions”. Factor 2 manifested troublesome feelings regarding chronic disease management (such as taking prescribed medication, controlling diet and exercising regularly) (including items 6 to 8), named “negative emotions in disease management”.

Table 2 EFA for NECIS

CFA was subsequently performed to confirm the two-factorial structure established in EFA. The result showed this two-factor CFA model fitted the data well (χ²/df=1.900; RMSEA=0.069; CFI=0.966; TLI=0.949; SRMR =0.045). All items had significant parameters on the designated factor (P <0.001), with factor loading coefficients ranging from 0.556 to 0.847.

Convergent Validity

The convergent validity of the NECIS was evaluated via examining factor loadings, AVE and CR (Table 3). Based on the result of CFA, the standardized factor loading coefficients of items in the respective constructs were above 0.50, which exceeded the acceptable level. The AVE values were above the acceptable level (0.50), with 0.556 for the factor “general negative emotions”, and 0.542 for the factor“negative emotions in disease management”. That indicated that items of each factor belonged to their respective factor. The CR value for the factor “general negative emotions” was 0.862, and for the factor “negative emotions in disease management” was 0.776. Both values were sufficient for the acceptable criterion (0.70), which indicated good composite reliability.

Table 3 Convergent Validity of the NECIS

Discriminant Validity

The correlation between the two constructs was 0.318, thus the squared correlation between the constructs (R²) was 0.101. The AVE values for both dimensions were larger than 0.101, which provided the evidence of sufficient discriminant validity of the NECIS.

Hypotheses Testing

Additional evidence of the construct validity was obtained through testing two hypotheses: 1) Participants with higher scores of CAS-R would report lower scores of NECIS; 2) Participants with higher scores of MSPSS would report lower scores of NECIS. Considering the data were not in normal distribution, Spearman correlation analysis was performed to examine the above relationships. As hypothesized, the result showed that patients with higher scores of CAS-R reported lower NECIS scores (r=−0.287, P<0.001), which confirmed hypothesis 1. The correlation coefficient between the NECIS and the MSPSS was −0.528 (P<0.001), which supported hypothesis 2.

Discussion

This present study examined the construct validity of the NECIS in patients with CAD at the primary clinical setting in mainland China. The result showed sound factorial validity, convergent and discriminant validity, also assumed relationships between negative emotions with other associated variables (perceived control and perceived social support) were supported. This study filled the gap regarding the construct validity of the NECIS when applied to patients living with CAD in mainland China.

Factorial Validity

This study performed both EFA and CFA to detect the internal structure of the NECIS. EFA was the data-driven statistical method to explore the internal structure of a measurement, which aimed to establish the construct of the tested measurement when administering in a certain population.⁴⁸ The result of EFA identified a two-factor structure of the NECIS, which was corresponding to its two parts of the contents: general negative emotions and troublesome feelings regarding chronic disease management. Then, a further CFA was performed to confirm whether the two-factor model could fit the data collected from this target sample. Results of the CFA illustrated this two-factor structure fitted the data well, which confirmed the proposed structure. The results of factor analysis supported the sound factorial validity of the NECIS. This was the first study to report the internal structure of the NECIS, and the present study filled this gap to provide evidence of factorial validity of this newly developed tool.

Convergent Validity and Discriminant validity

Convergent validity reflects that a set of items are presumed to measure the same construct.²⁷ In this study, it was assessed through factor loadings, AVE and CR. All these indicators were shown satisfactory, which established adequate convergent validity of the NECIS. Discriminant validity, however, reflected that a set of items are supposed to measure different constructs.⁴⁹ In this study, the AVE value for each factor higher than the squared correlation between the factors (R²) provided the evidence of satisfactory discriminant validity of the NECIS. Prior to this study, no evidence of convergent and discriminant validity for the NECIS was published. This study filled this gap to provide further evidence of the construct validity of the NECIS.

Hypothesis Testing

Except for the factor validity, convergent validity and discriminant validity, proposed hypotheses were tested to provide additional evidence of construct validity. The construct validity of a measurement could be established through developing and testing hypotheses regarding the assumed relationship with other variables based on previous evidence.⁵⁰ In order to test the conceptual accuracy of the NECIS, two hypotheses regarding the relationship between negative emotions with its associated variables (perceived control, perceived social support) were tested. Results of the correlation analysis confirmed both two hypotheses, which contributed to further support for construct validity of the NECIS.

Limitations

This study had several limitations. Firstly, a convenience sample limited the representativeness of the target population with CAD, which might restrict the generalization of the findings. Secondly, in this study, we tested construct validity only in patients with CAD, the subgroup of individuals with chronic illness. Therefore, we were not sure whether the construct validity of the NECIS was still acceptable when applied to those with other types of chronic illness except cardiovascular disease. Future researches could expand the subjects to those with different types of chronic illness. Thirdly, we did not compare the response from those who require a structured interview through reading the items aloud due to impaired vision with that from other respondents. There might exist the information bias from those who required a structured interview.

Conclusion

This study specially examined the construct validity of the NECIS in patients with CAD in mainland China. The results indicated that NECIS had satisfactory construct validity when applied into the target sample in the present study, which provided further evidence of the psychometrical properties of this newly developed scale. The NECIS had the potential to be a suitable measurement to assess negative emotions in clinical mental health service or psychology intervention for Chinese patients with CAD.