Psychometric properties of the Persian Health Care Climate Questionnaire (HCCQ-P): assessment of type 2 diabetes care supportiveness in Iran

Introduction

Health-care systems play a key role in responding to the growing problem of patients with type 2 diabetes and management of their complications.¹ Supporting and reinforcing patients’ role, especially in self-care, could lead to better disease-specific clinical outcomes, such as low Hb_A1c levels.² Thus, a productive relationship between patients and health-care providers (HCPs) is an indispensable component of effective care provision in type 2 diabetes and other chronic diseases.^2–4

Type 2 diabetes is one of the most prominent and growing chronic health conditions worldwide, with 415 million patients^5,6 and 5 million deaths in 2015 globally.⁶ Research evidence suggests that effective self-care, ie, adherence to medication regimen, changes in lifestyle such as managing weight, increasing physical activity and maintaining a healthy diet, can lead to better blood glucose control.^5–9

Self-determination theory, developed by Deci and Ryan,^10,11 was used in this study as a framework to study motivation and the ways in which people attempt to boost their energy and mobilize their efforts to perform activities of daily life. This theory could also be applicable in research on a wide range of disease-related behaviors, including preventive or therapeutic practices that healthy people or patients might adopt for a better health outcome. This theory tries to explain how the interplay between intrinsic and extrinsic factors could sustain individuals’ passions and their efforts to reach personal aspirations. A special focus of the theory is on sociocultural factors that could facilitate or undermine the individuals’ sense of autonomy, competence and relatedness, which were argued to be fundamental in maintaining motivation and performance.^5,8,12 Perceived support for autonomy, self-care competence and psychological relatedness based on the theory are crucial for optimal functioning of a patient in accordance with the prescribed medications or behavioral pattern.^{5,7–9,12–14}

A patient-centered and conducive health-care climate could provide the required support for the universally agreed needs of the patients; therefore, having an in-depth knowledge and understanding of its cyclical nature could offer a timely opportunity for a constructive and far-reaching look at the established organizational behavior to promote best practice performance in health-care settings.^5,7,9,15

Various studies on the impact of the health-care climate have shown that by affecting autonomy and independent motivation, selection of a healthy lifestyle and gaining a sense of self-care competency could be improved in patients with type 2 diabetes.^7–9,16

The Health Care Climate Questionnaire (HCCQ) was developed by Williams et al¹⁷ and measures the perceived needs of patients and extent of HCPs' support with regard to independence and success in motivating patients to accept greater responsibility for their own health.^12,18–20

The scale has also been applied to measure the influence of a protective care environment on various health topics, including smoking cessation²¹ and identification of perceived barriers to reduce sodium intake in patients with chronic kidney disease,²² as well as a feasibility study of physiotherapist training to present the theory-based self-management of osteoarthritis and lower back pain,²³ in addition to other subject areas, such as mental disabilities,²⁴ tobacco dependence,²⁵ diabetes-related topics,^9,15,27–29 and complex gender-based interplay between the clinical and individual-based outcomes.²⁶

Other related scales have been developed based on the HCCQ, such as the Virtual Health Inventory Questionnaire,¹⁸ Important Other Climate Questionnaire for cigarette smoking³⁰ and diet,³⁰ and a modified HCCQ for breast cancer patients.³¹ The HCCQ has been validated for use in other languages, such as German,¹⁹ French¹⁴ and Dutch.²⁴

Previous studies indicated that Iranian patients with diabetes generally do not receive the required quality of services. A part of this perceived deficiency may be associated with the health-care provision facilities and the overall perceived performance environment in the health-care centers.^32,33

Since this scale has not been validated for use in Persian-speaking populations, the main aim of this study was to evaluate the psychometric properties of the Persian version of the Health Care Climate Questionnaire (HCCQ-P) for use in future studies. Patients' perceived needs and HCPs’ support are universal phenomena in health-care provision, and cross-cultural validation of the HCCQ could help researchers and HCPs to work with a standard tool for the purposes of data collection and patient assessment.

Material and methods

Study objectives

The main objective of the present study was to translate the HCCQ and conduct psychometric validation of the HCCQ-P to assess its potential applicability in the study of health-care climate support for patients in Iran and other Persian-speaking countries. Therefore, content validity and reliability scores were estimated, and structural validity verification procedures based on the results of exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were performed.

Study sample

Based on the recommended sample size for accurate and precise factor analysis,³⁴ 177 patients with type 2 diabetes registered in the diabetes clinic of the Shahid Madani Hospital in the city of Khoy, north-west Iran, were recruited out of 1,600 active patients' files with a random sampling method. The inclusion criteria were type 2 diabetes, over 30 years of age and being a native resident of the city; and the exclusion criteria were type 1 diabetes, gestational diabetes, having mental disorders such as Alzheimer's disease, dementia or congenital mental retardation, or having a severe limiting disability such as quadriplegia or limiting cardiovascular disease.

Content validity

Standard translation/back-translation procedures^35-37 were performed to prepare a preliminary draft of the scale in Persian. In the first stage, the original instrument (Table S1) was translated into Persian by two professional translators; and in the next stage, the Persian version (Figure S1) was translated into English by two other professional translators who were familiar with the main study topic. In the third stage, the back-translated English version of the HCCQ was compared with the original version in terms of content, and idiomatic and semantic affinity. Minor corrections were made as a consequence and the final draft was approved by the researchers. The face and content validity of the translated version was checked by a group of experts consisting 15 academic staff of the Tabriz University of Medical Sciences, who had work experience in the field of type 2 diabetes. Clarity of the wording, placement of the items and scoring were checked by the experts at this stage, and their feedback was followed to make corrections. Content validity ratio (CVR) and content validity index (CVI) scores were calculated to determine the suitability of items on the translated questionnaire. A CVI of 0.80 or higher³⁸ for individual items and a CVR above 0.49 were deemed as acceptable cut-off points.³⁹ The translated questionnaire was also pilot tested on 20 patients regarding understandability of the questions and response items.

Reliability

Reliability of the HCCQ-P was assessed using Cronbach’s alpha and intraclass correlation coefficients (ICCs).

Data collection

Trained interviewers were employed for collecting face-to-face data about age, gender, marital status, occupation, level of education, place of residence, income level, type of medication (oral/injectable) and having hypertension or type 2 diabetes-related complications, in addition to the HCCQ-P items.

Procedure and ethical considerations

This study was approved by the Ethics Committee of Tabriz University of Medical Sciences (approval number IR.TBZMED.REC.1396.192) and conducted with permission from Shahid Madani Hospital’s administrative authorities and in accordance with the Helsinki Declaration.⁴⁰ Complete information was provided to the study participants at the beginning of the recruitment stage and written consent was obtained from the patients. Participants were also reassured that their information would be kept confidential and that they could withdraw from the study at any time without being questioned.

Statistical analysis

Individual participants’ total scores, their means and standard deviations were calculated after applying a reverse-coding mechanism for item 13, as recommended by the scale’s developer, owing to the negative wording of the question in that item.⁴¹

The Pearson’s coefficient of skewness and Fisher’s measure of kurtosis were used to assess the asymmetry of the study data at the next stage. The range of skewness coefficient was deemed to be between −1 and +1, and Fisher’s measure of kurtosis was considered to be between −1.96 and +1.96.⁴²

The ceiling and floor effects were checked for detecting outliers (ie, more than 15% of the respondents reached the lowest or highest possible scores).³⁴ The Cronbach’s alpha coefficient was calculated to appraise the scale’s internal consistency, and values greater than or equal to 0.7 were considered to be acceptable.³⁴ The test–retest procedure was used and the ICC was calculated by asking 20 patients with type 2 diabetes to fill in the study questionnaire after 20 days. ICC values below 0.4, between 0.41 and 0.6, and above 0.8 were considered as weak, moderate, and good and excellent indicators of the participants’ performance in the two measurements, respectively.³⁴ To facilitate the decision-making process, values above 0.7 were considered acceptable.⁴³

The correlation between the scale items’ scores and the total scale score was examined and found to be in the range of 0.82–0.91, representing a high consistency between items.

To perform psychometric verification of the translated questionnaire, half of the collected data, selected at random, was used for EFA and the rest for CFA. The Keyser–Meyer–Olkin (KMO) indicator was used to verify the adequacy of the sample size and Bartlett’s test of sphericity was devised to confirm the correlation between items and suitability of performing factor analysis. During the exploratory factor analysis, principal axis factoring and direct oblimin rotation were applied; eigenvalues greater than 1 were used to determine the number of factor solutions, and factor loadings greater than 0.3 were considered to assign questions to factors. The required modifications were also performed to improve the model fit in the CFA. All statistical analyses in this study were implemented with SPSS 20, STATA 15 and AMOS 20 software.

Results

Sample characteristics

In total, 177 patients with diabetes gave their consent to participate in the study. The normality of the study data was verified by skewness (−0.737) and kurtosis (−0.486) tests. The mean age of the participants was 57.49 years (SD=11.57), 64.4% of them were female and 85.3% were married. Other demographic characteristics of the recruited participants are shown in Table 1. The mean total score of the respondents was 81.34 (SD=21.24), in the range of 29–105.

Table 1 Sociodemographic characteristics of the participants in the psychometric properties appraisal of the Persian Health Care Climate Questionnaire (HCCQ-P)

Content validity

Quantitative and qualitative content validity appraisals were performed to ensure the integrity of the HCCQ-P. In the qualitative phase, the questionnaire was checked by an expert group in terms of wording, item allocation, grammar and scaling. The CVI and CVR were calculated based on the experts’ opinions in terms of relevance, clarity and simplicity of the items in the translated scale. The CVI values for all the items were equal to or above 0.8 and the CVR value for the total scale was 0.91.

Factorial (construct) validity

EFA

The KMO index for the study data was 0.94 and Bartlett’s test of sphericity was significant (χ²=−1745.31, P<0.001), which indicates the adequacy of the study sample for EFA application. For all 15 items of the scale, eigenvalues were greater than 1, explaining 71.74% of the total variance. The eigenvalues and other fit indices are presented in Table 2.

Table 2 Eigenvalues and fit indices in the psychometric properties appraisal of the Persian Health Care Climate Questionnaire (HCCQ-P)

CFA

CFA was conducted on the 15-item scale to test the fitness of the model obtained from EFA. As shown in Figure 1, all goodness-of-fit indices (root mean square error of approximation=0.14, comparative fit index=0.914, Tucker Lewis index=0.899, standardized root mean square residual=0.03) were satisfactory, which indicates the one-component model fit of the study data.

Figure 1 Visual representation of the items loadings in one component model obtained from confirmatory factor analysis (n=177) in the psychometric properties appraisal of the Persian Health Care Climate Questionnaire (HCCQ-P).

Figure 2 Scree plot of the eigenvalues in the PCA analysis of the study data for psychometric appraisal of the Persian Health Care Climate Questionnaire (HCCQ-P)

Feasibility

The floor effect was not observed in this study, ie, none of the participants obtained the lowest score (15), but 16.95% obtained the highest score (105), representing a trivial ceiling effect.

Reliability

The reliability index of Cronbach’s alpha was α=0.945, which is above the acceptable threshold.⁴³ Test–retest analysis was also performed to test the stability of the questionnaire over time, and the result was in the range of satisfactory values (ICC=0.979, P=0.000) (Table 3).

Table 3 Mean score and standard deviation, skewness, kurtosis and reliability indices in the psychometric properties appraisal of the Persian Health Care Climate Questionnaire (HCCQ-P) (N=177)

Discussion

Based on the results of previous studies, Iranian patients with type 2 diabetes were not entirely satisfied with the health-care services provided, and suffered from irreversible complications that were partly due to defects in their health care.^{32,33,44–46} The existence of a standard tool to identify the extent of support provided by HCPs for patients with type 2 diabetes could help these patients and contribute toward the effective implementation and impact evaluation of intervention programs.

The main purpose of this study was to investigate the psychometric properties of the HCCQ-P and assess its potential applicability in studies on the supportiveness of health-care delivery environments for health behavior change in patients with type 2 diabetes. The health-care climate is considered as one of the most influential elements in motivating behavioral change in patients with chronic diseases.^5,7–9 In addition, the use of the HCCQ in health-care environments can reduce the incidence of disease complications by positively affecting clinical results and improving health outcomes, especially the control of blood glucose levels and decrease in Hb_A1c levels.^7,47

As the distribution of the study attendants’ major sociodemographic characteristics was sought to be representative of the largest population of Iranian patients with type 2 diabetes, the study results demonstrate the applicability of the HCCQ-P as a suitable instrument for assessing the degree of health-care climate supportiveness for Iranian patients with type 2 diabetes. The psychometric analysis of the HCCQ-P on the studied Iranian patients produced similar results to the findings of previous studies on psychometric assessment of the HCCQ in France,¹⁴ the USA,³⁰ the Netherlands^18,24 and Germany.¹⁹

Content validity

The translated Persian version of the HCCQ was proven to be a valid scale based on the CVR and CVI scores. Based on the experts’ opinions, the HCCQ-P can be used to measure Iranian patients’ perceptions concerning the extent of autonomy support provided by the country’s health-care system in general and their HCPs in particular.

Construct validity

The study results showed an excellent KMO factor structure, which indicates sufficient sample size for factor analysis. The estimated index was similar to the findings obtained in the psychometric evaluation of the German version of the HCCQ.¹⁹ In this study, the EFA output indicated a one-factor solution as the best model fit (Figure 2). This solution was in agreement with the findings of the original study in verifying the psychometric properties of the HCCQ scale.¹⁷ Several previous studies have also reported a one-factor solution as the best model fit in HCCQ psychometric appraisal.^14,24 In the German version of the scale, however, two major factors were found: a factor with 14 items and a factor with only one item (item 13), which had a negative load.¹⁹ The results of CFA in the present study confirmed the fitness of the model obtained from EFA, which was consistent with the results reported in previous studies.^14,24

Feasibility

Based on the study results, a partial ceiling effect was observed (the highest score was achieved by 16.9%), which is consistent with the ceiling effect reported in the HCCQ psychometric assessment by Frielink et al²⁴ and Schmidt et al.¹⁹ However, no ceiling and floor effects were reported in the study by Czajkowska et al.¹⁴

Reliability

The estimated internal consistency index for the HCCQ-P was similar to the reported measure in the original HCCQ assessment study¹⁷ and other validation studies on HCCQ.^14,18,19 The test–retest reliability index in this study was higher than that obtained in the study by Czajkowska et al.¹⁴

Limitations

Despite an acceptable representation of the validity and reliability measures, the study findings should be interpreted with caution. The studied sample’s imbalanced gender ratio and distinct sociodemographic characteristics of the participants have the potential to cause bias in the study findings. The participants’ previous experiences in dealing with the health-care system, their trust in the health-care system, baseline mental status and overall life satisfaction are other factors that might be considered in the interpretation of the results.

As a brief tool, the HCCQ focuses on the overall supportiveness of the health-care delivery setting rather than the details and several dimensions of the appropriateness of the provided care. Further studies are required to verify these preliminary results by recruiting samples from various parts of the country, which has a very diverse socioeconomic, geographic and development profile.

The observed small ceiling effect in this study is a common limitation in research on patient satisfaction, since study respondents are generally reluctant to criticize their HCPs, for several reasons.¹⁹ However, sociocultural differences may exist based on literacy level, previous experiences, insurance coverage and social norms, which should be considered in the interpretation of the findings.

Long-term stability of the patients’ views should also be assessed in future studies because of the influences that contextual factors (eg, economic turbulence) may have on the patients’ prospects.

Convergent and divergent validities of the translated HCCQ were not studied in this research since a reliable scale was not identified by the authors to measure the health-care system supportiveness. Therefore, further assessment of the HCCQ-P’s construct validity using a comparable tool is recommended.

The study findings cannot be generalized beyond outpatient care settings since the sample was drawn only from the outpatient diabetes clinic. Additional studies are required if application of the HCCQ-P is to be considered on authentic and complex inpatient diabetes cases, as recommended in other relevant studies.¹⁹

Despite the above limitations, strengths of this study are the applied explicit cultural adaptation in the translation process and the high participation rate, which suggests a low risk of selection bias that would otherwise have had the potential to affect the findings.

Conclusion

The study results revealed that the Persian version of HCCQ has appropriate levels of validity, simplicity, reliability and feasibility to assess the Iranian health-care system’s supportiveness for patients with type 2 diabetes.

Since widely scattered Persian-speaking populations are living in different countries of West Asia, including Iran, Afghanistan and Tajikistan, cross-cultural validation of the HCCQ-P could be a promising recommendation for researchers across the country to assess its applicability in a wider geographical area and to compare the countries’ profiles. Application of the translated scale could also provide valid information about the mechanisms underlying the improvement of type 2 diabetes care and other chronic care in the West Asia region.