Validation of Diabetes Knowledge Questionnaire (DKQ) in the Taiwanese Population — Concurrent Validity with Diabetes-Specific Quality of Life Questionnaire Module

Introduction

Diabetes mellitus (DM) is a challenging chronic illness, leading over time to target organ damage individually;¹ therefore, DM imposes an economic burden globally.² The prevalence of diabetes has been steadily increasing worldwide, including in Taiwan.³ From 2005 to 2014 in Taiwan, the total population with DM increased by 66%; the prevalence of DM was approximately 50% in those aged ≥ 65 years.⁴ Self-care begins with diabetes knowledge and adherence to good self-care practices.⁵ Moreover, diabetes knowledge could be classified as different concepts, including diabetes etiology and symptoms (eg, type 1 or type 2 DM; polyuria/polydipsia),⁶ different levels of nursing (eg, glycemic targets⁷ and inspection of foot as elementary nursing; wound care⁸ and balanced diet as intermediate nursing), complications (eg, diabetic kidney disease⁹), and relevant treatments (eg, optimization of glucose control).¹⁰

To help patients with DM have better self-care behavior, a program named as pay for performance (P4P) was launched in Taiwan in 2001. The P4P program for patients with DM has demonstrated higher medication adherence,¹¹ fewer diabetes-related hospitalizations,¹² less lower extremity amputation,¹³ and better survival¹⁴ compared to non-P4P usual care. The P4P program covers patient empowerment, health literacy and self-care behaviors, resulting in reduced HbA1c level.^15–17 Among the positive effects of the P4P program, an important feature is that it may assist patients with DM in improving their health literacy, which is instrumental in improving diabetes knowledge and self-efficacy.¹⁸

Indeed, the importance of health literacy has been documented. Patient score of complication awareness and health literacy were significantly and negatively associated with the level of glycated hemoglobin (HbA1c).¹⁹ Significant and direct pathways were demonstrated from empowerment perceptions to health literacy; subsequently leading to self-efficacy, and finally to self-care behavior, which contributes to reduction in HbA1c levels.¹⁶ Some prior studies have demonstrated different factors associated with health literacy. For example, Kim et al reported that lower health literacy was associated with older age and having less education;²⁰ Asmelash et al found that occupation, education and marital status were significantly associated with attitude and practice towards glycemic control.²¹ Although prior evidence demonstrates the potential factors on diabetes knowledge, it is important for healthcare providers to have a validated instrument to assess diabetes knowledge for their patients with DM. With the use of a validated instrument assessing diabetes knowledge, healthcare providers may evaluate the effectiveness of a program on diabetes knowledge improvement.

The Diabetes Knowledge Questionnaire (DKQ), a 24-item questionnaire, has been developed by the Starr County Diabetes Education Study²² to evaluate patients’ diabetes knowledge. DKQ was firstly developed in Spanish and English, and has been translated and validated in many languages, including Simplified Chinese.²³ To make DKQ free to openly access in Chinese-using countries, the present study’s objective was to translate and validate DKQ (and make modifications if needed) among patients with type 2 DM into the Traditional Chinese language used in Taiwan. More specifically, factor structure, internal consistency, known-group validity, responsiveness, test–retest reliability, and concurrent validity were examined for the DKQ.

Materials and Methods

Participants and Procedure

The participants of this study were patients with type 2 DM, who were recruited from a certified health-promoting diabetes clinic with certified diabetes educators (CDE) including physicians, nutritionists, and registered nurses in Taiwan. Participants covered three subgroups for different psychometric testing: (1) existing patients with type 2 DM in the diabetes clinic for all psychometric testing, except for responsiveness and test–retest reliability, (2) newly arrived patients with type 2 DM who have not received a DKQ-specific diabetes education in the clinic for responsiveness, and (3) patients completing a re-test 3 months after the first test for test–retest reliability. For newly arrived patients with type 2 DM (subgroup 2), the certified diabetes educators (CDE) provided a diabetes education that took approximately 30 minutes. The content of DKQ was used as the DKQ-specific education material (Appendix). The DKQ is used as a pre-test and post-test. A pre-test is conducted before the first-time diabetes education. A post-test is conducted after the diabetes education session. Approval of this study was obtained from the National Cheng Kung University Human Research Ethics Committee (NCKU HREC-E-110–443-2). Moreover, the present study complies with the Declaration of Helsinki.

Patients with type 2 DM were invited to participate in this study after their regular visit to the clinic, and they could decide whether they wanted to participate. The paper-based questionnaires were administered one-on-one in a certified health-promoting diabetes clinic. Data collection of the two questionnaires take about 5 to 10 minutes to complete. The participants received an NT$ 50 (about USD 1.8) convenience store gift card after they completed the questionnaires. For the subgroup of newcomers, they received an NT$ 100 dollars gift card after completing both pre-test and post-test. Moreover, all the participants signed a written informed consent.

Instrument

Diabetes Knowledge Questionnaire (DKQ)

The 24-item DKQ (DKQ-24)²² was the shortened version of the 60-item DKQ. Response choices to the questions include (1) Yes, (2) No, and (3) I do not know. The items were scored as correct or incorrect, and the correct items were summed to obtain a total score; the higher the score, the better the diabetes knowledge. DKQ-24 contains 17 positively worded items and 7 negatively worded items, and attained a reliability coefficient of 0.78; the group-by-time interaction effect was statistically significant (P < 0.001).²² The DKQ was translated into Traditional Chinese for Taiwanese after the permission was obtained from the original authors, who have published the DKQ.²² In the translation procedure, the standard procedure that ensures the linguistic validity was adopted.²⁴ Specifically, forward translation, back translation, and reconciliation were carried out in the translation process. First, the DKQ was translated from English into Traditional Chinese version by a native Mandarin translator with a bachelor’s degree in English. Second, the Traditional Chinese version was translated back into English by a professional Taiwanese translator with an MA degree in English. Lastly, the translation was reviewed by two experts including a psychometrician and a physician with diabetes educator certification. Moreover, the simplified Chinese version of the DKQ developed by Dr. Hu²³ was also used as reference during the translation process.

Diabetes-Specific Quality of Life Module (DMQoL)

In addition to the DKQ, this study utilized the diabetes-specific quality of life questionnaire module (DMQoL) developed by Lin et al²⁵ to evaluate the quality of life of patients with type 2 DM. The DMQoL is a 10-item questionnaire with 5-point Likert scale. The DMQoL includes 10 items selected from 20 items, as shown in Table 1. For item 12, the response scale ranged from 1 (None at all) to 5 (Extremely). For the other items, the scale ranged from 1 (very dissatisfied) to 5 (most satisfied).

Table 1 Demographic Characteristics

Laboratory Data

Glycated hemoglobin, HbA1c, was measured by turbidimetric inhibition immunoassay (TINIA) method using Roche’s cobas c513. This method is certified by the National Glycohemoglobin Standardization Program.

Data Analysis

SPSS 23.0 (IBM SPSS Incorp., Armonk, NY) was used to analyze all the data. The demographics of patients with type 2 DM were analyzed using descriptive statistics. Categorical variables were reported using numbers and percentage, and continuous variables used mean and standard deviation (SD). Exploratory factor analysis (EFA) was used to examine the factors of the DKQ. Moreover, factor loadings derived from the EFA were used to remove items for improving the factor structure of the DKQ. Principle axis factoring with promax oblique rotation method was used to extract the number of factors based on an eigenvalue >1. Items with a factor loading less than 0.3 were removed unless an item had a strong reason to be retained; then, another EFA was conducted using the same procedure. Cronbach’s α and corrected item–total correlation was later used to verify the internal consistency of each factor score.

To evaluate the quality of each item of the DKQ, item difficulty and item discrimination were assessed. Item difficulty was calculated using the item difficulty index (ie, the number of correct responses to each test item of DKQ divided by the total number of responses including both correct and incorrect ones). Item discrimination using Truman Kelley’s “27% of sample” group size, values of 0.4 and above are regarded as high and less than 0.2 as low by R.L. Ebel.²⁶ Item discrimination was calculated using the Pearson correlation between participant responses to a particular item and total scores on all items on the test. Specifically, item discrimination of each item was determined by the relationship between the participant’s DKQ performance in each item and his/her overall scores.

Test–retest reliability of the entire DKQ was examined using the Pearson correlation coefficient. Test–retest reliability > 0.4 was considered to be acceptable according to following suggestions: test–retest reliability between 0.4 and 0.75 is fair to good; > 0.75 is excellent.^27,28 The concurrent validity of the DKQ was examined using the Pearson correlation with the DMQoL. The correlation between each factor of DKQ-18 and DMQoL score were examined separately in three education levels, ie ≤ junior high school, senior high school, and ≥ college. Known-group validity²⁹ of DKQ-18 was examined using two demographic characteristics, ie gender and education level, and one clinical characteristic, ie HbA1c level. Gender was analyzed by an independent t-test; education and HbA1c levels were analyzed by analysis of variance (ANOVA). Lastly, the responsiveness of the DKQ-18 was tested among 51 participants who received a pre-test and post-test of DKQ. The scores of pre-test and post-test were analyzed using paired-samples t-test and Cohen’s d. Lastly, hierarchical regression was constructed to explore the potential predictors for diabetes knowledge assessing using DKQ.

Results

A total of 425 patients with type 2 DM (mean±SD age=58.4±11.6 years), with females accounting for 197 (46.4%), were enrolled. The numbers of education levels are 104 (24.47%) for the subgroup of ≤ junior high school, 154 (36.24%) for the subgroup of senior high school, and 167 (39.29%) for the subgroup of ≥ college, respectively. The average duration of diabetes was 8.9±7.2 years. The HbA1c of each participant was the mean value of 5 latest HbA1c values with three months apart, accounting for 92.25% of 374 participants. In the pre-test and post-test group of 51 participants, 58.8% only has a single HbA1c value because they are new patients visiting this clinic. For diabetes treatment, 21 (4.94%) were on diet control only, 331 (77.88%) received oral anti-diabetic drugs, 7 (1.65%) insulin, GLP-1 agonists or both, and 66 (15.53%) oral antidiabetic agents (OAD) and injection therapy (Table 1).

Six items showing low factor loadings (ie, < 0.3) in the EFA were removed. The retained 18 items (ie, DKQ-18) were used to examine the factor structure and five factors were extracted. The DKQ-18 had six items loaded on the first factor (named as Diabetes etiology and symptoms); three items on the second factor (Intermediate nursing); four items on the third factor (Complications); two items on the fourth factor (Diet and treatment); and three items on the fifth factor (Elementary nursing). Moreover, the eigenvalue (1.099 to 3.345) and the explained variance (2.261 to 14.806) for the second EFA results were satisfactory, indicating that the five-factor structure is appropriate for the 18-item DKQ (ie, DKQ-18; Table 2).

Table 2 Rotated Factor Matrix of Exploratory Factor Analysis

After ensuring the suitable items and factor structure of the DKQ-18, the 18 DKQ items were tested for their item properties. Specifically, Table 3 presents their item difficulties (ranged between 0.285 and 0.993) and item discriminations (ranged between 0.199 and 0.578). Afterward, the entire DKQ-18 were examined for their internal consistency, which was satisfactory (Cronbach’s α= 0.732 and McDonald’s ω=0.748). Concurrent validity of the DKQ-18 was supported by the positive correlations with DMQoL (Table 4). Specifically, Factor 2 (Intermediate nursing) was significantly associated with DMQoL (r=0.210; P=0.007) in the college or above education group. Factor 4 (Diet and treatment) was significantly associated with DMQoL (r=0.201; P=0.04) in junior high school or below education group and the senior high school group (r= 0.200; P=0.01). Factor 5 (Elementary nursing) was significantly associated with DMQoL (r=0.288; P=0.003) in junior high school or below education group and the senior high school group (r=0.170; P=0.035).

Table 3 Item Difficulty and Item Discrimination of Each Item in DKQ-18 (N=425)

Table 4 Concurrent Validity of the DKQ-18 (18 Items Diabetes Knowledge Questionnaire) with the Diabetes Mellitus Quality of Life (DMQoL) Score

The known-group validity of the DKQ-18 was adequate as participants with different genders, different educational levels, or different HbA1c levels had significantly different scores in the DKQ-18 (Table 5). Women had significantly better DKQ-18 score (13.28±2.48 vs 12.57± 3.22; P< 0.011) when compared with men. Participants with a higher level of education showed better score in DKQ-18, with 11.70±2.992 (junior high school or below), 12.82±2.724 (senior high school), and 13.73±2.795 (college or above); p<0.001) for each respective education group. Participants with HbA1c≤7 had better score in DKQ-18 compared to group of HbA1c level >8.5 (13.35±2.664 vs 11.99±3.272; P=0.002). The test–retest reliability with an interval of three months on a subgroup of 77 patients with type 2 DM was 0.69. Regarding the responsiveness of the DKQ-18, the paired-samples t-test revealed a significant difference in the scores for pre-test score (13.41±4.88) and post-test score (19.27±3.27; P<0.0001) among the 51 newcomers. Moreover, the effect size was large (Cohen’s d=1.41).

Table 5 Known-Group Validity of DKQ-18

Predictors of diabetes knowledge: A two-level hierarchical multiple regression analysis was conducted with the demographic variables of gender, age, body mass index and level of education as a first block followed by the addition of a second block of variables: diabetes duration, HbA1c value and diabetes treatment. Table 6 presents the results of the regression analysis. In the first step of the model, demographic variables accounted for 8% of the variance in diabetes knowledge, and gender and level of education were significant predictors. In the second step, demographic variables, along with clinical variables enhanced the prediction of diabetes knowledge, explaining 11% of the total variance in diabetes knowledge, with diabetes duration and HbA1c value as significant predictors. Table 7 shows the six deleted items, where significance occurred rarely in terms of gender, education, HbA1c, and DM duration.

Table 6 Hierarchical Regression Analysis for Variables Predicting Diabetes Knowledge (n=425)

Table 7 t-Test and ANOVA of Six Items Deleted from DKQ-24

Discussion

Six items were removed from the DKQ based on the result of EFA. A potential reason for the poor psychometric performance of the six items could be explained by the extreme item difficulties (ie, either too hard or too simple). Among the 425 participants, 363 (85.41%) gave the wrong answer for item 1; 216 (50.82%) gave the wrong answer for item 7. For item 2, item 6, item 10, and item 11, most of the participants gave the correct answer, ranging from 70.35% to 88.24%. Therefore, these items were unable to effectively distinguish the differences among participants’ diabetes knowledge, resulting in poor performance in EFA loadings.

Among the six deleted items, gender difference only occurred in item 1 (p=0.023) (Table 7). In terms of education, a significant ANOVA test was obtained for item 2 and item 11. For item 2, p-value was 0.035 for group of ≤ junior high school vs group of senior high school; for item 11, p-value was 0.002 for group of senior high school vs group of ≥college. In terms of HbA1c, a significant ANOVA test was only obtained for item 10, with p-value 0.04 for group of HbA1c≤7 vs group of 7<HbA1c≤8.5. In terms of DM duration, no significant ANOVA test was obtained between subgroups of DM duration.

Five factors were extracted according to the EFA results, which are named as Diabetes etiology and symptoms (Factor 1), Intermediate nursing (Factor 2), Complications (Factor 3), Diet and treatment (Factor 4), and Elementary nursing (Factor 5). For Factor 1, it is named as Diabetes etiology and symptoms because this factor covers the cause of diabetes (item 3), insulin production (item 4), best way to check diabetes (item 9), insulin reaction (item 12), and signs of hypoglycemia and hyperglycemia (item 21 and 22). Factor 2 is named as Intermediate nursing because this factor covers the equal importance of diet, exercise and medication (item 13), the correct way to cleanse a cut wound (item 17), avoidance of tight socks (item 23), and the content of diabetes diet (item 24). Factor 3 is named as Complications because this factor covers poor circulation (item 14), diabetic neuropathy (item 19), and nerve damage from diabetes, ie diabetic neuropathy (item 20). Factor 4 is named as Diet and treatment because this factor includes high blood glucose in untreated diabetes (item 5) and the way of food preparation (item 18). Factor 5 is named as Elementary nursing because this factor describes the normal values for fasting blood glucose (item 8), cutting toenails with caution (item 16), and slow healing of cuts in patients with DM (item 15). The items in Factor 5 were Elementary nursing knowledge compared to Factor 2, Intermediate nursing knowledge.

The Cronbach’s α value of DKQ-18 in this study was 0.732 and McDonald’s ω=0.748, which are comparable to prior findings.^22,23,30,31 For example, Cronbach’s α of the original DKQ-24 was 0.78 in the Spanish version²² and 0.89 in the Simplified Chinese version.²³ Moreover, the DKQ-24 also had satisfactory internal consistency in the Urdu version (Cronbach’s α = 0.702)³⁰ and Croatian language (α = 0.740).³¹ However, prior evidence on internal consistency was based on Cronbach’s α and the present study extended the evidence of DKQ’s internal consistency to another robust method, ie, McDonald’s ω. In addition, internal consistency including McDonald’s ω is presented in this study. In the psychometrics field, there is a trend of using McDonald’s ω to replace Cronbach’s α because McDonald’s ω can provide more robust estimation than α in internal consistency.³² Moreover, to the best of our knowledge, this is the first study to demonstrate the test–retest reliability of the DKQ (although the present study used an 18-item version instead of a 24-item version); and the results revealed that the DKQ-18 had acceptable test–retest reliability in a three-month interval. In this study, participants with better HbA1c level (≤7) had better score in DKQ-18 compared to group of HbA1c level >8.5 (p=0.002), as was the case in the studies by Hu²³ and Bukhsh.³⁰

This study also measured the responsiveness of DKQ by using the DKQ as a test before and after diabetes education. After diabetes education, the improvements of the participants were reflected by increase in the DKQ score. The responsiveness with a large effect size (Cohen’s d=1.41) because the content of diabetes education covered the knowledge highlighted in the DKQ-18. Therefore, diabetes care professionals and researchers can use DKQ-18 as an evaluation tool to examine if their diabetes care programs are effective.

Health literacy played an influential role in diabetes knowledge;³³ however, associations of health literacy with glycemic control,³³ self-monitoring of blood glucose, foot care and medication adherence were inconclusive.¹⁸ Educated graduate and professional subjects had good level of knowledge about diabetes, frequent exercise and a better control over postprandial (PP) blood sugar as compared to those having lower level of education.³⁴ The different levels of nursing knowledge could be due to separating nursing knowledge into two different factors, which could be further explained based on the concurrent validity of the DKQ-18 and the DMQoL. First, the subgroup with the highest education level (≥ college) are already familiar with the knowledge in Factor 5 (Elementary nursing); therefore, no significant correlation was found between DKQ-18 and DMQoL. The two subgroups with a lower education level (≤ junior high school, high school) might be unfamiliar with the items in Factor 5: Elementary nursing. With learning, a significant correlation could be achieved between DKQ-18 and DMQoL. Second, Factor 2 (Intermediate nursing) is relatively more difficult for the subgroup of junior high school or below and the subgroup of high school. As for Factor 2, significant correlation was found between DKQ-18 and DMQoL for group of college or above but it was not the case for lower education subgroup. The lower education subgroups cannot comply with the Intermediate nursing knowledge; therefore, their knowledge was not reflected in their DMQoL scores.

According to education level, concurrent validity of DKQ-18 with DMQoL showed that “Intermediate nursing” (F2) was significantly associated with DMQoL in the group of college or above (Table 4). “Elementary nursing” (F5) was significantly associated with DMQoL in the group of junior high school or below and senior high school. “Diet and treatment” (F4) was significantly associated with DMQoL in the group of junior high school or below and group of senior high school. We have arguments as follows: First, the group with the highest education level (≥college) are already familiar with the knowledge in Factor 5 (Elementary nursing); therefore, no significant correlation was found between DKQ-18 score and DMQoL score. In contrast, the two groups with lower education level (≤junior high school, high school) might be initially unfamiliar with the items in Factor 5: Elementary nursing. With learning, those two groups could interpret items of elementary nursing; in this case, significant correlation could be achieved between DKQ-18 and DMQoL. Second, Factor 2 (Intermediate nursing) is relatively more difficult for the lower education level groups (group of junior high school or below, and group of high school). As for Factor 2, significant correlation was found between DKQ-18 and DMQoL for the group with college or above education levels, but this was not the case for lower education group. The lower education groups could not interpret the Intermediate nursing knowledge and could not comply with those intermediate nursing items; therefore, their knowledge was not reflected in their DMQoL scores.

Implications: The shorted DKQ-18 can serve as a tool to evaluate the knowledge of participants with type 2 diabetes. In addition, DKQ-guided diabetes education provides a time-saving and to-the-point education for patients with type 2 DM, improving the self-care by participants. More specifically, given that the DKQ-18 was found to have good responsiveness, future studies on developing programs in diabetes knowledge improvement can use DKQ-18 to evaluate whether a newly designed program is effective. Moreover, the present study found that the DKQ-18 can be constructed by five factors, future programs in diabetes knowledge improvement could consider using the five factors to design an in-depth program to tackle different aspects of diabetes knowledge, especially in implementing diabetes pay-for-performance programs in Taiwan.

A limitation of this study is that the participants of this study come from a single diabetes clinic and the DKQ was self-reported, which made recall bias³⁵ possible. Due to the learning effect issue, the responsiveness could be over-estimated. In particular, the participants with higher education levels may have better performance than those with lower education level after the diabetes education. Social desirability may have influenced the responses of participants.²³ That is, the participants may select either “Yes” or “No” answers instead of “I don’t know.” Not knowing the answer could be embarrassing to some participants. During the interviews, when some participants seemed unsure about the answer, they said that they think this should be “Yes.”

Conclusion

Knowledge of diabetes play an important role in diabetes self-management, which should be addressed through integrated education program. The shortened DKQ-18 could evaluate the diabetes knowledge of patients with type 2 DM. Participants with a higher level of education showed better scores in DKQ-18. Participants with HbA1c ≤ 7 also had better scores in DKQ-18 compared to group of HbA1c level > 8.5. Meanwhile, a DKQ-guided diabetes education could provide a to-The-point education for patients with type 2 DM, improving their self-management through a diabetes team.