The Psychometric Properties of General Adherence Scale in Chinese (GAS-C) in Patients with Type 2 Diabetes Using Insulin

Introduction

China has one of the highest rates of diabetes in the world. Research data in 2013 showed that the prevalence rate of diabetes in China was about 11.6%, and the prevalence rate of Pre-diabetes’ prevalence rate was approximately 50.1%, which meant that about 113.9 million Chinese were suffering from diabetes and about 493.4 million Chinese were pre-diabetic.^1,2 China is under tremendous pressure in the prevention and treatment of diabetes.

Studies have found that non-adherence in T2D patients is more common.³ Unless severe life-threatening complications occur, T2D patients’ adherence is more likely to be lower than other chronic diseases.⁴ A 2003 World Health Organization report showed that in developed countries, the treatment adherence of patients with T2D is only about 50%,⁵ In a longitudinal, large sample study, only 39.6% of patients with T2D were still following their doctor’s advice to take medicine after 24 months.⁶ Adherence was 62% in patients who used insulin for a long time.⁷

Patients with T2D have poor adherence and are prone to various acute and chronic complications, increasing hospitalization and death risk and imposing a huge economic burden on families and society.^8,9 A 2011 study conducted in Hangzhou, China, found that the average annual hospitalization cost for patients with T2D complications was $2000–3466.¹⁰ A 2018 survey conducted in multiple locations in China found that the median average cost per hospitalization for people with T2D with or without health insurance was $1552 and $1490, respectively.¹¹ A 2019 study of non-hospitalized T2D patients in the Chinese community found that the average direct medical burden for these patients was $682 a year, compared to $922 for those over the age of 70.¹²

Poor adherence to glucose management programs is the main reason for long-term unsatisfactory glycemic control in diabetic patients.¹³ For patients with T2D, it is essential to improve their adherence. Therefore, we need to conduct active research on adherence of patients with T2D and propose effective interventions based on the research results. T2D, patients need to follow their healthcare professionals’ advice in several areas, including healthy lifestyle, regular monitoring, taking or injecting medications, physical therapy, and preventing various acute and chronic complications.^14,15 Therefore, we can assess the patient’s adherence as a whole.

To date, Chinese researchers have conducted more studies on medication adherence in T2D patients. However, there is a lack of scientifically validated research tools to assess T2D patients’ general adherence in China. The General Adherence Scale (GAS) is a useful standardized measurement tool for studying general adherence in patients with chronic diseases, which was developed by DiMatteo and Hays. In the famous medical outcome study (MOS) undertaken by Rand Corporation of America, GAS was used to assess chronic patients’ general adherence to heart disease, hypertension, and diabetes.¹⁶ Many studies have proven that the scale is simple, easy to use, and has a good measuring effect, which can be applied to the study of adherence of patients such as T2D.^17–19 Therefore, we decided to revise the General Adherence Scale in Chinese (GAS-C) and select middle-aged and elderly Chinese patients with T2D as the research objects to evaluate the GAS-C’s reliability and validity.

Materials and Methods

The study consists of two parts:

study A: translating GAS into Chinese and testing its reliability and validity to form a Chinese version of GAS-C;

study B: performing a confirmatory factor analysis (CFA) of GAS-C.

This study was conducted in accordance with the Declaration of Helsinki. Before the research was carried out, we reported the research plan to the Ethics Committee of the Shanghai Pudong New Area Mental Health Center. After the committee’s review and approval (the approval number: 2,017,009), the study began. This study was conducted from May 2018 to september 2020 in the diabetes wards of several general hospitals in Haicheng City, Liaoning Province, China. Data collection began after the hospitalized T2D patient’s vital signs stabilized.

Participants

The inclusion criteria of the study subjects were:

1. patients with T2D who meet the WHO diagnostic criteria for diabetes,
2. use insulin to treat T2D for at least 4 weeks,
3. aged 45 to 74 years,
4. Han people who had been living in the survey site for at least five consecutive years at the time of the survey,
5. voluntary participation.

Subjects were excluded from this study if they met the following exclusion criteria:

1. those who were seriously ill and unable to complete the study. Including the current acute complications of diabetes, life-threatening medical conditions in the acute phase of other diseases other than diabetes, such as severe stroke, myocardial infarction or active malignancy, and terminal illness with a life expectancy less than 2 years (per health care provider or patient report).
2. those who had a disturbance of consciousness (Glasgow coma score lower than 15) and cognitive impairment (According to the Chinese “Mini-mental state examination” scoring standard: illiteracy, more than 14 points: primary school educational level, more than 19 points; middle school educational level and above: more than 24 points.).
3. those who were drug dependent or suffered from various severe mental illnesses (Including schizophrenia, schizotypal and delusional disorders; severe depressive episode; schizoaffective disorders; other mental disorders due to brain damage and dysfunction and to physical disease)

Finally, 136 subjects with T2D were included in the study A. Based on the N:q rule of 10–20 subjects per parameter, the 136 subjects for the five-item GAS questionnaire were sufficiency.^20,21 Study B contained 200 subjects, which meets CFA’s requirements for sample size.²²

Instruments

General Information Questionnaire

It consisted of some items about socio-demographic information of the subjects’ current treatment status and glycosylated hemoglobin level.

GAS

GAS was developed by DiMatteo and Hays, consists of five items, which is used to assess the overall tendency of patients with chronic diseases to comply with doctor’s recommendations in the last four weeks.^23,24 The response format of GAS is a six-point Likert scale, ranging from “None of the time” (1) to “All of the time” (6). Items 1 and 3 of the GAS were reverse coded. The total score of the scale is from 6 to 30. The GAS scale has one dimension, and its internal consistency reliability is acceptable (α=0.78). The correlation coefficient r of the test-retest reliability two years later is 0.41.²⁵ In this study, it will be revised in Chinese.

MMAS-8

MMAS-8 was developed by Morisky and is mainly used to investigate medication adherence of patients with chronic diseases. The scale contains eight items, and the total score is the sum of the scores of each item.²⁶ The MMAS-8 is a reliable and valid tool used to test whether patients comply with doctor’s orders to take their medications.^27,28 In this study, MMAS-8 was used under license to assess the criterion-related validity of the GAS-C.

Translation and Adaptation of the Scale

We contact the developer of GAS at first. After obtaining the developer’s permission, we started to revise the Chinese version of GAS.^23,25 We integrated the cross-culture revision method recommended by experts to revise GAS in Chinese.^29,30 We divide the process of translation and adaptation of GAS into (1) initial pre-translation, (2) summary of expert committee recommendations, (3) back-translation, (4) re-suggestions by expert committees, (5) content validity evaluation, (6) pre-testing and cognitive interviews (7) establishment of the final version of GAS-C.³¹

Initial Pre-Translation

First, two experts who are bilingual in Chinese and English (one bilingual expert is a team member who is familiar with the research, the other is a professor in the Department of English) translated GAS from English to Chinese back-to-back, resulting in two Chinese drafts of the scale.

Summary of Expert Committee Recommendations

Two bilingual experts were asked to work with the research team members to compare and analyze the two initial drafts. Then, we integrated out one initial Chinese draft of the scale.

Back-Translation

Two other bilingual experts whose native language is English back-to-back translated the prepared initial Chinese draft back into English to form two back-translated English scales.

Re-Suggestions by Expert Committees

One epidemiologist and four bilingual experts mentioned above were asked to meet with the research team members. They compared and analyzed the original English version of the scale, two draft English to Chinese versions, one integrated initial Chinese draft, and the two back-translated versions to finalize the initial Chinese version of GAS.

Content Validity Evaluation

Six physicians working on diabetes wards were invited to evaluate the content validity of the GAS-C. A four-point Likert scale was used to examine the degree of correlation between each item and GAS-C’s measurement objectives. A pre-test would be conducted after content validity meets the evaluation criteria.

Pre-Testing and Cognitive Interviews

Subsequently, among the inpatients in the diabetes ward of a general hospital, 15 patients with T2D who met the research criteria were selected for the prediction test. After the patients filled out the initial Chinese version of GAS, the researchers conducted a cognitive interview to investigate the comprehensibility and conceptual equivalence of the text in the questionnaire, find the problem, and make adjustments in time.

Establishment of the Final Version of GAS-C

There was no ambiguity in the subjects’ understanding of the initial Chinese version of the scale during the interview. The scale was filled in smoothly. The initial version could better reflect the original author’s intention, so it was decided to determine the initial Chinese version as the final Chinese version of the GAS.

Data Collection

A public health intern, as the investigator, introduced the study to the hospitalized patients at the bedside and asked them if they would like to participate in the study, if the inpatients agreed to participate and met the subjects criteria, then enter the study. The questionnaire was completed the day before the subject was discharged from the hospital. The subject was asked to sign an informed consent form before filling out the questionnaire. If the subject is illiterate or visually impaired, the questionnaire will be read aloud to him or her. After the subjects had answered according to his or her own judgment, the investigator would fill in the questionnaire instead of the subjects. During the questioning process, the investigator tried to be as calm, objective, non-intrusive as possible. After completing the questionnaire, the investigator asked if the subjects was willing to fill out the GAS questionnaire once more two weeks later. We randomly selected 100 subjects who agreed to be retested and surveyed them again using the GAS-C two weeks later when they returned to the clinic for a follow-up visit.

Statistical Analysis

Except for the use of AMOS 23.0 in the confirmatory factor analysis (CFA), the rest of the data analysis was performed using the SPSS 23.0. The subjects’ socio-demographic information was described by mean and standard deviation, frequency, percentage. Continuous variables were expressed as mean± standard deviation (SD), while categorical variables were expressed as counts and percentages. P<0.05 is a statistically significant difference.

Item Analysis

To test the appropriateness of each item of GAS-C and decide to keep or delete an item, we conducted an item analysis on the scale, used homogeneity test and other methods to screen the item. There are four retention criteria. If (1) statistics the correlation between each item and the total score, if the difference test P≥0.05 or the correlation coefficient r<0.4, delete the item,³² (2) In factor analysis, if the communalities of an item was less than 0.4, the item would be deleted,²¹ (3) If the Measure of Sampling Adequacy (MSA) of a certain item ≥0.5, it means that the item could be included in the factor analysis process, otherwise it would be deleted,³³ (4) If the value of the internal consistency coefficient α of the scale increases after deleting a specific item, then the item will be removed.³¹

Validity Analysis

We analyzed the content validity, construct validity, and criterion-related validity of GAS-C. According to the content validity measurement method proposed by Lynn,³⁴ six doctors working in the diabetes ward were invited to evaluate the content validity of the initial GAS-C. One researcher introduced the concept of general adherence and the purpose of this study to the six physicians before scoring. When all physicians reported that they had understood, the researcher invited them to read the five items of the GAS-C in turn and to judge the degree of correlation between each item and the measurement objectives of GAS-C. The scoring method is a four-point Likert response. 1= not relevant to 4= very relevant. After scoring, the item-level content validity index (I-CVI) and the scale-level content validity index (S-CVI) were calculated. S-CVI was calculated using the S-CVI/Ave method, that is, calculated the average of I-CVI for each item. According to Lynn’s content validity standards, if the number of experts is ≥6, those items with I-CVI ≥0.78 can be retained.³⁴ Under Polit’s recommendation, when the S-CVI/Ave≥0.9, the scale-level content validity is acceptable.³⁵

An exploratory factor analysis (EFA) was conducted to test the construct validity of the GAS-C. If the Kaiser-Meyer-Olkin measure of sampling adequacy (KMO)≥0.70,³³ and the difference of Bartlett’s Test of Sphericity Approx is statistically significant (p<0.05),³⁶ the GAS-C as a whole is judged to be suitable for factor analysis.³⁷ If the Measure of Sampling Adequacy (MSA) of an item≥ 0.5 indicates that the item could be included in the factor analysis process, otherwise, it would be deleted.³³ We chose the principal component analysis (PCA) combined with the Varimax orthogonal rotation method to analyze our data. The Kaiser’s eigenvalue>1 principle was used to extract common factors.³⁸ Factors with Initial Eigenvalues greater than one were retained.³⁷ At the same time, we used the scree test to assist our judgment, observed scree plot, and visually judged the number of extracted factors.³⁹

In conducting the confirmatory factor analysis (CFA), we used the following criteria to assess the goodness of the model. Ratio of chi-square to degrees of freedom (CMIN/df)<5; goodness of fit index (GFI), Tucker–Lewis index (TLI), comparative fit index (CFI) value >0.9; root mean square residual (RMR)<0.05; root mean square error of approximation (RMSEA) got a value of 0.08 or less, it would imply that the model was good fits with the data.^40–42

We analyzed the correlation between GAS-C and MMAS-8 to test the criterion-related validity of GAS-C. If the Pearson correlation coefficient r is statistically significant and greater than 0.4, the criterion-related validity is acceptable.⁴³

Reliability Analysis

We evaluated the reliability of GAS-C by analyzing internal consistency reliability, intraclass correlation coefficient (ICC), test-retest reliability. The Cronbach’s alpha value judged the internal consistency reliability of the GAS-C. If Cronbach’s alpha value is more than or equal to 0.70, the scale’s internal consistency reliability is appropriate.³¹ The GAS-C has five items. The folded-in-half items are unequal in length, so we calculated the unequal length Spearman-Brown coefficient. The judgment standard is the same as that of Cronbach’s alpha value, which is greater than or equal to 0.7 is good.³¹

Cicchetti suggests that an ICC is good between 0.6 and 0.74, and more than or equal to 0.75 is excellent.^44,45 We accepted the suggestion. The Pearson correlation coefficient r of test-retest reliability ≥0.7 is acceptable.^32,46

Ceiling Effect and Floor Effect

We also examined the data for the ceiling effect and floor effect to see if there was any response bias. The ceiling effect means the percentage of subjects achieving the maximum score on an item exceeded 15%. Similarly, The floor effect means the percentage of subjects achieving the minimum score on an item exceeded 15%.⁴⁷

Results

A total of 336 T2D patients were collected, from which 200 were randomly selected as Subject B for the CFA in Study B and the remaining 136 as Subject A for the reliability and validity analysis of the GAS-C in Study A. There was no missing data. Table 1 records the characteristics of the subjects.

Table 1 General Characteristics of the Participants in Sample A and B

Item Analysis

As seen in the item analysis (Table 2), the correlation coefficient r between each item score and the total scale score was Between 0.861 and 0.926, all greater than or equal to 0.4, and the differences were statistically significant (p<0.01). In factor analysis, the items’ communalities were between 0.740 and 0.862, all higher than 0.4. The five items’ MSA values were between 0.883 and 0.945, all greater than 0.5, indicating that the five items in the scale were all suitable for entering the factor analysis process.In the reliability analysis, the internal consistency reliability α of the GAS-C was 0.942. Each time an attempt was made to delete an item in turn, the alpha value decreases accordingly. We ended up keeping all the items.

Table 2 Summary of Item Analysis of GAS

Validity Analysis

Content Validity

The calculation results of content validity showed that the I-CVI of the five items were between 0.83 and 1, all higher than 0.78, and the S-CVI/Ave of GAS-C was 0.94. They all met the criteria for judging content validity.³⁴ GAS-C has good content validity (Table 3).

Table 3 I-CVI and S-CVI of GAS

Result of Constructing Validity

We carried out an EFA to test the construct validity of the GAS-C (Table 4). We chose PCA combined with the Varimax orthogonal rotation method to analyze our data. KMO value was equal to 0.899≥0.7. The difference of Bartlett’s Test had Statistical significance (p<0.05). The Chi-square value was equal to 611.821, and the degree of freedom (df) was 10 (p<0.001). These results indicated that there were common factors among the variables, and GAS-C was suitable for factor analysis.^33,36 The communalities of the items were between 0.740 and 0.862, all greater than or equal to 0.4. The MSA values were between 0.883 and 0.945, all greater than or equal to 0.5. Five items in the scale were suitable for the factor analysis process. A common factor was extracted by applying the Kaiser’s eigenvalue-over-one principle. It could explain 81.403% of the total variance. The Initial Eigenvalues was 4.070. Factor loadings of items of GAS-C range from 0.860 to 0.929. It is also clear from the scree plot that it is more appropriate to extract a common factor (Figure 1).

Table 4 Summary of Exploratory Factor Analysis

Figure 1 Scree plot. Notes:Only one factor have a Kaiser’s eigenvalue ≥1; extraction method, principal component analysis.

We used confirmatory factor analysis to validate the one-dimensionalproperties of GAS-C. CFA analysis showed that χ2= 10.161, df=5, CMIN/DF=2.032<5; GFI =0.981, CFI =0.996, TLI =0.992, their values all greater than 0.9; the RMR =0.011<0.05, and the RMSEA =0.072. All the fit indexes indicated that the one-factormodel was good fits with the data of GAS-C. Standardized factor loadings ranged between 0.88 for item 5 and 0.95 for item 2 (Figure 2).

Figure 2 Path diagram for GAS-C’s CFA. Abbreviations: CFA, confirmatory factor analysis; GAS-C, General Adherence Scale in Chinese.

Criterion-RelatedValidity

To understand GAS-C’s criterion-related validity, we performed a Pearson correlation analysis between GAS-C and MMAS-8. We found that the correlation coefficients r=0.542>0.4, p<0.001, which indicates that the criterion-related validity of GAS-C is acceptable.⁴³

Reliability Analysis

Internal consistency refers to the extent that all items in a scale contribute positively towards measuring the same construct. Cronbach’s alpha is the most common measure for assessing reliability and determining the items’ internal consistency in a scale³² A higher Cronbach’s alpha coefficient means a higher homogeneity between items examining the scale’s same dimension. The Cronbach’s alpha reliability coefficient was 0.942≥0.7, which means the GAS-C’s internal consistency reliability is ideal.³¹ GAS-C’s ICC was 0.941 (95% CI 0.924–0.955). Based on Cicchetti’s criteria, the ICC is excellent.⁴⁴ Two weeks later, we used GAS-C to measured 100 subjects again. The correlation coefficient r for the total GAS-C score obtained from the pre and post measurements was 0.772 p< 0.001. The test-retest reliability of GAS-C is acceptable.⁴³ The split-half method applies to reliability measures for those scales which assume unidimensionality of the construct. We chose the Spearman-Brown method to calculate the split-half reliability of the GAS-C. The GAS-C five items were randomly divided into two unequal parts by SPSS statistical software, with the first half containing items 1, 2, and 3 and the second half containing items 4 and 5. The correlation coefficient between the first half and the second half was 0.880. The unequal Spearman-Brown coefficient value was 0.939 ≥ 0.7, which indicates that the GAS-C’s split-half reliability was good.³¹

Ceiling Effect and Floor Effect

No one in this survey had the lowest total GAS-C score of 5. There was no floor effect on the data. Out of 136 subjects, 20 subjects achieved a maximum total score of 30, which was 14.71% of all people less than the 15% ceiling effect criterion. The data did not have ceiling effect too.⁴⁷ The subjects had little to no response bias in this survey.

Discussion

Poor adherence to glucose management programs is the main reason for long-term unsatisfactory glycemic control in diabetic patients.¹³ Although there are a variety of effective measures to control blood glucose in T2D patients, only patients with good adherence can benefit from it. Therefore, it is necessary to carry out scientific research to improve adherence in T2D patients. The adherence of T2D patients is a kind of general adherence. In addition to the adherence of taking medicine, it also includes a healthy lifestyle, regular monitoring of blood glucose, injection of drugs, physical therapy, and the prevention and treatment of chronic complications.¹⁵ There is currently no research tool available in China that can measure general adherence in T2D patients. Therefore, we revised the GAS and hope that the GAS-C could be used for adherence studies in Chinese T2D patients.

After translation, back-translation, content validity evaluation, pre-experimentation, and cognitive interviews, we finalized the GAS. During the pre-test, the measured subjects gave positive comments about the comprehensibility and convenience of the questionnaire.

GAS-C also has good content validity. EFA revealed that the GAS-C is one-dimensional, which is the same as the scale structure of the original GAS, and CFA further supported this conclusion. The correlation coefficient r for the criterion-related validity of GAS-C is 0.542, which is relatively modest. Because the MMSA-8 scale selected in this study is mainly used to measure the patient’s medication adherence, and the adherence of GAS refers to general adherence throughout the prevention, treatment, and health care of patients. General adherence is related to and distinct from medication adherence. Medication adherence is a part of general adherence, which may explain why GAS and MMSA-8 are related, but the correlation is not too high.

The Cronbach’s Alpha of the original GAS was 0.78 in the baseline, and it was 0.79 for the two-year follow-up.²⁵ The Cronbach’s Alpha of GAS in the other two studies were 0.81 and 0.87, respectively.^16,48 It reached 0.942 in this study, indicating that both the original GAS and the GAS-C had good internal consistency reliability. However, because these studies selected entirely different subjects, so there is little comparability between their alpha values. The correlation coefficient r of GAS’s test-retest reliability still reached 0.41 two years later,¹⁶ and the r of GAS-C’s test-retest reliability was 0.772 after two weeks. Since two years and two weeks are not comparable, it remains to be seen whether our revised scale continues to be stable after two years. If conditions permit, we will conduct further research on this issue in two years. In this study, we also analyzed the ICC and split-half reliability of GAS-C. The results were satisfactory, indicating that overall, the reliability of GAS-C is nice.

Non-adherence is more common in people with diabetes.⁴ How to improve the adherence of T2D patients and how to control their blood glucose better is still a common challenge for medical professionals. It is necessary to research T2D patients’ adherence in various aspects to get out of this dilemma. For example, further understanding the reasons for the occurrence of non-adherence behavior and the manifestation of non-adherence behavior, as well as to grade the degree of non-adherence of patients, and then to develop a reasonable and effective intervention method to address the problems identified.

An essential point in researching adherence is that the researcher should insist on using scientific and useful research tools to draw more credible conclusions. Since GAS uses a self-assessment format for subjects, the subjects are likely to miss valid information by making subjective judgments. For example, a male patient with T2D has to rely on taking medications to control his blood sugar better. He has proper daily adherence with his doctor’s instructions to maintain a healthy lifestyle, control diet, and exercise, but he often forgets to take medication, leading to poor glycemic control. When he fills in a self-assessment questionnaire such as GAS, the patient may think that he is following the doctor’s advice in all aspects except taking medication, so he will likely make a very positive evaluation of his adherence. As another example, a male T2D patient with poor adherence develops severe life-threatening complications and is currently forced to inject insulin strictly as prescribed by his physician. Although his blood glucose control is perfect after using insulin, he still has various prejudices about using insulin. He is unwilling to use it if left to his own choice. How do we assess his adherence objectively, despite his blood glucose meeting the goal of treatment?

So far, there is still no clear conclusion about whether using subjective self-assessment questionnaires or relying on objective clinical indicators such as glycated hemoglobin value can truly reflect patient’s adherence. Researchers may need to conduct more research and discuss it.

Strengths

GAS is widely used, but as far as we know, this is the first revised Chinese version of GAS. This study’s research objectives and the inclusion and exclusion criteria of the research subjects are relatively clear. The sample size of this study met the requirements of the revised scale and structural equation model. Due to the efforts of the investigators, there was no missing data from the survey. GAS-C has relatively good reliability and validity, CFA also verify the goodness of the GAS-C model. All of the above are the strengths of this study.

Limitations

There are several shortcomings in this study. Firstly, the selection of the subjects was not representative enough. China is a multi-ethnic country. Not only are lifestyles and dietary structures very different among ethnic groups, but the proportion of T2D patients also varies.² This study only selected the urban Han population as the research object. Whether GAS-C can be applied to T2D patients of other ethnic groups needs further research.

Secondly, there were limitations in the age selection of the study subjects. The age range of the subjects in this study was 45–74 years old. Since the onset of T2D patients in China is increasingly showing a tendency to be younger,⁴⁹ more attention should be paid to the study of adherence in adolescent or pre-adult T2D patients.⁵⁰ For convenience sampling, the study did not include adolescent or pre-adult study subjects, which is another deficiency. We will further expand the subject’s age range to make up for this deficiency in the future.

Another shortcoming of this study is that the GAS is used to evaluate a patient’s compliance with physician recommendations within the last four weeks. But subject to the conditions we had to collect data measuring the retest reliability only after two weeks, despite the precautions we have taken, which might happen with the Hawthorne effect and social desirability bias. We do not know the long-term prediction effect of GAS-C on the adherence of T2D patients. We cannot answer whether those T2D patients who showed high scores in the GAS-C test would have a better blood sugar control result. We will solve these problems in the future.

Conclusion

We revised the Chinese version of GAS, which has superior reliability and validity. The scale is one-dimensional and has five items. It can be applied to the Han people’s general adherence study of middle-aged and elderly type 2 diabetes patients using insulin in urban China.