Back to Journals » Patient Preference and Adherence » Volume 13

A cultural adaptation and validation study of a self-report measure of the extent of and reasons for medication nonadherence among patients with diabetes in Singapore

Authors Liau YW , Cheow C, Leung KTY, Tan H, Low SF, Cheen HHM , Lim WC, Tan LL, Tan JZY, Lee ES , Xu SJ, Tan CYK , Phang JW, Phang JK , Lam MH, Blalock DV, Voils CI , Yap KZ , Kwan YH 

Received 21 March 2019

Accepted for publication 27 June 2019

Published 29 July 2019 Volume 2019:13 Pages 1241—1252


Checked for plagiarism Yes

Review by Single anonymous peer review

Peer reviewer comments 2

Editor who approved publication: Professor Qizhi (Cathy) Yao

Yuan Wei Liau,1 Celine Cheow,1 Kenneth Tin Yau Leung,2 Hejing Tan,3 Suat Fern Low,4 Hua Heng McVin Cheen,5 Woan Chyi Lim,6 Li Ling Tan,6 Joyce Zhen Yin Tan,7 Eng Sing Lee,8 Sandra Jialun Xu,9 Corrinne Yong Koon Tan,10 Jie Wen Phang,11 Jie Kie Phang,12 Miao Hui Lam,2 Dan V Blalock,13,14 Corrine I Voils,15,16 Kai Zhen Yap,1 Yu Heng Kwan2,17

1Department of Pharmacy, Faculty of Science, National University of Singapore, Singapore, Singapore; 2Department of Pharmacy, Khoo Teck Puat Hospital, Singapore, Singapore; 3Department of Pharmacy, Woodlands Health Campus, Singapore, Singapore; 4Department of Pharmacy, Yishun Community Hospital, Singapore, Singapore; 5Department of Pharmacy, Singapore General Hospital, Singapore, Singapore; 6Department of Pharmacy, Tan Tock Seng Hospital, Singapore, Singapore; 7Department of Pharmacy, Admiralty Medical Centre, Singapore, Singapore; 8Clinical Research Unit, Family Medicine Development Division, National Healthcare Group Polyclinics, Singapore, Singapore; 9Department of Pharmacy, National Healthcare Group Polyclinics (Hougang), Singapore, Singapore; 10Pharmacy Transformation Office, National Healthcare Group Pharmacy, Singapore, Singapore; 11Department of Pharmacy, Ng Teng Fong General Hospital, Singapore, Singapore; 12Department of Rheumatology and Immunology, Singapore General Hospital, Singapore, Singapore; 13Center of Innovation to Accelerate Discovery and Practice Transformation (ADAPT), Durham Veterans Affairs Health Care System, Durham, NC, USA; 14Department of Psychiatry and Behavioral Sciences, Duke University School of Medicine, Durham, NC, USA; 15Department of Surgery, University of Wisconsin School of Medicine and Public Health, Wisconsin, WI, USA; 16William S Middleton Memorial Veterans Hospital, Wisconsin, WI, USA; 17Program in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore

Background: This self-report measure is a new instrument to measure the extent of and reasons for medication adherence separately. However, few studies have assessed its psychometric properties in diabetic patients and also in Asian populations.
Objectives: To validate this self-report measure in diabetic patients in Singapore.
Methods: We collected data prospectively using a questionnaire among 393 diabetic patients from hospitals in Singapore from July 2018 to January 2019. Using the COnsensus-based Standards for the selection of health Measurement INstruments framework, we assessed face validity, internal consistency, test–retest reliability, structural validity, and measurement error. We tested four a priori hypotheses on correlation of extent score with patient-reported outcome measures to assess construct validity. We examined cross-cultural validity via measurement invariance across gender, age groups, and languages.
Results: We performed cognitive interviews with 30 consenting English-literate, Chinese-literate, and Malay-literate (10 patients per language) diabetic patients (age range 48–76 years, 53% male, disease duration range 1–30 years) and face validity was supported. Among 393 patients (mean age: 59.4±12.2 years, 50.9% female, 52.4% Chinese), we showed moderate internal consistency (Cronbach’s alpha =0.67) and test–retest reliability (intra-class coefficient=0.56 [95% CI 0.37–0.70]). We calculated smallest detectable change as 0.80. We established construct validity by meeting all four hypotheses. We showed structural validity as confirmatory factor analysis confirmed a one-factor model, with excellent fit statistics (Comparative Fit Index=1.0; Tucker-Lewis Index=1.0; Root Mean Square Error of Approximation<0.001; Standardized Root Mean Residuals<0.001). Analysis of cross-cultural validity supported configural invariance model but not metric invariance and scalar invariance model. Caution must be taken against directly comparing extent scores across gender, age groups, and languages.
Conclusion: This self-report measure is valid and reliable in measuring medication adherence in diabetic patients in Singapore.

Keywords: adherence, patient-reported outcome, quality of life, psychometric, singapore, diabetes


Diabetes mellitus is a metabolic disorder characterized by resistance to insulin action, insufficient insulin secretion, or both,1 causing hyperglycemia, which is an increase in blood glucose level. It is predicted that the number of people with diabetes worldwide will increase from 382 million people in 2013 to 592 million by 2035.2

Medication adherence is necessary for improved health outcomes and lower health care costs, especially in chronic diseases like diabetes.3 Conversely, poor adherence in chronic diseases leads to worsening health outcomes,4,5 higher risk of hospitalization6 and higher mortality rate.7 Diabetic treatment regimens are complex, causing adherence to diabetic medications to be difficult8 and worse compared to other chronic diseases.9 Poor adherence to diabetic medications has been found to be related to poorer glycemic control10 and greater risk of diabetic complications,7 such as retinopathy, neuropathy, nephropathy, and cardiovascular disease. Improving medication adherence can be a more efficient strategy than changing or adding additional treatments.11

Nonadherence can be measured directly and indirectly. Direct methods include measurement of concentrations of drugs or their metabolites or markers but are used infrequently due to difficulty, cost, and inability to provide feedback at the point of care.12 Indirect methods are more frequently reported in literature than direct measures12 and include pill counts, electronic monitoring devices, pharmacy refill records, and self-reports. Although there is no “gold” standard for measuring nonadherence,13 self-reports have unique advantages over other methods such as ease of completion, inexpensive, and being able to provide immediate feedback at the point of care.14

Given the high prevalence and health care cost associated with diabetes, a variety of behavioral interventions have been developed to increase adherence to diabetes medications.15 It is important to have a reliable, valid measure that assesses the extent of nonadherence and the reasons for nonadherence to identify patients who would benefit from intervention, inform the type of intervention to be done, and evaluate the effectiveness of intervention. There are many self-report measures of medication nonadherence validated in other patient populations like the 4-item and 8-item Morisky Medication Adherence Scale [MMAS-4 and MMAS-8],16 Medication Adherence Report Scale,17 and Hill-Bone Compliance Scale [Hill-Bone].18 However, studies have concluded that these current measures have inadequate reliability and validity across populations due to the design of these measures.19 Voils developed a self-report measure that seeks to address two issues with the existing self-report measures. First, existing measures conflate the extent of and reasons for medication nonadherence. The Voils measure resolves this issue by separately assessing the extent of and reasons for medication nonadherence using appropriate psychometric models.20 Second, some measures are unclear when conceptualizing stability of nonadherence over time14 and thus the Voils measure sets a specific recall period of 7 days,14 which was validated through cognitive interviews. The measure has been used in randomized controlled trials to assess medication nonadherence in patients with hypertension,21 dyslipidemia,22 and Type 2 diabetes in America.23

However, the self-report measure by Voils has not been validated in Singapore, and there are limited medication nonadherence self-reports validated in Singapore. Singapore is a multicultural society with three dominant ethnic groups – Chinese (74.2%), Malays (13.4%), and Indians (9.2%)24 and there are significant proportions of residents (aged ≥15 years) who are literate only in one language (English: 10.6%, Chinese: 15.6%, Malay: 1.52%).24 In order to apply the measure to patient populations in Singapore, it must be culturally relevant and translated to the appropriate languages. Assessing adherence to medications for diabetes in Singapore is important because the prevalence of diabetes is increasing. The number of people with diabetes in Singapore has increased from 2.0% of the population in 1975 to 12.7% in 201225 and is projected to rise to 22.7% in 2035.26 Therefore, we aim to validate a self-report measure of the extent of and reasons for medication nonadherence by Voils in patients with diabetes in Singapore.


Study design

We conducted a two-phase study. The first phase involved cognitive interviews to assess cultural adaptation of the translated measures. The second phase involved a prospective cohort study, whereby all participants received the questionnaire, and then 2 weeks later a subset of those participants received the same questionnaire again. The goal of the first measurement component was to assess internal consistency and construct validity, whereas the goal of the repeated measure component was to assess the reliability of the extent of nonadherence scores across a short time period. We recruited patients with diabetes from multiple sites consisting of both public hospitals and primary care centers in Singapore from July 2018 to January 2019. Participants could choose English, Chinese, or Malay version of the questionnaire, depending on their language preference. The National Healthcare Group Domain Specific Review Board (Ref no.: 2018/00472) approved this study. All participants provided written informed consent.


We approached eligible participants for recruitment while they were in the waiting areas of the pharmacies. We included participants who were at least 21 years old, with diabetes treated with antidiabetic medications and on regular follow-up at these hospitals or primary care centers. We excluded participants who were not willing to provide informed consent.

The self-report measure

It is a self-administered, two-domain questionnaire that assesses the extent of and reasons for nonadherence over the past 7 days,14 with a Likert-type additive scale of five responses measuring frequency: none of the time, a little of the time, some of the time, most of the time, and every time.22 There are three items in the extent of nonadherence, which are: “I missed my medicine”, “I skipped a dose of my medication”, and “I did not take a dose of my medication”. The overall score was calculated by the average score of the three items. Higher scores indicate greater levels of nonadherence.20 There are 18 items in the reasons for nonadherence which stand on their own as descriptors. The reasons scale is a causal indicator model14 and higher scores indicate greater endorsement of each reason for nonadherence.20

Translation of the self-report measure

We followed the guidelines for translation and adaptation of self-report measures by the ISPOR Task Force for Translation and Cultural Adaptation.27 Two researchers performed the forward translation of the English version of the self-report measure to Chinese and Malay. Another two independent researchers who were unaware of the original self-report measure performed the back translation. A panel of four multidisciplinary and bilingual clinical pharmacists with experiences with diabetic patients, which were different from the four researchers who translated the self-report measure, evaluated the three versions (the original, the forward translated, and the back translated). They ensured that the content, wording, and cognitive level of the Chinese and Malay versions were equivalent to the English version and that they had been appropriately adapted linguistically to Singaporeans.

Cognitive testing of the self-report measure

We conducted cognitive interviews with 30 consenting English-literate, Chinese-literate, and Malay-literate (ten participants per language) diabetic patients to evaluate the cultural relevance, comprehensiveness, scope, and acceptability of the self-report measure. We selected participants to represent a range of age, gender, and disease duration. The participants completed the questionnaire in the presence of a trained interviewer (YWL, CC, MHL) and the time of start and completion of the questionnaire were recorded. We answered any questions raised by the participants specific to the understanding of the questions by using a prescribed set of interpretation notes to validate the interpretation of the questions. We asked the participants about questions they had trouble answering and possible rephrasing of the questions. We also asked the participants on the ease of understanding, ambiguity, comprehensiveness, inappropriateness, and relevance of the questions for medication nonadherence.

Prospective cohort study

For the cross-sectional study, we asked participants who had provided written consent to complete a questionnaire, which consisted of demographic characteristics, clinical information, and patient-reported outcome measures (PROMs). Demographic characteristics included age, gender, ethnicity, and highest education level attained. We obtained clinical data from medical records, including disease duration and HbA1c in the past 3 months. PROMs included the self-report adherence measure being validated,20 Culig Adherence Scale (CAS),28 Medication Adherence Visual Analogue Scale (MAVAS),29 and EuroQoL-5 Dimensions-5 Levels (EQ-5D-5L).30

For the longitudinal component, we assessed medication adherence at two time points. We conducted face-to-face interviews at the first time point, which was at baseline (Day 0). We then performed telephone follow-up on Day 14, when the interviewer asked the same questions in the same questionnaire used at the face-to-face interviews for test–retest reliability. The purpose of the 14-day interval was to verify the consistency of adherence scores recorded at baseline.31 We chose this time period as it was recommended by the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) framework to be sufficiently long enough to minimize recall bias while sufficiently short enough to fulfill the assumption of no significant change in medication adherence.32

Comparison measures

CAS is also a two-part questionnaire that measures the extent of and reasons for nonadherence28 separately with an indefinite duration of nonadherence. There is one item in the extent of nonadherence that asks, “When was the last time when you failed to take your medication?”, with six responses: last week, 1–2 weeks ago, 3–4 weeks ago, 1–3 months ago, more than 3 months ago and I never fail to take my medication on time. Higher scores indicate lower levels of nonadherence. There are 16 items in the reasons for nonadherence with a Likert-type additive scale of four responses: never, very rare (1–2 times a year), sometimes (3–5 times a year), and often (more than 5 times a year). Higher scores indicate greater occurrence of each reason for nonadherence. We chose CAS as it also separates extent and reasons of nonadherence and so is a good comparison with the self-report measure.

MAVAS measures the extent of nonadherence29 over the past 6 months. It is a one-item measure that asks, “What percent of time over the past 6 months did you take your prescribed medication?”, with a continuous scale from 0% (not adherent) to 100% (fully adherent). Higher scores indicate greater levels of nonadherence. We chose MAVAS as ordinal scales have shown to have higher sensitivity and reproducibility compared to discrete scales.33 It also permits easier administration and we can see changes in the measurement of nonadherence between continuous and ordinal scales.

EQ-5D-5L (ranging from − 0.5 to 1.0) is a generic health index measuring health utility30 on the day of questionnaire administration. Health utility is computed from five dimensions of health: mobility, self-care, usual activities, pain/discomfort, and anxiety/depression. It includes a visual analog scale which records patients’ self-rated health status on a graduated (0–100) scale. Higher scores in both scales show higher health-related quality of life (HRQoL).34 We chose EQ-5D-5L as it is well validated with measurement invariance across the English, Chinese, and Malay versions in Singapore.35

Statistical analyses

We performed all statistical analyses using STATA SE 14.0 for Windows (StataCorp, College Station, TX, USA) and followed the COSMIN framework. We used the Shapiro–Wilk test to investigate the normality of the distribution of continuous variables. Due to normal distribution of the variables, we tabulated descriptive statistics as mean (SD) and categorical variables as n (%).

As the self-report measure contains two related but distinct nonadherence constructs, each construct must be assessed by a different type of psychometric model.20 The extent of nonadherence is represented by an effect indicator model, where the common underlying latent variable (in this case level of nonadherence) determines the item responses.36

Reliability is defined as consistency of the measure, which is the degree to which the measurement is free from measurement error.37 We assessed internal consistency, the degree of the interrelatedness among the items,37 using Cronbach’s alpha, where values of 0.6–0.7 show acceptable level of reliability,38 0.7–0.8 shows satisfactory level,39 and ≥0.8 shows excellent level,38 assuming all items were similar and measured a single construct. We assessed test–retest reliability via intraclass correlation coefficient (ICC) (two-way mixed effects model, single measure), where based on the 95% confidence interval, values of 0.50–0.75, 0.75–0.90, and >0.90 show moderate, good, and excellent reliability, respectively.40 We calculated ICC for each extent item and the extent average score with time as fixed parameter and patients as random variable,41 assuming the construct was stable over the two time points. We assessed measurement error, the random error of a patient’s score that was not attributed to true changes in the construct,37 by analyzing the smallest detectable change (SDC) based on the 95% limits of agreement using the formula: SDC=1.96×Standard Error of Measurement in mean difference in the extent score of the two assessments in the test–retest reliability sample×√2. It is sufficiently low when SDC<Minimal Important Change,32 which is the minimum change in average extent score that is important to patients.37

We tested construct validity, the degree to which the extent scores were consistent with the hypotheses,37 using four a priori hypotheses:

  • 1) The self-report measure is moderately negatively correlated with CAS and MAVAS28,29
  • 2) The self-report measure is weakly positively correlated with HbA1c42
  • 3) The self-report measure is weakly negatively correlated with EQ-5D-5L43
  • We assessed the correlation between the self-report measure and other PROMs via Spearman’s rank correlation. The linear relationships were considered negligible if the magnitude of the correlation coefficient is <=0.3, weak if >0.3 and ≤0.5, moderate if >0.5 and ≤0.7, high if >0.7 and ≤0.9, and very high if >0.9.44 The self-report measure should correlate with stronger magnitude with instruments that measure similar domains.32 At least 75% of the results should be in accordance with the hypotheses to demonstrate construct validity.32 To reduce the likelihood of Type I error, we considered p-values <0.0125 as statistically significant after applying Bonferroni’s correction.

    We assessed structural validity, the degree to which extent scores are an adequate reflection of the dimensionality of the construct,37 using confirmatory factor analysis to confirm the goodness of fit of the self-report measure in relation to the construct, using the maximum-likelihood method.45 We assessed the increment model fit by Comparative Fit Index (CFI) and Tucker-Lewis Index (TLI), values >0.95 were indicative as good model fit.46 We assessed the absolute model fit by Root Mean Square Error of Approximation (RMSEA), and Standardized Root Mean Residuals (SRMR), RMSEA <0.06 and SRMR <0.08 were indicative of good fit.46

    We assessed cross-cultural validity because the self report measure was used in different “cultural” population in the study,46 by examining measurement invariance, the statistical property of a measurement that indicates that the same underlying construct is being measured across groups or across time,47 of extent scores of the self-report measure across age, gender, and language. As the median age of patients in the study was 60, we dichotomized the patients into <60 years old and ≥60 years old for age group. We created three language groups based on language of the questionnaire done: 1) English (n=199); 2) Chinese (n=112); and 3) Malay (n=94). We then conducted a multi-group confirmatory factor analysis by constructing the following three increasingly restrictive models step-wise: where all parameters were free (configural invariance), where factor loadings were invariant (metric invariance), where factor loadings and item intercepts were invariant (scalar invariance). For adequate sample size (total N>300) and sample sizes similar across groups, measurement invariance is seen if ∆CFI and ∆TLI ≤0.01, ∆RMSEA ≤0.15, ∆SRMR ≤0.03 (for metric invariance), and ∆SRMR ≤0.01 (for scalar invariance).48

    Reasons for nonadherence are represented by a causal indicator model, where each reason for nonadherence stands alone as a descriptive indicator because they may not be correlated.36 We considered any participants who scored ≥2 on any extent item as nonadherent, as seen in other validations of this scale and other investigations of the longitudinal predictive validity of this scale.22 We recorded descriptive statistics for the reasons items. We did not calculate Cronbach’s alpha because it is inappropriate for causal indicators.49


    Phase I: cognitive testing of the self-report measure

    We conducted cognitive interviews with 30 consenting participants with diabetes (age range 48–76 years, 53% male, disease duration range 1–30 years). The participants took an average of 4 minutes to complete the self-report measure. In the English version, seven participants pointed out that the phrasing of the extent of nonadherence questions was vague and five participants could not understand the difference between Item 2 (I skipped a dose of my medicine) and Item 3 (I did not take a dose of my medicine). In the Chinese version, two participants felt that the instructions needed clarification, as “please think about your pills only (not interferon)” was inadvertently left (based on the version validated in patients with hepatitis C infection). Three participants felt that inclusion of the stem “I missed my medication because … ” for the reasons for nonadherence items was unnecessary, so it was dropped. No participants identified problems with the wording of the extent of nonadherence items or response scale. For the Malay version, no issues were identified with the instructions, items, or response scales. The revised Chinese and Malay versions were tested in a second round of cognitive interviews.

    Phase II: prospective cohort study

    For the cross-sectional study, we recruited 393 participants who are available for analysis as shown in Table 1. The participants were equally distributed between the two genders (50.9% female). The mean age of the participants was 59.4±12.2 years. Most were Chinese (52.4%), married (72.6%) and spoke English (77.6%), and less than half completed secondary education (35.1%). The mean HbA1c obtained from the medical records was 8.3±1.6%.

    Table 1 Sociodemographic and clinical characteristics of patients

    Extent of nonadherence


    Figure 1 shows the score distribution of the average score of the extent of nonadherence in the self-report measure from the 393 participants. The extent score had a median of 1.3, with a range of 1–4. Most patients were adherent to their diabetic medications the past 7 days, as shown by the large proportion of patients scoring 1 (61.1%). Due to the small number of nonadherent patients, we dichotomized the scores to adherence (score of 1) and nonadherence (scores above 1). Table S1 shows descriptive statistics for the extent questions for all languages, English, Chinese, and Malay version.

    Figure 1 The scores of the extent of nonadherence. Notes: The scores are an average of the three extent items and ranged from 1 to 5. A score of 1 represents patients with complete adherence while a score of 5 represents patients with the poorest adherence over the past 7 days.


    The Cronbach’s alpha for all languages was 0.67 while the Cronbach’s alphas for English, Chinese, and Malay version were 0.65, 0.70, 0.73, respectively, demonstrating acceptable internal consistency. For the longitudinal component, we assessed 118 participants for test–retest reliability, which was moderate as ICCs of the individual extent items were 0.52, 0.54, and 0.52 for Item 1, 2, and 3, respectively, and the average score had an ICC of 0.56 (95% CI 0.37–0.70). We calculated the SDC as 0.80.

    Construct validity

    We fulfilled all four hypotheses about the magnitude and direction of correlation between the self-report measure and CAS, MAVAS, EQ-5D-5L, and HbA1c (Table 2) and all were statistically significant with p-values <0.0125, supporting construct validity. As expected, we observed stronger correlations with PROMs that measure adherence (ie, CAS and MAVAS) than those that measure other constructs, providing evidence of convergent and discriminant validity, respectively.

    Table 2 Construct validity of the self-report measure (n=393)

    Structural validity

    The one-factor model proposed for the confirmatory factor analysis of the self-report measure displayed the following fit indices (CFI=1.0; TLI=1.0; RMSEA<0.001; SRMR<0.001), which showed that the model yielded a good fit to the data, thus supporting structural validity.

    Cross-cultural validity

    The one-factor model proposed yielded great fit for age group, gender and language (CFI=1.000; TLI=1.000; RMSEA<0.001; SRMR<0.001 for all three). This showed that the single-factor structure of the self-report measure was equivalent across age group, gender, and language. However, the metric and scalar invariance model did not fit well according to descriptive fit indexes. ∆CFI, ∆TLI, ∆RMSEA, and ∆SRMR all did not meet the cutoff threshold (Table S2).

    Reasons for nonadherence

    Table 3 provides descriptive statistics for each reason of nonadherence across the 131 participants that reported nonadherence. Most participants reported forgetting as the most common reason (n=112, 73.2%), followed by being out of routine (n=92, 60.1%), being unable to meet the food requirements (n=76, 49.7%), not having medication with them (n=73, 47.7%), and being too late with their dose (n=73, 47.7%). The remaining reasons were on 13.1–36.6% of nonadherence occasions. On five nonadherence occasions (3.3%), the participants did not endorse any reasons for nonadherence despite them reporting nonadherence on the extent scale.

    Table 3 Endorsement of reasons for nonadherence


    This study provides support for adequate face validity, internal consistency, test–retest reliability, construct validity, and structural validity of the self-report adherence measure in Singapore. To our knowledge, this is the first psychometric validation of the self-report measure in measuring the extent of and reasons for medication nonadherence in diabetic patients in a multilingual Asian city. This study is also the first to show the cross-cultural validity of a medication nonadherence questionnaire in Asia. Having a large sample size, following the COSMIN framework rigorously and formulating hypotheses a priori for construct validity were the key strengths of this study.

    From the cognitive interviews, we made changes to the extent items of the English version, which were the addition of “by accident” to Item 1 and “on purpose” to Item 2, as participants reported that they could not understand the difference between both items. The multi-lingual Singaporeans that completed the self-report measure in this study as compared to English-speaking only Americans that Voils herself developed the original version from,20 might account for differences in understanding of the English items, that compelled us to clarify the wording. The original version that Voils developed was done through several rounds of cognitive interviews with monolingual Americans20 and thus, that version is still recommended to be used for Americans due to the extensive work and evidence of reliability and validity. In other multilingual countries, re-validation of the self-report measure is necessary for proper linguistic adaptation, to ensure the measure remains culturally relevant to the respective context. This is further supported with the higher Cronbach’s alpha in the Chinese and Malay versions as compared to the English version, which shows that the modification of the self-report measure was more appropriate for the Chinese and Malay versions than the English version.

    For the Chinese version, the stem “I missed my medication because …” for the reasons for nonadherence items was dropped as the participants voiced out that it was unnecessary. Although the stem was included in the original version as Voils found in her cognitive interviews that patients started to respond about the reasons for nonadherence items in general rather than as reasons for missing a dose in the past 7 days,20 the three participants during our cognitive interviews mentioned that they understood what was asked without needing the stem to be repeated for every reason.

    Our reported adherence rate (61%) is slightly higher than that observed in patients taking blood pressure medications (40%)20 and statins (57%).22 This may reflect that the actual level of adherence to medications in diabetic patients is higher than in other populations. Alternatively, this may reflect that our patient population was more likely to overestimate adherence due to social desirability bias, where patients give responses that followed the advice of their health care providers,50 non-response bias, where nonadherent patients were less likely to participate in the study,51 and recall bias.52 These three types of bias are well-known limitations of self-report measures.

    The test–retest reliability and structural validity of the extent of nonadherence were comparable to a study done by Voils et al.20 However, the internal consistency was found to be lower (alpha =0.65, 0.70, and 0.73 for English, Chinese and Malay, respectively) compared to Voils’ study on hypertensive patients (alpha=0.84)20 and 2014 (alpha=0.78–0.94 across four occasions),21 and Blalock’s 2019 study on dyslipidemia patients (alpha =0.90).22 For the English version, this may be due to the changes made to the extent items to clarify the meaning to participants. Because people may respond in one way to “by accident” and another way to “on purpose,” as there are two different aspects of medication nonadherence measured: unintentional (Item 1) and intentional (Item 2),53 the inter-item correlations will be attenuated.54 As Cronbach’s alpha measures both the homogeneity of the items and the homogeneity of what is being assessed (medication nonadherence),55 internal consistency may decrease with increasing clarity of the extent questions as shown by the lower Cronbach’s alpha. This suggests a need to refine the items further in future research. The alphas for the Chinese and Malay versions were acceptable, even if somewhat lower than the original version. Cultural adaptations of measures do not always yield identical psychometric properties as the original measure.

    Construct validity was supported in hypotheses testing, as there was fulfillment of all four a priori hypotheses in both magnitude and direction. We found a small, positive correlation between the self-report measure and HbA1c; this was expected given that adherence and glycemic control are different constructs, represent different time periods (7 days versus 3 months), and that variables other than adherence contribute to glycemic control (eg, dietary intake, genetics).42 We also found a small, negative correlation between the self-report measure and EQ-5D-5L, thus reflecting the fact that adherence and HRQoL are different constructs, both represent different time periods too (7 days versus day of administration), and that health status does not depend on adherence alone.56

    Analysis of cross-cultural validity supported the configural invariance model but not the metric invariance and scalar invariance models in the extent of nonadherence. Clinicians can still use the self-report measure in patients with varying gender, age groups, and languages, but caution must be taken against directly comparing the extent scores across these categories. Any direct comparison of the extent scores across languages should consider the fact that any observed differences could be due to measurement equivalence/invariance.

    The reasons of nonadherence in the self-report measure were similar to a qualitative study by Shiyanbola et al, where they reported diabetic patients were non-compliant to antidiabetics due to forgetting, concerns about side effects, doubts about effectiveness, medications not physically available to them, caregivers unavailable to facilitate use of medicines, and lack of knowledge on how to ask providers questions regarding medications.57 However, some reasons reported by Shiyanbola et al, such as fear of taking medicines, frustration, and tiredness of taking for long time and denial of having diabetes as the disease did not show any symptoms to patients,57 were not reflected in the self-report measure. The reasons of nonadherence were also similar to a cross-sectional study in American veterans by Weidenbacher et al, which included “I forgot”, “I ran out of medication”, “I had other medications to take”, “The medication caused side effects”, “I had other medications to take”, “I was feeling too sick to take it”.23 However, other reasons reported by Weidenbacher et al, such as reasons under the categories of “Negative expectations or worry” and “Does not take condition seriously”,23 were not shown in the self-report measure. The five participants who did not indicate any reasons for nonadherence despite reporting nonadherence on the extent scale might have the other reasons as stated by Shiyanbola et al57 and Weidenbacher et al,23 and thus further studies can be done to improve the comprehensiveness of this self-report measure. These five participants may not have indicated any reasons for nonadherence because they had no specific reason. They might have known that they did not take their medications but did not know why they did not do so.

    There are some limitations to this study. First, the convenience sampling procedure may result in selection bias as nonadherent patients were less likely to visit their physicians58 and therefore less likely to participate in the study.51 However, the aim of this study was to assess the psychometric properties of the self-report measure, and the relationship between variables should hold even if the mean level of adherence was inflated, provided there was sufficient variability. Therefore, the impact of having a convenience sample will be minimal in influencing the study results. Second, we did not estimate Minimal Important Change due to the difficulty in finding a proven intervention that improves medication adherence. Thus, we only reported SDC, in line with recommendations from COSMIN.37 Third, we did not assess criterion validity using an objective measure such as electronic drug monitoring or pill count,59 and thus future studies can consider validating the self-report measure against more objective measures. Fourth, we did not specify if the participants took oral hypoglycemic agents or insulin because we wanted to validate the self-report measure for use across a wider diabetic population. Insulin adherence is complicated due to feasibility challenges associated with prescription claims data and injectable medication days’ supply,60 and no quality measure is currently associated with adherence to insulin treatment.60 Further studies can validate this self-report measure in measuring insulin adherence specifically. Fifth, there were five participants who did not indicate any reasons for nonadherence despite reporting nonadherence on the extent scale. This was due to us not including an open-ended option for participants to report any reasons that were not included in the list. Though upon asking the 30 participants whether there were any other reasons to add during the cognitive interviews, they did not mention any. This sample size is sufficient enough according to COSMIN guidelines32 and we have sampled a wide breadth of patients through purposive sampling. Future studies can include additional in-depth interviews or focus group discussions to elucidate many reasons for nonadherence from a larger sample of Asians. Last, we did not mention intentional and unintentional nonadherence because we wanted to cover the reasons for nonadherence as representatively in our target population as possible for the self-report measure to have content validity. Future studies can use the self-report measure to examine what proportion of medication nonadherence is intentional versus unintentional in our target population.


    In conclusion, this study supports the face validity, internal consistency, test–retest reliability, construct validity and structural validity of the self-report measure to measure the extent of and reasons for medication nonadherence separately in diabetic patients in Singapore. This provides researchers and clinicians greater confidence to use the self-report measure to measure the extent of nonadherence in diabetic patients. Future interventions can be developed specifically for different reasons of nonadherence, and effectiveness of such interventions can be assessed with the self-report measure.


    Full version of the self-report measure will be available on The National Healthcare Group Pharmacy Transformation Office (NHGPTO) provided a non-restricted grant for this project. All patients provided written informed consent to participate in this research.


    Dr Voils was supported by a Research Career Scientist award from the Department of Veterans Affairs Health Services Research & Development Service (RCS 14-443). The authors report no other conflicts of interest in this work.


    1. Thomas CC, Philipson LH. Update on diabetes classification. Med Clin North Am. 2015;99(1):1–16. doi:10.1016/j.mcna.2014.08.015 

    2. Guariguata L, Whiting DR, Hambleton I, Beagley J, Linnenkamp U, Shaw JE. Global estimates of diabetes prevalence for 2013 and projections for 2035. Diabetes Res Clin Pract. 2014;103(2):137–149. doi:10.1016/j.diabres.2013.11.002

    3. Balkrishnan R. The importance of medication adherence in improving chronic-disease related outcomes: what we know and what we need to further know. Med Care. 2005;43(6):517–520.

    4. Teo V, Toh MR, Kwan YH, Raaj S, Tan SY, Tan JZ. Association between total daily doses with duration of hospitalization among readmitted patients in a multi-ethnic Asian population. Saudi Pharm J. 2015;23(4):388–396. doi:10.1016/j.jsps.2015.01.013

    5. Toh MR, Teo V, Kwan YH, Raaj S, Tan SY, Tan JZ. Association between number of doses per day, number of medications and patient’s non-compliance, and frequency of readmissions in a multi-ethnic Asian population. Preventive Med Reports. 2014;1:43–47. doi:10.1016/j.pmedr.2014.10.001

    6. Lau DT, Nau DP. Oral antihyperglycemic medication nonadherence and subsequent hospitalization among individuals with type 2 diabetes. Diabetes Care. 2004;27(9):2149–2153. doi:10.2337/diacare.27.9.2149

    7. Ho PM, Rumsfeld JS, Masoudi FA, et al. Effect of medication nonadherence on hospitalization and mortality among patients with diabetes mellitus. Arch Intern Med. 2006;166(17):1836–1841. doi:10.1001/archinte.166.17.1836

    8. Lerman I. Adherence to treatment: the key for avoiding long-term complications of diabetes. Arch Med Res. 2005;36(3):300–306. doi:10.1016/j.arcmed.2004.12.001

    9. DiMatteo MR. Variations in patients’ adherence to medical recommendations: a quantitative review of 50 years of research. Med Care. 2004;42(3):200–209.

    10. Lin LK, Sun Y, Heng BH, Chew DEK, Chong PN. Medication adherence and glycemic control among newly diagnosed diabetes patients. BMJ Open Diabetes Res Care. 2017;5(1):e000429. doi:10.1136/bmjdrc-2017-000429

    11. Haynes RB, Ackloo E, Sahota N, McDonald HP, Yao X. Interventions for enhancing medication adherence. Cochrane Database Syst Rev. 2014;20(11):CD000011. doi:10.1002/14651858.CD000011.pub4

    12. Vermeire E, Hearnshaw H, Van Royen P, Denekens J. Patient adherence to treatment: three decades of research. A comprehensive review. J Clin Pharm Ther. 2001;26(5):331–342.

    13. Osterberg L, Blaschke T. Adherence to medication. N Engl J Med. 2005;353(5):487–497. doi:10.1056/NEJMra050100

    14. Voils CI, Hoyle RH, Thorpe CT, Maciejewski ML, Yancy WS Jr. Improving the measurement of self-reported medication nonadherence. J Clin Epidemiol. 2011;64(3):250–254. doi:10.1016/j.jclinepi.2010.07.014

    15. Odegard PS, Capoccia K. Medication taking and diabetes. Diabetes Educ. 2007;33(6):1014–1029. doi:10.1177/0145721707308407

    16. Moon SJ, Lee WY, Hwang JS, Hong YP, Morisky DE. Accuracy of a screening tool for medication adherence: a systematic review and meta-analysis of the morisky medication adherence scale-8. PLoS One. 2017;12(11):e0187139. doi:10.1371/journal.pone.0187139

    17. Thompson K, Kulkarni J, Sergejew AA. Reliability and validity of a new Medication Adherence Rating Scale (MARS) for the psychoses. Schizophr Res. 2000;42(3):241–247.

    18. Kim MT, Hill MN, Bone LR, Levine DM. Development and testing of the hill-bone compliance to high blood pressure therapy scale. Prog Cardiovasc Nurs. 2000;15(3):90–96.

    19. Koschack J, Marx G, Schnakenberg J, Kochen MM, Himmel W. Comparison of two self-rating instruments for medication adherence assessment in hypertension revealed insufficient psychometric properties. J Clin Epidemiol. 2010;63(3):299–306. doi:10.1016/j.jclinepi.2009.06.011

    20. Voils CI, Maciejewski ML, Hoyle RH, et al. Initial validation of a self-report measure of the extent of and reasons for medication nonadherence. Med Care. 2012;50(12):1013–1019. doi:10.1097/MLR.0b013e318269e121

    21. Voils CI, King HA, Neelon B, et al. Characterizing weekly self-reported antihypertensive medication nonadherence across repeated occasions. Patient Prefer Adherence. 2014;8:643–650. doi:10.2147/PPA.S60715

    22. Blalock DV, Zullig LL, Bosworth HB, Taylor SS, Voils CI. Self-reported medication nonadherence predicts cholesterol levels over time. J Psychosom Res. 2019. doi:10.1016/j.jpsychores.2019.01.010

    23. Weidenbacher HJ, Beadles CA, Maciejewski ML, Reeve BB, Voils CI. Extent and reasons for nonadherence to antihypertensive, cholesterol, and diabetes medications: the association with depressive symptom burden in a sample of American veterans. Patient Prefer Adherence. 2015;9:327–336. doi:10.2147/PPA.S74531

    24. Wong WK. Census of Population 2010. Statistical Release 1: Demographic Characteristics, Education, Language and Religion. Singapore:: Singapore Department of Statistics; 2011.

    25. Lee WR. The changing demography of diabetes mellitus in Singapore. Diabetes Res Clin Pract. 2000;50(Suppl 2):S35–S39.

    26. Wong LY, Toh MP, Tham LW. Projection of prediabetes and diabetes population size in Singapore using a dynamic Markov model. J Diabetes. 2017;9(1):65–75. doi:10.1111/1753-0407.12384

    27. Wild D, Grove A, Martin M, et al. Principles of good practice for the translation and cultural adaptation Process for Patient-Reported Outcomes (PRO) measures: report of the ISPOR task force for translation and cultural adaptation. Value Health. 2005;8(2):94–104. doi:10.1111/j.1524-4733.2005.04054.x

    28. Culig J, Leppee M, Boskovic J, Eric M. Determining the difference in medication compliance between the general patient population and patients receiving antihypertensive therapy: a case study. Arch Pharm Res. 2011;34(7):1143–1152. doi:10.1007/s12272-011-0712-0

    29. Nau DP, Steinke DT, Williams LK, et al. Adherence analysis using visual analog scale versus claims-based estimation. Ann Pharmacother. 2007;41(11):1792–1797. doi:10.1345/aph.1K264

    30. Wang P, Luo N, Tai ES, Thumboo J. The EQ-5D-5L is more discriminative than the EQ-5D-3L in patients with diabetes in Singapore. Value Health Reg Issues. 2016;9:57–62. doi:10.1016/j.vhri.2015.11.003

    31. Leung AA, Daskalopoulou SS, Dasgupta K, et al. Hypertension Canada’s 2017 guidelines for diagnosis, risk assessment, prevention, and treatment of hypertension in adults. Can J Cardiol. 2017;33(5):557–576. doi:10.1016/j.cjca.2017.03.005

    32. Prinsen CA, Mokkink LB, Bouter LM, et al. COSMIN guideline for systematic reviews of patient-reported outcome measures. Qual Life Res. 2018;27(5):1147–1157. doi:10.1007/s11136-018-1798-3

    33. Grant S, Aitchison T, Henderson E, et al. A comparison of the reproducibility and the sensitivity to change of visual analogue scales, Borg scales, and Likert scales in normal subjects during submaximal exercise. Chest. 1999;116(5):1208–1217. doi:10.1378/chest.116.5.1208

    34. Herdman M, Gudex C, Lloyd A, et al. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Quality Life Res. 2011;20(10):1727–1736. doi:10.1007/s11136-011-9903-x

    35. Wang Y, Tan N-C, Tay E-G, Thumboo J, Luo N. Cross-cultural measurement equivalence of the 5-level EQ-5D (EQ-5D-5L) in patients with type 2 diabetes mellitus in Singapore. Health Qual Life Outcomes. 2015;13:103. doi:10.1186/s12955-015-0297-2

    36. Strube MJ. Effect indicator versus causal indicator measurement. Encycl Clin Psychol. 2014;1–5.

    37. Mokkink LB, Terwee CB, Patrick DL, et al. COSMIN Checklist Manual. Amsterdam: University Medical Center. 2012.

    38. Ursachi G, Horodnic IA, Zait A. How reliable are measurement scales? External factors with indirect influence on reliability estimators. Procedia Econ Finance. 2015;20:679–686. doi:10.1016/S2212-5671(15)00123-9

    39. Bland JM, Altman DG. Cronbach’s alpha. BMJ. 1997;314(7080):572. doi:10.1136/bmj.314.7080.572

    40. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155–163. doi:10.1016/j.jcm.2016.02.012

    41. McGraw KO, Wong SP. Forming inferences about some intraclass correlation coefficients. Psychol Methods. 1996;1(1):30. doi:10.1037/1082-989X.1.1.30

    42. Cohen HW, Shmukler C, Ullman R, Rivera CM, Walker EA. Measurements of medication adherence in diabetic patients with poorly controlled HbA(1c). Diabet Med. 2010;27(2):210–216. doi:10.1111/j.1464-5491.2009.02898.x

    43. Kastien-Hilka T, Rosenkranz B, Schwenkglenks M, Bennett BM, Sinanovic E. Association between health-related quality of life and medication adherence in pulmonary tuberculosis in South Africa. Front Pharmacol. 2017;8:919. doi:10.3389/fphar.2017.00919

    44. Mukaka MM. Statistics corner: A guide to appropriate use of correlation coefficient in medical research. Malawi Med J. 2012;24(3):69–71.

    45. Marsh HW, Balla JR, McDonald RP. Goodness-of-fit indexes in confirmatory factor analysis: the effect of sample size. Psychol Bull. 1988;103(3):391. doi:10.1037/0033-2909.103.3.391

    46. Mokkink LB, Prinsen C, Patrick DL, et al. COSMIN Methodology for Systematic Reviews of Patient-reported Outcome Measures (proms). User manual. 2018;78:1.

    47. Bialosiewicz S, Murphy K, Berry T. An introduction to measurement invariance testing: resource packet for participants. Am Eval Assoc. 2013;1–37.

    48. Chen FF. Sensitivity of goodness of fit indexes to lack of measurement invariance. Struct Equation Model. 2007;14(3):464–504. doi:10.1080/10705510701301834

    49. Bollen K, Lennox R. Conventional wisdom on measurement: a structural equation perspective. Psychol Bull. 1991;110(2):305. doi:10.1037/0033-2909.110.2.305

    50. Van de Mortel TF. Faking it: social desirability response bias in self-report research. Aust J Adv Nurs. 2008;25(4):40.

    51. Wang Y, Lee J, Toh MP, Tang WE, Ko Y. Validity and reliability of a self-reported measure of medication adherence in patients with Type 2 diabetes mellitus in Singapore. Diabet Med. 2012;29(9):e338–e344. doi:10.1111/j.1464-5491.2012.03733.x

    52. Tarrant MA, Manfredo MJ. Digit preference, recall bias, and nonresponse bias in self reports of angling participation. Leisure Sci. 1993;15(3):231–238. doi:10.1080/01490409309513202

    53. Fan JH, Lyons SA, Goodman MS, Blanchard MS, Kaphingst KA. Relationship between health literacy and unintentional and intentional medication nonadherence in medically underserved patients with type 2 diabetes. Diabetes Educ. 2016;42(2):199–208. doi:10.1177/0145721715624969

    54. Wroe AL. Intentional and unintentional nonadherence: a study of decision making. J Behav Med. 2002;25(4):355–372.

    55. Streiner DL. Starting at the beginning: an introduction to coefficient alpha and internal consistency. J Pers Assess. 2003;80(1):99–103. doi:10.1207/S15327752JPA8001_18

    56. Nazir R, Ur S, Azmi Hassali M, Saleem F, Bashir S, Aljadhey H. Does adherence to the therapeutic regimen associate with health related quality of life: findings from an observational study of type 2 diabetes mellitus patients in Pakistan. Pak J Pharm Sci. 2017;30(6):2159–2165.

    57. Shiyanbola OO, Brown CM, Ward EC. “I did not want to take that medicine”: African-Americans’ reasons for diabetes medication nonadherence and perceived solutions for enhancing adherence. Patient Prefer Adherence. 2018;12:409–421. doi:10.2147/PPA.S152146

    58. McNary A. Avoiding risk with nonadherent patients. Innov Clin Neurosci. 2015;12(3–4):37–40.

    59. Shi L, Liu J, Fonseca V, Walker P, Kalsekar A, Pawaskar M. Correlation between adherence rates measured by MEMS and self-reported questionnaires: a meta-analysis. Health Qual Life Outcomes. 2010;8:99. doi:10.1186/1477-7525-8-99

    60. Stolpe S, Kroes MA, Webb N, Wisniewski T. A systematic review of insulin adherence measures in patients with diabetes. J Manag Care Spec Pharm. 2016;22(11):1224–1246. doi:10.18553/jmcp.2016.22.11.1224

    Supplementary materials

    Table S1 Proportion of extent of nonadherence

    Table S2 Cross-cultural validity of the self-report measure (n=393)

    Creative Commons License This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.