Validating a Self-Reported Medication Nonadherence Measure in the Context of Multiple Chronic Diseases and Routes of Medication Administration Among Patients with Type 2 Diabetes

Introduction

Assessing medication nonadherence among patients with type 2 diabetes is a common clinical challenge. Having a simple way to accurately identify nonadherence is essential to safe and high-quality prescribing. If nonadherence is not correctly identified, providers may over- or under-prescribe medication, leading to toxicity or decreased therapeutic benefit, respectively. When clinicians rely on their clinical judgment alone, their nonadherence estimates are no better than a coin toss.^1–3 Despite the limitations of relying on clinical judgment, patients are not systematically asked about medication nonadherence in clinical settings. Although several tools have been developed and used to assess medication nonadherence,^4–6 they are not organized consistent with best practices for psychometric scaling, such as conflating the extent of nonadherence with reasons for nonadherence, leading to inappropriate assessment of reliability and validity.⁷

To meet the need for a reliable, valid medication nonadherence questionnaire, we previously created a two-domain measure which separates the measurement of “extent of” and “reasons for” nonadherence. This measure involves a three-item scale for extent of nonadherence (ie, missed doses), with additional (optional) items comprising a reasons for nonadherence index administered only to patients who self-report nonadherence. Our team then conducted two studies to refine and assess the validity of this measure. The first study was conducted among patients prescribed medications for hypertension and resulted in Version 1 of the measure.⁸ The second study was conducted in patients prescribed oral medications for hepatitis C to determine if the measure worked for assessing nonadherence to medications with a strict dosing regimen and to add relevant reasons for nonadherence in this context.⁹ The result, Version 2, has since been translated and implemented in multiple languages (Turkish,¹⁰ French, Spanish, Singaporean Malay and Singaporean Chinese,¹¹ Azeri Turkish, and Arabic; all available from the copyright holder, Duke University).

Although several studies have provided evidence for the reliability and validity of both versions of the measure across several populations prescribed oral medications, the measure’s performance for injectable diabetes medications is unclear. Dosing frequencies can range from daily to weekly or less often, and there may be unique reasons for nonadherence associated with injectable medications. Furthermore, many patients with type 2 diabetes are prescribed medications for multiple chronic diseases. The measure has been used to assess nonadherence to medications for a single condition. Unknown is whether patients with type 2 diabetes can accurately report nonadherence for comorbid chronic conditions such as hypertension and dyslipidemia through a single set of questions or whether accurate measurement requires separate questions for each condition.

To address these two implementation questions, we conducted two cognitive interview studies among patients with type 2 diabetes. Cognitive interviewing is a qualitative method applied to scale development and refinement.¹² The goal is to assess how respondents comprehend instructions, items, and response scales and how they generate their responses. We conducted the two studies concurrently, making relevant changes (eg, clarifying instructions) to both studies simultaneously.

Study 1

Design, Setting, and Participants

Individuals were eligible for cognitive interviews if they were 18 or older, diagnosed with Type 2 diabetes, and prescribed at least one injectable medication [fixed-dose insulin or glucagon-like peptide receptor agonist 1 (GLP-1)] and/or one oral medication for diabetes. Patients needed to have a hemoglobin A1c (HbA1c) of 8% or higher in the past 6 months, indicating uncontrolled blood glucose. This inclusion criterion was selected so that the participants were representative of those who would most benefit from a nonadherence assessment prior to adjusting the medication regimen. We conducted the interviews by video via WebEx or Zoom, and participants were required to have access to an internet-connected computer. We excluded patients on sliding scale insulin as their only diabetes medication due to the lack of a uniform dosing schedule. We also excluded patients who did not speak English, were residing in a nursing home or receiving home healthcare to administer medications, or who needed assistance from others to take their medications. Patients were seen at the UW Health Endocrinology, Diabetes, and Metabolism Clinics or on the inpatient wards by the UW Health Diabetes Management Service. The study was reviewed by the University of Wisconsin Health Sciences Institutional Review Board (IRB; Study ID: 2020–0215) and determined to meet the criteria for exempt human subjects research according to 45 CFR 46 Category 2. The study complies with the Declaration of Helsinki.

We identified potential participants by obtaining names and contact information of patients meeting eligibility criteria through an electronic medical record data pull. These patients received a recruitment letter with instructions to opt out. The study endocrinologist also identified patients through chart review of individuals with type 2 diabetes seen at the endocrinology clinic. We called patients to assess eligibility and schedule an interview.

Procedures

Prior to conducting interviews, the investigative team generated additional reasons for nonadherence unique to injectable medications based on clinical knowledge and existing literature.^13–15 These items were added to the existing Reasons for Nonadherence index, resulting in 28 unique reasons for nonadherence (Figure 1).

Figure 1 Final two-part measure of extent of, and reasons for, nonadherence to injectable diabetes medications. Notes: The measure is copyrighted by Duke University. For permission to use it, please contact Duke University’s Office of Translation and Commercialization.

Interviewers trained in cognitive interviewing led each session accompanied by a note-taker. At the beginning of the video call, the interviewer obtained verbal consent. The study met criteria for waiver of documentation of written informed consent as determined by the IRB. Participants were directed to a survey link in the chat box to complete on their own. They were instructed to skip items that they did not know how to answer. The survey link brought up one of two versions of the survey, the order randomly determined: 1) separate versions for injectable medicines and pills, and 2) a combined version in which participants were instructed, “When responding, please think about both your diabetes pills and injectable medicines.” Participants completed both versions of the measure with interviewer probes after each assessment. Each interview lasted up to 60 minutes.

Utilizing retrospective probing, participants completed the measure on their own, then were asked a set of questions by the interviewer about how they interpreted the instructions and items; the difficulty of recalling missed doses and reasons for missing doses over the last 7 days; how they constructed their responses; and completeness and repetition (in the case of separate versions) of their reasons for nonadherence. When participants completed both the separate and combined versions, we asked them to reflect on which version they preferred and which would provide the most accurate information. We conducted four rounds of interviews with different participants, making changes after each round to address them, and then evaluating them. Interviews were recorded and notes were taken.

At the end of each interview, participants clicked a link to a self-report survey to assess ethnicity, race, sex assigned at birth, gender, education level, working status, income level, marital status, insurance status, and numeracy. Then, the participants clicked a link to the Short-Test of Functional Health Literacy in Adults (S-TOFHLA). Based on their total scores, patients’ literacy can be classified as inadequate (0–16 points), marginal (17–22), or adequate (23–36).

Data collection and analysis occurred iteratively. After each interview, the note-taker and interviewer met to discuss and create interim summaries and for discussion among the investigative team. The team changed the survey and interview guide, which were subsequently tested in other participants. We did not have an a priori sample size target as we used our judgment and experience to determine when to cease data collection once saturation was achieved (ie, no new reasons emerged).

Results

We contacted 98 patients, of whom 44 were unable to be reached, 35 declined to participate, 4 were ineligible, and 1 was unable to scheduled. Fourteen patients with diabetes participated across all interviews. Respondents were 56.2 (SD: 9.3) years old on average, 39% were assigned male at birth, and all were Non-Hispanic, White persons (Table 1). The average S-TOFHLA score was 35.1 (range, 33–36), indicating adequate functional health literacy. Ten participants were prescribed both injectable and oral medications, one was prescribed only an oral medication (who responded to the injectable version hypothetically), and three were prescribed only injectable medications. The results of cognitive interviews are summarized in Table 2 and discussed below.

Table 1 Participant Characteristics

Table 2 Results of Cognitive Interviews for Study 1 (Oral and Injectable Diabetes Medications) and Study 2 (Diabetes, Blood Pressure, and Cholesterol Medications)

Instructions and Recall Period

In Round 1, participants were unsure about how to respond to the extent of nonadherence items if they took one or more doses “late.” We tested three different modifications across Rounds 2–4 to address this issue. The final revised instruction read, “If you took the medicine later than usual but still took it, it does not count as a missed dose.”

All participants indicated that they could remember missed doses and reasons for missing doses of daily medications over the past 7 days. A few participants indicated that they would be able to remember missed doses up to 2 or 3 weeks. These individuals described habitualizing their routine (eg, taking medications for several diseases together at a particular time of day) and either did not miss medications or always missed them for the same reasons. Other participants did not believe they could accurately remember missed doses for longer than 7 days.

Across the first three rounds, a few participants reported taking weekly injectable medications. They reported that they could remember missed doses for 4–6 weeks. We recognized that a 7-day recall period for weekly medications did not match the graded frequency response scale, which assumes the behavior occurs more than once in the recall period. To assess missed doses of a weekly medication, we considered two options: 1) keeping the 7-day recall period and creating a binary response scale or 2) extending the recall period and keeping the graded response scale. Because participants reported they could recall missed doses of weekly injectable medications over several weeks, and because we desired to keep the same response scale for ease of implementation, we purposefully recruited additional participants in Round 4 who were prescribed weekly injectable medications and queried about their ability to recall missed doses “in the past month.” All indicated they could respond accurately regarding weekly injectable medications across 1 month.

In Round 2, a few participants from both studies voiced that 7 days does not represent what they do over longer time periods, making them want to respond about their more typical pattern despite the 7-day instruction. Therefore, we added an instruction, “We recognize that the past 7 days may not represent what you do over longer time periods. However, we are only interested in the past 7 days.” Subsequent participants indicated this instruction was clear and that they only responded about the past 7 days.

Extent of Nonadherence Items

Universally, participants indicated that the extent of nonadherence items was appropriate for oral and injectable medications alike; the concept of “dose” fit both modes of administration. However, in Round 1, participants’ item responses highlighted the need to query about oral and injectable medications separately instead of together because the different timing and dosing instructions sometimes resulted in missed doses for one route of administration but not necessarily the other. Additionally, participants indicated that it was easier to respond, and thought they could provide more accurate responses with separate oral and injectable versions. Therefore, we eliminated the combined version in Rounds 2–4.

Many participants perceived a difference in intent among the three extent of nonadherence items, such that the first item (I missed my medicine) asked about unintentional nonadherence and the second item (I skipped a dose of my medicine) asked about intentional nonadherence. The third item (I did not take a dose of my medicine) was perceived as unintentional by some and as intentional by others. Due to these perceived differences, participants sometimes responded to the items differently (eg, response of none of the time to item 1 but a little of the time to item 2).

Reasons for Nonadherence

The initial items pertinent to oral medications were viewed as relevant to people with type 2 diabetes, even if they did not personally experience those reasons for missing doses. The new items relevant to injectable medications (Table 2) were perceived as relevant and interpreted as intended. Based on participant feedback in Round 1, we changed the item, “I could not afford my medicine” to “The medicine was too expensive” to avoid placing blame for cost-related reasons on the patient. In Round 2, one participant noted the absence of an item to capture not having enough time to take the insulin out of the refrigerator and wait for it to warm up as a reason for nonadherence. We added, “I did not have time to prepare my medicine” to capture this situation. This item was perceived as intended in Rounds 3 and 4.

Study 2

Design and Participants

We conducted cognitive interviews with adults meeting the same eligibility criteria as Study 1 who were prescribed at least one antihypertensive hypertension and one lipid-lowering medication.

Procedures

Recruitment and consent procedures were identical to Study 1. Participants received one of two versions of the survey, the order randomly determined: 1) three separate, disease-specific versions (one each for diabetes, blood pressure, and cholesterol-lowering medications) and 2) a universal version in which participants were told, “When responding, think about your medications for diabetes, blood pressure, and cholesterol.”

As in Study 1, interviewers asked how participants interpreted the instructions and items, how they constructed their responses, and the suitability of a 7-day recall period. Interviewers also asked participants to reflect on providing a more accurate answer with the disease-specific versus universal version and how the repetition of reasons for nonadherence in the disease-specific versions affected their responses. We also included questions to inform implementation in clinical practice, including whether participants would be willing to complete the measure at a clinical visit, whether they believed people would be hesitant to respond accurately, and whether they believed people would respond differently when asked by a nurse versus physician. As in Study 1, participants completed a demographic survey and the S-TOFHLA. Data collection and analysis occurred iteratively using the methods described for Study 1.

Results

We contacted 159 patients, of which 61 were unable to be contacted, 54 declined, and 32 were ineligible. We interviewed 12 participants. Respondents were 61.5 years old on average, and 42% were assigned male at birth, nine were Non-Hispanic White persons, and two were Non-Hispanic Black persons. The average S-TOFHLA score was 33.6 (range: 25–36), indicating adequate functional health literacy. Two participants were taking oral diabetes medications only, five were prescribed injectable medications only, and five were prescribed both medications All 12 participants were reported being prescribed at least one medication for blood pressure and cholesterol. The results of cognitive interviews are summarized in Table 2 and discussed below.

Instructions and Recall Period

In Round 1, two participants responded to one disease-specific version of the survey regarding the wrong disease (eg, to the diabetes version about blood pressure medications). Additionally, a few participants commented that respondents might not read all the instructions. To draw participants’ attention to the disease in question on the disease-specific versions, we added a heading at the top of each version (eg, “Blood Pressure Medicine”). Following this modification, no participant responded about the wrong disease.

As in Study 1, participants indicated that, for daily medications, they could remember missed doses and reasons for missing doses over the past 7 days. Participants with highly routinized schedules felt they could remember over longer time periods, but other participants did not. We did not change the recall period.

Extent of Nonadherence Items

Participants indicated that the extent of nonadherence items were appropriate for diabetes, blood pressure, and cholesterol medications. As in Study 1, many participants perceived the first item as addressing unintentional nonadherence, the second item as addressing intentional nonadherence, and the third item as addressing either.

Two participants from Round 2 – one of whom who took all medications together and one of whom did not miss any doses–felt they could respond accurately to either the combined or disease-specific versions because they either took or missed doses of all their medications together. Most participants, however – many of whom took medications more than once each day – felt that their responses to the combined version would not be as accurate as to the disease-specific versions. When completing the combined version, some of these participants thought about the one medication they missed; some thought of all their medications and averaged across them; and others thought that they would forget the one they missed because it was an exception. Therefore, we eliminate the combined version from Rounds 3 and 4.

Reasons for Nonadherence

We administered the existing list of reasons for nonadherence from our previously published Version 2⁹ for blood pressure, cholesterol, and oral diabetes medications and added to the list the new items for injectable diabetes medications. Participants indicated the lists of reasons for nonadherence were comprehensive for all three diseases. Participants reported that the repetition of reasons across the three disease-specific surveys did not bother them and felt it was necessary for accurate responding. Participants recognized the additional items related to injectable medications on the diabetes version and suggested placing items that differed across lists at the bottom to facilitate item interpretation and response.

Implementation in Clinical Practice

All participants indicated they would be willing to complete the measure while waiting for a medical appointment. Three participants provided caveats due to negative interactions in which their physicians assumed they were nonadherent. They wanted assurance that their responses would be used to help their care team provide better care instead of being the basis for a lecture from their clinicians. One of these participants noted that people of color may be reluctant to admit to nonadherence due to distrust of providers, which may disadvantage them because they could not benefit from interventions such as grants from pharmaceutical companies. All participants indicated they would respond similarly, regardless of whether the measure was administered by a nurse or a physician.

Discussion

Many patients have multiple chronic conditions and medications requiring different routes of administration. There is little guidance about how to assess self-reported nonadherence in these contexts. We used cognitive interviewing to investigate how participants interpret items and construct responses when considering oral versus injectable diabetes medications and when thinking about their behavior in the context of multiple chronic diseases. Our findings support and extend our previous research on improving the reliability and validity of our self-reported nonadherence measurement.

Previously, we validated the measure for oral administration, leaving the question of whether the items would work in the context of injectable medications. We learned the importance of assessing nonadherence to oral and injectable diabetes medications separately. Furthermore, we recognized that the response scale for the extent of nonadherence items is incompatible with the weekly dosing schedule of some injectable diabetes medications. Participants reported they could recall missed doses and reasons for doing so for weekly injectable medications for 1 month. This finding will need to be validated in a future quantitative study.

Although self-report measures are often administered in the context of one disease, clinicians would like to assess nonadherence to medications for multiple diseases.^16,17 Our results suggest that participants find it cognitively easier to respond, and report nonadherence more accurately, in separate, disease-specific versions. Whether responses are more valid with disease-specific versions is an empirical question to be addressed with quantitative methods. Disease-specific versions may be more useful to clinicians, as a global measure of nonadherence to “all of your medications” would require follow-up to obtain specific information. Our participants did not mind the extra time or effort required to complete multiple versions. However, this viewpoint may be unique to our participants, who were willing to volunteer for a survey-based research study. Future studies could investigate data completeness among large samples with varying numbers of disease-specific versions to see if missingness increases with more versions administered.

In addition to answering our two primary research questions, our findings confirm the need to administer multiple items to assess extent of nonadherence. Although single items may be less burdensome, they typically lead to greater measurement error (ie, reduced reliability and validity) compared to multiple-item scales. Thus, when constructing the measure, we followed best practices in scale development by including multiple items (ie, a minimum of three) designed to assess the same latent variable of extent of nonadherence, but with different grammar and word choices.¹⁸ We noted that participants may interpret and respond to the three extent items differently regarding intentional versus unintentional nonadherence. Because people miss medications intentionally and unintentionally, it is a strength to capture these related but different aspects as it increases our ability to detect nonadherence. Assessing both types does not appear to negatively affect reliability or produce results suggesting that two separate subscales are needed. Two extra brief items are expected to take less than 30 seconds to complete. This improvement in accuracy is likely to be worth the small amount of extra time.

This study has some limitations. Respondents were primarily Non-Hispanic, White, English-speaking persons who agreed to be in a research study and had high health literacy. Although we achieved saturation of ideas with our sample sizes, other issues may arise in future research with different populations and a larger sample size.¹⁹ Finally, new oral and injectable therapies are currently under investigation. Therefore, the reasons for nonadherence items emerging from this study may not capture disease-specific reasons that are specific to new regimens, and some reasons on this measure may be less relevant to new regimens.

Implications for Respondent Burden and Measure Implementation

Even an optimally designed questionnaire may lead to inaccurate responses if patients perceive that their clinicians will use self-reported nonadherence as a reason to lecture them. To encourage accurate responding, we need to assure participants that their responses will be used to provide better care, that their responses over the recall period may not represent what they do over longer time periods, and visually clarify which disease is being addressed when using multiple disease-specific questionnaires. Additionally, similar reasons for nonadherence that appear for multiple diseases could be grouped together, with disease- or route-specific reasons grouped separately.

Based on our current findings, we recommend administering the measure weekly for daily medications and monthly for weekly medications. Researchers and clinicians often wish to know about missed doses over longer time periods. Because participants likely cannot recall longer time periods accurately, we recommend administering the measure repeatedly. With digital health technologies such as text messaging and mobile applications, such administrations can be automated and enable participants to complete them in their everyday environment.^20,21

The two domains of extent of nonadherence and reasons for nonadherence are designed to achieve balance of response burden and information. The brief, three-item extent of nonadherence domain was created to assess missed doses across a wide range of diseases and can be administered alone or in combination with the reasons for nonadherence domain. The reasons for nonadherence domain was designed to be tailored to the population or disease in question. Whereas some reasons such as forgetting and medication cost apply to any population, other reasons do not. The reasons for nonadherence domain is based on a causal indicator-based latent variable model,⁷ so removal of irrelevant items – and addition of relevant items – is encouraged to maximize content validity. In cases in which the burden of a long list of reasons is deemed too great, clinicians and researchers may abbreviate the reasons for nonadherence (eg, select those that are most prevalent) or prioritize the diseases for which nonadherence is assessed.

Future quantitative studies with larger sample sizes are needed to validate participants’ beliefs about what contributes to more accurate responses. Research is also needed to address implementation questions such as the duration of administration required to represent participant experiences; how the measure can be implemented in clinical practice to maximize appropriateness, acceptability, and feasibility; how clinicians wish to receive results; in which settings the measure has impact; and, most importantly, whether routine screening improves clinical outcomes. For all future research directions, it is critically important to include ethnically and racially diverse samples.

Conclusion

This paper addresses important questions regarding implementation of a self-report nonadherence measure beyond oral medications in the context of a single disease. Lessons learned should greatly improve the collection of clinically meaningful nonadherence data in the context of the growing proportion of patients prescribed multiple classes of medications across different routes of administration. This measure could be used to facilitate constructive conversations between clinicians and patients to improve patient trust and adherence and to improve clinician confidence in decisions relevant to treatment intensification.^22,23 The measure may also allow more comprehensive collection of nonadherence data in research studies as well.