Exploring the Agreement Between Self-Reported Medication Adherence and Pharmacy Refill-Based Measures in Patients with Kidney Disease

Introduction

Poor adherence to medications is observed in up to 50% of patients with chronic diseases¹ and is associated with a higher risk of morbidity and mortality.² Patients with kidney disease, especially those with kidney failure (KF) on dialysis, have an unusually high pill burden,³ which could make them vulnerable to poor medication adherence. Kidney disease is also associated with a high prevalence of psychosocial issues such as anxiety and depression as well as lower health literacy which can promote nonadherence.^4–6 The adoption of strategies to improve adherence would therefore be highly beneficial among this patient population, and an accurate diagnosis of medication nonadherence is the first step in this approach.²

The detection of nonadherence is, however, not easy due to the absence of a reliable method that consistently identifies and quantifies adherence.^2,7 The prevalence of nonadherence can vary based on the method used to measure adherence, as exemplified by an Australian study of patients with pre-dialysis chronic kidney disease, which reported nonadherence rates of 43% and 60% among patients in the same cohort, using two different self-report tools.⁸ Medication refill data, curated from pharmacy insurance claims, have been increasingly used to measure adherence with the advent of computerized pharmacy refill records.⁹ The premise behind their use is the assumption that medication refills, which equate to medication possession, correspond to medication-taking behaviours.¹⁰ Constructs of medication possession based on pharmacy refill data, such as “medication possession ratio” (MPR), estimate medication availability by computing the ratio of the prescribed days’ supply filled by the patient to the total prescribed days in a refill interval, during the study period.¹¹ These medication refill measures are continuous metrics with higher values, closer to 1.0 (100%), indicating better adherence. However, patients are frequently categorised as “adherent” or “nonadherent” based on arbitrary thresholds using different cut-off scores of refill measures.¹² For instance, studies have frequently used a cut-off score of 0.8 (80%), assigning patients to “inadequate” (<0.8) or “adequate” adherence (≥0.8) categories.^11,13 Based on this paradigm, patients with a medication refill rate of 0.2 (20%) or 0.79 (79%) would be classified to the same category of “inadequate adherence”, while the latter patient is much closer to being “adherent” than the former. Furthermore, a cutoff threshold of 0.8 may not also be appropriate for all clinical settings; for instance, 80% adherence is insufficient for a kidney transplant recipient. To avoid the potential issues with categorization of adherence using arbitrary thresholds, it has been suggested to measure adherence as a continuous variable.⁷

Medication adherence is more frequently measured using self-reported, subjective adherence instruments such as the Morisky medication adherence scale (MMAS) and the Hill-Bone scale.¹⁴ The MMAS aggregates the scores drawn from responses to standardized questions and classifies patients into categories of high, medium and low adherence, using cut-off aggregate scores.¹⁵ The MMAS categories have been shown to correspond to clinical indicators of medication non-adherence, such as inadequate blood pressure control.^15,16 It has also been validated against objective adherence markers like the pharmacy refill rates, based on concordance between MMAS adherence categories and pharmacy refill adherence categories, and rates of non-persistent refills.¹³

Similar to the medication refill measures, eg, MPR, as highlighted above, it is possible that a lower MMAS score such as 1.5 may represent much poorer adherence than a higher score such as 5.75, even though both patients would be categorized to “inadequate adherence” using the recommended cut-off score (<6.0). Whether the MMAS score can be expressed as a continuous variable to reflect medication adherence just like the MPR scores has not been examined. In this context, we sought to explore the quantitative limits of agreement between aggregate scores of the MMAS adherence instrument and pharmacy refill rates as continuous variables, in a cohort of patients with chronic kidney disease, including patients with KF on dialysis. In the same cohort, we also assessed the agreement between categories of lower adherence for the two measures, which has been reported in the literature, for non-renal patient groups.

Materials and Methods

This cross-sectional study was conducted among patients attending the renal outpatient clinics for the treatment of chronic kidney disease (CKD) and those with KF who were undergoing dialysis in the Illawarra Shoalhaven Local Health District in New South Wales, Australia. The study aimed to examine the level of quantitative agreement between two measures of medication adherence and was part of a broader project, which explored the relationship between medication adherence and its potential contributing factors in the above cohort. Patients who were 18 years or older and had kidney disease were able to take part in the study. Patients were excluded if they had undergone kidney transplantation or were being actively worked up for a living donor transplant, had been diagnosed with dementia or intellectual impairment, had an anticipated life expectancy of fewer than 12 months, or were on a renal palliative care pathway. The test instruments were administered in English by research assistants, and patients who did not speak and read English were also excluded from the study.

This study was approved by the Illawarra Shoalhaven Local Health District/University of Wollongong Human Research Ethics Committee (HE14/398–HREC/14/WGONG/90) and complies with the Declaration of Helsinki. Participants in this study signed a consent form for the main study, as well as a separate consent form, approved by Services Australia, which authorized the use of pharmacy refill information from the Medicare Pharmaceutical Benefits Scheme (PBS) claims data, subject to patients specifically consenting for the release of such data. The Services Australia approval was obtained after a detailed review of the study protocol (vide reference number MI-7232).

Data Collection

Demographic and Clinical Data

Demographic data (age, gender, race, educational level and income), smoking status and clinical data (cause of renal disease and comorbidities) were collected using a combination of patient interviews and chart reviews.

Measures of Medication Adherence

Self-reported medication adherence was assessed using the 8-item Morisky medication adherence scale (MMAS).¹⁵ Based on their aggregate MMAS scores, participants were categorized as highly adherent (score = 8), moderately adherent (score = 6 to <8) or poorly adherent (score <6).¹⁵ For assessing quantitative agreement, we generated an MMAS score fraction by dividing the aggregate MMAS score of a given patient by 8, which is the maximum possible MMAS score. For instance, a raw MMAS score of 8 for a patient equated to an MMAS score fraction of 1.0 (8/8), while an aggregate MMAS score of 6 equated to a score fraction of 0.75 (6/8).

Two pharmacy refill-based measures, namely “medication possession ratio” (MPR) and “proportion of days covered” (PDC), were derived from the PBS medication refill data using the standard approach. MPR was computed as the total days of supply of the dispensed medication (supply days) divided by the total number of days in the study period (study days).^11,17 PDC was computed as the total number of days on which the patient had the medication available, or the days covered (covered days) divided by the number of days in the study period (study days).^17,18 MPR is insensitive to delayed script refills,¹⁹ while PDC takes into account delayed refills, since medication will not be available till refill.¹⁸ PDC also truncates any oversupply by capping adherence at 1.0 (100%), while MPR can go beyond 1.0, by allowing oversupply.¹¹

We analysed MPR and PDC for two groups of drugs, namely antihypertensive medications and other cardiometabolic drugs (including statins). In this study, we included cardiovascular drugs other than antihypertensives, such as, clopidogrel, amiodarone, isosorbide mononitrate, nicorandil and lipid lowering drugs such as statins, ezetimibe and fibrates as cardiometabolic drugs. Anti-platelet agents like clopidogrel were included among cardiometabolic drugs due to the standard practice of fixed-dose therapy, but oral anticoagulants were excluded since the majority of the patients on this class were on warfarin, which routinely has variable dosing and not suitable for medication dispensing data to assess adherence. In addition, aspirin was not included among the cardiometabolic drugs in the analysis because it is available without prescription and may not be captured by refill measures. When patients were on multiple ‘antihypertensive medications’ or “other cardiometabolic medications”, the average of their MPR and PDC was taken as the final MPR and PDC for the respective group. We did not examine the adherence to phosphate-binding agents because: that would be applicable only for KF patients; their prescribed dose could change frequently based on monthly blood test results and therefore make the estimation of the denominator of “study days” (see below) less reliable. It is important to recognize that patients are far less adherent to phosphate binders compared to antihypertensive drugs.²⁰

For the purposes of the study, PBS claims data obtained for each patient included the description of the medication, date of prescription, PBS item code, date of medication supply, patient category, such as general, concessional, and form category, such as original, repeat script. The PBS claims data were obtained for the period spanning from the patients’ enrolment to the main study to the conclusion of the study. Since the quantity of a medication dispensed at one refill is generally fixed, based on the standard pack size, and the prescribed dose of the medication can vary between patients, the “study days” for which the refilled medication supply is available can vary between patients. The “study days”, which varies with the prescribed dose, is the denominator for computing both MPR and PDC. The information on the prescribed dose corresponding to the refill period was obtained from the patient’s medical records to calculate “study days” and was used to compute MPR and PDC.

Medications studied in this research could be refilled only with a valid prescription from a registered medical practitioner in the Australian healthcare system. Since all patients are covered by the mandatory public health insurance scheme, known as Medicare²¹ and all outpatient medications are dispensed under the Pharmaceutical Benefits Scheme (PBS) of Medicare, it was assumed that the data regarding medication refills were comprehensive.

Refill data were included in the analysis only when three or more scripts covering a continuous period of three or more months were identified during the study period for each patient, since MPR and PDC have improved precision when calculated over at least three dispensings.¹¹ For patients who had refill-data covering an extended period of time, only the first 12–18 months of continuous therapy of each drug, which could be matched with the prescription information from medical records, was included in the calculation of MPR and PDC. If the same drug was prescribed at a different dose or the drug was switched with another agent from the same class, it was not counted again. However, if a different class of drug was used during the study period that was included as a different medication for the analysis if it satisfied the above criteria.

Statistical Methods

We described continuous variables using means and standard deviations and discrete variables using proportions. We compared continuous variables between CKD and KF groups using the Wilcoxon rank-sum test and categorical variables using the chi-square test. MPR and PDC were calculated for each dispensed medication of interest with the “medadherence package” for Stata¹⁷ using the dispensing dates, supply days and study days information from the refill and prescription data.

Association and Agreement Between MMAS and MPR / PDC Adherence Measures

Different methods were used to assess the association and concordance/agreement between measures of medication possession and self-reported measures. We examined the scores of MPR and PDC among high, medium and low adherence categories based on self-reported MMAS scores. We evaluated the trend of MPR/PDC scores across the MMAS categories using non-parametric tests of trend. We also undertook the analysis of covariance (ANCOVA) to explore the independent association between refill adherence rates and MMAS categories, after adjusting for age, patient group, whether CKD or KF on dialysis, educational status and annual income, due to their potential confounding effect on medication adherence. Conversely, we also looked at the MMAS aggregate scores in patients with adequate and inadequate adherence categories based on cut-off scores (<0.8 vs ≥0.8) of MPR and PDC for antihypertensive medications and other cardiometabolic medications. We examined the independent association between MMAS aggregate scores and MPR/PDC categories adjusting for age, patient group, educational status and annual income.

Since MMAS as well as MPR and PDC measure the same entity of medication adherence, we considered weighted Cohen’s kappa to be the most appropriate method to assess the agreement between categories of suboptimal adherence based on the two instruments. We examined the agreement of low versus medium and high adherence MMAS categories with pharmacy refill-based adherence categories of <0.8 versus ≥0.8 scores for MPR and PDC for antihypertensives and other cardiometabolic drugs.

Bland-Altman plots were used to evaluate the agreement between MMAS score fractions and MPR & PDC separately for antihypertensive drugs and other cardiometabolic drugs. In these analyses, we examined how many measurements were outside the limits of agreement, the mean bias and the limits of agreement for each analysis. Missing data were handled by list-wise deletion. Statistical significance was set at a two-tailed p-value of <0.05 for all comparisons. The statistical analyses were undertaken using Stata 16.0.

Results

Study Population

Of the 321 eligible participants (148 KF and 173 CKD patients) invited to participate in the study, 220 patients, 107 with KF undergoing dialysis and 113 with CKD not receiving dialysis, were recruited for the main study, after 66 (20.6%) patients declined to participate and 35 patients either withdrew consent or had an exclusion criterion identified before any study procedure was done. Out of this main study sample, 122 patients provided informed consent to release their PBS claims data. However, pharmacy refill data was obtained for only 116 patients since Services Australia consent forms were incomplete for six participants. Furthermore, three patients were not receiving antihypertensive or other cardiometabolic medications, resulting in a final refill cohort of 113 study participants (53 KF requiring dialysis and 60 early-stage CKD). The baseline characteristics are given in Table 1.

Table 1 Baseline Characteristics of the Study Sample Comparing Non-Dialysis CKD Patients and KF Patients

Table 2 provides the measures of self-reported and refill data derived medication adherence markers in the overall sample and the KF, CKD groups. MMAS scores were significantly lower in KF patients (p = 0.026) compared to CKD patients. The proportion of patients with low adherence (MMAS score <6.0) was greater in KF (37.7%) compared to CKD (25%); however, this difference was not statistically significant. Measures of antihypertensive medication adherence, assessed by MPR and PDC scores, as well as categories of low adherence (<0.8), were worse in KF patients compared to CKD patients (Table 2). Medication possession measures of other cardiometabolic medications, assessed by MPR and PDC scores, as well as categories of low adherence (<0.8), were worse but not significantly different between KF patients and CKD patients.

Table 2 Results of Measures of Self-Reported and Pharmacy Refill-Based Medication Adherence for the Overall Sample and CKD, KF Sub-Groups

The MPR and PDC scores for antihypertensive medications and other cardiometabolic medications for different MMAS adherence categories are shown in Table 3. There was a consistent and statistically significant drop in MPR and PDC for both classes of medications with worsening self-reported adherence. The non-parametric test for trend showed a significant declining trend in MPR and PDC scores for both antihypertensive and cardiometabolic medications among MMAS adherence categories. ANCOVA investigating the association between refill adherence scores and MMAS categories, adjusted for age, educational status, income status and patient group (KF on dialysis vs CKD), was also significant for all analyses (Table 3). Vice versa, the MMAS scores were significantly lower in the MPR and PDC low adherence category (score <0.8) for both antihypertensives and other cardiometabolic medications. ANCOVA investigating the independent association between MMAS scores and MPR as well as PDC categories, adjusted for age, educational status, income status and patient group, also showed significant association (Table 4).

Table 3 The Trend of Pharmacy Refill-Based Markers of Medication Adherence Among MMAS Categories of Self-Reported Adherence

Table 4 MMAS Scores in Patients with High vs Low Adherence Category Based on MPR and PDC for Antihypertensives and Other Cardiometabolic Medications

Cohen’s kappa statistics for agreement between categories of low adherence based on MMAS scores and measures derived from refill data are given in Table 5. There was fair agreement between the categories of the two methods of measuring adherence which was statistically significant.

Table 5 Cohen’s Kappa Exploring Agreement Between Low Adherence MMAS Category (MMAS Score <0.6) and Low Adherence Based on Pharmacy Refill-Based Measures for MPR & PDC (<0.8)

The results of our examination of the agreement between MMAS score fractions and MPR and PDC scores using the Bland-Altman plots are given in Figures 1–4. Less than 5% cases for MPR and PDC for both antihypertensive and other cardiometabolic medications were outside the limits of agreement. The mean bias ranged from −0.05 to −0.09, suggesting some concordance, but the limits of agreement were much broader, indicating such concordance is not strong. The mean bias and the limits of agreement between MMAS score fractions and measures of refill adherence measures are given in Table 6.

Table 6 Bias and Limits of Agreement for the Parameter Estimate as per Bland Altman Plots Evaluating Agreement Between MMAS Score and Pharmacy Refill-Based Measures

Figure 1 Bland Altman plots exploring agreement between MMAS and MPR of antihypertensives.

Figure 2 Bland Altman plots exploring agreement between MMAS and PDC of antihypertensives.

Figure 3 Bland Altman plots exploring agreement between MMAS and MPR of cardiometabolic medications.

Figure 4 Bland Altman plots exploring agreement between MMAS and PDC of cardiometabolic medications.

Discussion

The key findings from our study demonstrate a modest agreement between measures of adherence assessed by the self-reported MMAS scale and the pharmacy refill data, based on categorical analysis, as well as quantitative analysis of the limits of agreement using the Bland–Altman method. We also noted that the bias for the limits of agreement between MMAS and MPR or PDC was negative indicating that the self-reported scale did not overestimate or inflate adherence.²² The agreement between lower adherence categories of MMAS and pharmacy refill measures is consistent with existing literature.^13,23 However, our finding of a modest agreement between aggregate scores of MMAS expressed as a fraction and refill adherence measures, analysed as continuous variables, has not been previously reported in the literature. The discussion below focuses on the rationale for using adherence as a continuous measure and comparing the two instruments quantitatively.

Medication non-adherence is an important consideration in the management of chronic diseases.¹ However, there is far less certainty regarding the best method of measuring medication nonadherence or estimating its severity. Long-term medication taking is a highly complex behaviour with three phases, namely initiation, implementation, and persistence, and poor medication adherence can impact any of these phases. However, specific adherence measures may be sensitive to one phase of adherence, but may not detect poor adherence in a different phase with the same reliability.²⁴ For instance, electronic medication monitoring, which is perceived to be the most accurate method of tracking medication adherence, is not sensitive to non-adherence manifesting as non-initiation.²⁴ Different metrics derived from pharmacy refill data have been used to assess adherence objectively, such as MPR or PDC, categories of adherence based on treatment gaps, and time between prescription refills.^10,11 In our study, we have used MPR and PDC, which are the most widely used objective refill measures, and the latter has also been approved by Centres for Medicaid and Medicare Services (CMS) as an adherence measure to be used in its star rating program.²⁵

Even though MPR and PDC are continuous variables, researchers often dichotomize patients based on the adherence rates using arbitrary cut-off scores into adherent and non-adherent categories. The commonly used cut-off score of 0.8 (80%) and above used to define adherence^13,18 may be reasonable for many chronic diseases, striking a balance between the benefits of long-term therapy and the burden of uninterrupted medication taking. However, an adherence level of 0.8 (80%) may not be sufficient for immunosuppressive therapy in transplant recipients. Such arbitrary categorization of adherence may therefore not be relevant in all clinical settings. Categorization based on arbitrary cut-off scores also poses the risk of inadvertently compressing or amplifying the differences in medication adherence between patients. For example, if we consider two patients, one with a PDC of 0.75 and another with 0.25, the difference between them is compressed by placing them in the same category due to the 0.8 cut-off criteria, while the difference between a patient who has a PDC value of 0.75 and 0.85 is amplified by placing them in the different categories using the same criteria. On the other hand, using MPR or PDC as a continuous variable retains the quantitative information and provides the most unbiased estimate that is available even though analysing adherence as a continuous variable may require novel approaches.

Even though subjective measures of adherence like self-reported questionnaires may be perceived as less robust, they are far more practical and suitable for routine outpatient use as a measure of adherence in most research settings. It is noteworthy that lower adherence based on self-report tools such as MMAS has been associated with adverse health outcomes such as higher mortality and hospitalization as well as greater treatment costs per quality adjusted life-years.^26–28 Measurement of medication adherence is currently not used in routine clinical practice, but instruments like MMAS are easily useable in diverse clinical settings. Traditionally, self-reported questionnaires, including MMAS, have reported the results as categories based on cut-off scores of aggregate scores derived from the responses to standardized questions. However, whether a much lower aggregate score within the same category indicates a worse degree of nonadherence is unknown. We have noted that MMAS scores were lower in the lower adherence category based on a cut-off score of <0.8 for both MPR and PDC for antihypertensives and other cardiometabolic medications. To check whether the aggregate MMAS scores agree with the MPR or PDC scores, we tried to explore the agreement between refill adherence measures and the MMAS fractions as a continuous variable. It is widely perceived that self-reported measures tend to overestimate adherence, but we noted in the Bland-Altman analysis that the bias between MMAS fractions and MPR and PDC was negative, indicating the MMAS fraction scores were lower than MPR and PDC measurements. However, the limits of agreement were wide, indicating the concordance was not strong. Therefore, the current results should be interpreted with caution while future studies using a similar approach in diverse healthcare settings would be informative.

Our assessment of agreement between categories of low adherence based on MMAS and refill measures yielded results that are consistent with existing literature. Many studies have used the correlation between categories to report the agreement. However, since both tests are measuring the same attribute of medication adherence, Cohen’s kappa is the most appropriate test for assessing agreement between categories and Bland-Altman’s method is the most appropriate approach for examining the agreement between continuous variables, rather than correlation or regression.^29,30 We have used these recommended methods in our analyses.

Our study has strengths and limitations. This is the first study exploring agreement between refill measures and self-reported MMAS adherence scale as a continuous variable, and we have noted that there is modest agreement even with this comparison. An important issue in the calculation of MPR and PDC is the ability to accurately determine the denominator, which is the number of “study days” for which medication supply is available with a single refill. This is determined by the prescribed dose and the standard dispensing pack size. In some studies that involve refill data, this information is not readily available, which could introduce errors in the estimation of MPR and PDC, if a standard dosing regimen is assumed.¹¹ In our relatively small cohort, we had access to corresponding prescription information, which ensured the accuracy of the MPR and PDC results.

We have compared two commonly used methods of measuring adherence which have been widely used in published literature in this field. Nevertheless, it could be argued that MMAS is not a true continuous variable since this is computed by aggregating the scores from the responses to a number of rating questions, and therefore not suitable for parametric analysis. However, such analyses are common in the field of psychology and it has been argued that while the responses to a rating question like the Likert items may be ordinal, the scales which consist of sums across many such items can be considered interval and parametric statistics can be shown to be robust despite violations of some of the strict statistical assumptions.³¹ The MPR and PDC are inherently continuous measures and we believe the evaluation of quantitative agreement between them and MMAS score fractions is justified. Nevertheless, it must be acknowledged that a given MMAS score or fraction, used as a continuous variable, may not carry the same weight and reliability as the corresponding MPR or PDC.

Self-reported measures like MMAS are most suitable for evaluating adherence to a single medication, and most patients in our study had a high pill burden. It could also be argued that evaluation of non-adherence is more important in patients who have a heavy pill burden since there is a greater risk of developing non-adherence to at least some of the medications. Administering the MMAS questionnaire for each medication separately will be repetitive and impractical, and it is noteworthy that there was moderate agreement between self-reported measures and refill measures using various methods of evaluation, which may indicate that this approach is valid even for patients on multiple medications. We have not assessed the adherence to phosphate binders which impose a significant medication burden, especially for patients with kidney failure on dialysis.³ Since nonadherence to phosphate binders is common,²⁰ the agreement observed with antihypertensives and cardiometabolic medications may not have persisted with phosphate binders, which is a limitation of our study. However, the frequent dose changes of phosphate binders that commonly occur in clinical practice would have made the estimation of MPR and PDC less reliable and the assessment of agreement potentially inaccurate.

It should also be acknowledged that our sample size was relatively modest, drawn from a single local health district and included a heterogenous patient population (ie patients with early stage CKD and those with kidney failure requiring dialysis). These limitations should be considered when interpreting our findings and conclusions.

It is noteworthy that the prevalence of nonadherence reported in our study (31%) was lower than that reported in an Australian study which evaluated adherence in patients with kidney disease (43–60%),⁸ but closer to the pooled non-adherence rates (32.6%) in a systematic review of pre-dialysis chronic kidney disease patients.³² The variations in the nonadherence rates between studies may be due to variations in the instruments used to measure adherence. It is also possible that a greater proportion of nonadherent patients may have declined to participate in our study which could have skewed our results. This suggestion is based on the evidence that patients with unfavourable behaviours and profiles are often overrepresented among the non-participants in clinical research leading to selection bias.³³

The key message from our study is the feasibility of using a self-reported adherence tool like MMAS in evaluating adherence as a continuous variable, rather than as defined categories, especially since non-adherence is not an all-or-none behaviour, and the intensity of sub-optimal adherence can vary between individuals and within the same subject between different time periods.⁴ More research is needed in this area to draw more confident conclusions.