Validity and Reliability of the Thai Version of the 19-Item Compliance-Questionnaire-Rheumatology

Introduction

Rheumatoid arthritis (RA) is the most common chronic inflammatory arthritis with prevalence of 0.25% worldwide¹ and 0.12% in Thailand.² RA primarily affects the lining of the synovial joints and can cause progressive disability, socioeconomic burden and premature death.³ Early treatment can prevent irreversible disability and improves quality of life.⁴ Current recommendations suggest the use of disease-modifying antirheumatic drugs (DMARDs) and treat-to-target strategy aiming at remission or at least low disease activity to attain favorable outcomes.^5–7 To achieve this goal, adherence with the medication regimen is essential. Medication nonadherence is associated with higher disease activity, disease flares, and functional impairment.^8–11

Adherence to medication in patients with RA varies from 22 to 100%.^9,11–14 There are many methods to assess medication adherence including self-reports, physician’s estimation, drug biomarkers, pharmacy refills, electronic monitoring, and questionnaires.¹¹ Assessment of medication adherence in the clinical setting is unlikely to apply because it is time-consuming. However, physicians tend to substantially overestimate medication adherence.^15–18 It has been shown that sensitivity of physician’s estimation of poor adherence was quite low as 24%.¹⁵ There are many self-reported questionnaires developed for measuring medication adherence.¹⁹ The 19-item Compliance-Questionnaire-Rheumatology was developed by de Klerk et al in 1999.²⁰ The CQR-19 was validated against an electronic medication event monitoring system that records the time and date of opening and closing of the drug medication package through microelectronic circuitry integrated into the cap of a pill-bottle and is known as the Medication Event Monitoring System (MEMs). The sensitivity and specificity of the CQR-19 to detect good compliance were 95% and 62%, respectively. The predictive value to detect poor compliance was 86% and to detect good compliance was 83%.²¹ The CQR-19 has been translated into many languages and used in many countries,^22–25 including Thailand.²⁶ Therefore, we aimed to evaluate the psychometric properties and satisfaction of the Thai version of the CQR-19 (Thai CQR-19).

Materials and Methods

Study Design and Population

This cross-sectional study was conducted in the rheumatology clinic of Siriraj hospital in Bangkok from October 2020 to March 2022. Patients with RA that fulfilled the 1987 American College of Rheumatology (ACR) Diagnostic Criteria²⁷ or the 2010 ACR/European League Against Rheumatism (EULAR) Classification Criteria,²⁸ who were ≥ 18 years of age and literate were included. Patients with cognitive dysfunction, schizophrenia, psychosis, vision problems, or were unable to complete the questionnaire by themselves were excluded. This study was approved by the Scientific Ethics Committee of the Siriraj Institutional Review Board (COA Si no. 739/2020). Informed consent was obtained from all participants prior to enrollment. All procedures were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Thai Version of Compliance-Questionnaire-Rheumatology-19 (Thai CQR-19)

The English version of the CQR-19 (Supplement 1) was translated into Thai language (Supplement 2) using the framework suggested by Beaton et al²⁹ in a previous study.²⁶ The Thai version of the CQR-19 consists of 19 items that ask patients to specify to what degree they agree or disagree with each statement on a 4-point Likert scale (1, strongly disagree; 2, disagree; 3, agree; 4, strongly agree). There are six items stated negatively (items 4, 8, 9, 11, 12, and 19) and were therefore reversed. The total score is calculated by summing the items, subtracting 19, and then dividing by 0.57. This ensures that the CQR total score can vary from 0 (complete noncompliance) to 100 (perfect compliance).²¹ To interpret the CQR-19, the score for each item was weighted and calculated using a calculator provided in a Microsoft Excel spreadsheet.²¹ Individual use for medical purpose is free of charge. For the use of more than one person for research or commercial activities, permission from the corresponding author is required.

Data Collection and Clinical Evaluation

The study data were collected and managed using REDCap (Research Electronic Data Capture) tools hosted at the Siriraj Center of Excellence in Bioinformatics and Data Management, Faculty of Medicine Siriraj Hospital, Mahidol University.³⁰ REDCap is a secure, web-based application designed to support data capture for research studies. It provides 1) an intuitive interface for validated data entry; 2) audit trails to monitor data manipulation and export procedures; 3) automated export procedures for seamless data downloads to common statistical packages; and 4) procedures for the importing of data from external sources.

Demographic and disease-related variables including age, sex, educational levels, comorbid diseases, smoking and alcohol status, disease duration, and serological status were recorded. RA disease activity was assessed by patient global assessment (PGA) (0–10 cm visual analogue scale or VAS, physician global assessment (PhGA) (0–10 cm VAS), the number of swollen and tender joint assessed by their rheumatologists, and erythrocyte sedimentation rate (ESR). The disease activity score 28 (DAS28) was used to represent the RA disease activity. DAS28 is composite measurement comprised of the number of tender joint count 28 joints (TJC28), the number of swollen joint count 28 joints (SJC28), the ESR, and the PGA. DAS28 was calculated as followed, DAS28 = 0.56*sqrt(TJC28) + 0.28*sqrt(SJC28) + 0.70*Ln(ESR) + 0.014*PGA. A DAS28 of greater than 5.1 implies active disease, less than 3.2 low disease activity, and less than 2.6 remission.³¹ Functional status was assessed using the Thai version of the Health Assessment Questionnaire (Thai HAQ).³² Participants were asked to complete the Thai CQR-19²⁶ by themselves at baseline. All unanswered questions were explained to the participants until they understood and could answer these questions. The Thai CQR-19 satisfaction was assessed by numeric scale from 0 to 10. The score of 0 means unsatisfactory and a score of 10 means satisfactory. The completion time and number of unanswered questions were recorded. Participants were advised to take medications only dispensed during this visit. Participants were received Thai CQR-19 in prepared envelop and were asked to complete and returned to investigator within two days via mail. A few days prior to the follow-up visit date, the investigators telephoned the participants to remind them to bring the remaining medications to the clinic. At the 3–4 months of follow-up visit, participants were asked to complete the Thai CQR-19 by themselves. Investigators assessed the medication compliance by comparing the pill count of the prescribed DMARDs and prednisolone from the previous visit. For the participants who could not attend the clinic at follow-up visit, investigators telephoned and asked the participants to perform pill counts by themselves via video call. The pill count was calculated using the following equation and poor compliance was described as compliance ≤ 80%.^11,19,21,24

Statistical Analysis

Sample Size Calculation

To measure validity, the Cohen’s kappa (κ) of medication compliance between the Thai CQR-19 and pill count was set at 0.8.²⁰ The calculated sample size for criterion validation was 139 participants when the proportion of chance agreement was 0.5, the desired precision width was 0.1, and type I error was 0.05.³³ Based on an estimated Cronbach’s alpha of 0.8 from previous studies,^21,24 the calculated sample size for reliability testing was 103 participants, when type I error was 0.05 and power was 0.80.³⁴ For intraclass correlation coefficient (ICC) of test-retest reliability of 0.63,²³ 140 participants were required with desired precision width of 0.2 and type I error of 0.05.³⁵ Finally, a total sample size of 154 participants was required with a 10% drop-out rate.

Data Analysis

Categorical variables were reported as number and frequencies. Continuous variables were reported using the mean and standard deviations for normally distributed data and the median and interquartile ranges (IQR) were used for non-normal distribution. The criterion validity of the Thai CQR-19 was evaluated using pill counts. Agreement between the Thai CQR-19 at follow-up visit and the pill count was described using Cohen’s kappa and percentage of agreement. However, when a “kappa paradox” was present (a discrepancy between Cohen’s kappa and percentage of agreement), agreement was assessed using Gwet’s AC1.^36–39 To assess construct validity, Spearman correlation coefficients (r_s) were calculated for the relationships between the Thai CQR-19 scores, DAS28 scores and Thai HAQ scores. Internal consistency was tested using Cronbach’s alpha, and test-retest reliability was tested using ICC with two-way random model, single measures, and consistency. The completion time and number of unanswered questions were recorded as a proxy for feasibility and described as median with IQR and percentage, respectively. A P value of less than 0.05 was considered to be statistically significant. All analyses were performed using IBM SPSS statistics Version 18.0 for Windows.

Results

Participant Characteristics

A total of 156 participants were enrolled (Table 1). The mean age ± SD was 55.9 ± 11.7 years. Most (89.7%) were female. The median education duration (IQR) was 12 (6.0–16.0) years. The median disease duration (IQR) was 9.9 years (3.9–15.6). The majority of participants had mild disease activity [median DAS28 (IQR) = 2.67 (2.16–3.40)] and mild functional impairment [median HAQ (IQR) = 0.13 (0.00–0.61)]. Eighty-one (52%) were prescribed two different types of DMARDs [median (IQR), 2 (2.0–2.8)]. A minority (15.4%) took prednisolone. Methotrexate (85.9%), chloroquine (38.5%), sulfasalazine (30.1%), and leflunomide (27.6%) were the most commonly prescribed medications. Only 3.2% and 5.8% used target synthetic DMARDs and biologic DMARDs, respectively.

Table 1 Baseline Characteristics and Prescribed Medications of 156 Patients with RA

Feasibility

One hundred and fourteen (73.1%) participants completed all 19 items. The most frequent unanswered questions were items 8, 9, 11, 12 and 19 (Table 2). The median number of unanswered questions (IQR) was 0 (0–1). Most participants skipped 1–2 items (9.0% each). The median completion time (IQR) was 4.4 (3.2–6.2) minutes. Most participants satisfied the Thai CQR-19 questionnaire [median (IQR), 9 (8–10)].

Table 2 Unanswered Question of Thai CQR-19 Questionnaire

Criterion Validity

Five participants were lost to follow-up. Thus, 151 participants were included for criterion validity analyses and 156 were included for construct validity analyses. Based on the pill count, the median compliance (IQR) was 95.6% (88.9–100.0%) and most participants had good compliance (% compliance > 80% = 88%). There was no agreement between the Thai CQR-19 and the pill count based on the kappa coefficient (kappa = 0.043, p = 0.56), despite the high percentage of agreement (72.2%).⁴⁰ Gwet’s AC1 was calculated and showed moderate agreement between the Thai CQR-19 and the pill count (AC1 = 0.61, p < 0.001). Sensitivity and specificity to detect good compliance by the Thai-CQR-19 were 78.2% (95% CI: 71.2–85.2) and 27.8% (95% CI: 27.8–70.9), respectively with 88.9% (95% CI: 83.2–94.6) positive predictive value, and 14.7% (95% CI: 2.8–26.6) negative predictive value. The overall accuracy was 72.2% (95% CI: 65–79) (Table 3).

Table 3 Comparison of Medication Compliance Based on the Thai CQR-19 and Pill Count

Construct Validity

There was a weak correlation between the total Thai CQR-19 score and disease activity measured by DAS28 (r_s = 0.200, p = 0.01). The total Thai CQR-19 score was not correlated with functional disability measured by Thai HAQ (r_s = 0.092, p 0.25) (Figure 1).

Figure 1 Scatter plot showing the correlation of total Thai CQR-19 and DAS28 scores (A) and HAQ scores (B).

Reliability

One hundred and forty-eight (94.9%) participants returned the retest questionnaires. The median CQR-19 score (IQR) was 82.46 (70.18–89.47) on the first measurement and 80.70 (70.18–89.47) on the retest. Internal consistency, measured by Cronbach’s alpha, was 0.826. The test-retest reliability, measured by ICC, was 0.643 (95% CI 0.54–0.73, p < 0.001).

Discussion

This study demonstrates that the Thai CQR-19 questionnaire is moderately correlated with pill count and reliable in both internal consistency and test-retest. Similar to previous studies, the median compliance was high (95.6%). Salgado et al reported a median compliance (IQR) of 98% (83 −100%) when the medication compliance was assessed using MEMs.²⁴ Lee et al assessed compliance using pharmacy refills and reported a mean compliance ± SD of 89.4 ± 14.5%.²² However, the prevalence of good compliance (> 80% compliance) in this study was 88.1%, which was higher than de Klerk et al who reported of 52% using MEMs.²¹ This discrepancy may be caused by the different gold standard for medication compliance used in these studies. It has been shown that self-reported and pill count overestimate compliance, while MEMs underestimate the compliance.^11,41–43 Additionally, the “Hawthorne effect” may explain the high compliance we observed if participants modified their behavior in response to their awareness of being monitored. Moreover, some patients may deliberately not return medications.^11,44

Previous studies have described a good correlation between CQR-19 and compliance using MEMs, pharmacy refill, and self-reported questionnaire.^22–25 Initially, we found no agreement between the Thai CQR-19 and the pill count when analysed using Cohen’ s kappa. We found a discordance between kappa and the percentage of agreement due to asymmetric data.^38,45,46 In our study, the high prevalence of good compliance contributed to this discordance. Therefore, Gwet’s AC1 was applied, revealing a moderate agreement between the Thai CQR-19 and the pill count.

We found that the Thai CQR-19 score had a weak correlation with disease activity as measured by the DAS28. Consistent with previous studies, disease activity was not statistically significantly related to medication noncompliance.^26,47,48 Marras et al also found that DAS28 scores were not different between patients in good or poor compliance CQR-19 categories.⁴⁹ Similar to the Spanish adaptation with respect to divergent construct validity, the total Thai CQR-19 score and HAQ were not correlated (r_s = 0.092, p 0.25).²⁴ High disease activity and poor functional status may be either the cause or outcome of poor medication compliance. Patients who have active disease or impaired functional status may be more likely to try to take medications according to their physicians’ prescription, while patients who have poor compliance may have more active disease or functional disability. Due to the cross-sectional study design, we could not determine any temporal relationship between compliance and disease activity or functional status.

The test-retest reliability yielded an ICC of 0.64 for the total Thai CQR-19 score, which was comparable to the Korean (0.71), Spanish (0.76), Arabic (0.76), and Turkish (0.63) adaptations, respectively).^22–25 Internal consistency as assessed by Cronbach’s alpha (0.83) was also comparable to the Spanish (0.86), Arabic (0.79), and Turkish (0.83) adaptation, respectively.^23–25 These reliability results were adequate and acceptable.^50,51

The median completion time of approximately five minutes was shorter than reported in previous studies (English, 12 minutes; Turkish, 10–15 minutes).^21,23 The most common unanswered questions were items 8, 9, 11, 12 and 19. All were negative questions. Items 9 and 19 were also the most often unanswered questions in the original English language version of the CQR-19.²¹ Questions 9 and 19 asked about drug compliance on holidays and special weekends, and some participants did not recognize the special days so they could not respond. Likewise, some participants did not understand the word “miracle” in question 11. Question 8 asked if they disliked taking medication, some participants felt neutral about taking medication, so they did not answer this question. Item 12 was framed as a hypothetical situation, “If you can’t stand the medicines you might say: ‘throw it away, no matter what’”, a behavior that rarely occurs with Thai patients. This may explain why some participants left the answer box empty. Because of the complexity of some questions in the Thai-CQR19, we suggest that the questionnaire be completed by a trained interviewer rather than self-administered.

The strength of this study is that we included the largest sample size, when compared to validating versions in other languages.^21–25 We used RA as study population in this validation study because RA is the most common rheumatic disease requiring long-term treatment. Therefore, the results of this study should be applicable to patients with other chronic illnesses.There are some limitations in this study. First, this study was conducted during the COVID-19 pandemic and some participants could not attend the follow-up visit at the clinic. Therefore, we telephoned and asked the participants to perform pill counts by themselves. This may have led to an overestimate of compliance. However, when possible, we requested the participants to use video call to verify the pill count. Second, a few participants (less than 5%) took the remaining drugs from the previous visit, as a result, the pill count may be inaccurate in some cases. Third, this study was conducted in a tertiary care hospital where patients tend to have more severe disease and become more aware of how to manage their illness, resulting in higher compliance than might be observed elsewhere. Forth, the retest was done and returned using the post office service. Some participants did not return the retest questionnaire by mail within two days, but returned it in person at the follow-up visit. Therefore, we could not be sure whether the duration between test and retest was actually within two days. Finally, we did not assess the content validity of the Thai version questionnaire.

Conclusion

In conclusion, the Thai CQR-19 is a valid and reliable instrument to evaluate medication compliance in daily practice and epidemiological research. Although this questionnaire is intended to be self-administered, to ensure completeness and accuracy of the compliance assessment this questionnaire may be administered by interview.