Reliability and validity of the Maltese version of the Perception of Anticoagulant Treatment Questionnaire (PACT-Q)

Introduction

Anticoagulant therapy is the mainstay treatment for the primary and secondary prevention of thromboembolic complications in patients with atrial fibrillation (AF), venous thromboembolism (VTE) and mechanical heart valve replacement.¹ However, since it is a chronic treatment for most clinical indications, it can affect the health-related quality of life.^2,3 For instance, vitamin K antagonists (VKA), such as warfarin, have several food and drug interactions and require dose adjustment, therefore mandating periodic blood testing of the international normalized ratio (INR).¹

Since patients’ negative beliefs related to medications can result in non-adherence to chronic treatment and therefore reduced effectiveness,^4,5 specific questionnaires have been developed to assess the satisfaction associated with the anticoagulant treatment. These include the Perception of Anticoagulant Treatment Questionnaire (PACT-Q),⁶ the Duke Anticoagulation Satisfaction Scale (DASS),⁷ the Anti-Clot Treatment Scale (ACTS),⁸ the Deep Venous Thrombosis Quality of Life questionnaire (DVTQOL)⁹ and the Pulmonary Embolism Quality of Life Questionnaire (PEmb-QoL).¹⁰ However, there was no validated questionnaire available in the Maltese language.

We chose to translate the PACT-Q and the DASS because they have already been translated into several languages and applied to patients with a broad range of clinical indications to the anticoagulant treatment.^7,11–15 The aim of this study was to assess the psychometric properties (reliability and validity) of the Maltese version of the PACT-Q. The psychometric properties of the Maltese version of the DASS have been reported in a separate paper.

Materials and methods

The perception of anticoagulant treatment questionnaire (PACT-Q)

The PACT-Q is divided into two parts: the PACT-Q1 measures the expectations associated with the anticoagulant treatment and is administered prior to treatment initiation, while the PACT-Q2 measures the convenience and the satisfaction and is administered during anticoagulant treatment.^6,11 In the PACT-Q2, the “Convenience” dimension comprises 13 items (from the combination of the original sections B “Convenience” and C “Burden of Disease and Treatment”), while the “Anticoagulant Treatment Satisfaction” dimension comprises 7 items (section D).¹¹ All items can be answered according to a 5-point Likert scale (not at all, a little, moderate, a lot, extremely). During the analysis, the items of “Convenience” are reversed, summed and rescaled on a 0–100 scale; the items of “Anticoagulant Treatment Satisfaction” are summed and rescaled on a 0–100 scale. Therefore, higher total scores correspond to higher convenience/satisfaction.¹¹

Permission to translate and use the PACT-Q was obtained from Sanofi Aventis/Mapi Research Trust. The linguistic validation process followed published guidelines,^16,17 with two forward translations from English to Maltese and a backward translation from Maltese to English, performed by different people (a professional translator, a health psychologist, and a speech and language pathologist), all bilingual in English and Maltese. A pilot testing was initially performed by completing and discussing the questionnaire with 5 patients on long-term oral anticoagulant treatment (not included in the analysis).

Study population

We administered the Maltese version of the PACT-Q2 to 174 patients receiving warfarin treatment. They were enrolled from the Anticoagulation Clinics at Mater Dei Hospital (Msida) and at 5 Health Centers (Cospicua, Floriana, Mosta, Qormi, Rabat) in Malta. Blood samples for INR testing at Mater Dei Hospital are collected using traditional venepuncture and INR is performed using laboratory coagulometers, whilst at the Health Centres INR is tested using point-of-care devices. Patients with cognitive impairment, dementia or major psychiatric disorders (such as schizophrenia) were excluded.

Two authors (NR, CBX) distributed the questionnaires between July 2017 and February 2018. Since we considered patients already receiving the anticoagulant treatment, only the PACT-Q2 was administered. Patients were also asked to complete a form on sociodemographic data. To ensure anonymity, questionnaires were identified using a code. In case of missing answers, the researchers associated the code with the provided demographic details and patients were contacted by phone.

During the same period, 157 patients on warfarin enrolled from the same Anticoagulation Clinics completed the original English version of the PACT-Q2.

A random sample of 40 patients underwent the following test-retest after 1–2 weeks (10 patients for each type): Maltese–Maltese; English–English; Maltese–English; English–Maltese.

This study was approved by the University of Malta Research and Ethics Committee (Ref No 07/2016) and all patients signed a written informed consent form before inclusion.

Statistical analysis

Continuous variables were reported as mean±standard deviation (SD), while categorical variables were reported as counts and percentages. Continuous variables were compared using the Student’s independent samples t-test, while categorical variables were compared using the Chi-square or the Fisher’s exact tests, as appropriate.

We evaluated the reliability of the Maltese version of the PACT-Q2 through internal consistency and test–retest.¹⁸ The Cronbach’s alpha coefficient was used to assess the internal consistency, with a value ≥0.70 indicating high internal consistency.¹⁹

A test–retest was performed to assess reproducibility, and we calculated the intraclass correlation coefficients (ICC) for the intra-language correlation (Maltese-Maltese and English–English test–retest). Values between 0.60 and 0.74 are considered acceptable.²⁰ For the cross-language correlation (Maltese–English and English–Maltese test-retest pooled together), we calculated the raw and the adjusted cross-language correlation (dividing the raw cross-language correlation by the square-root of the product of the intra-language correlations, to adjust for score unreliability).^21,22

We evaluated the validity of the Maltese translation of the PACT-Q2 through floor and ceiling effect, factor analysis, construct validity and known-group validity. Floor and ceiling effect occur when more than 15% of the respondents achieve the lowest or the highest possible score, respectively.²³

In the factor analysis, convergent and discriminant validity were evaluated. The convergent validity criterion was considered met when the correlation between each item and its dimension was ≥0.40, while the discriminant validity criterion was considered met when each item showed higher correlation with its dimension than the other.²⁴ We conducted an exploratory factor analysis with varimax rotation to examine the structure of the PACT-Q2. A subsequent confirmatory factor analysis provided the following fit parameters: root mean square error of approximation (RMSEA ≤0.05 good fit, ≤0.08 acceptable fit); standardized root mean squared residual (SRMR ≤0.05 good fit, ≤0.10 acceptable fit); goodness-of-fit index (GFI), adjusted goodness-of-fit index (AGFI) and comparative fit index (CFI), with values ≥0.90 considered acceptable.²⁵

To examine construct validity, the Pearson’s correlation between different subscales was assessed.²⁶ Known-group validity was assessed through Pearson’s correlation between the score of each PACT-Q2 subscale and the following variables: increasing age; male sex; living alone; primary school education only; paid employment; atrial fibrillation; anticoagulant treatment duration (>5 years); INR in the therapeutic range at enrolment; time-within-therapeutic-range (TTR, calculated according to the Rosendaal method)²⁷ ≥70% in the previous year; hospitalization in the previous year; history of any bleeding during anticoagulant treatment (self-reported).

A sample size of at least 150 patients was planned, since recommendations suggest at least 50 patients²³ and previous validation studies enrolled around 100 patients.^28,29

The statistical software STATA SE v.12 (StataCorp LP, College Station, TX, USA) and SAS v. 9.4 (SAS Institute Inc, Cary, NC, USA) were used for statistical analysis, with two-tailed p-values<0.05 considered statistically significant.

Results

Study population

Baseline characteristics of the study population are summarized in Table 1. The comparison between patients who completed the Maltese and the English version of the questionnaires has been already reported.

Table 1 Characteristics of patients who completed the Maltese version of the PACT-Q2

There was no difference in the mean convenience score between the two cohorts (mean±SD 82.2±16.1 for the Maltese version vs 84.0±13.7 for the English version, p=0.28), while patients who completed the Maltese version showed a trend toward lower satisfaction score (65.2±11.5 vs 67.6±14.6, respectively, p=0.09), corresponding to lower anticoagulant treatment satisfaction.

Internal consistency

The internal consistency of the Maltese translation of the PACT-Q2 was good for the convenience subscale with Cronbach’s alpha coefficient of 0.86. Cronbach’s alpha was 0.62 for the satisfaction subscale, which is slightly below the standard acceptable cut-off of 0.70. However, the satisfaction subscale has only 7 items and one item (D2) showed poor correlation with the overall satisfaction subscale, showing both a low item-total correlation of ~0.3 and an increase of Cronbach’s alpha when deleted. However, D2 corresponds to the question “Do you feel that your anticoagulant treatment has decreased your symptoms?”, which might have a negative answer also in satisfied patients, if anticoagulation is used for stroke prevention in atrial fibrillation or mechanical heart valves, and therefore does not have any impact on patients’ symptoms.

The English version of the PACT-Q2 also showed good internal consistency in our cohort with the following Cronbach’s alpha coefficients: 0.86 for the convenience subscale and 0.75 for the satisfaction subscale (Table 2).

Table 2 Internal consistency of the Maltese and English versions of the PACT-Q2

Reproducibility

In the Maltese–Maltese test–retest, the ICC for the intra-language correlation was very good for the convenience subscale (0.87), but low (0.40) for the satisfaction subscale. In the English–English test–retest, ICC was 0.87 for the convenience subscale and 0.60 for the satisfaction subscale (Table S1).

For the cross-language correlation, the corresponding ICC was 0.51 and 0.52 and the adjusted ICC was 0.59 and 1.06 for the convenience and satisfaction subscales, respectively. When analyzed separately, ICC for the English–Maltese test–retest was 0.76 and 0.68 for the convenience and satisfaction subscales, while ICC for the Maltese–English test–retest was 0.43 and 0.41, respectively.

Floor and ceiling effect

When we analyzed the response distribution for each item of the PACT-Q2, reversing the items of the convenience subscale, a significant ceiling effect was observed for most of the questions. A significant floor effect was found only for question D2. The Maltese and the English version of the PACT-Q2 showed similar results in our study (Table S2).

We subsequently analyzed the results of each PACT-Q2 subscale: for the Maltese version, ceiling effect was 6.3% for convenience and 1.2% for satisfaction; for the English version, ceiling effect was 9.6% for convenience and 1.3% for satisfaction. Floor effect was 0% for all subscales in both languages.

Factor analysis

Results of the confirmatory factor analysis were acceptable. For the Maltese version of the PACT-Q2, SRMR=0.10 was within the acceptable limits; RMSEA=0.09 was slightly above the reference, while GFI=0.82, AGFI=0.78 and CFI=0.79 were slightly below the reference values (Table 3).

Table 3 Results of the confirmatory factor analysis

The rotated factor pattern is reported in Table S3. The convergent validity criterion was met by all items of the Maltese PACT-Q2, except B10, B11, C2 (for the convenience subscale) and D2, D3 (for the satisfaction subscale). All items met the discriminant validity criterion.

Correlation scale-subscales

There was no correlation between the convenience and the satisfaction subscales in the Maltese version of the PACT-Q2 (r=0.01, p=0.83), while a weak positive correlation was found in the English version (r=0.33, p<0.001).

Known-group validity

The Maltese version of the PACT-Q2 showed a negative correlation with previous bleeding. The correlation was statistically significant for the satisfaction subscale and borderline for the convenience subscale (Table 4).

Table 4 Correlation between the PACT-Q2 and sociodemographic or clinical characteristics

In the English version of the PACT-Q2, the convenience subscale showed a significant positive correlation with increasing age and male sex, and a significant negative correlation with full- or part-time paid employment, hospitalization and history of bleeding. The subscale satisfaction gave similar results, which were statistically significant only for male sex.

These findings suggest that advanced age and male sex are associated with greater satisfaction/convenience, while paid employment, hospitalization and previous bleeding are associated with lower satisfaction/convenience.

Discussion

To the best of our knowledge, this is the first time that the PACT-Q2 has been translated and validated in the Maltese language. The results of our study suggest that the Maltese version of the PACT-Q2 is a valid and reliable instrument. The psychometric properties were very good for the convenience subscale, whereas they were slightly lower for the satisfaction subscale.

The PACT-Q is a specific questionnaire that evaluates the quality of life of anticoagulated patients through simple questions. It was rigorously developed, translated in several languages and used in a number of studies enrolling patients with different clinical indications.^14,30–32 While the PACT-Q1 assesses the expectations of the anticoagulant treatment, the PACT-Q2 evaluates the satisfaction and is used for patients already receiving the anticoagulant treatment. Patient-reported outcomes should always be considered, because of the relationship between low satisfaction, poor adherence and treatment failure.^33–35 In our study, we translated the PACT-Q2 in Maltese and administered it to 174 patients on warfarin for different clinical indications, including atrial fibrillation, heart valve replacement and venous thromboembolism. A peculiarity of our study is the fact that during the same time-frame, the original English version of the PACT-Q2 was administered to 157 patients at the same centers, therefore allowing a comparison of the psychometric properties. This study design was possible because Malta is a bilingual country where both Maltese and English are official languages.³⁶

We found that the reliability of the Maltese translation of the PACT-Q2 was very good for the convenience subscale (Cronbach’s alpha 0.86, ICC 0.87), while it was less so for the satisfaction subscale (Cronbach’s alpha 0.62, ICC 0.40). However, the satisfaction subscale showed lower reliability also in the English version of the PACT-Q2 (Cronbach’s alpha 0.75, ICC 0.60 in our study; Cronbach alpha 0.76 in the study by Prins et al).¹¹ This finding can be partly explained by the lower number of items (13 questions in the convenience subscale vs 7 questions in the satisfaction subscale) and partly by a response bias. Response bias is common in patient-reported outcomes and occurs when participants’ responses are influenced by their belief of which answers are socially acceptable or which answers are expected by the researchers.³⁷ The satisfaction subscale might have been particularly susceptible to response bias, due to the fact that several questions (D4-D7) ask directly the level of satisfaction with different aspects of the anticoagulant treatment (the level of independence, the appointments, the anticoagulant drug and the overall satisfaction). Participants might have felt more obliged to show that they were satisfied with the service, appointments, anticoagulant drug rather than risk reprisal on their treatment, even though the informed consent specified anonymity of data. Furthermore, although the retest was performed within two weeks, changes in the level of satisfaction might have occurred due to intercurrent clinical complications or differences of experience of service provision during following appointments for INR testing.

Validity of the Maltese translation of the PACT-Q2 was good. We observed a significant ceiling effect for most of the PACT-Q2 items when analyzed individually. However, when we considered the two subscales, floor effect was 0% and ceiling effect did not exceed 10% (being 6.3% for convenience and 1.2% for satisfaction). These results were even better than the original study of the PACT-Q2 which reported a ceiling effect of 22.1% for convenience and 3.3% for satisfaction.¹¹ Results of the factor analysis were good, with fit parameters close to the acceptable cut-offs. Furthermore, all items met the discriminant validity criterion, while the convergent validity was met by all items except B10, B11, C2 (convenience subscale) and D2, D3 (anticoagulant treatment satisfaction subscale). Although previous studies did not report the fit parameters (RMSEA, SRMR, GFI, AGFI and CFI), items B10-B11 and D2-D3 did not meet the convergent validity criterion also in the original study by Prins et al.¹¹ Correlation between the two PACT-Q2 subscales was weak, as previously reported,¹¹ confirming that they cover different dimensions. The results of the known-group validity analysis showed that patients with history of bleeding had lower satisfaction. Although previous validation studies of the PACT-Q did not evaluate this group,^11,12 lower scores on the convenience dimension of the PACT-Q2 were reported after bleeding events in a prospective study enrolling 807 atrial fibrillation patients on warfarin.³¹ Furthermore, lower satisfaction in anticoagulated patients with history of bleeding was already reported in validation studies of other specific questionnaires.^7,38

Our study population shows some differences when compared to previous PACT-Q validation studies. Mean age was older (70 years), compared to 65 years in the study by Prins et al¹¹ and 58 years in the study by Mohamed et al.¹² We enrolled patients on oral anticoagulant treatment with VKA, while Prins et al¹¹ considered also patients on treatment with idraparinux, which is a parenteral drug injected subcutaneously once weekly. Finally, we had a higher proportion of patients with primary school level of education (62%), compared to the study by Mohamed et al¹² where only 28% had only primary school education or no education at all.

Our study has also some limitations which need to be acknowledged. First, we enrolled patients who were already on anticoagulant treatment; therefore, we could validate only the PACT-Q2. Second, although the PACT-Q has been developed for patients receiving different types of anticoagulants (oral or parenteral),⁶ we enrolled only patients on warfarin which was the main oral anticoagulant treatment in Malta at the time of patients enrolment. Nonetheless, the strengths of our study include the completeness of data, without any missing answers, and the rigorous process of translation and analysis.

Conclusion

Our results support the finding that the Maltese translation of the PACT-Q2 is a valid and reliable instrument, which can be used by health-care professionals when assessing Maltese-speaking anticoagulated patients.