The Oxford Participation and Activities Questionnaire (Ox-PAQ): development of a short form and index measure

Background

The Oxford Participation and Activities Questionnaire (Ox-PAQ) is a 23-item patient-reported outcome measure (PROM), theoretically grounded in the World Health Organisation International Classification of Functioning, Disability and Health.¹ The measure is a generic PROM. It is intended primary use is in clinical trials, as well as in the evaluation of interventions targeted at improving and/or maintaining participation and activity. However, as a generic measure it permits comparison across treatment groups, and populations.

The development of the Ox-PAQ has been reported elsewhere.^2–8 A key feature of the development process is its compliance with current best practice guidelines, such as those provided by the United States Food and Drug Administration (FDA) and European Medicines Agency (EMA).^9,10 The content of the measure was based upon interviews with lay people diagnosed with a variety of illness conditions. Subsequent exploratory factor analyses and validation of the measure in patient surveys^6–8 identified three domains; Routine Activities (14 items), Emotional Well-Being (5 items) and Social Engagement (4 items). Analyses indicated that the measure demonstrates good reliability and validity and low levels of missing data across all three domains. Furthermore, the operating characteristics of the instrument have been found to be stable across paper and computer forms of administration^7,8 To date no evidence has been presented suggesting an overall index score could be calculated for the measure. Such an index may prove useful as a general summary of the health of respondents, and can reduce the role of chance, due to multiple comparisons when analyses include all individual dimensions of a PROM, in trials where the measure is adopted as a secondary outcome. It is hypothesised here that a single index can be derived for the Ox-PAQ and results from this Index can also be obtained from a shorter form version. It is also hypothesised that results from the two measures would be virtually identical, and demonstrate a very similar level of correlation with another generic patient outcome measure. The methods adopted have been used on a variety of instruments to both provide summary scores and make them more economic in nature (and hence less of a burden to patients).^11–13

The purpose of this paper is to report (i) procedures undertaken to determine if the three dimensions of the Ox-PAQ can meaningfully be summed into a single index measure, and (ii) to determine if a subset of items will replicate the results of the single index but in a shorter measure. Two studies are reported, based upon two separate surveys, one detailing the development of the index score and Short Form Ox-PAQ (SF-Ox-PAQ), and a second to determine that the two Ox-PAQ measures have similar operating characteristics, in terms of convergent validity, by comparing results on the two measures with another generic index score of health, the EQ-5D-5L.¹⁴

Study 1: the development of the Ox-PAQ index and SF-Ox-PAQ

Methods

Ethical approval for the data collected in the analyses reported here was granted by the Medical Sciences Inter Divisional Research Ethics Committee of the University of Oxford (ref: R44281/RE001).

Participants

Three hundred and seventy three people with a primary diagnosis of chronic obstructive pulmonary disease (COPD), multiple sclerosis (MS) or Parkinson’s disease (PD) were recruited via their respective support organisations in the United Kingdom; the British Lung Foundation, MS Society and Parkinson’s UK. Informed consent to participate in the study was obtained from each participant. Participants must have had a confirmed diagnosis of the relevant condition, be 18 years of age or over and competent in the use of English.

Materials

Procedure

Recruitment was undertaken via adverts on the relevant charities website, and potential participants contacted a member of the research team via email. They were consented into the study and subsequently forwarded a unique link to the survey. An e-based version of the Ox-PAQ was administered via Qualtrics survey software.¹⁵ Demographic information (gender, age, primary condition, age at diagnosis, marital status, employment status and ethnicity) was collected prior to completion of the Ox-PAQ.

Statistical analysis

Data were checked for the presence of outliers and normality of distribution prior to statistical analysis. Furthermore, tests of significance were conducted to determine that factor analytic procedures could be meaningfully be carried out on the data.¹⁶ The three domain scores of the Ox-PAQ were subjected to higher order principal components analysis. Reliability was assessed via Cronbach’s alpha.¹⁷ A single score was calculated from the Ox-PAQ by summing all three domains and expressing results on a metric of 0–100. Higher scores indicate greater problems with activity and participation. Linear regression of items for each of the three Ox-PAQ dimensions were separately run against the Index to determine those which were significantly predictive of the Index. These items were summed to create an Index and this “Short Form” Index was compared to the parent version using the intra-class correlation coefficient (ICC). Due to the nature of the analyses, i.e. validating a parent measure against a shorter form, any respondents with missing data were removed. Data were analysed using SPSS Version 23.

Results

Three hundred and forty people fully completed the Ox-PAQ, a response rate of 91.4%. The sample comprised 140 males and 199 females, had a combined mean age of 61.52, a mean age at diagnosis of 52.51 and mean disease duration of 9.01 years. Detailed sample characteristics by disease group are reported in Table 1.

Table 1 Sample characteristics

The Kaiser-Meyer Olkin (KMO) and Bartlett’s Test measure of sampling adequacy were used to examine the appropriateness of factor analytic techniques. Chi-square was 590.04 with 3 degrees of freedom, which is significant at 0.001 level. The KMO statistic of 0.70 is also acceptable. Hence factor analysis is as an appropriate technique for further analysis of the data. In the principal components analysis one factor with an Eigenvalue in excess of one was identified, accounting for 80.52% of the variance. Each domain of the Ox-PAQ loaded on this one factor which had an eigenvalue of 2.42. Consequently, all three domains of the Ox-PAQ were summed to create a summary index score. The mean of the Summary Index figure was 43.16. (S.D. =23.66, min =0.00, max =96.73, n=340). Cronbach’s alpha value was calculated as 0.88.

Three linear regressions were undertaken of items in the three dimensions of the Ox-PAQ against the Summary Index. Seven items from the Routine Activity dimension were found to be statistically significant predictors of the Index, with an adjusted R² of 0.88. Three items from the Emotional Well-Being scale were found to be predictors of the Summary Index (R² = 0.77). The regression of items in the Social Engagement dimension indicated all four to be predictive of the Single Index (R²=0.88), leading to a total of 14 items (see Table 2). Cronbach’s alpha for the 14 items was calculated as 0.93. Results gained from the two indices are compared in Table 3 and indicate the Short Form replicates the results of the parent form to a very high degree. Furthermore, intra-class correlation coefficients of the Ox-PAQ Index and Short Form Ox-PAQ Index were found to be very high: ICC =0.99 (p<0.001) for each condition.

Table 2 Items found to be significantly predictive (p≤0.01) of their own scale scores in three independent linear regressions

Table 3 Comparison of results from the Ox-PAQ summary index against those derived from the SF-Ox-PAQ Summary Index across three conditions

Study 2: validation of the Ox-PAQ index and short form and comparison with results from the EQ-5D

Methods

Ethical approval for this aspect of the Ox-PAQ study was granted by the Medical Sciences Inter Divisional Research Ethics Committee of the University of Oxford (reference MSD-IDREC-C1-2014–089).

Participants

Recruitment of participants was undertaken over a period of 6 months with the assistance of three patient support organizations: the Motor Neurone Disease Association, MS Society, and Parkinson’s UK. The organizations advertised the study through various means, including social media, Web sites, print and electronic publications, research bulletin boards, and emails, inviting potential participants to contact the research team to express their interest in taking part. Participants were required to have a confirmed diagnosis of motor neurone disease (MND), multiple sclerosis (MS), or Parkinson’s disease (PD), as well as the ability to complete the survey independently. Participants were also required to be competent in the use of English, be aged 18 years or above, and be living in the UK.

Procedure

The survey booklet which contained questions on demographic variables (including sex, age, age at diagnosis), the Ox-PAQ (as detailed earlier), and the EQ-5D-5L. The EQ-5D-5L¹² is a five-item generic measure assessing mobility, self-care, usual activities, pain/discomfort, and anxiety/depression. Initially developed with questions answered on a three-point Likert scale, a revised version of the measure now incorporates a five-point Likert scale. The EQ-5D-5L includes a visual analogue scale (VAS) to indicate general health, with a score of zero reflecting worst health status and 100 indicating the best possible health status. Recent studies^18–20 suggest that the updated measure is both valid and reliable.

After contacting the research team by telephone or email, participants were sent the booklet of questionnaires and a written consent form for completion and return. A follow-up email or letter was sent to non-responders after 2 weeks.

Statistical analysis

A single Score was calculated for the Ox-PAQ and the SF-Ox-PAQ. Descriptive statistics were calculated for both measures, and results compared by intra-class correlation coefficient. Results were then correlated (Spearman’s rho) with results on the EQ-5D-5L Index and Visual Analogue Scale (VAS).

Results

A total of 334 participants completed the postal survey, a response rate of 86.5%. 97 (29.04%) respondents reported a diagnosis of MND, 100 (29.94%) a diagnosis of MS and 137 (41.02%) a diagnosis of PD. Mean age was 60.06 years (SD 12.10; range 24–88), mean age at diagnosis 52.82 years (SD 14.50; range 18–87) and mean disease duration 7.31 years (SD 7.52; range 0–50). The sample comprised 162 males (48.5%) and 172 females (51.5%). Mean scores for the measures for the entire sample were found to be very similar: a mean of 42.14 (SD 24.40, 39.42–44.86, n=309) on the Ox-PAQ compared to a mean of 42.67 (SD 24.43, 95% CI 39.95–45.39, n=309) on the SF-Ox-PAQ. A similar picture emerged when analyses were broken down by condition with scores on the two measures never varying by more than one point. As in the development survey reported above the two measures were found to be highly correlated (ICC=0.99, p<0.001, n=309). Furthermore, the measures were found to exhibit similar characteristics when compared to the EQ-5D. The Ox-PAQ and SF-Ox-PAQ were correlated with the EQ-5D-5L rho= −0.81 (p<0.001, n=308) and −0.80 (p<0.001, n=308) respectively (negative correlations are due to the fact that scoring conventions differ with high scores on the EQ-5D-5L indicating good health and high scores on the Ox-PAQ indicating poor health). A similar picture emerged when comparing results on the two measures with the EQ-5D VAS with the Ox-PAQ and SF-Ox-PAQ correlated with the VAS rho=−0.71 (p<0.001, n=307) and 0.70 (p<0.001, n=307) respectively.

Discussion

Single indices have been found to be useful summary scores of the overall impact of health conditions, whilst shorter form measures reduce respondent burden.^21,22 The results reported here suggest that a single index can be meaningfully gained from the Ox-PAQ. Higher order factor analysis supports the derivation of a single index summary score for the measure. Reliability of the Summary Index was assessed using Cronbach’s alpha and found to be high indicating that the index created by summing the three domains is internally consistent and reproducible. The construct validity of the measure was assessed by comparing the results of the Ox-PAQ Summary Index with the EQ-5D-5L.

Data from the Ox-PAQ can be presented in profile form as well as summarised in the Ox-PAQ Summary Index. Thus, the impact of treatment on specific domains of well-being and functioning can be evaluated with the profile scores, whilst the overall impact of the disease on participation and activities, as measured by the Ox-PAQ, can be assessed using the Summary Index. Such an index has the potential for use in the evaluation of different treatments as interpretation of a single figure can often be less complex than that of a profile of scores. Furthermore, the adoption of a single index measure of outcome can reduce the number of statistical comparisons and consequently reduce the role of chance in testing hypotheses about health outcomes.

Regression analyses suggested a sub-set of items from the Ox-PAQ could create a Short Form Ox-PAQ that can provide the Summary Index. The operating characteristics of the two Ox-PAQ Summary Indices were evaluated by assessing the construct validity against the EQ-5D-5L Index and VAS. Virtually identical correlations were found for the Ox-PAQ Single Index and SF-Ox-PAQ Single Index against the EQ-5D-5L scores.

It is recognised that the methods of recruitment for the reported studies were self-selecting in nature, and the samples may not therefore be fully representative of the illness groups that participated. Additionally it is acknowledged that the samples recruited for electronic completion of the measures may not be entirely representative of their illness groups, as not all will have access to electronically administered measures or be computer literate.

In conclusion, in instances where a detailed profile of scores is required the SF-Ox-PAQ will not be the instrument of choice, but in instances where an overall index is required, or where the additional burden of the Ox-PAQ is unacceptable, the SF-Ox-PAQ represents an economical and informative instrument. Further studies will provide evidence of the psychometric properties in additional disease groups.

Copies of the Ox-PAQ, a User Manual and licences for its use can be obtained from Clinical Outcomes at Oxford University Innovation: https://innovation.ox.ac.uk/clinical-outcomes/.