Administering the Routine Activities domain of the Oxford Participation and Activities Questionnaire as a stand-alone scale: the Oxford Routine Activities Measure

Introduction

The recently validated Oxford Participation and Activities Questionnaire (Ox-PAQ) is a 23-item patient-reported outcome measure, theoretically grounded in the World Health Organization International Classification of Functioning, Disability and Health.¹ The measure is specifically designed to assess participation and activity in people experiencing a range of health conditions. Its intended primary use is in clinical trials, as well as in the evaluation of interventions targeted at improving and/or maintaining participation and activity.

The development of the Ox-PAQ has been extensively reported elsewhere, which readers may wish to refer to.^2–4 A key feature of the development process is its compliance with current best practice guidelines, such as those provided by the US Food and Drug Administration and European Medicines Agency.^5,6 Subsequent validation of the measure^7–9 identified three domains: Routine Activities (14 items), Emotional Well-Being (5 items), and Social Engagement (4 items). Analyses indicated that the measure demonstrates sound reliability (internal and external) and validity (concurrent and known-groups).

The Ox-PAQ provides a broad assessment of health and well-being in terms of activities and participation. It is likely, however, that for many trials the Routine Activities domain will be an important or primary endpoint on which they may be powered. As is evident from current literature, the purpose of many interventions is to maximize the ability of participants to engage in routine activities for as long as possible, or to restore them to a level of functional ability that permits greater engagement in routine activities.^10–13

The omission of domains when administering previously validated multidomain instruments is generally discouraged in survey research. Doing so has the potential to disrupt their measurement properties, and thereby produce misleading or inaccurate results. A potential solution, however, is to assess the psychometric properties of a given domain as a stand-alone scale in its own right. Therefore, the purpose of the analysis reported here was to assess whether the Routine Activities domain of Ox-PAQ can be utilized as a stand-alone measure without compromising its psychometric integrity. From the responses of a sample independent of the initial validation survey, it was hypothesized that the 14 items would load onto one single factor and would demonstrate acceptable reliability and validity.

Methods

Ethical approval and consent

Ethical approval for the data collected in the analyses reported here was granted by the Medical Sciences Inter Divisional Research Ethics Committee of the University of Oxford (reference: R44281/RE001). Informed consent was gained electronically from all participants prior to their completion of the e-based survey detailed in this report.

Participants

Three hundred seventy-three people with a diagnosis of either chronic obstructive pulmonary disease (COPD), multiple sclerosis (MS), or Parkinson’s disease (PD) were recruited via their respective support organizations in the UK – the British Lung Foundation, MS Society, and Parkinson’s UK.

Materials

An e-based version of the Ox-PAQ was administered via Qualtrics survey software¹⁴ as part of a wider study assessing the responsiveness of the measure. Having been migrated from its paper-based equivalent, the e-based Ox-PAQ has been shown to be both usable and acceptable to respondents.¹⁵ Higher Ox-PAQ scores indicate greater problems with participation and activity.

Procedure

In response to advertisements placed by the support organizations identified above, participants contacted a member of the research team via email and were subsequently forwarded a unique link to the survey. Demographic information (gender, age, primary condition, age at diagnosis, marital status, employment status, and ethnicity) was collected prior to completion of the Ox-PAQ.

Statistical analysis

As is required with psychometric analyses, respondents with missing Ox-PAQ data were removed from the analysis. The 14 items of the Routine Activities domain were subject to factor analytic techniques using IBM SPSS Statistics Version 24.¹⁶ The Kaiser–Meyer–Olkin (KMO)^17,18 measure of sampling adequacy and Bartlett’s test of sphericity¹⁹ were calculated to ensure that the use of principal components analysis (PCA) was acceptable. PCA was subsequently performed, alongside calculation of eigenvalues and explained variance, in order to confirm the factor structure of the 14 items. Reliability was assessed through calculation of item-total correlations (ITCs) and Cronbach’s alpha value.²⁰ Validity was assessed through analysis of variance (ANOVA) and Tukey’s post-hoc tests²¹ as an assessment of known-groups validity.

Results

Three hundred and forty-one people fully completed the Ox-PAQ, a completion rate of 91.4%. The sample comprised 141 males and 200 females, had a combined mean age of 61.52, a mean age at diagnosis of 52.51 and mean disease duration of 9.01 years. Detailed sample characteristics by disease group can be viewed in Table 1.

Table 1 Sample characteristics by disease group and total sample Abbreviations: COPD, chronic obstructive pulmonary disease; MS, multiple sclerosis; PD, Parkinson’s disease; SD, standard deviation.

Factor structure

The data met the criteria for the use of PCA. Sampling adequacy was high (KMO=0.962), and Bartlett’s test of sphericity calculated the model (PCA) to be appropriate (χ²=4368.97, df=91, P=0.00). PCA confirmed that the 14 items loaded onto one single factor with an eigenvalue of 9.29 explaining 66.35% of variance. Factor loadings ranged from 0.902 to 0.635, with full details available in Table 2.

Table 2 Factor loadings and item-total correlations of the Ox-PAQ Routine Activities domain by item (presented by size) Notes: aItem numbers correspond with the 23-item Ox-PAQ. Adapted from Morley D, Dummett S, Kelly L, Dawson J, Fitzpatrick R, Jenkinson C. Validation of the Oxford Participation and Activities Questionnaire. Patient Relat Outcome Meas. 2016;7:73–80. Copyright © 2016 Morley et al.7 Abreviation: Ox-PAQ, Oxford Participation and Activities Questionnaire.

Reliability

Corrected ITCs ranged from 0.880 to 0.594, with full details available in Table 2. Cronbach’s alpha value was calculated at 0.96.

Validity

Mean scores and standard deviations of the Ox-PAQ Routine Activities domain by disease group are given in Table 3. ANOVA results indicate statistically significant differences between the three conditions; F(2, 338)=41.90, P<0.001; Post-hoc tests (Tukey’s HSD) at the 0.05 level of significance confirm significantly inferior scores for people with COPD when compared with those with PD (P<0.001), alongside significantly inferior scores for people with MS when compared with those with PD (P<0.001).

Table 3 Mean scores and standard deviations of the Ox-PAQ Routine Activities domain by disease group Abbreviations: COPD, chronic obstructive pulmonary disease; MS, multiple sclerosis; Ox-PAQ, Oxford Participation and Activities Questionnaire; PD, Parkinson’s disease.

Discussion

Results presented indicate that where researchers wish to focus primarily on the Routine Activities component of the Ox-PAQ, the Oxford Routine Activities Measure (Ox-RAM) can be legitimately incorporated as a stand-alone scale. As hypothesized, all 14 items loaded onto a single factor, explaining 66.35% of variance, a level typically seen when validating instruments through the use of PCA.^22,23 Additionally, all factor loadings are above the level of 0.6 and therefore regarded as high.²⁴

Reliability of the Ox-RAM is confirmed through ITCs that are in excess of previously defined criteria,²⁵ confirming that individual item scores are related to the overall Ox-RAM score. Cronbach’s alpha value was calculated at 0.96, thereby indicating strong internal consistency reliability.²⁶

The validity of the Ox-RAM is demonstrated through assessment of known-groups validity. Such an assessment is made where there are good reasons to hypothesize that scores on a measure of interest will differ between groups,²⁷ as has been incorporated in previous research.^28–31 ANOVA results confirm previously reported differences between people with Parkinson’s and those with MS.^7,31,32 No previous comparisons between neurological conditions and COPD exist, but it is evident that the Ox-RAM identifies highly significant differences between people with PD and those with COPD.

While results confirm the legitimate use of the Ox-RAM, it should be emphasized that where a detailed assessment of all areas of activity and participation highlighted as important during the development of the Ox-PAQ is required, all three domains of the full measure should be administered. The testing of further psychometric properties is required for both the Ox-RAM and the Ox-PAQ, with analysis of sensitivity to change currently being undertaken. Information regarding the use of the Ox-RAM and Ox-PAQ can be obtained from DM or CJ.

Acknowledgments

Development and validation of the Ox-PAQ were funded by the European Brain Council. We would also like to thank Actelion Pharmaceuticals Ltd for an unrestricted educational research grant, which was used to support part of this work.

The analysis reported here was presented at the 23rd Annual Conference of the International Society for Quality of Life Research, Copenhagen, Denmark, October 19–22, 2016 and was published in Quality of Life Research, Volume 25, Supplement 1.

Author contributions

All authors contributed toward data analysis, drafting and critically revising the paper, gave final approval of the version to be published, and agree to be accountable for all aspects of the work.

Disclosure

All authors are developers of the Ox-PAQ and will receive royalties from its use in commercial contexts. All authors have also undertaken consultancy work for Oxford University Innovation Ltd which holds the license for the Ox-PAQ. The authors report no other conflicts of interest in this work.

References

1.		World Health Organisation. International Classification of Functioning, Disability and Health. Geneva: World Health Organisation; 2001.
2.		Kelly L, Jenkinson C, Dummett S, Dawson J, Fitzpatrick R, Morley D. Development of the Oxford Participation and Activities Questionnaire: constructing an item pool. Patient Relat Outcome Meas. 2015;6:145–155.
3.		Kelly L, Dummett S, Dawson J, Fitzpatrick R, Jenkinson C, Morley D. Generating items for the Oxford Participation and Activities Questionnaire (Ox-PAQ). Qual Life Res. 2014;23(Suppl 1):81–82.
4.		Morley D, Dummett S, Kelly L, Dawson J, Fitzpatrick R, Jenkinson C. Pretesting the Oxford Participation and Activities Questionnaire: results from an expert review. Mov Disord. 2015;30(Suppl 1):S419.
5.		Food and Drug Administration, Department of Health and Human Sciences. Guidance to Industry. Patient Reported Outcome Measures. Use in Medical Product Development to Support Labelling Claims. Silver Spring, MD: Food and Drug Administration; 2009.
6.		EMA. Reflection paper on the regulatory guidance for the use of health-related quality of life (HRQL) measures in the evaluation of medicinal products; 2004. Available from: http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2009/09/WC500003637.pdf. Accessed November 1, 2017.
7.		Morley D, Dummett S, Kelly L, Dawson J, Fitzpatrick R, Jenkinson C. Validation of the Oxford Participation and Activities Questionnaire. Patient Relat Outcome Meas. 2016;7:73–80.
8.		Morley D, Dummett S, Dawson J, et al. The Oxford Participation and Activities Questionnaire (Ox-PAQ): psychometric analysis. Value Health. 2015;18(7):A742.
9.		Kelly L, Dummett S, Dawson J, Fitzpatrick R, Jenkinson C, Morley D. Psychometric evaluation of the Oxford participation and activities questionnaire (Ox-PAQ). J Neurol Sci. 2015;357(357 Suppl 1):e273.
10.		Sackley CM, Walker MF, Burton CR, et al. An occupational therapy intervention for residents with stroke related disabilities in UK care homes (OTCH): cluster randomised controlled trial. BMJ. 2015;350: h468.
11.		Rodgers H, Shaw L, Cant R, et al. Evaluating an extended rehabilitation service for stroke patients (EXTRAS): study protocol for a randomised controlled trial. Trials. 2015;16:205.
12.		Desveaux L, Beauchamp MK, Lee A, Ivers N, Goldstein R, Brooks D. Effects of a Community-Based, Post-Rehabilitation Exercise Program in COPD: Protocol for a Randomized Controlled Trial With Embedded Process Evaluation. JMIR Res Protoc. 2016;5(2):e63.
13.		Hornslien AG, Sandset EC, Wyller TB, Berge E. Scandinavian Candesartan Acute Stroke Trial (SCAST) Study Group. Effects of candesartan in acute stroke on activities of daily living and level of care at 6 months. J Hypertens. 2015;33(7):1487–1491.
14.		Qualtrics, 2005. Copyright Year: 2017. Provo, UT, USA.
15.		Kelly L, Dummett S, Dawson J, et al. Pretesting An E-Based Version Of The Oxford Participation & Activities Questionnaire (Ox-Paq). Value Health. 2016;19(7):A485.
16.		IBMCR. IBM SPSS Statistics for Windows, Version 24.0. Armonk, NY: IBM Corp; 2016.
17.		Kaiser HF. A second generation little jiffy. Psychometrika. 1970;35(4):401–415.
18.		Kaiser HF, Rice J, Little Jiffy MIV. Educ Psychol Meas. 1974;34(1):111–117.
19.		Bartlett MS. Tests of significance in factor analysis. British Journal of Statistical Psychology. 1950;3(2):77–85.
20.		Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16(3):297–334.
21.		Tukey JW. Comparing individual means in the analysis of variance. Biometrics. 1949;5(2):99–114.
22.		Peto V, Jenkinson C, Fitzpatrick R, Greenhall R. The development and validation of a short measure of functioning and well being for individuals with Parkinson’s disease. Qual Life Res. 1995;4(3):241–248.
23.		Jenkinson C, Fitzpatrick R, Brennan C, Bromberg M, Swash M. The amyotrophic lateral sclerosis assessment questionnaire (ALSAQ-40): tests of data quality, score reliability and response rate in a survey of patients. J Neurol. 1999;246(Suppl 3):III16–III21.
24.		Kline P. A general description of factor analysis. In: Kline P, editor. An Easy Guide to Factor Analysis. London: Routledge; 1994:1–13.
25.		Estabrooks CA, Squires JE, Hayduk LA, Cummings GG, Norton PG. Advancing the argument for validity of the Alberta Context Tool with healthcare aides in residential long-term care. BMC Med Res Methodol. 2011;11:107.
26.		Aaronson N, Alonso J, Burnam A, et al. Assessing health status and quality-of-life instruments: attributes and review criteria. Qual Life Res. 2002;11(3):193–205.
27.		Brazier J, Deverill M. A checklist for judging preference-based measures of health related quality of life: learning from psychometrics. Health Econ. 1999;8(1):41–51.
28.		Morley D, Selai C, Schrag A, Thompson AJ, Jahanshahi M. Refinement and validation of the Parental Illness Impact Scale. Parkinsonism Relat Disord. 2010;16(3):181–185.
29.		Papaioannou D, Brazier J, Parry G. How valid and responsive are generic health status measures, such as EQ-5D and SF-36, in schizophrenia? A systematic review. Value Health. 2011;14(6):907–920.
30.		Morley D, Dummett S, Kelly L, Dawson J, Jenkinson C. Evaluating the psychometric properties of an e-based version of the 39-item Parkinson’s Disease Questionnaire. Health Qual Life Outcomes. 2015; 13:5.
31.		Riazi A, Hobart JC, Lamping DL, et al. Using the SF-36 measure to compare the health impact of multiple sclerosis and Parkinson’s disease with normal population health profiles. J Neurol Neurosurg Psychiatry. 2003;74(6):710–714.
32.		Morley D, Selai C, Thompson A. Quality of life and psychosocial well-being in people with multiple sclerosis and Parkinson’s disease. 2007 International Society for Quality of Life Research Meeting Abstracts. Qual Life Res. 2007;A-114–Abstract #1393.