Development of the Oxford Participation and Activities Questionnaire: constructing an item pool

Introduction

There is a growing interest in the management of long-term conditions and keeping people active and participating in daily life.^1–4 Testing the effectiveness of interventions that aim to impact on activities and participation, however, can be challenging. In order to accurately evaluate these constructs, it is necessary to use a valid and reliable instrument that has both an explicit theoretical foundation and an empirical evidence supporting its measurement properties.^5–8 It is therefore crucial that the meaning of concepts such as “activities” and “participation” are clear in relation to the population being assessed.

Despite widespread debate in the academic literature regarding how to define “participation” and “activity”,^9–11 the World Health Organization (WHO) International Classification of Functioning, Disability, and Health (ICF)¹² provides the most widely used framework within which to define activity and participation. Activity is described as “the execution of a task or action by an individual”, while participation is described as “involvement in life situations”. Although initially described as two separate concepts, the final version of the ICF merges activity and participation into a single taxonomy which consists of nine domains (Table 1).

Table 1 ICF classification of participation and activity Abbreviation: ICF, International Classification of Functioning, Disability, and Health.

Definitions and applications of the ICF domains have been explored across a range of health conditions. Expert consensus using the Delphi technique has largely supported the use of ICF categories in neurological conditions such as spinal cord injury and multiple sclerosis^13,14 and other chronic conditions like rheumatoid arthritis, ischemic heart disease, and chronic pain.^15–17 The appropriateness of the ICF for use across a range of condition groups has also been supported through direct research among patient groups.^18,19 These studies have found some condition-specific areas, for example, glucose control and self-management for people with diabetes, less well represented within the ICF coding framework; however, this is somewhat expected in a generic tool kit.

The ICF has been widely reported, and as a consequence, there are a number of instruments available that aim to reflect the ICF framework. However, existing measures have not been developed using best practice guidelines, such as those laid down by the US Food and Drug Administration (FDA).⁷ At the initial stages of instrument development, FDA guidance stipulates that a clear and empirically informed conceptual framework must be used to inform items. Importantly, this empirical evidence must be gathered within the target population. The psychometric properties of the instrument must subsequently be assessed with an appropriate sample. A number of reviews and clinical commentaries have highlighted significant limitations in a number of these instruments,^20–25 including the need for evidence regarding adequate psychometric properties. The Oxford Participation and Activities Questionnaire (Ox-PAQ) initiative aims to address the need for a fully FDA-compliant patient-reported outcome measure (PROM) to assess participation and activity in patients experiencing a range of health conditions. It was envisioned that at least one item which defined each domain would be developed based on the patients’ perspective and language.

This paper reports the development of a generic item pool to inform the new measure, the Ox-PAQ. An important aspect of this process was the use of the patient perspective as this can highlight issues not found in the literature or issues that might not be considered relevant by health care professionals. Patient involvement also ensures that questions and responses are both appropriate and understandable to the potential respondent group.^5,7

Materials and methods

Several steps were taken to construct items relevant to activities and participation, according to the nine domains outlined in the ICF. In-depth interviews were conducted to explore how health influenced activities and participation in everyday life. Following a period of qualitative analysis of interview transcripts, statements were selected to represent ICF themes and recast as questionnaire items. Items were refined using both expert and patient opinion and a translatability assessment. The Ox-PAQ protocol is published elsewhere.²⁶ Ethical approval for the study was granted by the Medical Sciences Inter Divisional Research Ethics Committee of the University of Oxford (reference MSD-IDREC-C1-2013-064).

Sample

Participants were required to be living in the UK, at least 18 years old, and have a confirmed diagnosis in one of the following conditions: arthritis, cancer, chronic back pain, diabetes, motor neuron disease, multiple sclerosis, Parkinson’s disease, or spinal cord injury. These patient groups were selected to represent a range of conditions that affect key bodily systems, as well as have different symptoms, trajectories, and prognoses.

In-depth interviews

In-depth interviews were conducted to explore aspects of health that had an impact on participation and activities. The use of qualitative interviewing enabled important topics relating to a participant’s experience of health to be explored and helped to capture participants’ own language when discussing health. A topic schedule was devised, on the basis of a previous literature review and discussion among the authors, to guide the discussion and included prompts relating to the impact of health on daily routine and activities both inside the home (eg, general tasks, self-care, mobility, finances) and outside the home (eg, transport, travel, mobility). The impact of health on participation in social settings, work life, and relationships were also explored. Patients were free to discuss any other topics that they felt were relevant. Interview questions were not devised and presented in a manner that sequentially explored each ICF domain as it was important to allow for flexibility in the interview for areas of importance to the participant. This also facilitated the possibility for new topics to emerge during the course of the interview where relevant.

Potential participants were identified through relevant health care charities and local organizations. Recruitment information about the study was sent via participating charities to their members and/or advertised on their Internet message boards. Recruitment also took place using existing participants, where they suggested peers who might wish to be involved in the research. Participants were required to sign a consent form in order to take part.

Interviews took place in the participant’s own home or another preferred location, for example, their workplace. Interviews lasted for approximately 1 hour and were recorded and transcribed. Transcription of the interviews was outsourced, and their accuracy was checked by the research team on their return.

Analysis

Interview transcripts were analyzed using a modified version of the “Framework” method. Framework analysis is systematic and involves five stages: 1) familiarizing with the data gathered; 2) identifying a thematic framework; 3) indexing the transcripts according to the thematic framework; 4) charting the data to allow within-case and between-case comparison; and 5) mapping and interpreting data.^27–29 All members of the team read and discussed transcripts. Coding consistency was checked by SD, JD, LK, and DM. LK used the NVIVO computer software³⁰ to index transcripts. Coding was carried out using both an inductive (codes emerging during analysis) and a deductive (codes anticipated) approach. Charting was carried out using an EXCEL worksheet in which key topics from the interviews that related to activities and participation were placed on the horizontal tabs and the participant identification number was placed on the vertical tabs.

Representation of ICF domains and identifying generic statements

All authors reviewed the EXCEL charting document and constructed a set of statements that were applicable to the nine ICF domains. Following a period of consultation between the authors, the statements were amalgamated and recast as questionnaire items. Each item was checked for its applicability to people with different health conditions. A frequency response scale (never–always) was considered appropriate to the item stems and was reviewed during expert and patient refinements.

Expert refinement

Items were reviewed by an expert advisory board consisting of eleven clinicians and academics familiar with the ICF domains. Members of this group were asked whether they thought the domains appropriately addressed areas identified in the ICF, the appropriateness of the items, and the questionnaire layout. They were also asked to comment more generally on the questionnaire. Comments were collated and reviewed by DM and SD. All reviews took place via email.

Patient refinement and item translatability

Two rounds of cognitive interviewing were carried out to explore how respondents understood and answered the candidate items with the aim of improving the validity and acceptability of the questionnaire.^31,32 At the time of the initial in-depth interviews, participants were asked if they would be willing to take part in an interview to assess the items. Thirteen participants were subsequently interviewed (seven in round one, six in round two) and were chosen to reflect the spectrum of health conditions sampled. The “verbal probing” method was used during interviewing, which requires participants to complete the questionnaire unaided, followed by a focused interview.^31,33 Participants then explained the reasons for their answers to each item and commented on any ambiguities. This method of interviewing allowed the interviewer to query a participant’s understanding of an item and their interpretation of the instructions and response options.²⁷

After the first round of cognitive interviewing, a “translatability assessment” was carried out in collaboration with experts in the field. Translatability assessments evaluate the extent to which a measure can be meaningfully translated into another language so that it is conceptually equivalent to the source language and both culturally and linguistically appropriate to the target country.³⁴ Assessing the translatability of a measure during its development can therefore reduce problems encountered in future translations and avoid incorrect interpretations of items. A concept elaboration document was initially produced that aimed to clarify any ambiguities or nuances within the questionnaire. The translatability of the questionnaire was subsequently reviewed for the following languages: Afrikaans, Arabic, Finnish, French, German, Greek, Hindi, Russian, simplified Chinese, and Spanish. The second round of cognitive interviews followed to confirm that the adjustments made were acceptable to participants.

Analysis

Consistency of interpretation for all items was checked across participants. Comments were amalgamated into a single document to highlight reoccurring problems with items or interpretation. Participant comments were reviewed by SD and DM and adjustments made to items when there was consistent ambiguity or item misinterpretation evident.

Results

Characteristics

Thirty-seven participants, 16 men and 21 women, took part in in-depth interviews. The mean age of the sample was 55.57 years (SD [standard deviation] 12.2), and the sample included three to seven participants from each of the eight selected condition groups. Four participants had more than one health condition. Twenty-six (70.3%) participants were married or cohabiting, and 24 (64.8%) participants were employed. Participants who took part in the cognitive interviews consisted of five men and eight women. The mean age within the sample was 57 years (SD 12.9). Table 2 summarizes the characteristics of the sample.

Table 2 Participant characteristics Note: aFour participants (10.8%) taking part in the in-depth interviews reported having more than one condition. Abbreviations: MND, motor neuron disease; MS, multiple sclerosis; PD, Parkinson’s disease; SCI, spinal cord injury; SD, standard deviation.

Informing the item pool

Following the analysis of the interview transcripts, key topics raised were mapped onto the nine established ICF domains. As some domains in the ICF focus on carrying out tasks and others on integrating and participating in society, some overlap was evident; however, topics relating to each ICF domain are expanded in what follows.

1. Learning and applying knowledge

Participants described becoming more aware of their health and well-being during the course of their illness. Nine participants described how they learned skills to help them manage their health and used their experiences in relation to their health to inform day-to-day decisions, for example, knowledge and experience of how specific actions may affect a person’s well-being and could influence their behavior:

I mean I’ve learned to not necessarily adjust, but be aware of my diabetes. I refuse more sweet things in food for example. I give myself perhaps up to one treat a day, but not more than that. And as my levels in fact are all in the green at the moment, I haven’t really got a lot to worry about as far as the diabetes goes. [Diabetes 1]

Seven participants discussed using creative ways to carry out tasks and tried various methods to solve problems. One participant with arthritis, for example, described lying down on the floor to paint skirting boards instead of sitting or crouching to paint. In some cases, problem-solving was facilitated by other people in the participants’ everyday life:

My wife has no idea how to change a lightbulb, so between us […] between us, we get it done because I will give her exact instructions on how to do it. She’ll reach it, and I’ll tell her from sat down. So again, it’s not a case of “I can’t”, it’s “How do I?” [Spinal Cord Injury 3]

2. General tasks and demands

Twenty participants, particularly those with conditions affecting mobility, reported simple tasks and activities taking longer and requiring more effort than they would otherwise take due to their health.

[…] if I shave in the morning […] that probably takes, well, five minutes […] I’m just standing still in front of the basin. I can just about hold myself up for that time, obviously grabbing onto the basin for, you know, on and off […] once I’ve done that, I need to sit down for 10 minutes to recover. [Motor Neuron Disease 3]

The extra time and planning taken to complete simple tasks were a source of frustration and stress for participants. Fourteen were forced to consider simple tasks more carefully due to their health. The additional effort to carry out such tasks was somewhat highlighted when comparing abilities before and after acquiring a health condition:

[…] at one time, I could have gone up a ladder, without too much thought, today it takes a lot of thought. I still go up and down the ladder, but only after a lot more consideration and thought. [Arthritis 2]

       It’s hard […] if you look at where I’m now, compared to what I was then, it’s unbelievable, the difference. It’s a bigger effort for me to go upstairs to go for a pee than it was for me to drive to [city] and back and do a day’s work. [Motor Neuron Disease 3]

To account for tasks requiring more time and energy, participants often planned daily routines and activities more carefully. Planning routine activities enabled the participant to carry out general demands while conserving energy and minimizing fatigue or stress. General chores, for example, were carried out with rest intervals or over a period of days.

[…] if it’s [chores] in small bursts, then it’s fine. If I cleaned the house for two hours constantly, the bending and like if it’s anything lower kind of, and it involves bending and stretching, and that really aggravates it […] I can do things, and it’s not ’til the end of the day that it really has an impact and it just completely knocks me. So, I can do things, but it does affect things, kind of long-term […]. [Arthritis 5]

Despite tasks often taking longer or requiring more effort, participants largely preferred doing things without the assistance of others where possible.

3. Communication

Seven people with motor neuron disease, Parkinson’s disease, and multiple sclerosis experienced difficulties when communicating, especially in large groups, places that had background noise, or when they were also carrying out another task. One woman, for example, found it difficult to have a conversation while getting around:

They say, “Hey, you’re not answering”. I say, “Please don’t speak to me whilst I’m walking as I can’t do it, I can’t do both at the same time”, which is ridiculous, you take that so much for granted. [Multiple Sclerosis 1]

Six of the participants described earlier were aware that their speech could occasionally become affected due to their condition. At times, this caused four people to avoid social situations:

[…] my speech goes a bit […] I sound like I’ve had a few G and Ts [gin and tonics]. My speech is quite good at the moment, and then drops at other moments […] but I think it’s more obvious to me than to other people. [Multiple Sclerosis 5]

       I became more isolated […] I was avoiding [social] situations […] I tend to hide in the back of things until I felt confident […]. You get the stuttering and the stammering and the shakes and things like that […] if there’s a way of avoiding things, then you will. [Parkinson’s Disease 3]

Two participants with motor neuron disease had very prominent difficulties with speech, causing them to feel isolated and unable to communicate freely with others. Emails and electronic communication devices while useful did not provide a satisfying solution:

[It] frustrates me greatly that we can’t have a conversation. If someone spoke all I’d written to him [20-year-old son] in two years, that person could say all this in about five hours […]. Silence is fine normally, but when you are mute then it’s a different thing. [Motor Neuron Disease 5, written]

4. Mobility

Thirteen people with physical limitations described difficulties getting up in the morning and getting dressed. Five described needing help to get up in the morning due to limited mobility or general stiffness:

Well, getting up first thing in the morning is quite a trial because I’m generally very stiff. My feet hurt when I put the weight on them in the morning, it takes me quite a while to actually get up and get moving [Parkinson’s Disease 4]

       I can’t get up until [wife] dresses me. She puts my socks on, and then I get out of bed onto the edge of the bed using a frame […]. She pulls me upright and dresses me. I can’t dress myself, I can’t get a shirt over my head myself, and it’s impossible to stand up to get on pants or trousers [Motor Neuron Disease 4].

The ease of getting around the home varied within the sample. Those experiencing stiffness found it useful to have some movement throughout the day. Participants were, however, conscious that they needed to have rest periods:

Sitting about doing nothing is not good for me. I pay a price every day I wake up. It’s like having a quota of mobility available to you. Once you’ve used up that quota, no matter what the activity, whether it’s walking into town and back, whether it’s going bowling, and then cutting the grass in the garden […]. [Arthritis 2]

Nineteen participants experienced transport difficulties due to their health. Seven participants had to stop driving, and this had an impact on other areas of their lives (eg, work life). The ability to drive or move around gave people a sense of independence and mobility:

I can walk and drive, so it’s given me some of my life back [Arthritis 4]

5. Self-care

Participants discussed difficulties with carrying out self-care activities. Thirteen participants with mobility difficulties (eg, those with arthritis, Parkinson’s disease, motor neuron disease, and multiple sclerosis) described some difficulties with washing and dressing, often needing some degree of help:

I have a shower wheelchair […] and I do my hair without letting go. If I let go, I fall over [so] that’s the most safe way of doing it. The important thing is that I can take myself to urinate, but I need someone to help me turn round and sit down and take my trousers off when I evacuate my bowels and also pull my trousers up. [Motor Neuron Disease 4]

       […] if it’s [clothing] got small buttons on, [husband] will have to do those up, but […] when I’m buying new clothes, I make a point of looking at the fastenings […] I wear leggings, so I haven’t got [to] worry about zips and I’ll wear socks rather than tights, […] I have got some shoes […] with a fine buckle on them, and [husband] has to do that up for me. [Parkinson’s Disease 2]

One woman with a spinal cord injury had assistance with her morning self-care activities and looked upon this help from her personal assistant as a way in which she could achieve greater independence:

[…] I think having a PA [personal assistant] for me really helps, because I can live on my own, but then everything would take a lot longer – like now when I get up to go to work, I’m up at seven, I leave the house at seven-fifty, and I’ve had a shower, got dressed, and had my breakfast so I can do things really quick. If I were to live on my own, however, I probably would have to get up at half [past] five to get myself dressed, showered, and ready and everything. [Spinal Cord Injury 2]

The aforementioned observations therefore highlighted the balance between maintaining a sense of independence and dignity while also fulfilling the need for efficiency in everyday life. Ten participants discussed the benefits of maintaining a healthy diet, doing exercise and, where applicable, taking their medication in order to benefit their health:

It’s not affected exercise […] I make sure that my blood sugar’s over 14 [mmol/L], which is a reasonably high blood sugar, but if I then go to the gym, I’ll actually leave the gym with a blood sugar that’s completely normal and a fine number. [Diabetes 3]

6. Domestic life

Fourteen participants described becoming frustrated as their health affected their usual role or responsibilities within the home. Domestic activities which were often affected included cooking, doing the laundry, and DIY (“Do It Yourself” household maintenance):

I couldn’t bend over […] it’s not a low oven […] but I couldn’t bend over it to, you know, even put a baked potato in it, you know, and the absolute frustration […] if he [husband] didn’t come home from work in time, […] no one was fed […] it just didn’t happen [Back pain 3]

       I miss being able to do things round the house, you know, simple things like DIY which I did a lot and that frustrates me. [Motor Neuron Disease 4]

Two participants completed their grocery shopping online as going out to supermarkets was physically demanding. Nine participants discussed having some difficulties when going out to do the shopping. These participants had poor mobility or compromised motor skills. In particular, those with PD recalled difficulties they encountered when buying goods:

[…] shopping […] If I’m going out on my own, I try again to do it in the mornings, but […]. prior to getting to the checkout […] I actually organize my purse […] I take notes out rather than coins, and if I’m using a credit card, I take the card out before I get to the cash desk, so that I haven’t got to fumble with the zips or pockets of my purse […]. [Parkinson’s Disease 2]

       […] luckily our local shop has the scan as you go, which I like because I can do it at my own pace. I hate when I get to a checkout and I get rushed through, because I get shaky and, you know, people don’t understand when they’re behind somebody slow in a supermarket queue. [Parkinson’s Disease 4]

Difficulties with carrying out household chores or shopping fostered a sense of reliance on others and in some circumstances contributed to a feeling of dependence on others within the household.

7. Interpersonal interactions and relationships

Twenty-two participants described how aspects of family and social life were affected due to their health. Changes in a person’s ability to participate in areas of life that were important to them (eg, professional or social life) could put strain on family relationships:

[…] relationships become very difficult because, I think, we’re used to having [a] very rich social life […] you [are used to having] all the interactions with work and your work colleagues […] and then you have all the interactions with your friends that you see socially, and then you have your family and so there’s a three-part thing. Well, two of those three parts have sort of largely been removed and that puts too much pressure in a way on the family, which it can’t, you know, necessarily sustain really. [Motor Neuron Disease 3]

Other ways in which family relationships could be affected involved being excluded from enjoyable family experiences such as outings with children/grandchildren or participating in activities which may result in a detrimental effect on one’s health. Interactions with family or friends could also be affected by restrictions due to diet or pain:

[…] when you go somewhere to eat you have to tell people you’re diabetic. If I go to my sister-in-law, she’s not a very good cook, she has everything brought in from a delicatessen. Because I’m allergic to preservatives now I can’t eat it, I feel very sick. [Diabetes 5]

       When you’re in periods of pain, you don’t think straight sometimes, you can be a bit of a miserable bugger, and that obviously impacts on those around you. [Back pain 2]

Fourteen participants who had a spouse recognized the additional pressure their health had on their partner. Four felt that, despite the difficulties of maintaining a physical relationship with their partner, they had become closer:

Emotionally we’re very close now. I think it happens. Either you are forced together or you are pushed apart by something like this. [Motor Neuron Disease 4]

       My wife and I enjoy a very happy personal relationship, and you go to start some nice sexual activity and suddenly your hip says “Oh no you don’t” and it gives you a belt of pain. Believe me, it’s quite a deterrent. I’m lucky I’ve got a wife who understands. [Arthritis 2]

While many had to relinquish a certain amount of control in relationships, two participants felt that it was important to maintain some independence and had clear boundaries of what they considered to be an acceptable level of assistance:

My husband has to do everything for me really, apart from […] well, I’m not going to say apart from feed me because that would be the end, I don’t think I could stand that if he had to feed me. [Motor Neuron Disease 6]

8. Major life areas

Twenty-four participants participated in some form of paid work. Half of those who were employed worked part-time. Being employed in a job which was not physically demanding was essential for 12 respondents. However, it was also important for a job to offer the flexibility of being able to move around at suitable intervals to ease potential stiffness:

I found it difficult to sit for too long, but I used to be able to get up and walk about, being a pencil pusher, I was always sat down. So in one respect, it was easier because I was sitting down, in another respect, I needed to get up and move about. [Arthritis 3]

As a result of their health, eleven participants were forced to change their job role, resulting in a change in their financial circumstances and/or career prospects:

I was a car sprayer, I had to pack up that, and they gave me a job in the stores, where I could sit down a lot, so I was still able to work […] but it cost me a lot of money, downgrading. [Arthritis 4]

       […] I’ve reduced my salary hugely and I’ve reduced my opportunities hugely […] the trajectory that I was on, logically […] would have led to my becoming […] somebody in a more senior responsible role involved in projects […] I just had to make a decision as soon as I got diagnosed that that would be unsustainable. [Motor Neuron Disease 3]

       It’s wrecked my career. I was a high-flying program manager for [company] and I’ve never got back up to those levels […]. [Cancer 4]

When in the workplace, ten were restricted in the activities in which they could participate. One person with diabetes recalled being reluctant to disclose details about her health to work colleagues, yet, nondisclosure meant she was excluded from social aspects of the job:

It [diabetes] had an impact because they [work colleagues] ate out a lot. I had to make an excuse. In the end they knew about it. The colleagues were very nice about it. They made sure my food from the canteen was correct […]. Traveling was a problem because of eating out. I traveled a lot to US and Amsterdam. [Diabetes 5]

9. Community, social, and civic life

Participants discussed changes in lifestyle due to their health. While involvement in their “usual activities” often declined, many became involved in new or revised pastimes. For example, four participants found enjoyment through the use of the Internet to interact with others, and two became active through support groups:

[Routine has changed] Utterly. Then I was out and about in the parish, visiting, cycling to see people, taking services, going to meetings, etc, etc. Now I tend to be limited to being here, I use my laptop a lot, I write my blogs. [Motor Neuron Disease 4]

       […] if you’re used to being so active, and so busy, how do you fill your days? So, that was one of the reasons why I […] applied to be a Trustee of MNDA [MND Association] because I thought […] I could add value there and also it will help me […] to be busy and […] still feel as if I’m contributing. [Motor Neuron Disease 6]

       […] when you start becoming involved with other people in a support group, you start becoming aware of other people’s values, other people’s ways of dealing with the condition, and you get, in a sense, the satisfaction from knowing that you’re helping them. [Cancer 3]

Twelve participants maintained interest in recreational activities they did prior to having poor health. However, these activities were often modified so that they could still participate:

I used to belong to a rambling club […] I can walk the distance, I could walk 3, 4, 5 miles, but I have to start comparatively slowly, and I need to do it at my own pace. I’m not crawling, it’s reasonable, but it isn’t a rambling club type pace any more. [Arthritis 2]

       But what changed was, instead of me playing badminton pretty much solidly for the hour and me being the one going, “Yeah, yeah, I’ll stay on court. You go and sit down, I’ll stay on court”. I was playing a game, sit out a game, play a game, sit out a game. So I sort of halved the amount that I was doing, but it gave me something to look forward to every 3 weeks and it meant that I could keep in touch with people, and again, I’m the sort of person, I didn’t try and hide my illness. [Cancer 1]

Representation of the ICF themes

Following the analysis of the interview transcripts, statements which the research team considered to reflect each of the nine ICF theoretical domains were identified from the transcripts and collated. A total of 222 statements from participant transcripts were identified as candidate statements that could be transformed to items. The pool of statements was reviewed for applicability across a range of health conditions. Statements were removed if they were condition specific (eg, “When you go somewhere to eat you have to tell people you’re diabetic”) or a repetition of a previous statement (eg, “I find it difficult to get up in the morning” and “I have difficulty getting up in the morning”). The remaining statements were converted to item format to fit with the question stem: “How often during the past 4 weeks have you have difficulties with the following ….?” An item pool containing 95 generic items applicable to all health groups remained after this process. Items represented all nine activity and participation domains of the ICF. A final meeting to further reduce and refine the number of items based on repetition and adequate coverage of the ICF domains reduced the pool further to 24 items.

Expert refinement

All reviewers agreed that the items represented the ICF domains and that the questionnaire was appropriately formatted. Minor amendments to improve the clarity of the wording of items were made where necessary following the reviewers’ comments. Three new items were added to ensure all domains were adequately reflected. Twenty-seven items were therefore entered into testing among patients.

Patient refinement

Two rounds of cognitive interviewing were carried out ensuring that it was clear participants found the instructions easy to understand and that participants considered all items relevant to the construct. The first round largely confirmed that items were interpreted consistently among participants. Minor changes to item wording were carried out where necessary. Changes included making adjustments to examples of activities provided to expand on item meaning and the separation of two double-barreled items (ie, four items were derived from two items) to improve clarity. This resulted in 29 items in total.

A translatability assessment was carried out on the revised items. The translatability assessment resulted in minor adjustments to wording, the deletion of two items, and the division of one item into two items in order to improve clarity. All wordings of the final 28 items were found to be culturally and linguistically relevant to the selected countries.

The second round of cognitive interviewing was carried out to confirm that changes made from the translatability assessment were acceptable to respondents. All items were found to be acceptable, and the final 28 items were considered suitable to enter psychometric testing. Emphasis was placed on retaining a sufficient number of items to represent each of the nine ICF domains. Overall, cognitive interviewing confirmed that the questionnaire instructions, 28 items, and the response options were acceptable to participants. An example candidate item for each ICF domain is illustrated in Table 3.

Table 3 Candidate Ox-PAQ items Notes: *Item stem for all items: “How often during the past 4 weeks have you have difficulties with the following …?” Response options for all items: “Always”, “Rarely”, “Sometimes”, “Often”, “Never”. Abbreviations: Ox-PAQ, Oxfoxrd Participation and Activities Questionnaire; ICF, International Classification of Functioning, Disability, and Health.

Discussion

This paper documents the steps taken to inform an item pool relating to activities and participation, according to the nine domains specified in the ICF. The in-depth interviews facilitated the exploration of key issues relating to activities and participation from the patients’ perspective. As the final Ox-PAQ instrument is required to be suitable across conditions, participants from a range of conditions were included in the in-depth interviews. Analysis of interview transcripts informed the construction of 24 candidate questionnaire items, which were subsequently reviewed by a panel of experts with experience of using the ICF. Items were also tested for applicability across the specified health conditions during face-to-face cognitive interviews and assessed for their potential ease of translation to other languages. The final 28 items were found to be acceptable for inclusion in a generic instrument.

The results reported represent preliminary evidence to support content and face validity as recommended by best practice guidelines in PROM.^7,35,36 Current measures of participation and/or activity are largely disability and rehabilitation focused, and their early stages of development are particularly poorly reported. Many have failed to involve patients in generating items,^37–41 something that is considered fundamental if measures are to reflect the issues of greatest concern to those we are assessing.

The in-depth interviews informing this research confirmed that participants from a wide range of health groups are affected across the nine domains of activity and participation as outlined in the ICF. It should, however, be noted that these domains are not necessarily discrete and there appears an inherent degree of overlap. For example, participating fully in “domestic life” can be affected by the degree to which a person can carry out “general tasks and demands”. In the reported interviews, one illustration of this overlap was when a parent’s usual role of providing a meal for their children was affected as back pain prevented them from cooking. Limitations in relation to “self-care” can also affect participation in “interpersonal interactions and relationships”. For example, some participants indicated reluctance for their spouse to become responsible for certain aspects of care such as helping them go to the toilet. Defining “activity” and “participation” as separate entities can therefore be problematic, as has been debated in previous literature.^42,43 In general, ICF domains 1–5 have been considered “activities”, while domains 6–9 have been seen as primarily related to “participation”.^42–44 Broadly speaking, the analysis reported here supports this assertion, yet recognizes considerable overlap between domains and, therefore, between the two terms. The candidate Ox-PAQ items reflect this overlap, and some items may well represent more than one domain. Indeed, previous research which has specifically divided items into separate scales representing “activity” and “participation” in accordance with the ICF chapter’s support the strong overlap between the two constructs having found very strong correlations between the scales. Psychometric testing of results from the Ox-PAQ will determine the ultimate grouping of items in the questionnaire, although there is clear evidence that all aspects of the ICF are covered by items currently included in the instrument.

While specific ICF domains may be more applicable to some conditions than others, they were all applicable to some degree to the conditions included within our sample. This was supported by both our qualitative interviews and subsequent cognitive interviewing across a range of conditions. Further testing of the items among a larger sample will enable deeper investigation to the applicability of each item to the conditions included in this study. It will also validate this PROM for use in further conditions such as chronic obstructive pulmonary disease and people with mental health problems. It should be emphasized, however, that the Ox-PAQ is intended for use in any condition that has the potential to impact upon an individual’s ability to participate and be active in their daily life.

The development process reported here follows current best practice such as guidance provided by the FDA, placing patients at the heart of the item generation process. While it would have been preferable to have all of the health groups included in the in-depth interviews equally represented, the interview schedule and subsequent themes that emerged were generic in nature and thereby not biased to specific conditions. The research is also strengthened by the inclusion of a translatability assessment. Formal assessment of translatability during the development of a measure such as the Ox-PAQ is crucial given its intended use. For example, clinical trials are likely to be conducted within multicultural populations and across many countries, and if the questionnaire is to have maximum usefulness, it is essential that its content is applicable and readily understandable in many languages. Conducting a translatability assessment during the early stages of development makes any required changes considerably easier to incorporate than would be the case following psychometric analysis.

Conclusion

This paper has documented the method of developing an item pool for a generic, theoretically and empirically based PROM assessing activity and participation. Using a clear conceptual basis, in the form of the WHO ICF to inform item generation, 28 candidate items have been identified as suitable to enter further testing. A large-scale postal survey is currently in progress and will facilitate further refinement of the Ox-PAQ alongside an initial assessment of its psychometric properties. The final instrument is intended for use in clinical trials and interventions targeted at maintaining or improving activity and participation. Further details of the development process can be found at http://www.dph.ox.ac.uk/research/hsru/OxPAQ.⁴⁵

Acknowledgments

We would like to acknowledge the continued support and assistance of Dr Mary Baker, MBE, President, European Brain Council. All patient/personal identifiers have been removed or disguised so the patient/person(s) described is not identifiable. The Ox-PAQ initiative is funded by the European Brain Council.

Disclosure

The authors report no conflicts of interest in this work.

---

References

1.	Macfarlane J. Other physical consequences of disability. Handb Clin Neurol. 2013;110:315–322.
2.	Beer S, Khan F, Kesselring J. Rehabilitation interventions in multiple sclerosis: an overview. J Neurol. 2012;259(9):1994–2008.
3.	Blickem C, Kennedy A, Vassilev I, et al. Linking people with long-term health conditions to healthy community activities: development of Patient-Led Assessment for Network Support (PLANS). Health Expect. 2013;16(3):3.
4.	Provencher V, Bier N, Audet T, Gagnon L. [Long-term effect of a cognitive intervention on learning and participation in a significant leisure activity in early dementia of Alzheimer type: a case study]. Psychol Neuropsychiatr Vieil. 2009;7(2):131–140. French.
5.	Streiner D, Norman G. Health Measurement Scales: A Practical Guide to their Development and Use. 4th ed. Oxford, UK: Oxford University Press; 2008.
6.	Hays RD, Revicki D. Reliability and validity (including responsiveness). In: Fayers PM, Hays RD, editors. Assessing Quality of Life in Clinical Trials: Methods and Practice. Oxford, UK: Oxford University Press; 2005.
7.	US Department of Health and Human Services, Food and Drug Administration. Guidance for Industry: Patient-reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. Silver Spring, MD: US Department of Health and Human Services, Food and Drug Administration; 2009.
8.	Kerr C, Nixon A, Wild D. Assessing and demonstrating data saturation in qualitative inquiry supporting patient-reported outcomes research. Expert Rev Pharmacoecon Outcomes Res. 2010;10(3):269–281.
9.	Dijkers MP. Issues in the conceptualization and measurement of participation: an overview. Arch Phys Med Rehabil. 2010;91(9 Suppl):S5–S16. doi: 10.1016/j.ampr.2009.10.036.
10.	Heinemann AW, Tulsky D, Dijkers M, et al. Issues in participation measurement in research and clinical applications. Arch Phys Med Rehabil. 2010;91(9 Suppl):S72–S76. doi: 10.1016/j.ampr.2009.11.031.
11.	Whiteneck GG, Bogner JA, Heinemann AW. Advancing the measurement of participation. Arch Phys Med Rehabil. 2011;92(4):540–541.
12.	WHO. World Health Organization the International Classification of Functioning, Disability and Health (ICF). Geneva, Switzerland: WHO; 2001.
13.	Herrmann KH, Kirchberger I, Stucki G, Cieza A. The comprehensive ICF core sets for spinal cord injury from the perspective of physical therapists: a worldwide validation study using the Delphi technique. Spinal Cord. 2011;49(4):502–514.
14.	Renom M, Conrad A, Bascuñana H, et al. Content validity of the Comprehensive ICF Core Set for multiple sclerosis from the perspective of speech and language therapists. Int J Lang Commun Disord. 2014;49(6):672–686.
15.	Cieza A, Stucki A, Geyh S, et al. ICF Core Sets for chronic ischaemic heart disease. J Rehabil Med. 2004;44:94–99.
16.	Cieza A, Stucki G, Weigl M, et al. ICF Core Sets for chronic widespread pain. J Rehabil Med. 2004;44:63–68.
17.	Rauch A, Kirchberger I, Boldt C, Cieza A, Stucki G. Does the Comprehensive International Classification of Functioning, Disability and Health (ICF) Core Set for rheumatoid arthritis capture nursing practice? A Delphi survey. Int J Nurs Stud. 2009;46(10):1320–1334.
18.	Glässel A, Coenen M, Kollerits B, Cieza A. Validation of the extended ICF core set for stroke from the patient perspective using focus groups. Disabil Rehabil. 2012;34(2):157–166.
19.	Kirchberger I, Coenen M, Hierl FX, et al. Validation of the International Classification of Functioning, Disability and Health (ICF) core set for diabetes mellitus from the patient perspective using focus groups. Diabet Med. 2009;26(7):700–707.
20.	Salter K, Jutai JW, Teasell R, Foley NC, Bitensky J, Bayley M. Issues for selection of outcome measures in stroke rehabilitation: ICF activity. Disabil Rehabil. 2005;27(6):315–340.
21.	Salter K, Jutai JW, Teasell R, Foley NC, Bitensky J, Bayley M. Issues for selection of outcome measures in stroke rehabilitation: ICF Participation. Disabil Rehabil. 2005;27(9):507–528.
22.	Noonan VK, Miller WC, Noreau L. A review of instruments assessing participation in persons with spinal cord injury. Spinal Cord. 2009; 47(6):435–446.
23.	Magasi S, Post MW. A comparative review of contemporary participation measures’ psychometric properties and content coverage. Arch Phys Med Rehabil. 2010;91(9 Suppl):S17–S28. doi: 10.1016/j.ampr.2010.07.011.
24.	Wilkie R, Jordan JL, Muller S, Nicholls E, Healey EL, van der Windt DA. Measures of social function and participation in musculoskeletal populations: impact on Participation and Autonomy (IPA), Keele Assessment of Participation (KAP), Participation Measure for Post-Acute Care (PM-PAC), Participation Objective, Participation Subjective (POPS), Rating of Perceived Participation (ROPP), and The Participation Scale. Arthritis Care Res. 2011;63(11):20641.
25.	Tse T, Douglas J, Lentin P, Carey L. Measuring participation after stroke: a review of frequently used tools. Arch Phys Med Rehabil. 2013; 94(1):177–192.
26.	Morley D, Dummett S, Kelly L, Dawson J, Fitzpatrick R, Jenkinson C. The Oxford Participation and Activities Questionnaire: study protocol. Patient Relat Outcome Meas. 2013;5:1–6.
27.	McColl E. Developing questionnaires. In: Fayers P, Hays R, editors. Assessing Quality of Life in Clinical Trials. 2nd ed. Oxford, UK: Oxford Medical Publications; 2005.
28.	Pope C, Ziebland S, Mays N. Analysing qualitative data. Br Med J. 2000;320(7227):114–116.
29.	Ritchie J, Spencer L. Qualitative data analysis for applied policy research. In: Bryman A, Burgess RG, editors. Analysing Qualitative Data. London, UK: Routledge; 1994.
30.	NVivo qualitative data analysis software [computer program]. Version 9. Melbourne, Australia: QSR International Pty Ltd; 2010.
31.	Willis GB. Cognitive Interviewing: A Tool for Improving Questionnaire Design. London, UK: Sage Publications; 2005.
32.	Willis GB. Cognitive Interviewing: A “How To” Guide. NC, USA: Research Triangle Institute; 1999.
33.	Wilson M. Constructing Measures: An Item Response Modeling Approach. Mahwah, NJ: Lawrence Erlbaum Associates; 2005.
34.	Conway K, Patrick DL, Gauchon T, Acquadro C. Enhancing Cross-Cultural Appropriateness for Newly Developed Patient-Reported Outcome (PRO) Instruments: the Use of Translatability Assessment. Patient Reported Outcomes Newsl. 2010;44:9–12.
35.	European Medicine Agency. Reflection Paper on the Regulatory Guidance for the Use of Health-related Quality of Life (HRQL) Measures in the Evaluation of Medicinal Products. EMEA/CHMP/EWP139391/2004. London, UK: European Medicine Agency; 2004.
36.	Reeve B, Wyrwich K, Wu A, et al. ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research. Qual Life Res. 2013;22(8):1889–1905.
37.	Fougeyrollas P, Noreau L, Bergeron H, Cloutier R, Dion SA, St-Michel G. Social consequences of long term impairments and disabilities: conceptual approach and assessment of handicap. Int J Rehabil Res. 1998;21(2):127–141.
38.	Cardol M, de Haan RJ, van den Bos GA, de Jong BA, de Groot IJ. The development of a handicap assessment questionnaire: the Impact on Participation and Autonomy (IPA). Clin Rehabil. 1999;13(5):411–419.
39.	Wilkie R, Peat G, Thomas E, Hooper H, Croft PR. The Keele Assessment of Participation: a new instrument to measure participation restriction in population studies. Combined qualitative and quantitative examination of its psychometric properties. Qual Life Res. 2005;14(8):1889–1899.
40.	Ostir GV, Granger CV, Black T, et al. Preliminary results for the PAR-PRO: a measure of home and community participation. Arch Phys Med Rehabil. 2006;87(8):1043–1051.
41.	Gandek B, Sinclair SJ, Jette AM, Ware JE Jr. Development and initial psychometric evaluation of the participation measure for post-acute care (PM-PAC). Am J Phys Med Rehabil. 2007;86(1):57–71.
42.	Whiteneck G, Dijkers MP. Difficult to measure constructs: conceptual and methodological issues concerning participation and environmental factors. Arch Phys Med Rehabil. 2009;90(11 Suppl):S22–S35. doi: 10.1016/j.ampr.2006.06.009.
43.	Post MW, de Witte LP, Reichrath E, Verdonschot MM, Wijlhuizen GJ, Perenboom RJ. Development and validation of IMPACT-S, an ICF-based questionnaire to measure activities and participation. J Rehabil Med. 2008;40(8):620–627.
44.	van der Zee CH, Kap A, Rambaran Mishre R, Schouten EJ, Post MW. Responsiveness of four participation measures to changes during and after outpatient rehabilitation. J Rehabil Med. 2011;43(11):1003–1009.
45.	Ox-PAQ Initiative. Nuffield Department of Population Health; Medical Sciences Division; University of Oxford. Available from: http://www.ndph.ox.ac.uk/research/health-services-research-unit-hsru/research/oxpaq-initiative. Accessed April 30, 2015.