Test–Retest Reliability and Clinical and Research Validity of the ‘Quality of Life Impact and Concerns’ (QoLI&C) Measure

Hip dysplasia is a significant public health issue, representing the single largest cause for total hip arthroplasty in young adults.¹ Acetabular dysplasia (AD) in young adults arises from residual childhood developmental dysplasia of the hip (DDH) and adolescent-onset acetabular dysplasia.² Significantly, these young adults are experiencing chronic pain, insomnia,³ difficulty in walking and effects on social relationships at a time when they would normally be going to university, starting a career and/or having a family.⁴ To provide insight into patient-perceived and patient-significant concerns of hip dysplasia and its impact on everyday life, we developed a novel measure, the “Quality of Life Impact and Concerns” (QoLI&C) measure, and provided evidence on its psychometric features, in particular, its acceptability to patients, face and content validity, reliability and supportive evidence of construct validity.⁵

This short report provides additional evidence to support the validity of the QoLI&C measure, exploring its test-retest reliability. The report also draws attention to some deficiencies in the published measure, namely, repeated or similarly phrased or poorly phrased items. The revised measure is presented, along with ways to best use the measure to enhance its research and clinical utility.

Test–Retest Study

A small test-retest reliability study (n=53) was conducted recruiting participants predominantly from volunteer patients taking part in a “human movement” PhD study* being conducted at Cardiff University’s Human Gait Laboratory. Study participants were recruited from a message posted on social media hip-dysplasia websites (Facebook - hip dysplasia; Facebook - International Hip Dysplasia Institute) and two UK hip dysplasia charities (STEPS, DDH UK). This directed interested persons to information about the project located on a secure Cardiff University website. Participants were asked to complete the measure on two occasions, separated by 2–3 weeks. Participants firstly consented on-line to take part and then were directed to the measure. One of the research team (first author) then electronically re-contacted the participant 2 to 3 weeks after the first questionnaire was completed and provided a link to the “retest” questionnaire. The two questionnaires were identical in all respects save two. Firstly, the “test” (first) questionnaire asked the participant two opened questions relating, respectively, to their hopes and fears. Participants commonly provided extensive comments. These two questions were omitted from the “retest” (second) questionnaire, as it was perceived inappropriate and too onerous for a participant to complete them again just 2–3 weeks after completing the original questionnaire. Secondly, to enhance validity, the “retest’ questionnaire included two extra questions asking: (1) if there have been any changes in their quality of life and well-being because of their hip condition in the last 2–3 weeks; and, (2) if so, to outline what these were. To assess test-retest reliability, Pearson’s correlation coefficients were computed. Ethical approval was gained from Cardiff University.

Participants had all been diagnosed with hip dysplasia (Table 1). Their profile was similar to those of participants in the original validation study.⁵ Just over a half (55%) had dysplasia in the left hip, and 28% in both hips. Participants were predominantly female (75%), had a median age of 28 years (range, 19–56), and commonly were in full or part-time work (49%) or in further or higher education (30%). The median time from first concern over their hip condition to diagnosis was 2–5 years (mean, 3.4 years), but for 19% it was five or more years. Nearly all (96%) had consulted a primary care physician prior to diagnosis and 66% a physiotherapist. The majority were over 12 months post-surgery (left hip, 57%; right hip, 42%). The most common treatment for the left hip was joint conservation or triple/femoral osteotomy (64%). The most common treatment or the right hip was the “other” category (43%), involving a number of different treatments experienced over time, followed by joint conservation or triple/femoral osteotomy (21%).

Table 1 Respondent Characteristics **

Table 2 presents the test-retest results. For all the sub-scales and the overall QoLI&C measure, Pearson’s correlations for the two time points ranged from 0.892 to 0.968, interpreted as evidence of “good” reliability.⁶ This was also the case for “current level of pain” and questionnaire completion time. “Acceptable” reliability, with values of 0.736 to 0.736, was found for “satisfaction with care” and general health status. Looking overall, this small study provides supportive evidence on the test-retest reliability of the measure. None of the participants pointed to any change in their condition between completion of the “test” and the “retest” questionnaire.

Table 2 Test–Retest: Correlations

Refinements to the Measure to Enhance Clinical and Research Validity

Our further research suggested ways to refine the measure to increase both its clinical and research utility. These were of three kinds.

Firstly, on close inspection of the published measure,⁵ two unintentionally repeated item were found, one in Question 12 relating to the Pain sub-scale (“The aches and pains have become more frequent and more painful”), the other in Question 21 relating to the Quality of Life sub-scale (“My life is very disrupted by my hip condition”). This repetition has been removed. Secondly, two very similar questions were included in Question 14 as part of the Sports and Exercise sub-scale (“I want to stay fit and in shape, but because of my hip it has become harder and harder to find activities I can do” and “I want to keep trim and in shape but it has got harder and harder to find activities that I can do.”) The latter statement alone has been included in the revised questionnaire. Thirdly, a single “indignity of being young and having to be careful about what clothes and shoes to wear” was created within the Body Image sub scale. This replaced the previous two gender-specific items, asking female and male respondents respectively, “there is an indignity of being a young woman and not being able to go out, wearing heels and doing normal stuff” or “there is an indignity of being a young man and not being able to go out, play sports and doing normal.”

The revised measure, and associated on-line questionnaire**now comprises nine domains: Pain (10 items); Education and Work (3 items); Sports and Exercise (6 items); Family Life (3 items); Physical Intimacy (3 items); Social Life and Relationships with Others (5 items); Confidence and Social Esteem (6 items); Body Image (9 items); Emotion and Depression (7 items); and Overall Quality of Life (13 items). The wider questionnaire also includes: fixed choice questions relating to levels of pain, satisfaction with treatment and general health status; two open-ended questions relating to hopes and fears; an open-ended question on the potential usefulness of the measure in discussions with their clinicians; and a rating of the ease of completion the measure and time taken to complete it. [see Supplementary Materials]

The measure is intended to be used by persons aged 16 or over who have been diagnosed with hip dysplasia. All of the sub-scales bar one are appropriate for any such respondent. The one exception is the Physical Intimacy sub-scale, which is to be completed only by those aged 18 or more. Where a respondent is aged under 18, this sub-scale would be omitted in the computation of the QoLI&C measure, thus resulting in a 62-item measure. For all other respondents, the full 65-item scale would be computed.

We suggest that the 65- or 62-item measure, dependent on age, can valuably be used by persons with hip dysplasia in their interactions with healthcare practitioners, including those in primary care, to aid discussions and insight into the ways that hip dysplasia is affecting their everyday life. However, it is likely to be the case that patients in these interactions want to draw attention to particular issues, such as pain, emotional and social impact or quality of life, that is, either a particular sub-scale of the measure or particular items within a sub-scale. The actual value of the full measure may then be of lesser interest, except perhaps as a way for the individual to monitor changes over time, for example, due to treatment or felt greater impact of hip dysplasia on their life.

From a research and clinical utility perspective, the actual scores of the measure are likely to be most valuable as a way to provide an overview, summary of changes in the patient’s hip condition and/or to monitor both changes over time and the effect of different surgical and rehabilitation treatments. Potential time points for data collection relate, for example, to document changes over time from diagnosis onwards, or at post-surgery follow-ups, usually undertaken in the UK at 8 weeks, 3 months, 6 months and 12 months (at which point patients would generally be discharged), and re-using the measure if and when patients re-enter any further hip-related healthcare, surgical intervention or rehabilitation.

The information generated would be helpful both to demonstrate the most common areas of impact of particular hip surgery interventions and to identify whether or not the surgical interventions have impacted positively on areas of greatest concern to patients or whether there are areas important to the patient which remain unaffected. This may also show whether, and which, patients fully recover (for example, those with an early diagnosis and speedy hip conservation surgery) and who may remain least affected by their hip condition for years after the surgery.

As with any new or novel measure, further validation work remains to be done. Together with more evidence on test-retest reliability, there is a need to explore the responsiveness to change of the QoLI&C measure. We suggest that this should be generated through a prospective, observational study, undertaken in a routine clinical practice context, recruiting participants, ideally, from their first post-diagnosis healthcare interaction, following participants up at least for 12 months post-surgical operation(s). This study could also valuably collect data on the way the measure is used, both by patients and healthcare practitioners, in treatment decision-making and treatment monitoring, focusing in particular on if, and in what ways, the measure enhances patient-clinician discussions and aids exploration of the relative benefits of treatment options.

In conclusion, further evidence to support the validity, reliability and utility of the QoLI&C measures has been generated. We suggest that this supports use of the measure in clinical practice to aid patient-clinician discussions over treatment options, both in primary and secondary care, and as a way to monitor changes in the impact of hip dysplasia on patients’ everyday lives. The measure also had potential to be used in effectiveness studies comparing different types of surgical interventions and rehabilitation practices.