Development and Validation of a New Patient-Reported Outcome Measure in the Arabic Language for Patients with Knee Osteoarthritis in Saudi Arabia

Introduction

Knee osteoarthritis (OA) is a joint condition resulting from cartilage and underlying bone breakdown.¹ It can result from previous joint injury, abnormal joint or limb development, or inherited factors.¹ The risk is higher in individuals who are overweight or obese, older, have legs of different lengths, or have occupations that result in exceptional levels of joint stress.¹ Knee OA is a rapidly growing health concern and a significant source of disability among older adults. In 2010, approximately 250 million (3.6% of the population) individuals worldwide had knee OA.¹ It has become one of the ten most disabling diseases in developed countries.¹

In Saudi Arabia, knee OA is a primary cause of disability.² Substantial evidence from previous research supports indicates that the prevalence of clinically defined (57.2%) and radiographic knee OA in men (53.3%) and women (60.9%) increases with increasing age.³ Further, older adults are more affected by knee OA than younger adults.³ In older adults, the increasing frequency of knee OA may lead to knee-related disability, severe pain, and other symptoms, which impair quality of life (QoL), in addition to costly knee OA management.⁴

Clinicians and researchers are encouraged to use accurate outcome measures that capture pain, symptoms, activities of daily living (ADL), recreational activities, and QoL for elderly patients with knee OA.⁵ There has been a growing use of patient-reported outcomes (PRO) in the past two decades to measure different aspects of a patient’s health status and to adapt available PRO measures for specific cultures. Socio-cultural context plays a role in patient preference. Studies have reported that essential domains of QoL—such as pain, physical disability, and other symptoms, include mental and social symptoms. These are perceived differently by patients in different populations,⁶ as patients living in different cultures grow up with different expectations, activities, and social structures.⁷

Numerous PRO measures have been developed and validated in Western countries to assess QoL in patients with knee OA.⁸ Full knee flexion is significantly decreased in patients with knee OA.⁹ Yet, Saudi Arabian culture includes daily activities that require full knee flexion, such as kneeling to pray (five times per day), squatting for ablution, and sitting to eat on the floor. These activities differ from those undertaken in other cultures, suggesting that knee OA may affect Saudi patients’ QoL in ways that differ from that experienced by different cultures.⁸ Thus, a new Arabic PRO measure with excellent reliability and validity is required, similar to the International Classification of Functioning (ICF) Disability and Health framework.¹⁰ The available PRO measures were linked to ICF.¹¹ However, the currently available PRO measures that assess knee OA do not include personal or ecological factors as part of the assessment.¹¹ People’s attitudinal environments and socio-cultural contexts significantly influence their functioning. However, individual PRO measures have been developed specifically for patients with knee OA in different cultures, thus reflecting particular cultures and lifestyles.¹²

Even though several PROs have been used for Saudi patients with knee OA, they do not entirely capture the socio-cultural setting of Saudi Arabia.¹³ In light of these deficiencies, we aimed to develop a new PRO measure in Arabic that covers all concepts like the ICF framework relating to health, function, and participation encompassing environmental and individual elements according to the Saudi socio-cultural context. The psychometric qualities of this new measure, such as test-retest reliability, internal consistency, and construct validity, were also examined in Saudi Arabian patients with knee OA. We predicted that the new Arabic PRO measure would serve as a legitimate, trustworthy tool according to the Saudi socio-cultural context for patients with knee OA.

Materials and Methods

This study was performed in three phases according to the guidelines adopted from Health measurement scales: a practical guide to their development and use (5th edition).¹⁴ Phase one was to define the concept, generate items, and assess content validity to develop the new measure. Phase two was to examine the properties of the generated items. Phase three involved the assessment of the psychometric properties of the items (Figure 1).

Figure 1 Flowchart of the development and validation process of the new Arabic patient-reported outcome measure.

Participants

This study had a cross-sectional design and was conducted at the Orthopedic and Physiotherapy Departments of the following five hospitals located in Riyadh between September 2016 and March 2017: Saudi Arabia: King Khalid University Hospital (KKUH), King Faisal Specialist Hospital & Research Centre (KFSHRC), King Saud Medical City (KSMC), Dawadmi General Hospital (DGH), and Quwaieah General Hospital (QGH). Radiographic knee OA was confirmed by physicians according to the standards of the American College of Rheumatology.¹⁵ The inclusion criteria were as follows: males and females aged 55 years and older with knee OA diagnosed by a physician and able to read and understand basic Arabic language.^16,17 We excluded patients with a surgical history of lower limb fracture in the last six months, inflammatory rheumatoid arthritis, or evident cognitive disorder.

Ethics Statement

The Ethics Committee and Institutional Review Board (IRB) of the following institutions reviewed and approved the study protocol: Applied Medical Sciences, King Saud University (No: CAMS 143–36/37, April 2016), KKUH (No: 16/0300/IRB, May 2016), KFSH&RC (No: ORA/1171/37, Aug 2016), KSMC (No: H1QE-25, Oct 2016), and the Ministry of Health (No: 16–200E, May 2016). All participants read and signed the consent forms before study participation.

Study Phases

Phase One

In this phase, 29 patients were used to develop the new Arabic PRO measure as recommended by David et al.¹⁴

Item Generation

Two approaches were used to establish the items for this study. The first method involved a literature review of studies involving previous PRO measures. The second method used face-to-face interviews (semi-structured interviews) of 10 patients with knee OA to document their problems with the disease.

Item Selection

Five leading experts and four patients (two males and two females, aged ≥48, with high school education or above) selected the relevant items for the content validity index (CVI).¹⁸ The CVI is widely used for quantifying the content validity of measures. The participating examinees (experts and patients) rated the relevance of each item to its subdomain (1 = not relevant, 2 = somewhat relevant, 3 = quite relevant, 4 = highly relevant) to enable the calculation of item-level CVI (I-CVI). Ratings of 1 and 2 are referred to as “invalid content” by researchers who support the adoption of a 4-Likert-type technique, whilst ratings of 3 and 4 are referred to as “valid content”.¹⁹ Additionally, Lynn believes that a 4-point scale is preferable to smaller or bigger scales with an ambiguous middle score (eg, neutral).²⁰

Each item’s I-CVI was calculated by dividing the number of examinees who gave it a 3 or 4 by the total number of examinees. The probability of chance agreements was calculated as follows:¹⁸ . Pc was the probability of chance agreement; n was the number of experts, and A was the number approving with functional relevance. We used the I-CVI and the likelihood of chance agreement to calculate the kappa statistic, K*as follows: . Based on the following rating criteria, each item on the measure was rated as “fair”, “good”, or “excellent”. For fair, K* = 0.40–0.59; good, K* = 0.60–0.74; and excellent, K* > 0.74. Any item that received a good or fair rating was deleted.²¹

Item Formation

The expert panel of physiotherapists, orthopedists, and linguists reviewed the derived items to develop the first draft of the new Arabic PRO measure. They consider concept relevance, clarity or ambiguity, item length, the purpose of the phrasing, and item duplication. The scaling response, recall period, and wording of instructions for people completing the questionnaire were determined by consensus among the expert committee members.

Item Evaluation

Face-to-face interviews were used to evaluate the novel Arabic PRO measure on 15 fresh patients. Their suggestions and any issues about their circumstances that we had not addressed were added to a new draft. We carefully considered the patients’ feedback, made the necessary changes, and created the new Arabic PRO measure final draft.

Phase Two

In this phase, we calculated the percentage of missing data for each item; less than 5% was considered acceptable.¹⁴ Floor and ceiling effects²² were reported when more than 15% of the participants scored the lowest or highest possible subscale scores.²³ Therefore, in the new Arabic PRO scales, floor and ceiling effects were present if more than 15% of our patient collective achieved the highest (100 points) or lowest (0 points) scores.

Phase Three

In this phase, 73 patients with knee OA were enrolled to examine the psychometric properties, including the internal consistency, test-retest reliability, and content validity of the new Arabic PRO measure.

Measures

The Short Form-36 Health Survey (SF-36) is a generic PRO of health status consisting of 36 items and measures relating to physical functioning, physical role, bodily pain, vitality, general health, social functioning, emotional role, and mental health.²⁴ Each domain is rated from 0 to 100, with higher scores indicating better health status. The Saudi Arabian version of the SF-36 has been reported to be a reliable and valid measure.²³

The Visual Analogue Scale (VAS) is a measurement instrument for pain intensity that uses a horizontal line, 100 mm in length, anchored by two descriptive words: no pain and very severe pain. The VAS is reliable and valid for assessing patients with knee conditions.²⁵

Sample Size

Phase one was conducted with 29 patients to develop the new measure, as recommended by David et al.¹⁴ In phase three (validation), we calculated the subject-to-item ratio, which is frequently used to estimate sample sizes with a varying number of patients (2–20) per item. This method of determination was reported in a review conducted by Anthoine et al.²⁶ We determined the ratio to be two patients per item. Therefore, we asked patients to complete the new Arabic PRO measure and the SF-36 twice within a one-week interval.

Psychometric Properties

We calculated Cronbach’s alpha to measure internal consistency.²⁷ We considered an acceptable internal consistency if alpha values lie between 0.70 and 0.95.²³ We assessed test-retest reliability by calculating the intra-class correlation coefficient (ICC). ICC values of ≥0.70 were considered to indicate acceptable reliability.²³ A week was chosen for the repeated measurements to avoid answer recall, which may occur if the re-test period is short, and minimize clinical changes when the re-test period is too long. For construct validity, Spearman correlation coefficient (r_s) measure was used to determine the correlations among the new Arabic PRO measure, the SF-36, and the VAS. Coefficient correlation values >0.50 were considered acceptable.²⁸ We hypothesized that: (1) the body function subscale would be correlated fairly (positively) with the SF-36 physical functioning subscale; (2) SF-36 bodily pain; (3) body function would be correlated fairly (negatively) with the VAS; (4) activity and participation would be correlated fairly (positively) with the SF-36 physical functioning subscale; (5) the SF-36 social function; (6) environmental factors would be correlated fairly (positively) with the SF-36 physical functioning subscale; and (7) SF-36 vitality. Rule of thumb for cut point values of a correlation coefficient explained as 0.90 to 1.00 (very high correlation, 0.70 to 0.90 (high correlation), 0.50 to 0.70 (moderate correlation), 0.30 to 0.50 (low correlation), 0.00 to 0.30 (negligible correlation).²⁹

Statistical Analysis

The data gathered from experts and patients were used to examine content validity. The CVI was used to quantify content validity via the kappa statistic. All data were initially captured in Microsoft Excel 2007 version 12.0 (Microsoft, Redmond, WA, USA) spreadsheets and analyzed with the adopted equations.¹⁸

Descriptive statistics were used to examine the differences between participants regarding sex, age, body mass index (BMI), education, occupation, the severity of OA, knee involvement, knee OA duration, and eating position. The ceiling and floor effects for the new Arabic PRO measure and SF-36 were defined to be present if more than 15% of our patient collective achieved the highest (100 points) or lowest (0 points) scores.

Results

The study included 106 patients in total. Due to various medical issues, four patients withdrew from the study. We used the remaining 102 patients’ data to develop and validate the measure. In the first two rounds, 29 patients took part, and 73 patients took part in the third phase (Figure 2).

Figure 2 The flow of the study participants.

Patient Characteristics

The sample (n =73) comprised mostly female patients (59%), with a mean age of 58.1 years. Most were from Riyadh’s central region (80%) and were either self-employed or retired (70%). Most patients had completed at least their high school education (74%). The average BMI was 31.9 ± 5.5 kg/m². The participants reported the severity of knee OA as severe in 19%, moderate in 53%, and mild in 28% of patients. Symptom duration of knee OA was an average of 4.7 years, and 77% had bilateral knee OA (Table 1).

Table 1 Characteristics of the Participants

Item Generation

Four domains of the ICF framework and a pool of 81 items (Appendix A) were generated for the new Arabic measure based on a review of the relevant literature, examination of other PRO measures, and interview of 10 Saudi patients with knee OA to ensure that all relevant points were included.

Item Selection

Of the 29 patients participating in the first two phases, four (with education levels of high school or above) revised the pool of items with five leading experts. They selected 29 items by rating them based on the CVI. According to the CVI results, we retained three items, two were altered, six were combined, and five were added (Appendix B) based on experts’ advice and the OA items’ clinical relevance. The pre-final version of the new Arabic PRO measure was developed with 34 items within a framework (Figure 3).

Figure 3 A framework of the new Arabic patient-reported outcome measure.

Patients (n =15) with knee OA had varying educational levels and were asked to answer each item in the pre-final version during the interviews. We rewrote items that were misunderstood, ambiguous, or rarely answered with the help of the patients and by consensus among the experts. We created the final version of the new Arabic PRO measure (Appendix C) after deleting one item, rewording three items, and improving the instructions’ clarity. The translated Arabic version of PRO into English has been provided (Appendix D).

The floor and ceiling effects, mean and standard deviations (SDs), percentages relating to the new Arabic PRO measure, and SF-36 subscale scores were determined. The highest average scores for the new Arabic PRO measure subscales and SF-36 subscales were 70.5 for Body Structure and 51.5 for Mental Health. The lowest average scores were 46.7 for Activity & Participation and 38.5 for Physical Function. The ceiling and floor effects for the new Arabic PRO measure and SF-36 subscales were <15%, except for Body Structure, for which the effect was 47%.

Psychometric Properties

Table 2 presents the internal consistency results for the new Arabic PRO measure subscales. Cronbach’s alpha coefficients for the four subscales were acceptable to good. The highest Cronbach’s alpha was 0.85 for the subscale of Body Structure, while the lowest was 0.69 for the subscale of Activity & Participation.

Table 2 Internal Consistency of the New Arabic Patient-Reported Outcome Measure’s Subscales

Table 3 presents the test-retest reliability results for the new Arabic PRO measure subscales. Of the 73 patients participating in the third phase, 63 (86%) completed the second test in 7.4 days (on average). The highest ICC was 0.91 for the subscale of Activity & Participation, while the lowest was 0.88 for the subscale of environment factors. These results indicate excellent test-retest reliability.

Table 3 Test-Retest Reliability of the New Arabic Patient-Reported Outcome Measure’s Subscales

Table 4 provides the correlations among the subscales of the new Arabic PRO measure, SF-36, and VAS. Six of seven predefined hypotheses confirmed the construct validity of the measures. The correlation between the subscales Body Function and Physical Function (r_s= 0.63), Activity & Participation and Physical Function (r_s= 0.72), and Body Function and Bodily Pain (r_s= 0.58) were considered fair. In contrast, the subscales of Body Function and the VAS reported a negative correlation (r_s= –0.50). The correlations between the new Arabic PRO measure’s subscales and the SF-36 Physical Composite Scale (PCS: r_s= 0.34–0.69) were weak to fair compared to those with the Mental Composite Scale (MCS: r_s= 0.16–0.55).

Table 4 Relationship Between the New Arabic Patient-Reported Outcome Measure Subscales, Short Form-36 Health Survey Subscales, and Visual Analogue Scale

Discussion

The present study developed a new PRO measure in the Arabic language similar to the ICF framework and evaluated its psychometric properties among patients with knee OA in Saudi Arabia. The results supported the hypothesis that the new Arabic PRO measure is simple and relevant for patients with knee OA. This new Arabic PRO measure is a reliable and valid tool for evaluating and assessing the health status of patients with knee OA in Saudi Arabia.

Knee OA is among the ten most disabling conditions and is highly prevalent among older adults in developed countries.³⁰ More accurate assessment of the type of disability and functional impairment is vital for patients with knee OA.³¹ However, knee OA measurement is challenging, as there is a diverse spectrum of functional limitations and severities.³² As the World Health Organization (WHO) recommended,³³ we used the ICF framework and the ICF core set for OA to define the typical range of functional disabilities and environmental factors using a more systematic approach and a standard global language. Few studies have associated knee OA measures with the ICF core set for OA to better the understanding of the relationship between these measures and the ICF.^11,34

The Oxford Knee Score (OKS) offered items connected to the ICF main categories,³⁵ much as the PRO established here. The PROs of the Knee Injury and Osteoarthritis Outcome Score (KOOS) and the Western Ontario McMaster University Osteoarthritis Index (WOMAC) often evaluate activity and disability in patients with knee OA, but very few include patient participation.³⁶ However, neither the WOMAC nor the KOOS includes environmental components. Items of the praying position (“bending to the floor”) were left out of research done to adopt a modified WOMAC for the Arabic language. Consequently, a lower WOMAC score did not always accurately reflect the Saudi socio-cultural situation.¹³ Another Egyptian test of knee function that the Arabic culture has accepted has not only been given to knee OA patients.³⁷ Additionally, Saudi patients would also be unable to understand the Egyptian version due to cultural differences.

Most of the frequently employed knee-specific measures were reviewed in this study, focusing on knee OA measures to include information on signs and symptoms, body function, activity, and involvement. We used the ICF’s core-set category for knee OA to characterize the typical functional and physical difficulties patients experience.³² Only 29 items were selected from the things that obtained a 7 out of 9 on the CVI, precisely measures of knee OA to capture factors relating to signs and symptoms, body function, activity, and involvement. The ICF’s core-set category for knee OA was utilized to describe the normal range of functional and physical challenges patients face. Items that received a CVI score of 7 out of 9 were chosen (only 29 items). After reviewing the issues, the experts offered insightful commentary, which the committee members debated. Supplied the four adjustments listed below: Items are classified based on their clinical importance, the socio-cultural context in Saudi Arabia, frequency of use, and relationship to an ICF core-set category.

Our study encountered no difficulties while developing the new Arabic PRO measure items. The expert panel used the design of the KOOS to format the items, the wording of the instructions, and the scaling of responses; the scaling responses used in the KOOS employ descriptors along a continuum. This provided more flexibility and has often been applied for similar PROs, such as the OKS.³⁴ and the Osteoarthritis Knee and Hip Quality of Life.³⁸ Each item was measured on an adjectival rating scale from 0 to 4. A score for each subscale was transformed to a 0 to 100 scale, with a higher score indicating a better health status. Improvements in the quality of the new Arabic PRO measure included revision by an Arabic language expert who recommended the use of standard Arabic language that could be readily understood in Saudi culture and by individuals living in all regions of Saudi Arabia. Concerning the recall period, no single standard guideline was found to be best for all measures or all phenomena.³⁸ For this reason, the recall period was determined by consensus among experts.

Consistent with previous studies,^13,39 in the present study, most patients were female and had bilateral knee OA and high BMI scores. Saudis typically engage in five prayers daily and sit on the floor to eat, which requires full knee flexion; however, most patients in this study reported eating at the table (67%). This may explain why knee OA had a more significant effect on ADL in patients with knee OA in Saudi Arabia.⁴⁰ The ceiling and floor factor effect for the new measure subscales were low <15%, apart from the ceiling effect of the subscale of body structure, which was high. The ceiling factor of the subscale of body structure might be due to a significant proportion of patients having mild knee OA.

As shown in earlier studies, the internal consistency of the new Arabic PRO measure subscales was good and acceptable, indicating homogeneity with a high correlation among items.^23,41,42 The test-retest reliability was high, which meant that the stability of the new Arabic PRO measure overtime was good.⁴³ However, previous studies have reported that a period of one or two weeks was appropriate for determining test-retest reliability.²⁵ One week was chosen for repeated measurements to avoid answer recall, which could occur if the period is short, and to minimize clinical change, which could happen if the period is prolonged. Construct validity of the new Arabic PRO measure confirmed all predefined hypotheses,^38,44 except the correlations between the subscales of Environmental Factors and SF-36 Vitality were lower than expected.

The current study has several strengths. It was designed to develop a new Arabic PRO measure that captures concepts related to health, function, and participation, as well as environmental and personal factors for patients with knee OA in Saudi Arabia. Among the development phases, the new Arabic PRO measure covered many categories of the OA core sets. As stated, good coverage of the different classes is essential. For this reason, the ICF categories can serve to rate patients included in clinical studies with knee OA or to guide orthopedic and physical therapists’ assessment of patients with knee OA.³³ The study ensured that patients’ perspectives would be considered at all steps involved in developing the new Arabic PRO measure. We followed the recommended guidelines and used a robust statistical methodology to ensure the new Arabic PRO measure’s quality and avoid biased selection. Finally, the new Arabic PRO measure was developed in multiple settings using simple standard Arabic language to enhance its utility across all regions of Saudi Arabia and other Gulf-region countries.

The current study also has some limitations. First, the new Arabic PRO measure was tested in patients of an older age group with knee OA only. Second, the study could not report on the concurrent validity with standard knee OA measures such as the KOOS and WOMAC. SF-36 was considered to provide more general insights into patient health, and it may be more sensitive to changes than WOMAC in a heterogeneous population.⁴⁵ Third, we did not examine the new Arabic PRO measure’s responsiveness and sensitivity to patient condition changes. Finally, we developed the new Arabic PRO measure according to the Saudi cultural context; therefore, generalization of the study results to other Arabic-speaking countries may affect the external validity.

The new Arabic measure developed in this study allows for assessing important health factors using an international framework (ICF). It considers ADL unique to Saudi culture and environmental and individual characteristics. Therefore, researchers can compare local studies’ findings with those of other countries using the new Arabic measure, which is legitimate and dependable. Additionally, the new Arabic metric enables Saudi Arabia and other nations in the Gulf region to evaluate and determine the efficacy of any healthcare intervention for individuals with knee OA. We will need further research to see how well the new Arabic metric will evaluate therapeutic and surgical procedures and rehabilitation. Comparisons with other PRO measures could be helpful for the new Arabic measure (eg, the KOOS or the WOMAC). An additional investigation should look at the new Arabic measure in younger individuals with other knee problems.

Conclusion

This study found the 33-item new Arabic PRO measure to be a reliable, valid, and widely useable tool for patients with knee OA. More research is required to examine its application in different Arab nations and discover how responsive it is to ascertain the new Arabic PRO measure’s validity among older people. Further investigation of this new Arabic PRO measure for younger people with various knee problems is needed.