Development and Validation the Uterine Fibroid Symptom and Quality of Life Short-Form: Integrating Classical and Modern Test Theory

Introduction

Uterine fibroids (UFs), also known as uterine smooth muscle tumors or uterine myomas, are the most common benign tumors in the female reproductive system. Among women of childbearing age, the overall prevalence rate is approximately 60%-70%, and about 25% of these patients have obvious clinical symptoms that require treatment.^1,2 Patients with uterine fibroids, particularly premenopausal women undergoing medical or uterus-preserving treatment, often have a significant burden of symptoms, including moderate to severe abdominal pain,³ low back pain, urinary frequency and urgency, pain during intercourse,² and vaginal bleeding.⁴ Notably, several of these presenting symptoms are not specific to fibroids and may overlap with malignancy.⁵ Furthermore, fibroids can distort uterine anatomy, which may potentially obscure visualization or delay the early diagnosis of aggressive cancer subtypes.⁶ Given the high prevalence and symptom burden among women, UFs impose a significant healthcare burden on women’s individual health as well as on the healthcare and social security systems as a result of work productivity loss during treatment and disease recurrence.^7,8 Consequently, measuring UF-related symptoms and QoL status in a valid and reliable manner could support high-quality practice and comprehensive patient management.

As the only questionnaire designed to assess the full spectrum of fibroid-related symptoms and their impact on QoL, the English version of the Uterine Fibroid Symptom and Quality of Life (UFS-QoL) questionnaire, published in 2002,⁹ is applicable only to women with uterine fibroids who have retained their uterus and still have a normal menstrual cycle, and not to those who have undergone hysterectomy. The UFS-QoL questionnaire consists of 37 items, eight of which assess the severity of symptoms (single domain) and 29 assess health-related quality of life (HRQL) in six subscales (concern, activities, energy/mood, control, self-consciousness, and sexual function). All the responses were classified into five Likert-scale options. The UFS-QoL questionnaire has been used and validated in Brazilian Portuguese,^10,11 Spanish,¹² traditional Chinese,¹³ and simplified Chinese^14,15 as a disease-specific measure of health-related QoL.

For an instrument to be clinically useful, an obstacle to its routine use is its length. Although the 37 items of the Chinese UFS-QoL—which contains the same number of items as the original English version—assess overall patient information, the length might deter its clinical efficacy, as it is time-consuming for daily high-frequency care, especially in primary hospitals with low resource settings. Shorter questionnaires (one that achieves the same effect but with fewer items) that improve response rates and reduce participant burden are recommended as follow-up measures.^16,17 Our group evaluated the adaptability and clinical applicability of the Chinese UFS-QoL.¹⁵ We found that the symptom severity, activity, and mood subscales of the Chinese UFS-QoL scale were valid and reliable. However, psychometric evidence revealed that the self-consciousness and sexual functioning subscales exhibited ceiling effects > 15%. Notably, the self-consciousness domain showed poor internal consistency (α= 0.56) and demonstrated the lowest detectability, with both effect size and standardized response means being only 0.38 at 6- and 12-months post-treatment. We pointed out before applying the UFS-QoL to Chinese patients, the cross-cultural test criteria of adjusting the subscales to the Chinese culture, shortening or deleting the poor discrimination items to ensure the scale validity, and further optimizing and reevaluating the practicability of these items, should be considered. Therefore, there are both practical application challenges and psychometric evidence to pursue the construction of a shorter version of the UFS-QoL while maintaining its reliability, validity, and clinical applicability and enhancing routine clinical assessments.

In this study, we employed different psychometric techniques from both classical test theory (CTT) and modern test theory (item response theory [IRT])¹⁸ to develop a short-form version of the UFS-QoL. While CTT identifies the basic properties of the data, such as eigenvalues and factor loadings, IRT provides more detailed information at the item level. These techniques complement each other when developing short scales.^18,19 Using the short-form version of the UFS-QoL, we designed it to reduce the respondent burden and be short enough for use in clinical practice and research. We predicted that the short-form version would have similar psychometric properties and reproduce scores that would be useful in assessing the efficacy of therapies without the loss of information compared with the original 37-item UFS-QoL.

Materials and Methods

To develop the short-form scale, we extracted data from a prospective cohort study that included 20 centers (Uterine Fibroids Multicenter Network Information System: www.hifuctr.com) for a secondary analysis, which included patients who underwent self-selected hysterectomy, myomectomy, or high-intensity focused ultrasound (HIFU) therapy after being fully informed of the treatment options (the multicenter study was approved by a China-registered clinical trial ethics committee (ChiECRCT-2011034).²⁰ Details regarding the study design, data collection, and primary outcomes regarding the efficacy of the treatment have been published.²⁰ Prior to undergoing any study-related procedures at the clinical site, the patients completed the UFS-QoL questionnaire, the study short-form-36 (SF-36), and a brief sociodemographic questionnaire. Follow-up visits were scheduled at 6 and 12 months post-surgery, including complications, magnetic resonance imaging evaluation, overall treatment effect evaluation, the UFS-QoL questionnaire (for those who had undergone HIFU or myomectomy, because the instructions of the UFS-QoL questionnaire were based on the presence of uterine fibroids and menstrual periods), SF-36, and several health care utilization items were recorded.

Patient-Reported Outcome Measures

Uterine Fibroid Symptom and Quality of Life Questionnaire (UFS-QoL)

The UFS-QoL was developed from focus groups of women with uterine fibroids.^9,14 The Original UFS-QoL questionnaire consists of 37 items, 8 of which assess the severity of symptoms (single domain) and 29 of which assess HRQL with six subscales (concern, activities, energy/mood, control, self-consciousness, and sexual function). All the responses were classified into five Likert-scale options. A higher score on the questionnaire’s severity subscale indicates more severe symptoms, whereas a lower score on the HRQL subscales indicates poorer QoL.

Medical Outcomes of the Study Short-Form 36 (SF-36)

The Chinese SF-36 is a 36-item self-administered generic measure used to assess general health status and validated its cross-cultural application, reliability and validity.^21–24 SF-36 consists of eight subscales: physical functioning, role-physical, bodily pain, general health, vitality, social functioning, role-emotional, and mental health, as well as two composite scores: physical and mental. Individual subscale items were combined to form a subscale rating, which was then converted to a 0–100 scale.^25,26 Higher QoL scores correspond to a four-week recall period.²⁶

Item Selection

We selected two items per UFS-QoL scale using a method that maximizes content validity.^16,27 Due to the poor internal consistency and ability to detect self-consciousness in our previous study,¹⁵ we did not retain the three items of self-consciousness. At the same time, because sexual functioning showed poor adaptability and high correlation between the two items, we only kept one item of the two items. We used the CTT and IRT to select two items for each of the five subscales. For the inconsistent items extracted from the two methods, we used responsiveness to choose items with higher ability to detect changes in clinical therapy. Therefore, we administered the 11-item version (UFS-QoL-11) to test its factor structure, criterion-related validity, and responsiveness, following the recommendations for short-form scale development. A flow chart of item selection is shown in Figure 1.

Figure 1 Flow charts of items selection.

Based on the CTT to choose shortened items, we followed a method using regression analysis that maximizes the content validity of a 2-item scale. The first item had the highest correlation with the original scale. The second item selected was the one with the highest beta weight of the remaining items in multiple regression, with the original scale score as the dependent variable and individual items as predictor variables.

Item response theory offers detailed information at the item level, making it a powerful technique for developing short-form scales.¹⁸ The responses of participants to each item were used to estimate their location on a latent trait (ie, the level of symptoms). This latent trait is estimated using the responses on all individual items, and not, as in the CTT, via the sum score of all items. Within the IRT, every single item is defined by a discrimination parameter (alpha) and one or more location parameters or threshold parameters. The threshold parameters indicate the location on the scale of the latent continuum where the item best discriminates among individuals. The discrimination parameter reflects the true difference in theta per item and is comparable to factor loading.

To determine the candidate items for the short-form of the scale, we selected the items with two of the highest discriminatory parameters, as given by the IRT analyses. As the scale has five ordered response options (1 = “not at all”, 2 = “A little bit”, 3 = “Somewhat”, 4 = “A great deal”, 5 = “A very great deal”), we used the graded response model to estimate the item response parameters. In addition, we inspected the item information curves to select items that covered a similar range of latent traits as the full scale. Finally, we compared the effect sizes of inconsistent items to choose the final items.

Statistical Analysis

Spearman correlation coefficient was used to determine the strength of the UFS-QoL-11 correlation test with the original UFS-QoL. To examine construct validity in the UFS-QoL-11, we employed principal axis factor analysis with orthogonal rotation, which was used in conjunction with orthogonal rotation to determine the final number of factors based on their eigenvalues, congruence, and clinical significance. Each dataset was first analyzed for normality using the Kaiser–Meyer–Olkin (KMO) measure and Bartlett test of sphericity. A KMO value > 0.5 indicated acceptable structural validity. Cronbach’s α was used to determine the internal consistency of the UFS-QoL-11. Cronbach’s α coefficient, which ranges from 0 to 1, was used to determine the degree to which items on the subscales measure were related to the same concepts. A larger value indicated a smaller measurement error, which indicated a higher level of reliability. Criterion validity was assessed using correlations between baseline scores on the UFS-QoL-11 and SF-36, which were designed to measure general health.

We examined the evidence of known-group validity so that the UFS-QoL-11 could distinguish between clinically distinct groups by testing its ability to differentiate between patients based on health status (defined as SF-36-1). The ability to detect change was evaluated by comparing the 6-month pre-treatment and post-treatment scores to the 12-month scores at 6-month intervals. The effect size (change in mean score divided by baseline standard deviation)²⁸ and standardized response mean (change in mean score divided by change standard deviation) were computed. A value of 0.2 was considered to have a “small” effect, 0.5 a “moderate” one, and ≥ 0.8 a “large” effect.

Descriptive analyses (means and standard deviations [SD]) were performed using sociodemographic and clinical characteristics. Means of differences, 95% confidence intervals (CI), and statistical significance (P < 0.05) were tested using independent sample t-tests. The questionnaires were scored according to developers’ instructions. Version 9.1.3 of SAS was used to conduct the analyses. All statistical tests were predetermined, and no missing data imputations were performed. All statistical tests were conducted with a fixed type I error probability of 0.05 and a two-tailed design.²⁵

Results

According to the screening criteria, we chose items based on the CTT and IRT results. All assumptions of IRT were met (ie, the scale is unidimensional, and there was no noteworthy local dependence and answer authenticity) (see Supplements Table S1-S2). As shown in Table 1, the CTT and IRT parameters per item included item–total correlations ranging from 0.483 to 0.841 and IRT discrimination parameters ranging from 0.841 to 3.709, indicating adequate item discrimination ability. The control and sexual function subscales showed consistent results between the two methods. Table 2 demonstrates a shortened scale with nearly identical responsiveness to original UFS-QoL. By comparing the effect size and standardized response means (SRM) in other subscales, the symptom severity and activities subscale showed higher responsiveness to CTT items. However, Concern and Energy/Mood showed a higher ability to detect changes in IRT items. As shown in Table S3, all items had a distribution of <1% at every follow-up. The final UFS-QoL-11 questionnaire is presented in Supplementary information, Figure S1. The scoring method for the UFS-QoL-11, adapted from the original UFS-QoL scoring manual, is provided in the Supplementary Method. In Figure 2, the selected items showed strong correlations with the corresponding domains of the original UFS-QoL scale, with correlation coefficients ranging from 0.73 to 0.94 (all P < 0.001).

Table 1 Items Selection with CTT and IRT in Development Data Set (n =969)

Table 2 Responsiveness of Women with Uterine Fibroids After Treatment Among Original Scale, CTT and IRT in Development Data Set (n=969)

Figure 2 Correlation between selected items and original UFS-QoL in each subscale.

In the factor analysis, values of the KMO test > 0.7, and the statistical significance of Bartlett’s test indicated adequate sampling.^29,30 We used orthogonal rotation to isolate the potential UFS-QoL-11 factors (Table 3). In this study, the value of KMO was 0.86, and the value of Bartlett’s sphericity test was 2536.26 (P < 0.001), indicating that factor analysis was suitable for the data. The UFS-QoL-11 retains three factors: symptom severity, concern, and daily life interference. Factor loadings ranged from 0.543 to 0.897, the test variables explained 64.22% of the total variance, which was slightly higher than the original scale score of 63.60%.¹⁵

Table 3 Construct Validity of the UFS-QoL-11: Baseline Factor Loadings in Validation Data Set (n=970)

Cronbach’s alpha was calculated for each UFS-QoL-11 subscale to assess internal consistency. The UFS-QoL-11 questionnaire demonstrated good internal consistency and reliability for all the subscales (> 0.7) (Table 4). Cronbach’s α values were 0.81 for symptom severity, 0.83 for concern, and 0.84 for daily life interference (Table 4).There was a small increase in internal consistency for the UFS-QoL-11 compared with the full UFS-QoL. The degree of correlation between similar subscales on the UFS-QoL-11 and SF-36 was used to assess the reliability of the correlation testing for the UFS-QoL-11 (Supplements Table S4). All domains of the SF-36 correlated positively and moderately with the “daily life interference” domain of the UFS-QoL-11 with correlation coefficients ranging from 0.35 to 0.51 (all P < 0.001).

Table 4 Internal Consistency Reliability of UFS-QOL-11 Subscales in Validation Data Set (n=970)

The UFS-QoL-11 was sensitive enough to detect varying levels of current health status, particularly at 6 and 12 months post-surgery (Table S5). At 6 months, patients with good health had lower symptom severity scores than those with poor health (11.54±13.37 vs 17.07±17.97; mean difference 5.5, 95% CI 3.1–8.0), and higher scores for concern and daily life interference. Similar differences were observed at 12 months, with larger effect sizes (0.74–1.41), indicating good sensitivity to differences in health status. Similar but larger differences were observed at 12 months, with effect sizes ranging from 0.68 to 1.41 (all P < 0.001).

The responsiveness of UFS-QoL-11 was comparable to that of original UFS-QoL; all effect size>0.45, displayed a good ability to detect changes (Table 5). After treatment, there was a significant decrease in symptom severity scores and improvement in the concern and daily life interference subscale scores. The mean score change from baseline to the 12-month follow-up for symptom severity was −19.4 (P < 0.001), with an effect size of −0.78 and a SRM of −0.98 (Table 5). The mean change scores for the other two subscales (concern and daily life interference) ranged from 11.6 to 19.5, and the effect sizes of the two subscales were between 0.66 and 0.72. Moreover, the SRM values at 12 months after treatment were between 0.75 and 0.82.

Table 5 Responsiveness of Women with Uterine Fibroids After Treatment Using UFS-QoL-11 in Validation Data Set (n=970)

Discussion

In this study, we used the largest cohort of patients with UFs through different psychometric techniques, CTT and IRT, to arrive at a reliable and valid short-form of UFS-QoL, which is the only questionnaire designed to assess the symptoms of UFs and their impact on QoL. Importantly, for clinical application, we used responsiveness to confirm the inconsistency between the CTT and IRT. There was a strong correlation between original UFS-QoL and its 11-item short-form, UFS-QoL-11, which supports the use of the scale in other populations as well. Factor analysis supported the hypothesized three-factor structure in 11-item, symptom severity, concern, and daily life interference. The scale and its subscales demonstrated good internal consistency, structural validity, known-group validity, and responsiveness. Throughout the treatment process, UFS-QoL-11 would be feasible for monitoring symptoms and QoL in clinical routine and long-term practice, and would enable clinicians and researchers to distinguish between symptoms and QoL changes sensitively.

The CTT and IRT are important approaches for optimizing psychological scales. According to statistics, several items were good candidates for a short-form scale to provide maximum information using two methods. An important barrier was that the statistics did not provide a clear suggestion for the selection of the best items using different approaches.¹⁹ The Food and Drug Administration (FDA) claims that a patient-reported outcome (PRO) questionnaire is an instrument not only with satisfactory psychological criteria, but should also measure what it is intended to measure, which means that the item could be sensitive enough to assess the concept of interest with each subscale in Chinese patients with UFs.³¹ Therefore, we combined these statistical parameters with clinical applicability that ability to detect change score to confirm the item. We strongly urge the use of these 11 items when assessing patients or research participants to maximize the comparability of clinical assessments.

Brief measures are favorable for clinical practice to reduce patient burden and infringement upon encountering time.¹⁷ Overall, in the present sample and the original validation study, the UFS-QoL-11 performed as well as the full version in the measures of reliability, validity, and responsiveness as the full version.¹⁵ When comparing the correlations between the UFS-QoL-11 and the full scale, the correlations were nearly identical. In sum, the UFS-QoL-11 scale will provide comparable information about the latent trait of symptoms and QoL for each respondent as the full scale currently does. Simultaneously, its brevity will increase the likelihood that the assessment of symptoms and QoL will be implemented in everyday clinical practice. Therefore, the UFS-QoL-11 can be easily administered during a typical outpatient visit and may serve as a discussion point to help clinicians understand patients’ symptom experiences, especially when symptoms appear refractory to treatment or physiological improvement.

Although the results of the present study are encouraging, there are several limitations that should be highlighted so that the findings are considered with caution and in light of these deficiencies. First, although the UFS-QoL-11 demonstrated good internal consistency and concurrent validity, we could not examine other important psychometric properties such as test-retest reliability. Future studies should examine these psychometric properties for the purpose of a comprehensive psychometric evaluation of the UFS-QoL-11 among the Chinese. Second, the validation dataset (n = 970) was derived from the same prospective cohort as the development dataset (n = 969) through internal validation via sample splitting. This approach only provides preliminary psychometric evidence and cannot substitute for external validation in an independent cohort. As such, we have outlined our future research plan to conduct external validation of the UFS-QoL-11 in an independent multicenter cohort of Chinese women with symptomatic uterine fibroids, aiming to confirm its factor structure and psychometric properties across different populations. Further external validation is needed to assess the adaptability and clinical applicability of the UFS-QoL-11 in real-world settings.

Conclusion

Our findings demonstrated that the UFS-QoL-11 performs as well as the original 37-item version while reducing the administrative burden by 70%. Its brevity, if implemented, will increase the likelihood that the assessment of symptoms and QoL will be implemented in everyday clinical practice. As patients with UFs experience a high symptom burden and QoL interference, we hope that the development of UFS-QoL-11 will lead to more widespread, accurate, timely, and sensitive monitoring, in clinical and research settings.