Validation of EORTC IN-PATSAT32 for Chinese patients with gastrointestinal cancer

Introduction

Patient satisfaction can be defined as the extent to which an individual’s health care experiences match his or her expectations.¹ Evidence has emerged indicating that patient satisfaction is related to patients’ adherence to cancer treatments, and is a key indicator of health care quality.^2,3 Moreover, the collection of patient satisfaction information can be used to describe the status of the medical institutions in order to identify areas in need of improvement. However, instruments for assessing the cancer inpatients’ perceptions of the quality of hospital-based cancer care are rare.^4–11 In recent years, the European Organization for Research and Treatment of Cancer (EORTC) Quality of Life Group has developed the EORTC In-patient Satisfaction with Care Questionnaire 32 (EORTC IN-PATSAT32), a questionnaire to assess patient satisfaction, which has been validated in a multicenter study with patients from nine countries.^12,13 Until now, no studies have been carried out investigating cancer patients’ satisfaction in the People’s Republic of China, and validation of the EORTC IN-PATSAT32 is in need. This study aims to assess the reliability, validity, and acceptability of EORTC IN-PATSAT32 in Chinese patients with gastrointestinal cancer, and to provide the basis for its clinical application.

Patients and methods

Patients

A total of 106 inpatients with gastrointestinal cancer in Cangzhou Center Hospital from July 2013–March 2014 were consecutively selected for this study, including 56 nonmetastatic gastrointestinal cancer patients, 38 metastatic gastrointestinal cancer patients, and eleven nonstaged gastrointestinal cancer patients. Patients had to be diagnosed with gastrointestinal cancer, be aged 18 years or older, had to be hospitalized for at least 3 days, and they also had to be mentally fit to complete a questionnaire. All participants self-administered the EORTC IN-PATSAT32 and EORTC Quality of Life Questionnaire – Core 30 (EORTC QLQ-C30).

Instruments

EORTC QLQ-C30

The current EORTC QLQ-C30 is a 30-item, cancer-specific scale addressing various aspects of health-related quality of life and is classified into 15 domains including five functional subscales: physical functioning; role functioning; emotional functioning; cognitive functioning; and social functioning. In addition, there are three multi-item symptom subscales: fatigue; nausea/vomiting; and pain. Finally, there is a global quality of life subscale, and six single items addressing various symptoms and perceived financial impact.¹⁴

EORTC IN-PATSAT32

The EORTC IN-PATSAT32 is composed of 32 items assessing cancer patients’ perceptions of the quality of hospital doctors and nurses, as well as selected aspects of the care organization and hospital environment that are relevant across national settings. The questionnaire is organized into eleven multi-item scales, including doctors’ and nurses’ technical skills (for example, knowledge, experience, assessment of physical symptoms), interpersonal skills (for example, interest, willingness to listen), information provision (for example, about the disease, medical tests, and treatment), and availability (for example, time devoted to patients); other hospital staff members’ interpersonal skills; wait times; hospital access; and three single items consisting of information exchange, hospital comfort, and overall satisfaction. A “poor”, “fair”, “good”, “very good”, or “excellent” response scale is used to rate each aspect of care.

Basic information form

Additional data collected included patients’ age, sex, nationality, marital status, employment status, education level, tumor stage, therapy, and therapeutic goal.

Data collection procedures and analysis

Patients were invited to participate in the study after an interview. Prospective participants were given oral and written information about the aim of the study and the assessment to be conducted. Patients who agreed to participate completed the questionnaires once. Each scale score was obtained by averaging the item scores within the scales, and they were then linearly converted to a 0–100 scale. A higher score reflected a higher level of satisfaction. Internal consistency reliability was determined by Cronbach’s α coefficient for each scale and single item, with a Cronbach’s α coefficient ≥0.7 considered to be statistically significant and ≥0.8 for satisfactory reliability.¹⁵ Multitrait scaling analysis was adopted to examine the item’s convergent validity (item scale correlation >0.4 as the criterion) and item discriminant validity (an item–own scale correlation higher than the correlation with the other scales).¹⁶ The divergent validity for each scale in EORTC IN-PATSAT32 and EORTC QLQ-C30 was analyzed by evaluating Pearson’s correlation coefficient, with r<0.4 considered a weak correlation. Acceptability was evaluated using the ratio of questionnaires to the miss rate of each item. All data were analyzed using SPSS 21.0 software. P<0.05 was considered to be significant.

Ethics

This study was approved by the ethics committee of Cangzhou Center Hospital. Before taking the survey, participants were asked to provide signed informed consent to demonstrate their willingness to take part in this study and to ensure their rights of voluntary participation and privacy.

Results

Sociodemographic and clinical characteristics

Sociodemographic characteristics for the 106 participants are shown in Table 1. Missing values existed due to limitations in the clinical data collection. Of these 106 patients, 68 (64.8%) were male and 37 (35.2%) were female. The patients varied in age from 26–77 years old with a mean age of 59 years old. Meanwhile, 103 (98.1%) patients were of Han nationality, 100 (98.0%) had been married, 47 (46.5%) had a compulsory educational level, and 49 (46.7%) had retired. As for lesions, 57 (60.0%) patients had nonmetastatic tumors, while 38 (40.0%) patients had metastatic tumors. Surgery was the predominant therapy in 76 (72.4%) participants, and 79 (79.8%) patients aimed for a complete cure.

Table 1 Sociodemographic and clinical characteristics of patients (n=106) Abbreviation: n, number.

The floor effect and ceiling effect

The floor effect (a large proportion of patients scoring at the minimum) and ceiling effect (a large proportion of patients scoring at the maximum) were adopted to assess the range of scales. The results showed that the floor effect was absent in all scales. Meanwhile, a ceiling effect existed in all scales, which was lower than 20% for most scales, except for the doctors’ availability scale. The highest for the doctors’ availability scale was 22.6%, and the lowest for the other hospital staff members’ interpersonal skills as well as wait time scales were 4.8% for both (Table 2).

Table 2 The floor effect and ceiling effect Abbreviations: SATDTS, doctors’ technical skills; SATDIS, doctors’ interpersonal skills; SATDIP, doctors’ information provision; SATDAV, doctors’ availability; SATNTS, nurses’ technical skills; SATNIS, nurses’ interpersonal skills; SATNIP, nurses’ information provision; SATNAV, nurses’ availability; SATEXE, information exchange; SATOTH, other hospital staff members’ interpersonal skills; SATWAI, wait times; SATACC, hospital access; SATCOM, hospital comfort; SATGEN, overall satisfaction.

Patient satisfaction

Scores of scales in EORTC IN-PATSAT32 varied from 67.98±3.97 to 78.89±13.51. Scales including hospital comfort (67.98±13.97), wait times (69.40±13.31), and other hospital staff members’ interpersonal skills and information provision (72.54±12.01) had the lowest scores. In addition, satisfaction with doctors including doctors’ availability (78.89±13.51), doctors’ information provision (78.46±12.49), and doctors’ interpersonal skills (78.33±13.75) turned out to possess the highest scores.

Reliability

Internal consistency reliability was determined by Cronbach’s α coefficient for each scale (Table 3). The Cronbach’s α coefficients were >0.7 for all multi-item scales, ranging from 0.74–0.93. Moreover, the Cronbach’s α coefficients were >0.8 for each scale, except for the hospital comfort scale (0.74), indicating that there was satisfactory internal consistency reliability for EORTC IN-PATSAT32.

Table 3 Multitrait scaling analysis and Cronbach’s α coefficient for EORTC IN-PATSAT32 Notes: aThe scores ranged from 0–100, with a higher score representing more levels of satisfaction. bA Cronbach’s α coefficient ≥0.7 was considered to be statistically significant, and ≥0.8 was considered for satisfactory internal consistency reliability. cPearson’s correlation coefficient was adopted, and r<0.4 indicated a weak correlation, 0.4–0.6 indicated a mild correlation, >0.6 indicated a strong correlation. Abbreviations: EORTC IN-PATSAT32, European Organization for Research and Treatment of Cancer In-patient Satisfaction with Care Questionnaire 32; SD, standard deviation; SATDTS, doctors’ technical skills; SATDIS, doctors’ interpersonal skills; SATDIP, doctors’ information provision; SATDAV, doctors’ availability; SATNTS, nurses’ technical skills; SATNIS, nurses’ interpersonal skills; SATNIP, nurses’ information provision; SATNAV, nurses’ availability; SATOTH, other hospital staff members’ interpersonal skills; SATWAI, wait times; SATACC, hospital access.

Validity

Multitrait scaling analysis

Results for the multitrait scaling analysis are displayed in Table 3. Item–scale correlation in each scale (corrected for overlap) exceeded the 0.4 criterion for item convergent validity. Item discriminant validity tests showed that in 50.0% of the cases, an item had a significantly higher correlation with its own scale (corrected for overlap) than with the other scales, meeting the criterion of discriminant validity. Though the remaining correlations for an item with its own scale were high, they were still lower than the correlation between this item with the other scales.

Correlation between EORTC IN-PATSAT32 and EORTC QLQ-C30

Pearson’s correlation coefficients for each scale in EORTC IN-PATSAT32 and EORTC QLQ-C30 are shown in Table 4. A weak correlation between the scales and the single items of the EORTC IN-PATSAT32 and EORTC QLQ-C30 was discovered (r<0.4).

Table 4 Correlations between different scales in EORTC IN-PATSAT32 and EORTC QLQ-C30 Note: *Correlation is significant at the 0.05 level (2-tailed). Abbreviations: EORTC IN-PATSAT32, European Organization for Research and Treatment of Cancer In-patient Satisfaction with Care Questionnaire 32; EORTC QLQ-C30, European Organization for Research and Treatment of Cancer Quality of Life Questionnaire – Core 30; PF, physical function; RF, role function; EF, emotional function; CF, cognitive function; SF, social function; FA, fatigue; NV, nausea and vomiting; PA, pain; DY, dyspnea; SL, insomnia; AP, appetite loss; CO, constipation; DI, diarrhea; FI, financial difficulties; QOL, quality of life; SATDTS, doctors’ technical skills; SATDIS, doctors’ interpersonal skills; SATDIP, doctors’ information provision; SATDAV, doctors’ availability; SATNTS, nurses’ technical skills; SATNIS, nurses’ interpersonal skills; SATNIP, nurses’ information provision; SATNAV, nurses’ availability; SATEXE, information exchange; SATOTH, other hospital staff members’ interpersonal skills; SATWAI, wait times; SATACC, hospital access; SATCOM, hospital comfort; SATGEN, overall satisfaction.

Known-groups comparisons

Groups were formed based on patient age (<59 years old and ≥59 years old), and education level (lower than compulsory education and compulsory education or higher). Differences of EORTC IN-PATSAT32 scores in different groups were assessed using the Mann–Whitney U-test (Table 5). Patients younger than 59 years old scored higher than those who were older than 59 years old, except on the nurses’ availability and hospital comfort scales. However, the influence of age categories on the EORTC IN-PATSAT32 scales was not statistically significant (P>0.05), demonstrating a weak correlation between patient satisfaction and age. Additionally, patients who had finished compulsory education (or greater) achieved higher scores than those who had not. Moreover, significant differences in scores for patients with different educational levels across the doctors’ information provision, nurses’ availability, and other hospital staff members’ interpersonal skills scales were found (P<0.05).

Table 5 Known-groups comparisons of scale scores for patients with different ages and educational levels in the EORTC IN-PATSAT32 Abbreviations: EORTC IN-PATSAT32, European Organization for Research and Treatment of Cancer In-patient Satisfaction with Care Questionnaire 32; n, number; SATDTS, doctors’ technical skills; SATDIS, doctors’ interpersonal skills; SATDIP, doctors’ information provision; SATDAV, doctors’ availability; SATNTS, nurses’ technical skills; SATNIS, nurses’ interpersonal skills; SATNIP, nurses’ information provision; SATNAV, nurses’ availability; SATEXE, information exchange; SATOTH, other hospital staff members’ interpersonal skills; SATWAI, wait times; SATACC, hospital access; SATCOM, hospital comfort; SATGEN, overall satisfaction.

Acceptability

All participants self-administered the EORTC IN-PATSAT32. The questionnaire was easily understood with a low miss rate. Of all 32 questions, the miss rates were 0.0% for eight (25%) questions, 0.9% for 23 (71.9%) questions, and 1.9% for one (3.1%) question, which indicated that all questions were satisfactorily acceptable.

Discussion

This study is an attempt to address the need for a reliable, valid, and acceptable instrument to assess inpatient satisfaction. The overall results emerging from the psychometric tests of the EORTC IN-PATSAT32 provide support for the reliability, validity, and acceptability of the questionnaire when applied to Chinese inpatients with gastrointestinal cancer.

Patient satisfaction

We noticed that high scores were obtained in each scale, revealing generally favorable patient satisfaction. Meanwhile, satisfaction with doctors including doctors’ availability, doctors’ information provision, and doctors’ interpersonal skills possessed the highest scores, suggesting that Chinese patients tend to recognize doctors for their general ability, rather than nurses and other hospital staff members. Conversely, previous research showed that patients usually displayed a high level of satisfaction toward nurses instead of doctors.¹⁷

Reliability

We discovered that most of scales in EORTC IN-PATSAT32 had great internal consistency reliability, except for hospital comfort. Although the two items on the hospital comfort scale, including ease of access (parking, means of transport, etc) and the ease of finding one’s way to the different departments, could reflect the same issue (the hospital comfort scale); these items focused on conceptually different issues, resulting in a low correlation between them. Research conducted in Spain and by the EORTC drew the same result.^13,18

Validity

Through psychometric tests, the only deficiency of the EORTC IN-PATSAT32 was that it did not show good discriminant validity for Chinese patients, confirmed by our results that 65.5% items had a significantly lower correlation with its own scale than with the other scales. Among all scales, the doctors’ technical skills and nurses’ technical skills appeared to possess the highest scaling error rate. Since the patients surveyed could not judge technical skills given that their medical knowledge was poor, this may have led to the deviation in scores. Moreover, “the attention doctors/nurses paid to your physical problems” item correlated weakly with doctors’/nurses’ technical skills. Instead, it clearly correlated with the doctors’ and nurses’ availability. However, research from the EORTC showed that an item correlated significantly higher with its own scale than with other scales, and a study in Spain also showed that only a few correlation coefficients for one item with its own scale were smaller than those with other scales.^13,18 The difference between our study and previous research may be due to cultural diversity. Furthermore, satisfactory convergent validity was confirmed, which was similar to the findings of previous studies.^13,18

Consistent with our expectations, EORTC IN-PATSAT32 did not correlate significantly with EORTC QLQ-C30, indicating that the former questionnaire assesses conceptually different issues (patient satisfaction) from the latter questionnaire (patients’ quality of life). Similarly, studies from the EORTC and Spain have confirmed the same findings.^13,18

Acceptablity

Coinciding with previous studies, the EORTC IN-PATSAT32 revealed a favorable level of acceptability.^13,18 “The waiting time for obtaining results of medical tests” was shown to be the item that possessed the highest scaling miss rate of 6%, while in our study, its miss rate turned out to be very low. Besides, we discovered that the miss rates of all the items were relatively low, with the highest rate at 1.7%, indicating that the EORTC IN-PATSAT32 could be used by Chinese inpatients to a great degree.

Conclusion

In conclusion, the results of this study support the EORTC IN-PATSAT32 as a reliable, valid, and acceptable instrument for evaluating patients with gastrointestinal cancer, and it is also appropriate for measuring patient satisfaction among Chinese patients.

Author contributions

All authors met the following criteria:

1. substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data;
2. drafting the article or revising it critically for important intellectual content;
3. final approval of the version to be published; and
4. agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Disclosure

The authors report no conflicts of interest in this work.

---

References

1.		Pascoe GC. Patient satisfaction in primary health care: a literature review and analysis. Eval Program Plann. 1983;6(3–4):185–210.
2.		Guldvog B. Can patient satisfaction improve health among patients with angina pectoris? Int J Qual Health Care. 1999;11(3):233–240.
3.		Draper M, Cohen P, Buchan H. Seeking consumer views: what use are results of hospital patient satisfaction surveys? Int J Qual Health Care. 2001;13(6):463–468.
4.		Oberst MT. Methodology in behavioral and psychosocial cancer research. Patients’ perceptions of care. Measurement of quality and satisfaction. Cancer. 1984;53(10 Suppl):2366–2375.
5.		Wiggers JH, Donovan KO, Redman S, Sanson-Fisher RW. Cancer patient satisfaction with care. Cancer. 1990;66(3):610–616.
6.		Loblaw DA, Bezjak A, Bunston T. Development and testing of a visit-specific patient satisfaction questionnaire: the Princess Margaret Hospital Satisfaction With Doctor Questionnaire. J Clin Oncol. 1999;17(6):1931–1938.
7.		Loblaw DA, Bezjak A, Singh PM, et al. Psychometric refinement of an outpatient, visit-specific satisfaction with doctor questionnaire. Psychooncology. 2004;13(4):223–234.
8.		Defossez G, Mathoulin-Pelissier S, Ingrand I, et al; REPERES research network. Satisfaction with care among patients with non-metastatic breast cancer: development and first steps of validation of the REPERES-60 questionnaire. BMC Cancer. 2007;7:129.
9.		Yun YH, Kim SH, Lee KM, et al. Patient-reported assessment of quality care at end of life: development and validation of Quality Care Questionnaire-End of Life (QCQ-EOL). Eur J Cancer. 2006;42(14):2310–2317.
10.		Radwin L, Alster K, Rubin KM. Development and testing of the Oncology Patients’ Perceptions of the Quality of Nursing Care Scale. Oncol Nurs Forum. 2003;30(2):283–290.
11.		Abetz L, Coombs JH, Keininger DL, et al. Development of the cancer therapy satisfaction questionnaire: item generation and content validity testing. Value Health. 2005;8 Suppl 1:S41–S53.
12.		Sprangers MA, Cull A, Groenvold M, Bjordal K, Blazeby J, Aaronson NK. The European Organization for Research and Treatment of Cancer approach to developing questionnaire modules: an update and overview. EORTC Quality of Life Study Group. Qual Life Res. 1998;7(4):291–300.
13.		Brédart A, Bottomley A, Blazeby JM, et al; European Organisation for Research and Treatment of Cancer Quality of Life Group and Quality of Life Unit. An international prospective study of the EORTC cancer in-patient satisfaction with care measure (EORTC IN-PATSAT32). Eur J Cancer. 2005;41(14):2120–2131.
14.		Aaronson NK, Ahmedzai S, Bergman B, et al. The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst. 1993;85(5):365–376.
15.		Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16(3):297–334.
16.		Ware JE, Harris WJ, Gandek B, Rogers BW. MAP-R for Windows: Multitrait-Multi-Item Analysis Program – Revised User’s Guide. Boston, MA: Health Assessment Lab; 1996:161.
17.		Hjörleifsdóttir E, Hallberg IR, Gunnarsdóttir ED. Satisfaction with care in oncology outpatient clinics: psychometric characteristics of the Icelandic EORTC IN-PATSAT32 version. J Clin Nurs. 2010;19(13–14):1784–1794.
18.		Arraras JI, Vera R, Martínez M, et al. The EORTC cancer in-patient satisfaction with care questionnaire: EORTC IN-PATSAT32 Validation study for Spanish patients. Clin Transl Oncol. 2009;11(4):237–242.