Development of a Short Form of the Questionnaire Quality from the Patient’s Perspective for Palliative Care (QPP-PC)

Introduction

The number of persons with life-threatening illnesses who need palliative care is expected to increase because more people are living longer, often with cancer and other life-threatening illnesses, due to advances in medical treatment and technology.^1–3 Patients facing death may struggle with progressive losses, increased disability and complex symptoms.^4,5 Multidisciplinary healthcare team includes personnel from different disciplines working together regarding patient care.⁶ A team of multidisciplinary healthcare personnel is considered to be a central component of palliative care.⁷ Patients may suffer physically, psychosocially, and existentially and therefore require palliative care, including eg physicians, nurses, assistant nurses, priests, physiotherapists, occupational therapists or social workers. Palliative care aims to relieve suffering and promote well-being for persons and their families, living with life-threatening illnesses near the end of life.⁸ Patients should be confident that, when facing a life-threatening illness and in need of palliative and end-of-life care, they will receive high-quality person-centered care according to their needs and preferences.⁹ As an approach to nursing and healthcare, person-centered care may provide a more therapeutic interrelationship between healthcare personnel, patients and their families, underpinned by the value of seeing patients as equal partners in planning, developing and assessing healthcare.^10,11

Palliative care services may be described as specialized (exclusively providing palliative care) and non-specialized services (occasionally providing palliative care).⁷ In Norway, palliative care is provided by a public healthcare system in specialist- and community healthcare contexts.¹² The specialist health care serves patients in hospitals and specialist services. Community care comprises care for patients in eg nursing homes, home care and care provided by a general practitioner (GPs). Non-specialized palliative care (general palliative care) is provided as an integrated part of the services in both specialist- and community care. Specialized palliative care within the specialist healthcare is provided through palliative centres, palliative units in hospitals and palliative care teams. In the community, specialized palliative care is provided as palliative care teams, palliative units or beds in nursing homes, and cancer nurses and/or coordinators in the community.

Patients’ views on the quality of palliative care are important to improve person-centered palliative care, both globally^1,13-15 and in Norway.¹⁶ The general understanding of the quality of care is that it is multidimensional and may vary depending on the perspective.¹⁷ The effect of person-centered palliative care that is intended to improve quality of palliative care may be measured by patient-reported outcome measures (PROMs) which relate to patients’ experiences of care results (outcomes) in terms of changes in, for example, health status or health-related quality of life, due to the delivery of the healthcare.¹⁸ Patient-reported experience measures (PREMs) measure how patients experience the care received and should focus on aspects that are important to patients.^19,20 Several instruments have been developed to measure how patients experience the quality of palliative care provided to the patients, eg FAMCARE²¹ and the Canadian Health Care Evaluation Project (CANHELP).²² However, none of these instruments included was explicitly either founded on a theoretical model of care quality or the patient’s perspective of quality, or comprised questions about patients’ subjective importance.

In this study, palliative care quality is measured from the patients’ perspective with the adapted version of the Quality from the Patient’s Perspective (QPP) instrument which is based on a theoretical model of quality of care.²³ The model states that patients’ perceptions of what constitutes care quality are formed from their system of norms, expectations and experiences, and from their encounter with an existing care structure. QPP is a well-validated and frequently used instrument developed to measure the quality of care from patients’ perspectives,^23–26 and has been adapted to a variety of care contexts.^27–29 Recently, QPP instruments have been evaluated and recommended in two systematic reviews assessing instruments to measure patients’ perceptions of care quality.^30,31 The Quality from the Patient’s Perspective for Palliative Care (QPP-PC) is adapted to the palliative care context based on patients’ preferences for palliative care; both the validity and the reliability of the instrument are acceptable.³² The QPP-PC includes patients’ perspective of the care from a multidisciplinary personnel from both specialized and non-specialized palliative care services (eg inpatient hospice care and nursing home, and homecare). However, the QPP-PC questionnaire consists of 52 items. One main concern is that the questionnaire might be too long for patients in the palliative phase due to their increasingly frail health status with multiple symptoms. This could affect the response rate and the quality of the answers negatively. It is, however, of great importance that these views can also be assessed using short, yet valid and reliable instruments.

The aim of this study was therefore to develop a short form of the instrument Quality from the Patient’s Perspective for Palliative Care (QPP-PC) and to describe and compare patients’ perceptions of the subjective importance of the care aspects and their perceptions of the care received.

Methods

Settings, Participants and Procedures

A cross-sectional study was conducted in Norway between November 2013 and December 2014 in two inpatient hospices, two hospice day-care centers, two palliative units in nursing homes that specialized in palliative care and two home-care districts that provide non-specialized palliative care, in Norway. Patients admitted to these care settings had access to multidisciplinary healthcare personnel. A registered nurse (RN) in each ward was responsible for recruiting participants (RRN) according to the following inclusion criteria: adult (≥18 years), understands Norwegian, has no cognitive impairment, received care from the services for at least 3 days, and has an advanced, life-threatening illness in a late palliative phase (malignant or non-malignant). This was judged and guided by the RRN’s negative response to the question: “Would you be surprised if this patient died within the next year?”.³³ Patients included in the study should be aware of being in a palliative phase and receiving palliative care (judged by the RRN). The RRNs were encouraged to consult with patients’ physicians and the first author (TS) to discuss any uncertainties that arose about the inclusion criteria, and whether or not to include patients in the study. The RRNs asked patients to participate, and provided verbal and written information about voluntary participation, information about the study and how to fill out the questionnaire. The participants returned the questionnaire in a sealed envelope, which was stored in the RRNs’ offices until collection by the researcher. Help with filling out the questionnaire was offered as an interview with one of the researchers (TS). Of the 128 participating patients, 34 (27%) were interviewed. The interviews were conducted either in a private room in the ward or in the patients’ homes and were conducted such that each question in the questionnaire was read aloud to the respondent. The researcher then wrote the responses in the questionnaire after each question. The settings, participations and procedures have been described more comprehensively in previous publications.³⁴

Measures

A QPP instrument specific to palliative care (QPP-PC) had previously been developed, validated and used to measure the quality of care from the perspectives of patients with different life-threatening illnesses in diverse palliative care settings.^32,34,35 The QPP-PC includes questions related to a multidisciplinary staff that is often involved in the palliative care (physicians, nurses and other personnel, which refers to assistant nurses, priests, physiotherapists, occupational therapists or social workers).

Modifications of the QPP items and development of new items were mainly based on a review of the literature⁹ and of symptoms presented in the revised version of the Edmonton Symptom Assessment System (ESAS-r),^36,37 in addition to the research team’s expert knowledge in the field, based on experience working with patients in the palliative phase and by conducting research in the context of palliative care. The QPP-PC consists of 12 factors, which are made up of 49 items and 3 single items. The QPP-PC is based on the theoretical foundation (conceptual model) of the validated general instrument QPP,^23,24 which comprises four dimensions representing the quality of care: the medical–technical competence of the caregiver (MT), the physical–technical conditions of the care organization (PT), the identity-oriented approach of the caregivers (ID) and the sociocultural atmosphere of the care organization (SC). From this, care quality can be seen as patients’ perceptions of the actual care received (the perceived reality PR) and perceptions of how important the various care aspects are to them (the subjective importance of the care aspects SI).²⁴ Therefore, patients answered each item in two ways. First, patients scored their opinions of the quality of actual care received (PR) related to the sentence “This is what I experience … ” (eg nurses are respectful to me). Then, patients scored the subjective importance of care aspects (SI) related to the sentence “This is how important this is to me … ” (eg nurses are respectful to me). A 4-point Likert-type scale, ranging from 1 (do not agree at all) to 4 (fully agree), was used for PR, and for SI from 1 (of little or no importance) to 4 (of the very highest importance). A non-applicable alternative was available for both responses. In addition, the questionnaire consisted of 10 background questions.

Development of the short form of the QPP-PC was inspired by the following steps described by Goetz and colleagues:³⁸ documenting the validity of the original scale and the objective of its shortening, take the conceptual model into account, preserve content validity, preserve psychometric properties, document justification for the selection of each item and validate the short form in an independent sample.

Statistical Analysis

Data were analyzed using IBM SPSS Statistics version 24, AMOS Graphics. Confirmatory factor analysis (CFA) was used to measure the construct validity/psychometric properties of the QPP-PC short version.³⁹ Of the patients, 67% had fewer than three items of not-applicable answers or missed responses. These were included in the analysis. The maximum likelihood estimation was used to impute the remaining not-applicable answers/missed responses. The CFA analysis included 20 items. Primarily root-mean-square error of approximation (RMSEA) was used as a model fit index, although different fit indices are also reported. CFA was conducted and is presented for the SI scale because these scores were considered to reflect general values, compared with PR scores, which reflect specific conditions of the settings.²⁴ CFA was also conducted for the PR scale to check if it shows the same pattern.

The internal consistency was assessed using Cronbach’s α. Cronbach’s α analyses were carried out on dimensions of both subscales (PR and SI).

Descriptive statistics were used to examine patient characteristics of the patients included in the analysis. Paired sample t-tests were used to investigate patients’ perceptions of the care in terms of differences in SI and PR.⁴⁰

Results

Patient Characteristics

Of the 262 patients asked to participate, 191 patients returned the questionnaire (response rate = 73%) and 128 were included in the CFA (67%). The patients included in this present study were characterized by a mean age of 66 years (standard deviation SD = 11.68, range 41–92 years), 50% were male, most had cancer (77%), a medium-to-high level of education (78%) and 48% lived alone. Of the patients, 85% were admitted to services that specialized in palliative care (hospice inpatient care, hospice day care and palliative care units in nursing homes) (Table 1). For the whole sample (n= 191), there was no significant difference with regard to age (P = 0.569) and gender (P = 0.117) between the respondents and the patients who declined to participate in the study.³⁴

Table 1 Patient Characteristics (n = 128)

Development of the Short Form of the QPP-PC

Development of the short form of QPP-PC is presented according to the order in which they were performed in the development process.

Selecting Items for the Short Form of the QPP-PC

The items selected from the original QPP-PC for the short form and the modification of items are shown in Appendix 1. The 24 items for the short form were selected based on the following to preserve the content validity of the theoretical model underlying the QPP instrument: items in each factor that were perceived by the authors to represent the content of the factor; items that patients had scored to be of high SI (mean > 3.0) combined with items that received a low number of not applicable or missed scores (<10%).

Seven items were modified by merging items about physicians, nurses and other healthcare personnel; to one item incorporated the whole team. For example, the three items about how doctors, nurses and other healthcare personnel “understand how I experience my situation” were modified to “The personnel understand how I experience my situation.”

For three items the response alternatives were changed from a 4-point Likert-type scale, to yes/no, to be in line with changes previously made in the original QPP instrument (see Appendix 1). These three items were not included in the calculation of dimension scores, the CFA analyses and the computation of Cronbach alpha coefficients.

Preserve Psychometric Properties

CFA was used to evaluate the psychometric properties of the short form of QPP-PC for 20 of the 24 items from the SI scale (Table 2) that were perceived to best preserve the theoretical model underlying the QPP instrument. The remaining four items were retained as single items. The CFA of the QPP-PC short version showed an RMSEA value of 0.109.

Table 2 Results from the Confirmatory Factor Analysis (CFA) for the SIa Scale

Internal consistency was tested by Cronbach’s α. The Cronbach’s α values ranged between 0.64 and 0.85 for the dimensions on the SI scale (see Appendix 1).

The results from the CFA for the PR scale showed similar patterns to those shown by the SI scale. The RMSEA value was 0.112, and Cronbach’s α values ranged between 0.42 and 0.86.

Face and Content Validity of the Short Form of QPP-PC – Pilot Evaluation

The face and content validity were preserved by a pilot evaluation of the short form in a new sample, consisting of two representatives of patient organizations relevant to palliative care, and six nurses who frequently worked with patients in the palliative phases and were now attending for a master’s degree in advanced clinical nursing. In addition, the short form was discussed by a group of researchers with long experience of palliative care research who had worked with patients in palliative care. The results of the pilot evaluation led to minor changes only, related to layout and linguistics in five items. For example; the item about help for pain was added (and/or discomfort), the item about food and drink that I like was changed to “ … food and drink that I want”, the item about whether relatives receive the best possible help, support and care was added the word “information”.

Patients’ Perceptions of Quality of Palliative Care

Patients’ perceptions of quality of palliative care are presented in Table 3. The results show mean scores of SI that range from 3.37 to 3.50 at the dimension level and from 2.96 to 3.78 at the single item and item level. PR scores ranged between 3.30 and 3.51 at the dimension level and between 2.96 and 3.91 at the single item and item level.

Table 3 Comparison of patients’ perceptions of subjective importance and care received, by dimensions, items and single items

When comparing patients’ scores for the SI and PR scales, SI scales were statistically significantly higher for the identity-oriented dimension and, within this dimension, the items about the information on illness and symptoms, on what to expect in the near future (development of the illness and symptoms, health and function) and self-care. Statistically, significantly higher SI than PR scores were also present for the single item about medical care.

For the item about access to the necessary equipment in the physical–technical dimension and the single item about the atmosphere in the ward, the scores on the PR scale were significantly higher than those on the SI scale (Table 3).

Discussion

Methodological Considerations

The short version of the QPP-PC has been developed, based on the theoretical foundation of the validated general QPP instrument, and it comprises all four dimensions of the QPP. The development of the short form of the QPP-PC has been inspired by the steps taken by Goetz et al³⁸ to ensure a throughout process that preserves the validity of the instrument. In line with the steps presented by Goetz and colleagues, the validity of the original QPP instrument and QPP-PC has been previously evaluated and the instruments are built on a conceptual model, as described in the methods section. This present study has presented results regarding the content validity, the justification for the selection of items and the psychometric evaluation of the short form of QPP-PC. With regard to validating the short form in an independent sample, a pilot evaluation was performed. However, it is important to test the QPP-PC short form further in a larger independent sample of patients.

Face and Content Validity

According to Goetz et al,³⁸ the preservation of content validity in selecting items for the short form should be guided by what is considered important to participants answering the items in the instrument. Therefore, items that previously received high scores for importance to patients (SI scores), and did not have a high number of “not applicable” or missed scores, were selected for the short form, which is in line with previously stated selection criteria.⁴¹ The item about spiritual and existential care received SI score (mean= 2.96) slightly below the selection criteria and the amount of “not applicable” or missed scores was above the selection criteria (23.4%). This may be because the wording of the question has been too abstract. As this aspect of care is considered to be important based on previous research of patients’ preferences in palliative care,⁹ it was retained in the short version, but has been concretized by adding the words “Life questions”. A higher amount of “not applicable” or missed scores than stated in the inclusion criteria was present for the single items about help with personal hygiene (18.8%) and atmosphere on the ward (28.1%). This can be related to the context of care in which these patients were recruited from. The single item about the atmosphere in the ward was not applicable to those receiving homecare (Table 3). Likewise, help with personal hygiene was not applicable for most patients recruited from Hospice day care. However, these aspects of care are perceived important for patients receiving inpatient care, and therefore these items were kept in the instrument as single items.

Of the 24 items, 7 were modified by merging items about physicians, nurses and other healthcare personnel: so one item per care aspect incorporates the whole team. For example, the three items about how doctors, nurses and other healthcare personnel “understand how I experience my situation” were modified to “The personnel understand how I experience my situation.” A footnote is added to the instrument to give examples of personnel. Additionally, the item “I receive the best possible help for pain (and/or discomfort)” was intended to represent physical, psychosocial and existential pain and discomfort, in line with the principals of holistic palliative care.⁸ However, all of these aspects might not be clear for patients using the instrument. For future use of the short version it is therefore recommended to include these aspects in the wording of the item, eg: I receive the best possible help for pain (physical, psychosocial and existential pain and/or discomfort).

The pilot evaluation showed good face and content validity for the short form of the QPP-PC in that the instrument appears to measure the intended construct and that the instrument contains items that are adequate for measuring this construct.⁴² The pilot evaluation led to minor changes of layout and wordings of five items. Further, it will be important to test these items in a larger independent sample of patients.

Construct Validity

Based on the theoretical model underlying the QPP instrument developed from patients’ perceptions of quality of care, a CFA was conducted for the SI and PR scales. Ratings from the SI scale were presented because these scores were considered to reflect more general values compared with PR scores.²⁴ However, the results for the PR scale were similar to the results for the SI scale. In line with the original QPP-PC, three of the items are single items, and were not therefore included in the CFA analysis. In addition, the item about spiritual and existential care was retained as a single item because of patients’ previous scores as described above. The relationship between items, factors and dimension of QPP-PC has previously been described.³² In this present study, we have kept all selected items in the dimensions and factors, which they belong, according to the validation of QPP-PC long version.

The advantages of an instrument developed for patients with different illnesses who receive help from different services are many. However, the disadvantage is naturally higher proportions of the response alternative “not applicable” (coded as missed response), because not all aspects are relevant to all patients. CFA analysis cannot be performed when missed responses occur. The method used for dealing with not-applicable/missing in this present study was as follows: (1) only patients with fewer than three items of not-applicable answers or missed responses (67%) were included in the analysis (128 patients), (2) selected items had low not-applicable/missed response (and high subjective importance to the patients) and (3). In order to be able to perform a CFA, values were imputed for the remaining not applicable/missed response answers. Imputing not-applicable/missed response answers may lead to more favorable results. However, we consider this risk of bias to be minor, due to the reduction of imputation needed after excluding patients with more than two not-applicable or missed responses, with the inclusion of the remaining 128 patients.

Limitations may relate to the RMSEA value being slightly above the recommended value, indicating that the model fit was not optimal.³⁹ Acceptability limits appear to vary. Cut-off points ranging from 0.05 to 0.10 has previously been described and tested.⁴³ RMSEA values depend on several factors, such as sample size, and criticism has been raised for using universal cut-off values.⁴³ In the present study, the same sample was used for the development of the short version of the QPP-PC and the longer version of QPP-PC, to reduce the burden for a vulnerable patient population. Furthermore, the short version of the QPP-PC requires validation in a new and larger sample of patients.³⁸

Internal Consistency

The internal consistency was assessed using Cronbach’s α. Cronbach’s α analyses were carried out on dimensions of both subscales (SI and PR) and values >0.7 were regarded as desirable.⁴⁰ Two quality dimensions show acceptable reliability (Identity-oriented approach and Socio-cultural atmosphere), where the Cronbach’s α values ranged between 0.59 and 0.86 for the dimensions on the PR and SI scales. Both these dimensions consisted of a higher number of items. The weaker scores are present for the physical–technical dimension which contains three items only. A lower Cronbach’s α value for the physical–technical dimension is in line with previous studies using the QPP in different care contexts.^26,44 This might be explained by the following; First, these two dimensions consist of few items. The Cronbach’s α values are sensitive to the number of items, and low numbers may lead to low Cronbach’s α values. Second, these dimensions cover ratings of a more factual aspect where it can be quite logical that the internal scaled consistency is low. One may, for instance, rate the availability of necessary equipment favorably and the quality of food and drink as low. We believe these arguments combined could explain the low Cronbach alpha coefficients within these two dimensions. Even if some of the Cronbach’s α values were below the recommended level, the items were scored as being of high importance to patients. It is further recommended to interpret results at the item level for dimensions that have shown Cronbach’s α values below 0.7.

As a consequence of the somewhat high RMSEA value, and the lower Cronbach alpha coefficients on the dimensions of Medical-technical and Physical-technical conditions, we recommend the use of single-item scores on these two quality dimensions.

Discussion of the Results with Regard to Patients’ Perceptions of Care Quality

Most dimensions and single items on the SI scale were scored as of high or highest importance. High scores were also obtained for most dimensions and single items of the actual care received (PR scale). Significantly higher scores for SI than for PR scores were present for the ID dimension, and within this dimension were the items about information. Based on these results, the care area about information may be interpreted as an area for improvement.³⁴ Previous studies confirm that information given to patients in the late palliative phase needs to be improved.^45,46

The SI scores of the single item about medical care were significantly higher than the PR scores. However, the PR score was 3.63, which is considered to be high. So, even if the patients scored medical care received as high, they wished for even better medical care. This may be interpreted as medical care being of the utmost importance for these patients, which has been supported by previous research.^45–47

The PR scores of the item about access to the necessary equipment in the physical–technical dimension, and the single item about the atmosphere in the ward, were significantly higher than the SI score. This may reflect that patients received even better care than they had wished for, and may be considered as areas of strengths. Areas with high scores on both SI and PR may also be considered as areas of strengths, which in this study was, for example, care areas about pain relief, treating patients and relatives and friends with respect and the atmosphere in the ward. Patients’ scores at the dimension level for the QPP-PC short form are in line with scores using the long version;³² this strengthens the reliability of the instrument. One exception was for the dimension medical–technical competence which scored lower for the long version on both PR and SI scales.

Person-centered care involves placing patients as the center of care and healthcare professionals (HCPs) being respectful and responsive to patients’ and families’ life situations, preferences, needs and values.^48–50 This comprises the importance of patients’ voices being heard by facilitating an evaluation of the experience of palliative care using valid instruments (PREMs) that are feasible for this vulnerable group of patients. To provide high-quality person-centered care, it is therefore important for the care to be in line with what is considered important by the patients, and to improve areas that patients identify as areas for improvement. Further improvement is therefore needed to meet patients’ preferences for information.

The results of this present study have been supported by results in previous studies. However, the results should be used with caution, since these results were derived from the original long version of QPP-PC. The results might be different when patients are being presented in the short version.

Conclusion and Clinical Implications

The short form of the QPP-PC shows promising results and may be used with caution as a person-centered PREMs indicator evaluating the multidisciplinary palliative care for patients in a late palliative phase. However, the short version of the QPP-PC needs to be further validated in new and larger samples of patients. The advantages of a short form are both practical and ethical. It is ethically desirable to able more patients, including those with a more advanced illness, to participate in evaluating their care. A shorter form is easier to use, especially for this specific patient population, and therefore an important practical contribution to the original QPP-PC and other existing scales.

Patients’ perceptions of the subjective importance of palliative care aspects and quality of the care received were mostly high. The use of the QPP-PC short form illuminated areas of strength and improvement in the palliative care – in particular, we found that information given to patients needs improvement.

Knowledge of patients’ evaluation of palliative care, and further use of the short version of the QPP-PC to measure patients’ preferences and perceptions of care quality, are important to improve the quality of palliative care and to tailor the care according to patients’ preferences, thereby enhancing person-centered care.