Patient experiences: a systematic review of quality improvement interventions in a hospital setting

Introduction

In the era of value-based healthcare we strive for the most optimal outcomes and experiences from the perspective of the patient. Therefore, patient experience has become a key quality indicator for healthcare and is positively associated with patient safety and clinical effectiveness.¹ Measuring and analyzing experiences is seen to support improvement in healthcare quality governance, public accountability, and patient choice.^2–5 Through the years, a variety of patient experience measures have been developed and used in healthcare, among which are questionnaires, focus groups, and interviews. While such tools are supposed to drive quality improvement (QI), their use and effectiveness for this purpose has been questioned.^6,7 The lack of QI may be linked to methodological barriers (eg, using a survey with poor psychometric properties, infrequent data-collection, ineffective monitoring), hampering the assessment of effectiveness. Also the lack of local ownership for QI, limited training and education of staff for QI, as well as the absence of an organizational culture for change has a negative effect on the improvement of patient experiences.⁸,⁹ Moreover, patient experiences cover diverse domains, which all require appropriate measurement and different quality improvement initiatives.¹⁰

Previous systematic reviews examining one or more aspect of QI initiatives confirms the aforementioned barriers, and all conclude that the optimal approach for using experience data effectively is lacking.^11–13 The aim of this systematic review, compared to other reviews, was to broaden our scope to national as well as local patient experience measures in a hospital setting and gain more insight into the effectiveness of diverse QI initiatives and their influencing factors. The following research questions were addressed:

1. Which QI strategies are being used to improve patient experiences?
2. What is the effectiveness of QI interventions to improve patient experiences?
3. What are the barriers and promoters of QI interventions aimed at improving patient experiences?

Methods

The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines were used to design this review.¹⁴

Scope of the review

Patient experiences were defined as; “the sum of all interactions, shaped by the organization’s culture, that influence patient perceptions, across the continuum of care“.¹⁵ We limited our scope to patient experiences related to Picker’s eight domains of Person Centered Care; 1) Accessibility, 2) Effective treatment and trusted professionals, 3) Continuity of care and transitions, 4) Involvement in decisions and respect for preferences, needs, and values 5) Comprehensible information and support for self-care, 6) Involvement of and support for family and friends, 7) Emotional support, empathy, and respect, and 8) Attention for physical and environmental needs.¹⁶ Studies that were limited to evaluating patient satisfaction, rather than patient experience, were beyond the scope of this review. Patients generally tend to overrate their satisfaction, for example due to gratitude bias.¹⁷ Therefore, the validity and usefulness of satisfaction data is questionable.¹⁸

Information sources and search parameters

The following databases were searched on September 29, 2017: Embase, Medline OvidSP, Web of Science, Cochrane Central, PubMed Publisher, Scopus, PsycInfo, and Google Scholar.

Search terms were derived from previous studies^11,19 and our research questions. The thesaurus in Embase which formed the basis for the search strategies for the other electronic databases is shown in Figure 1.

Figure 1 Flowchart literature search.

Eligibility criteria

Included studies met the following criteria: 1) QI interventions that targets patient experiences; 2) patients’ experiences are examined pre- and post-intervention; 3) hospital setting; 4) written in English; and 5) published after 2006. Non-intervention studies and editorials, conference papers, reviews, books, interviews, or columns were excluded, as well as studies that could not be retrieved in full-text.

Data extraction

Two authors (CB and HB) independently screened titles and abstracts for inclusion. Eligible studies were evaluated in full-text by both authors. A third author (LdJV) was consulted when agreement was not reached. For all eligible studies, details about study design, patient experience, topic, measurements, sample size, interventions, and outcomes were extracted.

Data synthesis and analysis

Due to the variation of the used methodology, interventions, topics, heterogeneity of data, and method of reporting outcomes, we performed a narrative synthesis of all relevant themes within and across the studies.

Risk of bias

The methodological quality of the included studies was assessed independently by the same researchers using the Critical Appraisal Skills Program (CASP) Qualitative Checklist.²⁰ The checklist was adapted using two questions in order to assess and compare all eligible studies with diverse methodology. The question “Is a qualitative methodology appropriate?” was adapted into “Is a qualitative/quantitative methodology appropriate?” For quantitative studies, the question “Was the data analysis sufficiently rigorous?” was judged by considering size of the confidence intervals and by examining whether the following variables were considered: confounding factors, blinding of providers, and response rate. Studies that obtained negative ratings for at least five out of ten items (ie, “no”, “can’t tell”, or “unclear”) were excluded from this review.

Results

In Figure 1, a flow diagram of the search process is presented. After removal of duplicates, a total of 3,289 records were identified. Of these, 3,139 studies were excluded based on title and abstract. Of the remaining 150 full-text articles, 21 studies were in agreement with the inclusion criteria and were included for review.

Characteristics of included studies

The search resulted in 15 pre–post intervention studies, two qualitative studies,^21,22 three RCT’s,^23–25 and a longitudinal study.²⁶ One study was performed in Tanzania,²⁴ and the other studies in either Europe, the US, or Canada. The majority of studies (n=15) included patients from a specific department (eg, neurosurgery). One study focused on the transition of hospital to primary care in a radical prostatectomy pathway.²⁷ In 12 studies, patient experiences were assessed using an existing survey (eg, Hospital Consumer Assessment of Healthcare Providers and Systems, HCAHPS), and seven studies used a self-developed survey. The remaining two studies used informal interviews²¹ or a combination of methods.²² The study characteristics are summarized in Table 1.

Table 1 Study characteristics

Methodological quality

For all the differences of methodological design and quality, none of the 21 studies obtained more than five negative ratings, thus were all included (Table 2). All studies clearly described the aims of their research, used appropriate methodology and research design, and collected data in a way that addressed the research question. However, in six studies it could not be determined whether the recruitment strategy was appropriate to the aims of the research.^{21,22,28–31} Two qualitative studies^21,22 did not report on the relationship between researcher and participants and, for 14 out of 19 quantitative studies, patients remained anonymous during the entire study. Six studies did not report whether they had taken ethical issues into consideration. The rigor of data-analyses was rated insufficient in 14 studies mostly because they didn’t report statistical significance of pre–post changes in patient experience scores, or multiple comparisons were made without correcting for multiple testing. The latter increases the chance of false positives. Seven studies did not clearly describe their findings in relation to other studies or current practice.^{21,25,28,32–35} Lastly, three studies were rated “unclear,” because the authors did not consider the findings in relation to current practice or policy or they did not identify new areas for research.^21,32,33

Table 2 CASP quality assessment of included papers

QI interventions

Various QI strategies were applied (Table 3). These can be categorized into staff education, patient education, audit and feedback, clinician reminders, organizational change, promotion of self-management, and policy change.³⁶ The most common strategies are organizational change^{21,22,24,26–35,37,38} and staff education.^{23–25,29–32,34,37,39–42} These strategies all relate to changing ward procedures and staff behavior. Most studies applied multiple QI strategies,^{21,23–26,29–32,34,37–39,41,42} while other studies used only one of the aforementioned QI strategies.^{22,27,28,33,35,40} Eleven studies reported to use a specific change management approach or tool. These include Lean or Lean Six Sigma,^{24,29,30,32,33,38} Plan-Do-Study-Act,^22,34,35 Kotter’s Model of Change,⁴² and a 30-step-scenario.²⁷ One study used The CAHPS improvement guide.³⁷

Table 3 Interventions and results

QI outcomes

With the exception of one study,²⁷ all studies reported at least one improved patient experience score following intervention. A dichotomy can be approximately found; six studies focused on improving the interaction of staff with patients (eg, communication, compassion, respect),^{23,24,32,34,38,40} and 10 studies focused on improving processes (eg, waiting time, noise disturbance, pain management).^{21,22,27–31,35,39,42} Five studies had objectives in both areas.^{25,26,33,37,41} Fourteen studies examined whether statistically significant change had occurred following intervention. In these 14 studies, 106 pre–post comparisons were made, of which 38 pre–post improvements were labeled statistically significant by the researchers. Six of these studies were targeted on staff–patient interaction,^{23,24,32,34,38,40} and four studies on improving processes.^27,29,31,35 Within the studies focusing on improving interactions, 55% of the pre–post comparisons significantly improved, while this was 16% within studies of improving processes and 17% within studies who wanted to improve on both levels. Noteworthy is the fact that studies that in advance targeted on the improvement of one outcome measure, such as improving waiting experience,³⁵ compassionate care,³² ratings or sleep,³¹ nursing care,²⁵ or overall patient experience,²⁹ were most successful.

Barriers and promoters

Eighteen studies mentioned specific barriers for QI (Figure 2).^{22–27,31–35,37–42} These can be categorized into data-related, professional, and organizational barriers.⁸ Commonly reported data-related barriers were the risk of bias due to a small sample size^{23,32,37,38,42} or a low response rate,^25,26,40 and confounding by simultaneously applied interventions^{22,23,26,32,39,41,42} or a lack of blinded providers.^27,34,38,41 Furthermore, four studies mentioned that their QI intervention may have been too short to induce significant change.^24,26,35,37 Skepticism amongst staff about the necessity or usefulness of the proposed change was the most frequently reported professional barrier.^{25,26,33,35,37,39} Also, staff changes, especially at management level, were held responsible for not achieving objectives,^24,27,34,40 along with the lack of time required for a successful implementation.^{25,27,34,37,39,40} The organizational barriers mentioned were mostly related to a lack of engaged management for QI^24,26,27,37 or no culture of change.³³

Figure 2 QI initiative.

Fourteen studies mentioned specific promoters for QI (Figure 2).^{22–26,31,34,35,37–42} Several studies indicate that a QI intervention only succeeds if the organization supports system change and approaches this through engaged leadership.^{22,25,30,37–41} Staff must be involved in data collection and be given help and insight into the interpretation of departmental patient experience scores.^25,34,37 It is important to support staff by means of coaching, provision of information, education, and multi-disciplinary collaboration.^{23,25,26,34,35,37,39} Another way that may facilitate QI is to involve patients in designing QI interventions.^23,25,35,37 Finally, frequent or continuous assessment of patient experiences has been mentioned as an important element to maintain a culture of change in healthcare.^{31,34,37,38,42}

Discussion

The aim of this systematic review was to broaden our scope to national as well as local patient experience measures and gain more insight into the effectiveness of diverse QI initiatives and their influencing factors in a hospital setting. Although all studies reported positive results, they showed large variability in their methodology of QI initiatives which hamper the comparison of results. However, similarities were found in experienced barriers and the proposed promoters for QI.

QI strategies used to improve patient experiences

Most studies applied a combination of QI strategies. Organization change was one of the most frequently used QI strategies, probably because it encompasses a wide range of topics; from physical changes to the hospital surrounding, to changes in staff. Another frequently used QI strategy is staff education. About half of the included studies educated staff as part of their QI intervention. The other half reported resistance among staff,^25,26,35 discussed staff changes as a barrier for QI success,^27,32,38 or mentioned not having a culture that supports QI.³³ Besides involving staff, it may also be valuable to involve patients in QI efforts. Five studies involved patients in designing QI interventions by patient focus groups or participation in a patient and facility advisory council, and may well offer an additional strategy for QI.^{21,28,31,32,37} To reach its full potential, it is, however, important that staff members recognize and value patient involvement.^{23,25,35,37,43,44}

Effectiveness of QI interventions to improve patient experiences

It is noteworthy that studies which targeted improving interactions of staff with the patient seem more successful than studies which targeted improving processes. Furthermore, studies which targeted the improvement of one outcome measure in advance were all successful.^{29,31,32,35,45} Within the studies with multiple outcome measures,^{23,24,26,27,34,37,38,40,41} it often remained unclear whether they actually intended to improve all outcomes, this could be an explanation for the lack of significant change. Other explanations can be found in the mentioned data-related, professional, and organizational barriers (Figure 2). Obviously, the type of study design is also an important determinant of the results and their interpretation. Three of the studies were Randomized Controlled Trials (RCTs).^23–25 These studies were successful in improving patient–provider communication. An obvious advantage of an RCT is the possibility to assign differences in pre–post scores to the effects of the QI intervention. However, in clinical practice an RCT is not always feasible for practical and methodological reasons (eg, ethical issues and costs). The 11 studies reporting the use of a specific change management approach or tool (eg, Lean or Lean Six Sigma, Plan-Do-Study-Act) had no better results in terms of methodology or significance.

Seven studies reported improved patient experiences but did not examine whether this improvement was statistically significant,^{21,22,28,30,33,39,42} for example because this was beyond the scope of their research question. Data had served as a communication tool to establish the need for change³³ or to provide insight into the development or operation of a QI strategy.²²

Barriers and promoters for QI

Almost all studies reported on specific barriers or promoters for QI, and a relationship is assumed with (a lack of) significant results. For instance, four of the studies did not adequately report on the number of patients included, or included a small sample size.^28,30,39,42 The risk of a small sample size is that changes in score results reflect random fluctuations rather than actual improvement. Regarding professional and organizational barriers, the findings are in line with previous studies among healthcare professionals and managers^8,9,46 and frequently reported barriers for QI in other healthcare settings such as mental healthcare.^47,48 This highlights the importance of designing and implementing strategies to involve and educate staff.^9,12,49 Physician engagement may, for instance, be enhanced by developing clear and efficient communication channels with physicians by building trust, understanding, and identifying or developing physician leaders.⁵⁰

Promoters of QI interventions were focused around engagement of patients, staff, management, and culture. This is in line with previous systematic reviews on the use of patient experiences for QI^11,12 and qualitative studies on promoters and barriers for improving patient experiences in healthcare.^8,51 A barrier that was not identified in the current review was changing the employees’ mind-set from “provider-focused” to “patient-focused,” which is an important aspect of patient-centered care.^8,51

Strengths and limitations

A strength of this review is that outcomes, barriers, and promoters for QI were derived from the studies included as a valuable source for further QI work. Also, the findings of previous reviews^11–13 were extended by this, looking beyond national patient experience surveys and gaining insight into the effectiveness of QI. In clinical practice, it is usually the case that departments obtain national as well as local patient feedback using a variety of measures (eg, surveys, focus groups). The inclusion of a wide variety of patient experience measures can also be considered a limitation of the current review. The many differences between studies (eg, study design, type of patient experience measures) hamper the interpretation of results. The studies that did meet inclusion criteria were evaluated for their methodological quality using the CASP Qualitative Checklist. As its name already implies, this checklist was developed for qualitative studies and was, therefore, less appropriate for quantitative studies.

Implication for future policy and research

Knowledge on barriers and promoters provides a valuable source of information that can be used to guide future QI initiatives. Addressing data-related, professional, and organizational barriers may positively influence the effectiveness of QI interventions that target patient experiences. Ideally, healthcare organizations or hospital departments develop structured plans on how to use patient feedback for QI and methods to engage clinicians in this process. In current practice, such plans are often lacking.^19,52 Also, it is encouraged to include a follow-up assessment to examine changes in patient experience following QI intervention. This is important, as a change is an improvement only when the patient experiences it as such. Large-scale RCT’s are needed to determine whether improvements are actually the direct result of a QI intervention and also to compare the effectiveness of different QI strategies. Another potentially valuable direction for future research is to examine the extent to which patients could and should be involved in designing QI interventions. Just as experiences may differ between patients and staff, this could also be the case with their perceptions on future healthcare.

Conclusion

Despite the heterogeneity of methodology and methodological quality of studies reviewed, many lessons can be learned. A wide range of patient experience domains were targeted for QI, but outcome measures focused on improving communication and interaction were more successful than outcome measures focused on changing processes. Alongside this, studies with a small number of outcome measures were most effective, organizational change, and staff education were the most frequently used QI strategies in those cases. While most studies report positive outcomes, they also report on significant barriers and promoters that can influence QI work, not least a sound design of research. Furthermore, engagement of patients and all stakeholders at both departmental and management level is commonly recommended for successful QI. Future research should address barriers and promoters in order to enhance methodological quality and study outcomes.