Patient and Staff Satisfaction and Experience While Transforming Health Center Systems

Introduction

Enhancing patients’ healthcare experience requires that health centers attend to many areas within the overall care delivery system. The Quintuple Aim goals reinforce the importance of this multifaceted approach, with a focus on improved health outcomes, improved patient experience, improved staff experience, reduced costs, and improved equity. Originally established by the Institute for Healthcare Improvement as the Triple Aim,¹ this concept was expanded to the Quadruple Aim with the addition of staff experience in 2014.² Since the start of this project in 2017, the National Association of Community Health Centers (NACHC) adopted the Value Transformation Framework’s expanded focus on the Quintuple Aim with the addition of “equity” as a core goal and measure.

NACHC³ is the national membership organization for health centers that meet the Health Resources and Services Administration’s (HRSA) Health Center Program Requirements. NACHC works closely with health centers and their state and regional partners – primary care associations (PCAs) and health center controlled networks (HCCNs) – to deliver high-quality primary care services to communities in need. PCAs are state or regional membership organizations that offer training and technical assistance to safety-net providers. HCCNs are groups of health centers working together to support and enhance the use of health information technology to improve access and quality and lower costs.^4–7

The Value Transformation Framework (VTF) was developed in response to an articulated need from health centers to gather, synthesize, and translate the expansive body of evidence around systems change into actionable steps health centers could readily implement.⁸ This need is even more critical as health centers transition from a volume-based to value-based model of care.

Health centers provide care to nearly 30 million patients at almost 14,000 delivery sites across nearly 1400 organizations. The majority of health center patients have incomes below the Federal Poverty Level and face social and environmental risk factors that affect their health.⁹ Despite the complexity of serving high-risk patients with disproportionate burdens of disease, health centers consistently provide quality care that meets or exceeds the performance of private practice primary care offices,¹⁰ at lower cost.¹¹ It is estimated that health centers save the healthcare system $24 billion annually.¹¹

Yet, health centers and their respective clinicians and staff—burdened with competing initiatives, rapidly changing technologies, and the demand for practice transformation—are experiencing increased burnout and decreased satisfaction.^12–14 They need support in finding successful strategies to deliver better care at lower costs.¹⁵

In this project, NACHC aimed to transform health center systems and improve value, through application of the VTF conceptual model for systems change. In 2017, serving as the project lead, NACHC selected participant PCAs through a competitive application process. The PCAs then selected participant health center sites (hereafter “health center”). NACHC required that the PCA also serves as an HCCN or demonstrates a commitment to partner with a HCCN. The final selected organizations included four health centers each in Georgia and Iowa (Table 1).⁵

Table 1 Averaged Uniform Data Systems 2016 Population Measures for Participating Health Centers

The Value Transformation Framework (VTF) is a conceptual model that organizes evidence-based strategies that can enhance the capabilities of a health center’s Infrastructure, Care Delivery, and People systems (known in the framework as “Domains”).⁸ The framework divides the Domains into 15 “Change Areas” (Figure 1). Change Areas focus on concrete interventions and recommended action steps, offered through step-by-step Action Guides and other tools that translate knowledge and evidence into practice. The VTF is designed to support health centers in the transition from a volume- to value-based model of care and achievement of improved health outcomes, improved patient and staff experiences, and lower costs. Equity, central to the Value Transformation Framework from its inception, was separately named as a goal beginning in 2019 – thus becoming the Quintuple Aim goals. While other improvement models exist, the VTF is unique as a clear, standardized model that federally qualified health centers can use as an actionable pathway to overall systems change that advances value.

Figure 1 Graphic of the Value Transformation Framework (VTF). The VTF helps guide systems change in health centers by translating research and promising practices into manageable steps health centers can take to improve care and outcomes. At the core of the model is the health center and the patients it serves. Health centers’ complex systems are broken down into three manageable Domains: Infrastructure, Care Delivery, and People. Each Domain is divided into more detailed Change Areas, 15 in total (represented by the icons that encircle the 3 Domains). The Infrastructure Change Areas include: improvement strategy, health information technology, policy, payment and cost. The Care Delivery Change Areas include: population health management, patient-centered medical home, evidence-based care, care coordination and care management and social determinants of health. The People Change Areas include: patients, care teams, leadership, workforce and partnerships. The outer circle focuses on the goal of the VTF: to support health centers in making system changes by directing action in the 15 change areas and advancing toward value, defined as the Quintuple AIM (represented on the outer circle by: improved health outcomes, improved patient experience, improved staff experience, reduced cost and equity). The VTF was developed by the author, Modica.8

Through the lens of the VTF, this project focused on improving the Quintuple Aim goals, with a particular focus on colorectal cancer screening. It was hypothesized that by taking a systems approach to change through application of the VTF, health centers could not only increase the percentage of patients screened for colorectal cancer under the recommended guidelines but also improve performance in other clinical conditions and areas of systems change. Throughout this project, the participating health centers were guided through the areas of change with a focus on 10 evidence-based interventions.

NACHC combined the Value Transformation Framework, evidence-based colorectal cancer screening (CRCS) interventions, and a learning community model to drive health center system improvement. The Learning Community consisted of in-person training, networking opportunities, and regular calls by NACHC staff to provide ongoing project coaching, technical assistance, and support. A structured, yet flexible approach was necessary to provide actionable steps with the ability to perform them within the context of the center’s priorities.¹⁶

Members of this investigative team published the overall results of this project in a May 2019 report.¹⁶ The raw percentage of eligible patients screened for colorectal cancer increased from 33.2% in January 2017 to 46.5% in December 2017. This translates into an average increase of 3.3 (95% CI: 1.7, 5.0) eligible patients screened per month per health center over the year or 317 additional patients.

Methods

To determine if implementing changes associated with this systems transformation approach affects the experience of health center patients and staff, surveys were administered before, during, and after application of the VTF systems approach.

Investigators used two surveys to collect quantitative data for the evaluation of satisfaction and experience in relation to the implementation of the VTF’s systems approach to change. Drawing from approaches of other published healthcare satisfaction and experience surveys but recognizing the desire to measure elements specific to this project, the investigators developed tools to measure elements unique to the project. The surveys were reviewed by content experts and tested for face-validity. Consensus was reached by group discussion, and the final surveys were approved by the full project team.

Three data collection periods were defined for each survey, the first in March 2017 (baseline measure), the second in December 2017 (end of year one), and the last in December 2018 (end of year two). Each collection period lasted about two months.

Our analyses aimed to answer the following questions:

• Was implementation of the VTF’s systems approach to change associated with any changes in patient satisfaction with the health center or provider?
• Was implementation of the VTF’s systems approach to change associated with perceived changes to the patient visit experience or utilization of health center services and tools?

• Was implementation of the VTF’s systems approach to change associated with any changes in care team satisfaction or experience? Was implementation of the VTF’s systems approach to change associated with provider perceptions of changes in administrative performance?

This quality improvement project was submitted to the A.T. Still University (ATSU, Arizona) Institutional Review Board and deemed to not fall under the jurisdiction of the Board. All surveys were anonymous and did not collect any personally identifying information. Completion of the surveys was voluntary, and there were no consequences for those who chose not to complete the surveys.

Study Sample

Patients and staff of the participating health centers in Georgia and Iowa were the intended recipients of the experience surveys. Eight health centers participated in the first year of the project, and two survey collection periods (baseline and close of year one). Six of the centers continued in year two and participated in the third survey period (end of year two). For this paper, results will be presented reflecting the six health centers that participated in both years one and two of the project and all three survey periods.

Health center population size was determined using the publicly available health center data accessed via the Uniform Data System (UDS).⁵ Select data on patient demographics provided from each health center’s organizational profile can be found in (Table 2).

Table 2 Total Patients, Including the Percentage of Adults (18–64) and Older Adults (65 and Older), by Health Center

Patient Survey

The patient survey included a total of 11 items designed to measure patient experience within three domains: satisfaction with health center, staff and visit experience, and utilization of health center services and tools. Patients were asked to base their responses on their experience at the health center over the prior 12-month period. A 5-point Likert scale was used, and responses were coded as Always (5), Often (4), Sometimes (3), Rarely (2), Never (1).

The patient survey was paper-based and available in both English and Spanish. Each participating health center was tasked with distributing the survey to adult patients visiting the health center during the three collection periods.

Staff Survey

The staff survey comprised 25 items developed to assess the following domains: individual level satisfaction and experience, organizational level satisfaction and experience, employment history, and demographics including age, gender, race, and ethnicity. A 5-point Likert scale included the responses: Strongly agree (5), Somewhat agree (4), Neither agree nor disagree (3), Somewhat disagree (2), Strongly disagree (1).

The survey was designed to be completed by health center staff online via the Qualtrics Research Platform. A unique electronic link was provided by investigators to project leaders at each health center and subsequently emailed to staff.

To preserve confidentiality, staff were provided with instructions for creating a personal identification number (PIN) to be entered at the start of each survey. This PIN was only used to link pre- and post-surveys and could not be used to identify any employee.

Analysis

Descriptive statistics (counts and percentages) were calculated to summarize survey responses for each item at each administration. Data were substantially skewed, so means (standard deviations) did not accurately convey the central tendency of the distribution or item variability. Medians (interquartile ranges) did not well convey the magnitude of respondent agreement with survey items or changes in scores across administrations. We chose to provide the percentage of respondents who “agreed” with each statement (endorsed “Always” or “Often” for patients or “Strongly agree” or “Somewhat agree” for staff). Kruskal–Wallis tests were used to evaluate changes in responses over time, based on the ordinal (Likert-type) response scale. Data were analyzed using the Statistical Package for Social Sciences (SPSS 27; IBM Corp., Armonk, NY). A p-value <0.05 (two-tailed) was considered statistically significant.

Results

The total number of patient and staff surveys collected by each health center is presented in (Table 3).

Table 3 Response Counts by Health Center for Patient and Staff Experience Surveys During Each Collection Period

Patient Survey Results

During the three collection periods, a total of 721, 632, and 389 patient surveys were collected, for a project total of 1742. Patient participants were not asked if they had previously completed the survey and no demographic information was collected.

The questions assessing patients’ satisfaction and their experience with the health center were rated highly and did not change significantly over time. For the survey item, I am satisfied with my experience at the health center, 91% of respondents endorsed “Always” or “Often” at each administration.

Patient agreement about whether staff had asked about cancer screening increased significantly during the three survey periods from 30.5% during survey period 1 to 33.3% at survey period 2 and to 48.7% at period survey 3 (Jonckheere–Terpstra test of trend, p < 0.001).

An additional category of the patient survey inquired if patients used or would use the optional health center services of an online patient portal and receipt of text message reminders. The percentage of patients using the health center’s online patient portal decreased from 21.9% at baseline to 15.1% at the third administration (p < 0.001). The percentage of patients who reported that they would like to receive text message reminders regarding health screenings remained stable at approximately 57%.

The full patient survey and the percentage agreement for each item are provided in Appendix A.

Staff Survey Results

A total of 374 staff surveys were collected with 128, 139, and 107 surveys received during the collection periods. Individual staff member responses could not be linked over time, due to inconsistencies in the use of the self-created PIN. However, staff members were asked to select the category, which best represented their job title from a provided list.

Clinical Providers and Clinical Staff consistently accounted for the greatest number of responses. Staff members in the Health Information Technology category accounted for the fewest number of responses. The job category proportions for each collection period are outlined in Table 4.

Table 4 Percentage of Staff Survey Responses by Job Title per Survey Collection Period

Health center staff rated items associated with individual level satisfaction and experience highly. None of the item scores changed significantly throughout the project. The highest scoring item was My work adds value to my health center’s ability to provide care with most staff indicating they “strongly agree”. The lowest scoring item was There are opportunities for career advancement within my health center, although more than half reported agreement.

The questions included in organizational level satisfaction and experience also scored highly, although measuring slightly lower than the individual level items. Most staff reported agreement with the following items: If I have ideas about how to improve systems or processes, I feel others will listen to me; I have been provided sufficient training to help me fulfill my specific role(s) as a member of my larger health center team; and At my health center, we all work together and support one another in getting work done.

For the items Overall, I am satisfied with my job and, I look forward to coming to work each day, staff remained in agreement over all measurement periods. The full staff survey and the percentage agreement score for each item are provided in Appendix B.

Discussion

Despite the addition of processes and changes in health center operations, both patient and staff satisfaction and experience remained highly rated throughout the project.

Significantly, these changes were also associated with improvements in performance of colorectal cancer screening.¹⁶

These findings support various recommendations for the use of multicomponent interventions in the pursuit of improved clinical care and patient outcomes.^17–20 Satisfaction can remain high within the health center despite potential disruptions brought about through transformation efforts.

Patient Survey Discussion

Overall, patient satisfaction and experience remained highly rated at each health center throughout the project period. Together, the survey results indicate patient satisfaction and a positive patient experience and suggest that the additions to changes in organizational systems did not disrupt patient satisfaction or experience.

Responses to the survey items regarding patient portal use and text messaging preferences indicate use of these technologies is low. Reasons for not using the patient portal and opting out of text message reminders were not explored and could be incorporated into future surveys. Elucidating these patient preferences may identify better ways to keep patients up to date with recommended screenings and to connect patients with recommended services.

The inclusion criterion was an age over 18 years. A more targeted sample of established patients who meet the criteria for cancer screening may have produced different results. Additionally, as we could not control the local survey distribution process, the methods used for patient recruitment may have differed between health centers and could have impacted the number of surveys received.

The requirements a health center must meet to be certified as an FQHC²¹ and the demographics of patients typically served by health centers suggest that there could be a need for assistance with social factors that can affect health.^22,23 Future patient surveys can assess the levels of need and perceived interest in receiving help in these areas, along with evaluations of the level of help provided.

Staff Survey Discussion

Throughout this project, health center staff also rated their job satisfaction and experience highly. Responses grouped by job category indicated improvements in ratings in all groups, except for those who work in Health Information Technology. This may be due to the low number of responses in this category but should be further explored.

Of note, the results showed upward trends for the item At my health center, screening and management of chronic disease (depression, obesity, high blood pressure, diabetes) are the responsibility of the entire care team and not just the medical provider. The improvements observed for this item can be used to demonstrate the success of the VTF in guiding successful systems change.

As shown in other published literature,^24–26 results from patient surveys can also be used as an indicator of a positive staff experience and positive staff performance. The patient survey results from this project indicate that staff are performing well on the measured areas, including recommending cancer screenings and making the patient feel included when making care decisions. These patient survey results can be shared with staff to demonstrate high levels of patient satisfaction and experience even while managing a system transformation.

The limitations of this survey’s administration prevented the tracking of staff participants and the linking of their responses over time. Future projects should explore new ways to engage staff in survey participation and improved methods to ensure accurate participant tracking. Additionally, although staff satisfaction surveys are routinely administered by health centers and tracked over time, psychometric validation of these surveys has not been completed and thus there may be limitations in the survey sensitivity to change in attitudes and beliefs, particularly to subtle changes.

Conclusion

Changes in protocol and increases in job responsibilities that may take place as a part of transformation efforts are often associated with a decrease in job satisfaction, particularly during the period of change.²⁷ The results of these surveys indicate that experience and satisfaction scores remained high for both patients and staff throughout the implementation of systems change through application of the Value Transformation Framework. Along with the improvement observed in the associated clinical measures, these survey results serve as further evidence that this framework may be effective in the pursuit of improving the elements of healthcare service associated with the Quintuple Aim.

A potential limitation is that both surveys were subject to the “ceiling effect”, allowing little room for improvement. Mean scores were high (more positive) from the start and therefore most increases in score were statistically insignificant. However, we view the overall results positively; as an indication that patients and staff highly rated their satisfaction and experience, despite system changes and the addition of processes and procedures.

Limitations in survey design prevent a full review of the association between application of the Value Transformation Framework and specific areas of patient and staff satisfaction and experience. Investigators must weigh the advantages and disadvantages of survey use while keeping in mind the intended outcome. Further, the surveys did not collect demographic information or information about specific social determinants of health. Future work should include more details in these areas to inform specific health center responses. Despite the production of imperfect data, the survey results of this project are still usable by the health center participants and others interested in pursuing application of this framework. More thorough evaluations might best be achieved through the regular distribution of verified experience surveys^28–30 in addition to continual reporting of performance in clinical measures within recommended guidelines.

This project aimed to compile a holistic view of the potential effects of the interventions implemented on the satisfaction and experience of health center patients and staff. As one of the authors is designer and original author of the Value Transformation Framework,⁸ the importance of impartiality in viewing results is recognized and the need for continued study of patient and staff experience with implementation of the VTF. The survey results reported in this paper serve as snapshots in time that are encouraging, with no unintended consequences related to satisfaction or experience identified when a systems approach to change was undertaken through application of the Value Transformation Framework.

These results can be used to show that the implementation of the Value Transformation Framework in this project improved clinical outcomes without negatively impacting patient and staff satisfaction and experience. Therefore, the Value Transformation Framework may be a useful tool for other healthcare organizations initiating systems change and may be adaptable to a wide variety of areas for healthcare improvement.