Impact of Point of Care Quality Improvement Training and Coaching on Quality Perceptions of Health Care Workers: Implication for Quality Policy

Introduction

Infant mortality and morbidity are sensitive indicators for the success of health development. Although remain as a classic problem globally, a downward trend has been noticeable. Focused efforts to accelerate the process are now being made by improving the quality of antenatal, labor, and postnatal services.^1,2 The 2020 infant mortality rate in Indonesia was reduced by 19.55%,³ but this reduction ranked as 6th highest among ASEAN countries.⁴ The main cause of infant death was premature birth.⁵ The known risk factors for infant mortality include socio-demographics (such as low maternal education levels, maternal age that is too young or too old, living in rural areas), low economic status, smoking habits, and obesity during pregnancy.⁶ Most of the risk factors for infant mortality are structural and social capital determinants⁷ and the health system may largely be responsible,⁸ especially at the front line of the health care system.⁸ The front line (point of care) of healthcare services had important role in reducing infant mortality,⁹ where quality is a key element.^10,11 Efforts in quality services improvement will strengthen health systems, encourage the workers and improve health care performance.^12,13 Quality of care improvement yields a major problem in healthcare systems worldwide, therefore appropriate tool for measuring performance is mandatory to assess essential aspects for identification, planning and evaluating of quality improvement strategies.¹⁴

The ongoing healthcare service quality assessment policy in Indonesia is based on accreditation, regardless of being a primary or secondary level of healthcare service facility or even government or privately owned. Accreditation emphasizes that all comply with minimum standards of input, process, and output of the healthcare services. Accredited healthcare facilities should continuously maintain their accreditation status above their minimum standards to certify the business license. Experiences showed that fulfilling only the minimum standards were insufficient for improved outcomes and their impacts. The accreditation system has not led to significant changes in the effects of health care performance and outcomes.¹⁵ Therefore quality performance, supported by the quality management system of health services is an addition to quality assurance by the accreditation system. Quality Improvement (QI) of health services is still not structured, systematic and massively used with less emphasis on solving quality problems faced daily in health services. To meet the minimal standard in accreditation does not mean quality activities in institutional healthcare facilities are rooted in, or involve every health worker, which is why it should be further encouraged in the forefront service of a health facility.

Several studies in developed countries used various approaches in quality improvement projects. Such as a project to increase breast milk feeding in premature infants California, USA, with multihospital collaborative design to establish evaluation and improvement model.¹⁶ In the UK an approach has been established which is known as experience-based co-design for quality improvement in healthcare.¹⁷

WHO SEARO introduced the Point of Care Quality Improvement (POCQI) in 2017.^18,19 POCQI is one method in the health service quality management system with an emphasis on strengthening service processes that are under the control at the forefront of health services units. POCQI is a movement and practice in service quality management. As a movement, POCQI is expected to make positive changes in the mindset or perception of HCWs who deal directly with patients about the quality of health services. In practice the provision of input (e.g resources) is the responsibility of the healthcare system, while POCQI is expected to be able to make changes to the quality improvement activities of the service process. Finally, of course, it will affect the output and outcome of health services. Thus, in the context of the quality system, POCQI is more concentrated on the quality process and is not expected to be able to change directly the quality input or the quality structure. POCQI demands individual engagement from frontline service units that deal directly with patients to improve quality. The use of POCQI can improve newborn care practices to be more feasible, measurable and replicable, adapted to available resources, including units with limited resources.²⁰

It has been reported by WHO, that the implementation of POCQI in the South-East Asia Region particularly in Bangladesh, Bhutan, Nepal, and Sri Lanka had been conducted in hospitals. Meanwhile, in this study, the implementation of POCQI was performed not only in hospitals but also in PHCs.¹⁹ Collaborating with the Ministry of Health which facilitated POCQI implementation for infant healthcare services at PHCs and Hospital, some interventions such as training and coaching are expected to achieve better performance. Performance that can be observed in a short time is HCW’s understanding of quality, skills in implementing POCQI measures, and engagement in quality improvement. It is hoped that there will then be changes in the service quality process and outcomes in the form of better health and satisfaction. It will take quite a longer time to reach an impact in general. WHO stated that there were many studies published by POCQI teams from India, however, mostly were focused on maternal and newborn health, while the ultimate goal actually was related to reducing infant mortality.¹⁹

The main intention of this pilot study is to observe the implementation of POCQI in the real situation of infant health care. The longterm purpose of the study is to assess how prominent and influential the implementation of POCQI in the service system is essential for formulating quality policies and strategies for health services. The analysis is about HCW’s understanding of improving the quality of service, skills in POCQI measures, engagement in quality activities, as well as changes in the process and intermediate outcomes of health services in PHCs and hospitals before and after the implementation of POCQI for IHS.

Materials and Methods

Research Design

This is a mixed-method study with a convergence triangulation strategy.²¹ Quantitative research strategy was a pre-post-test design, while qualitative strategy was content analyses, conducted from May to November 2021. Measurement of quantitative assessment variables were carried out at baseline and end of study line. Training, assistance, observation and evaluation were also carried out. The implementation of POCQI training was made possible by collaborations with District Health Offices (DHOs) and their staff.

Training and Coaching as Intervention Tools

Researchers, together with the Ministry of Health conducted a need assessment study of POCQI implementation at PHCs and hospitals. Prior to implementation, an approach and coordination were carried out first with the local government, DHOs, and health facilities. As part of the trial, researchers developed operational modules for POCQI training. The selection of health care facilities for the POCQI trial is determined by the DHO. Then training was carried out for healthcare workers in the targeted, problematic healthcare facilities (HCWs) to involve healthcare workers and coaches in the training programs for each healthcare facility.

Sequential training is conducted online for groups of HCWs and coaches for 3 days in 3 weeks. Both groups received the same training materials on day 1 and 2, and specific cases in day 3 using the modules that had been built. All training materials and recordings are stored on a digital platform (cloud based), so that all participants can revisit anytime they wished. Every participant was expected to work in each unit, both at PHC and Hospital, by carrying out POCQI steps at least once a week or as needed, facilitated by coaches. The results of the activities are stored on a digital platform. POCQI activities in each unit develop according to the cases and problems encountered where each team can learn from one or another experiences.

Monitoring and feedback of activities can be done at any time as well by the research team through this digital platform. However, monitoring is also carried out offline once a month for each unit. All the training and coaching act as an intervention, while observed for changes are noted and discussed together accompanied by researchers as resource persons.

Study Context

Cianjur and Indramayu districts have the highest number of infant mortality in West Java Province (173/1000 live births) and (208/1000 live births).²² Cianjur District is a regency located in the southwest of West Java Province and is a mountainous area in the north part, and coastal in its south area. Meanwhile, Indramayu District is a regency located in the north of West Java Province and is a coastal and more flat area in general. The geographical conditions of the two districts are very different, but both have high infant mortality rates.

Cianjur District has 47 PHCs, two public hospitals, and two private hospitals. PHC has been 96% accredited. All public hospitals have been accredited, while only one new private hospital. Meanwhile, Indramayu District has 49 PHCs, four Public Hospitals, and six private hospitals. All PHCs have been accredited. For public hospitals, there are three accredited, while for private hospitals, there are four that have been accredited. The POCQI pilot study was carried out while the COVID-19 pandemic was still ongoing. All research-related activities are carried out with health protocols following government regulations.

Subject

The research population for both quantitative and qualitative studies is HCWs from PHCs and Hospitals. The types of HCWs who are respondents are doctors, specialists, nurses, midwives, and other health workers, depending on the POCQI team at the health service unit at the PHC or Hospital.

The total number of survey subjects was 100 respondents, 50 respondents from Indramayu District, and 50 people from Cianjur District. Thirty respondents were taken from the PHCs and 20 from the hospitals for each district. The unit of analysis used in this research is the PHC and the Hospital. Thus, the number of subjects was 40 respondents for hospitals and 60 respondents for the PHCs survey. The survey was conducted twice for baseline and end line. The dynamics of HCWs (e.g tour of duties) in the service unit caused some respondents to change during the baseline and end line. So that it affects the collection and analysis of data.

In the qualitative study, the types of health workers who are respondents are doctors, nurses, midwives, and other health workers, and triangulation is carried out on patients. Subjects for the qualitative study were 20 people per district, so the total number of respondents was 40 people.

Research Instruments

The research team developed the instrument for the quantitative study based on logic model theory covering input, process, and outcome,^23,24 understanding of POCQI using WHO POCQI book references.¹⁹ POCQI emphasizes the ability of HCWs in the forefront to make positive changes to the functions under their control, in this case, the service process.

As a pilot project, the current research focus is more on the acceptance of the POCQI model by HCWs in the leading service units. Observations put more emphasis on mental, cognitive, and skill changes in the POCQI steps of the trained HCWs, not on the impact of implementing the POCQI itself. The questionnaire consists of QI understanding, QI engagement, POCQI steps, process, and outcome (intermediate) variables.

QI understanding refers to participants’ understanding of quality improvement; QI engagement refers to the involvement of participants in quality improvement activities. POCQI steps refer to participants’ perception of their ability to perform POCQI steps. The process is the participant’s perception of the quality of the service process. The Outcome is the participant’s perception of the outcome of the quality of health services in his unit. These five variables construct a quality performance perceived by the HCWs themselves. Participants understand better and can assess and reflect on their own situation. Positive changes in participants’ perceptions by measuring their perception before and after intervention were chosen as a tool to measure the effect of the intervention.

There are two instruments developed due to the difference in characteristics of the two, one for PHC and one for the hospital. Question items are based on the perception of the respondents themselves, so this instrument is perceptional. The research instrument was compiled in May 2021, and the first instrument trial was carried out in Cianjur District with a sample of 30 people to see validity and reliability. After revisions of the instrument, a second research instrument trial was conducted in Cianjur and Indramayu districts with a total sample of 30 people. After that, enumerator training was performed to be standardized and training to conduct in-depth interviews. The number of enumerators was ten people or 5 enumerators per district.

The results of the validity and reliability of the instrument at the time of measurement are presented in Table 1.

Table 1 Validity and Reliability of the Instrument

Data Collection

Quantitative data was collected through surveys, while qualitative data was collected through in-depth interviews. Survey activities were used to obtain data on dimensions of QI understanding, QI engagement, POCQI activities/steps, process, and outputs, before and after the intervention. Meanwhile, in-depth interview activities were used to explore perceptions of quality and POCQI activities before and after the intervention.

Enumerators carried out survey data collection and in-depth interviews for baseline in June 2021. Data collection was carried out at six PHCs and four hospitals (two government hospitals and two private hospitals) for the two districts. The intervention on POCQI was carried out from June-November 2021. End-line data collection was conducted after the intervention has been completed by enumerators and researchers using the same methods.

Data Analysis

The quantitative analysis were aggregated for both districts based on the unit of analysis for the PHC and hospital. The questionnaire uses an ordinal Likert scale with ordinal data. Therefore, the ordinal data is first transformed by Rasch modeling and using the Winsteps application.²⁵ After each dimension can be assigned to numerical data, the data is processed using SPSS version 20.0. Statistical analyses comparing baseline and end-line using independent t-test. Qualitative data processing using maxQDA 2020. Qualitative data analysis uses thematic content analysis that focuses on perceived quality and POCQI steps.

Ethical Considerations

This research is by the Declaration of Helsinki and has obtained ethical clearance from the Universitas Padjadjaran Ethical Committee No 435/UN6.KEP/EC/2022. Informed consent was obtained from all subjects before participating in this study, and informants were registered under pseudonyms.

Results

The survey results in the PHC group (Table 2) showed that the five variables observed had a significant change in the mean, especially in QI understanding, QI engagement, and POCQI skills. In addition, there was an increase in the mean of the process and outcome variables, although the changes were insignificant. But overall, there was a significant change in the quality performance of the PHC group.

Table 2 Changes of Perceptions of Staff in the PHC Group (n=60)

The survey results in the Hospital group (Table 3) show that POCQI training and coaching in Infant Health Services have significantly changed all variables. The most significant change occurred in the POCQI skill variable, followed by QI engagement and QI understanding. In contrast to PPHC, the Hospital group saw significant positive changes in the process and outcome.

Table 3 Changes of Perceptions of Staff in the Hospital Group (n=40)

Qualitative results are presented in Tables 4 and 5 to see differences in perceptions of service quality and POCQI steps before and after the POCQI training and coaching intervention.

Table 4 Comparison of Perceived Quality of Infant Health Services Before and After POCQI Interventions in Indramayu and Cianjur Districts

Table 5 Comparison of POCQI Activities for Infant Health Services Before and After POCQI Interventions in Indramayu and Cianjur Districts

Qualitative Study: Quality Perception

Before Intervention

According to the patient, the infant health service at the PHC was not good, the queue at the registration section was quite long, about 1 hour. This made the patient feel bored and annoyed, so the patient writes his frustration in the suggestion box.

Yeah, the service wasn’t that great …. I have to queue at the registration section … quite a long time, one hour …. (R25)

Yes, I wrote it in the suggestion box because there are a lot of patients here, so the wait is a bit long, that’s all (R10)

Several respondents who visited the PHC who did not have a guaranteed health insurance card (HIC) said that the price for treatment at the PHC was affordable, that is Rp. 10,000 (approximately 0.7 USD).

Healthcare costs at the PHC are not expensive, the price is affordable for me … only Rp. 10,000, - because I don’t have a HIC, but the queues are long both at registration and service … (C3)

Perceptions of health workers about the quality of infant care at the PHC and at the hospital, still need to be improved in terms of quality improvement. Health workers have submitted requests such as additional human resources and infrastructure, but this is not easy to fulfill due to limited funding.

Every year we ask …. every year a request ….to the ministry … It’s been submitted, since my time it’s still like that. No additional HR. Then Rp 2.5 billion, yes, the ventilator … the incubator is good, the incubator is not cheap. How many incubators in a sophisticated Neonatal Care Intensive Unit (NICU) are Rp 2 billion, just buy 2–3 out. Maybe … (R14)

Another perception perceived by health workers regarding service quality is the lack of feedback from superiors on services provided to patients.

… … we have provided services to patients but there is no evaluation or response from superiors. (R9)

Respondents who came from health workers in private hospitals said the quality of infant care could not be provided properly due to the lack of human resources, facilities and infrastructure. Human resources are very limited because they are divided into a healthy baby room, a sick baby and if there is caesarean section (CS), they have to go to the operating room and get out of the baby’s room. Human Resources should be shared, causing babies not to be monitored properly because there is no connecting door, to facilitate baby monitoring. There are no NICU and pediatric intensive care unit (PICU) rooms so if there is an emergency it will take a long time to get a referral. In addition, the obstacle in improving the quality of service to children under five is the length of decision making when referring.

… The quality of baby care in our hospital is not good, because human resources are limited, the facilities and infrastructure are the same … we can’t monitor the baby’s condition all the time, because there is no connecting door in the room … we don’t have a NICU and PICU room … often slow in making decisions whether the baby will be referred or not …. because there must be a person in charge who will refer … (C9)

After Intervention

The patient’s perception of the quality of infant care at the PHC and Hospitals was considered good. Patient care from the emergency room (ER) to the completion of the delivery process is handled as well as possible.

The action is good, the midwife is responsive too, patient … very patient, ma’am … (R22)

I’m satisfied, the service is good, ma’am, it’s clear if you think about it, I think I understand (R11)

In government hospitals, patient was satisfied with the services provided. The service is in terms of facilities, infrastructure, friendliness of officers, clarity in action. The following is an excerpt from the interview:

I walk around the room every morning … I like to ask the patient the most or the patient’s parents. How come there are complaints or there are our services that are not so good … From infrastructure or for example from our officers who are not good. Thank God, all this time. Yes, if you come home in good health, from the beginning the child can’t walk and when he comes home he can walk alone and run alone. Thank God they are satisfied it seems.. (C11)

After the POCQI training, health workers are optimistic that they can improve quality because it is supported by leadership policies that support infant care, improvement of Standard Operating Procedure (SOPs), and the enthusiasm of health workers in improving quality. The better the quality of service, the better the patient’s assessment of hospitals and health centers.

The quality of service here is now increasing now there is an update to the SOP, so the SOP is changed and it must be carried out right …, previously this. SOP existed, but it was not implemented, sometimes it was just a display on the wall, even if it was implemented, not all of them, ma’am …. jumping up and down … now with the Head of the PHC and the doctor of the PHC … we are being monitored continuously …. (R26)

According to health worker respondents, PHC are currently starting to make improvements in providing services to patients, especially after the POCQI accreditation and training. The health center provides a play area for children. The maternal and child health (MCH) room is also separated from the general patient care (GPC) room, so the patient feels more comfortable Therefore, it is very rare to receive complaints from the public nowadays.

… So if the other health centers are GPC, MCH, integrated management of childhood illness (IMCI) are connected, yes, there can be a ride. Yesterday, we had an innovation, we took it, it was moved in one building, so we are waiting for the patient, not at the same time as the parents or elderly patient. The pink building even though …. well even if it isn’t, it’s not a new building, but the waiting area is for children, including the children’s playground. (C13)

In addition, according to one PHC respondent, POCQI improves the quality of infant health services, because with the POCQI activity, an examination of the nutritional status of infants, which had not previously been carried out in a special infection room, was carried out. The following is the respondent’s statement:

Yes …, after the POCQI was seen in the nutritionist’s room, they were actually a room specializing in COVID, they didn’t weigh the weight or measure the height, they might not even check it …, but with this POCQI we are reminded once again that weighing and measuring are important, because we didn’t think that way at first …, it turns out that there was a malnutrition that we didn’t handle before, because there were complaints of coughs, colds or fever, just go straight to the orchid room, without looking the nutritional status of the baby, basically they are heading towards COVID …. (C16)

Best Practices: Changes Made by Primary Health Care and Hospitals

1. Changes in the flow of services make it easier for patients and health workers to provide services to patients.
2. Establish standard operational procedures (SOPs) for each patient case served.
3. Establish SOPs for the use of facilities such as incubators.
4. Conduct an audit of SOP compliance.
5. Sort out the Child and Mother Health and Treatment Center service buildings.
6. Involve the family when providing services.
7. Complete handwashing facilities for patients and their families.
8. All health workers providing services are more patient-centered.
9. Public PHC and hospitals are more creative in providing service processes for the prevention of infant mortality.

The survey results in PHC and hospitals showed that after the POCQI intervention, there was an increase in QI understanding, QI engagement and POCQI skills, and changes in service quality processes and outcomes. These results are by the perceptions of health workers and patients carried out through in-depth interviews that health centers and hospitals have changed to be more focused on patient-centered.

Discussion

The survey results at the PHCs and hospitals showed an increase in comprehension, engagement, and POCQI skills, processes, and outcomes. However, the changes in the processes and outcomes of the PPHC were not significant. In both groups, there is a significant change in the performance of quality efforts. The results of this study are similar to those in Ethiopia, there was a change in understanding of QI and staff were able to make changes to services better after the intervention.²⁶ QI interventions in Ghana, Tanzania, and Uganda can improve maternal and newborn care and prevent infant mortality.^27,28

The most significant change occurred in the POCQI skill variable, followed by QI engagement and QI understanding. POCQI is relatively new to be trained on HCWs in forefront services. Therefore, it is reasonable if the change is the highest compared to other variables. Hospitals are relatively more often exposed to the quality of health services than PHCs. This is because the hospital accreditation and other quality management system has been around for much longer than the PHC. However, POCQI skills are also a new thing in the hospital’s quality management system.

Training and coaching POCQI for Infant Health Services has changed potential quality performance on average in the PHC group through the measurement instrument built. In addition, training and guidance interventions for service providers effectively improve service quality and performance.^29,30 The leading service units in PHC, no doubt, have considerable potential to improve services to the community. Previous study investigated factors which influenced the process of QI in three PHCs in Cianjur District, the researchers found there were four main factors which contributed to the QI process namely: leadership, enthusiasm of staff and multidisciplinary collaboration, QI culture integration, and PHC on going accreditation process.³¹

Based on qualitative results conducted on health workers at both the PHC and hospitals, it was stated that the capacity of the officers increased after receiving POCQI training. Increasing the capacity of health workers encourages an increase in the quality of health services. This result is in line with the findings of Kumar et al, who said that the POCQI approach managed by doctors and nurses has been proven to improve the quality of health services and can overcome contextual problems such as limited health service resources.³² The results of the meta-analysis show that public health models and frameworks can support capacity building for health workers.³³

The qualitative results also show that based on the patient’s perception after the POCQI training the quality of health services at the PHC and Hospitals is getting better. Patients say that health workers are more caring, more informative, the service rooms are cleaner and the service flow is clearer. This is in accordance with the opinion of Mosadeghrad that the quality of service depends on good care and treatment, improved health, a clean service environment, and interaction with service providers.³⁴ The patient’s perspective can be a tool to measure quality.³⁵ Perceptions of the quality of health services vary greatly depending on the context and stakeholder perspectives.³⁴ The quality of health services encourages the use of health services, perceptions of quality affect the health system.³⁶

The qualitative results also showed that after POCQI training, PHC and hospital staff became able to carry out the 4 POCQI steps and were able to make changes to improve the quality of infant health services properly. Plan-Do-Study-Action (PDSA), which is the third step, is a step to make changes when finding problems in infant health services. This is supported by the opinion of Sachan et al who said that PDSA triggers health workers to make ideas for changes to services so that service quality increases.²⁰

PHC, as a government-owned institution, is often stigmatized as the lowest health care facility or as a last resort in obtaining health services. However, this measurement shows a change in the performance of the quality of service at the PHC. Therefore, there is a very open opportunity for local governments to improve the performance of their health system. Therefore, strengthening the health system is an essential key so that the service system can be stronger and more resilient so that POCQI can run well.

Strengthening the health system in low and middle-income countries has proven to enhance health services to improve service quality.^37,38 The quality of health care in the US varies greatly between hospitals, cities, and states. Some cases do not even meet the standards, although some cases have met and exceeded service standards.³⁹ Moreover, the strategy of POCQI implementation also successfully in improving quality of care in 131 facilities in India.⁴⁰ PHCs and hospitals in the district have limited resources. PHC and public hospitals are under the responsibility of the DHO in the local health system. If DHO can support better resources, then leading service units can amplify service quality performance. A study in India conducted in a resource constraint setting by Sachan et al, found that a simple POCQI methodology could enhance the quality of care.²⁰

The implementation of POCQI makes all PHC and hospital public HCWs focus more on patient-centered care (PCC), which aims to improve service quality. PCC is a treatment that respects the patient deeply and is responsive to patient preferences, needs and values.⁴¹ PCC is one of the dimensions of high-quality health care, and its implementation has been shown to improve the overall quality of health care.⁴² Efforts to promote PCC must consider patients and their families, physicians and the health system.^43,44 Therefore, the implementation of PCC requires the role of all stakeholders and the commitment of health care organizations, including DHO.

The limitation of this study is that the intervention did not use a control group. The results of both quantitative and qualitative research are perceived by the respondents. The strength of this research is the mixed method concurrent triangulation method, so it can be used to see the similarities between the quantitative and qualitative results. Research on the implementation of POCQI in IHS with this model will be the basis for the development of knowledge and practice for the next POCQI interventions in Indonesia.

Implication for Policy

POCQI implementation has proven to transform primary and secondary service providers to provide better and more innovative service processes. Therefore, the Ministry of Health of Indonesia should establish POCQI as one of the essential quality policies in providing infant health services. Improving service quality is not enough only by improving the quality assurance (e.g existing accreditation system) but, more importantly changing the quality culture of HCFs and HCWs using POCQI. POCQI implementation encourages community decision making, health workers and national health system policies.⁴⁰

The quality policy in health care facilities should be managed by the state so that improving the quality of services in the forefront units can be implemented in all HCFs. Some countries that already have national institutions to enhance the quality of health services generally have better outcomes.^45–47

The implementation of POCQI to complement the quality management system in the local health system is very strategic in improving the health status of infants. Therefore, DHO needs to strengthen quality policies in the local health system through by strengthening PHC in its area by scaling up coaching POCQI for all public and private PHCs.

Conclusion

POCQI training and coaching can improve QI understanding, QI engagement, and POCQI skills, processes, and service outcomes at PHCs and hospitals. However, changes in the processes and outcomes of PHC are not significant. In addition, there has also been a significant change in the performance of quality. Therefore, the Ministry of Health must strengthen the quality policy by implementing the POCQI strategy and strengthening the quality management system in the National and Local Health Systems.