Assessing Service Quality Dimensions and Their Effect on Patients Satisfaction in Bahrain Primary Healthcare Using a Modified Version of the General Practice Assessment Questionnaire

Introduction

It is essential to continuously strive to enhance the quality of care in general practice healthcare settings. An important aspect of quality improvement includes understanding the patients’ perspective, which can help identify the target areas for quality improvement. Measuring patient experience is a method commonly used to assess the quality of care in general practice settings.¹ Moreover, patient satisfaction in their experience of a healthcare setting is highly important. It is associated with better compliance and clinical outcomes.² Satisfied patients are more likely to recommend the hospital to family and friends and keep coming back, which improves the hospital’s reputation and accreditation scores. Hospitals trying to improve patient satisfaction will have a significant competitive advantage over other hospitals because a positive patient experience comes from more than just the patient–clinician interaction and is influenced by the entire health system’s infrastructure. Understanding the patient’s experience allows identifying gaps and developing quality improvement initiatives that continuously modify healthcare services to achieve better outcomes.

There are many tools worldwide for evaluating the quality of primary care. Of these tools is the General Practice Assessment Questionnaire (GPAQ)^1,3,4 GPAQ is a modified version of what had been developed in the United States as the Primary Care Assessment Tool (PCAT) and General Practice Assessment Survey.³ GPAQ became widely used to assess primary healthcare systems. GPAQ evaluates several primary care qualities, including a general practitioner, access to primary care, nurse care, continuity of care, physicians’ communication skills, and enablement. Furthermore, the GPAQ features a more detailed evaluation of physicians’ interpersonal and communication skills with patients.^4–6 Therefore, GPAQ is widely used to assess patient’s satisfaction with their primary healthcare system. Although the latest version of the GPAQ-R2 is a reliable and valid survey used to measure patient satisfaction in their experience of general practice healthcare settings,^3–5 a modification to it is necessary to fit the application, the healthcare system, and culture of any population.^6–9

This study aimed to have an instrument to assess Bahrain healthcare that fits the system and the Bahrain culture. Therefore, we used the GPAQ-R2 and a modified shorter version of it, GPAQ-R2-BDF, to assess their reliability and validity at a primary healthcare setting in Bahrain. Since such an assessment has not been performed in Bahrain, it will also highlight the healthcare system’s strengths and weaknesses and put the grounds for more studies, mainly a longitudinal research design, to establish a high-quality healthcare system.

Methods

Study Approval and Design

A cross-sectional design protocol was submitted and approved by the Research Ethics Committee at the Royal Services of Bahrain Defense Force (BDF) Military Hospital (Approval # BDF/R&REC/2018-217) in early 2018. This study was carried out in accordance with the Declaration of Helsinki. All participants voluntarily agreed to participate. After a detailed explanation of the study’s objective to the participants, a written consent form was obtained from each participant. Furthermore, written consent was obtained from the parents of participants who were younger than 16 years old. Moreover, all participants had a right to withdraw at any time or skip a single question or segment of questions.

Clinic Setting

This study was conducted at the General Practice (GP) of the Bahrain Defence Force (BDF) Hospital, the largest hospital in Bahrain. BDF hospital is a military teaching hospital with 450 beds, 322 physicians and dentists, and 1072 nurses and practical nurses. The hospital is for military staff and entitled patients only.

The study population was selected as a random sample of patients who received treatments in primary healthcare at a particular time. A total of 25 physicians worked at the time of the data collection. The GP consists of different departments, such as general health and pediatric departments, obesity and diabetes clinic, a postnatal and vaccination clinic, and a travel unit.

Procedure

This study used the drop off technique of its participants. All participants were patients visiting the general practitioner for a specific service. While waiting for their service, a researcher approached each participant, described the study, and asked if he/she is willing to participate. Besides, participants were informed that participating in this study was optional and that it would not affect their treatment and follow-up in the hospital. Following the acceptance of participation, each participant handed informed consent for signing. Then, the researcher handed out the GPAQ-R2 questionnaire with an Arabic translation in the waiting room and informed the participants to fill out the questionnaire under the researcher’s supervision.

Instrument Development Process

The study tool was created by adapting the GPAQ-R2 © 2014, which was reproduced with the kind permission of the University of Manchester & University of Cambridge. All participants answered the whole questionnaire, which is composed of 46 items.¹⁰ The GPAQ-R2 questionnaire consists of 6 sections; the first section is composed of 11 items (Q1-Q11), and respondents were asked to evaluate their general practitioner (GP), the second section is composed of 8 items (Q12-Q19), and respondents were asked to assess the receptionists and appointments. The third section is about overall of the general practice (Q20-Q29). The fourth is composed of 7 items (Q30-Q36) about the nurse care, the fifth about is composed of 5 items (Q37-Q41) about overall care, and the last section is about patient’s socio-demographic information (Q42-Q46).

In the GPAQ-R2, the first eight questions asked how well the doctor performed several tasks, and responses used a 5-point Likert scale ranging from “very good” to “very poor”. A sixth option “does not apply” was also provided. In contrast, two of the last three questions (Q9-Q11) asked respondents about their confidence in the doctor’s trustworthiness and confidentiality and were rated on a scale of 1–4, with 1 being “yes, 2 being definitely”, 3 being “not at all” and 4 being “do not know/cannot say”. In the BDF-modified GPAQ-R2 (GPAQ-R2-BDF), we split these into two components: doctor care (Q1-Q8) and confidence and enablement (Q9 and Q10) at the BDF hospital. The Q9 and Q10 components were included to items (Q37-Q39) to cover confidence and enablement as well. Items (Q11 and Q36) were removed from the doctor care and nurse care, respectively, since these questions have a yes or no answer, while the other questions within the same categories better quantify the doctor and nurse care. In the receptionist and appointments section (GP access), items Q15, Q16, Q18, and Q19 were removed because Q15 and Q16 were not culturally applicable, and Q18 and Q19 were informative data. The overall care was named GP practice and contained items Q21, Q23, and Q25, whereas items Q26-Q29 were removed. The removed items were not applicable to the BDF hospital system. Furthermore, items Q20, Q22, and Q24 were removed since they were asking about the quickness in time, and questions Q21, Q23, and Q25 were asking about the rating in quickness, whereby the latter items were more applicable to the patient’s satisfaction and services. Finally, Q41 was also deleted since the culture in Bahrain, as in all the Arabic countries, if a patient is happy with something, he/she will recommend it to others. Overall, the modified GPAQ-R2-BDF was composed of 32 items: 26 for general practice, 1 for an overall experience, and 5 for patient’s socio-demographic data and then selected to construct an instrument suitable for the context of this study.

Following the suggested changes, face content validity was achieved using a table of specifications and expert opinions. Expert opinion was taken into consideration by sending the instrument to five experts, who have more than 30 years of experience in the field of general practice and asking them to judge whether the content and format of the modified instrument were suitable in comparison to the table of specification with the original GPAQ-R2. Following the expert opinion approval, the analysis of the data took place as recommended.

In this study, the scale was reversed. For instance, doctor and nurse care components were scaled as 5 for “very good” and 1 for “very poor”. Responses such as “does not apply” and “do not know” were reported but excluded from the statistical analysis. Also, for confidence and enablement scaled as 3 “yes, definitely”, 2 “yes to some extent” and 1 “no, not at all”. As for GP access, the scale used as 4 “very helpful” to 1 as “not at all helpful”, and for a GP practice, the scale used as 6 “excellent” to 1 as “very poor”.

Data Analysis

Keiser-Mayer-Olkin (KMO) analysis and Bartlett’s test of sphericity were performed to determine the data suitability for principal component analysis (PCA). The validity of the questionnaire was assessed using rotated factor analysis, Varimax rotation. Eigen value >1 was used to determine the factors. Furthermore, if there was a cross-loading of items, the item with the highest factor where it was loaded was assigned. The number of factors to be extracted was based on the Kaiser rule (i.e., Eigen value >1.0). Besides, Cronbach’s alpha was used to evaluate the reliability of the instrument. Responses such as “does not apply” and “do not know” were excluded from the statistical analysis. The results and the responses of each component or item of a component were expressed as mean ± standard deviations. Patient satisfaction scores were compared across different patient characteristics. The Student’s t-test was applied for testing between two groups or component items with the assumption of equal variance unless Levene’s test was significant. For evaluating responses of more than two, one-way analysis of variance (ANOVA) was used to compare the mean scale scores of the different factors followed by Tukey’s post hoc test to differentiate the significant group. P value of <0.05 was considered significant.

Results

Patients’ Characteristics

A total of 520 participants were enrolled in the study, and their response rate was 100%. Of the 520, 278 (53.5%) were females and 242 (46.5%) were males. The majority (78%) of participants’ age was between 16 and 44 years, whereas 21.7% of the participants’ age was between 45 and 64 years (Table 1). Seventy-eight percent of the participant were Bahrainis, and 22% were non-Bahrainis living in Bahrain. Furthermore, most of the patients (85.6%) visited the clinic at BDF for their health, and 27.5% of the participants had a long-standing health condition (Table 1).

Table 1 Demographic Characteristics of the Survey Participants

Validity and Reliability of GPAQ-R2

Although the GPAQ-R2 is a well standard instrument in assessing general practice, re-validation and measurement of its reliability in BDF hospital showed that it needed some adjustment to fit the Bahrain healthcare system and culture in some components, the GP access and GP practice (Table 2). In the latter two sections, the reliability and validity levels were less than 0.70, which are considered questionable for reliability and only acceptable for validity. Besides, the correlation matrix revealed close to zero values with negative and positive numbers within the same item.

Table 2 Validation and Reliability Scores of the Original GRAQ-R2 in Bahrain Healthcare System

Validity and Reliability of the Modified GPAQ-R2 (GPAQ-R2-BDF)

The whole modified instrument, GPAQ-R2-BDR, was found to be suitable for factor analysis with KMO of 0.920 and a significant Bartlett’s test of sphericity (p<0.000). PCA was used to identify the factors and the total variance. The factor analysis showed that the data on the questionnaire decomposed into five factors that represented 61.3% of the total variance. The rotated factor matrix analysis with Varimax rotation for each component of GPAQ-R2-BDF assured our initial categorization to five components with the items that we have proposed (Table 3). KMO values for all five components ranged between 0.704 and 0.928 with significant Bartlett’s test of sphericity (p<0.000).

Table 3 Rotated Factor Analysis*, Validity, and Reliability of the Modified GRAQ-R2-BDF

As mentioned above, a total of five components were extracted from the analysis. The first component is the doctor care and included 8 items. The second component is nurse care and included 5 items. The third component is confidence and enablement and included 5 items. The fourth component is GP access and included 4 items. The fifth component is GP practice and included 3 items. Reliability ranged from 0.77 to 0.936 (Table 3), and the overall reliability of the questionnaire was high (α= 0.927).

Service Quality Scores and Patients’ Satisfaction at BDF Outpatient Clinic

Since each component’s scale was different, we normalized the scale to a 100% score (Table 4). The patients’ satisfaction scores were highest (~90%) in confidence and enablement, and doctor care. These scores were significantly higher than nurse care, GP access, and GP practice (P<0.001). On the other hand, patients’ satisfaction scores were lowest (~77%) in GP access and GP practice, and were significantly less than nurse care (~87%) as well (p<0.001).

Table 4 Patients’ Scores and Satisfaction in Each Component of the GPAQ-R2-BDF

Demographic Variables and Patients’ Satisfaction at BDF Outpatient Clinic

Participants’ age did not reveal of any significant difference in any of the components of the instrument. However, male participants were more satisfied with GP access than females (p<0.001) (Table 5). Furthermore, Bahraini patients felt better about the doctors’ care than non-Bahraini patients (p<0.01) but not with nurse care, confidence and enablement, GP access or GP practice. Finally, patients with long-standing health conditions showed less satisfaction with doctor care, confidence and enablement, and GP access (p<0.05–0.01) (Table 5).

Table 5 Comparison of Mean Scale Score with Different Variables

Discussion

This study evaluated the modified version GPAQ-R2-BDF as instruments to assess the general practice at a BDF hospital in Bahrain. Based on the factor analysis, the reliability, validity, and correlation matrixes for all the instrument components of the modified GPAQ-R2-BDF were high enough to ensure the instrument’s robustness. Such modifications are revealed to fit the healthcare in Bahrain.

Using the original GPAQ-R2, the results showed questionable reliability values, acceptable validity, and low correlation matrix within the GP access and GP practice components. These results confirm that each country or population needs to modify the instrument to fit its health system and culture.^6–9 Therefore, a modified version was adapted. The rotated factor analysis results yielded five components, and the results of the reliability analysis suggested the tool’s reliability and validity.^3,7 Therefore, this study indicates that GPAQ-R2-BDF can be used as an assessment instrument of Bahrain’s general practice centers. Furthermore, the overall experience was remarkably close to the average of all components. On the other hand, instruments used in UK, Thailand and Japan, but not Australia and New Zealand, showed a remarkable difference between the average of all components and overall experience.⁶ This discrepancy might be due to how participants weigh selected items within a component and overall satisfaction.

We must stress that the GPAQ-R2-BDF did not add any new items but only removed some and re-categorized the components. The reasons for removing items were described in the methods. Although dichotomous questions are used to lose ambiguity, they may give a false representation of the reliability, validity, and correlation matrix.¹¹ Besides, such dichotomous answers may fit in certain communities and populations, but not all. For instance, in this study, removing the dichotomous questions within the doctor and nurse care components increased the correlation matrix in both components and the reliability and validity in the nurse care component. Additionally, removing specific items due to their inapplicability to the healthcare system and Bahrain culture increased the convergent validity between items within GP access and GP practice components.

The modified GPAQ-R2-BDF included patient ratings of many different aspects of a general practice setting, such as waiting time, receptionists, and satisfaction with the general practitioner and nurses. The study also reported the scores of the participants with different characteristics, such as age and gender. Overall, the GP access and GP practice scored the lowest values (~77%). Similarly, GP access has always been the least component to satisfy patients in different settings and countries.^6–9 The most critical points in GP access were getting through someone at GP over the phone and how easy to speak to a doctor or nurse over the phone, which brought the least satisfaction. Similarly, in a GP practice, the least satisfactory item was the rating of waiting time. As in most clinics, patients likely have to wait for some time, even if they had scheduled appointments.⁶ Arabs, in general, including Bahrainis, are willing to wait even if they have an appointment. Although 55.9% of the patients waited for more than 10 minutes, it was not surprising that the combined “very poor”, “poor”, and “satisfactory” rating was only 12.3%. However, in an Australian study of patients who waited more than 11 minutes, more than 65% reported low ratings.⁷ Therefore, cultural differences apply, and that is why the instrument should be modified accordingly.

Furthermore, unlike past studies, no significant difference was found between the satisfaction scores and age.⁷ In Australia, a study using GPAQ reported that older people were inclined to convey higher satisfaction than younger patients.⁷ Similarly in Japan, a study using a modified version of GPAQ revealed high satisfaction rates in a sample averaged age of 68 years.⁶ In the present study, due to an insufficient number of older participants, no conclusion can be determined between old age and better satisfaction.

On the other hand, gender difference was seen in GP access, where males were more satisfied than females. Besides, patients with long-standing health conditions were less satisfied with doctor care, confidence and enablement, and GP access than those who do not have long-standing health conditions. The continuity of care for patients with long-standing health conditions has also been lowly related scale in other populations.⁶ These individuals are more likely to give a better picture of the system because of their needs and their regular interaction with the clinic. Besides, such patients need to understand their detailed medical history, medication, and family condition.⁶ This is why a longitudinal research design on services and satisfaction of the healthcare system is more indicative.

We chose the drop-off technique to minimize bias in this study, whereby patients visiting the clinic were asked to participate in the study. Secondly, participants were assured that their responses would not affect their treatment, and they filled out the evaluation themselves. Although completing the questionnaire under the researcher’s supervision and in the waiting room may be an advantage to ensure proper participation, it is also a limitation. Filling out the questionnaire under the supervision and in the waiting room may have influenced the participants’ ratings and made them more reluctant to report negative experiences. Participants may be perceived that the hospital staff would access their responses. In this study, the satisfaction ratings were considered somewhat high but equivalent to other communities such as New Zealand, although the latter study was done using postal services.⁹ Both New Zealand and Bahrain are relatively rich countries and have a small population (4.8 and 1.7 million, respectively), which might be a reason for such resemblance in satisfaction ratings. However, the ratings may have been different if participants filled out the questionnaire online or mailed it. It has been proposed that postal or online rating is ~5% less than drop off type.⁹ Several studies found higher bias in the GPAQ scores of questionnaires filled out in a waiting room than those filled out elsewhere and mailed in by post.^4,12,13 Future studies should consider ways to fill out the questionnaire without supervision using an online questionnaire that participants can fill out at home and without supervision and ensure participants respond to all the questions.

Conclusions

The present findings demonstrated high reliability and validity for the GPAQ-R2-BDF, which supports its use to assess Bahrain’s GP healthcare system. Furthermore, such an instrument should be adapted in a longitudinal research design to assess Bahrain’s healthcare system better. Lastly, BDF hospital healthcare clinics showed concerns in GP access, GP practice, and quality services for patients with long-standing health conditions requiring strategic interventions to reach better quality services.